# Technical Documentation: ocrmypdf/OCRmyPDF

> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/ocrmypdf/OCRmyPDF)

### Overview (`README.md`)

<img src="docs/images/logo.svg" width="240" alt="OCRmyPDF">

[](https://github.com/ocrmypdf/OCRmyPDF/actions/workflows/build.yml) [ ![Homebrew version][homebrew] ![ReadTheDocs][docs] ![Python versions][pyversions]

[pypi]: https://img.shields.io/pypi/v/ocrmypdf.svg "PyPI version"
[homebrew]: https://img.shields.io/homebrew/v/ocrmypdf.svg "Homebrew version"
[docs]: https://readthedocs.org/projects/ocrmypdf/badge/?version=latest "RTD"
[pyversions]: https://img.shields.io/pypi/pyversions/ocrmypdf "Supported Python versions"

OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched or copy-pasted.

```bash
ocrmypdf                      # it's a scriptable command line program
   -l eng+fra                 # it supports multiple languages
   --rotate-pages             # it can fix pages that are misrotated
   --deskew                   # it can deskew crooked PDFs!
   --title "My PDF"           # it can change output metadata
   --jobs 4                   # it uses multiple cores by default
   --output-type pdfa         # it produces PDF/A by default
   input_scanned.pdf          # takes PDF input (or images)
   output_searchable.pdf      # produces validated PDF output
```

[See the release notes for details on the latest changes](https://ocrmypdf.readthedocs.io/en/latest/release_notes.html).

## Main features

- Generates a searchable [PDF/A](https://en.wikipedia.org/?title=PDF/A) file from a regular PDF
- Places OCR text accurately below the image to ease copy / paste
- Keeps the exact resolution of the original embedded images
- When possible, inserts OCR information as a "lossless" operation without disrupting any other content
- Optimizes PDF images, often producing files smaller than the input file
- If requested, deskews and/or cleans the image before performing OCR
- Validates input and output files
- Distributes work across all available CPU cores
- Uses [Tesseract OCR](https://github.com/tesseract-ocr/tesseract) engine to recognize more than [100 languages](https://github.com/tesseract-ocr/tessdata)
- Keeps your private data private.
- Scales properly to handle files with thousands of pages.
- Battle-tested on millions of PDFs.

<img src="misc/screencast/demo.svg" alt="Demo of OCRmyPDF in a terminal session">

For details: please consult the [documentation](https://ocrmypdf.readthedocs.io/en/latest/).

## Motivation

I searched the web for a free command line tool to OCR PDF files: I found many, but none of them were really satisfying:

- Either they produced PDF files with misplaced text under the image (making copy/paste impossible)
- Or they did not handle accents and multilingual characters
- Or they changed the resolution of the embedded images
- Or they generated ridiculously large PDF files
- Or they crashed when trying to OCR
- Or they did not produce valid PDF files
- On top of that none of them produced PDF/A files (format dedicated for long time storage)

...so I decided to develop my own tool.

## Installation

Linux, Windows, macOS and FreeBSD are supported. Docker images are also available, for both x64 and ARM.

| Operating system              | Install command               |
| ----------------------------- | ------------------------------|
| Debian, Ubuntu                | ``apt install ocrmypdf``      |
| Windows Subsystem for Linux   | ``apt install ocrmypdf``      |
| Fedora                        | ``dnf install ocrmypdf``      |
| macOS (Homebrew)              | ``brew install ocrmypdf``     |
| macOS (MacPorts)              | ``port install ocrmypdf``     |
| macOS (nix)                   | ``nix-env -i ocrmypdf``       |
| LinuxBrew                     | ``brew install ocrmypdf``     |
| FreeBSD                       | ``pkg install py-ocrmypdf``   |
| OpenBSD                       | ``pkg_add ocrmypdf``          |
| Ubuntu Snap                   | ``snap install ocrmypdf``     |

For everyone else, [see our documentation](https://ocrmypdf.readthedocs.io/en/latest/installation.html) for installation steps.

## Languages

OCRmyPDF uses Tesseract for OCR, and relies on its language packs. For Linux users, you can often find packages that provide language packs:

```bash

# Debian/Ubuntu users
apt-cache search tesseract-ocr # Display a list of all Tesseract language packs
apt-get install tesseract-ocr-chi-sim  # Example: Install Chinese Simplified language pack


# Arch Linux users
pacman -S tesseract-data-eng tesseract-data-deu # Example: Install the English and German language packs

# OpenBSD users
pkg_info -aQ tesseract  # Display a list of all Tesseract language packs
pkg_add tesseract-cym  # Example: Install the Welsh language pack

# brew macOS users
brew install tesseract-lang

# Fedora users
dnf search tesseract-langpack # Display a list of all Tesseract language packs 
dnf install tesseract-langpack-ita # Example: Install the Italian language pack


```

You can then pass the `-l LANG` argument to OCRmyPDF to give a hint as to what languages it should search for. Multiple languages can be requested.

OCRmyPDF supports Tesseract 4.1.1+. It will automatically use whichever version it finds first on the `PATH` environment variable. On Windows, if `PATH` does not provide a Tesseract binary, we use the highest version number that is installed according to the Windows Registry.

## Documentation and support

Once OCRmyPDF is installed, the built-in help which explains the command syntax and options can be accessed via:

```bash
ocrmypdf --help
```

Our [documentation is served on Read the Docs](https://ocrmypdf.readthedocs.io/en/latest/index.html).

Please report issues on our [GitHub issues](https://github.com/ocrmypdf/OCRmyPDF/issues) page, and follow the issue template for quick response.

## Feature demo

```bash
# Add an OCR layer and require PDF/A
ocrmypdf --output-type pdfa input.pdf output.pdf

# Convert an image to single page PDF
ocrmypdf input.jpg output.pdf

# Add OCR to a file in place (only modifies file on success)
ocrmypdf myfile.pdf myfile.pdf

# OCR with non-English languages (look up your language's ISO 639-3 code)
ocrmypdf -l fra LeParisien.pdf LeParisien.pdf

# OCR multilingual documents
ocrmypdf -l eng+fra Bilingual-English-French.pdf Bilingual-English-French.pdf

# Deskew (straighten crooked pages)
ocrmypdf --deskew input.pdf output.pdf
```

For more features, see the [documentation](https://ocrmypdf.readthedocs.io/en/latest/index.html).

## Requirements

In addition to the required Python version, OCRmyPDF requires external program installations of Ghostscript and Tesseract OCR. OCRmyPDF is pure Python, and runs on pretty much everything: Linux, macOS, Windows and FreeBSD.

## Plugins

OCRmyPDF provides a plugin interface allowing its capabilities to be extended or replaced. Here are some plugins we are aware of:

- [OCRmyPDF-AppleOCR](https://github.com/mkyt/ocrmypdf-AppleOCR): replaces the standard Tesseract OCR engine with Apple Vision Framework. Requires macOS.
- [OCRmyPDF-EasyOCR](https://github.com/ocrmypdf/OCRmyPDF-EasyOCR): replaces the standard Tesseract OCR engine with EasyOCR, a newer OCR engine based on PyTorch. GPU strongly recommended.
- [OCRmyPDF-PaddleOCR](https://github.com/clefru/ocrmypdf-paddleocr): replaces the standard Tesseract OCR engine with PaddleOCR, a powerful GPU accelerated OCR engine.

[paperless-ngx](https://docs.paperless-ngx.com/) provides integration of OCRmyPDF into a searchable document management system.

## Press & Media

- [Going paperless with OCRmyPDF](https://medium.com/@ikirichenko/going-paperless-with-ocrmypdf-e2f36143f46a)
- [Converting a scanned document into a compressed searchable PDF with redactions](https://medium.com/@treyharris/converting-a-scanned-document-into-a-compressed-searchable-pdf-with-redactions-63f61c34fe4c)
- [c't 1-2014, page 59](https://web.archive.org/web/20220115123833/https://www.heise.de/ct/ausgabe/2014-1-kurz-vorgestellt-Texterkennung-2279695.html): Detailed presentation of OCRmyPDF v1.0 in the leading German IT magazine c't
- [heise Open Source, 09/2014: Texterkennung mit OCRmyPDF](https://heise.de/-2356670)
- [heise Durchsuchbare PDF-Dokumente mit OCRmyPDF erstellen](https://www.heise.de/ratgeber/Durchsuchbare-PDF-Dokumente-mit-OCRmyPDF-erstellen-4607592.html)
- [Excellent Utilities: OCRmyPDF](https://www.linuxlinks.com/excellent-utilities-ocrmypdf-add-ocr-text-layer-scanned-pdfs/)
- [LinuxUser Texterkennung mit OCRmyPDF und Scanbd automatisieren](https://www.linux-community.de/ausgaben/linuxuser/2021/06/texterkennung-mit-ocrmypdf-und-scanbd-automatisieren/)
- [Y Combinator discussion](https://news.ycombinator.com/item?id=32028752)

## Business enquiries

OCRmyPDF would not be the software that it is today without companies and users choosing to provide support for feature development and consulting enquiries. We are happy to discuss all enquiries, whether for extending the existing feature set, or integrating OCRmyPDF into a larger system.

## License

The OCRmyPDF software is licensed under the Mozilla Public License 2.0 (MPL-2.0). This license permits integration of OCRmyPDF with other code, included commercial and closed source, but asks you to publish source-level modifications you make to OCRmyPDF.

Some components of OCRmyPDF have other licenses, as indicated by standard SPDX license identifiers or the DEP5 copyright and licensing information file. Generally speaking, non-core code is licensed under MIT, and the documentation and test files are licensed under Creative Commons ShareAlike 4.0 (CC-BY-SA 4.0).

## Disclaimer

The software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

---

### Overview (`misc/screencast/README.md`)

To regenerate
=============

Using asciinema and svg-term (`npm install svg-term-cli`).

Create `~/.config/asciinema/config` to disable prompt.

```
[record]

command = fish --init-command 'alias fish_prompt="echo \>\ "'
```

Run asciinema

```
asciinema rec new_input.cast
```

Re-record faster version with fewer pauses

```
asciinema rec demo.cast -c "asciinema play new_input.cast --speed 2 --idle-time-limit 0.5"
```

Convert to SVG
```
svg-term --in=misc/screencast/demo.cast --out=misc/screencast/demo.svg --window
```

---

### Index (`docs/index.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# OCRmyPDF documentation

:::{figure} images/logo.svg
:::

OCRmyPDF adds an optical character recognition (OCR) text layer to scanned PDF
files, allowing them to be searched.

PDF is the best format for storing and exchanging scanned documents.
Unfortunately, PDFs can be difficult to modify. OCRmyPDF makes it easy to apply
image processing and OCR (recognized, searchable text) to existing PDFs.

```{toctree}
:maxdepth: 1

introduction
releasenotes/index
installation
languages
jbig2
```

```{toctree}
:caption: Usage
:maxdepth: 2

cookbook
optimizer
docker
advanced
batch
cloud
performance
pdfsecurity
errors
```

```{toctree}
:caption: Developers
:maxdepth: 2

api
plugins
apiref
design_notes
contributing
maintainers
```

# Indices and tables

- {ref}`genindex`
- {ref}`modindex`
- {ref}`search`

---

### Releasenotes Index (`docs/releasenotes/index.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Release notes

OCRmyPDF uses [semantic versioning](http://semver.org/) for its
command line interface and its public API.

OCRmyPDF's output messages are not considered part of the stable interface -
that is, output messages may be improved at any release level, so parsing them
may be unreliable. Use the API to depend on precise behavior.

The public API may be useful in scripts that launch OCRmyPDF processes or that
wish to use some of its features for working with PDFs.

The most recent release of OCRmyPDF is . Any newer versions
referred to in these notes may exist the main branch but have not been
tagged yet.

OCRmyPDF typically supports the three most recent Python versions.

:::{note}
Attention maintainers: these release notes may be updated with information
about a forthcoming release that has not been tagged yet. A release is only
official when it's tagged and posted to PyPI.
:::

```{toctree}
:glob: true
:maxdepth: 1
:reversed: true

version*
```

---

### Advanced (`docs/advanced.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Advanced features

## Control of unpaper

OCRmyPDF uses `unpaper` to provide the implementation of the
`--clean` and `--clean-final` arguments.
[unpaper](https://github.com/Flameeyes/unpaper/blob/main/doc/basic-concepts.md)
provides a variety of image processing filters to improve images.

By default, OCRmyPDF uses only `unpaper` arguments that were found to
be safe to use on almost all files without having to inspect every page
of the file afterwards. This is particularly true when only `--clean`
is used, since that instructs OCRmyPDF to only clean the image before
OCR and not the final image.

However, if you wish to use the more aggressive options in `unpaper`,
you may use `--unpaper-args '...'` to override the OCRmyPDF's defaults
and forward other arguments to unpaper. This option will forward
arguments to `unpaper` without any knowledge of what that program
considers to be valid arguments. The string of arguments must be quoted
as shown in the examples below. No filename arguments may be included.
OCRmyPDF will assume it can append input and output filename of
intermediate images to the `--unpaper-args` string.

In this example, we tell `unpaper` to expect two pages of text on a
sheet (image), such as occurs when two facing pages of a book are
scanned. `unpaper` uses this information to deskew each independently
and clean up the margins of both.

```bash
ocrmypdf --clean --clean-final --unpaper-args '--layout double' input.pdf output.pdf
ocrmypdf --clean --clean-final --unpaper-args '--layout double --no-noisefilter' input.pdf output.pdf
```

:::{warning}
Some `unpaper` features will reposition text within the image.
`--clean-final` is recommended to avoid this issue.
:::

:::{warning}
Some `unpaper` features cause multiple input or output files to be
consumed or produced. OCRmyPDF requires `unpaper` to consume one
file and produce one file; errors will result if this assumption is not
met.
:::

:::{note}
`unpaper` uses uncompressed PBM/PGM/PPM files for its intermediate
files. For large images or documents, it can take a lot of temporary
disk space.
:::

## Control of OCR options

OCRmyPDF provides many features to control the behavior of the OCR
engine, Tesseract.

### OCR processing mode

:::{versionadded} 17.0.0
The `--mode` (`-m`) argument consolidates OCR processing options.
:::

OCRmyPDF provides a unified `--mode` argument to control how pages with
existing text are handled:

| Mode | Behavior | Legacy equivalent |
|------|----------|-------------------|
| `default` | Error if text is found | (no flag) |
| `force` | Rasterize all content and run OCR, keeping hyperlinks | `--force-ocr` |
| `force-ocr-no-links` | Like `force`, but also discard hyperlinks | |
| `skip` | Skip pages with existing text | `--skip-text` |
| `redo` | Re-OCR pages, stripping old OCR layer | `--redo-ocr` |

```bash
# Skip pages that already have text
ocrmypdf --mode skip input.pdf output.pdf
# or equivalently:
ocrmypdf -m skip input.pdf output.pdf

# Force OCR on all pages (rasterizes everything)
ocrmypdf --mode force input.pdf output.pdf

# Re-do OCR, replacing old invisible text
ocrmypdf --mode redo input.pdf output.pdf
```

The legacy flags (`--force-ocr`, `--skip-text`, `--redo-ocr`) remain as
silent aliases for backward compatibility.

### When OCR is skipped

If a page in a PDF seems to have text, by default OCRmyPDF will exit
without modifying the PDF. This is to ensure that PDFs that were
previously OCRed or were "born digital" rather than scanned are not
processed.

If `--mode skip` (or `--skip-text`) is issued, then no image processing or OCR will be
performed on pages that already have text. The page will be copied to
the output. This may be useful for documents that contain both "born
digital" and scanned content, or to use OCRmyPDF to normalize and
convert to PDF/A regardless of their contents.

If `--mode redo` (or `--redo-ocr`) is issued, then a detailed text analysis is performed.
Text is categorized as either visible or invisible. Invisible text (OCR)
is stripped out. Then an image of each page is created with visible text
masked out. The page image is sent for OCR, and any additional text is
inserted as OCR. If a file contains a mix of text and bitmap images that
contain text, OCRmyPDF will locate the additional text in images without
disrupting the existing text. Some PDF OCR solutions render text as
technically printable or visible in some way, perhaps by drawing it and
then painting over it. OCRmyPDF cannot distinguish this type of OCR
text from real text, so it will not be "redone".

If `--mode force` (or `--force-ocr`) is issued, then all pages will be rasterized to
images, discarding any hidden OCR text, rasterizing any printable
text, and flattening form fields or interactive objects into their visual
representation. This is useful for redoing OCR, for fixing OCR text
with a damaged character map (text is selectable but not searchable),
and destroying redacted information.

Hyperlinks (link annotations) are kept, so links to web pages and to other
places in the document still work in the rasterized output; they are clickable
areas over the page image. Other annotations are flattened into the image.
Use `--mode force-ocr-no-links` to rasterize in the same way but discard the
hyperlinks too, for example when the link targets themselves are sensitive.

### Tagged PDFs and structural markup

Some PDFs carry a logical structure tree (`/StructTreeRoot`), the markup that
makes a "Tagged PDF" — typically the result of layout analysis or a born-digital
export. By default OCRmyPDF treats this as a signal that the document may not need
OCR and exits, in the same way it stops on PDFs that already contain text. Use
`--tagged-pdf-mode ignore`, or one of `--mode skip`/`redo`/`force`, to process
such a file anyway.

OCRmyPDF cannot rebuild a structure tree to match newly recognized text. When
`--force-ocr` rasterizes pages, or `--redo-ocr` strips and rewrites the text layer,
the structure tree no longer corresponds to the page content, so it is discarded.
`--mode skip` leaves text pages untouched, so their structural markup is preserved.

:::{note}
Preservation under `--mode skip` only holds when the output is not converted to
PDF/A. PDF/A conversion is performed by Ghostscript, and Ghostscript 10.x discards
the structure tree during conversion (Ghostscript 9.x preserved it). Because the
default `--output-type auto` may fall back to Ghostscript, use
`--output-type pdf` if you need to guarantee that a Tagged PDF's structural markup
survives. Speculative PDF/A conversion, which does not use Ghostscript,
sidesteps this issue whenever the file passes pikepdf's PDF/A validator.
:::

### Time and image size limits

By default, OCRmyPDF permits tesseract to run for three minutes (180
seconds) per page. This is usually more than enough time to find all
text on a reasonably sized page with modern hardware.

If a page is skipped, it will be inserted without OCR. If preprocessing
was requested, the preprocessed image layer will be inserted.

If you want to adjust the amount of time spent on OCR, change
`--tesseract-timeout`. You can also automatically skip images that
exceed a certain number of megapixels with `--skip-big`. (A 300 DPI,
8.5×11" page image is 8.4 megapixels.)

```bash
# Allow 300 seconds for OCR; skip any page larger than 50 megapixels
ocrmypdf --tesseract-timeout 300 --skip-big 50 bigfile.pdf output.pdf
```

### OCR for huge images

Tesseract has internal limits on the size
of images it will process. By default,
`--tesseract-downsample-large-images` is enabled, and OCRmyPDF will
downsample images to fit Tesseract limits. (The limits are usually encountered
only for scanned images of oversized media, such as large maps or blueprints exceeding
110 cm or 43 inches in either dimension, and at high DPI.) This feature can disabled
using `--no-tesseract-downsample-large-images`.

`--tesseract-downsample-above Npixels` adjusts the threshold at which images
will be downsampled. By default, only images that exceed any of Tesseract's
internal limits are downsampled (32767 pixels on either dimension).

You will also need to set `--tesseract-timeout` high enough to allow
for processing.

Only the image sent for OCR is downsampled. The original image is
preserved.

```bash
# Allow 600 seconds for OCR on huge images
ocrmypdf --tesseract-timeout 600 \
    --tesseract-downsample-large-images \
    bigfile.pdf output.pdf

# Downsample images above 5000 pixels on the longest dimension to
# 5000 pixels
ocrmypdf --tesseract-timeout 120 \
    --tesseract-downsample-large-images \
    --tesseract-downsample-above 5000 \
    bigfile.pdf output_downsampled_ocr.pdf
```

### Overriding default tesseract

OCRmyPDF checks the system `PATH` for the `tesseract` binary.

Some relevant environment variables that influence Tesseract's behavior
include:

```{eval-rst}
.. envvar:: TESSDATA_PREFIX

   Overrides the path to Tesseract's data files. This can allow
   simultaneous installation of the "best" and "fast" training data
   sets. OCRmyPDF does not manage this environment variable.

   If you point ``TESSDATA_PREFIX`` at a hand-assembled ``tessdata``
   folder (for example, individual ``.traineddata`` files downloaded
   from tessdata_best), make sure it also contains the ``configs/``
   subdirectory with the ``hocr`` and ``txt`` files. OCRmyPDF requires
   these; without them Tesseract produces no output. See
   :ref:`Tesseract cannot open its config file <tesseract-config-missing>`.
```

```{eval-rst}
.. envvar:: OMP_THREAD_LIMIT

   Controls the number of threads Tesseract will use. OCRmyPDF will
   manage this environment variable if it is not already set.
```

For example, if you have a development build of Tesseract don't wish to
use the system installation, you can launch OCRmyPDF as follows:

```bash
env \
    PATH=/home/user/src/tesseract/api:$PATH \
    TESSDATA_PREFIX=/home/user/src/tesseract \
    ocrmypdf input.pdf output.pdf
```

In this example `TESSDATA_PREFIX` is required to redirect Tesseract to
an alternate folder for its "tessdata" files.

### Overriding other support programs

In addition to tesseract, OCRmyPDF uses the following external binaries:

- `gs` (Ghostscript)
- `unpaper`
- `pngquant`
- `jbig2`

In each case OCRmyPDF will search the `PATH` environment variable to
locate the binaries. By modifying the `PATH` environment variable, you
can override the binaries that OCRmyPDF uses.

### Changing Tesseract configuration variables

You can override Tesseract's default [control
parameters](https://tesseract-ocr.github.io/tessdoc/tess3/ControlParams.html)
with a configuration file.

As an example, this configuration will disable Tesseract's dictionary
for current language. Normally the dictionary is helpful for
interpolating words that are unclear, but it may interfere with OCR if
the document does not contain many words (for example, a list of part
numbers).

Create a file named "no-dict.cfg" with these contents:

```
load_system_dawg 0
language_model_penalty_non_dict_word 0
language_model_penalty_non_freq_dict_word 0
```

then run ocrmypdf as follows (along with any other desired arguments):

```bash
ocrmypdf --tesseract-config no-dict.cfg input.pdf output.pdf
```

:::{warning}
Some combinations of control parameters will break Tesseract or break
assumptions that OCRmyPDF makes about Tesseract's output.
:::

### Changing page segmentation mode

The directive `--tesseract-pagesegmode Nmode` forwards the desired page segmentation
mode to Tesseract OCR. The default is 3.

Page segmentation can improve OCR results when you know that a PDF ought to be
analyzed a particular way, such as PDFs whose pages contain only a single line of
text. For the vast majority of users, changing the page segmentation mode will only
make things worse.

As of June 2024, the Tesseract page segmentation modes are:

| ID  | Description                                                                                   |
| --- | --------------------------------------------------------------------------------------------- |
| 0   | Orientation and script detection (OSD) only.                                                  |
| 1   | Automatic page segmentation with OSD.                                                         |
| 2   | Automatic page segmentation, but no OSD, or OCR. (not implemented)                            |
| 3   | Fully automatic page segmentation, but no OSD. (Default)                                      |
| 4   | Assume a single column of text of variable sizes.                                             |
| 5   | Assume a single uniform block of vertically aligned text.                                     |
| 6   | Assume a single uniform block of text.                                                        |
| 7   | Treat the image as a single text line.                                                        |
| 8   | Treat the image as a single word.                                                             |
| 9   | Treat the image as a single word in a circle.                                                 |
| 10  | Treat the image as a single character.                                                        |
| 11  | Sparse text. Find as much text as possible in no particular order.                            |
| 12  | Sparse text with OSD.                                                                         |
| 13  | Raw line. Treat the image as a single text line, bypassing hacks that are Tesseract-specific. |

Modes 0, 1, 2, and 12 (all of those that enable orientation and script detection)
are not compatible with OCRmyPDF, which performs OSD in a separate step from OCR.
Their use may interfere with `--rotate-pages` and other features.

It is currently not possible to use advanced Tesseract OCR features, such as creating
OCR information, when using Tesseract through OCRmyPDF.

## Choosing a PDF rasterizer

:::{versionadded} 17.0.0
:::

rasterizing

: Converting a PDF page to an image for OCR processing.

OCRmyPDF supports two PDF rasterizers:

| Rasterizer | Package | Advantages | Disadvantages |
|------------|---------|------------|---------------|
| pypdfium2 | Python package | Faster, fewer version issues | Requires pypdfium2 package |
| Ghostscript | System binary | More widely packaged | Version consistency issues, restrictive AGPLv3 |

The `--rasterizer` argument controls which rasterizer is used:

```bash
# Automatic selection (default) - prefers pypdfium when available
ocrmypdf --rasterizer auto input.pdf output.pdf

# Force pypdfium2
ocrmypdf --rasterizer pypdfium input.pdf output.pdf

# Force Ghostscript
ocrmypdf --rasterizer ghostscript input.pdf output.pdf
```

pypdfium2 is a Python binding for pdfium, the PDF rendering library used
by Google Chrome and Chromium. It generally produces output identical to
Ghostscript but with better performance.

:::{note}
If pypdfium2 is not installed and `--rasterizer pypdfium` is requested,
OCRmyPDF will exit with an error. Install it with: `pip install pypdfium2`
:::

## Changing the PDF renderer

rendering

: Creating a new PDF from other data (such as an existing PDF).

:::{versionchanged} 17.0.0
The fpdf2 renderer is now the default, replacing the legacy hOCR renderer.
:::

OCRmyPDF uses PDF renderers to create the invisible text layer. The
renderer may be selected using `--pdf-renderer`. The default is
`auto` which selects `fpdf2`.

### The `fpdf2` renderer (default)

:::{versionadded} 17.0.0
:::

The fpdf2 renderer creates text layers using the fpdf2 library. It provides:

- Full multilingual support including RTL languages (Arabic, Hebrew, Persian)
- Accurate text positioning aligned with OCR bounding boxes
- Improved "Occulta" glyphless font handling:
  - Zero-width markers are properly handled
  - Double-width CJK characters are properly sized
- Direct OcrElement tree input (no hOCR intermediate format required)

The fpdf2 renderer is the recommended choice for all installations.

:::{note}
The fpdf2 renderer may be slightly slower than the legacy hocrtransform
renderer for some workloads. This is an area of ongoing optimization.
:::

In both renderers, a text-only layer is rendered and sandwiched (overlaid)
on to either the original PDF page, or newly rasterized version of the
original PDF page (when `--mode force` is used). In this way, loss
of PDF information is generally avoided. (You may need to disable PDF/A
conversion and optimization to eliminate all lossy transformations.)

### The `sandwich` renderer

The `sandwich` renderer uses Tesseract's text-only PDF feature,
which produces a PDF page that lays out the OCR in invisible text.

Currently some problematic PDF viewers like Mozilla PDF.js and macOS
Preview have problems with segmenting its text output, and
mightrunseveralwordstogether. It also does not implement right to left
fonts (Arabic, Hebrew, Persian). The output of this renderer cannot
be edited. The sandwich renderer is retained for testing.

When image preprocessing features like `--deskew` are used, the
original PDF will be rendered as a full page and the OCR layer will be
placed on top.

### Legacy renderer options

The `hocr` and `hocrdebug` renderer options are deprecated and
automatically redirect to `fpdf2`. They will be removed in a future version.

## Rendering and rasterizing options

:::{versionadded} 14.3.0
:::

The `--continue-on-soft-render-error` option allows OCRmyPDF to
proceed if a page cannot be rasterized/rendered. This is useful if you are
trying to get the best possible OCR from a PDF that is not well-formed,
and you are willing to accept some pages that may not visually match the
input, and that may not OCR well.

## Color conversion strategy

:::{versionadded} 15.0.0
:::

OCRmyPDF uses Ghostscript to convert PDF to PDF/A. In some cases, this
conversion requires color conversion. The default strategy is to convert
using the `LeaveColorUnchanged` strategy, which preserves the original
color space wherever possible (some rare color spaces might still be
converted).

Usually document scanners produce PDFs in the sRGB color space, and do
not need to be converted, so the default strategy is appropriate.

Suppose that you have a document that was prepared for professional
printing in a Separation or CMYK color space, and text was converted to
curves. In this case, you may want to use a different color conversion
strategy. The `--color-conversion-strategy` option allows you to select a
different strategy, such as `RGB`.

## Advanced Ghostscript tuning

:::{versionadded} 17.5.0
:::

OCRmyPDF intentionally hides most Ghostscript controls because Ghostscript
is a legacy code path. The preferred PDF/A pipeline in v17+ uses pypdfium2
as the rasterizer and pikepdf's PDF/A validator to check speculative PDF/A output, with
Ghostscript reserved as a fallback for PDFs that cannot be made compliant
without it. OCRmyPDF's separate optimizer (controlled by `--optimize`,
`--jpeg-quality`, `--png-quality`, etc.) is the supported way to shrink
output PDFs: it gives consistent results across input files, and isolates
Ghostscript so it can focus on producing a PDF/A with as few image
transformations as possible.

The two options below are exposed for advanced users who want to tune
Ghostscript's intermediate PDF/A output directly. Most users will get
more predictable results from the optimizer.

### `--ghostscript-jpeg-quality Q`

Sets Ghostscript's `-dJPEGQ` switch for images that Ghostscript chooses
to recompress to JPEG while building a PDF/A. `Q=0` requests maximum
compression and `Q=100` requests best quality; if the flag is omitted,
OCRmyPDF passes `95` (the historical default). This only affects images
Ghostscript transcodes — existing JPEGs pass through unchanged on modern
Ghostscript releases. For end-to-end JPEG quality tuning, prefer
`--jpeg-quality`, which is implemented by the OCRmyPDF optimizer and is
applied independently of whatever Ghostscript decides to do.

Note: setting both `--ghostscript-jpeg-quality` and `--jpeg-quality` can
result in double JPEG recompression, since the optimizer may re-encode
images that Ghostscript already recompressed. This can degrade quality
in subtle ways.

### `--ghostscript-jpeg-maxdpi DPI`

Enables Ghostscript's image downsampling and caps color, grayscale, and
monochrome image resolution to `DPI`. The downsample threshold is set to
`1.0`, so any image whose effective DPI exceeds the cap will be
downsampled.

Reducing JPEG quality is almost always a better trade than downsampling
at the same compression budget: a 400 DPI JPEG at modest quality usually
looks much better than a 200 DPI JPEG, because the JPEG codec can spend
bits where they count. Downsampling is also dangerous for PDFs that
combine a low-resolution color image with a high-resolution monochrome
mask — capping the mask resolution can produce visible quality loss.
For these reasons, prefer `--jpeg-quality` over `--ghostscript-jpeg-maxdpi`
unless you specifically want to force a hard DPI cap.

Example:

```bash
ocrmypdf --output-type pdfa \
    --ghostscript-jpeg-quality 80 \
    --ghostscript-jpeg-maxdpi 150 \
    in.pdf out.pdf
```

These options only take effect when Ghostscript is invoked for PDF/A
conversion (`--output-type pdfa`, `pdfa-1`, `pdfa-2`, or `pdfa-3`, or
when `--output-type auto` falls back to Ghostscript).

## PDF/A output modes

:::{versionchanged} 17.0.0
The default `--output-type` is now `auto` instead of `pdfa`.
:::

OCRmyPDF can produce PDF/A compliant output for long-term archival. The
`--output-type` argument controls PDF/A conversion:

| Output type | Behavior |
|-------------|----------|
| `auto` | Best-effort PDF/A without requiring Ghostscript (default) |
| `pdfa` | PDF/A-2b via Ghostscript |
| `pdfa-1` | PDF/A-1b via Ghostscript |
| `pdfa-2` | PDF/A-2b via Ghostscript (same as `pdfa`) |
| `pdfa-3` | PDF/A-3b via Ghostscript |
| `pdf` | Standard PDF, no PDF/A conversion |
| `none` | No output file (useful with `--sidecar`) |

### Non-embedded fonts and PDF/A

:::{versionadded} 17.8.0
OCRmyPDF now refuses to corrupt non-embedded CID text layers during PDF/A
conversion.
:::

PDF/A requires every font to be embedded. If your input already has a text
layer that uses *non-embedded* CID fonts — most commonly a CJK
(Chinese-Japanese-Korean) OCR layer
produced by Adobe Acrobat, which relies on the reader's system fonts —
Ghostscript would have to substitute and re-embed a replacement font to make
the file PDF/A. For CID-keyed (CJK) fonts this routinely corrupts the
character-to-Unicode mapping, so the text silently becomes garbage or stops
being searchable even though the page still *looks* correct.

Rather than emit corrupted output, OCRmyPDF detects this situation and:

- with `--output-type auto` (the default), produces a regular PDF instead of
  PDF/A, preserving the existing text layer exactly;
- with an explicit `--output-type pdfa` (or `pdfa-1`/`pdfa-2`/`pdfa-3`), stops
  with an error.

This is a Ghostscript limitation that OCRmyPDF cannot repair, because a
non-embedded font cannot be made PDF/A-compliant without re-embedding it. To
keep the existing text layer, use `--output-type pdf`. To produce PDF/A anyway,
re-run OCR with `--force-ocr`, which discards the original text layer and
rebuilds it with embedded fonts. Text layers whose fonts are *already embedded*
are converted to PDF/A normally.

### Speculative PDF/A conversion

:::{versionadded} 17.0.0
:::

:::{versionchanged} 17.13.0
Speculative conversion is checked by pikepdf's PDF/A validator
(`pikepdf.pdfa`) instead of veraPDF, so it no longer depends on veraPDF
being installed, and it now works for PDF/A-1b as well as PDF/A-2b and
PDF/A-3b.
:::

For all PDF/A output types, including the default `--output-type auto`,
OCRmyPDF first attempts a fast "speculative" PDF/A conversion that avoids
Ghostscript when possible:

1. Using `pikepdf.pdfa`, OCRmyPDF replaces the output intents with an sRGB ICC
   profile, removes image interpolation flags, removes annotations that are
   hidden or not viewable (as Ghostscript does, with a warning) and sets the
   Print flag on the others, adds the `/CIDSet` that PDF/A-1 requires
   for subset CID fonts, removes XMP metadata properties that PDF/A does not
   permit, and adds the PDF/A identification to the XMP metadata.
2. pikepdf's PDF/A validator checks the file as written, and OCRmyPDF
   checks the final output file again after its metadata and optimization
   steps, which rewrite it.
3. If validation passes, Ghostscript is skipped entirely.
4. If validation fails, OCRmyPDF falls back to Ghostscript.

pikepdf's validator is deliberately conservative: it approves only
constructs it recognizes and knows to conform. A file with a violation
fails; a file containing constructs the validator does not check is *not
checked*, which OCRmyPDF also treats as a reason to use Ghostscript. It is
not a replacement for a reference PDF/A validator such as veraPDF. Run
`ocrmypdf -v1` to see why a file was not approved. See pikepdf's
documentation of `pikepdf.pdfa` for what the validator covers.

This fast path avoids some Ghostscript limitations (such as image
transcoding) and is used whenever it can produce valid PDF/A. When it
cannot, because the input needs real conversion (for example, fonts that
are not embedded, or transparency in PDF/A-1), `auto` falls back to
Ghostscript so that it still produces PDF/A by default, matching OCRmyPDF 16
and earlier. If even Ghostscript cannot safely produce PDF/A, `auto` outputs
a regular PDF instead of failing.

### Choosing the PDF/A backend

:::{versionadded} 17.13.0
:::

`--pdfa-backend` selects how PDF/A is produced:

- `auto` (the default) tries speculative conversion first and falls back to
  Ghostscript, as described above.
- `internal` uses speculative conversion only and never uses Ghostscript to
  make PDF/A. JPEG images pass through unchanged and the result is checked
  by pikepdf's validator. If the validator does not approve the file,
  `--output-type pdfa` (or `pdfa-1`, `pdfa-2`, `pdfa-3`) fails with exit
  code 10 (`pdfa_conversion_failed`) and logs the validator's findings;
  `--output-type auto` outputs a regular PDF instead. Ghostscript need not
  be installed for PDF/A output.
- `ghostscript` always converts with Ghostscript, which re-encodes images
  and converts colours as needed, and requires Ghostscript to be installed.

The options that only Ghostscript applies, `--pdfa-image-compression`
(other than `auto`), `--ghostscript-jpeg-quality` and
`--ghostscript-jpeg-maxdpi`, select the Ghostscript backend when
`--pdfa-backend` is `auto`, and are an error with `--pdfa-backend internal`.
OCRmyPDF does no colour conversion of its own, so the same applies to
`--color-conversion-strategy` `CMYK`, `Gray` and `UseDeviceIndependentColor`;
`LeaveColorUnchanged` (the default) and `RGB` need no conversion in any file
pikepdf's validator approves, so they work with every backend.

```bash
# PDF/A without Ghostscript, or fail
ocrmypdf --output-type pdfa --pdfa-backend internal input.pdf output.pdf

# PDF/A from Ghostscript, as in OCRmyPDF 16 and earlier
ocrmypdf --output-type pdfa --pdfa-backend ghostscript input.pdf output.pdf
```

### PDF/A conversion flow

The following diagram illustrates the PDF/A conversion decision tree:

```{mermaid}
flowchart TD
    A[Start] --> B{--output-type?}
    B -->|pdf| C[Output standard PDF]
    B -->|pdfa/pdfa-N/auto| G{--pdfa-backend?}
    G -->|auto/internal| E[Attempt speculative conversion]
    G -->|ghostscript| D

    E --> F["Repair and declare PDF/A (pikepdf)"]
    F --> I[Validate with pikepdf]

    I --> J{Validation passed?}
    J -->|Yes| K[Output PDF/A - Ghostscript skipped]
    J -->|No, internal + pdfa| Y[Error: PDF/A conversion failed]
    J -->|No, internal + auto| L
    J -->|No, auto backend| H{Ghostscript available?}

    H -->|Yes| D[Ghostscript PDF/A conversion]
    H -->|No, auto| L[Output standard PDF]
    H -->|No, pdfa| X[Error: Ghostscript required]

    D --> N[Output PDF/A]

    style K fill:#90EE90
    style N fill:#90EE90
    style L fill:#FFB6C1
```

:::{warning}
**Breaking change:** If Ghostscript is not installed and the file does not
pass speculative conversion, `--output-type auto` will produce a standard
PDF instead of PDF/A. This is a change from previous versions where
Ghostscript was required and PDF/A was always produced.
:::

## Return code policy

OCRmyPDF writes all messages to `stderr`. `stdout` is reserved for
piping output files. `stdin` is reserved for piping input files.

The return codes generated by the OCRmyPDF are considered part of the
stable user interface. They may be imported from
`ocrmypdf.exceptions`.

```
/* Detailed source-code truncated for AI context efficiency. */
```

(tmpdir)=
## Changing temporary storage location

OCRmyPDF generates many temporary files during processing.

To change where temporary files are stored, change the `TMPDIR`
environment variable for ocrmypdf's environment. (Python's
`tempfile.gettempdir()` returns the root directory in which temporary
files will be stored.) For example, one could redirect `TMPDIR` to a
large RAM disk to avoid wear on HDD/SSD and potentially improve
performance.

On Windows, the `TEMP` environment variable is used instead.

## Debugging the intermediate files

OCRmyPDF normally saves its intermediate results to a temporary folder
and deletes this folder when it exits, whether it succeeded or failed.

If the `--keep-temporary-files` (`-k`) argument is issued on the
command line, OCRmyPDF will keep the temporary folder and print the location,
whether it succeeded or failed. An example message is:

```none
Temporary working files retained at:
/tmp/ocrmypdf.io.u20wpz07
```

When OCRmyPDF is launched as a snap, this corresponds to the snap filesystem, for instance:

> /tmp/snap-private-tmp/snap.ocrmypdf/tmp/ocrmypdf.io.u20wpz07

The organization of this folder is an implementation detail and subject
to change between releases. However the general organization is that
working files on a per page basis have the page number as a prefix
(starting with page 1), an infix indicates the processing stage, and a
suffix indicates the file type. Some important files include:

- `_rasterize.png` - what the input page looks like
- `_ocr.png` - the file that is sent to Tesseract for OCR; depending
  on arguments this may differ from the presentation image
- `_pp_deskew.png` - the image, after deskewing
- `_pp_clean.png` - the image, after cleaning with unpaper
- `_ocr_hocr.pdf` - the OCR file; appears as a blank page with invisible
  text embedded
- `_ocr_hocr.txt` - the OCR text (not necessarily all text on the page,
  if the page is mixed format)
- `fix_docinfo.pdf` - a temporary file created to fix the PDF DocumentInfo
  data structure
- `graft_layers.pdf` - the rendered PDF with OCR layers grafted on
- `pdfa.pdf` - `graft_layers.pdf` after conversion to PDF/A
- `pdfa.ps` - a PostScript file used by Ghostscript for PDF/A conversion
- `optimize.pdf` - the PDF generated before optimization
- `optimize.out.pdf` - the PDF generated by optimization
- `origin` - the input file
- `origin.pdf` - the input file or the input image converted to PDF
- `images/*` - images extracted during the optimization process; here
  the prefix indicates a PDF object ID not a page number

---

### Api (`docs/api.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Using the OCRmyPDF API

OCRmyPDF originated as a command line program and continues to have this
legacy, but parts of it can be imported and used in other Python
applications.

Some applications may want to consider running ocrmypdf from a
subprocess call anyway, as this provides isolation of its activities.

## Example

OCRmyPDF provides one high-level function to run its main engine from an
application.

```{versionchanged} 17.0
The {func}`ocrmypdf.ocr` function now accepts an {class}`~ocrmypdf.OcrOptions`
object as its first argument, providing a cleaner API with full type hints
and validation. The previous positional argument style remains supported.
```

### Modern API (recommended)

The recommended way to call {func}`ocrmypdf.ocr` is to construct an
{class}`~ocrmypdf.OcrOptions` object with all settings, then pass it
as the sole argument:

```python
import ocrmypdf
from ocrmypdf import OcrOptions

if __name__ == '__main__':  # To ensure correct behavior on Windows and macOS
    options = OcrOptions(
        input_file='input.pdf',
        output_file='output.pdf',
        deskew=True,
        languages=['eng'],
    )
    ocrmypdf.ocr(options)
```

{class}`~ocrmypdf.OcrOptions` is a Pydantic model that provides:

- Full type hints and IDE autocompletion
- Validation of option values at construction time
- Clear documentation of all available options

```{versionadded} 17.0
The {class}`~ocrmypdf.OcrOptions` class is now exported from the top-level
`ocrmypdf` module.
```

### Legacy API

For compatibility with OCRmyPDF < v17, the traditional calling style
with positional arguments is still fully supported:

```python
import ocrmypdf

if __name__ == '__main__':  # To ensure correct behavior on Windows and macOS
    ocrmypdf.ocr('input.pdf', 'output.pdf', deskew=True)
```

With this style, all of the command line arguments are available
and may be passed as equivalent keywords.

A few differences are that `verbose` and `quiet` are not available.
Instead, output should be managed by configuring logging.

### Parent process requirements

The {func}`ocrmypdf.ocr` function runs OCRmyPDF similar to command line
execution. To do this, it will:

- create worker processes or threads
- manage the signal flags of its worker processes
- execute other subprocesses (forking and executing other programs)

The Python process that calls {func}`ocrmypdf.ocr()` must be sufficiently
privileged to perform these actions.

There currently is no option to manage how jobs are scheduled other
than the argument `jobs=` which will limit the number of worker
processes.

Creating a child process to call {func}`ocrmypdf.ocr()` is suggested. That
way your application will survive and remain interactive even if
OCRmyPDF fails for any reason. For example:

```python
from multiprocessing import Process
import ocrmypdf
from ocrmypdf import OcrOptions


def ocrmypdf_process():
    options = OcrOptions(input_file='input.pdf', output_file='output.pdf')
    ocrmypdf.ocr(options)


def call_ocrmypdf_from_my_app():
    p = Process(target=ocrmypdf_process)
    p.start()
    p.join()
```

Programs that call {func}`ocrmypdf.ocr()` should also install a SIGBUS signal
handler (except on Windows), to raise an exception if access to a memory
mapped file fails. OCRmyPDF may use memory mapping.

Several OCR jobs may run concurrently in the same Python interpreter process.

Installing a plugin set mutates interpreter-global state, because of how OCRmyPDF's
plugins and Python's library import system work, and a running job cannot survive
that state being replaced underneath it. {func}`ocrmypdf.ocr()` therefore takes a
readers-writer lock: installing a plugin set takes it exclusively, so it waits for
every job already running, while a job holds it shared for its own run. A plugin set
is installed once per interpreter and reused thereafter, so a job that asks for a
plugin set some earlier job already installed never blocks.

The practical consequence is that concurrent jobs should all use the same plugin
set. Jobs requesting different plugin sets serialize against each other. They must
also use the same `max_image_mpixels`, since Pillow's decompression-bomb limit is
interpreter-global and the last job to set it wins. If you need to run jobs with
differing configurations, use separate processes.

Because a plugin set is installed only once, a plugin module given as a file path is
executed once per interpreter rather than once per call. Plugins must not rely on
being re-executed for each job, and must not store per-job state on the plugin
manager, which concurrent jobs may share.

Note that N concurrent jobs each configured with `jobs=M` may spawn up to N*M
workers, so size `jobs` accordingly.

:::{warning}
On Windows and macOS, the script that calls {func}`ocrmypdf.ocr()` must be
protected by an "ifmain" guard (`if __name__ == '__main__'`). If you do
not take at least one of these steps, process semantics will prevent
OCRmyPDF from working correctly.
:::

### Logging

OCRmyPDF will log under loggers named `ocrmypdf`. In addition, it
imports `pdfminer` and `PIL`, both of which post log messages under
those logging namespaces.

You can configure the logging as desired for your application or call
{func}`ocrmypdf.configure_logging` to configure logging the same way
OCRmyPDF itself does. The command line parameters such as `--quiet`
and `--verbose` have no equivalents in the API; you must use the
provided configuration function or do configuration in a way that suits
your use case.

### Progress monitoring

OCRmyPDF uses the `rich` package to implement its progress bars.
{func}`ocrmypdf.configure_logging` will set up logging output to
`sys.stderr` in a way that is compatible with the display of the
progress bar. Use `ocrmypdf.ocr(...progress_bar=False)` to disable
the progress bar.

### Standard output

OCRmyPDF is strict about not writing to standard output so that
users can safely use it in a pipeline and produce a valid output
file. A caller application will have to ensure it does not write to
standard output either, if it wants to be compatible with this
behavior and support piping to a file. Another benefit of running
OCRmyPDF in a child process, as recommended above, is that it will
not interfere with the parent process's standard output.

### Exceptions

OCRmyPDF may throw standard Python exceptions, `ocrmypdf.exceptions.*`
exceptions, some exceptions related to multiprocessing, and
{exc}`KeyboardInterrupt`. The parent process should provide an exception
handler. OCRmyPDF will clean up its temporary files and worker processes
automatically when an exception occurs.

When OCRmyPDF succeeds conditionally, it returns an integer exit code.

### Bundling with PyInstaller

OCRmyPDF ships a PyInstaller hook, which PyInstaller finds automatically,
so applications that call {func}`ocrmypdf.ocr` can be frozen without extra
`--collect-*` options. The hook bundles OCRmyPDF's builtin plugins, fonts
and color profiles, and the pikepdf data needed for PDF/A output.

The external programs are not bundled. Tesseract and Ghostscript must still
be installed on the target machine and be on the `PATH`, as must any optional
tools you use, such as unpaper or pngquant. Third party plugins must be
bundled by your application.

Like any frozen application that uses multiprocessing, call
{func}`multiprocessing.freeze_support` at the start of your
`if __name__ == '__main__':` block.

### Plugin Development Changes

```{versionchanged} 16.13
Plugin hooks now receive {class}`~ocrmypdf.OcrOptions` objects instead of
`argparse.Namespace`.
```

- {class}`~ocrmypdf.OcrOptions` provides the same attribute access as `Namespace` (duck-typing compatible)
- Plugin developers should update type hints: `from ocrmypdf import OcrOptions`
- Built-in plugins no longer modify options in-place for better immutability

Most existing plugins will continue working without modification due to the
duck-typing compatibility between {class}`~ocrmypdf.OcrOptions` and `Namespace`.

---

### Apiref (`docs/apiref.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# API reference

This page summarizes the rest of the public API. Generally speaking this
should be mainly of interest to plugin developers.

## ocrmypdf.api

```{eval-rst}
.. automodule:: ocrmypdf.api
    :members:
```

## ocrmypdf._options

```{eval-rst}
.. automodule:: ocrmypdf._options
    :members: OcrOptions
```

## ocrmypdf.exceptions

```{eval-rst}
.. automodule:: ocrmypdf.exceptions
    :members:
    :undoc-members:
```

## ocrmypdf.helpers

```{eval-rst}
.. automodule:: ocrmypdf.helpers
    :members:
```

## ocrmypdf.hocrtransform

```{eval-rst}
.. automodule:: ocrmypdf.hocrtransform
    :members:
```

## ocrmypdf.pdfa

```{eval-rst}
.. automodule:: ocrmypdf.pdfa
    :members:
```

## ocrmypdf.quality

```{eval-rst}
.. automodule:: ocrmypdf.quality
    :members:
```

## ocrmypdf.subprocess

```{eval-rst}
.. automodule:: ocrmypdf.subprocess
    :members:
```

---

### Batch (`docs/batch.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

Batch processing
================

This article provides information about running OCRmyPDF on multiple
files or configuring it as a service triggered by file system events.

Batch jobs
----------

Consider using the excellent [GNU
Parallel](https://www.gnu.org/software/parallel/) to apply OCRmyPDF to
multiple files at once.

Both `parallel` and `ocrmypdf` will try to use all available processors.
To maximize parallelism without overloading your system with processes,
consider using `parallel -j 2` to limit parallel to running two jobs at
once.

This command will run `ocrmypdf` on all files named `*.pdf` in the
current directory and write them to the previously created `output/`
folder. It will not search subdirectories.

The `--tag` argument tells parallel to print the filename as a prefix
whenever a message is printed, so that one can trace any errors to the
file that produced them.

:::{code} bash
parallel --tag -j 2 ocrmypdf '{}' 'output/{}' ::: *.pdf
:::

OCRmyPDF automatically repairs PDFs before parsing and gathering
information from them.

Directory trees
---------------

This will walk through a directory tree and run OCR on all files in
place, and printing each filename in between runs:

:::{code} bash
find . -name '*.pdf' -printf '%p\n' -exec ocrmypdf '{}' '{}' \;
:::

This only runs one `ocrmypdf` process at a time. This variation uses
`find` to create a directory list and `parallel` to parallelize runs of
`ocrmypdf`, again updating files in place.

:::{code} bash
find . -name '*.pdf' | parallel --tag -j 2 ocrmypdf '{}' '{}'
:::

In a Windows batch file, use

:::{code} bat
for /r %%f in (*.pdf) do ocrmypdf %%f %%f
:::

With a Docker container, you will need to stream through standard input
and output:

:::{code} bash
find . -name '*.pdf' -print0 | xargs -0 | while read pdf; do
    pdfout=$(mktemp)
    docker run --rm -i jbarlow83/ocrmypdf - - <$pdf >$pdfout && cp $pdfout $pdf
done
:::

### Sample script

This user contributed script also provides an example of batch
processing.

:::{literalinclude} ../misc/batch.py
---
caption: misc/batch.py
---
:::

### Synology DiskStations

Synology DiskStations (Network Attached Storage devices) can run the
Docker image of OCRmyPDF if the Synology [Docker
package](https://www.synology.com/en-global/dsm/packages/Docker) is
installed. Attached is a script to address particular quirks of using
OCRmyPDF on one of these devices.

At the time this script was written, it only worked for x86-based
Synology products. It is not known if it will work on ARM-based Synology
products. Further adjustments might be needed to deal with the
Synology\'s relatively limited CPU and RAM.

:::{literalinclude} ../misc/synology.py
---
caption: misc/synology.py - Sample script for Synology DiskStations
---
:::

### Huge batch jobs

If you have thousands of files to work with, contact the author.
Consulting work related to OCRmyPDF helps fund this open source project
and all inquiries are appreciated.

Hot (watched) folders
---------------------

### Watched folders with watcher.py

OCRmyPDF has a folder watcher called watcher.py, which is currently
included in source distributions but not part of the main program. It
may be used natively or may run in a Docker container. Native instances
tend to give better performance. watcher.py works on all platforms.

Users may need to customize the script to meet their requirements.

:::{code} bash
# Using uv (recommended)
uv sync --extra watcher

# Or using pip
pip3 install ocrmypdf[watcher]

env OCR_INPUT_DIRECTORY=/mnt/input-pdfs \
    OCR_OUTPUT_DIRECTORY=/mnt/output-pdfs \
    OCR_OUTPUT_STRUCTURE=YEAR_MONTH \
    python3 watcher.py
:::

:::{list-table} watcher.py environment variables
---
header-rows: 1
---

* - Environment variable
  - Description
* - OCR\_INPUT\_DIRECTORY
  - Set input directory to monitor (recursive)
* - OCR\_OUTPUT\_DIRECTORY
  - Set output directory (should not be under input)
* - OCR\_ARCHIVE\_DIRECTORY
  - Set archive directory for processed originals (should not be under input, requires `OCR_ON_SUCCESS_ARCHIVE` to be set)
* - OCR\_ON\_SUCCESS\_DELETE
  - Delete the input file if the exit code is 0 (OK). Takes precedence over `OCR_ON_SUCCESS_ARCHIVE` if both are set.
* - OCR\_ON\_SUCCESS\_ARCHIVE
  - Move the processed original file to `OCR_ARCHIVE_DIRECTORY` if the exit code is 0 (OK). Note that `OCR_ON_SUCCESS_DELETE` takes precedence over this option, i.e. if both options are set, the input file will be deleted.
* - OCR\_OUTPUT\_STRUCTURE (`--output-structure`)
  - How output (and archived) files are laid out under their destination
    directory. One of `FLAT` (default; all files directly in the
    destination directory), `YEAR_MONTH` (`{destination}/{year}/{month}/{filename}`),
    or `HIERARCHY` (mirrors the input directory tree, e.g.
    `input/a/b/c.pdf` → `{destination}/a/b/c.pdf`). Applies to both
    `OCR_OUTPUT_DIRECTORY` and `OCR_ARCHIVE_DIRECTORY`.
* - OCR\_ON\_CONFLICT (`--on-conflict`)
  - What to do when the intended destination file already exists. One of
    `SUFFIX` (default; write `name (1).pdf`, `name (2).pdf`, ... in the OS
    style), `SKIP` (log and leave the file unprocessed), or `OVERWRITE`
    (replace the existing file, the previous default behavior). Applies to
    both `OCR_OUTPUT_DIRECTORY` and `OCR_ARCHIVE_DIRECTORY`.
* - OCR\_OUTPUT\_DIRECTORY\_YEAR\_MONTH
  - **Deprecated** in favor of `OCR_OUTPUT_STRUCTURE=YEAR_MONTH`. Still
    honored, and logs a deprecation warning; if both variables are set,
    `OCR_OUTPUT_STRUCTURE` takes precedence.
* - OCR\_DESKEW
  - Apply deskew to crooked input PDFs
* - OCR\_JSON\_SETTINGS
  - A JSON string specifying any other arguments for `ocrmypdf.ocr`, e.g. `'OCR_JSON_SETTINGS={"rotate_pages": true, "optimize": "3"}'`.
* - OCR\_POLL\_NEW\_FILE\_SECONDS
  - Polling interval
* - OCR\_LOGLEVEL
  - Level of log messages (DEBUG, INFO, WARNING, ERROR, CRITICAL)
:::

One could configure a networked scanner or scanning computer to drop
files in the watched folder.

Output and archive filenames, and any directory components generated by
`OCR_OUTPUT_STRUCTURE`, are sanitized for compatibility with the output
filesystem: characters that are illegal on Windows or SMB shares
(`<>:"/\|?*` and control characters) are replaced with `_`, trailing dots
and spaces are stripped, and reserved DOS device names (`CON`, `PRN`,
`AUX`, `NUL`, `COM1`-`COM9`, `LPT1`-`LPT9`) are prefixed with `_`. This is
done unconditionally because the output directory may sit on a more
restrictive filesystem (e.g. an SMB share) than the input side, even when
the watcher itself runs on Linux or macOS.

:::{note}
If you write output to an SMB share and mount it on a Windows client, newly
written files may not appear immediately in Windows Explorer or `dir`. This
is the SMB client's own directory and file-info cache
(`DirectoryCacheLifetime` / `FileInfoCacheLifetime`, ~10 seconds by
default), not a delay introduced by the watcher. It can be tuned on the
client, e.g. with `Set-SmbClientConfiguration`; the watcher deliberately
does not attempt server-side workarounds for a client-side cache.
:::

### Watcher security model

`watcher.py` treats the input, output and archive directories as *data
only* — they may be writable by less-privileged users, but code must never
live in or be executed from them (a "Harvard architecture"). To enforce this,
the watcher validates its configuration at startup and refuses to run
(exiting with code 9, *invalid configuration*) if any of these conditions is
violated:

* Any of the input, output or archive directories overlaps a Python
  interpreter path — `sys.executable`, `sys.path`, the virtual environment,
  the site-packages directories, or any directory on `$PATH`. This prevents
  an attacker who can write to a data directory from modifying the interpreter
  or the code that runs.
* `OCR_JSON_SETTINGS` names a file that lives inside a data directory, or that
  is group- or world-writable. Keep the settings file outside the data
  directories and restrict it to owner-only write access (e.g. `chmod 600`).
* `OCR_JSON_SETTINGS` specifies a plugin (a `.py` path) located inside a data
  directory.

* The output or archive directory is the input directory or a subdirectory of
  it. Because the input directory is watched recursively, OCRmyPDF output (and
  archived originals) written there would be detected as new input and
  reprocessed endlessly. Keep the output and archive directories outside the
  input directory.

At runtime the watcher also refuses to follow symlinks or process non-regular
files (such as fifos or devices) in the input directory, and refuses to write
output onto a destination occupied by a non-regular file.

### Watched folders with Docker

The watcher service is included in the OCRmyPDF Docker image. To run it:

:::{code} bash
docker run \
    --volume <path to files to convert>:/input \
    --volume <path to store results>:/output \
    --volume <path to store processed originals>:/processed \
    --env OCR_OUTPUT_STRUCTURE=YEAR_MONTH \
    --env OCR_ON_SUCCESS_ARCHIVE=1 \
    --env OCR_DESKEW=1 \
    --env PYTHONUNBUFFERED=1 \
    --interactive --tty --entrypoint python3 \
    jbarlow83/ocrmypdf \
    /app/watcher.py
:::

:::{note}
The image runs as the non-root `app` user (uid 1000) by default, so it
may not be able to write to the `/output` and `/processed` volumes unless
you add a `--user` argument. The correct value depends on whether you use
rootful Docker, rootless Docker, or Podman -- see
{ref}`Bind-mounted volumes <docker-volumes>` for details.
:::

This service will watch for a file that matches `/input/\*.pdf`, convert
it to a OCRed PDF in `/output/`, and move the processed original to
`/processed`. The parameters to this image are:

:::{list-table} Watcher Docker Parameters
:header-rows: 1

* - Parameter
  - Description
* - `--volume <path to files to convert>:/input`
  - Files placed in this location will be OCRed
* - `--volume <path to store results>:/output`
  - This is where OCRed files will be stored
* - `--volume <path to store processed originals>:/processed`
  - Archive processed originals here
* - `--env OCR_OUTPUT_STRUCTURE=YEAR_MONTH`
  - Define environment variable `OCR_OUTPUT_STRUCTURE` to lay out output
    (and archived) files as `FLAT` (default), `YEAR_MONTH`
    (`{destination}/{year}/{month}/{filename}`), or `HIERARCHY` (mirrors
    the input directory tree). Replaces the deprecated
    `OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1`.
* - `--env OCR_ON_CONFLICT=SUFFIX`
  - Define environment variable `OCR_ON_CONFLICT` to control what happens
    when a destination file already exists: `SUFFIX` (default, appends
    " (1)", " (2)", ...), `SKIP`, or `OVERWRITE`
* - `--env OCR_ON_SUCCESS_ARCHIVE=1`
  - Define environment variable `OCR_ON_SUCCESS_ARCHIVE` to move processed originals
* - `--env OCR_DESKEW=1`
  - Define environment variable `OCR_DESKEW` to apply deskew to crooked input PDFs
* - `--env PYTHONBUFFERED=1`
  - This will force `STDOUT` to be unbuffered and allow you to see messages in docker logs
* - `--env OCR_LOGLEVEL='DEBUG'`
  - Level of log messages
* - `--env OCR_JSON_SETTINGS={"language":"deu+eng", "rotate_pages": true}`
  - A JSON string specifying any other arguments for `ocrmypdf.ocr`
:::

This service uses native operating system filesystem notifications to detect
changes. On some environments, such as filesystems shared on a slow network,
these notifications may be unreliable; set `OCR_USE_POLLING=1` to fall back to
polling in that case.

A configuration manager such as Docker Compose could be used to ensure
that the service is always available.

:::{literalinclude} ../misc/docker-compose.example.yml
---
caption: misc/docker-compose.example.yml
---
:::

### Caveats

-   `watchmedo` may not work properly on a networked file system,
    depending on the capabilities of the file system client and server.
-   This simple recipe does not filter for the type of file system
    event, so file copies, deletes and moves, and directory operations,
    will all be sent to ocrmypdf, producing errors in several cases.
    Disable your watched folder if you are doing anything other than
    copying files to it.
-   If the source and destination directory are the same, watchmedo may
    create an infinite loop.
-   On BSD, FreeBSD and older versions of macOS, you may need to
    increase the number of file descriptors to monitor more files, using
    `ulimit -n 1024` to watch a folder of up to 1024 files.

### Alternatives

-   On Linux, [systemd user
    services](https://wiki.archlinux.org/index.php/Systemd/User) can be
    configured to automatically perform OCR on a collection of files.
-   [Watchman](https://facebook.github.io/watchman/) is a more powerful
    alternative to `watchmedo`.

macOS Automator
---------------

You can use the Automator app with macOS, to create a Workflow or Quick
Action. Use a *Run Shell Script* action in your workflow. In the context
of Automator, the `PATH` may be set differently your Terminal\'s `PATH`;
you may need to explicitly set the PATH to include `ocrmypdf`. The
following example may serve as a starting point:



You may customize the command sent to ocrmypdf.

---

### Cloud (`docs/cloud.md`)

% SPDX-FileCopyrightText: 2025 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

(ocr-service)=

# Online deployments

OCRmyPDF is designed to be used as a command line tool, but it can be
used in a web service. This document describes some considerations for
doing so.

A basic web service implementation is provided in the source code
repository, as `misc/webservice.py`. It is only demonstration quality
and is not intended for production use.

OCRmyPDF is not designed for use as a public web service where a
malicious user could upload a chosen PDF. In particular, it is not
necessarily secure against PDF malware or PDFs that cause denial of
service. For further discussino of security, see
[security](security).

OCRmyPDF relies on Ghostscript, and therefore, if deployed online one
should be prepared to comply with Ghostscript\'s Affero GPL license, and
any other licenses.

Setting aside these concerns, a side effect of OCRmyPDF is that it may
incidentally sanitize PDFs containing certain types of malware. It
repairs the PDF with pikepdf/libqpdf, which could correct malformed PDF
structures that are part of an attack. When PDF/A output is selected
(the default), the input PDF is partially reconstructed by Ghostscript.
When `--force-ocr` is used, all pages are rasterized and reconverted to
PDF, which could remove malware in embedded images.

## Limiting CPU usage

OCRmyPDF will attempt to use all available CPUs and storage, so
executing `nice ocrmypdf` or limiting the number of jobs with the
`--jobs` argument may ensure the server remains responsive. Another
option would be to run OCRmyPDF jobs inside a Docker container, a
virtual machine, or a cloud instance, which can impose its own limits on
CPU usage and be terminated \"from orbit\" if it fails to complete.

## Temporary storage requirements

OCRmyPDF will use a large amount of temporary storage for its work,
proportional to the total number of pixels needed to rasterize the PDF.
The raster image of a 8.5×11\" color page at 300 DPI takes 25 MB
uncompressed; OCRmyPDF saves its intermediates as PNG, but that still
means it requires about 9 MB per intermediate based on average
compression ratios. Multiple intermediates per page are also required,
depending on the command line given. A rule of thumb would be to allow
100 MB of temporary storage per page in a file -- meaning that a small
cloud servers or small VM partitions should be provisioned with plenty
of extra space, if say, a 500 page file might be sent.

To change the temporary directory, see [tmpdir](#tmpdir).

On Amazon Web Services or other cloud vendors, consider setting your
temporary directory to [empheral
storage](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html).

## Timeouts

To prevent excessively long OCR jobs consider setting
`--tesseract-timeout` and/or `--skip-big` arguments. `--skip-big` is
particularly helpful if your PDFs include documents such as reports on
standard page sizes with large images attached - often large images are
not worth OCR\'ing anyway.

## Document management systems

If you are looking for a full document management system, consider
[paperless-ngx](https://github.com/paperless-ngx/paperless-ngx), which
is a web application that uses OCRmyPDF to automatically OCR and archive
documents.

## Commercial OCR alternatives

The author also provides professional services that include OCR and
building databases around PDFs, and is happy to provide consultation.

Abbyy Cloud OCR is viable commercial alternative with a web services
API. Amazon Textract, Google Cloud Vision, and Microsoft Azure Computer
Vision provide advanced OCR but have less PDF rendering capability.

---

### Contributing (`docs/contributing.md`)

% SPDX-FileCopyrightText: 2025 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Contributing guidelines

Contributions are welcome!

## Big changes

Please open a new issue to discuss or propose a major change. Not only
is it fun to discuss big ideas, but we might save each other\'s time
too. Perhaps some of the work you\'re contemplating is already half-done
in a development branch.

## AI-assisted contributions 🤖

AI-assisted contributions are permitted. Proactive disclosure of AI use
is recommended but not required; it's usually not hard to tell.

When reporting issues with AI assistance, we encourage you to
*stick to the facts* from a *user perspective*. Use the issue template,
and provide a reproducer.

AI speculation about how to solve an issue is unhelpful. Leave it out
of your issue report. If you contribute a full PR, you're
likely to get a solution that gets the details right.

We will dismiss "vibe coded" PRs that don't appear to understand the
project.

## Code style

We use `ruff` for code formatting.
The settings for these programs are in `pyproject.toml`. Pull requests
should follow the style guide. One difference we use from \"black\"
style is that strings shown to the user are always in double quotes
(`"`) and strings for internal uses are in single quotes (`'`).

## Tests

New features should come with tests that confirm their correctness.

### Tesseract OCR cache

Running Tesseract on every test would make the test suite far too slow, so
the results of most Tesseract invocations are recorded in `tests/cache/`
and replayed by the `tests/plugins/tesseract_cache.py` plugin. The cache is
committed to the repository, so a contributor with a different (or no)
Tesseract still gets the same OCR output.

Entries live at
`tests/cache/v<CACHE_VERSION>/<key>/<argv-slug>/{stdout,stderr,hocr,pdf,txt}.bin`.
The key for a file in `tests/resources/` is its stem plus the first 8 hex
digits of the SHA-256 of its contents; inputs generated during a test run
are keyed by stem alone, because their bytes vary from run to run.
`CACHE_INFO.json` records the Tesseract version that produced the tree and
`manifest.jsonl` logs each entry as it is created. Neither is touched on a
cache hit, so a normal test run leaves the working tree clean.

There are three ways the cache can go stale, and only one of them needs
your attention:

- **A test resource changed.** Nothing to do. The digest in the cache key
  changes with the file, so affected entries are regenerated automatically
  and the old ones can be deleted. `tests/test_tesseract_cache.py` fails if
  a digest-keyed folder no longer matches any resource, which is your cue
  to delete the orphaned folder.
- **OCRmyPDF changed the images it sends to Tesseract** (preprocessing,
  rasterization, DPI selection, image optimization on the OCR path). The
  cached answers now belong to different questions, and the cache would
  silently replay them. Bump `CACHE_VERSION` in
  `tests/plugins/tesseract_cache.py`, delete the old `tests/cache/v<N>/`
  tree, and regenerate.
- **Tesseract itself changed.** `pytest_sessionstart` in `tests/conftest.py`
  compares the installed Tesseract's major.minor version against
  `CACHE_INFO.json` and warns on a mismatch. Set
  `OCRMYPDF_TEST_CACHE_STRICT=1` to turn that warning into an immediate
  session abort, which is useful in CI. A mismatch is not automatically a
  problem — it means the cache no longer reflects what your Tesseract would
  produce, so regenerate before trusting or updating it.

To regenerate the cache:

```bash
rm -rf tests/cache/v1  # or whatever the current CACHE_VERSION is
pytest                 # with the reference Tesseract installed
pytest tests/test_tesseract_cache.py  # integrity check must pass
git add tests/cache
git commit -m "Regenerate Tesseract cache (tesseract 5.5.1)"
```

Always name the Tesseract version in the commit message, and regenerate the
whole tree with one Tesseract version rather than mixing versions.

### PDF/A validation

Speculative PDF/A conversion (see {doc}`advanced`) is validated by
pikepdf's PDF/A support, `pikepdf.pdfa`. Problems with the validator's
verdicts, and its differential testing against
[veraPDF](https://verapdf.org/) (`misc/pdfa_differential.py`), belong to
[pikepdf](https://github.com/pikepdf/pikepdf). OCRmyPDF's tests of the PDF/A
pipeline cross-check their output with veraPDF when it is installed.

## New dependencies

If you are proposing a change that will require a new dependency, we
prefer dependencies that are already packaged by Debian or Red Hat. This
makes life much easier for our downstream package maintainers. A package
that is only available on PyPI or GitHub, and not more widely packaged,
may not be accepted.

We are unlikely to accept a dependency on CUDA or other GPU-based
libraries, because these are still difficult to package and install on
many systems. We recommend implementing these changes as plugins.

Python dependencies must also be license-compatible. GPLv3 or AGPLv3 are
likely incompatible with the project\'s license, but LGPLv3 is
compatible.

## New non-Python dependencies

OCRmyPDF uses several external programs (Tesseract, Ghostscript and
others) for its functionality. In general we prefer to avoid adding new
external programs, and if we are to add external programs, we prefer
those that are already packaged by Debian or Red Hat.

## Plugins

Some new features may be a good fit for a plugin. Plugins are a way to
add features to OCRmyPDF without adding them to the core program.
Plugins are installed separately from OCRmyPDF. They are written in
Python and can be installed from PyPI. See the [plugin
documentation](https://ocrmypdf.readthedocs.io/en/latest/plugins.html).

We are happy to link users to your plugin from the documentation.

## Style guide: Is it OCRmyPDF or ocrmypdf?

The program/project is OCRmyPDF and the name of the executable or
library is ocrmypdf.

## Copyright and license

For contributions over 10 lines of code, please add your name to list of
copyright holders for that file. The core program is licensed under
MPL-2.0, test files and documentation under CC-BY-SA 4.0, and
miscellaneous files under MIT, with a few minor exceptions. Please
contribute only content that you own or have the right to contribute
under these licenses.

---

### Cookbook (`docs/cookbook.md`)

% SPDX-FileCopyrightText: 2025 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Cookbook

## Basic examples

### Help!

ocrmypdf has built-in help.

```bash
ocrmypdf --help
```

### Add an OCR layer and convert to PDF/A

```bash
ocrmypdf input.pdf output.pdf
```

### Add an OCR layer and output a standard PDF

```bash
ocrmypdf --output-type pdf input.pdf output.pdf
```

### Create a PDF/A with all color and grayscale images converted to JPEG

```bash
ocrmypdf --output-type pdfa --pdfa-image-compression jpeg input.pdf output.pdf
```

### Reduce JPEG quality with the optimizer

This is the recommended way to shrink JPEG content in the output. The
optimizer applies regardless of `--output-type`, so it works on both
plain PDFs and Ghostscript-produced PDF/A files.

```bash
ocrmypdf --optimize 2 --jpeg-quality 60 input.pdf output.pdf
```

### Modify a file in place

The file will only be overwritten if OCRmyPDF is successful.

```bash
ocrmypdf myfile.pdf myfile.pdf
```

### Correct page rotation

OCR will attempt to automatic correct the rotation of each page. This
can help fix a scanning job that contains a mix of landscape and
portrait pages.

```bash
ocrmypdf --rotate-pages myfile.pdf myfile.pdf
```

You can increase (decrease) the parameter `--rotate-pages-threshold` to
make page rotation more (less) aggressive. The threshold number is the
ratio of how confidence the OCR engine is that the document image should
be changed, compared to kept the same. The default value is quite
conservative; on some files it may not attempt rotations at all unless
it is very confident that the current rotation is wrong. A lower value
of `2.0` will produce more rotations, and more false positives. Run with
`-v1` to see the confidence level for each page to see if there may be a
better value for your files.

If the page is \"just a little off horizontal\", like a crooked picture,
then you want `--deskew`. `--rotate-pages` is for when the cardinal
angle is wrong.

### OCR languages other than English

OCRmyPDF assumes the document is in English unless told otherwise. OCR
quality may be poor if the wrong language is used.

```bash
ocrmypdf -l fra LeParisien.pdf LeParisien.pdf
ocrmypdf -l eng+fra Bilingual-English-French.pdf Bilingual-English-French.pdf
```

Language packs must be installed for all languages specified. See
`Installing additional language packs <lang-packs>`{.interpreted-text
role="ref"}.

Unfortunately, the Tesseract OCR engine has no ability to detect the
language when it is unknown.

### Produce PDF and text file containing OCR text

This produces a file named \"output.pdf\" and a companion text file
named \"output.txt\".

```bash
ocrmypdf --sidecar output.txt input.pdf output.pdf
```

:::{note}
The sidecar file contains the **OCR text** found by OCRmyPDF. If the
document contains pages that already have text, that text will not
appear in the sidecar. If the option `--pages` is used, only those pages
on which OCR was performed will be included in the sidecar. If certain
pages were skipped because of options like `--skip-big` or
`--tesseract-timeout`, those pages will not be in the sidecar.

If you don\'t want to generate the output PDF, use `--output-type=none`
to avoid generating one. Set the output filename to `-` (i.e. redirect
to stdout).

To extract all text from a PDF, whether generated from OCR or otherwise,
use a program like Poppler\'s `pdftotext` or `pdfgrep`.
:::

### OCR images, not PDFs

#### Option: use Tesseract

If you are starting with images, you can just use Tesseract directly to
convert images to PDFs:

```bash
tesseract my-image.jpg output-prefix pdf
```

```bash
# When there are multiple images
tesseract text-file-containing-list-of-image-filenames.txt output-prefix pdf
```

Tesseract\'s PDF output is quite good -- OCRmyPDF uses it internally, in
some cases. However, OCRmyPDF has many features not available in
Tesseract like image processing, metadata control, and PDF/A generation.

#### Option: use img2pdf

You can also use a program like
[img2pdf](https://gitlab.mister-muffin.de/josch/img2pdf) to convert your
images to PDFs, and then pipe the results to run ocrmypdf. The `-` tells
ocrmypdf to read standard input.

```bash
img2pdf my-images*.jpg | ocrmypdf - myfile.pdf
```

`img2pdf` is recommended because it does an excellent job at generating
PDFs without transcoding images.

#### Option: use OCRmyPDF (single images only)

For convenience, OCRmyPDF can also convert single images to PDFs on its
own. If the resolution (dots per inch, DPI) of an image is not set or is
incorrect, it can be overridden with `--image-dpi`. (As 1 inch is 2.54
cm, 1 dpi = 0.39 dpcm).

```bash
ocrmypdf --image-dpi 300 image.png myfile.pdf
```

If you have multiple images, you must use `img2pdf` to convert the
images to PDF.

#### Not recommended

We caution against using ImageMagick or Ghostscript to convert images to
PDF, since they may transcode images or produce downsampled images,
sometimes without warning.

(image-processing)=

## Image processing

OCRmyPDF perform some image processing on each page of a PDF, if
desired. The same processing is applied to each page. It is suggested
that the user review files after image processing as these commands
might remove desirable content, especially from poor quality scans.

-   `--rotate-pages` attempts to determine the correct orientation for
    each page and rotates the page if necessary.
-   `--remove-background` attempts to detect and remove a noisy
    background from grayscale or color images. Monochrome images are
    ignored. This should not be used on documents that contain color
    photos as it may remove them.
-   `--deskew` will correct pages that were scanned at a skewed angle by
    rotating them back into place.
-   `--clean` uses [unpaper](https://www.flameeyes.eu/projects/unpaper)
    to clean up pages before OCR, but does not alter the final output.
    This makes it less likely that OCR will try to find text in
    background noise.
-   `--clean-final` uses unpaper to clean up pages before OCR and
    inserts the page into the final output. You will want to review each
    page to ensure that unpaper did not remove something important.

:::{note}
In many cases image processing will rasterize PDF pages as images,
potentially losing quality.
:::

:::{warning}
`--clean-final` and `--remove-background` may leave undesirable visual
artifacts in some images where their algorithms have shortcomings. Files
should be visually reviewed after using these options.
:::

### Example: OCR and correct document skew (crooked scan)

Deskew:

```bash
ocrmypdf --deskew input.pdf output.pdf
```

Image processing commands can be combined. The order in which options
are given does not matter. OCRmyPDF always applies the steps of the
image processing pipeline in the same order (rotate, remove background,
deskew, clean).

```bash
ocrmypdf --deskew --clean --rotate-pages input.pdf output.pdf
```

Don\'t actually OCR my PDF
--------------------------

If you set `--ocr-engine none` OCRmyPDF will skip OCR entirely. This works if
all you want is PDF/A conversion or image processing such as `--clean`.

```bash
ocrmypdf --ocr-engine none --output-type pdfa input.pdf output.pdf
```

`--ocr-engine none` cannot detect page orientation or skew, so
`--rotate-pages` and `--deskew` have no effect with it. To rotate or deskew
pages without adding a text layer, use `--tesseract-timeout 0` instead.
Tesseract will still be used to measure orientation and skew, but will not
OCR the page.

```bash
ocrmypdf --tesseract-timeout 0 --rotate-pages --deskew input.pdf output.pdf
```

:::{versionchanged} v17.0.0

Prior to this version, `--tesseract-timeout 0` was recommended as the general
idiom to turn off OCR. `--ocr-engine none` is now preferred, except when
rotating or deskewing pages.

:::

:::{versionchanged} v14.1.0

Prior to this version, `--tesseract-timeout 0` would prevent other uses
of Tesseract, such as deskewing, from working. This is no longer the
case. Use `--tesseract-non-ocr-timeout` to control the timeout for
non-OCR operations, if needed.
:::

### Remove the OCR text layer from my PDF

To remove the invisible OCR text layer while keeping the original pages
exactly as they are -- no rasterizing, no change to images or visible
content, and a smaller output file -- use `--mode strip`:

```bash
ocrmypdf --mode strip input.pdf output.pdf
```

Why would you want to do this? Perhaps you have a PDF where OCR failed to
produce useful results and you simply want to get rid of it.

`--mode strip` removes only text drawn as *invisible* (PDF text render
mode 3), which is how OCRmyPDF and most OCR tools add a searchable layer
over a scanned page. Some OCR products -- and OCRmyPDF v2.2 and earlier --
instead draw *visible* text and paint an opaque image on top of it. That
text is part of the visible page, so `--mode strip` cannot remove it
without altering the page's appearance.

To strip *all* text, including such visible text, rasterize the whole page
into a \"bag of images\" PDF instead (this rebuilds every page as an image,
so the file usually grows and vector content is lost):

```bash
ocrmypdf --ocr-engine none --force-ocr input.pdf output.pdf
```

### Optimize images without performing OCR

You can also optimize all images without performing any OCR:

```bash
ocrmypdf --ocr-engine none --optimize 3 --skip-text input.pdf output.pdf
```

## Using v17 features

### Select a rasterizer

:::{versionadded} 17.0.0
:::

OCRmyPDF can use pypdfium2 or Ghostscript to rasterize PDF pages. pypdfium2
is generally faster and is preferred when available.

```bash
# Automatic selection (default) - prefers pypdfium when available
ocrmypdf --rasterizer auto input.pdf output.pdf

# Explicitly use pypdfium2 (requires pip install pypdfium2)
ocrmypdf --rasterizer pypdfium input.pdf output.pdf

# Explicitly use Ghostscript
ocrmypdf --rasterizer ghostscript input.pdf output.pdf
```

### PDF/A without Ghostscript

:::{versionadded} 17.0.0
:::

OCRmyPDF can often produce PDF/A without using Ghostscript for conversion:
it adds the PDF/A structures itself and checks the result with pikepdf's
PDF/A validator, and only uses Ghostscript for files that fail the check.
This is faster and avoids some Ghostscript limitations.

:::{versionchanged} 17.13.0
The check is made by pikepdf's PDF/A validator (`pikepdf.pdfa`); veraPDF is
no longer needed.
:::

```bash
# Best-effort PDF/A: speculative conversion, then Ghostscript, then plain PDF
ocrmypdf --output-type auto input.pdf output.pdf

# Required PDF/A: speculative conversion, then Ghostscript
ocrmypdf --output-type pdfa input.pdf output.pdf

# Required PDF/A without Ghostscript: fail if the validator does not approve
ocrmypdf --output-type pdfa --pdfa-backend internal input.pdf output.pdf

# Always convert with Ghostscript
ocrmypdf --output-type pdfa --pdfa-backend ghostscript input.pdf output.pdf

# A specific image compression also needs Ghostscript, and selects it
ocrmypdf --output-type pdfa --pdfa-image-compression lossless input.pdf output.pdf
```

### Using --mode instead of legacy flags

:::{versionadded} 17.0.0
:::

The `--mode` (`-m`) flag consolidates OCR behavior options:

```bash
# Instead of --skip-text
ocrmypdf --mode skip input.pdf output.pdf

# Instead of --force-ocr
ocrmypdf --mode force input.pdf output.pdf

# Instead of --redo-ocr
ocrmypdf --mode redo input.pdf output.pdf

# Short form
ocrmypdf -m skip input.pdf output.pdf
```

The legacy flags continue to work as aliases.

### Process only certain pages

You can ask OCRmyPDF to only apply [image processing](#image-processing)
and OCR to certain pages.

```bash
ocrmypdf --pages 2,3,13-17 input.pdf output.pdf
```

Hyphens denote a range of pages and commas separate page numbers. If you
prefer to use spaces, quote all of the page numbers:
`--pages '2, 3, 5, 7'`.

The token `end` (case-insensitive) is an alias for the last page in the
document. For example, `--pages 3-end` OCRs from page 3 through the
final page, and `--pages end` OCRs only the last page:

```bash
ocrmypdf --pages 3-end input.pdf output.pdf
ocrmypdf --pages end input.pdf output.pdf
```

OCRmyPDF will warn if your list of page numbers contains duplicates or
overlapping pages. (Repeated page numbers are de-duplicated automatically,
since the underlying set of pages is what matters.) OCRmyPDF does not
currently account for document page numbers, such as an introduction
section of a book that uses Roman numerals. It simply counts the number
of virtual pieces of paper since the start. If your list of pages is out
of numerical order, OCRmyPDF will sort it for you.

Regardless of the argument to `--pages`, OCRmyPDF will optimize all
pages/images in the file and convert it to PDF/A, unless you disable
those options. Both of these steps are \"whole file\" operations. In
this example, we want to OCR only the title and otherwise change the PDF
as little as possible:

```bash
ocrmypdf --pages 1 --output-type pdf --optimize 0 input.pdf output.pdf
```

## Redo existing OCR

To redo OCR on a file OCRed with other OCR software or a previous
version of OCRmyPDF and/or Tesseract, you may use the `--redo-ocr`
argument. (Normally, OCRmyPDF will exit with an error if asked to modify
a file with OCR.)

This may be helpful for users who want to take advantage of accuracy
improvements in Tesseract for files they previously OCRed with an
earlier version of Tesseract and OCRmyPDF.

```bash
ocrmypdf --redo-ocr input.pdf output.pdf
```

This method will replace OCR without rasterizing, reducing quality or
removing vector content. If a file contains a mix of pure digital text
and OCR, digital text will be ignored and OCR will be replaced. As such
this mode is incompatible with image processing options, since they
alter the appearance of the file.

In some cases, existing OCR cannot be detected or replaced. Files
produced by OCRmyPDF v2.2 or earlier, for example, are internally
represented as having visible text with an opaque image drawn on top.
This situation cannot be detected.

If `--redo-ocr` does not work, you can use `--force-ocr`, which will
force rasterization of all pages, potentially reducing quality or losing
vector content.

Improving OCR quality
---------------------

The [Image processing](#image-processing) features can improve OCR
quality.

Rotating pages and deskewing helps to ensure that the page orientation
is correct before OCR begins. Removing the background and/or cleaning
the page can also improve results. The `--oversample DPI` argument can
be specified to resample images to higher resolution before attempting
OCR; this can improve results as well.

OCR quality will suffer if the resolution of input images is not correct
(since the range of pixel sizes that will be checked for possible fonts
will also be incorrect).

## PDF optimization

By default OCRmyPDF will attempt to perform lossless optimizations on
the images inside PDFs after OCR is complete. Optimization is performed
even if no OCR text is found.

The `--optimize N` (short form `-O`) argument controls optimization,
where `N` ranges from 0 to 3 inclusive, analogous to the optimization
levels in the GCC compiler. `-O1` is the default.

For further details, see the section on [PDF optimization](optimizer).

```bash
ocrmypdf --optimize 3 in.pdf out.pdf  # Make it small
```

Some users may consider enabling lossy JBIG2. See:
`jbig2-lossy`{.interpreted-text role="ref"}.

:::{note}
Image processing and PDF/A conversion can also introduce lossy
transformations to your PDF images, even when `--optimize 1` is in use.
:::

Digitally signed PDFs
---------------------

OCRmyPDF cannot preserve digital signatures in PDFs and also add OCR to
them. By default, it will refuse to modify a signed PDF regardless of
other settings. You can override this behavior with
`--invalidate-digital-signatures`; as the name suggests, any digital
signatures will be invalidated.

OCRmyPDF cannot open documents that are encrypted with a digital
certificate.

Versions of OCRmyPDF prior to 14.4.0 would invalidate existing digital
signatures without warning.

---

### Design Notes (`docs/design_notes.md`)

% SPDX-FileCopyrightText: 2023 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Design notes

## Why doesn\'t OCRmyPDF use PyTesseract?

PyTesseract is a Python wrapper around the Tesseract OCR engine. When
OCRmyPDF was first written, PyTesseract used ABI bindings to call the
Tesseract library. This was not a good fit for OCRmyPDF because ABI
bindings can be fragile.

PyTesseract has since evolved calling the Tesseract executable,
abandoning the ABI approach and using the CLI instead, just like
OCRmyPDF does. If it were written from scratch today, OCRmyPDF might use
PyTesseract.

PyTesseract has more features don\'t particularly need PDF output, but
less features than OCRmyPDF\'s API for creating PDFs.

## What is `executor()`?

OCRmyPDF uses a custom concurrent executor which can support either
threads or processes with the same interface. This is useful because
OCRmyPDF can use either threads or processes to parallelize work,
whichever is more appropriate for the task at hand.

The interface is currently private and subject to change. In particular,
if experiments with asyncio and anyio are successful, the interface will
change.

---

### Docker (`docs/docker.md`)

# OCRmyPDF Docker image {#docker}

OCRmyPDF is also available in Docker images that packages recent
versions of all dependencies.

For users who already have Docker installed this may be an easy and
convenient option.

On platforms other than Linux, Docker runs in a virtual machine, and so
may be less performant. You may also want to adjust the Docker virtual
machine\'s memory and CPU allocation. On Linux, the Docker image runs
natively and performance is comparable to a system installation.

{#docker-install}
## Installing the Docker image

If you have [Docker](https://docs.docker.com/) installed on your system,
you can install a Docker image of the latest release.

If you can run this command successfully, your system is ready to
download and execute the image:

:::{code} bash
docker run hello-world
:::

:::{list-table} Docker Images
:header-rows: 1

* - Image
  - Architecture
  - Description
* - `jbarlow83/ocrmypdf-alpine`
  - x86_64 and arm64
  - Recommended image, based on Alpine Linux.
* - `jbarlow83/ocrmypdf-ubuntu`
  - x86_64 and arm64
  - Alternate image, based on Ubuntu. When the Alpine image is considered stable and available for arm64, this image will be deprecated.
* - `jbarlow83/ocrmypdf`
  - x86_64 and arm64
  - Currently an alias for ocrmypdf-ubuntu. When the Alpine image is considered stable and available for arm64, this name will point to the Alpine image. If you don\'t know about the difference between Alpine and Ubuntu, use this image.
:::

To install:

:::{code} bash
docker pull jbarlow83/ocrmypdf-alpine
:::

The `ocrmypdf` image is also available, but is deprecated and will be
removed in the future.

OCRmyPDF will use all available CPU cores. See the Docker documentation
for [adjusting memory and CPU on other
platforms](https://docs.docker.com/config/containers/resource_constraints/)
if you are using Docker on macOS or Windows, where you may need to
manually assign more resources. On Linux, all resources will be
available automatically.

The underlying operating system and other details in Docker images are
considered implementation details and **subject to change at minor
releases**. If you are modifying the image, you should pin the version
you intend to use.

## Using the Docker image on the command line

**Unlike typical Docker containers**, in this section the OCRmyPDF
Docker container is ephemeral -- it runs for one OCR job and terminates,
just like a command line program. We are using Docker to deliver an
application (as opposed to the more conventional case, where a Docker
container runs as a server). For that reason we usually use the `--rm`
argument to delete the container when it exits.

:::{note}
The image runs as a non-root user (`app`, uid/gid 1000) by default,
rather than as root. This is a defense-in-depth measure: a flaw in
OCRmyPDF or one of its dependencies cannot trivially act as root inside
the container. The examples below assume **rootless Docker** or
**Podman**; the differences for traditional *rootful* Docker are
described separately under *Special case: rootful Docker* below.
:::

To start a Docker container (instance of the image):

:::{code} bash
docker run --rm -i jbarlow83/ocrmypdf-alpine (... all other arguments here...) - -
:::

### Recommended: pipe through stdin and stdout

The easiest and most portable way to use the image is to send the input
file on stdin and read the output from stdout. This **avoids file
permission issues entirely** -- nothing is written to a mounted
directory, so it does not matter which user the container runs as, nor
whether you use rootless or rootful Docker. For convenience, create a
shell alias to hide the Docker command:

:::{code} bash
alias docker_ocrmypdf='docker run --rm -i jbarlow83/ocrmypdf-alpine'
docker_ocrmypdf --version  # runs docker version
docker_ocrmypdf - - <input.pdf >output.pdf
:::

Or in the wonderful [fish shell](https://fishshell.com/):

:::{code} fish
alias docker_ocrmypdf 'docker run --rm -i jbarlow83/ocrmypdf-alpine'
funcsave docker_ocrmypdf
:::

{#docker-volumes}
### Bind-mounted volumes

If you would rather mount a directory and pass file paths, you need to
consider which user owns the files OCRmyPDF writes back into that
directory. The image's default working directory is `/data`, so mounting
your files there lets you pass plain relative paths without an explicit
`--workdir`. Because the container runs as the non-root `app` user, the
right invocation otherwise depends on your container runtime.

**Rootless Docker (the assumed default).** Your own account runs the
daemon, so the container's `root` maps back to *your* unprivileged host
user, while every other container uid -- including the image's default
`app`/1000 -- maps to a *subordinate* uid. A directory you own on the
host therefore appears owned by `root` inside the container, so the
default `app` user usually **cannot write to it at all**. Run the job as
container-`root`, which under rootless Docker is still your ordinary host
user, so the write succeeds and the output is owned by you:

:::{code} bash
alias docker_ocrmypdf='docker run --rm -i --user 0:0 -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
docker_ocrmypdf input.pdf output.pdf
:::

**Podman.** Podman provides `--userns keep-id`, which maps your host uid
straight through into the container. Combined with `--user`, you run as
your own uid and own the output directly, otherwise you may get access
errors because the user ID is not mapped to the same UID as on the host:

:::{code} bash
alias podman_ocrmypdf='podman run --rm -i --user "$(id -u):$(id -g)" --userns keep-id -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
podman_ocrmypdf input.pdf output.pdf
:::

If you have SELinux enabled, you may additionally need to add the `:Z` [suffix to
the
volume](https://docs.podman.io/en/stable/markdown/podman-run.1.html#volume-v-source-volume-host-dir-container-dir-options)
or disable SELinux for the container using
`--security-opt label=disable`, which is suggested for system files as
they should not be re-labelled. Please refer to the „Note" section at
the end of the linked podman documentation for details. This results in
the following full command:

:::{code} bash
alias podman_ocrmypdf='podman run --rm -i --user "$(id -u):$(id -g)" --userns keep-id -v "$PWD:/data" --security-opt label=disable jbarlow83/ocrmypdf-alpine'
podman_ocrmypdf input.pdf output.pdf
:::

{#docker-rootful}
### Special case: rootful Docker

With a traditional root daemon, container uid *N* is the *same* uid *N*
on the host. Running the container as root would therefore fill your
mounted directory with root-owned files and -- more importantly -- a
container escape would run as real host root. Drop to your own uid so the
output is owned by you and the process stays unprivileged:

:::{code} bash
alias docker_ocrmypdf='docker run --rm -i --user "$(id -u):$(id -g)" -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
docker_ocrmypdf input.pdf output.pdf
:::

The non-root default and the `--user` override both reduce the risk here,
but rootless Docker or Podman remain the safer choice when available.

{#docker-lang-packs}
## Adding languages to the Docker image

By default the Docker image includes English, German, Simplified
Chinese, French, Portuguese and Spanish, the most popular languages for
OCRmyPDF users based on feedback. You may add other languages by
creating a new Dockerfile based on the public one.

:::{code} dockerfile
FROM jbarlow83/ocrmypdf

# The image runs as the non-root "app" user, so switch back to root for
# build steps that install packages, then drop back to "app".
USER root
# Example: add Italian
RUN apt-get update && apt-get install -y tesseract-ocr-ita
USER app
:::

To install language packs (training data) such as the
[tessdata\_best](https://github.com/tesseract-ocr/tessdata_best) suite
or custom data, you first need to determine the version of Tesseract
data files, which may differ from the Tesseract program version. Use
this command to determine the data file version:

:::{code} bash
docker run -i --rm --entrypoint /bin/ls jbarlow83/ocrmypdf /usr/share/tesseract-ocr
:::

As of 2021, the data file version is probably `4.00`.

You can then add new data with either a Dockerfile:

:::{code} dockerfile
FROM jbarlow83/ocrmypdf:{TAG}

# Example: add a tessdata_best file
COPY chi_tra_vert.traineddata /usr/share/tesseract-ocr/<data version>/tessdata/
:::

When creating your own image, you should always pin a specific version
of the OCRmyPDF Docker image. This ensures that your image will not
break when a new version of OCRmyPDF is released.

Alternately, you can copy training data into a Docker container as
follows:

:::{code} bash
docker cp mycustomtraining.traineddata name_of_container:/usr/share/tesseract-ocr/<tesseract version>/tessdata/
:::

Extending the Docker image
--------------------------

You can extend the Docker image with your own customizations, similar to
the way it is extended to add language packs. Because the image runs as
the non-root `app` user, switch to `USER root` for any build steps that
require root (installing packages, writing to system directories) and
back to `USER app` afterwards, as shown in the language pack example
above.

Note that the Docker image is subject to change at any time. For
example, the base image may be updated to a newer version of Ubuntu or
Debian. Such changes will be noted in the release notes but might occur
at minor versions releases, unless the way a \"casual\" user of the
Docker image is affected.

If you extend the Docker image, you should pin a specific version of the
OCRmyPDF Docker image.

Executing the test suite
------------------------

The OCRmyPDF test suite is installed with image. To run it:

:::{code} bash
docker run --rm --workdir /app --entrypoint python jbarlow83/ocrmypdf -m pytest
:::

Accessing the shell
-------------------

To use the shell in the Docker image:

:::{code} bash
docker run -it --entrypoint sh jbarlow83/ocrmypdf-alpine
:::

This shell runs as the non-root `app` user. If you need root inside the
container -- for example to install extra packages with `apk` or `apt` --
add `--user root`:

:::{code} bash
docker run -it --user root --entrypoint sh jbarlow83/ocrmypdf-alpine
:::

Using the OCRmyPDF web service wrapper
--------------------------------------

The OCRmyPDF Docker image includes an example, barebones web service
built with [Streamlit](https://streamlit.io/). It may be launched as follows:

:::{code} bash
docker run --entrypoint python -p 8501:8501 jbarlow83/ocrmypdf /app/webservice.py
:::

Then open <http://localhost:8501> in a web browser. Streamlit listens on
port 8501 by default; any arguments after `/app/webservice.py` are passed
to `streamlit run`, so to use a different port:

:::{code} bash
docker run --entrypoint python -p 5000:5000 jbarlow83/ocrmypdf \
    /app/webservice.py --server.port 5000
:::

An equivalent Docker Compose service:

:::{code} yaml
services:
  ocrmypdf:
    image: jbarlow83/ocrmypdf
    entrypoint: ["python", "/app/webservice.py"]
    ports:
      - "8501:8501"
:::

Unlike command line usage this program will open a socket and wait for
connections.

:::{warning}
The OCRmyPDF web service wrapper is intended for demonstration or
development. It provides no security, no authentication, no protection
against denial of service attacks, and no load balancing. It processes
one file at a time for each browser session.
:::

Unlike the rest of OCRmyPDF, this web service is licensed under the
Affero GPLv3 (AGPLv3) since Ghostscript is also licensed in this way.

In addition to the above, please read our
`general remarks on using OCRmyPDF as a service <ocr-service>`{.interpreted-text
role="ref"}.

---

### Errors (`docs/errors.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Common error messages

## Page already has text

:::{code}
ERROR -    1: page already has text! – aborting (use --force-ocr to force OCR)
:::

You ran ocrmypdf on a file that already contains printable text or a
hidden OCR text layer (it can\'t quite tell the difference). You
probably don\'t want to do this, because the file is already searchable.

As the error message suggests, your options are:

-   `ocrmypdf --force-ocr` to
    `rasterize <raster-vector>`{.interpreted-text role="ref"} all vector
    content and run OCR on the images. This is useful if a previous OCR
    program failed, or if the document contains a text watermark.
-   `ocrmypdf --skip-text` to skip OCR and other processing on any pages
    that contain text. Text pages will be copied into the output PDF
    without modification.
-   `ocrmypdf --redo-ocr` to scan the file for any existing OCR
    (non-printing text), remove it, and do OCR again. This is one way to
    take advantage of improvements in OCR accuracy. Printable vector
    text is excluded from OCR, so this can be used on files that contain
    a mix of digital and scanned files.

## Input file \'filename\' is not a valid PDF

OCRmyPDF checks files with pikepdf, a library that in turn uses libqpdf
to fixes errors in PDFs, before it tries to work on them. In most cases
this happens because the PDF is corrupt and truncated (incomplete file
copying) and not much can be done.

You can try rewriting the file with Ghostscript:

:::{code} bash
gs -o output.pdf -dSAFER -sDEVICE=pdfwrite input.pdf
:::

`pdftk` can also rewrite PDFs:

:::{code} bash
pdftk input.pdf cat output output.pdf
:::

Sometimes Acrobat can repair PDFs with its [Preflight
tool](https://helpx.adobe.com/acrobat/using/correcting-problem-areas-preflight-tool.html).

(tesseract-config-missing)=

## Tesseract cannot open its config file \'hocr\' or \'txt\'

:::{code}
ERROR - Tesseract cannot open its config file 'hocr'.
:::

OCRmyPDF asks Tesseract to produce `hocr` and `txt` output. Tesseract
reads the instructions for these output formats from configuration files
named `hocr` and `txt` that live in the `configs/` subdirectory of its
`tessdata` folder. If those files are missing, Tesseract prints
`read_params_file: Can't open hocr`, exits without error, and produces no
output.

This usually happens when a `tessdata` directory was assembled by hand --
for example, by downloading individual `.traineddata` files from
[tessdata_best](https://github.com/tesseract-ocr/tessdata_best) and
pointing `TESSDATA_PREFIX` at them -- because those repositories do not
include the `configs/` directory. A complete Tesseract installation from
your operating system\'s package manager includes it.

To fix this, ensure the `configs/hocr` and `configs/txt` files exist in
the `tessdata` directory that Tesseract is using. Copying the `configs/`
directory from a full Tesseract installation is sufficient. See
{envvar}`TESSDATA_PREFIX` for more on selecting an alternate `tessdata`
folder.

---

### Installation (`docs/installation.md`)

---
myst:
  substitutions:
    deb_12: |-
      :::{image} https://repology.org/badge/version-for-repo/debian_12/ocrmypdf.svg
      :alt: Debian 12
      :::
    deb_13: |-
      :::{image} https://repology.org/badge/version-for-repo/debian_13/ocrmypdf.svg
      :alt: Debian 13
      :::
    deb_unstable: |-
      :::{image} https://repology.org/badge/version-for-repo/debian_unstable/ocrmypdf.svg
      :alt: Debian unstable
      :::
    fedora_43: |-
      :::{image} https://repology.org/badge/version-for-repo/fedora_43/ocrmypdf.svg
      :alt: Fedora 43
      :::
    fedora_44: |-
      :::{image} https://repology.org/badge/version-for-repo/fedora_44/ocrmypdf.svg
      :alt: Fedora 44
      :::
    fedora_rawhide: |-
      :::{image} https://repology.org/badge/version-for-repo/fedora_rawhide/ocrmypdf.svg
      :alt: Fedora Rawhide
      :::
    latest: |-
      :::{image} https://img.shields.io/pypi/v/ocrmypdf.svg
      :alt: OCRmyPDF latest released version on PyPI
      :::
    ubu_2204: |-
      :::{image} https://repology.org/badge/version-for-repo/ubuntu_22_04/ocrmypdf.svg
      :alt: Ubuntu 22.04 LTS
      :::
    ubu_2404: |-
      :::{image} https://repology.org/badge/version-for-repo/ubuntu_24_04/ocrmypdf.svg
      :alt: Ubuntu 24.04 LTS
      :::
    ubu_2604: |-
      :::{image} https://repology.org/badge/version-for-repo/ubuntu_26_04/ocrmypdf.svg
      :alt: Ubuntu 26.04 LTS
      :::
---

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Installing OCRmyPDF

(latest)=

The easiest way to install OCRmyPDF is to follow the steps for your operating
system/platform. This version may be out of date, however.

These platforms have one-liner installs:

:::{list-table}
:header-rows: 0

* - Homebrew (macOS and Linux)
  - ``brew install ocrmypdf``
* - Debian, Ubuntu
  - ``apt install ocrmypdf``
* - Windows Subsystem for Linux
  - ``apt install ocrmypdf``
* - Fedora
  - ``dnf install ocrmypdf tesseract-osd``
* - Alpine Linux
  - ``apk add ocrmypdf``
* - macOS (MacPorts)
  - ``port install ocrmypdf``
* - FreeBSD
  - ``pkg install py311-ocrmypdf``
:::

More detailed procedures are outlined below. If you want to do a manual
install, or install a more recent version than your platform provides, read on.

If your operating system is older than those listed here, we recommend
installing OCRmyPDF with [Homebrew](#homebrew-linux), or using the
[Docker image](docker).

:::{contents} Platform-specific steps
:depth: 2
:local: true
:::

## Installing on Linux

### Debian 12 or newer and Ubuntu 22.04 or newer

:::{list-table}
:header-rows: 1

* - OCRmyPDF versions in Debian & Ubuntu
* - {{ latest }}
* - {{ deb_12 }} {{ deb_13 }} {{ deb_unstable }}
* - {{ ubu_2204 }} {{ ubu_2404 }} {{ ubu_2604 }}
:::

Users of Debian or Ubuntu may simply

```bash
apt install ocrmypdf
```

As indicated in the table above, Debian and Ubuntu releases may lag
behind the latest version, sometimes by several major versions. If the version
available for your platform is out of date, see
[Installing the latest version on Debian and Ubuntu](#ubuntu-lts-latest).

For full details on version availability for your platform, check the
[Debian Package Tracker](https://tracker.debian.org/pkg/ocrmypdf) or
[Ubuntu launchpad.net](https://launchpad.net/ocrmypdf).

:::{note}
The Debian and Ubuntu OCRmyPDF packages do not install the JBIG2 encoder.
OCRmyPDF works fine without it but will produce larger output files.
On Debian 13 or newer and Ubuntu 24.04 or newer, the encoder is packaged and
can be installed with `apt install jbig2`. On older releases, you can build
jbig2enc from source; OCRmyPDF will automatically detect it on the `PATH`.
See {ref}`jbig2`.
:::

(ubuntu-lts-latest)=

### Installing the latest version on Debian and Ubuntu

Debian and Ubuntu include an older version of OCRmyPDF - you can install that
with `apt install ocrmypdf`. To install the latest version, we recommend
installing the system package first, which brings in Tesseract, Ghostscript
and other system dependencies, and then using
[uv](https://docs.astral.sh/uv/) to install the latest OCRmyPDF from PyPI:

```bash
# Install system dependencies first
sudo apt-get update
sudo apt-get -y install ocrmypdf

# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

uv installs the `ocrmypdf` command in `~/.local/bin`. Open a new terminal
(or run `uv tool update-shell`) so that it is on your `PATH`, and then run
`ocrmypdf --version` to confirm that the latest version is used rather than the
older system version in `/usr/bin`.

If your system Python is too old for the latest OCRmyPDF, as on Ubuntu 22.04,
uv will download a suitable version of Python automatically.

To upgrade later:

```bash
uv tool upgrade ocrmypdf
```

:::{tip}
Ubuntu 22.04 ships Tesseract 4.1.1, which is the oldest version OCRmyPDF
supports. Tesseract 5 gives better results; it is available for Ubuntu LTS
releases from the
[alex-p/tesseract-ocr5 PPA](https://launchpad.net/~alex-p/+archive/ubuntu/tesseract-ocr5):
`sudo add-apt-repository ppa:alex-p/tesseract-ocr5`. Newer Debian and Ubuntu
releases already include Tesseract 5.
:::

Alternatively, use Homebrew on Linux for a full-featured installation (see
{ref}`homebrew-linux`).

To add JBIG2 encoding, on Debian 13 or newer and Ubuntu 24.04 or newer run
`sudo apt install jbig2`; otherwise see {ref}`jbig2`.

### Older Debian and Ubuntu releases

Ubuntu 22.04 LTS and Debian 12 are the oldest releases OCRmyPDF supports.
Ubuntu 20.04 and Debian 11 have reached the end of standard support, and their
Python, Tesseract and Ghostscript are too old for current OCRmyPDF. Consider
upgrading to a supported release.

If you cannot upgrade, the most convenient way to install a recent version of
OCRmyPDF is to use Homebrew on Linux:

```bash
brew install ocrmypdf
```

See {ref}`homebrew-linux` for more information on using Homebrew on Linux.
Alternatively, run OCRmyPDF from the [Docker image](docker), which carries its
own up-to-date dependencies.

### Fedora

:::{list-table}
:header-rows: 1

* - OCRmyPDF version
* - {{latest}}
* - {{fedora_43}} {{fedora_44}} {{fedora_rawhide}}
:::

Users of Fedora may simply

```bash
dnf install ocrmypdf tesseract-osd
```

`tesseract-osd` is not installed automatically, but OCRmyPDF needs it for
automatic page rotation (`--rotate-pages`). Tesseract language packs are named
`tesseract-langpack-<language>`, for example `tesseract-langpack-deu`.

For full details on version availability, check the [Fedora Package
Tracker](https://packages.fedoraproject.org/pkgs/ocrmypdf/ocrmypdf/).

If the version available for your platform is out of date, install the
system package first to satisfy system dependencies, then use uv to install
the latest version:

```bash
# Install system dependencies first
sudo dnf install ocrmypdf tesseract-osd

# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

Open a new terminal (or run `uv tool update-shell`) so that `~/.local/bin` is on
your `PATH`, then confirm the version with `ocrmypdf --version`. To install
from source instead, see [Installing HEAD revision from
sources](#installing-head-revision-from-sources).

:::{note}
OCRmyPDF for Fedora currently omits the JBIG2 encoder, and Fedora does not
package it. All JBIG2 patents expired in 2017. OCRmyPDF works fine without it
but will produce larger output files. If you build jbig2enc from source,
OCRmyPDF will automatically detect it on the `PATH`. To add JBIG2 encoding,
see {ref}`jbig2`.
:::

### RHEL, CentOS Stream, AlmaLinux and Rocky Linux 9 and 10

There is no OCRmyPDF package for Red Hat Enterprise Linux or its derivatives,
either in the base repositories or in EPEL. Install the system dependencies
from the base (AppStream) repositories:

```bash
sudo dnf install ghostscript tesseract tesseract-osd tesseract-langpack-eng
```

Other Tesseract languages are available as `tesseract-langpack-<language>`,
for example `tesseract-langpack-deu`.

The optional dependencies `unpaper` and, on version 9, `pngquant` are available
from [EPEL](https://docs.fedoraproject.org/en-US/epel/getting-started/). Once
EPEL is enabled:

```bash
sudo dnf install unpaper pngquant  # pngquant: version 9 only
```

Then install uv, if not already installed, and use it to install OCRmyPDF:

```bash
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

RHEL 9's default Python 3.9 is too old for OCRmyPDF; uv will download a
suitable version of Python automatically. Open a new terminal (or run
`uv tool update-shell`) so that `~/.local/bin` is on your `PATH`, then confirm
the version with `ocrmypdf --version`.

To add JBIG2 encoding, see {ref}`Installing the JBIG2 encoder <jbig2>`.

### Arch Linux (AUR)

:::{image} https://repology.org/badge/version-for-repo/aur/ocrmypdf.svg
:alt: ArchLinux
:target: https://repology.org/metapackage/ocrmypdf
:::

There is an [Arch User Repository (AUR) package for OCRmyPDF](https://aur.archlinux.org/packages/ocrmypdf/).
OCRmyPDF is not in the official Arch repositories, but its dependencies,
including Tesseract and its language data, are.

Installing AUR packages as root is not allowed, so you must first [setup a
non-root user](https://wiki.archlinux.org/title/Users_and_groups#User_management) and
[configure sudo](https://wiki.archlinux.org/title/Sudo#Configuration).
The official Docker image, `archlinux:latest`, does **not** have a
non-root user configured, so users of that image must follow these guides.

Next you should install the [base-devel package group](https://archlinux.org/packages/core/any/base-devel/). This includes the
standard tooling needed to build packages, such as a compiler and binary tools.

```bash
sudo pacman -S --needed base-devel git
```

Now you are ready to install the OCRmyPDF package.

```bash
git clone https://aur.archlinux.org/ocrmypdf.git
cd ocrmypdf
makepkg -sri
```

At this point you will have a working install of OCRmyPDF, but the Tesseract
install won’t include any OCR language data. You can install [the
tesseract-data packages](https://archlinux.org/packages/?q=tesseract-data) to add
supported languages. For example, for English:

```bash
sudo pacman -S tesseract-data-eng
```

As an alternative to this manual procedure, consider using an [AUR helper](https://wiki.archlinux.org/title/AUR_helpers). Such a tool will
automatically fetch, build and install the AUR package, resolve dependencies
(including dependencies on AUR packages), and ease the upgrade procedure.

If you have any difficulties with installation, check the repository package
page.

:::{note}
The JBIG2 encoder is an optional dependency of the OCRmyPDF AUR package.
OCRmyPDF works fine without it but will produce larger output files. The
encoder is available from [the jbig2enc AUR package](https://aur.archlinux.org/packages/jbig2enc/)
and may be installed using the same series of steps as for the OCRmyPDF AUR
package. Alternatively, it may be built manually from source following the
instructions in {ref}`Installing the JBIG2 encoder <jbig2>`. If jbig2enc is
installed, OCRmyPDF will automatically detect it.
:::

### Alpine Linux

:::{image} https://repology.org/badge/version-for-repo/alpine_edge/ocrmypdf.svg
:alt: Alpine Linux
:target: https://repology.org/metapackage/ocrmypdf
:::

OCRmyPDF is in the Alpine Linux `community` repository. To install it:

```bash
apk add ocrmypdf
```

If the version available for your Alpine release is out of date, consider the
Alpine-based [Docker image](docker), `jbarlow83/ocrmypdf-alpine`.

### Gentoo Linux

:::{image} https://repology.org/badge/version-for-repo/gentoo_ovl_guru/ocrmypdf.svg
:alt: Gentoo Linux
:target: https://repology.org/metapackage/ocrmypdf
:::

OCRmyPDF is available from the [GURU](https://wiki.gentoo.org/wiki/Project:GURU)
overlay, which must be enabled first. To install OCRmyPDF on Gentoo Linux, use
the following commands:

```bash
emerge --ask --noreplace app-eselect/eselect-repository dev-vcs/git
eselect repository enable guru
emaint sync --repo guru
emerge --ask app-text/OCRmyPDF
```

### Other Linux packages

OCRmyPDF is also packaged for other distributions, including NixOS (`ocrmypdf`),
Void Linux (`python3-ocrmypdf`) and openSUSE Tumbleweed. See the
[Repology](https://repology.org/metapackage/ocrmypdf/versions) page for a full
list.

In general, first install the OCRmyPDF package for your system to satisfy
system dependencies, then optionally use the procedure [Installing with
uv](#installing-with-python-pip) to install a more recent version:

```bash
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

:::{note}
The OCRmyPDF Snap package (`snap install ocrmypdf`) tracks each OCRmyPDF
release on the `stable` channel. Builds of the development branch are available
with `snap install ocrmypdf --edge`. The snap is strictly confined and can only
read and write files in your home folder or on removable media; for other
locations, use `snap run ocrmypdf - - <input.pdf >output.pdf`.
:::

(homebrew-linux)=

## Installing with Homebrew (macOS and Linux)

:::{image} https://img.shields.io/homebrew/v/ocrmypdf.svg
:alt: homebrew
:target: https://formulae.brew.sh/formula/ocrmypdf
:::

[Homebrew](https://brew.sh) provides a full-featured OCRmyPDF installation
on both macOS and Linux with all recommended dependencies. This is often
the easiest way to get a complete, up-to-date installation.

```bash
brew install ocrmypdf
```

This includes Tesseract, Ghostscript, jbig2enc, pngquant, unpaper and all
required dependencies. English language support is included by default. For
other languages:

```bash
brew install tesseract-lang  # Optional: Install all language packs
```

To upgrade later:

```bash
brew upgrade ocrmypdf
```

:::{tip}
**For Linux users:** Homebrew on Linux is an excellent choice when your
distribution's package is outdated, your distribution is too old for current
OCRmyPDF, or its package is missing optional dependencies like jbig2enc,
pngquant, or unpaper. Homebrew provides a consistent, full-featured
installation that works across many Linux distributions.

Install Homebrew on Linux: https://brew.sh
:::

## Installing on macOS

### Homebrew

See {ref}`homebrew-linux` above - the installation is identical on macOS.

### MacPorts

:::{image} https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fports.macports.org%2Fapi%2Fv1%2Fports%2Focrmypdf%2F%3Fformat%3Djson&query=version&label=MacPorts
:alt: Macports Version Information
:target: https://ports.macports.org/port/ocrmypdf
:::

OCRmyPDF is included in MacPorts:

```bash
sudo port install ocrmypdf
```

Note that while this will install tesseract you will need to install
the appropriate tesseract [language ports](https://ports.macports.org/search/?q=tesseract-&name=on),
which are named `tesseract-<language>`. For example, for German:

```bash
sudo port install tesseract-deu
```

### Manual installation on macOS

These instructions are for installing a more current version of OCRmyPDF than
is available from Homebrew. Note that Homebrew versions usually track
releases fairly closely.

If it's not already present, [install Homebrew](https://brew.sh/).

Update Homebrew and install OCRmyPDF's dependencies, without OCRmyPDF itself:

```bash
brew update
brew install --only-dependencies ocrmypdf
```

Homebrew's Tesseract includes English only. If you need other languages you
can optionally install them all:

(macos-all-languages)=

```bash
brew install tesseract-lang  # Optional: for all language packs
```

Install uv, if not already installed, and use it to install OCRmyPDF:

```bash
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

Open a new terminal (or run `uv tool update-shell`) so that `~/.local/bin` is
on your `PATH`. The command line program should now be available:

```bash
ocrmypdf --help
```

To upgrade later, run `uv tool upgrade ocrmypdf`.

## Installing on Windows

### Native Windows

% If you have a Windows that is not the Home edition, you can use Windows Sandbox to test on a blank Windows instance.
% https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/

:::{note}
Administrator privileges will be required for some of these steps.
:::

You must install the following for Windows:

- Tesseract 64-bit
- Ghostscript 64-bit (recommended; required for some PDF/A conversions)
- uv, which installs OCRmyPDF and, if needed, a suitable Python

Using the [winget](https://learn.microsoft.com/en-us/windows/package-manager/winget/)
package manager:

```powershell
winget install -e --id tesseract-ocr.tesseract
winget install -e --id astral-sh.uv
```

If you prefer not to use winget for uv, you can install it with its
standalone installer instead:

```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```

Ghostscript is not available from winget. You will need to install it manually,
[since it does not support automated installs
anymore](https://artifex.com/news/ghostscript-10.01.0-disabling-silent-install-option):

- [Ghostscript download page](https://ghostscript.com/releases/gsdnld.html)

Or alternately, using the [Chocolatey](https://chocolatey.org/) package
manager, install the following when running in an Administrator command
prompt:

```powershell
choco install tesseract
choco install ghostscript
choco install pngquant  # optional
```

and then install uv using either of the commands above.

Either set of commands will install the required software. At the moment there
is no single command to install OCRmyPDF and all of its dependencies on
Windows.

Open a new command prompt or PowerShell window, so that the newly installed
programs are on your `PATH`, and install OCRmyPDF. (This can be performed by a
user or Administrator.)

```powershell
uv tool install ocrmypdf
```

Open another new window, then confirm that OCRmyPDF is available:

```powershell
ocrmypdf --version
```

If the `ocrmypdf` command is not found, run `uv tool update-shell` and open a
new window. To upgrade later, run `uv tool upgrade ocrmypdf`.

OCRmyPDF will check the Windows Registry and standard locations in your Program Files
for third party software it needs (specifically, Tesseract and Ghostscript). To
override the versions OCRmyPDF selects, you can modify the `PATH` environment
variable. [Follow these directions](https://www.computerhope.com/issues/ch000549.htm#dospath)
to change the PATH.

:::{warning}
32-bit Windows is not supported.
:::

### Windows Subsystem for Linux

1. Install Ubuntu 24.04 or 26.04 LTS for Windows Subsystem for Linux, if not
   already installed.
2. Follow the procedure to install {ref}`the latest OCRmyPDF on Ubuntu <ubuntu-lts-latest>`.
3. In the Ubuntu (WSL) terminal, create a symlink so that the latest version is
   found when OCRmyPDF is run from Windows:

```bash
sudo ln -s ~/.local/bin/ocrmypdf /usr/local/bin/ocrmypdf
```

Then, from the Windows command prompt, confirm that the expected version from
PyPI ({{ latest }}) is installed:

```powershell
wsl ocrmypdf --version
```

You can then run OCRmyPDF in the Windows command prompt or Powershell, prefixing
`wsl`, and call it from Windows programs or batch files.

### Cygwin64

:::{warning}
Cygwin installation is not tested. uv is not available for Cygwin, and some
of OCRmyPDF's Python dependencies, such as pikepdf, pypdfium2 and uharfbuzz,
have no Cygwin binaries and must be compiled from source, which may fail. We
recommend using Windows Subsystem for Linux, the Docker image, or native
Windows instead.
:::

First install the following prerequisite Cygwin packages using `setup-x86_64.exe`:

```
python312 (or later)
python312-devel
python312-pip
python312-lxml
python312-imaging

   (where 312 should match the version of Python you installed)

gcc-g++
ghostscript
libexempi-devel
libffi-devel
pngquant
qpdf
libqpdf-devel
tesseract-ocr
tesseract-ocr-devel
```

Then open a Cygwin terminal (i.e. `mintty`), and install OCRmyPDF into a
virtual environment:

```bash
python3.12 -m venv --system-site-packages ~/ocrmypdf-venv
~/ocrmypdf-venv/bin/python -m pip install ocrmypdf
~/ocrmypdf-venv/bin/ocrmypdf --version
```

The optional dependency "unpaper" is currently not available under Cygwin.
Without it, certain options such as `--clean` will produce an error message.
However, the OCR-to-text-layer functionality is available.

### Docker

You can also [Install the Docker image](docker) on Windows. Ensure that
your command prompt can run the docker "hello world" container.

## Installing on FreeBSD

:::{image} https://repology.org/badge/version-for-repo/freebsd/ocrmypdf.svg
:alt: FreeBSD
:target: https://repology.org/project/ocrmypdf/versions
:::

```bash
pkg install py311-ocrmypdf
```

To install a more recent version, first install the system version with `pkg`
to satisfy system dependencies, then install uv from packages and use it to
install OCRmyPDF:

```bash
pkg install uv
uv tool install ocrmypdf
```

Some Python dependencies do not publish binaries for FreeBSD and will be
compiled from source, so this may require additional development packages.

## Installing the Docker image

For some users, installing the Docker image will be easier than
installing all of OCRmyPDF's dependencies. It is also a good option if your
operating system is too old to run current OCRmyPDF.

See [Installing the Docker image](docker) for more information.

(installing-with-python-pip)=

## Installing with uv (recommended)

We recommend using [uv](https://docs.astral.sh/uv/) for installing OCRmyPDF from PyPI.
uv is a fast, modern Python package manager that installs command line tools
in isolated environments, and can download a suitable version of Python if
your system's Python is too old.

For best results, first install [your platform's
version](https://repology.org/metapackage/ocrmypdf/versions) of
`ocrmypdf` using the instructions elsewhere in this document to satisfy system
dependencies. Then use uv to get the latest OCRmyPDF version.

On macOS and Linux:

```bash
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

On Windows:

```powershell
# Install uv, if not already installed
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# Install the latest OCRmyPDF
uv tool install ocrmypdf
```

Open a new terminal (or run `uv tool update-shell`) so that the `ocrmypdf`
command is on your `PATH`. Use `ocrmypdf --version` to confirm what version was
installed, and `uv tool upgrade ocrmypdf` to upgrade later.

To use OCRmyPDF as a Python library in your own project, add it as a
dependency of that project instead:

```bash
uv add ocrmypdf
```

### Installing with pipx

If you already use pipx for isolated command-line tool installations, it works
as well:

```bash
pipx install ocrmypdf
```

(requirements-for-pip-and-head-install)=

### Requirements for uv and source installs

OCRmyPDF currently requires these external programs and libraries to be
installed, and must be satisfied using the operating system package
manager. uv and other Python package managers cannot provide them.

:::{versionchanged} 17.0.0
Ghostscript is now optional. pypdfium2 can be used for PDF rasterization,
and verapdf can validate speculative PDF/A conversion.
:::

:::{versionchanged} 17.13.0
veraPDF is no longer used. Speculative PDF/A conversion is checked by
pikepdf's PDF/A support (`pikepdf.pdfa`), so OCRmyPDF requires
`pikepdf[pdfa]` 10.15 or newer, which brings in the jsonschema, referencing
and fonttools Python packages.
:::

The following versions are required:

- Python 3.11 or newer (3.12+ recommended; uv can install it for you)
- Tesseract 4.1.1 or newer
- One of: Ghostscript 9.54+ **or** pypdfium2 (Python package)
- Ghostscript 9.54+ (for PDF/A output of files that cannot be converted
  without it; optional)
- fpdf2 2.8 or newer (Python package)
- uharfbuzz (Python package)
- fonts-noto or equivalent (system package, recommended)
- jbig2enc 0.28 or newer (optional)
- pngquant 2.12.2 or newer (optional)
- unpaper 6.1 or newer (optional)

:::{note}
For the best user experience, install both Ghostscript and pypdfium2. pypdfium2 is
faster for rasterization, while Ghostscript is required for certain PDF/A
conversions.
:::

**Dependency summary:**

| Feature | Option 1 | Option 2 | Notes |
|---------|----------|----------|-------|
| PDF rasterization | pypdfium2 (Python) | Ghostscript (binary) | pypdfium2 preferred when available |
| PDF/A conversion | pikepdf (`pikepdf.pdfa`) | Ghostscript | pikepdf validates speculative conversion |
| Text rendering | fpdf2 + uharfbuzz | - | Required |
| OCR | tesseract-ocr | `--ocr-engine none` | Can be skipped entirely |

**Minimum viable installation:**
tesseract-ocr + (pypdfium2 OR Ghostscript) + fpdf2 + uharfbuzz

**Recommended installation:**
tesseract-ocr + pypdfium2 + Ghostscript + fpdf2 + uharfbuzz + fonts-noto + unpaper + pngquant + jbig2enc

We recommend 64-bit versions of all software. (32-bit versions are not
supported, although on Linux, they may still work.)

**fpdf2** and **uharfbuzz** are required dependencies that provide the text
layer rendering engine. fpdf2 generates the PDF text layer, while uharfbuzz
provides text shaping for proper multilingual support. These replace the
legacy hOCR-based renderer. They are Python packages, installed automatically
when you install OCRmyPDF with uv.

**fonts-noto** (or an equivalent comprehensive font package) is recommended
for proper text rendering, especially for non-Latin scripts. OCRmyPDF bundles
a Latin font only, and discovers the rest from the fonts installed on your
system.

- Debian/Ubuntu: `apt install fonts-noto`
- Fedora: `dnf install google-noto-fonts-all`
- macOS with Homebrew: Homebrew has no single Noto package; each family is a
  separate cask. Install at least
  `brew install --cask font-noto-sans font-noto-serif`, plus a cask per
  additional script you OCR, for example
  `brew install --cask font-noto-sans-arabic font-noto-sans-cjk`. Run
  `brew search font-noto` to list them all.

If OCRmyPDF warns that no installed font has glyphs for some of the text, the
message names the characters it could not render, for example
`'Ꮳ' U+13E3 CHEROKEE LETTER TSA`. Install the Noto font for that script — here,
`fonts-noto-core` on Debian or `font-noto-sans-cherokee` on Homebrew. The text
layer remains searchable and copyable either way; only its appearance when
highlighted in a PDF viewer is affected.

**pypdfium2** provides fast PDF page rasterization using the pdfium library
(the same library used by Google Chrome). It is installed automatically when
you install OCRmyPDF with uv, although some distribution packages of OCRmyPDF
omit it. When present, it is preferred over Ghostscript due to better
performance.

**Speculative PDF/A conversion** needs no extra software. OCRmyPDF creates
PDF/A by adding metadata and an ICC profile using pikepdf, then checks the
result with pikepdf's PDF/A validator. If validation passes, Ghostscript
is skipped entirely; with `--pdfa-backend internal`, Ghostscript is never
used for PDF/A. **veraPDF** is no longer used by OCRmyPDF itself; the
test suite uses it, if installed, to cross-check pikepdf's validator
(see [verapdf.org](https://verapdf.org/)).

**jbig2enc**, if present, will be used to optimize the encoding of
monochrome images. This can significantly reduce the file size of the
output file. It is not required.
[jbig2enc](https://github.com/agl/jbig2enc) is not available in some
distributions due to historical patent concerns, but all JBIG2 patents
expired in 2017. It can easily be built from source. To add JBIG2 encoding,
see {ref}`jbig2`.

:::{warning}
Lossy JBIG2 encoding (`--jbig2-lossy`) has been removed in v17.0.0 due to
well-documented risks of character substitution errors. Only lossless
JBIG2 compression is now supported.
:::

**pngquant**, if present, is optionally used to optimize the encoding of
PNG-style images in PDFs (actually, any that are that losslessly
encoded) by lossily quantizing to a smaller color palette. It is only
activated then the `--optimize` argument is `2` or `3`.

**unpaper**, if present, enables the `--clean` and `--clean-final`
command line options.

These are in addition to the Python packaging dependencies, meaning that
unfortunately, the `uv tool install` command cannot satisfy all of them.

(installing-head-revision-from-sources)=

## Installing HEAD revision from sources

If you have `git` and uv installed, you can install from source. uv will
alert you if Python dependencies cannot be installed; OCRmyPDF will alert you
at runtime if system dependencies are missing.

If you prefer to build everything from source, you will need to [build
pikepdf from
source](https://pikepdf.readthedocs.io/en/latest/installation.html#building-from-source).
First ensure you can build and install pikepdf.

To install the latest development version as a command line tool:

```bash
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

uv tool install git+https://github.com/ocrmypdf/OCRmyPDF.git
```

Or, to work from a local checkout allowing customization:

```bash
git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
cd OCRmyPDF
uv sync
```

This creates a virtual environment in `.venv` and installs OCRmyPDF in
editable mode along with its dependencies. Run the program with `uv run`, or
activate the environment:

```bash
uv run ocrmypdf --help

# or
source .venv/bin/activate
ocrmypdf --help
```

Note: when installed this way, `ocrmypdf` will only be accessible through
`uv run` or when the virtual environment is activated.

If not yet installed, the script will notify you about dependencies that
need to be installed. The script requires specific versions of the
dependencies. Older version than the ones mentioned in the release notes
are likely not to be compatible to OCRmyPDF.

## Optional Features

OCRmyPDF provides optional features and development tools. We recommend using `uv` as your package manager.

### Installing User Features

User features are available as optional dependencies (extras). When
installing OCRmyPDF as a tool:

```bash
uv tool install "ocrmypdf[heic]"                   # HEIF/HEIC image support
uv tool install "ocrmypdf[watcher]"                # File watching service
uv tool install "ocrmypdf[webservice]"             # Streamlit web UI
uv tool install "ocrmypdf[watcher,webservice]"     # Multiple features
```

Or, from a local source checkout:

```bash
uv sync --extra heic           # HEIF/HEIC image support
uv sync --extra watcher        # File watching service
uv sync --extra webservice     # Streamlit web UI
uv sync --extra watcher --extra webservice  # Multiple features
```

### Development Tools

Development tools use dependency groups:

```bash
# Testing infrastructure
uv sync --group test

# Documentation building
uv sync --group docs

# Enhanced Streamlit development
uv sync --group streamlit-dev

# All development groups
uv sync --all-groups
```

**Why use uv?**

- Modern, fast Python package manager
- Required for development (testing, docs)
- Better dependency resolution
- Consistent across all platforms

Install uv: `curl -LsSf https://astral.sh/uv/install.sh | sh` or visit https://docs.astral.sh/uv/

### For development

To install all of the development and test requirements:

```bash
git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
cd OCRmyPDF
uv sync --all-groups
```

To add JBIG2 encoding, see {ref}`jbig2`.

## Shell completions

Completions for `bash` and `fish` are available in the project's
`misc/completion` folder. The `bash` completions are likely `zsh`
compatible but this has not been confirmed. Package maintainers, please
install these at the appropriate locations for your system.

To manually install the `bash` completion, copy
`misc/completion/ocrmypdf.bash` to `/etc/bash_completion.d/ocrmypdf`
(rename the file).

To manually install the `fish` completion, copy
`misc/completion/ocrmypdf.fish` to
`~/.config/fish/completions/ocrmypdf.fish`.

## Note on 32-bit support

We don't support any 32-bit system, including 32-bit Python or 32-bit
Ghostscript on Windows.

---

### Introduction (`docs/introduction.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Introduction

OCRmyPDF is a Python application and library that adds text "layers" to images in
PDFs, making scanned image PDFs searchable. It uses OCR to guess the text
contained in images. OCRmyPDF also supports plugins
that enable customization of its processing steps, and it is highly tolerant
of PDFs containing scanned images and "born digital" content that doesn't
require text recognition.

## About OCR

[Optical character
recognition](https://en.wikipedia.org/wiki/Optical_character_recognition)
is a technology that converts images of typed or handwritten text, such as
in a scanned document, into computer text that can be selected, searched and copied.

OCRmyPDF uses
[Tesseract](https://github.com/tesseract-ocr/tesseract), a widely
available open source OCR engine, to perform OCR.

(raster-vector)=

## About PDFs

PDFs are page description files that attempt to preserve a layout
exactly. They contain [vector
graphics](http://vector-conversions.com/vectorizing/raster_vs_vector.html)
that can contain raster objects, such as scanned images. Because PDFs can
contain multiple pages (unlike many image formats) and can contain fonts
and text, they are a suitable format for exchanging scanned documents.

:::{image} images/bitmap_vs_svg.svg
:::

A PDF page may contain multiple images, even if it appears to have only
one image. Some scanners or scanning software may segment pages into
monochromatic text and color regions, for example, to enhance the compression
ratio and appearance of the page.

Rasterizing a PDF is the process of generating corresponding raster images.
OCR engines like Tesseract work with images, not scalable vector graphics
or mixed raster-vector-text graphics such as PDF.

## About PDF/A

[PDF/A](https://en.wikipedia.org/wiki/PDF/A) is an ISO-standardized
subset of the full PDF specification that is designed for archiving (the
'A' stands for Archive). PDF/A differs from PDF primarily by omitting
features that could complicate future file readability,
such as embedded Javascript, video, audio and references to external
fonts. All fonts and resources needed to interpret the PDF must be
contained within it. Because PDF/A disables Javascript and other types
of embedded content, it is likely more secure.

There are various conformance levels and versions, such as "PDF/A-2b".

In general, the preferred format for scanned documents is PDF/A. Some
governments and jurisdictions, US Courts in particular, [mandate the use
of PDF/A](https://pdfblog.com/2012/02/13/what-is-pdfa/) for scanned
documents.

Since most individuals scanning documents aim for long-term readability,
OCRmyPDF defaults to generating PDF/A-2b.

PDF/A does have a few drawbacks. Some PDF viewers display an alert
indicating that the file is in PDF/A format, which may confuse some users.
Additionally, it tends to result in larger files than standard PDFs because
it embeds certain resources, even if they are widely available. PDF/A
files can be digitally signed but may not be encrypted to ensure future
readability. Fortunately, converting from PDF/A to a regular PDF is
straightforward, and any PDF viewer can handle PDF/A files.

## What OCRmyPDF does

OCRmyPDF analyzes each page of a PDF to determine the required colorspace
and resolution (DPI) for capturing all the information on that page without
losing content. It uses a PDF rasterizer (pypdfium2 or
[Ghostscript](http://ghostscript.com/)) to convert each page to an image and
subsequently performs OCR on the rasterized image to generate an OCR "layer."
This layer is then integrated back into the original PDF.

:::{versionchanged} 17.0.0
OCRmyPDF now supports pypdfium2 as an alternative rasterizer to Ghostscript.
pypdfium2 is a Python binding for pdfium, the PDF rendering library used by
Google Chrome. The `--rasterizer auto` setting (default) prefers pypdfium2
when available.
:::

While it is possible to use a program like Ghostscript or ImageMagick to
obtain an image and then run that image through Tesseract OCR, this process
actually generates a new PDF, potentially resulting in the loss of various
details (such as the document's metadata). In contrast, OCRmyPDF can produce
a minimally altered PDF as the output.

OCRmyPDF also offers several image processing options, such as deskew, which
enhances the visual quality of files and the accuracy of OCR. When these
options are utilized, the OCR layer is integrated into the processed image.

By default, OCRmyPDF generates archival PDFs in the PDF/A format, which is
a more rigid subset of PDF features designed for long-term archives. If you
prefer regular PDFs, you can disable this feature using the
`--output-type pdf` option.

## Why you shouldn't do this manually

A PDF is similar to an HTML file, in that it contains document structure
along with images. While some PDFs may solely display a full-page image,
they often contain additional content that would be forfeited if not preserved.

A manual process could take one of these approaches:

1. Rasterize each page as an image, perform OCR on the images, and then merge the
   output into a PDF. This method preserves the layout of each page, but
   resamples all images potentially leading to quality loss, increased file size,
   and the introduction of compression artifacts, among other issues.
2. Extract each image, OCR, and combine the output into a PDF. This approach
   loses the context in which images are used in the PDF, potentially resulting
   in loss of information related to scaling and position of images. Some scanned
   PDFs contain multiple images segmented into black and white, grayscale
   and color regions, with stencil masks to prevent overlap, as this can
   enhance the appearance of a file while reducing file size.
   Reassembling these images can be challenging, and risks losing vector art
   or text that is not part of an image.

In cases where a PDF solely serves as a container for images without any
rotation, scaling, or cropping, the second approach can be lossless.

OCRmyPDF uses various strategies depending on input options and the input PDF
itself. Generally, it rasterizes a page for OCR and then integrates the OCR
data back into the original PDF. This approach allows it to handle complex
PDFs and preserve their content as much as possible.

Furthermore, OCRmyPDF supports a wide range of edge cases that have emerged
during several years of development. It accommodates PDF features like
images within Form XObjects and pages with UserUnit scaling. It also
supports less common image formats like non-monochrome 1-bit images and
provides warnings about files you may not want to OCR. Thanks to tools
like pikepdf and QPDF, it can auto-repair damaged PDFs. You don't need to
understand the intricacies of these issues; you should be able to use
OCRmyPDF with any PDF file, and expect reasonable results.

## Limitations

OCRmyPDF is subject to limitations imposed by the Tesseract OCR engine.
These limitations are inherent to any software relying on Tesseract:

- The OCR accuracy may not match that of commercial OCR solutions.
- It is incapable of recognizing handwriting.
- It may detect gibberish and report it as OCR output.
- Results may be subpar when a document contains languages not specified
  in the `-l LANG` argument.
- Tesseract may struggle to analyze the natural reading order of documents.
  For instance, it might fail to recognize two columns in a document and
  attempt to join text across columns.
- Poor quality scans can result in subpar OCR quality. In other words, the
  quality of the OCR output depends on the quality of the input.
- Tesseract does not provide information about the font family to which text
  belongs.
- Tesseract does not divide text into paragraphs or headings. It only provides
  the text and its bounding box. As such, the generated PDF does not
  contain any information about the document's structure.

### Ghostscript considerations

:::{versionchanged} 17.0.0
Ghostscript is no longer strictly required. OCRmyPDF can use pypdfium2
for rasterization and produce PDF/A without Ghostscript.
:::

:::{versionchanged} 17.13.0
PDF/A produced without Ghostscript is checked by pikepdf's PDF/A validator
(`pikepdf.pdfa`). veraPDF is no longer used.
:::

While Ghostscript remains a capable and feature-rich tool with a long history,
recent releases have introduced some compatibility challenges that OCRmyPDF
v17 addresses through alternative codepaths. When Ghostscript is used:

- PDFs containing JPEG 2000-encoded content may be converted to JPEG
  encoding, which may introduce compression artifacts, if Ghostscript
  PDF/A is enabled.
- Ghostscript may transcode grayscale and color images, potentially
  lossily, based on an internal algorithm. By default
  (`--pdfa-image-compression=auto`) OCRmyPDF selects lossless image
  compression at `-O0` so Ghostscript will not transcode lossless images
  to JPEG. At `-O1` (the default optimization level) and above, `auto`
  defers to Ghostscript's heuristic instead; `-O1` is a historical
  exception, kept for backwards compatibility because coercing it to
  lossless can substantially bloat output. You can override this by
  setting `--pdfa-image-compression` to `jpeg` or `lossless` to force all
  images to one type or the other. `lossless` passes existing JPEGs
  through untouched (re-encoding them losslessly would only inflate them)
  while encoding non-JPEG images losslessly.
  (Modern Ghostscript can copy JPEG images without transcoding them.)
  Advanced users can also tune Ghostscript's image recompression with
  `--ghostscript-jpeg-quality` and `--ghostscript-jpeg-maxdpi`; see
  [Advanced Ghostscript tuning](advanced.md#advanced-ghostscript-tuning).
  Most users should prefer `--jpeg-quality` (applied by the OCRmyPDF
  optimizer) over those Ghostscript-scoped controls.
- Ghostscript's PDF/A conversion removes any XMP metadata that is not
  one of the standard XMP metadata namespaces for PDFs. In particular,
  PRISM Metadata is removed.
- Ghostscript's PDF/A conversion may remove or deactivate
  hyperlinks and other active content.

Many of these limitations are avoided by the speculative PDF/A conversion
path, which OCRmyPDF tries first for all PDF/A output types and uses
whenever the result passes pikepdf's PDF/A validator. Use
`--pdfa-backend internal` to never use Ghostscript for PDF/A, or
`--pdfa-backend ghostscript` to always use it. Install pypdfium2
to avoid Ghostscript for rasterization as well.

You can use `--output-type pdf` to disable PDF/A conversion and produce
a standard, non-archival PDF.

Regarding OCRmyPDF itself:

- PDFs using transparency are not currently represented in the test
  suite

## Similar programs

To the author's knowledge, OCRmyPDF is the most feature-rich and
thoroughly tested command line OCR PDF conversion tool. If it does not
meet your needs, contributions and suggestions are welcome.

Ghostscript recently added three "pdfocr" output devices. They work by
rasterizing all content and converting all pages to a single colour space.

## Web front-ends

The Docker image of OCRmyPDF provides a web service front-end
that allows files to submitted over HTTP, and the results can be downloaded.
This is an HTTP server intended to demonstrate how OCRmyPDF can be
integrated into a web service. It is not intended to be deployed on the
public internet and does not provide any security measures.

In addition, the following third-party integrations are available:

- [Paperless-ngx](https://docs.paperless-ngx.com/) is a free software
  document management system that uses OCRmyPDF to perform OCR on
  uploaded documents.
- [Nextcloud OCR](https://github.com/janis91/ocr) is a free software
  plugin for the Nextcloud private cloud software.

OCRmyPDF is not designed to be secure against malware-bearing PDFs (see
[Using OCRmyPDF online](ocr-service)). Users should ensure they
comply with OCRmyPDF's licenses and the licenses of all dependencies. In
particular, OCRmyPDF requires Ghostscript, which is licensed under
AGPLv3.

---

### Jbig2 (`docs/jbig2.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

{#jbig2}

# Installing the JBIG2 encoder

Many Linux distributions do not include a JBIG2 encoder since JBIG2
encoding was patented for a long time. All known JBIG2 US patents have
expired as of 2017, but it is possible that unknown patents exist.

JBIG2 encoding is recommended for OCRmyPDF and is used to losslessly
create smaller PDFs. If JBIG2 encoding is not available, lower quality
CCITT encoding will be used for monochrome images.

JBIG2 decoding is not patented and is performed automatically by most
PDF viewers. It is widely supported and has been part of the PDF
specification since 2001.

JBIG2 encoding is automatically provided by these OCRmyPDF packages: 
- Docker image (both Ubuntu and Alpine)
- Snap package
- Alpine Linux package
- ArchLinux AUR package (as an optional dependency)
- Debian package
- Ubuntu package
- Homebrew on macOS

For all other platforms, you would need to build the JBIG2 encoder from
source:

:::{code} bash
git clone https://github.com/agl/jbig2enc
cd jbig2enc
./autogen.sh
./configure && make
[sudo] make install
:::

Dependencies include libtoolize and libleptonica, which on Ubuntu
systems are packaged as libtool and libleptonica-dev. On Fedora (35)
they are packaged as libtool and leptonica-devel. For this to work,
please make sure to install `autotools`, `automake`, `libtool`, `pkg-config`
and `leptonica` first if not already installed. Other dependencies might
be required depending on your system.

:::{code} bash
[sudo] apt install autotools-dev automake libtool libleptonica-dev pkg-config
:::

## JBIG2 Compression

OCRmyPDF uses JBIG2 lossless compression for bitonal (black and white)
images. This provides excellent compression ratios compared to the older
CCITT G4 standard, while preserving the exact pixel content of the
original image.

You can adjust the threshold for JBIG2 compression with
`--jbig2-threshold`. The default is 0.85.

:::{note}
Previous versions of OCRmyPDF supported a lossy JBIG2 mode
(`--jbig2-lossy`). This feature has been removed due to the well-known
risk of character substitution errors (e.g., 6/8 confusion). See
[JBIG2 disadvantages](https://en.wikipedia.org/wiki/JBIG2#Disadvantages)
for more information on why lossy JBIG2 is problematic. The `--jbig2-lossy`
and `--jbig2-page-group-size` arguments are now ignored with a warning.
:::

---

### Languages (`docs/languages.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

(lang-packs)=

# Installing additional language packs

OCRmyPDF uses Tesseract for OCR, and relies on its language packs for all languages.
On most platforms, English is installed with Tesseract by default, but not always.

Tesseract supports [most
languages](https://github.com/tesseract-ocr/tesseract/blob/main/doc/tesseract.1.asc#languages).
Languages are identified by standardized three-letter codes (called ISO 639-2 Alpha-3).
Tesseract's documentation also lists the three-letter code for your language.
Some are anglicized, e.g. Spanish is `spa` rather than `esp`, while others
are not, e.g. German is `deu` and French is `fra`.

Language packs (strictly speaking, Tesseract "traineddata" files) generally correspond
to the language in question, but different language packs are used in certain
situations. For German, the "Fraktur" language pack can assist with reading older
materials in the Fraktur typeface family (`deu_frak`). Some communities have changed
their script from Cyrillic to Latin; the Cyrillic version of Uzbek is available
as `uzb_cyrl` and the Latin version is `uzb`.

After you have installed a language pack, you can use it with `ocrmypdf -l <language>`,
for example `ocrmypdf -l spa`. For multilingual documents, you can specify
all languages to be expected, e.g. `ocrmypdf -l eng+fra` for English and French.
English is assumed by default unless other language(s) are specified.

For Linux users, you can often find packages that provide language
packs.

## Platform install steps

### Debian and Ubuntu (apt)

```bash
# Display a list of all Tesseract language packs
apt-cache search tesseract-ocr

# Install Chinese Simplified language pack
apt-get install tesseract-ocr-chi-sim
```

You can then pass the `-l LANG` argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either `-l eng+fra` (English and French) or
`-l eng -l fra`.

### Fedora

```bash
# Display a list of all Tesseract language packs
dnf search tesseract

# Install Chinese Simplified language pack
dnf install tesseract-langpack-chi_sim
```

You can then pass the `-l LANG` argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either `-l eng+fra` (English and French) or
`-l eng -l fra`.

### Arch Linux

```bash
# Display a list of all Tesseract language packs
pacman -Ss tesseract-data

# Install German language pack
pacman -S tesseract-data-deu
```

You can then pass the `-l LANG` argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either `-l eng+fra` (English and French) or
`-l eng -l fra`.

### Gentoo

On Gentoo the package `app-text/tessdata_fast`, which `app-text/tesseract` depends on, handles Tesseract languages.
It accepts USE flags to select what languages should be installed, these can be set in `/etc/portage/package.use`.
Alternatively one can globally set the [L10N use extension](https://wiki.gentoo.org/wiki/Localization/Guide#L10N) in `/etc/portage/make.conf`.
This enables these languages for all packages (e.g. including aspell).

```bash
# Display a list of all Tesseract language packs
equery uses app-text/tessdata_fast

# Add English and German language support for Tesseract only
echo 'app-text/tessdata_fast l10n_de l10n_en' >> /etc/portage/package.use

# Add global English and German language support (the `l10n_` from equery has to be omitted)
echo L10N="de en" >> /etc/portage/make.conf

# update system to reflect changed USE flags
emerge --update --deep --newuse @world
```

You can then pass the `-l LANG` argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either `-l eng+fra` (English and French) or
`-l eng -l fra`.

### macOS

You can install additional language packs by
{ref}`installing Tesseract using Homebrew with all language packs <macos-all-languages>`.

### Docker

Users of the OCRmyPDF Docker image should install language packs into a
derived Docker image as
{ref}`described in that section <docker-lang-packs>`.

### Windows

The Tesseract installer provided by Chocolatey currently includes only English language.
To install other languages, download the respective language pack (`.traineddata` file)
from <https://github.com/tesseract-ocr/tessdata/> and place it in
`C:\\Program Files\\Tesseract-OCR\\tessdata` (or wherever Tesseract OCR is installed).

## Custom language packs

If you have fine-tuned or trained Tesseract and generated custom trained data, you can
copy your `customlang.traineddata` file into your Tesseract "tessdata" folder, and
then use the `-l customlang` argument to tell OCRmyPDF to pass that language on to
Tesseract.

---

### Maintainers (`docs/maintainers.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Maintainer notes

This is for those who package OCRmyPDF for downstream use. (Thank you
for your hard work.)

## Known ports/packagers

OCRmyPDF has been ported to many platforms already. If you are
interesting in porting to a new platform, check with
[Repology](https://repology.org/projects/?search=ocrmypdf) to see the
status of that platform.

### Make sure you can package pikepdf

pikepdf, created by the same author, is a mixed Python and C++14 package
with much stiffer build requirements. If you want to use OCRmyPDF on
some novel platform or distribution, first make sure you can package
pikepdf.

### Core dependencies

:::{versionchanged} 17.0.0
Ghostscript is no longer strictly required. OCRmyPDF now supports alternative
codepaths for both PDF rasterization and PDF/A conversion.
:::

OCRmyPDF has the following runtime dependencies:

**For PDF rasterization** (converting PDF pages to images for OCR):

- `pypdfium2` (Python package) - OR -
- `ghostscript` (system binary)
- Recommendation: Install both for best compatibility

**For PDF/A conversion**:

- Speculative conversion with pikepdf, checked by pikepdf's PDF/A validator
  (`pikepdf.pdfa`, which needs the `pikepdf[pdfa]` extra: `jsonschema`,
  `referencing` and `fonttools`); Ghostscript is used for files that fail
  validation
- `ghostscript` (system binary)
- Recommendation: Install Ghostscript for files that need real conversion.
  veraPDF is no longer used at runtime; the test suite uses it, if
  installed, to cross-check pikepdf's validator.

**For OCR**:
- `tesseract-ocr` (system binary) - Required for MVP

**For text rendering** (expressing OCR results in PDF):
- `fpdf2` (Python package) - Required for text layer rendering
- `uharfbuzz` (Python package) - Required for text layer rendering
- Noto fonts (system package) - Recommended for text layer rendering.
  `fonts-noto` on Debian/Ubuntu, `google-noto-fonts-all` on Fedora; Homebrew
  has no single Noto package, only per-family casks such as `font-noto-sans`.

**Other dependencies**:
- `unpaper` (system binary) - Optional, enables `--clean` and `--clean-final`
- `pngquant` (system binary) - Optional, enables `--optimize 2` and `--optimize 3`
- `jbig2enc` (system binary) - Optional, improves compression of monochrome images

While Ghostscript remains a capable and feature-rich tool with a long history,
recent releases have introduced some compatibility challenges that OCRmyPDF v17
addresses through alternative codepaths. For the best user experience, packagers
should install both Ghostscript and the alternative tools (pypdfium2)
when available.

On Windows, OCRmyPDF will also check the registry for Tesseract and Ghostscript
locations.

Tesseract OCR relies on SIMD for performance and only has proper support
for this on ARM and x86\_64. Performance may be poor on other processor
architectures.

### Versioning scheme

OCRmyPDF uses hatch-vcs for versioning, which derives the version from
Git as a single source of truth. This may be unsuitable for some
distributions, e.g. to indicate that your distribution modifies OCRmyPDF
in some way.

You can patch the `__version__` variable in `src/ocrmypdf/_version.py`
if necessary, or set the environment variable
`SETUPTOOLS_SCM_PRETEND_VERSION` to the required version, if you need to
override versioning for some reason.

### jbig2enc

OCRmyPDF will use jbig2enc, a JBIG2 encoder, if one can be found. Some
distributions have shied away from packaging JBIG2 because it contains
patented algorithms, but all patents have expired since 2017. If
possible, consider packaging it too to improve OCRmyPDF's compression.

:::{note}
Lossy JBIG2 encoding has been removed in v17.0.0 due to well-documented
risks of character substitution errors. Previously we provided this feature
on a "caveat emptor" basis but in the interest of focusing and eliminating
risks, we decided to remove this option. Now, only lossless JBIG2 compression
is supported.
:::

### Dependency matrix for packagers

:::{versionadded} 17.0.0
:::

The following table summarizes the dependency options introduced in v17.0.0:

| Feature | Option 1 | Option 2 | Notes |
|---------|----------|----------|-------|
| PDF rasterization | pypdfium2 (Python) | ghostscript (binary) | pypdfium2 preferred when available |
| PDF/A conversion | pikepdf (`pikepdf.pdfa`) | ghostscript | pikepdf validates speculative conversion |
| Text rendering | fpdf2 (Python) | - | Required, replaces legacy hOCR renderer |
| OCR | tesseract-ocr | `--ocr-engine none` | Can be skipped entirely |

**Minimum viable installation:**

- tesseract-ocr + (pypdfium2 OR ghostscript) + fpdf2

**Recommended installation:**

- tesseract-ocr + pypdfium2 + ghostscript + fpdf2 + unpaper + pngquant + jbig2enc

:::{warning}
If Ghostscript is not installed, PDF/A output can only be produced for files
that pass pikepdf's PDF/A validator after speculative conversion. Other files
are output as a standard PDF (`--output-type auto`) or fail with an error
(`--output-type pdfa`).
:::

**Sample debian/control dependency specification**

```
Depends:
 fonts-noto,
 fpdf2 (>= 2.8),
 ghostscript (>= 9.55),  # Not strictly required, but best user experience
 icc-profiles-free,
 img2pdf,
 python3-coloredlogs,
 python3-deprecation,
 python3-fonttools (>= 4.40),  # for pikepdf.pdfa
 python3-jsonschema (>= 4.18),  # for pikepdf.pdfa
 python3-pdfminer (>= 20181108+dfsg-3),
 python3-pikepdf (>= 10.17),
 python3-referencing,  # for pikepdf.pdfa
 python3-pil,
 python3-pluggy,
 python3-reportlab,
 python3-rich,
 python3-uharfbuzz,  # Not currently in Debian
 tesseract-ocr (>= 5.0.0),
 zlib1g,
 ${misc:Depends},
 ${python3:Depends},
Recommends:
 cyclopts,   # Not currently in Debian
 jbig2
 paddleocr,  # Not currently in Debian
 pngquant,
 pypdfium2,  # Not currently in Debian
 unpaper,
Suggests:
 ocrmypdf-doc,
 python-watchfiles,
```

### Command line completions

Please ensure that command line completions are installed, as described
in the installation documentation.

### 32-bit Linux support

If you maintain a Linux distribution that supports 32-bit x86 or ARM,
OCRmyPDF should continue to work as long as all of its dependencies
continue to be available in 32-bit form. Please note we do not test on
32-bit platforms.

### HEIF/HEIC

HEIF/HEIC image support is an optional extra (`ocrmypdf[heic]`), backed by
the pillow-heif PyPI package; it is not installed by default and is not
included in the Docker image or snap. Its binary wheels bundle libheif,
libde265, and x265, which makes the wheel as a whole GPLv2-licensed, so
this stays opt-in rather than a default or bundled dependency. If not
installed, OCRmyPDF gracefully degrades and loses only support for this
feature.

---

### Optimizer (`docs/optimizer.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# PDF optimization

OCRmyPDF includes an image-oriented PDF optimizer. By default, the
optimizer runs with safe settings with the goal of improving compression
at no loss of quality. At higher optimization levels, lossy
optimizations may be applied and tuned. Optimization occurs after OCR,
and only if OCR succeeded. It does not perform other possible
optimizations such as deduplicating resources, consolidating fonts,
simplifying vector drawings, or anything of that nature.

:::{list-table} OCRmyPDF optimization settings
---
widths: 33 6 60
header-rows: 1
---

* - Optimization level
  - Shorthand
  - Description
* - ``--optimize 0``
  - ``-O0``
  - Disable most optimizations.
* - ``--optimize 1`` (default)
  - ``-O1``
  - Enables lossless optimizations, such as transcoding images to more
      efficient formats. Also compress other uncompressed objects in the
      PDF and enables the more efficient "object streams" within the PDF.
* - ``--optimize 2``
  - ``-O2``
  - All of the above, and enables lossy optimizations and color quantization.
* - ``--optimize 3``
  - ``-O3``
  - All of the above, and enables more aggressive optimizations and targets lower
      image quality.
:::

The exact type of optimizations performed will vary over time, and
depend on what third party tools are installed.

Despite optimizations, OCRmyPDF might still increase the overall file
size, since it must embed information about the recognized text, and
depending on the settings chosen, may not be able to represent the
output file as compactly as the input file.

## Optimizations that always occurs

OCRmyPDF will automatically replace obsolete or inferior compression
schemes such as RLE or LZW with superior schemes such as Deflate, and
convert monochrome images to CCITT G4. Since this is lossless, it always
occurs and there is no way to disable it. Other non-image compressed
objects are compressed as well.

## Fast web view

OCRmyPDF automatically optimizes PDFs for \"fast web view\" in Adobe
Acrobat\'s parlance, or equivalently, linearizes PDFs so that the
resources they reference are presented in the order a viewer needs them
for sequential display. This reduces the latency of viewing a PDF both
online and from local storage, in exchange for a slight increase in file
size.

To disable this optimization and all others, use
`ocrmypdf --optimize 0 ...` or the shorthand `-O0`.

Adobe Acrobat might not report the file as being \"fast web view\".

## Lossless optimizations

At optimization level `-O1` (the default), OCRmyPDF will also attempt
lossless image optimization.

If a JBIG2 encoder is available, then monochrome images will be
converted to JBIG2, with the potential for huge savings on large black
and white images, since JBIG2 is far more efficient than any other
monochrome (bi-level) compression. (All known US patents related to
JBIG2 have probably expired, but it remains the responsibility of the
user to supply a JBIG2 encoder such as
[jbig2enc](https://github.com/agl/jbig2enc). OCRmyPDF does not implement
JBIG2 encoding on its own.)

OCRmyPDF currently does not attempt to recompress losslessly compressed
objects more aggressively.

## Lossy optimizations

At optimization level `-O1`, `-O2` and `-O3`, OCRmyPDF will some attempt
loss image optimization.

If Ghostscript is used to create a PDF/A (the default), Ghostscript will
optimize some images by converting them to JPEG, which are lossy. If
`--output-type pdf` is used, there are no lossy optimizations. Ghostscript's
JPEG conversion is quite safe.

If `pngquant` is installed, OCRmyPDF will use it to perform quantize
paletted images to reduce their size.

The quality of JPEGs may be lowered, on the assumption that a lower
quality image may be suitable for storage after OCR. Use `--jpeg-quality`
to control the optimizer's JPEG quality target. The optimizer is the
recommended way to reduce JPEG image sizes: it applies consistently
regardless of whether Ghostscript was used to produce a PDF/A.

If you specifically need to tune Ghostscript's own PDF/A image handling
(for example, to force a hard DPI cap), see
[Advanced Ghostscript tuning](advanced.md#advanced-ghostscript-tuning)
for the separate `--ghostscript-jpeg-quality` and
`--ghostscript-jpeg-maxdpi` options.

It is not possible to optimize all image types. Uncommon image types may
be skipped by the optimizer.

---

### Pdfsecurity (`docs/pdfsecurity.md`)

(security)=

# PDF security issues

> OCRmyPDF should only be used on PDFs you trust. It is not designed to
> protect you against malware.

Recognizing that many users have an interest in handling PDFs and
applying OCR to PDFs they did not generate themselves, this article
discusses the security implications of PDFs and how users can protect
themselves.

The disclaimer applies: this software has no warranties of any kind.

## PDFs may contain malware

PDF is a rich, complex file format. The official PDF 1.7 specification,
ISO 32000:2008, is hundreds of pages long and references several annexes
each of which are similar in length. PDFs can contain video, audio, XML,
JavaScript and other programming, and forms. In some cases, they can
open internet connections to pre-selected URLs. All of these are
possible attack vectors.

In short, PDFs [may contain
viruses](https://security.stackexchange.com/questions/64052/can-a-pdf-file-contain-a-virus).

If you do not trust a PDF or its source, do not open it or use OCRmyPDF
on it. Consider using a Docker container or virtual machine to isolate
an untrusted PDF from your system.

## How OCRmyPDF processes PDFs

OCRmyPDF must open and interpret your PDF in order to insert an OCR
layer. First, it runs all PDFs through
[pikepdf](https://github.com/pikepdf/pikepdf), a library based on
[QPDF](https://github.com/qpdf/qpdf), a program that repairs PDFs with
syntax errors. This is done because, in the author\'s experience, a
significant number of PDFs in the wild, especially those created by
scanners, are not well-formed files. QPDF makes it more likely that
OCRmyPDF will succeed, but offers no security guarantees. QPDF is also
used to split the PDF into single page PDFs.

Finally, OCRmyPDF rasterizes each page of the PDF using
[Ghostscript](http://ghostscript.com/) in `-dSAFER` mode.

Depending on the options specified, OCRmyPDF may graft the OCR layer
into the existing PDF or it may essentially reconstruct (\"re-fry\") a
visually identical PDF that may be quite different at the binary level.
That said, OCRmyPDF is not a tool designed for sanitizing PDFs.

## Password protected PDFs

Password protected PDFs usually have two passwords, and owner and user
password. When the user password is set to empty, PDF readers will open
the file automatically and mark it as \"(SECURED)\". Password security
can also request certain restrictions on the PDF, but anyone can remove
these restrictions if they have either the owner *or* user password.
Passwords mainly present a barrier for casual users.

OCRmyPDF cannot remove passwords from PDFs. If you want to remove a
password from a PDF, you must use other software, such as `qpdf`.

If the owner and user password are set, a password is required for
`qpdf`. If only the owner password is set, then the password can be
stripped, even if one does not have the owner password. To remove the
password from a using QPDF, use:

:::{code} bash
qpdf --decrypt --password='abc123' input.pdf no_password.pdf
:::

Then you can run OCRmyPDF on the file.

In its default mode, OCRmyPDF generates PDF/A. Passwords may not be set
on PDF/A documents. If you want to set a password on the output PDF, you
must specify `--output-type pdf`.

## Signature images

Many programs exist which are capable of inserting an image of
someone\'s signature. On its own, this offers no security guarantees. It
is trivial to remove the signature image and apply it to other files.
This practice offers no real security.

## Digital signatures

Important documents can be digitally signed and certified to attest to
their authorship, approval or execution of a legal agreement. OCRmyPDF
will detect signed PDFs and will not modify them, unless the
`--invalidate-digital-signatures` option is used, which will invalidate
any signatures. (The signature may still be present in the PDF if
opened, but PDF readers will not validate it.)

A digital signature adds a cryptographic hash of the document to the
document, so tamper protection is provided. That also precludes OCRmyPDF
from modifying the document and preserving the signature.

Digital signatures are not the same as a signature image. A digital
signature is a cryptographic hash of the document that is encrypted with
the author\'s private key. The signature is decrypted with the author\'s
public key. The public key is usually distributed by a certificate
authority. The signature is then verified by the PDF reader. If the
document is modified, the signature will be invalidated.

## Certificate-encrypted PDFs

PDFs can be encrypted with a certificate. This is a more secure form of
encryption than a password. The certificate is usually issued by a
certificate authority. A certificate is used to encrypt the document
using the public key for the benefit of a specific recipient who
possesses the private key.

OCRmyPDF cannot open certificate-encrypted PDFs. If you have the
certificate, you can use other PDF software, such as Acrobat, to decrypt
the PDF.

---

### Performance (`docs/performance.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Performance

Some users have noticed that current versions of OCRmyPDF do not run as
quickly as some older versions (specifically 6.x and older). This is
because OCRmyPDF added image optimization as a postprocessing step, and
it is enabled by default.

## Speed

If running OCRmyPDF quickly is your main goal, you can use settings such
as:

-   `--optimize 0` to disable file size optimization
-   `--output-type pdf` to disable PDF/A generation
-   `--fast-web-view 999999` to disable fast web view optimization
-   `--skip-big` to skip large images, if some pages have large images

You can also avoid:

-   `--force-ocr`
-   Image preprocessing

## Memory

Peak memory is set by the number of *pixels* in the largest page, not by the
size of the PDF on disk. OCRmyPDF rasterizes each page at the resolution of the
images it contains, and there is no ceiling on that by default, so a 1200 dpi
scan costs four times as much as the same page at 600 dpi.

As a rule of thumb, budget about **15 bytes per pixel of the largest page, per
concurrent job**. A 34 megapixel page (letter size at 600 dpi) therefore peaks
at roughly 500 MB with `--jobs 1`, and roughly 2 GB with `--jobs 4`. Three
things account for that:

-   the OCR engine, which is the largest single consumer -- Tesseract peaks at
    about 12 bytes per pixel of the image it is given
-   rasterizing the page, which holds the renderer's bitmap and the copy taken
    from it at the same time
-   masking and writing the image that goes to OCR

They run one after another rather than at once, so the peak is the largest of
them, not their sum -- but each concurrent job pays it separately.

To bound memory on files with very large images, use `--max-ocr-image-mpixels`.
It downsamples the image sent to OCR when a page exceeds the given size, which
caps the largest consumer:

```bash
ocrmypdf --max-ocr-image-mpixels 8 --jobs 2 input.pdf output.pdf
```

The visible page is never downsampled, so the appearance of the output is
unaffected in every mode, including `--force-ocr`. What you trade is OCR
accuracy on very high resolution scans. Tesseract is tuned for roughly 300 dpi
and gains little above 400 dpi, so a cap around 8 megapixels is usually free for
letter or A4 pages; on documents with unusually small print, check the results
before adopting it.

Two other settings matter:

-   `--jobs` multiplies the peak, since each worker processes its own page.
    Lowering it is the bluntest way to fit a memory limit.
-   `--max-image-mpixels` is *not* a memory control. It is Pillow's
    decompression-bomb guard, and it aborts the job rather than downsampling.

---

### Plugins (`docs/plugins.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# Plugins

> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
> NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
> "OPTIONAL" in this document are to be interpreted as described in
> RFC 2119.

You can use plugins to customize the behavior of OCRmyPDF at certain points of
interest.

Currently, it is possible to:

- add new command line arguments
- override the decision for whether or not to perform OCR on a particular file
- modify the image is about to be sent for OCR
- modify the page image before it is converted to PDF
- replace the Tesseract OCR with another OCR engine that has similar behavior
- replace Ghostscript with another PDF to image converter (rasterizer) or
  PDF/A generator

OCRmyPDF plugins are based on the Python `pluggy` package and conform to its
conventions. Note that: plugins installed with as setuptools entrypoints are
not checked currently, because OCRmyPDF assumes you may not want to enable
plugins for all files.

See \[OCRmyPDF-EasyOCR\](<https://github.com/ocrmypdf/OCRmyPDF-EasyOCR>) for an
example of a straightforward, fully working plugin.

## Script plugins

Script plugins may be called from the command line, by specifying the name of a file.
Script plugins may be convenient for informal or "one-off" plugins, when a certain
batch of files needs a special processing step for example.

```bash
ocrmypdf --plugin ocrmypdf_example_plugin.py input.pdf output.pdf
```

Multiple plugins may be installed by issuing the `--plugin` argument multiple times.

## Packaged plugins

Installed plugins may be installed into the same virtual environment as OCRmyPDF
is installed into. They may be invoked using Python standard module naming.
If you are intending to distribute a plugin, please package it.

```bash
ocrmypdf --plugin ocrmypdf_fancypants.pockets.contents input.pdf output.pdf
```

OCRmyPDF does not automatically import plugins, because the assumption is that
plugins affect different files differently and you may not want them activated
all the time. The command line or `ocrmypdf.ocr(plugin='...')` must call
for them.

Third parties that wish to distribute packages for ocrmypdf should package them
as packaged plugins, and these modules should begin with the name `ocrmypdf_`
similar to `pytest` packages such as `pytest-cov` (the package) and
`pytest_cov` (the module).

:::{note}
We recommend plugin authors name their plugins with the prefix
`ocrmypdf-` (for the package name on PyPI) and `ocrmypdf_` (for the
module), just like pytest plugins. At the same time, please make it clear
that your package is not official.
:::

## Plugins

You can also create a plugin that OCRmyPDF will always automatically load if both are
installed in the same virtual environment, using a project entrypoint.
OCRmyPDF uses the entrypoint namespace "ocrmypdf".

For example, `pyproject.toml` would need to contain the following, for a plugin named
`ocrmypdf-exampleplugin`:

```toml
[project]
name = "ocrmypdf-exampleplugin"

[project.entry-points."ocrmypdf"]
exampleplugin = "exampleplugin.pluginmodule"
```

## Plugin requirements

OCRmyPDF generally uses multiple worker processes. When a new worker is started,
Python will import all plugins again, including all plugins that were imported earlier.
This means that the global state of a plugin in one worker will not be shared with
other workers. As such, plugin hook implementations should be stateless, relying
only on their inputs. Hook implementations may use their input parameters to
to obtain a reference to shared state prepared by another hook implementation.
Plugins must expect that other instances of the plugin will be running
simultaneously.

The `context` object that is passed to many hooks can be used to share information
about a file being worked on. Plugins must write private, plugin-specific data to
a subfolder named `{options.work_folder}/ocrmypdf-plugin-name`. Plugins MAY
read and write files in `options.work_folder`, but should be aware that their
semantics are subject to change.

OCRmyPDF will delete `options.work_folder` when it has finished OCRing
a file, unless invoked with `--keep-temporary-files`.

The documentation for some plugin hooks contain a detailed description of the
execution context in which they will be called.

Plugins should be prepared to work whether executed in worker threads or worker
processes. Generally, OCRmyPDF uses processes, but has a semi-hidden threaded
argument that simplifies debugging.

## Plugin hooks

A plugin may provide the following hooks. Hooks must be decorated with
`ocrmypdf.hookimpl`, for example:

```python
from ocrmypdf import hookimpl


@hookimpl
def add_options(parser):
    pass
```

The following is a complete list of hooks that are available, and when
they are called.

(firstresult)=

**Note on firstresult hooks**

If multiple plugins install implementations for this hook, they will be called in
the reverse of the order in which they are installed (i.e., last plugin wins).
When each hook implementation is called in order, the first implementation that
returns a value other than `None` will "win" and prevent execution of all other
hooks. As such, you cannot "chain" a series of plugin filters together in this
way. Instead, a single hook implementation should be responsible for any such
chaining operations.

## Examples

- OCRmyPDF's test suite contains several plugins that are used to simulate certain
  test conditions.
- [ocrmypdf-papermerge](https://github.com/papermerge/OCRmyPDF_papermerge) is
  a production plugin that integrates OCRmyPDF and the Papermerge document
  management system.

### Suppressing or overriding other plugins

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.initialize
```

### Custom command line arguments

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.add_options
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.check_options
```

### Plugin option models

Plugins can define their own option models using Pydantic. This allows plugins to:

- Define type-safe option structures with validation
- Add CLI arguments that map to their option model fields
- Access options via nested namespaces (e.g., `options.tesseract.timeout`)

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.register_options
```

Plugin options can be accessed in two ways:

1. **Flat access** (backward compatible): `options.tesseract_timeout`
2. **Nested access**: `options.tesseract.timeout`

Both access patterns are equivalent and return the same values.

:::{note}
**Plugin Interface Change**: Starting in OCRmyPDF v17.0.0, plugin hooks receive
`OcrOptions` objects instead of `argparse.Namespace` objects. Most plugins will
continue working due to duck-typing compatibility, but plugin developers should
update their type hints accordingly.
:::

### Migration guide for plugin developers

:::{versionadded} 17.0.0
:::

**Update imports:**

```python
from ocrmypdf._options import OcrOptions
```

**Update type hints:**

```python
# Before (v16 and earlier)
def check_options(options: argparse.Namespace) -> None: ...


# After (v17+)
def check_options(options: OcrOptions) -> None: ...
```

**Attribute access unchanged:**

```python
# These work exactly as before
options.languages
options.output_type
options.tesseract_timeout
```

**Remove in-place modifications:**

```python
# Before (v16 pattern - no longer recommended)
def check_options(options):
    options.some_computed_value = compute_value(options)


# After (v17 pattern - compute at point of use)
def some_function(options):
    computed = compute_value(options)
    use_computed(computed)
```

### Execution and progress reporting

```{eval-rst}
.. autoclass:: ocrmypdf.pluginspec.ProgressBar
    :members:
    :special-members: __init__, __enter__, __exit__
```

```{eval-rst}
.. autoclass:: ocrmypdf.pluginspec.Executor
    :members:
    :special-members: __call__
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.get_logging_console
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.get_executor
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.get_progressbar_class
```

### Applying special behavior before processing

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.validate
```

### PDF page to image

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.rasterize_pdf_page
```

### Modifying intermediate images

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.filter_ocr_image
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.filter_page_image
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.filter_pdf_page
```

### OCR engine

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.get_ocr_engine
```

```{eval-rst}
.. autoclass:: ocrmypdf.pluginspec.OcrEngine
    :members:

    .. automethod:: __str__
```

```{eval-rst}
.. autoclass:: ocrmypdf.pluginspec.OrientationConfidence
```

### PDF/A production

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.generate_pdfa
```

### PDF optimization

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.optimize_pdf
```

```{eval-rst}
.. autofunction:: ocrmypdf.pluginspec.is_optimization_enabled
```

### Working with OcrElement trees

:::{versionadded} 17.0.0
:::

OCRmyPDF v17 introduces the `OcrElement` dataclass for representing OCR
output in an engine-agnostic format. This enables plugins to work with
OCR results without parsing hOCR XML.

**Key classes:**

```python
from ocrmypdf import OcrElement, OcrClass, BoundingBox

# OcrElement - represents any OCR structural unit
page = OcrElement(
    ocr_class=OcrClass.PAGE, bbox=BoundingBox(0, 0, 612, 792), children=[...]
)

# BoundingBox - axis-aligned bounding box (left, top, right, bottom)
bbox = BoundingBox(left=100, top=50, right=300, bottom=80)

# OcrClass - constants for element types
OcrClass.PAGE  # "ocr_page"
OcrClass.LINE  # "ocr_line"
OcrClass.WORD  # "ocrx_word"
OcrClass.PARAGRAPH  # "ocr_par"
```

**Navigating the tree:**

```python
# Get all words in a page
words = page.words  # Returns list[OcrElement]

# Get all lines
lines = page.lines

# Get combined text
text = page.get_text_recursive()

# Iterate by class
for para in page.paragraphs:
    print(para.get_text_recursive())
```

**OCR engine plugins:**

Plugins implementing custom OCR engines can now output `OcrElement` trees
directly via the `generate_ocr()` method, bypassing hOCR entirely:

```python
from pathlib import Path
from ocrmypdf.pluginspec import OcrEngine
from ocrmypdf import OcrElement, OcrClass, BoundingBox


class MyOcrEngine(OcrEngine):
    def generate_ocr(
        self,
        input_file: Path,
        options,
        context,
    ) -> OcrElement:
        # Perform OCR and return OcrElement tree directly
        # No need to generate hOCR XML
        return OcrElement(
            ocr_class=OcrClass.PAGE,
            bbox=BoundingBox(0, 0, width, height),
            dpi=300,
            children=[
                OcrElement(
                    ocr_class=OcrClass.LINE,
                    bbox=BoundingBox(100, 50, 500, 80),
                    children=[
                        OcrElement(
                            ocr_class=OcrClass.WORD,
                            bbox=BoundingBox(100, 50, 200, 80),
                            text="Hello",
                        ),
                        # ... more words
                    ],
                ),
                # ... more lines
            ],
        )

    def supports_generate_ocr(self) -> bool:
        return True  # Indicate this engine uses generate_ocr()
```

This approach is simpler than generating hOCR and allows modern OCR
engines to integrate more naturally with OCRmyPDF.

---

### Releasenotes Version02 (`docs/releasenotes/version02.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v2

## v2.2-stable (2014-09-29)

OCRmyPDF versions 1 and 2 were implemented as shell scripts. OCRmyPDF
3.0+ is a fork that gradually replaced all shell scripts with Python
while maintaining the existing command line arguments. No one is
maintaining old versions.

For details on older versions, see the [final version of its release
notes](https://github.com/fritz-hh/OCRmyPDF/blob/7fd3dbdf42ca53a619412ce8add7532c5e81a9d1/RELEASE_NOTES.md).

---

### Releasenotes Version03 (`docs/releasenotes/version03.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v3

## v3.2.1

Changes

- Fixed {issue}`47`
  "convert() got and unexpected keyword argument 'dpi'" by upgrading to
  img2pdf 0.2
- Tweaked the Dockerfiles

## v3.2

New features

- Lossless reconstruction: when possible, OCRmyPDF will inject text
  layers without otherwise manipulating the content and layout of a PDF
  page. For example, a PDF containing a mix of vector and raster
  content would see the vector content preserved. Images may still be
  transcoded during PDF/A conversion. (`--deskew` and
  `--clean-final` disable this mode, necessarily.)
- New argument `--tesseract-pagesegmode` allows you to pass page
  segmentation arguments to Tesseract OCR. This helps for two column
  text and other situations that confuse Tesseract.
- Added a new "polyglot" version of the Docker image, that generates
  Tesseract with all languages packs installed, for the polyglots among
  us. It is much larger.

Changes

- JPEG transcoding quality is now 95 instead of the default 75. Bigger
  file sizes for less degradation.

## v3.1.1

Changes

- Fixed bug that caused incorrect page size and DPI calculations on
  documents with mixed page sizes

## v3.1

Changes

- Default output format is now PDF/A-2b instead of PDF/A-1b
- Python 3.5 and macOS El Capitan are now supported platforms - no
  changes were needed to implement support
- Improved some error messages related to missing input files
- Fixed {issue}`20`: uppercase .PDF extension not accepted
- Fixed an issue where OCRmyPDF failed to text that certain pages
  contained previously OCR'ed text, such as OCR text produced by
  Tesseract 3.04
- Inserts /Creator tag into PDFs so that errors can be traced back to
  this project
- Added new option `--pdf-renderer=auto`, to let OCRmyPDF pick the
  best PDF renderer. Currently it always chooses the 'hocrtransform'
  renderer but that behavior may change.
- Set up Travis CI automatic integration testing

## v3.0

New features

- Easier installation with a Docker container or Python's `pip`
  package manager
- Eliminated many external dependencies, so it's easier to setup
- Now installs `ocrmypdf` to `/usr/local/bin` or equivalent for
  system-wide access and easier typing
- Improved command line syntax and usage help (`--help`)
- Tesseract 3.03+ PDF page rendering can be used instead for better
  positioning of recognized text (`--pdf-renderer tesseract`)
- PDF metadata (title, author, keywords) are now transferred to the
  output PDF
- PDF metadata can also be set from the command line (`--title`,
  etc.)
- Automatic repairs malformed input PDFs if possible
- Added test cases to confirm everything is working
- Added option to skip extremely large pages that take too long to OCR
  and are often not OCRable (e.g. large scanned maps or diagrams);
  other pages are still processed (`--skip-big`)
- Added option to kill Tesseract OCR process if it seems to be taking
  too long on a page, while still processing other pages
  (`--tesseract-timeout`)
- Less common colorspaces (CMYK, palette) are now supported by
  conversion to RGB
- Multiple images on the same PDF page are now supported

Changes

- New, robust rewrite in Python 3.4+ with
  [ruffus](http://www.ruffus.org.uk/index.html) pipelines

- Now uses Ghostscript 9.14's improved color conversion model to
  preserve PDF colors

- OCR text is now rendered in the PDF as invisible text. Previous
  versions of OCRmyPDF incorrectly rendered visible text with an image
  on top.

- All "tasks" in the pipeline can be executed in parallel on any
  available CPUs, increasing performance

- The `-o DPI` argument has been phased out, in favor of
  `--oversample DPI`, in case we need `-o OUTPUTFILE` in the future

- Removed several dependencies, so it's easier to install. We no longer
  use:

  - GNU [parallel](https://www.gnu.org/software/parallel/)
  - [ImageMagick](http://www.imagemagick.org/script/index.php)
  - Python 2.7
  - Poppler
  - [MuPDF](http://mupdf.com/docs/) tools
  - shell scripts
  - Java and [JHOVE](http://jhove.sourceforge.net/)
  - libxml2

- Some new external dependencies are required or optional, compared to
  v2.x:

  - Ghostscript 9.14+
  - [qpdf](http://qpdf.sourceforge.net/) 5.0.0+
  - [Unpaper](https://github.com/Flameeyes/unpaper) 6.1 (optional)
  - some automatically managed Python packages

Release candidates^

- rc9:

  - Fix
    {issue}`118`:
    report error if ghostscript iccprofiles are missing
  - fixed another issue related to
    {issue}`111`: PDF
    rasterized to palette file
  - add support image files with a palette
  - don't try to validate PDF file after an exception occurs

- rc8:

  - Fix
    {issue}`111`:
    exception thrown if PDF is missing DocumentInfo dictionary

- rc7:

  - fix error when installing direct from pip, "no such file
    'requirements.txt'"

- rc6:

  - dropped libxml2 (Python lxml) since Python 3's internal XML parser
    is sufficient
  - set up Docker container
  - fix Unicode errors if recognized text contains Unicode characters
    and system locale is not UTF-8

- rc5:

  - dropped Java and JHOVE in favour of qpdf
  - improved command line error output
  - additional tests and bug fixes
  - tested on Ubuntu 14.04 LTS

- rc4:

  - dropped MuPDF in favour of qpdf
  - fixed some installer issues and errors in installation
    instructions
  - improve performance: run Ghostscript with multithreaded rendering
  - improve performance: use multiple cores by default
  - bug fix: checking for wrong exception on process timeout

- rc3: skipping version number intentionally to avoid confusion with
  Tesseract

- rc2: first release for public testing to test-PyPI, Github

- rc1: testing release process

## Compatibility notes

- `./OCRmyPDF.sh` script is still available for now
- Stacking the verbosity option like `-vvv` is no longer supported
- The configuration file `config.sh` has been removed. Instead, you
  can feed a file to the arguments for common settings:

```
ocrmypdf input.pdf output.pdf @settings.txt
```

where `settings.txt` contains *one argument per line*, for example:

```
-l
deu
--author
A. Merkel
--pdf-renderer
tesseract
```

Fixes

- Handling of filenames containing spaces: fixed

Notes and known issues

- Some dependencies may work with lower versions than tested, so try
  overriding dependencies if they are "in the way" to see if they work.
- `--pdf-renderer tesseract` will output files with an incorrect page
  size in Tesseract 3.03, due to a bug in Tesseract.
- PDF files containing "inline images" are not supported and won't be
  for the 3.0 release. Scanned images almost never contain inline
  images.

---

### Releasenotes Version04 (`docs/releasenotes/version04.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v4

## v4.5.6

- Fixed {issue}`156`,
  'NoneType' object has no attribute 'getObject' on pages with no
  optional /Contents record. This should resolve all issues related to
  pages with no /Contents record.
- Fixed {issue}`158`, ocrmypdf
  now stops and terminates if Ghostscript fails on an intermediate
  step, as it is not possible to proceed.
- Fixed {issue}`160`,
  exception thrown on certain invalid arguments instead of error
  message

## v4.5.5

- Automated update of macOS homebrew tap
- Fixed {issue}`154`, KeyError
  '/Contents' when searching for text on blank pages that have no
  /Contents record. Note: incomplete fix for this issue.

## v4.5.4

- Fixed `--skip-big` raising an exception if a page contains no images
  ({issue}`152`) (thanks
  to @TomRaz)
- Fixed an issue where pages with no images might trigger "cannot write
  mode P as JPEG"
  ({issue}`151`)

## v4.5.3

- Added a workaround for Ghostscript 9.21 and probably earlier versions
  would fail with the error message "VMerror -25", due to a Ghostscript
  bug in XMP metadata handling
- High Unicode characters (U+10000 and up) are no longer accepted for
  setting metadata on the command line, as Ghostscript may not handle
  them correctly.
- Fixed an issue where the `tess4` renderer would duplicate content
  onto output pages if tesseract failed or timed out
- Fixed `tess4` renderer not recognized when lossless reconstruction
  is possible

## v4.5.2

- Fixed {issue}`147`,
  `--pdf-renderer tess4 --clean` will produce an oversized page
  containing the original image in the bottom left corner, due to loss
  DPI information.
- Make "using Tesseract 4.0" warning less ominous
- Set up machinery for homebrew OCRmyPDF tap

## v4.5.1

- Fixed {issue}`137`,
  proportions of images with a non-square pixel aspect ratio would be
  distorted in output for `--force-ocr` and some other combinations
  of flags

## v4.5

- PDFs containing "Form XObjects" are now supported (issue
  {issue}`134`; PDF
  reference manual 8.10), and images they contain are taken into
  account when determining the resolution for rasterizing
- The Tesseract 4 Docker image no longer includes all languages,
  because it took so long to build something would tend to fail
- OCRmyPDF now warns about using `--pdf-renderer tesseract` with
  Tesseract 3.04 or lower due to issues with Ghostscript corrupting the
  OCR text in these cases

## v4.4.2

- The Docker images (ocrmypdf, ocrmypdf-polyglot, ocrmypdf-tess4) are
  now based on Ubuntu 16.10 instead of Debian stretch

  - This makes supporting the Tesseract 4 image easier
  - This could be a disruptive change for any Docker users who built
    customized these images with their own changes, and made those
    changes in a way that depends on Debian and not Ubuntu

- OCRmyPDF now prevents running the Tesseract 4 renderer with Tesseract
  3.04, which was permitted in v4.4 and v4.4.1 but will not work

## v4.4.1

- To prevent a [TIFF output
  error](https://github.com/python-pillow/Pillow/issues/2206) caused
  by img2pdf >= 0.2.1 and Pillow \<= 3.4.2, dependencies have been
  tightened
- The Tesseract 4.00 simultaneous process limit was increased from 1 to
  2, since it was observed that 1 lowers performance
- Documentation improvements to describe the `--tesseract-config`
  feature
- Added test cases and fixed error handling for `--tesseract-config`
- Tweaks to setup.py to deal with issues in the v4.4 release

## v4.4

- Tesseract 4.00 is now supported on an experimental basis.

  - A new rendering option `--pdf-renderer tess4` exploits Tesseract
    4's new text-only output PDF mode. See the documentation on PDF
    Renderers for details.
  - The `--tesseract-oem` argument allows control over the Tesseract
    4 OCR engine mode (tesseract's `--oem`). Use
    `--tesseract-oem 2` to enforce the new LSTM mode.
  - Fixed poor performance with Tesseract 4.00 on Linux

- Fixed an issue that caused corruption of output to stdout in some
  cases

- Removed test for Pillow JPEG and PNG support, as the minimum
  supported version of Pillow now enforces this

- OCRmyPDF now tests that the intended destination file is writable
  before proceeding

- The test suite now requires `pytest-helpers-namespace` to run (but
  not install)

- Significant code reorganization to make OCRmyPDF re-entrant and
  improve performance. All changes should be backward compatible for
  the v4.x series.

  - However, OCRmyPDF's dependency "ruffus" is not re-entrant, so no
    Python API is available. Scripts should continue to use the
    command line interface.

## v4.3.5

- Update documentation to confirm Python 3.6.0 compatibility. No code
  changes were needed, so many earlier versions are likely supported.

## v4.3.4

- Fixed "decimal.InvalidOperation: quantize result has too many digits"
  for high DPI images

## v4.3.3

- Fixed PDF/A creation with Ghostscript 9.20 properly
- Fixed an exception on inline stencil masks with a missing optional
  parameter

## v4.3.2

- Fixed a PDF/A creation issue with Ghostscript 9.20 (note: this fix
  did not actually work)

## v4.3.1

- Fixed an issue where pages produced by the "hocr" renderer after a
  Tesseract timeout would be rotated incorrectly if the input page was
  rotated with a /Rotate marker
- Fixed a file handle leak in LeptonicaErrorTrap that would cause a
  "too many open files" error for files around hundred pages of pages
  long when `--deskew` or `--remove-background` or other Leptonica
  based image processing features were in use, depending on the system
  value of `ulimit -n`
- Ability to specify multiple languages for multilingual documents is
  now advertised in documentation
- Reduced the file sizes of some test resources
- Cleaned up debug output
- Tesseract caching in test cases is now more cautious about false
  cache hits and reproducing exact output, not that any problems were
  observed

## v4.3

- New feature `--remove-background` to detect and erase the
  background of color and grayscale images

- Better documentation

- Fixed an issue with PDFs that draw images when the raster stack depth
  is zero

- ocrmypdf can now redirect its output to stdout for use in a shell
  pipeline

  - This does not improve performance since temporary files are still
    used for buffering
  - Some output validation is disabled in this mode

## v4.2.5

- Fixed an issue
  ({issue}`100`) with
  PDFs that omit the optional /BitsPerComponent parameter on images
- Removed non-free file milk.pdf

## v4.2.4

- Fixed an error
  ({issue}`90`) caused by
  PDFs that use stencil masks properly
- Fixed handling of PDFs that try to draw images or stencil masks
  without properly setting up the graphics state (such images are now
  ignored for the purposes of calculating DPI)

## v4.2.3

- Fixed an issue with PDFs that store page rotation (/Rotate) in an
  indirect object

- Integrated a few fixes to simplify downstream packaging (Debian)

  - The test suite no longer assumes it is installed
  - If running Linux, skip a test that passes Unicode on the command
    line

- Added a test case to check explicit masks and stencil masks

- Added a test case for indirect objects and linearized PDFs

- Deprecated the OCRmyPDF.sh shell script

## v4.2.2

- Improvements to documentation

## v4.2.1

- Fixed an issue where PDF pages that contained stencil masks would
  report an incorrect DPI and cause Ghostscript to abort
- Implemented stdin streaming

## v4.2

- ocrmypdf will now try to convert single image files to PDFs if they
  are provided as input
  ({issue}`15`)

  - This is a basic convenience feature. It only supports a single
    image and always makes the image fill the whole page.
  - For better control over image to PDF conversion, use `img2pdf`
    (one of ocrmypdf's dependencies)

- New argument `--output-type {pdf|pdfa}` allows disabling
  Ghostscript PDF/A generation

  - `pdfa` is the default, consistent with past behavior
  - `pdf` provides a workaround for users concerned about the
    increase in file size from Ghostscript forcing JBIG2 images to
    CCITT and transcoding JPEGs
  - `pdf` preserves as much as it can about the original file,
    including problems that PDF/A conversion fixes

- PDFs containing images with "non-square" pixel aspect ratios, such as
  200x100 DPI, are now handled and converted properly (fixing a bug
  that caused to be cropped)

- `--force-ocr` rasterizes pages even if they contain no images

  - supports users who want to use OCRmyPDF to reconstruct text
    information in PDFs with damaged Unicode maps (copy and paste text
    does not match displayed text)
  - supports reinterpreting PDFs where text was rendered as curves for
    printing, and text needs to be recovered
  - fixes issue
    {issue}`82`

- Fixes an issue where, with certain settings, monochrome images in
  PDFs would be converted to 8-bit grayscale, increasing file size
  ({issue}`79`)

- Support for Ubuntu 12.04 LTS "precise" has been dropped in favor of
  (roughly) Ubuntu 14.04 LTS "trusty"

  - Some Ubuntu "PPAs" (backports) are needed to make it work

- Support for some older dependencies dropped

  - Ghostscript 9.15 or later is now required (available in Ubuntu
    trusty with backports)
  - Tesseract 3.03 or later is now required (available in Ubuntu
    trusty)

- Ghostscript now runs in "safer" mode where possible

## v4.1.4

- Bug fix: monochrome images with an ICC profile attached were
  incorrectly converted to full color images if lossless reconstruction
  was not possible due to other settings; consequence was increased
  file size for these images

## v4.1.3

- More helpful error message for PDFs with version 4 security handler
- Update usage instructions for Windows/Docker users
- Fixed order of operations for matrix multiplication (no effect on most
  users)
- Add a few leptonica wrapper functions (no effect on most users)

## v4.1.2

- Replace IEC sRGB ICC profile with Debian's sRGB (from
  icc-profiles-free) which is more compatible with the MIT license
- More helpful error message for an error related to certain types of
  malformed PDFs

## v4.1

- `--rotate-pages` now only rotates pages when reasonably confidence
  in the orientation. This behavior can be adjusted with the new
  argument `--rotate-pages-threshold`
- Fixed problems in error checking if `unpaper` is uninstalled or
  missing at run-time
- Fixed problems with "RethrownJobError" errors during error handling
  that suppressed the useful error messages

## v4.0.7

- Minor correction to Ghostscript output settings

## v4.0.6

- Update install instructions
- Provide a sRGB profile instead of using Ghostscript's

## v4.0.5

- Remove some verbose debug messages from v4.0.4
- Fixed temporary that wasn't being deleted
- DPI is now calculated correctly for cropped images, along with other
  image transformations
- Inline images are now checked during DPI calculation instead of
  rejecting the image

## v4.0.4

Released with verbose debug message turned on. Do not use. Skip to
v4.0.5.

## v4.0.3

New features

- Page orientations detected are now reported in a summary comment

Fixes

- Show stack trace if unexpected errors occur
- Treat "too few characters" error message from Tesseract as a reason
  to skip that page rather than abort the file
- Docker: fix blank JPEG2000 issue by insisting on Ghostscript versions
  that have this fixed

## v4.0.2

Fixes

- Fixed compatibility with Tesseract 3.04.01 release, particularly its
  different way of outputting orientation information
- Improved handling of Tesseract errors and crashes
- Fixed use of chmod on Docker that broke most test cases

## v4.0.1

Fixes

- Fixed a KeyError if tesseract fails to find page orientation
  information

## v4.0

New features

- Automatic page rotation (`-r`) is now available. It uses ignores
  any prior rotation information on PDFs and sets rotation based on the
  dominant orientation of detectable text. This feature is fairly
  reliable but some false positives occur especially if there is not
  much text to work with.
  ({issue}`4`)
- Deskewing is now performed using Leptonica instead of unpaper.
  Leptonica is faster and more reliable at image deskewing than
  unpaper.

Fixes

- Fixed an issue where lossless reconstruction could cause some pages
  to be appear incorrectly if the page was rotated by the user in
  Acrobat after being scanned (specifically if it a /Rotate tag)
- Fixed an issue where lossless reconstruction could misalign the
  graphics layer with respect to text layer if the page had been
  cropped such that its origin is not (0, 0)
  ({issue}`49`)

Changes

- Logging output is now much easier to read
- `--deskew` is now performed by Leptonica instead of unpaper
  ({issue}`25`)
- libffi is now required
- Some changes were made to the Docker and Travis build environments to
  support libffi
- `--pdf-renderer=tesseract` now displays a warning if the Tesseract
  version is less than 3.04.01, the planned release that will include
  fixes to an important OCR text rendering bug in Tesseract 3.04.00.
  You can also manually install ./share/sharp2.ttf on top of pdf.ttf in
  your Tesseract tessdata folder to correct the problem.

---

### Releasenotes Version05 (`docs/releasenotes/version05.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v5

## v5.7.0

- Fixed an issue that caused poor CPU utilization on machines with more
  than 4 cores when running Tesseract 4. (Related to {issue}`217`.)

- The 'hocr' renderer has been improved. The 'sandwich' and 'tesseract'
  renderers are still better for most use cases, but 'hocr' may be
  useful for people who work with the PDF.js renderer in English/ASCII
  languages. ({issue}`225`)

  - It now formats text in a matter that is easier for certain PDF
    viewers to select and extract copy and paste text. This should
    help macOS Preview and PDF.js in particular.
  - The appearance of selected text and behavior of selecting text is
    improved.
  - The PDF content stream now uses relative moves, making it more
    compact and easier for viewers to determine when two words on the
    same line.
  - It can now deal with text on a skewed baseline.
  - Thanks to @cforcey for the pull request, @jbreiden for many
    helpful suggestions, @ctbarbour for another round of improvements,
    and @acaloiaro for an independent review.

## v5.6.3

- Suppress two debug messages that were too verbose

## v5.6.2

- Development branch accidentally tagged as release. Do not use.

## v5.6.1

- Fixed {issue}`219`: change
  how the final output file is created to avoid triggering permission
  errors when the output is a special file such as `/dev/null`
- Fixed test suite failures due to a qpdf 8.0.0 regression and Python
  3.5's handling of symlink
- The "encrypted PDF" error message was different depending on the type
  of PDF encryption. Now a single clear message appears for all types
  of PDF encryption.
- ocrmypdf is now in Homebrew. Homebrew users are advised to the
  version of ocrmypdf in the official homebrew-core formulas rather
  than the private tap.
- Some linting

## v5.6.0

- Fixed {issue}`216`: preserve
  "text as curves" PDFs without rasterizing file
- Related to the above, messages about rasterizing are more consistent
- For consistency versions minor releases will now get the trailing .0
  they always should have had.

## v5.5

- Add new argument `--max-image-mpixels`. Pillow 5.0 now raises an
  exception when images may be decompression bombs. This argument can
  be used to override the limit Pillow sets.
- Fixed output page cropped when using the sandwich renderer and OCR is
  skipped on a rotated and image-processed page
- A warning is now issued when old versions of Ghostscript are used in
  cases known to cause issues with non-Latin characters
- Fixed a few parameter validation checks for `-output-type pdfa-1` and
  `pdfa-2`

## v5.4.4

- Fixed {issue}`181`: fix
  final merge failure for PDFs with more pages than the system file
  handle limit (`ulimit -n`)
- Fixed {issue}`200`: an
  uncommon syntax for formatting decimal numbers in a PDF would cause
  qpdf to issue a warning, which ocrmypdf treated as an error. Now this
  the warning is relayed.
- Fixed an issue where intermediate PDFs would be created at version 1.3
  instead of the version of the original file. It's possible but
  unlikely this had side effects.
- A warning is now issued when older versions of qpdf are used since
  issues like
  {issue}`200` cause
  qpdf to infinite-loop
- Address issue
  {issue}`140`: if
  Tesseract outputs invalid UTF-8, escape it and print its message
  instead of aborting with a Unicode error
- Adding previously unlisted setup requirement, pytest-runner
- Update documentation: fix an error in the example script for Synology
  with Docker images, improved security guidance, advised
  `pip install --user`

## v5.4.3

- If a subprocess fails to report its version when queried, exit
  cleanly with an error instead of throwing an exception
- Added test to confirm that the system locale is Unicode-aware and
  fail early if it's not
- Clarified some copyright information
- Updated pinned requirements.txt so the homebrew formula captures more
  recent versions

## v5.4.2

- Fixed a regression from v5.4.1 that caused sidecar files to be
  created as empty files

## v5.4.1

- Add workaround for Tesseract v4.00alpha crash when trying to obtain
  orientation and the latest language packs are installed

## v5.4

- Change wording of a deprecation warning to improve clarity
- Added option to generate PDF/A-1b output if desired
  (`--output-type pdfa-1`); default remains PDF/A-2b generation
- Update documentation

## v5.3.3

- Fixed missing error message that should occur when trying to force
  `--pdf-renderer sandwich` on old versions of Tesseract
- Update copyright information in test files
- Set system `LANG` to UTF-8 in Dockerfiles to avoid UTF-8 encoding
  errors

## v5.3.2

- Fixed a broken test case related to language packs

## v5.3.1

- Fixed wrong return code given for missing Tesseract language packs
- Fixed "brew audit" crashing on Travis when trying to auto-brew

## v5.3

- Added `--user-words` and `--user-patterns` arguments which are
  forwarded to Tesseract OCR as words and regular expressions
  respective to use to guide OCR. Supplying a list of subject-domain
  words should assist Tesseract with resolving words.
  ({issue}`165`)
- Using a non Latin-1 language with the "hocr" renderer now warns about
  possible OCR quality and recommends workarounds
  ({issue}`176`)
- Output file path added to error message when that location is not
  writable
  ({issue}`175`)
- Otherwise valid PDFs with leading whitespace at the beginning of the
  file are now accepted

## v5.2

- When using Tesseract 3.05.01 or newer, OCRmyPDF will select the
  "sandwich" PDF renderer by default, unless another PDF renderer is
  specified with the `--pdf-renderer` argument. The previous behavior
  was to select `--pdf-renderer=hocr`.
- The "tesseract" PDF renderer is now deprecated, since it can cause
  problems with Ghostscript on Tesseract 3.05.00
- The "tess4" PDF renderer has been renamed to "sandwich". "tess4" is
  now a deprecated alias for "sandwich".

## v5.1

- Files with pages larger than 200" (5080 mm) in either dimension are
  now supported with `--output-type=pdf` with the page size preserved
  (in the PDF specification this feature is called UserUnit scaling).
  Due to Ghostscript limitations this is not available in conjunction
  with PDF/A output.

## v5.0.1

- Fixed {issue}`169`,
  exception due to failure to create sidecar text files on some
  versions of Tesseract 3.04, including the jbarlow83/ocrmypdf Docker
  image

## v5.0

- Backward incompatible changes

  > - Support for Python 3.4 dropped. Python 3.5 is now required.
  > - Support for Tesseract 3.02 and 3.03 dropped. Tesseract 3.04 or
  >   newer is required. Tesseract 4.00 (alpha) is supported.
  > - The OCRmyPDF.sh script was removed.

- Add a new feature, `--sidecar`, which allows creating "sidecar"
  text files which contain the OCR results in plain text. These OCR
  text is more reliable than extracting text from PDFs. Closes
  {issue}`126`.

- New feature: `--pdfa-image-compression`, which allows overriding
  Ghostscript's lossy-or-lossless image encoding heuristic and making
  all images JPEG encoded or lossless encoded as desired. Fixes
  {issue}`163`.

- Fixed {issue}`143`, added
  `--quiet` to suppress "INFO" messages

- Fixed {issue}`164`, a typo

- Removed the command line parameters `-n` and `--just-print` since
  they have not worked for some time (reported as Ubuntu bug
  [#1687308](https://bugs.launchpad.net/ubuntu/+source/ocrmypdf/+bug/1687308))

---

### Releasenotes Version06 (`docs/releasenotes/version06.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v6

## v6.2.5

- Disable a failing test due to Tesseract 4.0rc1 behavior change.
  Previously, Tesseract would exit with an error message if its
  configuration was invalid, and OCRmyPDF would intercept this message.
  Now Tesseract issues a warning, which OCRmyPDF v6.2.5 may relay or
  ignore. (In v7.x, OCRmyPDF will respond to the warning.)
- This release branch no longer supports using the optional PyMuPDF
  installation, since it was removed in v7.x.
- This release branch no longer supports macOS. macOS users should
  upgrade to v7.x.

## v6.2.4

- Backport Ghostscript 9.25 compatibility fixes, which removes support
  for setting Unicode metadata
- Backport blacklisting Ghostscript 9.24
- Older versions of Ghostscript are still supported

## v6.2.3

- Fixed compatibility with img2pdf >= 0.3.0 by rejecting input images
  that have an alpha channel
- This version will be included in Ubuntu 18.10

## v6.2.2

- Backport compatibility fixes for Python 3.7 and ruffus 2.7.0 from
  v7.0.0
- Backport fix to ignore masks when deciding what colors are on a page
- Backport some minor improvements from v7.0.0: better argument
  validation and warnings about the Tesseract 4.0.0 `--user-words`
  regression

## v6.2.1

- Fixed recent versions of Tesseract (after 4.0.0-beta1) not being
  detected as supporting the `sandwich` renderer ({issue}`271`).

## v6.2.0

- **Docker**: The Docker image `ocrmypdf-tess4` has been removed. The
  main Docker images, `ocrmypdf` and `ocrmypdf-polyglot` now use
  Ubuntu 18.04 as a base image, and as such Tesseract 4.0.0-beta1 is
  now the Tesseract version they use. There is no Docker image based on
  Tesseract 3.05 anymore.
- Creation of PDF/A-3 is now supported. However, there is no ability to
  attach files to PDF/A-3.
- Lists more reasons why the file size might grow.
- Fixed {issue}`262`,
  `--remove-background` error on PDFs contained colormapped
  (paletted) images.
- Fixed another XMP metadata validation issue, in cases where the input
  file's creation date has no timezone and the creation date is not
  overridden.

## v6.1.5

- Fixed {issue}`253`, a
  possible division by zero when using the `hocr` renderer.
- Fixed incorrectly formatted `<xmp:ModifyDate>` field inside XMP
  metadata for PDF/As. veraPDF flags this as a PDF/A validation
  failure. The error is caused the timezone and final digit of the
  seconds of modified time to be omitted, so at worst the modification
  time stamp is rounded to the nearest 10 seconds.

## v6.1.4

- Fixed {issue}`248`
  `--clean` argument may remove OCR from left column of text on
  certain documents. We now set `--layout none` to suppress this.
- The test cache was updated to reflect the change above.
- Change test suite to accommodate Ghostscript 9.23's new ability to
  insert JPEGs into PDFs without transcoding.
- XMP metadata in PDFs is now examined using `defusedxml` for safety.
- If an external process exits with a signal when asked to report its
  version, we now print the system error message instead of suppressing
  it. This occurred when the required executable was found but was
  missing a shared library.
- qpdf 7.0.0 or newer is now required as the test suite can no longer
  pass without it.

### Notes

- An apparent [regression in Ghostscript
  9.23](https://bugs.ghostscript.com/show_bug.cgi?id=699216) will
  cause some ocrmypdf output files to become invalid in rare cases; the
  workaround for the moment is to set `--force-ocr`.

## v6.1.3

- Fixed {issue}`247`,
  `/CreationDate` metadata not copied from input to output.
- A warning is now issued when Python 3.5 is used on files with a large
  page count, as this case is known to regress to single core
  performance. The cause of this problem is unknown.

## v6.1.2

- Upgrade to PyMuPDF v1.12.5 which includes a more complete fix to
  {issue}`239`.
- Add `defusedxml` dependency.

## v6.1.1

- Fixed text being reported as found on all pages if PyMuPDF is not
  installed.

## v6.1.0

- PyMuPDF is now an optional but recommended dependency, to alleviate
  installation difficulties on platforms that have less access to
  PyMuPDF than the author anticipated. (For version 6.x only) install
  OCRmyPDF with `pip install ocrmypdf[fitz]` to use it to its full
  potential.
- Fixed `FileExistsError` that could occur if OCR timed out while it
  was generating the output file.
  ({issue}`218`)
- Fixed table of contents/bookmarks all being redirected to page 1 when
  generating a PDF/A (with PyMuPDF). (Without PyMuPDF the table of
  contents is removed in PDF/A mode.)
- Fixed "RuntimeError: invalid key in dict" when table of
  contents/bookmarks titles contained the character `)`.
  ({issue}`239`)
- Added a new argument `--skip-repair` to skip the initial PDF repair
  step if the PDF is already well-formed (because another program
  repaired it).

## v6.0.0

- The software license has been changed to GPLv3 [it has since changed again].
  Test resource files and some individual sources may have other licenses.

- OCRmyPDF now depends on
  [PyMuPDF](https://pymupdf.readthedocs.io/en/latest/installation/).
  Including PyMuPDF is the primary reason for the change to GPLv3.

- Other backward incompatible changes

  - The `OCRMYPDF_TESSERACT`, `OCRMYPDF_QPDF`, `OCRMYPDF_GS` and
    `OCRMYPDF_UNPAPER` environment variables are no longer used.
    Change `PATH` if you need to override the external programs
    OCRmyPDF uses.
  - The `ocrmypdf` package has been moved to `src/ocrmypdf` to
    avoid issues with accidental import.
  - The function `ocrmypdf.exec.get_program` was removed.
  - The deprecated module `ocrmypdf.pageinfo` was removed.
  - The `--pdf-renderer tess4` alias for `sandwich` was removed.

- Fixed an issue where OCRmyPDF failed to detect existing text on
  pages, depending on how the text and fonts were encoded within the
  PDF. ({issue}`233,232`)

- Fixed an issue that caused dramatic inflation of file sizes when
  `--skip-text --output-type pdf` was used. OCRmyPDF now removes
  duplicate resources such as fonts, images and other objects that it
  generates. ({issue}`237`)

- Improved performance of the initial page splitting step. Originally
  this step was not believed to be expensive and ran in a process.
  Large file testing revealed it to be a bottleneck, so it is now
  parallelized. On a 700 page file with quad core machine, this change
  saves about 2 minutes. ({issue}`234`)

- The test suite now includes a cache that can be used to speed up test
  runs across platforms. This also does not require computing
  checksums, so it's faster. ({issue}`217`)

---

### Releasenotes Version07 (`docs/releasenotes/version07.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v7

## v7.4.0

- `--force-ocr` may now be used with the new `--threshold` and
  `--mask-barcodes` features
- pikepdf >= 0.9.1 is now required.
- Changed metadata handling to pikepdf 0.9.1. As a result, metadata
  handling of non-ASCII characters in Ghostscript 9.25 or later is
  fixed.
- chardet >= 3.0.4 is temporarily listed as required. pdfminer.six
  depends on it, but the most recent release does not specify this
  requirement.
  ({issue}`326`)
- python-xmp-toolkit and libexempi are no longer required.
- A new Docker image is now being provided for users who wish to access
  OCRmyPDF over a simple HTTP interface, instead of the command line.
- Increase tolerance of PDFs that overflow or underflow the PDF
  graphics stack.
  ({issue}`325`)

## v7.3.1

- Fixed performance regression from v7.3.0; fast page analysis was not
  selected when it should be.
- Fixed a few exceptions related to the new `--mask-barcodes` feature
  and improved argument checking
- Added missing detection of TrueType fonts that lack a Unicode mapping

## v7.3.0

- Added a new feature `--redo-ocr` to detect existing OCR in a file,
  remove it, and redo the OCR. This may be particularly helpful for
  anyone who wants to take advantage of OCR quality improvements in
  Tesseract 4.0. Note that OCR added by OCRmyPDF before version 3.0
  cannot be detected since it was not properly marked as invisible text
  in the earliest versions. OCR that constructs a font from visible
  text, such as Adobe Acrobat's ClearScan.

- OCRmyPDF's content detection is generally more sophisticated. It
  learns more about the contents of each PDF and makes better
  recommendations:

  - OCRmyPDF can now detect when a PDF contains text that cannot be
    mapped to Unicode (meaning it is readable to human eyes but
    copy-pastes as gibberish). In these cases it recommends
    `--force-ocr` to make the text searchable.
  - PDFs containing vector objects are now rendered at more
    appropriate resolution for OCR.
  - We now exit with an error for PDFs that contain Adobe LiveCycle
    Designer's dynamic XFA forms. Currently the open source community
    does not have tools to work with these files.
  - OCRmyPDF now warns when a PDF that contains Adobe AcroForms, since
    such files probably do not need OCR. It can work with these files.

- Added three new **experimental** features to improve OCR quality in
  certain conditions. The name, syntax and behavior of these arguments
  is subject to change. They may also be incompatible with some other
  features.

  - `--remove-vectors` which strips out vector graphics. This can
    improve OCR quality since OCR will not search artwork for readable
    text; however, it currently removes "text as curves" as well.
  - `--mask-barcodes` to detect and suppress barcodes in files. We
    have observed that barcodes can interfere with OCR because they
    are "text-like" but not actually textual.
  - `--threshold` which uses a more sophisticated thresholding
    algorithm than is currently in use in Tesseract OCR. This works
    around a [known issue in Tesseract
    4.0](https://github.com/tesseract-ocr/tesseract/issues/1990)
    with dark text on bright backgrounds.

- Fixed an issue where an error message was not reported when the
  installed Ghostscript was very old.

- The PDF optimizer now saves files with object streams enabled when
  the optimization level is `--optimize 1` or higher (the default).
  This makes files a little bit smaller, but requires PDF 1.5. PDF 1.5
  was first released in 2003 and is broadly supported by PDF viewers,
  but some rudimentary PDF parsers such as PyPDF2 do not understand
  object streams. You can use the command line tool
  `qpdf --object-streams=disable` or
  [pikepdf](https://github.com/pikepdf/pikepdf) library to remove
  them.

- New dependency: pdfminer.six 20181108. Note this is a fork of the
  Python 2-only pdfminer.

- Deprecation notice: At the end of 2018, we will be ending support for
  Python 3.5 and Tesseract 3.x. OCRmyPDF v7 will continue to work with
  older versions.

## v7.2.1

- Fixed compatibility with an API change in pikepdf 0.3.5.
- A kludge to support Leptonica versions older than 1.72 in the test
  suite was dropped. Older versions of Leptonica are likely still
  compatible. The only impact is that a portion of the test suite will
  be skipped.

## v7.2.0

**Lossy JBIG2 behavior change**

A user reported that ocrmypdf was in fact using JBIG2 in **lossy**
compression mode. This was not the intended behavior. Users should
[review the technical concerns with JBIG2 in lossy
mode](https://abbyy.technology/en:kb:tip:jbig2_compression_and_ocr)
and decide if this is a concern for their use case.

JBIG2 lossy mode does achieve higher compression ratios than any other
monochrome compression technology; for large text documents the savings
are considerable. JBIG2 lossless still gives great compression ratios
and is a major improvement over the older CCITT G4 standard.

Only users who have reviewed the concerns with JBIG2 in lossy mode
should opt-in. As such, lossy mode JBIG2 is only turned on when the new
argument `--jbig2-lossy` is issued. This is independent of the setting
for `--optimize`.

Users who did not install an optional JBIG2 encoder are unaffected.

(Thanks to user 'bsdice' for reporting this issue.)

**Other issues**

- When the image optimizer quantizes an image to 1 bit per pixel, it
  will now attempt to further optimize that image as CCITT or JBIG2,
  instead of keeping it in the "flate" encoding which is not efficient
  for 1 bpp images.
  ({issue}`297`)
- Images in PDFs that are used as soft masks (i.e. transparency masks
  or alpha channels) are now excluded from optimization.
- Fixed handling of Tesseract 4.0-rc1 which now accepts invalid
  Tesseract configuration files, which broke the test suite.

## v7.1.0

- Improve the performance of initial text extraction, which is done to
  determine if a file contains existing text of some kind or not. On
  large files, this initial processing is now about 20x times faster.
  ({issue}`299`)
- pikepdf 0.3.3 is now required.
- Fixed {issue}`231`, a
  problem with JPEG2000 images where image metadata was only available
  inside the JPEG2000 file.
- Fixed some additional Ghostscript 9.25 compatibility issues.
- Improved handling of KeyboardInterrupt error messages.
  ({issue}`301`)
- README.md is now served in GitHub markdown instead of
  reStructuredText.

## v7.0.6

- Blacklist Ghostscript 9.24, now that 9.25 is available and fixes many
  regressions in 9.24.

## v7.0.5

- Improve capability with Ghostscript 9.24, and enable the JPEG
  passthrough feature when this version in installed.
- Ghostscript 9.24 lost the ability to set PDF title, author, subject
  and keyword metadata to Unicode strings. OCRmyPDF will set ASCII
  strings and warn when Unicode is suppressed. Other software may be
  used to update metadata. This is a short term work around.
- PDFs generated by Kodak Capture Desktop, or generally PDFs that
  contain indirect references to null objects in their table of
  contents, would have an invalid table of contents after processing by
  OCRmyPDF that might interfere with other viewers. This has been
  fixed.
- Detect PDFs generated by Adobe LiveCycle, which can only be displayed
  in Adobe Acrobat and Reader currently. When these are encountered,
  exit with an error instead of performing OCR on the "Please wait"
  error message page.

## v7.0.4

- Fixed exception thrown when trying to optimize a certain type of PNG
  embedded in a PDF with the `-O2`
- Update to pikepdf 0.3.2, to gain support for optimizing some
  additional image types that were previously excluded from
  optimization (CMYK and grayscale). Fixes
  {issue}`285`.

## v7.0.3

- Fixed {issue}`284`, an error
  when parsing inline images that have are also image masks, by
  upgrading pikepdf to 0.3.1

## v7.0.2

- Fixed a regression with `--rotate-pages` on pages that already had
  rotations applied.
  ({issue}`279`)
- Improve quality of page rotation in some cases by rasterizing a
  higher quality preview image.
  ({issue}`281`)

## v7.0.1

- Fixed compatibility with img2pdf >= 0.3.0 by rejecting input images
  that have an alpha channel
- Add forward compatibility for pikepdf 0.3.0 (unrelated to img2pdf)
- Various documentation updates for v7.0.0 changes

## v7.0.0

- The core algorithm for combining OCR layers with existing PDF pages
  has been rewritten and improved considerably. PDFs are no longer
  split into single page PDFs for processing; instead, images are
  rendered and the OCR results are grafted onto the input PDF. The new
  algorithm uses less temporary disk space and is much more performant
  especially for large files.

- New dependency: [pikepdf](https://github.com/pikepdf/pikepdf).
  pikepdf is a powerful new Python PDF library driving the latest
  OCRmyPDF features, built on the QPDF C++ library (libqpdf).

- New feature: PDF optimization with `-O` or `--optimize`. After
  OCR, OCRmyPDF will perform image optimizations relevant to OCR PDFs.

  - If a JBIG2 encoder is available, then monochrome images will be
    converted, with the potential for huge savings on large black and
    white images, since JBIG2 is far more efficient than any other
    monochrome (bi-level) compression. (All known US patents related
    to JBIG2 have probably expired, but it remains the responsibility
    of the user to supply a JBIG2 encoder such as
    [jbig2enc](https://github.com/agl/jbig2enc). OCRmyPDF does not
    implement JBIG2 encoding.)
  - If `pngquant` is installed, OCRmyPDF will optionally use it to
    perform lossy quantization and compression of PNG images.
  - The quality of JPEGs can also be lowered, on the assumption that a
    lower quality image may be suitable for storage after OCR.
  - This image optimization component will eventually be offered as an
    independent command line utility.
  - Optimization ranges from `-O0` through `-O3`, where `0`
    disables optimization and `3` implements all options. `1`, the
    default, performs only safe and lossless optimizations. (This is
    similar to GCC's optimization parameter.) The exact type of
    optimizations performed will vary over time.

- Small amounts of text in the margins of a page, such as watermarks,
  page numbers, or digital stamps, will no longer prevent the rest of a
  page from being OCRed when `--skip-text` is issued. This behavior
  is based on a heuristic.

- Removed features

  - The deprecated `--pdf-renderer tesseract` PDF renderer was
    removed.
  - `-g`, the option to generate debug text pages, was removed
    because it was a maintenance burden and only worked in isolated
    cases. HOCR pages can still be previewed by running the
    hocrtransform.py with appropriate settings.

- Removed dependencies

  - `PyPDF2`
  - `defusedxml`
  - `PyMuPDF`

- The `sandwich` PDF renderer can be used with all supported versions
  of Tesseract, including that those prior to v3.05 which don't support
  `-c textonly`. (Tesseract v4.0.0 is recommended and more
  efficient.)

- `--pdf-renderer auto` option and the diagnostics used to select a
  PDF renderer now work better with old versions, but may make
  different decisions than past versions.

- If everything succeeds but PDF/A conversion fails, a distinct return
  code is now returned (`ExitCode.pdfa_conversion_failed (10)`) where
  this situation previously returned
  `ExitCode.invalid_output_pdf (4)`. The latter is now returned only
  if there is some indication that the output file is invalid.

- Notes for downstream packagers

  - There is also a new dependency on `python-xmp-toolkit` which in
    turn depends on `libexempi3`.
  - It may be necessary to separately `pip install pycparser` to
    avoid [another Python 3.7
    issue](https://github.com/eliben/pycparser/pull/135).

---

### Releasenotes Version08 (`docs/releasenotes/version08.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v8

## v8.3.2

- Dropped workaround for macOS that allowed it work without pdfminer.six,
  now a proper sdist release of pdfminer.six is available.
- pikepdf 1.5.0 is now required.

## v8.3.1

- Fixed an issue where PDFs with malformed metadata would be rendered as
  blank pages. {issue}`398`.

## v8.3.0

- Improved the strategy for updating pages when a new image of the page
  was produced. We now attempt to preserve more content from the
  original file, for annotations in particular.
- For PDFs with more than 100 pages and a sequence where one PDF page
  was replaced and one or more subsequent ones were skipped, an
  intermediate file would be corrupted while grafting OCR text, causing
  processing to fail. This is a regression, likely introduced in
  v8.2.4.
- Previously, we resized the images produced by Ghostscript by a small
  number of pixels to ensure the output image size was an exactly what
  we wanted. Having discovered a way to get Ghostscript to produce the
  exact image sizes we require, we eliminated the resizing step.
- Command line completions for `bash` are now available, in addition
  to `fish`, both in `misc/completion`. Package maintainers, please
  install these so users can take advantage.
- Updated requirements.
- pikepdf 1.3.0 is now required.

## v8.2.4

- Fixed a false positive while checking for a certain type of PDF that
  only Acrobat can read. We now more accurately detect Acrobat-only
  PDFs.
- OCRmyPDF holds fewer open file handles and is more prompt about
  releasing those it no longer needs.
- Minor optimization: we no longer traverse the table of contents to
  ensure all references in it are resolved, as changes to libqpdf have
  made this unnecessary.
- pikepdf 1.2.0 is now required.

## v8.2.3

- Fixed that `--mask-barcodes` would occasionally leave a unwanted
  temporary file named `junkpixt` in the current working folder.
- Fixed (hopefully) handling of Leptonica errors in an environment
  where a non-standard `sys.stderr` is present.
- Improved help text for `--verbose`.

## v8.2.2

- Fixed a regression from v8.2.0, an exception that occurred while
  attempting to report that `unpaper` or another optional dependency
  was unavailable.
- In some cases, `ocrmypdf [-c|--clean]` failed to exit with an error
  when `unpaper` is not installed.

## v8.2.1

- This release was canceled.

## v8.2.0

- A major improvement to our Docker image is now available thanks to
  hard work contributed by @mawi12345. The new Docker image,
  ocrmypdf-alpine, is based on Alpine Linux, and includes most of the
  functionality of three existed images in a smaller package. This
  image will replace the main Docker image eventually but for now all
  are being built. [See documentation for
  details](https://ocrmypdf.readthedocs.io/en/latest/docker.html).
- Documentation reorganized especially around the use of Docker images.
- Fixed a problem with PDF image optimization, where the optimizer
  would unnecessarily decompress and recompress PNG images, in some
  cases losing the benefits of the quantization it just had just
  performed. The optimizer is now capable of embedding PNG images into
  PDFs without transcoding them.
- Fixed a minor regression with lossy JBIG2 image optimization. All
  JBIG2 candidates images were incorrectly placed into a single
  optimization group for the whole file, instead of grouping pages
  together. This usually makes a larger JBIG2Globals dictionary and
  results in inferior compression, so it worked less well than
  designed. However, quality would not be impacted. Lossless JBIG2 was
  entirely unaffected.
- Updated dependencies, including pikepdf to 1.1.0. This fixes
  {issue}`358`.
- The install-time version checks for certain external programs have
  been removed from setup.py. These tests are now performed at
  run-time.
- The non-standard option to override install-time checks
  (`setup.py install --force`) is now deprecated and prints a
  warning. It will be removed in a future release.

## v8.1.0

- Added a feature, `--unpaper-args`, which allows passing arbitrary
  arguments to `unpaper` when using `--clean` or `--clean-final`.
  The default, very conservative unpaper settings are suppressed.
- The argument `--clean-final` now implies `--clean`. It was
  possible to issue `--clean-final` on its before this, but it would
  have no useful effect.
- Fixed an exception on traversing corrupt table of contents entries
  (specifically, those with invalid destination objects)
- Fixed an issue when using `--tesseract-timeout` and image
  processing features on a file with more than 100 pages.
  {issue}`347`
- OCRmyPDF now always calls `os.nice(5)` to signal to operating
  systems that it is a background process.

## v8.0.1

- Fixed an exception when parsing PDFs that are missing a required
  field. {issue}`325`
- pikepdf 1.0.5 is now required, to address some other PDF parsing
  issues.

## v8.0.0

No major features. The intent of this release is to sever support for
older versions of certain dependencies.

**Breaking changes**

- Dropped support for Tesseract 3.x. Tesseract 4.0 or newer is now
  required.
- Dropped support for Python 3.5.
- Some `ocrmypdf.pdfa` APIs that were deprecated in v7.x were
  removed. This functionality has been moved to pikepdf.

**Other changes**

- Fixed an unhandled exception when attempting to mask barcodes.
  {issue}`322`
- It is now possible to use ocrmypdf without pdfminer.six, to support
  distributions that do not have it or cannot currently use it (e.g.
  Homebrew). Downstream maintainers should include pdfminer.six if
  possible.
- A warning is now issue when PDF/A conversion removes some XMP
  metadata from the input PDF. (Only a "whitelist" of certain XMP
  metadata types are allowed in PDF/A.)
- Fixed several issues that caused PDF/As to be produced with
  nonconforming XMP metadata (would fail validation with veraPDF).
- Fixed some instances where invalid DocumentInfo from a PDF cause XMP
  metadata creation to fail.
- Fixed a few documentation problems.
- pikepdf 1.0.2 is now required.

---

### Releasenotes Version09 (`docs/releasenotes/version09.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v9

## v9.8.2

- Fixed an issue where OCRmyPDF would ignore text inside Form XObject when
  making certain decisions about whether a document already had text.
- Fixed file size increase warning to take overhead of small files into account.
- Added instructions for installing on Cygwin.

## v9.8.1

- Fixed an issue where unexpected files in the `%PROGRAMFILES%\gs` directory
  (Windows) caused an exception.
- Mark pdfminer.six 20200517 as supported.
- If jbig2enc is missing and optimization is requested, a warning is issued
  instead of an error, which was the intended behavior.
- Documentation updates.

## v9.8.0

- Fixed issue where only the first PNG (FlateDecode) image in a file would be
  considered for optimization. File sizes should be improved from here on.
- Fixed a startup crash when the chosen language was Japanese ({issue}`543`).
- Added options to configure polling and log level to watcher.py.

## v9.7.2

- Fixed an issue with `ocrmypdf.ocr(...language=)` not accepting a list of
  languages as documented.
- Updated setup.py to confirm that pdfminer.six version 20200402 is supported.

## v9.7.1

- Fixed version check failing when used with qpdf 10.0.0.
- Added some missing type annotations.
- Updated documentation to warn about need for "ifmain" guard and Windows.

## v9.7.0

- Fixed an error in watcher.py if `OCR_JSON_SETTINGS` was not defined.
- Ghostscript 9.51 is now blacklisted, due to numerous problems with this version.
- Added a workaround for a problem with "txtwrite" in Ghostscript 9.52.
- Fixed an issue where the incorrect number of threads used was shown when
  `OMP_THREAD_LIMIT` was manipulated.
- Removed a possible performance bottlenecks for files that use hundreds to
  thousands of images on the same page.
- Documentation improvements.
- Optimization will now be applied to some monochrome images that have a color
  profile defined instead of only black and white.
- ICC profiles are consulted when determining the simplified colorspace of an
  image.

## v9.6.1

- Documentation improvements - thanks to many users for their contributions!

  > - Fixed installation instructions for ArchLinux (@pigmonkey)
  > - Updated installation instructions for FreeBSD and other OSes (@knobix)
  > - Added instructions for using Docker Compose with watchdog (@ianalexander,
  >   @deisi)
  > - Other miscellany (@mb720, @toy, @caiofacchinato)
  > - Some scripts provided in the documentation have been migrated out so that
  >   they can be copied out as whole files, and to ensure syntax checking
  >   is maintained.

- Fixed an error that caused bash completions to fail on macOS. ({issue}`502,504`;
  @AlexanderWillner)

- Fixed a rare case where OCRmyPDF threw an exception while processing a PDF
  with the wrong object type in its `/Trailer /Info`. The error is now logged
  and incorrect object is ignored. ({issue}`497`)

- Removed potentially non-free file `enron1.pdf` and simplified the test that
  used it.

- Removed potentially non-free file `misc/media/logo.afdesign`.

## v9.6.0

- Fixed a regression with transferring metadata from the input PDF to the output
  PDF in certain situations.
- pdfminer.six is now supported up to version 2020-01-24.
- Messages are explaining page rotation decisions are now shown at the standard
  verbosity level again when `--rotate-pages`. In some previous version they
  were set to debug level messages that only appeared with the parameter `-v1`.
- Improvements to `misc/watcher.py`. Thanks to @ianalexander and @svenihoney.
- Documentation improvements.

## v9.5.0

- Added API functions to measure OCR quality.
- Modest improvements to handling PDFs with difficult/non compliant metadata.

## v9.4.0

- Updated recommended dependency versions.
- Improvements to test coverage and changes to facilitate better measurement of
  test coverage, such as when tests run in subprocesses.
- Improvements to error messages when Leptonica is not installed correctly.
- Fixed use of pytest "session scope" that may have caused some intermittent
  CI failures.
- When the argument `--keep-temporary-files` or verbosity is set to `-v1`,
  a debug log file is generated in the working temporary folder.

## v9.3.0

- Improved native Windows support: we now check in the obvious places in
  the "Program Files" folders installations of Tesseract and Ghostscript,
  rather than relying on the user to edit `PATH` to specify their location.
  The `PATH` environment variable can still be used to differentiate when
  multiple installations are present or the programs are installed to non-
  standard locations.
- Fixed an exception on parsing Ghostscript error messages.
- Added an improved example demonstrating how to set up a watched folder
  for automated OCR processing (thanks to @ianalexander for the contribution).

## v9.2.0

- Native Windows is now supported.
- Continuous integration moved to Azure Pipelines.
- Improved test coverage and speed of tests.
- Fixed an issue where a page that was originally a JPEG would be saved as a
  PNG, increasing file size. This occurred only when a preprocessing option
  was selected along with `--output-type=pdf` and all images on the original
  page were JPEGs. Regression since v7.0.0.
- OCRmyPDF no longer depends on the QPDF executable `qpdf` or `libqpdf`.
  It uses pikepdf (which in turn depends on `libqpdf`). Package maintainers
  should adjust dependencies so that OCRmyPDF no longer calls for libqpdf on
  its own. For users of Python binary wheels, this change means a separate
  installation of QPDF is no longer necessary. This change is mainly to
  simplify installation on Windows.
- Fixed a rare case where log messages from Tesseract would be discarded.
- Fixed incorrect function signature for pixFindPageForeground, causing
  exceptions on certain platforms/Leptonica versions.

## v9.1.1

- Expand the range of pdfminer.six versions that are supported.
- Fixed Docker build when using pikepdf 1.7.0.
- Fixed documentation to recommend using pip from get-pip.py.

## v9.1.0

- Improved diagnostics when file size increases at output. Now warns if JBIG2
  or pngquant were not available.
- pikepdf 1.7.0 is now required, to pick up changes that remove the need for
  a source install on Linux systems running Python 3.8.

## v9.0.5

- The Alpine Docker image (jbarlow83/ocrmypdf-alpine) has been dropped due to
  the difficulties of supporting Alpine Linux.
- The primary Docker image (jbarlow83/ocrmypdf) has been improved to take on
  the extra features that used to be exclusive to the Alpine image.
- No changes to application code.
- pdfminer.six version 20191020 is now supported.

## v9.0.4

- Fixed compatibility with Python 3.8 (but requires source install for the moment).
- Fixed Tesseract settings for `--user-words` and `--user-patterns`.
- Changed to pikepdf 1.6.5 (for Python 3.8).
- Changed to Pillow 6.2.0 (to mitigate a security vulnerability in earlier Pillow).
- A debug message now mentions when English is automatically selected if the locale
  is not English.

## v9.0.3

- Embed an encoded version of the sRGB ICC profile in the intermediate
  Postscript file (used for PDF/A conversion). Previously we included the
  filename, which required Postscript to run with file access enabled. For
  security, Ghostscript 9.28 enables `-dSAFER` and as such, no longer
  permits access to any file by default. This fix is necessary for
  compatibility with Ghostscript 9.28.
- Exclude a test that sometimes times out and fails in continuous integration
  from the standard test suite.

## v9.0.2

- The image optimizer now skips optimizing flate (PNG) encoded images in some
  situations where the optimization effort was likely wasted.
- The image optimizer now ignores images that specify arbitrary decode arrays,
  since these are rare.
- Fixed an issue that caused inversion of black and white in monochrome images.
  We are not certain but the problem seems to be linked to Leptonica 1.76.0 and
  older.
- Fixed some cases where the test suite failed if
  English or German Tesseract language packs were not installed.
- Fixed a runtime error if the Tesseract English language is not installed.
- Improved explicit closing of Pillow images after use.
- Actually fixed of Alpine Docker image build.
- Changed to pikepdf 1.6.3.

## v9.0.1

- Fixed test suite failing when either of optional dependencies unpaper and
  pngquant were missing.
- Attempted fix of Alpine Docker image build.
- Documented that FreeBSD ports are now available.
- Changed to pikepdf 1.6.1.

## v9.0.0

**Breaking changes**

- The `--mask-barcodes` experimental feature has been dropped due to poor
  reliability and occasional crashes, both due to the underlying library that
  implements this feature (Leptonica).
- The `-v` (verbosity level) parameter now accepts only `0`, `1`, and
  `2`.
- Dropped support for Tesseract 4.00.00-alpha releases. Tesseract 4.0 beta and
  later remain supported.
- Dropped the `ocrmypdf-polyglot` and `ocrmypdf-webservice` images.

**New features**

- Added a high level API for applications that want to integrate OCRmyPDF.
  Special thanks to Martin Wind (@mawi1988) whose made significant contributions
  to this effort.
- Added progress bars for long-running steps. ■■■■■■■□□
- We now create linearized ("fast web view") PDFs by default. The new parameter
  `--fast-web-view` provides control over when this feature is applied.
- Added a new `--pages` feature to limit OCR to only a specific page range.
  The list may contain commas or single pages, such as `1, 3, 5-11`.
- When the number of pages is small compared to the number of allowed jobs, we
  run Tesseract in multithreaded (OpenMP) mode when available. This should
  improve performance on files with low page counts.
- Removed dependency on `ruffus`, and with that, the non-reentrancy
  restrictions that previous made an API impossible.
- Output and logging messages overhauled so that ocrmypdf may be integrated
  into applications that use the logging module.
- pikepdf 1.6.0 is required.
- Added a logo. 😊

**Bug fixes**

- Pages with vector artwork are treated as full color. Previously, vectors
  were ignored when considering the colorspace needed to cover a page, which
  could cause loss of color under certain settings.
- Test suite now spawns processes less frequently, allowing more accurate
  measurement of code coverage.
- Improved test coverage.
- Fixed a rare division by zero (if optimization produced an invalid file).
- Updated Docker images to use newer versions.
- Fixed images encoded as JBIG2 with a colorspace other than `/DeviceGray`
  were not interpreted correctly.
- Fixed a OCR text-image registration (i.e. alignment) problem when the page
  when MediaBox had a nonzero corner.

---

### Releasenotes Version10 (`docs/releasenotes/version10.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v10

## v10.3.3

- Fixed a "KeyError: 'dpi'" error message when using `--threshold` on an image.
  ({issue}`607`)

## v10.3.2

- Fixed a case where we reported "no reason" for a file size increase, when we
  could determine the reason.
- Enabled support for pdfminer.six 20200726.

## v10.3.1

- Fixed a number of test suite failures with pdfminer.six older than version 20200402.
- Enabled support for pdfminer.six 20200720.

## v10.3.0

- Fixed an issue where we would consider images that were already JBIG2-encoded
  for optimization, potentially producing a less optimized image than the original.
  We do not believe this issue would ever cause an image to loss fidelity.
- Where available, pikepdf memory mapping is now used. This improves performance.
- When Leptonica 1.79+ is installed, use its new error handling API to avoid
  a "messy" redirection of stderr which was necessary to capture its error
  messages.
- For older versions of Leptonica, added a new thread level lock. This fixes a
  possible race condition in handling error conditions in Leptonica (although
  there is no evidence it ever caused issues in practice).
- Documentation improvements and more type hinting.

## v10.2.1

- Disabled calculation of text box order with pdfminer. We never needed this result
  and it is expensive to calculate on files with complex pre-existing text.
- Fixed plugin manager to accept `Path(plugin)` as a path to a plugin.
- Fixed some typing errors.
- Documentation improvements.

## v10.2.0

- Update Docker image to use Ubuntu 20.04.
- Fixed issue PDF/A acquires title "Untitled" after conversion. ({issue}`582`)
- Fixed a problem where, when using `--pdf-renderer hocr`, some text would
  be missing from the output when using a more recent version of Tesseract.
  Tesseract began adding more detailed markup about the semantics of text
  that our HOCR transform did not recognize, so it ignored them. This option is
  not the default. If necessary `--redo-ocr` also redoing OCR to fix such issues.
- Fixed an error in Python 3.9 beta, due to removal of deprecated
  `Element.getchildren()`. ({issue}`584`)
- Implemented support using the API with `BytesIO` and other file stream objects.
  ({issue}`545`)

## v10.1.1

- Fixed `OMP_THREAD_LIMIT` set to invalid value error messages on some input
  files. (The error was harmless, apart from less than optimal performance in
  some cases.)

## v10.1.0

- Previously, we `--clean-final` would cause an unpaper-cleaned page image to
  be produced twice, which was necessary in some cases but not in general. We
  now take this optimization opportunity and reuse the image if possible.
- We now provide PNG files as input to unpaper, since it accepts them, instead
  of generating PPM files which can be very large. This can improve performance
  and temporary disk usage.
- Documentation updated for plugins.

## v10.0.1

- Fixed regression when `-l lang1+lang2` is used from command line.

## v10.0.0

**Breaking changes**

- Support for pdfminer.six version 20181108 has been dropped, along with a
  monkeypatch that made this version work.
- Output messages are now displayed in color (when supported by the terminal)
  and prefixes describing the severity of the message are removed. As such
  programs that parse OCRmyPDF's log message will need to be revised. (Please
  consider using OCRmyPDF as a library instead.)
- The minimum version for certain dependencies has increased.
- Many API changes; see developer changes.
- The Python libraries pluggy and coloredlogs are now required.

**New features and improvements**

- PDF page scanning is now parallelized across CPUs, speeding up this phase
  dramatically for files with a high page counts.
- PDF page scanning is optimized, addressing some performance regressions.
- PDF page scanning is no longer run on pages that are not selected when the
  `--pages` argument is used.
- PDF page scanning is now independent of Ghostscript, ending our past reliance
  on this occasionally unstable feature in Ghostscript.
- A plugin architecture has been added, currently allowing one to more easily
  use a different OCR engine or PDF renderer from Tesseract and Ghostscript,
  respectively. A plugin can also override some decisions, such changing
  the OCR settings after initial scanning.
- Colored log messages.

**Developer changes**

- The test spoofing mechanism, used to test correct handling of failures in
  Tesseract and Ghostscript, has been removed in favor of using plugins for
  testing. The spoofing mechanism was fairly complex and required many special
  hacks for Windows.
- Code describing the resolution in DPI of images was refactored into a
  `ocrmypdf.helpers.Resolution` class.
- The module `ocrmypdf._exec` is now private to OCRmyPDF.
- The `ocrmypdf.hocrtransform` module has been updated to follow PEP8 naming
  conventions.
- Ghostscript is no longer used for finding the location of text in PDFs, and
  APIs related to this feature have been removed.
- Lots of internal reorganization to support plugins.

---

### Releasenotes Version11 (`docs/releasenotes/version11.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v11

## v11.7.3

- Exclude CCITT Group 3 images from being optimized. Some libraries
  OCRmyPDF uses do not seem to handle this obscure compression format properly.
  You may get errors or possible corrupted output images without this fix.

## v11.7.2

- Updated pinned versions in main.txt, primarily to upgrade Pillow to 8.1.2, due
  to recently disclosed security vulnerabilities in that software.
- The `--sidecar` parameter now causes an exception if set to the same file as
  the input or output PDF.

## v11.7.1

- Some exceptions while attempting image optimization were only logged at the debug
  level, causing them to be suppressed. These errors are now logged appropriately.
- Improved the error message related to `--unpaper-args`.
- Updated documentation to mention the new conda distribution.

## v11.7.0

- We now support using `--sidecar` in conjunction with `--pages`; these arguments
  used to be mutually exclusive. ({issue}`735`)
- Fixed a possible issue with PDF/A-1b generation. Acrobat complained that our PDFs use
  object streams. More robust PDF/A validators like veraPDF don't consider this a
  problem, but we'll honor Acrobat's objection from here on. This may increase file
  size of PDF/A-1b files. PDF/A-2b files will not be affected.

## v11.6.2

- Fixed a regression where the wrong page orientation would be produced when using
  arguments such as `--deskew --rotate-pages` ({issue}`730`).

## v11.6.1

- Fixed an issue with attempting optimize unusually narrow-width images by excluding
  these images from optimization ({issue}`732`).
- Remove an obsolete compatibility shim for a version of pikepdf that is no longer
  supported.

## v11.6.0

- OCRmyPDF will now automatically register plugins from the same virtual environment
  with an appropriate setuptools entrypoint.
- Refactor the plugin manager to remove unnecessary complications and make plugin
  registration more automatic.
- `PageContext` and `PdfContext` are now formally part of the API, as they
  should have been, since they were part of `ocrmypdf.pluginspec`.

## v11.5.0

- Fixed an issue where the output page size might differ by a fractional amount
  due to rounding, when `--force-ocr` was used and the page contained objects
  with multiple resolutions.
- When determining the resolution at which to rasterize a page, we now consider
  printed text on the page as requiring a higher resolution. This fixes issues
  with certain pages being rendered with unacceptably low resolution text, but
  may increase output file sizes in some workflows where low resolution text
  is acceptable.
- Added a workaround to fix an exception that occurs when trying to
  `import ocrmypdf.leptonica` on Apple ARM silicon (or potentially, other
  platforms that do not permit write+executable memory).

## v11.4.5

- Fixed an issue where files may not be closed when the API is used.
- Improved `setup.cfg` with better settings for test coverage.

## v11.4.4

- Fixed `AttributeError: 'NoneType' object has no attribute 'userunit'` ({issue}`700`),
  related to OCRmyPDF not properly forwarded an error message from pdfminer.six.
- Adjusted typing of some arguments.
- `ocrmypdf.ocr` now takes a `threading.Lock` for reasons outlined in the
  documentation.

## v11.4.3

- Removed a redundant debug message.
- Test suite now asserts that most patched functions are called when they should be.
- Test suite now skips a test that fails on two particular versions of piekpdf.

## v11.4.2

- Fixed support for Cygwin, hopefully.
- watcher.py: Fixed an issue with the OCR_LOGLEVEL not being interpreted.

## v11.4.1

- Fixed an issue where invalid pages ranges passed using the `pages` argument,
  such as "1-0" would cause unhandled exceptions.
- Accepted a user-contributed to the Synology demo script in misc/synology.py.
- Clarified documentation about change of temporary file location `ocrmypdf.io`.
- Fixed Python wheel tag which was incorrectly set to py35 even though we long
  since dropped support for Python 3.5.

## v11.4.0

- When looking for Tesseract and Ghostscript, we now check the Windows Registry to
  see if their installers registered the location of their executables. This should
  help Windows users who have installed these programs to non-standard
  locations.
- We now report on the progress of PDF/A conversion, since this operation is
  sometimes slow.
- Improved command line completions.
- The prefix of the temporary folder OCRmyPDF creates has been changed from
  `com.github.ocrmypdf` to `ocrmypdf.io`. Scripts that chose to depend on this
  prefix may need to be adjusted. (This has always been an implementation detail so is
  not considered part of the semantic versioning "contract".)
- Fixed {issue}`692`, where a particular file with malformed fonts would flood an
  internal message cue by generating so many debug messages.
- Fixed an exception on processing hOCR files with no page record. Tesseract
  is not known to generate such files.

## v11.3.4

- Fixed an error message 'called readLinearizationData for file that is not
  linearized' that may occur when pikepdf 2.1.0 is used. (Upgrading to pikepdf
  2.1.1 also fixes the issue.)
- File watcher now automatically includes `.PDF` in addition to `.pdf` to
  better support case sensitive file systems.
- Some documentation and comment improvements.

## v11.3.3

- If unpaper outputs non-UTF-8 data, quietly fix this rather than choke on the
  conversion. (Possibly addresses {issue}`671`.)

## v11.3.2

- Explicitly require pikepdf 2.0.0 or newer when running on Python 3.9. (There are
  concerns about the stability of pybind11 2.5.x with Python 3.9, which is used in
  pikepdf 1.x.)
- Fixed another issue related to page rotation.
- Fixed an issue where image marked as image masks were not properly considered
  as optimization candidates.
- On some systems, unpaper seems to be unable to process the PNGs we offer it
  as input. We now convert the input to PNM format, which unpaper always accepts.
  Fixes {issue}`665` and {issue}`667`.
- DPI sent to unpaper is now rounded to a more reasonable number of decimal digits.
- Debug and error messages from unpaper were being suppressed.
- Some documentation tweaks.

## v11.3.1

- Declare support for new versions: pdfminer.six 20201018 and pikepdf 2.x
- Fixed warning related to `--pdfa-image-compression` that appears at the wrong
  time.

## v11.3.0

- The "OCR" step is describing as "Image processing" in the output messages when
  OCR is disabled, to better explain the application's behavior.
- Debug logs are now only created when run as a command line, and not when OCR
  is performed for an API call. It is the calling application's responsibility
  to set up logging.
- For PDFs with a low number of pages, we gathered information about the input PDF
  in a thread rather than process (when there are more pages). When run as a
  thread, we did not close the file handle to the working PDF, leaking one file
  handle per call of `ocrmypdf.ocr`.
- Fixed an issue where debug messages send by child worker processes did not match
  the log settings of parent process, causing messages to be dropped. This affected
  macOS and Windows only where the parent process is not forked.
- Fixed the hookspec of rasterize_pdf_page to remove default parameters that
  were not handled in an expected way by pluggy.
- Fixed another issue with automatic page rotation ({issue}`658`) due to the issue above.

## v11.2.1

- Fixed an issue where optimization of a 1-bit image with a color palette or
  associated ICC that was optimized to JBIG2 could have its colors inverted.

## v11.2.0

- Fixed an issue with optimizing PNG-type images that had soft masks or image masks.
  This is a regression introduced in (or about) v11.1.0.
- Improved type checking of the `plugins` parameter for the `ocrmypdf.ocr`
  API call.

## v11.1.2

- Fixed hOCR renderer writing the text in roughly reverse order. This should not
  affect reasonably smart PDF readers that properly locate the position of all
  text, but may confuse those that rely on the order of objects in the content
  stream. ({issue}`642`)

## v11.1.1

- We now avoid using named temporary files when using pngquant allowing containerized
  pngquant installs to be used.
- Clarified an error message.
- Highest number of 1's in a release ever!

## v11.1.0

- Fixed page rotation issues: {issue}`634,589`.
- Fixed some cases where optimization created an invalid image such as a
  1-bit "RGB" image: {issue}`629,620`.
- Page numbers are now displayed in debug logs when pages are being grafted.
- ocrmypdf.optimize.rewrite_png and ocrmypdf.optimize.rewrite_png_as_g4 were
  marked deprecated. Strictly speaking these should have been internal APIs,
  but they were never hidden.
- As a precaution, pikepdf mmap-based file access has been disabled due to a
  rare race condition that causes a crash when certain objects are deallocated.
  The problem is likely in pikepdf's dependency pybind11.
- Extended the example plugin to demonstrate conversion to mono.

## v11.0.2

- Fixed {issue}`612`, TypeError exception. Fixed by eliminating unnecessary repair of
  input PDF metadata in memory.

## v11.0.1

- Blacklist pdfminer.six 20200720, which has a regression fixed in 20200726.
- Approve img2pdf 0.4 as it passes tests.
- Clarify that the GPL-3 portion of pdfa.py was removed with the changes in v11.0.0;
  the debian/copyright file did not properly annotate this change.

## v11.0.0

- Project license changed to Mozilla Public License 2.0. Some miscellaneous
  code is now under MIT license and non-code content/media remains under
  CC-BY-SA 4.0. License changed with approval of all people who were found
  to have contributed to GPLv3 licensed sections of the project. ({issue}`600`)
- Because the license changed, this is being treated as a major version number
  change; however, there are no known breaking changes in functional behavior
  or API compared to v10.x.

---

### Releasenotes Version12 (`docs/releasenotes/version12.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v12

## v12.7.2

- Fixed "invalid version number" error for Tesseract packaging with nonstandard
  version "5.0.0-rc1.20211030".
- Fixed use of deprecated `importlib.resources.read_binary`.
- Replace some uses of string paths with `pathlib.Path`.
- Fixed a leaked file handle when using `--output-type none`.
- Removed shims to support versions of pikepdf that are no longer supported.

## v12.7.1

- Declare support for pdfminer.six v20211012.

## v12.7.0

- Fixed test suite failure when using pikepdf 3.2.0 that was compiled with pybind11
  2.8.0. {issue}`843`
- Improve advice to user about using `--max-image-mpixels` if OCR fails for this
  reason.
- Minor documentation fixes. (Thanks to @mara004.)
- Don't require importlib-metadata and importlib-resources backports on versions of
  Python where the standard library implementation is sufficient.
  (Thanks to Marco Genasci.)

## v12.6.0

- Implemented `--output-type=none` to skip producing PDFs for applications that
  only want sidecar files ({issue}`787`).
- Fixed ambiguities in descriptions of behavior of `--jbig2-lossy`.
- Various improvements to documentation.

## v12.5.0

- Fixed build failure for the combination of PyPy 3.6 and pikepdf 3.0. This
  combination can work in a source build but does not work with wheels.
- Accepted bot that wanted to upgrade our deprecated requirements.txt.
- Documentation updates.
- Replace pkg_resources and install dependency on setuptools with
  importlib-metadata and importlib-resources.
- Fixed regression in hocrtransform causing text to be omitted when this
  renderer was used.
- Fixed some typing errors.

## v12.4.0

- When grafting text layers, use pikepdf's `unparse_content_stream` if available.
- Confirmed support for pluggy 1.0. (Thanks @QuLogic.)
- Fixed some typing issues, improved pre-commit settings, and fixed issues
  flagged by linters.
- PyPy 7.3.3 (=Python 3.6) is now supported. Note that PyPy does not necessarily
  run faster, because the vast majority of OCRmyPDF's execution time is spent
  running OCR or generally executing native code. However, PyPy may bring speed
  improvements in some areas.

## v12.3.3

- watcher.py: fixed interpretation of boolean env vars ({issue}`821`).
- Adjust CI scripts to test Tesseract 5 betas.
- Document our support for the Tesseract 5 betas.

## v12.3.2

- Indicate support for flask 2.x, watcher 2.x ({issue}`815, 816`).

## v12.3.1

- Fixed issue with selection of text when using the hOCR renderer ({issue}`813`).
- Fixed build errors with the Docker image by upgrading to a newer Ubuntu.
  Also set the timezone of this image to UTC.

## v12.3.0

- Fixed a regression introduced in Pillow 8.3.0. Pillow no longer rounds DPI
  for image resolutions. We now account for this ({issue}`802`).
- We no longer use some API calls that are deprecated in the latest versions of
  pikepdf.
- Improved error message when a language is requested that doesn't look like a
  typical ISO 639-2 code.
- Fixed some tests that attempted to symlink on Windows, breaking tests on a
  Windows desktop but not usually on CI.
- Documentation fixes (thanks to @mara004)

## v12.2.0

- Fixed invalid Tesseract version number on Windows ({issue}`795`).
- Documentation tweaks. Documentation build now depends on sphinx-issues package.

## v12.1.0

- For security reasons we now require Pillow >= 8.2.x. (Older versions will continue
  to work if upgrading is not an option.)
- The build system was reorganized to rely on `setup.cfg` instead of `setup.py`.
  All changes should work with previously supported versions of setuptools.
- The files in `requirements/*` are now considered deprecated but will be retained for v12.
  Instead use `pip install ocrmypdf[test]` instead of `requirements/test.txt`, etc.
  These files will be removed in v13.

## v12.0.3

- Expand the list of languages supported by the hocr PDF renderer.
  Several languages were previously considered not supported, particularly those
  non-European languages that use the Latin alphabet.
- Fixed a case where the exception stack trace was suppressed in verbose mode.
- Improved documentation around commercial OCR.

## v12.0.2

- Fixed exception thrown when using `--remove-background` on files containing small
  images ({issue}`769`).
- Improve documentation for description of adding language packs to the Docker image
  and corrected name of French language pack.

## v12.0.1

- Fixed "invalid version number" for untagged tesseract versions ({issue}`770`).

## v12.0.0

**Breaking changes**

- Due to recent security issues in pikepdf, Pillow and reportlab, we now require
  newer versions of these libraries and some of their dependencies. (If necessary,
  package maintainers may override these versions at their discretion; lower
  versions will often work.)
- We now use the "LeaveColorUnchanged" color conversion strategy when directing
  Ghostscript to create a PDF/A. Generally this is faster than performing a
  color conversion, which is not always necessary.
- OCR text is now packaged in a Form XObject. This makes it easier to isolate
  OCR from other document content. However, some poorly implemented PDF text
  extraction algorithms may fail to detect the text.
- Many API functions have stricter parameter checking or expect keyword arguments
  were they previously did not.
- Some deprecated functions in `ocrmypdf.optimize` were removed.
- The `ocrmypdf.leptonica` module is now deprecated, due to difficulties with
  the current strategy of ABI binding on newer platforms like Apple Silicon.
  It will be removed and replaced, either by repackaging Leptonica as an
  independent library using or using a different image processing library.
- Continuous integration moved to GitHub Actions.
- We no longer depend on `pytest_helpers_namespace` for testing.

**New features**

- New plugin hook: `get_progressbar_class`, for progress reporting,
  allowing developers to replace the standard console progress bar with some
  other mechanism, such as updating a GUI progress bar.
- New plugin hook: `get_executor`, for replacing the concurrency model.
  This is primarily to support execution on AWS Lambda, which does not support
  standard Python `multiprocessing` due to its lack of shared memory.
- New plugin hook: `get_logging_console`, for replacing the standard
  way OCRmyPDF outputs its messages.
- New plugin hook: `filter_pdf_page`, for modifying individual PDF
  pages produced by OCRmyPDF.
- OCRmyPDF now runs on nonstandard execution environments that do not have
  interprocess semaphores, such as AWS Lambda and Android Termux. If the environment
  does not have semaphores, OCRmyPDF will automatically select an alternate
  process executor that does not use semaphores.
- Continuous integration moved to GitHub Actions.
- We now generate an ARM64-compatible Docker image alongside the x64 image.
  Thanks to @andkrause for doing most of the work in a pull request several months
  ago, which we were finally able to integrate now. Also thanks to @0x326 for
  review comments.

**Fixes**

- Fixed a possible deadlock on attempting to flush `sys.stderr` when older
  versions of Leptonica are in use.
- Some worker processes inherited resources from their parents such as log
  handlers that may have also lead to deadlocks. These resources are now released.
- Improvements to test coverage.
- Removed vestiges of support for Tesseract versions older than 4.0.0-beta1 (
  which ships with Ubuntu 18.04).
- OCRmyPDF can now parse all of Tesseract version numbers, since several
  schemes have been in use.
- Fixed an issue with parsing PDFs that contain images drawn at a scale of 0. ({issue}`761`)
- Removed a frequently repeated message about disabling mmap.

---

### Releasenotes Version13 (`docs/releasenotes/version13.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v13

## v13.7.0

- Fixed an exception when attempting to run and Tesseract is not installed.
- Changed to SPDX license tracking and information files.

## v13.6.2

- Added a shim to prevent an "error during error handling" for Python 3.7 and 3.8.
- Modernized some type annotations.
- Improved annotations on our \_windows module to help IDEs and mypy figure out what
  we're doing.

## v13.6.1

- Require setuptools-scm 7.0.5 to avoid possible issues with source distributions in
  earlier versions of setuptools-scm.
- Suppress a spurious warning, improve tests, improve typing and other miscellany.

## v13.6.0

- Added a new `initialize` plugin hook, making it possible to suppress built-in
  plugins more easily, among other possibilities.
- Fixed an issue where unpaper would exit with a "wrong stream" error, probably
  related to images with an odd integer width. {issue}`887, 665`

## v13.5.0

- Added a new `optimize_pdf` plugin hook, making it possible to create plugins that
  replace or enhance OCRmyPDF's PDF optimizer.
- Removed all max version restrictions. Our new policy is to blacklist known-bad releases
  and only block known-bad versions of dependencies.
- The naming schema for object that holds all OCR text that OCRmyPDF inserts has
  changed. This has always been an implementation detail (and remains so), but possibly,
  someone was relying on it and would appreciate the heads-up.
- Cleanup.

## v13.4.7

- Fixed PermissionError when cleaning up temporary files in rare cases. {issue}`974`
- Fixed PermissionError when calling `os.nice` on platforms that lack it. {issue}`973`
- Suppressed some warnings from libxmp during tests.

## v13.4.6

- Convert error on corrupt ICC profiles into a warning. Thanks to @oscherler.

## v13.4.5

- Remove upper bound on pdfminer.six version.
- Documentation.

## v13.4.4

- Updated pdfminer.six version.
- Docker image changed to Ubuntu 22.04 now that it is released and provides the
  dependencies we need. This seems more consistent than our recent change to
  Debian.

## v13.4.3

- Fix error on pytest.skip() with older versions of pytest.
- Documentation updates.

## v13.4.2

- Worked around a
  [major regression in Ghostscript 9.56.0](https://bugs.ghostscript.com/show_bug.cgi?id=705187)
  where **all OCR text is stripped out of the PDF**. It simply removes all text,
  even generated by software other than OCRmyPDF. Fortunately, we can ask
  Ghostscript 9.56.0 to use its old behavior that worked correctly for our purposes.
  Users must avoid the combination (Ghostscript 9.56.0, ocrmypdf \<13.4.2) since
  older versions of OCRmyPDF have no way of detecting that this particular
  version of Ghostscript removes all OCR text.
- Marked pdfminer 20220319 as supported.
- Fixed some deprecation warnings from recent versions of Pillow and pytest.
- Test suite now covers Python 3.10 (Python 3.10 worked fine before, but was not
  being tested).
- Docker image now uses debian:bookworm-slim as the base image to fix the Docker
  image build.

## v13.4.1

- Temporarily make threads rather than processes the default executor worker, due
  to a persistent deadlock issue when processes are used. Add a new command line
  argument `--no-use-threads` to disable this.

## v13.4.0

- Fixed test failures when using pikepdf 5.0.0.
- Various improvements to the optimizer. In particular, we now recognize PDF images
  that are encoded with both deflate (PNG) and DCT (JPEG), and also produce PDF
  with images compressed with deflate and DCT, since this often yields file size
  improvements compared to plain DCT.

## v13.3.0

- Made a harmless but "scary" exception after failing to optimize an image less scary.
- Added a warning if a page image is too large for unpaper to clean. The image is
  passed through without cleaning. This is due to a hard-coded limitation in a
  C library used by unpaper so it cannot be rectified easily.
- We now use better default settings when calling img2pdf.
- We no longer try to optimize images that we failed to save in certain situations.
- We now account for some differences in text output from Tesseract 5 compared to
  Tesseract 4.
- Better handling of Ghostscript producing empty images when attempting to rasterize
  page images.

## v13.2.0

- Removed all runtime uses of distutils since it is deprecated in standard library. We
  previous used `distutils.version` to examine version numbers of dependencies
  at run time, and now use `packaging.version` for this. This is a new
  dependency.
- Fixed an error message advising the user that Ghostscript was not installed being
  suppressed when this condition actually happens.
- Fixed an issue with incorrect page number and totals being displayed in the progress
  bar. This was purely a display/presentation issue. {issue}`876`.

## v13.1.1

- Fixed issue with attempting to deskew a blank page on Tesseract 5. {issue}`868`.

## v13.1.0

- Changed to using Python concurrent.futures-based parallel execution instead of
  pools, since futures have now exceed pools in features.
- If a child worker is terminated (perhaps by the operating system or the user
  killing it in a task manager), the parallel task will fail an error message.
  Previously, the main ocrmypdf process would "hang" indefinitely, waiting for the
  child to report.
- Added new argument `--tesseract-thresholding` to provide control over Tesseract 5's
  threshold parameter.
- Documentation updates and changes. Better documentation for `--output-type none`,
  added a few releases ago. Removed some obsolete documentation.
- Improved bash completions - thanks to @FPille.

## v13.0.0

**Breaking changes**

- The deprecated module `ocrmypdf.leptonica` has been removed.
- We no longer depend on Leptonica (`liblept`) or CFFI (`libffi`,
  `python3-cffi`). (Note that Tesseract still requires Leptonica; OCRmyPDF no longer
  directly uses this library.)
- The argument `--remove-background` is temporarily disabled while we search for an
  alternative to the Leptonica implementation of this feature.
- The `--threshold` argument has been removed, since this also depended on Leptonica.
  Tesseract 5.x has implemented improvements to thresholding, so this feature will be
  redundant anyway.
- `--deskew` was previous calculated by a Leptonica algorithm. We now use a feature
  of Tesseract to find the appropriate the angle to deskew a page. The deskew angle
  according to Tesseract may differ from Leptonica's algorithm. At least in theory,
  Tesseract's deskew angle is informed by a more complex analysis than Leptonica,
  so this should improve results in general. We also use Pillow to perform the
  deskewing, which may affect the appearance of the image compared to Leptonica.
- Support for Python 3.6 was dropped, since this release is approaching end of life.
- We now require pikepdf 4.0 or newer. This, in turn, means that OCRmyPDF requires
  a system compatible with the manylinux2014 specification. This change was "forced"
  by Pillow not releasing manylinux2010 wheels anymore.
- We no longer provide requirements.txt-style files. Use `pip install ocrmypdf[...]`
  instead.
- Bumped required versions of several libraries.

**Fixes**

- Fixed an issue where OCRmyPDF failed to find Ghostscript on Windows even when
  installed, and would exit with an error.
- By removing Leptonica, we fixed all issues related to Leptonica on Apple
  Silicon or Leptonica failing to import on Windows.

---

### Releasenotes Version14 (`docs/releasenotes/version14.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v14

## v14.4.0

- Digitally signed PDFs are now detected. If the PDF is signed, OCRmyPDF will
  refuse to modify it. Previously, only encrypted PDFs were detected, not
  those that were signed but not encrypted. {issue}`1040`
- In addition, `--invalidate-digital-signatures` can be used to override the
  above behavior and modify the PDF anyway. {issue}`1040`
- tqdm progress bars replaced with "rich" progress bars. The rich library is
  a new dependency. Certain APIs that used tqdm are now deprecated and will
  be removed in the next major release.
- Improved integration with GitHub Releases. Thanks to @stumpylog.

## v14.3.0

- Renamed master branch to main.
- Improve PDF rasterization accuracy by using the `-dPDFSTOPONERROR` option
  to Ghostscript. Use `--continue-on-soft-render-error` if you want to render
  the PDF anyway. The plugin specification was adjusted to support this feature;
  plugin authors may want to adapt PDF rasterizing and rendering
  plugins. {issue}`1083`
- The calculated deskew angle is now recorded in the logged output. {issue}`1101`
- Metadata can now be unset by setting a metadata type such as `--title` to an
  empty string. {issue}`1117,1059`
- Fixed random order of languages due to use of a set. This may have caused output
  to vary when multiple languages were set for OCR. {issue}`1113`
- Clarified the optimization ratio reported in the log output.
- Documentation improvements.

## v14.2.1

- Fixed {issue}`977`, where images inside Form XObjects were always excluded
  from image optimization.

## v14.2.0

- Added `--tesseract-downsample-above` to downsample larger images even when
  they do not exceed Tesseract's internal limits. This can be used to speed
  up OCR, possibly sacrificing accuracy.
- Fixed resampling AttributeError on older Pillow. {issue}`1096`
- Removed an error about using Ghostscript on PDFs with that have the /UserUnit
  feature in use. Previously, Ghostscript would fail to process these PDFs,
  but in all supported versions it is now supported, so the error is no longer
  needed.
- Improved documentation around installing other language packs for Tesseract.

## v14.1.0

- Added `--tesseract-non-ocr-timeout`. This allows using Tesseract's deskew
  and other non-OCR features while disabling OCR using `--tesseract-timeout 0`.
- Added `--tesseract-downsample-large-images`. This downsamples larges images
  that exceed the maximum image size Tesseract can handle. Large images may still
  take a long time to process, but this allows them to be processed if that
  is desired.
- Fixed {issue}`1082`, an issue with snap packaged building.
- Change linter to ruff, fix lint errors, update documentation.

## v14.0.4

- Fixed {issue}`1066, 1075`, an exception when processing certain malformed PDFs.

## v14.0.3

- Fixed {issue}`1068`, avoid deleting /dev/null when running as root.
- Other documentation fixes.

## v14.0.2

- Fixed {issue}`1052`, an exception on attempting to process certain nonconforming PDFs.
- Explicitly documented that Windows 32-bit is no longer supported.
- Fixed source installation instructions.
- Other documentation fixes.

## v14.0.1

- Fixed some version checks done with smart version comparison.
- Added missing jbig2dec to Docker image.

## v14.0.0

- Dropped support for Python 3.7.
- Dropped support generally speaking, all dependencies older than what Ubuntu 20.04
  provides.
- Ghostscript 9.50 or newer is now required. Shims to support old versions were
  removed.
- Tesseract 4.1.1 or newer is now required. Shims to support old versions were
  removed.
- Docker image now uses Tesseract 5.
- Dropped setup.cfg configuration for pyproject.toml.
- Removed deprecation exception PdfMergeFailedError.
- A few more public domain test files were removed or replaced. We are aiming for
  100% compliance with SPDX and generally towards simplifying copyright.

---

### Releasenotes Version15 (`docs/releasenotes/version15.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v15

## v15.4.4

- Fixed documentation for installing Ghostscript on Windows. {issue}`1198`
- Added warning message about security issue in older versions of Ghostscript.

## v15.4.3

- Fixed deprecation warning in pikepdf older than 8.7.1; pikepdf >= 8.7.1 is
  now required.

## v15.4.2

- We now raise an exception on a certain class of PDFs that likely need an
  explicit color conversion strategy selected to display correctly
  for PDF/A conversion.
- Fixed an error that occurred while trying to write a log message after the
  debug log handler was removed.

## v15.4.1

- Fixed misc/watcher.py regressions: accept `--ocr-json-settings` as either
  filename or JSON string, as previously; and argument count mismatch.
  {issue}`1183,1185`
- We no longer attempt to set /ProcSet in the PDF output, since this is an
  obsolete PDF feature.
- Documentation improvements.

## v15.4.0

- Added new experimental APIs to support offline editing of the final text.
  Specifically, one can now generate hOCR files with OCRmyPDF, edit them with
  some other tool, and then finalize the PDF. They are experimental and
  subject to change, including details of how the working folder is used.
  There is no command line interface.
- Code reorganization: executors, progress bars, initialization and setup.
- Fixed test coverage in cases where the coverage tool did not properly trace
  into threads or subprocesses. This code was still being tested but appeared
  as not covered.
- In the test suite, reduced use of subprocesses and other techniques that
  interfere with coverage measurement.
- Improved error check for when we appear to be running inside a snap container
  and files are not available.
- Plugin specification now properly defines progress bars as a protocol rather
  than defining them as "tqdm-like".
- We now default to using "forkserver" process creation on POSIX platforms
  rather than fork, since this is method is more robust and avoids some
  issues when threads are present.
- Fixed an instance where the user's request to `--no-use-threads` was ignored.
- If a PDF does not have language metadata on its top level object, we add
  the OCR language.
- Replace some cryptic test error messages with more helpful ones.
- Debug messages for how OCRmyPDF picks the colorspace for a page are now
  more descriptive.

## v15.3.1

- Fixed an issue with logging settings for misc/watcher.py introduced in the
  previous release. {issue}`1180`
- We now attempt to preserve the input's extended attributes when creating
  the output file.
- For some reason, the macOS build now needs OpenSSL explicitly installed.
- Updated documentation on Docker performance concerns.

## v15.3.0

- Update misc/watcher.py to improve command line interface using Typer, and
  support `.env` specification of environment variables. Improved error
  messages. Thanks to @mflagg2814 for the PR that prompted this improvement.
- Improved error message when a file cannot be read because we are running in
  a snap container.

## v15.2.0

- Added a Docker image based on Alpine Linux. This image is smaller than the
  Ubuntu-based image and may be useful in some situations. Currently hosted at
  jbarlow83/ocrmypdf-alpine. Currently not available in ARM flavor.
- The Ubuntu Docker is now aliased to jbarlow83/ocrmypdf-ubuntu.
- Updated Docker documentation.

## v15.1.0

- We now require Pillow 10.0.1, due a serious security vulnerability in all earlier
  versions of that dependency. The vulnerability concerns WebP images and could
  be triggered in OCRmyPDF when creating a PDF from a malicious WebP image.
- Added some keyword arguments to `ocrmypdf.ocr` that were previously accepted
  but undocumented.
- Documentation updates and typing improvements.

## v15.0.2

- Added Python 3.12 to test matrix.
- Updated documentation for notes on Python 3.12, 32-bit support and some new
  features in v15.

## v15.0.1

- Wheels Python tag changed to py39.
- Marked as a expected fail a test that fails on recent Ghostscript versions.
- Clarified documentation and release notes around the extent of 32-bit support.
- Updated installation documentation to changes in v15.

## v15.0.0

- Dropped support for Python 3.8.
- Dropped support some older dependencies, specifically `coloredlogs` and
  `tqdm` in favor of rich - see `pyproject.toml` for details.
  Generally speaking, Ubuntu 22.04 is our new baseline system.
- Tightened version requirements for some dependencies.
- Dropped support for 32-bit Linux wheels. We strongly recommend a 64-bit operating
  system, and 64-bit versions of Python, Tesseract and Ghostscript to use OCRmyPDF.
  Many of our dependencies are dropping 32-bit builds (e.g. Pillow), and we are
  following suit. (Maintainers may still build 32-bit versions from source.)
- Changed to trusted release for PyPI publishing.
- pikepdf memory mapping is enabled again for improved performance, now that an
  issue with feature in pikepdf is fixed.
- `ocrmypdf.helpers.calculate_downsample` previously had two variants, one
  that took a `PIL.Image` and one that took a `tuple[int, int]`. The latter
  was removed.
- The snap version of ocrmypdf is now based on Ubuntu core22.
- We now account for situations where a small portion of an image on a page is drawn
  at high DPI (resolution). Previously, the entire page would be rasterized at the
  highest resolution of any feature, which caused performance problems. Now,
  the page is rasterized
  at a resolution based on the average DPI of the page, weighted by the area that
  each feature occupies. Typically, small areas of high resolution in PDFs are
  errors or quirks from the repeated use of assets and high resolution is not
  beneficial. {issue}`1010,1104,1004,1079,1010`
- Ghostscript color conversion strategy is now configurable using
  `--color-conversion-strategy`. {issue}`1143`
- JBIG2 threshold for optimization is now configurable using
  `--jbig2-threshold`. {issue}`1133`

---

### Releasenotes Version16 (`docs/releasenotes/version16.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v16

## v16.13.0

- Added detection and repair for Ghostscript 10.6 JPEG corruption. When GS 10.6
  truncates JPEG data by 1-15 bytes, OCRmyPDF now restores the original image
  bytes from the input PDF. A warning is issued when GS 10.6+ is detected.
  {issue}`1603`
- We continue to force re-optimization of JPEGs, since this catches some issues with corruption for situations where Ghostscript modifies an image. It is likely there are still cases where we cannot mitigate all corruption issues. {issue}`1585`
- Fixed handling of PDF page boxes (ArtBox, BleedBox) which were not being
  processed correctly in some cases. {issue}`1181,1360`
- Documentation: clarified podman usage instructions.

## v16.12.0

- Disable Ghostscript's subset fonts feature, which was found to corrupt text in certain
  PDFs. Thanks @mnaegler for identifying this issue. {issue}`1592`
- Users of Ghostscript 10.6.0+ reported that Ghostscript seems to generate corrupted
  JPEGs. We force re-optimization of these JPEGs to mitigate the corruption until
  Ghostscript fixes the issue. {issue}`1585`
- OCRmyPDF now avoids applying flate compression to large JPEG images, unless maximum
  optimization is requested, since flate+DCT compression reduces performances in PDF
  viewers with large images.
- Updated Dockerfiles to use more recent base operating systems.
- Updated build and test matrix to include Python 3.14.
- Minor documentation improvements.
- pikepdf >= 10.0.0 is now required.

## v16.11.1

- Fixed issue with Tesseract changing an error message related to skew. {issue}`1576`
- Dropped macOS 13 from build-test matrix since it is no longer supported by Apple.

## v16.11.0

- Deprecated "semfree" plugin in favor of falling back to threads if the platform
  does not support semaphores. Fixes an issue with Python 3.14.
- Fixed references to PDF/A compliances levels to be consistent with ISO nomenclature.
  Thanks @5HT2. {issue}`1557`
- Fixed an issue around using plugin_manager as an argument. {issue}`1555`
- Added OpenBSD install steps to README. {issue}`1554`
- Removed PyPy from test matrix due to declining support in third party libraries.
- Documentation improvements.

## v16.10.4

- Corrected build errors in Python 3.13.3 and 3.13.4.

## v16.10.3 (not released)

- Blocked optimization of images with pre-blended soft masks. {issue}`1536`
- Fixed warning from hypothesis on running tests.
- Release incomplete due to new test failures in Python 3.13.3 and 3.13.4.

## v16.10.2

- Blacklist pikepdf 9.8.0 due to an incompatible change.

## v16.10.1

- No changes affecting OCRmyPDF functionality for command line end users.
- webservice: made page specification easier to find in UI.
- webservice: fix download button downloads wrong file.
- Converted project documentation from rST to Markdown.
- Added README translation to Simplified Chinese. Thanks @HuaPai.
- Modernized license specification in pyproject.toml.
- Modernized SPDX license to REUSE.toml.

## v16.10.0

- Added hocr textangle processing, improving handling of text at angles.
  Thanks @0dinD {issue}`1467`
- Docker documentation updates related to podman. Thanks @rugk. {issue}`1489,1488`
- Dropped webservice.py's fragile use of ttyd. Instead, messages from ocrmypdf are
  printed to the console.
- Fixed broken test test_hocrtransform_matches_sandwich, which had become
  an invalid test. Thanks @QuLogic for reporting.
- Improved install instructions for Windows. Thanks @alex.

## v16.9.0

- Added hocr caption processing. Thanks @0dinD {issue}`1466`
- ocrmypdf-alpine Docker image is now built with Alpine 3.21.
- Fixed error handling of PDFs that contain invalid images with both ImageMask
  and ColorSpace defined. {issue}`1453`
- Fixed test suite regression when only older Ghostscripts are installed.
- Improved documetnation of \_progressbar.py. Thanks @QuentinFuxa. {issue}`1456`
- Disabling building of documentation as PDF on ReadTheDocs, as this caused
  complex build issues deemed not worth solving.

## v16.8.0

- Upgraded webservice.py demonstration using streamlit. It's now possible to
  exercise most of OCRmyPDF's functionality in a simple web UI.
- Added cache to Dockerfiles to improve build speed.
- Fixed numerous formatting errors in the documentation that prevented some
  parts of documentation from generating correctly.
- Improved OCR text rendering by suppressing negative-width spaces. Thanks
  @pajowu. {issue}`1446`
- Improved detecting of invisible text when using `--redo-ocr`. Thanks
  @pajowu. {issue}`1448``

## v16.7.0

- Fixed further issues with Docker build and updated some versions.
- Main Docker image returned to Ubuntu 24.04 since the fix in v16.6.2 resolved
  that concern.
- Code that previously sent Ghostscript output to stdout has been changed to
  output to temporary files, since Ghostscript was doing that anyway internally.
  This is a modest efficiency improvement.
- Fixed an issue with debug log output being parsed as rich markup. {issue}`1444`

## v16.6.2

- Remove invalid hyperlink annotations to satisfy Ghostscript 10.x during PDF/A
  conversion. {issue}`1425`

## v16.6.1

- Fixed some issues with Docker build, such as removing unnecessary content and using
  a stable Tesseract version.
- Reverted Docker image to Ubuntu 22.04 to access older/more stable Ghostscript
  for now.
- Clarified batch commands in documentation.
- Fixed an issue with JSON serialization and pickling of HOCRResult. {issue}`1427`

## v16.6.0

- Fixed an issue where damaged PDFs would fail with `--redo-ocr`. {issue}`1403`
- Fixed an error that prevented JBIG2 optimization on Windows if the image
  was optimized in an earlier step. {issue}`1396`
- Fixed an error detecting the version of unpaper 7.0.0. {issue}`1409`
- Fixed a performance regression when scanning pages. {issue}`1378`. Thanks @aliemjay.
- Fixed Alpine Docker image by enforcing Alpine 3.19. Alpine 3.20 includes a
  defective version of Tesseract OCR and so is not usable.
- Upgraded Ubuntu Docker image to use Ubuntu 24.04.
- Build and test scripts/actions switched to uv.
- When running in a container, we now remind the user that temporary folders
  are inside the container and may not be accessible.
- Fixed Linux test coverage matrix, which was missing some key versions.

## v16.5.0

- Fixed issue with interpreting PDFs that have images with array masks.
  {issue}`1377`
- Enabled testing on Python 3.13.
- Fixed a test that did not work correctly but still passed. {issue}`1382`
- Improved "PDF/A conversion failed" warning message to better describe implications.
- Updated documentation to better explain OCR_JSON_SETTINGS in batch processing.
- Build backend changed from setuptools to hatchling.

## v16.4.3

- Work around pdfminer.six issue where a token on the buffer boundary is incorrectly
  parsed as two tokens. {issue}`1361`
- New rules are applied to stencil masks and explicit masks when calculating the
  optimal page DPI for rendering. {issue}`1362`
- Fixed attempts to use an incompatible jbig2.EXE provided by TeX Live. {issue}`1363`

## v16.4.2

- Fixed order of filenames passed to Ghostscript for PDF/A generation. {issue}`1359`
- Suppressed missing jbig2dec warning message. {issue}`1358`
- Fixed calculation of image size when soft mask dimensions don't match image
  dimension. {issue}`1351`
- Several fixes to documentation. Thanks to users Iris and JoKalliauer
  who contributed these changes.
- Fixed error on processing PDFs that are missing certain image metadata. {issue}`1315`

## v16.4.1

- Fixed calculation of image printed area (used in finding weighted DPI for OCR).
  {issue}`1334`
- Fixed "NotImplementedError: not sure how to get colorspace" error
  messages in logs which simply records a failure to optimize images with
  print production colorspaces. {issue}`1315`

## v16.4.0

- Selecting the `osd` and `equ` pseudo-languages with `-l/--language` now
  exits with an error when using Tesseract OCR, because these are not
  regular Tesseract languages but implementation details implemented.
  Using them can cause Tesseract to crash.
- The hOCR renderer is more tolerant of extra whitespace in input files.
- watcher.py now changes the output file extension to .pdf when the input is not
  .pdf.
- Improved handling of PDFs that contain circularly referenced Form XObjects.
  {issue}`1321`
- Fixed Alpine Docker image for ARM64, which was not building correctly.
- Docker images now use pikepdf 9.0.0.
- Prevent use of Tesseract OCR 5.4.0, a version with known regressions.
- Disabled progressbar for "Linearizing" when `--no-progress-bar` set.
- Fixed some tests that warn about missing JBIG2 decoding via pikepdf, by
  installing the necessary libraries during tests.

## v16.3.1

- Fixed a test suite failure with Ghostscript 10.03.0+. {issue}`1316`
- Fixed an issue with the presentation of the "OCR" progress bar. {issue}`1313`

## v16.3.0

- Fixed progress bar not displaying for Ghostscript PDF/A conversion. {issue}`1313`
- Added progress bar for linearization. {issue}`1313`
- If `--rotate-pages-threshold` issued without `--rotate-pages` we now exit with
  an error since the user likely intended to use `--rotate-pages`. {issue}`1309`
- If Tesseract hOCR gives an invalid line box, print an error message instead of
  exiting with an error. {issue}`1312`

## v16.2.0

- Fixed issue 'NoneType' object has no attribute 'get' when optimizing certain PDFs.
  {issue}`1293,1271`
- Switched formatting from black to ruff.
- Added support for sending sidecar output to io.BytesIO.
- Added support for converting HEIF/HEIC images (the native image of iPhones and
  some other devices) to PDFs, when the appropriate pi-hief library is installed.
  This library is marked as a dependency, but maintainers may opt out if needed.
- We now default to downsampling large images that would exceed Tesseract's internal
  limits, but only if it cause processing to fail. Previously, this behavior only
  occurred if specifically requested on command line. It can still be configured
  and disabled. See the --tesseract command line options.
- Added Macports install instructions. Thanks @akierig.
- Improved logging output when an unexpected error occurs while trying to obtain
  the version of a third party program.

## v16.1.2

- Fixed test suite failure when using Ghostscript 10.3.
- Other minor corrections.

## v16.1.1

- Fixed PyPy 3.10 support.

## v16.1.0

- Improved hOCR renderer is now default for left to right languages.
- Improved handling of rotated pages. Previously, OCR text might be missing for
  pages that were rotated with a /Rotate tag on the page entry.
- Improved handling of cropped pages. Previously, in some cases a page with a
  crop box would not have its OCR applied correctly and misalignment between
  OCR text and visible text coudl occur.
- Documentation improvements, especially installation instructions for less
  common platforms.

## v16.0.4

- Fixed some issues for left-to-right text with the new hOCR renderer. It is still
  not default yet but will be made so soon. Right-to-left text is still in progress.
- Added an error to prevent use of several versions of Ghostscript that seem
  corrupt existing text in input PDFs. Newly generated OCR is not affected.
  For best results, use Ghostscript 10.02.1 or newer, which contains the fix
  for the issue.

## v16.0.3

- Changed minimum required Ghostscript to 9.54, to support users of RHEL 9 and its
  derivatives, since that is the latest version available there.
- Removed warning message about CVE-2023-43115, on the assumption that most
  distributions have backported the patch by now.

## v16.0.2

- Temporarily changed PDF text renderer back to sandwich by default to address
  regressions in macOS Preview.

## v16.0.1

- Fixed text rendering issue with new hOCR text renderer - extraneous byte order
  marks.
- Tightened dependencies.

## v16.0.0

- Added OCR text renderer, combined the best ideas of Tesseract's PDF
  generator and the older hOCR transformer renderer. The result is a hopefully
  permanent fix for wordssmushedtogetherwithoutspaces issues in extracted text,
  better registration/position of text on skewed baselines {issue}`1009`,
  fixes to character output when the German Fraktur script is used {issue}`1191`,
  proper rendering of right to left languages (Arabic, Hebrew, Persian) {issue}`1157`.
  Asian languages may still have excessive word breaks compared to expectations.
  The new renderer is the default; the old sandwich renderer is still available
  using `--pdf-renderer sandwich`; the old hOCR renderer is no more.
- The `ocrmypdf.hocrtransform` API has changed substantially.
- Support for Python 3.9 has been dropped. Python 3.10+ is now required.
- pikepdf >= 8.8.0 is now required.

---

### Releasenotes Version17 (`docs/releasenotes/version17.md`)

% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0

# v17

## v17.13.1

**Changes**

- OCRmyPDF now requires `pikepdf[pdfa]` 10.17 or later, and uses its new
  `content_scopes()` to find the fonts and images a PDF uses.

**Fixes**

- Fixed PDF/A conversion appearing to hang on a file whose form XObjects
  share one `/Resources` dictionary that lists all of them, a common layout.
  The search for fonts that are not embedded followed every path through the
  forms, which takes exponential time. The repair for JPEGs damaged by
  Ghostscript 10.6 had the same defect. {issue}`1763`
- Fixed a `RecursionError` when examining a page with a form XObject that
  draws itself. A page that draws form XObjects an exponential number of times
  no longer hangs.
- Fonts and images in form XObjects nested more than ten levels deep are no
  longer missed, and fonts used only by tiling patterns, Type 3 glyphs or soft
  masks are now checked for embedding.

## v17.13.0

**Changes**

- PDF/A made without Ghostscript ("speculative" conversion) is now validated
  with pikepdf's PDF/A support (`pikepdf.pdfa`) instead of veraPDF, and the
  final output is validated again after the metadata and optimization steps.
  veraPDF is no longer used, so the Ghostscript-free path is available
  whether or not veraPDF is installed, including in the Docker image. Files
  the validator does not approve go to Ghostscript as before; run with `-v1`
  to see why.
- Speculative conversion now supports PDF/A-1b, and repairs common problems
  that previously always sent a file to Ghostscript: it replaces other output
  intents with sRGB, removes image interpolation flags, sets the Print flag on
  annotations, adds the `/CIDSet` PDF/A-1 requires, and removes XMP properties
  and hidden annotations that PDF/A does not permit.
- New option `--pdfa-backend {auto,ghostscript,internal}` (API:
  `pdfa_backend=`). `auto`, the default, tries OCRmyPDF's own conversion and
  then Ghostscript. `internal` never uses Ghostscript, so JPEGs pass through
  unchanged; if the result is not approved, the `pdfa` output types fail with
  exit code 10 and `--output-type auto` outputs a regular PDF. `ghostscript`
  always uses Ghostscript. Options that only Ghostscript implements
  (`--pdfa-image-compression`, `--ghostscript-jpeg-quality`,
  `--ghostscript-jpeg-maxdpi`, and `--color-conversion-strategy` `CMYK`,
  `Gray` or `UseDeviceIndependentColor`) now select Ghostscript under `auto`
  and are an error with `internal`.
- OCRmyPDF now requires `pikepdf[pdfa]` 10.15 or later. The `pdfa` extra
  brings in `jsonschema`, `referencing` and `fonttools`, all packaged by
  Debian and Red Hat.
- `--force-ocr` now keeps hyperlinks, moving link annotations onto the
  rasterized page; so do `--deskew` and `--clean-final`. The new mode
  `--mode force-ocr-no-links` keeps the old behaviour. {issue}`605`
- When Ghostscript substitutes fonts that are not embedded, it now always uses
  its own fonts (`-dNONATIVEFONTMAP`), so output is the same on every
  platform; on macOS, native font lookup could embed tens of megabytes of
  system fonts. OCRmyPDF warns when fonts other than the standard 14 are
  substituted, or when a substitute loses bold or italic styling.
  {issue}`1369`
- A creation date without a time zone is now taken to be local time, and the
  output's dates get that zone, with a warning. Set `TZ` to choose another
  zone.
- In the default mode, a page that already has text now stops OCRmyPDF
  straight after the initial scan, instead of after OCRing the pages before
  it. The error names the page. {issue}`613`
- The check of the output file is much faster: each stream is checked in the
  cheapest way for its compression instead of being fully decoded, and a
  progress bar is shown. JPEG 2000 and CCITT images are now checked too, but
  JPEGs that are corrupt without being truncated are no longer detected.
  {issue}`1570`
- OCRmyPDF now ships a PyInstaller hook, so apps that use it can be bundled
  without extra options. Tesseract and Ghostscript must still be installed.
  {issue}`1024`
- New `PageInfo.has_visible_text`, which excludes invisible text such as an
  OCR layer. `has_text` is unchanged.

**Fixes**

- `--force-ocr` on a scan that already had an invisible OCR layer rasterized
  it at no less than 400 dpi, as if the text were visible, which could
  multiply the file size. {issue}`961`
- Process workers (`use_threads=False` on Windows and macOS) failed on every
  page with `'OcrOptions' object has no attribute 'tesseract'`.
  {issue}`1757`
- `ocrmypdf.ocr()` rejected thresholding method names such as
  `tesseract_thresholding='adaptive-otsu'`. {issue}`1460`
- `--rotate-pages --tesseract-timeout 0` detected page orientation but did not
  rotate the pages. The cookbook now recommends this combination for rotating
  or deskewing without OCR. {issue}`778`
- With `--output-type auto` and `--force-ocr`, output was labelled PDF/A
  without being validated or given the PDF/A declarations when veraPDF was
  not installed, as in the Docker image. {issue}`1751`
- Ghostscript's PDF/A conversion deleted most hyperlinks (those without the
  Print flag). Hidden annotations, which PDF/A does not permit, are now
  removed before Ghostscript runs, with the same warning as speculative
  conversion, instead of silently. {issue}`605`
- Text in fonts without `/ToUnicode`, which viewers extract through glyph
  names, was garbled or lost when Ghostscript made the PDF/A. {issue}`1297`
- XMP metadata with no document-info equivalent, such as `dc:contributor`
  and `dc:subject`, was lost when Ghostscript made the PDF/A. It is now
  copied with its whole value, including every language of a multilingual
  property. {issue}`1220`
- Images in PDFs from iText and pdftk were recompressed without their Flate
  predictor, growing the output by about 30%. {issue}`1620`
- Copied text from vertical Japanese and other vertical or rotated OCR lines
  had spurious spaces and characters out of order. {issue}`1244`
- Text set in the glyphless fallback font, used when no installed font
  covers a script (e.g. CJK), lost characters when selected or copied in
  Chrome and other pdfium-based viewers.
- A page made of full-page images at different resolutions was rasterized at
  a weighted average resolution rather than the highest. {issue}`948`
- The document language (`/Lang`) was not set for Tesseract's language codes
  such as `deu`, `fra` and `ces`. Languages without a two-letter code now get
  their three-letter code. {issue}`1749`
- Some LaTeX PDFs with malformed numbers in a content stream, such as
  `0.000-50131235`, stopped OCRmyPDF. It now warns and continues, as viewers
  do. {issue}`1054`
- A `/Rotate` stored as a real number, or a `cm` operator with a non-numeric
  operand, crashed the page scan.
- `--force-ocr --ocr-engine none` crashed on pages with no images.
- When OCRmyPDF's own PDF/A failed its final check, the Ghostscript fallback
  could fail with `FileExistsError`; on Windows it always did.
- With the default `--output-type auto`, the explanation for a file that grew
  now points to PDF/A conversion. {issue}`1369`
- Ghostscript 10.08's title `'Untitled'` (with quotes) for untitled files is
  removed again.
- The Docker image's web service could not find its Streamlit script. The
  Docker documentation for the web service is updated. {issue}`1753`
- The Docker watcher no longer turns off `deskew` given in
  `OCR_JSON_SETTINGS` when `OCR_DESKEW` is not set.

## v17.12.1

**Fixes**

- `ocrmypdf --version` and `ocrmypdf --help` printed to standard error
  instead of standard output, so `version=$(ocrmypdf --version)` came back
  empty and `ocrmypdf --help | less` showed nothing. OCRmyPDF redirects file
  descriptor 1 to standard error early in startup, before it parses the
  command line, so that a stray write from a plugin or a library cannot
  corrupt a PDF being written to standard output; argparse's own output was
  caught by that redirection too and is now routed back to the real standard
  output. Error messages continue to go to standard error. Broken since
  v17.8.0.

## v17.12.0

**Changes**

- OCRmyPDF now requires pikepdf 10.2 or later, up from pikepdf 10. This is the
  first release that provides `pikepdf.NamePath`, the `Object.as_int()` family
  of type-safe accessors, and a thread-local `explicit_conversion()`. OCRmyPDF
  uses all three to read optional values out of PDFs it did not write.
- HEIF/HEIC input support is now an opt-in extra, `ocrmypdf[heic]`, instead of
  a default dependency. The `pi-heif` package we previously depended on is
  discontinued; its successor `pillow-heif` bundles libheif, libde265 and x265
  in its wheels, making the wheel as a whole GPLv2-licensed, so we do not
  install it by default. Install `ocrmypdf[heic]` if you feed HEIC images to
  OCRmyPDF. The Docker image and snap do not include it. {issue}`1746`
- A Tagged PDF that is refused because it appears to contain real text now
  exits with code 6 (`already_done_ocr`) instead of code 2 (`input_file`),
  matching the exit code for a PDF with prior OCR. Scripts that check for exit
  code 2 on tagged PDFs need updating. Thanks @BetterAndBetterII.
  {issue}`1551`
- The snap package is now built and published from our CI rather than
  Launchpad, so the `stable` channel tracks releases again instead of being
  frozen at v16.4.2. It is based on core26 and includes a JBIG2 encoder.

**Fixes**

- Text copied or searched in a `--output-type pdfa` file came out as garbage
  (a stable substitution of unrelated characters, or nothing at all) when the
  PDF/A conversion ran through Ghostscript 9.56 through 10.04.x, which
  includes the Ghostscript in Ubuntu 24.04. fpdf2 writes each font's ToUnicode
  CMap as a single `bfchar` block, but the CMap specification allows at most
  100 entries per block, and those Ghostscript releases discard the whole map
  when a block is larger. Any page with more than 100 distinct glyphs was
  affected, which is nearly every page of prose. CFF CID-keyed fonts such as
  Noto Sans CJK have the same problem in their Encoding CMap, where those
  Ghostscript releases then also draw the wrong glyphs. OCRmyPDF now splits
  every CMap block to 100 entries as fpdf2 writes it, for every fpdf2 release.
  fpdf2 2.8.9 will split the ToUnicode blocks itself but not the Encoding
  blocks, so the fix stays in place. Thanks @olafure for the analysis
  ({issue}`1747`, fpdf2 issue py-pdf/fpdf2#1952).
- Words containing `fi`, `ff`, `fl` and similar pairs extracted with the
  letters missing (`con dentiality`, `e ects`) from `--output-type pdfa` files
  when the PDF/A conversion ran through Ghostscript 10.05.0 through 10.06.x.
  Those Ghostscript releases drop ToUnicode entries that expand to more than
  one character (Ghostscript bug 709030, fixed in 10.07.0), and the fpdf2
  renderer's HarfBuzz shaping had been forming optional Latin ligatures whose
  entries do exactly that. Invisible text in scripts that do not need shaping
  is now encoded one glyph per character, so no such entries exist to lose.
  Complex scripts still get shaped, and OCRmyPDF now warns when an affected
  Ghostscript is in use, since their conjunct mappings can still be dropped.
  Thanks @kmn5 ({issue}`1744`).
- A malformed PDF that stores something other than a dictionary at a
  structural key -- `/Resources`, `/Resources /XObject`, `/Root /AcroForm`,
  `/Root /MarkInfo`, `/Root /Names`, `/Root /PieceInfo`, an annotation's `/A`,
  or an image's `/SMask` -- is now tolerated everywhere rather than in the
  handful of places that had been hardened individually. Such a file is read as
  though the key were absent, which is what the well-guarded paths already did.
- An image XObject with no `/Subtype` no longer raises out of the optimizer.
- `PdfInfo` no longer aborts on a `/MarkInfo << /Marked 1 >>`, where a producer
  wrote a flag as an integer instead of a Boolean. Thanks @linhongyu510
  ({issue}`1742`).
- A page `/UserUnit` written as a PDF Real is now read exactly rather than via
  binary floating point, so the digits the file wrote are the digits used.
- `--sidecar` now refuses any spelling of the input or output file, not just a
  byte-identical one. `ocrmypdf in.pdf out.pdf --sidecar ./in.pdf` was accepted
  and then silently overwrote the input PDF with the OCR text. Paths are now
  compared after resolving `.`, `..`, symlinks and filesystem case. Thanks
  @linhongyu510.
- `--deskew` and `--rotate-pages` now warn that they will have no effect when
  combined with `--ocr-engine none`, since skew and orientation are measured by
  the OCR engine. Other image processing options are unaffected. Thanks
  @Anai-Guo. {issue}`1735`
- Installation documentation refreshed for current platforms: Ubuntu 26.04 and
  Fedora 43/44 added, Ubuntu 20.04 dropped, and `uv tool install` shown in
  place of pip.
- Documentation: corrected the list of platforms that ship a JBIG2 encoder,
  which now includes Debian and Ubuntu. Thanks @lcorbasson.
- Release process and CI improvements.

## v17.11.0

**Enhancements**

- New `--max-ocr-image-mpixels` downsamples the image sent to OCR when a page
  exceeds the given size, which bounds the largest consumer of memory. The
  visible page is never downsampled, so output appearance is unaffected in every
  mode; what it trades is OCR accuracy on very high resolution scans. A 34
  megapixel page that peaks at 492 MB peaks at 325 MB under
  `--max-ocr-image-mpixels 8`, and recognizes the same text. See the "Memory"
  section of the performance documentation for how to size a memory limit.
- `watcher.py` (the `watcher` extra) gained a configurable output layout and
  conflict policy. These are watcher-only changes; they do not affect the
  `ocrmypdf` library API.
    - New `OCR_OUTPUT_STRUCTURE` setting (`--output-structure`): `FLAT`
      (default, all outputs directly in the destination directory),
      `YEAR_MONTH` (`{destination}/{year}/{month}/{filename}`, same layout as
      the old `OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1`), or `HIERARCHY`, which
      mirrors the input directory tree under the destination, e.g.
      `input/a/b/c.pdf` → `output/a/b/c.pdf`.
    - New `OCR_ON_CONFLICT` setting (`--on-conflict`) controls what happens
      when the intended destination file already exists: `SUFFIX` (default)
      writes `name (1).pdf`, `name (2).pdf`, ... in the OS style; `SKIP` logs
      and leaves the file unprocessed; `OVERWRITE` is the old behavior.
      **Behavior change: the default is now `SUFFIX`, so existing output
      files are no longer silently overwritten.**
    - Both settings now apply equally to `OCR_OUTPUT_DIRECTORY` and to the
      archive directory used by `OCR_ON_SUCCESS_ARCHIVE`; previously the
      archive directory was always flat and silently overwrote on a name
      collision.
    - Output and archive filenames and directory components are sanitized
      for filesystems more restrictive than the input side (e.g. an SMB
      share): characters illegal on Windows/SMB (`<>:"/\|?*` and control
      characters) are replaced with `_`, trailing dots/spaces are stripped,
      and reserved DOS device names (`CON`, `PRN`, `AUX`, `NUL`, `COM1`-`9`,
      `LPT1`-`9`) are prefixed with `_`.
    - `OCR_OUTPUT_DIRECTORY_YEAR_MONTH` is now deprecated in favor of
      `OCR_OUTPUT_STRUCTURE=YEAR_MONTH`. It is still honored and logs a
      deprecation warning; if both are set, `OCR_OUTPUT_STRUCTURE` wins.
    - See the "Watched folders with watcher.py" section of the batch
      processing documentation for the full description, including a note
      for SMB users about client-side directory/file-info caching delays.

**Performance**

- The image sent to OCR is no longer decoded and re-encoded when nothing needs
  to change it. On a page with no pre-existing text to mask and no filtering
  plugin -- the ordinary case for a scanned document -- the rasterized page
  already *is* the OCR image, so it is linked rather than rewritten. Producing
  it was costing 3.3 seconds and a full-size buffer on a 34 megapixel page, out
  of about 15 seconds for the whole file. Building the mask is now deferred
  until a text area actually needs blanking, since that step is what forced the
  decode. The minimum Pillow version is raised to 12, because deciding whether
  anything decoded the image reads an attribute whose shape settled in Pillow 11
  (`pi-heif` already required Pillow 11.1, so the effective floor barely moves).
- Reduced peak memory on files with large images by about a third at default
  settings. On a 34 megapixel page the process tree peaked at 752 MB and now
  peaks at 492 MB. Two changes account for it: rasterizing a page no longer
  allocates a third full-page buffer to correct PDFium's rounding of the
  rendered size by a pixel or two, and freed heap memory is now returned to the
  operating system before the OCR engine runs, instead of counting against our
  resident set for as long as the engine is working. Rasterization is also
  faster, since correcting the size no longer resamples the whole page.
- Several OCR jobs may now run concurrently in a single Python process. The API
  previously held a lock for the whole duration of `ocrmypdf.ocr()`, so a second
  call in another thread had to wait for the first to finish. Plugin state is
  now guarded by a readers-writer lock: installing a plugin set takes it
  exclusively, and a job holds it shared for its run. An in-flight job therefore
  cannot have the plugin infrastructure it depends on replaced underneath it,
  while jobs that are past installation proceed concurrently.
- A plugin set is now installed once per interpreter and reused, rather than
  being reinstalled on every call. Previously each `ocrmypdf.ocr()` call
  re-executed plugin modules given as file paths and rebound them in
  `sys.modules`. Plugins must not rely on being re-executed for each job, and
  must not store per-job state on the plugin manager, which concurrent jobs
  requesting the same plugin set now share.
- Jobs requesting different plugin sets serialize against each other, since
  installing the second set must wait for the first set's jobs to finish. Use
  the same plugin set across concurrent jobs, or separate processes.
- Image optimization is substantially faster on documents with large images.
  An image stored as `/FlateDecode` with a PNG predictor already holds exactly
  what a PNG `IDAT` chunk holds, so it is now repackaged as a PNG directly
  instead of being decoded to a bitmap and re-encoded. On a 6-page document
  containing one 9000x9000 image, the optimization step went from 3.4s to 0.6s,
  and to 0.04s together with the JPEG change below; total runtime went from
  10.0s to 7.0s. Output is unchanged: the compressed
  data is reused verbatim. Images that are not in a directly repackageable form
  still take the previous path.
- During image optimization, we decoded all JPEGs, even if the code
  path was an optimization setting with the decoded JPEG would be never be
  re-encoded (below `--optimize 2`). We now decode only on code paths that
  use the decoded JPEG. Output is unchanged.
- An uncompressed image (one with no `/Filter`) no longer produces a spurious
  "could not be processed by the optimizer" warning. Such an image raised
  `IndexError` internally, which the optimizer's best-effort handler caught and
  reported as a warning; it is now recognized and skipped quietly.
- Removed an unreachable branch in the image optimizer that claimed to handle
  1 bit per component images in an ICC-based colorspace. An earlier check sends
  every 1 bpc image to the JBIG2 pass, which handles ICC-based images by
  neutralizing the profile before extracting, so the branch could never run.
- Removed the process-wide lock that serialized worker pools across all
  `Executor` instances. `Executor.pool_lock` is retained but no longer acquired,
  and is deprecated; it will be removed in a future major release. The invariant
  it protected - that only one progress bar renders on the shared console - is
  now enforced by the progress bar, which disables itself if another bar already
  owns the console.
- Note that N concurrent jobs each configured with `jobs=M` may now spawn up to
  N*M workers, where previously they were serialized to M. Size `jobs`
  accordingly.
- Known limitation of concurrent in-process jobs: they must use the same
  `max_image_mpixels`. Pillow's decompression-bomb limit is interpreter-global
  and the last job to set it wins. Use separate processes to run jobs with
  differing configurations.

**Fixes**

- Fixed a latent use-after-free in the pypdfium2 rasterizer. `to_pil()` lets
  Pillow alias PDFium's bitmap buffer for some formats -- grayscale renders
  among them, which is every mono and grayscale page -- and the buffer was freed
  immediately afterwards, leaving Pillow reading memory PDFium had released.
- Windows: OCRmyPDF no longer prints `[WinError 2] The system cannot find the
  file specified` warnings while it searches for Ghostscript and Tesseract
  ([#1671](https://github.com/ocrmypdf/OCRmyPDF/discussions/1671)). These
  messages came from probing registry keys that simply don't exist when the
  programs were installed by a package manager such as Scoop, or not installed
  at all. Since the search then continues elsewhere and usually succeeds, these
  failures are normal, and are now logged at debug level, naming the location
  that was searched. If a program genuinely cannot be found, OCRmyPDF still
  reports that as an error.
- Windows: fixed a crash when the `PROGRAMFILES` environment variable pointed to
  a folder that does not exist.
- `--mode strip` failed to remove OCR text layers that OCRmyPDF itself
  produced ({issue}`1730`). OCRmyPDF grafts its text layer as a Form XObject
  and stripping only examined the page content stream, so the invisible text
  was never found. The same flaw made `--mode redo` stack a second text layer
  on top of the old one instead of replacing it. Stripping now descends into
  Form XObjects. Thanks @Anai-Guo ({issue}`1732`).
- Stripping now also resolves page `/Resources` inherited from an ancestor
  `/Pages` node, rather than only looking at the page's own resources.
- A text layer that becomes empty after stripping is now removed from the page
  instead of being left behind as a vestigial Form XObject husk.
- Setting `clean_final` on an existing options object in the Python API no
  longer leaves `clean` unset. `--clean-final` implies `--clean`, but the rule
  lived in a field validator that only ran while the options object was being
  constructed, so assigning to the attribute afterwards silently skipped it.
- Validation errors for out-of-range or misspelled options now name the option
  they are about, and their wording comes from Pydantic rather than being
  hand-written, so it has changed slightly. For example, `--jobs 999` now
  reports `--jobs: Input should be less than or equal to 256`. The set of
  accepted values is unchanged.

## v17.10.0

- The `watcher.py` watched-folder helper (the `watcher` extra) has been
  modernized and security-hardened:
    - It now uses `watchfiles` instead of `watchdog`. Installing
      `ocrmypdf[watcher]` now pulls in `watchfiles`; native OS filesystem
      notifications are used by default, with `OCR_USE_POLLING=1` to force
      polling.
    - It enforces a "Harvard architecture" separation between data and code:
      at startup it refuses to run (exit code 9) if the input, output or
      archive directory overlaps any Python interpreter path (`sys.path`,
      the virtual environment, site-packages, or `$PATH`), if
      `OCR_JSON_SETTINGS` points at a file inside a data directory or one that
      is group/world-writable, or if it specifies a plugin located inside a
      data directory. It also refuses to run when the output or archive
      directory is the input directory or a subdirectory of it, which would
      otherwise cause OCRmyPDF output to be reprocessed in an endless loop.
    - At runtime it no longer follows symlinks or processes non-regular files
      (fifos, devices, etc.) in the watched directory, and refuses to write
      output onto a destination occupied by a non-regular file.
    - A password-protected PDF dropped into the watched folder no longer stops
      the watcher ({issue}`1715`). `pikepdf.PasswordError` does not derive from
      `pikepdf.PdfError`, so it escaped the handler that waits for a file to be
      fully written and tore down the watch loop, leaving files that arrived
      afterwards unprocessed. Encrypted files are now logged and skipped
      immediately — no amount of retrying will supply the password. More
      generally, no per-file error can stop the watcher now: failures are
      logged and watching continues. Thanks @christophdb for the report and a
      fix ({issue}`1716`).

  See the "Watcher security model" section of the batch processing
  documentation for details. Existing
  deployments where the data directories are kept separate from the application
  are unaffected; deployments that co-located data with the interpreter or its
  environment will need to relocate one or the other.
- Ghostscript 10.7.0 and later are no longer treated as affected by the JPEG
  passthrough truncation bug, which Ghostscript fixed in 10.07.0
  ({issue}`1726`). The version check had no upper bound, so users on a fixed
  Ghostscript still saw the "JPEG encoding errors" warning and, worse, silently
  had every JPEG lossily re-encoded at `--optimize 1` (the default) to work
  around a bug their Ghostscript did not have. Thanks @zuentec-droid for the
  detailed measurements and upstream analysis.
- The same JPEG re-encoding workaround no longer applies when Ghostscript did
  not produce the file at all. It was previously triggered by the mere presence
  of an affected Ghostscript, so `--output-type pdf` and files converted by the
  speculative PDF/A path — neither of which runs Ghostscript — paid the quality
  loss for nothing.

## v17.9.0

- OCRmyPDF now uses any Noto font installed on the system, not just the two
  dozen script families it knows by name ({issue}`1722`). Previously a document
  in, say, Cherokee or Vai was rendered with the glyphless fallback font even
  though the matching font was installed — a common situation on macOS, which
  ships around a hundred script-specific Noto faces. When the named fonts
  cannot cover a word, OCRmyPDF now searches the installed fonts for one that
  can.
- The "no installed font has glyphs" warning now names the characters it could
  not render, with their codepoints and Unicode names, so it is clear which
  font to install. Text that mixes scripts no single font covers is now
  reported as such, instead of advising the user to install fonts they may
  already have.
- Fixed the macOS font installation instructions, which recommended a Homebrew
  package (`font-noto`) that does not exist ({issue}`1722`). Homebrew has no
  single Noto package; each family is a separate cask. The Fedora package name
  was also corrected to `google-noto-fonts-all`.
- Font providers may now implement the optional `GlyphSearchingFontProvider`
  protocol to participate in coverage-based font search.
- Fixed `--jpeg-quality`/`--jpg-quality` having no effect on the CLI: the
  value was silently dropped before reaching the optimizer, which then
  always used its own built-in default JPEG quality regardless of what was
  requested ({issue}`1723`). The same bug affected the Python API's
  `jpg_quality` parameter. `ocrmypdf.ocr()` now accepts `jpeg_quality`
  (matching the CLI flag name) as the canonical parameter; `jpg_quality`
  still works but is deprecated.
- Hardened PDF parsing against malformed (non-dictionary) `/Resources`,
  `/XObject`, and `/FontDescriptor` entries, which previously crashed
  `ocrmypdf.ocr()` with `AttributeError`/`TypeError`/`ValueError` on
  otherwise-processable files, both during PDF/A font scanning and general
  image scanning ({issue}`1713`). Thanks @mvanhorn for the initial fix.
- Release process improvements: fixed a CI bug where every push to main
  after a release was tagged would incorrectly revert the just-published
  GitHub release back to draft status.

## v17.8.1

- Improved the `--tesseract-pagesegmode` help text to point to
  `tesseract --help-extra`, since Tesseract 5.5.2 moved the page segmentation
  mode documentation there from `tesseract --help`. Thanks @sokai.
- Internal refactoring: completed a project-wide mypy type-checking pass
  (`--check-untyped-defs` is now enabled, and the mypy pre-commit hook is now
  blocking rather than advisory), fixing several latent edge-case bugs
  surfaced along the way.
- Release process improvements: migrated from pre-commit to prek for local
  git hooks, and added a dedicated lint job to CI.
- Improved typing strictness for `Path`.

## v17.8.0

- `--output-type auto` (the default) again produces PDF/A whenever it can,
  matching OCRmyPDF 16's "PDF/A by default" behavior. It first tries the fast
  Ghostscript-free conversion (validated by veraPDF when available) and now
  falls back to Ghostscript when that cannot produce PDF/A, only emitting a
  regular PDF when even Ghostscript cannot safely convert (for example, an
  input with non-embedded CID/CJK fonts, per {issue}`1561`). A consequence is
  that the default path may once again invoke Ghostscript, which is slower and
  may transcode images; use `--output-type pdf` to skip PDF/A conversion
  entirely.
- Fixed detection of veraPDF 1.30.0 and newer: recent builds print JVM
  warnings before their version string, which caused OCRmyPDF to report
  veraPDF as unavailable and skip the fast PDF/A path.
- OCRmyPDF no longer silently corrupts a non-embedded CID (CJK) text layer when
  producing PDF/A ({issue}`1561`). PDF/A requires all fonts to be embedded, so
  Ghostscript substitutes and re-embeds non-embedded CID fonts — such as the OCR
  text layer Adobe Acrobat adds to scanned CJK documents — which mangles the
  text and destroys searchability. OCRmyPDF now detects non-embedded CID fonts
  before conversion: with `--output-type auto` (the default) it produces a
  regular PDF and preserves the existing text layer, and with an explicit
  `--output-type pdfa*` it stops with an error rather than emit corrupted
  output. Use `--output-type pdf` to keep the text layer, or `--force-ocr` to
  rebuild it with embedded fonts.
- Writing the output PDF to standard output (`ocrmypdf input.pdf -`) is now
  protected against corruption at the operating system level. Previously
  OCRmyPDF relied on no in-process code — third-party libraries, plugins, or
  stray `print()` calls — ever writing to stdout; a single accidental write
  would silently corrupt the PDF. The command line program now saves the real
  stdout at startup, before plugins are loaded or any worker process/thread is
  started, and redirects file descriptor 1 to stderr, so that only OCRmyPDF's
  final PDF output can reach stdout. A consequence is that a plugin which
  intentionally prints to stdout will have that output redirected to stderr.
- Added the public API function {func}`ocrmypdf.configure_stdout_protection`,
  which installs this same protection. Like {func}`ocrmypdf.configure_logging`,
  it is optional and intended for callers that want command-line-like behavior;
  applications that manage their own standard output should not call it.
- Fixed an uncaught `UnicodeDecodeError` when processing a PDF whose
  `/DocumentInfo` dictionary contains a `/Name` key encoded in Latin-1 (or
  another non-UTF-8 encoding), such as `/Saks#e5r`. `repair_docinfo_nuls` now
  treats such a block as malformed, logs a message, and continues instead of
  crashing the pipeline ({issue}`1540`). Current pikepdf releases tolerate these
  keys by surrogate-escaping them, but older versions raised while iterating the
  dictionary.

## v17.7.1

- Fixed a severe, Windows-specific performance regression in the "Scanning
  contents" phase, most visible with `--redo-ocr` ({issue}`1662`). Since
  v16.4.3, OCRmyPDF forced pdfminer's read buffer to 256 MiB to work around a
  pdfminer bug that mishandled tokens split across the buffer boundary
  ({issue}`1361`). On Windows, CPython's `BufferedReader.read()` eagerly
  allocates a buffer of the requested size on every read, so the oversized
  buffer made each of pdfminer's thousands of reads cost tens of milliseconds
  (this allocation is lazy, and effectively free, on Linux). The underlying
  pdfminer bug was fixed upstream in pdfminer.six 20250327
  ([#1030](https://github.com/pdfminer/pdfminer.six/pull/1030)), with a
  follow-up for tokens split across streams in 20260107
  ([#1158](https://github.com/pdfminer/pdfminer.six/pull/1158)), so the
  workaround has been removed and the minimum pdfminer.six version raised to
  20260107.
- The font discovery used to build the OCR text layer now finds variable fonts
  such as `NotoSansArabic[wdth,wght].ttf`, the form shipped by Homebrew casks
  and current Google Fonts releases. Previously only static `-Regular.ttf`/`.otf`
  files were matched, so users who had installed the correct Noto font still got
  the glyphless fallback and a "No font found" warning ({issue}`1652`).
- Font discovery is now language-aware for CJK: each Chinese, Japanese, and
  Korean language maps to its own per-language Noto family (NotoSansSC, TC, HK,
  JP, KR), with the pan-CJK super font kept as a shared fallback, since the
  per-language fonts are region subsets that may lack glyphs from other scripts.
- The warning shown when no installed font has glyphs for some text was reworded
  to explain the consequence — the text is still added as a searchable, copyable
  layer but appears blank when highlighted in a viewer — and to name the specific
  font family to install.

## v17.7.0

- The Docker images now run as a non-root user (`app`, uid/gid 1000) by default
  rather than as root, as a defense-in-depth measure. If you bind-mount a
  directory for input and output, you may now need to add a `--user` argument so
  the container can write to it; the correct value differs for rootless Docker,
  Podman, and rootful Docker, and is described in the Docker documentation.
  Piping the input and output through stdin/stdout still works with no
  permission setup.
- The Docker images now default their working directory to `/data`, so files in
  a directory mounted there can be given as relative paths without an explicit
  `--workdir`.
- The Ubuntu Docker image now installs Tesseract 5 from the Ubuntu archive
  instead of the third-party `alex-p/tesseract-ocr5` PPA, and the base images
  were updated to Ubuntu 26.04 and Alpine 3.24.
- Fixed a missing space in the error message shown when OCRmyPDF cannot access
  its working directory inside a Docker container.
- Updated packaged dependencies, including the optional web service stack
  (starlette, tornado, python-multipart) and cryptography.

## v17.6.0

- When the optimizer encounters an image it cannot process (for example, an
  exotic colorspace that cannot be transcoded), it now logs a concise warning
  that the image was left unchanged rather than printing an alarming
  traceback. The output file was already valid in these cases; only the
  reporting was misleading. The full traceback is still available at debug
  verbosity (`-v 1`) ({issue}`846`).
- `--pdfa-image-compression=auto` (the default) now selects lossless image
  compression at `-O0` so Ghostscript no longer transcodes lossless images to
  JPEG during PDF/A generation. At `-O1` and above, `auto` continues to defer
  to Ghostscript's heuristic, which may recompress images lossily. `-O1` (the
  default level) is kept as a historical exception because coercing it to
  lossless can substantially bloat output; users who want guaranteed lossless
  image handling should pass `--pdfa-image-compression=lossless` or use `-O0`
  ({issue}`1124`).
- `--pdfa-image-compression=lossless` now passes existing JPEG images through
  unchanged rather than re-encoding them with a lossless codec. Re-encoding an
  already-lossy JPEG losslessly cannot recover quality and only inflates the
  file, so JPEGs are preserved while non-JPEG images are encoded losslessly.
- OCRmyPDF now validates and repairs malformed page-boundary boxes
  (``/MediaBox``, ``/CropBox``, ``/TrimBox``, ``/ArtBox``, ``/BleedBox``) in its
  input, following the PDF 2.0 specification. Coordinates written in invalid
  exponential notation are reinterpreted ({issue}`1398`); rectangles whose
  corners are given in reversed order are normalized, which previously crashed
  with ``NegativeDimensionError`` ({issue}`1526`); and a crop/trim/art/bleed box
  that falls outside the MediaBox is clamped to their intersection, or discarded
  when that intersection is empty, which previously produced an output with a
  zero-height effective page that some viewers refused to open ({issue}`1400`).
  When a box is discarded, clamped, or reinterpreted, OCRmyPDF logs a warning
  recommending visual inspection of the output. Thanks @ajdlinux for the initial
  fix in PR #1691.
- OCRmyPDF now discards an embedded Adobe full-text search index
  (``/Root/PieceInfo/SearchIndex``) from its output. This proprietary index,
  produced by Acrobat's "Embed Index" feature, is read only by Adobe Acrobat;
  other viewers ignore it and search the text on the fly. Because any change to
  a PDF invalidates the index, retaining it after OCRmyPDF rewrites the document
  would leave a stale index that returns incorrect search results in Acrobat.
  Modern viewers rebuild a search index on demand, so there is no loss of
  search capability.
- OCRmyPDF now discards embedded per-page thumbnail images (the optional
  ``/Thumb`` image XObject on a page) from its output. OCRmyPDF alters page
  appearance (deskew, clean, rasterize, re-render) and plugins may edit pages
  arbitrarily, so a retained thumbnail would be stale and no longer match its
  page. Embedded thumbnails are a navigation aid that modern viewers generate
  on demand, so there is no loss of functionality.
- Fixed a regression in OCR quality for PDFs that paint a 1-bit image mask
  (stencil) with a gray or colored fill color. Previously such pages were
  rasterized as 1-bit black-and-white before OCR, so Ghostscript dithered
  mid-tone text into an unreadable stipple and Tesseract failed to recognize
  it. The rasterizer now inspects the fill color used to paint a mask and
  promotes the page to grayscale or full color as needed, so the distinction
  is preserved for the OCR engine. This applies to both the Ghostscript and
  pypdfium rasterizers. {issue}`1688`
- The default 1-bit raster device for Ghostscript is now ``pngmonod``
  (error-diffusion) instead of ``pngmono`` (ordered dithering). It produces
  better input for OCR on faint or anti-aliased scans at negligible cost and
  no change to output file size, since the rasterized image is an
  intermediate that is discarded after OCR.
- When rasterizing pages with Ghostscript, OCRmyPDF now enables text and
  graphics anti-aliasing (``-dTextAlphaBits=4 -dGraphicsAlphaBits=4``) for the
  grayscale and color raster devices. Ghostscript 10.x renders aliased glyphs
  that OCR frequently misreads as extra word breaks or substituted characters;
  anti-aliasing materially improves OCR accuracy on the Ghostscript
  rasterization path, especially for small fonts at moderate resolution. The
  1-bit monochrome devices are unaffected, since they perform their own
  anti-aliased downscaling and older Ghostscript versions reject alpha-bit
  options on them. Note that the default rasterizer (``--rasterizer auto``)
  prefers pypdfium2, which already anti-aliases; this change benefits users who
  select ``--rasterizer ghostscript`` or do not have pypdfium2 installed.
  OCRmyPDF now also logs which rasterizer rendered each page at debug verbosity
  (``-v 1``), and the ``--rasterizer`` help text explains the OCR-quality
  trade-off, to make such reports easier to diagnose. {issue}`1439`
- When Tesseract reports a page with many diacritics, OCRmyPDF still logs its
  interpreted "lots of diacritics - possibly poor OCR" hint, but now also emits
  Tesseract's raw message at debug verbosity (``-v 1``) so the original wording
  is available for diagnosis. {issue}`1566`
- Added ``--mode strip``, which removes the invisible OCR text layer from a PDF
  in place. Unlike ``--ocr-engine none --force-ocr``, it does not rasterize the
  page, so images and visible content are preserved unchanged and the output is
  smaller rather than larger. Only text drawn as invisible (PDF text render mode
  3) is removed; some OCR engines -- and OCRmyPDF v2.2 and earlier -- express
  text as visible glyphs covered by an opaque image, and that text cannot be
  removed this way. {issue}`1435`

## v17.5.0

- Added support for the ``end`` alias in ``--pages``, denoting the last page
  of the document. For example, ``--pages 3-end`` OCRs from page 3 through
  the final page. {issue}`1615`
- Added ``--ghostscript-jpeg-quality`` and ``--ghostscript-jpeg-maxdpi``
  advanced options for tuning Ghostscript's PDF/A output. The optimizer's
  ``--jpeg-quality`` remains the recommended file-size control.
- Fixed pypdfium2 rasterizer clipping content when the CropBox was smaller
  than the MediaBox (e.g. JSTOR or cropped PDFs). {issue}`1685`
- Fixed Form XObject cycle detection in the optimizer's image xref scan.
  Self-referential or DAG-shaped Form graphs (notably from PowerPoint
  exports) previously produced floods of recursion warnings and could hang
  for minutes. {issue}`1321`
- Tesseract config errors are now surfaced as ``TesseractConfigError`` with
  actionable guidance, instead of crashing later with a confusing
  ``FileNotFoundError`` on the missing hOCR output. {issue}`1687`
- Refreshed the Chinese README translation. Thanks @cislunarspace.
- Internal refactoring of the ``_exec`` and ``subprocess`` modules to
  separate probing from execution.
- CI dependency updates.

## v17.4.2

- Fixed Python API unconditionally overriding ``PIL.Image.MAX_IMAGE_PIXELS``
  when the caller did not explicitly set ``max_image_mpixels``. Host
  applications (e.g. Paperless-NGX) that configure the PIL limit before
  invoking ``ocrmypdf.ocr()`` now have their setting respected. The CLI
  default of 250 megapixels is unchanged. {issue}`1665`
- Updated uv.lock to avoid pinning a vulnerable version of Pillow. {issue}`1666`

## v17.4.1

- Fixed RTL text extraction order in the fpdf2 renderer. Arabic lam-alef
  ligatures and other multi-character CMap entries were garbled by the bidi
  algorithm during text extraction. {issue}`1655`
- Fixed ``work_folder`` not being set in ``PdfContext`` options when using
  the Python API. Thanks @bluebox-steven. {issue}`1613`
- Updated Ghostscript JPEG corruption warning to include the detected version
  number, confirming the bug persists in Ghostscript 10.7.0.
- Internal refactoring.
- CI dependency updates.

## v17.4.0

- Added ``--no-overwrite`` / ``-n`` option to prevent overwriting output files.
  If the destination file already exists, OCRmyPDF exits with code 5
  (``OutputFileAccessError``). {issue}`1642`
- Fixed text layer stretching in the fpdf2 renderer for widely-spaced words.
  The horizontal scaling (Tz) was incorrectly stretched to fill inter-word gaps
  instead of relying on Td positioning, causing text selection to highlight far
  beyond the actual word boundaries. {issue}`1635`
- Fixed ``optimize=2`` or ``optimize=3`` crash when using the Python API without
  explicitly setting ``jpg_quality`` or ``png_quality``. {issue}`1641`
- Fixed ``verapdf`` availability check crashing with ``NotADirectoryError`` on
  some platforms. {issue}`1638`

## v17.3.0

- Fixed Python API ignoring the ``language`` parameter, always defaulting to
  ``eng``. The API now correctly maps ``language`` to OcrOptions ``languages``
  and splits ``+``-separated codes (e.g. ``eng+deu``) to match CLI behavior.
  {issue}`1640`
- Fixed Python API producing empty OCR output because ``tesseract_timeout``
  defaulted to 0, causing Tesseract to time out immediately. The default is
  now ``None``, falling back to the plugin's 180-second timeout. {issue}`1636`
- Fixed OCR text layer displacement on PDFs with non-zero MediaBox origins
  (e.g. JSTOR or cropped PDFs). The coordinate transformation matrix is now
  always computed, not skipped when rotation is zero. {issue}`1630`
- Restored image overlay support (``--image``) for the hocrtransform tool,
  enabling sandwich PDF output with the fpdf2 renderer. {issue}`1634`
- Docker: updated Alpine base image to 3.23.
- Documentation restructured into per-major-version release notes files.
- Release process improvements.

## v17.2.0

- Fixed incorrect word spacing in poppler-based PDF viewers and tools (Evince,
  pdftotext, and others) where words on the same line appeared separated by
  double newlines. This works around a poppler bug where Tz (horizontal scaling)
  is not carried across BT/ET boundaries. {issue}`1632`
- Fixed OCR text layer being visible instead of invisible due to incorrect fpdf2
  text rendering mode attribute. This caused OCR text to appear when images were
  removed from the PDF. {issue}`1631`
- Fixed OCR text layer misalignment with non-zero mediabox origins, which
  affected cropped PDFs and JSTOR PDFs generated by iText. The ``--redo-ocr``
  mode would shift text vertically on these files. {issue}`1630`
- Fixed Ghostscript rasterization failure with very low DPI values (below 10).
  OCRmyPDF now renders at a minimum of 10 DPI and resizes the output to match
  the originally requested dimensions. {issue}`1612`

## v17.1.0

- Added `--tagged-pdf-mode` to allow skipping the TaggedPDF error message, if desired.
- Fixed an issue where deflated JPEGs (FlateDecode + DCTDecode) were counted as
  lossless images for the purpose of determining whether to compress to JPEG,
  causing file size inflation with some workflows (`--mode force` in particular).

## v17.0.1

- Fixed output file size inflation when using pypdfium as rasterizer and force-ocr
  mode.

## v17.0.0

**Breaking changes**

- **Plugin interface migration**: Plugin hooks now receive `OcrOptions` objects instead of
  `argparse.Namespace` objects. Most plugins will continue working due to duck-typing
  compatibility, but plugin developers should update their type hints from `Namespace`
  to `OcrOptions`.
- Built-in plugins no longer modify options in-place, improving immutability and
  code clarity.
- **Lossy JBIG2 removed**: The `--jbig2-lossy` and `--jbig2-page-group-size` options have been
  removed due to well-documented risks of character substitution errors. These options are now
  deprecated and will emit warnings if used. Only lossless JBIG2 compression is supported.
- **PDF/A output behavior change**: If neither Ghostscript nor verapdf is installed,
  `--output-type auto` (the new default) will produce a standard PDF instead of PDF/A. This is
  a change from previous versions where Ghostscript was required and PDF/A was always produced.
  This configuration is rare but users should be aware of the change.

**New features**

- **pypdfium2 rasterizer**: Added optional pypdfium2-based PDF rasterization plugin as an
  alternative to Ghostscript for page rendering. Use `--rasterizer pypdfium` to enable
  (requires `pip install pypdfium2`). The default `--rasterizer auto` prefers pypdfium when
  available and falls back to Ghostscript.
- **Pluggable OCR engines**: New `--ocr-engine` option allows selecting OCR engines:
  - `auto` (default): Uses Tesseract
  - `tesseract`: Explicit Tesseract selection
  - `none`: Skip OCR entirely for PDF processing-only workflows

  This prepares the foundation for future third-party OCR engine plugins.
- **Smart PDF/A conversion**: New `--output-type auto` (now the default) produces best-effort
  PDF/A output without requiring Ghostscript when the verapdf validator is available. Falls back
  to traditional Ghostscript conversion when needed.
- **verapdf integration**: Added optional verapdf validation for fast PDF/A conversion. When
  available, OCRmyPDF attempts speculative PDF/A conversion using pikepdf, validates with verapdf,
  and skips Ghostscript if validation passes.
- **Optional Ghostscript**: As a consequence of the changes above, Ghostscript is no longer a required dependency. It is optional.
- **fpdf2 text renderer**: Replaced legacy hOCR text renderer with new fpdf2-based implementation,
  providing better multilingual support and more accurate text positioning.
- **Improved Occulta glyphless font**: The new Occulta font provides better handling of
  zero-width markers and double-width CJK characters for accurate text layer positioning.
- **Expanded multilingual font support**: Added FontProvider infrastructure with language-aware
  font selection for Devanagari (Hindi, Sanskrit, Marathi, Nepali), CJK (Chinese, Japanese,
  Korean), Arabic script, and many other scripts. System font discovery reduces package size.
- **Simplified mode selection**: New `--mode` (`-m`) argument consolidates processing options:
  - `default`: Error if text is found (standard behavior)
  - `force`: Rasterize all content and run OCR (replaces `--force-ocr`)
  - `skip`: Skip pages with existing text (replaces `--skip-text`)
  - `redo`: Re-OCR pages, stripping old text layer (replaces `--redo-ocr`)

  Legacy flags remain as silent aliases for backward compatibility.

**API improvements**

- Centralized validation logic in the `OcrOptions` Pydantic model
- Removed scattered option mutation throughout the codebase
- Better type safety for plugin development
- Simplified plugin option handling
- New `OcrElement`, `OcrClass`, and `BoundingBox` exports for OCR engine plugin developers
- Extended `OcrEngine` ABC with `generate_ocr()` method for direct OCR tree output, eliding the need to translate a modern engine's output to hOCR or directly write to PDF.

**Bug fixes**

- Fixed double-compression of already-deflated JPEGs.
- Fixed tesseract_cache plugin to properly handle cache misses.
- Fixed handling of PDF page boxes (ArtBox, BleedBox) which were not being processed correctly.
- Added thread safety lock to pypdfium plugin for concurrent operations.
- Improved pdfminer.six compatibility with explicit word spacing.

**Documentation**

- Updated cookbook to replace deprecated `--tesseract-timeout 0` with `--ocr-engine none`.
- Added comprehensive plugin documentation for new OCR engine framework.

**Dependency changes**

- Requires: one of `pypdfium2` or `ghostscript` for PDF rasterization (PDF to image)
  - Preferred: both
- Requires: one of `verapdf` or `ghostscript` for PDF/A generation
  - Preferred: both
- Recommended: `pypdfium2` for PDF rasterization (new dependency)
- Recommended: `ghostscript` (used to be Required)
- Recommended: Noto fonts for improved OCR text positioning
- Optional: `verapdf` for fast PDF/A validation (new dependency)
- Requires: `fpdf2` for text layer rendering (new dependency)
- Recommended: replace `typer` with `cyclopts` in misc scripts (new dependency)
- See docs/maintainers.md for details.

**Migration guide for plugin developers**

- Update imports: `from ocrmypdf._options import OcrOptions`
- Update type hints: `def check_options(options: OcrOptions)` instead of `options: Namespace`
- Attribute access remains unchanged: `options.languages`, `options.output_type`, etc.
- Remove any in-place option modifications - compute values at point of use instead
- Most existing plugins will continue working without changes due to duck-typing

---

--- METRICS ---
- Files Extracted: 39
- Estimated Token Budget: ~90579 tokens
- Canonical Reference: https://codewiki.google/github.com/ocrmypdf/OCRmyPDF