Technical Documentation: ocrmypdf/OCRmyPDF
ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference
Overview (`README.md`)
OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched or copy-pasted.
ocrmypdf # it's a scriptable command line program
-l eng+fra # it supports multiple languages
--rotate-pages # it can fix pages that are misrotated
--deskew # it can deskew crooked PDFs!
--title "My PDF" # it can change output metadata
--jobs 4 # it uses multiple cores by default
--output-type pdfa # it produces PDF/A by default
input_scanned.pdf # takes PDF input (or images)
output_searchable.pdf # produces validated PDF outputSee the release notes for details on the latest changes.
Main features
- Generates a searchable PDF/A file from a regular PDF
- Places OCR text accurately below the image to ease copy / paste
- Keeps the exact resolution of the original embedded images
- When possible, inserts OCR information as a "lossless" operation without disrupting any other content
- Optimizes PDF images, often producing files smaller than the input file
- If requested, deskews and/or cleans the image before performing OCR
- Validates input and output files
- Distributes work across all available CPU cores
- Uses Tesseract OCR engine to recognize more than 100 languages
- Keeps your private data private.
- Scales properly to handle files with thousands of pages.
- Battle-tested on millions of PDFs.
For details: please consult the documentation.
Motivation
I searched the web for a free command line tool to OCR PDF files: I found many, but none of them were really satisfying:
- Either they produced PDF files with misplaced text under the image (making copy/paste impossible)
- Or they did not handle accents and multilingual characters
- Or they changed the resolution of the embedded images
- Or they generated ridiculously large PDF files
- Or they crashed when trying to OCR
- Or they did not produce valid PDF files
- On top of that none of them produced PDF/A files (format dedicated for long time storage)
...so I decided to develop my own tool.
Installation
Linux, Windows, macOS and FreeBSD are supported. Docker images are also available, for both x64 and ARM.
| Operating system | Install command |
|---|---|
| Debian, Ubuntu | apt install ocrmypdf |
| Windows Subsystem for Linux | apt install ocrmypdf |
| Fedora | dnf install ocrmypdf |
| macOS (Homebrew) | brew install ocrmypdf |
| macOS (MacPorts) | port install ocrmypdf |
| macOS (nix) | nix-env -i ocrmypdf |
| LinuxBrew | brew install ocrmypdf |
| FreeBSD | pkg install py-ocrmypdf |
| OpenBSD | pkg_add ocrmypdf |
| Ubuntu Snap | snap install ocrmypdf |
For everyone else, see our documentation for installation steps.
Languages
OCRmyPDF uses Tesseract for OCR, and relies on its language packs. For Linux users, you can often find packages that provide language packs:
# Debian/Ubuntu users
apt-cache search tesseract-ocr # Display a list of all Tesseract language packs
apt-get install tesseract-ocr-chi-sim # Example: Install Chinese Simplified language pack
# Arch Linux users
pacman -S tesseract-data-eng tesseract-data-deu # Example: Install the English and German language packs
# OpenBSD users
pkg_info -aQ tesseract # Display a list of all Tesseract language packs
pkg_add tesseract-cym # Example: Install the Welsh language pack
# brew macOS users
brew install tesseract-lang
# Fedora users
dnf search tesseract-langpack # Display a list of all Tesseract language packs
dnf install tesseract-langpack-ita # Example: Install the Italian language packYou can then pass the -l LANG argument to OCRmyPDF to give a hint as to what languages it should search for. Multiple languages can be requested.
OCRmyPDF supports Tesseract 4.1.1+. It will automatically use whichever version it finds first on the PATH environment variable. On Windows, if PATH does not provide a Tesseract binary, we use the highest version number that is installed according to the Windows Registry.
Documentation and support
Once OCRmyPDF is installed, the built-in help which explains the command syntax and options can be accessed via:
ocrmypdf --helpOur documentation is served on Read the Docs.
Please report issues on our GitHub issues page, and follow the issue template for quick response.
Feature demo
# Add an OCR layer and require PDF/A
ocrmypdf --output-type pdfa input.pdf output.pdf
# Convert an image to single page PDF
ocrmypdf input.jpg output.pdf
# Add OCR to a file in place (only modifies file on success)
ocrmypdf myfile.pdf myfile.pdf
# OCR with non-English languages (look up your language's ISO 639-3 code)
ocrmypdf -l fra LeParisien.pdf LeParisien.pdf
# OCR multilingual documents
ocrmypdf -l eng+fra Bilingual-English-French.pdf Bilingual-English-French.pdf
# Deskew (straighten crooked pages)
ocrmypdf --deskew input.pdf output.pdfFor more features, see the documentation.
Requirements
In addition to the required Python version, OCRmyPDF requires external program installations of Ghostscript and Tesseract OCR. OCRmyPDF is pure Python, and runs on pretty much everything: Linux, macOS, Windows and FreeBSD.
Plugins
OCRmyPDF provides a plugin interface allowing its capabilities to be extended or replaced. Here are some plugins we are aware of:
- OCRmyPDF-AppleOCR: replaces the standard Tesseract OCR engine with Apple Vision Framework. Requires macOS.
- OCRmyPDF-EasyOCR: replaces the standard Tesseract OCR engine with EasyOCR, a newer OCR engine based on PyTorch. GPU strongly recommended.
- OCRmyPDF-PaddleOCR: replaces the standard Tesseract OCR engine with PaddleOCR, a powerful GPU accelerated OCR engine.
paperless-ngx provides integration of OCRmyPDF into a searchable document management system.
Press & Media
- Going paperless with OCRmyPDF
- Converting a scanned document into a compressed searchable PDF with redactions
- c't 1-2014, page 59: Detailed presentation of OCRmyPDF v1.0 in the leading German IT magazine c't
- heise Open Source, 09/2014: Texterkennung mit OCRmyPDF
- heise Durchsuchbare PDF-Dokumente mit OCRmyPDF erstellen
- Excellent Utilities: OCRmyPDF
- LinuxUser Texterkennung mit OCRmyPDF und Scanbd automatisieren
- Y Combinator discussion
Business enquiries
OCRmyPDF would not be the software that it is today without companies and users choosing to provide support for feature development and consulting enquiries. We are happy to discuss all enquiries, whether for extending the existing feature set, or integrating OCRmyPDF into a larger system.
License
The OCRmyPDF software is licensed under the Mozilla Public License 2.0 (MPL-2.0). This license permits integration of OCRmyPDF with other code, included commercial and closed source, but asks you to publish source-level modifications you make to OCRmyPDF.
Some components of OCRmyPDF have other licenses, as indicated by standard SPDX license identifiers or the DEP5 copyright and licensing information file. Generally speaking, non-core code is licensed under MIT, and the documentation and test files are licensed under Creative Commons ShareAlike 4.0 (CC-BY-SA 4.0).
Disclaimer
The software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
Overview (`misc/screencast/README.md`)
To regenerate
Using asciinema and svg-term (npm install svg-term-cli).
Create ~/.config/asciinema/config to disable prompt.
[record]
command = fish --init-command 'alias fish_prompt="echo \>\ "'Run asciinema
asciinema rec new_input.castRe-record faster version with fewer pauses
asciinema rec demo.cast -c "asciinema play new_input.cast --speed 2 --idle-time-limit 0.5"Convert to SVG
svg-term --in=misc/screencast/demo.cast --out=misc/screencast/demo.svg --windowIndex (`docs/index.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
OCRmyPDF documentation
:::{figure} images/logo.svg
:::
OCRmyPDF adds an optical character recognition (OCR) text layer to scanned PDF
files, allowing them to be searched.
PDF is the best format for storing and exchanging scanned documents.
Unfortunately, PDFs can be difficult to modify. OCRmyPDF makes it easy to apply
image processing and OCR (recognized, searchable text) to existing PDFs.
:maxdepth: 1
introduction
releasenotes/index
installation
languages
jbig2:caption: Usage
:maxdepth: 2
cookbook
optimizer
docker
advanced
batch
cloud
performance
pdfsecurity
errors:caption: Developers
:maxdepth: 2
api
plugins
apiref
design_notes
contributing
maintainersIndices and tables
- {ref}
genindex - {ref}
modindex - {ref}
search
Releasenotes Index (`docs/releasenotes/index.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Release notes
OCRmyPDF uses semantic versioning for its
command line interface and its public API.
OCRmyPDF's output messages are not considered part of the stable interface -
that is, output messages may be improved at any release level, so parsing them
may be unreliable. Use the API to depend on precise behavior.
The public API may be useful in scripts that launch OCRmyPDF processes or that
wish to use some of its features for working with PDFs.
The most recent release of OCRmyPDF is . Any newer versions
referred to in these notes may exist the main branch but have not been
tagged yet.
OCRmyPDF typically supports the three most recent Python versions.
:::{note}
Attention maintainers: these release notes may be updated with information
about a forthcoming release that has not been tagged yet. A release is only
official when it's tagged and posted to PyPI.
:::
:glob: true
:maxdepth: 1
:reversed: true
version*Advanced (`docs/advanced.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Advanced features
Control of unpaper
OCRmyPDF uses unpaper to provide the implementation of the--clean and --clean-final arguments.
unpaper
provides a variety of image processing filters to improve images.
By default, OCRmyPDF uses only unpaper arguments that were found to
be safe to use on almost all files without having to inspect every page
of the file afterwards. This is particularly true when only --clean
is used, since that instructs OCRmyPDF to only clean the image before
OCR and not the final image.
However, if you wish to use the more aggressive options in unpaper,
you may use --unpaper-args '...' to override the OCRmyPDF's defaults
and forward other arguments to unpaper. This option will forward
arguments to unpaper without any knowledge of what that program
considers to be valid arguments. The string of arguments must be quoted
as shown in the examples below. No filename arguments may be included.
OCRmyPDF will assume it can append input and output filename of
intermediate images to the --unpaper-args string.
In this example, we tell unpaper to expect two pages of text on a
sheet (image), such as occurs when two facing pages of a book are
scanned. unpaper uses this information to deskew each independently
and clean up the margins of both.
ocrmypdf --clean --clean-final --unpaper-args '--layout double' input.pdf output.pdf
ocrmypdf --clean --clean-final --unpaper-args '--layout double --no-noisefilter' input.pdf output.pdf:::{warning}
Some unpaper features will reposition text within the image.--clean-final is recommended to avoid this issue.
:::
:::{warning}
Some unpaper features cause multiple input or output files to be
consumed or produced. OCRmyPDF requires unpaper to consume one
file and produce one file; errors will result if this assumption is not
met.
:::
:::{note}unpaper uses uncompressed PBM/PGM/PPM files for its intermediate
files. For large images or documents, it can take a lot of temporary
disk space.
:::
Control of OCR options
OCRmyPDF provides many features to control the behavior of the OCR
engine, Tesseract.
OCR processing mode
:::{versionadded} 17.0.0
The --mode (-m) argument consolidates OCR processing options.
:::
OCRmyPDF provides a unified --mode argument to control how pages with
existing text are handled:
| Mode | Behavior | Legacy equivalent |
|---|---|---|
default |
Error if text is found | (no flag) |
force |
Rasterize all content and run OCR, keeping hyperlinks | --force-ocr |
force-ocr-no-links |
Like force, but also discard hyperlinks |
|
skip |
Skip pages with existing text | --skip-text |
redo |
Re-OCR pages, stripping old OCR layer | --redo-ocr |
# Skip pages that already have text
ocrmypdf --mode skip input.pdf output.pdf
# or equivalently:
ocrmypdf -m skip input.pdf output.pdf
# Force OCR on all pages (rasterizes everything)
ocrmypdf --mode force input.pdf output.pdf
# Re-do OCR, replacing old invisible text
ocrmypdf --mode redo input.pdf output.pdfThe legacy flags (--force-ocr, --skip-text, --redo-ocr) remain as
silent aliases for backward compatibility.
When OCR is skipped
If a page in a PDF seems to have text, by default OCRmyPDF will exit
without modifying the PDF. This is to ensure that PDFs that were
previously OCRed or were "born digital" rather than scanned are not
processed.
If --mode skip (or --skip-text) is issued, then no image processing or OCR will be
performed on pages that already have text. The page will be copied to
the output. This may be useful for documents that contain both "born
digital" and scanned content, or to use OCRmyPDF to normalize and
convert to PDF/A regardless of their contents.
If --mode redo (or --redo-ocr) is issued, then a detailed text analysis is performed.
Text is categorized as either visible or invisible. Invisible text (OCR)
is stripped out. Then an image of each page is created with visible text
masked out. The page image is sent for OCR, and any additional text is
inserted as OCR. If a file contains a mix of text and bitmap images that
contain text, OCRmyPDF will locate the additional text in images without
disrupting the existing text. Some PDF OCR solutions render text as
technically printable or visible in some way, perhaps by drawing it and
then painting over it. OCRmyPDF cannot distinguish this type of OCR
text from real text, so it will not be "redone".
If --mode force (or --force-ocr) is issued, then all pages will be rasterized to
images, discarding any hidden OCR text, rasterizing any printable
text, and flattening form fields or interactive objects into their visual
representation. This is useful for redoing OCR, for fixing OCR text
with a damaged character map (text is selectable but not searchable),
and destroying redacted information.
Hyperlinks (link annotations) are kept, so links to web pages and to other
places in the document still work in the rasterized output; they are clickable
areas over the page image. Other annotations are flattened into the image.
Use --mode force-ocr-no-links to rasterize in the same way but discard the
hyperlinks too, for example when the link targets themselves are sensitive.
Tagged PDFs and structural markup
Some PDFs carry a logical structure tree (/StructTreeRoot), the markup that
makes a "Tagged PDF" — typically the result of layout analysis or a born-digital
export. By default OCRmyPDF treats this as a signal that the document may not need
OCR and exits, in the same way it stops on PDFs that already contain text. Use--tagged-pdf-mode ignore, or one of --mode skip/redo/force, to process
such a file anyway.
OCRmyPDF cannot rebuild a structure tree to match newly recognized text. When--force-ocr rasterizes pages, or --redo-ocr strips and rewrites the text layer,
the structure tree no longer corresponds to the page content, so it is discarded.--mode skip leaves text pages untouched, so their structural markup is preserved.
:::{note}
Preservation under --mode skip only holds when the output is not converted to
PDF/A. PDF/A conversion is performed by Ghostscript, and Ghostscript 10.x discards
the structure tree during conversion (Ghostscript 9.x preserved it). Because the
default --output-type auto may fall back to Ghostscript, use--output-type pdf if you need to guarantee that a Tagged PDF's structural markup
survives. Speculative PDF/A conversion, which does not use Ghostscript,
sidesteps this issue whenever the file passes pikepdf's PDF/A validator.
:::
Time and image size limits
By default, OCRmyPDF permits tesseract to run for three minutes (180
seconds) per page. This is usually more than enough time to find all
text on a reasonably sized page with modern hardware.
If a page is skipped, it will be inserted without OCR. If preprocessing
was requested, the preprocessed image layer will be inserted.
If you want to adjust the amount of time spent on OCR, change--tesseract-timeout. You can also automatically skip images that
exceed a certain number of megapixels with --skip-big. (A 300 DPI,
8.5×11" page image is 8.4 megapixels.)
# Allow 300 seconds for OCR; skip any page larger than 50 megapixels
ocrmypdf --tesseract-timeout 300 --skip-big 50 bigfile.pdf output.pdfOCR for huge images
Tesseract has internal limits on the size
of images it will process. By default,--tesseract-downsample-large-images is enabled, and OCRmyPDF will
downsample images to fit Tesseract limits. (The limits are usually encountered
only for scanned images of oversized media, such as large maps or blueprints exceeding
110 cm or 43 inches in either dimension, and at high DPI.) This feature can disabled
using --no-tesseract-downsample-large-images.
--tesseract-downsample-above Npixels adjusts the threshold at which images
will be downsampled. By default, only images that exceed any of Tesseract's
internal limits are downsampled (32767 pixels on either dimension).
You will also need to set --tesseract-timeout high enough to allow
for processing.
Only the image sent for OCR is downsampled. The original image is
preserved.
# Allow 600 seconds for OCR on huge images
ocrmypdf --tesseract-timeout 600 \
--tesseract-downsample-large-images \
bigfile.pdf output.pdf
# Downsample images above 5000 pixels on the longest dimension to
# 5000 pixels
ocrmypdf --tesseract-timeout 120 \
--tesseract-downsample-large-images \
--tesseract-downsample-above 5000 \
bigfile.pdf output_downsampled_ocr.pdfOverriding default tesseract
OCRmyPDF checks the system PATH for the tesseract binary.
Some relevant environment variables that influence Tesseract's behavior
include:
.. envvar:: TESSDATA_PREFIX
Overrides the path to Tesseract's data files. This can allow
simultaneous installation of the "best" and "fast" training data
sets. OCRmyPDF does not manage this environment variable.
If you point ``TESSDATA_PREFIX`` at a hand-assembled ``tessdata``
folder (for example, individual ``.traineddata`` files downloaded
from tessdata_best), make sure it also contains the ``configs/``
subdirectory with the ``hocr`` and ``txt`` files. OCRmyPDF requires
these; without them Tesseract produces no output. See
:ref:`Tesseract cannot open its config file <tesseract-config-missing>`... envvar:: OMP_THREAD_LIMIT
Controls the number of threads Tesseract will use. OCRmyPDF will
manage this environment variable if it is not already set.For example, if you have a development build of Tesseract don't wish to
use the system installation, you can launch OCRmyPDF as follows:
env \
PATH=/home/user/src/tesseract/api:$PATH \
TESSDATA_PREFIX=/home/user/src/tesseract \
ocrmypdf input.pdf output.pdfIn this example TESSDATA_PREFIX is required to redirect Tesseract to
an alternate folder for its "tessdata" files.
Overriding other support programs
In addition to tesseract, OCRmyPDF uses the following external binaries:
gs(Ghostscript)unpaperpngquantjbig2
In each case OCRmyPDF will search the PATH environment variable to
locate the binaries. By modifying the PATH environment variable, you
can override the binaries that OCRmyPDF uses.
Changing Tesseract configuration variables
You can override Tesseract's default control
parameters
with a configuration file.
As an example, this configuration will disable Tesseract's dictionary
for current language. Normally the dictionary is helpful for
interpolating words that are unclear, but it may interfere with OCR if
the document does not contain many words (for example, a list of part
numbers).
Create a file named "no-dict.cfg" with these contents:
load_system_dawg 0
language_model_penalty_non_dict_word 0
language_model_penalty_non_freq_dict_word 0then run ocrmypdf as follows (along with any other desired arguments):
ocrmypdf --tesseract-config no-dict.cfg input.pdf output.pdf:::{warning}
Some combinations of control parameters will break Tesseract or break
assumptions that OCRmyPDF makes about Tesseract's output.
:::
Changing page segmentation mode
The directive --tesseract-pagesegmode Nmode forwards the desired page segmentation
mode to Tesseract OCR. The default is 3.
Page segmentation can improve OCR results when you know that a PDF ought to be
analyzed a particular way, such as PDFs whose pages contain only a single line of
text. For the vast majority of users, changing the page segmentation mode will only
make things worse.
As of June 2024, the Tesseract page segmentation modes are:
| ID | Description |
|---|---|
| 0 | Orientation and script detection (OSD) only. |
| 1 | Automatic page segmentation with OSD. |
| 2 | Automatic page segmentation, but no OSD, or OCR. (not implemented) |
| 3 | Fully automatic page segmentation, but no OSD. (Default) |
| 4 | Assume a single column of text of variable sizes. |
| 5 | Assume a single uniform block of vertically aligned text. |
| 6 | Assume a single uniform block of text. |
| 7 | Treat the image as a single text line. |
| 8 | Treat the image as a single word. |
| 9 | Treat the image as a single word in a circle. |
| 10 | Treat the image as a single character. |
| 11 | Sparse text. Find as much text as possible in no particular order. |
| 12 | Sparse text with OSD. |
| 13 | Raw line. Treat the image as a single text line, bypassing hacks that are Tesseract-specific. |
Modes 0, 1, 2, and 12 (all of those that enable orientation and script detection)
are not compatible with OCRmyPDF, which performs OSD in a separate step from OCR.
Their use may interfere with --rotate-pages and other features.
It is currently not possible to use advanced Tesseract OCR features, such as creating
OCR information, when using Tesseract through OCRmyPDF.
Choosing a PDF rasterizer
:::{versionadded} 17.0.0
:::
rasterizing
: Converting a PDF page to an image for OCR processing.
OCRmyPDF supports two PDF rasterizers:
| Rasterizer | Package | Advantages | Disadvantages |
|---|---|---|---|
| pypdfium2 | Python package | Faster, fewer version issues | Requires pypdfium2 package |
| Ghostscript | System binary | More widely packaged | Version consistency issues, restrictive AGPLv3 |
The --rasterizer argument controls which rasterizer is used:
# Automatic selection (default) - prefers pypdfium when available
ocrmypdf --rasterizer auto input.pdf output.pdf
# Force pypdfium2
ocrmypdf --rasterizer pypdfium input.pdf output.pdf
# Force Ghostscript
ocrmypdf --rasterizer ghostscript input.pdf output.pdfpypdfium2 is a Python binding for pdfium, the PDF rendering library used
by Google Chrome and Chromium. It generally produces output identical to
Ghostscript but with better performance.
:::{note}
If pypdfium2 is not installed and --rasterizer pypdfium is requested,
OCRmyPDF will exit with an error. Install it with: pip install pypdfium2
:::
Changing the PDF renderer
rendering
: Creating a new PDF from other data (such as an existing PDF).
:::{versionchanged} 17.0.0
The fpdf2 renderer is now the default, replacing the legacy hOCR renderer.
:::
OCRmyPDF uses PDF renderers to create the invisible text layer. The
renderer may be selected using --pdf-renderer. The default isauto which selects fpdf2.
The `fpdf2` renderer (default)
:::{versionadded} 17.0.0
:::
The fpdf2 renderer creates text layers using the fpdf2 library. It provides:
- Full multilingual support including RTL languages (Arabic, Hebrew, Persian)
- Accurate text positioning aligned with OCR bounding boxes
- Improved "Occulta" glyphless font handling:
- Zero-width markers are properly handled
- Double-width CJK characters are properly sized
- Direct OcrElement tree input (no hOCR intermediate format required)
The fpdf2 renderer is the recommended choice for all installations.
:::{note}
The fpdf2 renderer may be slightly slower than the legacy hocrtransform
renderer for some workloads. This is an area of ongoing optimization.
:::
In both renderers, a text-only layer is rendered and sandwiched (overlaid)
on to either the original PDF page, or newly rasterized version of the
original PDF page (when --mode force is used). In this way, loss
of PDF information is generally avoided. (You may need to disable PDF/A
conversion and optimization to eliminate all lossy transformations.)
The `sandwich` renderer
The sandwich renderer uses Tesseract's text-only PDF feature,
which produces a PDF page that lays out the OCR in invisible text.
Currently some problematic PDF viewers like Mozilla PDF.js and macOS
Preview have problems with segmenting its text output, and
mightrunseveralwordstogether. It also does not implement right to left
fonts (Arabic, Hebrew, Persian). The output of this renderer cannot
be edited. The sandwich renderer is retained for testing.
When image preprocessing features like --deskew are used, the
original PDF will be rendered as a full page and the OCR layer will be
placed on top.
Legacy renderer options
The hocr and hocrdebug renderer options are deprecated and
automatically redirect to fpdf2. They will be removed in a future version.
Rendering and rasterizing options
:::{versionadded} 14.3.0
:::
The --continue-on-soft-render-error option allows OCRmyPDF to
proceed if a page cannot be rasterized/rendered. This is useful if you are
trying to get the best possible OCR from a PDF that is not well-formed,
and you are willing to accept some pages that may not visually match the
input, and that may not OCR well.
Color conversion strategy
:::{versionadded} 15.0.0
:::
OCRmyPDF uses Ghostscript to convert PDF to PDF/A. In some cases, this
conversion requires color conversion. The default strategy is to convert
using the LeaveColorUnchanged strategy, which preserves the original
color space wherever possible (some rare color spaces might still be
converted).
Usually document scanners produce PDFs in the sRGB color space, and do
not need to be converted, so the default strategy is appropriate.
Suppose that you have a document that was prepared for professional
printing in a Separation or CMYK color space, and text was converted to
curves. In this case, you may want to use a different color conversion
strategy. The --color-conversion-strategy option allows you to select a
different strategy, such as RGB.
Advanced Ghostscript tuning
:::{versionadded} 17.5.0
:::
OCRmyPDF intentionally hides most Ghostscript controls because Ghostscript
is a legacy code path. The preferred PDF/A pipeline in v17+ uses pypdfium2
as the rasterizer and pikepdf's PDF/A validator to check speculative PDF/A output, with
Ghostscript reserved as a fallback for PDFs that cannot be made compliant
without it. OCRmyPDF's separate optimizer (controlled by --optimize,--jpeg-quality, --png-quality, etc.) is the supported way to shrink
output PDFs: it gives consistent results across input files, and isolates
Ghostscript so it can focus on producing a PDF/A with as few image
transformations as possible.
The two options below are exposed for advanced users who want to tune
Ghostscript's intermediate PDF/A output directly. Most users will get
more predictable results from the optimizer.
`--ghostscript-jpeg-quality Q`
Sets Ghostscript's -dJPEGQ switch for images that Ghostscript chooses
to recompress to JPEG while building a PDF/A. Q=0 requests maximum
compression and Q=100 requests best quality; if the flag is omitted,
OCRmyPDF passes 95 (the historical default). This only affects images
Ghostscript transcodes — existing JPEGs pass through unchanged on modern
Ghostscript releases. For end-to-end JPEG quality tuning, prefer--jpeg-quality, which is implemented by the OCRmyPDF optimizer and is
applied independently of whatever Ghostscript decides to do.
Note: setting both --ghostscript-jpeg-quality and --jpeg-quality can
result in double JPEG recompression, since the optimizer may re-encode
images that Ghostscript already recompressed. This can degrade quality
in subtle ways.
`--ghostscript-jpeg-maxdpi DPI`
Enables Ghostscript's image downsampling and caps color, grayscale, and
monochrome image resolution to DPI. The downsample threshold is set to1.0, so any image whose effective DPI exceeds the cap will be
downsampled.
Reducing JPEG quality is almost always a better trade than downsampling
at the same compression budget: a 400 DPI JPEG at modest quality usually
looks much better than a 200 DPI JPEG, because the JPEG codec can spend
bits where they count. Downsampling is also dangerous for PDFs that
combine a low-resolution color image with a high-resolution monochrome
mask — capping the mask resolution can produce visible quality loss.
For these reasons, prefer --jpeg-quality over --ghostscript-jpeg-maxdpi
unless you specifically want to force a hard DPI cap.
Example:
ocrmypdf --output-type pdfa \
--ghostscript-jpeg-quality 80 \
--ghostscript-jpeg-maxdpi 150 \
in.pdf out.pdfThese options only take effect when Ghostscript is invoked for PDF/A
conversion (--output-type pdfa, pdfa-1, pdfa-2, or pdfa-3, or
when --output-type auto falls back to Ghostscript).
PDF/A output modes
:::{versionchanged} 17.0.0
The default --output-type is now auto instead of pdfa.
:::
OCRmyPDF can produce PDF/A compliant output for long-term archival. The--output-type argument controls PDF/A conversion:
| Output type | Behavior |
|---|---|
auto |
Best-effort PDF/A without requiring Ghostscript (default) |
pdfa |
PDF/A-2b via Ghostscript |
pdfa-1 |
PDF/A-1b via Ghostscript |
pdfa-2 |
PDF/A-2b via Ghostscript (same as pdfa) |
pdfa-3 |
PDF/A-3b via Ghostscript |
pdf |
Standard PDF, no PDF/A conversion |
none |
No output file (useful with --sidecar) |
Non-embedded fonts and PDF/A
:::{versionadded} 17.8.0
OCRmyPDF now refuses to corrupt non-embedded CID text layers during PDF/A
conversion.
:::
PDF/A requires every font to be embedded. If your input already has a text
layer that uses non-embedded CID fonts — most commonly a CJK
(Chinese-Japanese-Korean) OCR layer
produced by Adobe Acrobat, which relies on the reader's system fonts —
Ghostscript would have to substitute and re-embed a replacement font to make
the file PDF/A. For CID-keyed (CJK) fonts this routinely corrupts the
character-to-Unicode mapping, so the text silently becomes garbage or stops
being searchable even though the page still looks correct.
Rather than emit corrupted output, OCRmyPDF detects this situation and:
- with
--output-type auto(the default), produces a regular PDF instead of
PDF/A, preserving the existing text layer exactly; - with an explicit
--output-type pdfa(orpdfa-1/pdfa-2/pdfa-3), stops
with an error.
This is a Ghostscript limitation that OCRmyPDF cannot repair, because a
non-embedded font cannot be made PDF/A-compliant without re-embedding it. To
keep the existing text layer, use --output-type pdf. To produce PDF/A anyway,
re-run OCR with --force-ocr, which discards the original text layer and
rebuilds it with embedded fonts. Text layers whose fonts are already embedded
are converted to PDF/A normally.
Speculative PDF/A conversion
:::{versionadded} 17.0.0
:::
:::{versionchanged} 17.13.0
Speculative conversion is checked by pikepdf's PDF/A validator
(pikepdf.pdfa) instead of veraPDF, so it no longer depends on veraPDF
being installed, and it now works for PDF/A-1b as well as PDF/A-2b and
PDF/A-3b.
:::
For all PDF/A output types, including the default --output-type auto,
OCRmyPDF first attempts a fast "speculative" PDF/A conversion that avoids
Ghostscript when possible:
- Using
pikepdf.pdfa, OCRmyPDF replaces the output intents with an sRGB ICC
profile, removes image interpolation flags, removes annotations that are
hidden or not viewable (as Ghostscript does, with a warning) and sets the
Print flag on the others, adds the/CIDSetthat PDF/A-1 requires
for subset CID fonts, removes XMP metadata properties that PDF/A does not
permit, and adds the PDF/A identification to the XMP metadata. - pikepdf's PDF/A validator checks the file as written, and OCRmyPDF
checks the final output file again after its metadata and optimization
steps, which rewrite it. - If validation passes, Ghostscript is skipped entirely.
- If validation fails, OCRmyPDF falls back to Ghostscript.
pikepdf's validator is deliberately conservative: it approves only
constructs it recognizes and knows to conform. A file with a violation
fails; a file containing constructs the validator does not check is not
checked, which OCRmyPDF also treats as a reason to use Ghostscript. It is
not a replacement for a reference PDF/A validator such as veraPDF. Runocrmypdf -v1 to see why a file was not approved. See pikepdf's
documentation of pikepdf.pdfa for what the validator covers.
This fast path avoids some Ghostscript limitations (such as image
transcoding) and is used whenever it can produce valid PDF/A. When it
cannot, because the input needs real conversion (for example, fonts that
are not embedded, or transparency in PDF/A-1), auto falls back to
Ghostscript so that it still produces PDF/A by default, matching OCRmyPDF 16
and earlier. If even Ghostscript cannot safely produce PDF/A, auto outputs
a regular PDF instead of failing.
Choosing the PDF/A backend
:::{versionadded} 17.13.0
:::
--pdfa-backend selects how PDF/A is produced:
auto(the default) tries speculative conversion first and falls back to
Ghostscript, as described above.internaluses speculative conversion only and never uses Ghostscript to
make PDF/A. JPEG images pass through unchanged and the result is checked
by pikepdf's validator. If the validator does not approve the file,--output-type pdfa(orpdfa-1,pdfa-2,pdfa-3) fails with exit
code 10 (pdfa_conversion_failed) and logs the validator's findings;--output-type autooutputs a regular PDF instead. Ghostscript need not
be installed for PDF/A output.ghostscriptalways converts with Ghostscript, which re-encodes images
and converts colours as needed, and requires Ghostscript to be installed.
The options that only Ghostscript applies, --pdfa-image-compression
(other than auto), --ghostscript-jpeg-quality and--ghostscript-jpeg-maxdpi, select the Ghostscript backend when--pdfa-backend is auto, and are an error with --pdfa-backend internal.
OCRmyPDF does no colour conversion of its own, so the same applies to--color-conversion-strategy CMYK, Gray and UseDeviceIndependentColor;LeaveColorUnchanged (the default) and RGB need no conversion in any file
pikepdf's validator approves, so they work with every backend.
# PDF/A without Ghostscript, or fail
ocrmypdf --output-type pdfa --pdfa-backend internal input.pdf output.pdf
# PDF/A from Ghostscript, as in OCRmyPDF 16 and earlier
ocrmypdf --output-type pdfa --pdfa-backend ghostscript input.pdf output.pdfPDF/A conversion flow
The following diagram illustrates the PDF/A conversion decision tree:
flowchart TD
A[Start] --> B{--output-type?}
B -->|pdf| C[Output standard PDF]
B -->|pdfa/pdfa-N/auto| G{--pdfa-backend?}
G -->|auto/internal| E[Attempt speculative conversion]
G -->|ghostscript| D
E --> F["Repair and declare PDF/A (pikepdf)"]
F --> I[Validate with pikepdf]
I --> J{Validation passed?}
J -->|Yes| K[Output PDF/A - Ghostscript skipped]
J -->|No, internal + pdfa| Y[Error: PDF/A conversion failed]
J -->|No, internal + auto| L
J -->|No, auto backend| H{Ghostscript available?}
H -->|Yes| D[Ghostscript PDF/A conversion]
H -->|No, auto| L[Output standard PDF]
H -->|No, pdfa| X[Error: Ghostscript required]
D --> N[Output PDF/A]
style K fill:#90EE90
style N fill:#90EE90
style L fill:#FFB6C1:::{warning}
Breaking change: If Ghostscript is not installed and the file does not
pass speculative conversion, --output-type auto will produce a standard
PDF instead of PDF/A. This is a change from previous versions where
Ghostscript was required and PDF/A was always produced.
:::
Return code policy
OCRmyPDF writes all messages to stderr. stdout is reserved for
piping output files. stdin is reserved for piping input files.
The return codes generated by the OCRmyPDF are considered part of the
stable user interface. They may be imported fromocrmypdf.exceptions.
/* Detailed source-code truncated for AI context efficiency. */(tmpdir)=
Changing temporary storage location
OCRmyPDF generates many temporary files during processing.
To change where temporary files are stored, change the TMPDIR
environment variable for ocrmypdf's environment. (Python'stempfile.gettempdir() returns the root directory in which temporary
files will be stored.) For example, one could redirect TMPDIR to a
large RAM disk to avoid wear on HDD/SSD and potentially improve
performance.
On Windows, the TEMP environment variable is used instead.
Debugging the intermediate files
OCRmyPDF normally saves its intermediate results to a temporary folder
and deletes this folder when it exits, whether it succeeded or failed.
If the --keep-temporary-files (-k) argument is issued on the
command line, OCRmyPDF will keep the temporary folder and print the location,
whether it succeeded or failed. An example message is:
Temporary working files retained at:
/tmp/ocrmypdf.io.u20wpz07When OCRmyPDF is launched as a snap, this corresponds to the snap filesystem, for instance:
/tmp/snap-private-tmp/snap.ocrmypdf/tmp/ocrmypdf.io.u20wpz07
The organization of this folder is an implementation detail and subject
to change between releases. However the general organization is that
working files on a per page basis have the page number as a prefix
(starting with page 1), an infix indicates the processing stage, and a
suffix indicates the file type. Some important files include:
_rasterize.png- what the input page looks like_ocr.png- the file that is sent to Tesseract for OCR; depending
on arguments this may differ from the presentation image_pp_deskew.png- the image, after deskewing_pp_clean.png- the image, after cleaning with unpaper_ocr_hocr.pdf- the OCR file; appears as a blank page with invisible
text embedded_ocr_hocr.txt- the OCR text (not necessarily all text on the page,
if the page is mixed format)fix_docinfo.pdf- a temporary file created to fix the PDF DocumentInfo
data structuregraft_layers.pdf- the rendered PDF with OCR layers grafted onpdfa.pdf-graft_layers.pdfafter conversion to PDF/Apdfa.ps- a PostScript file used by Ghostscript for PDF/A conversionoptimize.pdf- the PDF generated before optimizationoptimize.out.pdf- the PDF generated by optimizationorigin- the input fileorigin.pdf- the input file or the input image converted to PDFimages/*- images extracted during the optimization process; here
the prefix indicates a PDF object ID not a page number
Api (`docs/api.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Using the OCRmyPDF API
OCRmyPDF originated as a command line program and continues to have this
legacy, but parts of it can be imported and used in other Python
applications.
Some applications may want to consider running ocrmypdf from a
subprocess call anyway, as this provides isolation of its activities.
Example
OCRmyPDF provides one high-level function to run its main engine from an
application.
The {func}`ocrmypdf.ocr` function now accepts an {class}`~ocrmypdf.OcrOptions`
object as its first argument, providing a cleaner API with full type hints
and validation. The previous positional argument style remains supported.Modern API (recommended)
The recommended way to call {func}ocrmypdf.ocr is to construct an
{class}~ocrmypdf.OcrOptions object with all settings, then pass it
as the sole argument:
import ocrmypdf
from ocrmypdf import OcrOptions
if __name__ == '__main__': # To ensure correct behavior on Windows and macOS
options = OcrOptions(
input_file='input.pdf',
output_file='output.pdf',
deskew=True,
languages=['eng'],
)
ocrmypdf.ocr(options){class}~ocrmypdf.OcrOptions is a Pydantic model that provides:
- Full type hints and IDE autocompletion
- Validation of option values at construction time
- Clear documentation of all available options
The {class}`~ocrmypdf.OcrOptions` class is now exported from the top-level
`ocrmypdf` module.Legacy API
For compatibility with OCRmyPDF < v17, the traditional calling style
with positional arguments is still fully supported:
import ocrmypdf
if __name__ == '__main__': # To ensure correct behavior on Windows and macOS
ocrmypdf.ocr('input.pdf', 'output.pdf', deskew=True)With this style, all of the command line arguments are available
and may be passed as equivalent keywords.
A few differences are that verbose and quiet are not available.
Instead, output should be managed by configuring logging.
Parent process requirements
The {func}ocrmypdf.ocr function runs OCRmyPDF similar to command line
execution. To do this, it will:
- create worker processes or threads
- manage the signal flags of its worker processes
- execute other subprocesses (forking and executing other programs)
The Python process that calls {func}ocrmypdf.ocr() must be sufficiently
privileged to perform these actions.
There currently is no option to manage how jobs are scheduled other
than the argument jobs= which will limit the number of worker
processes.
Creating a child process to call {func}ocrmypdf.ocr() is suggested. That
way your application will survive and remain interactive even if
OCRmyPDF fails for any reason. For example:
from multiprocessing import Process
import ocrmypdf
from ocrmypdf import OcrOptions
def ocrmypdf_process():
options = OcrOptions(input_file='input.pdf', output_file='output.pdf')
ocrmypdf.ocr(options)
def call_ocrmypdf_from_my_app():
p = Process(target=ocrmypdf_process)
p.start()
p.join()Programs that call {func}ocrmypdf.ocr() should also install a SIGBUS signal
handler (except on Windows), to raise an exception if access to a memory
mapped file fails. OCRmyPDF may use memory mapping.
Several OCR jobs may run concurrently in the same Python interpreter process.
Installing a plugin set mutates interpreter-global state, because of how OCRmyPDF's
plugins and Python's library import system work, and a running job cannot survive
that state being replaced underneath it. {func}ocrmypdf.ocr() therefore takes a
readers-writer lock: installing a plugin set takes it exclusively, so it waits for
every job already running, while a job holds it shared for its own run. A plugin set
is installed once per interpreter and reused thereafter, so a job that asks for a
plugin set some earlier job already installed never blocks.
The practical consequence is that concurrent jobs should all use the same plugin
set. Jobs requesting different plugin sets serialize against each other. They must
also use the same max_image_mpixels, since Pillow's decompression-bomb limit is
interpreter-global and the last job to set it wins. If you need to run jobs with
differing configurations, use separate processes.
Because a plugin set is installed only once, a plugin module given as a file path is
executed once per interpreter rather than once per call. Plugins must not rely on
being re-executed for each job, and must not store per-job state on the plugin
manager, which concurrent jobs may share.
Note that N concurrent jobs each configured with jobs=M may spawn up to N*M
workers, so size jobs accordingly.
:::{warning}
On Windows and macOS, the script that calls {func}ocrmypdf.ocr() must be
protected by an "ifmain" guard (if __name__ == '__main__'). If you do
not take at least one of these steps, process semantics will prevent
OCRmyPDF from working correctly.
:::
Logging
OCRmyPDF will log under loggers named ocrmypdf. In addition, it
imports pdfminer and PIL, both of which post log messages under
those logging namespaces.
You can configure the logging as desired for your application or call
{func}ocrmypdf.configure_logging to configure logging the same way
OCRmyPDF itself does. The command line parameters such as --quiet
and --verbose have no equivalents in the API; you must use the
provided configuration function or do configuration in a way that suits
your use case.
Progress monitoring
OCRmyPDF uses the rich package to implement its progress bars.
{func}ocrmypdf.configure_logging will set up logging output tosys.stderr in a way that is compatible with the display of the
progress bar. Use ocrmypdf.ocr(...progress_bar=False) to disable
the progress bar.
Standard output
OCRmyPDF is strict about not writing to standard output so that
users can safely use it in a pipeline and produce a valid output
file. A caller application will have to ensure it does not write to
standard output either, if it wants to be compatible with this
behavior and support piping to a file. Another benefit of running
OCRmyPDF in a child process, as recommended above, is that it will
not interfere with the parent process's standard output.
Exceptions
OCRmyPDF may throw standard Python exceptions, ocrmypdf.exceptions.*
exceptions, some exceptions related to multiprocessing, and
{exc}KeyboardInterrupt. The parent process should provide an exception
handler. OCRmyPDF will clean up its temporary files and worker processes
automatically when an exception occurs.
When OCRmyPDF succeeds conditionally, it returns an integer exit code.
Bundling with PyInstaller
OCRmyPDF ships a PyInstaller hook, which PyInstaller finds automatically,
so applications that call {func}ocrmypdf.ocr can be frozen without extra--collect-* options. The hook bundles OCRmyPDF's builtin plugins, fonts
and color profiles, and the pikepdf data needed for PDF/A output.
The external programs are not bundled. Tesseract and Ghostscript must still
be installed on the target machine and be on the PATH, as must any optional
tools you use, such as unpaper or pngquant. Third party plugins must be
bundled by your application.
Like any frozen application that uses multiprocessing, call
{func}multiprocessing.freeze_support at the start of yourif __name__ == '__main__': block.
Plugin Development Changes
Plugin hooks now receive {class}`~ocrmypdf.OcrOptions` objects instead of
`argparse.Namespace`.- {class}
~ocrmypdf.OcrOptionsprovides the same attribute access asNamespace(duck-typing compatible) - Plugin developers should update type hints:
from ocrmypdf import OcrOptions - Built-in plugins no longer modify options in-place for better immutability
Most existing plugins will continue working without modification due to the
duck-typing compatibility between {class}~ocrmypdf.OcrOptions and Namespace.
Apiref (`docs/apiref.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
API reference
This page summarizes the rest of the public API. Generally speaking this
should be mainly of interest to plugin developers.
ocrmypdf.api
.. automodule:: ocrmypdf.api
:members:ocrmypdf._options
.. automodule:: ocrmypdf._options
:members: OcrOptionsocrmypdf.exceptions
.. automodule:: ocrmypdf.exceptions
:members:
:undoc-members:ocrmypdf.helpers
.. automodule:: ocrmypdf.helpers
:members:ocrmypdf.hocrtransform
.. automodule:: ocrmypdf.hocrtransform
:members:ocrmypdf.pdfa
.. automodule:: ocrmypdf.pdfa
:members:ocrmypdf.quality
.. automodule:: ocrmypdf.quality
:members:ocrmypdf.subprocess
.. automodule:: ocrmypdf.subprocess
:members:Batch (`docs/batch.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Batch processing
This article provides information about running OCRmyPDF on multiple
files or configuring it as a service triggered by file system events.
Batch jobs
Consider using the excellent GNU
Parallel to apply OCRmyPDF to
multiple files at once.
Both parallel and ocrmypdf will try to use all available processors.
To maximize parallelism without overloading your system with processes,
consider using parallel -j 2 to limit parallel to running two jobs at
once.
This command will run ocrmypdf on all files named *.pdf in the
current directory and write them to the previously created output/
folder. It will not search subdirectories.
The --tag argument tells parallel to print the filename as a prefix
whenever a message is printed, so that one can trace any errors to the
file that produced them.
:::{code} bash
parallel --tag -j 2 ocrmypdf '{}' 'output/{}' ::: *.pdf
:::
OCRmyPDF automatically repairs PDFs before parsing and gathering
information from them.
Directory trees
This will walk through a directory tree and run OCR on all files in
place, and printing each filename in between runs:
:::{code} bash
find . -name '*.pdf' -printf '%p\n' -exec ocrmypdf '{}' '{}' ;
:::
This only runs one ocrmypdf process at a time. This variation usesfind to create a directory list and parallel to parallelize runs ofocrmypdf, again updating files in place.
:::{code} bash
find . -name '*.pdf' | parallel --tag -j 2 ocrmypdf '{}' '{}'
:::
In a Windows batch file, use
:::{code} bat
for /r %%f in (*.pdf) do ocrmypdf %%f %%f
:::
With a Docker container, you will need to stream through standard input
and output:
:::{code} bash
find . -name '*.pdf' -print0 | xargs -0 | while read pdf; do
pdfout=$(mktemp)
docker run --rm -i jbarlow83/ocrmypdf - - <$pdf >$pdfout && cp $pdfout $pdf
done
:::
Sample script
This user contributed script also provides an example of batch
processing.
:::{literalinclude} ../misc/batch.py
caption: misc/batch.py
:::
Synology DiskStations
Synology DiskStations (Network Attached Storage devices) can run the
Docker image of OCRmyPDF if the Synology Docker
package is
installed. Attached is a script to address particular quirks of using
OCRmyPDF on one of these devices.
At the time this script was written, it only worked for x86-based
Synology products. It is not known if it will work on ARM-based Synology
products. Further adjustments might be needed to deal with the
Synology's relatively limited CPU and RAM.
:::{literalinclude} ../misc/synology.py
caption: misc/synology.py - Sample script for Synology DiskStations
:::
Huge batch jobs
If you have thousands of files to work with, contact the author.
Consulting work related to OCRmyPDF helps fund this open source project
and all inquiries are appreciated.
Hot (watched) folders
Watched folders with watcher.py
OCRmyPDF has a folder watcher called watcher.py, which is currently
included in source distributions but not part of the main program. It
may be used natively or may run in a Docker container. Native instances
tend to give better performance. watcher.py works on all platforms.
Users may need to customize the script to meet their requirements.
:::{code} bash
Using uv (recommended)
uv sync --extra watcher
Or using pip
pip3 install ocrmypdf[watcher]
env OCR_INPUT_DIRECTORY=/mnt/input-pdfs
OCR_OUTPUT_DIRECTORY=/mnt/output-pdfs
OCR_OUTPUT_STRUCTURE=YEAR_MONTH
python3 watcher.py
:::
:::{list-table} watcher.py environment variables
header-rows: 1
- Environment variable
- Description
- OCR_INPUT_DIRECTORY
- Set input directory to monitor (recursive)
- OCR_OUTPUT_DIRECTORY
- Set output directory (should not be under input)
- OCR_ARCHIVE_DIRECTORY
- Set archive directory for processed originals (should not be under input, requires
OCR_ON_SUCCESS_ARCHIVEto be set)
- OCR_ON_SUCCESS_DELETE
- Delete the input file if the exit code is 0 (OK). Takes precedence over
OCR_ON_SUCCESS_ARCHIVEif both are set.
- OCR_ON_SUCCESS_ARCHIVE
- Move the processed original file to
OCR_ARCHIVE_DIRECTORYif the exit code is 0 (OK). Note thatOCR_ON_SUCCESS_DELETEtakes precedence over this option, i.e. if both options are set, the input file will be deleted.
- OCR_OUTPUT_STRUCTURE (
--output-structure) - How output (and archived) files are laid out under their destination
directory. One ofFLAT(default; all files directly in the
destination directory),YEAR_MONTH({destination}/{year}/{month}/{filename}),
orHIERARCHY(mirrors the input directory tree, e.g.input/a/b/c.pdf→{destination}/a/b/c.pdf). Applies to bothOCR_OUTPUT_DIRECTORYandOCR_ARCHIVE_DIRECTORY.
- OCR_OUTPUT_STRUCTURE (
- OCR_ON_CONFLICT (
--on-conflict) - What to do when the intended destination file already exists. One of
SUFFIX(default; writename (1).pdf,name (2).pdf, ... in the OS
style),SKIP(log and leave the file unprocessed), orOVERWRITE
(replace the existing file, the previous default behavior). Applies to
bothOCR_OUTPUT_DIRECTORYandOCR_ARCHIVE_DIRECTORY.
- OCR_ON_CONFLICT (
- OCR_OUTPUT_DIRECTORY_YEAR_MONTH
- Deprecated in favor of
OCR_OUTPUT_STRUCTURE=YEAR_MONTH. Still
honored, and logs a deprecation warning; if both variables are set,OCR_OUTPUT_STRUCTUREtakes precedence.
- OCR_DESKEW
- Apply deskew to crooked input PDFs
- OCR_JSON_SETTINGS
- A JSON string specifying any other arguments for
ocrmypdf.ocr, e.g.'OCR_JSON_SETTINGS={"rotate_pages": true, "optimize": "3"}'.
- OCR_POLL_NEW_FILE_SECONDS
- Polling interval
- OCR_LOGLEVEL
- Level of log messages (DEBUG, INFO, WARNING, ERROR, CRITICAL)
:::
One could configure a networked scanner or scanning computer to drop
files in the watched folder.
Output and archive filenames, and any directory components generated byOCR_OUTPUT_STRUCTURE, are sanitized for compatibility with the output
filesystem: characters that are illegal on Windows or SMB shares
(<>:"/\|?* and control characters) are replaced with _, trailing dots
and spaces are stripped, and reserved DOS device names (CON, PRN,AUX, NUL, COM1-COM9, LPT1-LPT9) are prefixed with _. This is
done unconditionally because the output directory may sit on a more
restrictive filesystem (e.g. an SMB share) than the input side, even when
the watcher itself runs on Linux or macOS.
:::{note}
If you write output to an SMB share and mount it on a Windows client, newly
written files may not appear immediately in Windows Explorer or dir. This
is the SMB client's own directory and file-info cache
(DirectoryCacheLifetime / FileInfoCacheLifetime, ~10 seconds by
default), not a delay introduced by the watcher. It can be tuned on the
client, e.g. with Set-SmbClientConfiguration; the watcher deliberately
does not attempt server-side workarounds for a client-side cache.
:::
Watcher security model
watcher.py treats the input, output and archive directories as data
only — they may be writable by less-privileged users, but code must never
live in or be executed from them (a "Harvard architecture"). To enforce this,
the watcher validates its configuration at startup and refuses to run
(exiting with code 9, invalid configuration) if any of these conditions is
violated:
Any of the input, output or archive directories overlaps a Python
interpreter path —sys.executable,sys.path, the virtual environment,
the site-packages directories, or any directory on$PATH. This prevents
an attacker who can write to a data directory from modifying the interpreter
or the code that runs.OCR_JSON_SETTINGSnames a file that lives inside a data directory, or that
is group- or world-writable. Keep the settings file outside the data
directories and restrict it to owner-only write access (e.g.chmod 600).OCR_JSON_SETTINGSspecifies a plugin (a.pypath) located inside a data
directory.The output or archive directory is the input directory or a subdirectory of
it. Because the input directory is watched recursively, OCRmyPDF output (and
archived originals) written there would be detected as new input and
reprocessed endlessly. Keep the output and archive directories outside the
input directory.
At runtime the watcher also refuses to follow symlinks or process non-regular
files (such as fifos or devices) in the input directory, and refuses to write
output onto a destination occupied by a non-regular file.
Watched folders with Docker
The watcher service is included in the OCRmyPDF Docker image. To run it:
:::{code} bash
docker run
--volume :/input
--volume :/output
--volume :/processed
--env OCR_OUTPUT_STRUCTURE=YEAR_MONTH
--env OCR_ON_SUCCESS_ARCHIVE=1
--env OCR_DESKEW=1
--env PYTHONUNBUFFERED=1
--interactive --tty --entrypoint python3
jbarlow83/ocrmypdf
/app/watcher.py
:::
:::{note}
The image runs as the non-root app user (uid 1000) by default, so it
may not be able to write to the /output and /processed volumes unless
you add a --user argument. The correct value depends on whether you use
rootful Docker, rootless Docker, or Podman -- see
{ref}Bind-mounted volumes <docker-volumes> for details.
:::
This service will watch for a file that matches /input/\*.pdf, convert
it to a OCRed PDF in /output/, and move the processed original to/processed. The parameters to this image are:
:::{list-table} Watcher Docker Parameters
:header-rows: 1
- Parameter
- Description
--volume :/input- Files placed in this location will be OCRed
--volume :/output- This is where OCRed files will be stored
--volume :/processed- Archive processed originals here
--env OCR_OUTPUT_STRUCTURE=YEAR_MONTH- Define environment variable
OCR_OUTPUT_STRUCTUREto lay out output
(and archived) files asFLAT(default),YEAR_MONTH
({destination}/{year}/{month}/{filename}), orHIERARCHY(mirrors
the input directory tree). Replaces the deprecatedOCR_OUTPUT_DIRECTORY_YEAR_MONTH=1.
--env OCR_ON_CONFLICT=SUFFIX- Define environment variable
OCR_ON_CONFLICTto control what happens
when a destination file already exists:SUFFIX(default, appends
" (1)", " (2)", ...),SKIP, orOVERWRITE
--env OCR_ON_SUCCESS_ARCHIVE=1- Define environment variable
OCR_ON_SUCCESS_ARCHIVEto move processed originals
--env OCR_DESKEW=1- Define environment variable
OCR_DESKEWto apply deskew to crooked input PDFs
--env PYTHONBUFFERED=1- This will force
STDOUTto be unbuffered and allow you to see messages in docker logs
--env OCR_LOGLEVEL='DEBUG'- Level of log messages
--env OCR_JSON_SETTINGS={"language":"deu+eng", "rotate_pages": true}- A JSON string specifying any other arguments for
ocrmypdf.ocr
:::
This service uses native operating system filesystem notifications to detect
changes. On some environments, such as filesystems shared on a slow network,
these notifications may be unreliable; set OCR_USE_POLLING=1 to fall back to
polling in that case.
A configuration manager such as Docker Compose could be used to ensure
that the service is always available.
:::{literalinclude} ../misc/docker-compose.example.yml
caption: misc/docker-compose.example.yml
:::
Caveats
watchmedomay not work properly on a networked file system,
depending on the capabilities of the file system client and server.- This simple recipe does not filter for the type of file system
event, so file copies, deletes and moves, and directory operations,
will all be sent to ocrmypdf, producing errors in several cases.
Disable your watched folder if you are doing anything other than
copying files to it. - If the source and destination directory are the same, watchmedo may
create an infinite loop. - On BSD, FreeBSD and older versions of macOS, you may need to
increase the number of file descriptors to monitor more files, usingulimit -n 1024to watch a folder of up to 1024 files.
Alternatives
- On Linux, systemd user
services can be
configured to automatically perform OCR on a collection of files. - Watchman is a more powerful
alternative towatchmedo.
macOS Automator
You can use the Automator app with macOS, to create a Workflow or Quick
Action. Use a Run Shell Script action in your workflow. In the context
of Automator, the PATH may be set differently your Terminal's PATH;
you may need to explicitly set the PATH to include ocrmypdf. The
following example may serve as a starting point:
You may customize the command sent to ocrmypdf.
Cloud (`docs/cloud.md`)
% SPDX-FileCopyrightText: 2025 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
(ocr-service)=
Online deployments
OCRmyPDF is designed to be used as a command line tool, but it can be
used in a web service. This document describes some considerations for
doing so.
A basic web service implementation is provided in the source code
repository, as misc/webservice.py. It is only demonstration quality
and is not intended for production use.
OCRmyPDF is not designed for use as a public web service where a
malicious user could upload a chosen PDF. In particular, it is not
necessarily secure against PDF malware or PDFs that cause denial of
service. For further discussino of security, see
security.
OCRmyPDF relies on Ghostscript, and therefore, if deployed online one
should be prepared to comply with Ghostscript's Affero GPL license, and
any other licenses.
Setting aside these concerns, a side effect of OCRmyPDF is that it may
incidentally sanitize PDFs containing certain types of malware. It
repairs the PDF with pikepdf/libqpdf, which could correct malformed PDF
structures that are part of an attack. When PDF/A output is selected
(the default), the input PDF is partially reconstructed by Ghostscript.
When --force-ocr is used, all pages are rasterized and reconverted to
PDF, which could remove malware in embedded images.
Limiting CPU usage
OCRmyPDF will attempt to use all available CPUs and storage, so
executing nice ocrmypdf or limiting the number of jobs with the--jobs argument may ensure the server remains responsive. Another
option would be to run OCRmyPDF jobs inside a Docker container, a
virtual machine, or a cloud instance, which can impose its own limits on
CPU usage and be terminated "from orbit" if it fails to complete.
Temporary storage requirements
OCRmyPDF will use a large amount of temporary storage for its work,
proportional to the total number of pixels needed to rasterize the PDF.
The raster image of a 8.5×11" color page at 300 DPI takes 25 MB
uncompressed; OCRmyPDF saves its intermediates as PNG, but that still
means it requires about 9 MB per intermediate based on average
compression ratios. Multiple intermediates per page are also required,
depending on the command line given. A rule of thumb would be to allow
100 MB of temporary storage per page in a file -- meaning that a small
cloud servers or small VM partitions should be provisioned with plenty
of extra space, if say, a 500 page file might be sent.
To change the temporary directory, see tmpdir.
On Amazon Web Services or other cloud vendors, consider setting your
temporary directory to empheral
storage.
Timeouts
To prevent excessively long OCR jobs consider setting--tesseract-timeout and/or --skip-big arguments. --skip-big is
particularly helpful if your PDFs include documents such as reports on
standard page sizes with large images attached - often large images are
not worth OCR'ing anyway.
Document management systems
If you are looking for a full document management system, consider
paperless-ngx, which
is a web application that uses OCRmyPDF to automatically OCR and archive
documents.
Commercial OCR alternatives
The author also provides professional services that include OCR and
building databases around PDFs, and is happy to provide consultation.
Abbyy Cloud OCR is viable commercial alternative with a web services
API. Amazon Textract, Google Cloud Vision, and Microsoft Azure Computer
Vision provide advanced OCR but have less PDF rendering capability.
Contributing (`docs/contributing.md`)
% SPDX-FileCopyrightText: 2025 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Contributing guidelines
Contributions are welcome!
Big changes
Please open a new issue to discuss or propose a major change. Not only
is it fun to discuss big ideas, but we might save each other's time
too. Perhaps some of the work you're contemplating is already half-done
in a development branch.
AI-assisted contributions 🤖
AI-assisted contributions are permitted. Proactive disclosure of AI use
is recommended but not required; it's usually not hard to tell.
When reporting issues with AI assistance, we encourage you to
stick to the facts from a user perspective. Use the issue template,
and provide a reproducer.
AI speculation about how to solve an issue is unhelpful. Leave it out
of your issue report. If you contribute a full PR, you're
likely to get a solution that gets the details right.
We will dismiss "vibe coded" PRs that don't appear to understand the
project.
Code style
We use ruff for code formatting.
The settings for these programs are in pyproject.toml. Pull requests
should follow the style guide. One difference we use from "black"
style is that strings shown to the user are always in double quotes
(") and strings for internal uses are in single quotes (').
Tests
New features should come with tests that confirm their correctness.
Tesseract OCR cache
Running Tesseract on every test would make the test suite far too slow, so
the results of most Tesseract invocations are recorded in tests/cache/
and replayed by the tests/plugins/tesseract_cache.py plugin. The cache is
committed to the repository, so a contributor with a different (or no)
Tesseract still gets the same OCR output.
Entries live attests/cache/v<CACHE_VERSION>/<key>/<argv-slug>/{stdout,stderr,hocr,pdf,txt}.bin.
The key for a file in tests/resources/ is its stem plus the first 8 hex
digits of the SHA-256 of its contents; inputs generated during a test run
are keyed by stem alone, because their bytes vary from run to run.CACHE_INFO.json records the Tesseract version that produced the tree andmanifest.jsonl logs each entry as it is created. Neither is touched on a
cache hit, so a normal test run leaves the working tree clean.
There are three ways the cache can go stale, and only one of them needs
your attention:
- A test resource changed. Nothing to do. The digest in the cache key
changes with the file, so affected entries are regenerated automatically
and the old ones can be deleted.tests/test_tesseract_cache.pyfails if
a digest-keyed folder no longer matches any resource, which is your cue
to delete the orphaned folder. - OCRmyPDF changed the images it sends to Tesseract (preprocessing,
rasterization, DPI selection, image optimization on the OCR path). The
cached answers now belong to different questions, and the cache would
silently replay them. BumpCACHE_VERSIONintests/plugins/tesseract_cache.py, delete the oldtests/cache/v<N>/
tree, and regenerate. - Tesseract itself changed.
pytest_sessionstartintests/conftest.py
compares the installed Tesseract's major.minor version againstCACHE_INFO.jsonand warns on a mismatch. SetOCRMYPDF_TEST_CACHE_STRICT=1to turn that warning into an immediate
session abort, which is useful in CI. A mismatch is not automatically a
problem — it means the cache no longer reflects what your Tesseract would
produce, so regenerate before trusting or updating it.
To regenerate the cache:
rm -rf tests/cache/v1 # or whatever the current CACHE_VERSION is
pytest # with the reference Tesseract installed
pytest tests/test_tesseract_cache.py # integrity check must pass
git add tests/cache
git commit -m "Regenerate Tesseract cache (tesseract 5.5.1)"Always name the Tesseract version in the commit message, and regenerate the
whole tree with one Tesseract version rather than mixing versions.
PDF/A validation
Speculative PDF/A conversion (see {doc}advanced) is validated by
pikepdf's PDF/A support, pikepdf.pdfa. Problems with the validator's
verdicts, and its differential testing against
veraPDF (misc/pdfa_differential.py), belong to
pikepdf. OCRmyPDF's tests of the PDF/A
pipeline cross-check their output with veraPDF when it is installed.
New dependencies
If you are proposing a change that will require a new dependency, we
prefer dependencies that are already packaged by Debian or Red Hat. This
makes life much easier for our downstream package maintainers. A package
that is only available on PyPI or GitHub, and not more widely packaged,
may not be accepted.
We are unlikely to accept a dependency on CUDA or other GPU-based
libraries, because these are still difficult to package and install on
many systems. We recommend implementing these changes as plugins.
Python dependencies must also be license-compatible. GPLv3 or AGPLv3 are
likely incompatible with the project's license, but LGPLv3 is
compatible.
New non-Python dependencies
OCRmyPDF uses several external programs (Tesseract, Ghostscript and
others) for its functionality. In general we prefer to avoid adding new
external programs, and if we are to add external programs, we prefer
those that are already packaged by Debian or Red Hat.
Plugins
Some new features may be a good fit for a plugin. Plugins are a way to
add features to OCRmyPDF without adding them to the core program.
Plugins are installed separately from OCRmyPDF. They are written in
Python and can be installed from PyPI. See the plugin
documentation.
We are happy to link users to your plugin from the documentation.
Style guide: Is it OCRmyPDF or ocrmypdf?
The program/project is OCRmyPDF and the name of the executable or
library is ocrmypdf.
Copyright and license
For contributions over 10 lines of code, please add your name to list of
copyright holders for that file. The core program is licensed under
MPL-2.0, test files and documentation under CC-BY-SA 4.0, and
miscellaneous files under MIT, with a few minor exceptions. Please
contribute only content that you own or have the right to contribute
under these licenses.
Cookbook (`docs/cookbook.md`)
% SPDX-FileCopyrightText: 2025 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Cookbook
Basic examples
Help!
ocrmypdf has built-in help.
ocrmypdf --helpAdd an OCR layer and convert to PDF/A
ocrmypdf input.pdf output.pdfAdd an OCR layer and output a standard PDF
ocrmypdf --output-type pdf input.pdf output.pdfCreate a PDF/A with all color and grayscale images converted to JPEG
ocrmypdf --output-type pdfa --pdfa-image-compression jpeg input.pdf output.pdfReduce JPEG quality with the optimizer
This is the recommended way to shrink JPEG content in the output. The
optimizer applies regardless of --output-type, so it works on both
plain PDFs and Ghostscript-produced PDF/A files.
ocrmypdf --optimize 2 --jpeg-quality 60 input.pdf output.pdfModify a file in place
The file will only be overwritten if OCRmyPDF is successful.
ocrmypdf myfile.pdf myfile.pdfCorrect page rotation
OCR will attempt to automatic correct the rotation of each page. This
can help fix a scanning job that contains a mix of landscape and
portrait pages.
ocrmypdf --rotate-pages myfile.pdf myfile.pdfYou can increase (decrease) the parameter --rotate-pages-threshold to
make page rotation more (less) aggressive. The threshold number is the
ratio of how confidence the OCR engine is that the document image should
be changed, compared to kept the same. The default value is quite
conservative; on some files it may not attempt rotations at all unless
it is very confident that the current rotation is wrong. A lower value
of 2.0 will produce more rotations, and more false positives. Run with-v1 to see the confidence level for each page to see if there may be a
better value for your files.
If the page is "just a little off horizontal", like a crooked picture,
then you want --deskew. --rotate-pages is for when the cardinal
angle is wrong.
OCR languages other than English
OCRmyPDF assumes the document is in English unless told otherwise. OCR
quality may be poor if the wrong language is used.
ocrmypdf -l fra LeParisien.pdf LeParisien.pdf
ocrmypdf -l eng+fra Bilingual-English-French.pdf Bilingual-English-French.pdfLanguage packs must be installed for all languages specified. SeeInstalling additional language packs <lang-packs>{.interpreted-text
role="ref"}.
Unfortunately, the Tesseract OCR engine has no ability to detect the
language when it is unknown.
Produce PDF and text file containing OCR text
This produces a file named "output.pdf" and a companion text file
named "output.txt".
ocrmypdf --sidecar output.txt input.pdf output.pdf:::{note}
The sidecar file contains the OCR text found by OCRmyPDF. If the
document contains pages that already have text, that text will not
appear in the sidecar. If the option --pages is used, only those pages
on which OCR was performed will be included in the sidecar. If certain
pages were skipped because of options like --skip-big or--tesseract-timeout, those pages will not be in the sidecar.
If you don't want to generate the output PDF, use --output-type=none
to avoid generating one. Set the output filename to - (i.e. redirect
to stdout).
To extract all text from a PDF, whether generated from OCR or otherwise,
use a program like Poppler's pdftotext or pdfgrep.
:::
OCR images, not PDFs
Option: use Tesseract
If you are starting with images, you can just use Tesseract directly to
convert images to PDFs:
tesseract my-image.jpg output-prefix pdf# When there are multiple images
tesseract text-file-containing-list-of-image-filenames.txt output-prefix pdfTesseract's PDF output is quite good -- OCRmyPDF uses it internally, in
some cases. However, OCRmyPDF has many features not available in
Tesseract like image processing, metadata control, and PDF/A generation.
Option: use img2pdf
You can also use a program like
img2pdf to convert your
images to PDFs, and then pipe the results to run ocrmypdf. The - tells
ocrmypdf to read standard input.
img2pdf my-images*.jpg | ocrmypdf - myfile.pdfimg2pdf is recommended because it does an excellent job at generating
PDFs without transcoding images.
Option: use OCRmyPDF (single images only)
For convenience, OCRmyPDF can also convert single images to PDFs on its
own. If the resolution (dots per inch, DPI) of an image is not set or is
incorrect, it can be overridden with --image-dpi. (As 1 inch is 2.54
cm, 1 dpi = 0.39 dpcm).
ocrmypdf --image-dpi 300 image.png myfile.pdfIf you have multiple images, you must use img2pdf to convert the
images to PDF.
Not recommended
We caution against using ImageMagick or Ghostscript to convert images to
PDF, since they may transcode images or produce downsampled images,
sometimes without warning.
(image-processing)=
Image processing
OCRmyPDF perform some image processing on each page of a PDF, if
desired. The same processing is applied to each page. It is suggested
that the user review files after image processing as these commands
might remove desirable content, especially from poor quality scans.
--rotate-pagesattempts to determine the correct orientation for
each page and rotates the page if necessary.--remove-backgroundattempts to detect and remove a noisy
background from grayscale or color images. Monochrome images are
ignored. This should not be used on documents that contain color
photos as it may remove them.--deskewwill correct pages that were scanned at a skewed angle by
rotating them back into place.--cleanuses unpaper
to clean up pages before OCR, but does not alter the final output.
This makes it less likely that OCR will try to find text in
background noise.--clean-finaluses unpaper to clean up pages before OCR and
inserts the page into the final output. You will want to review each
page to ensure that unpaper did not remove something important.
:::{note}
In many cases image processing will rasterize PDF pages as images,
potentially losing quality.
:::
:::{warning}--clean-final and --remove-background may leave undesirable visual
artifacts in some images where their algorithms have shortcomings. Files
should be visually reviewed after using these options.
:::
Example: OCR and correct document skew (crooked scan)
Deskew:
ocrmypdf --deskew input.pdf output.pdfImage processing commands can be combined. The order in which options
are given does not matter. OCRmyPDF always applies the steps of the
image processing pipeline in the same order (rotate, remove background,
deskew, clean).
ocrmypdf --deskew --clean --rotate-pages input.pdf output.pdfDon\'t actually OCR my PDF
If you set --ocr-engine none OCRmyPDF will skip OCR entirely. This works if
all you want is PDF/A conversion or image processing such as --clean.
ocrmypdf --ocr-engine none --output-type pdfa input.pdf output.pdf--ocr-engine none cannot detect page orientation or skew, so--rotate-pages and --deskew have no effect with it. To rotate or deskew
pages without adding a text layer, use --tesseract-timeout 0 instead.
Tesseract will still be used to measure orientation and skew, but will not
OCR the page.
ocrmypdf --tesseract-timeout 0 --rotate-pages --deskew input.pdf output.pdf:::{versionchanged} v17.0.0
Prior to this version, --tesseract-timeout 0 was recommended as the general
idiom to turn off OCR. --ocr-engine none is now preferred, except when
rotating or deskewing pages.
:::
:::{versionchanged} v14.1.0
Prior to this version, --tesseract-timeout 0 would prevent other uses
of Tesseract, such as deskewing, from working. This is no longer the
case. Use --tesseract-non-ocr-timeout to control the timeout for
non-OCR operations, if needed.
:::
Remove the OCR text layer from my PDF
To remove the invisible OCR text layer while keeping the original pages
exactly as they are -- no rasterizing, no change to images or visible
content, and a smaller output file -- use --mode strip:
ocrmypdf --mode strip input.pdf output.pdfWhy would you want to do this? Perhaps you have a PDF where OCR failed to
produce useful results and you simply want to get rid of it.
--mode strip removes only text drawn as invisible (PDF text render
mode 3), which is how OCRmyPDF and most OCR tools add a searchable layer
over a scanned page. Some OCR products -- and OCRmyPDF v2.2 and earlier --
instead draw visible text and paint an opaque image on top of it. That
text is part of the visible page, so --mode strip cannot remove it
without altering the page's appearance.
To strip all text, including such visible text, rasterize the whole page
into a "bag of images" PDF instead (this rebuilds every page as an image,
so the file usually grows and vector content is lost):
ocrmypdf --ocr-engine none --force-ocr input.pdf output.pdfOptimize images without performing OCR
You can also optimize all images without performing any OCR:
ocrmypdf --ocr-engine none --optimize 3 --skip-text input.pdf output.pdfUsing v17 features
Select a rasterizer
:::{versionadded} 17.0.0
:::
OCRmyPDF can use pypdfium2 or Ghostscript to rasterize PDF pages. pypdfium2
is generally faster and is preferred when available.
# Automatic selection (default) - prefers pypdfium when available
ocrmypdf --rasterizer auto input.pdf output.pdf
# Explicitly use pypdfium2 (requires pip install pypdfium2)
ocrmypdf --rasterizer pypdfium input.pdf output.pdf
# Explicitly use Ghostscript
ocrmypdf --rasterizer ghostscript input.pdf output.pdfPDF/A without Ghostscript
:::{versionadded} 17.0.0
:::
OCRmyPDF can often produce PDF/A without using Ghostscript for conversion:
it adds the PDF/A structures itself and checks the result with pikepdf's
PDF/A validator, and only uses Ghostscript for files that fail the check.
This is faster and avoids some Ghostscript limitations.
:::{versionchanged} 17.13.0
The check is made by pikepdf's PDF/A validator (pikepdf.pdfa); veraPDF is
no longer needed.
:::
# Best-effort PDF/A: speculative conversion, then Ghostscript, then plain PDF
ocrmypdf --output-type auto input.pdf output.pdf
# Required PDF/A: speculative conversion, then Ghostscript
ocrmypdf --output-type pdfa input.pdf output.pdf
# Required PDF/A without Ghostscript: fail if the validator does not approve
ocrmypdf --output-type pdfa --pdfa-backend internal input.pdf output.pdf
# Always convert with Ghostscript
ocrmypdf --output-type pdfa --pdfa-backend ghostscript input.pdf output.pdf
# A specific image compression also needs Ghostscript, and selects it
ocrmypdf --output-type pdfa --pdfa-image-compression lossless input.pdf output.pdfUsing --mode instead of legacy flags
:::{versionadded} 17.0.0
:::
The --mode (-m) flag consolidates OCR behavior options:
# Instead of --skip-text
ocrmypdf --mode skip input.pdf output.pdf
# Instead of --force-ocr
ocrmypdf --mode force input.pdf output.pdf
# Instead of --redo-ocr
ocrmypdf --mode redo input.pdf output.pdf
# Short form
ocrmypdf -m skip input.pdf output.pdfThe legacy flags continue to work as aliases.
Process only certain pages
You can ask OCRmyPDF to only apply image processing
and OCR to certain pages.
ocrmypdf --pages 2,3,13-17 input.pdf output.pdfHyphens denote a range of pages and commas separate page numbers. If you
prefer to use spaces, quote all of the page numbers:--pages '2, 3, 5, 7'.
The token end (case-insensitive) is an alias for the last page in the
document. For example, --pages 3-end OCRs from page 3 through the
final page, and --pages end OCRs only the last page:
ocrmypdf --pages 3-end input.pdf output.pdf
ocrmypdf --pages end input.pdf output.pdfOCRmyPDF will warn if your list of page numbers contains duplicates or
overlapping pages. (Repeated page numbers are de-duplicated automatically,
since the underlying set of pages is what matters.) OCRmyPDF does not
currently account for document page numbers, such as an introduction
section of a book that uses Roman numerals. It simply counts the number
of virtual pieces of paper since the start. If your list of pages is out
of numerical order, OCRmyPDF will sort it for you.
Regardless of the argument to --pages, OCRmyPDF will optimize all
pages/images in the file and convert it to PDF/A, unless you disable
those options. Both of these steps are "whole file" operations. In
this example, we want to OCR only the title and otherwise change the PDF
as little as possible:
ocrmypdf --pages 1 --output-type pdf --optimize 0 input.pdf output.pdfRedo existing OCR
To redo OCR on a file OCRed with other OCR software or a previous
version of OCRmyPDF and/or Tesseract, you may use the --redo-ocr
argument. (Normally, OCRmyPDF will exit with an error if asked to modify
a file with OCR.)
This may be helpful for users who want to take advantage of accuracy
improvements in Tesseract for files they previously OCRed with an
earlier version of Tesseract and OCRmyPDF.
ocrmypdf --redo-ocr input.pdf output.pdfThis method will replace OCR without rasterizing, reducing quality or
removing vector content. If a file contains a mix of pure digital text
and OCR, digital text will be ignored and OCR will be replaced. As such
this mode is incompatible with image processing options, since they
alter the appearance of the file.
In some cases, existing OCR cannot be detected or replaced. Files
produced by OCRmyPDF v2.2 or earlier, for example, are internally
represented as having visible text with an opaque image drawn on top.
This situation cannot be detected.
If --redo-ocr does not work, you can use --force-ocr, which will
force rasterization of all pages, potentially reducing quality or losing
vector content.
Improving OCR quality
The Image processing features can improve OCR
quality.
Rotating pages and deskewing helps to ensure that the page orientation
is correct before OCR begins. Removing the background and/or cleaning
the page can also improve results. The --oversample DPI argument can
be specified to resample images to higher resolution before attempting
OCR; this can improve results as well.
OCR quality will suffer if the resolution of input images is not correct
(since the range of pixel sizes that will be checked for possible fonts
will also be incorrect).
PDF optimization
By default OCRmyPDF will attempt to perform lossless optimizations on
the images inside PDFs after OCR is complete. Optimization is performed
even if no OCR text is found.
The --optimize N (short form -O) argument controls optimization,
where N ranges from 0 to 3 inclusive, analogous to the optimization
levels in the GCC compiler. -O1 is the default.
For further details, see the section on PDF optimization.
ocrmypdf --optimize 3 in.pdf out.pdf # Make it smallSome users may consider enabling lossy JBIG2. See:jbig2-lossy{.interpreted-text role="ref"}.
:::{note}
Image processing and PDF/A conversion can also introduce lossy
transformations to your PDF images, even when --optimize 1 is in use.
:::
Digitally signed PDFs
OCRmyPDF cannot preserve digital signatures in PDFs and also add OCR to
them. By default, it will refuse to modify a signed PDF regardless of
other settings. You can override this behavior with--invalidate-digital-signatures; as the name suggests, any digital
signatures will be invalidated.
OCRmyPDF cannot open documents that are encrypted with a digital
certificate.
Versions of OCRmyPDF prior to 14.4.0 would invalidate existing digital
signatures without warning.
Design Notes (`docs/design_notes.md`)
% SPDX-FileCopyrightText: 2023 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Design notes
Why doesn\'t OCRmyPDF use PyTesseract?
PyTesseract is a Python wrapper around the Tesseract OCR engine. When
OCRmyPDF was first written, PyTesseract used ABI bindings to call the
Tesseract library. This was not a good fit for OCRmyPDF because ABI
bindings can be fragile.
PyTesseract has since evolved calling the Tesseract executable,
abandoning the ABI approach and using the CLI instead, just like
OCRmyPDF does. If it were written from scratch today, OCRmyPDF might use
PyTesseract.
PyTesseract has more features don't particularly need PDF output, but
less features than OCRmyPDF's API for creating PDFs.
What is `executor()`?
OCRmyPDF uses a custom concurrent executor which can support either
threads or processes with the same interface. This is useful because
OCRmyPDF can use either threads or processes to parallelize work,
whichever is more appropriate for the task at hand.
The interface is currently private and subject to change. In particular,
if experiments with asyncio and anyio are successful, the interface will
change.
Docker (`docs/docker.md`)
OCRmyPDF Docker image {#docker}
OCRmyPDF is also available in Docker images that packages recent
versions of all dependencies.
For users who already have Docker installed this may be an easy and
convenient option.
On platforms other than Linux, Docker runs in a virtual machine, and so
may be less performant. You may also want to adjust the Docker virtual
machine's memory and CPU allocation. On Linux, the Docker image runs
natively and performance is comparable to a system installation.
{#docker-install}
Installing the Docker image
If you have Docker installed on your system,
you can install a Docker image of the latest release.
If you can run this command successfully, your system is ready to
download and execute the image:
:::{code} bash
docker run hello-world
:::
:::{list-table} Docker Images
:header-rows: 1
- Image
- Architecture
- Description
jbarlow83/ocrmypdf-alpine- x86_64 and arm64
- Recommended image, based on Alpine Linux.
jbarlow83/ocrmypdf-ubuntu- x86_64 and arm64
- Alternate image, based on Ubuntu. When the Alpine image is considered stable and available for arm64, this image will be deprecated.
jbarlow83/ocrmypdf- x86_64 and arm64
- Currently an alias for ocrmypdf-ubuntu. When the Alpine image is considered stable and available for arm64, this name will point to the Alpine image. If you don't know about the difference between Alpine and Ubuntu, use this image.
:::
To install:
:::{code} bash
docker pull jbarlow83/ocrmypdf-alpine
:::
The ocrmypdf image is also available, but is deprecated and will be
removed in the future.
OCRmyPDF will use all available CPU cores. See the Docker documentation
for adjusting memory and CPU on other
platforms
if you are using Docker on macOS or Windows, where you may need to
manually assign more resources. On Linux, all resources will be
available automatically.
The underlying operating system and other details in Docker images are
considered implementation details and subject to change at minor
releases. If you are modifying the image, you should pin the version
you intend to use.
Using the Docker image on the command line
Unlike typical Docker containers, in this section the OCRmyPDF
Docker container is ephemeral -- it runs for one OCR job and terminates,
just like a command line program. We are using Docker to deliver an
application (as opposed to the more conventional case, where a Docker
container runs as a server). For that reason we usually use the --rm
argument to delete the container when it exits.
:::{note}
The image runs as a non-root user (app, uid/gid 1000) by default,
rather than as root. This is a defense-in-depth measure: a flaw in
OCRmyPDF or one of its dependencies cannot trivially act as root inside
the container. The examples below assume rootless Docker or
Podman; the differences for traditional rootful Docker are
described separately under Special case: rootful Docker below.
:::
To start a Docker container (instance of the image):
:::{code} bash
docker run --rm -i jbarlow83/ocrmypdf-alpine (... all other arguments here...) - -
:::
Recommended: pipe through stdin and stdout
The easiest and most portable way to use the image is to send the input
file on stdin and read the output from stdout. This avoids file
permission issues entirely -- nothing is written to a mounted
directory, so it does not matter which user the container runs as, nor
whether you use rootless or rootful Docker. For convenience, create a
shell alias to hide the Docker command:
:::{code} bash
alias docker_ocrmypdf='docker run --rm -i jbarlow83/ocrmypdf-alpine'
docker_ocrmypdf --version # runs docker version
docker_ocrmypdf - - output.pdf
:::
Or in the wonderful fish shell:
:::{code} fish
alias docker_ocrmypdf 'docker run --rm -i jbarlow83/ocrmypdf-alpine'
funcsave docker_ocrmypdf
:::
{#docker-volumes}
Bind-mounted volumes
If you would rather mount a directory and pass file paths, you need to
consider which user owns the files OCRmyPDF writes back into that
directory. The image's default working directory is /data, so mounting
your files there lets you pass plain relative paths without an explicit--workdir. Because the container runs as the non-root app user, the
right invocation otherwise depends on your container runtime.
Rootless Docker (the assumed default). Your own account runs the
daemon, so the container's root maps back to your unprivileged host
user, while every other container uid -- including the image's defaultapp/1000 -- maps to a subordinate uid. A directory you own on the
host therefore appears owned by root inside the container, so the
default app user usually cannot write to it at all. Run the job as
container-root, which under rootless Docker is still your ordinary host
user, so the write succeeds and the output is owned by you:
:::{code} bash
alias docker_ocrmypdf='docker run --rm -i --user 0:0 -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
docker_ocrmypdf input.pdf output.pdf
:::
Podman. Podman provides --userns keep-id, which maps your host uid
straight through into the container. Combined with --user, you run as
your own uid and own the output directly, otherwise you may get access
errors because the user ID is not mapped to the same UID as on the host:
:::{code} bash
alias podman_ocrmypdf='podman run --rm -i --user "$(id -u):$(id -g)" --userns keep-id -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
podman_ocrmypdf input.pdf output.pdf
:::
If you have SELinux enabled, you may additionally need to add the :Z suffix to
the
volume
or disable SELinux for the container using--security-opt label=disable, which is suggested for system files as
they should not be re-labelled. Please refer to the „Note" section at
the end of the linked podman documentation for details. This results in
the following full command:
:::{code} bash
alias podman_ocrmypdf='podman run --rm -i --user "$(id -u):$(id -g)" --userns keep-id -v "$PWD:/data" --security-opt label=disable jbarlow83/ocrmypdf-alpine'
podman_ocrmypdf input.pdf output.pdf
:::
{#docker-rootful}
Special case: rootful Docker
With a traditional root daemon, container uid N is the same uid N
on the host. Running the container as root would therefore fill your
mounted directory with root-owned files and -- more importantly -- a
container escape would run as real host root. Drop to your own uid so the
output is owned by you and the process stays unprivileged:
:::{code} bash
alias docker_ocrmypdf='docker run --rm -i --user "$(id -u):$(id -g)" -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
docker_ocrmypdf input.pdf output.pdf
:::
The non-root default and the --user override both reduce the risk here,
but rootless Docker or Podman remain the safer choice when available.
{#docker-lang-packs}
Adding languages to the Docker image
By default the Docker image includes English, German, Simplified
Chinese, French, Portuguese and Spanish, the most popular languages for
OCRmyPDF users based on feedback. You may add other languages by
creating a new Dockerfile based on the public one.
:::{code} dockerfile
FROM jbarlow83/ocrmypdf
The image runs as the non-root "app" user, so switch back to root for
build steps that install packages, then drop back to "app".
USER root
Example: add Italian
RUN apt-get update && apt-get install -y tesseract-ocr-ita
USER app
:::
To install language packs (training data) such as the
tessdata_best suite
or custom data, you first need to determine the version of Tesseract
data files, which may differ from the Tesseract program version. Use
this command to determine the data file version:
:::{code} bash
docker run -i --rm --entrypoint /bin/ls jbarlow83/ocrmypdf /usr/share/tesseract-ocr
:::
As of 2021, the data file version is probably 4.00.
You can then add new data with either a Dockerfile:
:::{code} dockerfile
FROM jbarlow83/ocrmypdf:{TAG}
Example: add a tessdata_best file
COPY chi_tra_vert.traineddata /usr/share/tesseract-ocr//tessdata/
:::
When creating your own image, you should always pin a specific version
of the OCRmyPDF Docker image. This ensures that your image will not
break when a new version of OCRmyPDF is released.
Alternately, you can copy training data into a Docker container as
follows:
:::{code} bash
docker cp mycustomtraining.traineddata name_of_container:/usr/share/tesseract-ocr/
:::
Extending the Docker image
You can extend the Docker image with your own customizations, similar to
the way it is extended to add language packs. Because the image runs as
the non-root app user, switch to USER root for any build steps that
require root (installing packages, writing to system directories) and
back to USER app afterwards, as shown in the language pack example
above.
Note that the Docker image is subject to change at any time. For
example, the base image may be updated to a newer version of Ubuntu or
Debian. Such changes will be noted in the release notes but might occur
at minor versions releases, unless the way a "casual" user of the
Docker image is affected.
If you extend the Docker image, you should pin a specific version of the
OCRmyPDF Docker image.
Executing the test suite
The OCRmyPDF test suite is installed with image. To run it:
:::{code} bash
docker run --rm --workdir /app --entrypoint python jbarlow83/ocrmypdf -m pytest
:::
Accessing the shell
To use the shell in the Docker image:
:::{code} bash
docker run -it --entrypoint sh jbarlow83/ocrmypdf-alpine
:::
This shell runs as the non-root app user. If you need root inside the
container -- for example to install extra packages with apk or apt --
add --user root:
:::{code} bash
docker run -it --user root --entrypoint sh jbarlow83/ocrmypdf-alpine
:::
Using the OCRmyPDF web service wrapper
The OCRmyPDF Docker image includes an example, barebones web service
built with Streamlit. It may be launched as follows:
:::{code} bash
docker run --entrypoint python -p 8501:8501 jbarlow83/ocrmypdf /app/webservice.py
:::
Then open http://localhost:8501 in a web browser. Streamlit listens on
port 8501 by default; any arguments after /app/webservice.py are passed
to streamlit run, so to use a different port:
:::{code} bash
docker run --entrypoint python -p 5000:5000 jbarlow83/ocrmypdf
/app/webservice.py --server.port 5000
:::
An equivalent Docker Compose service:
:::{code} yaml
services:
ocrmypdf:
image: jbarlow83/ocrmypdf
entrypoint: ["python", "/app/webservice.py"]
ports:
- "8501:8501"
:::
Unlike command line usage this program will open a socket and wait for
connections.
:::{warning}
The OCRmyPDF web service wrapper is intended for demonstration or
development. It provides no security, no authentication, no protection
against denial of service attacks, and no load balancing. It processes
one file at a time for each browser session.
:::
Unlike the rest of OCRmyPDF, this web service is licensed under the
Affero GPLv3 (AGPLv3) since Ghostscript is also licensed in this way.
In addition to the above, please read ourgeneral remarks on using OCRmyPDF as a service <ocr-service>{.interpreted-text
role="ref"}.
Errors (`docs/errors.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Common error messages
Page already has text
:::{code}
ERROR - 1: page already has text! – aborting (use --force-ocr to force OCR)
:::
You ran ocrmypdf on a file that already contains printable text or a
hidden OCR text layer (it can't quite tell the difference). You
probably don't want to do this, because the file is already searchable.
As the error message suggests, your options are:
ocrmypdf --force-ocrtorasterize <raster-vector>{.interpreted-text role="ref"} all vector
content and run OCR on the images. This is useful if a previous OCR
program failed, or if the document contains a text watermark.ocrmypdf --skip-textto skip OCR and other processing on any pages
that contain text. Text pages will be copied into the output PDF
without modification.ocrmypdf --redo-ocrto scan the file for any existing OCR
(non-printing text), remove it, and do OCR again. This is one way to
take advantage of improvements in OCR accuracy. Printable vector
text is excluded from OCR, so this can be used on files that contain
a mix of digital and scanned files.
Input file \'filename\' is not a valid PDF
OCRmyPDF checks files with pikepdf, a library that in turn uses libqpdf
to fixes errors in PDFs, before it tries to work on them. In most cases
this happens because the PDF is corrupt and truncated (incomplete file
copying) and not much can be done.
You can try rewriting the file with Ghostscript:
:::{code} bash
gs -o output.pdf -dSAFER -sDEVICE=pdfwrite input.pdf
:::
pdftk can also rewrite PDFs:
:::{code} bash
pdftk input.pdf cat output output.pdf
:::
Sometimes Acrobat can repair PDFs with its Preflight
tool.
(tesseract-config-missing)=
Tesseract cannot open its config file \'hocr\' or \'txt\'
:::{code}
ERROR - Tesseract cannot open its config file 'hocr'.
:::
OCRmyPDF asks Tesseract to produce hocr and txt output. Tesseract
reads the instructions for these output formats from configuration files
named hocr and txt that live in the configs/ subdirectory of itstessdata folder. If those files are missing, Tesseract printsread_params_file: Can't open hocr, exits without error, and produces no
output.
This usually happens when a tessdata directory was assembled by hand --
for example, by downloading individual .traineddata files from
tessdata_best and
pointing TESSDATA_PREFIX at them -- because those repositories do not
include the configs/ directory. A complete Tesseract installation from
your operating system's package manager includes it.
To fix this, ensure the configs/hocr and configs/txt files exist in
the tessdata directory that Tesseract is using. Copying the configs/
directory from a full Tesseract installation is sufficient. See
{envvar}TESSDATA_PREFIX for more on selecting an alternate tessdata
folder.
Installation (`docs/installation.md`)
myst:
substitutions:
deb_12: |-
:::{image} https://repology.org/badge/version-for-repo/debian_12/ocrmypdf.svg
:alt: Debian 12
:::
deb_13: |-
:::{image} https://repology.org/badge/version-for-repo/debian_13/ocrmypdf.svg
:alt: Debian 13
:::
deb_unstable: |-
:::{image} https://repology.org/badge/version-for-repo/debian_unstable/ocrmypdf.svg
:alt: Debian unstable
:::
fedora_43: |-
:::{image} https://repology.org/badge/version-for-repo/fedora_43/ocrmypdf.svg
:alt: Fedora 43
:::
fedora_44: |-
:::{image} https://repology.org/badge/version-for-repo/fedora_44/ocrmypdf.svg
:alt: Fedora 44
:::
fedora_rawhide: |-
:::{image} https://repology.org/badge/version-for-repo/fedora_rawhide/ocrmypdf.svg
:alt: Fedora Rawhide
:::
latest: |-
:::{image} https://img.shields.io/pypi/v/ocrmypdf.svg
:alt: OCRmyPDF latest released version on PyPI
:::
ubu_2204: |-
:::{image} https://repology.org/badge/version-for-repo/ubuntu_22_04/ocrmypdf.svg
:alt: Ubuntu 22.04 LTS
:::
ubu_2404: |-
:::{image} https://repology.org/badge/version-for-repo/ubuntu_24_04/ocrmypdf.svg
:alt: Ubuntu 24.04 LTS
:::
ubu_2604: |-
:::{image} https://repology.org/badge/version-for-repo/ubuntu_26_04/ocrmypdf.svg
:alt: Ubuntu 26.04 LTS
:::
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Installing OCRmyPDF
(latest)=
The easiest way to install OCRmyPDF is to follow the steps for your operating
system/platform. This version may be out of date, however.
These platforms have one-liner installs:
:::{list-table}
:header-rows: 0
- Homebrew (macOS and Linux)
brew install ocrmypdf
- Debian, Ubuntu
apt install ocrmypdf
- Windows Subsystem for Linux
apt install ocrmypdf
- Fedora
dnf install ocrmypdf tesseract-osd
- Alpine Linux
apk add ocrmypdf
- macOS (MacPorts)
port install ocrmypdf
- FreeBSD
pkg install py311-ocrmypdf
:::
More detailed procedures are outlined below. If you want to do a manual
install, or install a more recent version than your platform provides, read on.
If your operating system is older than those listed here, we recommend
installing OCRmyPDF with Homebrew, or using the
Docker image.
:::{contents} Platform-specific steps
:depth: 2
:local: true
:::
Installing on Linux
Debian 12 or newer and Ubuntu 22.04 or newer
:::{list-table}
:header-rows: 1
- OCRmyPDF versions in Debian & Ubuntu
- {{ latest }}
- {{ deb_12 }} {{ deb_13 }} {{ deb_unstable }}
- {{ ubu_2204 }} {{ ubu_2404 }} {{ ubu_2604 }}
:::
- {{ ubu_2204 }} {{ ubu_2404 }} {{ ubu_2604 }}
Users of Debian or Ubuntu may simply
apt install ocrmypdfAs indicated in the table above, Debian and Ubuntu releases may lag
behind the latest version, sometimes by several major versions. If the version
available for your platform is out of date, see
Installing the latest version on Debian and Ubuntu.
For full details on version availability for your platform, check the
Debian Package Tracker or
Ubuntu launchpad.net.
:::{note}
The Debian and Ubuntu OCRmyPDF packages do not install the JBIG2 encoder.
OCRmyPDF works fine without it but will produce larger output files.
On Debian 13 or newer and Ubuntu 24.04 or newer, the encoder is packaged and
can be installed with apt install jbig2. On older releases, you can build
jbig2enc from source; OCRmyPDF will automatically detect it on the PATH.
See {ref}jbig2.
:::
(ubuntu-lts-latest)=
Installing the latest version on Debian and Ubuntu
Debian and Ubuntu include an older version of OCRmyPDF - you can install that
with apt install ocrmypdf. To install the latest version, we recommend
installing the system package first, which brings in Tesseract, Ghostscript
and other system dependencies, and then using
uv to install the latest OCRmyPDF from PyPI:
# Install system dependencies first
sudo apt-get update
sudo apt-get -y install ocrmypdf
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install the latest OCRmyPDF
uv tool install ocrmypdfuv installs the ocrmypdf command in ~/.local/bin. Open a new terminal
(or run uv tool update-shell) so that it is on your PATH, and then runocrmypdf --version to confirm that the latest version is used rather than the
older system version in /usr/bin.
If your system Python is too old for the latest OCRmyPDF, as on Ubuntu 22.04,
uv will download a suitable version of Python automatically.
To upgrade later:
uv tool upgrade ocrmypdf:::{tip}
Ubuntu 22.04 ships Tesseract 4.1.1, which is the oldest version OCRmyPDF
supports. Tesseract 5 gives better results; it is available for Ubuntu LTS
releases from the
alex-p/tesseract-ocr5 PPA:sudo add-apt-repository ppa:alex-p/tesseract-ocr5. Newer Debian and Ubuntu
releases already include Tesseract 5.
:::
Alternatively, use Homebrew on Linux for a full-featured installation (see
{ref}homebrew-linux).
To add JBIG2 encoding, on Debian 13 or newer and Ubuntu 24.04 or newer runsudo apt install jbig2; otherwise see {ref}jbig2.
Older Debian and Ubuntu releases
Ubuntu 22.04 LTS and Debian 12 are the oldest releases OCRmyPDF supports.
Ubuntu 20.04 and Debian 11 have reached the end of standard support, and their
Python, Tesseract and Ghostscript are too old for current OCRmyPDF. Consider
upgrading to a supported release.
If you cannot upgrade, the most convenient way to install a recent version of
OCRmyPDF is to use Homebrew on Linux:
brew install ocrmypdfSee {ref}homebrew-linux for more information on using Homebrew on Linux.
Alternatively, run OCRmyPDF from the Docker image, which carries its
own up-to-date dependencies.
Fedora
:::{list-table}
:header-rows: 1
- OCRmyPDF version
- {{latest}}
- {{fedora_43}} {{fedora_44}} {{fedora_rawhide}}
:::
- {{fedora_43}} {{fedora_44}} {{fedora_rawhide}}
Users of Fedora may simply
dnf install ocrmypdf tesseract-osdtesseract-osd is not installed automatically, but OCRmyPDF needs it for
automatic page rotation (--rotate-pages). Tesseract language packs are namedtesseract-langpack-<language>, for example tesseract-langpack-deu.
For full details on version availability, check the Fedora Package
Tracker.
If the version available for your platform is out of date, install the
system package first to satisfy system dependencies, then use uv to install
the latest version:
# Install system dependencies first
sudo dnf install ocrmypdf tesseract-osd
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install the latest OCRmyPDF
uv tool install ocrmypdfOpen a new terminal (or run uv tool update-shell) so that ~/.local/bin is on
your PATH, then confirm the version with ocrmypdf --version. To install
from source instead, see Installing HEAD revision from
sources.
:::{note}
OCRmyPDF for Fedora currently omits the JBIG2 encoder, and Fedora does not
package it. All JBIG2 patents expired in 2017. OCRmyPDF works fine without it
but will produce larger output files. If you build jbig2enc from source,
OCRmyPDF will automatically detect it on the PATH. To add JBIG2 encoding,
see {ref}jbig2.
:::
RHEL, CentOS Stream, AlmaLinux and Rocky Linux 9 and 10
There is no OCRmyPDF package for Red Hat Enterprise Linux or its derivatives,
either in the base repositories or in EPEL. Install the system dependencies
from the base (AppStream) repositories:
sudo dnf install ghostscript tesseract tesseract-osd tesseract-langpack-engOther Tesseract languages are available as tesseract-langpack-<language>,
for example tesseract-langpack-deu.
The optional dependencies unpaper and, on version 9, pngquant are available
from EPEL. Once
EPEL is enabled:
sudo dnf install unpaper pngquant # pngquant: version 9 onlyThen install uv, if not already installed, and use it to install OCRmyPDF:
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install the latest OCRmyPDF
uv tool install ocrmypdfRHEL 9's default Python 3.9 is too old for OCRmyPDF; uv will download a
suitable version of Python automatically. Open a new terminal (or runuv tool update-shell) so that ~/.local/bin is on your PATH, then confirm
the version with ocrmypdf --version.
To add JBIG2 encoding, see {ref}Installing the JBIG2 encoder <jbig2>.
Arch Linux (AUR)
:::{image} https://repology.org/badge/version-for-repo/aur/ocrmypdf.svg
:alt: ArchLinux
:target: https://repology.org/metapackage/ocrmypdf
:::
There is an Arch User Repository (AUR) package for OCRmyPDF.
OCRmyPDF is not in the official Arch repositories, but its dependencies,
including Tesseract and its language data, are.
Installing AUR packages as root is not allowed, so you must first setup a
non-root user and
configure sudo.
The official Docker image, archlinux:latest, does not have a
non-root user configured, so users of that image must follow these guides.
Next you should install the base-devel package group. This includes the
standard tooling needed to build packages, such as a compiler and binary tools.
sudo pacman -S --needed base-devel gitNow you are ready to install the OCRmyPDF package.
git clone https://aur.archlinux.org/ocrmypdf.git
cd ocrmypdf
makepkg -sriAt this point you will have a working install of OCRmyPDF, but the Tesseract
install won’t include any OCR language data. You can install the
tesseract-data packages to add
supported languages. For example, for English:
sudo pacman -S tesseract-data-engAs an alternative to this manual procedure, consider using an AUR helper. Such a tool will
automatically fetch, build and install the AUR package, resolve dependencies
(including dependencies on AUR packages), and ease the upgrade procedure.
If you have any difficulties with installation, check the repository package
page.
:::{note}
The JBIG2 encoder is an optional dependency of the OCRmyPDF AUR package.
OCRmyPDF works fine without it but will produce larger output files. The
encoder is available from the jbig2enc AUR package
and may be installed using the same series of steps as for the OCRmyPDF AUR
package. Alternatively, it may be built manually from source following the
instructions in {ref}Installing the JBIG2 encoder <jbig2>. If jbig2enc is
installed, OCRmyPDF will automatically detect it.
:::
Alpine Linux
:::{image} https://repology.org/badge/version-for-repo/alpine_edge/ocrmypdf.svg
:alt: Alpine Linux
:target: https://repology.org/metapackage/ocrmypdf
:::
OCRmyPDF is in the Alpine Linux community repository. To install it:
apk add ocrmypdfIf the version available for your Alpine release is out of date, consider the
Alpine-based Docker image, jbarlow83/ocrmypdf-alpine.
Gentoo Linux
:::{image} https://repology.org/badge/version-for-repo/gentoo_ovl_guru/ocrmypdf.svg
:alt: Gentoo Linux
:target: https://repology.org/metapackage/ocrmypdf
:::
OCRmyPDF is available from the GURU
overlay, which must be enabled first. To install OCRmyPDF on Gentoo Linux, use
the following commands:
emerge --ask --noreplace app-eselect/eselect-repository dev-vcs/git
eselect repository enable guru
emaint sync --repo guru
emerge --ask app-text/OCRmyPDFOther Linux packages
OCRmyPDF is also packaged for other distributions, including NixOS (ocrmypdf),
Void Linux (python3-ocrmypdf) and openSUSE Tumbleweed. See the
Repology page for a full
list.
In general, first install the OCRmyPDF package for your system to satisfy
system dependencies, then optionally use the procedure Installing with
uv to install a more recent version:
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install the latest OCRmyPDF
uv tool install ocrmypdf:::{note}
The OCRmyPDF Snap package (snap install ocrmypdf) tracks each OCRmyPDF
release on the stable channel. Builds of the development branch are available
with snap install ocrmypdf --edge. The snap is strictly confined and can only
read and write files in your home folder or on removable media; for other
locations, use snap run ocrmypdf - - output.pdf.
:::
(homebrew-linux)=
Installing with Homebrew (macOS and Linux)
:::{image} https://img.shields.io/homebrew/v/ocrmypdf.svg
:alt: homebrew
:target: https://formulae.brew.sh/formula/ocrmypdf
:::
Homebrew provides a full-featured OCRmyPDF installation
on both macOS and Linux with all recommended dependencies. This is often
the easiest way to get a complete, up-to-date installation.
brew install ocrmypdfThis includes Tesseract, Ghostscript, jbig2enc, pngquant, unpaper and all
required dependencies. English language support is included by default. For
other languages:
brew install tesseract-lang # Optional: Install all language packsTo upgrade later:
brew upgrade ocrmypdf:::{tip}
For Linux users: Homebrew on Linux is an excellent choice when your
distribution's package is outdated, your distribution is too old for current
OCRmyPDF, or its package is missing optional dependencies like jbig2enc,
pngquant, or unpaper. Homebrew provides a consistent, full-featured
installation that works across many Linux distributions.
Install Homebrew on Linux: https://brew.sh
:::
Installing on macOS
Homebrew
See {ref}homebrew-linux above - the installation is identical on macOS.
MacPorts
:::{image} https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fports.macports.org%2Fapi%2Fv1%2Fports%2Focrmypdf%2F%3Fformat%3Djson&query=version&label=MacPorts
:alt: Macports Version Information
:target: https://ports.macports.org/port/ocrmypdf
:::
OCRmyPDF is included in MacPorts:
sudo port install ocrmypdfNote that while this will install tesseract you will need to install
the appropriate tesseract language ports,
which are named tesseract-<language>. For example, for German:
sudo port install tesseract-deuManual installation on macOS
These instructions are for installing a more current version of OCRmyPDF than
is available from Homebrew. Note that Homebrew versions usually track
releases fairly closely.
If it's not already present, install Homebrew.
Update Homebrew and install OCRmyPDF's dependencies, without OCRmyPDF itself:
brew update
brew install --only-dependencies ocrmypdfHomebrew's Tesseract includes English only. If you need other languages you
can optionally install them all:
(macos-all-languages)=
brew install tesseract-lang # Optional: for all language packsInstall uv, if not already installed, and use it to install OCRmyPDF:
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install the latest OCRmyPDF
uv tool install ocrmypdfOpen a new terminal (or run uv tool update-shell) so that ~/.local/bin is
on your PATH. The command line program should now be available:
ocrmypdf --helpTo upgrade later, run uv tool upgrade ocrmypdf.
Installing on Windows
Native Windows
% If you have a Windows that is not the Home edition, you can use Windows Sandbox to test on a blank Windows instance.
% https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/
:::{note}
Administrator privileges will be required for some of these steps.
:::
You must install the following for Windows:
- Tesseract 64-bit
- Ghostscript 64-bit (recommended; required for some PDF/A conversions)
- uv, which installs OCRmyPDF and, if needed, a suitable Python
Using the winget
package manager:
winget install -e --id tesseract-ocr.tesseract
winget install -e --id astral-sh.uvIf you prefer not to use winget for uv, you can install it with its
standalone installer instead:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"Ghostscript is not available from winget. You will need to install it manually,
since it does not support automated installs
anymore:
Or alternately, using the Chocolatey package
manager, install the following when running in an Administrator command
prompt:
choco install tesseract
choco install ghostscript
choco install pngquant # optionaland then install uv using either of the commands above.
Either set of commands will install the required software. At the moment there
is no single command to install OCRmyPDF and all of its dependencies on
Windows.
Open a new command prompt or PowerShell window, so that the newly installed
programs are on your PATH, and install OCRmyPDF. (This can be performed by a
user or Administrator.)
uv tool install ocrmypdfOpen another new window, then confirm that OCRmyPDF is available:
ocrmypdf --versionIf the ocrmypdf command is not found, run uv tool update-shell and open a
new window. To upgrade later, run uv tool upgrade ocrmypdf.
OCRmyPDF will check the Windows Registry and standard locations in your Program Files
for third party software it needs (specifically, Tesseract and Ghostscript). To
override the versions OCRmyPDF selects, you can modify the PATH environment
variable. Follow these directions
to change the PATH.
:::{warning}
32-bit Windows is not supported.
:::
Windows Subsystem for Linux
- Install Ubuntu 24.04 or 26.04 LTS for Windows Subsystem for Linux, if not
already installed. - Follow the procedure to install {ref}
the latest OCRmyPDF on Ubuntu. - In the Ubuntu (WSL) terminal, create a symlink so that the latest version is
found when OCRmyPDF is run from Windows:
sudo ln -s ~/.local/bin/ocrmypdf /usr/local/bin/ocrmypdfThen, from the Windows command prompt, confirm that the expected version from
PyPI ({{ latest }}) is installed:
wsl ocrmypdf --versionYou can then run OCRmyPDF in the Windows command prompt or Powershell, prefixingwsl, and call it from Windows programs or batch files.
Cygwin64
:::{warning}
Cygwin installation is not tested. uv is not available for Cygwin, and some
of OCRmyPDF's Python dependencies, such as pikepdf, pypdfium2 and uharfbuzz,
have no Cygwin binaries and must be compiled from source, which may fail. We
recommend using Windows Subsystem for Linux, the Docker image, or native
Windows instead.
:::
First install the following prerequisite Cygwin packages using setup-x86_64.exe:
python312 (or later)
python312-devel
python312-pip
python312-lxml
python312-imaging
(where 312 should match the version of Python you installed)
gcc-g++
ghostscript
libexempi-devel
libffi-devel
pngquant
qpdf
libqpdf-devel
tesseract-ocr
tesseract-ocr-develThen open a Cygwin terminal (i.e. mintty), and install OCRmyPDF into a
virtual environment:
python3.12 -m venv --system-site-packages ~/ocrmypdf-venv
~/ocrmypdf-venv/bin/python -m pip install ocrmypdf
~/ocrmypdf-venv/bin/ocrmypdf --versionThe optional dependency "unpaper" is currently not available under Cygwin.
Without it, certain options such as --clean will produce an error message.
However, the OCR-to-text-layer functionality is available.
Docker
You can also Install the Docker image on Windows. Ensure that
your command prompt can run the docker "hello world" container.
Installing on FreeBSD
:::{image} https://repology.org/badge/version-for-repo/freebsd/ocrmypdf.svg
:alt: FreeBSD
:target: https://repology.org/project/ocrmypdf/versions
:::
pkg install py311-ocrmypdfTo install a more recent version, first install the system version with pkg
to satisfy system dependencies, then install uv from packages and use it to
install OCRmyPDF:
pkg install uv
uv tool install ocrmypdfSome Python dependencies do not publish binaries for FreeBSD and will be
compiled from source, so this may require additional development packages.
Installing the Docker image
For some users, installing the Docker image will be easier than
installing all of OCRmyPDF's dependencies. It is also a good option if your
operating system is too old to run current OCRmyPDF.
See Installing the Docker image for more information.
(installing-with-python-pip)=
Installing with uv (recommended)
We recommend using uv for installing OCRmyPDF from PyPI.
uv is a fast, modern Python package manager that installs command line tools
in isolated environments, and can download a suitable version of Python if
your system's Python is too old.
For best results, first install your platform's
version ofocrmypdf using the instructions elsewhere in this document to satisfy system
dependencies. Then use uv to get the latest OCRmyPDF version.
On macOS and Linux:
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install the latest OCRmyPDF
uv tool install ocrmypdfOn Windows:
# Install uv, if not already installed
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# Install the latest OCRmyPDF
uv tool install ocrmypdfOpen a new terminal (or run uv tool update-shell) so that the ocrmypdf
command is on your PATH. Use ocrmypdf --version to confirm what version was
installed, and uv tool upgrade ocrmypdf to upgrade later.
To use OCRmyPDF as a Python library in your own project, add it as a
dependency of that project instead:
uv add ocrmypdfInstalling with pipx
If you already use pipx for isolated command-line tool installations, it works
as well:
pipx install ocrmypdf(requirements-for-pip-and-head-install)=
Requirements for uv and source installs
OCRmyPDF currently requires these external programs and libraries to be
installed, and must be satisfied using the operating system package
manager. uv and other Python package managers cannot provide them.
:::{versionchanged} 17.0.0
Ghostscript is now optional. pypdfium2 can be used for PDF rasterization,
and verapdf can validate speculative PDF/A conversion.
:::
:::{versionchanged} 17.13.0
veraPDF is no longer used. Speculative PDF/A conversion is checked by
pikepdf's PDF/A support (pikepdf.pdfa), so OCRmyPDF requirespikepdf[pdfa] 10.15 or newer, which brings in the jsonschema, referencing
and fonttools Python packages.
:::
The following versions are required:
- Python 3.11 or newer (3.12+ recommended; uv can install it for you)
- Tesseract 4.1.1 or newer
- One of: Ghostscript 9.54+ or pypdfium2 (Python package)
- Ghostscript 9.54+ (for PDF/A output of files that cannot be converted
without it; optional) - fpdf2 2.8 or newer (Python package)
- uharfbuzz (Python package)
- fonts-noto or equivalent (system package, recommended)
- jbig2enc 0.28 or newer (optional)
- pngquant 2.12.2 or newer (optional)
- unpaper 6.1 or newer (optional)
:::{note}
For the best user experience, install both Ghostscript and pypdfium2. pypdfium2 is
faster for rasterization, while Ghostscript is required for certain PDF/A
conversions.
:::
Dependency summary:
| Feature | Option 1 | Option 2 | Notes |
|---|---|---|---|
| PDF rasterization | pypdfium2 (Python) | Ghostscript (binary) | pypdfium2 preferred when available |
| PDF/A conversion | pikepdf (pikepdf.pdfa) |
Ghostscript | pikepdf validates speculative conversion |
| Text rendering | fpdf2 + uharfbuzz | - | Required |
| OCR | tesseract-ocr | --ocr-engine none |
Can be skipped entirely |
Minimum viable installation:
tesseract-ocr + (pypdfium2 OR Ghostscript) + fpdf2 + uharfbuzz
Recommended installation:
tesseract-ocr + pypdfium2 + Ghostscript + fpdf2 + uharfbuzz + fonts-noto + unpaper + pngquant + jbig2enc
We recommend 64-bit versions of all software. (32-bit versions are not
supported, although on Linux, they may still work.)
fpdf2 and uharfbuzz are required dependencies that provide the text
layer rendering engine. fpdf2 generates the PDF text layer, while uharfbuzz
provides text shaping for proper multilingual support. These replace the
legacy hOCR-based renderer. They are Python packages, installed automatically
when you install OCRmyPDF with uv.
fonts-noto (or an equivalent comprehensive font package) is recommended
for proper text rendering, especially for non-Latin scripts. OCRmyPDF bundles
a Latin font only, and discovers the rest from the fonts installed on your
system.
- Debian/Ubuntu:
apt install fonts-noto - Fedora:
dnf install google-noto-fonts-all - macOS with Homebrew: Homebrew has no single Noto package; each family is a
separate cask. Install at leastbrew install --cask font-noto-sans font-noto-serif, plus a cask per
additional script you OCR, for examplebrew install --cask font-noto-sans-arabic font-noto-sans-cjk. Runbrew search font-nototo list them all.
If OCRmyPDF warns that no installed font has glyphs for some of the text, the
message names the characters it could not render, for example'Ꮳ' U+13E3 CHEROKEE LETTER TSA. Install the Noto font for that script — here,fonts-noto-core on Debian or font-noto-sans-cherokee on Homebrew. The text
layer remains searchable and copyable either way; only its appearance when
highlighted in a PDF viewer is affected.
pypdfium2 provides fast PDF page rasterization using the pdfium library
(the same library used by Google Chrome). It is installed automatically when
you install OCRmyPDF with uv, although some distribution packages of OCRmyPDF
omit it. When present, it is preferred over Ghostscript due to better
performance.
Speculative PDF/A conversion needs no extra software. OCRmyPDF creates
PDF/A by adding metadata and an ICC profile using pikepdf, then checks the
result with pikepdf's PDF/A validator. If validation passes, Ghostscript
is skipped entirely; with --pdfa-backend internal, Ghostscript is never
used for PDF/A. veraPDF is no longer used by OCRmyPDF itself; the
test suite uses it, if installed, to cross-check pikepdf's validator
(see verapdf.org).
jbig2enc, if present, will be used to optimize the encoding of
monochrome images. This can significantly reduce the file size of the
output file. It is not required.
jbig2enc is not available in some
distributions due to historical patent concerns, but all JBIG2 patents
expired in 2017. It can easily be built from source. To add JBIG2 encoding,
see {ref}jbig2.
:::{warning}
Lossy JBIG2 encoding (--jbig2-lossy) has been removed in v17.0.0 due to
well-documented risks of character substitution errors. Only lossless
JBIG2 compression is now supported.
:::
pngquant, if present, is optionally used to optimize the encoding of
PNG-style images in PDFs (actually, any that are that losslessly
encoded) by lossily quantizing to a smaller color palette. It is only
activated then the --optimize argument is 2 or 3.
unpaper, if present, enables the --clean and --clean-final
command line options.
These are in addition to the Python packaging dependencies, meaning that
unfortunately, the uv tool install command cannot satisfy all of them.
(installing-head-revision-from-sources)=
Installing HEAD revision from sources
If you have git and uv installed, you can install from source. uv will
alert you if Python dependencies cannot be installed; OCRmyPDF will alert you
at runtime if system dependencies are missing.
If you prefer to build everything from source, you will need to build
pikepdf from
source.
First ensure you can build and install pikepdf.
To install the latest development version as a command line tool:
# Install uv, if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install git+https://github.com/ocrmypdf/OCRmyPDF.gitOr, to work from a local checkout allowing customization:
git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
cd OCRmyPDF
uv syncThis creates a virtual environment in .venv and installs OCRmyPDF in
editable mode along with its dependencies. Run the program with uv run, or
activate the environment:
uv run ocrmypdf --help
# or
source .venv/bin/activate
ocrmypdf --helpNote: when installed this way, ocrmypdf will only be accessible throughuv run or when the virtual environment is activated.
If not yet installed, the script will notify you about dependencies that
need to be installed. The script requires specific versions of the
dependencies. Older version than the ones mentioned in the release notes
are likely not to be compatible to OCRmyPDF.
Optional Features
OCRmyPDF provides optional features and development tools. We recommend using uv as your package manager.
Installing User Features
User features are available as optional dependencies (extras). When
installing OCRmyPDF as a tool:
uv tool install "ocrmypdf[heic]" # HEIF/HEIC image support
uv tool install "ocrmypdf[watcher]" # File watching service
uv tool install "ocrmypdf[webservice]" # Streamlit web UI
uv tool install "ocrmypdf[watcher,webservice]" # Multiple featuresOr, from a local source checkout:
uv sync --extra heic # HEIF/HEIC image support
uv sync --extra watcher # File watching service
uv sync --extra webservice # Streamlit web UI
uv sync --extra watcher --extra webservice # Multiple featuresDevelopment Tools
Development tools use dependency groups:
# Testing infrastructure
uv sync --group test
# Documentation building
uv sync --group docs
# Enhanced Streamlit development
uv sync --group streamlit-dev
# All development groups
uv sync --all-groupsWhy use uv?
- Modern, fast Python package manager
- Required for development (testing, docs)
- Better dependency resolution
- Consistent across all platforms
Install uv: curl -LsSf https://astral.sh/uv/install.sh | sh or visit https://docs.astral.sh/uv/
For development
To install all of the development and test requirements:
git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
cd OCRmyPDF
uv sync --all-groupsTo add JBIG2 encoding, see {ref}jbig2.
Shell completions
Completions for bash and fish are available in the project'smisc/completion folder. The bash completions are likely zsh
compatible but this has not been confirmed. Package maintainers, please
install these at the appropriate locations for your system.
To manually install the bash completion, copymisc/completion/ocrmypdf.bash to /etc/bash_completion.d/ocrmypdf
(rename the file).
To manually install the fish completion, copymisc/completion/ocrmypdf.fish to~/.config/fish/completions/ocrmypdf.fish.
Note on 32-bit support
We don't support any 32-bit system, including 32-bit Python or 32-bit
Ghostscript on Windows.
Introduction (`docs/introduction.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Introduction
OCRmyPDF is a Python application and library that adds text "layers" to images in
PDFs, making scanned image PDFs searchable. It uses OCR to guess the text
contained in images. OCRmyPDF also supports plugins
that enable customization of its processing steps, and it is highly tolerant
of PDFs containing scanned images and "born digital" content that doesn't
require text recognition.
About OCR
Optical character
recognition
is a technology that converts images of typed or handwritten text, such as
in a scanned document, into computer text that can be selected, searched and copied.
OCRmyPDF uses
Tesseract, a widely
available open source OCR engine, to perform OCR.
(raster-vector)=
About PDFs
PDFs are page description files that attempt to preserve a layout
exactly. They contain vector
graphics
that can contain raster objects, such as scanned images. Because PDFs can
contain multiple pages (unlike many image formats) and can contain fonts
and text, they are a suitable format for exchanging scanned documents.
:::{image} images/bitmap_vs_svg.svg
:::
A PDF page may contain multiple images, even if it appears to have only
one image. Some scanners or scanning software may segment pages into
monochromatic text and color regions, for example, to enhance the compression
ratio and appearance of the page.
Rasterizing a PDF is the process of generating corresponding raster images.
OCR engines like Tesseract work with images, not scalable vector graphics
or mixed raster-vector-text graphics such as PDF.
About PDF/A
PDF/A is an ISO-standardized
subset of the full PDF specification that is designed for archiving (the
'A' stands for Archive). PDF/A differs from PDF primarily by omitting
features that could complicate future file readability,
such as embedded Javascript, video, audio and references to external
fonts. All fonts and resources needed to interpret the PDF must be
contained within it. Because PDF/A disables Javascript and other types
of embedded content, it is likely more secure.
There are various conformance levels and versions, such as "PDF/A-2b".
In general, the preferred format for scanned documents is PDF/A. Some
governments and jurisdictions, US Courts in particular, mandate the use
of PDF/A for scanned
documents.
Since most individuals scanning documents aim for long-term readability,
OCRmyPDF defaults to generating PDF/A-2b.
PDF/A does have a few drawbacks. Some PDF viewers display an alert
indicating that the file is in PDF/A format, which may confuse some users.
Additionally, it tends to result in larger files than standard PDFs because
it embeds certain resources, even if they are widely available. PDF/A
files can be digitally signed but may not be encrypted to ensure future
readability. Fortunately, converting from PDF/A to a regular PDF is
straightforward, and any PDF viewer can handle PDF/A files.
What OCRmyPDF does
OCRmyPDF analyzes each page of a PDF to determine the required colorspace
and resolution (DPI) for capturing all the information on that page without
losing content. It uses a PDF rasterizer (pypdfium2 or
Ghostscript) to convert each page to an image and
subsequently performs OCR on the rasterized image to generate an OCR "layer."
This layer is then integrated back into the original PDF.
:::{versionchanged} 17.0.0
OCRmyPDF now supports pypdfium2 as an alternative rasterizer to Ghostscript.
pypdfium2 is a Python binding for pdfium, the PDF rendering library used by
Google Chrome. The --rasterizer auto setting (default) prefers pypdfium2
when available.
:::
While it is possible to use a program like Ghostscript or ImageMagick to
obtain an image and then run that image through Tesseract OCR, this process
actually generates a new PDF, potentially resulting in the loss of various
details (such as the document's metadata). In contrast, OCRmyPDF can produce
a minimally altered PDF as the output.
OCRmyPDF also offers several image processing options, such as deskew, which
enhances the visual quality of files and the accuracy of OCR. When these
options are utilized, the OCR layer is integrated into the processed image.
By default, OCRmyPDF generates archival PDFs in the PDF/A format, which is
a more rigid subset of PDF features designed for long-term archives. If you
prefer regular PDFs, you can disable this feature using the--output-type pdf option.
Why you shouldn't do this manually
A PDF is similar to an HTML file, in that it contains document structure
along with images. While some PDFs may solely display a full-page image,
they often contain additional content that would be forfeited if not preserved.
A manual process could take one of these approaches:
- Rasterize each page as an image, perform OCR on the images, and then merge the
output into a PDF. This method preserves the layout of each page, but
resamples all images potentially leading to quality loss, increased file size,
and the introduction of compression artifacts, among other issues. - Extract each image, OCR, and combine the output into a PDF. This approach
loses the context in which images are used in the PDF, potentially resulting
in loss of information related to scaling and position of images. Some scanned
PDFs contain multiple images segmented into black and white, grayscale
and color regions, with stencil masks to prevent overlap, as this can
enhance the appearance of a file while reducing file size.
Reassembling these images can be challenging, and risks losing vector art
or text that is not part of an image.
In cases where a PDF solely serves as a container for images without any
rotation, scaling, or cropping, the second approach can be lossless.
OCRmyPDF uses various strategies depending on input options and the input PDF
itself. Generally, it rasterizes a page for OCR and then integrates the OCR
data back into the original PDF. This approach allows it to handle complex
PDFs and preserve their content as much as possible.
Furthermore, OCRmyPDF supports a wide range of edge cases that have emerged
during several years of development. It accommodates PDF features like
images within Form XObjects and pages with UserUnit scaling. It also
supports less common image formats like non-monochrome 1-bit images and
provides warnings about files you may not want to OCR. Thanks to tools
like pikepdf and QPDF, it can auto-repair damaged PDFs. You don't need to
understand the intricacies of these issues; you should be able to use
OCRmyPDF with any PDF file, and expect reasonable results.
Limitations
OCRmyPDF is subject to limitations imposed by the Tesseract OCR engine.
These limitations are inherent to any software relying on Tesseract:
- The OCR accuracy may not match that of commercial OCR solutions.
- It is incapable of recognizing handwriting.
- It may detect gibberish and report it as OCR output.
- Results may be subpar when a document contains languages not specified
in the-l LANGargument. - Tesseract may struggle to analyze the natural reading order of documents.
For instance, it might fail to recognize two columns in a document and
attempt to join text across columns. - Poor quality scans can result in subpar OCR quality. In other words, the
quality of the OCR output depends on the quality of the input. - Tesseract does not provide information about the font family to which text
belongs. - Tesseract does not divide text into paragraphs or headings. It only provides
the text and its bounding box. As such, the generated PDF does not
contain any information about the document's structure.
Ghostscript considerations
:::{versionchanged} 17.0.0
Ghostscript is no longer strictly required. OCRmyPDF can use pypdfium2
for rasterization and produce PDF/A without Ghostscript.
:::
:::{versionchanged} 17.13.0
PDF/A produced without Ghostscript is checked by pikepdf's PDF/A validator
(pikepdf.pdfa). veraPDF is no longer used.
:::
While Ghostscript remains a capable and feature-rich tool with a long history,
recent releases have introduced some compatibility challenges that OCRmyPDF
v17 addresses through alternative codepaths. When Ghostscript is used:
- PDFs containing JPEG 2000-encoded content may be converted to JPEG
encoding, which may introduce compression artifacts, if Ghostscript
PDF/A is enabled. - Ghostscript may transcode grayscale and color images, potentially
lossily, based on an internal algorithm. By default
(--pdfa-image-compression=auto) OCRmyPDF selects lossless image
compression at-O0so Ghostscript will not transcode lossless images
to JPEG. At-O1(the default optimization level) and above,auto
defers to Ghostscript's heuristic instead;-O1is a historical
exception, kept for backwards compatibility because coercing it to
lossless can substantially bloat output. You can override this by
setting--pdfa-image-compressiontojpegorlosslessto force all
images to one type or the other.losslesspasses existing JPEGs
through untouched (re-encoding them losslessly would only inflate them)
while encoding non-JPEG images losslessly.
(Modern Ghostscript can copy JPEG images without transcoding them.)
Advanced users can also tune Ghostscript's image recompression with--ghostscript-jpeg-qualityand--ghostscript-jpeg-maxdpi; see
Advanced Ghostscript tuning.
Most users should prefer--jpeg-quality(applied by the OCRmyPDF
optimizer) over those Ghostscript-scoped controls. - Ghostscript's PDF/A conversion removes any XMP metadata that is not
one of the standard XMP metadata namespaces for PDFs. In particular,
PRISM Metadata is removed. - Ghostscript's PDF/A conversion may remove or deactivate
hyperlinks and other active content.
Many of these limitations are avoided by the speculative PDF/A conversion
path, which OCRmyPDF tries first for all PDF/A output types and uses
whenever the result passes pikepdf's PDF/A validator. Use--pdfa-backend internal to never use Ghostscript for PDF/A, or--pdfa-backend ghostscript to always use it. Install pypdfium2
to avoid Ghostscript for rasterization as well.
You can use --output-type pdf to disable PDF/A conversion and produce
a standard, non-archival PDF.
Regarding OCRmyPDF itself:
- PDFs using transparency are not currently represented in the test
suite
Similar programs
To the author's knowledge, OCRmyPDF is the most feature-rich and
thoroughly tested command line OCR PDF conversion tool. If it does not
meet your needs, contributions and suggestions are welcome.
Ghostscript recently added three "pdfocr" output devices. They work by
rasterizing all content and converting all pages to a single colour space.
Web front-ends
The Docker image of OCRmyPDF provides a web service front-end
that allows files to submitted over HTTP, and the results can be downloaded.
This is an HTTP server intended to demonstrate how OCRmyPDF can be
integrated into a web service. It is not intended to be deployed on the
public internet and does not provide any security measures.
In addition, the following third-party integrations are available:
- Paperless-ngx is a free software
document management system that uses OCRmyPDF to perform OCR on
uploaded documents. - Nextcloud OCR is a free software
plugin for the Nextcloud private cloud software.
OCRmyPDF is not designed to be secure against malware-bearing PDFs (see
Using OCRmyPDF online). Users should ensure they
comply with OCRmyPDF's licenses and the licenses of all dependencies. In
particular, OCRmyPDF requires Ghostscript, which is licensed under
AGPLv3.
Jbig2 (`docs/jbig2.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
{#jbig2}
Installing the JBIG2 encoder
Many Linux distributions do not include a JBIG2 encoder since JBIG2
encoding was patented for a long time. All known JBIG2 US patents have
expired as of 2017, but it is possible that unknown patents exist.
JBIG2 encoding is recommended for OCRmyPDF and is used to losslessly
create smaller PDFs. If JBIG2 encoding is not available, lower quality
CCITT encoding will be used for monochrome images.
JBIG2 decoding is not patented and is performed automatically by most
PDF viewers. It is widely supported and has been part of the PDF
specification since 2001.
JBIG2 encoding is automatically provided by these OCRmyPDF packages:
- Docker image (both Ubuntu and Alpine)
- Snap package
- Alpine Linux package
- ArchLinux AUR package (as an optional dependency)
- Debian package
- Ubuntu package
- Homebrew on macOS
For all other platforms, you would need to build the JBIG2 encoder from
source:
:::{code} bash
git clone https://github.com/agl/jbig2enc
cd jbig2enc
./autogen.sh
./configure && make
[sudo] make install
:::
Dependencies include libtoolize and libleptonica, which on Ubuntu
systems are packaged as libtool and libleptonica-dev. On Fedora (35)
they are packaged as libtool and leptonica-devel. For this to work,
please make sure to install autotools, automake, libtool, pkg-config
and leptonica first if not already installed. Other dependencies might
be required depending on your system.
:::{code} bash
[sudo] apt install autotools-dev automake libtool libleptonica-dev pkg-config
:::
JBIG2 Compression
OCRmyPDF uses JBIG2 lossless compression for bitonal (black and white)
images. This provides excellent compression ratios compared to the older
CCITT G4 standard, while preserving the exact pixel content of the
original image.
You can adjust the threshold for JBIG2 compression with--jbig2-threshold. The default is 0.85.
:::{note}
Previous versions of OCRmyPDF supported a lossy JBIG2 mode
(--jbig2-lossy). This feature has been removed due to the well-known
risk of character substitution errors (e.g., 6/8 confusion). See
JBIG2 disadvantages
for more information on why lossy JBIG2 is problematic. The --jbig2-lossy
and --jbig2-page-group-size arguments are now ignored with a warning.
:::
Languages (`docs/languages.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
(lang-packs)=
Installing additional language packs
OCRmyPDF uses Tesseract for OCR, and relies on its language packs for all languages.
On most platforms, English is installed with Tesseract by default, but not always.
Tesseract supports most
languages.
Languages are identified by standardized three-letter codes (called ISO 639-2 Alpha-3).
Tesseract's documentation also lists the three-letter code for your language.
Some are anglicized, e.g. Spanish is spa rather than esp, while others
are not, e.g. German is deu and French is fra.
Language packs (strictly speaking, Tesseract "traineddata" files) generally correspond
to the language in question, but different language packs are used in certain
situations. For German, the "Fraktur" language pack can assist with reading older
materials in the Fraktur typeface family (deu_frak). Some communities have changed
their script from Cyrillic to Latin; the Cyrillic version of Uzbek is available
as uzb_cyrl and the Latin version is uzb.
After you have installed a language pack, you can use it with ocrmypdf -l <language>,
for example ocrmypdf -l spa. For multilingual documents, you can specify
all languages to be expected, e.g. ocrmypdf -l eng+fra for English and French.
English is assumed by default unless other language(s) are specified.
For Linux users, you can often find packages that provide language
packs.
Platform install steps
Debian and Ubuntu (apt)
# Display a list of all Tesseract language packs
apt-cache search tesseract-ocr
# Install Chinese Simplified language pack
apt-get install tesseract-ocr-chi-simYou can then pass the -l LANG argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either -l eng+fra (English and French) or-l eng -l fra.
Fedora
# Display a list of all Tesseract language packs
dnf search tesseract
# Install Chinese Simplified language pack
dnf install tesseract-langpack-chi_simYou can then pass the -l LANG argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either -l eng+fra (English and French) or-l eng -l fra.
Arch Linux
# Display a list of all Tesseract language packs
pacman -Ss tesseract-data
# Install German language pack
pacman -S tesseract-data-deuYou can then pass the -l LANG argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either -l eng+fra (English and French) or-l eng -l fra.
Gentoo
On Gentoo the package app-text/tessdata_fast, which app-text/tesseract depends on, handles Tesseract languages.
It accepts USE flags to select what languages should be installed, these can be set in /etc/portage/package.use.
Alternatively one can globally set the L10N use extension in /etc/portage/make.conf.
This enables these languages for all packages (e.g. including aspell).
# Display a list of all Tesseract language packs
equery uses app-text/tessdata_fast
# Add English and German language support for Tesseract only
echo 'app-text/tessdata_fast l10n_de l10n_en' >> /etc/portage/package.use
# Add global English and German language support (the `l10n_` from equery has to be omitted)
echo L10N="de en" >> /etc/portage/make.conf
# update system to reflect changed USE flags
emerge --update --deep --newuse @worldYou can then pass the -l LANG argument to OCRmyPDF to give a hint as
to what languages it should search for. Multiple languages can be
requested using either -l eng+fra (English and French) or-l eng -l fra.
macOS
You can install additional language packs by
{ref}installing Tesseract using Homebrew with all language packs <macos-all-languages>.
Docker
Users of the OCRmyPDF Docker image should install language packs into a
derived Docker image as
{ref}described in that section <docker-lang-packs>.
Windows
The Tesseract installer provided by Chocolatey currently includes only English language.
To install other languages, download the respective language pack (.traineddata file)
from https://github.com/tesseract-ocr/tessdata/ and place it inC:\\Program Files\\Tesseract-OCR\\tessdata (or wherever Tesseract OCR is installed).
Custom language packs
If you have fine-tuned or trained Tesseract and generated custom trained data, you can
copy your customlang.traineddata file into your Tesseract "tessdata" folder, and
then use the -l customlang argument to tell OCRmyPDF to pass that language on to
Tesseract.
Maintainers (`docs/maintainers.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Maintainer notes
This is for those who package OCRmyPDF for downstream use. (Thank you
for your hard work.)
Known ports/packagers
OCRmyPDF has been ported to many platforms already. If you are
interesting in porting to a new platform, check with
Repology to see the
status of that platform.
Make sure you can package pikepdf
pikepdf, created by the same author, is a mixed Python and C++14 package
with much stiffer build requirements. If you want to use OCRmyPDF on
some novel platform or distribution, first make sure you can package
pikepdf.
Core dependencies
:::{versionchanged} 17.0.0
Ghostscript is no longer strictly required. OCRmyPDF now supports alternative
codepaths for both PDF rasterization and PDF/A conversion.
:::
OCRmyPDF has the following runtime dependencies:
For PDF rasterization (converting PDF pages to images for OCR):
pypdfium2(Python package) - OR -ghostscript(system binary)- Recommendation: Install both for best compatibility
For PDF/A conversion:
- Speculative conversion with pikepdf, checked by pikepdf's PDF/A validator
(pikepdf.pdfa, which needs thepikepdf[pdfa]extra:jsonschema,referencingandfonttools); Ghostscript is used for files that fail
validation ghostscript(system binary)- Recommendation: Install Ghostscript for files that need real conversion.
veraPDF is no longer used at runtime; the test suite uses it, if
installed, to cross-check pikepdf's validator.
For OCR:
tesseract-ocr(system binary) - Required for MVP
For text rendering (expressing OCR results in PDF):
fpdf2(Python package) - Required for text layer renderinguharfbuzz(Python package) - Required for text layer rendering- Noto fonts (system package) - Recommended for text layer rendering.
fonts-notoon Debian/Ubuntu,google-noto-fonts-allon Fedora; Homebrew
has no single Noto package, only per-family casks such asfont-noto-sans.
Other dependencies:
unpaper(system binary) - Optional, enables--cleanand--clean-finalpngquant(system binary) - Optional, enables--optimize 2and--optimize 3jbig2enc(system binary) - Optional, improves compression of monochrome images
While Ghostscript remains a capable and feature-rich tool with a long history,
recent releases have introduced some compatibility challenges that OCRmyPDF v17
addresses through alternative codepaths. For the best user experience, packagers
should install both Ghostscript and the alternative tools (pypdfium2)
when available.
On Windows, OCRmyPDF will also check the registry for Tesseract and Ghostscript
locations.
Tesseract OCR relies on SIMD for performance and only has proper support
for this on ARM and x86_64. Performance may be poor on other processor
architectures.
Versioning scheme
OCRmyPDF uses hatch-vcs for versioning, which derives the version from
Git as a single source of truth. This may be unsuitable for some
distributions, e.g. to indicate that your distribution modifies OCRmyPDF
in some way.
You can patch the __version__ variable in src/ocrmypdf/_version.py
if necessary, or set the environment variableSETUPTOOLS_SCM_PRETEND_VERSION to the required version, if you need to
override versioning for some reason.
jbig2enc
OCRmyPDF will use jbig2enc, a JBIG2 encoder, if one can be found. Some
distributions have shied away from packaging JBIG2 because it contains
patented algorithms, but all patents have expired since 2017. If
possible, consider packaging it too to improve OCRmyPDF's compression.
:::{note}
Lossy JBIG2 encoding has been removed in v17.0.0 due to well-documented
risks of character substitution errors. Previously we provided this feature
on a "caveat emptor" basis but in the interest of focusing and eliminating
risks, we decided to remove this option. Now, only lossless JBIG2 compression
is supported.
:::
Dependency matrix for packagers
:::{versionadded} 17.0.0
:::
The following table summarizes the dependency options introduced in v17.0.0:
| Feature | Option 1 | Option 2 | Notes |
|---|---|---|---|
| PDF rasterization | pypdfium2 (Python) | ghostscript (binary) | pypdfium2 preferred when available |
| PDF/A conversion | pikepdf (pikepdf.pdfa) |
ghostscript | pikepdf validates speculative conversion |
| Text rendering | fpdf2 (Python) | - | Required, replaces legacy hOCR renderer |
| OCR | tesseract-ocr | --ocr-engine none |
Can be skipped entirely |
Minimum viable installation:
- tesseract-ocr + (pypdfium2 OR ghostscript) + fpdf2
Recommended installation:
- tesseract-ocr + pypdfium2 + ghostscript + fpdf2 + unpaper + pngquant + jbig2enc
:::{warning}
If Ghostscript is not installed, PDF/A output can only be produced for files
that pass pikepdf's PDF/A validator after speculative conversion. Other files
are output as a standard PDF (--output-type auto) or fail with an error
(--output-type pdfa).
:::
Sample debian/control dependency specification
Depends:
fonts-noto,
fpdf2 (>= 2.8),
ghostscript (>= 9.55), # Not strictly required, but best user experience
icc-profiles-free,
img2pdf,
python3-coloredlogs,
python3-deprecation,
python3-fonttools (>= 4.40), # for pikepdf.pdfa
python3-jsonschema (>= 4.18), # for pikepdf.pdfa
python3-pdfminer (>= 20181108+dfsg-3),
python3-pikepdf (>= 10.17),
python3-referencing, # for pikepdf.pdfa
python3-pil,
python3-pluggy,
python3-reportlab,
python3-rich,
python3-uharfbuzz, # Not currently in Debian
tesseract-ocr (>= 5.0.0),
zlib1g,
${misc:Depends},
${python3:Depends},
Recommends:
cyclopts, # Not currently in Debian
jbig2
paddleocr, # Not currently in Debian
pngquant,
pypdfium2, # Not currently in Debian
unpaper,
Suggests:
ocrmypdf-doc,
python-watchfiles,Command line completions
Please ensure that command line completions are installed, as described
in the installation documentation.
32-bit Linux support
If you maintain a Linux distribution that supports 32-bit x86 or ARM,
OCRmyPDF should continue to work as long as all of its dependencies
continue to be available in 32-bit form. Please note we do not test on
32-bit platforms.
HEIF/HEIC
HEIF/HEIC image support is an optional extra (ocrmypdf[heic]), backed by
the pillow-heif PyPI package; it is not installed by default and is not
included in the Docker image or snap. Its binary wheels bundle libheif,
libde265, and x265, which makes the wheel as a whole GPLv2-licensed, so
this stays opt-in rather than a default or bundled dependency. If not
installed, OCRmyPDF gracefully degrades and loses only support for this
feature.
Optimizer (`docs/optimizer.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
PDF optimization
OCRmyPDF includes an image-oriented PDF optimizer. By default, the
optimizer runs with safe settings with the goal of improving compression
at no loss of quality. At higher optimization levels, lossy
optimizations may be applied and tuned. Optimization occurs after OCR,
and only if OCR succeeded. It does not perform other possible
optimizations such as deduplicating resources, consolidating fonts,
simplifying vector drawings, or anything of that nature.
:::{list-table} OCRmyPDF optimization settings
widths: 33 6 60 header-rows: 1
- Optimization level
- Shorthand
- Description
--optimize 0-O0- Disable most optimizations.
--optimize 1(default)-O1- Enables lossless optimizations, such as transcoding images to more
efficient formats. Also compress other uncompressed objects in the
PDF and enables the more efficient "object streams" within the PDF.
--optimize 2-O2- All of the above, and enables lossy optimizations and color quantization.
--optimize 3-O3- All of the above, and enables more aggressive optimizations and targets lower
image quality.
:::
The exact type of optimizations performed will vary over time, and
depend on what third party tools are installed.
Despite optimizations, OCRmyPDF might still increase the overall file
size, since it must embed information about the recognized text, and
depending on the settings chosen, may not be able to represent the
output file as compactly as the input file.
Optimizations that always occurs
OCRmyPDF will automatically replace obsolete or inferior compression
schemes such as RLE or LZW with superior schemes such as Deflate, and
convert monochrome images to CCITT G4. Since this is lossless, it always
occurs and there is no way to disable it. Other non-image compressed
objects are compressed as well.
Fast web view
OCRmyPDF automatically optimizes PDFs for "fast web view" in Adobe
Acrobat's parlance, or equivalently, linearizes PDFs so that the
resources they reference are presented in the order a viewer needs them
for sequential display. This reduces the latency of viewing a PDF both
online and from local storage, in exchange for a slight increase in file
size.
To disable this optimization and all others, useocrmypdf --optimize 0 ... or the shorthand -O0.
Adobe Acrobat might not report the file as being "fast web view".
Lossless optimizations
At optimization level -O1 (the default), OCRmyPDF will also attempt
lossless image optimization.
If a JBIG2 encoder is available, then monochrome images will be
converted to JBIG2, with the potential for huge savings on large black
and white images, since JBIG2 is far more efficient than any other
monochrome (bi-level) compression. (All known US patents related to
JBIG2 have probably expired, but it remains the responsibility of the
user to supply a JBIG2 encoder such as
jbig2enc. OCRmyPDF does not implement
JBIG2 encoding on its own.)
OCRmyPDF currently does not attempt to recompress losslessly compressed
objects more aggressively.
Lossy optimizations
At optimization level -O1, -O2 and -O3, OCRmyPDF will some attempt
loss image optimization.
If Ghostscript is used to create a PDF/A (the default), Ghostscript will
optimize some images by converting them to JPEG, which are lossy. If--output-type pdf is used, there are no lossy optimizations. Ghostscript's
JPEG conversion is quite safe.
If pngquant is installed, OCRmyPDF will use it to perform quantize
paletted images to reduce their size.
The quality of JPEGs may be lowered, on the assumption that a lower
quality image may be suitable for storage after OCR. Use --jpeg-quality
to control the optimizer's JPEG quality target. The optimizer is the
recommended way to reduce JPEG image sizes: it applies consistently
regardless of whether Ghostscript was used to produce a PDF/A.
If you specifically need to tune Ghostscript's own PDF/A image handling
(for example, to force a hard DPI cap), see
Advanced Ghostscript tuning
for the separate --ghostscript-jpeg-quality and--ghostscript-jpeg-maxdpi options.
It is not possible to optimize all image types. Uncommon image types may
be skipped by the optimizer.
Pdfsecurity (`docs/pdfsecurity.md`)
(security)=
PDF security issues
OCRmyPDF should only be used on PDFs you trust. It is not designed to
protect you against malware.
Recognizing that many users have an interest in handling PDFs and
applying OCR to PDFs they did not generate themselves, this article
discusses the security implications of PDFs and how users can protect
themselves.
The disclaimer applies: this software has no warranties of any kind.
PDFs may contain malware
PDF is a rich, complex file format. The official PDF 1.7 specification,
ISO 32000:2008, is hundreds of pages long and references several annexes
each of which are similar in length. PDFs can contain video, audio, XML,
JavaScript and other programming, and forms. In some cases, they can
open internet connections to pre-selected URLs. All of these are
possible attack vectors.
In short, PDFs may contain
viruses.
If you do not trust a PDF or its source, do not open it or use OCRmyPDF
on it. Consider using a Docker container or virtual machine to isolate
an untrusted PDF from your system.
How OCRmyPDF processes PDFs
OCRmyPDF must open and interpret your PDF in order to insert an OCR
layer. First, it runs all PDFs through
pikepdf, a library based on
QPDF, a program that repairs PDFs with
syntax errors. This is done because, in the author's experience, a
significant number of PDFs in the wild, especially those created by
scanners, are not well-formed files. QPDF makes it more likely that
OCRmyPDF will succeed, but offers no security guarantees. QPDF is also
used to split the PDF into single page PDFs.
Finally, OCRmyPDF rasterizes each page of the PDF using
Ghostscript in -dSAFER mode.
Depending on the options specified, OCRmyPDF may graft the OCR layer
into the existing PDF or it may essentially reconstruct ("re-fry") a
visually identical PDF that may be quite different at the binary level.
That said, OCRmyPDF is not a tool designed for sanitizing PDFs.
Password protected PDFs
Password protected PDFs usually have two passwords, and owner and user
password. When the user password is set to empty, PDF readers will open
the file automatically and mark it as "(SECURED)". Password security
can also request certain restrictions on the PDF, but anyone can remove
these restrictions if they have either the owner or user password.
Passwords mainly present a barrier for casual users.
OCRmyPDF cannot remove passwords from PDFs. If you want to remove a
password from a PDF, you must use other software, such as qpdf.
If the owner and user password are set, a password is required forqpdf. If only the owner password is set, then the password can be
stripped, even if one does not have the owner password. To remove the
password from a using QPDF, use:
:::{code} bash
qpdf --decrypt --password='abc123' input.pdf no_password.pdf
:::
Then you can run OCRmyPDF on the file.
In its default mode, OCRmyPDF generates PDF/A. Passwords may not be set
on PDF/A documents. If you want to set a password on the output PDF, you
must specify --output-type pdf.
Signature images
Many programs exist which are capable of inserting an image of
someone's signature. On its own, this offers no security guarantees. It
is trivial to remove the signature image and apply it to other files.
This practice offers no real security.
Digital signatures
Important documents can be digitally signed and certified to attest to
their authorship, approval or execution of a legal agreement. OCRmyPDF
will detect signed PDFs and will not modify them, unless the--invalidate-digital-signatures option is used, which will invalidate
any signatures. (The signature may still be present in the PDF if
opened, but PDF readers will not validate it.)
A digital signature adds a cryptographic hash of the document to the
document, so tamper protection is provided. That also precludes OCRmyPDF
from modifying the document and preserving the signature.
Digital signatures are not the same as a signature image. A digital
signature is a cryptographic hash of the document that is encrypted with
the author's private key. The signature is decrypted with the author's
public key. The public key is usually distributed by a certificate
authority. The signature is then verified by the PDF reader. If the
document is modified, the signature will be invalidated.
Certificate-encrypted PDFs
PDFs can be encrypted with a certificate. This is a more secure form of
encryption than a password. The certificate is usually issued by a
certificate authority. A certificate is used to encrypt the document
using the public key for the benefit of a specific recipient who
possesses the private key.
OCRmyPDF cannot open certificate-encrypted PDFs. If you have the
certificate, you can use other PDF software, such as Acrobat, to decrypt
the PDF.
Performance (`docs/performance.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Performance
Some users have noticed that current versions of OCRmyPDF do not run as
quickly as some older versions (specifically 6.x and older). This is
because OCRmyPDF added image optimization as a postprocessing step, and
it is enabled by default.
Speed
If running OCRmyPDF quickly is your main goal, you can use settings such
as:
--optimize 0to disable file size optimization--output-type pdfto disable PDF/A generation--fast-web-view 999999to disable fast web view optimization--skip-bigto skip large images, if some pages have large images
You can also avoid:
--force-ocr- Image preprocessing
Memory
Peak memory is set by the number of pixels in the largest page, not by the
size of the PDF on disk. OCRmyPDF rasterizes each page at the resolution of the
images it contains, and there is no ceiling on that by default, so a 1200 dpi
scan costs four times as much as the same page at 600 dpi.
As a rule of thumb, budget about 15 bytes per pixel of the largest page, per
concurrent job. A 34 megapixel page (letter size at 600 dpi) therefore peaks
at roughly 500 MB with --jobs 1, and roughly 2 GB with --jobs 4. Three
things account for that:
- the OCR engine, which is the largest single consumer -- Tesseract peaks at
about 12 bytes per pixel of the image it is given - rasterizing the page, which holds the renderer's bitmap and the copy taken
from it at the same time - masking and writing the image that goes to OCR
They run one after another rather than at once, so the peak is the largest of
them, not their sum -- but each concurrent job pays it separately.
To bound memory on files with very large images, use --max-ocr-image-mpixels.
It downsamples the image sent to OCR when a page exceeds the given size, which
caps the largest consumer:
ocrmypdf --max-ocr-image-mpixels 8 --jobs 2 input.pdf output.pdfThe visible page is never downsampled, so the appearance of the output is
unaffected in every mode, including --force-ocr. What you trade is OCR
accuracy on very high resolution scans. Tesseract is tuned for roughly 300 dpi
and gains little above 400 dpi, so a cap around 8 megapixels is usually free for
letter or A4 pages; on documents with unusually small print, check the results
before adopting it.
Two other settings matter:
--jobsmultiplies the peak, since each worker processes its own page.
Lowering it is the bluntest way to fit a memory limit.--max-image-mpixelsis not a memory control. It is Pillow's
decompression-bomb guard, and it aborts the job rather than downsampling.
Plugins (`docs/plugins.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
Plugins
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
RFC 2119.
You can use plugins to customize the behavior of OCRmyPDF at certain points of
interest.
Currently, it is possible to:
- add new command line arguments
- override the decision for whether or not to perform OCR on a particular file
- modify the image is about to be sent for OCR
- modify the page image before it is converted to PDF
- replace the Tesseract OCR with another OCR engine that has similar behavior
- replace Ghostscript with another PDF to image converter (rasterizer) or
PDF/A generator
OCRmyPDF plugins are based on the Python pluggy package and conform to its
conventions. Note that: plugins installed with as setuptools entrypoints are
not checked currently, because OCRmyPDF assumes you may not want to enable
plugins for all files.
See [OCRmyPDF-EasyOCR](https://github.com/ocrmypdf/OCRmyPDF-EasyOCR) for an
example of a straightforward, fully working plugin.
Script plugins
Script plugins may be called from the command line, by specifying the name of a file.
Script plugins may be convenient for informal or "one-off" plugins, when a certain
batch of files needs a special processing step for example.
ocrmypdf --plugin ocrmypdf_example_plugin.py input.pdf output.pdfMultiple plugins may be installed by issuing the --plugin argument multiple times.
Packaged plugins
Installed plugins may be installed into the same virtual environment as OCRmyPDF
is installed into. They may be invoked using Python standard module naming.
If you are intending to distribute a plugin, please package it.
ocrmypdf --plugin ocrmypdf_fancypants.pockets.contents input.pdf output.pdfOCRmyPDF does not automatically import plugins, because the assumption is that
plugins affect different files differently and you may not want them activated
all the time. The command line or ocrmypdf.ocr(plugin='...') must call
for them.
Third parties that wish to distribute packages for ocrmypdf should package them
as packaged plugins, and these modules should begin with the name ocrmypdf_
similar to pytest packages such as pytest-cov (the package) andpytest_cov (the module).
:::{note}
We recommend plugin authors name their plugins with the prefixocrmypdf- (for the package name on PyPI) and ocrmypdf_ (for the
module), just like pytest plugins. At the same time, please make it clear
that your package is not official.
:::
Plugins
You can also create a plugin that OCRmyPDF will always automatically load if both are
installed in the same virtual environment, using a project entrypoint.
OCRmyPDF uses the entrypoint namespace "ocrmypdf".
For example, pyproject.toml would need to contain the following, for a plugin namedocrmypdf-exampleplugin:
[project]
name = "ocrmypdf-exampleplugin"
[project.entry-points."ocrmypdf"]
exampleplugin = "exampleplugin.pluginmodule"Plugin requirements
OCRmyPDF generally uses multiple worker processes. When a new worker is started,
Python will import all plugins again, including all plugins that were imported earlier.
This means that the global state of a plugin in one worker will not be shared with
other workers. As such, plugin hook implementations should be stateless, relying
only on their inputs. Hook implementations may use their input parameters to
to obtain a reference to shared state prepared by another hook implementation.
Plugins must expect that other instances of the plugin will be running
simultaneously.
The context object that is passed to many hooks can be used to share information
about a file being worked on. Plugins must write private, plugin-specific data to
a subfolder named {options.work_folder}/ocrmypdf-plugin-name. Plugins MAY
read and write files in options.work_folder, but should be aware that their
semantics are subject to change.
OCRmyPDF will delete options.work_folder when it has finished OCRing
a file, unless invoked with --keep-temporary-files.
The documentation for some plugin hooks contain a detailed description of the
execution context in which they will be called.
Plugins should be prepared to work whether executed in worker threads or worker
processes. Generally, OCRmyPDF uses processes, but has a semi-hidden threaded
argument that simplifies debugging.
Plugin hooks
A plugin may provide the following hooks. Hooks must be decorated withocrmypdf.hookimpl, for example:
from ocrmypdf import hookimpl
@hookimpl
def add_options(parser):
passThe following is a complete list of hooks that are available, and when
they are called.
(firstresult)=
Note on firstresult hooks
If multiple plugins install implementations for this hook, they will be called in
the reverse of the order in which they are installed (i.e., last plugin wins).
When each hook implementation is called in order, the first implementation that
returns a value other than None will "win" and prevent execution of all other
hooks. As such, you cannot "chain" a series of plugin filters together in this
way. Instead, a single hook implementation should be responsible for any such
chaining operations.
Examples
- OCRmyPDF's test suite contains several plugins that are used to simulate certain
test conditions. - ocrmypdf-papermerge is
a production plugin that integrates OCRmyPDF and the Papermerge document
management system.
Suppressing or overriding other plugins
.. autofunction:: ocrmypdf.pluginspec.initializeCustom command line arguments
.. autofunction:: ocrmypdf.pluginspec.add_options.. autofunction:: ocrmypdf.pluginspec.check_optionsPlugin option models
Plugins can define their own option models using Pydantic. This allows plugins to:
- Define type-safe option structures with validation
- Add CLI arguments that map to their option model fields
- Access options via nested namespaces (e.g.,
options.tesseract.timeout)
.. autofunction:: ocrmypdf.pluginspec.register_optionsPlugin options can be accessed in two ways:
- Flat access (backward compatible):
options.tesseract_timeout - Nested access:
options.tesseract.timeout
Both access patterns are equivalent and return the same values.
:::{note}
Plugin Interface Change: Starting in OCRmyPDF v17.0.0, plugin hooks receiveOcrOptions objects instead of argparse.Namespace objects. Most plugins will
continue working due to duck-typing compatibility, but plugin developers should
update their type hints accordingly.
:::
Migration guide for plugin developers
:::{versionadded} 17.0.0
:::
Update imports:
from ocrmypdf._options import OcrOptionsUpdate type hints:
# Before (v16 and earlier)
def check_options(options: argparse.Namespace) -> None: ...
# After (v17+)
def check_options(options: OcrOptions) -> None: ...Attribute access unchanged:
# These work exactly as before
options.languages
options.output_type
options.tesseract_timeoutRemove in-place modifications:
# Before (v16 pattern - no longer recommended)
def check_options(options):
options.some_computed_value = compute_value(options)
# After (v17 pattern - compute at point of use)
def some_function(options):
computed = compute_value(options)
use_computed(computed)Execution and progress reporting
.. autoclass:: ocrmypdf.pluginspec.ProgressBar
:members:
:special-members: __init__, __enter__, __exit__.. autoclass:: ocrmypdf.pluginspec.Executor
:members:
:special-members: __call__.. autofunction:: ocrmypdf.pluginspec.get_logging_console.. autofunction:: ocrmypdf.pluginspec.get_executor.. autofunction:: ocrmypdf.pluginspec.get_progressbar_classApplying special behavior before processing
.. autofunction:: ocrmypdf.pluginspec.validatePDF page to image
.. autofunction:: ocrmypdf.pluginspec.rasterize_pdf_pageModifying intermediate images
.. autofunction:: ocrmypdf.pluginspec.filter_ocr_image.. autofunction:: ocrmypdf.pluginspec.filter_page_image.. autofunction:: ocrmypdf.pluginspec.filter_pdf_pageOCR engine
.. autofunction:: ocrmypdf.pluginspec.get_ocr_engine.. autoclass:: ocrmypdf.pluginspec.OcrEngine
:members:
.. automethod:: __str__.. autoclass:: ocrmypdf.pluginspec.OrientationConfidencePDF/A production
.. autofunction:: ocrmypdf.pluginspec.generate_pdfaPDF optimization
.. autofunction:: ocrmypdf.pluginspec.optimize_pdf.. autofunction:: ocrmypdf.pluginspec.is_optimization_enabledWorking with OcrElement trees
:::{versionadded} 17.0.0
:::
OCRmyPDF v17 introduces the OcrElement dataclass for representing OCR
output in an engine-agnostic format. This enables plugins to work with
OCR results without parsing hOCR XML.
Key classes:
from ocrmypdf import OcrElement, OcrClass, BoundingBox
# OcrElement - represents any OCR structural unit
page = OcrElement(
ocr_class=OcrClass.PAGE, bbox=BoundingBox(0, 0, 612, 792), children=[...]
)
# BoundingBox - axis-aligned bounding box (left, top, right, bottom)
bbox = BoundingBox(left=100, top=50, right=300, bottom=80)
# OcrClass - constants for element types
OcrClass.PAGE # "ocr_page"
OcrClass.LINE # "ocr_line"
OcrClass.WORD # "ocrx_word"
OcrClass.PARAGRAPH # "ocr_par"Navigating the tree:
# Get all words in a page
words = page.words # Returns list[OcrElement]
# Get all lines
lines = page.lines
# Get combined text
text = page.get_text_recursive()
# Iterate by class
for para in page.paragraphs:
print(para.get_text_recursive())OCR engine plugins:
Plugins implementing custom OCR engines can now output OcrElement trees
directly via the generate_ocr() method, bypassing hOCR entirely:
from pathlib import Path
from ocrmypdf.pluginspec import OcrEngine
from ocrmypdf import OcrElement, OcrClass, BoundingBox
class MyOcrEngine(OcrEngine):
def generate_ocr(
self,
input_file: Path,
options,
context,
) -> OcrElement:
# Perform OCR and return OcrElement tree directly
# No need to generate hOCR XML
return OcrElement(
ocr_class=OcrClass.PAGE,
bbox=BoundingBox(0, 0, width, height),
dpi=300,
children=[
OcrElement(
ocr_class=OcrClass.LINE,
bbox=BoundingBox(100, 50, 500, 80),
children=[
OcrElement(
ocr_class=OcrClass.WORD,
bbox=BoundingBox(100, 50, 200, 80),
text="Hello",
),
# ... more words
],
),
# ... more lines
],
)
def supports_generate_ocr(self) -> bool:
return True # Indicate this engine uses generate_ocr()This approach is simpler than generating hOCR and allows modern OCR
engines to integrate more naturally with OCRmyPDF.
Releasenotes Version02 (`docs/releasenotes/version02.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v2
v2.2-stable (2014-09-29)
OCRmyPDF versions 1 and 2 were implemented as shell scripts. OCRmyPDF
3.0+ is a fork that gradually replaced all shell scripts with Python
while maintaining the existing command line arguments. No one is
maintaining old versions.
For details on older versions, see the final version of its release
notes.
Releasenotes Version03 (`docs/releasenotes/version03.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v3
v3.2.1
Changes
- Fixed {issue}
47
"convert() got and unexpected keyword argument 'dpi'" by upgrading to
img2pdf 0.2 - Tweaked the Dockerfiles
v3.2
New features
- Lossless reconstruction: when possible, OCRmyPDF will inject text
layers without otherwise manipulating the content and layout of a PDF
page. For example, a PDF containing a mix of vector and raster
content would see the vector content preserved. Images may still be
transcoded during PDF/A conversion. (--deskewand--clean-finaldisable this mode, necessarily.) - New argument
--tesseract-pagesegmodeallows you to pass page
segmentation arguments to Tesseract OCR. This helps for two column
text and other situations that confuse Tesseract. - Added a new "polyglot" version of the Docker image, that generates
Tesseract with all languages packs installed, for the polyglots among
us. It is much larger.
Changes
- JPEG transcoding quality is now 95 instead of the default 75. Bigger
file sizes for less degradation.
v3.1.1
Changes
- Fixed bug that caused incorrect page size and DPI calculations on
documents with mixed page sizes
v3.1
Changes
- Default output format is now PDF/A-2b instead of PDF/A-1b
- Python 3.5 and macOS El Capitan are now supported platforms - no
changes were needed to implement support - Improved some error messages related to missing input files
- Fixed {issue}
20: uppercase .PDF extension not accepted - Fixed an issue where OCRmyPDF failed to text that certain pages
contained previously OCR'ed text, such as OCR text produced by
Tesseract 3.04 - Inserts /Creator tag into PDFs so that errors can be traced back to
this project - Added new option
--pdf-renderer=auto, to let OCRmyPDF pick the
best PDF renderer. Currently it always chooses the 'hocrtransform'
renderer but that behavior may change. - Set up Travis CI automatic integration testing
v3.0
New features
- Easier installation with a Docker container or Python's
pip
package manager - Eliminated many external dependencies, so it's easier to setup
- Now installs
ocrmypdfto/usr/local/binor equivalent for
system-wide access and easier typing - Improved command line syntax and usage help (
--help) - Tesseract 3.03+ PDF page rendering can be used instead for better
positioning of recognized text (--pdf-renderer tesseract) - PDF metadata (title, author, keywords) are now transferred to the
output PDF - PDF metadata can also be set from the command line (
--title,
etc.) - Automatic repairs malformed input PDFs if possible
- Added test cases to confirm everything is working
- Added option to skip extremely large pages that take too long to OCR
and are often not OCRable (e.g. large scanned maps or diagrams);
other pages are still processed (--skip-big) - Added option to kill Tesseract OCR process if it seems to be taking
too long on a page, while still processing other pages
(--tesseract-timeout) - Less common colorspaces (CMYK, palette) are now supported by
conversion to RGB - Multiple images on the same PDF page are now supported
Changes
New, robust rewrite in Python 3.4+ with
ruffus pipelinesNow uses Ghostscript 9.14's improved color conversion model to
preserve PDF colorsOCR text is now rendered in the PDF as invisible text. Previous
versions of OCRmyPDF incorrectly rendered visible text with an image
on top.All "tasks" in the pipeline can be executed in parallel on any
available CPUs, increasing performanceThe
-o DPIargument has been phased out, in favor of--oversample DPI, in case we need-o OUTPUTFILEin the futureRemoved several dependencies, so it's easier to install. We no longer
use:- GNU parallel
- ImageMagick
- Python 2.7
- Poppler
- MuPDF tools
- shell scripts
- Java and JHOVE
- libxml2
Some new external dependencies are required or optional, compared to
v2.x:
Release candidates^
rc9:
- Fix
{issue}118:
report error if ghostscript iccprofiles are missing - fixed another issue related to
{issue}111: PDF
rasterized to palette file - add support image files with a palette
- don't try to validate PDF file after an exception occurs
- Fix
rc8:
- Fix
{issue}111:
exception thrown if PDF is missing DocumentInfo dictionary
- Fix
rc7:
- fix error when installing direct from pip, "no such file
'requirements.txt'"
- fix error when installing direct from pip, "no such file
rc6:
- dropped libxml2 (Python lxml) since Python 3's internal XML parser
is sufficient - set up Docker container
- fix Unicode errors if recognized text contains Unicode characters
and system locale is not UTF-8
- dropped libxml2 (Python lxml) since Python 3's internal XML parser
rc5:
- dropped Java and JHOVE in favour of qpdf
- improved command line error output
- additional tests and bug fixes
- tested on Ubuntu 14.04 LTS
rc4:
- dropped MuPDF in favour of qpdf
- fixed some installer issues and errors in installation
instructions - improve performance: run Ghostscript with multithreaded rendering
- improve performance: use multiple cores by default
- bug fix: checking for wrong exception on process timeout
rc3: skipping version number intentionally to avoid confusion with
Tesseractrc2: first release for public testing to test-PyPI, Github
rc1: testing release process
Compatibility notes
./OCRmyPDF.shscript is still available for now- Stacking the verbosity option like
-vvvis no longer supported - The configuration file
config.shhas been removed. Instead, you
can feed a file to the arguments for common settings:
ocrmypdf input.pdf output.pdf @settings.txtwhere settings.txt contains one argument per line, for example:
-l
deu
--author
A. Merkel
--pdf-renderer
tesseractFixes
- Handling of filenames containing spaces: fixed
Notes and known issues
- Some dependencies may work with lower versions than tested, so try
overriding dependencies if they are "in the way" to see if they work. --pdf-renderer tesseractwill output files with an incorrect page
size in Tesseract 3.03, due to a bug in Tesseract.- PDF files containing "inline images" are not supported and won't be
for the 3.0 release. Scanned images almost never contain inline
images.
Releasenotes Version04 (`docs/releasenotes/version04.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v4
v4.5.6
- Fixed {issue}
156,
'NoneType' object has no attribute 'getObject' on pages with no
optional /Contents record. This should resolve all issues related to
pages with no /Contents record. - Fixed {issue}
158, ocrmypdf
now stops and terminates if Ghostscript fails on an intermediate
step, as it is not possible to proceed. - Fixed {issue}
160,
exception thrown on certain invalid arguments instead of error
message
v4.5.5
- Automated update of macOS homebrew tap
- Fixed {issue}
154, KeyError
'/Contents' when searching for text on blank pages that have no
/Contents record. Note: incomplete fix for this issue.
v4.5.4
- Fixed
--skip-bigraising an exception if a page contains no images
({issue}152) (thanks
to @TomRaz) - Fixed an issue where pages with no images might trigger "cannot write
mode P as JPEG"
({issue}151)
v4.5.3
- Added a workaround for Ghostscript 9.21 and probably earlier versions
would fail with the error message "VMerror -25", due to a Ghostscript
bug in XMP metadata handling - High Unicode characters (U+10000 and up) are no longer accepted for
setting metadata on the command line, as Ghostscript may not handle
them correctly. - Fixed an issue where the
tess4renderer would duplicate content
onto output pages if tesseract failed or timed out - Fixed
tess4renderer not recognized when lossless reconstruction
is possible
v4.5.2
- Fixed {issue}
147,--pdf-renderer tess4 --cleanwill produce an oversized page
containing the original image in the bottom left corner, due to loss
DPI information. - Make "using Tesseract 4.0" warning less ominous
- Set up machinery for homebrew OCRmyPDF tap
v4.5.1
- Fixed {issue}
137,
proportions of images with a non-square pixel aspect ratio would be
distorted in output for--force-ocrand some other combinations
of flags
v4.5
- PDFs containing "Form XObjects" are now supported (issue
{issue}134; PDF
reference manual 8.10), and images they contain are taken into
account when determining the resolution for rasterizing - The Tesseract 4 Docker image no longer includes all languages,
because it took so long to build something would tend to fail - OCRmyPDF now warns about using
--pdf-renderer tesseractwith
Tesseract 3.04 or lower due to issues with Ghostscript corrupting the
OCR text in these cases
v4.4.2
The Docker images (ocrmypdf, ocrmypdf-polyglot, ocrmypdf-tess4) are
now based on Ubuntu 16.10 instead of Debian stretch- This makes supporting the Tesseract 4 image easier
- This could be a disruptive change for any Docker users who built
customized these images with their own changes, and made those
changes in a way that depends on Debian and not Ubuntu
OCRmyPDF now prevents running the Tesseract 4 renderer with Tesseract
3.04, which was permitted in v4.4 and v4.4.1 but will not work
v4.4.1
- To prevent a TIFF output
error caused
by img2pdf >= 0.2.1 and Pillow <= 3.4.2, dependencies have been
tightened - The Tesseract 4.00 simultaneous process limit was increased from 1 to
2, since it was observed that 1 lowers performance - Documentation improvements to describe the
--tesseract-config
feature - Added test cases and fixed error handling for
--tesseract-config - Tweaks to setup.py to deal with issues in the v4.4 release
v4.4
Tesseract 4.00 is now supported on an experimental basis.
- A new rendering option
--pdf-renderer tess4exploits Tesseract
4's new text-only output PDF mode. See the documentation on PDF
Renderers for details. - The
--tesseract-oemargument allows control over the Tesseract
4 OCR engine mode (tesseract's--oem). Use--tesseract-oem 2to enforce the new LSTM mode. - Fixed poor performance with Tesseract 4.00 on Linux
- A new rendering option
Fixed an issue that caused corruption of output to stdout in some
casesRemoved test for Pillow JPEG and PNG support, as the minimum
supported version of Pillow now enforces thisOCRmyPDF now tests that the intended destination file is writable
before proceedingThe test suite now requires
pytest-helpers-namespaceto run (but
not install)Significant code reorganization to make OCRmyPDF re-entrant and
improve performance. All changes should be backward compatible for
the v4.x series.- However, OCRmyPDF's dependency "ruffus" is not re-entrant, so no
Python API is available. Scripts should continue to use the
command line interface.
- However, OCRmyPDF's dependency "ruffus" is not re-entrant, so no
v4.3.5
- Update documentation to confirm Python 3.6.0 compatibility. No code
changes were needed, so many earlier versions are likely supported.
v4.3.4
- Fixed "decimal.InvalidOperation: quantize result has too many digits"
for high DPI images
v4.3.3
- Fixed PDF/A creation with Ghostscript 9.20 properly
- Fixed an exception on inline stencil masks with a missing optional
parameter
v4.3.2
- Fixed a PDF/A creation issue with Ghostscript 9.20 (note: this fix
did not actually work)
v4.3.1
- Fixed an issue where pages produced by the "hocr" renderer after a
Tesseract timeout would be rotated incorrectly if the input page was
rotated with a /Rotate marker - Fixed a file handle leak in LeptonicaErrorTrap that would cause a
"too many open files" error for files around hundred pages of pages
long when--deskewor--remove-backgroundor other Leptonica
based image processing features were in use, depending on the system
value ofulimit -n - Ability to specify multiple languages for multilingual documents is
now advertised in documentation - Reduced the file sizes of some test resources
- Cleaned up debug output
- Tesseract caching in test cases is now more cautious about false
cache hits and reproducing exact output, not that any problems were
observed
v4.3
New feature
--remove-backgroundto detect and erase the
background of color and grayscale imagesBetter documentation
Fixed an issue with PDFs that draw images when the raster stack depth
is zeroocrmypdf can now redirect its output to stdout for use in a shell
pipeline- This does not improve performance since temporary files are still
used for buffering - Some output validation is disabled in this mode
- This does not improve performance since temporary files are still
v4.2.5
- Fixed an issue
({issue}100) with
PDFs that omit the optional /BitsPerComponent parameter on images - Removed non-free file milk.pdf
v4.2.4
- Fixed an error
({issue}90) caused by
PDFs that use stencil masks properly - Fixed handling of PDFs that try to draw images or stencil masks
without properly setting up the graphics state (such images are now
ignored for the purposes of calculating DPI)
v4.2.3
Fixed an issue with PDFs that store page rotation (/Rotate) in an
indirect objectIntegrated a few fixes to simplify downstream packaging (Debian)
- The test suite no longer assumes it is installed
- If running Linux, skip a test that passes Unicode on the command
line
Added a test case to check explicit masks and stencil masks
Added a test case for indirect objects and linearized PDFs
Deprecated the OCRmyPDF.sh shell script
v4.2.2
- Improvements to documentation
v4.2.1
- Fixed an issue where PDF pages that contained stencil masks would
report an incorrect DPI and cause Ghostscript to abort - Implemented stdin streaming
v4.2
ocrmypdf will now try to convert single image files to PDFs if they
are provided as input
({issue}15)- This is a basic convenience feature. It only supports a single
image and always makes the image fill the whole page. - For better control over image to PDF conversion, use
img2pdf
(one of ocrmypdf's dependencies)
- This is a basic convenience feature. It only supports a single
New argument
--output-type {pdf|pdfa}allows disabling
Ghostscript PDF/A generationpdfais the default, consistent with past behaviorpdfprovides a workaround for users concerned about the
increase in file size from Ghostscript forcing JBIG2 images to
CCITT and transcoding JPEGspdfpreserves as much as it can about the original file,
including problems that PDF/A conversion fixes
PDFs containing images with "non-square" pixel aspect ratios, such as
200x100 DPI, are now handled and converted properly (fixing a bug
that caused to be cropped)--force-ocrrasterizes pages even if they contain no images- supports users who want to use OCRmyPDF to reconstruct text
information in PDFs with damaged Unicode maps (copy and paste text
does not match displayed text) - supports reinterpreting PDFs where text was rendered as curves for
printing, and text needs to be recovered - fixes issue
{issue}82
- supports users who want to use OCRmyPDF to reconstruct text
Fixes an issue where, with certain settings, monochrome images in
PDFs would be converted to 8-bit grayscale, increasing file size
({issue}79)Support for Ubuntu 12.04 LTS "precise" has been dropped in favor of
(roughly) Ubuntu 14.04 LTS "trusty"- Some Ubuntu "PPAs" (backports) are needed to make it work
Support for some older dependencies dropped
- Ghostscript 9.15 or later is now required (available in Ubuntu
trusty with backports) - Tesseract 3.03 or later is now required (available in Ubuntu
trusty)
- Ghostscript 9.15 or later is now required (available in Ubuntu
Ghostscript now runs in "safer" mode where possible
v4.1.4
- Bug fix: monochrome images with an ICC profile attached were
incorrectly converted to full color images if lossless reconstruction
was not possible due to other settings; consequence was increased
file size for these images
v4.1.3
- More helpful error message for PDFs with version 4 security handler
- Update usage instructions for Windows/Docker users
- Fixed order of operations for matrix multiplication (no effect on most
users) - Add a few leptonica wrapper functions (no effect on most users)
v4.1.2
- Replace IEC sRGB ICC profile with Debian's sRGB (from
icc-profiles-free) which is more compatible with the MIT license - More helpful error message for an error related to certain types of
malformed PDFs
v4.1
--rotate-pagesnow only rotates pages when reasonably confidence
in the orientation. This behavior can be adjusted with the new
argument--rotate-pages-threshold- Fixed problems in error checking if
unpaperis uninstalled or
missing at run-time - Fixed problems with "RethrownJobError" errors during error handling
that suppressed the useful error messages
v4.0.7
- Minor correction to Ghostscript output settings
v4.0.6
- Update install instructions
- Provide a sRGB profile instead of using Ghostscript's
v4.0.5
- Remove some verbose debug messages from v4.0.4
- Fixed temporary that wasn't being deleted
- DPI is now calculated correctly for cropped images, along with other
image transformations - Inline images are now checked during DPI calculation instead of
rejecting the image
v4.0.4
Released with verbose debug message turned on. Do not use. Skip to
v4.0.5.
v4.0.3
New features
- Page orientations detected are now reported in a summary comment
Fixes
- Show stack trace if unexpected errors occur
- Treat "too few characters" error message from Tesseract as a reason
to skip that page rather than abort the file - Docker: fix blank JPEG2000 issue by insisting on Ghostscript versions
that have this fixed
v4.0.2
Fixes
- Fixed compatibility with Tesseract 3.04.01 release, particularly its
different way of outputting orientation information - Improved handling of Tesseract errors and crashes
- Fixed use of chmod on Docker that broke most test cases
v4.0.1
Fixes
- Fixed a KeyError if tesseract fails to find page orientation
information
v4.0
New features
- Automatic page rotation (
-r) is now available. It uses ignores
any prior rotation information on PDFs and sets rotation based on the
dominant orientation of detectable text. This feature is fairly
reliable but some false positives occur especially if there is not
much text to work with.
({issue}4) - Deskewing is now performed using Leptonica instead of unpaper.
Leptonica is faster and more reliable at image deskewing than
unpaper.
Fixes
- Fixed an issue where lossless reconstruction could cause some pages
to be appear incorrectly if the page was rotated by the user in
Acrobat after being scanned (specifically if it a /Rotate tag) - Fixed an issue where lossless reconstruction could misalign the
graphics layer with respect to text layer if the page had been
cropped such that its origin is not (0, 0)
({issue}49)
Changes
- Logging output is now much easier to read
--deskewis now performed by Leptonica instead of unpaper
({issue}25)- libffi is now required
- Some changes were made to the Docker and Travis build environments to
support libffi --pdf-renderer=tesseractnow displays a warning if the Tesseract
version is less than 3.04.01, the planned release that will include
fixes to an important OCR text rendering bug in Tesseract 3.04.00.
You can also manually install ./share/sharp2.ttf on top of pdf.ttf in
your Tesseract tessdata folder to correct the problem.
Releasenotes Version05 (`docs/releasenotes/version05.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v5
v5.7.0
Fixed an issue that caused poor CPU utilization on machines with more
than 4 cores when running Tesseract 4. (Related to {issue}217.)The 'hocr' renderer has been improved. The 'sandwich' and 'tesseract'
renderers are still better for most use cases, but 'hocr' may be
useful for people who work with the PDF.js renderer in English/ASCII
languages. ({issue}225)- It now formats text in a matter that is easier for certain PDF
viewers to select and extract copy and paste text. This should
help macOS Preview and PDF.js in particular. - The appearance of selected text and behavior of selecting text is
improved. - The PDF content stream now uses relative moves, making it more
compact and easier for viewers to determine when two words on the
same line. - It can now deal with text on a skewed baseline.
- Thanks to @cforcey for the pull request, @jbreiden for many
helpful suggestions, @ctbarbour for another round of improvements,
and @acaloiaro for an independent review.
- It now formats text in a matter that is easier for certain PDF
v5.6.3
- Suppress two debug messages that were too verbose
v5.6.2
- Development branch accidentally tagged as release. Do not use.
v5.6.1
- Fixed {issue}
219: change
how the final output file is created to avoid triggering permission
errors when the output is a special file such as/dev/null - Fixed test suite failures due to a qpdf 8.0.0 regression and Python
3.5's handling of symlink - The "encrypted PDF" error message was different depending on the type
of PDF encryption. Now a single clear message appears for all types
of PDF encryption. - ocrmypdf is now in Homebrew. Homebrew users are advised to the
version of ocrmypdf in the official homebrew-core formulas rather
than the private tap. - Some linting
v5.6.0
- Fixed {issue}
216: preserve
"text as curves" PDFs without rasterizing file - Related to the above, messages about rasterizing are more consistent
- For consistency versions minor releases will now get the trailing .0
they always should have had.
v5.5
- Add new argument
--max-image-mpixels. Pillow 5.0 now raises an
exception when images may be decompression bombs. This argument can
be used to override the limit Pillow sets. - Fixed output page cropped when using the sandwich renderer and OCR is
skipped on a rotated and image-processed page - A warning is now issued when old versions of Ghostscript are used in
cases known to cause issues with non-Latin characters - Fixed a few parameter validation checks for
-output-type pdfa-1andpdfa-2
v5.4.4
- Fixed {issue}
181: fix
final merge failure for PDFs with more pages than the system file
handle limit (ulimit -n) - Fixed {issue}
200: an
uncommon syntax for formatting decimal numbers in a PDF would cause
qpdf to issue a warning, which ocrmypdf treated as an error. Now this
the warning is relayed. - Fixed an issue where intermediate PDFs would be created at version 1.3
instead of the version of the original file. It's possible but
unlikely this had side effects. - A warning is now issued when older versions of qpdf are used since
issues like
{issue}200cause
qpdf to infinite-loop - Address issue
{issue}140: if
Tesseract outputs invalid UTF-8, escape it and print its message
instead of aborting with a Unicode error - Adding previously unlisted setup requirement, pytest-runner
- Update documentation: fix an error in the example script for Synology
with Docker images, improved security guidance, advisedpip install --user
v5.4.3
- If a subprocess fails to report its version when queried, exit
cleanly with an error instead of throwing an exception - Added test to confirm that the system locale is Unicode-aware and
fail early if it's not - Clarified some copyright information
- Updated pinned requirements.txt so the homebrew formula captures more
recent versions
v5.4.2
- Fixed a regression from v5.4.1 that caused sidecar files to be
created as empty files
v5.4.1
- Add workaround for Tesseract v4.00alpha crash when trying to obtain
orientation and the latest language packs are installed
v5.4
- Change wording of a deprecation warning to improve clarity
- Added option to generate PDF/A-1b output if desired
(--output-type pdfa-1); default remains PDF/A-2b generation - Update documentation
v5.3.3
- Fixed missing error message that should occur when trying to force
--pdf-renderer sandwichon old versions of Tesseract - Update copyright information in test files
- Set system
LANGto UTF-8 in Dockerfiles to avoid UTF-8 encoding
errors
v5.3.2
- Fixed a broken test case related to language packs
v5.3.1
- Fixed wrong return code given for missing Tesseract language packs
- Fixed "brew audit" crashing on Travis when trying to auto-brew
v5.3
- Added
--user-wordsand--user-patternsarguments which are
forwarded to Tesseract OCR as words and regular expressions
respective to use to guide OCR. Supplying a list of subject-domain
words should assist Tesseract with resolving words.
({issue}165) - Using a non Latin-1 language with the "hocr" renderer now warns about
possible OCR quality and recommends workarounds
({issue}176) - Output file path added to error message when that location is not
writable
({issue}175) - Otherwise valid PDFs with leading whitespace at the beginning of the
file are now accepted
v5.2
- When using Tesseract 3.05.01 or newer, OCRmyPDF will select the
"sandwich" PDF renderer by default, unless another PDF renderer is
specified with the--pdf-rendererargument. The previous behavior
was to select--pdf-renderer=hocr. - The "tesseract" PDF renderer is now deprecated, since it can cause
problems with Ghostscript on Tesseract 3.05.00 - The "tess4" PDF renderer has been renamed to "sandwich". "tess4" is
now a deprecated alias for "sandwich".
v5.1
- Files with pages larger than 200" (5080 mm) in either dimension are
now supported with--output-type=pdfwith the page size preserved
(in the PDF specification this feature is called UserUnit scaling).
Due to Ghostscript limitations this is not available in conjunction
with PDF/A output.
v5.0.1
- Fixed {issue}
169,
exception due to failure to create sidecar text files on some
versions of Tesseract 3.04, including the jbarlow83/ocrmypdf Docker
image
v5.0
Backward incompatible changes
- Support for Python 3.4 dropped. Python 3.5 is now required.
- Support for Tesseract 3.02 and 3.03 dropped. Tesseract 3.04 or
newer is required. Tesseract 4.00 (alpha) is supported. - The OCRmyPDF.sh script was removed.
Add a new feature,
--sidecar, which allows creating "sidecar"
text files which contain the OCR results in plain text. These OCR
text is more reliable than extracting text from PDFs. Closes
{issue}126.New feature:
--pdfa-image-compression, which allows overriding
Ghostscript's lossy-or-lossless image encoding heuristic and making
all images JPEG encoded or lossless encoded as desired. Fixes
{issue}163.Fixed {issue}
143, added--quietto suppress "INFO" messagesFixed {issue}
164, a typoRemoved the command line parameters
-nand--just-printsince
they have not worked for some time (reported as Ubuntu bug
#1687308)
Releasenotes Version06 (`docs/releasenotes/version06.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v6
v6.2.5
- Disable a failing test due to Tesseract 4.0rc1 behavior change.
Previously, Tesseract would exit with an error message if its
configuration was invalid, and OCRmyPDF would intercept this message.
Now Tesseract issues a warning, which OCRmyPDF v6.2.5 may relay or
ignore. (In v7.x, OCRmyPDF will respond to the warning.) - This release branch no longer supports using the optional PyMuPDF
installation, since it was removed in v7.x. - This release branch no longer supports macOS. macOS users should
upgrade to v7.x.
v6.2.4
- Backport Ghostscript 9.25 compatibility fixes, which removes support
for setting Unicode metadata - Backport blacklisting Ghostscript 9.24
- Older versions of Ghostscript are still supported
v6.2.3
- Fixed compatibility with img2pdf >= 0.3.0 by rejecting input images
that have an alpha channel - This version will be included in Ubuntu 18.10
v6.2.2
- Backport compatibility fixes for Python 3.7 and ruffus 2.7.0 from
v7.0.0 - Backport fix to ignore masks when deciding what colors are on a page
- Backport some minor improvements from v7.0.0: better argument
validation and warnings about the Tesseract 4.0.0--user-words
regression
v6.2.1
- Fixed recent versions of Tesseract (after 4.0.0-beta1) not being
detected as supporting thesandwichrenderer ({issue}271).
v6.2.0
- Docker: The Docker image
ocrmypdf-tess4has been removed. The
main Docker images,ocrmypdfandocrmypdf-polyglotnow use
Ubuntu 18.04 as a base image, and as such Tesseract 4.0.0-beta1 is
now the Tesseract version they use. There is no Docker image based on
Tesseract 3.05 anymore. - Creation of PDF/A-3 is now supported. However, there is no ability to
attach files to PDF/A-3. - Lists more reasons why the file size might grow.
- Fixed {issue}
262,--remove-backgrounderror on PDFs contained colormapped
(paletted) images. - Fixed another XMP metadata validation issue, in cases where the input
file's creation date has no timezone and the creation date is not
overridden.
v6.1.5
- Fixed {issue}
253, a
possible division by zero when using thehocrrenderer. - Fixed incorrectly formatted
<xmp:ModifyDate>field inside XMP
metadata for PDF/As. veraPDF flags this as a PDF/A validation
failure. The error is caused the timezone and final digit of the
seconds of modified time to be omitted, so at worst the modification
time stamp is rounded to the nearest 10 seconds.
v6.1.4
- Fixed {issue}
248--cleanargument may remove OCR from left column of text on
certain documents. We now set--layout noneto suppress this. - The test cache was updated to reflect the change above.
- Change test suite to accommodate Ghostscript 9.23's new ability to
insert JPEGs into PDFs without transcoding. - XMP metadata in PDFs is now examined using
defusedxmlfor safety. - If an external process exits with a signal when asked to report its
version, we now print the system error message instead of suppressing
it. This occurred when the required executable was found but was
missing a shared library. - qpdf 7.0.0 or newer is now required as the test suite can no longer
pass without it.
Notes
- An apparent regression in Ghostscript
9.23 will
cause some ocrmypdf output files to become invalid in rare cases; the
workaround for the moment is to set--force-ocr.
v6.1.3
- Fixed {issue}
247,/CreationDatemetadata not copied from input to output. - A warning is now issued when Python 3.5 is used on files with a large
page count, as this case is known to regress to single core
performance. The cause of this problem is unknown.
v6.1.2
- Upgrade to PyMuPDF v1.12.5 which includes a more complete fix to
{issue}239. - Add
defusedxmldependency.
v6.1.1
- Fixed text being reported as found on all pages if PyMuPDF is not
installed.
v6.1.0
- PyMuPDF is now an optional but recommended dependency, to alleviate
installation difficulties on platforms that have less access to
PyMuPDF than the author anticipated. (For version 6.x only) install
OCRmyPDF withpip install ocrmypdf[fitz]to use it to its full
potential. - Fixed
FileExistsErrorthat could occur if OCR timed out while it
was generating the output file.
({issue}218) - Fixed table of contents/bookmarks all being redirected to page 1 when
generating a PDF/A (with PyMuPDF). (Without PyMuPDF the table of
contents is removed in PDF/A mode.) - Fixed "RuntimeError: invalid key in dict" when table of
contents/bookmarks titles contained the character).
({issue}239) - Added a new argument
--skip-repairto skip the initial PDF repair
step if the PDF is already well-formed (because another program
repaired it).
v6.0.0
The software license has been changed to GPLv3 [it has since changed again].
Test resource files and some individual sources may have other licenses.OCRmyPDF now depends on
PyMuPDF.
Including PyMuPDF is the primary reason for the change to GPLv3.Other backward incompatible changes
- The
OCRMYPDF_TESSERACT,OCRMYPDF_QPDF,OCRMYPDF_GSandOCRMYPDF_UNPAPERenvironment variables are no longer used.
ChangePATHif you need to override the external programs
OCRmyPDF uses. - The
ocrmypdfpackage has been moved tosrc/ocrmypdfto
avoid issues with accidental import. - The function
ocrmypdf.exec.get_programwas removed. - The deprecated module
ocrmypdf.pageinfowas removed. - The
--pdf-renderer tess4alias forsandwichwas removed.
- The
Fixed an issue where OCRmyPDF failed to detect existing text on
pages, depending on how the text and fonts were encoded within the
PDF. ({issue}233,232)Fixed an issue that caused dramatic inflation of file sizes when
--skip-text --output-type pdfwas used. OCRmyPDF now removes
duplicate resources such as fonts, images and other objects that it
generates. ({issue}237)Improved performance of the initial page splitting step. Originally
this step was not believed to be expensive and ran in a process.
Large file testing revealed it to be a bottleneck, so it is now
parallelized. On a 700 page file with quad core machine, this change
saves about 2 minutes. ({issue}234)The test suite now includes a cache that can be used to speed up test
runs across platforms. This also does not require computing
checksums, so it's faster. ({issue}217)
Releasenotes Version07 (`docs/releasenotes/version07.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v7
v7.4.0
--force-ocrmay now be used with the new--thresholdand--mask-barcodesfeatures- pikepdf >= 0.9.1 is now required.
- Changed metadata handling to pikepdf 0.9.1. As a result, metadata
handling of non-ASCII characters in Ghostscript 9.25 or later is
fixed. - chardet >= 3.0.4 is temporarily listed as required. pdfminer.six
depends on it, but the most recent release does not specify this
requirement.
({issue}326) - python-xmp-toolkit and libexempi are no longer required.
- A new Docker image is now being provided for users who wish to access
OCRmyPDF over a simple HTTP interface, instead of the command line. - Increase tolerance of PDFs that overflow or underflow the PDF
graphics stack.
({issue}325)
v7.3.1
- Fixed performance regression from v7.3.0; fast page analysis was not
selected when it should be. - Fixed a few exceptions related to the new
--mask-barcodesfeature
and improved argument checking - Added missing detection of TrueType fonts that lack a Unicode mapping
v7.3.0
Added a new feature
--redo-ocrto detect existing OCR in a file,
remove it, and redo the OCR. This may be particularly helpful for
anyone who wants to take advantage of OCR quality improvements in
Tesseract 4.0. Note that OCR added by OCRmyPDF before version 3.0
cannot be detected since it was not properly marked as invisible text
in the earliest versions. OCR that constructs a font from visible
text, such as Adobe Acrobat's ClearScan.OCRmyPDF's content detection is generally more sophisticated. It
learns more about the contents of each PDF and makes better
recommendations:- OCRmyPDF can now detect when a PDF contains text that cannot be
mapped to Unicode (meaning it is readable to human eyes but
copy-pastes as gibberish). In these cases it recommends--force-ocrto make the text searchable. - PDFs containing vector objects are now rendered at more
appropriate resolution for OCR. - We now exit with an error for PDFs that contain Adobe LiveCycle
Designer's dynamic XFA forms. Currently the open source community
does not have tools to work with these files. - OCRmyPDF now warns when a PDF that contains Adobe AcroForms, since
such files probably do not need OCR. It can work with these files.
- OCRmyPDF can now detect when a PDF contains text that cannot be
Added three new experimental features to improve OCR quality in
certain conditions. The name, syntax and behavior of these arguments
is subject to change. They may also be incompatible with some other
features.--remove-vectorswhich strips out vector graphics. This can
improve OCR quality since OCR will not search artwork for readable
text; however, it currently removes "text as curves" as well.--mask-barcodesto detect and suppress barcodes in files. We
have observed that barcodes can interfere with OCR because they
are "text-like" but not actually textual.--thresholdwhich uses a more sophisticated thresholding
algorithm than is currently in use in Tesseract OCR. This works
around a known issue in Tesseract
4.0
with dark text on bright backgrounds.
Fixed an issue where an error message was not reported when the
installed Ghostscript was very old.The PDF optimizer now saves files with object streams enabled when
the optimization level is--optimize 1or higher (the default).
This makes files a little bit smaller, but requires PDF 1.5. PDF 1.5
was first released in 2003 and is broadly supported by PDF viewers,
but some rudimentary PDF parsers such as PyPDF2 do not understand
object streams. You can use the command line toolqpdf --object-streams=disableor
pikepdf library to remove
them.New dependency: pdfminer.six 20181108. Note this is a fork of the
Python 2-only pdfminer.Deprecation notice: At the end of 2018, we will be ending support for
Python 3.5 and Tesseract 3.x. OCRmyPDF v7 will continue to work with
older versions.
v7.2.1
- Fixed compatibility with an API change in pikepdf 0.3.5.
- A kludge to support Leptonica versions older than 1.72 in the test
suite was dropped. Older versions of Leptonica are likely still
compatible. The only impact is that a portion of the test suite will
be skipped.
v7.2.0
Lossy JBIG2 behavior change
A user reported that ocrmypdf was in fact using JBIG2 in lossy
compression mode. This was not the intended behavior. Users should
review the technical concerns with JBIG2 in lossy
mode
and decide if this is a concern for their use case.
JBIG2 lossy mode does achieve higher compression ratios than any other
monochrome compression technology; for large text documents the savings
are considerable. JBIG2 lossless still gives great compression ratios
and is a major improvement over the older CCITT G4 standard.
Only users who have reviewed the concerns with JBIG2 in lossy mode
should opt-in. As such, lossy mode JBIG2 is only turned on when the new
argument --jbig2-lossy is issued. This is independent of the setting
for --optimize.
Users who did not install an optional JBIG2 encoder are unaffected.
(Thanks to user 'bsdice' for reporting this issue.)
Other issues
- When the image optimizer quantizes an image to 1 bit per pixel, it
will now attempt to further optimize that image as CCITT or JBIG2,
instead of keeping it in the "flate" encoding which is not efficient
for 1 bpp images.
({issue}297) - Images in PDFs that are used as soft masks (i.e. transparency masks
or alpha channels) are now excluded from optimization. - Fixed handling of Tesseract 4.0-rc1 which now accepts invalid
Tesseract configuration files, which broke the test suite.
v7.1.0
- Improve the performance of initial text extraction, which is done to
determine if a file contains existing text of some kind or not. On
large files, this initial processing is now about 20x times faster.
({issue}299) - pikepdf 0.3.3 is now required.
- Fixed {issue}
231, a
problem with JPEG2000 images where image metadata was only available
inside the JPEG2000 file. - Fixed some additional Ghostscript 9.25 compatibility issues.
- Improved handling of KeyboardInterrupt error messages.
({issue}301) - README.md is now served in GitHub markdown instead of
reStructuredText.
v7.0.6
- Blacklist Ghostscript 9.24, now that 9.25 is available and fixes many
regressions in 9.24.
v7.0.5
- Improve capability with Ghostscript 9.24, and enable the JPEG
passthrough feature when this version in installed. - Ghostscript 9.24 lost the ability to set PDF title, author, subject
and keyword metadata to Unicode strings. OCRmyPDF will set ASCII
strings and warn when Unicode is suppressed. Other software may be
used to update metadata. This is a short term work around. - PDFs generated by Kodak Capture Desktop, or generally PDFs that
contain indirect references to null objects in their table of
contents, would have an invalid table of contents after processing by
OCRmyPDF that might interfere with other viewers. This has been
fixed. - Detect PDFs generated by Adobe LiveCycle, which can only be displayed
in Adobe Acrobat and Reader currently. When these are encountered,
exit with an error instead of performing OCR on the "Please wait"
error message page.
v7.0.4
- Fixed exception thrown when trying to optimize a certain type of PNG
embedded in a PDF with the-O2 - Update to pikepdf 0.3.2, to gain support for optimizing some
additional image types that were previously excluded from
optimization (CMYK and grayscale). Fixes
{issue}285.
v7.0.3
- Fixed {issue}
284, an error
when parsing inline images that have are also image masks, by
upgrading pikepdf to 0.3.1
v7.0.2
- Fixed a regression with
--rotate-pageson pages that already had
rotations applied.
({issue}279) - Improve quality of page rotation in some cases by rasterizing a
higher quality preview image.
({issue}281)
v7.0.1
- Fixed compatibility with img2pdf >= 0.3.0 by rejecting input images
that have an alpha channel - Add forward compatibility for pikepdf 0.3.0 (unrelated to img2pdf)
- Various documentation updates for v7.0.0 changes
v7.0.0
The core algorithm for combining OCR layers with existing PDF pages
has been rewritten and improved considerably. PDFs are no longer
split into single page PDFs for processing; instead, images are
rendered and the OCR results are grafted onto the input PDF. The new
algorithm uses less temporary disk space and is much more performant
especially for large files.New dependency: pikepdf.
pikepdf is a powerful new Python PDF library driving the latest
OCRmyPDF features, built on the QPDF C++ library (libqpdf).New feature: PDF optimization with
-Oor--optimize. After
OCR, OCRmyPDF will perform image optimizations relevant to OCR PDFs.- If a JBIG2 encoder is available, then monochrome images will be
converted, with the potential for huge savings on large black and
white images, since JBIG2 is far more efficient than any other
monochrome (bi-level) compression. (All known US patents related
to JBIG2 have probably expired, but it remains the responsibility
of the user to supply a JBIG2 encoder such as
jbig2enc. OCRmyPDF does not
implement JBIG2 encoding.) - If
pngquantis installed, OCRmyPDF will optionally use it to
perform lossy quantization and compression of PNG images. - The quality of JPEGs can also be lowered, on the assumption that a
lower quality image may be suitable for storage after OCR. - This image optimization component will eventually be offered as an
independent command line utility. - Optimization ranges from
-O0through-O3, where0
disables optimization and3implements all options.1, the
default, performs only safe and lossless optimizations. (This is
similar to GCC's optimization parameter.) The exact type of
optimizations performed will vary over time.
- If a JBIG2 encoder is available, then monochrome images will be
Small amounts of text in the margins of a page, such as watermarks,
page numbers, or digital stamps, will no longer prevent the rest of a
page from being OCRed when--skip-textis issued. This behavior
is based on a heuristic.Removed features
- The deprecated
--pdf-renderer tesseractPDF renderer was
removed. -g, the option to generate debug text pages, was removed
because it was a maintenance burden and only worked in isolated
cases. HOCR pages can still be previewed by running the
hocrtransform.py with appropriate settings.
- The deprecated
Removed dependencies
PyPDF2defusedxmlPyMuPDF
The
sandwichPDF renderer can be used with all supported versions
of Tesseract, including that those prior to v3.05 which don't support-c textonly. (Tesseract v4.0.0 is recommended and more
efficient.)--pdf-renderer autooption and the diagnostics used to select a
PDF renderer now work better with old versions, but may make
different decisions than past versions.If everything succeeds but PDF/A conversion fails, a distinct return
code is now returned (ExitCode.pdfa_conversion_failed (10)) where
this situation previously returnedExitCode.invalid_output_pdf (4). The latter is now returned only
if there is some indication that the output file is invalid.Notes for downstream packagers
- There is also a new dependency on
python-xmp-toolkitwhich in
turn depends onlibexempi3. - It may be necessary to separately
pip install pycparserto
avoid another Python 3.7
issue.
- There is also a new dependency on
Releasenotes Version08 (`docs/releasenotes/version08.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v8
v8.3.2
- Dropped workaround for macOS that allowed it work without pdfminer.six,
now a proper sdist release of pdfminer.six is available. - pikepdf 1.5.0 is now required.
v8.3.1
- Fixed an issue where PDFs with malformed metadata would be rendered as
blank pages. {issue}398.
v8.3.0
- Improved the strategy for updating pages when a new image of the page
was produced. We now attempt to preserve more content from the
original file, for annotations in particular. - For PDFs with more than 100 pages and a sequence where one PDF page
was replaced and one or more subsequent ones were skipped, an
intermediate file would be corrupted while grafting OCR text, causing
processing to fail. This is a regression, likely introduced in
v8.2.4. - Previously, we resized the images produced by Ghostscript by a small
number of pixels to ensure the output image size was an exactly what
we wanted. Having discovered a way to get Ghostscript to produce the
exact image sizes we require, we eliminated the resizing step. - Command line completions for
bashare now available, in addition
tofish, both inmisc/completion. Package maintainers, please
install these so users can take advantage. - Updated requirements.
- pikepdf 1.3.0 is now required.
v8.2.4
- Fixed a false positive while checking for a certain type of PDF that
only Acrobat can read. We now more accurately detect Acrobat-only
PDFs. - OCRmyPDF holds fewer open file handles and is more prompt about
releasing those it no longer needs. - Minor optimization: we no longer traverse the table of contents to
ensure all references in it are resolved, as changes to libqpdf have
made this unnecessary. - pikepdf 1.2.0 is now required.
v8.2.3
- Fixed that
--mask-barcodeswould occasionally leave a unwanted
temporary file namedjunkpixtin the current working folder. - Fixed (hopefully) handling of Leptonica errors in an environment
where a non-standardsys.stderris present. - Improved help text for
--verbose.
v8.2.2
- Fixed a regression from v8.2.0, an exception that occurred while
attempting to report thatunpaperor another optional dependency
was unavailable. - In some cases,
ocrmypdf [-c|--clean]failed to exit with an error
whenunpaperis not installed.
v8.2.1
- This release was canceled.
v8.2.0
- A major improvement to our Docker image is now available thanks to
hard work contributed by @mawi12345. The new Docker image,
ocrmypdf-alpine, is based on Alpine Linux, and includes most of the
functionality of three existed images in a smaller package. This
image will replace the main Docker image eventually but for now all
are being built. See documentation for
details. - Documentation reorganized especially around the use of Docker images.
- Fixed a problem with PDF image optimization, where the optimizer
would unnecessarily decompress and recompress PNG images, in some
cases losing the benefits of the quantization it just had just
performed. The optimizer is now capable of embedding PNG images into
PDFs without transcoding them. - Fixed a minor regression with lossy JBIG2 image optimization. All
JBIG2 candidates images were incorrectly placed into a single
optimization group for the whole file, instead of grouping pages
together. This usually makes a larger JBIG2Globals dictionary and
results in inferior compression, so it worked less well than
designed. However, quality would not be impacted. Lossless JBIG2 was
entirely unaffected. - Updated dependencies, including pikepdf to 1.1.0. This fixes
{issue}358. - The install-time version checks for certain external programs have
been removed from setup.py. These tests are now performed at
run-time. - The non-standard option to override install-time checks
(setup.py install --force) is now deprecated and prints a
warning. It will be removed in a future release.
v8.1.0
- Added a feature,
--unpaper-args, which allows passing arbitrary
arguments tounpaperwhen using--cleanor--clean-final.
The default, very conservative unpaper settings are suppressed. - The argument
--clean-finalnow implies--clean. It was
possible to issue--clean-finalon its before this, but it would
have no useful effect. - Fixed an exception on traversing corrupt table of contents entries
(specifically, those with invalid destination objects) - Fixed an issue when using
--tesseract-timeoutand image
processing features on a file with more than 100 pages.
{issue}347 - OCRmyPDF now always calls
os.nice(5)to signal to operating
systems that it is a background process.
v8.0.1
- Fixed an exception when parsing PDFs that are missing a required
field. {issue}325 - pikepdf 1.0.5 is now required, to address some other PDF parsing
issues.
v8.0.0
No major features. The intent of this release is to sever support for
older versions of certain dependencies.
Breaking changes
- Dropped support for Tesseract 3.x. Tesseract 4.0 or newer is now
required. - Dropped support for Python 3.5.
- Some
ocrmypdf.pdfaAPIs that were deprecated in v7.x were
removed. This functionality has been moved to pikepdf.
Other changes
- Fixed an unhandled exception when attempting to mask barcodes.
{issue}322 - It is now possible to use ocrmypdf without pdfminer.six, to support
distributions that do not have it or cannot currently use it (e.g.
Homebrew). Downstream maintainers should include pdfminer.six if
possible. - A warning is now issue when PDF/A conversion removes some XMP
metadata from the input PDF. (Only a "whitelist" of certain XMP
metadata types are allowed in PDF/A.) - Fixed several issues that caused PDF/As to be produced with
nonconforming XMP metadata (would fail validation with veraPDF). - Fixed some instances where invalid DocumentInfo from a PDF cause XMP
metadata creation to fail. - Fixed a few documentation problems.
- pikepdf 1.0.2 is now required.
Releasenotes Version09 (`docs/releasenotes/version09.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v9
v9.8.2
- Fixed an issue where OCRmyPDF would ignore text inside Form XObject when
making certain decisions about whether a document already had text. - Fixed file size increase warning to take overhead of small files into account.
- Added instructions for installing on Cygwin.
v9.8.1
- Fixed an issue where unexpected files in the
%PROGRAMFILES%\gsdirectory
(Windows) caused an exception. - Mark pdfminer.six 20200517 as supported.
- If jbig2enc is missing and optimization is requested, a warning is issued
instead of an error, which was the intended behavior. - Documentation updates.
v9.8.0
- Fixed issue where only the first PNG (FlateDecode) image in a file would be
considered for optimization. File sizes should be improved from here on. - Fixed a startup crash when the chosen language was Japanese ({issue}
543). - Added options to configure polling and log level to watcher.py.
v9.7.2
- Fixed an issue with
ocrmypdf.ocr(...language=)not accepting a list of
languages as documented. - Updated setup.py to confirm that pdfminer.six version 20200402 is supported.
v9.7.1
- Fixed version check failing when used with qpdf 10.0.0.
- Added some missing type annotations.
- Updated documentation to warn about need for "ifmain" guard and Windows.
v9.7.0
- Fixed an error in watcher.py if
OCR_JSON_SETTINGSwas not defined. - Ghostscript 9.51 is now blacklisted, due to numerous problems with this version.
- Added a workaround for a problem with "txtwrite" in Ghostscript 9.52.
- Fixed an issue where the incorrect number of threads used was shown when
OMP_THREAD_LIMITwas manipulated. - Removed a possible performance bottlenecks for files that use hundreds to
thousands of images on the same page. - Documentation improvements.
- Optimization will now be applied to some monochrome images that have a color
profile defined instead of only black and white. - ICC profiles are consulted when determining the simplified colorspace of an
image.
v9.6.1
Documentation improvements - thanks to many users for their contributions!
- Fixed installation instructions for ArchLinux (@pigmonkey)
- Updated installation instructions for FreeBSD and other OSes (@knobix)
- Added instructions for using Docker Compose with watchdog (@ianalexander,
@deisi) - Other miscellany (@mb720, @toy, @caiofacchinato)
- Some scripts provided in the documentation have been migrated out so that
they can be copied out as whole files, and to ensure syntax checking
is maintained.
Fixed an error that caused bash completions to fail on macOS. ({issue}
502,504;
@AlexanderWillner)Fixed a rare case where OCRmyPDF threw an exception while processing a PDF
with the wrong object type in its/Trailer /Info. The error is now logged
and incorrect object is ignored. ({issue}497)Removed potentially non-free file
enron1.pdfand simplified the test that
used it.Removed potentially non-free file
misc/media/logo.afdesign.
v9.6.0
- Fixed a regression with transferring metadata from the input PDF to the output
PDF in certain situations. - pdfminer.six is now supported up to version 2020-01-24.
- Messages are explaining page rotation decisions are now shown at the standard
verbosity level again when--rotate-pages. In some previous version they
were set to debug level messages that only appeared with the parameter-v1. - Improvements to
misc/watcher.py. Thanks to @ianalexander and @svenihoney. - Documentation improvements.
v9.5.0
- Added API functions to measure OCR quality.
- Modest improvements to handling PDFs with difficult/non compliant metadata.
v9.4.0
- Updated recommended dependency versions.
- Improvements to test coverage and changes to facilitate better measurement of
test coverage, such as when tests run in subprocesses. - Improvements to error messages when Leptonica is not installed correctly.
- Fixed use of pytest "session scope" that may have caused some intermittent
CI failures. - When the argument
--keep-temporary-filesor verbosity is set to-v1,
a debug log file is generated in the working temporary folder.
v9.3.0
- Improved native Windows support: we now check in the obvious places in
the "Program Files" folders installations of Tesseract and Ghostscript,
rather than relying on the user to editPATHto specify their location.
ThePATHenvironment variable can still be used to differentiate when
multiple installations are present or the programs are installed to non-
standard locations. - Fixed an exception on parsing Ghostscript error messages.
- Added an improved example demonstrating how to set up a watched folder
for automated OCR processing (thanks to @ianalexander for the contribution).
v9.2.0
- Native Windows is now supported.
- Continuous integration moved to Azure Pipelines.
- Improved test coverage and speed of tests.
- Fixed an issue where a page that was originally a JPEG would be saved as a
PNG, increasing file size. This occurred only when a preprocessing option
was selected along with--output-type=pdfand all images on the original
page were JPEGs. Regression since v7.0.0. - OCRmyPDF no longer depends on the QPDF executable
qpdforlibqpdf.
It uses pikepdf (which in turn depends onlibqpdf). Package maintainers
should adjust dependencies so that OCRmyPDF no longer calls for libqpdf on
its own. For users of Python binary wheels, this change means a separate
installation of QPDF is no longer necessary. This change is mainly to
simplify installation on Windows. - Fixed a rare case where log messages from Tesseract would be discarded.
- Fixed incorrect function signature for pixFindPageForeground, causing
exceptions on certain platforms/Leptonica versions.
v9.1.1
- Expand the range of pdfminer.six versions that are supported.
- Fixed Docker build when using pikepdf 1.7.0.
- Fixed documentation to recommend using pip from get-pip.py.
v9.1.0
- Improved diagnostics when file size increases at output. Now warns if JBIG2
or pngquant were not available. - pikepdf 1.7.0 is now required, to pick up changes that remove the need for
a source install on Linux systems running Python 3.8.
v9.0.5
- The Alpine Docker image (jbarlow83/ocrmypdf-alpine) has been dropped due to
the difficulties of supporting Alpine Linux. - The primary Docker image (jbarlow83/ocrmypdf) has been improved to take on
the extra features that used to be exclusive to the Alpine image. - No changes to application code.
- pdfminer.six version 20191020 is now supported.
v9.0.4
- Fixed compatibility with Python 3.8 (but requires source install for the moment).
- Fixed Tesseract settings for
--user-wordsand--user-patterns. - Changed to pikepdf 1.6.5 (for Python 3.8).
- Changed to Pillow 6.2.0 (to mitigate a security vulnerability in earlier Pillow).
- A debug message now mentions when English is automatically selected if the locale
is not English.
v9.0.3
- Embed an encoded version of the sRGB ICC profile in the intermediate
Postscript file (used for PDF/A conversion). Previously we included the
filename, which required Postscript to run with file access enabled. For
security, Ghostscript 9.28 enables-dSAFERand as such, no longer
permits access to any file by default. This fix is necessary for
compatibility with Ghostscript 9.28. - Exclude a test that sometimes times out and fails in continuous integration
from the standard test suite.
v9.0.2
- The image optimizer now skips optimizing flate (PNG) encoded images in some
situations where the optimization effort was likely wasted. - The image optimizer now ignores images that specify arbitrary decode arrays,
since these are rare. - Fixed an issue that caused inversion of black and white in monochrome images.
We are not certain but the problem seems to be linked to Leptonica 1.76.0 and
older. - Fixed some cases where the test suite failed if
English or German Tesseract language packs were not installed. - Fixed a runtime error if the Tesseract English language is not installed.
- Improved explicit closing of Pillow images after use.
- Actually fixed of Alpine Docker image build.
- Changed to pikepdf 1.6.3.
v9.0.1
- Fixed test suite failing when either of optional dependencies unpaper and
pngquant were missing. - Attempted fix of Alpine Docker image build.
- Documented that FreeBSD ports are now available.
- Changed to pikepdf 1.6.1.
v9.0.0
Breaking changes
- The
--mask-barcodesexperimental feature has been dropped due to poor
reliability and occasional crashes, both due to the underlying library that
implements this feature (Leptonica). - The
-v(verbosity level) parameter now accepts only0,1, and2. - Dropped support for Tesseract 4.00.00-alpha releases. Tesseract 4.0 beta and
later remain supported. - Dropped the
ocrmypdf-polyglotandocrmypdf-webserviceimages.
New features
- Added a high level API for applications that want to integrate OCRmyPDF.
Special thanks to Martin Wind (@mawi1988) whose made significant contributions
to this effort. - Added progress bars for long-running steps. ■■■■■■■□□
- We now create linearized ("fast web view") PDFs by default. The new parameter
--fast-web-viewprovides control over when this feature is applied. - Added a new
--pagesfeature to limit OCR to only a specific page range.
The list may contain commas or single pages, such as1, 3, 5-11. - When the number of pages is small compared to the number of allowed jobs, we
run Tesseract in multithreaded (OpenMP) mode when available. This should
improve performance on files with low page counts. - Removed dependency on
ruffus, and with that, the non-reentrancy
restrictions that previous made an API impossible. - Output and logging messages overhauled so that ocrmypdf may be integrated
into applications that use the logging module. - pikepdf 1.6.0 is required.
- Added a logo. 😊
Bug fixes
- Pages with vector artwork are treated as full color. Previously, vectors
were ignored when considering the colorspace needed to cover a page, which
could cause loss of color under certain settings. - Test suite now spawns processes less frequently, allowing more accurate
measurement of code coverage. - Improved test coverage.
- Fixed a rare division by zero (if optimization produced an invalid file).
- Updated Docker images to use newer versions.
- Fixed images encoded as JBIG2 with a colorspace other than
/DeviceGray
were not interpreted correctly. - Fixed a OCR text-image registration (i.e. alignment) problem when the page
when MediaBox had a nonzero corner.
Releasenotes Version10 (`docs/releasenotes/version10.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v10
v10.3.3
- Fixed a "KeyError: 'dpi'" error message when using
--thresholdon an image.
({issue}607)
v10.3.2
- Fixed a case where we reported "no reason" for a file size increase, when we
could determine the reason. - Enabled support for pdfminer.six 20200726.
v10.3.1
- Fixed a number of test suite failures with pdfminer.six older than version 20200402.
- Enabled support for pdfminer.six 20200720.
v10.3.0
- Fixed an issue where we would consider images that were already JBIG2-encoded
for optimization, potentially producing a less optimized image than the original.
We do not believe this issue would ever cause an image to loss fidelity. - Where available, pikepdf memory mapping is now used. This improves performance.
- When Leptonica 1.79+ is installed, use its new error handling API to avoid
a "messy" redirection of stderr which was necessary to capture its error
messages. - For older versions of Leptonica, added a new thread level lock. This fixes a
possible race condition in handling error conditions in Leptonica (although
there is no evidence it ever caused issues in practice). - Documentation improvements and more type hinting.
v10.2.1
- Disabled calculation of text box order with pdfminer. We never needed this result
and it is expensive to calculate on files with complex pre-existing text. - Fixed plugin manager to accept
Path(plugin)as a path to a plugin. - Fixed some typing errors.
- Documentation improvements.
v10.2.0
- Update Docker image to use Ubuntu 20.04.
- Fixed issue PDF/A acquires title "Untitled" after conversion. ({issue}
582) - Fixed a problem where, when using
--pdf-renderer hocr, some text would
be missing from the output when using a more recent version of Tesseract.
Tesseract began adding more detailed markup about the semantics of text
that our HOCR transform did not recognize, so it ignored them. This option is
not the default. If necessary--redo-ocralso redoing OCR to fix such issues. - Fixed an error in Python 3.9 beta, due to removal of deprecated
Element.getchildren(). ({issue}584) - Implemented support using the API with
BytesIOand other file stream objects.
({issue}545)
v10.1.1
- Fixed
OMP_THREAD_LIMITset to invalid value error messages on some input
files. (The error was harmless, apart from less than optimal performance in
some cases.)
v10.1.0
- Previously, we
--clean-finalwould cause an unpaper-cleaned page image to
be produced twice, which was necessary in some cases but not in general. We
now take this optimization opportunity and reuse the image if possible. - We now provide PNG files as input to unpaper, since it accepts them, instead
of generating PPM files which can be very large. This can improve performance
and temporary disk usage. - Documentation updated for plugins.
v10.0.1
- Fixed regression when
-l lang1+lang2is used from command line.
v10.0.0
Breaking changes
- Support for pdfminer.six version 20181108 has been dropped, along with a
monkeypatch that made this version work. - Output messages are now displayed in color (when supported by the terminal)
and prefixes describing the severity of the message are removed. As such
programs that parse OCRmyPDF's log message will need to be revised. (Please
consider using OCRmyPDF as a library instead.) - The minimum version for certain dependencies has increased.
- Many API changes; see developer changes.
- The Python libraries pluggy and coloredlogs are now required.
New features and improvements
- PDF page scanning is now parallelized across CPUs, speeding up this phase
dramatically for files with a high page counts. - PDF page scanning is optimized, addressing some performance regressions.
- PDF page scanning is no longer run on pages that are not selected when the
--pagesargument is used. - PDF page scanning is now independent of Ghostscript, ending our past reliance
on this occasionally unstable feature in Ghostscript. - A plugin architecture has been added, currently allowing one to more easily
use a different OCR engine or PDF renderer from Tesseract and Ghostscript,
respectively. A plugin can also override some decisions, such changing
the OCR settings after initial scanning. - Colored log messages.
Developer changes
- The test spoofing mechanism, used to test correct handling of failures in
Tesseract and Ghostscript, has been removed in favor of using plugins for
testing. The spoofing mechanism was fairly complex and required many special
hacks for Windows. - Code describing the resolution in DPI of images was refactored into a
ocrmypdf.helpers.Resolutionclass. - The module
ocrmypdf._execis now private to OCRmyPDF. - The
ocrmypdf.hocrtransformmodule has been updated to follow PEP8 naming
conventions. - Ghostscript is no longer used for finding the location of text in PDFs, and
APIs related to this feature have been removed. - Lots of internal reorganization to support plugins.
Releasenotes Version11 (`docs/releasenotes/version11.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v11
v11.7.3
- Exclude CCITT Group 3 images from being optimized. Some libraries
OCRmyPDF uses do not seem to handle this obscure compression format properly.
You may get errors or possible corrupted output images without this fix.
v11.7.2
- Updated pinned versions in main.txt, primarily to upgrade Pillow to 8.1.2, due
to recently disclosed security vulnerabilities in that software. - The
--sidecarparameter now causes an exception if set to the same file as
the input or output PDF.
v11.7.1
- Some exceptions while attempting image optimization were only logged at the debug
level, causing them to be suppressed. These errors are now logged appropriately. - Improved the error message related to
--unpaper-args. - Updated documentation to mention the new conda distribution.
v11.7.0
- We now support using
--sidecarin conjunction with--pages; these arguments
used to be mutually exclusive. ({issue}735) - Fixed a possible issue with PDF/A-1b generation. Acrobat complained that our PDFs use
object streams. More robust PDF/A validators like veraPDF don't consider this a
problem, but we'll honor Acrobat's objection from here on. This may increase file
size of PDF/A-1b files. PDF/A-2b files will not be affected.
v11.6.2
- Fixed a regression where the wrong page orientation would be produced when using
arguments such as--deskew --rotate-pages({issue}730).
v11.6.1
- Fixed an issue with attempting optimize unusually narrow-width images by excluding
these images from optimization ({issue}732). - Remove an obsolete compatibility shim for a version of pikepdf that is no longer
supported.
v11.6.0
- OCRmyPDF will now automatically register plugins from the same virtual environment
with an appropriate setuptools entrypoint. - Refactor the plugin manager to remove unnecessary complications and make plugin
registration more automatic. PageContextandPdfContextare now formally part of the API, as they
should have been, since they were part ofocrmypdf.pluginspec.
v11.5.0
- Fixed an issue where the output page size might differ by a fractional amount
due to rounding, when--force-ocrwas used and the page contained objects
with multiple resolutions. - When determining the resolution at which to rasterize a page, we now consider
printed text on the page as requiring a higher resolution. This fixes issues
with certain pages being rendered with unacceptably low resolution text, but
may increase output file sizes in some workflows where low resolution text
is acceptable. - Added a workaround to fix an exception that occurs when trying to
import ocrmypdf.leptonicaon Apple ARM silicon (or potentially, other
platforms that do not permit write+executable memory).
v11.4.5
- Fixed an issue where files may not be closed when the API is used.
- Improved
setup.cfgwith better settings for test coverage.
v11.4.4
- Fixed
AttributeError: 'NoneType' object has no attribute 'userunit'({issue}700),
related to OCRmyPDF not properly forwarded an error message from pdfminer.six. - Adjusted typing of some arguments.
ocrmypdf.ocrnow takes athreading.Lockfor reasons outlined in the
documentation.
v11.4.3
- Removed a redundant debug message.
- Test suite now asserts that most patched functions are called when they should be.
- Test suite now skips a test that fails on two particular versions of piekpdf.
v11.4.2
- Fixed support for Cygwin, hopefully.
- watcher.py: Fixed an issue with the OCR_LOGLEVEL not being interpreted.
v11.4.1
- Fixed an issue where invalid pages ranges passed using the
pagesargument,
such as "1-0" would cause unhandled exceptions. - Accepted a user-contributed to the Synology demo script in misc/synology.py.
- Clarified documentation about change of temporary file location
ocrmypdf.io. - Fixed Python wheel tag which was incorrectly set to py35 even though we long
since dropped support for Python 3.5.
v11.4.0
- When looking for Tesseract and Ghostscript, we now check the Windows Registry to
see if their installers registered the location of their executables. This should
help Windows users who have installed these programs to non-standard
locations. - We now report on the progress of PDF/A conversion, since this operation is
sometimes slow. - Improved command line completions.
- The prefix of the temporary folder OCRmyPDF creates has been changed from
com.github.ocrmypdftoocrmypdf.io. Scripts that chose to depend on this
prefix may need to be adjusted. (This has always been an implementation detail so is
not considered part of the semantic versioning "contract".) - Fixed {issue}
692, where a particular file with malformed fonts would flood an
internal message cue by generating so many debug messages. - Fixed an exception on processing hOCR files with no page record. Tesseract
is not known to generate such files.
v11.3.4
- Fixed an error message 'called readLinearizationData for file that is not
linearized' that may occur when pikepdf 2.1.0 is used. (Upgrading to pikepdf
2.1.1 also fixes the issue.) - File watcher now automatically includes
.PDFin addition to.pdfto
better support case sensitive file systems. - Some documentation and comment improvements.
v11.3.3
- If unpaper outputs non-UTF-8 data, quietly fix this rather than choke on the
conversion. (Possibly addresses {issue}671.)
v11.3.2
- Explicitly require pikepdf 2.0.0 or newer when running on Python 3.9. (There are
concerns about the stability of pybind11 2.5.x with Python 3.9, which is used in
pikepdf 1.x.) - Fixed another issue related to page rotation.
- Fixed an issue where image marked as image masks were not properly considered
as optimization candidates. - On some systems, unpaper seems to be unable to process the PNGs we offer it
as input. We now convert the input to PNM format, which unpaper always accepts.
Fixes {issue}665and {issue}667. - DPI sent to unpaper is now rounded to a more reasonable number of decimal digits.
- Debug and error messages from unpaper were being suppressed.
- Some documentation tweaks.
v11.3.1
- Declare support for new versions: pdfminer.six 20201018 and pikepdf 2.x
- Fixed warning related to
--pdfa-image-compressionthat appears at the wrong
time.
v11.3.0
- The "OCR" step is describing as "Image processing" in the output messages when
OCR is disabled, to better explain the application's behavior. - Debug logs are now only created when run as a command line, and not when OCR
is performed for an API call. It is the calling application's responsibility
to set up logging. - For PDFs with a low number of pages, we gathered information about the input PDF
in a thread rather than process (when there are more pages). When run as a
thread, we did not close the file handle to the working PDF, leaking one file
handle per call ofocrmypdf.ocr. - Fixed an issue where debug messages send by child worker processes did not match
the log settings of parent process, causing messages to be dropped. This affected
macOS and Windows only where the parent process is not forked. - Fixed the hookspec of rasterize_pdf_page to remove default parameters that
were not handled in an expected way by pluggy. - Fixed another issue with automatic page rotation ({issue}
658) due to the issue above.
v11.2.1
- Fixed an issue where optimization of a 1-bit image with a color palette or
associated ICC that was optimized to JBIG2 could have its colors inverted.
v11.2.0
- Fixed an issue with optimizing PNG-type images that had soft masks or image masks.
This is a regression introduced in (or about) v11.1.0. - Improved type checking of the
pluginsparameter for theocrmypdf.ocr
API call.
v11.1.2
- Fixed hOCR renderer writing the text in roughly reverse order. This should not
affect reasonably smart PDF readers that properly locate the position of all
text, but may confuse those that rely on the order of objects in the content
stream. ({issue}642)
v11.1.1
- We now avoid using named temporary files when using pngquant allowing containerized
pngquant installs to be used. - Clarified an error message.
- Highest number of 1's in a release ever!
v11.1.0
- Fixed page rotation issues: {issue}
634,589. - Fixed some cases where optimization created an invalid image such as a
1-bit "RGB" image: {issue}629,620. - Page numbers are now displayed in debug logs when pages are being grafted.
- ocrmypdf.optimize.rewrite_png and ocrmypdf.optimize.rewrite_png_as_g4 were
marked deprecated. Strictly speaking these should have been internal APIs,
but they were never hidden. - As a precaution, pikepdf mmap-based file access has been disabled due to a
rare race condition that causes a crash when certain objects are deallocated.
The problem is likely in pikepdf's dependency pybind11. - Extended the example plugin to demonstrate conversion to mono.
v11.0.2
- Fixed {issue}
612, TypeError exception. Fixed by eliminating unnecessary repair of
input PDF metadata in memory.
v11.0.1
- Blacklist pdfminer.six 20200720, which has a regression fixed in 20200726.
- Approve img2pdf 0.4 as it passes tests.
- Clarify that the GPL-3 portion of pdfa.py was removed with the changes in v11.0.0;
the debian/copyright file did not properly annotate this change.
v11.0.0
- Project license changed to Mozilla Public License 2.0. Some miscellaneous
code is now under MIT license and non-code content/media remains under
CC-BY-SA 4.0. License changed with approval of all people who were found
to have contributed to GPLv3 licensed sections of the project. ({issue}600) - Because the license changed, this is being treated as a major version number
change; however, there are no known breaking changes in functional behavior
or API compared to v10.x.
Releasenotes Version12 (`docs/releasenotes/version12.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v12
v12.7.2
- Fixed "invalid version number" error for Tesseract packaging with nonstandard
version "5.0.0-rc1.20211030". - Fixed use of deprecated
importlib.resources.read_binary. - Replace some uses of string paths with
pathlib.Path. - Fixed a leaked file handle when using
--output-type none. - Removed shims to support versions of pikepdf that are no longer supported.
v12.7.1
- Declare support for pdfminer.six v20211012.
v12.7.0
- Fixed test suite failure when using pikepdf 3.2.0 that was compiled with pybind11
2.8.0. {issue}843 - Improve advice to user about using
--max-image-mpixelsif OCR fails for this
reason. - Minor documentation fixes. (Thanks to @mara004.)
- Don't require importlib-metadata and importlib-resources backports on versions of
Python where the standard library implementation is sufficient.
(Thanks to Marco Genasci.)
v12.6.0
- Implemented
--output-type=noneto skip producing PDFs for applications that
only want sidecar files ({issue}787). - Fixed ambiguities in descriptions of behavior of
--jbig2-lossy. - Various improvements to documentation.
v12.5.0
- Fixed build failure for the combination of PyPy 3.6 and pikepdf 3.0. This
combination can work in a source build but does not work with wheels. - Accepted bot that wanted to upgrade our deprecated requirements.txt.
- Documentation updates.
- Replace pkg_resources and install dependency on setuptools with
importlib-metadata and importlib-resources. - Fixed regression in hocrtransform causing text to be omitted when this
renderer was used. - Fixed some typing errors.
v12.4.0
- When grafting text layers, use pikepdf's
unparse_content_streamif available. - Confirmed support for pluggy 1.0. (Thanks @QuLogic.)
- Fixed some typing issues, improved pre-commit settings, and fixed issues
flagged by linters. - PyPy 7.3.3 (=Python 3.6) is now supported. Note that PyPy does not necessarily
run faster, because the vast majority of OCRmyPDF's execution time is spent
running OCR or generally executing native code. However, PyPy may bring speed
improvements in some areas.
v12.3.3
- watcher.py: fixed interpretation of boolean env vars ({issue}
821). - Adjust CI scripts to test Tesseract 5 betas.
- Document our support for the Tesseract 5 betas.
v12.3.2
- Indicate support for flask 2.x, watcher 2.x ({issue}
815, 816).
v12.3.1
- Fixed issue with selection of text when using the hOCR renderer ({issue}
813). - Fixed build errors with the Docker image by upgrading to a newer Ubuntu.
Also set the timezone of this image to UTC.
v12.3.0
- Fixed a regression introduced in Pillow 8.3.0. Pillow no longer rounds DPI
for image resolutions. We now account for this ({issue}802). - We no longer use some API calls that are deprecated in the latest versions of
pikepdf. - Improved error message when a language is requested that doesn't look like a
typical ISO 639-2 code. - Fixed some tests that attempted to symlink on Windows, breaking tests on a
Windows desktop but not usually on CI. - Documentation fixes (thanks to @mara004)
v12.2.0
- Fixed invalid Tesseract version number on Windows ({issue}
795). - Documentation tweaks. Documentation build now depends on sphinx-issues package.
v12.1.0
- For security reasons we now require Pillow >= 8.2.x. (Older versions will continue
to work if upgrading is not an option.) - The build system was reorganized to rely on
setup.cfginstead ofsetup.py.
All changes should work with previously supported versions of setuptools. - The files in
requirements/*are now considered deprecated but will be retained for v12.
Instead usepip install ocrmypdf[test]instead ofrequirements/test.txt, etc.
These files will be removed in v13.
v12.0.3
- Expand the list of languages supported by the hocr PDF renderer.
Several languages were previously considered not supported, particularly those
non-European languages that use the Latin alphabet. - Fixed a case where the exception stack trace was suppressed in verbose mode.
- Improved documentation around commercial OCR.
v12.0.2
- Fixed exception thrown when using
--remove-backgroundon files containing small
images ({issue}769). - Improve documentation for description of adding language packs to the Docker image
and corrected name of French language pack.
v12.0.1
- Fixed "invalid version number" for untagged tesseract versions ({issue}
770).
v12.0.0
Breaking changes
- Due to recent security issues in pikepdf, Pillow and reportlab, we now require
newer versions of these libraries and some of their dependencies. (If necessary,
package maintainers may override these versions at their discretion; lower
versions will often work.) - We now use the "LeaveColorUnchanged" color conversion strategy when directing
Ghostscript to create a PDF/A. Generally this is faster than performing a
color conversion, which is not always necessary. - OCR text is now packaged in a Form XObject. This makes it easier to isolate
OCR from other document content. However, some poorly implemented PDF text
extraction algorithms may fail to detect the text. - Many API functions have stricter parameter checking or expect keyword arguments
were they previously did not. - Some deprecated functions in
ocrmypdf.optimizewere removed. - The
ocrmypdf.leptonicamodule is now deprecated, due to difficulties with
the current strategy of ABI binding on newer platforms like Apple Silicon.
It will be removed and replaced, either by repackaging Leptonica as an
independent library using or using a different image processing library. - Continuous integration moved to GitHub Actions.
- We no longer depend on
pytest_helpers_namespacefor testing.
New features
- New plugin hook:
get_progressbar_class, for progress reporting,
allowing developers to replace the standard console progress bar with some
other mechanism, such as updating a GUI progress bar. - New plugin hook:
get_executor, for replacing the concurrency model.
This is primarily to support execution on AWS Lambda, which does not support
standard Pythonmultiprocessingdue to its lack of shared memory. - New plugin hook:
get_logging_console, for replacing the standard
way OCRmyPDF outputs its messages. - New plugin hook:
filter_pdf_page, for modifying individual PDF
pages produced by OCRmyPDF. - OCRmyPDF now runs on nonstandard execution environments that do not have
interprocess semaphores, such as AWS Lambda and Android Termux. If the environment
does not have semaphores, OCRmyPDF will automatically select an alternate
process executor that does not use semaphores. - Continuous integration moved to GitHub Actions.
- We now generate an ARM64-compatible Docker image alongside the x64 image.
Thanks to @andkrause for doing most of the work in a pull request several months
ago, which we were finally able to integrate now. Also thanks to @0x326 for
review comments.
Fixes
- Fixed a possible deadlock on attempting to flush
sys.stderrwhen older
versions of Leptonica are in use. - Some worker processes inherited resources from their parents such as log
handlers that may have also lead to deadlocks. These resources are now released. - Improvements to test coverage.
- Removed vestiges of support for Tesseract versions older than 4.0.0-beta1 (
which ships with Ubuntu 18.04). - OCRmyPDF can now parse all of Tesseract version numbers, since several
schemes have been in use. - Fixed an issue with parsing PDFs that contain images drawn at a scale of 0. ({issue}
761) - Removed a frequently repeated message about disabling mmap.
Releasenotes Version13 (`docs/releasenotes/version13.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v13
v13.7.0
- Fixed an exception when attempting to run and Tesseract is not installed.
- Changed to SPDX license tracking and information files.
v13.6.2
- Added a shim to prevent an "error during error handling" for Python 3.7 and 3.8.
- Modernized some type annotations.
- Improved annotations on our _windows module to help IDEs and mypy figure out what
we're doing.
v13.6.1
- Require setuptools-scm 7.0.5 to avoid possible issues with source distributions in
earlier versions of setuptools-scm. - Suppress a spurious warning, improve tests, improve typing and other miscellany.
v13.6.0
- Added a new
initializeplugin hook, making it possible to suppress built-in
plugins more easily, among other possibilities. - Fixed an issue where unpaper would exit with a "wrong stream" error, probably
related to images with an odd integer width. {issue}887, 665
v13.5.0
- Added a new
optimize_pdfplugin hook, making it possible to create plugins that
replace or enhance OCRmyPDF's PDF optimizer. - Removed all max version restrictions. Our new policy is to blacklist known-bad releases
and only block known-bad versions of dependencies. - The naming schema for object that holds all OCR text that OCRmyPDF inserts has
changed. This has always been an implementation detail (and remains so), but possibly,
someone was relying on it and would appreciate the heads-up. - Cleanup.
v13.4.7
- Fixed PermissionError when cleaning up temporary files in rare cases. {issue}
974 - Fixed PermissionError when calling
os.niceon platforms that lack it. {issue}973 - Suppressed some warnings from libxmp during tests.
v13.4.6
- Convert error on corrupt ICC profiles into a warning. Thanks to @oscherler.
v13.4.5
- Remove upper bound on pdfminer.six version.
- Documentation.
v13.4.4
- Updated pdfminer.six version.
- Docker image changed to Ubuntu 22.04 now that it is released and provides the
dependencies we need. This seems more consistent than our recent change to
Debian.
v13.4.3
- Fix error on pytest.skip() with older versions of pytest.
- Documentation updates.
v13.4.2
- Worked around a
major regression in Ghostscript 9.56.0
where all OCR text is stripped out of the PDF. It simply removes all text,
even generated by software other than OCRmyPDF. Fortunately, we can ask
Ghostscript 9.56.0 to use its old behavior that worked correctly for our purposes.
Users must avoid the combination (Ghostscript 9.56.0, ocrmypdf <13.4.2) since
older versions of OCRmyPDF have no way of detecting that this particular
version of Ghostscript removes all OCR text. - Marked pdfminer 20220319 as supported.
- Fixed some deprecation warnings from recent versions of Pillow and pytest.
- Test suite now covers Python 3.10 (Python 3.10 worked fine before, but was not
being tested). - Docker image now uses debian:bookworm-slim as the base image to fix the Docker
image build.
v13.4.1
- Temporarily make threads rather than processes the default executor worker, due
to a persistent deadlock issue when processes are used. Add a new command line
argument--no-use-threadsto disable this.
v13.4.0
- Fixed test failures when using pikepdf 5.0.0.
- Various improvements to the optimizer. In particular, we now recognize PDF images
that are encoded with both deflate (PNG) and DCT (JPEG), and also produce PDF
with images compressed with deflate and DCT, since this often yields file size
improvements compared to plain DCT.
v13.3.0
- Made a harmless but "scary" exception after failing to optimize an image less scary.
- Added a warning if a page image is too large for unpaper to clean. The image is
passed through without cleaning. This is due to a hard-coded limitation in a
C library used by unpaper so it cannot be rectified easily. - We now use better default settings when calling img2pdf.
- We no longer try to optimize images that we failed to save in certain situations.
- We now account for some differences in text output from Tesseract 5 compared to
Tesseract 4. - Better handling of Ghostscript producing empty images when attempting to rasterize
page images.
v13.2.0
- Removed all runtime uses of distutils since it is deprecated in standard library. We
previous useddistutils.versionto examine version numbers of dependencies
at run time, and now usepackaging.versionfor this. This is a new
dependency. - Fixed an error message advising the user that Ghostscript was not installed being
suppressed when this condition actually happens. - Fixed an issue with incorrect page number and totals being displayed in the progress
bar. This was purely a display/presentation issue. {issue}876.
v13.1.1
- Fixed issue with attempting to deskew a blank page on Tesseract 5. {issue}
868.
v13.1.0
- Changed to using Python concurrent.futures-based parallel execution instead of
pools, since futures have now exceed pools in features. - If a child worker is terminated (perhaps by the operating system or the user
killing it in a task manager), the parallel task will fail an error message.
Previously, the main ocrmypdf process would "hang" indefinitely, waiting for the
child to report. - Added new argument
--tesseract-thresholdingto provide control over Tesseract 5's
threshold parameter. - Documentation updates and changes. Better documentation for
--output-type none,
added a few releases ago. Removed some obsolete documentation. - Improved bash completions - thanks to @FPille.
v13.0.0
Breaking changes
- The deprecated module
ocrmypdf.leptonicahas been removed. - We no longer depend on Leptonica (
liblept) or CFFI (libffi,python3-cffi). (Note that Tesseract still requires Leptonica; OCRmyPDF no longer
directly uses this library.) - The argument
--remove-backgroundis temporarily disabled while we search for an
alternative to the Leptonica implementation of this feature. - The
--thresholdargument has been removed, since this also depended on Leptonica.
Tesseract 5.x has implemented improvements to thresholding, so this feature will be
redundant anyway. --deskewwas previous calculated by a Leptonica algorithm. We now use a feature
of Tesseract to find the appropriate the angle to deskew a page. The deskew angle
according to Tesseract may differ from Leptonica's algorithm. At least in theory,
Tesseract's deskew angle is informed by a more complex analysis than Leptonica,
so this should improve results in general. We also use Pillow to perform the
deskewing, which may affect the appearance of the image compared to Leptonica.- Support for Python 3.6 was dropped, since this release is approaching end of life.
- We now require pikepdf 4.0 or newer. This, in turn, means that OCRmyPDF requires
a system compatible with the manylinux2014 specification. This change was "forced"
by Pillow not releasing manylinux2010 wheels anymore. - We no longer provide requirements.txt-style files. Use
pip install ocrmypdf[...]
instead. - Bumped required versions of several libraries.
Fixes
- Fixed an issue where OCRmyPDF failed to find Ghostscript on Windows even when
installed, and would exit with an error. - By removing Leptonica, we fixed all issues related to Leptonica on Apple
Silicon or Leptonica failing to import on Windows.
Releasenotes Version14 (`docs/releasenotes/version14.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v14
v14.4.0
- Digitally signed PDFs are now detected. If the PDF is signed, OCRmyPDF will
refuse to modify it. Previously, only encrypted PDFs were detected, not
those that were signed but not encrypted. {issue}1040 - In addition,
--invalidate-digital-signaturescan be used to override the
above behavior and modify the PDF anyway. {issue}1040 - tqdm progress bars replaced with "rich" progress bars. The rich library is
a new dependency. Certain APIs that used tqdm are now deprecated and will
be removed in the next major release. - Improved integration with GitHub Releases. Thanks to @stumpylog.
v14.3.0
- Renamed master branch to main.
- Improve PDF rasterization accuracy by using the
-dPDFSTOPONERRORoption
to Ghostscript. Use--continue-on-soft-render-errorif you want to render
the PDF anyway. The plugin specification was adjusted to support this feature;
plugin authors may want to adapt PDF rasterizing and rendering
plugins. {issue}1083 - The calculated deskew angle is now recorded in the logged output. {issue}
1101 - Metadata can now be unset by setting a metadata type such as
--titleto an
empty string. {issue}1117,1059 - Fixed random order of languages due to use of a set. This may have caused output
to vary when multiple languages were set for OCR. {issue}1113 - Clarified the optimization ratio reported in the log output.
- Documentation improvements.
v14.2.1
- Fixed {issue}
977, where images inside Form XObjects were always excluded
from image optimization.
v14.2.0
- Added
--tesseract-downsample-aboveto downsample larger images even when
they do not exceed Tesseract's internal limits. This can be used to speed
up OCR, possibly sacrificing accuracy. - Fixed resampling AttributeError on older Pillow. {issue}
1096 - Removed an error about using Ghostscript on PDFs with that have the /UserUnit
feature in use. Previously, Ghostscript would fail to process these PDFs,
but in all supported versions it is now supported, so the error is no longer
needed. - Improved documentation around installing other language packs for Tesseract.
v14.1.0
- Added
--tesseract-non-ocr-timeout. This allows using Tesseract's deskew
and other non-OCR features while disabling OCR using--tesseract-timeout 0. - Added
--tesseract-downsample-large-images. This downsamples larges images
that exceed the maximum image size Tesseract can handle. Large images may still
take a long time to process, but this allows them to be processed if that
is desired. - Fixed {issue}
1082, an issue with snap packaged building. - Change linter to ruff, fix lint errors, update documentation.
v14.0.4
- Fixed {issue}
1066, 1075, an exception when processing certain malformed PDFs.
v14.0.3
- Fixed {issue}
1068, avoid deleting /dev/null when running as root. - Other documentation fixes.
v14.0.2
- Fixed {issue}
1052, an exception on attempting to process certain nonconforming PDFs. - Explicitly documented that Windows 32-bit is no longer supported.
- Fixed source installation instructions.
- Other documentation fixes.
v14.0.1
- Fixed some version checks done with smart version comparison.
- Added missing jbig2dec to Docker image.
v14.0.0
- Dropped support for Python 3.7.
- Dropped support generally speaking, all dependencies older than what Ubuntu 20.04
provides. - Ghostscript 9.50 or newer is now required. Shims to support old versions were
removed. - Tesseract 4.1.1 or newer is now required. Shims to support old versions were
removed. - Docker image now uses Tesseract 5.
- Dropped setup.cfg configuration for pyproject.toml.
- Removed deprecation exception PdfMergeFailedError.
- A few more public domain test files were removed or replaced. We are aiming for
100% compliance with SPDX and generally towards simplifying copyright.
Releasenotes Version15 (`docs/releasenotes/version15.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v15
v15.4.4
- Fixed documentation for installing Ghostscript on Windows. {issue}
1198 - Added warning message about security issue in older versions of Ghostscript.
v15.4.3
- Fixed deprecation warning in pikepdf older than 8.7.1; pikepdf >= 8.7.1 is
now required.
v15.4.2
- We now raise an exception on a certain class of PDFs that likely need an
explicit color conversion strategy selected to display correctly
for PDF/A conversion. - Fixed an error that occurred while trying to write a log message after the
debug log handler was removed.
v15.4.1
- Fixed misc/watcher.py regressions: accept
--ocr-json-settingsas either
filename or JSON string, as previously; and argument count mismatch.
{issue}1183,1185 - We no longer attempt to set /ProcSet in the PDF output, since this is an
obsolete PDF feature. - Documentation improvements.
v15.4.0
- Added new experimental APIs to support offline editing of the final text.
Specifically, one can now generate hOCR files with OCRmyPDF, edit them with
some other tool, and then finalize the PDF. They are experimental and
subject to change, including details of how the working folder is used.
There is no command line interface. - Code reorganization: executors, progress bars, initialization and setup.
- Fixed test coverage in cases where the coverage tool did not properly trace
into threads or subprocesses. This code was still being tested but appeared
as not covered. - In the test suite, reduced use of subprocesses and other techniques that
interfere with coverage measurement. - Improved error check for when we appear to be running inside a snap container
and files are not available. - Plugin specification now properly defines progress bars as a protocol rather
than defining them as "tqdm-like". - We now default to using "forkserver" process creation on POSIX platforms
rather than fork, since this is method is more robust and avoids some
issues when threads are present. - Fixed an instance where the user's request to
--no-use-threadswas ignored. - If a PDF does not have language metadata on its top level object, we add
the OCR language. - Replace some cryptic test error messages with more helpful ones.
- Debug messages for how OCRmyPDF picks the colorspace for a page are now
more descriptive.
v15.3.1
- Fixed an issue with logging settings for misc/watcher.py introduced in the
previous release. {issue}1180 - We now attempt to preserve the input's extended attributes when creating
the output file. - For some reason, the macOS build now needs OpenSSL explicitly installed.
- Updated documentation on Docker performance concerns.
v15.3.0
- Update misc/watcher.py to improve command line interface using Typer, and
support.envspecification of environment variables. Improved error
messages. Thanks to @mflagg2814 for the PR that prompted this improvement. - Improved error message when a file cannot be read because we are running in
a snap container.
v15.2.0
- Added a Docker image based on Alpine Linux. This image is smaller than the
Ubuntu-based image and may be useful in some situations. Currently hosted at
jbarlow83/ocrmypdf-alpine. Currently not available in ARM flavor. - The Ubuntu Docker is now aliased to jbarlow83/ocrmypdf-ubuntu.
- Updated Docker documentation.
v15.1.0
- We now require Pillow 10.0.1, due a serious security vulnerability in all earlier
versions of that dependency. The vulnerability concerns WebP images and could
be triggered in OCRmyPDF when creating a PDF from a malicious WebP image. - Added some keyword arguments to
ocrmypdf.ocrthat were previously accepted
but undocumented. - Documentation updates and typing improvements.
v15.0.2
- Added Python 3.12 to test matrix.
- Updated documentation for notes on Python 3.12, 32-bit support and some new
features in v15.
v15.0.1
- Wheels Python tag changed to py39.
- Marked as a expected fail a test that fails on recent Ghostscript versions.
- Clarified documentation and release notes around the extent of 32-bit support.
- Updated installation documentation to changes in v15.
v15.0.0
- Dropped support for Python 3.8.
- Dropped support some older dependencies, specifically
coloredlogsandtqdmin favor of rich - seepyproject.tomlfor details.
Generally speaking, Ubuntu 22.04 is our new baseline system. - Tightened version requirements for some dependencies.
- Dropped support for 32-bit Linux wheels. We strongly recommend a 64-bit operating
system, and 64-bit versions of Python, Tesseract and Ghostscript to use OCRmyPDF.
Many of our dependencies are dropping 32-bit builds (e.g. Pillow), and we are
following suit. (Maintainers may still build 32-bit versions from source.) - Changed to trusted release for PyPI publishing.
- pikepdf memory mapping is enabled again for improved performance, now that an
issue with feature in pikepdf is fixed. ocrmypdf.helpers.calculate_downsamplepreviously had two variants, one
that took aPIL.Imageand one that took atuple[int, int]. The latter
was removed.- The snap version of ocrmypdf is now based on Ubuntu core22.
- We now account for situations where a small portion of an image on a page is drawn
at high DPI (resolution). Previously, the entire page would be rasterized at the
highest resolution of any feature, which caused performance problems. Now,
the page is rasterized
at a resolution based on the average DPI of the page, weighted by the area that
each feature occupies. Typically, small areas of high resolution in PDFs are
errors or quirks from the repeated use of assets and high resolution is not
beneficial. {issue}1010,1104,1004,1079,1010 - Ghostscript color conversion strategy is now configurable using
--color-conversion-strategy. {issue}1143 - JBIG2 threshold for optimization is now configurable using
--jbig2-threshold. {issue}1133
Releasenotes Version16 (`docs/releasenotes/version16.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v16
v16.13.0
- Added detection and repair for Ghostscript 10.6 JPEG corruption. When GS 10.6
truncates JPEG data by 1-15 bytes, OCRmyPDF now restores the original image
bytes from the input PDF. A warning is issued when GS 10.6+ is detected.
{issue}1603 - We continue to force re-optimization of JPEGs, since this catches some issues with corruption for situations where Ghostscript modifies an image. It is likely there are still cases where we cannot mitigate all corruption issues. {issue}
1585 - Fixed handling of PDF page boxes (ArtBox, BleedBox) which were not being
processed correctly in some cases. {issue}1181,1360 - Documentation: clarified podman usage instructions.
v16.12.0
- Disable Ghostscript's subset fonts feature, which was found to corrupt text in certain
PDFs. Thanks @mnaegler for identifying this issue. {issue}1592 - Users of Ghostscript 10.6.0+ reported that Ghostscript seems to generate corrupted
JPEGs. We force re-optimization of these JPEGs to mitigate the corruption until
Ghostscript fixes the issue. {issue}1585 - OCRmyPDF now avoids applying flate compression to large JPEG images, unless maximum
optimization is requested, since flate+DCT compression reduces performances in PDF
viewers with large images. - Updated Dockerfiles to use more recent base operating systems.
- Updated build and test matrix to include Python 3.14.
- Minor documentation improvements.
- pikepdf >= 10.0.0 is now required.
v16.11.1
- Fixed issue with Tesseract changing an error message related to skew. {issue}
1576 - Dropped macOS 13 from build-test matrix since it is no longer supported by Apple.
v16.11.0
- Deprecated "semfree" plugin in favor of falling back to threads if the platform
does not support semaphores. Fixes an issue with Python 3.14. - Fixed references to PDF/A compliances levels to be consistent with ISO nomenclature.
Thanks @5HT2. {issue}1557 - Fixed an issue around using plugin_manager as an argument. {issue}
1555 - Added OpenBSD install steps to README. {issue}
1554 - Removed PyPy from test matrix due to declining support in third party libraries.
- Documentation improvements.
v16.10.4
- Corrected build errors in Python 3.13.3 and 3.13.4.
v16.10.3 (not released)
- Blocked optimization of images with pre-blended soft masks. {issue}
1536 - Fixed warning from hypothesis on running tests.
- Release incomplete due to new test failures in Python 3.13.3 and 3.13.4.
v16.10.2
- Blacklist pikepdf 9.8.0 due to an incompatible change.
v16.10.1
- No changes affecting OCRmyPDF functionality for command line end users.
- webservice: made page specification easier to find in UI.
- webservice: fix download button downloads wrong file.
- Converted project documentation from rST to Markdown.
- Added README translation to Simplified Chinese. Thanks @HuaPai.
- Modernized license specification in pyproject.toml.
- Modernized SPDX license to REUSE.toml.
v16.10.0
- Added hocr textangle processing, improving handling of text at angles.
Thanks @0dinD {issue}1467 - Docker documentation updates related to podman. Thanks @rugk. {issue}
1489,1488 - Dropped webservice.py's fragile use of ttyd. Instead, messages from ocrmypdf are
printed to the console. - Fixed broken test test_hocrtransform_matches_sandwich, which had become
an invalid test. Thanks @QuLogic for reporting. - Improved install instructions for Windows. Thanks @alex.
v16.9.0
- Added hocr caption processing. Thanks @0dinD {issue}
1466 - ocrmypdf-alpine Docker image is now built with Alpine 3.21.
- Fixed error handling of PDFs that contain invalid images with both ImageMask
and ColorSpace defined. {issue}1453 - Fixed test suite regression when only older Ghostscripts are installed.
- Improved documetnation of _progressbar.py. Thanks @QuentinFuxa. {issue}
1456 - Disabling building of documentation as PDF on ReadTheDocs, as this caused
complex build issues deemed not worth solving.
v16.8.0
- Upgraded webservice.py demonstration using streamlit. It's now possible to
exercise most of OCRmyPDF's functionality in a simple web UI. - Added cache to Dockerfiles to improve build speed.
- Fixed numerous formatting errors in the documentation that prevented some
parts of documentation from generating correctly. - Improved OCR text rendering by suppressing negative-width spaces. Thanks
@pajowu. {issue}1446 - Improved detecting of invisible text when using
--redo-ocr. Thanks
@pajowu. {issue}`1448``
v16.7.0
- Fixed further issues with Docker build and updated some versions.
- Main Docker image returned to Ubuntu 24.04 since the fix in v16.6.2 resolved
that concern. - Code that previously sent Ghostscript output to stdout has been changed to
output to temporary files, since Ghostscript was doing that anyway internally.
This is a modest efficiency improvement. - Fixed an issue with debug log output being parsed as rich markup. {issue}
1444
v16.6.2
- Remove invalid hyperlink annotations to satisfy Ghostscript 10.x during PDF/A
conversion. {issue}1425
v16.6.1
- Fixed some issues with Docker build, such as removing unnecessary content and using
a stable Tesseract version. - Reverted Docker image to Ubuntu 22.04 to access older/more stable Ghostscript
for now. - Clarified batch commands in documentation.
- Fixed an issue with JSON serialization and pickling of HOCRResult. {issue}
1427
v16.6.0
- Fixed an issue where damaged PDFs would fail with
--redo-ocr. {issue}1403 - Fixed an error that prevented JBIG2 optimization on Windows if the image
was optimized in an earlier step. {issue}1396 - Fixed an error detecting the version of unpaper 7.0.0. {issue}
1409 - Fixed a performance regression when scanning pages. {issue}
1378. Thanks @aliemjay. - Fixed Alpine Docker image by enforcing Alpine 3.19. Alpine 3.20 includes a
defective version of Tesseract OCR and so is not usable. - Upgraded Ubuntu Docker image to use Ubuntu 24.04.
- Build and test scripts/actions switched to uv.
- When running in a container, we now remind the user that temporary folders
are inside the container and may not be accessible. - Fixed Linux test coverage matrix, which was missing some key versions.
v16.5.0
- Fixed issue with interpreting PDFs that have images with array masks.
{issue}1377 - Enabled testing on Python 3.13.
- Fixed a test that did not work correctly but still passed. {issue}
1382 - Improved "PDF/A conversion failed" warning message to better describe implications.
- Updated documentation to better explain OCR_JSON_SETTINGS in batch processing.
- Build backend changed from setuptools to hatchling.
v16.4.3
- Work around pdfminer.six issue where a token on the buffer boundary is incorrectly
parsed as two tokens. {issue}1361 - New rules are applied to stencil masks and explicit masks when calculating the
optimal page DPI for rendering. {issue}1362 - Fixed attempts to use an incompatible jbig2.EXE provided by TeX Live. {issue}
1363
v16.4.2
- Fixed order of filenames passed to Ghostscript for PDF/A generation. {issue}
1359 - Suppressed missing jbig2dec warning message. {issue}
1358 - Fixed calculation of image size when soft mask dimensions don't match image
dimension. {issue}1351 - Several fixes to documentation. Thanks to users Iris and JoKalliauer
who contributed these changes. - Fixed error on processing PDFs that are missing certain image metadata. {issue}
1315
v16.4.1
- Fixed calculation of image printed area (used in finding weighted DPI for OCR).
{issue}1334 - Fixed "NotImplementedError: not sure how to get colorspace" error
messages in logs which simply records a failure to optimize images with
print production colorspaces. {issue}1315
v16.4.0
- Selecting the
osdandequpseudo-languages with-l/--languagenow
exits with an error when using Tesseract OCR, because these are not
regular Tesseract languages but implementation details implemented.
Using them can cause Tesseract to crash. - The hOCR renderer is more tolerant of extra whitespace in input files.
- watcher.py now changes the output file extension to .pdf when the input is not
.pdf. - Improved handling of PDFs that contain circularly referenced Form XObjects.
{issue}1321 - Fixed Alpine Docker image for ARM64, which was not building correctly.
- Docker images now use pikepdf 9.0.0.
- Prevent use of Tesseract OCR 5.4.0, a version with known regressions.
- Disabled progressbar for "Linearizing" when
--no-progress-barset. - Fixed some tests that warn about missing JBIG2 decoding via pikepdf, by
installing the necessary libraries during tests.
v16.3.1
- Fixed a test suite failure with Ghostscript 10.03.0+. {issue}
1316 - Fixed an issue with the presentation of the "OCR" progress bar. {issue}
1313
v16.3.0
- Fixed progress bar not displaying for Ghostscript PDF/A conversion. {issue}
1313 - Added progress bar for linearization. {issue}
1313 - If
--rotate-pages-thresholdissued without--rotate-pageswe now exit with
an error since the user likely intended to use--rotate-pages. {issue}1309 - If Tesseract hOCR gives an invalid line box, print an error message instead of
exiting with an error. {issue}1312
v16.2.0
- Fixed issue 'NoneType' object has no attribute 'get' when optimizing certain PDFs.
{issue}1293,1271 - Switched formatting from black to ruff.
- Added support for sending sidecar output to io.BytesIO.
- Added support for converting HEIF/HEIC images (the native image of iPhones and
some other devices) to PDFs, when the appropriate pi-hief library is installed.
This library is marked as a dependency, but maintainers may opt out if needed. - We now default to downsampling large images that would exceed Tesseract's internal
limits, but only if it cause processing to fail. Previously, this behavior only
occurred if specifically requested on command line. It can still be configured
and disabled. See the --tesseract command line options. - Added Macports install instructions. Thanks @akierig.
- Improved logging output when an unexpected error occurs while trying to obtain
the version of a third party program.
v16.1.2
- Fixed test suite failure when using Ghostscript 10.3.
- Other minor corrections.
v16.1.1
- Fixed PyPy 3.10 support.
v16.1.0
- Improved hOCR renderer is now default for left to right languages.
- Improved handling of rotated pages. Previously, OCR text might be missing for
pages that were rotated with a /Rotate tag on the page entry. - Improved handling of cropped pages. Previously, in some cases a page with a
crop box would not have its OCR applied correctly and misalignment between
OCR text and visible text coudl occur. - Documentation improvements, especially installation instructions for less
common platforms.
v16.0.4
- Fixed some issues for left-to-right text with the new hOCR renderer. It is still
not default yet but will be made so soon. Right-to-left text is still in progress. - Added an error to prevent use of several versions of Ghostscript that seem
corrupt existing text in input PDFs. Newly generated OCR is not affected.
For best results, use Ghostscript 10.02.1 or newer, which contains the fix
for the issue.
v16.0.3
- Changed minimum required Ghostscript to 9.54, to support users of RHEL 9 and its
derivatives, since that is the latest version available there. - Removed warning message about CVE-2023-43115, on the assumption that most
distributions have backported the patch by now.
v16.0.2
- Temporarily changed PDF text renderer back to sandwich by default to address
regressions in macOS Preview.
v16.0.1
- Fixed text rendering issue with new hOCR text renderer - extraneous byte order
marks. - Tightened dependencies.
v16.0.0
- Added OCR text renderer, combined the best ideas of Tesseract's PDF
generator and the older hOCR transformer renderer. The result is a hopefully
permanent fix for wordssmushedtogetherwithoutspaces issues in extracted text,
better registration/position of text on skewed baselines {issue}1009,
fixes to character output when the German Fraktur script is used {issue}1191,
proper rendering of right to left languages (Arabic, Hebrew, Persian) {issue}1157.
Asian languages may still have excessive word breaks compared to expectations.
The new renderer is the default; the old sandwich renderer is still available
using--pdf-renderer sandwich; the old hOCR renderer is no more. - The
ocrmypdf.hocrtransformAPI has changed substantially. - Support for Python 3.9 has been dropped. Python 3.10+ is now required.
- pikepdf >= 8.8.0 is now required.
Releasenotes Version17 (`docs/releasenotes/version17.md`)
% SPDX-FileCopyrightText: 2022 James R. Barlow
% SPDX-License-Identifier: CC-BY-SA-4.0
v17
v17.13.1
Changes
- OCRmyPDF now requires
pikepdf[pdfa]10.17 or later, and uses its newcontent_scopes()to find the fonts and images a PDF uses.
Fixes
- Fixed PDF/A conversion appearing to hang on a file whose form XObjects
share one/Resourcesdictionary that lists all of them, a common layout.
The search for fonts that are not embedded followed every path through the
forms, which takes exponential time. The repair for JPEGs damaged by
Ghostscript 10.6 had the same defect. {issue}1763 - Fixed a
RecursionErrorwhen examining a page with a form XObject that
draws itself. A page that draws form XObjects an exponential number of times
no longer hangs. - Fonts and images in form XObjects nested more than ten levels deep are no
longer missed, and fonts used only by tiling patterns, Type 3 glyphs or soft
masks are now checked for embedding.
v17.13.0
Changes
- PDF/A made without Ghostscript ("speculative" conversion) is now validated
with pikepdf's PDF/A support (pikepdf.pdfa) instead of veraPDF, and the
final output is validated again after the metadata and optimization steps.
veraPDF is no longer used, so the Ghostscript-free path is available
whether or not veraPDF is installed, including in the Docker image. Files
the validator does not approve go to Ghostscript as before; run with-v1
to see why. - Speculative conversion now supports PDF/A-1b, and repairs common problems
that previously always sent a file to Ghostscript: it replaces other output
intents with sRGB, removes image interpolation flags, sets the Print flag on
annotations, adds the/CIDSetPDF/A-1 requires, and removes XMP properties
and hidden annotations that PDF/A does not permit. - New option
--pdfa-backend {auto,ghostscript,internal}(API:pdfa_backend=).auto, the default, tries OCRmyPDF's own conversion and
then Ghostscript.internalnever uses Ghostscript, so JPEGs pass through
unchanged; if the result is not approved, thepdfaoutput types fail with
exit code 10 and--output-type autooutputs a regular PDF.ghostscript
always uses Ghostscript. Options that only Ghostscript implements
(--pdfa-image-compression,--ghostscript-jpeg-quality,--ghostscript-jpeg-maxdpi, and--color-conversion-strategyCMYK,GrayorUseDeviceIndependentColor) now select Ghostscript underauto
and are an error withinternal. - OCRmyPDF now requires
pikepdf[pdfa]10.15 or later. Thepdfaextra
brings injsonschema,referencingandfonttools, all packaged by
Debian and Red Hat. --force-ocrnow keeps hyperlinks, moving link annotations onto the
rasterized page; so do--deskewand--clean-final. The new mode--mode force-ocr-no-linkskeeps the old behaviour. {issue}605- When Ghostscript substitutes fonts that are not embedded, it now always uses
its own fonts (-dNONATIVEFONTMAP), so output is the same on every
platform; on macOS, native font lookup could embed tens of megabytes of
system fonts. OCRmyPDF warns when fonts other than the standard 14 are
substituted, or when a substitute loses bold or italic styling.
{issue}1369 - A creation date without a time zone is now taken to be local time, and the
output's dates get that zone, with a warning. SetTZto choose another
zone. - In the default mode, a page that already has text now stops OCRmyPDF
straight after the initial scan, instead of after OCRing the pages before
it. The error names the page. {issue}613 - The check of the output file is much faster: each stream is checked in the
cheapest way for its compression instead of being fully decoded, and a
progress bar is shown. JPEG 2000 and CCITT images are now checked too, but
JPEGs that are corrupt without being truncated are no longer detected.
{issue}1570 - OCRmyPDF now ships a PyInstaller hook, so apps that use it can be bundled
without extra options. Tesseract and Ghostscript must still be installed.
{issue}1024 - New
PageInfo.has_visible_text, which excludes invisible text such as an
OCR layer.has_textis unchanged.
Fixes
--force-ocron a scan that already had an invisible OCR layer rasterized
it at no less than 400 dpi, as if the text were visible, which could
multiply the file size. {issue}961- Process workers (
use_threads=Falseon Windows and macOS) failed on every
page with'OcrOptions' object has no attribute 'tesseract'.
{issue}1757 ocrmypdf.ocr()rejected thresholding method names such astesseract_thresholding='adaptive-otsu'. {issue}1460--rotate-pages --tesseract-timeout 0detected page orientation but did not
rotate the pages. The cookbook now recommends this combination for rotating
or deskewing without OCR. {issue}778- With
--output-type autoand--force-ocr, output was labelled PDF/A
without being validated or given the PDF/A declarations when veraPDF was
not installed, as in the Docker image. {issue}1751 - Ghostscript's PDF/A conversion deleted most hyperlinks (those without the
Print flag). Hidden annotations, which PDF/A does not permit, are now
removed before Ghostscript runs, with the same warning as speculative
conversion, instead of silently. {issue}605 - Text in fonts without
/ToUnicode, which viewers extract through glyph
names, was garbled or lost when Ghostscript made the PDF/A. {issue}1297 - XMP metadata with no document-info equivalent, such as
dc:contributor
anddc:subject, was lost when Ghostscript made the PDF/A. It is now
copied with its whole value, including every language of a multilingual
property. {issue}1220 - Images in PDFs from iText and pdftk were recompressed without their Flate
predictor, growing the output by about 30%. {issue}1620 - Copied text from vertical Japanese and other vertical or rotated OCR lines
had spurious spaces and characters out of order. {issue}1244 - Text set in the glyphless fallback font, used when no installed font
covers a script (e.g. CJK), lost characters when selected or copied in
Chrome and other pdfium-based viewers. - A page made of full-page images at different resolutions was rasterized at
a weighted average resolution rather than the highest. {issue}948 - The document language (
/Lang) was not set for Tesseract's language codes
such asdeu,fraandces. Languages without a two-letter code now get
their three-letter code. {issue}1749 - Some LaTeX PDFs with malformed numbers in a content stream, such as
0.000-50131235, stopped OCRmyPDF. It now warns and continues, as viewers
do. {issue}1054 - A
/Rotatestored as a real number, or acmoperator with a non-numeric
operand, crashed the page scan. --force-ocr --ocr-engine nonecrashed on pages with no images.- When OCRmyPDF's own PDF/A failed its final check, the Ghostscript fallback
could fail withFileExistsError; on Windows it always did. - With the default
--output-type auto, the explanation for a file that grew
now points to PDF/A conversion. {issue}1369 - Ghostscript 10.08's title
'Untitled'(with quotes) for untitled files is
removed again. - The Docker image's web service could not find its Streamlit script. The
Docker documentation for the web service is updated. {issue}1753 - The Docker watcher no longer turns off
deskewgiven inOCR_JSON_SETTINGSwhenOCR_DESKEWis not set.
v17.12.1
Fixes
ocrmypdf --versionandocrmypdf --helpprinted to standard error
instead of standard output, soversion=$(ocrmypdf --version)came back
empty andocrmypdf --help | lessshowed nothing. OCRmyPDF redirects file
descriptor 1 to standard error early in startup, before it parses the
command line, so that a stray write from a plugin or a library cannot
corrupt a PDF being written to standard output; argparse's own output was
caught by that redirection too and is now routed back to the real standard
output. Error messages continue to go to standard error. Broken since
v17.8.0.
v17.12.0
Changes
- OCRmyPDF now requires pikepdf 10.2 or later, up from pikepdf 10. This is the
first release that providespikepdf.NamePath, theObject.as_int()family
of type-safe accessors, and a thread-localexplicit_conversion(). OCRmyPDF
uses all three to read optional values out of PDFs it did not write. - HEIF/HEIC input support is now an opt-in extra,
ocrmypdf[heic], instead of
a default dependency. Thepi-heifpackage we previously depended on is
discontinued; its successorpillow-heifbundles libheif, libde265 and x265
in its wheels, making the wheel as a whole GPLv2-licensed, so we do not
install it by default. Installocrmypdf[heic]if you feed HEIC images to
OCRmyPDF. The Docker image and snap do not include it. {issue}1746 - A Tagged PDF that is refused because it appears to contain real text now
exits with code 6 (already_done_ocr) instead of code 2 (input_file),
matching the exit code for a PDF with prior OCR. Scripts that check for exit
code 2 on tagged PDFs need updating. Thanks @BetterAndBetterII.
{issue}1551 - The snap package is now built and published from our CI rather than
Launchpad, so thestablechannel tracks releases again instead of being
frozen at v16.4.2. It is based on core26 and includes a JBIG2 encoder.
Fixes
- Text copied or searched in a
--output-type pdfafile came out as garbage
(a stable substitution of unrelated characters, or nothing at all) when the
PDF/A conversion ran through Ghostscript 9.56 through 10.04.x, which
includes the Ghostscript in Ubuntu 24.04. fpdf2 writes each font's ToUnicode
CMap as a singlebfcharblock, but the CMap specification allows at most
100 entries per block, and those Ghostscript releases discard the whole map
when a block is larger. Any page with more than 100 distinct glyphs was
affected, which is nearly every page of prose. CFF CID-keyed fonts such as
Noto Sans CJK have the same problem in their Encoding CMap, where those
Ghostscript releases then also draw the wrong glyphs. OCRmyPDF now splits
every CMap block to 100 entries as fpdf2 writes it, for every fpdf2 release.
fpdf2 2.8.9 will split the ToUnicode blocks itself but not the Encoding
blocks, so the fix stays in place. Thanks @olafure for the analysis
({issue}1747, fpdf2 issue py-pdf/fpdf2#1952). - Words containing
fi,ff,fland similar pairs extracted with the
letters missing (con dentiality,e ects) from--output-type pdfafiles
when the PDF/A conversion ran through Ghostscript 10.05.0 through 10.06.x.
Those Ghostscript releases drop ToUnicode entries that expand to more than
one character (Ghostscript bug 709030, fixed in 10.07.0), and the fpdf2
renderer's HarfBuzz shaping had been forming optional Latin ligatures whose
entries do exactly that. Invisible text in scripts that do not need shaping
is now encoded one glyph per character, so no such entries exist to lose.
Complex scripts still get shaped, and OCRmyPDF now warns when an affected
Ghostscript is in use, since their conjunct mappings can still be dropped.
Thanks @kmn5 ({issue}1744). - A malformed PDF that stores something other than a dictionary at a
structural key --/Resources,/Resources /XObject,/Root /AcroForm,/Root /MarkInfo,/Root /Names,/Root /PieceInfo, an annotation's/A,
or an image's/SMask-- is now tolerated everywhere rather than in the
handful of places that had been hardened individually. Such a file is read as
though the key were absent, which is what the well-guarded paths already did. - An image XObject with no
/Subtypeno longer raises out of the optimizer. PdfInfono longer aborts on a/MarkInfo << /Marked 1 >>, where a producer
wrote a flag as an integer instead of a Boolean. Thanks @linhongyu510
({issue}1742).- A page
/UserUnitwritten as a PDF Real is now read exactly rather than via
binary floating point, so the digits the file wrote are the digits used. --sidecarnow refuses any spelling of the input or output file, not just a
byte-identical one.ocrmypdf in.pdf out.pdf --sidecar ./in.pdfwas accepted
and then silently overwrote the input PDF with the OCR text. Paths are now
compared after resolving.,.., symlinks and filesystem case. Thanks
@linhongyu510.--deskewand--rotate-pagesnow warn that they will have no effect when
combined with--ocr-engine none, since skew and orientation are measured by
the OCR engine. Other image processing options are unaffected. Thanks
@Anai-Guo. {issue}1735- Installation documentation refreshed for current platforms: Ubuntu 26.04 and
Fedora 43/44 added, Ubuntu 20.04 dropped, anduv tool installshown in
place of pip. - Documentation: corrected the list of platforms that ship a JBIG2 encoder,
which now includes Debian and Ubuntu. Thanks @lcorbasson. - Release process and CI improvements.
v17.11.0
Enhancements
- New
--max-ocr-image-mpixelsdownsamples the image sent to OCR when a page
exceeds the given size, which bounds the largest consumer of memory. The
visible page is never downsampled, so output appearance is unaffected in every
mode; what it trades is OCR accuracy on very high resolution scans. A 34
megapixel page that peaks at 492 MB peaks at 325 MB under--max-ocr-image-mpixels 8, and recognizes the same text. See the "Memory"
section of the performance documentation for how to size a memory limit. watcher.py(thewatcherextra) gained a configurable output layout and
conflict policy. These are watcher-only changes; they do not affect theocrmypdflibrary API.- New
OCR_OUTPUT_STRUCTUREsetting (--output-structure):FLAT
(default, all outputs directly in the destination directory),YEAR_MONTH({destination}/{year}/{month}/{filename}, same layout as
the oldOCR_OUTPUT_DIRECTORY_YEAR_MONTH=1), orHIERARCHY, which
mirrors the input directory tree under the destination, e.g.input/a/b/c.pdf→output/a/b/c.pdf. - New
OCR_ON_CONFLICTsetting (--on-conflict) controls what happens
when the intended destination file already exists:SUFFIX(default)
writesname (1).pdf,name (2).pdf, ... in the OS style;SKIPlogs
and leaves the file unprocessed;OVERWRITEis the old behavior.
Behavior change: the default is nowSUFFIX, so existing output
files are no longer silently overwritten. - Both settings now apply equally to
OCR_OUTPUT_DIRECTORYand to the
archive directory used byOCR_ON_SUCCESS_ARCHIVE; previously the
archive directory was always flat and silently overwrote on a name
collision. - Output and archive filenames and directory components are sanitized
for filesystems more restrictive than the input side (e.g. an SMB
share): characters illegal on Windows/SMB (<>:"/\|?*and control
characters) are replaced with_, trailing dots/spaces are stripped,
and reserved DOS device names (CON,PRN,AUX,NUL,COM1-9,LPT1-9) are prefixed with_. OCR_OUTPUT_DIRECTORY_YEAR_MONTHis now deprecated in favor ofOCR_OUTPUT_STRUCTURE=YEAR_MONTH. It is still honored and logs a
deprecation warning; if both are set,OCR_OUTPUT_STRUCTUREwins.- See the "Watched folders with watcher.py" section of the batch
processing documentation for the full description, including a note
for SMB users about client-side directory/file-info caching delays.
- New
Performance
- The image sent to OCR is no longer decoded and re-encoded when nothing needs
to change it. On a page with no pre-existing text to mask and no filtering
plugin -- the ordinary case for a scanned document -- the rasterized page
already is the OCR image, so it is linked rather than rewritten. Producing
it was costing 3.3 seconds and a full-size buffer on a 34 megapixel page, out
of about 15 seconds for the whole file. Building the mask is now deferred
until a text area actually needs blanking, since that step is what forced the
decode. The minimum Pillow version is raised to 12, because deciding whether
anything decoded the image reads an attribute whose shape settled in Pillow 11
(pi-heifalready required Pillow 11.1, so the effective floor barely moves). - Reduced peak memory on files with large images by about a third at default
settings. On a 34 megapixel page the process tree peaked at 752 MB and now
peaks at 492 MB. Two changes account for it: rasterizing a page no longer
allocates a third full-page buffer to correct PDFium's rounding of the
rendered size by a pixel or two, and freed heap memory is now returned to the
operating system before the OCR engine runs, instead of counting against our
resident set for as long as the engine is working. Rasterization is also
faster, since correcting the size no longer resamples the whole page. - Several OCR jobs may now run concurrently in a single Python process. The API
previously held a lock for the whole duration ofocrmypdf.ocr(), so a second
call in another thread had to wait for the first to finish. Plugin state is
now guarded by a readers-writer lock: installing a plugin set takes it
exclusively, and a job holds it shared for its run. An in-flight job therefore
cannot have the plugin infrastructure it depends on replaced underneath it,
while jobs that are past installation proceed concurrently. - A plugin set is now installed once per interpreter and reused, rather than
being reinstalled on every call. Previously eachocrmypdf.ocr()call
re-executed plugin modules given as file paths and rebound them insys.modules. Plugins must not rely on being re-executed for each job, and
must not store per-job state on the plugin manager, which concurrent jobs
requesting the same plugin set now share. - Jobs requesting different plugin sets serialize against each other, since
installing the second set must wait for the first set's jobs to finish. Use
the same plugin set across concurrent jobs, or separate processes. - Image optimization is substantially faster on documents with large images.
An image stored as/FlateDecodewith a PNG predictor already holds exactly
what a PNGIDATchunk holds, so it is now repackaged as a PNG directly
instead of being decoded to a bitmap and re-encoded. On a 6-page document
containing one 9000x9000 image, the optimization step went from 3.4s to 0.6s,
and to 0.04s together with the JPEG change below; total runtime went from
10.0s to 7.0s. Output is unchanged: the compressed
data is reused verbatim. Images that are not in a directly repackageable form
still take the previous path. - During image optimization, we decoded all JPEGs, even if the code
path was an optimization setting with the decoded JPEG would be never be
re-encoded (below--optimize 2). We now decode only on code paths that
use the decoded JPEG. Output is unchanged. - An uncompressed image (one with no
/Filter) no longer produces a spurious
"could not be processed by the optimizer" warning. Such an image raisedIndexErrorinternally, which the optimizer's best-effort handler caught and
reported as a warning; it is now recognized and skipped quietly. - Removed an unreachable branch in the image optimizer that claimed to handle
1 bit per component images in an ICC-based colorspace. An earlier check sends
every 1 bpc image to the JBIG2 pass, which handles ICC-based images by
neutralizing the profile before extracting, so the branch could never run. - Removed the process-wide lock that serialized worker pools across all
Executorinstances.Executor.pool_lockis retained but no longer acquired,
and is deprecated; it will be removed in a future major release. The invariant
it protected - that only one progress bar renders on the shared console - is
now enforced by the progress bar, which disables itself if another bar already
owns the console. - Note that N concurrent jobs each configured with
jobs=Mmay now spawn up to
N*M workers, where previously they were serialized to M. Sizejobs
accordingly. - Known limitation of concurrent in-process jobs: they must use the same
max_image_mpixels. Pillow's decompression-bomb limit is interpreter-global
and the last job to set it wins. Use separate processes to run jobs with
differing configurations.
Fixes
- Fixed a latent use-after-free in the pypdfium2 rasterizer.
to_pil()lets
Pillow alias PDFium's bitmap buffer for some formats -- grayscale renders
among them, which is every mono and grayscale page -- and the buffer was freed
immediately afterwards, leaving Pillow reading memory PDFium had released. - Windows: OCRmyPDF no longer prints
[WinError 2] The system cannot find the file specifiedwarnings while it searches for Ghostscript and Tesseract
(#1671). These
messages came from probing registry keys that simply don't exist when the
programs were installed by a package manager such as Scoop, or not installed
at all. Since the search then continues elsewhere and usually succeeds, these
failures are normal, and are now logged at debug level, naming the location
that was searched. If a program genuinely cannot be found, OCRmyPDF still
reports that as an error. - Windows: fixed a crash when the
PROGRAMFILESenvironment variable pointed to
a folder that does not exist. --mode stripfailed to remove OCR text layers that OCRmyPDF itself
produced ({issue}1730). OCRmyPDF grafts its text layer as a Form XObject
and stripping only examined the page content stream, so the invisible text
was never found. The same flaw made--mode redostack a second text layer
on top of the old one instead of replacing it. Stripping now descends into
Form XObjects. Thanks @Anai-Guo ({issue}1732).- Stripping now also resolves page
/Resourcesinherited from an ancestor/Pagesnode, rather than only looking at the page's own resources. - A text layer that becomes empty after stripping is now removed from the page
instead of being left behind as a vestigial Form XObject husk. - Setting
clean_finalon an existing options object in the Python API no
longer leavescleanunset.--clean-finalimplies--clean, but the rule
lived in a field validator that only ran while the options object was being
constructed, so assigning to the attribute afterwards silently skipped it. - Validation errors for out-of-range or misspelled options now name the option
they are about, and their wording comes from Pydantic rather than being
hand-written, so it has changed slightly. For example,--jobs 999now
reports--jobs: Input should be less than or equal to 256. The set of
accepted values is unchanged.
v17.10.0
The
watcher.pywatched-folder helper (thewatcherextra) has been
modernized and security-hardened:- It now uses
watchfilesinstead ofwatchdog. Installingocrmypdf[watcher]now pulls inwatchfiles; native OS filesystem
notifications are used by default, withOCR_USE_POLLING=1to force
polling. - It enforces a "Harvard architecture" separation between data and code:
at startup it refuses to run (exit code 9) if the input, output or
archive directory overlaps any Python interpreter path (sys.path,
the virtual environment, site-packages, or$PATH), ifOCR_JSON_SETTINGSpoints at a file inside a data directory or one that
is group/world-writable, or if it specifies a plugin located inside a
data directory. It also refuses to run when the output or archive
directory is the input directory or a subdirectory of it, which would
otherwise cause OCRmyPDF output to be reprocessed in an endless loop. - At runtime it no longer follows symlinks or processes non-regular files
(fifos, devices, etc.) in the watched directory, and refuses to write
output onto a destination occupied by a non-regular file. - A password-protected PDF dropped into the watched folder no longer stops
the watcher ({issue}1715).pikepdf.PasswordErrordoes not derive frompikepdf.PdfError, so it escaped the handler that waits for a file to be
fully written and tore down the watch loop, leaving files that arrived
afterwards unprocessed. Encrypted files are now logged and skipped
immediately — no amount of retrying will supply the password. More
generally, no per-file error can stop the watcher now: failures are
logged and watching continues. Thanks @christophdb for the report and a
fix ({issue}1716).
See the "Watcher security model" section of the batch processing
documentation for details. Existing
deployments where the data directories are kept separate from the application
are unaffected; deployments that co-located data with the interpreter or its
environment will need to relocate one or the other.- It now uses
Ghostscript 10.7.0 and later are no longer treated as affected by the JPEG
passthrough truncation bug, which Ghostscript fixed in 10.07.0
({issue}1726). The version check had no upper bound, so users on a fixed
Ghostscript still saw the "JPEG encoding errors" warning and, worse, silently
had every JPEG lossily re-encoded at--optimize 1(the default) to work
around a bug their Ghostscript did not have. Thanks @zuentec-droid for the
detailed measurements and upstream analysis.The same JPEG re-encoding workaround no longer applies when Ghostscript did
not produce the file at all. It was previously triggered by the mere presence
of an affected Ghostscript, so--output-type pdfand files converted by the
speculative PDF/A path — neither of which runs Ghostscript — paid the quality
loss for nothing.
v17.9.0
- OCRmyPDF now uses any Noto font installed on the system, not just the two
dozen script families it knows by name ({issue}1722). Previously a document
in, say, Cherokee or Vai was rendered with the glyphless fallback font even
though the matching font was installed — a common situation on macOS, which
ships around a hundred script-specific Noto faces. When the named fonts
cannot cover a word, OCRmyPDF now searches the installed fonts for one that
can. - The "no installed font has glyphs" warning now names the characters it could
not render, with their codepoints and Unicode names, so it is clear which
font to install. Text that mixes scripts no single font covers is now
reported as such, instead of advising the user to install fonts they may
already have. - Fixed the macOS font installation instructions, which recommended a Homebrew
package (font-noto) that does not exist ({issue}1722). Homebrew has no
single Noto package; each family is a separate cask. The Fedora package name
was also corrected togoogle-noto-fonts-all. - Font providers may now implement the optional
GlyphSearchingFontProvider
protocol to participate in coverage-based font search. - Fixed
--jpeg-quality/--jpg-qualityhaving no effect on the CLI: the
value was silently dropped before reaching the optimizer, which then
always used its own built-in default JPEG quality regardless of what was
requested ({issue}1723). The same bug affected the Python API'sjpg_qualityparameter.ocrmypdf.ocr()now acceptsjpeg_quality
(matching the CLI flag name) as the canonical parameter;jpg_quality
still works but is deprecated. - Hardened PDF parsing against malformed (non-dictionary)
/Resources,/XObject, and/FontDescriptorentries, which previously crashedocrmypdf.ocr()withAttributeError/TypeError/ValueErroron
otherwise-processable files, both during PDF/A font scanning and general
image scanning ({issue}1713). Thanks @mvanhorn for the initial fix. - Release process improvements: fixed a CI bug where every push to main
after a release was tagged would incorrectly revert the just-published
GitHub release back to draft status.
v17.8.1
- Improved the
--tesseract-pagesegmodehelp text to point totesseract --help-extra, since Tesseract 5.5.2 moved the page segmentation
mode documentation there fromtesseract --help. Thanks @sokai. - Internal refactoring: completed a project-wide mypy type-checking pass
(--check-untyped-defsis now enabled, and the mypy pre-commit hook is now
blocking rather than advisory), fixing several latent edge-case bugs
surfaced along the way. - Release process improvements: migrated from pre-commit to prek for local
git hooks, and added a dedicated lint job to CI. - Improved typing strictness for
Path.
v17.8.0
--output-type auto(the default) again produces PDF/A whenever it can,
matching OCRmyPDF 16's "PDF/A by default" behavior. It first tries the fast
Ghostscript-free conversion (validated by veraPDF when available) and now
falls back to Ghostscript when that cannot produce PDF/A, only emitting a
regular PDF when even Ghostscript cannot safely convert (for example, an
input with non-embedded CID/CJK fonts, per {issue}1561). A consequence is
that the default path may once again invoke Ghostscript, which is slower and
may transcode images; use--output-type pdfto skip PDF/A conversion
entirely.- Fixed detection of veraPDF 1.30.0 and newer: recent builds print JVM
warnings before their version string, which caused OCRmyPDF to report
veraPDF as unavailable and skip the fast PDF/A path. - OCRmyPDF no longer silently corrupts a non-embedded CID (CJK) text layer when
producing PDF/A ({issue}1561). PDF/A requires all fonts to be embedded, so
Ghostscript substitutes and re-embeds non-embedded CID fonts — such as the OCR
text layer Adobe Acrobat adds to scanned CJK documents — which mangles the
text and destroys searchability. OCRmyPDF now detects non-embedded CID fonts
before conversion: with--output-type auto(the default) it produces a
regular PDF and preserves the existing text layer, and with an explicit--output-type pdfa*it stops with an error rather than emit corrupted
output. Use--output-type pdfto keep the text layer, or--force-ocrto
rebuild it with embedded fonts. - Writing the output PDF to standard output (
ocrmypdf input.pdf -) is now
protected against corruption at the operating system level. Previously
OCRmyPDF relied on no in-process code — third-party libraries, plugins, or
strayprint()calls — ever writing to stdout; a single accidental write
would silently corrupt the PDF. The command line program now saves the real
stdout at startup, before plugins are loaded or any worker process/thread is
started, and redirects file descriptor 1 to stderr, so that only OCRmyPDF's
final PDF output can reach stdout. A consequence is that a plugin which
intentionally prints to stdout will have that output redirected to stderr. - Added the public API function {func}
ocrmypdf.configure_stdout_protection,
which installs this same protection. Like {func}ocrmypdf.configure_logging,
it is optional and intended for callers that want command-line-like behavior;
applications that manage their own standard output should not call it. - Fixed an uncaught
UnicodeDecodeErrorwhen processing a PDF whose/DocumentInfodictionary contains a/Namekey encoded in Latin-1 (or
another non-UTF-8 encoding), such as/Saks#e5r.repair_docinfo_nulsnow
treats such a block as malformed, logs a message, and continues instead of
crashing the pipeline ({issue}1540). Current pikepdf releases tolerate these
keys by surrogate-escaping them, but older versions raised while iterating the
dictionary.
v17.7.1
- Fixed a severe, Windows-specific performance regression in the "Scanning
contents" phase, most visible with--redo-ocr({issue}1662). Since
v16.4.3, OCRmyPDF forced pdfminer's read buffer to 256 MiB to work around a
pdfminer bug that mishandled tokens split across the buffer boundary
({issue}1361). On Windows, CPython'sBufferedReader.read()eagerly
allocates a buffer of the requested size on every read, so the oversized
buffer made each of pdfminer's thousands of reads cost tens of milliseconds
(this allocation is lazy, and effectively free, on Linux). The underlying
pdfminer bug was fixed upstream in pdfminer.six 20250327
(#1030), with a
follow-up for tokens split across streams in 20260107
(#1158), so the
workaround has been removed and the minimum pdfminer.six version raised to - The font discovery used to build the OCR text layer now finds variable fonts
such asNotoSansArabic[wdth,wght].ttf, the form shipped by Homebrew casks
and current Google Fonts releases. Previously only static-Regular.ttf/.otf
files were matched, so users who had installed the correct Noto font still got
the glyphless fallback and a "No font found" warning ({issue}1652). - Font discovery is now language-aware for CJK: each Chinese, Japanese, and
Korean language maps to its own per-language Noto family (NotoSansSC, TC, HK,
JP, KR), with the pan-CJK super font kept as a shared fallback, since the
per-language fonts are region subsets that may lack glyphs from other scripts. - The warning shown when no installed font has glyphs for some text was reworded
to explain the consequence — the text is still added as a searchable, copyable
layer but appears blank when highlighted in a viewer — and to name the specific
font family to install.
v17.7.0
- The Docker images now run as a non-root user (
app, uid/gid 1000) by default
rather than as root, as a defense-in-depth measure. If you bind-mount a
directory for input and output, you may now need to add a--userargument so
the container can write to it; the correct value differs for rootless Docker,
Podman, and rootful Docker, and is described in the Docker documentation.
Piping the input and output through stdin/stdout still works with no
permission setup. - The Docker images now default their working directory to
/data, so files in
a directory mounted there can be given as relative paths without an explicit--workdir. - The Ubuntu Docker image now installs Tesseract 5 from the Ubuntu archive
instead of the third-partyalex-p/tesseract-ocr5PPA, and the base images
were updated to Ubuntu 26.04 and Alpine 3.24. - Fixed a missing space in the error message shown when OCRmyPDF cannot access
its working directory inside a Docker container. - Updated packaged dependencies, including the optional web service stack
(starlette, tornado, python-multipart) and cryptography.
v17.6.0
- When the optimizer encounters an image it cannot process (for example, an
exotic colorspace that cannot be transcoded), it now logs a concise warning
that the image was left unchanged rather than printing an alarming
traceback. The output file was already valid in these cases; only the
reporting was misleading. The full traceback is still available at debug
verbosity (-v 1) ({issue}846). --pdfa-image-compression=auto(the default) now selects lossless image
compression at-O0so Ghostscript no longer transcodes lossless images to
JPEG during PDF/A generation. At-O1and above,autocontinues to defer
to Ghostscript's heuristic, which may recompress images lossily.-O1(the
default level) is kept as a historical exception because coercing it to
lossless can substantially bloat output; users who want guaranteed lossless
image handling should pass--pdfa-image-compression=losslessor use-O0
({issue}1124).--pdfa-image-compression=losslessnow passes existing JPEG images through
unchanged rather than re-encoding them with a lossless codec. Re-encoding an
already-lossy JPEG losslessly cannot recover quality and only inflates the
file, so JPEGs are preserved while non-JPEG images are encoded losslessly.- OCRmyPDF now validates and repairs malformed page-boundary boxes
(/MediaBox,/CropBox,/TrimBox,/ArtBox,/BleedBox) in its
input, following the PDF 2.0 specification. Coordinates written in invalid
exponential notation are reinterpreted ({issue}1398); rectangles whose
corners are given in reversed order are normalized, which previously crashed
withNegativeDimensionError({issue}1526); and a crop/trim/art/bleed box
that falls outside the MediaBox is clamped to their intersection, or discarded
when that intersection is empty, which previously produced an output with a
zero-height effective page that some viewers refused to open ({issue}1400).
When a box is discarded, clamped, or reinterpreted, OCRmyPDF logs a warning
recommending visual inspection of the output. Thanks @ajdlinux for the initial
fix in PR #1691. - OCRmyPDF now discards an embedded Adobe full-text search index
(/Root/PieceInfo/SearchIndex) from its output. This proprietary index,
produced by Acrobat's "Embed Index" feature, is read only by Adobe Acrobat;
other viewers ignore it and search the text on the fly. Because any change to
a PDF invalidates the index, retaining it after OCRmyPDF rewrites the document
would leave a stale index that returns incorrect search results in Acrobat.
Modern viewers rebuild a search index on demand, so there is no loss of
search capability. - OCRmyPDF now discards embedded per-page thumbnail images (the optional
/Thumbimage XObject on a page) from its output. OCRmyPDF alters page
appearance (deskew, clean, rasterize, re-render) and plugins may edit pages
arbitrarily, so a retained thumbnail would be stale and no longer match its
page. Embedded thumbnails are a navigation aid that modern viewers generate
on demand, so there is no loss of functionality. - Fixed a regression in OCR quality for PDFs that paint a 1-bit image mask
(stencil) with a gray or colored fill color. Previously such pages were
rasterized as 1-bit black-and-white before OCR, so Ghostscript dithered
mid-tone text into an unreadable stipple and Tesseract failed to recognize
it. The rasterizer now inspects the fill color used to paint a mask and
promotes the page to grayscale or full color as needed, so the distinction
is preserved for the OCR engine. This applies to both the Ghostscript and
pypdfium rasterizers. {issue}1688 - The default 1-bit raster device for Ghostscript is now
pngmonod
(error-diffusion) instead ofpngmono(ordered dithering). It produces
better input for OCR on faint or anti-aliased scans at negligible cost and
no change to output file size, since the rasterized image is an
intermediate that is discarded after OCR. - When rasterizing pages with Ghostscript, OCRmyPDF now enables text and
graphics anti-aliasing (-dTextAlphaBits=4 -dGraphicsAlphaBits=4) for the
grayscale and color raster devices. Ghostscript 10.x renders aliased glyphs
that OCR frequently misreads as extra word breaks or substituted characters;
anti-aliasing materially improves OCR accuracy on the Ghostscript
rasterization path, especially for small fonts at moderate resolution. The
1-bit monochrome devices are unaffected, since they perform their own
anti-aliased downscaling and older Ghostscript versions reject alpha-bit
options on them. Note that the default rasterizer (--rasterizer auto)
prefers pypdfium2, which already anti-aliases; this change benefits users who
select--rasterizer ghostscriptor do not have pypdfium2 installed.
OCRmyPDF now also logs which rasterizer rendered each page at debug verbosity
(-v 1), and the--rasterizerhelp text explains the OCR-quality
trade-off, to make such reports easier to diagnose. {issue}1439 - When Tesseract reports a page with many diacritics, OCRmyPDF still logs its
interpreted "lots of diacritics - possibly poor OCR" hint, but now also emits
Tesseract's raw message at debug verbosity (-v 1) so the original wording
is available for diagnosis. {issue}1566 - Added
--mode strip, which removes the invisible OCR text layer from a PDF
in place. Unlike--ocr-engine none --force-ocr, it does not rasterize the
page, so images and visible content are preserved unchanged and the output is
smaller rather than larger. Only text drawn as invisible (PDF text render mode- is removed; some OCR engines -- and OCRmyPDF v2.2 and earlier -- express
text as visible glyphs covered by an opaque image, and that text cannot be
removed this way. {issue}1435
- is removed; some OCR engines -- and OCRmyPDF v2.2 and earlier -- express
v17.5.0
- Added support for the
endalias in--pages, denoting the last page
of the document. For example,--pages 3-endOCRs from page 3 through
the final page. {issue}1615 - Added
--ghostscript-jpeg-qualityand--ghostscript-jpeg-maxdpi
advanced options for tuning Ghostscript's PDF/A output. The optimizer's--jpeg-qualityremains the recommended file-size control. - Fixed pypdfium2 rasterizer clipping content when the CropBox was smaller
than the MediaBox (e.g. JSTOR or cropped PDFs). {issue}1685 - Fixed Form XObject cycle detection in the optimizer's image xref scan.
Self-referential or DAG-shaped Form graphs (notably from PowerPoint
exports) previously produced floods of recursion warnings and could hang
for minutes. {issue}1321 - Tesseract config errors are now surfaced as
TesseractConfigErrorwith
actionable guidance, instead of crashing later with a confusingFileNotFoundErroron the missing hOCR output. {issue}1687 - Refreshed the Chinese README translation. Thanks @cislunarspace.
- Internal refactoring of the
_execandsubprocessmodules to
separate probing from execution. - CI dependency updates.
v17.4.2
- Fixed Python API unconditionally overriding
PIL.Image.MAX_IMAGE_PIXELS
when the caller did not explicitly setmax_image_mpixels. Host
applications (e.g. Paperless-NGX) that configure the PIL limit before
invokingocrmypdf.ocr()now have their setting respected. The CLI
default of 250 megapixels is unchanged. {issue}1665 - Updated uv.lock to avoid pinning a vulnerable version of Pillow. {issue}
1666
v17.4.1
- Fixed RTL text extraction order in the fpdf2 renderer. Arabic lam-alef
ligatures and other multi-character CMap entries were garbled by the bidi
algorithm during text extraction. {issue}1655 - Fixed
work_foldernot being set inPdfContextoptions when using
the Python API. Thanks @bluebox-steven. {issue}1613 - Updated Ghostscript JPEG corruption warning to include the detected version
number, confirming the bug persists in Ghostscript 10.7.0. - Internal refactoring.
- CI dependency updates.
v17.4.0
- Added
--no-overwrite/-noption to prevent overwriting output files.
If the destination file already exists, OCRmyPDF exits with code 5
(OutputFileAccessError). {issue}1642 - Fixed text layer stretching in the fpdf2 renderer for widely-spaced words.
The horizontal scaling (Tz) was incorrectly stretched to fill inter-word gaps
instead of relying on Td positioning, causing text selection to highlight far
beyond the actual word boundaries. {issue}1635 - Fixed
optimize=2oroptimize=3crash when using the Python API without
explicitly settingjpg_qualityorpng_quality. {issue}1641 - Fixed
verapdfavailability check crashing withNotADirectoryErroron
some platforms. {issue}1638
v17.3.0
- Fixed Python API ignoring the
languageparameter, always defaulting toeng. The API now correctly mapslanguageto OcrOptionslanguages
and splits+-separated codes (e.g.eng+deu) to match CLI behavior.
{issue}1640 - Fixed Python API producing empty OCR output because
tesseract_timeout
defaulted to 0, causing Tesseract to time out immediately. The default is
nowNone, falling back to the plugin's 180-second timeout. {issue}1636 - Fixed OCR text layer displacement on PDFs with non-zero MediaBox origins
(e.g. JSTOR or cropped PDFs). The coordinate transformation matrix is now
always computed, not skipped when rotation is zero. {issue}1630 - Restored image overlay support (
--image) for the hocrtransform tool,
enabling sandwich PDF output with the fpdf2 renderer. {issue}1634 - Docker: updated Alpine base image to 3.23.
- Documentation restructured into per-major-version release notes files.
- Release process improvements.
v17.2.0
- Fixed incorrect word spacing in poppler-based PDF viewers and tools (Evince,
pdftotext, and others) where words on the same line appeared separated by
double newlines. This works around a poppler bug where Tz (horizontal scaling)
is not carried across BT/ET boundaries. {issue}1632 - Fixed OCR text layer being visible instead of invisible due to incorrect fpdf2
text rendering mode attribute. This caused OCR text to appear when images were
removed from the PDF. {issue}1631 - Fixed OCR text layer misalignment with non-zero mediabox origins, which
affected cropped PDFs and JSTOR PDFs generated by iText. The--redo-ocr
mode would shift text vertically on these files. {issue}1630 - Fixed Ghostscript rasterization failure with very low DPI values (below 10).
OCRmyPDF now renders at a minimum of 10 DPI and resizes the output to match
the originally requested dimensions. {issue}1612
v17.1.0
- Added
--tagged-pdf-modeto allow skipping the TaggedPDF error message, if desired. - Fixed an issue where deflated JPEGs (FlateDecode + DCTDecode) were counted as
lossless images for the purpose of determining whether to compress to JPEG,
causing file size inflation with some workflows (--mode forcein particular).
v17.0.1
- Fixed output file size inflation when using pypdfium as rasterizer and force-ocr
mode.
v17.0.0
Breaking changes
- Plugin interface migration: Plugin hooks now receive
OcrOptionsobjects instead ofargparse.Namespaceobjects. Most plugins will continue working due to duck-typing
compatibility, but plugin developers should update their type hints fromNamespace
toOcrOptions. - Built-in plugins no longer modify options in-place, improving immutability and
code clarity. - Lossy JBIG2 removed: The
--jbig2-lossyand--jbig2-page-group-sizeoptions have been
removed due to well-documented risks of character substitution errors. These options are now
deprecated and will emit warnings if used. Only lossless JBIG2 compression is supported. - PDF/A output behavior change: If neither Ghostscript nor verapdf is installed,
--output-type auto(the new default) will produce a standard PDF instead of PDF/A. This is
a change from previous versions where Ghostscript was required and PDF/A was always produced.
This configuration is rare but users should be aware of the change.
New features
pypdfium2 rasterizer: Added optional pypdfium2-based PDF rasterization plugin as an
alternative to Ghostscript for page rendering. Use--rasterizer pypdfiumto enable
(requirespip install pypdfium2). The default--rasterizer autoprefers pypdfium when
available and falls back to Ghostscript.Pluggable OCR engines: New
--ocr-engineoption allows selecting OCR engines:auto(default): Uses Tesseracttesseract: Explicit Tesseract selectionnone: Skip OCR entirely for PDF processing-only workflows
This prepares the foundation for future third-party OCR engine plugins.
Smart PDF/A conversion: New
--output-type auto(now the default) produces best-effort
PDF/A output without requiring Ghostscript when the verapdf validator is available. Falls back
to traditional Ghostscript conversion when needed.verapdf integration: Added optional verapdf validation for fast PDF/A conversion. When
available, OCRmyPDF attempts speculative PDF/A conversion using pikepdf, validates with verapdf,
and skips Ghostscript if validation passes.Optional Ghostscript: As a consequence of the changes above, Ghostscript is no longer a required dependency. It is optional.
fpdf2 text renderer: Replaced legacy hOCR text renderer with new fpdf2-based implementation,
providing better multilingual support and more accurate text positioning.Improved Occulta glyphless font: The new Occulta font provides better handling of
zero-width markers and double-width CJK characters for accurate text layer positioning.Expanded multilingual font support: Added FontProvider infrastructure with language-aware
font selection for Devanagari (Hindi, Sanskrit, Marathi, Nepali), CJK (Chinese, Japanese,
Korean), Arabic script, and many other scripts. System font discovery reduces package size.Simplified mode selection: New
--mode(-m) argument consolidates processing options:default: Error if text is found (standard behavior)force: Rasterize all content and run OCR (replaces--force-ocr)skip: Skip pages with existing text (replaces--skip-text)redo: Re-OCR pages, stripping old text layer (replaces--redo-ocr)
Legacy flags remain as silent aliases for backward compatibility.
API improvements
- Centralized validation logic in the
OcrOptionsPydantic model - Removed scattered option mutation throughout the codebase
- Better type safety for plugin development
- Simplified plugin option handling
- New
OcrElement,OcrClass, andBoundingBoxexports for OCR engine plugin developers - Extended
OcrEngineABC withgenerate_ocr()method for direct OCR tree output, eliding the need to translate a modern engine's output to hOCR or directly write to PDF.
Bug fixes
- Fixed double-compression of already-deflated JPEGs.
- Fixed tesseract_cache plugin to properly handle cache misses.
- Fixed handling of PDF page boxes (ArtBox, BleedBox) which were not being processed correctly.
- Added thread safety lock to pypdfium plugin for concurrent operations.
- Improved pdfminer.six compatibility with explicit word spacing.
Documentation
- Updated cookbook to replace deprecated
--tesseract-timeout 0with--ocr-engine none. - Added comprehensive plugin documentation for new OCR engine framework.
Dependency changes
- Requires: one of
pypdfium2orghostscriptfor PDF rasterization (PDF to image)- Preferred: both
- Requires: one of
verapdforghostscriptfor PDF/A generation- Preferred: both
- Recommended:
pypdfium2for PDF rasterization (new dependency) - Recommended:
ghostscript(used to be Required) - Recommended: Noto fonts for improved OCR text positioning
- Optional:
verapdffor fast PDF/A validation (new dependency) - Requires:
fpdf2for text layer rendering (new dependency) - Recommended: replace
typerwithcycloptsin misc scripts (new dependency) - See docs/maintainers.md for details.
Migration guide for plugin developers
- Update imports:
from ocrmypdf._options import OcrOptions - Update type hints:
def check_options(options: OcrOptions)instead ofoptions: Namespace - Attribute access remains unchanged:
options.languages,options.output_type, etc. - Remove any in-place option modifications - compute values at point of use instead
- Most existing plugins will continue working without changes due to duck-typing
--- METRICS ---
- Files Extracted: 39
- Estimated Token Budget: ~90579 tokens
- Canonical Reference: https://codewiki.google/github.com/ocrmypdf/OCRmyPDF