olmocr Documentation

File: README.md

A toolkit for converting PDFs and other image-based document formats into clean, readable, plain text format.

Try the online demo: https://olmocr.allenai.org/

Features:

Convert PDF, PNG, and JPEG based documents into clean Markdown
Support for equations, tables, handwriting, and complex formatting
Automatically removes headers and footers
Convert into text with a natural reading order, even in the presence of
figures, multi-column layouts, and insets
Efficient, less than $200 USD per million pages converted
(Based on a 7B parameter VLM, so it requires a GPU)

News

October 21, 2025 - v0.4.0 - New model release, boosts olmOCR-bench score by ~4 points using synthetic data and introduces RL training.
August 13, 2025 - v0.3.0 - New model release, fixes auto-rotation detection, and hallucinations on blank documents.
July 24, 2025 - v0.2.1 - New model release, scores 3 points higher on olmOCR-Bench, also runs significantly faster because it's default FP8, and needs much fewer retries per document.
July 23, 2025 - v0.2.0 - New cleaned up trainer code, makes it much simpler to train olmOCR models yourself.
June 17, 2025 - v0.1.75 - Switch from sglang to vllm based inference pipeline, updated docker image to CUDA 12.8.
May 23, 2025 - v0.1.70 - Official docker support and images are now available! See Docker usage
May 19, 2025 - v0.1.68 - olmOCR-Bench launch, scoring 77.4. Launch includes 2 point performance boost in olmOCR pipeline due to bug fixes with prompts.
Mar 17, 2025 - v0.1.60 - Performance improvements due to better temperature selection in sampling.
Feb 25, 2025 - v0.1.58 - Initial public launch and demo.

Benchmark

olmOCR-Bench:
We also ship a comprehensive benchmark suite covering over 7,000 test cases across 1,400 documents to help measure performance of OCR systems.

	ArXiv	Oldscansmath	Tables	Oldscans	Headers&footers	Multicolumn	Longtinytext	Base	Overall
Mistral OCR API	77.2	67.5	60.6	29.3	93.6	71.3	77.1	99.4	72.0±1.1
Marker 1.10.1	83.8	66.8	72.9	33.5	86.6	80.0	85.7	99.3	76.1±1.1
MinerU 2.5.4*	76.6	54.6	84.9	33.7	96.6	78.2	83.5	93.7	75.2±1.1
DeepSeek-OCR	77.2	73.6	80.2	33.3	96.1	66.4	79.4	99.8	75.7±1.0
Nanonets-OCR2-3B	75.4	46.1	86.8	40.9	32.1	81.9	93.0	99.6	69.5±1.1
PaddleOCR-VL*	85.7	71.0	84.1	37.8	97.0	79.9	85.7	98.5	80.0±1.0
Infinity-Parser 7B*	84.4	83.8	85.0	47.9	88.7	84.2	86.4	99.8	82.5±?
Chandra OCR 0.1.0*	82.2	80.3	88.0	50.4	90.8	81.2	92.3	99.9	83.1±0.9

olmOCR v0.4.0	83.0	82.3	84.9	47.7	96.1	83.7	81.9	99.7	82.4±1.1

Installation

System Dependencies

You will need to install poppler-utils and additional fonts for rendering PDF images.

Install dependencies (Ubuntu/Debian):

bash

sudo apt-get update
sudo apt-get install poppler-utils ttf-mscorefonts-installer msttcorefonts fonts-crosextra-caladea fonts-crosextra-carlito gsfonts lcdf-typetools

Python Installation

Set up a conda environment and install olmocr. The requirements for running olmOCR
are difficult to install in an existing python environment, so please do make a clean python environment to install into.

bash

conda create -n olmocr python=3.11
conda activate olmocr

Choose the installation option that matches your use case:

Option 1: Remote Inference (Lightweight)

If you plan to use a remote vLLM server with the --server flag, install the base package:

bash

pip install olmocr

This avoids installing heavy GPU dependencies like PyTorch (~2GB+).

Option 2: Local GPU Inference

Requirements:

Recent NVIDIA GPU (tested on RTX 4090, L40S, A100, H100) with at least 12 GB of GPU RAM
30GB of free disk space

For running inference with your own GPU:

bash

pip install olmocr[gpu] --extra-index-url https://download.pytorch.org/whl/cu128

# Recommended: Install flash infer for faster inference on GPU
pip install https://download.pytorch.org/whl/cu128/flashinfer/flashinfer_python-0.2.5%2Bcu128torch2.7-cp38-abi3-linux_x86_64.whl

Option 3: Beaker Cluster Execution

For submitting jobs to Beaker clusters with the --beaker flag:

bash

pip install olmocr[beaker]

Option 4: Benchmark Suite

For running the olmOCR benchmark suite:

bash

pip install olmocr[bench]

Combined Installation

You can combine multiple options:

bash

# GPU + Beaker support
pip install olmocr[gpu,beaker] --extra-index-url https://download.pytorch.org/whl/cu128

# GPU + Benchmark support
pip install olmocr[gpu,bench] --extra-index-url https://download.pytorch.org/whl/cu128

Troubleshooting

If you run into errors about too many open files, update your ulimit:

bash

ulimit -n 65536

Usage Examples

For quick testing, try the web demo.

Convert a Single PDF (Local GPU):

bash

# Download a sample PDF
curl -o olmocr-sample.pdf https://olmocr.allenai.org/papers/olmocr_3pg_sample.pdf

# Convert it to markdown
olmocr ./localworkspace --markdown --pdfs olmocr-sample.pdf

Convert an Image file:

bash

olmocr ./localworkspace --markdown --pdfs random_page.png

Convert Multiple PDFs:

bash

olmocr ./localworkspace --markdown --pdfs tests/gnarly_pdfs/*.pdf

Use Remote Inference Server:

bash

olmocr ./localworkspace --server http://remote-server:8000/v1 --model allenai/olmOCR-2-7B-1025-FP8 --markdown --pdfs *.pdf

With the --markdown flag, results will be stored as markdown files inside of ./localworkspace/markdown/.

Note: You can also use python -m olmocr.pipeline instead of olmocr if you prefer.

Viewing Results

The ./localworkspace/ workspace folder will then have both Dolma and markdown files (if using --markdown).

bash

cat localworkspace/markdown/olmocr-sample.md

text

olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models
...

Using an Inference Provider or External Server

If you have a vLLM server already running elsewhere (or any inference platform implementing the OpenAI API), you can point olmOCR to use it instead of spawning a local instance.

Installation for Remote Inference:

bash

# Lightweight installation - no GPU dependencies needed
pip install olmocr

Using an External Server:

bash

# Use external vLLM server instead of local one
olmocr ./localworkspace --server http://remote-server:8000/v1 --model allenai/olmOCR-2-7B-1025-FP8 --markdown --pdfs tests/gnarly_pdfs/*.pdf

The served model name in vLLM needs to match the value provided in --model.

Example vLLM Server Launch:

bash

vllm serve allenai/olmOCR-2-7B-1025-FP8 --max-model-len 16384

Verified External Providers

We have tested olmOCR-2-7B-1025-FP8 on these external model providers and confirmed that they work

	$/1M Input tokens	$/1M Output tokens	Example Command
Cirrascale	$0.07	$0.15	`olmocr ./workspace --server https://ai2endpoints.cirrascale.ai/api --api_key sk-XXXXXXX --workers 1 --max_concurrent_requests 20 --model olmOCR-2-7B-1025 --pdfs tests/gnarly_pdfs/*.pdf`
DeepInfra	$0.09	$0.19	`olmocr ./workspace --server https://api.deepinfra.com/v1/openai --api_key DfXXXXXXX --workers 1 --max_concurrent_requests 20 --model allenai/olmOCR-2-7B-1025 --pdfs tests/gnarly_pdfs/*.pdf`
Parasail	$0.10	$0.20	`olmocr ./workspace --server https://api.parasail.io/v1 --api_key psk-XXXXX --workers 1 --max_concurrent_requests 20 --model allenai/olmOCR-2-7B-1025 --pdfs tests/gnarly_pdfs/*.pdf`

Notes on arguments

--server: Defines the OpenAI-compatible endpoint: ex https://api.deepinfra.com/v1/openai
--api_key: Your API key, bassed in via Authorization Bearer HTTP header
--max_concurrent_requests: Max concurrent requests that will be in-flight to the inference provider at one time
--workers: Max number of page groups that will be processed at once. You may want to set this to 1 so that you finish one group of stuff before moving on.
--pages_per_group: You may want a smaller number of pages per group as many external provides have lower concurrent request limits
--model: The model identifier, ex. allenai/olmOCR-2-7B-1025, different providers have different names, and if you run locally, you can use olmocr
Other arguments work the same as with local inference

Multi-node / Cluster Usage

If you want to convert millions of PDFs using multiple nodes running in parallel, olmOCR supports
reading PDFs from AWS S3 and coordinating work using an AWS S3 output bucket.

Start the first worker node:

bash

olmocr s3://my_s3_bucket/pdfworkspaces/exampleworkspace --pdfs s3://my_s3_bucket/jakep/gnarly_pdfs/*.pdf

This sets up a simple work queue in your AWS bucket and starts converting PDFs.

On subsequent worker nodes:

bash

olmocr s3://my_s3_bucket/pdfworkspaces/exampleworkspace

They will automatically start grabbing items from the same workspace queue.

Using Beaker for Cluster Execution

If you are at Ai2 and want to linearize millions of PDFs efficiently using beaker, install with Beaker support:

bash

pip install olmocr[gpu,beaker] --extra-index-url https://download.pytorch.org/whl/cu128

Then use the --beaker flag to prepare the workspace locally and launch N GPU workers in the cluster:

bash

olmocr s3://my_s3_bucket/pdfworkspaces/exampleworkspace --pdfs s3://my_s3_bucket/jakep/gnarly_pdfs/*.pdf --beaker --beaker_gpus 4

Using Docker

Pull the Docker image (large, includes the model, ~30GB):

bash

docker pull alleninstituteforai/olmocr:latest-with-model

For advanced users who want to manage their own model downloads, we also provide a base image without the model:

bash

docker pull alleninstituteforai/olmocr:latest

Quick Start - Process PDFs

Process a single PDF in your current directory:

bash

docker run --gpus all \
  -v $(pwd):/workspace \
  alleninstituteforai/olmocr:latest-with-model \
  -c "olmocr /workspace/output --markdown --pdfs /workspace/sample.pdf"

Process multiple PDFs:

bash

docker run --gpus all \
  -v /path/to/pdfs:/input \
  -v /path/to/output:/output \
  alleninstituteforai/olmocr:latest-with-model \
  -c "olmocr /output --markdown --pdfs /input/*.pdf"

Interactive Mode

Run the container interactively for exploration and debugging:

bash

docker run -it --gpus all alleninstituteforai/olmocr:latest-with-model

Visit our Docker repository on Docker Hub for more information.

Full Documentation

To see all available options:

bash

olmocr --help
usage: pipeline.py [-h] [--pdfs [PDFS ...]] [--model MODEL] [--workspace_profile WORKSPACE_PROFILE] [--pdf_profile PDF_PROFILE] [--pages_per_group PAGES_PER_GROUP] [--max_page_retries MAX_PAGE_RETRIES] [--max_page_error_rate MAX_PAGE_ERROR_RATE] [--workers WORKERS]
                   [--apply_filter] [--stats] [--markdown] [--target_longest_image_dim TARGET_LONGEST_IMAGE_DIM] [--target_anchor_text_len TARGET_ANCHOR_TEXT_LEN] [--guided_decoding] [--gpu-memory-utilization GPU_MEMORY_UTILIZATION] [--max_model_len MAX_MODEL_LEN]
                   [--tensor-parallel-size TENSOR_PARALLEL_SIZE] [--data-parallel-size DATA_PARALLEL_SIZE] [--port PORT] [--server SERVER] [--beaker] [--beaker_workspace BEAKER_WORKSPACE] [--beaker_cluster BEAKER_CLUSTER] [--beaker_gpus BEAKER_GPUS] [--beaker_priority BEAKER_PRIORITY]
                   workspace

Manager for running millions of PDFs through a batch inference pipeline

positional arguments:
  workspace             The filesystem path where work will be stored, can be a local folder, or an s3 path if coordinating work with many workers, s3://bucket/prefix/

options:
  -h, --help            show this help message and exit
  --pdfs [PDFS ...]     Path to add pdfs stored in s3 to the workspace, can be a glob path s3://bucket/prefix/*.pdf or path to file containing list of pdf paths
  --model MODEL         Path where the model is located, allenai/olmOCR-7B-0725-FP8 is the default, can be local, s3, or hugging face.
  --workspace_profile WORKSPACE_PROFILE
                        S3 configuration profile for accessing the workspace
  --pdf_profile PDF_PROFILE
                        S3 configuration profile for accessing the raw pdf documents
  --pages_per_group PAGES_PER_GROUP
                        Aiming for this many pdf pages per work item group
  --max_page_retries MAX_PAGE_RETRIES
                        Max number of times we will retry rendering a page
  --max_page_error_rate MAX_PAGE_ERROR_RATE
                        Rate of allowable failed pages in a document, 1/250 by default
  --workers WORKERS     Number of workers to run at a time
  --apply_filter        Apply basic filtering to English pdfs which are not forms, and not likely seo spam
  --stats               Instead of running any job, reports some statistics about the current workspace
  --markdown            Also write natural text to markdown files preserving the folder structure of the input pdfs
  --target_longest_image_dim TARGET_LONGEST_IMAGE_DIM
                        Dimension on longest side to use for rendering the pdf pages
  --target_anchor_text_len TARGET_ANCHOR_TEXT_LEN
                        Maximum amount of anchor text to use (characters), not used for new models
  --guided_decoding     Enable guided decoding for model YAML type outputs

VLLM arguments:
  --gpu-memory-utilization GPU_MEMORY_UTILIZATION
                        Fraction of VRAM vLLM may pre-allocate for KV-cache (passed through to vllm serve).
  --max_model_len MAX_MODEL_LEN
                        Upper bound (tokens) vLLM will allocate KV-cache for, lower if VLLM won't start
  --tensor-parallel-size TENSOR_PARALLEL_SIZE, -tp TENSOR_PARALLEL_SIZE
                        Tensor parallel size for vLLM
  --data-parallel-size DATA_PARALLEL_SIZE, -dp DATA_PARALLEL_SIZE
                        Data parallel size for vLLM
  --port PORT           Port to use for the VLLM server
  --server SERVER       URL of external vLLM (or other compatible provider)
                        server (e.g., http://hostname:port). If provided,
                        skips spawning local vLLM instance

beaker/cluster execution:
  --beaker              Submit this job to beaker instead of running locally
  --beaker_workspace BEAKER_WORKSPACE
                        Beaker workspace to submit to
  --beaker_cluster BEAKER_CLUSTER
                        Beaker clusters you want to run on
  --beaker_gpus BEAKER_GPUS
                        Number of gpu replicas to run
  --beaker_priority BEAKER_PRIORITY
                        Beaker priority level for the job

Code overview

There are some nice reusable pieces of the code that may be useful for your own projects:

A prompting strategy to get really good natural text parsing using ChatGPT 4o - buildsilver.py
Basic filtering by language and SEO spam removal - filter.py
SFT Finetuning code for Qwen2.5-VL - train.py
GRPO RL Trainer - grpo_train.py
Synthetic data generation - mine_html_templates.py
Processing millions of PDFs through a finetuned model using VLLM - pipeline.py
Viewing Dolma docs created from PDFs - dolmaviewer.py

Team

olmOCR is developed and maintained by the AllenNLP team, backed by the Allen Institute for Artificial Intelligence (AI2).
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
To learn more about who specifically contributed to this codebase, see our contributors page.

License

olmOCR is licensed under Apache 2.0.
A full copy of the license can be found on GitHub.

Citing

For olmOCR v1 and OlmOCR-bench:

bibtex

@misc{olmocrbench,
      title={{olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models}},
      author={Jake Poznanski and Jon Borchardt and Jason Dunkelberger and Regan Huff and Daniel Lin and Aman Rangapur and Christopher Wilhelm and Kyle Lo and Luca Soldaini},
      year={2025},
      eprint={2502.18443},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.18443},
}

For olmOCR v2 Unit Testing Rewards with RL:

bibtex

@misc{olmocr2,
      title={olmOCR 2: Unit Test Rewards for Document OCR}, 
      author={Jake Poznanski and Luca Soldaini and Kyle Lo},
      year={2025},
      eprint={2510.19817},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2510.19817}, 
}

File: CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

Unreleased

[v0.4.27](https://github.com/allenai/olmocr/releases/tag/v0.4.27) - 2026-03-12

[v0.4.26](https://github.com/allenai/olmocr/releases/tag/v0.4.26) - 2026-03-06

[v0.4.25](https://github.com/allenai/olmocr/releases/tag/v0.4.25) - 2026-01-25

[v0.4.24](https://github.com/allenai/olmocr/releases/tag/v0.4.24) - 2026-01-23

[v0.4.23](https://github.com/allenai/olmocr/releases/tag/v0.4.23) - 2026-01-23

[v0.4.22](https://github.com/allenai/olmocr/releases/tag/v0.4.22) - 2026-01-23

[v0.4.21](https://github.com/allenai/olmocr/releases/tag/v0.4.21) - 2026-01-23

[v0.4.20](https://github.com/allenai/olmocr/releases/tag/v0.4.20) - 2026-01-20

[v0.4.19](https://github.com/allenai/olmocr/releases/tag/v0.4.19) - 2026-01-20

[v0.4.18](https://github.com/allenai/olmocr/releases/tag/v0.4.18) - 2026-01-20

[v0.4.17](https://github.com/allenai/olmocr/releases/tag/v0.4.17) - 2026-01-20

[v0.4.16](https://github.com/allenai/olmocr/releases/tag/v0.4.16) - 2026-01-12

[v0.4.15](https://github.com/allenai/olmocr/releases/tag/v0.4.15) - 2026-01-09

[v0.4.14](https://github.com/allenai/olmocr/releases/tag/v0.4.14) - 2026-01-08

[v0.4.13](https://github.com/allenai/olmocr/releases/tag/v0.4.13) - 2026-01-08

[v0.4.12](https://github.com/allenai/olmocr/releases/tag/v0.4.12) - 2025-12-10

[v0.4.11](https://github.com/allenai/olmocr/releases/tag/v0.4.11) - 2025-12-10

[v0.4.10](https://github.com/allenai/olmocr/releases/tag/v0.4.10) - 2025-12-09

[v0.4.9](https://github.com/allenai/olmocr/releases/tag/v0.4.9) - 2025-12-09

[v0.4.7](https://github.com/allenai/olmocr/releases/tag/v0.4.7) - 2025-12-01

[v0.4.6](https://github.com/allenai/olmocr/releases/tag/v0.4.6) - 2025-11-17

[v0.4.5](https://github.com/allenai/olmocr/releases/tag/v0.4.5) - 2025-11-14

[v0.4.4](https://github.com/allenai/olmocr/releases/tag/v0.4.4) - 2025-11-04

[v0.4.3](https://github.com/allenai/olmocr/releases/tag/v0.4.3) - 2025-10-31

[v0.4.2](https://github.com/allenai/olmocr/releases/tag/v0.4.2) - 2025-10-22

[v0.4.1](https://github.com/allenai/olmocr/releases/tag/v0.4.1) - 2025-10-22

[v0.4.0](https://github.com/allenai/olmocr/releases/tag/v0.4.0) - 2025-10-22

[v0.3.9](https://github.com/allenai/olmocr/releases/tag/v0.3.9) - 2025-10-07

[v0.3.8](https://github.com/allenai/olmocr/releases/tag/v0.3.8) - 2025-10-06

[v0.3.7](https://github.com/allenai/olmocr/releases/tag/v0.3.7) - 2025-10-06

[v0.3.6](https://github.com/allenai/olmocr/releases/tag/v0.3.6) - 2025-09-29

[v0.3.4](https://github.com/allenai/olmocr/releases/tag/v0.3.4) - 2025-08-31

[v0.3.3](https://github.com/allenai/olmocr/releases/tag/v0.3.3) - 2025-08-15

[v0.3.2](https://github.com/allenai/olmocr/releases/tag/v0.3.2) - 2025-08-14

[v0.3.1](https://github.com/allenai/olmocr/releases/tag/v0.3.1) - 2025-08-14

[v0.3.0](https://github.com/allenai/olmocr/releases/tag/v0.3.0) - 2025-08-13

[v0.2.3](https://github.com/allenai/olmocr/releases/tag/v0.2.3) - 2025-08-04

[v0.2.2](https://github.com/allenai/olmocr/releases/tag/v0.2.2) - 2025-08-04

[v0.2.1](https://github.com/allenai/olmocr/releases/tag/v0.2.1) - 2025-07-23

[v0.2.0](https://github.com/allenai/olmocr/releases/tag/v0.2.0) - 2025-07-23

[v0.1.76](https://github.com/allenai/olmocr/releases/tag/v0.1.76) - 2025-06-23

[v0.1.75](https://github.com/allenai/olmocr/releases/tag/v0.1.75) - 2025-06-17

[v0.1.74](https://github.com/allenai/olmocr/releases/tag/v0.1.74) - 2025-06-17

[v0.1.73](https://github.com/allenai/olmocr/releases/tag/v0.1.73) - 2025-06-17

[v0.1.72](https://github.com/allenai/olmocr/releases/tag/v0.1.72) - 2025-06-17

[v0.1.71](https://github.com/allenai/olmocr/releases/tag/v0.1.71) - 2025-05-30

[v0.1.70](https://github.com/allenai/olmocr/releases/tag/v0.1.70) - 2025-05-23

[v0.1.69](https://github.com/allenai/olmocr/releases/tag/v0.1.69) - 2025-05-20

[v0.1.68](https://github.com/allenai/olmocr/releases/tag/v0.1.68) - 2025-05-19

[v0.1.60](https://github.com/allenai/olmocr/releases/tag/v0.1.60) - 2025-03-17

[v0.1.58](https://github.com/allenai/olmocr/releases/tag/v0.1.58) - 2025-02-15

[v0.1.53](https://github.com/allenai/olmocr/releases/tag/v0.1.53) - 2025-02-14

Fixed git checks
Added gemini and claude runners and a viewer.

File: RELEASE_PROCESS.md

GitHub Release Process

Steps

Update the version in olmocr/version.py.
Run the release script:
bash
```
./scripts/release.sh
```
This will commit the changes to the CHANGELOG and version.py files and then create a new tag in git
which will trigger a workflow on GitHub Actions that handles the rest.

Fixing a failed release

If for some reason the GitHub Actions release workflow failed with an error that needs to be fixed, you'll have to delete both the tag and corresponding release from GitHub. After you've pushed a fix, delete the tag from your local clone with

bash

git tag -l | xargs git tag -d && git fetch -t

Then repeat the steps above.

File: .readthedocs.yaml

version: 2

sphinx:
configuration: docs/source/conf.py
fail_on_warning: true

python:
version: "3.8"
install:
- method: pip
path: .
extra_requirements:
- dev

File: docs/source/index.md

olmocr

{toctree}

:maxdepth: 2
:hidden:
:caption: Getting started

installation
overview

{toctree}

:hidden:
:caption: Development

CHANGELOG
CONTRIBUTING
License <https://raw.githubusercontent.com/allenai/olmocr/main/LICENSE>
GitHub Repository <https://github.com/allenai/olmocr>

Indices and tables

{eval-rst}

* :ref:`genindex`
* :ref:`modindex`

File: docs/source/installation.md

Installation

olmocr supports Python >= 3.8.

Installing with `pip`

olmocr is available on PyPI. Just run

bash

pip install olmocr

Installing from source

To install olmocr from source, first clone the repository:

bash

git clone https://github.com/allenai/olmocr.git
cd olmocr

Then run

bash

pip install -e .

File: docs/source/overview.md

Overview

File: olmocr/bench/README.md

olmOCR-Bench

Dataset Link: https://huggingface.co/datasets/allenai/olmOCR-bench

We develop olmOCR-Bench in order to automatically and effectively evaluate document-level OCR of various tools.

olmOCR-Bench works by testing various "facts" about document pages at the PDF-level.
Our intention is that each "fact" is very simple, unambiguous, and machine-checkable, similar to a unit test. For example, once your document has been OCRed, we may check that a particular sentence appears exactly somewhere on the page.

We stay away from soft metrics like edit distance comparisons, because they may assign lower scores for parses of the document that differ from the reference, but may in fact still be correct. For example, on a document containing multiple distinct articles: you want the text of each article to be grouped together, but the relative order of the two articles may not be critical. Also, some documents may have critical details, like switching x and y in an equation that can make all the difference in understanding, but would appear as just a single character edit in an edit-distance metric.

olmOCR-bench operates on single page PDFs directly. We make this choice because PDFs do preserve some digital metadata and information which may be helpful to some OCR systems. Almost any other format can be converted to a PDF, but not the reverse, so we try to preserve these original documents where possible.

We have run the benchmark against some contemporary OCR pipelines, but it is really easy
to run it against your own OCR tools. Your tool just needs to support Markdown or plain text output.

Results

	ArXiv	Oldscansmath	Tables	Oldscans	Headers&footers	Multicolumn	Longtinytext	Base	Overall
Mistral OCR API	77.2	67.5	60.6	29.3	93.6	71.3	77.1	99.4	72.0±1.1
Marker 1.10.1	83.8	66.8	72.9	33.5	86.6	80.0	85.7	99.3	76.1±1.1
MinerU 2.5.4*	76.6	54.6	84.9	33.7	96.6	78.2	83.5	93.7	75.2±1.1
DeepSeek-OCR	77.2	73.6	80.2	33.3	96.1	66.4	79.4	99.8	75.7±1.0
Nanonets-OCR2-3B	75.4	46.1	86.8	40.9	32.1	81.9	93.0	99.6	69.5±1.1
PaddleOCR-VL*	85.7	71.0	84.1	37.8	97.0	79.9	85.7	98.5	80.0±1.0
Infinity-Parser 7B*	84.4	83.8	85.0	47.9	88.7	84.2	86.4	99.8	82.5±?
Chandra OCR 0.1.0*	82.2	80.3	88.0	50.4	90.8	81.2	92.3	99.9	83.1±0.9

olmOCR (first release)	63.3	67.5	62.3	38.6	93.4	67.6	54.8	97.9	68.2±1.1
v0.1.60 + Dynamic temp scaling	71.4	73.1	65.6	40.5	93.2	76.6	64.9	96.7	72.8±1.2
v0.1.68 + Better prompting	76.3	76.0	70.2	43.2	94.1	77.5	71.9	96.8	75.8±1.0
v0.2.0 + New trainer, YAML, img resize, Qwen 2.5 VL	78.8	77.5	71.9	45.4	94.2	78.6	81.4	99.8	78.5±1.1
v0.3.0 + Handle blank pages	78.6	79.9	72.9	43.9	95.1	77.3	81.2	98.9	78.5±1.1
v0.4.0 + Synth data, RLVR, souping	83.0	82.3	84.9	47.7	96.1	83.7	81.9	99.7	82.4±1.1

^{_{Results are reproduced in-house, except those marked with *, which are reported by model authors.}}

Sourcing Documents and Tests

We define 7 distinct document types that we found olmOCR (or its earlier iterations) often struggled to process and defined custom acquisition strategies for each (described below). We removed documents that both contained PII and were not meant for public dissemination. We also decontaminate against documents that appear in olmOCR-Mix via URL level deduplication. To scale creation of test cases over these documents, we combined manual design and review with prompting GPT-4o.

Document Types

arXiv Math (AR): We downloaded a recent set of papers from the math subset of arXiv, selecting manuscripts with a single TeX source file and corresponding rendered PDF. To select a candidate LATEX expression from a page to use in a test, we (1) ran olmOCR to identify candidate pages with TeX, (2) match pages back to original TeX source, and (3) validate matched TeX rendering compatibility with KaTeX. We manually verify the final set of test cases to exclude instances where custom macros produce renderings that deviate from standard LATEX and to split multi-part equations into smaller test cases.
Old Scans Math (OSM): We crawl old, public domain math textbooks from the Internet Archive, extracting random pages from these documents. We similarly use olmOCR to find candidate pages with formulas, but this time manually annotate each formula on the page to use as test cases.
Tables (TA): We sampled more documents from the same internal crawled PDF repository used to create olmOCR-Mix and filtered to those which had tables using a simple prompt with Gemini-Flash-2.0. On pages with tables, we prompted Gemini-Flash-2.0 for the relationships between randomly chosen cells. We manually reviewed those tests for accuracy.
Old Scans (OS): We sampled historical letters and typewritten documents with existing human transcriptions from the Library of Congress digital archives. We then wrote a small script to generate Natural Reading Order cases consisting of sentences that were naturally before or after one another in the original human transcriptions. We manually added test cases to cover some headers/footers which should have been excluded from any OCR version of these documents. All of the test cases then underwent a second pass of human review for accuracy.
Headers Footers (HF): We sampled documents from the same internally crawled PDF repository as olmOCR-Mix. We used DocLayout-YOLO to identify page regions labeled as headers or footers using the abandon category. To extract the text from these header/footer regions, we visually mask out the rest of the document and prompt Gemini-Flash-2.0 for the content. These extracted snippets are added as test cases that should be absent in linearized output. We manually reviewed to remove mistakenly filtered text and to set conditions such as limiting the search area to the first N or last N characters.
Multi Column (MC): We visually sample documents from our internal crawled PDF repository to find documents with multi-column layouts and multiple articles on one page. We use Claude-Sonnet-3.7 to render those pages to HTML, and from that HTML, we extract text segments before/after one another. We manually review each entry for accuracy. We purposely select simple text blocks from coherent regions of the document, and avoid including any math formulas, superscripts, or subscripts in these tests.
Long Tiny Text (LTT): We crawled documents from the Internet Archive containing a large amount of dense, small print on a single page. Such documents include pages from a dictionary or pages of references from academic papers. We then generate test cases using Gemini-Flash-2.0 and verify them manually.

Benchmark Principles

As we created olmOCR-bench, we also kept a few general rules in mind:

We expect your OCR system to output a plain-text Unicode document in a reading order that would be considered natural.
Documents from the benchmark should fit on a standard A4 piece of paper and still be readable to a human.
Markdown syntax is allowed, but ignored. Ex. if we are looking for the word "enlightenment" to appear on a page, and your system outputs "**enlightenment**" in Markdown bold, that still counts.
olmOCR-bench is not position sensitive, ex. we check that a sentence or math equation appears anywhere on a page. The exception to this is header/footer tests where we want to find simple page numbers appearing in the first or last few characters of a page.
Tables can be in either Markdown syntax, or as an html ``.
Math equations must render with Katex and be delimeted with $, $$, \(, or \[.
Math equations are not position sensitive either, so if we are checking for
$ 3x^2 $ to appear on a page, then outputting $ \int_a^b{ 3x ^ 2dx} $ counts.
We normalize all Unicode to NFC before running the benchmark, so if your OCR model outputs é vs e + ◌́ then either way should not affect your benchmark score.
We normalize all the different variants of hyphens to the ascii -, all the variants of double quoets to ascii " and all variants of single quotes/apostrophes to ascii '. You should score the same on the benchmark if you output - vs —
All facts checked about documents are either pass/fail. We want it to be very clear if your OCR system fails a test, and if so, what output would make it pass.

olmOCR-Bench Test classes

Text presence
- This task makes sure that a given small piece of text (ex. 1-3 sentence level) is present within
  a parsed document. Soft/fuzzy matching is allowed, as well as specifying if the text must be in the first N or last N characters of the document. Case sensitive by default.
Text absense
- This task makes sure that a given piece of next does NOT appear in the OCR'ed version of a document. We generally want our OCR systems to filter out content like headers/footers/page numbers from documents. The same fuzzy matching as in Text Presence tests is allowed.
Natural Reading Order
- This task ensures that blocks of text which are present have a defined order relative to one another. For example,
  on a document that contains multiple news articles on one page, you'd want to see that the first sentence of the
  first article appears after the heading of that article. But, you may be okay with swapping the order of those
  two articles.
Table Accuracy
- Both Markdown and HTML based tables are supported. These tests check that a cell with a given text exists somewhere in the table, and that its neighbors have certain properties. Ex. A cell exists on this page with text "4.5%" and above that is a cell with the text "2.4%". However, it's important to note that some tests depend on rowspan and colspan information being present in the table, which is only available with HTML based tables. This means that a model outputting only markdown tables cannot achieve a max score on this section.
Math Formula Accuracy
- We render a given Latex style equation using Katex in a headless browser. And then see if it exists anywhere in the final OCRed document. Matching is performed on a relative symbol level, ex. in "\f\relax{x} = \int_{-\infty}^\infty
  x^2dx" we check that a ∫ appears to the left of a x, x appears to the left of dx, etc...

Downloading and running the benchmark

Currently the full benchmark data is located here:
https://huggingface.co/datasets/allenai/olmOCR-bench

To run a benchmark, first install the bench requirements

bash

conda create -n olmocr python=3.11
conda activate olmocr

git clone https://github.com/allenai/olmocr.git
cd olmocr

# Install olmocr and the requirements needed to run the benchmark
pip install -e .[bench]

# Configure playwright headless browser to run the math rendering tests
playwright install chromium

# Now clone the benchmark data from hugging face, this includes the PDFs and JSON annotation data
huggingface-cli download --repo-type dataset --resume-download allenai/olmOCR-bench --local-dir ./olmOCR-bench

Convert your documents

bash

# You will need to install the [gpu] subset of olmocr dependencies to run gpu inference
# Then convert using using olmocr.bench.convert, see the olmocr/bench/runners directory for options
pip install olmocr[gpu] --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/
python -m olmocr.bench.convert olmocr_pipeline --dir ./olmOCR-bench/bench_data

# OR, you can use the pipeline to convert the benchmark PDFs and move them into the final format
python -m olmocr.pipeline ./localworkspace --markdown --pdfs ./olmOCR-bench/bench_data/pdfs/**/*.pdf 
python olmocr/bench/scripts/workspace_to_bench.py localworkspace/ olmOCR-bench/bench_data/olmocr --bench-path ./olmOCR-bench/

Now run the benchmark

bash

python -m olmocr.bench.benchmark --dir ./olmOCR-bench/bench_data

Previewing the benchmark questions

We have an internal data annotation tool that can be used to review the questions in the benchmark, and make edits.

bash

python -m olmocr.bench.review_app --port 5000 --debug ./olmOCR-bench/bench_data/multi_column.jsonl --force

File: olmocr/bench/sample_data/olmocr_pipeline/blank_book_pg1_pg1_repeat1.md

[Empty file]

File: olmocr/bench/sample_data/olmocr_pipeline/buildingnotes_pg1_repeat1.md

Master - 7 1/4 - 36"
Master Bath - 7 1/4 - 30"
Laundry - 4 3/4 - 36"
Bath - 7 1/4 - 24"
MUD - 7 - 36"
UTIL - 8 1/4 - 36"
DOWN BATH - 7 1/4 - 32"
BUT KIT - 6 3/4 - 30
PANTRY - 4 3/4 - 24
6 WEST - 32 9/8 - 32
6 WEST BATH 5" - 24"

File: olmocr/bench/sample_data/olmocr_pipeline/discoverworld_crazy_table4_pg1_repeat1.md

Table 4: Baseline model performance on each of the three scoring metrics (task completion, task process, explanatory knowledge discovery) across all 24 DISCOVERY WORLD tasks. Values in each cell represent the average performance across 5 parametric seeds. Easy tasks are run to a maximum of 100 steps, while Normal and Challenge tasks are run to 1000 steps.

#	Topic	Task	ReACT	Plan+Execute	Hypothesizer
1	Proteomics	Clustering	0.87	0.20	0.20
2	Chemistry	Exploring Combinations and Hill Climbing	0.88	0.40	0.40
3	Archaeology	Correlations	0.88	0.40	0.60
4	Easy	Single substances	0.87	1.00	1.00
5	Normal	Mix of 3 substances	0.82	0.00	0.00
6	Challenge	Mix of 4 substances	0.90	0.40	0.00
7	Easy	Simple instrument	0.27	0.60	0.00
8	Normal	Instrument Use	0.72	0.40	0.30
9	Challenge	Correlation	0.46	0.20	0.00
10	Reactor Lab	Regression	0.42	0.00	0.40
11	Normal	Linear regression	0.44	0.00	0.20
12	Challenge	Quadratic regression	0.43	0.00	0.20
13	Easy	Simplified rules	0.80	0.20	0.20
14	Normal	Presence rules	0.91	0.60	0.00
15	Challenge	Logical Rules	0.89	0.40	0.00
16	Easy	Open-ended discovery	0.78	0.60	0.00
17	Normal	Multiple instruments	0.58	0.00	0.13
18	Challenge	Novel instruments	0.55	0.00	0.00
19	Easy	Look-up variables	0.33	0.00	0.00
20	Normal	Measure 2 variables	0.51	0.00	0.05
21	Challenge	Measure 5 variables	0.43	0.00	0.00
22	Easy	Rosetta-stone style linguistic discovery of alien language	0.40	0.40	0.20
23	Normal	Noun and verb	0.20	0.00	0.00
24	Challenge	Noun, adj., and verb	0.49	0.00	0.00
Average (Easy)	0.59	0.38	0.25	0.56	0.18
Average (Normal)	0.63	0.18	0.14	0.64	0.18
Average (Challenge)	0.63	0.18	0.10	0.50	0.15

Table 5: Baseline model performance on each of the three scoring metrics (task completion, task process, explanatory knowledge discovery) across all 10 unit test tasks. Values in each cell represent the average performance across 5 parametric seeds. Unit tests tasks are run to a maximum of 100 steps.

#	Unit Test Topic	ReACT	Plan+Execute	Hypothesizer
25	Multi-turn dialog with an agent	1.00	1.00	1.00
26	Measure an object with an instrument	0.87	0.60	0.73
27	Pick-and-place object	0.90	0.80	0.80
28	Pick-and-give object	1.00	1.00	1.00
29	Read DiscoveryFeed posts	1.00	1.00	0.90
30	Move through doors	0.58	0.20	0.25
31	Using keys with doors	0.69	0.20	0.54
32	Navigate to a specific room in a house	0.20	0.20	0.20
33	Search an environment for an object	0.80	0.80	0.60
34	Interact with a moving agent	0.60	0.20	0.53
Average (Unit Tests)	0.76	0.60	0.66	0.44

4.2 Baseline Agent Models

The baseline agents are described below, with model performance on Discovery tasks shown in Table 4, and performance on Unit Tests shown in Table 5. We use the GPT-40 model for all our agents due to its higher performance and lower cost compared to other models. For space we provide

File: olmocr/bench/sample_data/olmocr_pipeline/earnings_pg1_repeat1.md

Recently Issued Accounting Pronouncements

Recently Adopted Accounting Pronouncement

In November 2023, the Financial Accounting Standards Board, or FASB, issued a new accounting standard requiring disclosures of significant expenses in operating segments. We adopted this standard in our fiscal year 2025 annual report. Refer to Note 16 of the Notes to the Consolidated Financial Statements in Part IV, Item 15 of this Annual Report on Form 10-K for further information.

Recent Accounting Pronouncements Not Yet Adopted

In December 2023, the FASB issued a new accounting standard which includes new and updated income tax disclosures, including disaggregation of information in the rate reconciliation and income taxes paid. We expect to adopt this standard in our fiscal year 2026 annual report. We do not expect the adoption of this standard to have a material impact on our Consolidated Financial Statements other than additional disclosures.

In November 2024, the FASB issued a new accounting standard requiring disclosures of certain additional expense information on an annual and interim basis, including, among other items, the amounts of purchases of inventory, employee compensation, depreciation and intangible asset amortization included within each income statement expense caption, as applicable. We expect to adopt this standard in our fiscal year 2028 annual report. We do not expect the adoption of this standard to have a material impact on our Consolidated Financial Statements other than additional disclosures.

Note 2 - Business Combination

Termination of the Arm Share Purchase Agreement

In February 2022, NVIDIA and SoftBank Group Corp, or SoftBank, announced the termination of the Share Purchase Agreement whereby NVIDIA would have acquired Arm from SoftBank. The parties agreed to terminate it due to significant regulatory challenges preventing the completion of the transaction. We recorded an acquisition termination cost of $1.4 billion in fiscal year 2023 reflecting the write-off of the prepayment provided at signing.

Note 3 - Stock-Based Compensation

Stock-based compensation expense is associated with RSUs, PSUs, market-based PSUs, and our ESPP.

Consolidated Statements of Income include stock-based compensation expense, net of amounts capitalized into inventory and subsequently recognized to cost of revenue, as follows:

Year Ended	Jan 29, 2023	Jan 28, 2024	Jan 29, 2023
(In millions)	$138	$141	$138
Cost of revenue	$178	$141	$138
Research and development	3,423	2,532	1,892
Sales, general and administrative	1,136	876	680
Total	$4,737	$3,549	$2,710

Stock-based compensation capitalized in inventories was not significant during fiscal years 2025, 2024, and 2023.

File: olmocr/bench/sample_data/olmocr_pipeline/lincoln_letter_pg1_repeat1.md

Executive Mansion,

Washington City,

January 15th, 1864

Major General Hitchcock, Commissioner of Exchanges, is authorized and directed to offer Brigadier General Trimble, now a prisoner of war in Fort McHenry, in exchange for Major White, who is held as a prisoner at Richmond. He is also directed to send forward the offer of exchange by Henry M. Warfield, Esq. of Baltimore, under a flag of truce, and give him a pass to City Point.

Abraham Lincoln

File: olmocr/bench/sample_data/olmocr_pipeline/math_2503_04086_pg1_repeat1.md

Proof. Let $S$ be the generating set associated with $D$ as described in Proposition 2.5. By the circulant diagonalization theorem, the spectrum of $G_R(D) = \Gamma(R, S)$ is the multiset ${\lambda_g}_{g \in R}$ where

$$\lambda_g = \sum_{s \in S} \zeta_n^{\psi(gs)} = \sum_{i=1}^k \left[ \sum_{s, Rs = I_i} \zeta_n^{\psi(gs)} \right].$$

We remark that by Corollary 2.7, if $s \in R$ such that $Rs = I_i = Rx_i$ then $s$ has a unique representation of the form $s = ux_i$ where $u \in (R/\text{Ann}_R(x_i))^\times$ and $\hat{u}$ is a fixed lift of $u$ to $R^\times$. With this presentation, we can write

$$\sum_{s, Rs = I_i} \zeta_n^{\psi(gs)} = \sum_{u \in (R/\text{Ann}R(x_i))^\times} \zeta_n^{\psi(gux_i)} = \sum{u \in (R/\text{Ann}_R(x_i))^\times} \zeta_n^{\psi_xi(gu)} = c(g, R/\text{Ann}_R(x_i)).$$

Here we recall that $\psi_xi$ is the induced linear functional on $R/\text{Ann}R(x_i)$. We conclude that $\lambda_g = \sum{i=1}^k c(g, R/\text{Ann}_R(x_i)).$ \hfill $\square$

The following corollary is simple yet important for our future work on perfect state transfers on gcd-graphs.

Corollary 4.17. Suppose that $g' = ug$ for some $u \in R^\times$. Then $\lambda_g = \lambda_{g'}$.

Acknowledgements

We thank the Department of Mathematics and Computer Science at Lake Forest College for their generous financial support through an Overleaf subscription. We also thank Ján Mináč for his constant encouragement and support.

References

Reza Akhtar, Megan Boggess, Tiffany Jackson-Henderson, Isidora Jiménez, Rachel Karpman, Amanda Kinzel, and Dan Pritikin, On the unitary Cayley graph of a finite ring, Electron. J. Combin. 16 (2009), no. 1, Research Paper 117, 13 pages.
Milan Bašić, Aleksandar Ilić, and Aleksandar Stamenković, Maximal diameter of integral circulant graphs, Information and Computation 301 (2024), 105208.
Maria Chudnovsky, Michal Cizek, Logan Crew, Ján Mináč, Tung T. Nguyen, Sophie Spirkl, and Nguyễn Duy Tấn, On prime Cayley graphs, arXiv:2401.06062, to appear in Journal of Combinatorics (2024).
Thomas Honold, Characterization of finite frobenius rings, Archiv der Mathematik 76 (2001), no. 6, 406–415.
Irving Kaplansky, Elementary divisors and modules, Transactions of the American Mathematical Society 66 (1949), no. 2, 464–491.
Walter Klotz and Torsten Sander, Some properties of unitary Cayley graphs, The Electronic Journal of Combinatorics 14 (2007), no. 1, R45, 12 pages.
Erich Lamprecht, Allgemeine theorie der Gaußschen Summen in endlichen kommutativen Ringen, Mathematische Nachrichten 9 (1953), no. 3, 149–196.
Ján Mináč, Tung T Nguyen, and Nguyen Duy Tấn, Isomorphic gcd-graphs over polynomial rings, arXiv preprint arXiv:2411.01768 (2024).
______, On the gcd graphs over polynomial rings, arXiv preprint arXiv:2409.01929 (2024).

File: olmocr/bench/sample_data/olmocr_pipeline/mathfuncs_colswitch_pg1_repeat1.md

The 20 Most Important Mathematical Equations

A journey through the most elegant and influential formulas in mathematics

1. Euler’s Identity	3. The Fundamental Theorem of Calculus
( e^{i\pi} + 1 = 0 )	( \int_a^b f(x) , dx = F(b) - F(a) )
Connects five fundamental constants (e, i, π, 1, 0), revealing the profound relationship between exponential functions and trigonometry.	Establishes that differentiation and integration are inverse operations. If F is an antiderivative of f, the definite integral equals F(b) - F(a). Revolutionized mathematical problem-solving.

2. Pythagorean Theorem	4. Maxwell’s Equations
( a^2 + b^2 = c^2 )	( \nabla \cdot \mathbf{E} = \frac{\rho}{\varepsilon_0} )
In right triangles, the hypotenuse squared equals the sum of the squares of the other sides. Cornerstone of geometry with applications in navigation and architecture.	( \nabla \cdot \mathbf{B} = 0 )
	( \nabla \times \mathbf{E} = -\frac{\partial \mathbf{B}}{\partial t} )
	( \nabla \times \mathbf{B} = \mu_0 \mathbf{J} + \mu_0 \varepsilon_0 \frac{\partial \mathbf{E}}{\partial t} )
	Unified electricity and magnetism as manifestations of the same force. Describes electromagnetic field behavior, predicting waves traveling at light speed. Enabled technologies from radio to smartphones.

File: olmocr/bench/sample_data/olmocr_pipeline/mathfuncs_pg1_repeat1.md

The 20 Most Important Mathematical Equations

A journey through the most elegant and influential formulas in mathematics

1. Euler's Identity	2. Pythagorean Theorem
( e^{i\pi} + 1 = 0 )	( a^2 + b^2 = c^2 )

Connects five fundamental constants (e, i, π, 1, 0), revealing the profound relationship between exponential functions and trigonometry.

In right triangles, the hypotenuse squared equals the sum of the squares of the other sides. Cornerstone of geometry with applications in navigation and architecture.

3. The Fundamental Theorem of Calculus	4. Maxwell's Equations
( \int_{a}^{b} f(x) , dx = F(b) - F(a) )	( \nabla \cdot \mathbf{E} = \frac{Q}{\varepsilon_0} )

Establishes that differentiation and integration are inverse operations. If ( F ) is an antiderivative of ( f ), the definite integral equals ( F(b) - F(a) ). Revolutionized mathematical problem-solving.

Unified electricity and magnetism as manifestations of the same force. Describes electromagnetic field behavior, predicting waves traveling at light speed. Enabled technologies from radio to smartphones.

File: olmocr/bench/sample_data/olmocr_pipeline/mattsnotes_pg1_repeat1.md

V-February Flow

Data Components:

Code:
The-Stack-V2

CodeText:
SE, whatever we've scraped

WebText:
HQ DCLM

DATA MIXES

~85% Source Code
~10% CodeText
~5% Webtext

~85% The-stack-V2
~15% CodeText
~0% Webtext

~100% Source Code

Deepseek Coder

StarCoder 2

Arctic

File: olmocr/bench/sample_data/olmocr_pipeline/mattsnotes_pg2_repeat1.md

P1: 100% Source code
P2: 80% code
20% language

Code Data Recipe [StackCoder]

Order by Repo ✓
Call Heuristic Filters ✗
Group by Repo, lang → minhash ✓
Pack into Repo-level docs □
Select PL's □
Pack into FIM tokens ✗

✓: Eng Done
X: Eng definitely NOT done
D: so so easy

Use Preprocessed code/text, web/text

File: olmocr/bench/sample_data/olmocr_pipeline/mattsnotes_pg3_repeat1.md

ARCH + TRAINING

Pick Arch like OLMO-IB
OR replicate a 3D model
Follow standard LR flow

Eval:

Hacky nonsense for now

File: olmocr/bench/sample_data/olmocr_pipeline/multi_column_miss_pg1_repeat1.md

stakeholders has occurred in other nations, with groups and individuals refusing to risk being appropriated into the industry’s public relations ambitions. It now looks like that with vigilance, tobacco control advocates can easily foment disruption of its efforts to take its place alongside other industries—often with considerable social credit—in the hope that it might gain by association.

Tobacco industry posturing about its corporate responsibility can never hide the ugly consequences of its ongoing efforts to “work with all relevant stakeholders for the preservation of opportunities for informed adults to consume tobacco products” (translation: “we will build alliances with others who want to profit from tobacco use, to do all we can to counteract effective tobacco control”). BAT has 15.4% and Philip Morris 16.4% of the global cigarette market. With 4.9 million smokers currently dying from tobacco use each year, and the industry unblinkingly concurring that its products are addictive, this leaves BAT to argue why it should not be held to be largely accountable for the annual deaths of some 754 600 smokers, and Philip Morris some 803 600 smokers.

REFERENCES
1 British American Tobacco. Social Report. http://www.bat.com/204pp
2 Wroe D. Tobacco ad campaign angers MPs. The Age (Melbourne) 2004; May 17 http://www.theage.com.au/articles/2004/05/16/108464669771.html?oneclick=true
3 Hirschhorn N. Corporate social responsibility and the tobacco industry: hope or hype? Tobacco Control 2004;13:447–53.
4 Ethical Corporation Asia 2004. Conference website. http://www.ethicalcorp.com/asia2004/
5 Chapman S, Shatenstein S. Extreme corporate makeover: tobacco companies, corporate responsibility and the corruption of “ethics”. Globalink petition. http://petition.globalink.org/view.php?code=extreme.
6 Mackay J, Erikson M. The tobacco atlas. Geneva: World Health Organization, 2002.

INDUSTRY WATCH

Corporate social responsibility and the tobacco industry: hope or hype?

N Hirschhorn

Corporate social responsibility (CSR) emerged from a realisation among transnational corporations of the need to account for and redress their adverse impact on society: specifically, on human rights, labour practices, and the environment. Two transnational tobacco companies have recently adopted CSR: Philip Morris, and British American Tobacco. This report explains the origins and theory behind CSR; examines internal company documents from Philip Morris showing the company’s deliberations on the matter, and the company’s perspective on its own behaviour; and reflects on whether marketing tobacco is antithetical to social responsibility.

Over the past three decades increasing pressure from non-governmental organisations (NGOs), governments and the United Nations, has required transnational corporations (TNCs) to examine and redress the adverse impact their businesses have on society and the environment. Many have responded by taking up what is known as “corporate social responsibility” (CSR); only recently have two major cigarette companies followed suit: Philip Morris (PM) and British American Tobacco (BAT). This report first provides the context and development of CSR; then, from internal company documents, examines how PM came to its own version. This paper examines whether a tobacco company espousing CSR should be judged simply as a corporate entity along standards of business ethics, or as an irretrievably negative force in the realm of public health, thereby rendering CSR an oxymoron.

CORPORATE SOCIAL RESPONSIBILITY: THE CONTEXT

The term “corporate social responsibility” is in vogue at the moment but as a concept it is vague and means different things to different people. Some writers on CSR trace its American roots to the 19th century when large industries engaged in philanthropy and established great public institutions, a form of “noblesse oblige”. But the notion that corporations should be required to return more to society because of their impact on society was driven by pressures from the civil rights, peace, and environmental movements of the last half century. The unprecedented expansion of power and influence of TNCs over the past three decades has accelerated global trade and development, but also environmental damage and abuses of

Abbreviations: ASH, Action on Smoking and Health; BAT, British American Tobacco; CERES, Coalition for Environmentally Responsible Economies; CSR, corporate social responsibility; DJSI, Dow Jones Sustainability Index; GCAC, Global Corporate Affairs Council; GRI, Global Reporting Initiative; MSA, Master Settlement Agreement; NGOs, non-governmental organisations; PM, Philip Morris; TNCs, transnational corporations; UNEP, United Nations Environment Program

File: olmocr/bench/sample_data/olmocr_pipeline/olmo2-pg4_pg1_repeat1.md

Table 1 Composition of the pretraining data for OLMo 2. The OLMo 2 1124 Mix is composed of StarCoder (Li et al., 2023b; Kocetkov et al., 2022), peS2o (Soldaini and Lo, 2023), web text from DCLM (Li et al., 2024) and Wiki come from Dolma 1.7 (Soldaini et al., 2024). arXiv comes from Red-Pajama (Together AI, 2023), while OpenWebMath (Paster et al., 2023) and Algebraic Stack come from ProofPile II (Azerbayev et al., 2023).

Source	Type	Tokens	Words	Bytes	Docs
DCLM-Baseline	Web pages	3.71T	3.32T	21.32T	2.95B
StarCoder	Code	83.0B	70.0B	459B	78.7M
filtered version from OLMoE Mix	Academic papers	58.6B	51.1B	413B	38.8M
peS2o from Dolma 1.7	Math web pages	12.2B	11.1B	47.2B	2.89M
arXiv	Math proofs code	11.8B	10.8B	44.0B	2.83M
OpenWebMath	Encyclopedic	3.7B	3.16B	16.2B	6.17M
Wikipedia & Wikibooks from Dolma 1.7
Total		3.90T	3.48T	22.38T	3.08B

2.1.1 Pretraining data: OLMo 2 Mix 1124

The mix used for this stage is shown in Table 1. It consists of approximately 3.9 trillion tokens, with over 95% derived from web data. We refer to this set as OLMo 2 Mix 1124. This is the same pretraining data used in OLMoE (Muennighoff et al., 2024).

We combine data from DCLM (Li et al., 2024) and Dolma 1.7 (Soldaini et al., 2024). From DCLM, we use the “baseline 1.0” mix. From Dolma, we use the arXiv (Together AI, 2023), OpenWebMath (Paster et al., 2023), Algebraic Stack, peS2o (Soldaini and Lo, 2023), and Wikipedia subsets. arXiv, OpenWebMath, and Algebraic Stack were originally part of ProofPile II (Azerbayev et al., 2023).

Finally, we include code from StarCoder (Li et al., 2023b), which is derived from permissively-licensed repositories from GitHub (Kocetkov et al., 2022). In an attempt to include higher quality code, we remove any document from a repository with fewer than 2 stars on GitHub. Further, through manual inspection of this source, we found it to contain documents encoded in binary format or containing mostly numerical content; to remove them, we discarded documents whose most frequent word constitutes over 30% of the document, or whose top-2 most frequent words constitute over 50% of the document. To mitigate possible training loss spikes, we remove documents with repeated sequences of 32 or more n-grams. We report details and show effectiveness of this intervention in Section §3.1.

2.1.2 Mid-training data: Dolmino Mix 1124

After the initial pretraining stage on mostly web data, we further train with a mixture of web data that has been more restrictively filtered for quality and a collection of domain-specific high quality data, much of which is synthetic. The purpose of this mixture is to imbue the model with math-centric skills and provide focused exposure to STEM references and high quality text. We generate several variants of this mixture, with varying sizes, but generally refer to this mixture as Dolmino Mix 1124. The base sources from which Dolmino Mix 1124 is subsampled are described in Table 2. We refer the reader to Section §4 for a deep dive detailing our processes for experimenting and curating data for this mix.

File: olmocr/bench/sample_data/olmocr_pipeline/openstax_caculus_pg_273_pg1_repeat1.md

3.4 EXERCISES

For the following exercises, the given functions represent the position of a particle traveling along a horizontal line.

a. Find the velocity and acceleration functions.

b. Determine the time intervals when the object is slowing down or speeding up.

( s(t) = 2t^3 - 3t^2 - 12t + 8 )
( s(t) = 2t^3 - 15t^2 + 36t - 10 )
( s(t) = \frac{t}{1 + t^2} )
A rocket is fired vertically upward from the ground. The distance ( s ) in feet that the rocket travels from the ground after ( t ) seconds is given by ( s(t) = -16t^2 + 560t ).

a. Find the velocity of the rocket 3 seconds after being fired.

b. Find the acceleration of the rocket 3 seconds after being fired.

A ball is thrown downward with a speed of 8 ft/s from the top of a 64-foot-tall building. After ( t ) seconds, its height above the ground is given by ( s(t) = -16t^2 - 8t + 64 ).

a. Determine how long it takes for the ball to hit the ground.

b. Determine the velocity of the ball when it hits the ground.

The position function ( s(t) = t^2 - 3t - 4 ) represents the position of the back of a car backing out of a driveway and then driving in a straight line, where ( s ) is in feet and ( t ) is in seconds. In this case, ( s(t) = 0 ) represents the time at which the back of the car is at the garage door, so ( s(0) = -4 ) is the starting position of the car, 4 feet inside the garage.

a. Determine the velocity of the car when ( s(t) = 0 ).

b. Determine the velocity of the car when ( s(t) = 14 ).

The position of a hummingbird flying along a straight line in ( t ) seconds is given by ( s(t) = 3t^3 - 7t ) meters.

a. Determine the velocity of the bird at ( t = 1 ) sec.

b. Determine the acceleration of the bird at ( t = 1 ) sec.

c. Determine the acceleration of the bird when the velocity equals 0.

A potato is launched vertically upward with an initial velocity of 100 ft/s from a potato gun at the top of an 85-foot-tall building. The distance in feet that the potato travels from the ground after ( t ) seconds is given by ( s(t) = -16t^2 + 100t + 85 ).

a. Find the velocity of the potato after 0.5 s and 5.75 s.

b. Find the speed of the potato at 0.5 s and 5.75 s.

c. Determine when the potato reaches its maximum height.

d. Find the acceleration of the potato at 0.5 s and 1.5 s.

e. Determine how long the potato is in the air.

f. Determine the velocity of the potato upon hitting the ground.

The position function ( s(t) = t^3 - 8t ) gives the position in miles of a freight train where east is the positive direction and ( t ) is measured in hours.

a. Determine the direction the train is traveling when ( s(t) = 0 ).

b. Determine the direction the train is traveling when ( s(t) = 0 ).

c. Determine the time intervals when the train is slowing down or speeding up.

The following graph shows the position ( y = s(t) ) of an object moving along a straight line.

a. Use the graph of the position function to determine the time intervals when the velocity is positive, negative, or zero.

b. Sketch the graph of the velocity function.

c. Use the graph of the velocity function to determine the time intervals when the acceleration is positive, negative, or zero.

d. Determine the time intervals when the object is speeding up or slowing down.

File: olmocr/bench/sample_data/olmocr_pipeline/small_page_size_pg1_repeat1.md

any—was very trifling. Since the use of bones has, however, become general, the turnip crop has been, in many instances, ten-fold, and in few less than four or five-fold its former bulk. All the succeeding crops of grain and seeds have been amazingly increased, and, upon the four or five-shift system, there is no doubt the land will go on progressively improving, requiring a less quantity of bones annually, from its increased fertility and power."

On light loams, the returns to the Doncaster Committee give bones a preference to farm-yard dung. And we learn that, upon the calcareous soil of the Yorkshire Wolds, heavy crops of turnips have been raised from 16 bushels per acre of bones, while in the same field, and under similar circumstances, but manured from the farm-yard at the rate of from 8 to 10 tons per acre, the turnips have been of the most inferior description.

On peat soils, if previously drained and laid dry, their advantages are reported to be so striking, that from fifteen to twenty bushels of dust per acre, drilled, have been also found to very far surpass the ordinary dressing of stable-dung, and even of lime and pigeons'-dung.

On gravels, the reports are meagre and contradictory, though perhaps reconcilable in principle, as it has been justly observed, that "a gravelly soil may embrace every variety of texture and quality, from the light dry sand to the water-logged yellow clay—preserving in each the necessary admixture of stones and grit." To wet gravel, their application has been found decidedly unfavourable.

ANALYSIS.

An examination of the component parts of soils, and of the power of bones, when applied to them as manure, would go far to explain the irregularity of their different effects upon various kinds of soil. Bone is known to consist of about equal parts of earthy and animal matter; the former chiefly composed of gypsum—which is of so indestructible a nature as to have been termed, by early chemists, the "earth of bones"—and a small portion of carbonate of lime; from which we may conclude that probably half the weight of bones is in the greater part consumed by plants as direct nourishment in their state of growth, and that the remainder is more gradually absorbed by the soil, as well also as by the plants; for lime, though in small amount, is always present, in greater or less quantity, in all vegetable substances.

"The quantity of earthy matter varies according to the age of the animal; and, in like manner, the quantity of animal matter varies also in proportion to the condition of the animal. In the best kinds of bones for manure, viz., those from fat young animals, perhaps the following proportions may give an approximation to the relative quantities of each in 100 parts:

| Earthy and saline matter | 40 |
| Cartilage and jelly | 40 |
| Fatty matter | 20 |

The soft parts thus form, in the best bone, about sixty, and upon an average, perhaps, amount to fifty per cent., which are almost entirely constituted of the same elements of plants, and all of them, sooner or later, liable to be dissolved and absorbed by the roots. The cartilage, indeed, when the bones have been buried in a dry situation, is very indestructible; but when exposed to the action of air, water, soil, and vegetation, will probably pass into the state of jelly, and be dissolved, or otherwise decomposed,

Doncaster Report, p. 8.

File: olmocr/bench/sample_data/olmocr_pipeline/test-graphical-text_pg1_repeat1.md

THE POWER OF STORYTELLING
FOR LEADERS
ดร.วิทย์ สิทธิเวคิน

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/ff0f0b22c55d8b90dd77d153f48e144fc9db_pg2_pg1_repeat1.md

Lassa Fever in Post-Conflict Sierra Leone

Jeffrey G. Shaffer1, Donald S. Grant2, John S. Schieffelin3, Matt L. Boisen4,5, Augustine Goba2,9, Jessica N. Hartnett4, Danielle C. Levy4, Rachael E. Yenni4, Lina M. Moses4,6, Mohammed Fullah2, Mambo Momoh2, Mbalu Fonnie2, Richard Fonnie2, Lansana Kanneh2, Veronica J. Koroma2, Kande Kargbo2, Darin Ottomassathien9, Ivana J. Muncy5, Abigail B. Jones5, Megan M. Illic7, Peter C. Kulakosky8, Allyson M. Haislip4, Christopher M. Bishop4, Deborah H. Elliot3, Bethany L. Brown5, Hu Zhu9, Kathryn M. Hastie10, Kristian G. Andersen11,12, Stephen K. Gire11,12, Shervin Tabrizi11,12, Ridhi Tariyal12, Mathew Stremlau11,12, Alex Matschiner7, Darryl B. Sampey7, Jennifer S. Spence6, Robert W. Cross4,13, Joan B. Geisbert13, Onikepe A. Folarin14, Christian T. Happi14, Kelly R. Pitts5, F. Jon Geske5, Thomas W. Geisbert13, Erica Ollmann Saphire9,15, James E. Robinson3, Russell B. Wilson8, Pardis C. Sabeti10,11,16, Lee A. Henderson8, S. Humarr Khan25, Daniel G. Bausch61, Luis M. Branco8,17, Robert F. Garry4,12,17*, the Viral Hemorrhagic Fever Consortium†

1 Department of Biostatistics and Bioinformatics, Tulane School of Public Health and Tropical Medicine, New Orleans, Louisiana, United States of America, 2 Lassa Fever Program, Kenema Government Hospital, Kenema, Sierra Leone, 3 Sections of Infectious Disease, Departments of Pediatrics and Internal Medicine, School of Medicine, Tulane University, New Orleans, Louisiana, United States of America, 4 Department of Microbiology and Immunology, Tulane University, New Orleans, Louisiana, United States of America, 5 Corgenix, Inc., Broomfield, Colorado, United States of America, 6 Department of Tropical Medicine, Tulane School of Public Health and Tropical Medicine, New Orleans, Louisiana, United States of America, 7 Biofactura Inc., Rockville, Maryland, United States of America, 8 Autoimmune Technologies, LLC, New Orleans, Louisiana, United States of America, 9 Vybion Inc., Ithaca, New York, United States of America, 10 Department of Immunology and Microbial Science, The Scripps Research Institute, La Jolla, California, United States of America, 11 FAS Center for Systems Biology, Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, Massachusetts, United States of America, 12 Broad Institute, Cambridge, Massachusetts, United States of America, 13 Department of Microbiology and Immunology, University of Texas Medical Branch at Galveston, Galveston, Texas, United States of America, 14 Department of Biological Sciences, College of Natural Sciences, Redeemers University, Redemption City, Ogun State, Nigeria, 15 The Skaggs Institute for Chemical Biology, The Scripps Research Institute, La Jolla, California, United States of America, 16 Department of Immunology and Infectious Disease, Harvard School of Public Health, Boston, Massachusetts, United States of America, 17 Zalgen Labs, LLC, Germantown, Maryland, United States of America

Abstract

Background: Lassa fever (LF), an often-fatal hemorrhagic disease caused by Lassa virus (LASV), is a major public health threat in West Africa. When the violent civil conflict in Sierra Leone (1991 to 2002) ended, an international consortium assisted in restoration of the LF program at Kenema Government Hospital (KGH) in an area with the world’s highest incidence of the disease.

Methodology/Principal Findings: Clinical and laboratory records of patients presenting to the KGH Lassa Ward in the post-conflict period were organized electronically. Recombinant antigen-based LF immunoassays were used to assess LASV antigenemia and LASV-specific antibodies in patients who met criteria for suspected LF. KGH has been reestablished as a center for LF treatment and research, with over 500 suspected cases now presenting yearly. Higher case fatality rates (CFRs) in LF patients were observed compared to studies conducted prior to the civil conflict. Different criteria for defining LF stages and differences in sensitivity of assays likely account for these differences. The highest incidence of LF in Sierra Leone was observed during the dry season. LF cases were observed in ten of Sierra Leone’s thirteen districts, with numerous cases from outside the traditional endemic zone. Deaths in patients presenting with LASV antigenemia were skewed towards individuals less than 29 years of age. Women self-reporting as pregnant were significantly overrepresented among LASV antigenemic patients. The CFR of ribavirin-treated patients presenting early in acute infection was lower than in untreated subjects.

Conclusions/Significance: Lassa fever remains a major public health threat in Sierra Leone. Outreach activities should expand because LF may be more widespread in Sierra Leone than previously recognized. Enhanced case finding to ensure rapid diagnosis and treatment is imperative to reduce mortality. Even with ribavirin treatment, there was a high rate of fatalities underscoring the need to develop more effective and/or supplemental treatments for LF.

Citation: Shaffer JG, Grant DS, Schieffelin JS, Boisen ML, Goba A, et al. (2014) Lassa Fever in Post-Conflict Sierra Leone. PLoS Negl Trop Dis 8(3): e2748. doi:10.1371/journal.pntd.0002748

Editor: Brian Bird, Centers for Disease Control and Prevention, United States of America

Received August 10, 2013; Accepted February 3, 2014; Published March 20, 2014

Copyright: © 2014 Shaffer et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work was supported by National Institute of Allergy and Infectious Diseases grants/contracts AI067188, A082119, AI2008031, AI2009061, AI104216, AI104621, P20GM103501, HHSN272200900018C, HHSN272200900049C, HHSN272201000022C, and an Investigators in the Pathogenesis of Infectious Disease award from the Burroughs Wellcome Fund. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/ff1fc6a205ad039139ce566851b6b260c929_pg1_pg1_repeat1.md

RTG Degradation Primer and Application to MMRTG

Nuclear and Emerging Technology for Space (NETS) 2015
February 23-26, 2015
Abstract 5107

Presenting Author: Tom Hammel, Teledyne Energy Systems
Co-Authors: Russell Bennett, Teledyne Energy Systems
Robert Sievers, Teledyne Energy Systems
Bill Otting, Aerojet Rocketdyne

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/ff3d6e051903fe5ca9bc172ece14964c5632_pg1_pg1_repeat1.md

[Binary file]

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/ff4f7dad78081cff727d19ab51c181d4a661_pg1_pg1_repeat1.md

Molecular markers of breast cancer metastasis
Weigelt, B.

Citation for published version (APA):
Weigelt, B. (2005). Molecular markers of breast cancer metastasis

General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).

Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: http://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/ff518b1240a66978f22035528ccb029450b5_pg2_pg1_repeat1.md

[Binary file]

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/ffaac214730d2b8c2ec842e3618ccb9c4259_pg1_pg1_repeat1.md

[Empty file]

File: olmocr/bench/sample_data/olmocr_pipeline/headers_footers/fff590bed29a2854ac1f874dad5752ede1aa_pg1_pg1_repeat1.md

User’s Manual

Model 475
DSP Gaussmeter

Lake Shore Cryotronics, Inc.
575 McCorkle Blvd.
Westerville, Ohio 43082-8888 USA

E-mail Addresses:
[email protected]
[email protected]

Visit Our Website At:
www.lakeshore.com

Fax: (614) 891-1392
Telephone: (614) 891-2243

Methods and apparatus disclosed and described herein have been developed solely on company funds of Lake Shore Cryotronics, Inc. No government or other contractual support or relationship whatsoever has existed which in any way affects or mitigates proprietary rights of Lake Shore Cryotronics, Inc. in these developments. Methods and apparatus disclosed herein may be subject to U.S. Patents existing or applied for. Lake Shore Cryotronics, Inc. reserves the right to add, improve, modify, or withdraw functions, design modifications, or products at any time without notice. Lake Shore shall not be liable for errors contained herein or for incidental or consequential damages in connection with furnishing, performance, or use of this material.

File: olmocr/train/README.md

olmOCR Training Guide

This guide provides comprehensive instructions for training olmOCR models, including what you need to reproduce https://huggingface.co/allenai/olmOCR-2-7B-1025-FP8 on your own hardware.

Environment setup

The first step is to setup your python/conda environment, and set things up the same way as for running olmocr.

Then, add in some extra training requirements:

bash

pip install .[train]
pip install transformers==4.52.4
pip install flash-attn>=2.8.0.post2 --no-build-isolation

Dataset Format

The training data should be organized as pairs of PDF files and their corresponding markdown annotations:

Important: Each PDF needs to be a single page only!

text

data/
├── document1.pdf
├── document1.md
├── document2.pdf
├── document2.md
└── ...

Each markdown file should contain:

YAML front matter with metadata
The extracted text content

Example markdown format:

markdown

---
primary_language: en
is_rotation_valid: True
rotation_correction: 0
is_table: False
is_diagram: False
---
Document text goes here...

The easiest way to grab a lot of files in this format is to use prepare_olmocrmix.py which will automatically download and prepare
olmOCR-mix-1025 for your environment.

bash

# Caution, requires ~200GB of disk space

# You can pick a specific split and subset to download, or just run all these commands in order to get everything
python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 00_documents --split train                       
python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 00_documents --split eval

python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 01_books --split train                       
python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 01_books --split eval

python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 02_loc_transcripts --split train                       
python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 02_loc_transcripts --split eval

python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 03_national_archives --split train                       
python -m olmocr.data.prepare_olmocrmix --dataset-path allenai/olmOCR-mix-1025 --destination ~/olmOCR-mix-1025-extracted --subset 03_national_archives --split eval

The second easiest way to get training data is to use olmOCR to convert a bunch of documents and then adjust the results to better suit you. Once you have an olmOCR workspace with some converted PDFs,
you can use the olmocr.data.prepare_workspace script to extract those into the proper format.

bash

python -m olmocr.pipeline ./localworkspace --pdfs /home/username/pdfs/*.pdf
python -m olmocr.data.prepare_workspace ./localworkspace ./localtrainingdata

./localtrainingdata will now have data in the proper format, each PDF split into a single page, etc, that you can use for fine tuning.

Setup your config

Reference configs for released models:

olmOCR-7B-0725-FP8 was trained with qwen25_vl_olmocrv2_2epoch.yaml
olmOCR-7B-0825-FP8 was trained with qwen25_vl_olmocrv3_rotation_1epoch.yaml
olmOCR-2-7B-1025-FP8 was trained with qwen25_vl_olmocrv4_rotation_1epoch_mix_1025_filtered.yaml

These are setup to train on a single B200 GPU, and training will take around 24-48 hours (~$300 if renting). A single epoch on our full model is ~270,000 page, so it's quite a big endeavour.

You may need to adjust the paths in the dataset section of the config to match where you downloaded and extracted the training mix. And also set the output_dir to a location where you would like to save checkpoints.

Finetuning Configs

If you would like to finetune on a smaller amount of local data, with a LoRA adapter, try the following

Basic finetuning config: qwen25_vl_olmocrv4_finetuning.yaml

You'll still need to adjust your paths accordingly.

Launch training job

bash

python -m olmocr.train.train --config olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_mix_1025_filtered.yaml

Prepare Checkpoints and Quantize

After training is done, you will need to call prepare_checkpoint.py to take the saved checkpoints
and get them ready for use with VLLM.

bash

python -m olmocr.train.prepare_olmocr_checkpoint [source dir]/checkpoint-xxxx [destination]

If you just finetuned a LoRA adapter, you should run the prepare_checkpoint.py script as above, it will merge the weights into a full model and adjust the configs so you can easily run the model.

FP8 Quantization

The training process operates in BF16 weights natively, even if you are just training a LoRA adapter.
We recommend doing an FP8 quantization step, whose performance is solidly in the error bars of the raw
bfloat16 model, but uses less memory and inferences around 12% faster.

bash

python -m olmocr.train.compress_checkpoint --config olmocr/train/quantization_configs/qwen2_5vl_w8a8_fp8.yaml [destination] [destination-FP8]

GRPO Training

olmOCR-2-7B-1025-FP8 adds an additional training step with GRPO RL based training
occuring on a synthetic version of olmOCR-bench.

olmOCR-synthmix-1025 was created by having Claude Sonnet take real PDF documents,
then convert them into HTML templates. Those HTML templates were then rendered, and converted into synthetic olmOCR-bench style benchmarks.
We then ran a GPRO based training process with a reward based on the benchmark score on this synthetic benchmark.

You can use the following command to download the synthetic dataset:

bash

hf download allenai/olmOCR-synthmix-1025 --repo-type dataset --local-dir olmOCR-synthmix-1025

This was generated using mine_html_templates.py, and is in the same format as other olmOCR-bench test cases. This means you can measure performance against these cases directly in the same way as running any other olmOCR-bench test suite.

The following scripts show how to start training, which is performed on an 8xH100 GPU node. One GPU is dedicated to running VLLM, while the other 7 are used to run training. At the moment, this code is quite specialized to our cluster here, but we hope to make it easier to run elsewhere in future releases.

bash

./scripts/train/grpotrainer-beaker-multi-gpu-augusta.sh --num-gpus 8      --model_name s3://ai2-oe-data/jakep/olmocr/qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_mix1025_more_rotation-8372 --train_bench_data_folder /data/jakep/grpo_data_mixes/olmocr-synthmix-1025-v2-rotate10p/bench_data --reward_bench 1.0 --reward_front_matter 1.0 --reward_eos 1.0 --beta 0.01 --name promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_finalrun1 --seed 1 --gradient_accumulation_steps 28 --learning_rate 2e-6 --preemptible

./scripts/train/grpotrainer-beaker-multi-gpu-augusta.sh --num-gpus 8      --model_name s3://ai2-oe-data/jakep/olmocr/qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_mix1025_more_rotation-8372 --train_bench_data_folder /data/jakep/grpo_data_mixes/olmocr-synthmix-1025-v2-rotate10p/bench_data --reward_bench 1.0 --reward_front_matter 1.0 --reward_eos 1.0 --beta 0.01 --name promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_importanceseq_finalrun1 --seed 1 --importance_sampling_level sequence --gradient_accumulation_steps 28 --learning_rate 2e-6 --preemptible

6 seeds were run, 3 with importance sampling level=sequence, and 3 with importance sampling level=token, and then merged into a final checkpoint. Souping can be done by passing more arguments to the prepare_checkpoint script.

bash

# Final souping command for olmocr-2-7b-1025
python -m olmocr.train.prepare_checkpoint s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_finalrun1-multigpu-01K60YDRKCJY82TKF0FP6WE4VA/checkpoint-306/ s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_finalrun2-multigpu-01K60YGB5Y2G15BG8CX4H1QW23/checkpoint-306/ s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_finalrun3-multigpu-01K60YGM2QEKJK9FC94JJG5YDP/checkpoint-306/ s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_importanceseq_finalrun3-multigpu-01K60YJBGC3AR7STTNH23BWH8A/checkpoint-306/ s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_importanceseq_finalrun2-multigpu-01K60YJ315K1GYCPN8VADTN7C3/checkpoint-306/ s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_importanceseq_finalrun1-multigpu-01K60YHSHCNS9RZWSF9E56J9FB/checkpoint-306/ s3://ai2-oe-data/jakep/olmocr-grpo-checkpoints/promptv4_mix1025_more_rotation_multigpu_v1_beta_01_lr2e-6_frontmatter1_0_eos_28gen_synthmix-1025_rotate10p_soupersoup

Notes for AI2

If you are a collaborator of AI2, you can use the following scripts to run training and inference

bash

# Run training using Beaker
scripts/train/newtrainer-beaker.sh --config [config file]

# Prepare checkpoint from an interactive session with WEKA
python -m olmocr.train.prepare_olmocr_checkpoint [source] [destination]

# Compress the prepared model checkpoint to FP8
scripts/train/compress_model.sh <recipe_path> <input_model_path> <output_model_path>[--calibration-pdfs PATTERN]

# Run olmOCR bench
scripts/run_benchmark.sh --model [destination]

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day2_capped.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day2-capped

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

evaluation_strategy: steps
eval_steps: 500
save_strategy: steps
save_steps: 500
save_total_limit: 5
load_best_model_at_end: true
metric_for_best_model: eval_processed_00_documents_eval_s2pdf_loss
greater_is_better: false

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_json.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-json

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

evaluation_strategy: steps
eval_steps: 500
save_strategy: steps
save_steps: 500
save_total_limit: 5
load_best_model_at_end: false # Needs to be false because it has a problem restoring checkpoints for some reason
metric_for_best_model: eval_processed_00_documents_eval_s2pdf_loss
greater_is_better: false

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_json_1280.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-1280

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_json_1280_noanchor.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-1280-noanchor

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_json_1600.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-1600

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1600
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_json_lr5e-5.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-json-lr5e-5

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 5e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_json_wsd.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-json-wsd

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: warmup_stable_decay
lr_scheduler_kwargs:
num_decay_steps: 2000
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_yaml_1280_noanchor.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-yaml-1280-noanchor

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_yaml_1280_noanchor_128batch.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-yaml-1280-noanchor-128batch

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 128

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 5e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_yaml_1280_noanchor_latexnormalize.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-yaml-1280-noanchor-latexnormalize

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: LatexBracketNormalizer
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_day3_yaml_1280_noanchor_newprompt.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-day3-yaml-1280-noanchor-newprompt

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_fresh_prompt.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default-fresh-prompt

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: NewYamlFinetuningPromptWithAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_fresh_prompt_no_doc_anchoring.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default-fresh-prompt-no-anchor

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: NewYamlFinetuningPromptWithAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_grad_acc32.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default-gradacc_32

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_image_1280.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default_image1280

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_image_1280_no_doc_anchor.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default_image1280_no_doc_anchor

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_image_1600.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default_image1600

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1600
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_image_1600_no_doc_anchor.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default_image1600_no_doc_anchor

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1600
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_json.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default-json

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 6000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_b100_x1_default_no_doc_anchor.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-finetune-default_no_doc_anchor

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

gradient_checkpointing: False

Learning rate

learning_rate: 1e-6
lr_scheduler_type: cosine
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1288_soupfull0.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-1288-soupfull0

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 100
data_seed: 101

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1288_soupfull1.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-1288-soupfull1

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 200
data_seed: 201

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1288_soupfull2.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-1288-soupfull2

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1epoch.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_1epoch_customtrainer

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1epoch_muon2e-5.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_1epoch_muon_2e-5

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: muon
weight_decay: 0.01
max_grad_norm: 1.0

Muon optimizer specific settings

muon_momentum: 0.95
muon_lr_multiplier_head: 4.4 # 0.22 / 0.05 = 4.4
muon_lr_multiplier_embed: 12.0 # 0.6 / 0.05 = 12.0
muon_lr_multiplier_scalar: 0.8 # 0.04 / 0.05 = 0.8

Adam parameters for non-muon groups

adam_beta1: 0.8
adam_beta2: 0.95
adam_epsilon: 1e-10

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1epoch_muon2e-6.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_1epoch_muon_2e-6

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-6
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: muon
weight_decay: 0.01
max_grad_norm: 1.0

Muon optimizer specific settings

muon_momentum: 0.95
muon_lr_multiplier_head: 4.4 # 0.22 / 0.05 = 4.4
muon_lr_multiplier_embed: 12.0 # 0.6 / 0.05 = 12.0
muon_lr_multiplier_scalar: 0.8 # 0.04 / 0.05 = 0.8

Adam parameters for non-muon groups

adam_beta1: 0.8
adam_beta2: 0.95
adam_epsilon: 1e-10

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_1epoch_muon6e-6.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_1epoch_muon_6e-6

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 6e-6
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: muon
weight_decay: 0.01
max_grad_norm: 1.0

Muon optimizer specific settings

muon_momentum: 0.95
muon_lr_multiplier_head: 4.4 # 0.22 / 0.05 = 4.4
muon_lr_multiplier_embed: 12.0 # 0.6 / 0.05 = 12.0
muon_lr_multiplier_scalar: 0.8 # 0.04 / 0.05 = 0.8

Adam parameters for non-muon groups

adam_beta1: 0.8
adam_beta2: 0.95
adam_epsilon: 1e-10

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_2epoch.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_2epoch

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 2.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_2epoch_promptfirst.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_2epoch_promptfirst

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 2.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_soup0.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_soup0

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 0.5

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 100
data_seed: 101

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_soup1.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_soup1

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 0.5

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 200
data_seed: 201

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_soup2.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2_soup2

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 0.5

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_tokflip.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-tokflip

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.0001
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_tokflip1k.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-tokflip1k

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.001
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_tokflip3k.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-tokflip3k

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.003
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_tokflip500.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-tokflip500

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.005
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen25_vl_olmocrv2_tokflip_2ep.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv2-tokflip-2ep

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.0001
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 2.0

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen2_vl_b100_x1_day3_json.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2-vl-7b-finetune-day3-json

Model configuration

model:
name: Qwen/Qwen2-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1024
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: 3000
- name: FinetuningPrompt
- name: JSONOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.2.0/qwen2_vl_b100_x1_day3_yaml.yaml

Example OlmOCR Training Configuration

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2-vl-7b-finetune-day3-yaml

Model configuration

model:
name: Qwen/Qwen2-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1280
- name: StaticLengthDocumentAnchoring
target_anchor_text_len: -1
- name: FinetuningPrompt
- name: FrontMatterOutputFormat
- name: InstructUserMessages
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /weka/oe-data-default/jakep/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_1epoch.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_1epoch_prompt_first

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: PDFRenderer
target_longest_image_dim: 1288
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_1epoch.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_1epoch_prompt_first_rotation

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_2epoch.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_2epoch_prompt_first_rotation_fixed

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 2

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_2epoch_resumable.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_2epoch_prompt_first_rotation_fixed_resumable

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 2

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_augment_1epoch.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_1epoch_prompt_first_rotation_augments

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: AugraphyBasicAugmentations
probability: 0.25
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_augment_1epoch_resumable.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_1epoch_prompt_first_rotation_augments_resumable

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: AugraphyBasicAugmentations
probability: 0.25
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_augment_2epoch_resumable.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_2epoch_prompt_first_rotation_augments_resumable

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: AugraphyBasicAugmentations
probability: 0.25
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 2

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_localtest.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_1epoch_prompt_first_rotation

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /home/ubuntu/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: AugraphyBasicAugmentations
probability: 1.0
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /home/ubuntu/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /home/ubuntu/olmOCR-mix-0225/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /home/ubuntu/olmOCR-mix-0225/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /home/ubuntu/olmocr-trainer
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.3.0/qwen25_vl_olmocrv3_rotation_tokflip_1epoch.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv3_1epoch_prompt_first_rotation_tokflip

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0225/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.0001
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0225/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_finetuning.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-finetuning
run_name: qwen25_vl_olmocrv4_finetuning-demo

Model configuration

model:
name: allenai/olmOCR-2-7B-1025
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: true
lora_rank: 8
lora_alpha: 32
lora_dropout: 0.1
lora_target_modules:
- q_proj
- v_proj
- k_proj
- o_proj

Dataset configuration

dataset:

train:
- name: finetuning_data
root_dir: /root/test-berkshire-data/train
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.02
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"

eval:
- name: eval_finetuning_data
root_dir: /root/test-berkshire-data/test
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /root/olmocr-trainer/finetuning
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_baseline

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0825/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0825/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /data/olmOCR-mix-0825/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /data/olmOCR-mix-0825/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_filtering.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_filtering

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0825/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0825/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_filtering_mix_0925_more_rotation.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_filtering_mix0925

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0925/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.01
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0925/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline
- name: processed_02_loc_transcripts
root_dir: /data/olmOCR-mix-0925/processed_02_loc_transcripts/
pipeline: *basic_pipeline
- name: processed_03_national_archives
root_dir: /data/olmOCR-mix-0925/processed_03_national_archives/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /data/olmOCR-mix-0925/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /data/olmOCR-mix-0925/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline
- name: processed_02_loc_transcripts_eval
root_dir: /data/olmOCR-mix-0925/processed_02_loc_transcripts_eval/
pipeline: *basic_pipeline
- name: processed_03_national_archives_eval
root_dir: /data/olmOCR-mix-0925/processed_03_national_archives_eval/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_filtering_mix_0925_preempt.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_filtering_mix0925_preempt

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0925/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0925/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline
- name: processed_02_loc_transcripts
root_dir: /data/olmOCR-mix-0925/processed_02_loc_transcripts/
pipeline: *basic_pipeline
- name: processed_03_national_archives
root_dir: /data/olmOCR-mix-0925/processed_03_national_archives/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_filtering_noise.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_filtering_noise

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0825/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: RandomTokenFlipper
token_flip_rate: 0.0001
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0825/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_filtering_preempt.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_filtering_preempt

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0825/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0825/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_localtest.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_baseline

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /home/ubuntu/olmOCR-mix-0825/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: ReformatLatexBoldItalic
- name: DatasetTextRuleFilter
# - name: FilterOutRotatedDocuments
- name: PDFRenderer
target_longest_image_dim: 1288
# - name: RotationAugmentation
# probability: 0.002
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /home/ubuntu/olmOCR-mix-0825/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /home/ubuntu/olmOCR-mix-0825/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /home/ubuntu/olmOCR-mix-0825/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /tmp/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_mix_0925_more_rotation.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_mix0925_more_rotation

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-0925/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.01
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-0925/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline
- name: processed_02_loc_transcripts
root_dir: /data/olmOCR-mix-0925/processed_02_loc_transcripts/
pipeline: *basic_pipeline
- name: processed_03_national_archives
root_dir: /data/olmOCR-mix-0925/processed_03_national_archives/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-data-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

File: olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_rotation_1epoch_mix_1025.yaml

Example OlmOCR Training Configuration with Torch Compile

Project metadata

project_name: olmocr-qwen-vl-training
run_name: qwen2.5-vl-7b-olmocrv4_1epoch_promptv4_mix1025_more_rotation

Model configuration

model:
name: Qwen/Qwen2.5-VL-7B-Instruct
trust_remote_code: true
torch_dtype: bfloat16
use_flash_attention: true
attn_implementation: flash_attention_2

LoRA settings (disabled by default)

use_lora: false

lora_rank: 8

lora_alpha: 32

lora_dropout: 0.1

lora_target_modules:

- q_proj

- v_proj

- k_proj

- o_proj

Dataset configuration

dataset:

train:
- name: processed_01_books_train_iabooks
root_dir: /data/olmOCR-mix-1025/processed_01_books_train_iabooks/
pipeline: &basic_pipeline
- name: FrontMatterParser
front_matter_class: PageResponse
- name: FilterOutRotatedDocuments
- name: ReformatLatexBoldItalic
- name: PDFRenderer
target_longest_image_dim: 1288
- name: RotationAugmentation
probability: 0.02
- name: NewYamlFinetuningPromptWithNoAnchoring
- name: FrontMatterOutputFormat
- name: InstructUserMessages
prompt_first: true
- name: Tokenizer
masking_index: -100
end_of_message_token: "<|im_end|>"
- name: processed_00_documents_train_s2pdf
root_dir: /data/olmOCR-mix-1025/processed_00_documents_train_s2pdf/
pipeline: *basic_pipeline
- name: processed_02_loc_transcripts
root_dir: /data/olmOCR-mix-1025/processed_02_loc_transcripts/
pipeline: *basic_pipeline
- name: processed_03_national_archives
root_dir: /data/olmOCR-mix-1025/processed_03_national_archives/
pipeline: *basic_pipeline

eval:
- name: processed_00_documents_eval_s2pdf
root_dir: /data/olmOCR-mix-1025/processed_00_documents_eval_s2pdf/
pipeline: *basic_pipeline
- name: processed_01_books_eval_iabooks
root_dir: /data/olmOCR-mix-1025/processed_01_books_eval_iabooks/
pipeline: *basic_pipeline
- name: processed_02_loc_transcripts_eval
root_dir: /data/olmOCR-mix-1025/processed_02_loc_transcripts_eval/
pipeline: *basic_pipeline
- name: processed_03_national_archives_eval
root_dir: /data/olmOCR-mix-1025/processed_03_national_archives_eval/
pipeline: *basic_pipeline

Training configuration

training:
output_dir: /weka/oe-training-default/jakep/olmocr-trainer/
num_train_epochs: 1

Batch size and accumulation

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 32

gradient_checkpointing: False

collator_max_token_len: 8192

Learning rate

learning_rate: 2e-5
lr_scheduler_type: linear
warmup_ratio: 0.1

Optimization

optim: adamw_torch
weight_decay: 0.01
max_grad_norm: 1.0

Torch compile settings

torch_compile: true
torch_compile_backend: inductor
torch_compile_mode: default
torch_compile_fullgraph: false
torch_compile_dynamic: false

seed: 300
data_seed: 301

Evaluation and checkpointing

report_to:
- wandb

Docker & Self-Hosted Deployment Verified OCI

File: README.md Copy Chapter

News Copy Section

Benchmark Copy Section

Installation Copy Section

System Dependencies

Python Installation

Usage Examples Copy Section

Viewing Results

Using an Inference Provider or External Server Copy Section

Verified External Providers

Multi-node / Cluster Usage Copy Section

Using Beaker for Cluster Execution

Using Docker Copy Section

Quick Start - Process PDFs

Interactive Mode

Full Documentation Copy Section

Code overview Copy Chapter

Team Copy Chapter

License Copy Chapter

Citing Copy Chapter

File: CHANGELOG.md Copy Chapter

Changelog

Unreleased Copy Chapter

[v0.4.27](https://github.com/allenai/olmocr/releases/tag/v0.4.27) - 2026-03-12 Copy Chapter

[v0.4.26](https://github.com/allenai/olmocr/releases/tag/v0.4.26) - 2026-03-06 Copy Chapter

[v0.4.25](https://github.com/allenai/olmocr/releases/tag/v0.4.25) - 2026-01-25 Copy Chapter

[v0.4.24](https://github.com/allenai/olmocr/releases/tag/v0.4.24) - 2026-01-23 Copy Chapter

[v0.4.23](https://github.com/allenai/olmocr/releases/tag/v0.4.23) - 2026-01-23 Copy Chapter

[v0.4.22](https://github.com/allenai/olmocr/releases/tag/v0.4.22) - 2026-01-23 Copy Chapter

[v0.4.21](https://github.com/allenai/olmocr/releases/tag/v0.4.21) - 2026-01-23 Copy Chapter

[v0.4.20](https://github.com/allenai/olmocr/releases/tag/v0.4.20) - 2026-01-20 Copy Chapter

[v0.4.19](https://github.com/allenai/olmocr/releases/tag/v0.4.19) - 2026-01-20 Copy Chapter

[v0.4.18](https://github.com/allenai/olmocr/releases/tag/v0.4.18) - 2026-01-20 Copy Chapter

[v0.4.17](https://github.com/allenai/olmocr/releases/tag/v0.4.17) - 2026-01-20 Copy Chapter

[v0.4.16](https://github.com/allenai/olmocr/releases/tag/v0.4.16) - 2026-01-12 Copy Chapter

[v0.4.15](https://github.com/allenai/olmocr/releases/tag/v0.4.15) - 2026-01-09 Copy Chapter

[v0.4.14](https://github.com/allenai/olmocr/releases/tag/v0.4.14) - 2026-01-08 Copy Chapter

[v0.4.13](https://github.com/allenai/olmocr/releases/tag/v0.4.13) - 2026-01-08 Copy Chapter

[v0.4.12](https://github.com/allenai/olmocr/releases/tag/v0.4.12) - 2025-12-10 Copy Chapter

[v0.4.11](https://github.com/allenai/olmocr/releases/tag/v0.4.11) - 2025-12-10 Copy Chapter

[v0.4.10](https://github.com/allenai/olmocr/releases/tag/v0.4.10) - 2025-12-09 Copy Chapter

[v0.4.9](https://github.com/allenai/olmocr/releases/tag/v0.4.9) - 2025-12-09 Copy Chapter

[v0.4.7](https://github.com/allenai/olmocr/releases/tag/v0.4.7) - 2025-12-01 Copy Chapter

[v0.4.6](https://github.com/allenai/olmocr/releases/tag/v0.4.6) - 2025-11-17 Copy Chapter

[v0.4.5](https://github.com/allenai/olmocr/releases/tag/v0.4.5) - 2025-11-14 Copy Chapter

[v0.4.4](https://github.com/allenai/olmocr/releases/tag/v0.4.4) - 2025-11-04 Copy Chapter

[v0.4.3](https://github.com/allenai/olmocr/releases/tag/v0.4.3) - 2025-10-31 Copy Chapter

[v0.4.2](https://github.com/allenai/olmocr/releases/tag/v0.4.2) - 2025-10-22 Copy Chapter

[v0.4.1](https://github.com/allenai/olmocr/releases/tag/v0.4.1) - 2025-10-22 Copy Chapter

[v0.4.0](https://github.com/allenai/olmocr/releases/tag/v0.4.0) - 2025-10-22 Copy Chapter

[v0.3.9](https://github.com/allenai/olmocr/releases/tag/v0.3.9) - 2025-10-07 Copy Chapter

[v0.3.8](https://github.com/allenai/olmocr/releases/tag/v0.3.8) - 2025-10-06 Copy Chapter

[v0.3.7](https://github.com/allenai/olmocr/releases/tag/v0.3.7) - 2025-10-06 Copy Chapter

[v0.3.6](https://github.com/allenai/olmocr/releases/tag/v0.3.6) - 2025-09-29 Copy Chapter

[v0.3.4](https://github.com/allenai/olmocr/releases/tag/v0.3.4) - 2025-08-31 Copy Chapter

[v0.3.3](https://github.com/allenai/olmocr/releases/tag/v0.3.3) - 2025-08-15 Copy Chapter

[v0.3.2](https://github.com/allenai/olmocr/releases/tag/v0.3.2) - 2025-08-14 Copy Chapter

[v0.3.1](https://github.com/allenai/olmocr/releases/tag/v0.3.1) - 2025-08-14 Copy Chapter

[v0.3.0](https://github.com/allenai/olmocr/releases/tag/v0.3.0) - 2025-08-13 Copy Chapter

[v0.2.3](https://github.com/allenai/olmocr/releases/tag/v0.2.3) - 2025-08-04 Copy Chapter

[v0.2.2](https://github.com/allenai/olmocr/releases/tag/v0.2.2) - 2025-08-04 Copy Chapter

[v0.2.1](https://github.com/allenai/olmocr/releases/tag/v0.2.1) - 2025-07-23 Copy Chapter

[v0.2.0](https://github.com/allenai/olmocr/releases/tag/v0.2.0) - 2025-07-23 Copy Chapter

[v0.1.76](https://github.com/allenai/olmocr/releases/tag/v0.1.76) - 2025-06-23 Copy Chapter

[v0.1.75](https://github.com/allenai/olmocr/releases/tag/v0.1.75) - 2025-06-17 Copy Chapter

[v0.1.74](https://github.com/allenai/olmocr/releases/tag/v0.1.74) - 2025-06-17 Copy Chapter

[v0.1.73](https://github.com/allenai/olmocr/releases/tag/v0.1.73) - 2025-06-17 Copy Chapter

[v0.1.72](https://github.com/allenai/olmocr/releases/tag/v0.1.72) - 2025-06-17 Copy Chapter

[v0.1.71](https://github.com/allenai/olmocr/releases/tag/v0.1.71) - 2025-05-30 Copy Chapter

[v0.1.70](https://github.com/allenai/olmocr/releases/tag/v0.1.70) - 2025-05-23 Copy Chapter

[v0.1.69](https://github.com/allenai/olmocr/releases/tag/v0.1.69) - 2025-05-20 Copy Chapter

[v0.1.68](https://github.com/allenai/olmocr/releases/tag/v0.1.68) - 2025-05-19 Copy Chapter

[v0.1.60](https://github.com/allenai/olmocr/releases/tag/v0.1.60) - 2025-03-17 Copy Chapter

[v0.1.58](https://github.com/allenai/olmocr/releases/tag/v0.1.58) - 2025-02-15 Copy Chapter

[v0.1.53](https://github.com/allenai/olmocr/releases/tag/v0.1.53) - 2025-02-14 Copy Chapter

File: RELEASE_PROCESS.md Copy Chapter

GitHub Release Process

Steps Copy Chapter

Fixing a failed release Copy Chapter

File: README.md

News

Benchmark

Installation

Usage Examples

Using an Inference Provider or External Server

Multi-node / Cluster Usage

Using Docker

Full Documentation

Code overview

Team

License

Citing

File: CHANGELOG.md

Unreleased

[v0.4.27](https://github.com/allenai/olmocr/releases/tag/v0.4.27) - 2026-03-12

[v0.4.26](https://github.com/allenai/olmocr/releases/tag/v0.4.26) - 2026-03-06

[v0.4.25](https://github.com/allenai/olmocr/releases/tag/v0.4.25) - 2026-01-25

[v0.4.24](https://github.com/allenai/olmocr/releases/tag/v0.4.24) - 2026-01-23

[v0.4.23](https://github.com/allenai/olmocr/releases/tag/v0.4.23) - 2026-01-23

[v0.4.22](https://github.com/allenai/olmocr/releases/tag/v0.4.22) - 2026-01-23

[v0.4.21](https://github.com/allenai/olmocr/releases/tag/v0.4.21) - 2026-01-23

[v0.4.20](https://github.com/allenai/olmocr/releases/tag/v0.4.20) - 2026-01-20

[v0.4.19](https://github.com/allenai/olmocr/releases/tag/v0.4.19) - 2026-01-20

[v0.4.18](https://github.com/allenai/olmocr/releases/tag/v0.4.18) - 2026-01-20

[v0.4.17](https://github.com/allenai/olmocr/releases/tag/v0.4.17) - 2026-01-20

[v0.4.16](https://github.com/allenai/olmocr/releases/tag/v0.4.16) - 2026-01-12

[v0.4.15](https://github.com/allenai/olmocr/releases/tag/v0.4.15) - 2026-01-09

[v0.4.14](https://github.com/allenai/olmocr/releases/tag/v0.4.14) - 2026-01-08

[v0.4.13](https://github.com/allenai/olmocr/releases/tag/v0.4.13) - 2026-01-08

[v0.4.12](https://github.com/allenai/olmocr/releases/tag/v0.4.12) - 2025-12-10

[v0.4.11](https://github.com/allenai/olmocr/releases/tag/v0.4.11) - 2025-12-10

[v0.4.10](https://github.com/allenai/olmocr/releases/tag/v0.4.10) - 2025-12-09

[v0.4.9](https://github.com/allenai/olmocr/releases/tag/v0.4.9) - 2025-12-09

[v0.4.7](https://github.com/allenai/olmocr/releases/tag/v0.4.7) - 2025-12-01

[v0.4.6](https://github.com/allenai/olmocr/releases/tag/v0.4.6) - 2025-11-17

[v0.4.5](https://github.com/allenai/olmocr/releases/tag/v0.4.5) - 2025-11-14

[v0.4.4](https://github.com/allenai/olmocr/releases/tag/v0.4.4) - 2025-11-04

[v0.4.3](https://github.com/allenai/olmocr/releases/tag/v0.4.3) - 2025-10-31

[v0.4.2](https://github.com/allenai/olmocr/releases/tag/v0.4.2) - 2025-10-22

[v0.4.1](https://github.com/allenai/olmocr/releases/tag/v0.4.1) - 2025-10-22

[v0.4.0](https://github.com/allenai/olmocr/releases/tag/v0.4.0) - 2025-10-22

[v0.3.9](https://github.com/allenai/olmocr/releases/tag/v0.3.9) - 2025-10-07

[v0.3.8](https://github.com/allenai/olmocr/releases/tag/v0.3.8) - 2025-10-06

[v0.3.7](https://github.com/allenai/olmocr/releases/tag/v0.3.7) - 2025-10-06

[v0.3.6](https://github.com/allenai/olmocr/releases/tag/v0.3.6) - 2025-09-29

[v0.3.4](https://github.com/allenai/olmocr/releases/tag/v0.3.4) - 2025-08-31

[v0.3.3](https://github.com/allenai/olmocr/releases/tag/v0.3.3) - 2025-08-15

[v0.3.2](https://github.com/allenai/olmocr/releases/tag/v0.3.2) - 2025-08-14

[v0.3.1](https://github.com/allenai/olmocr/releases/tag/v0.3.1) - 2025-08-14

[v0.3.0](https://github.com/allenai/olmocr/releases/tag/v0.3.0) - 2025-08-13

[v0.2.3](https://github.com/allenai/olmocr/releases/tag/v0.2.3) - 2025-08-04

[v0.2.2](https://github.com/allenai/olmocr/releases/tag/v0.2.2) - 2025-08-04

[v0.2.1](https://github.com/allenai/olmocr/releases/tag/v0.2.1) - 2025-07-23

[v0.2.0](https://github.com/allenai/olmocr/releases/tag/v0.2.0) - 2025-07-23

[v0.1.76](https://github.com/allenai/olmocr/releases/tag/v0.1.76) - 2025-06-23

[v0.1.75](https://github.com/allenai/olmocr/releases/tag/v0.1.75) - 2025-06-17

[v0.1.74](https://github.com/allenai/olmocr/releases/tag/v0.1.74) - 2025-06-17

[v0.1.73](https://github.com/allenai/olmocr/releases/tag/v0.1.73) - 2025-06-17

[v0.1.72](https://github.com/allenai/olmocr/releases/tag/v0.1.72) - 2025-06-17

[v0.1.71](https://github.com/allenai/olmocr/releases/tag/v0.1.71) - 2025-05-30

[v0.1.70](https://github.com/allenai/olmocr/releases/tag/v0.1.70) - 2025-05-23

[v0.1.69](https://github.com/allenai/olmocr/releases/tag/v0.1.69) - 2025-05-20

[v0.1.68](https://github.com/allenai/olmocr/releases/tag/v0.1.68) - 2025-05-19

[v0.1.60](https://github.com/allenai/olmocr/releases/tag/v0.1.60) - 2025-03-17

[v0.1.58](https://github.com/allenai/olmocr/releases/tag/v0.1.58) - 2025-02-15

[v0.1.53](https://github.com/allenai/olmocr/releases/tag/v0.1.53) - 2025-02-14

File: RELEASE_PROCESS.md

Steps

Fixing a failed release

File: .readthedocs.yaml

File: docs/source/index.md

olmocr

Indices and tables

File: docs/source/installation.md

Installing with `pip`

Installing from source

File: docs/source/overview.md

File: olmocr/bench/README.md

Results