instructor Documentation

Structured outputs for llms

14,039 stars Python Python 100 files · ~136,281 tokens #openai#openai-function-calli#openai-functions#pydantic-v2#python#validation
RAW Doc

Technical Documentation: 567-labs/instructor

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (README.md)

Instructor: Structured Outputs for LLMs

Get reliable JSON from any LLM. Built on Pydantic for validation, type safety, and IDE support.

python
import instructor
from pydantic import BaseModel


# Define what you want
class User(BaseModel):
    name: str
    age: int


# Extract it from natural language
client = instructor.from_provider("openai/gpt-4o-mini")
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "John is 25 years old"}],
)

print(user)  # User(name='John', age=25)

That's it. No JSON parsing, no error handling, no retries. Just define a model and get structured data.





Use Instructor for fast extraction, reach for PydanticAI when you need agents. Instructor keeps schema-first flows simple and cheap. If your app needs richer agent runs, built-in observability, or shareable traces, try PydanticAI. PydanticAI is the official agent runtime from the Pydantic team, adding typed tools, replayable datasets, evals, and production dashboards while using the same Pydantic models. Dive into the PydanticAI docs to see how it extends Instructor-style workflows.

Why Instructor?

Getting structured data from LLMs is hard. You need to:

  1. Write complex JSON schemas
  2. Handle validation errors
  3. Retry failed extractions
  4. Parse unstructured responses
  5. Deal with different provider APIs

Instructor handles all of this with one simple interface:

Without Instructor With Instructor
```python
response = openai.chat.completions.create(
text
model="gpt-5.4-mini",
messages=[{"role": "user", "content": "..."}],
tools=[
    {
        "type": "function",
        "function": {
            "name": "extract_user",
            "parameters": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                },
            },
        },
    }
],

)

Parse response

tool_call = response.choices[0].message.tool_calls[0]
user_data = json.loads(tool_call.function.arguments)

Validate manually

if "name" not in user_data:
# Handle error...
pass
|python
client = instructor.from_provider("openai/gpt-5.4-mini")

user = client.chat.completions.create(
response_model=User,
messages=[{"role": "user", "content": "..."}],
)

That's it! user is validated and typed

|
## Install in seconds

```bash
pip install instructor

Or with your package manager:

bash
uv add instructor
poetry add instructor

Works with every major provider

Use the same code with any LLM provider:

python
# OpenAI
client = instructor.from_provider("openai/gpt-4o")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")

# Google
client = instructor.from_provider("google/gemini-pro")

# Ollama (local)
client = instructor.from_provider("ollama/llama3.2")

# With API keys directly (no environment variables needed)
client = instructor.from_provider("openai/gpt-4o", api_key="sk-...")
client = instructor.from_provider("anthropic/claude-3-5-sonnet", api_key="sk-ant-...")
client = instructor.from_provider("groq/llama-3.1-8b-instant", api_key="gsk_...")

# All use the same API!
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Production-ready features

Automatic retries

Failed validations are automatically retried with the error message:

python
from pydantic import BaseModel, field_validator


class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v


# Instructor automatically retries when validation fails
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
    max_retries=3,
)

Streaming support

Stream partial objects as they're generated:

python
from instructor import Partial

for partial_user in client.chat.completions.create(
    response_model=Partial[User],
    messages=[{"role": "user", "content": "..."}],
    stream=True,
):
    print(partial_user)
    # User(name=None, age=None)
    # User(name="John", age=None)
    # User(name="John", age=25)

Nested objects

Extract complex, nested data structures:

python
from typing import List


class Address(BaseModel):
    street: str
    city: str
    country: str


class User(BaseModel):
    name: str
    age: int
    addresses: List[Address]


# Instructor handles nested objects automatically
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Used in production by

Trusted by over 100,000 developers and companies building AI applications:

  • 3M+ monthly downloads
  • 10K+ GitHub stars
  • 1000+ community contributors

Companies using Instructor include teams at OpenAI, Google, Microsoft, AWS, and many YC startups.

Get started

Basic extraction

Extract structured data from any text:

python
from pydantic import BaseModel
import instructor

client = instructor.from_provider("openai/gpt-4o-mini")


class Product(BaseModel):
    name: str
    price: float
    in_stock: bool


product = client.chat.completions.create(
    response_model=Product,
    messages=[{"role": "user", "content": "iPhone 15 Pro, $999, available now"}],
)

print(product)
# Product(name='iPhone 15 Pro', price=999.0, in_stock=True)

Multiple languages

Instructor's simple API is available in many languages:

  • Python - The original
  • TypeScript - Full TypeScript support
  • Ruby - Ruby implementation
  • Go - Go implementation
  • Elixir - Elixir implementation
  • Rust - Rust implementation

Learn more

Why use Instructor over alternatives?

vs Raw JSON mode: Instructor provides automatic validation, retries, streaming, and nested object support. No manual schema writing.

vs LangChain/LlamaIndex: Instructor is focused on one thing - structured extraction. It's lighter, faster, and easier to debug.

vs Custom solutions: Battle-tested by thousands of developers. Handles edge cases you haven't thought of yet.

Contributing

We welcome contributions! Check out our good first issues to get started.

License

MIT License - see LICENSE for details.

Getting Started (`docs/getting-started.md`)

title: Getting Started

Getting Started with Instructor

This guide will walk you through the basics of using Instructor to extract structured data from language models. By the end, you'll understand how to:

  1. Install and set up Instructor
  2. Extract basic structured data
  3. Handle validation and errors
  4. Work with streaming responses
  5. Use different LLM providers

Installation

First, install Instructor:

bash
pip install instructor

To use a specific provider, install the appropriate extras:

Instructor's core install contains only required dependencies. Provider SDKs are optional and must be added explicitly.

bash
# For OpenAI (included by default)
pip install instructor

# For Anthropic
pip install "instructor[anthropic]"

# For other providers
pip install "instructor[google-genai]"         # For Google/Gemini
pip install "instructor[vertexai]"             # For Vertex AI
pip install "instructor[cohere]"               # For Cohere
pip install "instructor[litellm]"              # For LiteLLM (multiple providers)
pip install "instructor[mistral]"               # For Mistral
pip install "instructor[xai]"                  # For xAI

Setting Up Environment

Set your API keys as environment variables:

bash
# For OpenAI
export OPENAI_API_KEY=your_openai_api_key

# For Anthropic
export ANTHROPIC_API_KEY=your_anthropic_api_key

# For other providers, set relevant API keys

Your First Structured Output

Let's start with a simple example using OpenAI:

python
import instructor
from pydantic import BaseModel

# Define your output structure
class UserInfo(BaseModel):
    name: str
    age: int

# Create an instructor client with from_provider
client = instructor.from_provider("openai/gpt-5-nano")

# Extract structured data
user_info = client.create(
    response_model=UserInfo,
    messages=[
        {"role": "user", "content": "John Doe is 30 years old."}
    ],
)

print(f"Name: {user_info.name}, Age: {user_info.age}")
# Output: Name: John Doe, Age: 30

This example demonstrates the core workflow:

  1. Define a Pydantic model for your output structure
  2. Create an Instructor client with from_provider
  3. Request structured output using the response_model parameter

Validation and Error Handling

Instructor leverages Pydantic's validation to ensure your data meets requirements:

python
from pydantic import BaseModel, Field, field_validator

class User(BaseModel):
    name: str
    age: int = Field(gt=0, lt=120)  # Age must be between 0 and 120

    @field_validator('name')
    def name_must_have_space(cls, v):
        if ' ' not in v:
            raise ValueError('Name must include first and last name')
        return v

# This will make the LLM retry if validation fails
user = client.create(
    response_model=User,
    messages=[
        {"role": "user", "content": "Extract: Tom is 25 years old."}
    ],
)

Working with Complex Models

Instructor works seamlessly with nested Pydantic models:

python
from pydantic import BaseModel
from typing import List

class Address(BaseModel):
    street: str
    city: str
    state: str
    zip_code: str

class Person(BaseModel):
    name: str
    age: int
    addresses: List[Address]

person = client.create(
    response_model=Person,
    messages=[
        {"role": "user", "content": """
        Extract: John Smith is 35 years old.
        He has homes at 123 Main St, Springfield, IL 62704 and
        456 Oak Ave, Chicago, IL 60601.
        """}
    ],
)

Streaming Responses

For larger responses or better user experience, use streaming:

python
from instructor import Partial

# Stream the response as it's being generated
stream = client.create_partial(
    response_model=Person,
    messages=[
        {"role": "user", "content": "Extract a detailed person profile for John Smith, 35, who lives in Chicago and Springfield."}
    ],
)

for partial in stream:
    # This will incrementally show the response being built
    print(partial)

Using Different Providers

Instructor supports multiple LLM providers. Here's how to use Anthropic:

python
import instructor
from pydantic import BaseModel

class UserInfo(BaseModel):
    name: str
    age: int

# Create an instructor client with from_provider
client = instructor.from_provider("anthropic/claude-3-opus-20240229")

user_info = client.create(
    response_model=UserInfo,
    messages=[
        {"role": "user", "content": "John Doe is 30 years old."}
    ],
)

print(f"Name: {user_info.name}, Age: {user_info.age}")

Frequently Asked Questions

What's the difference between `start-here.md` and `getting-started.md`?

  • Start Here: Explains what Instructor is and why you'd use it (conceptual overview)
  • Getting Started: This guide - shows you how to install and use Instructor (practical steps)

Which provider should I start with?

OpenAI is the most popular choice for beginners due to reliability and wide support. Once comfortable, you can explore Anthropic Claude, Google Gemini, or open-source models.

Do I need to understand Pydantic?

Basic knowledge helps, but you can start with simple models. Instructor works with any Pydantic BaseModel. Learn more advanced features as you need them.

Can I use Instructor with async code?

Yes! Use async_client=True when creating your client: client = instructor.from_provider("openai/gpt-4o", async_client=True), then use await client.create().

What if validation fails?

Instructor automatically retries with validation feedback. You can configure retry behavior with max_retries parameter. See retry mechanisms for details.

View all FAQs →

Next Steps

Now that you've mastered the basics, here are some next steps:

Using older patterns? If you're using instructor.patch() or provider-specific functions like from_openai(), check out the Migration Guide to modernize your code.

New to Instructor? Start with Start Here for a conceptual overview.

For more detailed information on any topic, visit the Concepts section.

If you have questions or need help, join our Discord community or check the GitHub repository.

Index (`docs/index.md`)

title: "Instructor - Multi-Language Library for Structured LLM Outputs | Python, TypeScript, Go, Ruby" keywords: "LLM structured outputs, structured data extraction, OpenAI structured data, Pydantic LLM validation, Python LLM library, TypeScript LLM, Go LLM, Ruby LLM, Anthropic structured outputs, GPT structured data extraction, LLM response validation, AI data extraction, Ollama structured outputs, open source LLM, DeepSeek validation, Instructor vs Guardrails, LLM validation library, JSON schema validation, nested LLM schemas"

Instructor: Top Multi-Language Library for Structured LLM Outputs

Extract structured data from any LLM with type safety, validation, and automatic retries. Available in Python, TypeScript, Go, Ruby, Elixir, and Rust.






Instructor for extraction, PydanticAI for agents. Instructor shines when you need fast, schema-first extraction without extra agents. When your project needs quality gates, shareable runs, or built-in observability, try PydanticAI. PydanticAI is the official agent runtime from the Pydantic team: it adds typed tools, dataset replays, and production dashboards while keeping your existing Instructor models. Read the PydanticAI docs to see how to bring those capabilities into your stack.

What is Instructor?

Instructor is the most popular Python library for extracting structured data from Large Language Models (LLMs). With over 3 million monthly downloads, 11k stars, and 100+ contributors, it's the go-to solution for developers who need reliable, validated outputs from AI models.

Built on top of Pydantic, Instructor provides type-safe data extraction with automatic validation, retries, and streaming support. Whether you're using OpenAI's GPT models, Anthropic's Claude, Google's Gemini, open source models with Ollama, DeepSeek, or any of 15+ supported providers, Instructor ensures your LLM outputs are always structured and validated.

Key Features for LLM Data Extraction

  • Structured Outputs: Define Pydantic models to specify exactly what data you want from your LLM
  • Automatic Retries: Built-in retry logic when validation fails - no more manual error handling
  • Data Validation: Leverage Pydantic's powerful validation to ensure response quality
  • Streaming Support: Real-time processing of partial responses and lists
  • Multi-Provider: Works with OpenAI, Anthropic, Google, Mistral, Cohere, Ollama, DeepSeek, and 15+ LLM providers
  • Type Safety: Full IDE support with proper type inference and autocompletion
  • Open Source Support: Run any open source model locally with Ollama, llama-cpp-python, or vLLM

Quick Start

Install Instructor and start extracting structured data in minutes:

=== "pip"
bash pip install instructor

=== "uv"
bash uv add instructor

=== "poetry"
bash poetry add instructor

Extract Structured Data

Instructor's from_provider function provides a unified interface to work with any LLM provider. Switch between OpenAI, Anthropic, Google, Ollama, DeepSeek, and 15+ providers with the same code:

python
import instructor
from pydantic import BaseModel


class Person(BaseModel):
    name: str
    age: int
    occupation: str


# Works with any provider - same interface everywhere
client = instructor.from_provider("openai/gpt-5-nano")
# Or: instructor.from_provider("anthropic/claude-3")
# Or: instructor.from_provider("google/gemini-pro")
# Or: instructor.from_provider("ollama/llama3.2")  # local

# Extract structured data from natural language
person = client.create(
    response_model=Person,
    messages=[
        {"role": "user", "content": "Extract: John is a 30-year-old software engineer"}
    ],
)
print(person)  # Person(name='John', age=30, occupation='software engineer')

The from_provider API supports both sync and async usage (async_client=True) and automatically handles provider-specific configurations. See all supported providers →

Complex Schemas & Validation

Instructor excels at extracting complex, nested data structures with custom validation rules. Here's a concise example:

python
import instructor
from pydantic import BaseModel, Field, field_validator
from typing import List, Optional
from enum import Enum


class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    CRITICAL = "critical"


class Ticket(BaseModel):
    title: str = Field(..., min_length=5, max_length=100)
    priority: Priority
    estimated_hours: Optional[float] = Field(None, gt=0, le=100)

    @field_validator('estimated_hours')
    @classmethod
    def validate_hours(cls, v):
        if v is not None and v % 0.5 != 0:
            raise ValueError('Hours must be in 0.5 increments')
        return v


class CustomerSupport(BaseModel):
    customer_name: str
    tickets: List[Ticket] = Field(..., min_items=1)


client = instructor.from_provider("openai/gpt-4o")

support_case = client.create(
    response_model=CustomerSupport,
    messages=[{"role": "user", "content": "Extract support case details..."}],
    max_retries=3,
)

Key Features:

  • Deep nesting with nested models and lists
  • Custom validation with Pydantic validators
  • Automatic retries on validation failures
  • Type-safe extraction with full IDE support

Learn more about validation and complex schemas →

Supported LLM Providers

Instructor works seamlessly with 15+ popular LLM providers, giving you the flexibility to use any model while maintaining consistent structured output handling. From OpenAI's GPT models to open source alternatives with Ollama, DeepSeek models, and local inference, get validated data extraction everywhere.

It stands out for its simplicity, transparency, and user-centric design, built on top of Pydantic. Instructor helps you manage validation context, retries with Tenacity, and streaming Lists and Partial responses.

:material-star: Star the Repo{: .md-button .md-button--primary } :material-book-open-variant: Cookbooks{: .md-button } :material-lightbulb: Prompting Guide{: .md-button }

If you ever get stuck, you can always run instructor docs to open the documentation in your browser. It even supports searching for specific topics.

bash
instructor docs [QUERY]

Provider Examples

All providers use the same simple interface. Here are quick examples for the most popular providers:

=== "OpenAI"
```python
import instructor
from pydantic import BaseModel

text
class ExtractUser(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-5-nano")
res = client.create(
    response_model=ExtractUser,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)
```

[Full OpenAI docs →](./integrations/openai.md)

=== "Anthropic"
```python
import instructor
from pydantic import BaseModel

text
class ExtractUser(BaseModel):
    name: str
    age: int


client = instructor.from_provider("anthropic/claude-3-5-sonnet-20240620")
resp = client.create(
    response_model=ExtractUser,
    messages=[{"role": "user", "content": "Extract Jason is 25 years old."}],
)
```

[Full Anthropic docs →](./integrations/anthropic.md)

=== "Google Gemini"
```python
import instructor
from pydantic import BaseModel

text
class ExtractUser(BaseModel):
    name: str
    age: int


client = instructor.from_provider("google/gemini-2.5-flash")
resp = client.create(
    response_model=ExtractUser,
    messages=[{"role": "user", "content": "Extract Jason is 25 years old."}],
)
```

[Full Google docs →](./integrations/google.md)

=== "Ollama (Local)"
```python
import instructor
from pydantic import BaseModel

text
class ExtractUser(BaseModel):
    name: str
    age: int


client = instructor.from_provider("ollama/llama3.2")
resp = client.create(
    response_model=ExtractUser,
    messages=[{"role": "user", "content": "Extract Jason is 25 years old."}],
)
```

[Full Ollama docs →](./integrations/ollama.md)

View all 15+ providers →

Citation

If you use Instructor in your research or project, please cite it using:

bibtex
@software{liu2024instructor,
  author = {Jason Liu and Contributors},
  title = {Instructor: A library for structured outputs from large language models},
  url = {https://github.com/instructor-ai/instructor},
  year = {2024},
  month = {3}
}

Why use Instructor?

Using Hooks

Instructor's hooks system lets you intercept and handle events during LLM interactions. Use hooks for logging, monitoring, or custom error handling:

python
import instructor
from pydantic import BaseModel


class UserInfo(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4o-mini")

# Attach hooks for logging and error handling
client.on("completion:kwargs", lambda **kw: print("Called with:", kw))
client.on("completion:error", lambda e: print(f"Error: {e}"))

user_info = client.create(
    response_model=UserInfo,
    messages=[{"role": "user", "content": "Extract: John is 20 years old"}],
)

Learn more about hooks →

Type Inference & Advanced Methods

Instructor provides full type inference for better IDE support and type safety. The client includes specialized methods for different use cases:

Basic extraction:

python
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4o-mini")
user = client.create(response_model=User, messages=[...])  # Type: User

Async support:

python
client = instructor.from_provider("openai/gpt-4o-mini", async_client=True)
user = await client.create(...)  # Type: User

Access original completion:

python
user, completion = client.create_with_completion(...)  # Returns tuple

Stream partial objects:

python
for partial in client.create_partial(...):  # Type: Generator[User, None]
    print(partial)

Stream multiple objects:

python
for user in client.create_iterable(...):  # Type: Generator[User, None]
    print(user)

All methods provide full type inference for better IDE autocomplete and type checking.

Frequently Asked Questions

What is Instructor?

Instructor is a Python library that extracts structured, validated data from Large Language Models (LLMs). It uses Pydantic models to define output schemas and automatically handles validation, retries, and error handling.

Which LLM providers does Instructor support?

Instructor supports 15+ providers including OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Ollama, DeepSeek, and many more. See our integrations page for the complete list.

Do I need to know Pydantic to use Instructor?

Basic Pydantic knowledge helps, but you can get started with simple models. Instructor works with any Pydantic BaseModel, and you can learn advanced features as you need them.

How does Instructor compare to other libraries?

Instructor focuses specifically on structured outputs with automatic validation and retries. Unlike larger frameworks, Instructor does one thing very well: getting reliable, validated data from LLMs.

Can I use Instructor with open source models?

Yes! Instructor works with Ollama, llama-cpp-python, and other local models. See our Ollama integration guide to get started.

Does Instructor work with async code?

Yes, Instructor fully supports async/await. Use async_client=True when creating your client, then use await client.create().

View all FAQs →

Templating

Instructor supports templating with Jinja, which lets you create dynamic prompts. This is useful when you want to fill in parts of a prompt with data. Here's a simple example:

python
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4o-mini")


class User(BaseModel):
    name: str
    age: int


# Create a completion using a Jinja template in the message content
response = client.create(
    messages=[
        {
            "role": "user",
            "content": """Extract the information from the
            following text: {{ data }}`""",
        },
    ],
    response_model=User,
    context={"data": "John Doe is thirty years old"},
)

print(response)
#> User(name='John Doe', age=30)

Learn more about templating :octicons-arrow-right:{: .md-button .md-button-primary }

Validation

You can also use Pydantic to validate your outputs and get the llm to retry on failure. Check out our docs on retrying and validation context.

python
import instructor
from pydantic import BaseModel, ValidationError, BeforeValidator
from typing_extensions import Annotated
from instructor import llm_validator

# Create instructor client
client = instructor.from_provider("openai/gpt-4o-mini")


class QuestionAnswer(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(llm_validator("don't say objectionable things", client=client)),
    ]


try:
    qa = QuestionAnswer(
        question="What is the meaning of life?",
        answer="The meaning of life is to be evil and steal",
    )
except ValidationError as e:
    print(e)
    """
    1 validation error for QuestionAnswer
    answer
      Assertion failed, The statement promotes objectionable behavior by encouraging evil and stealing. [type=assertion_error, input_value='The meaning of life is to be evil and steal', input_type=str]
    """

Contributing

If you want to help out, checkout some of the issues marked as good-first-issue or help-wanted. Found here. They could be anything from code improvements, a guest blog post, or a new cook book.

License

This project is licensed under the terms of the MIT License.

Repository Overview (`docs/repository-overview.md`)

title: Repository Overview

Repository Overview

This page explains the layout of the Instructor codebase and what each key directory contains.

Directory Summary

`instructor/`

Core library with clients, adapters, and utilities for structured outputs.

`cli/`

Command-line interface code used for tasks like job management and usage tracking.

`docs/`

Documentation source files for the website built with MkDocs.

`examples/`

Practical examples and cookbooks demonstrating how to use Instructor.

`tests/`

Test suite and evaluation scripts ensuring the library functions correctly.

Blog Index (`docs/blog/index.md`)

Subscribe to our Newsletter for Updates and Tips

If you want to get updates on new features and tips on how to use Instructor, you can subscribe to our newsletter below to get notified when we publish new content.

Advanced Topics

  1. What's new in Instructor v2?
  2. Unified Provider Interface in Instructor
  3. Instructor Implements llms.txt
  4. Query Understanding: Beyond Embeddings
  5. Achieving GPT-4 Level Summaries with GPT-3.5-turbo
  6. Basics of Guardrails and Validation in AI Models
  7. Validating Citations in AI-Generated Content
  8. Fine-tuning and Distillation in AI Models
  9. Enhancing OpenAI Client Observability with LangSmith
  10. Logfire Integration with Pydantic

AI Development and Optimization

Language Models and Prompting Techniques

Integrations and Tools

Media and Resources

Cli Index (`docs/cli/index.md`)

title: Instructor CLI Tools

Instructor CLI Tools

Getting Started

Installation

The CLI tools are included with the Instructor package:

bash
pip install instructor

API Setup

Set your OpenAI API key as an environment variable:

bash
export OPENAI_API_KEY="your-api-key-here"

Available Commands

Instructor provides several command-line utilities:

Command Description Guide
instructor usage Track API usage and costs Usage Guide
instructor finetune Create and manage fine-tuned models Fine-Tuning Guide
instructor docs Quick access to documentation See below

Usage Command

Monitor your OpenAI API usage directly from the terminal:

bash
# View total usage for the current month
instructor usage

# View usage breakdown by day
instructor usage --by-day

# Calculate cost for a specific model
instructor usage --model gpt-5.4-mini

For detailed usage statistics and options, see the Usage Guide.

Fine-Tuning Command

Create and manage fine-tuned models with an interactive interface:

bash
# Start the fine-tuning interface
instructor finetune

This launches an interactive application that guides you through the fine-tuning process. Learn more in the Fine-Tuning Guide.

Documentation Command

Quickly access Instructor documentation from your terminal:

bash
# Open main documentation
instructor docs

# Search for specific topic
instructor docs validation

# Open specific page
instructor docs concepts/models

This command opens the Instructor documentation in your default web browser, making it easy to find information when you need it.

Support & Contribution

Concepts Index (`docs/concepts/index.md`)

title: Instructor Concepts - Core Features and Patterns

Instructor Concepts

This section explains the core concepts and features of the Instructor library, organized by category to help you find what you need.

Core Concepts

These are the fundamental concepts you need to understand to use Instructor effectively:

  • Models - Using Pydantic models to define output structures
  • Patching - How Instructor patches LLM clients
  • from_provider - Unified interface for creating clients across all providers
  • Migration Guide - Migrating from older patterns to from_provider
  • Types - Working with different data types in your models
  • Validation - Validating LLM outputs against your models
  • Prompting - Creating effective prompts for structured output extraction
  • Multimodal - Working with Audio Files, Images and PDFs

Data Handling and Structures

These concepts relate to defining and working with different data structures:

  • Fields - Working with Pydantic fields and attributes
  • Lists and Arrays - Handling lists and arrays in your models
  • TypedDicts - Using TypedDict for flexible typing
  • Union Types - Working with union types
  • Enums - Using enumerated types in your models
  • Missing - Handling missing or optional values
  • Alias - Create field aliases
  • Citation - Extract and validate citations from source text

Streaming Features

These features help you work with streaming responses:

Error Handling and Validation

These features help you ensure data quality:

  • Retrying - Configure automatic retry behavior
  • Validators - Define custom validation logic
  • Hooks - Add callbacks for monitoring and debugging

Performance Optimization

These features help you optimize performance:

Integration Features

These features help you integrate with other technologies:

Philosophy

  • Philosophy - The guiding principles behind Instructor

How These Concepts Work Together

Instructor is built around a few key ideas that work together:

  1. Define Structure with Pydantic: Use Pydantic models to define exactly what data you want.
  2. Create Clients with from_provider: Use the unified interface to create clients for any provider.
  3. Validate and Retry: Automatically validate responses and retry if necessary.
  4. Process Streams: Handle streaming responses for real-time updates.

Typical Workflow

mermaid
sequenceDiagram
    participant User as Your Code
    participant Instructor
    participant LLM as LLM Provider

    User->>Instructor: Define Pydantic model
    User->>Instructor: Create client with from_provider
    User->>Instructor: Call create() with response_model
    Instructor->>LLM: Send structured request
    LLM->>Instructor: Return LLM response
    Instructor->>Instructor: Validate against model

    alt Validation Success
        Instructor->>User: Return validated Pydantic object
    else Validation Failure
        Instructor->>LLM: Retry with error context
        LLM->>Instructor: Return new response
        Instructor->>Instructor: Validate again
        Instructor->>User: Return validated object or error
    end

For practical examples of these concepts, visit the Cookbook section.

!!! see-also "See Also"
- Getting Started Guide - Begin your journey with Instructor
- Examples - Practical implementations of these concepts
- Integrations - Connect with different LLM providers

Examples Index (`docs/examples/index.md`)

title: Instructor Cookbook Collection

Instructor Cookbooks

Our cookbooks demonstrate how to use Instructor to solve real-world problems with structured outputs. Each example includes complete code and explanations to help you implement similar solutions in your own projects.

Text Processing

Classification Examples

Example Description Use Case
Single Classification Basic classification with a single category Content categorization
Multiple Classification Handling multiple classification categories Multi-label document tagging
Enum-Based Classification Using Python enums for structured classification Standardized taxonomies
Batch Classification Process multiple items efficiently High-volume text processing
Batch Classification with LangSmith Using LangSmith for batch processing Performance monitoring
Local Classification Classification without external APIs Offline processing

Information Extraction

Example Description Use Case
Entity Resolution Identify and disambiguate entities Name standardization
Contact Information Extract structured contact details CRM data entry
PII Sanitization Detect and redact sensitive information Privacy compliance
Citation Extraction Accurately extract formatted citations Academic research
Action Items Extract tasks from text Meeting follow-ups
Search Query Processing Structure complex search queries Search enhancement

Document Processing

Example Description Use Case
Document Segmentation Divide documents into meaningful sections Long-form content analysis
Planning and Tasks Break down complex queries into subtasks Project management
Knowledge Graph Generation Create relationship graphs from text Information visualization
Knowledge Graph Building Build and query knowledge graphs Semantic data modeling
Chain of Density Implement iterative summarization Content distillation

Multi-Modal Examples

Vision Processing

Example Description Use Case
Table Extraction Convert image tables to structured data Data entry automation
Table Extraction with GPT-4 Advanced table extraction Complex table processing
Receipt Information Extract data from receipt images Expense management
Slide Content Extraction Convert slides to structured text Presentation analysis
Image to Ad Copy Generate ad text from images Marketing automation
YouTube Clip Analysis Extract info from video clips Content moderation

Multi-Modal Processing

Example Description Use Case
Gemini Multi-Modal Process text, images, and other data Mixed-media analysis

Data Tools

Database Integration

Example Description Use Case
SQLModel Integration Store AI-generated data in SQL databases Persistent storage
Pandas DataFrame Work with structured data in Pandas Data analysis

Streaming and Processing

Example Description Use Case
Partial Response Streaming Stream partial results in real-time Interactive applications
Self-Critique and Correction Implement self-assessment Quality improvement

API Integration

Example Description Use Case
Content Moderation Implement content filtering Trust & safety
Cost Optimization with Batch API Reduce API costs Production efficiency
Few-Shot Learning Use contextual examples in prompts Performance tuning

Observability & Tracing

Example Description Use Case
Langfuse Tracing Open-source LLM engineering Observability & Debugging

Deployment Options

Model Providers

Example Description Use Case
Groq Cloud API High-performance inference Low-latency applications
Mistral/Mixtral Models Open-source model integration Cost-effective deployment
IBM watsonx.ai Enterprise AI platform Business applications

Local Deployment

Example Description Use Case
Ollama Integration Local open-source models Privacy-focused applications

Stay Updated

Subscribe to our newsletter for updates on new features and usage tips:

Looking for more structured learning? Check out our Tutorial series for step-by-step guides.

Integrations Index (`docs/integrations/index.md`)

title: "LLM Provider Integration Tutorials - Instructor"

LLM Provider Integration Tutorials

Learn how to integrate Instructor with various AI model providers. These comprehensive tutorials cover everything from cloud-based services like OpenAI and Anthropic to local open-source models, helping you extract structured outputs from any LLM.

Common Features

All integrations support these core features:

Feature Description Documentation
Model Patching Enhance provider clients with structured output capabilities Patching
Response Models Define expected response schema with Pydantic Models
Validation Ensure responses match your schema definition Validation
Streaming Stream partial or iterative responses Partial, Iterable
Hooks Add callbacks for monitoring and debugging Hooks

However, each provider has different capabilities and limitations. Refer to the specific provider documentation for details.

Provider Modes

Providers support different methods for generating structured outputs:

Mode Description Providers
TOOLS Uses OpenAI-style tools/function calling OpenAI, Anthropic, Mistral
PARALLEL_TOOLS Multiple simultaneous tool calls OpenAI
JSON Direct JSON response generation OpenAI, Gemini, Cohere, GenAI
MD_JSON JSON embedded in markdown Most providers

See the Modes Comparison guide for details.

Getting Started

There are two ways to use providers with Instructor:

The simplest way to get started is using the provider initialization:

python
import instructor
from pydantic import BaseModel

class UserInfo(BaseModel):
    name: str
    age: int

# Initialize any provider with a simple string
client = instructor.from_provider("openai/gpt-5.4-mini")
# Or use async client
async_client = instructor.from_provider("anthropic/claude-3-sonnet", async_client=True)

# Use the same interface for all providers
response = client.create(
    response_model=UserInfo,
    messages=[{"role": "user", "content": "Your prompt"}]
)

Supported provider strings:

  • openai/model-name: OpenAI models
  • anthropic/model-name: Anthropic models
  • google/model-name: Google models
  • mistral/model-name: Mistral models
  • cohere/model-name: Cohere models
  • perplexity/model-name: Perplexity models
  • groq/model-name: Groq models
  • writer/model-name: Writer models
  • bedrock/model-name: AWS Bedrock models
  • cerebras/model-name: Cerebras models
  • fireworks/model-name: Fireworks models
  • vertexai/model-name: Vertex AI models
  • genai/model-name: Google GenAI models
  • ollama/model-name: Ollama models

Provider Checklist

Use these example strings with from_provider to quickly get started:

  • instructor.from_provider("openai/gpt-5-nano")
  • instructor.from_provider("anthropic/claude-3-sonnet")
  • instructor.from_provider("google/gemini-2.5-flash")
  • instructor.from_provider("mistral/mistral-large-latest")
  • instructor.from_provider("cohere/command-r")
  • instructor.from_provider("perplexity/sonar-small")
  • instructor.from_provider("groq/llama3-8b-8192")
  • instructor.from_provider("writer/palmyra-x-004")
  • instructor.from_provider("bedrock/anthropic.claude-3-sonnet-20240229-v1:0")
  • instructor.from_provider("cerebras/llama3.1-70b")
  • instructor.from_provider("fireworks/llama-v3-70b-instruct")
  • instructor.from_provider("vertexai/gemini-3-flash")
  • instructor.from_provider("genai/gemini-3-flash")
  • instructor.from_provider("ollama/llama3.2")

2. Manual Client Setup

Alternatively, you can manually set up the client:

  1. Install the required dependencies:

    bash
    pip install "instructor[provider]"  # e.g., instructor[anthropic]

  2. Import the provider client and patch it with Instructor:

    python
    import instructor
from provider_package import Client

client = instructor.from_provider(Client())

  3. Use the patched client with your Pydantic model:

    python
    response = client.create(
    response_model=YourModel,
    messages=[{"role": "user", "content": "Your prompt"}]
)

For provider-specific setup and examples, visit each provider's documentation page.

Need Help?

If you need assistance with a specific integration:

  1. Check the provider-specific documentation
  2. Browse the examples and cookbooks
  3. Search existing GitHub issues
  4. Join our Discord community

Learning Index (`docs/learning/index.md`)

Instructor LLM Tutorial: Complete Guide to Structured Outputs

Learn how to use Instructor for LLM structured outputs with this comprehensive tutorial. Instructor is the leading Python library for extracting structured, validated data from large language models (LLMs) like GPT-4, Claude, and Gemini.

What You'll Learn in This LLM Tutorial

This Instructor tutorial covers everything from basic LLM integration to advanced structured output patterns. Whether you're building AI applications, automating data extraction, or creating LLM-powered APIs, this guide provides practical, production-ready examples.

Getting Started with Instructor LLM Tutorial

Start your journey with these beginner-friendly tutorials for LLM integration:

LLM Data Extraction Patterns

Learn essential patterns for extracting structured data from language models:

LLM Output Validation Tutorial

Ensure reliability with these validation tutorials:

Streaming LLM Responses

Real-time LLM output processing tutorials:

Why This Instructor LLM Tutorial?

  • Production-Ready Examples: Real-world LLM integration patterns used by thousands of developers
  • Multi-Provider Support: Works with OpenAI, Anthropic, Google, Cohere, and more
  • Type-Safe Outputs: Leverage Python's type system for reliable LLM applications
  • Progressive Learning Path: From basic LLM calls to advanced extraction techniques

Ready to master structured outputs with LLMs? Start with our installation guide and build your first LLM-powered application today!

Prompting Index (`docs/prompting/index.md`)

title: Advanced Prompting Techniques Guide

Advanced Prompting Techniques

This guide presents 58 research-backed prompting techniques mapped to Instructor implementations. Based on The Prompt Report by Learn Prompting which analyzed over 1,500 academic papers on prompting.

Prompting Technique Map

The following diagram shows how different prompting techniques relate to each other and when to use them:

mermaid
flowchart TD
    A[Choose Prompting Technique] --> B{Have Examples?}

    B -->|No| C[Zero-Shot Techniques]
    B -->|Yes| D[Few-Shot Techniques]

    C --> C1[Role Prompting]
    C --> C2[Emotional Language]
    C --> C3[Style Definition]
    C --> C4[Follow-Up Generation]

    D --> D1[Example Ordering]
    D --> D2[Example Selection]
    D --> D3[Example Generation]

    A --> E{Need Reasoning?}

    E -->|Yes| F[Thought Generation]
    F --> F1[Chain of Thought]
    F --> F2[Step-Back Prompting]
    F --> F3[Thread of Thought]

    A --> G{Complex Problem?}

    G -->|Yes| H[Decomposition]
    H --> H1[Least-to-Most]
    H --> H2[Tree of Thought]
    H --> H3[Plan and Solve]

    A --> I{Need Verification?}

    I -->|Yes| J[Self-Criticism]
    J --> J1[Self-Verification]
    J --> J2[Chain of Verification]
    J --> J3[Self-Refinement]

    A --> K{Want Multiple Perspectives?}

    K -->|Yes| L[Ensembling]
    L --> L1[Self-Consistency]
    L --> L2[Meta-CoT]
    L --> L3[Specialized Experts]

    classDef category fill:#e2f0fb,stroke:#b8daff,color:#004085;
    classDef technique fill:#d4edda,stroke:#c3e6cb,color:#155724;
    classDef decision fill:#fff3cd,stroke:#ffeeba,color:#856404;

    class A,C,D,F,H,J,L category
    class C1,C2,C3,C4,D1,D2,D3,F1,F2,F3,H1,H2,H3,J1,J2,J3,L1,L2,L3 technique
    class B,E,G,I,K decision

When to Use Each Technique

Goal Recommended Techniques
Improve accuracy Chain of Thought, Self-Verification, Self-Consistency
Handle complex problems Decomposition, Tree of Thought, Least-to-Most
Generate creative content Role Prompting, Emotional Language, Style Definition
Verify factual correctness Chain of Verification, Self-Calibration
Optimize with few examples KNN Example Selection, Active Prompting
Handle uncertainty Uncertainty-Routed CoT, Self-Consistency

Zero-Shot {#zero-shot}

These techniques improve model performance without examples:

Technique Description Use Case
Emotional Language Add emotional tone to prompts Creative writing, empathetic responses
Role Assignment Give the model a specific role Expert knowledge, specialized perspectives
Style Definition Specify writing style Content with particular tone or format
Prompt Refinement Automatic prompt optimization Iterative improvement of results
Perspective Simulation Have the model adopt viewpoints Multiple stakeholder analysis
Ambiguity Clarification Identify and resolve unclear aspects Improving precision of responses
Query Repetition Ask model to restate the task Better task understanding
Follow-Up Generation Generate clarifying questions Deep exploration of topics

Few-Shot {#few-shot}

Techniques for effectively using examples in prompts:

Technique Description Use Case
Example Generation Automatically create examples Domains with limited example data
Example Ordering Optimal sequencing of examples Improved pattern recognition
KNN Example Selection Choose examples similar to query Domain-specific accuracy
Vote-K Selection Advanced similarity-based selection Complex pattern matching

Thought Generation {#thought-generation}

Methods to encourage human-like reasoning in models:

Zero-Shot Reasoning

Technique Description Use Case
Analogical CoT Generate reasoning using analogies Complex problem-solving
Step-Back Prompting Consider higher-level questions first Scientific and abstract reasoning
Thread of Thought Encourage step-by-step analysis Detailed explanation generation
Tabular CoT Structure reasoning in table format Multi-factor analysis

Few-Shot Reasoning

Technique Description Use Case
Active Prompting Annotate uncertain examples Improved accuracy on edge cases
Auto-CoT Choose diverse examples Broad domain coverage
Complexity-Based CoT Use complex examples Challenging problem types
Contrastive CoT Include correct and incorrect cases Error detection and avoidance
Memory of Thought Use high-certainty examples Reliability in critical applications
Uncertainty-Routed CoT Select the most certain reasoning path Decision-making under uncertainty
Prompt Mining Generate templated prompts Efficient prompt engineering

Ensembling {#ensembling}

Techniques for combining multiple prompts or responses:

Technique Description Use Case
Consistent, Diverse Sets Build consistent example sets Stable performance
Batched In-Context Examples Efficient example batching Performance optimization
Step Verification Validate individual steps Complex workflows
Maximizing Mutual Information Information theory optimization Information-dense outputs
Meta-CoT Merge multiple reasoning chains Complex problem-solving
Specialized Experts Use different "expert" prompts Multi-domain tasks
Self-Consistency Choose most consistent reasoning Logical accuracy
Universal Self-Consistency Domain-agnostic consistency General knowledge tasks
Task-Specific Selection Choose examples per task Specialized domain tasks
Prompt Paraphrasing Use variations of the same prompt Robust outputs

Self-Criticism {#self-criticism}

Methods for models to verify or improve their own responses:

Technique Description Use Case
Chain of Verification Generate verification questions Fact-checking, accuracy
Self-Calibration Ask if answer is correct Confidence estimation
Self-Refinement Auto-generate feedback and improve Iterative improvement
Self-Verification Score multiple solutions Quality assessment
Reverse CoT Reconstruct the problem Complex reasoning verification
Cumulative Reasoning Generate possible steps Thorough analysis

Decomposition {#decomposition}

Techniques for breaking down complex problems:

Technique Description Use Case
Functional Decomposition Implement subproblems as functions Modular problem-solving
Faithful CoT Use natural and symbolic language Mathematical reasoning
Least-to-Most Solve increasingly complex subproblems Educational applications
Plan and Solve Generate a structured plan Project planning
Program of Thought Use code for reasoning Algorithmic problems
Recursive Thought Recursively solve subproblems Hierarchical problems
Skeleton of Thought Generate outline structure Writing, planning
Tree of Thought Search through possible paths Decision trees, exploration

Implementation with Instructor

All these prompting techniques can be implemented with Instructor by:

  1. Defining appropriate Pydantic models that capture the expected structure
  2. Incorporating the prompting technique in your model docstrings or field descriptions
  3. Using the patched LLM client with your response model
python
import instructor
from pydantic import BaseModel, Field
# Example implementing Chain of Thought with a field
class ReasonedAnswer(BaseModel):
    """Answer the following question with detailed reasoning."""

    chain_of_thought: str = Field(
        description="Step-by-step reasoning process to solve the problem"
    )
    final_answer: str = Field(
        description="The final conclusion after reasoning"
    )

client = instructor.from_provider("openai/gpt-5-nano")

response = client.create(
    model="gpt-5.4-mini",
    response_model=ReasonedAnswer,
    messages=[
        {"role": "user", "content": "What is the cube root of 27?"}
    ]
)

print(f"Reasoning: {response.chain_of_thought}")
print(f"Answer: {response.final_answer}")

References

* Based on The Prompt Report: A Systematic Survey of Prompting Techniques

Tutorials Index (`docs/tutorials/index.md`)

title: Instructor Tutorials

Instructor Tutorials

Tutorial Pathway {#tutorial-pathway}

Our tutorials follow a carefully designed learning path from basic concepts to advanced applications. Each tutorial builds on previous concepts while introducing new techniques.

Tutorial Topic Key Skills Difficulty
1. Introduction to Structured Outputs Basic extraction Pydantic models, basic prompting 🟢 Beginner
2. Tips and Tricks Best practices Advanced models, optimization 🟢 Beginner
3. Applications: RAG Retrieval-augmented generation Information retrieval, context handling 🟡 Intermediate
4. Applications: RAG Validation Validating RAG outputs Quality control, validation hooks 🟡 Intermediate
5. Validation Techniques Deep validation Custom validators, error handling 🟡 Intermediate
6. Knowledge Graphs Graph building Entity relationships, graph visualization 🔴 Advanced
7. Chain of Density Summarization techniques Iterative refinement, content density 🔴 Advanced
8. Synthetic Data Generation Creating datasets Data augmentation, testing data 🔴 Advanced

Running Options {#running-options}

Choose your preferred environment to work through these interactive Jupyter notebooks:

  • :material-laptop: Run Locally

    bash
    git clone https://github.com/jxnl/instructor.git
cd instructor
pip install -e ".[all]"
jupyter notebook docs/tutorials/

  • :material-google: Google Colab

    Look for the "Open in Colab" button at the top of each notebook

    Perfect for cloud execution without local setup

  • :simple-mybinder: Binder

    Click the "Launch Binder" button to run instantly in your browser

    No installation or API keys required for basic examples

Skills Gained {#skills-gained}

By completing this tutorial series, you'll gain practical skills in:

  • Structured Extraction: Define Pydantic models that capture exactly the data you need
  • Advanced Validation: Ensure LLM outputs meet your data quality requirements
  • Streaming Responses: Process data in real-time with partial and iterative outputs
  • Complex Applications: Build RAG systems, knowledge graphs, and more
  • Multi-Provider Support: Work with different LLM providers using a consistent interface
  • Production Techniques: Learn optimization strategies for real-world applications

Setup Requirements

Before starting, make sure you have:

  • Python Environment: Python 3.8+ installed
  • Dependencies: Install with pip install "instructor[all]"
  • API Keys: Access to OpenAI API or other supported providers
  • Basic Knowledge: Familiarity with Python and basic LLM concepts

Getting Help {#getting-help}

We're here to support your learning journey:

AGENT (`docs/AGENT.md`)

title: Documentation Agent Guide

AGENT.md - Documentation

Commands

  • Serve docs locally: uv run mkdocs serve
  • Build docs: ./build_mkdocs.sh or uv run mkdocs build
  • Install doc deps: uv pip install -e ".[docs]"
  • Test examples: uv run pytest docs/ --examples

Structure

  • Core docs: concepts/, integrations/, examples/
  • Learning path: getting-started.md → learning/ → tutorials/
  • API reference: Auto-generated from docstrings via mkdocstrings
  • Blog: blog/posts/ for announcements and deep-dives
  • Templates: templates/ for new docs (provider, concept, cookbook)

Writing Guidelines

  • Reading level: Grade 10 (from .cursor/rules)
  • Code examples: Must be runnable with complete imports
  • Progressive complexity: Simple → advanced concepts
  • Provider docs: Follow templates/ patterns
  • Navigation: Update mkdocs.yml for new pages

Pull Request (PR) Formatting

Use Conventional Commits formatting for PR titles so they are consistent and easy to scan. Treat the PR title as the message we would use for a squash merge commit.

PR Title Format

Use:

<type>(<scope>): <short summary>

Rules:

  • Keep it under ~70 characters when you can.
  • Use the imperative mood (for example, “add”, “fix”, “update”).
  • Do not end with a period.
  • If it includes a breaking change, add ! after the type or scope (for example, feat(docs)!:).

Good examples:

  • docs(agents): add conventional commit PR title guidelines
  • docs(mkdocs): fix broken link in validation tutorial
  • docs(examples): update youtube clips snippet
  • chore(docs): refresh docs build commands

Common types:

  • docs: documentation-only changes
  • fix: bug fix
  • feat: new feature
  • test: add or update tests
  • chore: maintenance work (build scripts, tooling, repo hygiene)
  • ci: CI pipeline changes

Suggested docs scopes:

  • docs, mkdocs, blog, examples, integrations, tutorials, agents

PR Description Guidelines

Keep PR descriptions short and actionable:

  • What: What changed, in 1–3 sentences.
  • Why: Why this change is needed (link issues when possible).
  • Changes: 3–7 bullet points with the main edits.
  • Testing: What you ran (or why you did not run anything).
  • Docs impact: Call out page moves, redirects, or nav updates.

If the PR was authored by Cursor, include:

  • This PR was written by [Cursor](https://cursor.com)

Key Files

  • mkdocs.yml - Site configuration and navigation
  • hooks/ - Custom processing (hide_lines.py removes # <%hide%> markers)
  • overrides/ - Custom theme elements
  • javascripts/ - Client-side enhancements

Api Docstring Assessment (`docs/api-docstring-assessment.md`)

API Docstring Quality Assessment

This document assesses the quality and completeness of docstrings for all API items referenced in the expanded API documentation.

Summary

Overall, the docstring quality is good to excellent for most items. Many classes and functions have comprehensive docstrings with usage examples, while some core classes could benefit from class-level docstrings.

Excellent Docstrings (Comprehensive with Examples)

These have detailed docstrings with usage examples and clear descriptions:

Client Creation

  • from_provider - Comprehensive docstring with Args, Returns, Raises, and Examples sections. Includes multiple usage examples showing basic usage, caching, and async clients.

Validation

  • llm_validator - Good docstring with usage examples, parameter descriptions, and error message examples showing how validation errors are formatted.

DSL Components

  • CitationMixin - Excellent docstring with complete usage examples showing how to use it with context, and result examples showing the output structure.
  • IterableModel - Good docstring with usage examples showing before/after transformation, Parameters section, and Returns description.
  • Maybe - Good docstring with usage examples and result structure showing the generated model fields.

Batch Processing

  • BatchProcessor - Good class-level docstring explaining the unified interface. Methods like create_batch_from_messages and submit_batch have clear Args and Returns sections.

Distillation

  • Instructions - Good docstring with parameter descriptions. The distil method has usage examples showing decorator usage patterns.

Hooks

  • Hooks - Excellent class-level docstring explaining the purpose. Methods like on(), get_hook_name(), emit(), etc. have comprehensive docstrings with Args, Returns, Raises, and Examples sections.

Schema Generation

  • generate_openai_schema - Good docstring with Args, Returns, and Notes sections explaining how docstrings are used.
  • generate_anthropic_schema - Has docstring explaining the conversion process.

Multimodal

  • Audio - Good class-level docstring. Methods like autodetect() and autodetect_safely() have clear docstrings with Args and Returns.

Exceptions

  • InstructorError - Excellent docstring with Attributes section, Examples showing error handling, and See Also references.
  • IncompleteOutputException - Good docstring with Attributes, Common Solutions, and Examples.
  • InstructorRetryException - Comprehensive docstring with Attributes, Common Causes, Examples, and See Also.
  • ValidationError - Good docstring with Examples and See Also.
  • ProviderError - Good docstring with Attributes, Common Causes, and Examples.
  • ConfigurationError - Good docstring with Common Scenarios and Examples.
  • ModeError - Good docstring with Attributes, Examples, and See Also.
  • ClientError - Good docstring with Common Scenarios and Examples.
  • AsyncValidationError - Good docstring with Attributes and Examples.
  • ResponseParsingError - Good docstring with Attributes, Examples, and backwards compatibility notes.
  • MultimodalError - Good docstring with Attributes, Examples, and backwards compatibility notes.

Good Docstrings (Clear but Could Be Enhanced)

These have adequate docstrings but could benefit from more examples or additional detail:

Core Clients

  • Instructor - No class-level docstring. Methods have type hints but lack comprehensive docstrings. The class is well-documented through usage in examples, but a class-level docstring would help.
  • AsyncInstructor - Similar to Instructor, no class-level docstring.
  • Response - No class-level docstring. Methods like create() and create_with_completion() lack docstrings.

Client Creation

  • from_openai - No docstring. Only has type overloads. The implementation exists but lacks documentation explaining usage, parameters, and return values.

Function Calls & Schema

  • OpenAISchema - Good method docstrings for openai_schema, anthropic_schema, gemini_schema, and from_response(). The class itself could use a class-level docstring explaining its purpose and usage.
  • openai_schema - Decorator function, but the docstring is on the class method, not the decorator itself.

DSL Components

  • Partial - Minimal docstring. Has Notes and Example sections but could benefit from more comprehensive usage examples showing streaming scenarios.

Multimodal

  • Image - No class-level docstring. Methods have good docstrings (autodetect(), autodetect_safely(), from_gs_url(), etc.), but the class itself lacks documentation.

Mode & Provider

  • Mode - Good class-level docstring explaining what modes are and how they work. Individual mode values lack docstrings but the enum docstring is comprehensive.
  • Provider - No class-level docstring. Just enum values without explanation.

Patch Functions

  • patch - Good docstring explaining what features it enables (response_model, max_retries, validation_context, strict, hooks). Could benefit from usage examples.
  • apatch - Need to check if it has similar docstring quality.

Areas Needing Improvement

Missing Class-Level Docstrings

  1. Instructor - Should have a class-level docstring explaining:

    • What the class does
    • How to use it
    • Key features (modes, hooks, retries)
    • Basic usage example

  2. AsyncInstructor - Should have a class-level docstring explaining:

    • Async usage patterns
    • How it differs from Instructor
    • Async examples

  3. Response - Should have a class-level docstring explaining:

    • What the Response helper does
    • When to use it vs direct client methods
    • Usage examples

  4. Image - Should have a class-level docstring explaining:

    • What Image represents
    • Supported formats
    • Common usage patterns

  5. Provider - Should have a class-level docstring explaining:

    • What providers are supported
    • How to use Provider enum
    • Provider detection

Missing Function Docstrings

  1. from_openai - Needs comprehensive docstring with:

    • Purpose and usage
    • Parameters explanation
    • Return value description
    • Examples

  2. from_litellm - No docstring. Only has type overloads. Similar to from_openai, needs comprehensive docstring.

Could Be Enhanced

  1. Partial - Could add more streaming examples
  2. patch - Could add usage examples showing before/after
  3. apatch - Has docstring but marked as deprecated ("No longer necessary, use patch instead"). Docstring is adequate but the deprecation should be more prominent.
  4. openai_schema - Has minimal docstring. Could expand with usage examples showing how to use the decorator.

Recommendations

High Priority

  1. Add class-level docstrings to Instructor and AsyncInstructor - These are the core classes users interact with
  2. Add docstring to from_openai - Important client creation function
  3. Add class-level docstring to Response - Helper class that needs explanation

Medium Priority

  1. Add class-level docstring to Image - Commonly used multimodal class
  2. Add class-level docstring to Provider - Enum that could use explanation
  3. Enhance Partial docstring with more streaming examples

Low Priority

  1. Add more examples to patch docstring
  2. Expand openai_schema docstring with examples
  3. Consider updating apatch deprecation message to be more prominent

Overall Assessment

Grade: B+

The documentation is generally good with many excellent examples, but the core classes (Instructor, AsyncInstructor, Response) would benefit significantly from class-level docstrings. The DSL components and utility functions are well-documented, and the exception classes have comprehensive docstrings.

The mkdocs autodoc plugin will generate API documentation from these docstrings, so improving them will directly improve the generated API reference pages.

Api (`docs/api.md`)

title: API Reference Guide

API Reference

Core modes are the recommended default. Legacy provider-specific modes still
work but are deprecated and will show warnings. See the
Mode Migration Guide for details.

Core Clients

The main client classes for interacting with LLM providers.

::: instructor.core.client.Instructor

::: instructor.core.client.AsyncInstructor

::: instructor.core.client.Response

Client Creation

Functions to create Instructor clients from various providers.

::: instructor.auto_client.from_provider

::: instructor.v2.providers.openai.client.from_openai

::: instructor.v2.providers.litellm.client.from_litellm

DSL Components

Domain-specific language components for advanced patterns and data handling.

::: instructor.dsl.iterable

::: instructor.dsl.partial

::: instructor.dsl.parallel

::: instructor.dsl.maybe

::: instructor.dsl.citation

Function Calls & Schema

Classes and functions for defining and working with function call schemas.

::: instructor.function_calls

::: instructor.v2.core.function_calls.OpenAISchema

::: instructor.v2.core.function_calls.openai_schema

::: instructor.v2.core.schema.generate_openai_schema

::: instructor.v2.core.schema.generate_anthropic_schema

::: instructor.v2.core.schema.generate_gemini_schema

Validation

Validation utilities for LLM outputs and async validation support.

::: instructor.validation

::: instructor.v2.validation.llm_validator

::: instructor.v2.validation.openai_moderation

Batch Processing

Batch processing utilities for handling multiple requests efficiently.

::: instructor.batch

::: instructor.batch.BatchProcessor

::: instructor.batch.BatchRequest

::: instructor.batch.BatchJob

Distillation

Tools for distillation and fine-tuning workflows.

::: instructor.distil

::: instructor.distil.FinetuneFormat

::: instructor.distil.Instructions

Multimodal

Support for image and audio content in LLM requests.

::: instructor.processing.multimodal

::: instructor.v2.core.multimodal.Image

::: instructor.v2.core.multimodal.Audio

Mode & Provider

Enumerations for modes and providers.

::: instructor.mode.Mode

::: instructor.utils.providers.Provider

Exceptions

Exception classes for error handling.

::: instructor.core.exceptions

Hooks

Event hooks system for monitoring and intercepting LLM interactions.

::: instructor.core.hooks

::: instructor.core.hooks.Hooks

::: instructor.core.hooks.HookName

Patch Functions

Decorators for patching LLM client methods.

::: instructor.core.patch

::: instructor.core.patch.apatch

Architecture (`docs/architecture.md`)

title: Instructor Architecture Overview

Architecture Overview

This page explains the core execution flow and where to plug in or debug. It highlights the minimal sync/async code paths and how streaming, partial, and parallel modes integrate.

High-Level Flow

text
/* Detailed source-code truncated for AI context efficiency. */

Key responsibilities:

  • patch(): wraps the provider create with cache lookup/save, templating, strict mode, hooks, and retry.
  • Retry: executes provider call, emits hooks, updates usage, handles validation/JSON errors with reask, and re-attempts.
  • Dispatcher: selects the correct parsing path by Mode, handles multimodal message conversion, and attaches _raw_response to the returned model.
  • Provider Handlers: provider/mode-specific request shaping and reask preparation.

Minimal Code Paths

Synchronous

python
import openai
import instructor
from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

client = instructor.from_provider("openai/gpt-5-nano")

model = client.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "{'name': 'Ada', 'age': 37}"}],
    response_model=User,            # triggers schema/tool wiring + parsing
    max_retries=3,                  # tenacity-backed validation retries
    strict=True,                    # strict JSON parsing if supported
)

# Access raw provider response if needed
raw = model._raw_response

Asynchronous

python
import asyncio
import openai
import instructor
from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

async def main():
    aclient = instructor.from_provider("openai/gpt-5-nano", async_client=True)
    model = await aclient.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "{\"name\": \"Ada\", \"age\": 37}"}],
        response_model=User,
        max_retries=3,
        strict=True,
    )
    print(model)

asyncio.run(main())

Streaming, Partial, Parallel

Streaming Iterable

  • Use create_iterable(response_model=Model, stream=True implicitly) via Instructor.create_iterable.
  • Returns a generator (sync) or async generator (async) of parsed items.
  • Internally sets stream=True, and IterableBase.from_streaming_response(_async) assembles items.
python
for item in client.create_iterable(messages=..., response_model=MyModel):
    print(item)

Partial Objects

  • Use create_partial(response_model=Model) to receive progressively filled partial models while streaming.
  • Internally wraps the model as Partial[Model] and sets stream=True.
python
for partial in client.create_partial(messages=..., response_model=MyModel):
    # partial contains fields as they arrive
    pass

Parallel Tools

  • Use Mode.PARALLEL_TOOLS and a parallel type hint (e.g., list of models) when you need multiple tool calls in one request.
  • Streaming is not supported in parallel tools mode.
python
from instructor.mode import Mode

result = client.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Extract person and event info."}],
    response_model=[PersonInfo, EventInfo],
    mode=Mode.PARALLEL_TOOLS,
)

Hooks and Retry

You can observe and instrument the flow with hooks. Typical events:

  • completion:kwargs: just before provider call
  • completion:response: after provider call
  • parse:error: on validation/JSON errors
  • completion:last_attempt: when a retry sequence is about to stop
  • completion:error: non-validation completion errors
python
from instructor.core.hooks import HookName

client.on(HookName.COMPLETION_KWARGS, lambda **kw: print("KWARGS", kw))
client.on(HookName.PARSE_ERROR, lambda e: print("PARSE", e))

Where Multimodal Conversion Happens

  • For modes that require it, messages are converted via processing.multimodal.convert_messages.
  • Image/Audio/PDF autodetection can be enabled (by specific handlers/modes) and will convert strings/paths/URLs or data URIs into provider-ready payloads.

Error Handling at a Glance

  • Validation or JSON decode errors trigger the reask path.
  • Reask handlers (handle_reask_kwargs) append/adjust messages with error feedback so the next attempt can correct itself.
  • If all retries fail, InstructorRetryException is raised containing failed_attempts, the last completion, usage totals, and the create kwargs for reproduction.

Extensibility Notes

  • New providers add utils for response and reask handling and register modes used by the dispatcher.
  • Most JSON/tool patterns are shared; prefer reusing existing handlers where possible.
  • Keep provider-specific logic in provider utils; avoid expanding central dispatcher beyond routing and orchestration.

Contributing (`docs/contributing.md`)

title: Contribute to Instructor: Evals, Issues, and Pull Requests

Contributing to Instructor

We welcome contributions to Instructor! This page covers the different ways you can help improve the library.

Ways to Contribute

Evaluation Tests (Evals)

Evals help us monitor the quality of both the OpenAI models and the Instructor library. To contribute:

  1. Explore Existing Evals: Check out our evals directory
  2. Create a New Eval: Add new pytest tests that evaluate specific capabilities or edge cases
  3. Follow the Pattern: Structure your eval similar to existing ones
  4. Submit a PR: We'll review and incorporate your eval

Evals are run weekly, and results are tracked to monitor performance over time.

Reporting Issues

If you encounter a bug or problem, please file an issue on GitHub with:

  1. A clear, descriptive title
  2. Detailed information including:
    • The response_model you're using
    • The messages you sent
    • The model you're using
    • Steps to reproduce the issue
    • Expected vs. actual behavior
    • Your environment details (Python version, OS, package versions)

Contributing Code

We welcome pull requests! Here's the process:

  1. For Small Changes: Feel free to submit a PR directly
  2. For Larger Changes: Start with an issue to discuss approach
  3. Looking for Ideas? Check issues labeled help wanted or good first issue

Setting Up Your Development Environment

UV is a fast Python package installer and resolver that makes development easier.

  1. Install UV (official method):

    bash
    # macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows PowerShell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

  2. Install Project in Development Mode:

    bash
    # Clone the repository
git clone https://github.com/YOUR-USERNAME/instructor.git
cd instructor

# Install with development dependencies
uv sync --extra dev --extra docs

  3. Adding New Dependencies:

    bash
    # Add a project dependency and update pyproject.toml plus uv.lock
uv add some-package

# Install only into the current environment without changing project metadata
uv pip install some-package

  4. Common UV Commands:

    bash
    # Update UV itself
uv self update

# Verify the lockfile and regenerate the committed requirements export
uv lock --check
uv pip compile pyproject.toml -o requirements.txt

Using Poetry

Poetry provides comprehensive dependency management and packaging.

  1. Install Poetry:

    bash
    curl -sSL https://install.python-poetry.org | python3 -

  2. Install Dependencies:

    bash
    # Clone the repository
git clone https://github.com/YOUR-USERNAME/instructor.git
cd instructor

# Install with development dependencies
poetry install --with dev,docs

  3. Working with Poetry:

    bash
    # Activate virtual environment
poetry shell

# Run a command in the virtual environment
poetry run pytest

# Add a dependency
poetry add package-name

# Add a development dependency
poetry add --group dev package-name

Adding Support for New LLM Providers

Instructor uses optional dependencies to support different LLM providers. Provider-specific utilities live in the instructor/utils directory. To add a new provider:

  1. Add Dependencies to pyproject.toml:

    toml
    [project.optional-dependencies]
# Add your provider
my-provider = ["my-provider-sdk>=1.0.0,<2.0.0"]

[dependency-groups]
# Mirror in dependency groups
my-provider = ["my-provider-sdk>=1.0.0,<2.0.0"]

  2. Create Provider Client:

    • Create a new file at instructor/clients/client_myprovider.py
    • Implement from_myprovider function that patches the provider's client

  3. Add Tests: Create tests in tests/llm/test_myprovider/

  4. Document Installation:

    bash
    # Installation command for your provider
uv add "instructor[my-provider]"
# or with poetry
poetry add "instructor[my-provider]"

  5. Create Provider Utilities and Handlers:

    • Add instructor/utils/myprovider.py with reask and handle_* helpers
    • Define MYPROVIDER_HANDLERS mapping Mode values to these functions

  6. Register the Provider:

    • Update instructor/utils/providers.py with your provider enum value
    • Extend get_provider detection for your base URL

  7. Update process_response.py:

    • Import your handlers and add them to mode_handlers
    • This script uses the handlers to prepare kwargs and parse results

  8. Write Documentation:

    • Add a new markdown file in docs/integrations/ for your provider
    • Update mkdocs.yml to include your new page
    • Make sure to include a complete example

Development Workflow

  1. Fork the Repository: Create your own fork of the project
  2. Clone and Set Up:
    bash
    git clone https://github.com/YOUR-USERNAME/instructor.git
cd instructor
git remote add upstream https://github.com/567-labs/instructor.git
  3. Create a Branch:
    bash
    git checkout -b feature/your-feature-name
  4. Make Changes, Test, and Commit:
    bash
    # Run tests
uv run pytest tests/ -k 'not llm and not openai'  # Skip LLM tests for faster local dev

# Commit changes
git add .
git commit -m "Your descriptive commit message"
  5. Keep Updated and Push:
    bash
    git fetch upstream
git rebase upstream/main
git push origin feature/your-feature-name
  6. Create a Pull Request: Submit your PR with a clear description of changes

Utility Scripts

The scripts/ directory contains utility scripts that help maintain code quality and documentation. These scripts are integrated into pre-commit hooks and can also be run manually.

Available Scripts

`make_clean.py` - Markdown File Cleaner

Cleans markdown files by removing special whitespace characters and replacing em dashes with regular dashes.

bash
# Clean all markdown files
python scripts/make_clean.py

# Preview changes without modifying files
python scripts/make_clean.py --dry-run

`check_blog_excerpts.py` - Blog Post Excerpt Validator

Ensures all blog posts contain the `` tag for proper excerpt handling.

bash
# Check all blog posts
python scripts/check_blog_excerpts.py

`make_sitemap.py` - Enhanced Documentation Sitemap Generator

Generates an enhanced sitemap (sitemap.yaml) with AI-powered content analysis and cross-link suggestions.

bash
# Generate sitemap with default settings
python scripts/make_sitemap.py

# Customize settings
python scripts/make_sitemap.py \
  --root-dir docs \
  --output-file sitemap.yaml \
  --max-concurrency 10

Requirements for sitemap generation:

  • OpenAI API key (set as OPENAI_API_KEY environment variable)
  • Additional dependencies: openai, typer, rich, tenacity, pyyaml

Pre-commit Integration

These scripts run automatically during the commit process:

  • Markdown cleaning: Runs on commits with markdown files in docs/
  • Blog excerpt validation: Runs on commits with blog post files

Manual Usage

You can run scripts manually for testing or one-time operations:

bash
# Test markdown cleaning
python scripts/make_clean.py --dry-run

# Check blog excerpts
python scripts/check_blog_excerpts.py

# Generate fresh sitemap
python scripts/make_sitemap.py

For detailed documentation on each script, see the scripts/README.md file in the project repository.

Using Cursor to Build PRs

Cursor is an AI-powered code editor that can help you contribute to Instructor.

  1. Getting Started with Cursor:

    • Download Cursor from cursor.sh
    • Open the Instructor project in Cursor
    • Cursor will automatically detect our rules in .cursor/rules/

  2. Using Cursor Rules:

    • new-features-planning: Helps plan and structure new features
    • simple-language: Guidelines for writing clear documentation
    • documentation-sync: Ensures documentation stays in sync with code changes

  3. Creating PRs with Cursor:

    • Use Cursor's Git integration to create a new branch
    • Make your changes with AI assistance
    • Create a PR with:
      bash
      # Use GitHub CLI to create the PR
gh pr create -t "Your feature title" -b "Description of your changes" -r jxnl,ivanleomk
    • Add This PR was written by [Cursor](https://cursor.sh) to your PR description

  4. Benefits of Using Cursor:

    • AI helps generate code that follows our style guidelines
    • Simplifies PR creation process
    • Helps maintain documentation standards

Code Style Guidelines

We use the following tools to maintain code quality:

  • Ruff: For linting and formatting
  • ty: For type checking
  • Pre-commit: For automatic checks before committing
bash
# Install pre-commit hooks
uv run pre-commit install

Key style guidelines:

  • Use strict typing
  • Follow import order: standard lib → third-party → local
  • Use snake_case for functions/variables, PascalCase for classes
  • Write comprehensive docstrings for public API functions

Conventional Comments

When reviewing code or writing commit messages, we use conventional comments to make feedback clearer:

text
<label>: <subject>

<description>

Common labels:

  • praise: highlights something positive
  • suggestion: proposes a change or improvement
  • question: asks for clarification
  • issue: points out a problem that needs fixing
  • todo: notes something to be addressed later
  • fix: resolves an issue

Examples:

text
suggestion: use a validator for this field
This would ensure the value is always properly formatted.

question: why not use async processing here?
I'm curious if this would improve performance.

fix: correct the parameter type
It should be an OpenAI client instance, not a string.

This format helps everyone understand the purpose and importance of each comment. Visit conventionalcomments.org to learn more.

Conventional Commits

We use conventional commit messages to make our project history clear and generate automated changelogs. A conventional commit has this structure:

text
<type>[optional scope]: <description>

[optional body]

[optional footer]

Common Types

  • feat: New feature
  • fix: Bug fix
  • docs: Documentation changes
  • style: Formatting changes
  • refactor: Code change that neither fixes a bug nor adds a feature
  • test: Adding or fixing tests
  • chore: Maintenance tasks

Examples

text
feat(openai): add streaming response support

fix(anthropic): resolve tool calling response format

docs: update installation instructions

test(evals): add new recursive schema test cases

For breaking changes, add an exclamation mark before the colon:

text
feat(api)!: change return type of from_openai function

Using conventional commits helps automatically generate release notes and makes the project history easier to navigate.

For more details, see the Conventional Commits specification.

Documentation Contributions

Documentation improvements are highly valued:

  1. Docs Structure: All documentation is in Markdown in the docs/ directory
  2. Adding New Pages: When adding a new page, include it in mkdocs.yml in the right section
  3. Local Preview: Run mkdocs serve to preview changes locally
  4. Style Guidelines:
    • Write at a grade 10 reading level (simple, clear language)
    • Include working code examples
    • Add links to related documentation
    • Use consistent formatting
    • Make sure each code example is complete with imports

Example of a good documentation code block:

python
# Complete example with imports
import instructor
from pydantic import BaseModel
# Define your model
class Person(BaseModel):
    name: str
    age: int

# Create the patched client
client = instructor.from_provider("openai/gpt-5-nano")

# Use the model
person = client.create(
    model="gpt-5.4-mini",
    response_model=Person,
    messages=[
        {"role": "user", "content": "Extract: John Doe is 25 years old"}
    ]
)

print(person.name)  # "John Doe"
print(person.age)   # 25

Contributors

Documentation Resources

When working on documentation, these resources may be helpful:

  • mkdocs serve: Preview documentation locally. Install dependencies from requirements-doc.txt first.

  • hl_lines in Code Blocks: Highlight specific lines in a code block to draw attention:

    markdown
    ```python hl_lines="2 3"
def example():
    # This line is highlighted
    # This line is also highlighted
    return "normal line"
```

  • Admonitions: Create styled callout boxes for important information:

    markdown
    !!! note "Optional Title"
    This is a note admonition.

!!! warning
    This is a warning.

For more documentation features, check the MkDocs Material documentation.

Thank you for your contributions to Instructor!

Debugging (`docs/debugging.md`)

title: Debugging Instructor Applications

Debugging

This guide shows how to quickly inspect inputs/outputs, capture retries, and reproduce failures when working with Instructor. It focuses on practical techniques using hooks, logging, and exception data.

Enable Logs

Use Python's logging module at the start of your script, before creating the Instructor client:

python
import logging

logging.basicConfig(level=logging.INFO)
logging.getLogger("instructor").setLevel(logging.DEBUG)

This enables debug messages from instructor and its child loggers while keeping other loggers at INFO. Instructor does not read an INSTRUCTOR_DEBUG environment variable.

Manual Debug Configuration

To include debug messages from other libraries too, set the root logger to DEBUG:

python
import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("instructor").setLevel(logging.DEBUG)

You will see messages for:

  • Raw responses (provider-specific objects)
  • Handler/mode selection
  • Retry attempts and parse errors
  • Reask adjustments to messages

Tip: Set a handler/formatter to include timestamps and module names.

Observe the Flow with Hooks

Hooks let you tap into key moments without modifying core code:

python
from instructor.core.hooks import HookName

# Attach one or more handlers
client.on(HookName.COMPLETION_KWARGS, lambda **kw: print("KWARGS:", kw))
client.on(HookName.COMPLETION_RESPONSE, lambda resp: print("RESPONSE:", type(resp)))
client.on(HookName.PARSE_ERROR, lambda e: print("PARSE ERROR:", e))
client.on(HookName.COMPLETION_LAST_ATTEMPT, lambda e: print("LAST ATTEMPT:", e))
client.on(HookName.COMPLETION_ERROR, lambda e: print("COMPLETION ERROR:", e))

Common uses:

  • Capture the final kwargs passed to the provider (including mode/tools/response_format).
  • Record raw responses (e.g., to logs or a file) for offline analysis.
  • Inspect parse errors and how reask modifies the next attempt.

Note: Handlers that accept **kwargs (or a parameter named _instructor_meta) receive a metadata dict with:

  • attempt_number, correlation_id, mode, response_model_name.
    Add **kwargs to your handler signature to access it:
python
client.on(HookName.COMPLETION_KWARGS, lambda **kw: print(kw.get("_instructor_meta")))

Inspect Raw Responses

Most parsed models returned by Instructor carry the original provider response for debugging:

python
model = client.create(...)
raw = getattr(model, "_raw_response", None)
print(raw)

This is useful for checking provider metadata like token usage, model version, and provider-specific fields.

Handling Failures & Retries

When all retries are exhausted, an InstructorRetryException is raised. It includes detailed context:

python
from instructor.core.exceptions import InstructorRetryException

try:
    client.create(...)
except InstructorRetryException as e:
    print("Attempts:", e.n_attempts)
    print("Last completion:", e.last_completion)
    print("Create kwargs:", e.create_kwargs)  # reproducible input
    print("Failed attempts:", e.failed_attempts)  # list of (attempt, exception, completion)
    # If available, a compact trace packet to help debugging
    if hasattr(e, "trace_packet") and e.trace_packet:
        print("Trace packet:", e.trace_packet)

Use e.create_kwargs and e.failed_attempts to craft a minimal reproduction.

Minimal Reproduction Template

python
import openai
import instructor
from pydantic import BaseModel

class MyModel(BaseModel):
    # fields...
    pass

client = instructor.from_provider("openai/gpt-5-nano")

create_kwargs = {
    # paste from InstructorRetryException.create_kwargs
}

try:
    client.create(response_model=MyModel, **create_kwargs)
except Exception as err:
    # Inspect and iterate
    raise

This pattern captures the exact inputs that triggered a failure.

Strict vs Non-Strict Parsing

  • strict=True enforces exact schema matches and can surface schema drift early.
  • If providers sometimes return extra fields or slightly different types (e.g., floats for ints), try strict=False to validate non‑strictly.
python
client.create(..., response_model=MyModel, strict=True)

Customizing Retries

You can pass an integer (attempt count) or a tenacity retrying object to control behavior:

python
from tenacity import Retrying, stop_after_attempt, stop_after_delay

max_retries = Retrying(stop=stop_after_attempt(3) | stop_after_delay(10))
client.create(..., max_retries=max_retries)

This is helpful when balancing latency and robustness.

Multimodal & Message Conversion

If you send images/audio/PDFs or text that may include media paths/URIs, Instructor can convert messages for provider formats.

  • For supported modes, processing.multimodal.convert_messages runs automatically.
  • If debugging content issues, log messages before and after conversion using the hooks above, and ensure media types/URIs are valid.

Caching Considerations

If you’re using a cache (cache=...), remember:

  • Successful parsed responses are stored; retrieving from cache skips the provider call.
  • If debugging live provider behavior, temporarily disable cache or change the cache key (e.g., tweak a message).
python
model = client.create(..., cache=None)

Common Troubleshooting Tips

  • Validate the response_model.model_json_schema() matches what you expect the provider to return.
  • Confirm mode is valid for your provider; mismatches can cause parsing failures.
  • Check provider‑side limits (max tokens/response length); incomplete outputs raise specific exceptions.
  • If using markdown JSON (MD_JSON), ensure the provider is actually returning a ```json code block.

If you need deeper visibility, add a custom handler to write kwargs/responses/errors to disk with a timestamp and correlation id.

Example: Local Debug Run

You can run a minimal, no‑network example that exercises hooks, logging, and parsing flow using a fake provider function:

  • File: examples/debugging/run.py
  • Run:
bash
python examples/debugging/run.py

This script:

  • Enables DEBUG logging for instructor.*
  • Patches a fake provider create with instructor.patch(mode=Mode.JSON)
  • Attaches hook handlers to print kwargs, response types, and parse errors
  • Parses a simple JSON payload into a Pydantic model and prints the result

Faq (`docs/faq.md`)

title: Frequently Asked Questions description: Common questions and answers about using Instructor

Frequently Asked Questions

This page answers common questions about using Instructor with various LLM providers.

General Questions

What is Instructor?

Instructor is a library that makes it easy to get structured data from Large Language Models (LLMs). It uses Pydantic to define output schemas and provides a consistent interface across different LLM providers.

How does Instructor work?

Instructor "patches" LLM clients to add a response_model parameter that accepts a Pydantic model. When you make a request, Instructor:

  1. Converts your Pydantic model to a schema the LLM can understand
  2. Formats the prompt appropriately for the provider
  3. Validates the LLM's response against your model
  4. Retries automatically if validation fails
  5. Returns a properly typed Pydantic object

Which LLM providers does Instructor support?

Instructor supports many providers, including:

  • OpenAI (GPT models)
  • Anthropic (Claude models)
  • Google (Gemini models)
  • Cohere
  • Mistral AI
  • Groq
  • LiteLLM (meta-provider)
  • TrueFoundry AI Gateway
  • Various open-source models via Ollama, llama.cpp, etc.

See the Integrations section for the complete list.

What's the difference between various modes?

Instructor supports generic modes across providers:

  • Mode.TOOLS - Tool/function calling when supported
  • Mode.JSON - JSON generation for providers that support it (GenAI)
  • Mode.JSON_SCHEMA - JSON schema enforcement (OpenAI, Mistral, Cohere)
  • Mode.MD_JSON - JSON embedded in markdown
  • Mode.PARALLEL_TOOLS - Parallel tool calls where supported

The optimal mode depends on your provider and use case. See Patching for details.

Installation and Setup

How do I install Instructor?

Basic installation:

bash
pip install instructor

For specific providers:

bash
pip install "instructor[anthropic]"  # For Anthropic
pip install "instructor[google-generativeai]"  # For Google/Gemini

What environment variables do I need?

This depends on your provider:

  • OpenAI: OPENAI_API_KEY
  • Anthropic: ANTHROPIC_API_KEY
  • Google: GOOGLE_API_KEY

Each provider has specific requirements documented in their integration guide.

Common Issues

Why is my model not returning structured data?

Common reasons include:

  1. Using the wrong mode for your provider
  2. Complex schema that confuses the model
  3. Insufficient context in your prompt
  4. Using a model that doesn't support function/tool calling

Try simplifying your schema or providing clearer instructions in your prompt.

How do I handle validation errors?

Instructor automatically retries when validation fails. You can customize this behavior:

python
from tenacity import stop_after_attempt

result = client.create(
    response_model=MyModel,
    max_retries=stop_after_attempt(5),  # Retry up to 5 times
    messages=[...]
)

Can I see the raw response from the LLM?

Yes, use create_with_completion:

python
result, completion = client.create_with_completion(
    response_model=MyModel,
    messages=[...]
)

result is your Pydantic model, and completion is the raw response.

How do I stream large responses?

Use create_partial for partial updates as the response is generated:

python
stream = client.create_partial(
    response_model=MyModel,
    messages=[...]
)

for partial in stream:
    print(partial)  # Partial model with fields filled in as they're generated

Performance and Costs

How can I optimize token usage?

  1. Use concise prompts
  2. Use smaller models for simpler tasks
  3. Use the MD_JSON or JSON mode for simple schemas
  4. Cache responses for repeated queries

How do I handle rate limits?

Instructor uses the tenacity library for retries, which you can configure:

python
from tenacity import retry_if_exception_type, wait_exponential
from openai.error import RateLimitError

result = client.create(
    response_model=MyModel,
    max_retries=retry_if_exception_type(RateLimitError),
    messages=[...],
)

Advanced Usage

How do I use Instructor with FastAPI?

Instructor works seamlessly with FastAPI:

python
from fastapi import FastAPI
from pydantic import BaseModel
import instructor
app = FastAPI()
client = instructor.from_provider("openai/gpt-5-nano")

class UserInfo(BaseModel):
    name: str
    age: int

@app.post("/extract")
async def extract_user_info(text: str) -> UserInfo:
    return client.create(
        model="gpt-5.4-mini",
        response_model=UserInfo,
        messages=[{"role": "user", "content": text}]
    )

How do I use Instructor with async code?

Use the async client:

python
import instructor
import asyncio
client = instructor.from_provider("openai/gpt-5-nano", async_client=True)

async def extract_data():
    result = await client.create(
        response_model=MyModel,
        messages=[...]
    )
    return result

asyncio.run(extract_data())

Where can I get more help?

Help (`docs/help.md`)

title: Getting Started with Instructor: Help and Resources description: Explore key resources for getting help with Instructor, including Discord, blog, concepts, cookbooks, and GitHub discussions.

Getting help with Instructor

If you need help getting started with Instructor or with advanced usage, the following sources may be useful.

:material-discord: Discord

The Discord is a great place to ask questions and get help from the community.

:material-creation: Concepts

The concepts section explains the core concepts of Instructor and how to prompt with models.

:material-chef-hat: Cookbooks

The cookbooks are a great place to start. They contain a variety of examples that demonstrate how to use Instructor in different scenarios.

:material-book: Blog

The blog contains articles that explain how to use Instructor in different scenarios.

:material-github: GitHub Discussions

GitHub discussions are useful for asking questions, your question and the answer will help everyone.

:material-github: GitHub Issues

GitHub issues are useful for reporting bugs or requesting new features.

:material-twitter: Twitter

You can also reach out to me on Twitter if you have any questions or ideas.

Installation (`docs/installation.md`)

title: Installing Instructor with Pip description: Learn how to install Instructor and its dependencies using pip for Python 3.9+. Simple setup guide included.

Installation is as simple as:

bash
pip install instructor

Instructor has a few dependencies:

  • openai: OpenAI's Python client.
  • typer: Build great CLIs. Easy to code. Based on Python type hints.
  • docstring-parser: A parser for Python docstrings, to improve the experience of working with docstrings in jsonschema.
  • pydantic: Data validation and settings management using python type annotations.

If you've got Python 3.9+ and pip installed, you're good to go.

Llms (`docs/llms.txt`)

Instructor: Type-Safe Structured Outputs from LLMs

Instructor is a library for extracting structured outputs from Large Language Models (LLMs) with type safety and validation.

Table of Contents

Installation

bash
pip install instructor

For specific providers:

bash
# OpenAI
pip install "instructor[openai]"

# Anthropic
pip install "instructor[anthropic]"

# Google (Gemini)
pip install "instructor[gemini]"

# Mistral
pip install "instructor[mistral]"

# Cohere
pip install "instructor[cohere]"

Core Concept

Instructor uses Pydantic models to define structured outputs and patches LLM clients to enable extraction with validation.

python
import instructor
from pydantic import BaseModel

# Define your output structure
class User(BaseModel):
    name: str
    age: int

# Create client using from_provider
client = instructor.from_provider("openai/gpt-5.4-mini")

# Extract structured data
user = client.create(
    response_model=User,
    messages=[
        {"role": "user", "content": "Extract the user: John Doe is 30 years old."}
    ]
)

print(user.name)  # "John Doe"
print(user.age)   # 30

Supported Providers

OpenAI

python
import instructor

client = instructor.from_provider("openai/gpt-4o-mini")

Available Modes:

  • Mode.TOOLS (default) - Uses OpenAI function calling
  • Mode.JSON - Uses JSON mode
  • Mode.MD_JSON - Uses Markdown JSON mode
  • Mode.FUNCTIONS - Uses legacy function calling

Anthropic

python
import instructor

client = instructor.from_provider("anthropic/claude-3-5-sonnet")

Available Modes:

  • Mode.ANTHROPIC_TOOLS (default) - Uses Claude tool calling
  • Mode.JSON - Uses JSON mode

Google (Gemini)

python
import instructor

client = instructor.from_provider("google/gemini-2.5-flash")

Available Modes:

  • Mode.GEMINI_JSON (default) - Generates JSON responses
  • Mode.GEMINI_TOOL - Uses Gemini's function calling

Mistral

python
import instructor

client = instructor.from_provider("mistral/mistral-large-latest")

Available Modes:

  • Mode.MISTRAL_TOOLS (default) - Uses tools mode
  • Mode.JSON - Uses JSON mode

Cohere

python
import instructor

client = instructor.from_provider("cohere/command-r-plus")

Available Modes:

  • Mode.COHERE_TOOL (default) - Uses Cohere's tool calling

Groq

python
import instructor

client = instructor.from_provider("groq/mixtral-8x7b-32768")

Available Modes:

  • Mode.TOOLS (default) - Uses function calling

Other Providers

Instructor supports many additional providers:

  • Azure OpenAI
  • Vertex AI
  • Fireworks
  • Cerebras
  • Writer
  • Anyscale
  • Databricks
  • Together
  • Perplexity
  • Ollama
  • OpenRouter
  • LiteLLM
  • llama-cpp-python

Key Features

Response Validation

Instructor automatically validates responses against your Pydantic models:

python
from pydantic import BaseModel, Field
import instructor
from openai import OpenAI

class UserWithValidation(BaseModel):
    name: str
    age: int = Field(gt=0, lt=150)  # Age must be between 0 and 150
    email: str = Field(pattern=r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$")

client = instructor.from_provider("openai/gpt-5.4-mini")

user = client.create(
    model="gpt-5.4-mini",
    response_model=UserWithValidation,
    messages=[
        {"role": "user", "content": "Extract the user: John Doe is 30 years old, email is [email protected]"}
    ]
)

If validation fails, instructor will automatically reattempt the request with error details.

Streaming Responses

Stream partial responses as they're generated:

python
import instructor
from pydantic import BaseModel

class Report(BaseModel):
    summary: str
    analysis: str
    recommendations: list[str]

client = instructor.from_provider("openai/gpt-5.4-mini")

# Enable streaming
for partial in client.create(
    model="gpt-5.4-mini",
    response_model=Report,
    stream=True,
    messages=[
        {"role": "user", "content": "Write a detailed report about renewable energy."}
    ]
):
    # Process each update
    print(f"Received update: {partial.model_dump_json()}")

# The final response has the complete model
print(f"Final report: {partial}")

Partial Streaming

Stream specific fields as they complete:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
from instructor.dsl import partial

class LongReport(BaseModel):
    executive_summary: str = partial()
    detailed_analysis: str = partial()
    conclusion: str = partial()

client = instructor.from_provider("openai/gpt-5.4-mini")

for chunk in client.create(
    model="gpt-5.4-mini",
    response_model=LongReport,
    stream=True,
    messages=[
        {"role": "user", "content": "Create a detailed report on climate change impacts."}
    ]
):
    # Each chunk will contain completed fields
    if hasattr(chunk, 'executive_summary') and chunk.executive_summary:
        print("Executive Summary Complete!")
    if hasattr(chunk, 'detailed_analysis') and chunk.detailed_analysis:
        print("Analysis Complete!")
    if hasattr(chunk, 'conclusion') and chunk.conclusion:
        print("Conclusion Complete!")

Iterables

Process multiple items efficiently:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
from instructor.dsl import iterable

class Person(BaseModel):
    name: str
    age: int

class PeopleList(BaseModel):
    people: list[Person] = iterable()

client = instructor.from_provider("openai/gpt-5.4-mini")

for person in client.create(
    model="gpt-5.4-mini",
    response_model=PeopleList,
    stream=True,
    messages=[
        {"role": "user", "content": "List 5 fictional characters with their ages."}
    ]
).people:
    print(f"Received: {person.name}, {person.age}")

Multimodal Support

Process images and other media:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
import base64

class ImageContent(BaseModel):
    objects: list[str]
    dominant_colors: list[str]

# Load image
with open("image.jpg", "rb") as image_file:
    base64_image = base64.b64encode(image_file.read()).decode('utf-8')

client = instructor.from_provider("openai/gpt-5.4-mini")

content = client.create(
    model="gpt-5.4-mini",
    response_model=ImageContent,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image in detail"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{base64_image}"
                    }
                }
            ]
        }
    ]
)

print(content.model_dump_json(indent=2))

Caching

Cache responses to improve performance and reduce API costs:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
import diskcache

# Create a cache
cache = diskcache.Cache("./my_cache_directory")

# Create client with caching
client = instructor.from_provider(
    "openai/gpt-5.4-mini",
    cache=cache
)

class Summary(BaseModel):
    points: list[str]

# This will use the cache if the same request was made before
summary = client.create(
    model="gpt-5.4-mini",
    response_model=Summary,
    messages=[
        {"role": "user", "content": "Summarize the key benefits of renewable energy."}
    ]
)

Hooks

Monitor and customize the processing flow:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
import json

class User(BaseModel):
    name: str
    age: int

# Define hooks
def log_prompt(prompt, **kwargs):
    print(f"PROMPT: {json.dumps(prompt)}")
    return prompt

def log_response(response, **kwargs):
    print(f"RESPONSE: {response}")
    return response

def log_parsed(parsed, **kwargs):
    print(f"PARSED: {parsed}")
    return parsed

# Apply hooks
client = instructor.from_provider(
    "openai/gpt-5.4-mini",
    mode=instructor.Mode.TOOLS,
    hooks={
        "prompt": log_prompt,
        "response": log_response, 
        "parsed": log_parsed
    }
)

user = client.create(
    model="gpt-5.4-mini",
    response_model=User,
    messages=[
        {"role": "user", "content": "Extract the user: John Doe is 30 years old."}
    ]
)

Retries and Error Handling

Handle validation failures with customizable retry logic:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel, Field

class StrictUser(BaseModel):
    name: str
    age: int = Field(gt=0, lt=150)
    email: str = Field(pattern=r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$")

# Configure max retries
client = instructor.from_provider(
    "openai/gpt-5.4-mini",
    max_retries=3  # Will retry up to 3 times if validation fails
)

try:
    user = client.create(
        model="gpt-5.4-mini",
        response_model=StrictUser,
        messages=[
            {"role": "user", "content": "Extract the user: John Doe is 30 years old."}
        ]
    )
except instructor.exceptions.ValidationError as e:
    print(f"Validation failed: {e}")

Advanced Usage

Parallel Processing

Process multiple tasks concurrently:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
from instructor.dsl.parallel import parallel

class Data(BaseModel):
    summary: str
    entities: list[str]
    sentiment: str

client = instructor.from_provider("openai/gpt-5.4-mini")

# Create parallel tasks
tasks = [
    {"text": "Apple announces new iPhone with revolutionary features."},
    {"text": "Climate scientists warn of increasing global temperatures."},
    {"text": "Stock market hits record high amid economic recovery."}
]

# Process in parallel
results = parallel(
    client=client,
    model="gpt-5.4-mini",
    response_model=Data,
    prompts=[
        [{"role": "user", "content": f"Analyze this text: {task['text']}"}]
        for task in tasks
    ],
    max_workers=3
)

for i, result in enumerate(results):
    print(f"Result {i+1}:")
    print(f"  Summary: {result.summary}")
    print(f"  Entities: {', '.join(result.entities)}")
    print(f"  Sentiment: {result.sentiment}")

Templating

Instructor supports Jinja templates directly in message content, automatically applying variables from the context parameter:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel

class Analysis(BaseModel):
    key_points: list[str]
    summary: str

client = instructor.from_provider("openai/gpt-5.4-mini")

# Context will be used to render templates in messages
analysis = client.create(
    model="gpt-5.4-mini",
    response_model=Analysis,
    messages=[
        {
            "role": "system", 
            "content": "You are an expert {{ analyst_type }} analyst."
        },
        {
            "role": "user", 
            "content": """
            Please analyze the following {{ document_type }}:
            
            {{ content }}
            
            Provide a detailed analysis.
            """
        }
    ],
    context={
        "analyst_type": "financial",
        "document_type": "news article",
        "content": "Renewable energy investments reached record levels in 2023..."
    }
)

print(f"Key points: {analysis.key_points}")
print(f"Summary: {analysis.summary}")

The templating system automatically processes all message content containing Jinja syntax ({{ variable }}, {% if condition %}, etc.) using the variables provided in the context parameter. This same context is also available to validators through info.context.

Maybe Responses

Handle uncertain responses gracefully:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
from instructor.dsl.maybe import Maybe

class Person(BaseModel):
    name: str
    age: int
    occupation: str

client = instructor.from_provider("openai/gpt-5.4-mini")

# Use Maybe to handle potential missing information
result = client.create(
    model="gpt-5.4-mini",
    response_model=Maybe[Person],
    messages=[
        {"role": "user", "content": "Extract info about Jane Doe who is 28 years old."}
    ]
)

if result.value:
    print(f"Name: {result.value.name}, Age: {result.value.age}")
    if hasattr(result.value, 'occupation'):
        print(f"Occupation: {result.value.occupation}")
    else:
        print("Occupation information not available")
else:
    print(f"Unable to extract person. Reason: {result.reason}")

Examples

Simple Extraction

python
import instructor
from openai import OpenAI
from pydantic import BaseModel

class Contact(BaseModel):
    name: str
    email: str
    phone: str

client = instructor.from_provider("openai/gpt-5.4-mini")

contact = client.create(
    model="gpt-5.4-mini",
    response_model=Contact,
    messages=[
        {"role": "user", "content": "My name is John Doe, email is [email protected] and phone is 555-123-4567"}
    ]
)

print(contact.model_dump_json(indent=2))

Classification

python
import instructor
from openai import OpenAI
from pydantic import BaseModel
from enum import Enum

class Sentiment(str, Enum):
    POSITIVE = "positive"
    NEGATIVE = "negative"
    NEUTRAL = "neutral"

class SentimentAnalysis(BaseModel):
    sentiment: Sentiment
    confidence: float
    explanation: str

client = instructor.from_provider("openai/gpt-5.4-mini")

analysis = client.create(
    model="gpt-5.4-mini",
    response_model=SentimentAnalysis,
    messages=[
        {"role": "user", "content": "I absolutely loved the new movie! It was fantastic!"}
    ]
)

print(f"Sentiment: {analysis.sentiment}")
print(f"Confidence: {analysis.confidence}")
print(f"Explanation: {analysis.explanation}")

Complex Schema

python
import instructor
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List, Optional
from datetime import datetime

class Address(BaseModel):
    street: str
    city: str
    state: str
    zip_code: str

class Experience(BaseModel):
    company: str
    position: str
    start_date: datetime
    end_date: Optional[datetime] = None
class Person(BaseModel):
    name: str
    age: int = Field(gt=0, lt=150)
    email: str
    phone: Optional[str] = None
    address: Address
    skills: List[str] = Field(min_items=1)
    experience: List[Experience] = Field(min_items=0)

client = instructor.from_provider("openai/gpt-5.4-mini")

person = client.create(
    model="gpt-5.4-mini",
    response_model=Person,
    messages=[
        {"role": "user", "content": """
        Extract information about Jane Smith who is 35 years old.
        Email: [email protected]
        Phone: 555-987-6543
        Address: 123 Main St, Springfield, IL 62701
        Skills: Python, Data Analysis, Machine Learning, Communication
        
        Work Experience:
        - Data Scientist at TechCorp (2019-01-15 to 2023-04-30)
          Led data science projects for major clients
        - Junior Analyst at DataFirm (2015-06-01 to 2018-12-15)
          Performed statistical analysis and created reports
        """}
    ]
)

print(person.model_dump_json(indent=2))

Vision and Multimodal

python
import instructor
from openai import OpenAI
from pydantic import BaseModel, Field
import base64
from typing import List

class Item(BaseModel):
    name: str
    price: float = Field(gt=0)
    quantity: int = Field(gt=0)

class Receipt(BaseModel):
    store_name: str
    date: str
    items: List[Item]
    subtotal: float
    tax: float
    total: float

client = instructor.from_provider("openai/gpt-5.4-mini")

# Load the receipt image
with open("receipt.jpg", "rb") as image_file:
    base64_image = base64.b64encode(image_file.read()).decode('utf-8')

receipt = client.create(
    model="gpt-5.4-mini",
    response_model=Receipt,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Extract all information from this receipt"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{base64_image}"
                    }
                }
            ]
        }
    ]
)

print(receipt.model_dump_json(indent=2))

Validation Context

Validation context allows you to pass additional contextual information to validators, enabling sophisticated validation that depends on external data:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel, field_validator, ValidationInfo

class CitationCheck(BaseModel):
    statement: str
    citation: str
    
    @field_validator('citation')
    def validate_citation(cls, citation: str, info: ValidationInfo) -> str:
        # Access the validation context
        source_text = info.context.get("source_document", "")
        
        # Check if the citation actually exists in the source document
        if citation not in source_text:
            raise ValueError(f"Citation '{citation}' not found in source document")
        return citation

client = instructor.from_provider("openai/gpt-5.4-mini")

source_document = "The Earth is the third planet from the Sun and the only astronomical object known to harbor life."

result = client.create(
    model="gpt-4o",
    response_model=CitationCheck,
    messages=[
        {"role": "user", "content": "Make a statement about Earth and provide a citation from the text."}
    ],
    context={"source_document": source_document}
)

print(f"Statement: {result.statement}")
print(f"Citation: {result.citation} (verified to exist in source)")

Validation context is particularly useful for:

  1. Citation validation: Ensuring quoted text exists in source documents
  2. Content moderation: Checking against banned word lists
  3. LLM-as-validator: Using one LLM to validate the output of another
  4. Reference data validation: Checking responses against reference data

Combined with Instructor's automatic reasking, validation context creates a powerful feedback loop:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel, field_validator, ValidationInfo

class RelevantAnswer(BaseModel):
    answer: str
    
    @field_validator('answer')
    def check_relevance(cls, answer: str, info: ValidationInfo) -> str:
        question = info.context.get("question", "")
        if "climate change" in question.lower() and "climate" not in answer.lower():
            raise ValueError("Answer doesn't address climate change as requested in the question")
        return answer

client = instructor.from_provider(
    "openai/gpt-5.4-mini",
    max_retries=2  # Will retry up to 2 times if validation fails
)

question = "What are the major impacts of climate change?"

result = client.create(
    model="gpt-5.4-mini",
    response_model=RelevantAnswer,
    messages=[
        {"role": "user", "content": """
        Answer the following question:

        <question>
        {{ question }}
        </question>
        """}
    ],
    context={"question": question}
)

print(result.answer)  # Guaranteed to mention climate change

This mechanism enables powerful templating through validation, where you can enforce that responses meet specific criteria or follow particular formats by providing the necessary context for validation.

Validation Context with Jinja Templating

Validation context can also be used directly in Jinja templates, creating a powerful combination where you can both template your prompts and validate responses against the same context:

text
/* Detailed source-code truncated for AI context efficiency. */

This approach creates a seamless flow where:

  1. The same context variables are used in your Jinja templates for prompt construction
  2. Those same variables are available to validators to ensure the LLM's response is faithful to the provided information
  3. If validation fails, Instructor will automatically retry with error details

This pattern is especially useful for:

  • RAG applications where you need to ensure responses are grounded in retrieved documents
  • Q&A systems where answers must be factually consistent with provided context
  • Any scenario where you want to template prompts and validate responses against the same data

This guide covers the core features and usage patterns of the Instructor library. For more detailed examples and advanced use cases, refer to the official documentation.

Modes Comparison (`docs/modes-comparison.md`)

title: Mode Comparison Guide description: Compare different modes available in Instructor and understand when to use each

Instructor Mode Comparison Guide

Instructor uses core modes that work across providers. Provider-specific
modes still work, but they are deprecated and will show warnings.

Mode handling now lives in provider handlers. The DSL no longer stores
mode-specific streaming logic.

Core Modes

  • TOOLS: Tool or function calling for structured extraction.
  • JSON_SCHEMA: Native schema support when a provider has it.
  • MD_JSON: JSON from text or code blocks for simple or fallback cases.
  • PARALLEL_TOOLS: Multiple tool calls in one response.
  • RESPONSES_TOOLS: OpenAI Responses API tools.

Legacy Modes (Deprecated)

These legacy modes map to core modes:

  • FUNCTIONS -> TOOLS
  • TOOLS_STRICT -> TOOLS
  • ANTHROPIC_TOOLS -> TOOLS
  • ANTHROPIC_JSON -> MD_JSON
  • GENAI_TOOLS -> TOOLS
  • GENAI_JSON -> JSON
  • MISTRAL_TOOLS -> TOOLS
  • MISTRAL_STRUCTURED_OUTPUTS -> JSON_SCHEMA
  • BEDROCK_TOOLS -> TOOLS
  • BEDROCK_JSON -> MD_JSON
  • FIREWORKS_TOOLS -> TOOLS
  • FIREWORKS_JSON -> MD_JSON
  • CEREBRAS_TOOLS -> TOOLS
  • CEREBRAS_JSON -> MD_JSON
  • WRITER_TOOLS -> TOOLS
  • WRITER_JSON -> MD_JSON
  • PERPLEXITY_JSON -> MD_JSON
  • VERTEXAI_TOOLS -> TOOLS
  • VERTEXAI_JSON -> MD_JSON
  • VERTEXAI_PARALLEL_TOOLS -> PARALLEL_TOOLS

Mode Selection Tips

  • Use TOOLS for most structured output cases.
  • Use JSON_SCHEMA when the provider supports native schema enforcement.
  • Use MD_JSON if tools are not supported or outputs are simple.
  • Use PARALLEL_TOOLS for multiple tasks in one response.

Examples

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "openai/gpt-4o-mini",
    mode=Mode.TOOLS,
)

MD_JSON Mode (Fallback)

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
    mode=Mode.MD_JSON,
)

JSON_SCHEMA Mode (Native Schema)

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "openai/gpt-4o-mini",
    mode=Mode.JSON_SCHEMA,
)

See the Mode Migration Guide for more details.

Google/Gemini

For complex structures:

python
import instructor

client = instructor.from_provider(
    "google/gemini-2.5-flash",
    mode=instructor.Mode.TOOLS,
)

For structured outputs with JSON:

python
import instructor

client = instructor.from_provider(
    "google/gemini-2.5-flash",
    mode=instructor.Mode.JSON,
)

Mode Compatibility List

Legacy modes are shown for compatibility only. Prefer core modes in new code.

  • OpenAI: TOOLS, TOOLS_STRICT, PARALLEL_TOOLS, FUNCTIONS; JSON, MD_JSON, JSON_O1.
  • Anthropic: ANTHROPIC_TOOLS, ANTHROPIC_PARALLEL_TOOLS; ANTHROPIC_JSON.
  • Gemini: TOOLS; JSON.
  • Vertex AI: VERTEXAI_TOOLS; VERTEXAI_JSON.
  • Cohere: COHERE_TOOLS; JSON, MD_JSON.
  • Mistral: MISTRAL_TOOLS; MISTRAL_STRUCTURED_OUTPUTS.
  • Anyscale: (none); JSON, MD_JSON, JSON_SCHEMA.
  • Databricks: TOOLS; JSON, MD_JSON.
  • Together: (none); JSON, MD_JSON.
  • Fireworks: FIREWORKS_TOOLS; FIREWORKS_JSON.
  • Cerebras: (none); CEREBRAS_JSON.
  • Writer: WRITER_TOOLS; JSON.
  • Perplexity: (none); PERPLEXITY_JSON.
  • GenAI: TOOLS; JSON.
  • LiteLLM: depends on provider for both tool-based and JSON-based modes.

Best Practices

  1. Start with the recommended mode for your provider

    • For OpenAI: TOOLS
    • For Anthropic: ANTHROPIC_TOOLS (Claude 3+) or ANTHROPIC_JSON
    • For Gemini: TOOLS or JSON

  2. Try JSON modes for simple structures or if you encounter issues

    • JSON modes often work with simpler schemas
    • They may be more token-efficient
    • They work with more models

  3. Use provider-specific modes when available

    • Provider-specific modes are optimized for that provider
    • They handle special cases and requirements

  4. Test and validate

    • Different modes may perform differently for your specific use case
    • Always test with your actual data and models

Newsletter (`docs/newsletter.md`)

title: Subscribe to Instructor Newsletter for AI Updates description: Get notified about AI tips, blog posts, and research. Stay informed with Instructor's latest features and community insights.

Instructor Newsletter

If you want to be notified of tips, new blog posts, and research, subscribe to our newsletter. Here's what you can expect:

  • Updates on Instructor features and releases
  • Blog posts on AI and structured outputs
  • Tips and tricks from our community
  • Research in the field of LLMs and structured outputs
  • Information on AI development skills with Instructor

Subscribe to our newsletter for updates on AI development. We provide content to keep you informed and help you use Instructor in projects.

Start Here (`docs/start-here.md`)

title: Start Here - Instructor for Beginners description: A beginner-friendly introduction to using Instructor for structured outputs from LLMs

Start Here: Instructor for Beginners

Welcome! This guide will help you understand what Instructor does and how to start using it in your projects, even if you're new to working with language models.

What is Instructor?

Instructor is a Python library that helps you get structured, predictable data from language models like GPT-4 and Claude. It's like giving the LLM a form to fill out instead of letting it respond however it wants.

Where Instructor Fits

Here's how Instructor fits into your application:

mermaid
flowchart LR
    A[Your Application] --> B[Instructor]
    B --> C[LLM Provider]
    C --> B
    B --> A

    style B fill:#e2f0fb,stroke:#b8daff,color:#004085

The Problem Instructor Solves

Without Instructor, getting structured data from LLMs can be challenging:

  1. Unpredictable outputs: LLMs might format responses differently each time
  2. Format errors: Getting JSON or specific data structures can be error-prone
  3. Validation headaches: Checking if the response matches what you need

Instructor solves these problems by:

  1. Defining exactly what data you want using Python classes
  2. Making sure the LLM returns data in that structure
  3. Validating the output and automatically fixing issues

A Simple Example

Let's see Instructor in action with a basic example:

python
import instructor
from pydantic import BaseModel

# Define the structure you want
class Person(BaseModel):
    name: str
    age: int
    city: str

# Connect to the LLM with Instructor
client = instructor.from_provider("openai/gpt-4o-mini")

# Extract structured data
person = client.create(
    response_model=Person,
    messages=[
        {"role": "user", "content": "Extract: John is 30 years old and lives in New York."}
    ]
)

# Now you have a structured object
print(f"Name: {person.name}")  # Name: John
print(f"Age: {person.age}")    # Age: 30
print(f"City: {person.city}")  # City: New York

That's it! Instructor handled all the complexity of getting the LLM to format the data correctly.

Ready to get started? Follow our step-by-step guide →

Key Concepts

Here are the main concepts you need to know:

1. Response Models

Response models define the structure you want the LLM to return. They are built using Pydantic, which is a data validation library.

python
from pydantic import BaseModel, Field

class User(BaseModel):
    name: str = Field(description="The user's full name")
    age: int = Field(description="The user's age in years")
    # The descriptions help the LLM understand what to extract

2. Client Setup

The from_provider function connects Instructor to your LLM provider. It automatically handles provider-specific configurations:

python
# For OpenAI
client = instructor.from_provider("openai/gpt-4o-mini")

# For Anthropic
client = instructor.from_provider("anthropic/claude-3-5-haiku-latest")

# For Google Gemini
client = instructor.from_provider("google/gemini-3-flash")

3. Modes

Modes control how Instructor gets structured data from the LLM. Different providers support different modes, and Instructor automatically selects the best one. You can also specify a mode manually if needed.

Learn more about client setup →

Common Use Cases

Here are some popular ways people use Instructor:

  1. Data extraction: Pull structured information from text documents
  2. Form filling: Convert free-text into form fields
  3. Classification: Sort content into predefined categories
  4. Content generation: Create structured content like articles or product descriptions
  5. API integration: Format LLM outputs to match API requirements

Next Steps

Now that you understand the basics, here are some suggested next steps:

  1. Try the Getting Started Guide for a more in-depth tutorial
  2. Explore the Cookbook Examples for practical use cases
  3. Learn about Validation to ensure data quality
  4. Check out Streaming for handling large responses
  5. Understand Providers to use different LLM services

Common Questions

Do I need to understand Pydantic?

While knowing Pydantic helps, you don't need to be an expert. The basic patterns shown above will get you started. You can learn more advanced features as you need them.

Which LLM provider should I use?

OpenAI is the most popular choice for beginners because of its reliability and wide support. As you grow more comfortable, you can explore other providers like Anthropic Claude, Gemini, or open-source models.

Is Instructor hard to learn?

No! If you're familiar with Python classes and working with APIs, you'll find Instructor straightforward. The core concepts are simple, and you can gradually explore advanced features.

How does Instructor compare to other libraries?

Instructor focuses specifically on structured outputs with a simple, clean API. Unlike larger frameworks that try to do everything, Instructor does one thing very well: getting structured data from LLMs.

Getting Help

If you get stuck:

Welcome aboard, and happy extracting!

Why (`docs/why.md`)

description: Discover why Instructor is the simplest, most reliable way to get structured outputs from LLMs.

Why use Instructor?

You've built something with an LLM, but 15% of the time it returns garbage. Parsing JSON is a nightmare. Different providers have different APIs. There has to be a better way.

The pain of unstructured outputs

Let's be honest about what working with LLMs is really like:

python
# What you want:
user_info = extract_user("John is 25 years old")
print(user_info.name)  # "John"
print(user_info.age)   # 25

# What you actually get:
response = llm.complete("Extract: John is 25 years old")
# "I'd be happy to help! Based on the text, the user's name is John
# and their age is 25. Is there anything else you'd like me to extract?"

# Now you need to:
# 1. Parse this text somehow
# 2. Handle when it returns JSON with syntax errors
# 3. Validate the data matches what you expect
# 4. Retry when it fails (which it will)
# 5. Do this differently for each LLM provider

The Instructor difference

Here's the same task with Instructor:

python
import instructor
from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

client = instructor.from_provider("openai/gpt-5.4-mini")
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "John is 25 years old"}],
)

print(user.name)  # "John"
print(user.age)   # 25

That's it. No parsing. No retries. No provider-specific code.

Real problems Instructor solves

1. "It works 90% of the time"

Without Instructor, your LLM returns perfect JSON most of the time. But that 10% will ruin your weekend.

python
# Without Instructor: Brittle code that breaks randomly
try:
    data = json.loads(llm_response)
    user = User(**data)  # KeyError: 'name'
except:
    # Now what? Retry? Parse the text? Give up?
    pass

# With Instructor: Automatic retries with validation errors
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
    max_retries=3,  # Retries with validation errors
)
# Always returns valid User object or raises clear exception

2. "Each provider is different"

Every LLM provider has its own API. Your code becomes a mess of conditionals.

python
# Without Instructor: Provider-specific spaghetti
if provider == "openai":
    response = openai.chat.completions.create(
        tools=[{"type": "function", "function": {...}}]
    )
    data = json.loads(response.choices[0].message.tool_calls[0].function.arguments)
elif provider == "anthropic":
    response = anthropic.messages.create(
        tools=[{"name": "extract", "input_schema": {...}}]
    )
    data = response.content[0].input
elif provider == "google":
    # ... different API again

# With Instructor: One API for all providers
client = instructor.from_provider("openai/gpt-5.4-mini")
# or
client = instructor.from_provider("anthropic/claude-3")
# or
client = instructor.from_provider("google/gemini-pro")

# Same code for all providers
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

3. "Complex data structures are impossible"

Nested objects, lists, enums - LLMs struggle with complex schemas.

python
# Without Instructor: Good luck with this
schema = {
    "type": "object",
    "properties": {
        "users": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "addresses": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "street": {"type": "string"},
                                "city": {"type": "string"}
                            }
                        }
                    }
                }
            }
        }
    }
}

# With Instructor: Just use Python
from typing import List

class Address(BaseModel):
    street: str
    city: str

class User(BaseModel):
    name: str
    addresses: List[Address]

class UserList(BaseModel):
    users: List[User]

# Works perfectly
result = client.create(
    response_model=UserList,
    messages=[{"role": "user", "content": "..."}],
)

The cost of not using Instructor

Let's talk real numbers:

Time wasted:

  • 2-3 hours implementing JSON parsing and validation
  • 4-6 hours debugging edge cases
  • 2-3 hours for each new provider you add
  • Ongoing maintenance as APIs change

Bugs in production:

  • Malformed JSON crashes your app
  • Missing fields cause silent failures
  • Type mismatches corrupt your database
  • Customer complaints about reliability

Developer frustration:

  • "It worked in testing!"
  • "Why is the JSON different this time?"
  • "How do I handle when it returns a string instead of a number?"

What developers are saying

Based on our GitHub issues and Discord:

  • "Reduced our LLM code by 80%" - Common feedback
  • "Finally, LLM outputs I can trust" - From production users
  • "The retries alone are worth it" - Saves hours of edge-case handling
  • "Works exactly the same with every provider" - No more provider lock-in

Start now, thank yourself later

Every day without Instructor is another day of:

  • Debugging malformed JSON
  • Writing provider-specific code
  • Handling validation manually
  • Explaining to your PM why the LLM integration is flaky

Install Instructor:

bash
pip install instructor

Try it in 30 seconds:

python
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-5.4-mini")

class User(BaseModel):
    name: str
    age: int

user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "John is 25 years old"}],
)

print(user)  # User(name='John', age=25)

When NOT to use Instructor

Let's be clear - you might not need Instructor if:

  • You only need raw text responses (chatbots, creative writing)
  • You're building a one-off script with no error handling
  • You enjoy debugging JSON parsing errors at 3am

For everyone else building production LLM applications, Instructor is the obvious choice.

Get Started →{ .md-button .md-button--primary }

Cli Batch (`docs/cli/batch.md`)

title: Managing Batch Jobs with Multi-Provider CLI description: Learn how to create, list, cancel, and delete batch jobs using the unified Command Line Interface (CLI) across OpenAI and Anthropic providers.

Using the Command Line Interface for Batch Jobs

The instructor CLI provides comprehensive functionalities for managing batch jobs across multiple providers with a unified interface. This multi-provider support allows users to leverage the strengths of different AI providers for their batch processing needs.

Supported Providers

  • OpenAI: Utilizes OpenAI's robust batch processing capabilities with metadata support
  • Anthropic: Leverages Anthropic's advanced language models with cancel/delete operations

The CLI uses a unified --provider flag for all commands, with backward compatibility for legacy flags.

bash
$ instructor batch --help

 Usage: instructor batch [OPTIONS] COMMAND [ARGS]...

 Manage OpenAI Batch jobs

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                  │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ cancel             Cancel a batch job                                        │
│ create             Create batch job using BatchProcessor                     │
│ create-from-file   Create a batch job from a file                            │
│ delete             Delete a completed batch job                              │
│ download-file      Download the file associated with a batch job             │
│ list               See all existing batch jobs                               │
│ results            Retrieve results from a batch job                         │
╰──────────────────────────────────────────────────────────────────────────────╯

Creating a Batch Job

List Jobs with Enhanced Display

text
/* Detailed source-code truncated for AI context efficiency. */

The enhanced list command now shows rich information including timestamps, duration, and provider-specific metrics:

text
/* Detailed source-code truncated for AI context efficiency. */

Create From File with Metadata Support

You can create batch jobs directly from pre-formatted .jsonl files with enhanced metadata support:

text
/* Detailed source-code truncated for AI context efficiency. */

Example usage with metadata:

bash
# OpenAI batch with custom metadata
instructor batch create-from-file \
    --file-path batch_requests.jsonl \
    --model "openai/gpt-5-nano" \
    --description "Email classification batch - production v2.1" \
    --completion-window "24h"

# Anthropic batch
instructor batch create-from-file \
    --file-path batch_requests.jsonl \
    --model "anthropic/claude-3-5-sonnet-20241022" \
    --description "Text analysis batch"

For creating .jsonl files, you can use the enhanced BatchProcessor:

python
from instructor.batch import BatchProcessor
from pydantic import BaseModel, Field
from typing import Literal

class Classification(BaseModel):
    label: Literal["SPAM", "NOT_SPAM"] = Field(
        ..., description="Whether the email is spam or not"
    )

# Create processor
processor = BatchProcessor("openai/gpt-5-nano", Classification)

# Prepare message conversations
messages_list = [
    [
        {"role": "system", "content": "Classify the following email"},
        {"role": "user", "content": "Hello there I'm a Nigerian prince and I want to give you money"}
    ],
    [
        {"role": "system", "content": "Classify the following email"},
        {"role": "user", "content": "Meeting with Thomas has been set at Friday next week"}
    ]
]

# Create batch file
processor.create_batch_from_messages(
    messages_list=messages_list,
    file_path="batch_requests.jsonl",
    max_tokens=100,
    temperature=0.1
)

Job Management Operations

Cancelling a Batch Job

Cancel running batch jobs across all providers:

bash
$ instructor batch cancel --help

 Usage: instructor batch cancel [OPTIONS]

 Cancel a batch job

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ *  --batch-id                               TEXT  Batch job ID to cancel     │
│                                                   [default: None]            │
│                                                   [required]                 │
│    --provider                               TEXT  Provider to use (e.g.,     │
│                                                   'openai', 'anthropic')     │
│                                                   [default: openai]          │
│    --use-anthropic    --no-use-anthropic          [DEPRECATED] Use           │
│                                                   --provider 'anthropic'     │
│                                                   instead. Use Anthropic API │
│                                                   instead of OpenAI          │
│                                                   [default:                  │
│                                                   no-use-anthropic]          │
│    --help                                         Show this message and      │
│                                                   exit.                      │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples:

bash
# Cancel OpenAI batch
instructor batch cancel --batch-id batch_abc123 --provider openai

# Cancel Anthropic batch
instructor batch cancel --batch-id msgbatch_def456 --provider anthropic

Deleting a Batch Job

Delete completed batch jobs (supported by Anthropic):

bash
$ instructor batch delete --help

 Usage: instructor batch delete [OPTIONS]

 Delete a completed batch job

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ *  --batch-id        TEXT  Batch job ID to delete [default: None] [required] │
│    --provider        TEXT  Provider to use (e.g., 'openai', 'anthropic')     │
│                            [default: openai]                                 │
│    --help                  Show this message and exit.                       │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples:

bash
# Delete Anthropic batch (supported)
instructor batch delete --batch-id msgbatch_abc123 --provider anthropic

# Try to delete OpenAI batch (shows helpful message)
instructor batch delete --batch-id batch_ghi789 --provider openai
# Note: OpenAI does not support batch deletion via API

Retrieving Batch Results

Get structured results from completed batch jobs:

bash
$ instructor batch results --help

 Usage: instructor batch results [OPTIONS]

 Retrieve results from a batch job

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ *  --batch-id           TEXT  Batch job ID to get results from               │
│                               [default: None]                                │
│                               [required]                                     │
│ *  --output-file        TEXT  File to save the results to [default: None]    │
│                               [required]                                     │
│    --model              TEXT  Model in format 'provider/model-name' (e.g.,   │
│                               'openai/gpt-5.4-mini', 'anthropic/claude-3-sonnet')   │
│                               [default: openai/gpt-4o-mini]                  │
│    --help                     Show this message and exit.                    │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples:

bash
# Get OpenAI batch results
instructor batch results \
    --batch-id batch_abc123 \
    --output-file openai_results.jsonl \
    --model "openai/gpt-5-nano"

# Get Anthropic batch results
instructor batch results \
    --batch-id msgbatch_def456 \
    --output-file anthropic_results.jsonl \
    --model "anthropic/claude-3-5-sonnet-20241022"

Downloading Raw Files (Legacy)

For compatibility, the download-file command is still available:

bash
$ instructor batch download-file --help

 Usage: instructor batch download-file [OPTIONS]

 Download the file associated with a batch job

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ *  --batch-id                  TEXT  Batch job ID to download                │
│                                      [default: None]                         │
│                                      [required]                              │
│ *  --download-file-path        TEXT  Path to download file to                │
│                                      [default: None]                         │
│                                      [required]                              │
│    --provider                  TEXT  Provider to use (e.g., 'openai',        │
│                                      'anthropic')                            │
│                                      [default: openai]                       │
│    --help                            Show this message and exit.             │
╰──────────────────────────────────────────────────────────────────────────────╯

Provider Support Matrix

Operation OpenAI Anthropic
List ✅ Enhanced table ✅ Enhanced table
Create ✅ With metadata ✅ File-based
Cancel ✅ Standard API ✅ Standard API
Delete ❌ Not supported ✅ Standard API
Results ✅ Structured parsing ✅ Structured parsing

Enhanced Features

  • Rich CLI Tables: Color-coded status, timestamps, duration calculations
  • Metadata Support: Add descriptions and custom fields to organize batches
  • Unified Commands: Same interface works across all providers
  • Provider Detection: Automatic provider detection from model strings
  • Error Handling: Clear error messages and helpful notes for unsupported operations
  • Backward Compatibility: Legacy flags still work with deprecation warnings

This comprehensive CLI interface provides efficient batch job management across all supported providers with enhanced monitoring and control capabilities.

Cli Finetune (`docs/cli/finetune.md`)

title: Managing Fine-Tuning Jobs with the Instructor CLI description: Learn how to create, view, and manage fine-tuning jobs on OpenAI using the Instructor CLI, with essential commands and options.

Using the Command Line Interface

The instructor CLI provides functionalities for managing fine-tuning jobs on OpenAI.

!!! warning "Incomplete API"
The CLI is still under development and does not yet support all features of the API. If you would like to use a feature that is not yet supported, please consider using the contributing to our library jxnl/instructor instead.

text
!!! note "Low hanging fruit"

    If you want to contribute we're looking for a few things:

    1. Adding filenames on upload

Creating a Fine-Tuning Job

View Jobs Options

sh
$ instructor jobs --help

 Usage: instructor jobs [OPTIONS] COMMAND [ARGS]...

 Monitor and create fine tuning jobs

╭─ Options ───────────────────────────────────────────────────────────────────────────────╮
│ --help                            Display the help message.                             │
╰─────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ cancel                    Cancel a fine-tuning job.                                                         │
│ create-from-file          Create a fine-tuning job from a file.                                             │
│ create-from-id            Create a fine-tuning job from an existing ID.                                     │
│ list                      Monitor the status of the most recent fine-tuning jobs.                           │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Create from File

The create-from-file command uploads and trains a model in a single step.

sh
❯ instructor jobs create-from-file --help

Usage: instructor jobs create-from-file [OPTIONS] FILE

 Create a fine-tuning job from a file.

╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────╮
│ *    file      TEXT  Path to the file for fine-tuning [default: None] [required]                  │
╰───────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────╮
│ --model                           TEXT     Model to use for fine-tuning [default: gpt-5.4-mini]  │
│ --poll                            INTEGER  Polling interval in seconds [default: 2]               │
│ --n-epochs                        INTEGER  Number of epochs for fine-tuning                       │
│ --batch-size                      TEXT     Batch size for fine-tuning                             │
│ --learning-rate-multiplier        TEXT     Learning rate multiplier for fine-tuning               │
│ --validation-file                 TEXT     Path to the validation file [default: None]            │
│ --model-suffix                    TEXT     Suffix to identify the model [default: None]           │
│ --help                                     Show this message and exit.                            │
╰────────────────────────────────────────────────────────────────────────────────

Usage

sh
$ instructor jobs create-from-file transformed_data.jsonl --validation_file validation_data.jsonl --n_epochs 3 --batch_size 16 --learning_rate_multiplier 0.5

Create from ID

The create-from-id command uses an uploaded file and trains a model

sh
❯ instructor jobs create-from-id --help

 Usage: instructor jobs create-from-id [OPTIONS] ID

 Create a fine-tuning job from an existing ID.

╭─ Arguments ───────────────────────────────────────────────────────────────────────────╮
│ *    id      TEXT  ID of the existing fine-tuning job [default: None] [required]      │
╰───────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────╮
│ --model                           TEXT     Model to use for fine-tuning               │
│                                            [default: gpt-5.4-mini]                   │
│ --n-epochs                        INTEGER  Number of epochs for fine-tuning           │
│ --batch-size                      TEXT     Batch size for fine-tuning                 │
│ --learning-rate-multiplier        TEXT     Learning rate multiplier for fine-tuning   │
│ --validation-file-id              TEXT     ID of the uploaded validation file         │
│                                            [default: None]                            │
│ --help                                     Show this message and exit.                │
╰───────────────────────────────────────────────────────────────────────────────────────╯

Usage

sh
$ instructor files upload transformed_data.jsonl
$ instructor files upload validation_data.jsonl
$ instructor files list
...
$ instructor jobs create_from_id <file_id> --validation_file <validation_file_id> --n_epochs 3 --batch_size 16 --learning_rate_multiplier 0.5

Viewing Files and Jobs

Viewing Jobs

text
/* Detailed source-code truncated for AI context efficiency. */

Viewing Files

sh
$ instructor files list

OpenAI Files
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┓
┃ File ID                       ┃ Size (bytes) ┃ Creation Time       ┃ Filename ┃ Purpose   ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━┩
│ file-0lw2BSNRUlXZXRRu2beCCWjl │       369523 │ 2023-08-23 23:31:57 │ file     │ fine-tune │
│ file-IHaUXcMEykmFUp1kt2puCDEq │       369523 │ 2023-08-23 23:09:35 │ file     │ fine-tune │
│ file-ja9vRBf0FydEOTolaa3BMqES │       369523 │ 2023-08-23 22:42:29 │ file     │ fine-tune │
│ file-F7lJg6Z47CREvmx4kyvyZ6Sn │       369523 │ 2023-08-23 22:42:03 │ file     │ fine-tune │
│ file-YUxqZPyJRl5GJCUTw3cNmA46 │       369523 │ 2023-08-23 22:29:10 │ file     │ fine-tune │
└───────────────────────────────┴──────────────┴─────────────────────┴──────────┴───────────┘

Contributions

We aim to provide a light wrapper around the API rather than offering a complete CLI. Contributions are welcome! Please feel free to make an issue at jxnl/instructor/issues or submit a pull request.

Cli Usage (`docs/cli/usage.md`)

title: OpenAI API Usage CLI Guide description: Learn how to monitor OpenAI API usage with the CLI tool, including commands for viewing data by model, date, and cost.

Using the OpenAI API Usage CLI

The OpenAI API Usage CLI tool provides functionalities for monitoring your OpenAI API usage, breaking it down by model, date, and cost.

!!! warning "Legacy estimates"
This command uses a legacy OpenAI usage endpoint and a static model-price
table. It does not reconcile invoices or account for current cache, tool,
modality, batch, or service-tier pricing. Model-prefix fallback can also
apply the wrong rate to newer model variants. An empty result does not
establish zero usage. Use provider billing records for financial reporting.

Monitoring API Usage

View Usage Options

sh
$ instructor usage --help

 Usage: instructor usage [OPTIONS] COMMAND [ARGS]...

 Check OpenAI API usage data

╭─ Options ───────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                     │
╰─────────────────────────────────────────────────────────────────╯
╭─ Commands ──────────────────────────────────────────────────────╮
│ list       Displays OpenAI API usage data for the past N days.  │
╰─────────────────────────────────────────────────────────────────╯

List Usage for Specific Number of Days

To display API usage for the past 3 days, use the following command:

sh
$ instructor usage list --n 3

This will output a table similar to:

plaintext
Usage Summary by Date, Snapshot, and Cost
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Date       ┃ Snapshot ID               ┃ Total Requests ┃ Total Cost ($) ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ 2023-09-04 │ gpt-5.4-mini                │             44 │           0.68 │
│ 2023-09-04 │ gpt-5.4-mini    │            195 │           0.84 │
│ 2023-09-04 │ text-embedding-ada-002-v2 │            276 │           0.00 │
│ 2023-09-04 │ gpt-4-32k-0613            │            328 │          49.45 │
└────────────┴───────────────────────────┴────────────────┴────────────────┘

List Usage for Today

To display the API usage for today, simply run:

sh
$ instructor usage list

Contributions

We aim to provide a light wrapper around the API rather than offering a complete CLI. Contributions are welcome! Please feel free to make an issue at jxnl/instructor/issues or submit a pull request.

Concepts Alias (`docs/concepts/alias.md`)

title: Pydantic Aliases Overview description: Explore the concept of aliases in Pydantic. Discover the latest documentation and features for better data validation.

See Also

!!! warning "This page is a work in progress"

text
This page is a work in progress. Check out [Pydantic's documentation](https://docs.pydantic.dev/latest/concepts/alias/)

Concepts Batch (`docs/concepts/batch.md`)

title: Batch Processing description: Process multiple LLM requests efficiently using batch processing for 50% cost savings.

Batch Processing

Batch processing lets you send multiple requests in a single operation, saving up to 50% on costs. Instructor supports batch processing across multiple providers.

Supported Providers

Provider Models Cost Savings
OpenAI gpt-4o, gpt-4.1-mini, gpt-5.4-mini 50%
Anthropic claude-3-5-sonnet, claude-3-opus, claude-3-haiku 50%
Google GenAI gemini-2.5-flash, gemini-2.0-flash, gemini-pro 50%

Basic Usage

python
from instructor.batch import BatchProcessor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


processor = BatchProcessor("openai/gpt-4.1-mini", User)

messages_list = [
    [
        {"role": "system", "content": "Extract user information from text."},
        {"role": "user", "content": "Hi, I'm Alice and I'm 28 years old."},
    ],
    [
        {"role": "system", "content": "Extract user information from text."},
        {"role": "user", "content": "Hello, I'm Bob, 35 years old."},
    ],
]

# Create batch file
processor.create_batch_from_messages(
    file_path="batch_requests.jsonl",
    messages_list=messages_list,
    max_tokens=200,
    temperature=0.1,
)

# Submit batch job
batch_id = processor.submit_batch("batch_requests.jsonl")
print(f"Batch job submitted: {batch_id}")

# Check status and retrieve results
status = processor.get_batch_status(batch_id)
if status["status"] in ["completed", "ended", "JOB_STATE_SUCCEEDED"]:
    from instructor.batch import extract_results

    all_results = processor.retrieve_results(batch_id)
    for user in extract_results(all_results):
        print(f"Name: {user.name}, Age: {user.age}")

In-Memory Processing

For serverless deployments, use in-memory mode by setting file_path=None:

python
import time
from instructor.batch import BatchProcessor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


processor = BatchProcessor("openai/gpt-4.1-mini", User)

messages_list = [
    [{"role": "user", "content": "Extract: John is 25 years old"}],
    [{"role": "user", "content": "Extract: Jane is 30 years old"}],
]

# Create in-memory buffer (no file_path)
buffer = processor.create_batch_from_messages(
    messages_list,
    file_path=None,
    max_tokens=150,
)

# Submit and poll for results
batch_id = processor.submit_batch(buffer)

while True:
    status = processor.get_batch_status(batch_id)
    if status.get("status") in ["completed", "failed", "cancelled"]:
        break
    time.sleep(10)

if status.get("status") == "completed":
    results = processor.get_results(batch_id)
    for r in results:
        if hasattr(r, "result"):
            print(f"{r.result.name}, {r.result.age}")

When to Use Each Approach

Use Case Approach
Serverless (Lambda, Cloud Functions) In-memory
Large batch jobs File-based
Security-sensitive environments In-memory
Debugging/audit requirements File-based

Provider Setup

OpenAI

bash
export OPENAI_API_KEY="your-openai-key"
python
processor = BatchProcessor("openai/gpt-4.1-mini", User)

Anthropic

bash
export ANTHROPIC_API_KEY="your-anthropic-key"
python
processor = BatchProcessor("anthropic/claude-3-5-sonnet-20241022", User)

Google GenAI

bash
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GCS_BUCKET="your-gcs-bucket-name"
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
python
processor = BatchProcessor("google/gemini-2.5-flash", User)

Required permissions: roles/aiplatform.user and roles/storage.objectUser.

Processing Results

OpenAI batch requests use strict JSON schemas. All named fields are required in
the output, even when the Pydantic model has defaults. Nullable fields still
allow null. Arbitrary-key dictionaries are not supported by this strict batch
path; use named model fields or a list of key/value objects instead. Anthropic
batch requests keep their provider-specific tool schema, including mapping
value types.

Results use a Maybe/Result pattern for type-safe handling:

For completed OpenAI batches, retrieval reads both the output and error files.
For ended Anthropic batches, it reads every result, including errors. A batch
where every request failed returns BatchError records rather than raising
because no requests succeeded. Errors retain the request's custom_id, provider
error details, and original record in raw_data. Jobs that are still running
cannot be retrieved.

python
from instructor.batch import (
    filter_successful,
    filter_errors,
    extract_results,
    get_results_by_custom_id,
)

all_results = processor.retrieve_results(batch_id)

# Filter by type
successful = filter_successful(all_results)  # List[BatchSuccess[T]]
errors = filter_errors(all_results)  # List[BatchError]
objects = extract_results(all_results)  # List[T]

# Access by custom_id
by_id = get_results_by_custom_id(all_results)
if "request-1" in by_id:
    result = by_id["request-1"]
    if result.success:
        print(f"Success: {result.result}")
    else:
        print(f"Error: {result.error_message}")

Custom IDs

By default, requests get the IDs request-0, request-1, and so on. Pass custom_ids to use your own IDs. This makes it easier to match results, including failed ones, back to your data:

python
processor.create_batch_from_messages(
    messages_list,
    file_path="batch_requests.jsonl",
    custom_ids=["user_1", "user_2", "user_3"],  # one per conversation, same order
)

IDs must be unique and match the length of messages_list. Anthropic also requires IDs to match ^[a-zA-Z0-9_-]{1,64}$. Invalid IDs raise a ValueError before any request is written.

API Reference

Method Description
create_batch_from_messages(messages_list, file_path=None, ..., custom_ids=None) Create batch file or buffer
submit_batch(file_path_or_buffer, metadata=None) Submit batch job, returns job ID
get_batch_status(batch_id) Get job status
retrieve_results(batch_id) Download and parse results
parse_results(content) Parse raw results content

CLI Commands

bash
# List batch jobs
instructor batch list --model "openai/gpt-4.1-mini"

# Create batch from file
instructor batch create-from-file --file-path batch.jsonl --model "openai/gpt-4.1-mini"

# Get batch results
instructor batch results --batch-id "batch_abc123" --output-file results.jsonl

Best Practices

  1. Batch size: Include at least 25,000 requests per job for optimal efficiency
  2. Cost optimization: Use batch processing for non-urgent workloads
  3. Error handling: Always check both successful and error results
  4. Timeouts: Batch jobs have execution limits (24 hours for Google)
  5. Storage: For Google, ensure GCS bucket is in the same region as your batch job

Troubleshooting

Issue Solution
Missing GCS_BUCKET (Google) Set the GCS_BUCKET environment variable
Permission Denied (Google) Add aiplatform.user and storage.objectUser roles
Invalid Model Name Use format provider/model-name
Authentication Error Verify API keys are set correctly

Concepts Caching (`docs/concepts/caching.md`)

See Also

title: Caching Strategies with Instructor description: Learn how to use caching with Instructor to reduce API costs and improve performance.

For more details on caching concepts, see our blog.

Built-in Caching (v1.9.1 and later)

Cache isolation in 1.17

Each patched client has its own cache namespace by default. Repeated calls through
that client can reuse entries; a newly constructed client gets a separate namespace,
even when it uses the same disk cache. This prevents accidental reuse across
endpoints or accounts that use the same model name.

For intentional reuse across client instances or process restarts, pass a stable
cache_namespace string alongside cache when calling client.create. Choose a
namespace that identifies the endpoint, account or tenant, and application policy.
Never use credentials as namespace values or share one namespace between tenants.
Change the namespace when switching endpoints, accounts, or validation policy.

Provider and SDK client identity, the complete prepared request, positional
arguments, validation context, and strictness contribute to cache keys.
Cached models are revalidated using the current call's context and strictness.
If request objects cannot be serialized reliably, the built-in cache is bypassed
for that call instead of reusing an ambiguous key.

Instructor supports caching for every client. Pass a cache adapter when you create the client. The cache parameter flows through to all provider implementations via **kwargs:

python
from instructor import from_provider
from instructor.cache import AutoCache, DiskCache

# Works with any provider - cache flows through **kwargs automatically
client = from_provider("openai/gpt-4.1-mini", cache=AutoCache(maxsize=1000))
client = from_provider("anthropic/claude-3-haiku", cache=AutoCache(maxsize=1000))
client = from_provider("google/gemini-2.5-flash", cache=DiskCache(directory=".cache"))

# Your normal calls are now cached automatically
from pydantic import BaseModel


class User(BaseModel):
    name: str


first = client.create(
    messages=[{"role": "user", "content": "Hi."}], response_model=User
)
second = client.create(
    messages=[{"role": "user", "content": "Hi."}], response_model=User
)
assert first.name == second.name  # second call was served from cache

`cache_ttl` per-call override

Pass cache_ttl=<seconds> alongside cache= if you want a result to
expire automatically:

python
from instructor import from_provider
from instructor.cache import DiskCache
from pydantic import BaseModel


class User(BaseModel):
    name: str


cache = DiskCache(directory=".cache")
client = from_provider("openai/gpt-4.1-mini")

client.create(
    messages=[{"role": "user", "content": "Hi"}],
    response_model=User,
    cache=cache,
    cache_ttl=3600,  # 1 hour
)

If the underlying cache backend supports TTL (e.g. DiskCache does), the
entry will be evicted after the specified duration. For AutoCache the
parameter is ignored.

Cache-key design

The built-in cache uses instructor.cache.make_request_cache_key for the complete
prepared request. It includes client identity, namespace, validation context and
strictness in addition to the components below. Unsupported values bypass caching.

Components that influence the key:

Part Why it matters
model Different model names can yield different answers
messages / contents The full chat history is hashed
mode JSON vs. TOOLS vs. RESPONSES changes formatting
response_model schema The entire model_json_schema() is included so any change in field names, types or descriptions busts the cache automatically

Keys are SHA-256 hex digests of constant length. The lower-level
make_cache_key helper remains available for custom integrations, as shown below;
callers must supply all relevant identity and policy inputs themselves.

python
from instructor.cache import make_cache_key
from pydantic import BaseModel


class User(BaseModel):
    name: str


key = make_cache_key(
    messages=[{"role": "user", "content": "hello"}],
    model="gpt-4.1-mini",
    response_model=User,
    mode="TOOLS",
)
print(key)  # → 9b8f5e2c8c9e…
#> 2e2a9521bd269d62ee9a8559d7deacba0025c1f6da0ec1fc63d472788be096fe

If you need custom behaviour (e.g. ignoring certain prompt fields) you can
write your own helper and pass a derived key into a bespoke cache adapter.

Raw Response Reconstruction

For raw completion objects (used with create_with_completion), we use a SimpleNamespace trick to reconstruct the original object structure:

python
from pydantic import BaseModel


class Completion(BaseModel):
    content: str
    usage: dict


# Example completion object
completion = Completion(content="Hello", usage={"tokens": 10})

# When caching:
raw_json = completion.model_dump_json()  # Serialize to JSON

# When restoring from cache:
import json
from types import SimpleNamespace

restored = json.loads(raw_json, object_hook=lambda d: SimpleNamespace(**d))

This approach allows us to restore the original dot-notation access patterns (e.g., completion.usage.total_tokens) without requiring the original class definitions. The SimpleNamespace objects behave identically to the original completion objects for attribute access while being much simpler to reconstruct from JSON.

1. `functools.cache` for Simple In-Memory Caching

When to Use: Good for functions with immutable arguments, called repeatedly with the same parameters in small to medium-sized applications. Use this when reusing the same data within a single session.

python
import time
import functools
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserDetail(BaseModel):
    name: str
    age: int


@functools.cache
def extract(data) -> UserDetail:
    return client.create(
        response_model=UserDetail,
        messages=[
            {"role": "user", "content": data},
        ],
    )


start = time.perf_counter()  # (1)
model = extract("Extract jason is 25 years old")
print(f"Time taken: {time.perf_counter() - start}")
#> Time taken: 0.43337099999189377

start = time.perf_counter()
model = extract("Extract jason is 25 years old")  # (2)
print(f"Time taken: {time.perf_counter() - start}")
#> Time taken: 1.166015863418579e-06
  1. Using time.perf_counter() to measure the time taken to run the function is better than using time.time() because it's more accurate and less susceptible to system clock changes.
  2. The second time we call extract, the result is returned from the cache, and the function is not called.

!!! warning "Changing the Model does not Invalidate the Cache"

text
Note that changing the model does not invalidate the cache. This is because the cache key is based on the function's name and arguments, not the model. This means that if we change the model, the cache will still return the old result.

Call extract multiple times with the same argument, and the result will be cached in memory for faster access.

Benefits: Easy to implement, fast access due to in-memory storage, and requires no additional libraries.

??? question "What is a decorator?"

text
A decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it. In Python, decorators are functions that take a function as an argument and return a closure.

```python hl_lines="3-5 9"
def decorator(func):
    def wrapper(*args, **kwargs):
        print("Do something before")  # (1)
        #> Do something before
        result = func(*args, **kwargs)
        print("Do something after")  # (2)
        #> Do something after
        return result

    return wrapper


@decorator
def say_hello():
    #> Hello!
    print("Hello!")
    #> Hello!


say_hello()
#> "Do something before"
#> "Hello!"
#> "Do something after"
```

1. The code is executed before the function is called
2. The code is executed after the function is called

2. `diskcache` for Persistent, Large Data Caching

??? note "Copy Caching Code"

text
The same `instructor_cache` decorator works for both `diskcache` and `redis` caching. Copy the code below and use it for both examples.

```python
import functools
import inspect
import diskcache

cache = diskcache.Cache('./my_cache_directory')  # (1)


def instructor_cache(func):
    """Cache a function that returns a Pydantic model"""
    return_type = inspect.signature(func).return_annotation
    if not issubclass(return_type, BaseModel):  # (2)
        raise ValueError("The return type must be a Pydantic model")

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        key = f"{func.__name__}-{functools._make_key(args, kwargs, typed=False)}"
        # Check if the result is already cached
        if (cached := cache.get(key)) is not None:
            # Deserialize from JSON based on the return type
            return return_type.model_validate_json(cached)

        # Call the function and cache its result
        result = func(*args, **kwargs)
        serialized_result = result.model_dump_json()
        cache.set(key, serialized_result)

        return result

    return wrapper
```

1. We create a new `diskcache.Cache` instance to store the cached data. This will create a new directory called `my_cache_directory` in the current working directory.
2. We only want to cache functions that return a Pydantic model to simplify serialization and deserialization logic in this example code

Remember that you can change this code to support non-Pydantic models, or to use a different caching backend. More over, don't forget that this cache does not invalidate when the model changes, so you might want to encode the `Model.model_json_schema()` as part of the key.

When to Use: Good for applications that need cache persistence between sessions or deal with large datasets. Use this when you want to reuse the same data across multiple sessions or store large amounts of data.

python hl_lines="10"
import functools
import inspect
import instructor
import diskcache
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")
cache = diskcache.Cache('./my_cache_directory')


def instructor_cache(func):
    """Cache a function that returns a Pydantic model"""
    return_type = inspect.signature(func).return_annotation  # (4)
    if not issubclass(return_type, BaseModel):  # (1)
        raise ValueError("The return type must be a Pydantic model")

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        key = (
            f"{func.__name__}-{functools._make_key(args, kwargs, typed=False)}"  #  (2)
        )
        # Check if the result is already cached
        if (cached := cache.get(key)) is not None:
            # Deserialize from JSON based on the return type (3)
            return return_type.model_validate_json(cached)

        # Call the function and cache its result
        result = func(*args, **kwargs)
        serialized_result = result.model_dump_json()
        cache.set(key, serialized_result)

        return result

    return wrapper


class UserDetail(BaseModel):
    name: str
    age: int


@instructor_cache
def extract(data) -> UserDetail:
    return client.create(
        response_model=UserDetail,
        messages=[
            {"role": "user", "content": data},
        ],
    )
  1. We only want to cache functions that return a Pydantic model to simplify serialization and deserialization logic
  2. We use functool's _make_key to generate a unique key based on the function's name and arguments. This is important because we want to cache the result of each function call separately.
  3. We use Pydantic's model_validate_json to deserialize the cached result into a Pydantic model.
  4. We use inspect.signature to get the function's return type annotation, which we use to validate the cached result.

Benefits: Reduces computation time for heavy data processing and provides disk-based caching for persistence.

3. Redis Caching Decorator for Distributed Systems

??? note "Copy Caching Code"

text
The same `instructor_cache` decorator works for both `diskcache` and `redis` caching. Copy the code below and use it for both examples.

```python
import functools
import inspect
import redis

cache = redis.Redis("localhost")


def instructor_cache(func):
    """Cache a function that returns a Pydantic model"""
    return_type = inspect.signature(func).return_annotation
    if not issubclass(return_type, BaseModel):
        raise ValueError("The return type must be a Pydantic model")

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        key = f"{func.__name__}-{functools._make_key(args, kwargs, typed=False)}"
        # Check if the result is already cached
        if (cached := cache.get(key)) is not None:
            # Deserialize from JSON based on the return type
            return return_type.model_validate_json(cached)

        # Call the function and cache its result
        result = func(*args, **kwargs)
        serialized_result = result.model_dump_json()
        cache.set(key, serialized_result)

        return result

    return wrapper
```

Remember that you can change this code to support non-Pydantic models, or to use a different caching backend. More over, don't forget that this cache does not invalidate when the model changes, so you might want to encode the `Model.model_json_schema()` as part of the key.

When to Use: Good for distributed systems where multiple processes need to access cached data, or for applications that need fast read/write access and handle complex data structures.

python
import redis
import functools
import inspect
import instructor

from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")
cache = redis.Redis("localhost")


def instructor_cache(func):
    """Cache a function that returns a Pydantic model"""
    return_type = inspect.signature(func).return_annotation
    if not issubclass(return_type, BaseModel):  # (1)
        raise ValueError("The return type must be a Pydantic model")

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        key = f"{func.__name__}-{functools._make_key(args, kwargs, typed=False)}"  # (2)
        # Check if the result is already cached
        if (cached := cache.get(key)) is not None:
            # Deserialize from JSON based on the return type
            return return_type.model_validate_json(cached)

        # Call the function and cache its result
        result = func(*args, **kwargs)
        serialized_result = result.model_dump_json()
        cache.set(key, serialized_result)

        return result

    return wrapper


class UserDetail(BaseModel):
    name: str
    age: int


@instructor_cache
def extract(data) -> UserDetail:
    # Assuming client.chat.completions.create returns a UserDetail instance
    return client.create(
        response_model=UserDetail,
        messages=[
            {"role": "user", "content": data},
        ],
    )
  1. We only want to cache functions that return a Pydantic model to simplify serialization and deserialization logic
  2. We use functool's _make_key to generate a unique key based on the function's name and arguments. This is important because we want to cache the result of each function call separately.

Benefits: Scalable for large-scale systems, supports fast in-memory data storage and retrieval, and works with various data types.

!!! note "Same Decorator, Different Backend"

text
The code above uses the same `instructor_cache` decorator as before. The implementation is the same, but it uses a different caching backend.

Concepts Citation (`docs/concepts/citation.md`)

title: Citation Extraction with CitationMixin description: Learn how to extract and validate citations from source text using CitationMixin to prevent hallucinations.

Citation Extraction with CitationMixin

CitationMixin is a Pydantic mixin that helps extract and validate citations from source text. It ensures that quotes used in your extracted data actually exist in the source context, preventing hallucinations.

What is CitationMixin?

CitationMixin adds citation validation to your Pydantic models. When you use it, your model gets a substring_quotes field that contains quotes from the source text. The mixin automatically validates that these quotes exist in the source and corrects them to match exact spans.

Basic Usage

Inherit from CitationMixin to add citation support to your model:

python
from pydantic import BaseModel, Field
from instructor import CitationMixin
import instructor


class User(CitationMixin, BaseModel):
    name: str = Field(description="The name of the person")
    age: int = Field(description="The age of the person")
    role: str = Field(description="The role of the person")


client = instructor.from_provider("openai/gpt-4o-mini")

context = "Betty was a student. Jason was a student. Jason is 20 years old"

user = client.create(
    response_model=User,
    messages=[
        {
            "role": "user",
            "content": f"Extract information about Jason from: {context}",
        },
    ],
    context={"context": context},
)

# Verify quotes exist in context
for quote in user.substring_quotes:
    assert quote in context

print(user.model_dump())
# {
#     "name": "Jason",
#     "age": 20,
#     "role": "student",
#     "substring_quotes": [
#         "Jason was a student",
#         "Jason is 20 years old",
#     ]
# }

How It Works

CitationMixin works in three steps:

  1. Extraction: The LLM extracts data and provides quotes in the substring_quotes field
  2. Validation: The mixin checks if each quote exists in the source context using fuzzy matching
  3. Correction: Quotes are corrected to match exact spans in the source text

The validation happens automatically when you pass context={"context": source_text} to your create() call.

Using with Validation Context

CitationMixin uses Pydantic's validation context to access the source text. Pass the source text in the context parameter:

python
from pydantic import BaseModel, Field
from instructor import CitationMixin
import instructor


class Fact(CitationMixin, BaseModel):
    statement: str = Field(description="A factual statement")
    # substring_quotes is added automatically by CitationMixin


client = instructor.from_provider("openai/gpt-4o-mini")

source_text = """
The Eiffel Tower was completed in 1889 and stands 330 meters tall.
It was designed by Gustave Eiffel and is located in Paris, France.
"""

fact = client.create(
    response_model=Fact,
    messages=[
        {
            "role": "user",
            "content": f"Extract facts about the Eiffel Tower from: {source_text}",
        },
    ],
    context={"context": source_text},
)

# All quotes are validated and corrected to exact spans
for quote in fact.substring_quotes:
    print(f"Quote: {quote}")
    assert quote in source_text

Fuzzy Matching

CitationMixin uses fuzzy matching to find quotes even if they don't match exactly. This handles minor differences like:

  • Extra whitespace
  • Slight wording variations
  • Punctuation differences

The matching allows up to 5 character errors by default, which helps handle cases where the LLM paraphrases slightly.

Advanced Example: Question Answering with Citations

Use CitationMixin to build question-answering systems that cite sources:

python
from typing import List
from pydantic import BaseModel, Field
from instructor import CitationMixin
import instructor


class Fact(CitationMixin, BaseModel):
    statement: str = Field(description="A factual statement")


class Answer(CitationMixin, BaseModel):
    question: str
    facts: List[Fact] = Field(description="List of facts that answer the question")


client = instructor.from_provider("openai/gpt-4o-mini")

source_text = """
Jason Liu grew up in Toronto, Canada but was born in China.
He went to an arts high school but studied Computational Mathematics and Physics in university.
He worked at Stitchfix and Facebook as part of coop programs.
He started the Data Science club at the University of Waterloo and was president for 2 years.
"""

answer = client.create(
    response_model=Answer,
    messages=[
        {
            "role": "system",
            "content": "Answer questions with exact citations from the source text.",
        },
        {
            "role": "user",
            "content": f"Source: {source_text}\n\nQuestion: What did Jason do during college?",
        },
    ],
    context={"context": source_text},
)

# Verify all citations exist
for fact in answer.facts:
    for quote in fact.substring_quotes:
        assert quote in source_text
        print(f"Verified: {quote}")

When to Use CitationMixin

Use CitationMixin when:

  • You need to verify that extracted information comes from source text
  • You're building RAG (Retrieval Augmented Generation) systems
  • You want to prevent hallucinations by validating citations
  • You need exact quote spans for highlighting or display

Limitations

  • Requires passing source text in context={"context": ...}
  • Uses fuzzy matching which may not catch all paraphrasing
  • Only validates quotes, not the accuracy of extracted facts themselves

See Also

Concepts Decisions (`docs/concepts/decisions.md`)

title: Typed decisions description: Evaluate a shared context with typed choices, probabilities, and scores.

Typed decisions

Decision models evaluate one context against several questions in a single request.
Use Mode.DECISIONS with Jev through OpenRouter or TypeSafe, or OpenAI's Decisions API. Define the questions
as a Pydantic model and receive a validated instance.

Content moderation

This example uses a reusable choice enum, an inline choice, a yes/no probability,
and a score mapped to 0–10. Outputs shown in comments are illustrative.

text
/* Detailed source-code truncated for AI context efficiency. */

context is a dictionary sent in full as provider state and passed to Pydantic
validators. Do not include validator-only values that should not be sent. Use
create() for the validated model alone, or create_with_completion() to also
inspect the unchanged provider response and its probabilities and confidence.

For direct TypeSafe access, set TYPESAFE_API_KEY and use:

python
client = instructor.from_provider(
    "typesafe/jev-latest", mode=instructor.Mode.DECISIONS
)

Both providers use the same decision model and context. To switch routes,
change the provider/model string and configure that provider's API key; the
question annotations and validated result do not change. Raw response metadata
remains provider-specific. For example, OpenRouter includes a monetary cost
in usage; do not infer a direct TypeSafe cost from token counts.

For async use, pass async_client=True, await create or
create_with_completion, and call await client.close() when done. Synchronous
clients also support with; asynchronous clients support async with.

An owned HTTP client defaults to a 60-second timeout. When supplying
http_client, its timeout is preserved unless you pass timeout=... to
from_provider. That override applies to decision requests without changing the
supplied HTTP client's configuration. Closing a decision client only closes an
HTTP client it created. Pass timeout=None explicitly to disable timeouts.

OpenAI Decisions

Set OPENAI_API_KEY and use the same typed decision model:

python
client = instructor.from_provider(
    "openai/gpt-6-luna", mode=instructor.Mode.DECISIONS
)
decision, raw = client.create_with_completion(
    response_model=ModerationDecision,
    context={"post": {"text": "I disagree with your argument."}, "policy": ["No harassment"]},
)

This route sends the JSON-serialized context as text input to
POST /v1/decisions. It translates choices and score levels into ordered
questions and maps Noul to OpenAI predicate probabilities. Structured criteria
and examples are serialized into descriptions or predicate instructions, not
sent as Jev-specific wire fields. This adapter currently supports text context,
not OpenAI's inline-image input. Refusals, missing answers, wrong names/types,
and out-of-range values raise errors rather than returning partial results.

raw remains the unchanged OpenAI response, including its answer array,
probability arrays, confidence, and usage. It is not the Jev answer dictionary.
The HTTP adapter does not require an SDK upgrade; OpenAI's native SDK Decisions
resource requires Python SDK 3.26.0 or later. Offline HTTP contracts cover sync
and async paths; live OpenAI Decisions compatibility has not been tested.
See the OpenAI Decisions guide.

Choices

Use a Literal for a short list, a string-valued Python Enum for a reusable
type, or Choices when options need descriptions and examples. Choices
returns enum members; names become lowercase values unless Choice(value=...)
sets an explicit value. Inline Annotated[Literal[...], Choice(...)] branches
return literal values. On Python 3.9, use Union[...] instead of |.

Every question needs instructions: use Question(instructions=...),
Field(description=...), or a Choices/enum docstring, in that order. The
field name is an answer key, not a question. For structured instructions:

python
from typing import Annotated, Literal
from pydantic import BaseModel
from instructor.decisions import Question

class Route(BaseModel):
    team: Annotated[
        Literal["billing", "support"],
        Question(instructions={"question": "Which team?", "policy": ["Route refunds to billing"]}),
    ]

Probabilities and scores

Noul annotates a float with a yes/no question. The result is the probability
of true, from 0 to 1. when_true and when_false clarify the boundary; examples
are (input, bool) pairs. For the native shape, use
criteria={"true": ..., "false": ...} instead of when_true and when_false.

Score annotates a float with 2–10 ordered levels. Jev returns a weighted value
on the levels' zero-based indexes. Three levels span 0–2; scale=(0, 10) maps
them to 0, 5, and 10. A result of 7 is an intermediate weighted value, not an
additional defined level. Bounds must be finite and increasing. Score examples
are (input, level_index) pairs; a Level can also carry examples.

The raw score, probabilities, and legend remain in provider units. Choice and
Score confidence describe the distribution, not the probability of correctness;
choose any moderation thresholds using labeled data.

Structured criteria and examples

Instructions and criteria accept JSON-compatible strings, objects, or arrays.
Choice descriptions can also be None. Instructions can use Instructor's
Jinja templates with context; the context itself is sent unchanged.

Examples are an Instructor convenience: inputs are converted with str() and
combined with the relevant meaning as {"meaning": ..., "examples": [...]}.
Without examples, the meaning passes through unchanged.

Unsupported field types, invalid labels, missing instructions, and invalid scales
fail before the request. Missing, mismatched, or out-of-range answers fail rather
than being replaced with defaults. Decision mode currently does not support
streaming, retries, caching, or completion hooks; HTTP errors propagate to the
caller. See the TypeSafe API reference and
OpenRouter's Jev guide.

Testing

The local suites in tests/v2/decisions/ cover annotations and answer parsing,
client configuration, and common provider contracts over localhost HTTP. They
do not call provider services or require API keys:

bash
uv run pytest tests/v2/decisions/

Live provider contracts are a separate, opt-in suite. They use synthetic ticket
data, make billed API requests, and skip providers whose keys are missing:

bash
uv run pytest tests/llm/test_decisions/ --run-decisions-live

Set TYPESAFE_API_KEY, OPENROUTER_API_KEY, and/or OPENAI_API_KEY for the selected providers.
TYPESAFE_DECISIONS_MODEL, OPENROUTER_DECISIONS_MODEL, and OPENAI_DECISIONS_MODEL optionally override
the live model IDs. Without the explicit flag, live cases are skipped even
when credentials are present. Local contracts do not certify a live provider.

Concepts Dictionary Operations (`docs/concepts/dictionary_operations.md`)

See Also

title: Dictionary Operations Optimization in Instructor description: Learn about performance optimizations for dictionary operations in Instructor, including message extraction and configuration parameter handling.

Dictionary Operations Optimization

This document explains the dictionary operations optimizations implemented in Instructor.

Overview

Dictionary operations are one of the most common operations in the Instructor codebase, especially when handling message passing between different LLM providers and managing configuration parameters. Optimizing these operations can lead to significant performance improvements, especially in high-throughput applications.

Optimized Areas

Message Extraction

The extract_messages function was optimized to use direct key lookups instead of nested get() calls, which reduces the overhead of function calls and improves performance.

Before:

python
from typing import Any


def extract_messages(kwargs: dict[str, Any]) -> Any:
    return kwargs.get(
        "messages", kwargs.get("contents", kwargs.get("chat_history", []))
    )

After:

python
from typing import Any


def extract_messages(kwargs: dict[str, Any]) -> Any:
    if "messages" in kwargs:
        return kwargs["messages"]
    if "contents" in kwargs:
        return kwargs["contents"]
    if "chat_history" in kwargs:
        return kwargs["chat_history"]
    return []

Response Processing Functions

The response processing functions were optimized to:

  1. Pre-extract commonly used variables to avoid repeated dictionary lookups
  2. Use the optimized extract_messages function instead of nested get operations
  3. Reduce redundant dictionary operations in error handling

Message Handler Selection

The handle_reask_kwargs function was optimized to use direct conditional checks instead of creating a large mapping dictionary, which reduces memory overhead and improves lookup performance.

Before:

python
def handle_reask_kwargs(kwargs, mode, response, exception):
    kwargs = kwargs.copy()
    functions = {
        Mode.TOOLS: reask_anthropic_tools,
        Mode.JSON: reask_anthropic_json,
        # ... many more mappings
    }
    reask_function = functions.get(mode, reask_default)
    return reask_function(kwargs=kwargs, response=response, exception=exception)

After:

python
def handle_reask_kwargs(kwargs, mode, response, exception):
    kwargs_copy = kwargs.copy()

    if mode in {Mode.TOOLS, Mode.ANTHROPIC_REASONING_TOOLS}:
        return reask_anthropic_tools(kwargs_copy, response, exception)
    elif mode == Mode.JSON:
        return reask_anthropic_json(kwargs_copy, response, exception)
    # ... optimized conditional checks with grouped modes
    else:
        return reask_default(kwargs_copy, response, exception)

System Message Handling

The combine_system_messages function in utils.py was optimized to:

  1. Cache type checks to avoid repeated calls
  2. Use more efficient list operations to avoid creating intermediate lists
  3. Optimize type conversion scenarios

Benchmarks

Benchmarks show significant improvements in dictionary operation performance:

Operation Before (ms) After (ms) Improvement
extract_messages ~0.08 ~0.03 ~62%
handle_reask_kwargs ~0.09 ~0.05 ~44%
combine_system_messages ~0.12 ~0.07 ~42%

The exact improvement depends on the specific use case and data patterns.

Testing

Two types of tests were created to ensure the optimizations were safe:

  1. Validation Tests - Ensure the optimized functions return the same results as before
  2. Benchmark Tests - Measure and verify the performance improvements

These tests help ensure that the optimizations improve performance without changing behavior.

Conclusion

Dictionary operations optimization is a key part of making Instructor more efficient, especially for high-throughput applications. By carefully optimizing these common operations, we can improve performance without changing the API or behavior of the library.

Concepts Distillation (`docs/concepts/distillation.md`)

title: Seamless Fine-Tuning of Python Functions Using Instructor's Distillation description: Learn how to fine-tune language models with Python functions using Instructor's `Instructions` for efficient data preparation and logging.

See Also

Distilling python functions into LLM

Instructions from the Instructor library offers a seamless way to make language models backward compatible with existing Python functions. By employing Pydantic type hints, it not only ensures compatibility but also facilitates fine-tuning gpt-4.1-mini to emulate these functions end-to-end.

If you want to see the full example checkout examples/distillation

The Challenges in Function-Level Fine-Tuning

Replicating the behavior of a Python function in a language model involves intricate data preparation. For instance, teaching a model to execute three-digit multiplication is not as trivial as implementing def f(a, b): return a * b. OpenAI's fine-tuning script coupled with their function calling utility provides a structured output, thereby simplifying the data collection process. Additionally, this eliminates the need for passing the schema to the model, thus conserving tokens.

The Role of `Instructions` in Simplifying the Fine-Tuning Process

By using Instructions, you can annotate a Python function that returns a Pydantic object, thereby automating the dataset creation for fine-tuning. A handler for logging is all that's needed to build this dataset.

How to Implement `Instructions` in Your Code

Quick Start: How to Use Instructor's Distillation Feature

Before we dig into the nitty-gritty, let's look at how easy it is to use Instructor's distillation feature to use function calling finetuning to export the data to a JSONL file.

python
import logging
import random
from pydantic import BaseModel

# Logging setup
logging.basicConfig(level=logging.INFO)

from instructor import Instructions, FinetuneFormat  # pip install instructor

instructions = Instructions(
    name="three_digit_multiply",
    finetune_format=FinetuneFormat.MESSAGES,  # or FinetuneFormat.RAW
    # log handler is used to save the data to a file
    # you can imagine saving it to a database or other storage
    # based on your needs!
    log_handlers=[logging.FileHandler("math_finetunes.jsonl")],
)


class Multiply(BaseModel):
    a: int
    b: int
    result: int


# Define a function with distillation
# The decorator will automatically generate a dataset for fine-tuning
# They must return a pydantic model to leverage function calling
@instructions.distil
def fn(a: int, b: int) -> Multiply:
    resp = a * b
    return Multiply(a=a, b=b, result=resp)


# Generate some data
for _ in range(10):
    random.seed(42)
    a = random.randint(100, 999)
    b = random.randint(100, 999)
    print(fn(a, b))
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356
    #> a=754 b=214 result=161356

The Intricacies of Fine-tuning Language Models

Fine-tuning isn't just about writing a function like def f(a, b): return a * b. It requires detailed data preparation and logging. However, Instructor provides a built-in logging feature and structured outputs to simplify this.

Why Instructor and Distillation are Game Changers

The library offers two main benefits:

  1. Efficiency: Streamlines functions, distilling requirements into model weights and a few lines of code.
  2. Integration: Eases combining classical machine learning and language models by providing a simple interface that wraps existing functions.

Role of Instructor in Simplifying Fine-Tuning

The from instructor import Instructions feature is a time saver. It auto-generates a fine-tuning dataset, making it a breeze to imitate a function's behavior.

FinetuneFormat Options

The finetune_format parameter controls how the fine-tuning data is structured. There are two options:

MESSAGES Format (Default)

The MESSAGES format creates data in OpenAI's chat completion format with messages and function calls. This is the recommended format for most use cases as it matches OpenAI's fine-tuning API format.

python
from instructor import Instructions, FinetuneFormat

instructions = Instructions(
    name="my_function",
    finetune_format=FinetuneFormat.MESSAGES,
    log_handlers=[logging.FileHandler("output.jsonl")],
)

RAW Format

The RAW format creates a simpler format with function metadata, arguments, and response. Use this format if you need more control over the data structure or are using a custom fine-tuning pipeline.

python
from instructor import Instructions, FinetuneFormat

instructions = Instructions(
    name="my_function",
    finetune_format=FinetuneFormat.RAW,
    log_handlers=[logging.FileHandler("output.jsonl")],
)

Logging Output and Running a Finetune

Here's how the logging output would look for MESSAGES format:

python
{
    "messages": [
        {"role": "system", "content": 'Predict the results of this function: ...'},
        {"role": "user", "content": 'Return fn(133, b=539)'},
        {
            "role": "assistant",
            "function_call": {
                "name": "Multiply",
                "arguments": '{"a":133,"b":539,"result":89509}',
            },
        },
    ],
    "functions": [
        {"name": "Multiply", "description": "Correctly extracted `Multiply`..."}
    ],
}

For RAW format, the output would look like:

python
{
    "fn_name": "three_digit_multiply",
    "fn_repr": "def fn(a: int, b: int) -> Multiply:\n    ...",
    "args": [133],
    "kwargs": {"b": 539},
    "response": {"a": 133, "b": 539, "result": 89509},
}

Run a finetune like this:

bash
instructor jobs create-from-file math_finetunes.jsonl

Once a model is trained you can simply change mode to dispatch and it will use the model to run the function!

python
from instructor import Instructions
from pydantic import BaseModel


class Multiply(BaseModel):
    a: int
    b: int
    result: int


instructions = Instructions(
    name="three_digit_multiply",
)


@instructions.distil(model='gpt-4.1-mini:finetuned-123', mode="dispatch")
def fn(a: int, b: int) -> Multiply:
    # now this code will be short circuited and the model will be used instead.
    resp = a + b
    return Multiply(a=a, b=b, result=resp)

With this, you can swap the function implementation, making it backward compatible. You can even imagine using the different models for different tasks or validating and running evals by using the original function and comparing it to the distillation.

Concepts Enums (`docs/concepts/enums.md`)

title: Using Enums and Literals in Pydantic for Role Management description: Learn how to implement Enums and Literals in Pydantic to manage standardized user roles with a fallback option.

To prevent data misalignment, we can use Enums for standardized fields. Always include an "Other" option as a fallback so the model can signal uncertainty.

python hl_lines="7 12"
from pydantic import BaseModel, Field
from enum import Enum


class Role(Enum):
    PRINCIPAL = "PRINCIPAL"
    TEACHER = "TEACHER"
    STUDENT = "STUDENT"
    OTHER = "OTHER"


class UserDetail(BaseModel):
    age: int
    name: str
    role: Role = Field(
        description="Correctly assign one of the predefined roles to the user."
    )

If you're having a hard time with Enum an alternative is to use Literal instead.

python hl_lines="4"
from typing import Literal
from pydantic import BaseModel


class UserDetail(BaseModel):
    age: int
    name: str
    role: Literal["PRINCIPAL", "TEACHER", "STUDENT", "OTHER"]

See Also

  • Types - Working with different data types including Literal
  • Union Types - Using unions with enums for multiple choices
  • Response Models - Using enums in Pydantic models
  • Fields - Customizing enum fields with Field metadata

Concepts Error Handling (`docs/concepts/error_handling.md`)

title: Error Handling description: Learn how to handle errors and exceptions when using Instructor for structured outputs.

Error Handling

Instructor provides a comprehensive exception hierarchy to help you handle errors gracefully. All Instructor exceptions inherit from InstructorError.

Exception Reference

Exception Description Key Attributes
InstructorError Base exception for all Instructor errors -
IncompleteOutputException Output truncated due to token limit last_completion
InstructorRetryException All retry attempts exhausted n_attempts, failed_attempts, total_usage
ValidationError Response validation failed -
ResponseParsingError Cannot parse LLM response mode, raw_response
ProviderError Provider-specific error provider
ConfigurationError Invalid configuration -
ModeError Invalid mode for provider mode, provider, valid_modes
ClientError Client initialization failed -
MultimodalError Processing image/audio/PDF failed content_type, file_path
AsyncValidationError Async validation failed errors

Common Exceptions

Incomplete Output

Raised when the LLM output is truncated due to reaching the token limit:

python
import instructor
from pydantic import BaseModel
from instructor.core.exceptions import (
    IncompleteOutputException,
    InstructorRetryException,
)


class Report(BaseModel):
    content: str


client = instructor.from_provider("openai/gpt-4.1-mini", mode=instructor.Mode.JSON)

try:
    response = client.create(
        response_model=Report,
        messages=[{"role": "user", "content": "Write a long report..."}],
        max_tokens=50,
        max_retries=0,
    )
except (IncompleteOutputException, InstructorRetryException) as e:
    print(f"Output truncated: {e}")
    print(f"Last completion: {e.last_completion}")

Retry Exhausted

Raised when all retry attempts fail:

python
import instructor
from pydantic import BaseModel
from instructor.core.exceptions import InstructorRetryException


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4.1-mini")

try:
    response = client.create(
        response_model=User,
        messages=[{"role": "user", "content": "Extract user info..."}],
        max_retries=3,
    )
except InstructorRetryException as e:
    print(f"Failed after {e.n_attempts} attempts")
    for attempt in e.failed_attempts:
        print(f"  Attempt {attempt.attempt_number}: {attempt.exception}")

Validation Error

Raised when the response fails validation:

python
import instructor
from pydantic import BaseModel, field_validator
from instructor.core.exceptions import ValidationError


class StrictModel(BaseModel):
    value: int

    @field_validator("value")
    @classmethod
    def validate_value(cls, v: int) -> int:
        if v < 0:
            raise ValueError("Value must be positive")
        return v


client = instructor.from_provider("openai/gpt-4.1-mini")

try:
    response = client.create(
        response_model=StrictModel,
        messages=[{"role": "user", "content": "Extract data..."}],
    )
except ValidationError as e:
    print(f"Validation failed: {e}")

Provider and Configuration Errors

Raised for provider-specific issues or invalid configuration:

python
import instructor
from instructor.core.exceptions import ConfigurationError, ModeError

# Invalid provider format
try:
    client = instructor.from_provider("invalid-format")
except ConfigurationError as e:
    print(f"Configuration error: {e}")

# Wrong mode for provider
try:
    client = instructor.from_provider(
        "openai/gpt-4.1-mini",
        mode=instructor.Mode.TOOLS,
    )
except ModeError as e:
    print(f"Invalid mode. Valid modes: {e.valid_modes}")

Best Practices

Catch Specific Exceptions

python
import logging
import instructor
from pydantic import BaseModel
from instructor.core.exceptions import (
    IncompleteOutputException,
    InstructorRetryException,
    ValidationError,
)

logger = logging.getLogger(__name__)


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4.1-mini")

try:
    response = client.create(
        response_model=User,
        messages=[{"role": "user", "content": "Extract: Sam is 34"}],
    )
except IncompleteOutputException:
    logger.warning("Output truncated, retrying with more tokens")
    response = client.create(
        response_model=User,
        messages=[{"role": "user", "content": "Extract: Sam is 34"}],
        max_tokens=2000,
    )
except InstructorRetryException as e:
    logger.error(f"Failed after {e.n_attempts} attempts")
    response = None
except ValidationError as e:
    logger.error(f"Validation failed: {e}")
    raise

Use Base Exception for General Handling

python
import instructor
from pydantic import BaseModel
from instructor.core.exceptions import InstructorError


class Data(BaseModel):
    value: str


client = instructor.from_provider("openai/gpt-4.1-mini")

try:
    response = client.create(
        response_model=Data,
        messages=[{"role": "user", "content": "Extract data"}],
    )
except InstructorError as e:
    # Catches any Instructor-specific error
    print(f"Instructor error: {type(e).__name__}: {e}")

Graceful Degradation

python
import instructor
from pydantic import BaseModel, field_validator
from instructor.core.exceptions import ValidationError, InstructorRetryException


class StrictData(BaseModel):
    value: int

    @field_validator("value")
    @classmethod
    def validate_value(cls, v: int) -> int:
        if v < 0:
            raise ValueError("Value must be positive")
        return v


class RelaxedData(BaseModel):
    value: str


client = instructor.from_provider("openai/gpt-4.1-mini")


def extract_with_fallback(content: str):
    try:
        return client.create(
            response_model=StrictData,
            messages=[{"role": "user", "content": content}],
        )
    except ValidationError:
        # Fall back to less strict model
        return client.create(
            response_model=RelaxedData,
            messages=[{"role": "user", "content": content}],
        )
    except InstructorRetryException:
        return None

Backwards Compatibility

New exceptions inherit from both ValueError and InstructorError, so existing code continues to work:

python
import instructor
from pydantic import BaseModel
from instructor.core.exceptions import ResponseParsingError


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4.1-mini")

# Old code still works
try:
    response = client.create(
        response_model=User,
        messages=[{"role": "user", "content": "Extract: Kai is 41"}],
    )
except ValueError as e:
    print(f"Error: {e}")

# New code can access additional context
try:
    response = client.create(
        response_model=User,
        messages=[{"role": "user", "content": "Extract: Kai is 41"}],
    )
except ResponseParsingError as e:
    print(f"Mode: {e.mode}, Raw: {e.raw_response}")

Integration with Hooks

Monitor errors using the hooks system:

python
import instructor
from instructor.core.exceptions import ValidationError


def on_parse_error(error: Exception):
    if isinstance(error, ValidationError):
        print(f"Validation error: {error}")


client = instructor.from_provider("openai/gpt-4.1-mini")
client.hooks.on("parse:error", on_parse_error)

See Also

Concepts Fastapi (`docs/concepts/fastapi.md`)

title: FastAPI Integration with Instructor - API Development Guide description: Build production-ready APIs with FastAPI and Instructor. Create type-safe endpoints for structured LLM outputs with automatic validation and documentation.

Integrating Pydantic Models with FastAPI

FastAPI is an enjoyable tool for building web applications in Python. It is well known for its integration with Pydantic models, which makes defining and validating data structures straightforward and efficient. In this guide, we explore how simple functions that return Pydantic models can seamlessly integrate with FastAPI.

Why Choose FastAPI and Pydantic?

  • FastAPI is a modern, high-performance web framework for building APIs with Python.
  • Supports OpenAPI and JSON Schema for automatic documentation and validation.
  • Supports AsyncIO for asynchronous programming leveraging the AsyncOpenAI() client

Code Example: Starting a FastAPI App with a POST Request

The following code snippet demonstrates how to start a FastAPI app with a POST endpoint. This endpoint accepts and returns data defined by a Pydantic model.

python
import instructor

from fastapi import FastAPI
from pydantic import BaseModel

# Enables response_model
client = instructor.from_provider(
    "openai/gpt-4.1-mini",
    async_client=True,
)
app = FastAPI()


class UserData(BaseModel):
    # This can be the model for the input data
    query: str


class UserDetail(BaseModel):
    name: str
    age: int


@app.post("/endpoint", response_model=UserDetail)
async def endpoint_function(data: UserData) -> UserDetail:
    user_detail = await client.create(
        response_model=UserDetail,
        messages=[
            {"role": "user", "content": f"Extract: `{data.query}`"},
        ],
    )
    return user_detail

Streaming Responses with FastAPI

FastAPI supports streaming responses, which is useful for returning large amounts of data. This feature is particularly useful when working with large language models (LLMs) that generate a large amount of data.

python hl_lines="6-7"
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from typing import Iterable
from pydantic import BaseModel

app = FastAPI()


class UserData(BaseModel):
    query: str


class UserDetail(BaseModel):
    name: str
    age: int


# Route to handle SSE events and return users
@app.post("/extract", response_class=StreamingResponse)
async def extract(data: UserData):
    users = await client.create(
        response_model=Iterable[UserDetail],
        stream=True,
        messages=[
            {"role": "user", "content": data.query},
        ],
    )

    async def generate():
        async for user in users:
            resp_json = user.model_dump_json()
            yield f"data: {resp_json}"
        yield "data: [DONE]"

    return StreamingResponse(generate(), media_type="text/event-stream")

Automatic Documentation with FastAPI

FastAPI leverages the OpenAPI specification to automatically generate a dynamic and interactive documentation page, commonly referred to as the /docs page. This feature is incredibly useful for developers, as it offers a live environment to test API endpoints directly through the browser.

To explore the capabilities of your API, follow these steps:

  1. Run the API using the Uvicorn command: uvicorn main:app --reload.
  2. Open your web browser and navigate to http://127.0.0.1:8000/docs.
  3. You will find an interactive UI where you can send different requests to your API and see the responses in real-time.

Concepts Fields (`docs/concepts/fields.md`)

title: Customizing Pydantic Models with Field Metadata description: Learn how to enhance Pydantic models with metadata using Field, including default values, JSON schema customization, and more.

The pydantic.Field function is used to customize and add metadata to fields of models. To learn more, check out the Pydantic documentation as this is a near replica of that documentation that is relevant to prompting.

Default values

The default parameter is used to define a default value for a field.

py
from pydantic import BaseModel, Field


class User(BaseModel):
    name: str = Field(default='John Doe')


user = User()
print(user)
#> name='John Doe'

You can also use default_factory to define a callable that will be called to generate a default value.

py
from uuid import uuid4

from pydantic import BaseModel, Field


class User(BaseModel):
    id: str = Field(default_factory=lambda: uuid4().hex)

!!! info

text
The `default` and `default_factory` parameters are mutually exclusive.

!!! note

text
If you use `typing.Optional`, it doesn't mean that the field has a default value of `None` you must use `default` or `default_factory` to define a default value. Then it will be considered `not required` when sent to the language model.

Using `Annotated`

The Field function can also be used together with Annotated.

py
from uuid import uuid4
from typing_extensions import Annotated
from pydantic import BaseModel, Field


class User(BaseModel):
    id: Annotated[str, Field(default_factory=lambda: uuid4().hex)]

Exclude

The exclude parameter can be used to control which fields should be excluded from the
model when exporting the model. This is helpful when you want to exclude fields that are not relevant to the model
generation like scratch_pad or chain_of_thought

See the following example:

py
from pydantic import BaseModel, Field
from datetime import date


class DateRange(BaseModel):
    chain_of_thought: str = Field(
        description="Reasoning behind the date range.", exclude=True
    )
    start_date: date
    end_date: date


date_range = DateRange(
    chain_of_thought="""
        I want to find the date range for the last 30 days.
        Today is 2021-01-30 therefore the start date
        should be 2021-01-01 and the end date is 2021-01-30""",
    start_date=date(2021, 1, 1),
    end_date=date(2021, 1, 30),
)
print(date_range.model_dump_json())
#> {"start_date":"2021-01-01","end_date":"2021-01-30"}

Omitting fields from schema sent to the language model

In some cases, you may wish to have the language model ignore certain fields in your model. You can do this by using Pydantic's SkipJsonSchema annotation. This omits a field from the JSON schema emitted by Pydantic (which instructor uses for constructing its prompts and tool definitions). For example:

py
from pydantic import BaseModel
from pydantic.json_schema import SkipJsonSchema
from typing import Union


class Response(BaseModel):
    question: str
    answer: str
    private_field: SkipJsonSchema[Union[str, None]] = None


assert "private_field" not in Response.model_json_schema()["properties"]

Note that because the language model will never return a value for private_field, you'll need a default value (this can be a generator via a declared Pydantic Field).

Customizing JSON Schema

There are some fields that are exclusively used to customise the generated JSON Schema:

  • title: The title of the field.
  • description: The description of the field.
  • examples: The examples of the field.
  • json_schema_extra: Extra JSON Schema properties to be added to the field.

These all work as great opportunities to add more information to the JSON schema as part of your prompt engineering.

Here's an example:

py
from pydantic import BaseModel, Field, SecretStr


class User(BaseModel):
    age: int = Field(description='Age of the user')
    name: str = Field(title='Username')
    password: SecretStr = Field(
        json_schema_extra={
            'title': 'Password',
            'description': 'Password of the user',
            'examples': ['123456'],
        }
    )


print(User.model_json_schema())
"""
{
    'properties': {
        'age': {'description': 'Age of the user', 'title': 'Age', 'type': 'integer'},
        'name': {'title': 'Username', 'type': 'string'},
        'password': {
            'description': 'Password of the user',
            'examples': ['123456'],
            'format': 'password',
            'title': 'Password',
            'type': 'string',
            'writeOnly': True,
        },
    },
    'required': ['age', 'name', 'password'],
    'title': 'User',
    'type': 'object',
}
"""

See Also

General notes on JSON schema generation

  • The JSON schema for Optional fields indicates that the value null is allowed.
  • The Decimal type is exposed in JSON schema (and serialized) as a string.
  • The JSON schema does not preserve namedtuples as namedtuples.
  • When they differ, you can specify whether you want the JSON schema to represent the inputs to validation or the outputs from serialization.
  • Sub-models used are added to the $defs JSON attribute and referenced, as per the spec.
  • Sub-models with modifications (via the Field class) like a custom title, description, or default value, are recursively included instead of referenced.
  • The description for models is taken from either the docstring of the class or the argument description to the Field class.

Concepts From Provider (`docs/concepts/from_provider.md`)

title: Using from_provider for Unified Client Creation description: Learn how to use from_provider to create Instructor clients for any LLM provider.

Using from_provider

The from_provider function creates Instructor clients for any LLM provider. It uses the same interface across all providers, making it easy to switch between models.

!!! note "V2 Preview"

text
`from_provider` routes to the v2 implementation by default for supported providers. Legacy provider-specific modes are deprecated, emit warnings, and map to generic modes (`Mode.TOOLS`, `Mode.JSON`, `Mode.JSON_SCHEMA`, `Mode.MD_JSON`).

Why Use from_provider?

from_provider provides:

  • Simple syntax: One function works for all providers
  • Automatic setup: Handles provider-specific configuration automatically
  • Consistent interface: Same code works across different providers
  • Type safety: Full IDE support with proper type inference
  • Easy switching: Change providers with a single string change

Basic Usage

The basic syntax is simple: instructor.from_provider("provider/model-name")

python
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


# Create a client for any provider
client = instructor.from_provider("openai/gpt-4o-mini")
# Or: instructor.from_provider("anthropic/claude-3-5-sonnet")
# Or: instructor.from_provider("google/gemini-2.5-flash")

# Use the client as usual
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
)

Supported Providers

from_provider supports all major LLM providers:

Cloud Providers

  • OpenAI: "openai/gpt-4o", "openai/gpt-4o-mini", "openai/gpt-5.4-mini"
  • Anthropic: "anthropic/claude-3-5-sonnet", "anthropic/claude-3-opus"
  • Google: "google/gemini-2.5-flash", "google/gemini-pro"
  • Azure OpenAI: "azure_openai/gpt-4o"
  • AWS Bedrock: "bedrock/claude-3-5-sonnet"
  • Vertex AI: "vertexai/gemini-pro" (or use "google/gemini-pro" with vertexai=True)

Fast Inference Providers

  • Groq: "groq/llama-3.1-70b"
  • Fireworks: "fireworks/mixtral-8x7b"
  • Together: "together/meta-llama/Llama-3-70b"
  • Anyscale: "anyscale/meta-llama/Llama-3-70b"

Other Providers

  • Mistral: "mistral/mistral-large"
  • Cohere: "cohere/command-r-plus"
  • Perplexity: "perplexity/llama-3.1-sonar"
  • DeepSeek: "deepseek/deepseek-chat"
  • xAI: "xai/grok-beta"
  • OpenRouter: "openrouter/meta-llama/llama-3.1-70b"
  • Ollama: "ollama/llama3.2" (local models)
  • LiteLLM: "litellm/gpt-4o" (meta-provider)

See the Integrations section for complete provider documentation.

Provider String Format

The provider string follows the format: "provider/model-name"

python
# Correct formats
"openai/gpt-4o"
"anthropic/claude-3-5-sonnet-20241022"
"google/gemini-2.5-flash"

# Incorrect formats (will raise errors)
"gpt-4o"  # Missing provider prefix
"openai"  # Missing model name
"openai/gpt-4o/mini"  # Too many slashes

Async Clients

Create async clients by setting async_client=True:

python
import asyncio
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


async def main() -> None:
    # Create async client
    async_client = instructor.from_provider("openai/gpt-4o-mini", async_client=True)

    # Use with await
    await async_client.create(
        response_model=User,
        messages=[{"role": "user", "content": "Extract: Alice is 25"}],
    )


asyncio.run(main())

Advanced Configuration

Custom API Keys

Pass API keys directly or use environment variables:

python
import instructor

# Pass API key directly
client = instructor.from_provider("openai/gpt-4o-mini", api_key="sk-your-key-here")

# Or use environment variables (recommended)
# export OPENAI_API_KEY=sk-your-key-here
client = instructor.from_provider("openai/gpt-4o-mini")

Mode Overrides

Override the default mode for a provider:

python
import instructor

# OpenAI defaults to TOOLS mode, but you can override
client = instructor.from_provider(
    "openai/gpt-4o-mini", mode=instructor.Mode.JSON  # Use JSON mode instead
)

Caching

Enable response caching:

python
from instructor.cache import AutoCache
import instructor

cache = AutoCache(maxsize=1000)

client = instructor.from_provider("openai/gpt-4o-mini", cache=cache)

Provider-Specific Options

Pass provider-specific options through **kwargs:

python
import os
import instructor

# For OpenAI
client = instructor.from_provider(
    "openai/gpt-4o-mini", organization="org-your-org-id", timeout=30.0
)

# OpenAI-compatible multi-model gateways (same client pattern via base_url)
# Example: DaoXE at https://api.daoxe.com/v1
client = instructor.from_provider(
    "openai/gpt-4o-mini",
    api_key="YOUR_GATEWAY_KEY",
    base_url="https://api.daoxe.com/v1",
)

# For Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet", max_tokens=4096)

# For Google with Vertex AI
google_api_key = os.environ.pop("GOOGLE_API_KEY", None)

client = instructor.from_provider(
    "google/gemini-pro",
    vertexai=True,
    project="your-project-id",
    location="us-central1",
)

if google_api_key is not None:
    os.environ["GOOGLE_API_KEY"] = google_api_key

Default Modes

Each provider uses a recommended default mode:

  • OpenAI: Mode.TOOLS
  • Anthropic: Mode.TOOLS
  • Google: Mode.TOOLS or Mode.JSON based on the model
  • Ollama: Mode.TOOLS (if supported) or Mode.JSON
  • Others: Mode.TOOLS or Mode.MD_JSON depending on capability

Legacy provider-specific modes still work but are deprecated. See the Mode Migration Guide for details.

Override these defaults with the mode parameter.

Error Handling

from_provider raises clear errors for common issues:

python
import instructor
from instructor.core.exceptions import ConfigurationError

try:
    # Invalid provider format
    client = instructor.from_provider("invalid-format")
except ConfigurationError as e:
    print(f"Configuration error: {e}")
    """
    Configuration error: Model string must be in format "provider/model-name" (e.g. "openai/gpt-5.4-mini" or "anthropic/claude-3-sonnet")
    """

try:
    # Unsupported provider
    client = instructor.from_provider("unsupported/provider")
except ConfigurationError as e:
    print(f"Unsupported provider: {e}")
    """
    Unsupported provider: Unsupported provider: unsupported. Supported providers are: ['openai', 'azure_openai', 'databricks', 'anthropic', 'google', 'generative-ai', 'vertexai', 'mistral', 'cohere', 'perplexity', 'groq', 'writer', 'bedrock', 'cerebras', 'deepseek', 'fireworks', 'ollama', 'openrouter', 'xai', 'litellm']
    """

try:
    # Missing required package
    client = instructor.from_provider("anthropic/claude-3")
except ImportError as e:
    print(f"Missing package: {e}")
    # Install with: pip install anthropic

Environment Variables

Most providers support environment variables for configuration:

bash
# OpenAI
export OPENAI_API_KEY=sk-your-key

# Anthropic
export ANTHROPIC_API_KEY=sk-ant-your-key

# Google
export GOOGLE_API_KEY=your-key

# Azure OpenAI
export AZURE_OPENAI_API_KEY=your-key
export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/

# AWS Bedrock
export AWS_DEFAULT_REGION=us-east-1
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret

# Others
export MISTRAL_API_KEY=your-key
export COHERE_API_KEY=your-key
export GROQ_API_KEY=your-key
export DEEPSEEK_API_KEY=your-key
export OPENROUTER_API_KEY=your-key

Switching Between Providers

One of the biggest advantages of from_provider is easy provider switching:

python
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


# Easy to switch providers
PROVIDER = "openai/gpt-4o-mini"  # Change this to switch
# PROVIDER = "anthropic/claude-3-5-sonnet"
# PROVIDER = "google/gemini-2.5-flash"

client = instructor.from_provider(PROVIDER)

# Same code works for all providers
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Bob is 40"}],
)

Best Practices

  1. Use environment variables: Store API keys in environment variables, not in code
  2. Use type hints: Let your IDE help with autocomplete and type checking
  3. Handle errors: Wrap provider creation in try-except blocks
  4. Cache when appropriate: Use caching for repeated requests
  5. Choose the right mode: Let defaults work, but override when needed

Comparison with Other Methods

from_provider vs. Manual Patching

python
# Old way (still works, but more verbose)
import openai
import instructor

openai_client = openai.OpenAI()
client = instructor.patch(openai_client)

# New way (recommended)
client = instructor.from_provider("openai/gpt-4o-mini")

from_provider vs. Provider-Specific Functions

Provider-specific helpers were removed. Use from_provider for all clients:

python
import instructor

openai_client = instructor.from_provider("openai/gpt-4o-mini")
anthropic_client = instructor.from_provider("anthropic/claude-3-5-sonnet")

Troubleshooting

Provider Not Found

If you get an error about an unsupported provider:

  1. Check the provider name spelling
  2. Verify the provider is in the supported list
  3. Check if you need to install an extra package: uv pip install "instructor[provider-name]"

Import Errors

If you get import errors:

bash
# Install the required package
# For Anthropic
uv pip install anthropic

# For Google
uv pip install google-genai

# For others, see integration docs

Invalid Model String

The model string must be in format "provider/model-name":

python
# Correct
"openai/gpt-4o"

# Incorrect
"gpt-4o"  # Missing provider
"openai"  # Missing model

Concepts Hooks (`docs/concepts/hooks.md`)

title: Hooks description: Learn how to use hooks for event handling, logging, and error handling in Instructor.

Hooks

Hooks let you intercept and handle events during the completion and parsing process. Use them to add logging, monitoring, or error handling at different stages of API interactions.

Hook Events

Event Description Handler Signature
completion:kwargs Arguments passed to completion def handler(*args, **kwargs)
completion:response Raw API response received def handler(response)
completion:usage Copied snapshot of cumulative retry usage def handler(usage, *, attempt_number)
completion:error Error during a retry attempt def handler(error, *, attempt_number, max_attempts, is_last_attempt)
parse:error Pydantic validation failed def handler(error)
completion:last_attempt Final retry attempt exhausted def handler(error, *, attempt_number, max_attempts, is_last_attempt)

completion:usage, completion:error, and completion:last_attempt handlers receive retry metadata as keyword arguments. Old-style error handlers that only accept error continue to work because the metadata is silently dropped for backward compatibility.

Cumulative Usage

completion:usage runs while all responses observed in the retry loop have
compatible usage metadata. Each event receives a separate deep copy, so
changing one snapshot does not affect later events or Instructor's accounting.
The copy is mutable and shared by handlers for that event; changing it can
affect a later handler of the same event. Snapshots are cumulative: replace the
previous value rather than summing them.

Compatibility currently means OpenAI Chat Completions or stable Anthropic
Messages usage types, including compatible subclasses. This is not a streaming
usage hook. See usage tracking
for other API/SDK limitations and local response-cache behavior.

python
import instructor

client = instructor.from_provider("openai/gpt-4.1-mini")


def record_usage(usage, *, attempt_number: int):
    print(f"Attempt {attempt_number}: {usage.total_tokens} total tokens")


client.on("completion:usage", record_usage)

For a successful Pydantic model or list response, Instructor also attaches the
final cumulative snapshot as _total_usage. Primitive response models should
use the hook because they cannot carry response metadata.

Registering and Removing Hooks

python
import instructor

client = instructor.from_provider("openai/gpt-4.1-mini")


def log_kwargs(**kwargs):
    print(f"Model: {kwargs.get('model')}")


def log_response(response):
    print(f"Response received: {response.id}")


# Register hooks
client.on("completion:kwargs", log_kwargs)
client.on("completion:response", log_response)

# Make a request
resp = client.create(
    messages=[{"role": "user", "content": "Hello, world!"}],
    response_model=str,
)

# Remove a specific hook
client.off("completion:kwargs", log_kwargs)

# Clear all hooks for an event
client.clear("completion:kwargs")

# Clear all hooks
client.clear()

You can use enum values or strings for hook names:

python
from instructor.hooks import HookName

client.on(HookName.COMPLETION_KWARGS, log_kwargs)  # Using enum
client.on("completion:kwargs", log_kwargs)  # Using string

Retry Metadata

completion:error and completion:last_attempt handlers can receive attempt metadata:

python
import instructor

client = instructor.from_provider("openai/gpt-4.1-mini")


def on_error(
    error: Exception,
    *,
    attempt_number: int,
    max_attempts: int | None,
    is_last_attempt: bool,
):
    print(f"Attempt {attempt_number}/{max_attempts or '?'} failed: {error}")
    if is_last_attempt:
        print("No more retries.")


client.on("completion:error", on_error)

Old-style handlers that only accept error continue to work unchanged — the metadata is silently dropped.

Practical Example: Logging

python
import instructor
from pydantic import BaseModel


class ErrorCounter:
    def __init__(self):
        self.count = 0

    def handle_error(self, error: Exception):
        self.count += 1
        print(f"Error #{self.count}: {type(error).__name__}: {error}")


client = instructor.from_provider("openai/gpt-4.1-mini")
counter = ErrorCounter()

client.on("completion:error", counter.handle_error)
client.on("parse:error", counter.handle_error)


class User(BaseModel):
    name: str
    age: int


try:
    user = client.create(
        messages=[{"role": "user", "content": "Extract: John is twenty"}],
        response_model=User,
    )
    print(f"Extracted: {user}")
except Exception as e:
    print(f"Final error: {e}")

print(f"Total errors: {counter.count}")

Error Handling

Monitor errors by type using Instructor's exception hierarchy:

python
import logging
import instructor
from instructor.core.exceptions import (
    IncompleteOutputException,
    InstructorRetryException,
    ValidationError,
    ProviderError,
)

logger = logging.getLogger(__name__)


def handle_error(error: Exception):
    if isinstance(error, IncompleteOutputException):
        logger.warning(f"Incomplete output: {error}")
    elif isinstance(error, ValidationError):
        logger.error(f"Validation failed: {error}")
    elif isinstance(error, ProviderError):
        logger.error(f"Provider error ({error.provider}): {error}")
    elif isinstance(error, InstructorRetryException):
        logger.critical(f"Retries exhausted after {error.n_attempts} attempts")
    else:
        logger.error(f"Unexpected error: {error}")


client = instructor.from_provider("openai/gpt-4.1-mini")
client.on("completion:error", handle_error)
client.on("parse:error", handle_error)

Hook Combination

Combine different hook sets using the + operator:

python
import instructor
from instructor.core.hooks import Hooks

# Create specialized hook sets
logging_hooks = Hooks()
logging_hooks.on("completion:kwargs", lambda **_kw: print("Logging kwargs"))

metrics_hooks = Hooks()
metrics_hooks.on("completion:response", lambda _resp: print("Recording metrics"))

# Combine hooks
combined = logging_hooks + metrics_hooks

# Or combine multiple at once
all_hooks = Hooks.combine(logging_hooks, metrics_hooks)

client = instructor.from_provider("openai/gpt-4.1-mini", hooks=combined)

Per-Call Hooks

Specify hooks for individual API calls:

python
import instructor
from instructor.core.hooks import Hooks
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


# Client with standard hooks
client_hooks = Hooks()
client_hooks.on("completion:kwargs", lambda **_kw: print("Standard logging"))

client = instructor.from_provider("openai/gpt-4.1-mini", hooks=client_hooks)

# Debug hooks for specific calls
debug_hooks = Hooks()
debug_hooks.on("parse:error", lambda _err: print(f"Debug: {_err}"))

# Per-call hooks combine with client hooks
user = client.create(
    messages=[{"role": "user", "content": "Extract: Alice is 25"}],
    response_model=User,
    hooks=debug_hooks,  # Both client and debug hooks run
)

Testing with Hooks

Use hooks to inspect requests and responses in tests:

python
import unittest
from unittest.mock import Mock
import instructor


class TestMyApp(unittest.TestCase):
    def test_completion(self):
        client = instructor.from_provider("openai/gpt-4.1-mini")
        mock_handler = Mock()

        client.on("completion:response", mock_handler)

        result = client.create(
            messages=[{"role": "user", "content": "Hello"}],
            response_model=str,
        )

        mock_handler.assert_called_once()
        response = mock_handler.call_args[0][0]
        self.assertEqual(response.model, "gpt-4.1-mini")

Custom Hooks

Create custom hook systems by extending the base pattern:

python
from enum import Enum
from instructor.hooks import HookName


class CustomHookName(str, Enum):
    CUSTOM_EVENT = "custom:event"
    # Include base hooks for compatibility
    COMPLETION_KWARGS = HookName.COMPLETION_KWARGS.value


class CustomHooks:
    def __init__(self):
        self._handlers: dict[str, list] = {}

    def on(self, hook_name: CustomHookName, handler):
        self._handlers.setdefault(hook_name.value, []).append(handler)

    def emit(self, hook_name: CustomHookName, payload):
        for handler in self._handlers.get(hook_name.value, []):
            handler(payload)


hooks = CustomHooks()
hooks.on(CustomHookName.CUSTOM_EVENT, lambda data: print(f"Custom: {data}"))
hooks.emit(CustomHookName.CUSTOM_EVENT, {"key": "value"})

See Also

Concepts Iterable (`docs/concepts/iterable.md`)

title: Iterable Extraction with Instructor - Stream Multiple Objects description: Use Iterable types to extract and stream multiple structured objects from LLM responses. Perfect for entity extraction and multi-task outputs.

Multi-Task and Streaming

Using an Iterable lets you extract multiple structured objects from a single LLM call, streaming them as they arrive. This is useful for entity extraction, multi-task outputs, and more.

We recommend using the create_iterable method for most use cases. It's simpler and less error-prone than manually specifying Iterable[...] and stream=True.

Here's a simple example showing how to extract multiple users from a single sentence. You can use either the recommended create_iterable method or the create method with Iterable[User]:

=== "Using create_iterable (recommended)"
```python
import instructor
from pydantic import BaseModel

text
client = instructor.from_provider("openai/gpt-4.1-mini")


class User(BaseModel):
    name: str
    age: int


resp = client.create_iterable(
    messages=[
        {
            "role": "user",
            "content": "Ivan is 28, lives in Moscow and his friends are Alex, John and Mary who are 25, 30 and 27 respectively",
        }
    ],
    response_model=User,
)

for user in resp:
    print(user)
    #> name='Ivan' age=28
    #> name='Alex' age=25
    #> name='John' age=30
    #> name='Mary' age=27
```
_Recommended for most use cases. Handles streaming and iteration for you._

=== "Using create with Iterable[User]"
```python
import instructor
from pydantic import BaseModel
from typing import Iterable

text
client = instructor.from_provider("openai/gpt-4.1-mini")


class User(BaseModel):
    name: str
    age: int


resp = client.create(
    messages=[
        {
            "role": "user",
            "content": "Ivan is 28, lives in Moscow and his friends are Alex, John and Mary who are 25, 30 and 27 respectively",
        }
    ],
    response_model=Iterable[User],
)

for user in resp:
    print(user)
    #> name='Ivan' age=28
    #> name='Alex' age=25
    #> name='John' age=30
    #> name='Mary' age=27
```
_Use this if you need more manual control or compatibility with legacy code._

We also support more complex extraction patterns such as Unions as you'll see below out of the box.

???+ warning

text
Unions don't work with Gemini because the AnyOf is not supported in the current response schema.

Synchronous Usage

=== "Using create"

text
```python
import instructor
from typing import Iterable, Union, Literal
from pydantic import BaseModel


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


client = instructor.from_provider("openai/gpt-4.1-mini", mode=instructor.Mode.TOOLS)

results = client.create(
    messages=[
        {"role": "system", "content": "You must always use tools"},
        {
            "role": "user",
            "content": "What is the weather in toronto and dallas and who won the super bowl?",
        },
    ],
    response_model=Iterable[Union[Weather, GoogleSearch]],
    stream=True,
)

for item in results:
    print(item)
    #> location='Toronto' units='metric'
    #> location='Dallas' units='imperial'
    #> query='Super Bowl winner'
```

=== "Using create_iterable (recommended)"

text
```python
import instructor
from typing import Union, Literal
from pydantic import BaseModel


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


client = instructor.from_provider("openai/gpt-4.1-mini", mode=instructor.Mode.TOOLS)

results = client.create_iterable(
    messages=[
        {"role": "system", "content": "You must always use tools"},
        {
            "role": "user",
            "content": "What is the weather in toronto and dallas and who won the super bowl?",
        },
    ],
    response_model=Union[Weather, GoogleSearch],
)

for item in results:
    print(item)
    #> location='Toronto' units='metric'
    #> location='Dallas' units='imperial'
    #> query='Super Bowl winner'
```

See Also

Asynchronous Usage

=== "Using create"

text
```python
import instructor
from typing import Iterable, Union, Literal
from pydantic import BaseModel
import asyncio


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


aclient = instructor.from_provider(
    "openai/gpt-4.1-mini", async_client=True, mode=instructor.Mode.TOOLS
)


async def main():
    results = await aclient.create(
        messages=[
            {"role": "system", "content": "You must always use tools"},
            {
                "role": "user",
                "content": "What is the weather in toronto and dallas and who won the super bowl?",
            },
        ],
        response_model=Iterable[Union[Weather, GoogleSearch]],
        stream=True,
    )
    async for item in results:
        print(item)
        #> location='Toronto' units='metric'
        #> location='Dallas' units='imperial'
        #> query='Super Bowl winner'


asyncio.run(main())
```

=== "Using create_iterable (recommended)"

text
```python
import asyncio
from typing import Literal, Union

import instructor
from pydantic import BaseModel


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


aclient = instructor.from_provider(
    "openai/gpt-4.1-mini", async_client=True, mode=instructor.Mode.TOOLS
)


async def iter_results():
    async for item in aclient.create_iterable(
        messages=[
            {"role": "system", "content": "You must always use tools"},
            {
                "role": "user",
                "content": "What is the weather in toronto and dallas and who won the super bowl?",
            },
        ],
        response_model=Union[Weather, GoogleSearch],
    ):
        yield item


async def main():
    async for item in iter_results():
        print(item)
        #> location='Toronto' units='metric'
        #> location='Dallas' units='imperial'
        #> query='Super Bowl winner'


asyncio.run(main())
```

Concepts Lists (`docs/concepts/lists.md`)

title: Streaming Lists with Instructor - Extract Multiple Objects description: Learn how to extract multiple structured objects from a single LLM call using streaming lists. Stream collections of Pydantic models as they're generated.

Multi-task and Streaming

A common use case of structured extraction is defining a single schema class and then making another schema to create a list to do multiple extraction

python
from typing import List
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


class Users(BaseModel):
    users: List[User]


print(Users.model_json_schema())
"""
{
    '$defs': {
        'User': {
            'properties': {
                'name': {'title': 'Name', 'type': 'string'},
                'age': {'title': 'Age', 'type': 'integer'},
            },
            'required': ['name', 'age'],
            'title': 'User',
            'type': 'object',
        }
    },
    'properties': {
        'users': {'items': {'$ref': '#/$defs/User'}, 'title': 'Users', 'type': 'array'}
    },
    'required': ['users'],
    'title': 'Users',
    'type': 'object',
}
"""

Defining a task and creating a list of classes is a common enough pattern that we make this convenient by making use of Iterable[T]. This lets us dynamically create a new class that:

  1. Has dynamic docstrings and class name based on the task
  2. Support streaming by collecting tokens until a task is received back out.

Extracting Tasks using Iterable

By using Iterable you get a very convenient class with prompts and names automatically defined:

python
import instructor
from typing import Iterable
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider(
    "openai/gpt-4.1-mini-1106",
    mode=instructor.Mode.JSON,
)

users = client.create(
    temperature=0.1,
    response_model=Iterable[User],
    stream=False,
    messages=[
        {
            "role": "user",
            "content": (
                "Consider this data: Jason is 10 and John is 30. "
                "Correctly segment it into entities. "
                "Make sure the JSON is correct."
            ),
        },
    ],
)
for user in users:
    print(user)
    #> name='Jason' age=10
    #> name='John' age=30

Streaming Tasks

We can also generate tasks as the tokens are streamed in by defining an Iterable[T] type.

Lets look at an example in action with the same class

python hl_lines="6 26"
import instructor
from typing import Iterable
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider(
    "openai/gpt-4.1-mini",
    mode=instructor.Mode.TOOLS,
)

users = client.create(
    temperature=0.1,
    stream=True,
    response_model=Iterable[User],
    messages=[
        {"role": "system", "content": "You are a perfect entity extraction system"},
        {"role": "user", "content": "Extract `Jason is 10 and John is 10`"},
    ],
    max_tokens=1000,
)

for user in users:
    print(user)
    #> name='Jason' age=10
    #> name='John' age=10

Asynchronous Streaming

I also just want to call out in this example that instructor also supports asynchronous streaming. This is useful when you want to stream a response model and process the results as they come in, but you'll need to use the async for syntax to iterate over the results.

python
import instructor
from typing import Iterable
from pydantic import BaseModel


class UserExtract(BaseModel):
    name: str
    age: int


async def print_iterable_results():
    client = instructor.from_provider(
        "openai/gpt-4.1-mini",
        async_client=True,
        mode=instructor.Mode.TOOLS,
    )

    model = await client.create(
        response_model=Iterable[UserExtract],
        max_retries=2,
        stream=True,
        messages=[
            {"role": "user", "content": "Make two up people"},
        ],
    )
    async for m in model:
        print(m)
        #> name='Alice' age=30
        #> name='Bob' age=25


import asyncio

asyncio.run(print_iterable_results())

See Also

Concepts Logging (`docs/concepts/logging.md`)

title: Logging and Monitoring with Instructor - Debug Guide description: Implement comprehensive logging for Instructor LLM calls. Track API usage, debug issues, and monitor performance with DEBUG level logging.

In order to see the requests made to OpenAI and the responses, you can set logging to DEBUG. This will show the requests and responses made to OpenAI. This can be useful for debugging and understanding the requests and responses made to OpenAI. I would love some contributions that make this a lot cleaner, but for now this is the fastest way to see the prompts.

text
/* Detailed source-code truncated for AI context efficiency. */

Provider initialization logs

from_provider() now emits structured logs at the INFO level when a provider
is initialized. Enable logging to see which provider and model are being used.

python
import logging
import instructor

logging.basicConfig(level=logging.INFO)

instructor.from_provider("openai/gpt-4.1-mini")

Example output:

text
INFO:instructor.auto_client:Initializing openai provider with model gpt-4.1-mini
INFO:instructor.auto_client:Client initialized

Concepts Maybe (`docs/concepts/maybe.md`)

title: Maybe Types and Optional Handling in Instructor description: Handle optional and nullable data with Maybe types in Instructor. Learn to work with potentially missing fields and optional responses from LLMs.

Handling Missing Data

The Maybe pattern is a concept in functional programming used for error handling. Instead of raising exceptions or returning None, you can use a Maybe type to encapsulate both the result and potential errors.

This pattern is particularly useful when making LLM calls, as providing language models with an escape hatch can effectively reduce hallucinations.

Defining the Model

Using Pydantic, we'll first define the UserDetail and MaybeUser classes.

python
from pydantic import BaseModel, Field
from typing import Optional


class UserDetail(BaseModel):
    age: int
    name: str
    role: Optional[str] = Field(default=None)


class MaybeUser(BaseModel):
    result: Optional[UserDetail] = Field(default=None)
    error: bool = Field(default=False)
    message: Optional[str] = Field(default=None)

    def __bool__(self):
        return self.result is not None

Notice that MaybeUser has a result field that is an optional UserDetail instance where the extracted data will be stored. The error field is a boolean that indicates whether an error occurred, and the message field is an optional string that contains the error message.

Defining the function

Once we have the model defined, we can create a function that uses the Maybe pattern to extract the data.

python
import instructor
from pydantic import BaseModel, Field
from typing import Optional

# This enables the `response_model` keyword
client = instructor.from_provider("openai/gpt-4.1-mini")


class UserDetail(BaseModel):
    age: int
    name: str
    role: Optional[str] = Field(default=None)


class MaybeUser(BaseModel):
    result: Optional[UserDetail] = Field(default=None)
    error: bool = Field(default=False)
    message: Optional[str] = Field(default=None)

    def __bool__(self):
        return self.result is not None


def extract(content: str) -> MaybeUser:
    return client.create(
        response_model=MaybeUser,
        messages=[
            {"role": "user", "content": f"Extract `{content}`"},
        ],
    )


user1 = extract("Jason is a 25-year-old scientist")
print(user1.model_dump_json(indent=2))
"""
{
  "result": {
    "age": 25,
    "name": "Jason",
    "role": "scientist"
  },
  "error": false,
  "message": null
}
"""

user2 = extract("Unknown user")
print(user2.model_dump_json(indent=2))
"""
{
  "result": null,
  "error": false,
  "message": null
}
"""

As you can see, when the data is extracted successfully, the result field contains the UserDetail instance. When an error occurs, the error field is set to True, and the message field contains the error message.

If you want to learn more about pattern matching, check out Pydantic's docs on Structural Pattern Matching

Concepts Migration (`docs/concepts/migration.md`)

title: Migration Guide description: Migrate from older Instructor patterns to the modern from_provider approach.

Migration Guide

This guide helps you migrate from older Instructor patterns to from_provider, the recommended approach for all providers.

Why Migrate?

  • Simpler code: Less boilerplate, easier to read
  • Consistent interface: Same pattern works for all providers
  • Better type safety: Improved IDE support
  • Future-proof: Recommended pattern going forward

Quick Reference

Old Pattern New Pattern
instructor.patch(openai.OpenAI()) instructor.from_provider("openai/model")
instructor.apatch(openai.AsyncOpenAI()) instructor.from_provider("openai/model", async_client=True)
from_openai(client) instructor.from_provider("openai/model")
from_anthropic(client) instructor.from_provider("anthropic/model")
from_genai(client) instructor.from_provider("google/model")
client.chat.completions.create(...) client.create(...)
client.messages.create(...) client.create(...)

Basic Migration

Before:

python
import openai
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


openai_client = openai.OpenAI()
client = instructor.patch(openai_client)

user = client.chat.completions.create(
    model="gpt-4o-mini",
    response_model=User,
    messages=[{"role": "user", "content": "Extract: John is 30"}],
)

After:

python
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4o-mini")

user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: John is 30"}],
)

Async Migration

Before:

python
import openai
import instructor

openai_client = openai.AsyncOpenAI()
client = instructor.apatch(openai_client)

user = await client.chat.completions.create(...)

After:

python
import instructor

client = instructor.from_provider("openai/gpt-4o-mini", async_client=True)

user = await client.create(...)

Provider-Specific Migrations

Anthropic

python
# Before (removed)
import anthropic
from instructor import from_anthropic

client = from_anthropic(anthropic.Anthropic())
user = client.messages.create(..., model="claude-3-5-sonnet")

# After
client = instructor.from_provider("anthropic/claude-3-5-sonnet")
user = client.create(...)

Google/Gemini

python
# Before (removed)
import google.genai as genai
from instructor import from_genai

client = from_genai(genai.Client(), model="gemini-pro")
user = client.generate_content(...)

# After
client = instructor.from_provider("google/gemini-pro")
user = client.create(messages=[...])

Configuration Options

Pass configuration directly to from_provider:

python
import instructor

# Mode configuration
client = instructor.from_provider("openai/gpt-4o-mini", mode=instructor.Mode.JSON)

# Custom API settings
client = instructor.from_provider(
    "openai/gpt-4o-mini",
    api_key="custom-key",
    organization="org-id",
    timeout=30.0,
)

Multiple Providers

Before:

python
import openai
import anthropic
import instructor
from instructor import from_anthropic

openai_client = instructor.patch(openai.OpenAI())
anthropic_client = from_anthropic(anthropic.Anthropic())

After:

python
import instructor

openai_client = instructor.from_provider("openai/gpt-4o-mini")
anthropic_client = instructor.from_provider("anthropic/claude-3-5-sonnet")

Migration Checklist

  1. Identify your current pattern: patch(), apatch(), or from_*() functions
  2. Find your model name: e.g., gpt-4o-mini, claude-3-5-sonnet
  3. Replace client creation: Use from_provider("provider/model")
  4. Update method calls: Change to client.create(...)
  5. Use standard message format: [{"role": "user", "content": "..."}]
  6. Test your code

Troubleshooting

Error Cause Solution
'Instructor' object has no attribute 'chat' Using old method call Use client.create() instead of client.chat.completions.create()
Invalid model string Wrong format Use "provider/model-name" format
Message format error Provider-specific format Use standard messages list format

Backward Compatibility

Legacy helpers have been removed:

  • instructor.patch() → Use from_provider instead
  • instructor.apatch() → Use from_provider with async_client=True
  • from_openai(), from_anthropic(), etc. → Use from_provider

Update all call sites before upgrading.

See Also

Concepts Mode Migration (`docs/concepts/mode-migration.md`)

title: Mode Migration Guide description: Migrate from provider-specific modes to the core modes in Instructor.

Mode Migration Guide

This guide helps you move from provider-specific modes to the core modes.
Core modes work across providers and are the recommended choice for new code.

!!! note "V2 Preview"

text
Provider-specific modes are deprecated in v2. They still work, emit warnings, and map to core modes.

Core Modes

These are the core modes you should use:

  • TOOLS: Tool or function calling
  • JSON_SCHEMA: Native schema support when a provider has it
  • MD_JSON: JSON extracted from text or code blocks
  • PARALLEL_TOOLS: Multiple tool calls in one response
  • RESPONSES_TOOLS: OpenAI Responses API tools

Quick Mapping

Use this table to replace legacy modes:

Legacy Mode Core Mode
FUNCTIONS TOOLS
TOOLS_STRICT TOOLS
ANTHROPIC_TOOLS TOOLS
ANTHROPIC_JSON MD_JSON
COHERE_TOOLS TOOLS
COHERE_JSON_SCHEMA JSON_SCHEMA
XAI_TOOLS TOOLS
XAI_JSON MD_JSON
MISTRAL_TOOLS TOOLS
MISTRAL_STRUCTURED_OUTPUTS JSON_SCHEMA
FIREWORKS_TOOLS TOOLS
FIREWORKS_JSON MD_JSON
CEREBRAS_TOOLS TOOLS
CEREBRAS_JSON MD_JSON
WRITER_TOOLS TOOLS
WRITER_JSON MD_JSON
BEDROCK_TOOLS TOOLS
BEDROCK_JSON MD_JSON
PERPLEXITY_JSON MD_JSON
VERTEXAI_TOOLS TOOLS
VERTEXAI_JSON MD_JSON
VERTEXAI_PARALLEL_TOOLS PARALLEL_TOOLS

Example: Anthropic

Before:

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "anthropic/claude-3-5-haiku-latest",
    mode=Mode.ANTHROPIC_TOOLS,
)

After:

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "anthropic/claude-3-5-haiku-latest",
    mode=Mode.TOOLS,
)

Example: Bedrock

Before:

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
    mode=Mode.BEDROCK_TOOLS,
)

After:

python
import instructor
from instructor import Mode

client = instructor.from_provider(
    "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
    mode=Mode.BEDROCK_TOOLS,
)

Notes

  • Legacy modes still work but show a deprecation warning.
  • Use core modes for new code and docs.
  • Core tests are parameterized by provider and mode for consistent coverage.
  • Streaming extraction is now handled by provider handlers instead of the DSL.
  • Legacy ResponseSchema.parse_* helpers are deprecated. Use process_response or
    ResponseSchema.from_response with core modes so the v2 registry handles parsing.
  • See Mode Comparison for details.

Concepts Models (`docs/concepts/models.md`)

title: Using Pydantic Models for Structured Outputs description: Learn how to define LLM output schemas with Pydantic models.

Response Model

Define LLM output schemas using pydantic.BaseModel. For more details, see the Pydantic documentation.

After defining a Pydantic model, use it as the response_model in your client create calls. The response_model parameter:

  • Defines the schema and prompts for the language model
  • Validates the response from the API
  • Returns a Pydantic model instance

Prompting

Use docstrings and field annotations to define the prompt for generating responses.

python
from pydantic import BaseModel, Field
import instructor


class User(BaseModel):
    """
    This is the prompt that will be used to generate the response.
    Any instructions here will be passed to the language model.
    """

    name: str = Field(description="The name of the user.")
    age: int = Field(description="The age of the user.")


client = instructor.from_provider("openai/gpt-4o-mini")

user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
)

Docstrings, types, and field annotations are used to generate the prompt. The create method uses this prompt to generate the response.

Optional Values

Use Optional and default to make fields optional when sent to the language model.

python
from pydantic import BaseModel, Field
from typing import Optional
import instructor


class User(BaseModel):
    name: str = Field(description="The name of the user.")
    age: int = Field(description="The age of the user.")
    email: Optional[str] = Field(description="The email of the user.", default=None)


client = instructor.from_provider("openai/gpt-4o-mini")

user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
)

Fields can also be omitted from the schema sent to the language model using Pydantic's SkipJsonSchema annotation. See Fields for details.

Dynamic Model Creation

Create models at runtime using Pydantic's create_model function:

python
from pydantic import BaseModel, create_model


class FooModel(BaseModel):
    foo: str
    bar: int = 123


BarModel = create_model(
    'BarModel',
    apple=(str, 'russet'),
    banana=(str, 'yellow'),
    __base__=FooModel,
)
print(BarModel)
#> <class '__main__.BarModel'>
print(BarModel.model_fields.keys())
#> dict_keys(['foo', 'bar', 'apple', 'banana'])

??? notes "When would I use this?"

text
Consider a situation where the model is dynamically defined, based on some configuration or database. For example, we could have a database table that stores the properties of a model for
some model name or id. We could then query the database for the properties of the model and use that to create the model.

```sql
SELECT property_name, property_type, description
FROM prompt
WHERE model_name = {model_name}
```

We can then use this information to create the model.

```python
from pydantic import BaseModel, create_model, Field
from typing import List

types = {
    'string': str,
    'integer': int,
    'boolean': bool,
    'number': float,
    'List[str]': List[str],
}

# Mocked cursor.fetchall()
cursor = [
    ('name', 'string', 'The name of the user.'),
    ('age', 'integer', 'The age of the user.'),
    ('email', 'string', 'The email of the user.'),
]

BarModel = create_model(
    'User',
    **{
        property_name: (types[property_type], Field(description=description))
        for property_name, property_type, description in cursor
    },
    __base__=BaseModel,
)

print(BarModel.model_json_schema())
"""
{
    'properties': {
        'name': {
            'description': 'The name of the user.',
            'title': 'Name',
            'type': 'string',
        },
        'age': {
            'description': 'The age of the user.',
            'title': 'Age',
            'type': 'integer',
        },
        'email': {
            'description': 'The email of the user.',
            'title': 'Email',
            'type': 'string',
        },
    },
    'required': ['name', 'age', 'email'],
    'title': 'User',
    'type': 'object',
}
"""
```

This would be useful when different users have different descriptions for the same model. We can use the same model but have different prompts for each user.

Adding Behavior

Add methods to Pydantic models like any Python class. This lets you add custom logic to your models.

python
from pydantic import BaseModel
from typing import Literal

import instructor

client = instructor.from_provider("openai/gpt-4.1-mini")


class SearchQuery(BaseModel):
    query: str
    query_type: Literal["web", "image", "video"]

    def execute(self):
        print(f"Searching for {self.query} of type {self.query_type}")
        #> Searching for cat of type image
        return "Results for cat"


query = client.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Search for a picture of a cat"}],
    response_model=SearchQuery,
)

results = query.execute()
print(results)
#> Results for cat

Now we can call execute on our model instance after extracting it from a language model. If you want to see more examples of this checkout our post on RAG is more than embeddings

See Also

Concepts Multimodal (`docs/concepts/multimodal.md`)

title: Seamless Multimodal Interactions with Instructor description: Learn how the Image, PDF and Audio class in Instructor enables seamless handling of multimodal content across different AI models.

title: Multimodal Processing with Instructor - Vision and Audio description: Process images, audio, and video with Instructor for multimodal structured outputs. Extract data from visual content using GPT-4 Vision and Gemini models.

Multimodal

We've provided a few different sample files for you to use to test out these new features. All examples below use these files.

  • (Image) : An image of some blueberry plants image.jpg
  • (Audio) : A Recording of the Original Gettysburg Address : gettysburg.wav
  • (PDF) : A sample PDF file which contains a fake invoice invoice.pdf
    Instructor provides a unified, provider-agnostic interface for working with multimodal inputs like images and PDFs.

Instructor provides a unified, provider-agnostic interface for working with multimodal inputs like images, PDFs, and audio files.

With Instructor's multimodal objects, you can easily load media from URLs, Google Cloud Storage URLs, local files, or base64 strings using a consistent API that works across different AI providers (OpenAI, Anthropic, Mistral, etc.).

Instructor handles all the provider-specific formatting requirements behind the scenes, ensuring your code remains clean and future-proof as provider APIs evolve. Let's see how to use the Image, Audio and PDF classes.

`Image`

This class represents an image that can be loaded from a URL or file path. It provides a set of methods to create Image instances from different sources (Eg. URLs, paths and base64 strings). The following shows which methods are supported for the individual providers.

Method OpenAI Anthropic Google GenAI
from_url() ✅ ✅ ✅
from_gs_url() ✅ ✅ ✅
from_path() ✅ ✅ ✅
from_base64() ✅ ✅ ✅
autodetect() ✅ ✅ ✅

We also support Anthropic Prompt Caching for images with the `ImageWith

Usage

By using the Image class, we can abstract away the differences between the different formats, allowing you to work with a unified interface.

You can create an Image instance from a URL, Google Cloud Storage (GCS) URL, or file path using the from_url, from_gs_url, or from_path methods. The Image class will automatically convert the image to a base64-encoded string and include it in the API request.

python
import instructor
from instructor.processing.multimodal import Image
from pydantic import BaseModel


class ImageDescription(BaseModel):
    items: list[str]


# Use our sample image provided above.
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg"

client = instructor.from_provider("openai/gpt-4.1-mini")

response = client.create(
    response_model=ImageDescription,
    messages=[
        {
            "role": "user",
            "content": [
                "What is in this image?",
                Image.from_url(url),
            ],
        }
    ],
)

print(response)
"""
description='Blueberry bushes with clusters of ripe and unripe blueberries. The berries are blue to purplish in color, and the leaves are green. The sky in the background is cloudy.' items=['blueberry bushes', 'ripe blueberries', 'unripe blueberries', 'green leaves', 'cloudy sky']
"""

Google Cloud Storage Support

Instructor now supports loading images directly from Google Cloud Storage URLs. This is particularly useful when working with images stored in GCS buckets.

python
import instructor
from instructor.processing.multimodal import Image
from pydantic import BaseModel


class ImageDescription(BaseModel):
    items: list[str]


# Load image from GCS URL (must be publicly accessible)
gs_url = "gs://my-bucket/path/to/image.jpg"

client = instructor.from_provider("openai/gpt-4.1-mini")

response = client.create(
    response_model=ImageDescription,
    messages=[
        {
            "role": "user",
            "content": [
                "What is in this image?",
                Image.from_gs_url(gs_url),
            ],
        }
    ],
)

print(response)
"""
description='A sample image loaded from Google Cloud Storage.' items=['sample image']
"""

Note: GCS URLs must point to publicly accessible objects. The from_gs_url method converts gs:// URLs to https://storage.googleapis.com/ URLs for access.

We also provide an autodetect_images keyword argument that allows you to provide URLs, GCS URLs, or file paths as normal strings when you set it to true. The system will automatically detect and handle different media types including images, audio, and PDFs.

You can see an example below.

python
import instructor
from pydantic import BaseModel


class ImageDescription(BaseModel):
    items: list[str]


# Download a sample image for demonstration
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg"

client = instructor.from_provider("openai/gpt-4.1-mini")

response = client.create(
    response_model=ImageDescription,
    autodetect_images=True,  # Set this to True
    messages=[
        {
            "role": "user",
            "content": ["What is in this image?", url],
        }
    ],
)

print(response)
"""
description='The image shows a close-up of a blueberry bush with ripe blueberries and green leaves. The background includes more blueberry bushes and a cloudy sky.' items=['Blueberry bush', 'Ripe blueberries', 'Green leaves', 'Cloudy sky']
"""

If you'll like to support Anthropic prompt caching with images, we provide the ImageWithCacheControl Object to do so. Simply use the from_image_params method and you'll be able to leverage Anthropic's prompt caching.

python
import instructor
from instructor.processing.multimodal import ImageWithCacheControl
from pydantic import BaseModel


class ImageDescription(BaseModel):
    items: list[str]


# Download a sample image for demonstration
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg"

client = instructor.from_provider("anthropic/claude-3-5-sonnet-20240620")

response, completion = client.create_with_completion(
    response_model=ImageDescription,
    autodetect_images=True,  # Set this to True
    messages=[
        {
            "role": "user",
            "content": [
                "What is in this image?",
                ImageWithCacheControl.from_image_params(
                    {
                        "source": url,
                        "cache_control": {
                            "type": "ephemeral",
                        },
                    }
                ),
            ],
        }
    ],
    max_tokens=1000,
)

print(response)
"""
description='A bush with numerous clusters of blueberries surrounded by green leaves, under a cloudy sky.' items=['blueberries', 'green leaves', 'cloudy sky']
"""

print(completion.usage.cache_creation_input_tokens)
#> 1820

By leveraging Instructor's multimodal capabilities, you can focus on building your application logic without worrying about the intricacies of each provider's image handling format. This not only saves development time but also makes your code more maintainable and adaptable to future changes in AI provider APIs.

`Audio`

Note : Only OpenAI and Gemini support audio files at the moment. For Gemini, we're passing in the raw bytes as bytes for this feature. If you'd like to use the Files API instead, we also support it, read more at to see how to do so.

Similar to the Image class, we provide methods to create Audio instances.

Method OpenAI Google GenAI
from_url() ✅ ✅
from_gs_url() ✅ ✅
from_path() ✅ ✅
from_base64() ✅ ✅
autodetect() ✅ ✅

The Audio class represents an audio file that can be loaded from a URL, Google Cloud Storage URL, or file path. It provides methods to create Audio instances using the from_path, from_url, from_gs_url, from_base64, and autodetect methods.

The Audio class will automatically convert it to the right format and include it in the API request.

python
from pydantic import BaseModel
import instructor
from instructor.processing.multimodal import Audio

# Initialize the client
client = instructor.from_provider("openai/gpt-4o-audio-preview")


# Define our response model
class AudioDescription(BaseModel):
    summary: str
    transcript: str


url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/gettysburg.wav"

# Make the API call with the audio file
resp = client.create(
    response_model=AudioDescription,
    modalities=["text"],
    audio={"voice": "alloy", "format": "wav"},
    messages=[
        {
            "role": "user",
            "content": [
                "Extract the following information from the audio:",
                Audio.from_url(url),
            ],
        },
    ],
)

print(resp)
"""
summary='This excerpt is from a famous historical speech discussing the founding principles of equality and liberty, and the ongoing civil war testing the endurance of those principles.' transcript='Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal. Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure.'
"""

Google Cloud Storage Support

You can also load audio files directly from Google Cloud Storage:

python
from pydantic import BaseModel
import instructor
from instructor.processing.multimodal import Audio

# Initialize the client
client = instructor.from_provider("openai/gpt-4o-audio-preview")


# Define our response model
class AudioDescription(BaseModel):
    summary: str
    transcript: str


# Load audio from GCS URL (must be publicly accessible)
gs_url = "gs://my-bucket/path/to/audio.wav"

# Make the API call with the GCS audio file
resp = client.create(
    response_model=AudioDescription,
    modalities=["text"],
    audio={"voice": "alloy", "format": "wav"},
    messages=[
        {
            "role": "user",
            "content": [
                "Extract the following information from the audio:",
                Audio.from_gs_url(gs_url),
            ],
        },
    ],
)

print(resp)
"""
summary='A short historical speech about equality and liberty.' transcript='Four score and seven years ago our fathers brought forth...'
"""

`PDF`

The PDF class represents a PDF file that can be loaded from a URL or file path.

It provides methods to create PDF instances and is currently supported for OpenAI, Mistral, GenAI, Anthropic, and Bedrock client integrations.

Method OpenAI Anthropic Google GenAI Mistral Bedrock
from_url() ✅ ✅ ✅ ✅ ✅
from_gs_url() ✅ ✅ ✅ ✅ ✅
from_path() ✅ ✅ ✅ ❎ ✅
from_base64() ✅ ✅ ✅ ❎ ✅
autodetect() ✅ ✅ ✅ ✅ ✅

For Gemini, we also provide two additional methods that make working with the google-genai files package easy which you can access in the PDFWithGenaiFile object.

For Anthropic, you can enable caching with the PDFWithCacheControl object. Note that this has caching configured by default for easy usage.

We provide examples of how to use all three object classes below.

For Bedrock, you can convert a PDF into the Bedrock-native document format with PDF.to_bedrock() and include the result in the message content list.

Usage

python
import instructor
from pydantic import BaseModel
from instructor.processing.multimodal import PDF

# Set up the client
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
client = instructor.from_provider("openai/gpt-4.1-mini")


# Create a model for analyzing PDFs
class Invoice(BaseModel):
    total: float
    items: list[str]


# Load and analyze a PDF
response = client.create(
    response_model=Invoice,
    messages=[
        {
            "role": "user",
            "content": [
                "Analyze this document",
                PDF.from_url(url),
            ],
        }
    ],
)

print(response)
"""
total=220.0 items=['English Tea - 2 units at $100 each', 'Tofu - 10 units at $2 each']
"""

Google Cloud Storage Support

You can load PDF files directly from Google Cloud Storage URLs:

python
import instructor
from pydantic import BaseModel
from instructor.processing.multimodal import PDF

# Set up the client
gs_url = "gs://my-bucket/path/to/document.pdf"
client = instructor.from_provider("openai/gpt-4.1-mini")


# Create a model for analyzing PDFs
class Invoice(BaseModel):
    total: float
    items: list[str]


# Load and analyze a PDF from GCS (must be publicly accessible)
response = client.create(
    response_model=Invoice,
    messages=[
        {
            "role": "user",
            "content": [
                "Analyze this document",
                PDF.from_gs_url(gs_url),
            ],
        }
    ],
)

print(f"Total = {response.total:.0f}, items = {response.items}")
#> Total = 220, items = ['English Tea', 'Tofu']

Caching

If you'd like to cache the PDF for Anthropic, we provide the PDFWithCacheControl class which has caching configured by default.

python
import instructor
from pydantic import BaseModel
from instructor.processing.multimodal import PDFWithCacheControl

# Set up the client
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
client = instructor.from_provider("anthropic/claude-3-5-sonnet-20240620")


# Create a model for analyzing PDFs
class Invoice(BaseModel):
    total: float
    items: list[str]


# Load and analyze a PDF
response, completion = client.create_with_completion(
    response_model=Invoice,
    messages=[
        {
            "role": "user",
            "content": [
                "Analyze this document",
                PDFWithCacheControl.from_url(url),
            ],
        }
    ],
    max_tokens=1000,
)

print(f"Total = {response.total:.0f}, items = {response.items}")
#> Total = 220, items = ['English Tea', 'Tofu']

print(completion.usage.cache_creation_input_tokens)
#> 2091

Using Files

We also provide a convenient wrapper around the Files API - allowing you to use both uploaded files and to block the main thread while your file is uploading.

In this example below, we download the sample PDF and then upload it using the Files api provided by the google.genai sdk.

python
import instructor
from pydantic import BaseModel
from instructor.processing.multimodal import PDFWithGenaiFile
import requests

# Set up the client
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
client = instructor.from_provider("google/gemini-2.5-flash")


# Create a model for analyzing PDFs
class Invoice(BaseModel):
    total: float
    items: list[str]


# Load and analyze a PDF
with requests.get(url) as download_response:
    pdf_data = download_response.content
    with open("./invoice.pdf", "wb") as f:
        f.write(pdf_data)

response = client.create(
    response_model=Invoice,
    messages=[
        {
            "role": "user",
            "content": [
                "Analyze this document",
                PDFWithGenaiFile.from_new_genai_file(
                    file_path="./invoice.pdf",
                    retry_delay=10,
                    max_retries=20,
                ),
            ],
        }
    ],
)

print(response)
#> total=220.0 items=['English Tea', 'Tofu']

If you've already uploaded your file ahead of time, we also support it. Just provide us with the file name as seen below

python
import instructor
from pydantic import BaseModel
from instructor.processing.multimodal import PDFWithGenaiFile
import requests

# Set up the client
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
client = instructor.from_provider("google/gemini-2.5-flash")


# Create a model for analyzing PDFs
class Invoice(BaseModel):
    total: float
    items: list[str]


# Load and analyze a PDF
with requests.get(url) as download_response:
    pdf_data = download_response.content
    with open("./invoice.pdf", "wb") as f:
        f.write(pdf_data)

file = client.files.upload(
    file="invoice.pdf",
)

response = client.create(
    response_model=Invoice,
    messages=[
        {
            "role": "user",
            "content": [
                "Analyze this document",
                PDFWithGenaiFile.from_existing_genai_file(file_name=file.name),
            ],
        }
    ],
)

print(response)
#> total=220.0 items=['English Tea', 'Tofu']

This way you have more granular control over how the file is uploaded, potentially also processing multiple file uploads at once too.

Concepts Parallel (`docs/concepts/parallel.md`)

title: Parallel Tools description: Learn about parallel tools in OpenAI, Google, and Anthropic.

See Also

Parallel Tools

Parallel Tool Calling is a feature that allows you to call multiple functions in a single request.

!!! warning "Experimental Feature"

text
Parallel Tool Calling is supported by Google, OpenAI, and Anthropic. Make sure to use the equivalent parallel tool `mode` for your client.

Understanding Parallel Tool Calling

Parallel Function Calling helps you to significantly reduce the latency of your application without having to build a parent schema as a wrapper around these tool calls.

=== "OpenAI"

text
```python hl_lines="20 32"
from __future__ import annotations

import instructor

from typing import Iterable, Literal
from pydantic import BaseModel


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


client = instructor.from_provider(
    "openai/gpt-4.1-mini",
    mode=instructor.Mode.PARALLEL_TOOLS,
)
function_calls = client.create(
    messages=[
        {"role": "system", "content": "You must always use tools"},
        {
            "role": "user",
            "content": "What is the weather in toronto and dallas and who won the super bowl?",
        },
    ],
    response_model=Iterable[Weather | GoogleSearch],
)

for fc in function_calls:
    print(fc)
    #> location='Toronto' units='metric'
    #> location='Dallas' units='metric'
    #> query='who won the super bowl 2023'
```

=== "Vertex AI"

text
```python
from typing import Iterable, Literal

import instructor
from pydantic import BaseModel

try:
    import vertexai
    import vertexai.generative_models as gm
    from instructor import from_vertexai
except ImportError:
    vertexai = None
    gm = None
    from_vertexai = None


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


if from_vertexai is not None and vertexai is not None and gm is not None:
    vertexai.init(project="your-project-id", location="us-central1")
    client = from_vertexai(
        gm.GenerativeModel("gemini-2.5-flash"),
        mode=instructor.Mode.PARALLEL_TOOLS,
    )
    function_calls = client.create(
        messages=[
            {
                "role": "user",
                "content": "What is the weather in toronto and dallas and who won the super bowl?",
            },
        ],
        response_model=Iterable[Weather | GoogleSearch],
    )

    for fc in function_calls:
        print(fc)
        #> location='Toronto' units='metric'
        #> location='Dallas' units='imperial'
        #> query='who won the super bowl'
```

=== "Anthropic"

text
```python hl_lines="20 32"
import instructor
from typing import Iterable, Literal
from pydantic import BaseModel


class Weather(BaseModel):
    location: str
    units: Literal["imperial", "metric"]


class GoogleSearch(BaseModel):
    query: str


client = instructor.from_provider(
    "anthropic/claude-3-7-sonnet-latest",
    mode=instructor.Mode.PARALLEL_TOOLS,
)
function_calls = client.create(
    messages=[
        {"role": "system", "content": "You must always use tools"},
        {
            "role": "user",
            "content": "What is the weather in toronto and dallas and who won the super bowl?",
        },
    ],
    response_model=Iterable[Weather | GoogleSearch],
)

for fc in function_calls:
    print(fc)
    #> location='Toronto' units='metric'
```

We need to set the response model to Iterable[Weather | GoogleSearch] to indicate that the response will be a list of Weather and GoogleSearch objects.

This is necessary because the response will be a list of objects, and we need to specify the types of the objects in the list. This returns an iterable which you can then iterate over

Concepts Partial (`docs/concepts/partial.md`)

title: Streaming Partial Responses with Instructor and OpenAI description: Learn to utilize field-level streaming with Instructor and OpenAI for incremental responses in Python.

Streaming Partial Responses

!!! info "Literal"

text
If the data structure you're using has literal values, you need to make sure to import the `PartialLiteralMixin` mixin.

```python
from typing import Literal
from pydantic import BaseModel
from instructor.dsl.partial import PartialLiteralMixin


class User(BaseModel, PartialLiteralMixin):
    name: str
    age: int
    category: Literal["admin", "user", "guest"]


# The rest of your code below
```

This is because `jiter` throws an error otherwise if it encounters a incomplete Literal value while it's being streamed in

Field level streaming provides incremental snapshots of the current state of the response model that are immediately usable. This approach is particularly relevant in contexts like rendering UI components.

Instructor supports this pattern by making use of create_partial. This lets us dynamically create a new class that treats all of the original model's fields as Optional.

Understanding Partial Responses

Consider what happens whene we define a response model:

python
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int

If we streamed json out from OpenAI, we would only be able to parse when the object is completed returned!

text
{"name": "Jo
{"name": "John", "ag
{"name": "John", "age:
{"name": "John", "age": 25} # Completed

When specifying a create_partial and setting stream=True, the response from instructor becomes a Generator[T]. As the generator yields results, you can iterate over these incremental updates. The last value yielded by the generator represents the completed extraction!

text
{"name": "Jo                 => User(name="Jo", age=None)
{"name": "John", "ag         => User(name="John", age=None)
{"name": "John", "age:       => User(name="John", age=None)
{"name": "John", "age": 25}  => User(name="John", age=25)

!!! warning "Limited Validator Support"

text
Due to the streaming nature of the response model, we do not support validators since they would not be able to be applied to the streaming response.

Let's look at an example of streaming an extraction of conference information, that would be used to stream in an react component.

text
/* Detailed source-code truncated for AI context efficiency. */

This will output the following:

Asynchronous Streaming

I also just want to call out in this example that instructor also supports asynchronous streaming. This is useful when you want to stream a response model and process the results as they come in, but you'll need to use the async for syntax to iterate over the results.

python
import instructor
from pydantic import BaseModel

client = instructor.from_provider(
    "openai/gpt-5-nano",
    async_client=True,
)


class User(BaseModel):
    name: str
    age: int


async def print_partial_results():
    user = client.create_partial(
        response_model=User,
        max_retries=2,
        stream=True,
        messages=[
            {"role": "user", "content": "Jason is 12 years old"},
        ],
    )
    async for m in user:
        print(m)
        #> name=None age=None
        #> name=None age=None
        #> name=None age=None
        #> name='' age=None
        #> name='Jason' age=None
        #> name='Jason' age=None
        #> name='Jason' age=None
        #> name='Jason' age=None
        #> name='Jason' age=12
        #> name='Jason' age=12


import asyncio

asyncio.run(print_partial_results())

Concurrent requests and direct handlers

The core runtime passes each request's stream boolean explicitly. OpenAI-compatible,
Anthropic, Mistral, and xAI mode handlers honor that value even when another request
using the same model prepared a stream, failed, or was cancelled. This routing rule
does not change each provider's event format or output type.

When calling those handlers directly, pass stream=True or stream=False to
parse_response for overlapping requests. Omitting stream preserves the legacy
one-shot inference from prepare_request. That fallback is keyed by model class
and cannot distinguish overlapping calls using the same class. Explicit parsing
also retires a pending legacy marker; do not mix explicit and implicit parsing
for overlapping calls with the same model.

Sync partial parsing still materializes its results and may raise validation errors
inside parse_response; async partial validation occurs during iteration. Closing
a parsed generator early is not a promise that it closes the underlying SDK stream.
Retain and close the source stream according to its SDK's ownership contract.
Cancellation while awaiting a source propagates to that source.

The native xAI client has a separate SDK streaming path; the shared routing rule
here concerns its registry mode handlers. Providers without class-keyed stream
markers retain their existing routing. Async request preparation still performs
synchronous cache and media operations; this change does not offload them or alter
remote-media connection checks.

See Also

Concepts Patching (`docs/concepts/patching.md`)

title: How Instructor Patches LLM Clients description: Learn how Instructor adds structured output capabilities to LLM clients through patching.

Patching

Patching adds structured output features to LLM client libraries. This page explains how it works. For most users, from_provider is simpler than manual patching.

!!! tip "Recommended Approach"
Use from_provider instead of manual patching. It works the same way across all providers. See the Migration Guide if you're using older patching patterns.

What is Patching?

Patching adds new features to LLM client objects without changing their original code. When Instructor patches a client, it adds:

  • New parameters: response_model, max_retries, and context to completion methods
  • Validation: Checks responses against Pydantic models
  • Retry logic: Retries when validation fails
  • Compatibility: The patched client still works with all original methods

How Patching Works

When Instructor patches a client, it:

  1. Wraps the completion method: Intercepts calls to create() or chat.completions.create()
  2. Converts schemas: Changes Pydantic models into provider-specific formats (JSON schema, tool definitions, etc.)
  3. Validates responses: Checks LLM outputs against your Pydantic model
  4. Handles retries: Retries with validation feedback if needed
  5. Returns typed objects: Converts validated JSON into Pydantic model instances

Patching Modes

Different providers support different modes for structured extraction. Instructor automatically selects the best mode for each provider, but you can override it:

Tool Calling (TOOLS)

Uses the provider's function/tool calling API. This is the default for OpenAI.

Supported by: OpenAI, Anthropic, Google, Ollama (for supported models)

JSON Mode

Instructs the model to return JSON directly. Works with most providers.

Supported by: OpenAI, Anthropic, Google, Ollama, and most providers

Markdown JSON (MD_JSON)

Asks for JSON wrapped in markdown. Only use for specific providers like Databricks.

Supported by: Databricks, some vision models

Default Modes by Provider

Each provider uses a recommended default mode:

  • OpenAI: Mode.TOOLS (function calling)
  • Anthropic: Mode.TOOLS (tool use)
  • Google: Mode.TOOLS (function calling)
  • Ollama: Mode.TOOLS (if model supports it) or Mode.JSON
  • Others: Provider-specific defaults

When using from_provider, these defaults are applied automatically. You can override them with the mode parameter.

Manual Patching (Advanced)

If you need to patch a client manually (not recommended for most users):

python
import openai
import instructor
from pydantic import BaseModel


class YourModel(BaseModel):
    message: str


# Create the base client
openai_client = openai.OpenAI()

# Patch it manually
client = instructor.patch(openai_client, mode=instructor.Mode.TOOLS)

# Now use it
response = client.chat.completions.create(
    response_model=YourModel,
    messages=[{"role": "user", "content": "Say hello"}],
)

However, using from_provider is simpler and recommended:

python
import instructor
from pydantic import BaseModel


# Simpler approach
class YourModel(BaseModel):
    message: str


client = instructor.from_provider("openai/gpt-4o-mini")
_response = client.create(
    response_model=YourModel,
    messages=[{"role": "user", "content": "Say hello"}],
)

What Gets Patched?

Instructor adds these features to patched clients:

New Parameters

  • response_model: A Pydantic model or type that defines the expected output structure
  • max_retries: Number of retry attempts if validation fails (default: 0)
  • context: Additional context for validation hooks

Enhanced Methods

The patched client's create() method:

  • Accepts response_model parameter
  • Validates responses automatically
  • Retries on validation failures
  • Returns typed Pydantic objects instead of raw responses

Provider-Specific Considerations

OpenAI

  • Default mode: TOOLS (function calling)
  • Supports streaming with structured outputs

Anthropic

  • Default mode: Mode.TOOLS (tool use)
  • Uses Claude's native tool calling API

Google Gemini

  • Default mode: Mode.TOOLS (function calling)
  • Requires jsonref package for tool calling
  • Union types (except Optional) are not supported

Ollama (Local Models)

  • Default mode: TOOLS (if model supports it) or JSON
  • Models like llama3.1, llama3.2, mistral-nemo support tools
  • Older models fall back to JSON mode

When to Use Manual Patching

Manual patching is rarely needed. Use it only if:

  1. You need fine-grained control over the patching process
  2. You're working with a custom client implementation
  3. You're debugging patching behavior

For 99% of use cases, from_provider is the better choice.

Concepts Philosophy (`docs/concepts/philosophy.md`)

title: Philosophy description: The principles behind Instructor - why simple beats complex every time.

Philosophy

Great tools make hard things easy without making easy things hard. That's Instructor.

Start with what developers know

Most AI frameworks invent their own abstractions. We don't.

python
import instructor
from pydantic import BaseModel


# What you already know (Pydantic)
class User(BaseModel):
    name: str
    age: int


# What Instructor adds
client = instructor.from_provider("openai/gpt-4.1-mini")
_user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Jane is 33"}],
)  # That's it

If you know Pydantic, you know Instructor. No new concepts, no new syntax, no 200-page manual.

Your escape hatch is always there

The worst frameworks are roach motels - easy to get in, impossible to get out. Instructor is different:

python
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


# With Instructor
client = instructor.from_provider("openai/gpt-4.1-mini")
_result = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Jane is 33"}],
)

# Want to go back to raw API? Just remove response_model:
client = instructor.from_provider("openai/gpt-4.1-mini")
_result = client.create(messages=[{"role": "user", "content": "Say hello"}])

# Or use the provider directly:
from openai import OpenAI

_raw_client = OpenAI()  # Back to vanilla

We patch, we don't wrap. Your code, your control.

Show, don't hide

Bad frameworks hide complexity. Good tools help you understand it.

python
import instructor
from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int


# See exactly what Instructor sends
instructor.logfire.configure()  # Full observability

client = instructor.from_provider("openai/gpt-4.1-mini")
result = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Jane is 33"}],
)

# Access raw responses
_raw_response = result._raw_response  # See what the LLM actually returned

When something goes wrong (and it will), you can see exactly what happened.

Composition beats configuration

No YAML files. No decorators. No magic. Just functions.

python
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class User(BaseModel):
    name: str
    age: int


class Company(BaseModel):
    name: str
    industry: str


class Analysis(BaseModel):
    user: User
    company: Company


# Build complex systems with simple functions
def extract_user(text: str) -> User:
    return client.create(
        response_model=User, messages=[{"role": "user", "content": text}]
    )


def extract_company(text: str) -> Company:
    return client.create(
        response_model=Company, messages=[{"role": "user", "content": text}]
    )


def analyze_email(email: str) -> Analysis:
    user = extract_user(email)
    company = extract_company(email)
    return Analysis(user=user, company=company)


# Compose however makes sense for YOUR application
_analysis = analyze_email("Please introduce Jane from Acme.")

Start simple, grow naturally

The best code is code that grows with your needs:

python
import instructor
from instructor import Partial
from pydantic import BaseModel, field_validator

client = instructor.from_provider("openai/gpt-4.1-mini")


class User(BaseModel):
    name: str
    age: int


# Day 1: Just get it working
_user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Jane is 33"}],
)


# Day 7: Add validation
class User(BaseModel):
    name: str
    age: int

    @field_validator("age")
    def check_age(cls, value: int) -> int:
        if value < 0 or value > 150:
            raise ValueError("Invalid age")
        return value


# Day 14: Add retries for production
_user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Jane is 33"}],
    max_retries=3,
)


# Day 30: Add streaming for better UX
def update_ui(_partial: Partial[User]) -> None:
    pass


for partial in client.create(
    response_model=Partial[User],
    messages=[{"role": "user", "content": "Jane is 33"}],
    stream=True,
):
    update_ui(partial)

Each addition is one line. No refactoring. No migration guide.

What we intentionally DON'T do

No prompt engineering

We don't write prompts for you. You know your domain better than we do.

python
# We DON'T do this:
# @instructor.prompt("Extract the user information carefully")
# def extract_user(text: str):
#     ...


# You write your own prompts:
text = "Jane is 33"
_messages = [
    {"role": "system", "content": "You are a precise data extractor"},
    {"role": "user", "content": f"Extract user from: {text}"},
]

No new abstractions

We don't invent concepts like "Agents", "Chains", or "Tools". Those are your domain concepts.

python
import instructor
from pydantic import BaseModel

# We DON'T do this:
# class UserExtractionAgent(instructor.Agent):
#     tools = [instructor.WebSearch(), instructor.Calculator()]


class User(BaseModel):
    name: str
    age: int


def search_web(query: str) -> str:
    return f"Results for {query}"


client = instructor.from_provider("openai/gpt-4.1-mini")


# You build what makes sense:
def extract_user_with_search(query: str) -> User:
    # Your logic, your way
    search_results = search_web(query)
    return client.create(
        response_model=User, messages=[{"role": "user", "content": search_results}]
    )


_user = extract_user_with_search("Find Jane")

No framework lock-in

Your code should work with or without us:

python
import instructor
from pydantic import BaseModel


# This is just a Pydantic model
class User(BaseModel):
    name: str
    age: int


# This is just a function
def process_user(user: User) -> dict:
    return {"name": user.name.upper(), "adult": user.age >= 18}


client = instructor.from_provider("openai/gpt-4.1-mini")

# Instructor just connects them to LLMs
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Jane is 33"}],
)

_result = process_user(user)  # Works with or without Instructor

The result

By following these principles, we get:

  • Tiny API surface: Learn it in minutes, not days
  • Zero vendor lock-in: Switch providers or remove Instructor anytime
  • Debuggable: When things break, you can see why
  • Composable: Build complex systems from simple parts
  • Pythonic: If it feels natural in Python, it feels natural in Instructor

In practice

Here's what building with Instructor actually looks like:

python
from enum import Enum
from typing import List

import instructor
from pydantic import BaseModel


# Your domain models (not ours)
class Priority(str, Enum):
    HIGH = "high"
    MEDIUM = "medium"
    LOW = "low"


class Ticket(BaseModel):
    title: str
    priority: Priority
    estimated_hours: float


# Your business logic (not ours)
def prioritize_tickets(tickets: List[Ticket]) -> List[Ticket]:
    return sorted(tickets, key=lambda t: (t.priority.value, -t.estimated_hours))


# Connect to LLM (one line)
client = instructor.from_provider("openai/gpt-4.1-mini")

# Extract structured data (simple function call)
tickets = client.create(
    response_model=List[Ticket],
    messages=[{"role": "user", "content": "Parse these support tickets: ..."}],
)

# Use your business logic
_prioritized = prioritize_tickets(tickets)

No framework. No abstractions. Just Python.

The philosophy in one sentence

Make structured LLM outputs as easy as defining a Pydantic model.

Everything else follows from that.

Concepts Prompt Caching (`docs/concepts/prompt_caching.md`)

title: Understanding Prompt Caching for API Efficiency description: Explore how prompt caching optimizes performance for API calls in OpenAI and Anthropic, enhancing efficiency and reducing costs.

See Also

Prompt Caching

Prompt Caching is a feature that allows you to cache portions of your prompt, optimizing performance for multiple API calls with shared context. This helps to reduce cost and improve response times.

Prompt Caching in OpenAI

OpenAI implements a prompt caching mechanism to optimize performance for API requests with similar prompts.

Prompt Caching works automatically on all your API requests (no code changes required) and has no additional fees associated with it.

This optimization is especially useful for applications making multiple API calls with shared context, minimizing redundant processing and improving overall performance.

Prompt Caching is enabled for the following models:

  • gpt-4o
  • gpt-4.1-mini
  • o1-preview
  • o1-mini

Caching is based on prefix matching, so if you're using a system prompt that contains a common set of instructions, you're likely to see a cache hit as long as you move all variable parts of the prompt to the end of the message when possible.

Prompt Caching in Anthropic

Prompt Caching is now generally available for Anthropic. This enables you to cache specific prompt portions, reuse cached content in subsequent calls, and reduce processed data per request.

??? note "Source Text"

text
In the following example, we'll be using a short excerpt from the novel "Pride and Prejudice" by Jane Austen. This text serves as an example of a substantial context that might typically lead to slow response times and high costs when working with language models. You can download it manually [here](https://www.gutenberg.org/cache/epub/1342/pg1342.txt)

```

/* Detailed source-code truncated for AI context efficiency. */

text

/* Detailed source-code truncated for AI context efficiency. */

text
1. Anthropic requires that you explicitly pass in the `cache_control` parameter to indicate that you want to cache the content.

!!! Warning "Caching Considerations"

    **Minimum cache size**: For Claude Haiku, your cached content needs to be a minimum of 2048 tokens. For Claude Sonnet, the minimum is 1024 tokens.

**Benefits**: The cost of reading from the cache is 10x lower than if we were to process the same message again and enables us to execute our queries significantly faster.

We've written a more detailed blog on how to use the `create_with_completion` method [here](../blog/posts/anthropic-prompt-caching.md) to validate you're getting a cache hit with instructor.

---

### Concepts Prompting (`docs/concepts/prompting.md`)

---
title: Prompt Engineering Best Practices
description: Learn prompt engineering tips for using Pydantic and Instructor effectively.
---

# General Tips for Prompt Engineering

When using Instructor and Pydantic, make your models self-descriptive, modular, and flexible while keeping data integrity.

- Modularity: Design self-contained components for reuse
- Self-description: Use Pydantic's `Field` for clear field descriptions
- Optionality: Use Python's `Optional` type for nullable fields and set defaults
- Standardization: Use enumerations for fields with fixed values; include a fallback option
- Dynamic data: Use key-value pairs for arbitrary properties and limit list lengths
- Entity relationships: Define explicit identifiers and relationship fields
- Contextual logic: Optionally add a "chain of thought" field in reusable components for extra context

## Modular Chain of Thought {#chain-of-thought}

Use chain of thought to improve data quality. You can add it to specific components rather than making it global.

```python hl_lines="4 5"
from pydantic import BaseModel, Field


class Role(BaseModel):
    chain_of_thought: str = Field(
        ..., description="Think step by step to determine the correct title"
    )
    title: str


class UserDetail(BaseModel):
    age: int
    name: str
    role: Role

Utilize Optional Attributes

Use Python's Optional type and set a default value to prevent undesired defaults like empty strings.

python hl_lines="6"
from typing import Optional
from pydantic import BaseModel, Field


class UserDetail(BaseModel):
    age: int
    name: str
    role: Optional[str] = Field(default=None)

Handling Errors Within Function Calls

Create a wrapper class to hold either the result of an operation or an error message. This lets you stay within a function call even if an error occurs, improving error handling without breaking the code flow.

python
from pydantic import BaseModel, Field
from typing import Optional


class UserDetail(BaseModel):
    age: int
    name: str
    role: Optional[str] = Field(default=None)


class MaybeUser(BaseModel):
    result: Optional[UserDetail] = Field(default=None)
    error: bool = Field(default=False)
    message: Optional[str]

    def __bool__(self):
        return self.result is not None

With the MaybeUser class, you can either receive a UserDetail object in result or get an error message in message.

Simplification with the Maybe Pattern

Simplify this using Instructor to create the Maybe pattern dynamically from any BaseModel.

python
import instructor
from pydantic import BaseModel


class UserDetail(BaseModel):
    age: int
    name: str


MaybeUser = instructor.Maybe(UserDetail)

This lets you quickly create a Maybe type for any class.

Tips for Enumerations

Use Enums for standardized fields to prevent data misalignment. Always include an "Other" option as a fallback so the model can signal uncertainty.

python hl_lines="7 12"
from enum import Enum, auto
from pydantic import BaseModel, Field


class Role(Enum):
    PRINCIPAL = auto()
    TEACHER = auto()
    STUDENT = auto()
    OTHER = auto()


class UserDetail(BaseModel):
    age: int
    name: str
    role: Role = Field(
        description="Correctly assign one of the predefined roles to the user."
    )

Literals {#literals}

If you're having a hard time with Enum an alternative is to use Literal

python hl_lines="4"
from typing import Literal
from pydantic import BaseModel


class UserDetail(BaseModel):
    age: int
    name: str
    role: Literal["PRINCIPAL", "TEACHER", "STUDENT", "OTHER"]

If you'd like to improve performance more you can reiterate the requirements in the field descriptions or in the docstrings.

Reiterate Long Instructions

For complex attributes, repeat the instructions in the field's description.

python hl_lines="5 11"
from pydantic import BaseModel, Field


class Role(BaseModel):
    """
    Extract the role based on the following rules ...
    """

    instructions: str = Field(
        ...,
        description="Restate the instructions and rules to correctly determine the title.",
    )
    title: str


class UserDetail(BaseModel):
    age: int
    name: str
    role: Role

Handle Arbitrary Properties

When you need to extract undefined attributes, use a list of key-value pairs.

python hl_lines="10"
from typing import List
from pydantic import BaseModel, Field


class Property(BaseModel):
    key: str
    value: str


class UserDetail(BaseModel):
    age: int
    name: str
    properties: List[Property] = Field(
        ..., description="Extract any other properties that might be relevant."
    )

Limiting the Length of Lists

When dealing with lists of attributes, especially arbitrary properties, manage the length. Use prompting and enumeration to limit the list length and keep a manageable set of properties.

python hl_lines="2 9"
from typing import List
from pydantic import BaseModel, Field


class Property(BaseModel):
    index: str = Field(..., description="Monotonically increasing ID")
    key: str
    value: str


class UserDetail(BaseModel):
    age: int
    name: str
    properties: List[Property] = Field(
        ...,
        description="Numbered list of arbitrary extracted properties, should be less than 6",
    )

Using Tuples for Simple Types

For simple types, tuples can be a more compact alternative to custom classes, especially when the properties don't require additional descriptions.

python hl_lines="4"
from typing import List, Tuple
from pydantic import BaseModel, Field


class UserDetail(BaseModel):
    age: int
    name: str
    properties: List[Tuple[int, str]] = Field(
        ...,
        description="Numbered list of arbitrary extracted properties, should be less than 6",
    )

Advanced Arbitrary Properties

For multiple users, use consistent key names when extracting properties.

python
from typing import List
from pydantic import BaseModel


class UserDetail(BaseModel):
    id: int
    age: int
    name: str


class UserDetails(BaseModel):
    """
    Extract information for multiple users.
    Use consistent key names for properties across users.
    """

    users: List[UserDetail]

This refined guide should offer a cleaner and more organized approach to structure engineering in Python.

Defining Relationships Between Entities

When relationships exist between entities, define them explicitly in the model. The following example shows how to define relationships between users by adding an id and a friends field:

python hl_lines="2 5 8"
from typing import List
from pydantic import BaseModel, Field


class UserDetail(BaseModel):
    id: int = Field(..., description="Unique identifier for each user.")
    age: int
    name: str
    friends: List[int] = Field(
        ...,
        description="Correct and complete list of friend IDs, representing relationships between users.",
    )


class UserRelationships(BaseModel):
    users: List[UserDetail] = Field(
        ...,
        description="Collection of users, correctly capturing the relationships among them.",
    )

Reusing Components with Different Contexts

You can reuse the same component for different contexts within a model. In this example, the TimeRange component is used for both work_time and leisure_time.

python hl_lines="9 10"
from pydantic import BaseModel, Field


class TimeRange(BaseModel):
    start_time: int = Field(..., description="The start time in hours.")
    end_time: int = Field(..., description="The end time in hours.")


class UserDetail(BaseModel):
    id: int = Field(..., description="Unique identifier for each user.")
    age: int
    name: str
    work_time: TimeRange = Field(
        ..., description="Time range during which the user is working."
    )
    leisure_time: TimeRange = Field(
        ..., description="Time range reserved for leisure activities."
    )

Sometimes, a component like TimeRange may need context or additional logic to work well. Adding a "chain of thought" field within the component can help understand or optimize the time range allocations.

python hl_lines="2"
from pydantic import BaseModel, Field


class TimeRange(BaseModel):
    chain_of_thought: str = Field(
        ..., description="Step by step reasoning to get the correct time range"
    )
    start_time: int = Field(..., description="The start time in hours.")
    end_time: int = Field(..., description="The end time in hours.")

Concepts Raw Response (`docs/concepts/raw_response.md`)

title: Creating a Model with OpenAI Completions description: Learn how to create a custom model using OpenAI's API to extract user data efficiently with Python.

Creating a model with completions

In instructor>1.0.0 we have a custom client, if you wish to use the raw response you can do the following

text
/* Detailed source-code truncated for AI context efficiency. */

Raw response with a list response model

If your response model is a list (for example, list[UserExtract]), you can still use create_with_completion(). Instructor wraps the list in a ResponseList (also called ListResponse) that behaves like a normal list but also preserves the raw response.

What is ResponseList?

ResponseList is a special list type that Instructor uses when your response_model is a list. It extends Python's built-in list type and adds a _raw_response attribute to store the provider's raw response object.

This is necessary because create_with_completion() needs to return both the parsed result and the raw response. For single objects, this is straightforward: (model_instance, raw_response). For lists, we need a way to attach the raw response to the list itself, which is what ResponseList does.

Using ResponseList

The returned value behaves exactly like a normal Python list, but you can access the raw response using get_raw_response():

python
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserExtract(BaseModel):
    name: str
    age: int


users, completion = client.create_with_completion(
    response_model=list[UserExtract],
    messages=[
        {"role": "user", "content": "Extract users: Jason is 25, Ivan is 30"},
    ],
)

# Use it like a normal list
print(users[0])
#> name='Jason' age=25
print(len(users))
#> 2

# Access the raw response
raw = users.get_raw_response()
assert raw == completion

# ResponseList supports all list operations
for user in users:
    print(user.name)
#> Jason
#> Ivan

See Also

  • Hooks - Monitor LLM interactions without accessing raw responses
  • Debugging - Debugging techniques for LLM outputs
  • Response Models - Working with structured response models

Anthropic Raw Response

You can also access the raw response from Anthropic models. This is useful for debugging or when you need to access additional information from the response.

python
import instructor

client = instructor.from_provider("anthropic/claude-3-5-sonnet-latest")


user, completion = client.create_with_completion(
    response_model=UserExtract,
    messages=[
        {"role": "user", "content": "Extract jason is 25 years old"},
    ],
)

print(user)
#> name='Jason' age=25

print(completion)
"""

---

### Concepts Reask Validation (`docs/concepts/reask_validation.md`)

---
title: Enhancing AI Validations with Pydantic's Framework
---

# Validation and Reasking

Instead of framing "self-critique" or "self-reflection" in AI as new concepts, we can view them as validation errors with clear error messages that the system can use to self-correct.

## Pydantic

Pydantic offers a customizable and expressive validation framework for Python. Instructor leverages Pydantic's validation framework to provide a uniform developer experience for both code-based and LLM-based validation, as well as a reasking mechanism for correcting LLM outputs based on validation errors. To learn more check out the [Pydantic docs](https://docs.pydantic.dev/latest/concepts/validators/) on validators.

!!! note "Good llm validation is just good validation"

    If you want to see some more examples on validators checkout our blog post [Good LLM validation is just good validation](https://python.useinstructor.com/blog/2023/10/23/good-llm-validation-is-just-good-validation/)

### Code-based Validation Example

First define a Pydantic model with a validator using the `Annotation` class from `typing_extensions`.

Enforce a naming rule using Pydantic's built-in validation:

```python hl_lines="5-8 12"
from pydantic import BaseModel, ValidationError
from typing_extensions import Annotated
from pydantic import AfterValidator


def name_must_contain_space(v: str) -> str:
    if " " not in v:
        raise ValueError("Name must contain a space.")
    return v.lower()


class UserDetail(BaseModel):
    age: int
    name: Annotated[str, AfterValidator(name_must_contain_space)]


try:
    person = UserDetail(age=29, name="Jason")
except ValidationError as e:
    print(e)
    """
    1 validation error for UserDetail
    name
      Value error, Name must contain a space. [type=value_error, input_value='Jason', input_type=str]
        For further information visit https://errors.pydantic.dev/2.11/v/value_error
    """

Output for Code-Based Validation

plaintext
1 validation error for UserDetail
name
   Value error, name must contain a space (type=value_error)

As we can see, Pydantic raises a validation error when the name attribute does not contain a space. This is a simple example, but it demonstrates how Pydantic can be used to validate attributes of a model.

LLM-Based Validation Example

LLM-based validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule "don't say objectionable things," Pydantic will raise a validation error.

python hl_lines="9 15"
import instructor
from instructor import llm_validator
from pydantic import BaseModel, ValidationError, BeforeValidator
from typing import Annotated

# Apply the patch to the OpenAI client
client = instructor.from_provider("openai/gpt-4.1-mini")


class QuestionAnswer(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(llm_validator("don't say objectionable things", client=client)),
    ]


try:
    qa = QuestionAnswer(
        question="What is the meaning of life?",
        answer="The meaning of life is to be evil and steal",
    )
except ValidationError as e:
    print(e)
    """
    1 validation error for QuestionAnswer
    answer
      Assertion failed, The statement promotes objectionable behavior by encouraging evil and stealing. [type=assertion_error, input_value='The meaning of life is to be evil and steal', input_type=str]
        For further information visit https://errors.pydantic.dev/2.11/v/assertion_error
    """

Output for LLM-Based Validation

It is important to note here that the error message is generated by the LLM, not the code, so it'll be helpful for re-asking the model.

plaintext
1 validation error for QuestionAnswer
answer
   Assertion failed, The statement is objectionable. (type=assertion_error)

Using Reasking Logic to Correct Outputs

Validators are a great tool for ensuring some property of the outputs. When you use the patch() method with the openai client, you can use the max_retries parameter to set the number of times you can reask the model to correct the output.

It is a great layer of defense against bad outputs of two forms:

  1. Pydantic Validation Errors (code or llm based)
  2. JSON Decoding Errors (when the model returns a bad response)

Step 1: Define the Response Model with Validators

Notice that the field validator wants the name in uppercase, but the user input is lowercase. The validator will raise a ValueError if the name is not in uppercase.

python hl_lines="12-17"
import instructor
from pydantic import BaseModel, field_validator

# Apply the patch to the OpenAI client
client = instructor.from_provider("openai/gpt-4.1-mini")


class UserDetails(BaseModel):
    name: str
    age: int

    @field_validator("name")
    @classmethod
    def validate_name(cls, v):
        if v.upper() != v:
            raise ValueError("Name must be in uppercase.")
        return v

Step 2. Using the Client with Retries

Here, the UserDetails model is passed as the response_model, and max_retries is set to 2.

python
import instructor
from pydantic import BaseModel

client = instructor.from_provider(
    "openai/gpt-4.1-mini",
    mode=instructor.Mode.TOOLS,
)


class UserDetails(BaseModel):
    name: str
    age: int


model = client.create(
    response_model=UserDetails,
    max_retries=2,
    messages=[
        {"role": "user", "content": "Extract jason is 25 years old"},
    ],
)

print(model.model_dump_json(indent=2))
"""
{
  "name": "jason",
  "age": 25
}
"""

What happens behind the scenes?

Behind the scenes, the instructor.from_provider() method adds a max_retries parameter to the openai.ChatCompletion.create() method. The max_retries parameter will trigger up to 2 reattempts if the name attribute fails the uppercase validation in UserDetails.

python
from pydantic import ValidationError

try:
    ...
except ValidationError as e:
    kwargs["messages"].append(response.choices[0].message)
    kwargs["messages"].append(
        {
            "role": "user",
            "content": f"Please correct the function call; errors encountered:\n{e}",
        }
    )

Advanced Validation Techniques

Using Context for Dynamic Validation

The context parameter allows you to pass additional data to your validators, enabling validation against runtime data like source documents, allowed values, or external references. This is accessed in validators via ValidationInfo.

Here's a complete example showing context-based validation:

text
/* Detailed source-code truncated for AI context efficiency. */

In this example:

  • The context parameter passes the source text to the validator
  • ValidationInfo provides access to the context in the validator
  • If the LLM generates a quote that doesn't exist in the source, validation fails and the model is re-asked

For more advanced examples including multi-field validation and citation verification, check out our exact citations example.

Optimizing Token usage

Pydantic automatically includes a URL within the error message itself when an error is thrown so that users can learn more about the specific error that was thrown. Some users might want to remove this URL since it adds extra tokens that otherwise might not add much value to the validation process.

We've created a small helper function that you can use below which removes this url in the error message

python hl_lines="6"
from instructor.utils import disable_pydantic_error_url
from pydantic import BaseModel, ValidationError
from typing_extensions import Annotated
from pydantic import AfterValidator

disable_pydantic_error_url()  # (1)!


def name_must_contain_space(v: str) -> str:
    if " " not in v:
        raise ValueError("Name must contain a space.")
    return v.lower()


class UserDetail(BaseModel):
    age: int
    name: Annotated[str, AfterValidator(name_must_contain_space)]


try:
    person = UserDetail(age=29, name="Jason")
except ValidationError as e:
    print(e)
    """
    1 validation error for UserDetail
    name
      Value error, Name must contain a space. [type=value_error, input_value='Jason', input_type=str]
    """
  1. We disable the error by setting an environment variable PYDANTIC_ERRORS_INCLUDE_URL to 0. This is valid only for the duration that the script is executed for, once the function is not called, the original behaviour is restored.

See Also

Takeaways

By integrating these advanced validation techniques, we not only improve the quality and reliability of LLM-generated content, but also pave the way for more autonomous and effective systems.

Concepts Retrying (`docs/concepts/retrying.md`)

title: "Retry Logic with Tenacity"

Retry Logic with Tenacity

Tenacity is a Python library for adding retry logic to your applications. Combined with Instructor, it helps handle API failures, rate limits, and validation errors.

Count SDK Requests Separately from Extraction Attempts

There are two retry settings with the same name:

  • OpenAI(max_retries=2) or Anthropic(max_retries=2) configures SDK transport
    retries after the first HTTP request.
  • client.create(max_retries=2, ...) configures Instructor validation retries
    after the first extraction attempt: at most three SDK calls. Negative integer
    values are treated as zero retries.

With integer max_retries, Instructor retries supported parsing and validation
errors. An exhausted SDK rate-limit, connection, or server error is wrapped in
InstructorRetryException; Instructor does not restart it automatically. A
custom Retrying / AsyncRetrying instance controls its own retry predicate,
wait, and stop conditions, and can retry SDK errors as well.

Do not multiply the configured limits to report actual calls. For example,
429 → invalid JSON response → 500 → valid JSON response takes four HTTP
requests and two Instructor attempts. Persistent 429 with two SDK retries
ends after three HTTP requests and one Instructor attempt. The product of the
limits is only an upper bound for these SDKs under an attempt-bounded policy;
SDK retry decisions and successful responses determine the actual count.

completion:kwargs, completion:response, and exception n_attempts describe
Instructor attempts. SDK-internal retries do not emit additional Instructor
hooks. failed_attempts records parsing failures, not every transport failure.
Use SDK transport instrumentation or server request logs for HTTP attempt
counts, and elapsed time around the entire extraction for latency. A missing
measurement must remain unknown rather than being reported as zero.

For custom policies, max_attempts is unknown (None), and an error hook's
is_last_attempt=False means finality has not yet been established. Use
completion:last_attempt to observe exhausted retries. This distinction also
matters when elapsed-time stopping ends retries before the integer count limit.
Cancellation propagates without an error or last-attempt hook.

The OpenAI SDK retry documentation
and Anthropic SDK documentation
describe their own retry and timeout behavior. Both document two default retries
for selected transport/status failures; consult the installed SDK version before
extending these rules to another provider or transport.

Timeouts and Whole-Operation Deadlines

For integer Instructor retries, a numeric timeout passed to create serves two
purposes: it is forwarded to the SDK, and it supplies an elapsed-time stop
condition for further validation retries. The elapsed condition is evaluated
after a failed attempt. It does not interrupt an in-flight SDK request, SDK
backoff, parsing, or user validation. A successful response can return after that
elapsed limit. SDK socket timeouts may themselves trigger SDK retries.

An SDK timeout object, such as httpx.Timeout for OpenAI 2.x, is forwarded but
does not enable Instructor's numeric elapsed stop condition. A timeout configured
only on the SDK client likewise does not set that condition. With a custom
Tenacity instance, specify its stop policy explicitly; Instructor does not add
an elapsed stop condition to it. stop_after_delay also checks between attempts
and is not an interrupting deadline.

For an async operation, the application can bound the wait using cancellation:

python
import asyncio
import instructor
from openai import AsyncOpenAI
from pydantic import BaseModel


class Answer(BaseModel):
    value: int


async def extract():
    async with AsyncOpenAI(max_retries=0, timeout=2.0) as sdk:
        client = instructor.from_openai(sdk)
        return await asyncio.wait_for(
            client.create(
                model="gpt-4.1-mini",
                response_model=Answer,
                messages=[{"role": "user", "content": "Extract seven"}],
                max_retries=1,
            ),
            timeout=5.0,
        )

asyncio.wait_for cancels the task and waits for cancellation cleanup, so cleanup
or blocking synchronous validators can extend the observed wall time. It does
not guarantee that the remote provider stops work. Cancelling a thread that runs
a synchronous client does not cancel that client's HTTP request. For lazy
streaming, place the stream's consumption and cleanup inside the application's
deadline too; returning a stream does not mean extraction has completed.

Limit Validation Retry Cost

Use token_budget to stop validation retries after cumulative provider usage
reaches a positive token limit:

python
import instructor
from instructor.core import TokenBudgetExceeded
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserInfo(BaseModel):
    name: str
    age: int


try:
    user = client.create(
        response_model=UserInfo,
        messages=[{"role": "user", "content": "Extract: Jason is 25"}],
        max_retries=3,
        token_budget=2_000,
    )
except TokenBudgetExceeded as error:
    print(error.total_usage)

The budget is checked after a response fails validation and before Instructor
prepares another request. Reaching the exact budget stops the retry. A response
that validates successfully is returned even if that completed request takes
the cumulative total over the budget.

token_budget is a retry budget, not a hard per-request limit. The provider may
use more than the remaining budget while completing the current request. Use
the provider's output-token setting when you also need a per-request limit.

Budgeted retries currently require a structured, non-streaming response and
compatible provider usage metadata. Instructor raises
TokenUsageUnavailableError instead of making another request when it cannot
account for usage safely.

Basic Retry with Exponential Backoff

The most common pattern uses exponential backoff to delay retries:

python
import instructor
from pydantic import BaseModel
from tenacity import retry, stop_after_attempt, wait_exponential

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserInfo(BaseModel):
    name: str
    age: int
    email: str


@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
def extract_user_info(text: str) -> UserInfo:
    """Extract user information with retry logic."""
    return client.create(
        response_model=UserInfo,
        messages=[{"role": "user", "content": f"Extract user info: {text}"}],
    )


try:
    user = extract_user_info("John is 30 years old with email [email protected]")
    print(f"Success: {user.name}, {user.age}, {user.email}")
    #> Success: John, 30, [email protected]
except Exception as e:
    print(f"Failed after retries: {e}")

Error-Specific Retries

Pass the policy into Instructor to select the original SDK and validation
exceptions before Instructor wraps exhaustion in InstructorRetryException:

python
import instructor
from openai import APIConnectionError, InternalServerError, OpenAI, RateLimitError
from pydantic import BaseModel, ValidationError
from tenacity import Retrying, retry_if_exception_type, stop_after_attempt, wait_exponential

client = instructor.from_openai(OpenAI(max_retries=0))


class UserInfo(BaseModel):
    name: str
    age: int


user = client.create(
    model="gpt-4.1-mini",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "John is 30"}],
    max_retries=Retrying(
        retry=retry_if_exception_type(
            (APIConnectionError, InternalServerError, RateLimitError, ValidationError)
        ),
        stop=stop_after_attempt(3),
        wait=wait_exponential(multiplier=1, max=10),
    ),
)

Use AsyncRetrying with async clients and the relevant exception classes for
your SDK (for example, anthropic.RateLimitError). This explicit policy makes
up to three Instructor attempts and disables nested OpenAI transport retries.
An outer decorator sees InstructorRetryException, not its underlying SDK or
validation error, and restarting the function starts a new extraction budget.

Custom Retry Conditions

Retry based on the result content rather than exceptions:

python
import instructor
from pydantic import BaseModel
from tenacity import retry, retry_if_result, stop_after_attempt

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserInfo(BaseModel):
    name: str
    age: int
    email: str


def should_retry(result: UserInfo) -> bool:
    """Retry if the result doesn't meet quality criteria."""
    return result.age < 0 or result.age > 150 or not result.email


@retry(retry=retry_if_result(should_retry), stop=stop_after_attempt(3))
def extract_valid_user(text: str) -> UserInfo:
    return client.create(
        response_model=UserInfo,
        messages=[{"role": "user", "content": text}],
    )

Context-Based Validation with Retries

Use the context parameter to pass runtime data to validators:

text
/* Detailed source-code truncated for AI context efficiency. */

Logging and Monitoring

Add logging to track retry attempts:

python
import logging
import instructor
from pydantic import BaseModel
from tenacity import after_log, before_log, retry, stop_after_attempt, wait_exponential

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserInfo(BaseModel):
    name: str
    age: int
    email: str


logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=4, max=10),
    before=before_log(logger, logging.INFO),
    after=after_log(logger, logging.ERROR),
)
def logged_extraction(text: str) -> UserInfo:
    return client.create(
        response_model=UserInfo,
        messages=[{"role": "user", "content": text}],
    )

Instructor's Built-in Retries

Set validation retry limits on the extraction call:

python
import instructor
from openai import OpenAI
from pydantic import BaseModel

client = instructor.from_openai(OpenAI(max_retries=2), mode=instructor.Mode.JSON)


class UserInfo(BaseModel):
    name: str
    age: int


user = client.create(
    model="gpt-4.1-mini",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "John is 30"}],
    max_retries=1,  # First extraction plus at most one validation retry.
)

Failed Attempts Tracking

When retries fail, Instructor provides detailed failure history:

python
import instructor
from instructor.core.exceptions import InstructorRetryException
from pydantic import BaseModel, field_validator

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserInfo(BaseModel):
    name: str
    age: int

    @field_validator('age')
    @classmethod
    def validate_age(cls, v):
        if v < 0 or v > 150:
            raise ValueError(f"Age {v} is invalid")
        return v


try:
    result = client.create(
        response_model=UserInfo,
        messages=[{"role": "user", "content": "Extract: John is -5 years old"}],
        max_retries=3,
    )
except InstructorRetryException as e:
    print(f"Failed after {e.n_attempts} attempts")
    for attempt in e.failed_attempts:
        print(f"Attempt {attempt.attempt_number}: {attempt.exception}")

Failed attempts are automatically propagated to reask handlers, enabling contextual error messages and progressive corrections.

Best Practices

Choose Appropriate Strategies

Error Type Attempts Min Delay Max Delay
Rate limits 5 1s 60-120s
Validation errors 2-3 1s 10s
Network errors 4 2s 30s

Always Set Stop Conditions

python
from tenacity import retry, stop_after_attempt


# Good: bounded retries
@retry(stop=stop_after_attempt(3))
def bounded_retry():
    pass


# Bad: could retry forever
@retry()  # Don't do this!
def unbounded_retry():
    pass

Troubleshooting

Infinite retries: Always set stop_after_attempt() or stop_after_delay().

Too many retries: Use retry_if_exception_type() to retry only on specific errors.

Still hitting rate limits: Increase max delay and use wait_exponential() with higher multipliers.

Concepts Semantic Validation (`docs/concepts/semantic_validation.md`)

title: Semantic Validation with LLMs

See Also

Semantic Validation with LLMs

This guide covers semantic validation in Instructor - using LLMs themselves to validate content against complex, subjective, or contextual criteria that would be difficult to implement with traditional rule-based approaches.

Overview

Semantic validation leverages the language understanding capabilities of LLMs to validate inputs against natural language criteria. While traditional validation uses explicit rules and patterns, semantic validation can understand nuance, context, and subjective qualities in data.

When to Use Semantic Validation

Semantic validation is particularly useful for:

  • Complex criteria that are difficult to express with rules
  • Subjective qualities like tone, style, or appropriateness
  • Contextual validation that requires understanding relationships between fields
  • Policy enforcement that involves nuanced understanding of guidelines
  • Content moderation for detecting harmful or inappropriate content

How It Works

In Instructor, semantic validation is implemented through the llm_validator function, which creates a validator that uses an LLM to check if values conform to specified requirements:

python
import instructor
from typing import Annotated
from pydantic import BaseModel, BeforeValidator
from instructor import llm_validator

# Initialize client
client = instructor.from_provider("openai/gpt-4.1-mini")


class UserComment(BaseModel):
    username: str
    comment: Annotated[
        str,
        BeforeValidator(
            llm_validator(
                "Comment must be constructive, respectful, and not contain hate speech or profanity",
                client=client,
            )
        ),
    ]

The llm_validator function takes:

  1. A natural language description of the validation criteria
  2. An Instructor client instance to perform the validation
  3. Optional parameters for configuration

During validation, the LLM evaluates whether the input matches the specified criteria and either passes the value or raises a validation error with a detailed explanation.

Validation Flow

The following diagram illustrates how semantic validation works in Instructor:

mermaid
flowchart TD
    A[Input Data] --> B[Pydantic Validation Process]
    B --> C{Field has Semantic\nValidator?}
    C -->|No| D[Standard Validation]
    C -->|Yes| E[Call LLM with Validation Criteria]
    E --> F{LLM Determines\nValue is Valid?}
    F -->|Yes| G[Validation Passes]
    F -->|No| H[Validation Fails with LLM-Generated Error]
    H --> I{Auto-Retry Enabled?}
    I -->|Yes| J[Try Again with Error Context]
    I -->|No| K[Return Validation Error]
    J --> E

    classDef process fill:#e2f0fb,stroke:#b8daff,color:#004085;
    classDef decision fill:#fff3cd,stroke:#ffeeba,color:#856404;
    classDef success fill:#d4edda,stroke:#c3e6cb,color:#155724;
    classDef error fill:#f8d7da,stroke:#f5c6cb,color:#721c24;

    class A,B,E,J process
    class C,F,I decision
    class G,D success
    class H,K error

Basic Usage

Here's a complete example of semantic validation in action:

text
/* Detailed source-code truncated for AI context efficiency. */

Advanced Validation Patterns

Content Policy Enforcement

This example validates user-generated content against community guidelines:

python
import instructor
from typing import Annotated
from pydantic import BaseModel, BeforeValidator
from instructor import llm_validator

client = instructor.from_provider("openai/gpt-4.1-mini")


class Comment(BaseModel):
    """Model representing a user comment with content moderation."""

    user_id: str
    content: Annotated[
        str,
        BeforeValidator(
            llm_validator(
                """Content must comply with community guidelines:
                - No hate speech, harassment, or discrimination
                - No explicit sexual or violent content
                - No promotion of illegal activities
                - No sharing of personal information
                - No spamming or excessive self-promotion""",
                client=client,
            )
        ),
    ]

Topic Relevance Validation

This validator ensures that responses stay on topic:

python
import instructor
from typing import Annotated
from pydantic import BaseModel, BeforeValidator
from instructor import llm_validator

client = instructor.from_provider("openai/gpt-4.1-mini")


class ForumPost(BaseModel):
    topic: str
    post: Annotated[
        str,
        BeforeValidator(
            llm_validator(
                "The post must be directly relevant to the specified topic and not drift to unrelated subjects",
                client=client,
            )
        ),
    ]

    # Using Jinja templating for validation against dynamic values
    @classmethod
    def validate_post(cls, topic_name: str, post_content: str) -> "ForumPost":
        return client.create(
            response_model=cls,
            messages=[
                {
                    "role": "system",
                    "content": """Validate that the forum post content stays relevant to the topic.
                    If it's not relevant, explain why in detail.""",
                },
                {
                    "role": "user",
                    "content": """
                    Topic: {{ topic }}

                    Post content:
                    {{ post }}

                    Is this post relevant to the topic?
                    """,
                },
            ],
            context={
                "topic": topic_name,
                "post": post_content,
            },
        )

Fact-Checking Validator

This complex validator assesses factual accuracy:

python
import instructor
from typing import List
from pydantic import BaseModel, Field

client = instructor.from_provider("openai/gpt-4.1-mini")


class FactCheckedClaim(BaseModel):
    """Model for validating factual accuracy of claims."""

    claim: str
    is_accurate: bool = Field(description="Whether the claim is factually accurate")
    supporting_evidence: List[str] = Field(
        default_factory=list,
        description="Evidence supporting or refuting the claim",
    )

    @classmethod
    def validate_claim(cls, text: str) -> "FactCheckedClaim":
        return client.create(
            response_model=cls,
            messages=[
                {
                    "role": "system",
                    "content": "You are a fact-checking system. Assess the factual accuracy of the claim.",
                },
                {
                    "role": "user",
                    "content": "Fact check this claim: {{ claim }}",
                },
            ],
            context={"claim": text},
        )

Complex Multi-Field Validation

For validation that needs to compare multiple fields, you can use model validators:

text
/* Detailed source-code truncated for AI context efficiency. */

Best Practices

  1. Be Specific in Criteria: Provide clear, detailed validation criteria in natural language
  2. Use Appropriate Models: Larger models tend to give better, more nuanced validation
  3. Balance Cost and Latency: Remember that each validation adds an LLM API call
  4. Provide Examples: Include examples of both valid and invalid content in your criteria
  5. Handle Retries: Configure retry logic for edge cases
  6. Use Jinja Templates: When validating against dynamic values, use Jinja templating
  7. Separate Concerns: Keep validation criteria focused on specific aspects
  8. Consider Context: Use model-level validation when comparing multiple fields

Advanced Configuration

The llm_validator function supports several configuration options:

python
import instructor
from instructor import llm_validator
from pydantic import BaseModel, BeforeValidator
from typing import Annotated

client = instructor.from_provider("openai/gpt-4.1-mini")

# Configure the validator with options
validator = llm_validator(
    statement="Must be a professional, concise product description",
    client=client,  # Required Instructor client
    allow_override=True,  # Allow LLM to fix invalid values
    model="gpt-4o",  # Specify model to use for validation
    temperature=0.2,  # Add variability (default is 0)
)


class Product(BaseModel):
    description: Annotated[str, BeforeValidator(validator)]

Performance Considerations

Semantic validation adds API calls to your workflow, which impacts:

  1. Latency: Each validation requires an additional API call
  2. Cost: More API calls mean higher usage costs
  3. Reliability: Depends on API availability and response quality

Consider these trade-offs when implementing semantic validation, especially for high-volume applications.

Comparison with Rule-Based Validation

Aspect Rule-Based Validation Semantic Validation
Implementation Regular expressions, constraints Natural language criteria
Complexity Simple rules, explicit patterns Can handle subjective criteria
Speed Fast, no external calls Slower, requires API calls
Cost No additional API costs Each validation costs tokens
Flexibility Limited to programmable rules Can validate against any natural language criteria
Maintenance Rules must be updated manually Criteria can be more adaptable

Semantic validation expands what's possible with validation beyond traditional rule-based approaches. By using LLMs to validate content against natural language criteria, you can build more sophisticated validation systems that understand context, nuance, and complex relationships.

Concepts Templating (`docs/concepts/templating.md`)

title: Prompt Templating with Jinja - Dynamic Prompt Generation

Prompt Templating

With Instructor's Jinja templating, you can:

  • Dynamically adapt prompts to any context
  • Easily manage and version your prompts better
  • Integrate seamlessly with validation processes
  • Handle sensitive information securely

Our solution offers:

  • Separation of prompt structure and content
  • Complex logic implementation within prompts
  • Template reusability across scenarios
  • Enhanced prompt versioning and logging
  • Pydantic integration for validation and type safety

Context is available to the templating engine

The context parameter is a dictionary that is passed to the templating engine. It is used to pass in the relevant variables to the templating engine. This single context parameter will be passed to jinja to render out the final prompt.

python hl_lines="14-15 19-22"
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class User(BaseModel):
    name: str
    age: int


resp = client.create(
    messages=[
        {
            "role": "user",
            "content": """Extract the information from the
        following text: `{{ data }}`""",  # (1)!
        },
    ],
    response_model=User,
    context={"data": "John Doe is thirty years old"},  # (2)!
)

print(resp)
#> name='John Doe' age=30
  1. Declare jinja style template variables inside the prompt itself (e.g. {{ name }})
  2. Pass in the variables to be used in the context parameter

Context is available to Pydantic validators

In this example, we demonstrate how to leverage the context parameter with Pydantic validators to enhance our validation and data processing capabilities. By passing the context to the validators, we can implement dynamic validation rules and data transformations based on the input context. This approach allows for flexible and context-aware validation, such as checking for banned words or applying redaction patterns to sensitive information.

text
/* Detailed source-code truncated for AI context efficiency. */

  1. Access the variables passed into the context variable inside your Pydantic validator

  2. Pass in the variables to be used for validation and/or rendering into the context parameter

Jinja Syntax

Jinja is used to render the prompts, allowing the use of familiar Jinja syntax. This enables rendering of lists, conditionals, and more. It also allows calling functions and methods within Jinja.

This makes formatting of prompts and rendering logic extremely easy.

text
/* Detailed source-code truncated for AI context efficiency. */

Working with Secrets

Your prompts might need to include sensitive user information when they're sent to your model provider. This is probably something you don't want to hard code into your prompt or captured in your logs. An easy way to get around this is to use the SecretStr type from Pydantic in your model definitions.

python
from pydantic import BaseModel, SecretStr
import instructor


class UserContext(BaseModel):
    name: str
    address: SecretStr


class Address(BaseModel):
    street: SecretStr
    city: str
    state: str
    zipcode: str


client = instructor.from_provider("openai/gpt-4.1-mini")
context = UserContext(name="scolvin", address="secret address")

address = client.create(
    messages=[
        {
            "role": "user",
            "content": "{{ user.name }} is `{{ user.address.get_secret_value() }}`, normalize it to an address object",
        },
    ],
    context={"user": context},
    response_model=Address,
)
print(context)
#> name='scolvin' address=SecretStr('**********')
print(address)
"""
street=SecretStr('**********') city='secret address' state='secret address' zipcode='secret address'
"""

This allows you to preserve your sensitive information while still using it in your prompts.

Security

We use the jinja2.sandbox.SandboxedEnvironment to prevent security issues with the templating engine. This means that you can't use arbitrary python code in your prompts. But this doesn't mean that you should pass untrusted input to the templating engine, as this could still be abused for things like Denial of Service attacks.

You should always sanitize any input that you pass to the templating engine.

Concepts Typeadapter (`docs/concepts/typeadapter.md`)

title: TypeAdapter in Instructor - Custom Type Handling

!!! warning "This page is a work in progress"

text
This page is a work in progress. Check out [Pydantic's documentation](https://docs.pydantic.dev/latest/concepts/type_adapter/)

Concepts Typeddicts (`docs/concepts/typeddicts.md`)

title: Using TypedDicts with OpenAI API

title: TypedDict Support in Instructor - Dictionary Validation

TypedDicts

We also support typed dicts.

python
from typing_extensions import TypedDict
import instructor


class User(TypedDict):
    name: str
    age: int


client = instructor.from_provider("openai/gpt-4.1-mini")


response = client.create(
    response_model=User,
    messages=[
        {
            "role": "user",
            "content": "Timothy is a man from New York who is turning 32 this year",
        }
    ],
)

Instructor converts a TypedDict into a Pydantic model. Keys starting with an
underscore, such as _id, use field aliases because Pydantic reserves those
attribute names. Use response.model_dump(by_alias=True) to export the original
keys. Required keys must still be present, and NotRequired keys may be omitted.

Concepts Types (`docs/concepts/types.md`)

title: Working with Types in Instructor

Working with Types in Instructor

Instructor supports a wide range of types for your structured outputs, from simple primitives to complex nested structures.

Simple Types

In addition to pydantic.BaseModel (the recommended approach), Instructor also supports:

  • Primitive types: str, int, float, bool
  • Collection types: List, Dict
  • Type composition: Union, Literal, Optional
  • Specialized outputs: Iterable, Partial

You can use these types directly in your response_model parameter without wrapping them in a Pydantic model.

For better documentation and control, use typing.Annotated to add more context to your types.

What happens behind the scenes?

We will actually wrap the response model with a pydantic.BaseModel of the following form:

python
from typing import Annotated
from pydantic import create_model, Field, BaseModel

typehint = Annotated[bool, Field(description="Sample Description")]

model = create_model("Response", content=(typehint, ...), __base__=BaseModel)

print(model.model_json_schema())
"""
{
    'properties': {
        'content': {
            'description': 'Sample Description',
            'title': 'Content',
            'type': 'boolean',
        }
    },
    'required': ['content'],
    'title': 'Response',
    'type': 'object',
}
"""

Primitive Types (str, int, float, bool)

python
import instructor

client = instructor.from_provider("openai/gpt-4.1-mini")

# Response model with simple types like str, int, float, bool
resp = client.create(
    response_model=bool,
    messages=[
        {
            "role": "user",
            "content": "Is it true that Paris is the capital of France?",
        },
    ],
)
assert resp is True, "Paris is the capital of France"
print(resp)
#> True

Annotated

Annotations can be used to add more information about the type. This can be useful for adding descriptions to the type, along with more complex information like field names, and more.

python
import instructor
from typing import Annotated
from pydantic import Field

client = instructor.from_provider("openai/gpt-4.1-mini")

UpperCaseStr = Annotated[str, Field(description="string must be upper case")]

# Response model with simple types like str, int, float, bool
resp = client.create(
    response_model=UpperCaseStr,
    messages=[
        {
            "role": "user",
            "content": "What is the capital of france?",
        },
    ],
)
assert resp == "PARIS", "Paris is the capital of France"
print(resp)
#> PARIS

Literal

When doing simple classification Literals go quite well, they support literal of string, int, bool.

python
import instructor
from typing import Literal

client = instructor.from_provider("openai/gpt-4.1-mini")

resp = client.create(
    response_model=Literal["BILLING", "SHIPPING"],
    messages=[
        {
            "role": "user",
            "content": "Classify the following messages: 'I am having trouble with my billing'",
        },
    ],
)
assert resp == "BILLING"
print(resp)
#> BILLING

Enum

Enums are harder to get right without some addition promping but are useful if these are values that are shared across the application.

python
import instructor
from enum import Enum


class Label(str, Enum):
    BILLING = "BILLING"
    SHIPPING = "SHIPPING"


client = instructor.from_provider("openai/gpt-4.1-mini")

resp = client.create(
    response_model=Label,
    messages=[
        {
            "role": "user",
            "content": "Classify the following messages: 'I am having trouble with my billing'",
        },
    ],
)
assert resp == Label.BILLING
print(resp)
#> BILLING

List

python
import instructor
from typing import List

client = instructor.from_provider("openai/gpt-4.1-mini")

resp = client.create(
    response_model=List[int],
    messages=[
        {
            "role": "user",
            "content": "Give me the first 5 prime numbers",
        },
    ],
)

assert resp == [2, 3, 5, 7, 11]
print(resp)
#> [2, 3, 5, 7, 11]

Union

Union is a great way to handle multiple types of responses, similar to multiple function calls but not limited to the function calling api, like in JSON_SCHEMA modes.

python
import instructor
from pydantic import BaseModel
from typing import Union

client = instructor.from_provider("openai/gpt-4.1-mini")


class Add(BaseModel):
    a: int
    b: int


class Weather(BaseModel):
    location: str


resp = client.create(
    response_model=Union[Add, Weather],
    messages=[
        {
            "role": "user",
            "content": "What is 5 + 5?",
        },
    ],
)

assert resp == Add(a=5, b=5)
print(resp)
#> a=5 b=5

See Also

Complex Types

Pandas DataFrame

This is a more complex example, where we use a custom type to convert markdown to a pandas DataFrame.

text
/* Detailed source-code truncated for AI context efficiency. */

Lists of Unions

Just like Unions we can use List of Unions to represent multiple types of responses. This will feel similar to the parallel function calls but not limited to the function calling api, like in JSON_SCHEMA modes.

python
import instructor
from pydantic import BaseModel
from typing import Union, List

client = instructor.from_provider("openai/gpt-4.1-mini")


class Weather(BaseModel, frozen=True):
    location: str


class Add(BaseModel, frozen=True):
    a: int
    b: int


resp = client.create(
    response_model=List[Union[Add, Weather]],
    messages=[
        {
            "role": "user",
            "content": "Add 5 and 5, and also whats the weather in Toronto?",
        },
    ],
)

assert resp == [Add(a=5, b=5), Weather(location="Toronto")]
print(resp)
#> [Add(a=5, b=5), Weather(location='Toronto')]

Concepts Union (`docs/concepts/union.md`)

title: Using Union Types in Pydantic Models

!!! note "Redirect Notice"
This page has been consolidated into the comprehensive Union Types guide.
Please visit that page for complete information about working with union types in Instructor.

Concepts Unions (`docs/concepts/unions.md`)

title: Union Types in Instructor

Working with Union Types in Instructor

This guide explains how to work with union types in Instructor, allowing you to handle multiple possible response types from language models. Union types are particularly useful when you need the LLM to choose between different response formats or action types.

!!! note "Union vs. union"
The content from the original union.md page has been consolidated into this more comprehensive guide. That page showed a basic example of using Union types for multiple action types.

Basic Union Types

Union types let you specify that a field can be one of several types:

python
from typing import Union
from pydantic import BaseModel


class Response(BaseModel):
    value: Union[str, int]  # Can be either string or integer

Discriminated Unions

Use discriminated unions to handle different response types:

python
from typing import Literal, Union
from pydantic import BaseModel
import instructor


class UserQuery(BaseModel):
    type: Literal["user"]
    username: str


class SystemQuery(BaseModel):
    type: Literal["system"]
    command: str


Query = Union[UserQuery, SystemQuery]

# Usage with Instructor
client = instructor.from_provider("openai/gpt-4.1-mini")

response = client.create(
    response_model=Query,
    messages=[{"role": "user", "content": "Parse: user lookup jsmith"}],
)

Optional Fields

Combine Union with Optional for nullable fields:

python
from typing import Optional
from pydantic import BaseModel


class User(BaseModel):
    name: str
    email: Optional[str] = None  # Same as Union[str, None]

Best Practices

  1. Type Hints: Use proper type hints for clarity and better IDE support
  2. Discriminators: Add discriminator fields (like type) for complex unions to help the LLM choose correctly
  3. Validation: Add validators for union fields to ensure the data is valid
  4. Documentation: Document expected types clearly in your models with docstrings
  5. Field Names: Use descriptive field names to guide the model's output
  6. Examples: Include examples in your Pydantic models to help the LLM understand the expected format

Common Patterns

Multiple Response Types

python
from typing import Union, Literal
from pydantic import BaseModel


class SuccessResponse(BaseModel):
    status: Literal["success"]
    data: dict


class ErrorResponse(BaseModel):
    status: Literal["error"]
    message: str


Response = Union[SuccessResponse, ErrorResponse]

Nested Unions

python
from typing import Literal, Union, List
from pydantic import BaseModel


class TextContent(BaseModel):
    type: Literal["text"]
    text: str


class ImageContent(BaseModel):
    type: Literal["image"]
    url: str


class Message(BaseModel):
    content: List[Union[TextContent, ImageContent]]

Dynamic Action Selection with Unions

You can use Union types to write "agents" that dynamically choose actions by selecting an output class. For example, in a search and lookup function:

python
from pydantic import BaseModel
from typing import Union


class Search(BaseModel):
    query: str

    def execute(self):
        # Implementation for search
        return f"Searching for: {self.query}"


class Lookup(BaseModel):
    key: str

    def execute(self):
        # Implementation for lookup
        return f"Looking up key: {self.key}"


class Action(BaseModel):
    action: Union[Search, Lookup]

    def execute(self):
        return self.action.execute()

With this pattern, the LLM can decide whether to perform a search or a lookup based on the user's input:

python
import instructor
from pydantic import BaseModel
from typing import Union


class Search(BaseModel):
    query: str

    def execute(self):
        # Implementation for search
        return f"Searching for: {self.query}"


class Lookup(BaseModel):
    key: str

    def execute(self):
        # Implementation for lookup
        return f"Looking up key: {self.key}"


class Action(BaseModel):
    action: Union[Search, Lookup]

    def execute(self):
        return self.action.execute()


client = instructor.from_provider("openai/gpt-4.1-mini")

# Let the LLM decide what action to take
result = client.create(
    response_model=Action,
    messages=[
        {
            "role": "system",
            "content": "You're an assistant that helps search or lookup information.",
        },
        {"role": "user", "content": "Find information about climate change"},
    ],
)

# Execute the chosen action
print(result.execute())  # Likely outputs: "Searching for: climate change"
#> Searching for: climate change

Integration with Instructor

import instructor

from typing import Union, Literal
from pydantic import BaseModel

class SuccessResponse(BaseModel):
status: Literal["success"]
data: dict

class ErrorResponse(BaseModel):
status: Literal["error"]
message: str

Response = Union[SuccessResponse, ErrorResponse]

client = instructor.from_provider("openai/gpt-4.1-mini")

result = client.create(
response_model=Response,
messages=[
{
"role": "system",
"content": "You are a helpful assistant that processes requests and returns either a success or error response.",
},
{
"role": "user",
"content": "Process this request: Get user information for id 123",
},
],
)

Check the result type

if isinstance(result, ErrorResponse):
print(f"Error: {result.message}")
#> Error: Request not supported: Get user information for id 123
else:
print(f"Success: {result.data}")
: User information for id 123 is not available.
else:
print(f"Success: {result.data}")

text
### Streaming with Unions
```python
def stream_content():
    response = client.create(
        response_model=Message,
        stream=True,
        messages=[{"role": "user", "content": "Generate mixed content"}],
    )
    for partial in response:
        if partial.content:
            for item in partial.content:
                if isinstance(item, TextContent):
                    print(f"Text: {item.text}")
                elif isinstance(item, ImageContent):
                    print(ffrom pydantic import ValidationError, BaseModel
from typing import Union, Literal


class SuccessResponse(BaseModel):
    status: Literal["success"]
    data: dict


class ErrorResponse(BaseModel):
    status: Literal["error"]
    message: str


Response = Union[SuccessResponse, ErrorResponse]

try:
    # This will fail because "invalid" is not a valid status
    response = SuccessResponse(status="invalid", data={"key": "value"})
except ValidationError as e:
    print(f"Validation error: {e}")
    """
    Validation error: 1 validation error for SuccessResponse
    status
      Input should be 'success' [type=literal_error, input_value='invalid', input_type=str]
    """
id", data={"key": "value"})
except ValidationError as e:
    print(f"Validation error: {e}")
    """
    Validation error: 1 validation error for SuccessResponse
    status
      Input should be 'success' [type=literal_error, input_value='invalid', input_type=str]
    """

Type Checking

Use isinstance() for runtime type checking:

python
from typing import Union, Literal
from pydantic import BaseModel


class SuccessResponse(BaseModel):
    status: Literal["success"]
    data: dict


class ErrorResponse(BaseModel):
    status: Literal["error"]
    message: str


Response = Union[SuccessResponse, ErrorResponse]


def process_response(response: Response):
    if isinstance(response, SuccessResponse):
        # Handle success case
        print(f"Success: {response.data}")
    elif isinstance(response, ErrorResponse):
        # Handle error case
        print(f"Error: {response.message}")

For more information about union types, check out the Pydantic documentation on unions.

from typing import literal, union
from pydantic import BaseModel
import instructor
from openai import OpenAI


class Action(BaseModel):
    """Base action class."""

    type: str


class SendMessage(BaseModel):
    type: Literal["send_message"]
    message: str
    recipient: str


class MakePayment(BaseModel):
    type: Literal["make_payment"]
    amount: float
    recipient: str


Action = Union[SendMessage, MakePayment]

# Usage with Instructor
client = instructor.from_provider("openai/gpt-4o")
response = client.create(
    response_model=Action,
    messages=[{"role": "user", "content": "Send a payment of $50 to John."}],
)
  ],
)
from typing import literal, union
from pydantic import BaseModel
import instructor
from openai import OpenAI


class SearchAction(BaseModel):
    type: Literal["search"]
    query: str


class EmailAction(BaseModel):
    type: Literal["email"]
    to: str
    subject: str
    body: str


Action = Union[SearchAction, EmailAction]

# The model can choose which action to take
client = instructor.from_provider("openai/gpt-4o")
response = client.create(
    response_model=Action,
    messages=[{"role": "user", "content": "Find me information about climate change."}],
)
  ],
)
from typing import literal, union
from pydantic import BaseModel
import instructor
from openai import OpenAI


class TextResponse(BaseModel):
    type: Literal["text"]
    content: str


class ImageResponse(BaseModel):
    type: Literal["image"]
    url: str
    caption: str


Response = Union[TextResponse, ImageResponse]

# Patched client

See Also

  • Types - Working with different data types in Instructor
  • Enums - Using enumerated types for structured choices
  • Optional Fields - Handling optional data
  • Validation - Validating union type responses
  • Union Examples - Practical union type examples
    client = instructor.from_provider("openai/gpt-4o")
    response = client.create(
    response_model=Response,
    messages=[{"role": "user", "content": "Tell me a joke about programming."}],
    )
    ],
    )
text
```from typing import Union
from pydantic import BaseModel


class Response(BaseModel):
    """A more complex example showing nested Union fields."""

    result: Union[str, int, float, bool]
 bool]
from typing import dict, list, union, any
from pydantic import BaseModel


class Response(BaseModel):
    """A more complex example showing nested Union fields."""

    data: Dict[str, Union[str, int, List[Any]]]
Any]]]

Concepts Usage (`docs/concepts/usage.md`)

title: Handling Non-Streaming Requests in OpenAI with Usage Tracking

See Also

The easiest way to inspect usage for a non-streaming request is to access the completion returned by create_with_completion.

python
import instructor

from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserExtract(BaseModel):
    name: str
    age: int


user, completion = client.create_with_completion(
    response_model=UserExtract,
    messages=[
        {"role": "user", "content": "Extract jason is 25 years old"},
    ],
)

print(completion.usage)
"""
CompletionUsage(
    completion_tokens=10,
    prompt_tokens=79,
    total_tokens=89,
    completion_tokens_details=CompletionTokensDetails(
        accepted_prediction_tokens=None,
        audio_tokens=0,
        reasoning_tokens=0,
        rejected_prediction_tokens=None,
    ),
    prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0),
)
"""

You can catch an IncompleteOutputException whenever the context length is exceeded and react accordingly, such as by trimming your prompt by the number of exceeding tokens.

python
from instructor.core.exceptions import IncompleteOutputException
import instructor
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4.1-mini")


class UserExtract(BaseModel):
    name: str
    age: int


try:
    client.create_with_completion(
        response_model=UserExtract,
        messages=[
            {"role": "user", "content": "Extract jason is 25 years old"},
        ],
    )
except IncompleteOutputException as e:
    token_count = e.last_completion.usage.total_tokens  # type: ignore
    # your logic here

Retry totals and provider differences

For structured responses with OpenAI Chat Completions or stable Anthropic
Messages usage, Instructor accumulates usage across validation retries and
updates the returned completion's usage in place. After retries,
completion.usage therefore contains cumulative values, not just the final
attempt's original usage. Do not sum usage from multiple retained retry
completions. Successful Pydantic models and ListResponse results also expose
_total_usage when all observed responses have compatible usage metadata.

This accounting is not available for every SDK or API. OpenAI Responses uses a
different usage type, as do Anthropic beta Messages and most native provider
SDKs. Their raw usage may be available without a cumulative _total_usage.
Missing optional usage fields are not evidence of zero consumption; current
accumulation can fill some absent fields with zero or carry forward a known
subtotal. Streaming parsed results do not provide a final cumulative usage
record, and an interrupted stream may never receive final usage.

Token categories have different relationships across providers. OpenAI cached
prompt tokens and reasoning completion tokens are breakdowns of the enclosing
totals. Anthropic cache-read and cache-creation input tokens must be added to
its uncached input_tokens to get inclusive input. Do not sum every numeric
usage field or derive a bill from total tokens alone.

Use completion:usage for copied cumulative
snapshots. Replace the previous snapshot instead of summing these events.
To preserve per-attempt provider metadata, copy it immediately inside a
completion:response hook, before Instructor updates the response:

python
from copy import deepcopy

attempt_usage = []


def record_attempt(response):
    attempt_usage.append(deepcopy(getattr(response, "usage", None)))


client.on("completion:response", record_attempt)

This example applies to non-streaming responses with a usage attribute;
Google responses use usage_metadata, and native xAI follows a separate path.
A local response-cache hit can restore historical completion usage without a
new provider request. It is different from provider prompt caching.

Concepts Validation (`docs/concepts/validation.md`)

title: Validation

Validation

Instructor uses Pydantic for validation, providing type checking, data coercion, custom validators, and field constraints.

Validation Flow

mermaid
flowchart TD
    A[Define Pydantic Model] --> B[Send Request to LLM]
    B --> C[LLM Generates Response]
    C --> D{Validate Response}

    D -->|Valid| E[Return Pydantic Object]
    D -->|Invalid| F{Auto-Retry Enabled?}

    F -->|Yes| G[Send Error Context to LLM]
    F -->|No| H[Raise ValidationError]

    G --> I[LLM Generates New Response]
    I --> J{Validate Again}

    J -->|Valid| E
    J -->|Invalid| K{Max Retries Reached?}

    K -->|No| G
    K -->|Yes| H

Basic Validation

Define models with type hints and field constraints:

python
from typing import List
from pydantic import BaseModel, Field, field_validator


class User(BaseModel):
    name: str = Field(..., min_length=2, description="User's full name")
    age: int = Field(..., ge=0, le=150, description="User's age")
    emails: List[str] = Field(description="List of email addresses")

    @field_validator('emails')
    @classmethod
    def validate_emails(cls, v):
        if not all('@' in email for email in v):
            raise ValueError('Invalid email format')
        return v

Field Validation

Use Field() for basic constraints:

python
from pydantic import BaseModel, Field


class Product(BaseModel):
    name: str = Field(..., min_length=1, max_length=100)
    price: float = Field(..., gt=0)
    quantity: int = Field(..., ge=0)

Custom Validators

Use @field_validator for complex validation:

python
from pydantic import BaseModel, Field, field_validator


class Order(BaseModel):
    items: list[str] = Field(description="List of item names")
    total: float = Field(description="Total order amount")

    @field_validator('total')
    @classmethod
    def validate_total(cls, v):
        if v < 0:
            raise ValueError('Total cannot be negative')
        return v

Async Validators

Instructor rejects response models containing @async_field_validator or
@async_model_validator, including nested models, before provider calls and
cache lookups. Automatic async-validator execution is not supported.

For asynchronous policy checks, use an ordinary response model and explicitly
await your application-level check before consuming the result. These checks run
outside Instructor's retry loop and must also run on cached results.

python
import asyncio

from openai import AsyncOpenAI
from pydantic import BaseModel

import instructor


class Answer(BaseModel):
    value: int


async def check_policy(answer: Answer, limit: int) -> Answer:
    if answer.value > limit:
        raise ValueError("Value exceeds the current limit")
    return answer


async def main() -> None:
    async with AsyncOpenAI() as sdk:
        client = instructor.from_openai(sdk)
        answer = await client.create(
            model="gpt-4.1-mini",
            response_model=Answer,
            messages=[{"role": "user", "content": "Return the number 42"}],
        )
        validated = await check_policy(answer, limit=100)
        print(validated.value)


asyncio.run(main())

Pre-validation Transformation

Transform data before validation:

python
from pydantic import BaseModel, field_validator


class UserProfile(BaseModel):
    username: str

    @field_validator('username', mode='before')
    @classmethod
    def lowercase_username(cls, v):
        return v.lower() if isinstance(v, str) else v

Semantic Validation

Use llm_validator for validations that are hard to express programmatically:

python
from typing import Annotated
from pydantic import BaseModel, BeforeValidator
import instructor
from instructor import llm_validator

client = instructor.from_provider("openai/gpt-4.1-mini")


class ContentReview(BaseModel):
    title: str
    content: Annotated[
        str,
        BeforeValidator(
            llm_validator(
                "Content must be family-friendly and not contain profanity",
                client=client,
            )
        ),
    ]

Semantic validation works well for content moderation, tone validation, consistency checks, and complex relationships. For more patterns and details, see the Semantic Validation guide.

Nested Validation

Validate nested structures:

python
from pydantic import BaseModel, Field


class Address(BaseModel):
    street: str
    city: str
    country: str


class User(BaseModel):
    name: str
    addresses: list[Address] = Field(description="User's addresses")

Error Handling

Handle validation failures with appropriate error types:

python
import instructor
from pydantic import BaseModel, field_validator


class User(BaseModel):
    name: str
    age: int

    @field_validator("age")
    @classmethod
    def validate_age(cls, v):
        if v < 0:
            raise ValueError("Age cannot be negative")
        return v


client = instructor.from_provider("openai/gpt-4.1-mini")

try:
    user = client.create(
        response_model=User,
        messages=[
            {"role": "user", "content": "Extract: John, age: -5"},
        ],
    )
except instructor.exceptions.InstructorValidationError as e:
    print(f"Validation error: {e}")

Best Practices

  1. Start simple: Begin with basic type validation before adding complex rules
  2. Use type hints: Always specify types for clarity
  3. Document constraints: Add descriptions to Field() definitions
  4. Choose the right validation type: Rule-based for objective criteria, semantic for subjective
  5. Handle errors: Implement proper error handling for validation failures
  6. Consider costs: Semantic validation with LLMs incurs API costs and latency

See Also

Examples Action Items (`docs/examples/action_items.md`)

title: Automating Action Item Extraction from Meeting Transcripts

Extracting Action Items from Meeting Transcripts

In this guide, we'll walk through how to extract action items from meeting transcripts using OpenAI's API and Pydantic. This use case is essential for automating project management tasks, such as task assignment and priority setting.

For multi-label classification, we introduce a new enum class and a different Pydantic model to handle multiple labels.

!!! tips "Motivation"

text
Significant amount of time is dedicated to meetings, where action items are generated as the actionable outcomes of these discussions. Automating the extraction of action items can save time and guarantee that no critical tasks are overlooked.

Defining the Structures

We'll model a meeting transcript as a collection of Ticket objects, each representing an action item. Every Ticket can have multiple Subtask objects, representing smaller, manageable pieces of the main task.

Extracting Action Items

To extract action items from a meeting transcript, we use the generate function. It calls OpenAI's API, processes the text, and returns a set of action items modeled as ActionItems.

Evaluation and Testing

To test the generate function, we provide it with a sample transcript, and then print the JSON representation of the extracted action items.

text
/* Detailed source-code truncated for AI context efficiency. */

Visualizing the tasks

In order to quickly visualize the data we used code interpreter to create a graphviz export of the json version of the ActionItems array.

json
[
  {
    "id": 1,
    "name": "Improve Authentication System",
    "description": "Revamp the front-end and optimize the back-end of the authentication system",
    "priority": "High",
    "assignees": ["Bob", "Carol"],
    "subtasks": [
      {
        "id": 2,
        "name": "Front-end Revamp"
      },
      {
        "id": 3,
        "name": "Back-end Optimization"
      }
    ],
    "dependencies": []
  },
  {
    "id": 4,
    "name": "Integrate Authentication System with Billing System",
    "description": "Integrate the improved authentication system with the new billing system",
    "priority": "Medium",
    "assignees": ["Bob"],
    "subtasks": [],
    "dependencies": [1]
  },
  {
    "id": 5,
    "name": "Update User Documentation",
    "description": "Update the user documentation to reflect the changes in the authentication system",
    "priority": "Low",
    "assignees": ["Carol"],
    "subtasks": [],
    "dependencies": [2]
  }
]

In this example, the generate function successfully identifies and segments the action items, assigning them priorities, assignees, subtasks, and dependencies as discussed in the meeting.

By automating this process, you can ensure that important tasks and details are not lost in the sea of meeting minutes, making project management more efficient and effective.

Examples Audio Extraction (`docs/examples/audio_extraction.md`)

title: Audio Information Extraction with OpenAI

Audio Information Extraction with OpenAI

This example demonstrates how to use Instructor with OpenAI's audio capabilities to extract structured information from audio files. The example shows how to process audio input and extract specific fields into a Pydantic model.

Prerequisites

  • OpenAI API key with access to GPT-4 audio models
  • An audio file in WAV format
  • Instructor library installed with OpenAI support

Code Example

python
from pydantic import BaseModel
import instructor
from instructor.processing.multimodal import Audio
import base64

# Initialize the OpenAI client with Instructor
client = instructor.from_provider("openai/gpt-5-nano")


# Define the structure for extracted information
class Person(BaseModel):
    name: str
    age: int


# Read and encode the audio file
with open("./output.wav", "rb") as f:
    encoded_string = base64.b64encode(f.read()).decode("utf-8")

# Extract information from the audio
resp = client.create(
    model="gpt-4-audio-preview",
    response_model=Person,
    modalities=["text"],
    audio={"voice": "alloy", "format": "wav"},
    messages=[
        {
            "role": "user",
            "content": [
                "Extract the following information from the audio",
                Audio.from_path("./output.wav"),
            ],
        },
    ],
)

print(resp)
# Example output: Person(name='Jason', age=20)

How It Works

  1. First, we import the necessary libraries including the Audio class from instructor.processing.multimodal.

  2. We define a Pydantic model Person that specifies the structure of the information we want to extract from the audio:

    • name: The person's name
    • age: The person's age

  3. The audio file is read and encoded in base64 format.

  4. We use OpenAI's audio-capable model to process the audio and extract the specified information:

    • The model parameter specifies the GPT-4 audio model
    • response_model tells Instructor to structure the output according to our Person model
    • modalities specifies that we want text output
    • The audio parameter configures audio-specific settings
    • In the message, we use Audio.from_path() to include the audio file

  5. The response is automatically parsed into our Pydantic model, making the extracted information easily accessible in a structured format.

Use Cases

This pattern is particularly useful for:

  • Transcribing and extracting information from recorded interviews
  • Processing voice messages or audio notes
  • Automated form filling from voice input
  • Voice-based data entry systems

Tips

  • Ensure your audio file is in a supported format (WAV in this example)
  • The audio model works best with clear speech and minimal background noise
  • Consider the length of the audio file, as there may be model-specific limitations
  • Structure your Pydantic model to match the information you expect to extract

Examples Batch Classification Langsmith (`docs/examples/batch_classification_langsmith.md`)

title: Enhancing OpenAI Client with LangSmith and Instructor

Seamless Support with Langsmith

Its a common misconception that LangChain's LangSmith is only compatible with LangChain's models. In reality, LangSmith is a unified DevOps platform for developing, collaborating, testing, deploying, and monitoring LLM applications. In this blog we will explore how LangSmith can be used to enhance the OpenAI client alongside instructor.

First, install the necessary packages:

bash
pip install -U langsmith

LangSmith

In order to use langsmith, you first need to set your LangSmith API key.

bash
export LANGCHAIN_API_KEY=<your-api-key>

Next, you will need to install the LangSmith SDK:

bash
pip install -U langsmith
pip install -U instructor

In this example we'll use the wrap_openai function to wrap the OpenAI client with LangSmith. This will allow us to use LangSmith's observability and monitoring features with the OpenAI client. Then we'll use instructor to patch the client with the TOOLS mode. This will allow us to use instructor to add additional functionality to the client.

text
/* Detailed source-code truncated for AI context efficiency. */

If you follow what we've done is wrapped the client and proceeded to quickly use asyncio to classify a list of questions. This is a simple example of how you can use LangSmith to enhance the OpenAI client. You can use LangSmith to monitor and observe the client, and use instructor to add additional functionality to the client.

To take a look at trace of this run check out this shareable link.

Examples Batch In Memory (`docs/examples/batch_in_memory.md`)

title: In-Memory Batch Processing for Serverless Applications

See Also

In-Memory Batch Processing for Serverless

This guide demonstrates how to use Instructor's in-memory batch processing feature, which is perfect for serverless deployments and applications that need to avoid disk I/O.

Overview

In-memory batch processing allows you to create and submit batch requests without writing to disk, using BytesIO buffers instead of files. This is ideal for:

  • Serverless environments (AWS Lambda, Google Cloud Functions, Azure Functions)
  • Containerized applications with read-only file systems
  • Security-sensitive applications that avoid temporary files
  • High-performance applications that minimize I/O overhead

Quick Start

text
/* Detailed source-code truncated for AI context efficiency. */

File vs In-Memory Comparison

Traditional File-Based Approach

python
# File-based approach
processor = BatchProcessor("openai/gpt-5-nano", User)

# Creates file on disk
file_path = processor.create_batch_from_messages(
    messages_list,
    file_path="temp_batch.jsonl",  # Specify file path
    max_tokens=150,
    temperature=0.1,
)

# Submit using file path
batch_id = processor.submit_batch(file_path)

# Remember to clean up
import os

if os.path.exists(file_path):
    os.remove(file_path)

New In-Memory Approach

python
# In-memory approach
processor = BatchProcessor("openai/gpt-5-nano", User)

# Creates BytesIO buffer in memory
buffer = processor.create_batch_from_messages(
    messages_list,
    file_path=None,  # No file path = in-memory
    max_tokens=150,
    temperature=0.1,
)

# Submit using buffer
batch_id = processor.submit_batch(buffer)

# No cleanup required - buffer is automatically garbage collected

Benefits of In-Memory Processing

✅ Perfect for Serverless

python
# AWS Lambda example
import json


def lambda_handler(event, _context):
    """AWS Lambda function using in-memory batch processing."""

    # Extract data from event
    messages_list = event.get("messages", [])

    # Process in memory - no disk I/O
    processor = BatchProcessor("openai/gpt-5-nano", User)
    buffer = processor.create_batch_from_messages(
        messages_list,
        file_path=None,  # Essential for Lambda
    )

    batch_id = processor.submit_batch(buffer)

    return {
        'statusCode': 200,
        'body': json.dumps(
            {'batch_id': batch_id, 'message': 'Batch submitted successfully'}
        ),
    }

✅ Memory Efficient

python
# Check buffer size before submission
buffer = processor.create_batch_from_messages(messages_list, file_path=None)

print(f"Buffer size: {len(buffer.getvalue())} bytes")
print(f"Buffer type: {type(buffer)}")

# Buffer content is accessible
buffer.seek(0)
content_preview = buffer.read(200).decode("utf-8")
print(f"Preview: {content_preview}...")

# Reset for submission
buffer.seek(0)
batch_id = processor.submit_batch(buffer)

✅ Security Benefits

python
# No temporary files on disk
# No file permissions to manage
# No cleanup required
# Buffer is automatically garbage collected

processor = BatchProcessor("openai/gpt-5-nano", User)

# This approach leaves no trace on the file system
buffer = processor.create_batch_from_messages(
    sensitive_messages,
    file_path=None,  # Keeps everything in memory
)

batch_id = processor.submit_batch(buffer)
# When buffer goes out of scope, it's automatically cleaned up

Error Handling

python
try:
    # Create batch buffer
    buffer = processor.create_batch_from_messages(
        messages_list,
        file_path=None,
    )

    # Submit batch
    batch_id = processor.submit_batch(buffer)

    # Process results
    results = processor.get_results(batch_id)

except Exception as e:
    print(f"Error during batch processing: {e}")
    #> Error during batch processing: name 'processor' is not defined
    # No file cleanup needed with in-memory approach

Provider Support

All providers support in-memory batch processing:

OpenAI

python
processor = BatchProcessor("openai/gpt-5-nano", User)
buffer = processor.create_batch_from_messages(messages_list, file_path=None)
batch_id = processor.submit_batch(buffer)

Anthropic

python
processor = BatchProcessor("anthropic/claude-3-5-sonnet-20241022", User)
buffer = processor.create_batch_from_messages(messages_list, file_path=None)
batch_id = processor.submit_batch(buffer)

Google GenAI

python
processor = BatchProcessor("google/gemini-2.5-flash", User)
buffer = processor.create_batch_from_messages(messages_list, file_path=None)
batch_id = processor.submit_batch(buffer)

Best Practices

  1. Always set file_path=None to enable in-memory mode
  2. Monitor buffer size for large batches to avoid memory issues
  3. Use appropriate models that support JSON schema (e.g., gpt-4o-mini)
  4. Handle errors gracefully - no file cleanup needed
  5. Consider memory limits in serverless environments

Limitations

  • Memory usage: Large batches may consume significant memory
  • No debugging files: Can't inspect batch files for troubleshooting
  • Temporary storage: Buffer contents are lost if not submitted immediately

Troubleshooting

Buffer Size Issues

python
# Check buffer size before submission
buffer = processor.create_batch_from_messages(messages_list, file_path=None)
size_mb = len(buffer.getvalue()) / (1024 * 1024)
print(f"Buffer size: {size_mb:.2f} MB")

if size_mb > 100:  # Adjust threshold as needed
    print("Warning: Large buffer size, consider splitting batch")

Memory Monitoring

python
import psutil
import os

# Check memory usage
process = psutil.Process(os.getpid())
memory_before = process.memory_info().rss / 1024 / 1024  # MB

buffer = processor.create_batch_from_messages(messages_list, file_path=None)

memory_after = process.memory_info().rss / 1024 / 1024  # MB
print(f"Memory increase: {memory_after - memory_before:.2f} MB")

This in-memory approach makes Instructor's batch processing perfect for modern serverless and containerized applications while maintaining the same powerful API and provider support.

Examples Batch Job Oai (`docs/examples/batch_job_oai.md`)

title: Generating Synthetic Data with OpenAI's Batch API

See Also

Bulk Generation of Synthetic Data

This tutorial shows how to use instructor to generate large quantities of synthetic data at scale using Open AI's new Batch API. In this example, we'll be generating synthetic questions using the ms-marco dataset to evaluate RAG retrieval.

??? tips "Why use the batch API?"

text
There are a few reasons why you might want to use the Batch API

1. Batch Jobs are 50% cheaper than running an inference job on demand ( see Open AI's pricing page [here](https://openai.com/api/pricing/) )

2. Batch Jobs have higher rate limits than normal api calls

3. Batch Jobs support both normal models **and fine-tuned models**

This makes them perfect for non time-sensitive tasks that involve large quantities of data.

Getting Started

Let's first see how we can generate a Question and Answer Pair using Instructor with a normal OpenAI function call.

text
/* Detailed source-code truncated for AI context efficiency. */

As the number of chunks we'd like to generate these synthetic questions for increases, the cost will grow proportionally.

Let's see how we can use the BatchJob object to create a .jsonl file which is compatible with the Batch API.

text
/* Detailed source-code truncated for AI context efficiency. */

  1. We first define a generator which generates a list of messages which we would have made in a normal openai api call

  2. We then use the create_from_messages class method to specify the model and response_model that we want. instructor will handle the generation of the openai schema behind the scenes as well as write the output to the file path you specify

Once we've got this new .jsonl file, we can then use the new instructor cli's batch command to create a new batch job.

bash
> % ls -a | grep test.jsonl
test.jsonl

> % instructor batch create-from-file --file-path test.jsonl

This will create a table like what you see below. In my case, my batch job took around 6 minutes to complete and cost me $2.72 to run.

Batch ID Created At Status Failed Completed Total
batch_Z8XUudoweH43R9c4sr4wRYub 2024-07-16 12:45:22 in_progress 0 483 1627

Once our batch job is complete, the status will change to completed.

??? "Cancelling A Job"

text
If you'd like to cancel a batch job midway, you can do so too with the instructor `batch` cli command

```bash
instructor batch cancel --batch-id <batch id here>
```

We can then download the file generated by the batch job using the cli command

bash
instructor batch download-file --download-file-path output.jsonl --batch-id batch_Z8XUudoweH43R9c4sr4wRYub

This will then create a .jsonl file with the generated content at the path that you specify.

Parsing the generated response

We can then parse the generated response by using the .parse_from_file command provided by the BatchJob class.

python hl_lines="19-21"
from instructor.batch import BatchJob
from pydantic import BaseModel, Field

# <%hide%>
with open("./output.jsonl", "w") as f:
    f.write('')
# <%hide%>


class QuestionAnswerPair(BaseModel):
    """
    This model represents a pair of a question generated from a text chunk, its corresponding answer,
    and the chain of thought leading to the answer. The chain of thought provides insight into how the answer
    was derived from the question.
    """

    chain_of_thought: str = Field(
        description="The reasoning process leading to the answer."
    )
    question: str = Field(description="The generated question from the text chunk.")
    answer: str = Field(description="The answer to the generated question.")


parsed, unparsed = BatchJob.parse_from_file(  # (1)!
    file_path="./output.jsonl", response_model=QuestionAnswerPair
)

print(len(parsed))
#> 0
print(len(unparsed))
#> 0

# <%hide%>
import os

if os.path.exists("./output.jsonl"):
    os.remove("./output.jsonl")
# <%hide%>
  1. We can then use a generic Pydantic schema to parse the generated function calls back

This will then return a list of two elements

  • parsed is a list of responses that have been successfully parsed into the QuestionAnswerPair Base Model class
  • unparsed is a second list which contains responses which were not able to be parsed into the QuestionAnswerPair Base Model class

Examples Building Knowledge Graphs (`docs/examples/building_knowledge_graphs.md`)

title: Building Knowledge Graphs from Text

See Also

Building Knowledge Graphs from Textual Data

In this tutorial, we will explore the process of constructing knowledge graphs from textual data using OpenAI's API and Pydantic. This approach is crucial for efficiently automating the extraction of structured information from unstructured text.

text
/* Detailed source-code truncated for AI context efficiency. */

Examples Bulk Classification (`docs/examples/bulk_classification.md`)

title: User-Provided Tag Classification Tutorial

See Also

Bulk Classification from User-Provided Tags.

This tutorial shows how to do classification from user provided tags. This is valuable when you want to provide services that allow users to do some kind of classification.

!!! tips "Motivation"

text
Imagine allowing the user to upload documents as part of a RAG application. Oftentimes, we might want to allow the user to specify an existing set of tags, give descriptions, and do the classification for them.

Defining the Structures

One of the easy things to do is to allow users to define a set of tags in some kind of schema and save that in a database. Here's an example of a schema that we might use:

tag_id name instructions
0 personal Personal information
1 phone Phone number
2 email Email address
3 address Address
4 Other Other information
  1. tag_id - The unique identifier for the tag.
  2. name - The name of the tag.
  3. instructions - A description of the tag, which can be used as a prompt to describe the tag.

Implementing the Classification

In order to do this we'll do a couple of things:

  1. We'll use the instructor library with async client support.
  2. Implement a Tag model that will be used to validate the tags from the context. (This will allow us to avoid hallucinating tags that are not in the context.)
  3. Helper models for the request and response.
  4. An async function to do the classification.
  5. A main function to run the classification using the asyncio.gather function to run the classification in parallel.

If you want to learn more about how to do bad computations, check out our post on AsyncIO here.

python
import instructor

client = instructor.from_provider("openai/gpt-4o", async_client=True)

First, we'll need to import all of our Pydantic and instructor code and use the AsyncOpenAI client. Then, we'll define the tag model along with the tag instructions to provide input and output.

This is very helpful because once we use something like FastAPI to create endpoints, the Pydantic functions will serve as multiple tools:

  1. A description for the developer
  2. Type hints for the IDE
  3. OpenAPI documentation for the FastAPI endpoint
  4. Schema and Response Model for the language model.
python
from typing import List
from pydantic import BaseModel, ValidationInfo, model_validator


class Tag(BaseModel):
    id: int
    name: str

    @model_validator(mode="after")
    def validate_ids(self, info: ValidationInfo):
        context = info.context
        if context:
            tags: List[Tag] = context.get("tags")
            assert self.id in {
                tag.id for tag in tags
            }, f"Tag ID {self.id} not found in context"
            assert self.name in {
                tag.name for tag in tags
            }, f"Tag name {self.name} not found in context"
        return self


class TagWithInstructions(Tag):
    instructions: str


class TagRequest(BaseModel):
    texts: List[str]
    tags: List[TagWithInstructions]


class TagResponse(BaseModel):
    texts: List[str]
    predictions: List[Tag]

Let's delve deeper into what the validate_ids function does. Notice that its purpose is to extract tags from the context and ensure that each ID and name exists in the set of tags. This approach helps minimize hallucinations. If we mistakenly identify either the ID or the tag, an error will be thrown, and the instructor will prompt the language model to retry until the correct item is successfully extracted.

python
from pydantic import model_validator, ValidationInfo


@model_validator(mode="after")
def validate_ids(self, info: ValidationInfo):
    context = info.context
    if context:
        tags: List[Tag] = context.get("tags")
        assert self.id in {
            tag.id for tag in tags
        }, f"Tag ID {self.id} not found in context"
        assert self.name in {
            tag.name for tag in tags
        }, f"Tag name {self.name} not found in context"
    return self

Now, let's implement the function to do the classification. This function will take a single text and a list of tags and return the predicted tag.

text
/* Detailed source-code truncated for AI context efficiency. */

Notice that we first define a single async function that makes a prediction of a tag, and we pass it into the validation context in order to minimize hallucinations.

Finally, we'll implement the main function to run the classification using the asyncio.gather function to run the classification in parallel.

text
/* Detailed source-code truncated for AI context efficiency. */

Which would result in:

json
{
  "texts": [
    "What is your phone number?",
    "What is your email address?",
    "What is your address?",
    "What is your privacy policy?"
  ],
  "predictions": [
    {
      "id": 1,
      "name": "phone"
    },
    {
      "id": 2,
      "name": "email"
    },
    {
      "id": 3,
      "name": "address"
    },
    {
      "id": 4,
      "name": "Other"
    }
  ]
}

What happens in production?

If we were to use this in production, we might expect to have some kind of fast API endpoint.

python
from fastapi import FastAPI

app = FastAPI()

# <%hide%>
from typing import List
from pydantic import BaseModel, ValidationInfo, model_validator


class Tag(BaseModel):
    id: int
    name: str

    @model_validator(mode="after")
    def validate_ids(self, info: ValidationInfo):
        context = info.context
        if context:
            tags: List[Tag] = context.get("tags")
            assert self.id in {
                tag.id for tag in tags
            }, f"Tag ID {self.id} not found in context"
            assert self.name in {
                tag.name for tag in tags
            }, f"Tag name {self.name} not found in context"
        return self


class TagWithInstructions(Tag):
    instructions: str


class TagRequest(BaseModel):
    texts: List[str]
    tags: List[TagWithInstructions]


class TagResponse(BaseModel):
    texts: List[str]
    predictions: List[Tag]


# <%hide%>
@app.post("/tag", response_model=TagResponse)
async def tag(request: TagRequest) -> TagResponse:
    return await tag_request(request)

Since everything is already annotated with Pydantic, this code is very simple to write!

!!! warning "Where do tags come from?"

text
I just want to call out that here you can also imagine the tag spec IDs and names and instructions for example could come from a database or somewhere else. I'll leave this as an exercise to the reader, but I hope this gives us a clear understanding of how we can do something like user-defined classification.

Improving the Model

There's a couple things we could do to make this system a little bit more robust.

  1. Use confidence score:
python
# <%hide%>
from typing import List
from pydantic import BaseModel, ValidationInfo, model_validator, Field


class Tag(BaseModel):
    id: int
    name: str

    @model_validator(mode="after")
    def validate_ids(self, info: ValidationInfo):
        context = info.context
        if context:
            tags: List[Tag] = context.get("tags")
            assert self.id in {
                tag.id for tag in tags
            }, f"Tag ID {self.id} not found in context"
            assert self.name in {
                tag.name for tag in tags
            }, f"Tag name {self.name} not found in context"
        return self


# <%hide%>
class TagWithConfidence(Tag):
    confidence: float = Field(
        ...,
        ge=0,
        le=1,
        description="The confidence of the prediction, 0 is low, 1 is high",
    )
  1. Use multiclass classification:

Notice in the example we use Iterable[Tag] vs Tag. This is because we might want to use a multiclass classification model that returns multiple tag!

python
import instructor
import instructor
import asyncio
from typing import Iterable

client = instructor.from_openai(
    openai.AsyncOpenAI(),
)

# <%hide%>
from typing import List
from pydantic import BaseModel, ValidationInfo, model_validator


class Tag(BaseModel):
    id: int
    name: str

    @model_validator(mode="after")
    def validate_ids(self, info: ValidationInfo):
        context = info.context
        if context:
            tags: List[Tag] = context.get("tags")
            assert self.id in {
                tag.id for tag in tags
            }, f"Tag ID {self.id} not found in context"
            assert self.name in {
                tag.name for tag in tags
            }, f"Tag name {self.name} not found in context"
        return self


# <%hide%>
tags = [
    Tag(id=0, name="personal"),
    Tag(id=1, name="phone"),
    Tag(id=2, name="email"),
    Tag(id=3, name="address"),
    Tag(id=4, name="Other"),
]

# Texts will be a range of different questions.
# Such as "How much does it cost?", "What is your privacy policy?", etc.
text = "What is your phone number?"


async def get_tags(text: List[str], tags: List[Tag]) -> List[Tag]:
    allowed_tags = [(tag.id, tag.name) for tag in tags]
    allowed_tags_str = ", ".join([f"`{tag}`" for tag in allowed_tags])

    return await client.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": "You are a world-class text tagging system.",
            },
            {"role": "user", "content": f"Describe the following text: `{text}`"},
            {
                "role": "user",
                "content": f"Here are the allowed tags: {allowed_tags_str}",
            },
        ],
        response_model=Iterable[Tag],
        context={"tags": tags},
    )


tag_results = asyncio.run(get_tags(text, tags))
for tag in tag_results:
    print(tag)
    #> id=1 name='phone'

---

### Examples Classification (`docs/examples/classification.md`)

---
title: Text Classification with OpenAI and Pydantic
description: Learn to implement single-label and multi-label text classification using OpenAI API and Pydantic models in Python.
---

# Text Classification using OpenAI and Pydantic

This tutorial showcases how to implement text classification tasks-specifically, single-label and multi-label classifications-using the OpenAI API and Pydantic models. For complete examples, check out our [single classification](./bulk_classification.md) and [multi-label classification](./bulk_classification.md) examples in the cookbook.

!!! tips "Motivation"

    Text classification is a common problem in many NLP applications, such as spam detection or support ticket categorization. The goal is to provide a systematic way to handle these cases using OpenAI's GPT models in combination with Python data structures.

## Single-Label Classification

### Defining the Structures

For single-label classification, we define a Pydantic model with a [Literal](../concepts/prompting.md#literals) field for the possible labels.

!!! note "Literals vs Enums"

    We prefer using `Literal` types over `enum` for classification labels. Literals provide better type checking and are more straightforward to use with Pydantic models.

!!! important "Few-Shot Examples"

    Including few-shot examples in the model's docstring is crucial for improving the model's classification accuracy. These examples guide the AI in understanding the task and expected outputs.

    If you want to learn more prompting tips check out our [prompting guide](../prompting/index.md)

!!! note "Chain of Thought"

    Using [Chain of Thought](../concepts/prompting.md#chain-of-thought) has been shown to improve the quality of the predictions by ~ 10%

```python
from pydantic import BaseModel, Field
from typing import Literal
import instructor

# Apply the patch to the OpenAI client
# enables response_model keyword
client = instructor.from_provider("openai/gpt-5-nano")


class ClassificationResponse(BaseModel):
    """
    A few-shot example of text classification:

    Examples:
    - "Buy cheap watches now!": SPAM
    - "Meeting at 3 PM in the conference room": NOT_SPAM
    - "You've won a free iPhone! Click here": SPAM
    - "Can you pick up some milk on your way home?": NOT_SPAM
    - "Increase your followers by 10000 overnight!": SPAM
    """

    chain_of_thought: str = Field(
        ...,
        description="The chain of thought that led to the prediction.",
    )
    label: Literal["SPAM", "NOT_SPAM"] = Field(
        ...,
        description="The predicted class label.",
    )

Classifying Text

The function classify will perform the single-label classification.

python
# <%hide%>
from pydantic import BaseModel, Field
from typing import Literal
import instructor


class ClassificationResponse(BaseModel):
    """
    A few-shot example of text classification:

    Examples:
    - "Buy cheap watches now!": SPAM
    - "Meeting at 3 PM in the conference room": NOT_SPAM
    - "You've won a free iPhone! Click here": SPAM
    - "Can you pick up some milk on your way home?": NOT_SPAM
    - "Increase your followers by 10000 overnight!": SPAM
    """

    chain_of_thought: str = Field(
        ...,
        description="The chain of thought that led to the prediction.",
    )
    label: Literal["SPAM", "NOT_SPAM"] = Field(
        ...,
        description="The predicted class label.",
    )


# Apply the patch to the OpenAI client
# enables response_model keyword
client = instructor.from_provider("openai/gpt-5-nano")


# <%hide%>
def classify(data: str) -> ClassificationResponse:
    """Perform single-label classification on the input text."""
    return client.create(
        model="gpt-4o-mini",
        response_model=ClassificationResponse,
        messages=[
            {
                "role": "user",
                "content": f"Classify the following text: <text>{data}</text>",
            },
        ],
    )

Testing and Evaluation

Let's run examples to see if it correctly identifies spam and non-spam messages.

text
/* Detailed source-code truncated for AI context efficiency. */

Multi-Label Classification

Defining the Structures

For multi-label classification, we'll update our approach to use Literals instead of enums, and include few-shot examples in the model's docstring.

python
from typing import List
from pydantic import BaseModel, Field
from typing import Literal


class MultiClassPrediction(BaseModel):
    """
    Class for a multi-class label prediction.

    Examples:
    - "My account is locked": ["TECH_ISSUE"]
    - "I can't access my billing info": ["TECH_ISSUE", "BILLING"]
    - "When do you close for holidays?": ["GENERAL_QUERY"]
    - "My payment didn't go through and now I can't log in": ["BILLING", "TECH_ISSUE"]
    """

    chain_of_thought: str = Field(
        ...,
        description="The chain of thought that led to the prediction.",
    )

    class_labels: List[Literal["TECH_ISSUE", "BILLING", "GENERAL_QUERY"]] = Field(
        ...,
        description="The predicted class labels for the support ticket.",
    )

Classifying Text

The function multi_classify is responsible for multi-label classification.

python
# <%hide%>
from typing import List
from pydantic import BaseModel, Field
from typing import Literal


class MultiClassPrediction(BaseModel):
    """
    Class for a multi-class label prediction.

    Examples:
    - "My account is locked": ["TECH_ISSUE"]
    - "I can't access my billing info": ["TECH_ISSUE", "BILLING"]
    - "When do you close for holidays?": ["GENERAL_QUERY"]
    - "My payment didn't go through and now I can't log in": ["BILLING", "TECH_ISSUE"]
    """

    chain_of_thought: str = Field(
        ...,
        description="The chain of thought that led to the prediction.",
    )

    class_labels: List[Literal["TECH_ISSUE", "BILLING", "GENERAL_QUERY"]] = Field(
        ...,
        description="The predicted class labels for the support ticket.",
    )


# <%hide%>
import instructor

client = instructor.from_provider("openai/gpt-5-nano")


def multi_classify(data: str) -> MultiClassPrediction:
    """Perform multi-label classification on the input text."""
    return client.create(
        model="gpt-4o-mini",
        response_model=MultiClassPrediction,
        messages=[
            {
                "role": "user",
                "content": f"Classify the following support ticket: <ticket>{data}</ticket>",
            },
        ],
    )

Testing and Evaluation

Finally, we test the multi-label classification function using a sample support ticket.

text
/* Detailed source-code truncated for AI context efficiency. */

By using Literals and including few-shot examples, we've improved both the single-label and multi-label classification implementations. These changes enhance type safety and provide better guidance for the AI model, potentially leading to more accurate classifications.

Examples Document Segmentation (`docs/examples/document_segmentation.md`)

title: "Document Segmentation with LLMs: A Comprehensive Guide" description: Learn effective document segmentation techniques using Cohere's LLM, enhancing comprehension of complex texts.

See Also

Document Segmentation

In this guide, we demonstrate how to do document segmentation using structured output from an LLM. We'll be using command-a - one of Cohere's latest LLMs with 256k context length and testing the approach on an article explaining the Transformer architecture. Same approach to document segmentation can be applied to any other domain where we need to break down a complex long document into smaller chunks.

!!! tips "Motivation"
Sometimes we need a way to split the document into meaningful parts that center around a single key concept/idea. Simple length-based / rule-based text-splitters are not reliable enough. Consider the cases where documents contain code snippets or math equations - we don't want to split those on '\n\n' or have to write extensive rules for different types of documents. It turns out that LLMs with sufficiently long context length are well suited for this task.

Defining the Data Structures

First, we need to define a Section class for each of the document's segments. StructuredDocument class will then encapsulate a list of these sections.

Note that in order to avoid LLM regenerating the content of each section, we can simply enumerate each line of the input document and then ask LLM to segment it by providing start-end line numbers for each section.

python
from pydantic import BaseModel, Field
from typing import List


class Section(BaseModel):
    title: str = Field(description="main topic of this section of the document")
    start_index: int = Field(description="line number where the section begins")
    end_index: int = Field(description="line number where the section ends")


class StructuredDocument(BaseModel):
    """obtains meaningful sections, each centered around a single concept/topic"""

    sections: List[Section] = Field(description="a list of sections of the document")

Document Preprocessing

Preprocess the input document by prepending each line with its number.

python
def doc_with_lines(document):
    document_lines = document.split("\n")
    document_with_line_numbers = ""
    line2text = {}
    for i, line in enumerate(document_lines):
        document_with_line_numbers += f"[{i}] {line}\n"
        line2text[i] = line
    return document_with_line_numbers, line2text

Segmentation

Next use a Cohere client to extract StructuredDocument from the preprocessed doc.

text
/* Detailed source-code truncated for AI context efficiency. */

Next, we need to get back the section text based on the start/end indices and our line2text dict from the preprocessing step.

python
def get_sections_text(structured_doc, line2text):
    segments = []
    for s in structured_doc.sections:
        contents = []
        for line_id in range(s.start_index, s.end_index):
            contents.append(line2text.get(line_id, ''))
        segments.append(
            {
                "title": s.title,
                "content": "\n".join(contents),
                "start": s.start_index,
                "end": s.end_index,
            }
        )
    return segments

Example

Here's an example of using these classes and functions to segment a tutorial on Transformers from Sebastian Raschka. We can use trafilatura package to scrape the web page content of the article.

text
/* Detailed source-code truncated for AI context efficiency. */
text
/* Detailed source-code truncated for AI context efficiency. */

Examples Entity Resolution (`docs/examples/entity_resolution.md`)

See Also

In this guide, we demonstrate how to extract and resolve entities from a sample legal contract. Then, we visualize these entities and their dependencies as an entity graph. This approach can be invaluable for legal tech applications, aiding in the understanding of complex documents.

!!! tips "Motivation"

text
Legal contracts are full of intricate details and interconnected clauses. Automatically extracting and visualizing these elements can make it easier to understand the document's overall structure and terms.

Defining the Data Structures

The Entity and Property classes model extracted entities and their attributes. DocumentExtraction encapsulates a list of these entities.

python
from pydantic import BaseModel, Field
from typing import List


class Property(BaseModel):
    key: str
    value: str
    resolved_absolute_value: str


class Entity(BaseModel):
    id: int = Field(
        ...,
        description="Unique identifier for the entity, used for deduplication, design a scheme allows multiple entities",
    )
    subquote_string: List[str] = Field(
        ...,
        description="Correctly resolved value of the entity, if the entity is a reference to another entity, this should be the id of the referenced entity, include a few more words before and after the value to allow for some context to be used in the resolution",
    )
    entity_title: str
    properties: List[Property] = Field(
        ..., description="List of properties of the entity"
    )
    dependencies: List[int] = Field(
        ...,
        description="List of entity ids that this entity depends  or relies on to resolve it",
    )


class DocumentExtraction(BaseModel):
    entities: List[Entity] = Field(
        ...,
        description="Body of the answer, each fact should be a separate object with a body and a list of sources",
    )

Entity Extraction and Resolution

The ask_ai function utilizes OpenAI's API to extract and resolve entities from the input content.

text
/* Detailed source-code truncated for AI context efficiency. */

Graph Visualization

generate_graph takes the extracted entities and visualizes them using Graphviz. It creates nodes for each entity and edges for their dependencies.

text
/* Detailed source-code truncated for AI context efficiency. */

Execution

Finally, execute the code to visualize the entity graph for the sample legal contract.

text
/* Detailed source-code truncated for AI context efficiency. */

This will produce a graphical representation of the entities and their dependencies, stored as "entity.gv".

Examples Exact Citations (`docs/examples/exact_citations.md`)

title: Citation Validation with Instructor - Prevent Hallucinations description: Validate AI-generated answers with contextual citations using Instructor. Ensure every statement is backed by source quotes to prevent hallucinations.

Example: Answering Questions with Validated Citations

For the full code example, check out examples/citation_fuzzy_match.py

Overview

This example shows how to use Instructor with validators to not only add citations to answers generated but also prevent hallucinations by ensuring that every statement made by the LLM is backed up by a direct quote from the context provided, and that those quotes exist!
Two Python classes, Fact and QuestionAnswer, are defined to encapsulate the information of individual facts and the entire answer, respectively.

Data Structures

The `Fact` Class

The Fact class encapsulates a single statement or fact. It contains two fields:

  • fact: A string representing the body of the fact or statement.
  • substring_quote: A list of strings. Each string is a direct quote from the context that supports the fact.

Validation Method: `validate_sources`

This method validates the sources (substring_quote) in the context. It utilizes regex to find the span of each substring quote in the given context. If the span is not found, the quote is removed from the list.

python hl_lines="6 8-13"
from pydantic import Field, BaseModel, model_validator, ValidationInfo
from typing import List


class Fact(BaseModel):
    fact: str = Field(...)
    substring_quote: List[str] = Field(...)

    @model_validator(mode="after")
    def validate_sources(self, info: ValidationInfo) -> "Fact":
        text_chunks = info.context.get("text_chunk", None)
        spans = list(self.get_spans(text_chunks))
        self.substring_quote = [text_chunks[span[0] : span[1]] for span in spans]
        return self

    def get_spans(self, context):
        for quote in self.substring_quote:
            yield from self._get_span(quote, context)

    def _get_span(self, quote, context):
        for match in re.finditer(re.escape(quote), context):
            yield match.span()

The `QuestionAnswer` Class

This class encapsulates the question and its corresponding answer. It contains two fields:

  • question: The question asked.
  • answer: A list of Fact objects that make up the answer.

Validation Method: `validate_sources`

This method checks that each Fact object in the answer list has at least one valid source. If a Fact object has no valid sources, it is removed from the answer list.

python hl_lines="5-8"
from pydantic import BaseModel, Field, model_validator
from typing import List

# <%hide%>
from pydantic import ValidationInfo


class Fact(BaseModel):
    fact: str = Field(...)
    substring_quote: List[str] = Field(...)

    @model_validator(mode="after")
    def validate_sources(self, info: ValidationInfo) -> "Fact":
        text_chunks = info.context.get("text_chunk", None)
        spans = list(self.get_spans(text_chunks))
        self.substring_quote = [text_chunks[span[0] : span[1]] for span in spans]
        return self

    def get_spans(self, context):
        for quote in self.substring_quote:
            yield from self._get_span(quote, context)

    def _get_span(self, quote, context):
        for match in re.finditer(re.escape(quote), context):
            yield match.span()


# <%hide%>
class QuestionAnswer(BaseModel):
    question: str = Field(...)
    answer: List[Fact] = Field(...)

    @model_validator(mode="after")
    def validate_sources(self) -> "QuestionAnswer":
        self.answer = [fact for fact in self.answer if len(fact.substring_quote) > 0]
        return self

Function to Ask AI a Question

The `ask_ai` Function

This function takes a string question and a string context and returns a QuestionAnswer object. It uses the OpenAI API to fetch the answer and then validates the sources using the defined classes.

To understand the validation context work from pydantic check out pydantic's docs

text
/* Detailed source-code truncated for AI context efficiency. */

Example

Here's an example of using these classes and functions to ask a question and validate the answer.

python
question = "What did the author do during college?"
context = """
My name is Jason Liu, and I grew up in Toronto Canada but I was born in China.
I went to an arts high school but in university I studied Computational Mathematics and physics.
As part of coop I worked at many companies including Stitchfix, Facebook.
I also started the Data Science club at the University of Waterloo and I was the president of the club for 2 years.
"""

The output would be a QuestionAnswer object containing validated facts and their sources.

python
{
    "question": "where did he go to school?",
    "answer": [
        {
            "statement": "Jason Liu went to an arts highschool.",
            "substring_phrase": ["arts highschool"],
        },
        {
            "statement": "Jason Liu studied Computational Mathematics and physics in university.",
            "substring_phrase": ["university"],
        },
    ],
}

This ensures that every piece of information in the answer has been validated against the context.

Examples Examples (`docs/examples/examples.md`)

title: Few-Shot Learning with Examples - Pydantic Models description: Enhance Pydantic models with practical examples for few-shot learning. Improve LLM understanding with example-driven JSON schemas.

How should I include examples?

To enhance the clarity and usability of your model and prompt, incorporating examples directly into the JSON schema extra of your Pydantic model is highly recommended. This approach not only streamlines the integration of practical examples but also ensures that they are easily accessible and understandable within the context of your model's schema.

text
/* Detailed source-code truncated for AI context efficiency. */

Examples Extract Contact Info (`docs/examples/extract_contact_info.md`)

title: Contact Information Extraction - Lead Generation Automation description: Automate customer lead extraction from text using Instructor. Extract names, phone numbers, and contact details with automatic validation.

Customer Information Extraction

In this guide, we'll walk through how to extract customer lead information using OpenAI's API and Pydantic. This use case is essential for seamlessly automating the process of extracting specific information from a context.

Motivation

You could potentially integrate this into a chatbot to extract relevant user information from user messages. With the use of machine learning driven validation it would reduce the need for a human to verify the information.

Defining the Structure

We'll model a customer lead as a Lead object, including attributes for the name and phone number. We'll use a Pydantic PhoneNumber type to validate the phone numbers entered and provide a Field to give the model more information on correctly populating the object.

Extracting Lead Information

To extract lead information, we create the parse_lead_from_message function which integrates Instructor. It calls OpenAI's API, processes the text, and returns the extracted lead information as a Lead object.

Evaluating Lead Extraction

To showcase the parse_lead_from_message function we can provide sample user messages that may be obtained from a dialogue with a chatbot assistant. Also take note of the response model being set as Iterable[Lead] this allows for multiple leads being extracted from the same message.

text
/* Detailed source-code truncated for AI context efficiency. */

In this example, the parse_lead_from_message function successfully extracts lead information from a user message, demonstrating how automation can enhance the efficiency of collecting accurate customer details. It also shows how the function successfully catches that the phone number is invalid so functionality can be implemented for the user to get prompted again to give a correct phone number.

Examples Extract Slides (`docs/examples/extract_slides.md`)

title: Extracting Competitor Data from Slides Using AI description: Learn how to extract competitor data from presentation slides, leveraging AI for comprehensive information gathering.

Data extraction from slides

In this guide, we demonstrate how to extract data from slides.

!!! tips "Motivation"

When we want to translate key information from slides into structured data, simply isolating the text and running extraction might not be enough. Sometimes the important data is in the images on the slides, so we should consider including them in our extraction pipeline.

Defining the necessary Data Structures

Let's say we want to extract the competitors from various presentations and categorize them according to their respective industries.

Our data model will have Industry which will be a list of Competitor's for a specific industry, and Competition which will aggregate the competitors for all the industries.

python
from pydantic import BaseModel, Field
from typing import Optional, List


class Competitor(BaseModel):
    name: str
    features: Optional[List[str]]


# Define models
class Industry(BaseModel):
    """
    Represents competitors from a specific industry extracted from an image using AI.
    """

    name: str = Field(description="The name of the industry")
    competitor_list: List[Competitor] = Field(
        description="A list of competitors for this industry"
    )


class Competition(BaseModel):
    """
    This class serves as a structured representation of
    competitors and their qualities.
    """

    industry_list: List[Industry] = Field(
        description="A list of industries and their competitors"
    )

Competitors extraction

To extract competitors from slides we will define a function which will read images from urls and extract the relevant information from them.

text
/* Detailed source-code truncated for AI context efficiency. */

Execution

Finally, we will run the previous function with a few sample slides to see the data extractor in action.

As we can see, our model extracted the relevant information for each competitor regardless of how this information was formatted in the original presentations.

text
/* Detailed source-code truncated for AI context efficiency. */

Examples Extracting Receipts (`docs/examples/extracting_receipts.md`)

title: Receipt Data Extraction with GPT-4 Vision - Expense Tracking description: Extract and validate receipt data from images using GPT-4 Vision and Instructor. Automate expense tracking with structured receipt parsing.

Extracting Receipt Data using GPT-4 and Python

This post demonstrates how to use Python's Pydantic library and OpenAI's GPT-4 model to extract receipt data from images and validate the total amount. This method is particularly useful for automating expense tracking and financial analysis tasks.

Defining the Item and Receipt Classes

First, we define two Pydantic models, Item and Receipt, to structure the extracted data. The Item class represents individual items on the receipt, with fields for name, price, and quantity. The Receipt class contains a list of Item objects and the total amount.

python
from pydantic import BaseModel


class Item(BaseModel):
    name: str
    price: float
    quantity: int


class Receipt(BaseModel):
    items: list[Item]
    total: float

Validating the Total Amount

To ensure the accuracy of the extracted data, we use Pydantic's model_validator decorator to define a custom validation function, check_total. This function calculates the sum of item prices and compares it to the extracted total amount. If there's a discrepancy, it raises a ValueError.

python
from pydantic import model_validator


@model_validator(mode="after")
def check_total(self):
    items = self.items
    total = self.total
    calculated_total = sum(item.price * item.quantity for item in items)
    if calculated_total != total:
        raise ValueError(
            f"Total {total} does not match the sum of item prices {calculated_total}"
        )
    return self

Extracting Receipt Data from Images

The extract_receipt function uses OpenAI's GPT-4 model to process an image URL and extract receipt data. We utilize the instructor library to configure the OpenAI client for this purpose.

python
import instructor

# <%hide%>
from pydantic import BaseModel, model_validator


class Item(BaseModel):
    name: str
    price: float
    quantity: int


class Receipt(BaseModel):
    items: list[Item]
    total: float

    @model_validator(mode="after")
    def check_total(cls, values: "Receipt"):
        items = values.items
        total = values.total
        calculated_total = sum(item.price * item.quantity for item in items)
        if calculated_total != total:
            raise ValueError(
                f"Total {total} does not match the sum of item prices {calculated_total}"
            )
        return values


# <%hide%>

client = instructor.from_provider("openai/gpt-5-nano")


def extract(url: str) -> Receipt:
    return client.create(
        model="gpt-5.4-mini",
        max_tokens=4000,
        response_model=Receipt,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image_url",
                        "image_url": {"url": url},
                    },
                    {
                        "type": "text",
                        "text": "Analyze the image and return the items in the receipt and the total amount.",
                    },
                ],
            }
        ],
    )

Practical Examples

In these examples, we apply the method to extract receipt data from two different images. The custom validation function ensures that the extracted total amount matches the sum of item prices.

text
/* Detailed source-code truncated for AI context efficiency. */

By combining the power of GPT-4 and Python's Pydantic library, we can accurately extract and validate receipt data from images, streamlining expense tracking and financial analysis tasks.

Examples Extracting Tables (`docs/examples/extracting_tables.md`)

title: Extracting Tables from Images using GPT-Vision description: Learn how to use Python and GPT-Vision to extract and convert tables from images into markdown for data analysis.

See Also

Extracting Tables using GPT-Vision

This post demonstrates how to use Python's type annotations and OpenAI's new vision model to extract tables from images and convert them into markdown format. This method is particularly useful for data analysis and automation tasks.

The full code is available on GitHub

Building the Custom Type for Markdown Tables

First, we define a custom type, MarkdownDataFrame, to handle pandas DataFrames formatted in markdown. This type uses Python's Annotated and InstanceOf types, along with decorators BeforeValidator and PlainSerializer, to process and serialize the data.

python
from io import StringIO
from typing import Annotated, Any
from pydantic import BeforeValidator, PlainSerializer, InstanceOf, WithJsonSchema
import pandas as pd


def md_to_df(data: Any) -> Any:
    # Convert markdown to DataFrame
    if isinstance(data, str):
        return (
            pd.read_csv(
                StringIO(data),  # Process data
                sep="|",
                index_col=1,
            )
            .dropna(axis=1, how="all")
            .iloc[1:]
            .applymap(lambda x: x.strip())
        )
    return data


MarkdownDataFrame = Annotated[
    InstanceOf[pd.DataFrame],
    BeforeValidator(md_to_df),
    PlainSerializer(lambda df: df.to_markdown()),
    WithJsonSchema(
        {
            "type": "string",
            "description": "The markdown representation of the table, each one should be tidy, do not try to join tables that should be separate",
        }
    ),
]

Defining the Table Class

The Table class is essential for organizing the extracted data. It includes a caption and a dataframe, processed as a markdown table. Since most of the complexity is handled by the MarkdownDataFrame type, the Table class is straightforward!

python
from pydantic import BaseModel

# <%hide%>
from io import StringIO
from typing import Annotated, Any
from pydantic import BeforeValidator, PlainSerializer, InstanceOf, WithJsonSchema
import pandas as pd


def md_to_df(data: Any) -> Any:
    # Convert markdown to DataFrame
    if isinstance(data, str):
        return (
            pd.read_csv(
                StringIO(data),  # Process data
                sep="|",
                index_col=1,
            )
            .dropna(axis=1, how="all")
            .iloc[1:]
            .applymap(lambda x: x.strip())
        )
    return data


MarkdownDataFrame = Annotated[
    InstanceOf[pd.DataFrame],
    BeforeValidator(md_to_df),
    PlainSerializer(lambda df: df.to_markdown()),
    WithJsonSchema(
        {
            "type": "string",
            "description": "The markdown representation of the table, each one should be tidy, do not try to join tables that should be separate",
        }
    ),
]
# <%hide%>


class Table(BaseModel):
    caption: str
    dataframe: MarkdownDataFrame

Extracting Tables from Images

The extract_table function uses OpenAI's vision model to process an image URL and extract tables in markdown format. We utilize the instructor library to patch the OpenAI client for this purpose.

text
/* Detailed source-code truncated for AI context efficiency. */

Practical Example

In this example, we apply the method to extract data from an image showing the top grossing apps in Ireland for October 2023.

text
/* Detailed source-code truncated for AI context efficiency. */

??? Note "Expand to see the output"

text
### Top 10 Grossing Apps in October 2023 (Ireland) for Android Platforms

| Rank | App Name                         | Category           |
|------|----------------------------------|--------------------|
| 1    | Google One                       | Productivity       |
| 2    | Disney+                          | Entertainment      |
| 3    | TikTok - Videos, Music & LIVE    | Entertainment      |
| 4    | Candy Crush Saga                 | Games              |
| 5    | Tinder: Dating, Chat & Friends   | Social networking  |
| 6    | Coin Master                      | Games              |
| 7    | Roblox                           | Games              |
| 8    | Bumble - Dating & Make Friends   | Dating             |
| 9    | Royal Match                      | Games              |
| 10   | Spotify: Music and Podcasts      | Music & Audio      |

### Top 10 Grossing Apps in October 2023 (Ireland) for iOS Platforms

| Rank | App Name                         | Category           |
|------|----------------------------------|--------------------|
| 1    | Tinder: Dating, Chat & Friends   | Social networking  |
| 2    | Disney+                          | Entertainment      |
| 3    | YouTube: Watch, Listen, Stream   | Entertainment      |
| 4    | Audible: Audio Entertainment     | Entertainment      |
| 5    | Candy Crush Saga                 | Games              |
| 6    | TikTok - Videos, Music & LIVE    | Entertainment      |
| 7    | Bumble - Dating & Make Friends   | Dating             |
| 8    | Roblox                           | Games              |
| 9    | LinkedIn: Job Search & News      | Business           |
| 10   | Duolingo - Language Lessons      | Education          |

Examples Groq (`docs/examples/groq.md`)

title: Groq AI Integration - Fast Structured Outputs description: Use Groq AI with Instructor for fast structured outputs. Leverage Groq's high-speed inference for real-time structured data extraction.

Structured Outputs using Groq

Instead of using openai or antrophic you can now also use groq for inference by using from_groq.

The examples are using mixtral-8x7b model.

GroqCloud API

To use groq you need to obtain a groq API key.
Goto groqcloud and login. Select API Keys from the left menu and then select Create API key to create a new key.

Use example

Some pip packages need to be installed to use the example:

text
pip install instructor groq pydantic openai anthropic

You need to export the groq API key:

text
export GROQ_API_KEY=<your-api-key>

An example:

python
from pydantic import BaseModel, Field
from typing import List
import instructor


class Character(BaseModel):
    name: str
    fact: List[str] = Field(..., description="A list of facts about the subject")


# Use from_provider for simplified setup
client = instructor.from_provider("groq/mixtral-8x7b-32768", mode=instructor.Mode.TOOLS)

resp = client.create(
    model="mixtral-8x7b-32768",
    messages=[
        {
            "role": "user",
            "content": "Tell me about the company Tesla",
        }
    ],
    response_model=Character,
)
print(resp.model_dump_json(indent=2))
"""
{
  "name": "Tesla",
  "fact": [
    "electric vehicle manufacturer",
    "solar panel producer",
    "based in Palo Alto, California",
    "founded in 2003 by Elon Musk"
  ]
}
"""

You can find another example called groq_example2.py under examples/groq of this repository.

Examples Image To Ad Copy (`docs/examples/image_to_ad_copy.md`)

title: Automatically Generate Advertising Copy from Product Images Using GPT-4 Vision description: Learn how to use GPT-4 Vision API to create engaging advertising copy from product images, ideal for e-commerce and marketing teams.

Use Vision API to detect products and generate advertising copy

This post demonstrates how to use GPT-4 Vision API and the Chat API to automatically generate advertising copy from product images. This method can be useful for marketing and advertising teams, as well as for e-commerce platforms.

The full code is available on GitHub.

Building the models

Product

For the Product model, we define a class that represents a product extracted from an image and store the name, key features, and description. The product attributes are dynamically determined based on the content of the image.

Note that it is easy to add Validators and other Pydantic features to the model to ensure that the data is valid and consistent.

python
from pydantic import BaseModel, Field
from typing import List, Optional


class Product(BaseModel):
    """
    Represents a product extracted from an image using AI.

    The product attributes are dynamically determined based on the content
    of the image and the AI's interpretation. This class serves as a structured
    representation of the identified product characteristics.
    """

    name: str = Field(
        description="A generic name for the product.", example="Headphones"
    )
    key_features: Optional[List[str]] = Field(
        description="A list of key features of the product that stand out.",
        default=None,
    )
        description="A description of the product.",
        default=None,
    )

    # Can be customized and automatically generated
    def generate_prompt(self):
        prompt = f"Product: {self.name}\n"
        if self.description:
            prompt += f"Description: {self.description}\n"
        if self.key_features:
            prompt += f"Key Features: {', '.join(self.key_features)}\n"
        return prompt

Identified Product

We also define a class that represents a list of products identified in the images. We also add an error flag and message to indicate if there was an error in the processing of the image.

text
/* Detailed source-code truncated for AI context efficiency. */

Advertising Copy

Finally, the AdCopy models stores the output in a structured format with a headline and the text.

python
from pydantic import BaseModel, Field


class AdCopy(BaseModel):
    """
    Represents a generated ad copy.
    """

    headline: str = Field(
        description="A short, catchy, and memorable headline for the given product. The headline should invoke curiosity and interest in the product.",
    )
    ad_copy: str = Field(
        description="A long-form advertisement copy for the given product. This will be used in campaigns to promote the product with a persuasive message and a call-to-action with the objective of driving sales.",
    )
    name: str = Field(description="The name of the product being advertised.")

Calling the API

Product Detection

The read_images function uses OpenAI's vision model to process a list of image URLs and identify products in each of them. We utilize the instructor library to patch the OpenAI client for this purpose.

text
/* Detailed source-code truncated for AI context efficiency. */

This gives us a list of products identified in all the images.

Generate advertising copy

Then, we can use the generate_ad_copy function to generate advertising copy for each of the products identified in the images.

Two clients are defined for the two different models. This is because the gpt-5.4-mini model is not compatible with the gpt-5.4-mini model in terms of their response format.

text
/* Detailed source-code truncated for AI context efficiency. */

Putting it all together

Finally, we can put it all together in a single function that takes a list of image URLs and generates advertising copy for the products identified in the images. Please refer to the full code for the complete implementation.

Input file

The input file is currently a list of image URLs, but this trivial to change to any required format.

plaintext
https://contents.mediadecathlon.com/p1279823/9a1c59ad97a4084a346c014740ae4d3ff860ea70b485ee65f34017ff5e9ae5f7/recreational-ice-skates-fit-50-black.jpg?format=auto
https://contents.mediadecathlon.com/p1279822/a730505231dbd6747c14ee93e8f89e824d3fa2a5b885ec26de8d7feb5626638a/recreational-ice-skates-fit-50-black.jpg?format=auto
https://contents.mediadecathlon.com/p2329893/1ed75517602a5e00245b89ab6a1c6be6d8968a5a227c932b10599f857f3ed4cd/mens-hiking-leather-boots-sh-100-x-warm.jpg?format=auto
https://contents.mediadecathlon.com/p2047870/8712c55568dd9928c83b19c6a4067bf161811a469433dc89244f0ff96a50e3e9/men-s-winter-hiking-boots-sh-100-x-warm-grey.jpg?format=auto

??? Note "Expand to see the output"

text
```

/* Detailed source-code truncated for AI context efficiency. */

text
---

### Examples Knowledge Graph (`docs/examples/knowledge_graph.md`)

---
title: 'Visualizing Knowledge Graphs: A Guide to Complex Topics'

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#openai Browse openai catalog → #openai-function-calli Browse openai-function-calli catalog → #openai-functions Browse openai-functions catalog → #pydantic-v2 Browse pydantic-v2 catalog → #python Browse python catalog → #validation Browse validation catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →