txtai Documentation

Api/Cluster Copy Section

Distributed embeddings clusters

The API supports combining multiple API instances into a single logical embeddings index. An example configuration is shown below.

cluster:
    shards:
        - http://127.0.0.1:8002
        - http://127.0.0.1:8003

This configuration aggregates the API instances above as index shards. Data is evenly split among each of the shards at index time. Queries are run in parallel against each shard and the results are joined together. This method allows horizontal scaling and supports very large index clusters.

This method is only recommended for data sets in the 1 billion+ records. The ANN libraries can easily support smaller data sizes and this method is not worth the additional complexity. At this time, new shards can not be added after building the initial index.

See the link below for a detailed example covering distributed embeddings clusters.

Api/Configuration Copy Section

Configuration

Configuration is set through YAML. In most cases, YAML keys map to fields names in Python. The example in the previous section gave a full-featured example covering a wide array of configuration options.

Each section below describes the available configuration settings.

Embeddings Copy Chapter

The configuration parser expects a top level embeddings key to be present in the YAML. All embeddings configuration is supported.

The following example defines an embeddings index.

path: index path
writable: true

embeddings:
  path: vector model
  content: true

Three top level settings are available to control where indexes are saved and if an index is a read-only index.

path Copy Section

path: string

Path to save and load the embeddings index. Each API instance can only access a single index at a time.

writable Copy Section

writable: boolean

Sets this embeddings index as writable (true) or read-only (false). This allows serving a read-only index. Defaults to read-only.

reindex Copy Section

reindex: boolean

Enables re-indexing for this embeddings index (defaults to disabled).

Re-indexing accepts new configuration and should only be enabled when the configuration comes from trusted sources.

cloud Copy Section

Cloud storage settings can be set under a cloud top level configuration group.

Agent Copy Chapter

Agents are defined under a top level agent key. Each key under the agent key is the name of the agent. Constructor parameters can be passed under this key.

The following example defines an agent.

agent:
    researcher:
        tools:
            - websearch

llm:
    path: Qwen/Qwen3-4B-Instruct-2507

Pipeline Copy Chapter

Pipelines are loaded as top level configuration parameters. Pipeline names are automatically detected in the YAML configuration and created upon startup. All pipelines are supported.

The following example defines a series of pipelines. Note that entries below are the lower-case names of the pipeline class.

caption:

extractor:
  path: model path

labels:

summary:

tabular:

translation:

Under each pipeline name, configuration settings for the pipeline can be set.

Workflow Copy Chapter

Workflows are defined under a top level workflow key. Each key under the workflow key is the name of the workflow. Under that is a tasks key with each task definition.

The following example defines a workflow.

workflow:
  sumtranslate:
    tasks:
        - action: summary
        - action: translation

schedule Copy Section

Schedules a workflow using a cron expression.

workflow:
  index:
    schedule:
      cron: 0/10 * * * * *
      elements: ["api params"] 
    tasks:
      - task: service
        url: api url
      - action: index

tasks Copy Section

tasks: list

Expects a list of workflow tasks. Each element defines a single workflow task. All task configuration is supported.

A shorthand syntax for creating tasks is supported. This syntax will automatically map task strings to an action:value pair.

Example below.

workflow:
  index:
    tasks:
      - action1
      - action2

Each task element supports the following additional arguments.

action

action: string|list

Both single and multi-action tasks are supported.

The action parameter works slightly different when passed via configuration. The parameter(s) needs to be converted into callable method(s). If action is a pipeline that has been defined in the current configuration, it will use that pipeline as the action.

There are three special action names index, upsert and search. If index or upsert are used as the action, the task will collect workflow data elements and load them into defined the embeddings index. If search is used, the task will execute embeddings queries for each input data element.

Otherwise, the action must be a path to a callable object or function. The configuration parser will resolve the function name and use that as the task action.

task

task: string

Optionally sets the type of task to create. For example, this could be a file task or a retrieve task. If this is not specified, a generic task is created. The list of workflow tasks can be found here.

args

args: list

Optional list of static arguments to pass to the workflow task. These are combined with workflow data to pass to each __call__.

Api/Customization Copy Section

Customization

The txtai API has a number of features out of the box that are designed to help get started quickly. API services can also be augmented with custom code and functionality. The two main ways to do this are with extensions and dependencies.

Extensions add a custom endpoint. Dependencies add middleware that executes with each request. See the sections below for more.

Extensions Copy Chapter

While the API is extremely flexible and complex logic can be executed through YAML-driven workflows, some may prefer to create an endpoint in Python. API extensions define custom Python endpoints that interact with txtai applications.

See the link below for a detailed example.

Dependencies Copy Chapter

txtai has a default API token authorization method that works well in many cases. Dependencies can also add custom logic with each request. This could be an additional authorization step and/or an authentication method.

See the link below for a detailed example.

Api/Index Copy Section

API

txtai has a full-featured API, backed by FastAPI, that can optionally be enabled for any txtai process. All functionality found in txtai can be accessed via the API.

The following is an example configuration and startup script for the API.

Note: This configuration file enables all functionality. For memory-bound systems, splitting pipelines into multiple instances is a best practice.

# Index file path
path: /tmp/index

# Allow indexing of documents
writable: True

# Enbeddings index
embeddings:
  path: sentence-transformers/nli-mpnet-base-v2

# Extractive QA
extractor:
  path: distilbert-base-cased-distilled-squad

# Zero-shot labeling
labels:

# Similarity
similarity:

# Text segmentation
segmentation:
    sentences: true

# Text summarization
summary:

# Text extraction
textractor:
    paragraphs: true
    minlength: 100
    join: true

# Transcribe audio to text
transcription:

# Translate text between languages
translation:

# Workflow definitions
workflow:
    sumfrench:
        tasks:
            - action: textractor
              task: url
            - action: summary
            - action: translation
              args: ["fr"]
    sumspanish:
        tasks:
            - action: textractor
              task: url
            - action: summary
            - action: translation
              args: ["es"]

Assuming this YAML content is stored in a file named config.yml, the following command starts the API process.

CONFIG=config.yml uvicorn "txtai.api:app"

Uvicorn is a full-featured production-ready server. See the Uvicorn deployment guide for more on configuration options.

Connect to API Copy Chapter

The default port for the API is 8000. See the uvicorn link above to change this.

txtai has a number of language bindings which abstract the API (see links below). Alternatively, code can be written to connect directly to the API. Documentation for a live running instance can be found at the /docs url (i.e. http://localhost:8000/docs). The following example runs a workflow using cURL.

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"sumfrench", "elements": ["https://github.com/neuml/txtai"]}'

Local instance Copy Chapter

A local instance can be instantiated. In this case, a txtai application runs internally, without any network connections, providing the same consolidated functionality. This enables running txtai in Python with configuration.

The configuration above can be run in Python with:

from txtai import Application

# Load and run workflow
app = Application(config.yml)
app.workflow("sumfrench", ["https://github.com/neuml/txtai"])

See this link for a full list of methods.

Run with containers Copy Chapter

The API can be containerized and run. This will bring up an API instance without having to install Python, txtai or any dependencies on your machine!

See this section for more information.

Supported language bindings Copy Chapter

The following programming languages have bindings with the txtai API:

• Python
• JavaScript
• Java
• Rust
• Go

The API also supports hosting OpenAI-compatible and Model Context Protocol (MCP) endpoints.

See the links below for detailed examples covering the API.

Api/Mcp Copy Section

Model Context Protocol

The Model Context Protocol (MCP) is an open standard that enables developers to build secure, two-way connections between their data sources and AI-powered tools.

The API can be configured to handle MCP requests. All enabled endpoints set in the API configuration are automatically added as MCP tools.

mcp: boolean|dict

When mcp is a boolean, default arguments are used. When mcp is a dictionary it supports the following options.

mcp:
  clientargs: http client options (dict)
  mcpargs: FastApiMCP options (dict)

See the following links for details on the options.

• HTTP Client Arguments
• FastApiMCP Arguments

Once this configuration option is added, a new route is added to the application /mcp.

The Model Context Protocol Inspector tool is a quick way to explore how the MCP tools are exported through this interface.

Run the following and go to the local URL specified.

npx @modelcontextprotocol/inspector node build/index.js

Enter http://localhost:8000/mcp to see the full list of tools available.

Api/Methods Copy Section

Methods

::: txtai.api.API
options:
inherited_members: true
filters:
- "!del"
- "!flows"
- "!function"
- "!indexes"
- "!limit"
- "!pipes"
- "!read"
- "!resolve"
- "!weights"

Api/Openai Copy Section

OpenAI-compatible API

The API can be configured to serve an OpenAI-compatible API as shown below.

openai: True

See the link below for a detailed example.

Api/Security Copy Section

Security

The default implementation of an API service runs via HTTP and is fully open. If the service is being run as a prototype on an internal network, that may be fine. In most scenarios, the connection should at least be encrypted. Authorization is another built-in feature that requires a valid API token with each request. See below for more.

HTTPS Copy Chapter

The default API service command starts a Uvicorn server as a HTTP service on port 8000. To run a HTTPS service, consider the following options.

• TLS Proxy Server. Recommended choice. With this configuration, the txtai API service runs as a HTTP service only accessible on the localhost/local network. The proxy server handles all encryption and redirects requests to local services. See this example configuration for more.

• Uvicorn SSL Certificate. Another option is setting the SSL certificate on the Uvicorn service. This works in simple situations but gets complex when hosting multiple txtai or other related services.

Authorization Copy Chapter

Authorization requires a valid API token with each API request. This token is sent as a HTTP Authorization header.

Server

CONFIG=config.yml TOKEN=<sha256 encoded token> uvicorn "txtai.api:app"

Client

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \ 
  -d '{"name":"sumfrench", "elements": ["https://github.com/neuml/txtai"]}'

It's important to note that HTTPS must be enabled using one of the methods mentioned above. Otherwise, tokens will be exchanged as clear text.

Authentication and Authorization can be fully customized. See the dependencies section for more.

Safe Open Copy Chapter

Many components in txtai work with local files and have no restrictions on what files and urls can be accessed. The textractor pipeline and retrieve tasks both have a built-in option called safeopen which is enabled by default when running the API and applications. It enforces the following rules:

• Local files must be in the safeopen directory (defaults to temp dir)
• URLs must be public URLs (also has checks for redirects and DNS rebind attempts)

The following workflow only allows accessing files stored in a specific directory along with public urls.

workflow:
  textract:
    tasks:
      - action: textractor

A number of of pipelines support reading local files. For these it's recommended to wrap those pipelines as workflows with a retrieve task. Note that pipelines can be created in-line and don't need to be exposed via the API.

workflow:
  caption:
    tasks:
      - task: retrieve
        action: caption

Additionally, internal pipelines can be created but not exposed via the API.

routes:
  caption: False

caption:

workflow:
  caption:
    tasks:
      - task: retrieve
        action: caption

Agent/Configuration Copy Section

Configuration

An agent takes two main arguments, an LLM and a list of tools.

The txtai agent framework is built with smolagents. Additional options can be passed in the Agent constructor.

from datetime import datetime

from txtai import Agent

wikipedia = {
    "name": "wikipedia",
    "description": "Searches a Wikipedia database",
    "provider": "huggingface-hub",
    "container": "neuml/txtai-wikipedia"
}

arxiv = {
    "name": "arxiv",
    "description": "Searches a database of scientific papers",
    "provider": "huggingface-hub",
    "container": "neuml/txtai-arxiv"
}

def today() -> str:
    """
    Gets the current date and time

    Returns:
        current date and time
    """

    return datetime.today().isoformat()

agent = Agent(
    model="Qwen/Qwen3-4B-Instruct-2507",
    tools=[today, wikipedia, arxiv, "websearch"],
)

model Copy Chapter

model: string|llm instance

LLM model path or LLM pipeline instance. The llm parameter is also supported for backwards compatibility.

See the LLM pipeline for more information.

tools Copy Chapter

tools: list

List of tools to supply to the agent. Supports the following configurations.

function Copy Section

A function tool takes the following dictionary fields.

A function or callable method can also be directly supplied in the tools list. In this case, the fields are inferred from the method documentation.

embeddings Copy Section

Embeddings indexes have built-in support. Provide the following dictionary configuration to add an embeddings index as a tool.

tool Copy Section

The following shortcut strings load tools directly. Passing a Tool instance is also supported.

instructions Copy Chapter

instructions: string|path

Supports loading an agents.md file. Can be provided directly as a string or as a path to a file.

Read more about agents.md here

template Copy Chapter

template: string

Customize the prompt template used by this agent. Supports Jinja templates. Uses a default template when this parameter is not provided.
Must include {{ text }} and {{ memory }} placeholders.

memory Copy Chapter

memory: int

Keeps a rolling window of memory inputs and outputs. These are added to future prompts and serve as "agent memory".

Supports storing memory by session to enable multiple conversation threads. Defaults to shared memory when not set. See the method documentation for more information.

method Copy Chapter

method: code|tool

Sets the agent method. Supports either a code or tool (default) calling agent. A code agent generates Python code and executes that. A tool calling agent generates JSON blocks and calls the agents within those blocks.

Additional options can be directly passed. See CodeAgent or ToolCallingAgent for a list of parameters.

Read more here.

Agent/Index Copy Section

Agent

An agent automatically creates workflows to answer multi-faceted user requests. Agents iteratively prompt and/or interface with tools to
step through a process and ultimately come to an answer for a request.

Agent prompting with agents.md and skill.md are also supported. Read the configuration for more on how to setup those up.

Agents excel at complex tasks where multiple tools and/or methods are required. They incorporate a level of randomness similar to different
people working on the same task. When the request is simple and/or there is a rule-based process, other methods such as RAG and Workflows
should be explored.

The following code snippet defines a basic agent.

from datetime import datetime

from txtai import Agent

wikipedia = {
    "name": "wikipedia",
    "description": "Searches a Wikipedia database",
    "provider": "huggingface-hub",
    "container": "neuml/txtai-wikipedia"
}

arxiv = {
    "name": "arxiv",
    "description": "Searches a database of scientific papers",
    "provider": "huggingface-hub",
    "container": "neuml/txtai-arxiv"
}

def today() -> str:
    """
    Gets the current date and time

    Returns:
        current date and time
    """

    return datetime.today().isoformat()

agent = Agent(
    model="Qwen/Qwen3-4B-Instruct-2507",
    tools=[today, wikipedia, arxiv, "websearch"],
    max_steps=10,
)

The agent above has access to two embeddings databases (Wikipedia and ArXiv) and the web. Given the user's input request, the agent decides the best tool to solve the task.

Example Copy Chapter

The first example will solve a problem with multiple data points. See below.

agent("Which city has the highest population, Boston or New York?")

This requires looking up the population of each city before knowing how to answer the question. Multiple search requests are run to generate a final answer.

Agentic RAG Copy Chapter

Standard retrieval augmented generation (RAG) runs a single vector search to obtain a context and builds a prompt with the context + input question. Agentic RAG is a more complex process that goes through multiple iterations. It can also utilize multiple databases to come to a final conclusion.

The example below aggregates information from multiple sources and builds a report on a topic.

researcher = """
You're an expert researcher looking to write a paper on {topic}.
Search for websites, scientific papers and Wikipedia related to the topic.
Write a report with summaries and references (with hyperlinks).
Write the text as Markdown.
"""

agent(researcher.format(topic="alien life"))

Agent Teams Copy Chapter

Agents can also be tools. This enables the concept of building "Agent Teams" to solve problems. The previous example can be rewritten as a list of agents.

from txtai import Agent, LLM

llm = LLM("Qwen/Qwen3-4B-Instruct-2507")

websearcher = Agent(
    model=llm,
    tools=["websearch"],
)

wikiman = Agent(
    model=llm,
    tools=[{
        "name": "wikipedia",
        "description": "Searches a Wikipedia database",
        "provider": "huggingface-hub",
        "container": "neuml/txtai-wikipedia"
    }],
)

researcher = Agent(
    model=llm,
    tools=[{
        "name": "arxiv",
        "description": "Searches a database of scientific papers",
        "provider": "huggingface-hub",
        "container": "neuml/txtai-arxiv"
    }],
)

agent = Agent(
    model=llm,
    tools=[{
        "name": "websearcher",
        "description": "I run web searches, there is no answer a web search can't solve!",
        "target": websearcher
    }, {
        "name": "wikiman",
        "description": "Wikipedia has all the answers, I search Wikipedia and answer questions",
        "target": wikiman
    }, {
        "name": "researcher",
        "description": "I'm a science guy. I search arXiv to get all my answers.",
        "target": researcher
    }],
    max_steps=10
)

This provides another level of intelligence to the process. Instead of just a single tool execution, each agent-tool combination has it's own reasoning engine.

agent("""
Research fundamental concepts about Signal Processing and build a comprehensive report.
Write the output in Markdown.
""")

More examples

Check out this Agent Quickstart Example. Additional examples are listed below.

Embeddings/Configuration/Ann Copy Section

ANN

Approximate Nearest Neighbor (ANN) index configuration for storing vector embeddings.

backend Copy Chapter

backend: faiss|hnsw|annoy|ggml|milvus|numpy|torch|turbovec|zvec|pgvector|sqlite|custom

Sets the ANN backend. Defaults to faiss. Additional backends are available via the ann extras package. Set custom backends via setting this parameter to the fully resolvable class string.

Backend-specific settings are set with a corresponding configuration object having the same name as the backend (i.e. annoy, faiss, or hnsw). These are optional and set to defaults if omitted.

faiss Copy Section

faiss:
    components: comma separated list of components - defaults to "IDMap,Flat" for small
                indices and "IVFx,Flat" for larger indexes where
                x = min(4 * sqrt(embeddings count), embeddings count / 39)
                automatically calculates number of IVF cells when omitted (supports "IVF,Flat")
    nprobe: search probe setting (int) - defaults to x/16 (as defined above)
            for larger indexes
    nflip: same as nprobe - only used with binary hash indexes
    quantize: store vectors with x-bit precision vs 32-bit (boolean|int)
              true sets 8-bit precision, false disables, int sets specified
              precision
    mmap: load as on-disk index (boolean) - trade query response time for a
          smaller RAM footprint, defaults to false
    sample: percent of data to use for model training (0.0 - 1.0)
            reduces indexing time for larger (>1M+ row) indexes, defaults to 1.0

Faiss supports both floating point and binary indexes. Floating point indexes are the default. Binary indexes are used when indexing scalar-quantized datasets.

See the following Faiss documentation links for more information.

• Guidelines for choosing an index
• Index configuration summary
• Index Factory
• Binary Indexes
• Search Tuning

Note: For macOS users, an existing bug in an upstream package restricts the number of processing threads to 1. This limitation is managed internally to prevent system crashes.

hnsw Copy Section

hnsw:
    efconstruction:  ef_construction param for init_index (int) - defaults to 200
    m: M param for init_index (int) - defaults to 16
    randomseed: random-seed param for init_index (int) - defaults to 100
    efsearch: ef search param (int) - defaults to None and not set

See Hnswlib documentation for more information on these parameters.

annoy Copy Section

annoy:
    ntrees: number of trees (int) - defaults to 10
    searchk: search_k search setting (int) - defaults to -1

See Annoy documentation for more information on these parameters. Note that annoy indexes can not be modified after creation, upserts/deletes and other modifications are not supported.

ggml Copy Section

ggml:
    gpu: enable GPU - defaults to True
    quantize: sets the tensor quantization - defaults to F32
    querysize: query buffer size - defaults to 64

The GGML backend is a k-nearest neighbors backend. It stores tensors using GGML and GGUF. It supports GPU-enabled operations and supports quantization. GGML is the framework used by llama.cpp.

See this for a list of quantization types.

milvus Copy Section

milvus:
    m: number of HNSW links per element (int) - defaults to 50

numpy Copy Section

numpy:
    safetensors: stores vectors using the safetensors format
                 defaults to NumPy array storage

The NumPy backend is a k-nearest neighbors backend. It's designed for simplicity and works well with smaller datasets that fit into memory.

torch Copy Section

torch:
    safetensors: stores vectors using the safetensors format - defaults
                 to NumPy array storage if quantization is disabled
    quantize:
        type: quantization type (fp4, nf4, int8)
        blocksize: quantization block size parameter

The Torch backend is a k-nearest neighbors backend like NumPy. It supports GPU-enabled operations. It also has support for quantization which enables fitting larger arrays into GPU memory.

When quantization is enabled, vectors are always stored in safetensors. Note that macOS support for quantization is limited.

turbovec Copy Section

turbovec:
    bitwidth: number of bits to store each vector dimension as. Supports 2, 3 or 4.

The turbovec backend is a k-nearest neighbors backend powered by the TurboQuant algorithm.

zvec Copy Section

zvec:
    m: number of HNSW links per element (int) - defaults to 50

The zvec backend is an embedded, path-based vector index.

pgvector Copy Section

pgvector:
    url: database url connection string, alternatively can be set via
         ANN_URL environment variable
    schema: database schema to store vectors - defaults to being
            determined by the database
    table: database table to store vectors - defaults to `vectors`
    precision: vector float precision (half or full) - defaults to `full`
    efconstruction:  ef_construction param (int) - defaults to 200
    m: M param for init_index (int) - defaults to 16

The pgvector backend stores embeddings in a Postgres database. See the pgvector documentation for more information on these parameters. See the SQLAlchemy documentation for more information on how to construct url connection strings.

sqlite Copy Section

sqlite:
    quantize: store vectors with x-bit precision vs 32-bit (boolean|int)
              true sets 8-bit precision, false disables, int sets specified
              precision
    table: database table to store vectors - defaults to `vectors`

The SQLite backend stores embeddings in a SQLite database using sqlite-vec. This backend supports 1-bit and 8-bit quantization at the storage level.

See this note on how to run this ANN on MacOS.

Embeddings/Configuration/Cloud Copy Section

Cloud

The following describes parameters used to sync indexes with cloud storage. Cloud object storage, the Hugging Face Hub and custom providers are all supported.

Parameters are set via the embeddings.load and embeddings.save methods.

provider Copy Chapter

provider: string

Cloud provider. Can be one of the following:

• Cloud object storage. Set to one of these providers. Use the text shown in the Provider Constant column as lower case.

• Hugging Face Hub. Set to huggingface-hub.

• Custom providers. Set to the full class path of the custom provider.

container Copy Chapter

container: string

Container/bucket/directory/repository name. Embeddings will be stored in the container with the filename specified by the path configuration.

Cloud object storage configuration Copy Chapter

In addition to the above common configuration, the cloud object storage provider has the following additional configuration parameters. Note that some cloud providers do not need any of these parameters and can use implicit authentication with service accounts.

See the libcloud documentation for more information on these parameters.

key Copy Section

key: string

Provider-specific access key. Can also be set via ACCESS_KEY environment variable. Ensure the configuration file is secured if added to the file. When using implicit authentication, set this to a value such as 'using-implicit-auth'.

secret Copy Section

secret: string

Provider-specific access secret. Can also be set via ACCESS_SECRET environment variable. Ensure the configuration file is secured if added to the file. When using implicit authentication, this option is not required.

prefix Copy Section

prefix: string

Optional object prefix. Object storage doesn't have the concept of a directory but a prefix is similar. For example, a prefix could be base/dir. This helps with organizing data in an object storage bucket.

More can be found at the following links.

• Organizing objects using prefixes
• libcloud container method documentation

host Copy Section

host: string

Optional server host name. Set when using a local cloud storage server.

port Copy Section

port: int

Optional server port. Set when using a local cloud storage server.

token Copy Section

token: string

Optional temporary session token

region Copy Section

region: string

Optional parameter to specify the storage region, provider-specific.

Hugging Face Hub configuration Copy Chapter

The huggingface-hub provider supports the following additional configuration parameters. More on these parameters can be found in the Hugging Face Hub's documentation.

revision Copy Section

revision: string

Optional Git revision id which can be a branch name, a tag, or a commit hash

cache Copy Section

cache: string

Path to the folder where cached files are stored

token Copy Section

token: string|boolean

Token to be used for the download. If set to True, the token will be read from the Hugging Face config folder.

Embeddings/Configuration/Database Copy Section

Database

Databases store metadata, text and binary content.

content Copy Chapter

content: boolean|sqlite|duckdb|client|url|custom

Enables content storage. When true, the default storage engine, sqlite will be used to save metadata.

Client-server connections are supported with either client or a full connection URL. When set to client, the CLIENT_URL environment variable must be set to the full connection URL. See the SQLAlchemy documentation for more information on how to construct connection strings for client-server databases.

Add custom storage engines via setting this parameter to the fully resolvable class string.

Content storage specific settings are set with a corresponding configuration object having the same name as the content storage engine (i.e. duckdb or sqlite). These are optional and set to defaults if omitted.

client Copy Section

schema:  default database schema for the session - defaults to being
         determined by the database

Additional settings for client-server databases. Also supported when the content=url.

sqlite Copy Section

sqlite:
    wal: enable write-ahead logging - allows concurrent read/write operations,
         defaults to false

Additional settings for SQLite.

objects Copy Chapter

objects: boolean|image|pickle

Enables object storage. Supports storing binary content. Requires content storage to also be enabled.

Object encoding options are:

• standard: Default encoder when boolean set. Encodes and decodes objects as byte arrays.
• image: Image encoder. Encodes and decodes objects as image objects.
• pickle: Pickle encoder. Encodes and decodes objects with the pickle module. Supports arbitrary objects.

functions Copy Chapter

functions: list

List of functions with user-defined SQL functions. Each list element must be one of the following:

• function
• callable object
• dict with fields for name, argcount, function and deterministic

An example can be found here.

expressions Copy Chapter

expressions: list

List of expression shortcuts. Each list element must be a dict with the following fields.

• name: name of the expression
• expression: SQL expression, defaults to name when empty
• index: if this expression should have a database index, defaults to False when not provided

The expression can be a json data column, sql function or anything that can be run as a SQL snippet.

query Copy Chapter

query:
    path: sets the path for the query model - this can be any model on the
          Hugging Face Model Hub or a local file path.
    prefix: text prefix to prepend to all inputs
    maxlength: maximum generated sequence length

Query translation model. Translates natural language queries to txtai compatible SQL statements.

Embeddings/Configuration/General Copy Section

General

General configuration options.

keyword Copy Chapter

keyword: boolean|string

Enables sparse keyword indexing for this embeddings.

When set to a boolean, this parameter creates a BM25 index for full text search. When set to a string, it expects a keyword method.

It also implicitly disables the defaults setting for vector search.

sparse Copy Chapter

sparse: boolean|path

Enables sparse vector indexing for this embeddings.

When set to True, this parameter creates a sparse vector index using the default sparse index model. When set to a string, it expects a local or Hugging Face model path.

It also implicitly disables the defaults setting for vector search.

dense Copy Chapter

dense: boolean|string

Alias for the vector model path. When set to True, the default transformers vector model is used.

hybrid Copy Chapter

hybrid: boolean

Enables hybrid (sparse + dense) indexing for this embeddings.

When enabled, this parameter creates a BM25 index for full text search. It has no effect on the defaults or path settings.

defaults Copy Chapter

defaults: boolean

Uses default vector model path when enabled (default setting is True) and path is not provided. See this link for an example.

indexes Copy Chapter

indexes: dict

Key value pairs defining subindexes for this embeddings. Each key is the index name and the value is the full configuration. This configuration can use any of the available configurations in a standard embeddings instance.

autoid Copy Chapter

format: int|uuid function

Sets the auto id generation method. When this is not set, an autogenerated numeric sequence is used. This also supports UUID generation functions. For example, setting this value to uuid4 will generate random UUIDs. Setting this to uuid5 will generate deterministic UUIDs for each input data row.

columns Copy Chapter

columns:
    text: name of the text column
    object: name of the object column
    store: limit json data fields to this list of columns

Sets the text and object column names. Defaults to text and object if not provided.

store sets a list of columns to store in the JSON data field. When this isn't provided, all columns are stored (default). When store is set to None, no JSON columns are stored. This is useful is a field is only needed at indexing time but not search time.

format Copy Chapter

format: json|pickle

Sets the configuration storage format. Defaults to json.

Note that pickle configuration is deprecated and will raise an error with the default settings. Its only kept for reading legacy indexes and requires setting the ALLOW_PICKLE environment variable. Only enable for local and/or trusted sources.

Embeddings/Configuration/Graph Copy Section

Graph

Enable graph storage via the graph parameter. This component requires the graph extras package.

When enabled, a graph network is built using the embeddings index. Graph nodes are synced with each embeddings index operation (index/upsert/delete). Graph edges are created using the embeddings index upon completion of each index/upsert/delete embeddings index call.

backend Copy Chapter

backend: networkx|rdbms|custom

Sets the graph backend. Defaults to networkx.

Add custom graph storage engines via setting this parameter to the fully resolvable class string.

The rdbms backend has the following additional settings.

rdbms Copy Section

url: database url connection string, alternatively can be set via the
     GRAPH_URL environment variable
schema: database schema to store graph - defaults to being
        determined by the database
nodes: table to store node data, defaults to `nodes`
edges: table to store edge data, defaults to `edges`

batchsize Copy Chapter

batchsize: int

Batch query size, used to query embeddings index - defaults to 256.

limit Copy Chapter

limit: int

Maximum number of results to return per embeddings query - defaults to 15.

minscore Copy Chapter

minscore: float

Minimum score required to consider embeddings query matches - defaults to 0.1.

approximate Copy Chapter

approximate: boolean

When true, queries only run for nodes without edges - defaults to true.

topics Copy Chapter

topics:
    algorithm: community detection algorithm (string), options are
               louvain (default), greedy, lpa
    level: controls number of topics (string), options are best (default) or first
    resolution: controls number of topics (int), larger values create more
                topics (int), defaults to 100
    labels: scoring index method used to build topic labels (string)
            options are bm25 (default), tfidf, sif
    terms: number of frequent terms to use for topic labels (int), defaults to 4
    stopwords: optional list of stop words to exclude from topic labels
    categories: optional list of categories used to group topics, allows
                granular topics with broad categories grouping topics

Enables topic modeling. Defaults are tuned so that in most cases these values don't need to be changed (except for categories). These parameters are available for advanced use cases where one wants full control over the community detection process.

copyattributes Copy Chapter

copyattributes: boolean|list

Copy these attributes from input dictionaries in the insert method. If this is set to True, all attributes are copied. Otherwise, only the
attributes specified in this list are copied to the graph as attributes.

Embeddings/Configuration/Index Copy Section

Configuration

The following describes available embeddings configuration. These parameters are set in the Embeddings constructor via either the config parameter or as keyword arguments.

Configuration is designed to be optional and set only when needed. Out of the box, sensible defaults are picked to get up and running fast. For example:

from txtai import Embeddings

embeddings = Embeddings()

Creates a new embeddings instance, using all-MiniLM-L6-v2 as the vector model, Faiss as the ANN index backend and content disabled.

from txtai import Embeddings

embeddings = Embeddings(content=True)

Is the same as above except it adds in SQLite for content storage.

The following sections link to all the available configuration options.

[ANN](./ann) Copy Chapter

The default vector index backend is Faiss.

[Cloud](./cloud) Copy Chapter

Embeddings databases can optionally be synced with cloud storage.

[Database](./database) Copy Chapter

Content storage is disabled by default. When enabled, SQLite is the default storage engine.

[General](./general) Copy Chapter

General configuration that doesn't fit elsewhere.

[Graph](./graph) Copy Chapter

An accomplying graph index can be created with an embeddings database. This enables topic modeling, path traversal and more. NetworkX is the default graph index.

[Scoring](./scoring) Copy Chapter

Sparse keyword indexing and word vectors term weighting.

[Vectors](./vectors) Copy Chapter

Vector search is enabled by converting text and other binary data into embeddings vectors. These vectors are then stored in an ANN index. The vector model is optional and a default model is used when not provided.

Embeddings/Configuration/Scoring Copy Section

Scoring

Enable scoring support via the scoring parameter.

This scoring instance can serve two purposes, depending on the settings.

One use case is building sparse/keyword indexes. This occurs when the terms parameter is set to True.

The other use case is with word vector term weighting. This feature has been available since the initial version but isn't quite as common anymore.

The following covers the available options.

method Copy Chapter

method: bm25|tfidf|sif|pgtext|sparse|custom

Sets the scoring method. Add custom scoring via setting this parameter to the fully resolvable class string.

pgtext Copy Section

schema: database schema to store keyword index - defaults to being
        determined by the database

Additional settings for Postgres full-text keyword indexes.

sparse Copy Section

path: sparse vector model path
vectormethod: vector embeddings method
vectornormalize: enable vector embeddings normalization (boolean)
gpu: boolean|int|string|device
normalize: enable score normalization (boolean|float|string|dict)
batch: Sets the transform batch size
encodebatch: Sets the encode batch size
vectors: additional model init args
encodeargs: additional encode() args
backend: ivfsparse|pgsparse

Sparse vector scoring options. The sparse scoring instance combines a sparse vector model with a sparse approximate nearest neighbor index (ANN). This method supports both vector normalization and score normalization.

Vector normalization normalizes all vectors to have a magnitude of 1. By extension, all generated scores will be 0 to 1.

Score normalization scales the output between 0 and 1. This setting supports:

• True for default scale normalization
• float normalize using this as the scale factor
• "bayes" for Bayesian normalization using dynamic candidate score statistics
• {method: "bayes", alpha: 1.0, beta: null} for Bayesian normalization with optional custom parameters

ivfsparse

ivfsparse:
  sample: percent of data to use for model training (0.0 - 1.0)
  nfeatures: top n features to use for model training (int)
  nlist: desired number of clusters (int)
  nprobe: search probe setting (int)
  minpoints: minimum number of points for a cluster (int)

Inverted file (IVF) index with flat vector file storage and sparse array support.

pgsparse

Sparse ANN backed by Postgres. Supports same options as the pgvector ANN.

terms Copy Chapter

terms: boolean|dict

Enables term frequency sparse arrays for a scoring instance. This is the backend for sparse keyword indexes.

Supports a dict with the parameters cachelimit and cutoff.

cachelimit is the maximum amount of resident memory in bytes to use during indexing before flushing to disk. This parameter is an int.

cutoff is used during search to determine what constitutes a common term. This parameter is a float, i.e. 0.1 for a cutoff of 10%.

When terms is set to True, default parameters are used for the cachelimit and cutoff. Normally, these defaults are sufficient.

normalize Copy Chapter

normalize: boolean|str|dict

Enables normalized scoring (ranging from 0 to 1). This setting supports:

• True for standard score normalization
• "bayes" | "bb25" for Bayesian normalization using dynamic candidate score statistics
• {method: "bayes", alpha: 1.0, beta: null} for Bayesian normalization with optional custom parameters

When standard normalization is enabled, statistics from the index are used to calculate normalized scores.
When Bayesian/BB25 normalization is enabled, it uses positive-score candidates, dynamic beta=median(scores), adaptive
alpha_eff=alpha/std(scores) and a sigmoid transform (likelihood-only variant with flat prior) to map scores to [0, 1].

Bayesian normalization references:

• https://github.com/instructkr/bb25
• https://github.com/cognica-io/bayesian-bm25

tokenizer Copy Chapter

tokenizer: dict

Set tokenization rules. Passes these arguments to the underlying Tokenization pipeline.

Embeddings/Configuration/Vectors Copy Section

Vectors

The following covers available vector model configuration options.

path Copy Chapter

path: string

Sets the path for a vectors model. When using a transformers/sentence-transformers model, this can be any model on the
Hugging Face Hub or a local file path. Otherwise, it must be a local file path to a word embeddings model.

method Copy Chapter

method: transformers|sentence-transformers|llama.cpp|litellm|model2vec|external|
        words

Embeddings method to use. If the method is not provided, it is inferred using the path.

sentence-transformers, llama.cpp, litellm, model2vec and words require the vectors extras package to be installed.

transformers Copy Section

Builds embeddings using a transformers model. While this can be any transformers model, it works best with
models trained to build embeddings.

mean, cls, last and late pooling are supported and automatically inferred from the model. The pooling method can be overwritten by changing the method
from transformers to meanpooling, clspooling, lastpooling or latepooling respectively.

Setting maxlength to True enables truncating inputs to the max_seq_length. Setting maxlength to an integer will truncate inputs to that value. When omitted (default), the maxlength will be set to either the model or tokenizer maxlength.

Supports loading ONNX models directly. See this section for more.

sentence-transformers Copy Section

Same as transformers but loads models with the sentence-transformers library.

llama.cpp Copy Section

Builds embeddings using a llama.cpp model. Supports both local and remote GGUF paths on the HF Hub.

litellm Copy Section

Builds embeddings using a LiteLLM model. See the LiteLLM documentation for the options available with LiteLLM models.

model2vec Copy Section

Builds embeddings using a Model2Vec model. Model2Vec is a knowledge-distilled version of a transformers model with static vectors.

words Copy Section

Builds embeddings using a word embeddings model and static vectors. While Transformers models are preferred in most cases, this method can be useful for low resource and historical languages where there isn't much linguistic data available.

pca

pca: int

Removes n principal components from generated embeddings. When enabled, a TruncatedSVD model is built to help with dimensionality reduction. After pooling of vectors creates a single embedding, this method is applied.

external Copy Section

Embeddings are created via an external model or API. Requires setting the transform parameter to a function that translates data into embeddings.

transform

transform: function

When method is external, this function transforms input content into embeddings. The input to this function is a list of data. This method must return either a numpy array, list of numpy arrays or list of float arrays.

Note that this method is deprecated and will raise an error with the default settings. Adding a custom Vectors implementation is a cleaner approach. The ALLOW_RESOLVE_TRANSFORM environment variable must be set to enable. Only enable for trusted and/or reviewed code.

gpu Copy Chapter

gpu: boolean|int|string|device

Set the target device. Supports true/false, device id, device string and torch device instance. This is automatically derived if omitted.

The sentence-transformers method supports encoding with multiple GPUs. This can be enabled by setting the gpu parameter to all.

batch Copy Chapter

batch: int

Sets the transform batch size. This parameter controls how input streams are chunked and vectorized.

encodebatch Copy Chapter

encodebatch: int

Sets the encode batch size. This parameter controls the underlying vector model batch size. This often corresponds to a GPU batch size, which controls GPU memory usage.

dimensionality Copy Chapter

dimensionality: int

Enables truncation of vectors to this dimensionality. This is only useful for models trained to store more important information in earlier dimensions such as Matryoshka Representation Learning (MRL).

quantize Copy Chapter

quantize: int|boolean

Enables scalar vector quantization at the specified precision. Supports 1-bit through 8-bit quantization. Scalar quantization transforms continuous floating point values to discrete unsigned integers. The faiss, pgvector, numpy and torch ANN backends support storing these vectors.

This parameter supports booleans for backwards compatibility. When set to true/false, this flag sets faiss.quantize.

In addition to vector-level quantization, some ANN backends have the ability to quantize vectors at the storage layer. See the ANN configuration options for more.

instructions Copy Chapter

instructions:
    query: prefix for queries
    data: prefix for indexing

Instruction-based models use prefixes to modify how embeddings are computed. This is especially useful with asymmetric search, which is when the query and indexed data are of vastly different lengths. In other words, short queries with long documents.

txtai automatically loads prompts stored in config_sentence_transformers.json except if this parameter is set. For some older models such as E5-base, instructions still need to be provided via this parameter.

models Copy Chapter

models: dict

Loads and stores vector models in this cache. This is primarily used with subindexes but can be set on any embeddings instance. This prevents the same model from being loaded multiple times when working with multiple embeddings instances.

tokenize Copy Chapter

tokenize: boolean

Enables string tokenization (defaults to false). This method applies tokenization rules that only work with English language text. It's not recommended for use with recent vector models.

vectors Copy Chapter

vectors: dict

Passes these additional parameters to the underlying vector model.

center Copy Section

center: boolean | dict

Centers normalized late-interaction token vectors and normalizes them again before MUVERA or LEMUR encoding. Centering is applied to
both query and document vectors. true uses batch scope and false disables centering. When omitted, centering defaults to batch
scope only for late-interaction models that load more than one torch.nn.Linear layer; models with zero or one loaded linear layer keep
their previous output.

Dictionary configuration supports the following scopes:

center:
    scope: document | batch | collection

document subtracts the mean of each document or query independently. batch subtracts the mean of all real token rows in the current
model batch. collection requires a caller-provided, one-dimensional mean vector through either mean (an array-like value) or path
(a safetensors file containing a center.mean tensor), but not both. Zero-padding rows are never centered and remain zero.

muvera Copy Section

muvera:
    repetitions: defaults 20
    hashes: defaults to 5
    projection: defaults 16

Settings to control the size of MUVERA fixed dimensional outputs. Default is 20 * 2^5 * 16 = 10,240 dimensions.

lemur Copy Section

lemur:
    path: local artifact directory or Hugging Face Hub path

Loads a trained LEMUR fixed dimensional encoder. LEMUR artifacts are corpus-specific and must be
created with LemurTrainer before configuring an embeddings index. Query vectors are summed learned features and document vectors
are ordinary least squares weights over the artifact's stored token sample. Each artifact contains config.json and
model.safetensors; the Safetensors file contains inference state only. See the trainer page for score filtering and Faiss exact/IVF
search behavior.

trust_remote_code Copy Section

trust_remote_code: boolean

Parameter for trusting the code from Hugging Face models with custom implementations.

Embeddings/Format Copy Section

Index format

This section documents the txtai index format. Each component is designed to ensure open access to the underlying data in a programmatic and platform independent way

If an underlying library has an index format, that is used. Otherwise, txtai persists content with MessagePack serialization.

To learn more about how these components work together, read the Index Guide and Query Guide.

ANN Copy Chapter

Approximate Nearest Neighbor (ANN) index configuration for storing vector embeddings.

Core Copy Chapter

Core embeddings index files.

Database Copy Chapter

Databases store metadata, text and binary content.

Graph Copy Chapter

Graph nodes and edges for an embeddings index

Scoring Copy Chapter

Sparse/keyword indexing

Embeddings/Index Copy Section

Embeddings

Embeddings databases are the engine that delivers semantic search. Data is transformed into embeddings vectors where similar concepts will produce similar vectors. Indexes both large and small are built with these vectors. The indexes are used to find results that have the same meaning, not necessarily the same keywords.

The following code snippet shows how to build and search an embeddings index.

from txtai import Embeddings

# Create embeddings model, backed by sentence-transformers & transformers
embeddings = Embeddings(path="sentence-transformers/nli-mpnet-base-v2")

data = [
  "US tops 5 million confirmed virus cases",
  "Canada's last fully intact ice shelf has suddenly collapsed, " +
  "forming a Manhattan-sized iceberg",
  "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
  "The National Park Service warns against sacrificing slower friends " +
  "in a bear attack",
  "Maine man wins $1M from $25 lottery ticket",
  "Make huge profits without work, earn up to $100,000 a day"
]

# Index the list of text
embeddings.index(data)

print(f"{'Query':20} Best Match")
print("-" * 50)

# Run an embeddings search for each query
for query in ("feel good story", "climate change", "public health story", "war",
              "wildlife", "asia", "lucky", "dishonest junk"):
    # Extract uid of first result
    # search result format: (uid, score)
    uid = embeddings.search(query, 1)[0][0]

    # Print text
    print(f"{query:20} {data[uid]}")

Build Copy Chapter

An embeddings instance is configuration-driven based on what is passed in the constructor. Vectors are stored with the option to also store content. Content storage enables additional filtering and data retrieval options.

The example above sets a specific embeddings vector model via the path parameter. An embeddings instance with no configuration can also be created.

embeddings = Embeddings()

In this case, when loading and searching for data, the default transformers vector model is used to vectorize data. See the model guide for current model recommentations.

Index Copy Chapter

After creating a new embeddings instance, the next step is adding data to it.

embeddings.index(rows)

The index method takes an iterable and supports the following formats for each element.

• (id, data, tags) - default processing format

• (id, data)

  Same as above but without tags.

• data

Single element to index. In this case, unique id's will automatically be generated. Note that for generated id's, upsert and delete calls require a separate search to get the target ids.

When the data field is a dictionary, text is passed via the text key, binary objects via the object key. Note that content must be enabled to store metadata and objects to store binary object data. The id and tags keys will be extracted, if provided.

The input iterable can be a list or generator. Generators help with indexing very large datasets as only portions of the data is in memory at any given time.

More information on indexing can be found in the index guide.

Search Copy Chapter

Once data is indexed, it is ready for search.

embeddings.search(query, limit)

The search method takes two parameters, the query and query limit. The results format is different based on whether content is stored or not.

• List of (id, score) when content is not stored
• List of {**query columns} when content is stored

Both natural language and SQL queries are supported. More information can be found in the query guide.

Resource management Copy Chapter

Embeddings databases are context managers. The following blocks automatically close and free resources upon completion.

# Create a new Embeddings database, index data and save
with Embeddings() as embeddings:
  embeddings.index(rows)
  embeddings.save(path)

# Search a saved Embeddings database
with Embeddings().load(path) as embeddings:
  embeddings.search(query)

While calling close isn't always necessary (resources will be garbage collected), it's best to free shared resources like database connections as soon as they aren't needed.

More examples Copy Chapter

See this link for a full list of embeddings examples.

Embeddings/Indexing Copy Section

Index guide

This section gives an in-depth overview on how to index data with txtai. We'll cover vectorization, indexing/updating/deleting data and the various components of an embeddings database.

Vectorization Copy Chapter

The most compute intensive step in building an index is vectorization. The path parameter sets the path to the vector model. There is logic to automatically detect the vector model method but it can also be set directly.

The batch and encodebatch parameters control the vectorization process. Larger values for batch will pass larger batches to the vectorization method. Larger values for encodebatch will pass larger batches for each vector encode call. In the case of GPU vector models, larger values will consume more GPU memory.

Data is buffered to temporary storage during indexing as embeddings vectors can be quite large (for example 768 dimensions of float32 is 768 * 4 = 3072 bytes per vector). Once vectorization is complete, a mmapped array is created with all vectors for Approximate Nearest Neighbor (ANN) indexing.

The terms ANN and dense vector index are used interchangeably throughout txtai's documentation.

Setting a backend Copy Chapter

As mentioned above, computed vectors are stored in an ANN. There are various index backends that can be configured. Faiss is the default backend.

Content storage Copy Chapter

Embeddings indexes can optionally store content. When this is enabled, the input content is saved in a database alongside the computed vectors. This enables filtering on additional fields and content retrieval.

The columns used for text, object and JSON data storage are set via column configuration.

Index vs Upsert Copy Chapter

Data is loaded into an index with either an index or upsert call.

embeddings.index([(uid, text, None) for uid, text in enumerate(data)])
embeddings.upsert([(uid, text, None) for uid, text in enumerate(data)])

The index call will build a brand new index replacing an existing one. upsert will insert or update records. upsert ops do not require a full index rebuild.

Save Copy Chapter

Indexes can be stored in a directory using the save method.

embeddings.save("/path/to/save")

Compressed indexes are also supported.

embeddings.save("/path/to/save/index.tar.gz")

In addition to saving indexes locally, they can also be persisted to cloud storage.

embeddings.save("/path/to/save/index.tar.gz", cloud={...})

This is especially useful when running in a serverless context or otherwise running on temporary compute. Cloud storage is only supported with compressed indexes.

Embeddings indexes can be restored using the load method.

embeddings.load("/path/to/load")

Delete Copy Chapter

Content can be removed from the index with the delete method. This method takes a list of ids to delete.

embeddings.delete(ids)

Reindex Copy Chapter

When content storage is enabled, reindex can be called to rebuild the index with new settings. For example, the backend can be switched from faiss to hnsw or the vector model can be updated. This prevents having to go back to the original raw data.

embeddings.reindex(path="sentence-transformers/all-MiniLM-L6-v2", backend="hnsw")

Graph Copy Chapter

Enabling a graph network adds a semantic graph at index time as data is being vectorized. Vector embeddings are used to automatically create relationships in the graph. Relationships can also be manually specified at index time.

# Manual relationships by id
embeddings.index([{"id": "0", "text": "...", "relationships": ["2"]}])

# Manual relationships with additional edge attributes
embeddings.index(["id": "0", "text": "...", "relationships": [
    {"id": "2", "type": "MEMBER_OF"}
]])

Additionally, graphs can be used for topic modeling. Dimensionality reduction with UMAP combined with HDBSCAN is a popular topic modeling method found in a number of libraries. txtai takes a different approach using community detection algorithms to build topic clusters.

This approach has the advantage of only having to vectorize data once. It also has the advantage of better topic precision given there isn't a dimensionality reduction operation (UMAP). Semantic graph examples are shown below.

Get a mapping of discovered topics to associated ids.

embeddings.graph.topics

Show the most central nodes in the index.

embeddings.graph.centrality()

Graphs are persisted alongside an embeddings index. Each save and load will also save and load the graph.

Sparse vectors Copy Chapter

Scoring instances can create a standalone sparse keyword indexes (BM25, TF-IDF) and sparse vector indexes (SPLADE). This enables hybrid search when there is an available dense vector index.

The terms sparse vector index, keyword index, terms index and scoring index are used interchangeably throughout txtai's documentation.

See this link to learn more.

Subindexes Copy Chapter

An embeddings instance can optionally have associated subindexes, which are also embeddings databases. This enables indexing additional fields, vector models and much more.

Word vectors Copy Chapter

When using word vector backed models with scoring set, a separate call is required before calling index as follows:

embeddings.score(rows)
embeddings.index(rows)

Both calls are required to support generator-backed iteration of data with word vectors models.

Embeddings/Methods Copy Section

Methods

::: txtai.embeddings.Embeddings
options:
filters:
- "!columns"
- "!createann"
- "!createcloud"
- "!createdatabase"
- "!creategraph"
- "!createids"
- "!createindexes"
- "!createscoring"
- "!checkarchive"
- "!configure"
- "!defaultallowed"
- "!defaults"
- "!initindex"
- "!loadquery"
- "!loadvectors"

Embeddings/Query Copy Section

Query guide

This section covers how to query data with txtai. The simplest way to search for data is building a natural language string with the desired content to find. txtai also supports querying with SQL. We'll cover both methods here.

Natural language queries Copy Chapter

In the simplest case, the query is text and the results are index text that is most similar to the query text.

embeddings.search("feel good story")
embeddings.search("wildlife")

The queries above search the index for similarity matches on feel good story and wildlife. If content storage is enabled, a list of {**query columns} is returned. Otherwise, a list of (id, score) tuples are returned.

SQL Copy Chapter

txtai supports more complex queries with SQL. This is only supported if content storage is enabled. txtai has a translation layer that analyzes input SQL statements and combines similarity results with content stored in a relational database.

SQL queries are run through embeddings.search like natural language queries but the examples below only show the SQL query for conciseness.

embeddings.search("SQL query")

Similar clause Copy Section

The similar clause is a txtai function that enables similarity searches with SQL.

SELECT id, text, score FROM txtai WHERE similar('feel good story')

The similar clause takes the following arguments:

similar("query", "number of candidates", "index", "weights")

The txtai query layer joins results from two separate components, a relational store and a similarity index. With a similar clause, a similarity search is run and those ids are fed to the underlying database query.

The number of candidates should be larger than the desired number of results when applying additional filter clauses. This ensures that limit results are still returned after applying additional filters. If the number of candidates is not specified, it is defaulted as follows:

• For a single query filter clause, the default is the query limit
• With multiple filtering clauses, the default is 10x the query limit

The index name is only applicable when subindexes are enabled. This specifies the index to use for the query.

Weights sets the hybrid score weights when an index has both a sparse and dense index.

Dynamic columns Copy Section

Content can be indexed in multiple ways when content storage is enabled. Remember that input documents take the form of (id, data, tags) tuples. If data is a string or binary content, it's indexed and searchable with similar() clauses.

If data is a dictionary, then all fields in the dictionary are stored and available via SQL. The text field or field specified in the index configuration is indexed and searchable with similar() clauses.

For example:

embeddings.index([{"text": "text to index", "flag": True,
                   "actiondate": "2022-01-01"}])

With the above input data, queries can now have more complex filters.

SELECT text, flag, actiondate FROM txtai WHERE similar('query') AND flag = 1
AND actiondate >= '2022-01-01'

txtai's query layer automatically detects columns and translates queries into a format that can be understood by the underlying database.

Nested dictionaries/JSON is supported and can be escaped with bracket statements.

embeddings.index([{"text": "text to index",
                   "parent": {"child element": "abc"}}])

SELECT text FROM txtai WHERE [parent.child element] = 'abc'

Note the bracket statement escaping the nested column with spaces in the name.

Bind parameters Copy Section

txtai has support for SQL bind parameters.

# Query with a bind parameter for similar clause
query = "SELECT id, text, score FROM txtai WHERE similar(:x)"
results = embeddings.search(query, parameters={"x": "feel good story"})

# Query with a bind parameter for column filter
query = "SELECT text, flag, actiondate FROM txtai WHERE flag = :x"
results = embeddings.search(query, parameters={"x": 1})

Aggregation queries Copy Section

The goal of txtai's query language is to closely support all functions in the underlying database engine. The main challenge is ensuring dynamic columns are properly escaped into the engines native query function.

Aggregation query examples.

SELECT count(*) FROM txtai WHERE similar('feel good story') AND score >= 0.15
SELECT max(length(text)) FROM txtai WHERE similar('feel good story')
AND score >= 0.15
SELECT count(*), flag FROM txtai GROUP BY flag ORDER BY count(*) DESC

Binary objects Copy Chapter

txtai has support for storing and retrieving binary objects. Binary objects can be retrieved as shown in the example below.

# Create embeddings index with content and object storage enabled
embeddings = Embeddings(content=True, objects=True)

# Get an image
request = open("demo.gif", "rb")

# Insert record
embeddings.index([(
  "txtai",
  {"text": "txtai executes machine-learning workflows.",
   "object": request.read()}
)])

# Query txtai and get associated object
query = "SELECT object FROM txtai WHERE similar('machine learning') LIMIT 1"
result = embeddings.search(query)[0]["object"]

# Query binary content with a bind parameter
query = "SELECT object FROM txtai WHERE similar(:x) LIMIT 1"
results = embeddings.search(query, parameters={"x": request.read()})

Custom SQL functions Copy Chapter

Custom, user-defined SQL functions extend selection, filtering and ordering clauses with additional logic. For example, the following snippet defines a function that translates text using a translation pipeline.

# Translation pipeline
translate = Translation()

# Create embeddings index
embeddings = Embeddings(path="sentence-transformers/nli-mpnet-base-v2",
                        content=True,
                        functions=[translate]})

# Run a search using a custom SQL function
embeddings.search("""
SELECT
  text,
  translation(text, 'de', null) 'text (DE)',
  translation(text, 'es', null) 'text (ES)',
  translation(text, 'fr', null) 'text (FR)'
FROM txtai WHERE similar('feel good story')
LIMIT 1
""")

Expressions Copy Chapter

Expression shortcuts expand into more complex SQL snippets. This is useful for making SQL queries more concise. Indexing is also available on expressions as a performance improvement.

The following example indexes a json extraction field (filepath) and the length of each field.

# Create embeddings index
embeddings = Embeddings(
  path="sentence-transformers/nli-mpnet-base-v2",
  content=True,
  expressions=[
    {"name": "filepath", "index": True},
    {"name": "textlength", "expression": "length(text)", "index": True}
  ]
)

embeddings.search("SELECT textlength, filepath FROM txtai LIMIT 1")

Query translation Copy Chapter

Natural language queries with filters can be converted to txtai-compatible SQL statements with query translation. For example:

embeddings.search("feel good story since yesterday")

can be converted to a SQL statement with a similar clause and date filter.

select id, text, score from txtai where similar('feel good story') and
entry >= date('now', '-1 day')

This requires setting a query translation model. The default query translation model is t5-small-txtsql but this can easily be finetuned to handle different use cases.

Hybrid search Copy Chapter

When an embeddings database has both a sparse and dense index, both indexes will be queried and the results will be equally weighted unless otherwise specified.

embeddings.search("query", weights=0.5)
embeddings.search(
  "SELECT id, text, score FROM txtai WHERE similar('query', 0.5)"
)

Graph search Copy Chapter

If an embeddings database has an associated graph network, graph searches can be run. The search syntax below uses openCypher. Follow the preceding link to learn more about this syntax.

Additionally, standard embeddings searches can be returned as graphs.

# Find all paths between id: 0 and id: 5 between 1 and 3 hops away
embeddings.search("""
MATCH P=({id: 0})-[*1..3]->({id: 5})
RETURN P
""")

# Find related nodes for query matches
embeddings.search("""
  MATCH P=(A)-[]->(B)
  WHERE SIMILAR(A, "query")
  RETURN B
  ORDER BY A.score DESC
  LIMIT 10
""")

# Standard embeddings search as graph
embeddings.search("query", graph=True)

Subindexes Copy Chapter

Subindexes can be queried as follows:

# Build index with subindexes
embeddings = Embeddings(
  content=True,
  defaults=False,
  indexes={
    "keyword": {
      "keyword": True
    },
    "dense":{
      "dense": True
    }
  }
)
embeddings.index(stream())

# Query with index parameter
embeddings.search("query", index="keyword")

# Specify with SQL
embeddings.search("""
SELECT id, text, score FROM txtai
WHERE similar('query', 'keyword')
""")

Combined index architecture Copy Chapter

txtai has multiple storage and indexing components. Content is stored in an underlying database along with an approximate nearest neighbor (ANN) index, keyword index and graph network. These components combine to deliver similarity search alongside traditional structured search.

The ANN index stores ids and vectors for each input element. When a natural language query is run, the query is translated into a vector and a similarity query finds the best matching ids. When a database is added into the mix, an additional step is executed. This step takes those ids and effectively inserts them as part of the underlying database query. The same steps apply with keyword indexes except a term frequency index is used to find the best matching ids.

Dynamic columns are supported via the underlying engine. For SQLite, data is stored as JSON and dynamic columns are converted into json_extract clauses. Client-server databases are supported via SQLAlchemy and dynamic columns are supported provided the underlying engine has JSON support.

Pipeline/Image/Caption Copy Section

Caption

The caption pipeline reads a list of images and returns a list of captions for those images.

Example Copy Chapter

The following shows a simple example using this pipeline.

from txtai.pipeline import Caption

# Create and run pipeline
caption = Caption()
caption("path to image file")

See the link below for a more detailed example.

Configuration-driven example Copy Chapter

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml Copy Section

# Create pipeline using lower case class name
caption:

# Run pipeline with workflow
workflow:
  caption:
    tasks:
      - action: caption

Run with Workflows Copy Section

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("caption", ["path to image file"]))

Run with API Copy Section

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"caption", "elements":["path to image file"]}'

Methods Copy Chapter

Python documentation for the pipeline.

::: txtai.pipeline.Caption.__init__ Copy Section

::: txtai.pipeline.Caption.__call__ Copy Section

Pipeline/Image/Imagehash Copy Section

ImageHash

The image hash pipeline generates perceptual image hashes. These hashes can be used to detect near-duplicate images. This method is not backed by machine learning models and not intended to find conceptually similar images.

Example Copy Chapter

The following shows a simple example using this pipeline.

from txtai.pipeline import ImageHash

# Create and run pipeline
ihash = ImageHash()
ihash("path to image file")

See the link below for a more detailed example.

Configuration-driven example Copy Chapter

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml Copy Section

# Create pipeline using lower case class name
imagehash:

# Run pipeline with workflow
workflow:
  imagehash:
    tasks:
      - action: imagehash

Run with Workflows Copy Section

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("imagehash", ["path to image file"]))

Run with API Copy Section

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"imagehash", "elements":["path to image file"]}'

Methods Copy Chapter

Python documentation for the pipeline.

::: txtai.pipeline.ImageHash.__init__ Copy Section

::: txtai.pipeline.ImageHash.__call__ Copy Section

Pipeline/Image/Objects Copy Section

Objects

The Objects pipeline reads a list of images and returns a list of detected objects.

Example Copy Chapter

The following shows a simple example using this pipeline.

from txtai.pipeline import Objects

# Create and run pipeline
objects = Objects()
objects("path to image file")

See the link below for a more detailed example.

Configuration-driven example Copy Chapter

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml Copy Section

# Create pipeline using lower case class name
objects:

# Run pipeline with workflow
workflow:
  objects:
    tasks:
      - action: objects

Run with Workflows Copy Section

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("objects", ["path to image file"]))

Run with API Copy Section

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"objects", "elements":["path to image file"]}'

Methods Copy Chapter

Python documentation for the pipeline.

::: txtai.pipeline.Objects.__init__ Copy Section

::: txtai.pipeline.Objects.__call__ Copy Section

Pipeline/Data/Filetohtml Copy Section

File To HTML

The File To HTML pipeline transforms files to HTML. It supports the following text extraction backends.

Apache Tika Copy Chapter

Apache Tika detects and extracts metadata and text from over a thousand different file types. See this link for a list of supported document formats.

Apache Tika requires Java to be installed. An alternative to that is starting a separate Apache Tika service via this Docker Image and setting these environment variables.

Docling Copy Chapter

Docling parses documents and exports them to the desired format with ease and speed. This is a library that has rapidly gained popularity starting in late 2024. Docling excels in parsing formatting elements from PDFs (tables, sections etc).

See this link for a list of supported document formats.

LiteParse Copy Chapter

LiteParse is a fast, helpful, and open-source document parser. It supports PDF out of the box with additional format support via third-party packages.

See this link for more.

Example Copy Chapter

The following shows a simple example using this pipeline.

from txtai.pipeline import FileToHTML

# Create and run pipeline
html = FileToHTML()
html("/path/to/file")

Configuration-driven example Copy Chapter

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml Copy Section

# Create pipeline using lower case class name
filetohtml:

# Run pipeline with workflow
workflow:
  html:
    tasks:
      - action: filetohtml

Run with Workflows Copy Section

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("html", ["/path/to/file"]))

Run with API Copy Section

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"html", "elements":["/path/to/file"]}'

Methods Copy Chapter

Python documentation for the pipeline.

::: txtai.pipeline.FileToHTML.__init__ Copy Section

::: txtai.pipeline.FileToHTML.__call__ Copy Section

Pipeline/Data/Htmltomd Copy Section

HTML To Markdown

The HTML To Markdown pipeline transforms HTML to Markdown.

Markdown formatting is applied for headings, blockquotes, lists, code, tables and text. Visual formatting is also included (bold, italic etc).

This pipeline searches for the best node that has relevant text, often found with an article, main or body tag.

The HTML to Markdown pipeline requires the BeautifulSoup4 library to be installed.

Example Copy Chapter

The following shows a simple example using this pipeline.

from txtai.pipeline import HTMLToMarkdown

# Create and run pipeline
md = HTMLToMarkdown()
md("<html>This is a test</html>")

Configuration-driven example Copy Chapter

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml Copy Section

# Create pipeline using lower case class name
htmltomarkdown:

# Run pipeline with workflow
workflow:
  markdown:
    tasks:
      - action: htmltomarkdown

Run with Workflows Copy Section

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("markdown", ["<html>This is a test</html>"]))

Run with API Copy Section

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"markdown", "elements":["<html>This is a test</html>"]}'

Methods Copy Chapter

Python documentation for the pipeline.

::: txtai.pipeline.HTMLToMarkdown.__init__ Copy Section

::: txtai.pipeline.HTMLToMarkdown.__call__ Copy Section

Pipeline/Data/Segmentation Copy Section

Segmentation

The Segmentation pipeline segments text into semantic units.

Example Copy Chapter

The following shows a simple example using this pipeline.

from txtai.pipeline import Segmentation

# Create and run pipeline
segment = Segmentation(sentences=True)
segment("This is a test. And another test.")

# Load third-party chunkers
segment = Segmentation(chunker="semantic")
segment("This is a test. And another test.")

The Segmentation pipeline supports segmenting sentences, lines, paragraphs and sections using a rules-based approach. Each of these modes can be set when creating the pipeline. Third-party chunkers are also supported via the chunker parameter.

Configuration-driven example Copy Chapter

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml Copy Section

# Create pipeline using lower case class name
segmentation:
  sentences: true

# Run pipeline with workflow
workflow:
  segment:
    tasks:
      - action: segmentation

Run with Workflows Copy Section

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("segment", ["This is a test. And another test."]))

Run with API Copy Section

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"segment", "elements":["This is a test. And another test."]}'

Methods Copy Chapter

Python documentation for the pipeline.

::: txtai.pipeline.Segmentation.__init__ Copy Section

::: txtai.pipeline.Segmentation.__call__ Copy Section