turbovec Documentation

File: README.md Copy Chapter

Ten million 768-dimensional embeddings take 31 GB of RAM as float32. turbovec holds them in 4 GB and searches them 4.4× faster than FAISS, at matching recall.

turbovec is an in-process vector index for Python and Rust. It stores each vector at 2 or 4 bits per dimension using Google Research's TurboQuant and searches that compressed form directly with agent-optimised SIMD kernels.

• Fast. 4.4× FAISS FastScan at 4-bit and 2.2× at 2-bit, on ARM and x86. A staged search scans one bit plane and rescores the shortlist exactly.
• Low memory. 7.8× smaller than float32 at 4-bit, 15.5× at 2-bit, at the same recall as FAISS.
• Online. No training step, no rebuilds: the codebook comes from the maths, not your data. Add a vector and it is searchable; remove one in microseconds.
• Incremental. sync(path) writes only what changed since the last call — one fsync, crash-safe at any byte, milliseconds however large the index.
• Local. A library, not a service: nothing leaves your process. Pair it with an open-source embedding model for a fully offline stack.
• Filtered. Pass an allowlist of ids or a bitmask; the kernel skips what the filter excludes and returns up to k results from the allowed set.

Python Copy Chapter

pip install turbovec

import numpy as np
from turbovec import TurboQuantIndex

vectors = np.load("embeddings.npy").astype(np.float32)   # shape (n, 1536)

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)

scores, indices = index.search(vectors[:1], k=5)
print(indices)         # [[    0 86544  3385 84492 24967]]   int64 slot positions
print(scores.shape)    # (1, 5)                             float32 inner products

index.sync("index.tv")                  # first sync writes the file
index.add(more_vectors)
index.sync("index.tv")                  # later syncs write just the change, durably

index = TurboQuantIndex.load("index.tv")   # loads either a sync'd or a write() file

Inputs are 2-D float32 arrays of shape (n, dim); other dtypes raise rather than silently convert. Scores are inner products, so normalise your vectors if you want cosine similarity.

Need ids that survive deletes? IdMapIndex keys everything by your own uint64 ids:

from turbovec import IdMapIndex

index = IdMapIndex(dim=1536, bit_width=4)
index.add_with_ids(vectors, ids)               # ids: uint64 array, one per row

scores, ids = index.search(query, k=10)        # your ids, not slot positions
index.remove(1002)                             # O(1)

# Filter: only ids another system approved — a SQL predicate, an ACL, a time window.
allowed = np.array([1001, 1003, 1042], dtype=np.uint64)
scores, ids = index.search(query, k=10, allowlist=allowed)   # shape (1, min(k, 3))

Framework integrations are drop-in replacements for each framework's in-memory store — same public surface, same persistence, same retriever wiring:
LangChain (pip install turbovec[langchain]) ·
LlamaIndex (turbovec[llama-index]) ·
Haystack (turbovec[haystack]) ·
Agno (turbovec[agno]).

Rust Copy Chapter

cargo add turbovec

use turbovec::{IdMapIndex, TurboQuantIndex};

let mut index = TurboQuantIndex::new(1536, 4)?;
index.add(&vectors);                          // &[f32], row-major, n × 1536
let results = index.search(&queries, 10);     // scores and slot indices, nq × 10
index.write("index.tv")?;
let index = TurboQuantIndex::load("index.tv")?;

let mut ids = IdMapIndex::new(1536, 4)?;
ids.add_with_ids(&vectors, &[1001, 1002, 1003])?;
let (scores, found) = ids.search(&queries, 10);
ids.remove(1002);

The Rust API mirrors the Python one; docs/api.md covers both, including the file formats, sync(), calibration and the staged search.

How it compares Copy Chapter

All figures: 100K OpenAI embeddings (d=1536 and d=3072) and GloVe (d=200), k=64, 1K queries, median of 5 runs, on a GCP c4a-standard-8 (ARM) and c3-standard-8 (x86), against FAISS at a matched bit rate. Scripts, raw JSON and every chart are in docs/benchmarks.md.

Both always return the true nearest neighbour within the first 8 results on the OpenAI corpora. The full suite — every chart, the raw JSON and the scripts to re-run it — is in docs/benchmarks.md.

When to use something else Copy Chapter

• You need a graph index. turbovec scans every vector. At 100K vectors a query costs 0.13–0.25 ms on all eight cores of the benchmark boxes and 0.5–2.0 ms on one thread (k=64, 1,000-query batches, d=1536 to d=3072), and the cost grows linearly with the corpus. For hundreds of millions of vectors behind strict latency, an HNSW or IVF index in front of it is the right shape, and turbovec is not one.
• You need exact float results. Quantization is lossy. On the OpenAI corpora the top result matches the float ground truth 96–98% of the time at 4-bit (90–93% at 2-bit) and is always within the first 8 returned; low-dimensional embeddings such as GloVe d=200 lose more. Check recall on a sample of your own data.
• You need a server. turbovec is a library with a file format. There is no network API, replication or multi-tenant service; the integrations above are how it slots into a stack that has those.
• Your vectors are not multiples of 8 wide, or wider than 16,384. Those are the dimension limits. Bit widths are 2 and 4.

How it works Copy Chapter

Normalise each vector and store its length. Rotate every vector by one shared random orthogonal matrix, after which each coordinate follows a known distribution whatever the input. Quantise each coordinate with a Lloyd-Max codebook computed for that distribution — 4 levels at 2-bit, 16 at 4-bit — and bit-pack. At search time the query is rotated once and scored against the codebook with SIMD lookup tables; a per-vector scalar fixed at encode time removes the inner-product bias quantization introduces. An optional one-shot calibrate(sample) fits two scalars per coordinate for data that drifts from the asymptotic distribution (low-dimensional and word-vector embeddings gain most).

The full walk-through, with the maths and the references, is in docs/how-it-works.md.

Building from source Copy Chapter

Python wheel and Rust crate

pip install maturin
cd turbovec-python && maturin build --release && pip install target/wheels/*.whl

cargo build --release

x86_64 builds target x86-64-v2 (SSE4.2, Nehalem 2008+) via .cargo/config.toml; the AVX-512 and AVX2 kernels are #[target_feature]-gated and chosen at runtime with is_x86_feature_detected!, so one binary runs everywhere and uses what the CPU has. Running the benchmark suite is described in docs/benchmarks.md.

References Copy Chapter

• TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate (ICLR 2026) — the paper this implements
• RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search (SIGMOD 2024) — the per-vector length-renormalization correction
• FAISS Fast accumulation of PQ and AQ codes — turbovec's x86 kernel adapts FastScan's pack layout, nibble-LUT scoring and u16 accumulator strategy

File: docs/integrations/agno.md Copy Chapter

Agno integration

turbovec.agno.TurboQuantVectorDb is an Agno VectorDb backed by an IdMapIndex. It implements the same public surface as agno.vectordb.lancedb.LanceDb (the closest in-tree single-machine backend) so this can be swapped in wherever LanceDb is used.

Install Copy Chapter

pip install turbovec[agno]

Basic usage Copy Chapter

from agno.agent import Agent
from agno.knowledge import Knowledge
from agno.knowledge.embedder.openai import OpenAIEmbedder
from turbovec.agno import TurboQuantVectorDb

vector_db = TurboQuantVectorDb(embedder=OpenAIEmbedder())

knowledge = Knowledge(vector_db=vector_db)
knowledge.add_content(text_content="Turbovec compresses vectors to 4 bits per dimension.")

agent = Agent(knowledge=knowledge)
agent.print_response("What does turbovec do?")

Constructor Copy Chapter

TurboQuantVectorDb(
    *,
    id: Optional[str] = None,
    name: Optional[str] = None,
    similarity_threshold: Optional[float] = None,
    embedder: Embedder,                              # required
    bit_width: int = 4,
    search_type: SearchType = SearchType.vector,
    distance: Distance = Distance.cosine,
    reranker: Optional[Reranker] = None,
    path: Optional[str] = None,
)

Similarity modes Copy Chapter

The distance parameter selects how scores are computed. It is fixed for the lifetime of the store:

• Distance.cosine (default). Document embeddings are L2-normalized at insert time and query embeddings at search time, so the kernel's raw score is cosine similarity in [-1, 1] for embeddings of any magnitude, and similarity_threshold compares against that cosine directly, so it behaves as a true [0, 1] relevance cutoff. A negative cosine clamps to 0, so any positive threshold drops it. Zero vectors are kept as-is and score 0 against everything.
• Distance.max_inner_product. Vectors are stored and queried raw: ranking is by raw inner product (magnitude-aware). The (ip + 1) / 2 mapping agno defines for this mode saturates at 0/1 once raw scores leave [-1, 1], so similarity_threshold values are dataset-relative here — calibrate against your embedder, or leave the threshold unset.

Insert / upsert Copy Chapter

insert and upsert follow the same (content_hash, documents, filters) signature as LanceDb. The internal doc_id is derived as md5(f"{base_id}_{content_hash}") where base_id is doc.id (or md5(content) when missing). The contract: the same (base_id, content_hash) pair always produces the same internal id, and the same base_id with a different content_hash is treated as a new entry — letting you keep content versions side-by-side.

Because doc_id is derived from base_id + content_hash (not from name, content_id, or metadata), two documents can collide on the same doc_id — a repeated explicit doc.id, or two documents with identical content and no id. When that happens both are stored and both remain individually deletable — keep-all, matching LanceDb's append-only behavior. (This differs from the LangChain store, which keeps the last write per id.) At query time, duplicate-content hits collapse to a single search result — see Filtered search.

from agno.knowledge.document import Document

docs = [Document(id="doc-1", name="paper.pdf", content="...", meta_data={"source": "arxiv"})]
vector_db.insert(content_hash="v1", documents=docs)

# Same doc with a new content_hash → new stored entry.
vector_db.insert(content_hash="v2", documents=docs)

Documents without embeddings are embedded via self.embedder before insertion. If embedding fails (get_embedding returns None) the call raises ValueError rather than silently dropping the document.

Filtered search Copy Chapter

Filters are resolved to an allowlist before scoring — the kernel only ever inserts allowed candidates into the per-query heap. You always get up to limit results from the filtered set; no over-fetching, no recall hit on selective filters.

results = vector_db.search(
    "quantum computing applications",
    limit=5,
    filters={"source": "arxiv", "year": 2024},      # AND of exact equality
)

Dict filters use AND-of-exact-equality on Document.meta_data. List-style FilterExpr filters (Agno's structured filter type) are silently ignored, matching LanceDb's behaviour.

Search results are deduplicated by content: after filtering (and reranking, when a reranker is set), hits with identical content (keyed by md5 of the text) collapse to the first occurrence, matching LanceDb.search. When duplicate-content documents are stored — e.g. the same text inserted under two content_hashes — a search can therefore return fewer than limit results; there is no over-fetch to refill the list.

Existence checks Copy Chapter

vector_db.name_exists("paper.pdf")          # bool — by Document.name
vector_db.id_exists("derived-md5-id")        # bool — by the internally-derived id
vector_db.content_hash_exists("v1")          # O(1) — set lookup, not a scan

Delete Copy Chapter

vector_db.delete_by_id(derived_id)               # by internal id
vector_db.delete_by_name("paper.pdf")             # by Document.name
vector_db.delete_by_metadata({"source": "web"})   # AND-of-equality on meta_data
vector_db.delete_by_content_id("cid-42")          # by Document.content_id
vector_db.drop()                                  # clear all
vector_db.delete()                                # returns False, deletes nothing — use drop()

Each delete_by_* returns True iff at least one document was removed. delete_by_name / delete_by_content_id / delete_by_metadata remove only the documents matching that exact predicate, even when other stored documents share the same derived doc_id. delete_by_id removes every document under that internal id.

update_metadata Copy Chapter

vector_db.update_metadata("cid-42", {"reviewed": True})

Merges the given metadata into meta_data of every document with the matching content_id. Overrides the base class's no-op warning.

Save / load Copy Chapter

vector_db = TurboQuantVectorDb(embedder=embedder, path="./my-store")
vector_db.create()                          # loads from path if existing

# ... insert documents ...

vector_db.save()                            # persists to path

Writes two files under the given folder path:

• index.tvim — the IdMapIndex payload.
• docstore.json — JSON-encoded document text, metadata, and id maps.

create() starts a fresh empty index only when the folder holds neither file. A folder holding just one of them is a partial save, and create() raises FileNotFoundError rather than starting empty — starting empty there would let the next save() overwrite the surviving file.

Document metadata must be JSON-serializable — same constraint Agno's LanceDb imposes on its payload column. The side-car carries a schema_version field; loaders refuse to deserialize unknown versions, and validate that the side-car's id maps are consistent with the loaded index.tvim (a mismatched or out-of-sync pair raises at load rather than failing later at query time).

The similarity mode is recorded in docstore.json. Because the store is constructed (with a distance) before create() loads the files, a recorded mode that conflicts with the constructor's raises ValueError — construct with the matching distance to load. A save written before the mode field existed holds raw, unnormalized vectors: it loads as Distance.max_inner_product — exactly the scoring it was written under — and self.distance is updated to reflect that.

save is atomic with respect to the destination: both files are written to sibling temp files and moved into place, so a failed save (e.g. non-JSON-serializable metadata) leaves a store previously saved at the same path intact.

The store also supports pickle (e.g. for multiprocessing workers, provided the embedder/reranker are picklable) and copy.copy / copy.deepcopy — both copies return a fully independent store (there is no shallow copy that shares the underlying index).

Async Copy Chapter

The lifecycle, write, and read methods have async counterparts: async_create, async_drop, async_exists, async_name_exists, async_get_count, async_insert, async_upsert, async_search. The remaining methods (the delete_by_* family, update_metadata, save, id_exists, content_hash_exists, optimize) are sync-only. The async paths call the embedder's async_get_embedding / async_get_embeddings_batch_and_usage for genuine async embedding generation. Agno's Embedder base class always defines both, so an embedder that inherits them without implementing them raises NotImplementedError on the async paths — use the sync methods with such an embedder.

async_create, async_drop, async_insert, async_upsert, and async_search run the index work on a worker thread (asyncio.to_thread) so the event loop stays responsive while a large insert or search is in flight. That is the shape Agno's own sync-backed vector DBs use (chromadb, pgvector, cassandra, pineconedb all wrap their sync bodies in asyncio.to_thread). async_exists, async_name_exists, and async_get_count answer inline — they are O(1) reads, and a thread hop would cost more than it saves.

Cancellation is only partial, and the distinction matters:

• asyncio.wait_for, task.cancel(), or a client disconnect returns control to the awaiting caller promptly — that part now works, where previously the coroutine ran to completion and the timeout never fired.
• It does not decide what happened to the insert. If the worker thread had already started, it runs the call to completion — work inside the Rust core is not interruptible at all — and the insert commits in full. If the executor was saturated, the call is cancelled before it ever starts and nothing is inserted. A cancelled async_insert is "outcome unknown": it may have fully committed, or may never have begun. Retry through async_upsert on the same content_hash if you need the retry to be idempotent.
• What is guaranteed: the outcome is all-or-nothing. The store is never left in a torn state.
• Timing out does not make the work go away: the loop's shutdown (asyncio.run on the way out, or loop.shutdown_default_executor()) waits for the worker thread, so a process that exits right after a short timeout can still block for the rest of the in-flight call.

Thread safety Copy Chapter

The store is safe for concurrent multi-threaded use:

• Reads run concurrently and scale. search, the existence checks, and get_count take no lock; the underlying index releases the GIL during scoring, so independent searches from multiple threads overlap and scale.
• Writes serialize. insert, upsert, the delete_by_* family, update_metadata, drop, and save serialize on a per-store lock. The async_* variants delegate to the same locked bodies.
• A read overlapping a write sees pre- or post-write state — never a torn one. Under heavy concurrent churn a search may transiently return fewer than limit results (hits deleted mid-search are skipped).

What the contract does not cover:

• No cross-call atomicity. A caller-side check-then-act sequence (id_exists then delete_by_id) can interleave with other writers. Batch writes are not atomic with respect to readers: a search overlapping an upsert can briefly see both the old and new generation of a content_hash.
• save serializes with writes (so it always snapshots a consistent store); reads may proceed during a save.
• The embedder and reranker are invoked outside the store's lock and must be thread-safe themselves.
• Two stores writing to the same path is safe. Concurrent save calls to one destination from several threads each publish atomically and the last writer wins; a caller never sees a torn file, and never an error caused only by the other writer. Which writer wins is not defined.
• Multi-process access is not supported.

Known limitations Copy Chapter

• Vector search only. search_type=SearchType.keyword and SearchType.hybrid are not supported (would require an external BM25 / lexical index). Constructor raises ValueError on those.
• No L2 distance. Distance.cosine and Distance.max_inner_product are the supported metrics; Distance.l2 raises ValueError (the underlying kernel scores by inner product).
• Embeddings are not retained after quantization. Stored vectors are the quantized form; the original full-precision embedding can't be recovered.
• JSON-serializable metadata only. Non-JSON-serializable values fail at save() time.

File: docs/integrations/haystack.md Copy Chapter

Haystack integration

turbovec.haystack.TurboQuantDocumentStore is a Haystack DocumentStore backed by an IdMapIndex. It implements the same public surface as haystack.document_stores.in_memory.InMemoryDocumentStore, so anywhere that store is written to or read from directly it can be swapped in. The query half of a RAG pipeline is not part of that surface — see Using in a Haystack Pipeline for the retriever you have to bring yourself.

Install Copy Chapter

pip install turbovec[haystack]

Basic usage Copy Chapter

from haystack import Document
from turbovec.haystack import TurboQuantDocumentStore

store = TurboQuantDocumentStore()
store.write_documents([
    Document(content="...", embedding=[...], meta={"source": "a"}),
    Document(content="...", embedding=[...], meta={"source": "b"}),
])

results = store.embedding_retrieval(query_embedding=[...], top_k=5)

Documents must have pre-computed embeddings — TurboQuantDocumentStore doesn't invoke an embedder. Pipe a Haystack embedder component upstream if your documents arrive without embeddings.

Constructor Copy Chapter

TurboQuantDocumentStore(
    dim: Optional[int] = None,
    bit_width: int = 4,
    *,
    embedding_similarity_function: Literal["dot_product", "cosine"] = "cosine",
    async_executor: Optional[ThreadPoolExecutor] = None,
    return_embedding: bool = False,
)

Similarity modes Copy Chapter

embedding_similarity_function selects how scores are computed. It is fixed for the lifetime of the store:

• "cosine" (default). Document embeddings are L2-normalized at write time and query embeddings at retrieval time, so raw scores are cosine similarity in [-1, 1] for embeddings of any magnitude, ranking matches InMemoryDocumentStore's cosine branch, and scale_score=True maps scores into [0, 1] via (s + 1) / 2 preserving order. Zero vectors are kept as-is and score 0 against everything (matching the reference, which substitutes a norm of 1 for zero-norm vectors).
• "dot_product". Vectors are stored and queried raw: scores are raw inner products and ranking is magnitude-aware — matching InMemoryDocumentStore's dot-product branch. scale_score=True applies the reference's expit(s / 100) sigmoid.

`DuplicatePolicy` Copy Chapter

write_documents takes a policy argument controlling how id collisions are handled:

from haystack.document_stores.types import DuplicatePolicy

store.write_documents(docs, policy=DuplicatePolicy.FAIL)      # raise if any id collides
store.write_documents(docs, policy=DuplicatePolicy.SKIP)      # silently skip colliding ids
store.write_documents(docs, policy=DuplicatePolicy.OVERWRITE) # remove-then-re-add colliding ids
# DuplicatePolicy.NONE is treated as FAIL.

Returns the number of documents actually written (so SKIP may return less than len(docs)).

Under FAIL (and NONE), documents are committed one at a time in batch order and the DuplicateDocumentError is raised on the first colliding id — every non-duplicate document before the collision stays persisted, matching InMemoryDocumentStore's post-exception state exactly.

Delete Copy Chapter

store.delete_documents(["id-1", "id-2"])     # by id; missing ids are silently ignored
store.delete_by_filter(filters)               # by filter; returns count
store.delete_all_documents()                  # clear everything

delete_documents and delete_by_filter are O(1) per matching document via the inner IdMapIndex.

Filters Copy Chapter

filter_documents(filters), embedding_retrieval(..., filters=...), and the other filter-aware helpers accept the full Haystack filter DSL:

filters = {
    "operator": "AND",
    "conditions": [
        {"field": "meta.source", "operator": "==", "value": "manual"},
        {"field": "meta.version", "operator": ">=", "value": 2},
    ],
}

# All docs matching the filter (no vector search):
docs = store.filter_documents(filters=filters)

# Top-k nearest to a query, filtered:
results = store.embedding_retrieval(
    query_embedding=[...],
    top_k=5,
    filters=filters,
)

Filter evaluation is delegated to haystack.utils.filters.document_matches_filter — anything Haystack's own stores support, we support.

embedding_retrieval validates query_embedding up front and raises ValueError("query_embedding should be a non-empty list of floats.") for an empty or non-numeric vector, matching InMemoryDocumentStore. A negative top_k also raises, where the reference returns n - 1 documents.

For embedding_retrieval, filters are resolved to an allowlist before scoring rather than via post-filtering. Selective filters return up to top_k matches from the filtered set; you never get fewer than top_k results just because the filter happened to exclude the top-scoring candidates.

Metadata helpers Copy Chapter

store.count_documents_by_filter(filters)                          # int
store.count_unique_metadata_by_filter(filters, ["source", "tag"]) # dict[str, int]
store.update_by_filter(filters, {"reviewed": True})               # bulk metadata update; returns count

store.get_metadata_fields_info()
# {"source": {"type": "keyword"}, "version": {"type": "int"}, ...}

store.get_metadata_field_min_max("version")     # {"min": 1, "max": 5}
store.get_metadata_field_unique_values("source")
# (["a", "b", "c"], 3)

update_by_filter updates metadata only — embeddings are quantized at write time and not re-encoded.

Async Copy Chapter

Every public method has an *_async variant:

await store.write_documents_async(docs)
results = await store.embedding_retrieval_async(query_embedding=q, top_k=5)
await store.delete_documents_async(["id-1"])

By default they run on a single-threaded executor owned by the store. Pass an async_executor= to the constructor to share an executor across stores (or to use more workers).

Save / load Copy Chapter

store.save_to_disk("./my-store")
# ... later ...
store = TurboQuantDocumentStore.load_from_disk("./my-store")

Writes two files under the given folder path:

• index.tvim — the IdMapIndex payload (quantized vectors + id maps).
• docstore.json — JSON-encoded document text, metadata, and id maps.

Document metadata must be JSON-serializable — the same constraint InMemoryDocumentStore.save_to_disk imposes. If the docstore.json side-car is out of sync with its index.tvim (a partial copy, a stale backup, tampering), load_from_disk raises a ValueError immediately rather than failing later with a KeyError at query time.

save_to_disk is atomic with respect to the destination: both files are written to sibling temp files and moved into place, so a failed save (e.g. non-JSON-serializable metadata) leaves a store previously saved at the same path intact.

The store also supports pickle (e.g. for multiprocessing workers; the restored store owns a fresh async executor) and copy.copy / copy.deepcopy — both copies return a fully independent store (there is no shallow copy that shares the underlying index).

Using in a Haystack Pipeline Copy Chapter

TurboQuantDocumentStore implements to_dict / from_dict so it can be serialized as part of a Haystack Pipeline. to_dict captures the component config (dim, bit_width, embedding_similarity_function, return_embedding); persisting the stored documents is the job of save_to_disk / load_from_disk.

Plug into a standard RAG pipeline, with two differences from InMemoryDocumentStore worth knowing before you wire it up.

No paired retriever ships. In Haystack the query half of a pipeline is a store-specific @component retriever, and core's InMemoryEmbeddingRetriever hard-rejects any store that is not the in-memory one. turbovec does not ship a retriever, so supply your own thin component that calls store.embedding_retrieval(...), or query the store directly outside the pipeline.

Reloading a serialized pipeline needs an allowlist. to_dict / from_dict work, but on haystack-ai 3.x — permitted by the declared haystack-ai>=2.23.0 floor — deserializing a pipeline that references an out-of-tree store raises DeserializationError unless the module is trusted. InMemoryDocumentStore is exempt because Haystack trusts its own module:

Pipeline.loads(pipeline.dumps())                                    # DeserializationError
Pipeline.loads(pipeline.dumps(), allowed_modules=["turbovec.haystack"])   # OK

HAYSTACK_DESERIALIZATION_ALLOWLIST sets the same thing process-wide.

The sentence-transformers embedders live in their own integration package (pip install sentence-transformers-haystack, which requires haystack-ai 2.24 or newer):

from haystack import Pipeline
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.embedders.sentence_transformers import (
    SentenceTransformersDocumentEmbedder,
)

store = TurboQuantDocumentStore()                 # dim inferred from first batch
indexing = Pipeline()
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder(
    model="sentence-transformers/all-MiniLM-L6-v2",
))
indexing.add_component("writer", DocumentWriter(document_store=store))
indexing.connect("embedder.documents", "writer.documents")

indexing.run({"embedder": {"documents": my_docs}})

Thread safety Copy Chapter

The store is safe for concurrent multi-threaded use:

• Reads run concurrently and scale. embedding_retrieval, filter_documents, the count and metadata helpers, and storage take no lock; the underlying index releases the GIL during scoring, so independent retrievals from multiple threads overlap and scale.
• Writes serialize. write_documents, delete_documents / delete_all_documents / delete_by_filter, update_by_filter, and save_to_disk serialize on a per-store lock. The *_async variants delegate to the same locked bodies.
• A read overlapping a write sees pre- or post-write state — never a torn one. Under heavy concurrent churn a retrieval may transiently return fewer than top_k documents (hits deleted mid-retrieval are skipped).

What the contract does not cover:

• No cross-call atomicity. A caller-side check-then-act sequence (count_documents then filter_documents) can interleave with other writers. Batch writes are not atomic with respect to readers: a retrieval overlapping an OVERWRITE write can briefly see a document id under both its old and new entry.
• save_to_disk serializes with writes (so it always snapshots a consistent store); reads may proceed during a save.
• to_dict / from_dict and the executor lifecycle are assumed single-threaded.
• Two stores writing to the same path is safe. Concurrent save_to_disk calls to one destination from several threads each publish atomically and the last writer wins; a caller never sees a torn file, and never an error caused only by the other writer. Which writer wins is not defined.
• Multi-process access is not supported.

Known limitations Copy Chapter

• Embeddings are not retained. embedding_retrieval(..., return_embedding=True) is accepted for signature compatibility but Document.embedding is always None on retrieved docs — turbovec discards the full-precision vector after quantization.
• JSON-serializable metadata only. Document metadata is stored as JSON in the side-car. Non-JSON-serializable values (custom objects, sets, etc.) fail at save time — the same constraint InMemoryDocumentStore.save_to_disk imposes.
• dim is locked on the first add. Subsequent calls with a different shape raise ValueError. If you need to change dim, construct a fresh store.

File: docs/integrations/langchain.md Copy Chapter

LangChain integration

turbovec.langchain.TurboQuantVectorStore is a LangChain VectorStore backed by an IdMapIndex. It implements the same public surface as langchain_core.vectorstores.in_memory.InMemoryVectorStore and can be used as a drop-in replacement wherever the in-memory store is used.

Install Copy Chapter

pip install turbovec[langchain]

Basic usage Copy Chapter

from langchain_huggingface import HuggingFaceEmbeddings
from turbovec.langchain import TurboQuantVectorStore

embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-base-en-v1.5")

store = TurboQuantVectorStore.from_texts(
    texts=["Document 1...", "Document 2...", "Document 3..."],
    embedding=embeddings,
    bit_width=4,
)

retriever = store.as_retriever(search_kwargs={"k": 5})

The dimensionality of the underlying quantized index is inferred from the embedding model on the first add_* call — no need to specify it up front.

Construction Copy Chapter

# No-arg: lazy. dim is inferred from the first add.
store = TurboQuantVectorStore(embeddings)

# from_texts: same lazy behaviour, plus immediate ingest.
store = TurboQuantVectorStore.from_texts(texts, embeddings, bit_width=4)

# Pre-built index: bring your own IdMapIndex (e.g. one loaded from disk).
from turbovec import IdMapIndex
store = TurboQuantVectorStore(embeddings, index=IdMapIndex(1536, 4))

bit_width is 2 or 4 and is fixed once the index is created.

Similarity modes Copy Chapter

The similarity keyword (on the constructor and from_texts/afrom_texts) selects how scores are computed. It is fixed for the lifetime of the store:

• "cosine" (default). Document vectors are L2-normalized before they reach the quantized index and query vectors are normalized before search, so scores are true cosine similarity in [-1, 1] and ranking matches InMemoryVectorStore regardless of embedding magnitude. Zero vectors are kept as-is and score 0 against everything (matching the reference's behavior).
• "dot_product". Vectors are stored and queried raw: scores are raw inner products and ranking is magnitude-aware. The (sim + 1) / 2 relevance mapping still applies for continuity, but it is not clamped to [0, 1] in this mode — an unbounded inner product has no calibrated relevance, so clamping would silently collapse every score >= 1.0 onto exactly 1.0 and defeat score_threshold retrieval. Requesting a relevance-score fn in this mode emits a UserWarning, and out-of-range values reach LangChain's own out-of-range warning. score_threshold retrieval is only meaningful in this mode if your embeddings are unit-normalized upstream.

store = TurboQuantVectorStore(embeddings, similarity="dot_product")

The similarity keyword is a turbovec extension: InMemoryVectorStore computes cosine unconditionally, so code written against the reference behaves identically under the default.

Adding with explicit ids Copy Chapter

store.add_texts(
    texts=["a", "b", "c"],
    ids=["doc-a", "doc-b", "doc-c"],
    metadatas=[{"source": "x"}, {"source": "y"}, {"source": "z"}],
)

# add_documents honours per-Document.id, falling back to a UUID per
# document if .id is missing — partial ids are not dropped wholesale.
store.add_documents([
    Document(id="explicit", page_content="..."),
    Document(page_content="..."),                  # gets a UUID
])

If an id is already present, add_texts upserts — the existing entry is removed and the new one added with the same id. This matches the typical user expectation that re-indexing a document with the same id should replace it, not duplicate it.

Ids must be str. A None entry in an explicit ids list is replaced with a generated UUID; any other non-str id (including bool/int) raises TypeError — naming the offending id, its type, and its position — before anything is stored. This is stricter than InMemoryVectorStore, which accepts non-str ids and then corrupts them through JSON persistence (an int 2 coexisting with the str "2" collapses to one document across dump/load).

Async equivalents (aadd_texts, aadd_documents) use the embedding model's aembed_documents so they benefit from concurrent embedding generation when the model supports it.

Search Copy Chapter

# By string query (uses the embedding function)
docs = store.similarity_search("what is turbovec?", k=5)

# With scores
docs_and_scores = store.similarity_search_with_score("...", k=5)

# By raw vector
import numpy as np
qvec = np.random.randn(768).astype(np.float32)
qvec /= np.linalg.norm(qvec)
docs = store.similarity_search_by_vector(qvec.tolist(), k=5)

Under the default similarity="cosine" mode, scores are cosine similarity — higher is better, range [-1, 1] — for embeddings of any magnitude (see Similarity modes).

similarity_search_with_relevance_scores and as_retriever(search_type="similarity_score_threshold") work: the cosine is mapped to [0, 1] via (sim + 1) / 2 (clamped to absorb the tiny overshoot caused by quantization noise). The clamp is cosine-only — see Similarity modes for dot_product.

similarity_search_with_score_by_vector returns (document, score) pairs for a precomputed query vector.

Async equivalents (asimilarity_search, asimilarity_search_with_score, asimilarity_search_by_vector, asimilarity_search_with_score_by_vector, aget_by_ids) are all implemented.

Filters Copy Chapter

similarity_search, similarity_search_with_score, and similarity_search_by_vector all accept a filter keyword:

# Dict — AND of exact equality on Document.metadata.
docs = store.similarity_search(
    "query", k=5, filter={"source": "manual", "version": 2},
)

# Callable — predicate over the Document.
docs = store.similarity_search(
    "query", k=5, filter=lambda doc: doc.metadata.get("score", 0) > 0.8,
)

The callable form matches the Callable[[Document], bool] convention used by InMemoryVectorStore, so predicates ported from there work unchanged. The dict form is a turbovec convenience on top of it — InMemoryVectorStore itself takes callables only.

A dict entry requires the key to be present: filter={"source": None} matches documents that store source=None, not documents with no source key at all. To match on absence, use the callable form (lambda doc: "source" not in doc.metadata).

Filters are resolved to an id allowlist before scoring; the kernel only ever inserts allowed documents into the per-query heap. You get up to k results from the filtered set, never fewer than k because the filter happened to exclude the top-scoring candidates.

Document retrieval by id Copy Chapter

docs = store.get_by_ids(["doc-a", "doc-c"])
# Missing ids are silently skipped.

aget_by_ids is also available.

Delete Copy Chapter

store.delete(["doc-a", "doc-b"])  # missing ids silently skipped, returns None

Delete is O(1) per id. delete(None) is a no-op (matches the InMemoryVectorStore contract).

Save / load Copy Chapter

store.dump("./my-store")
# ... later ...
store = TurboQuantVectorStore.load("./my-store", embedding=embeddings)

Writes two files under the given folder path:

• index.tvim — the IdMapIndex payload (see api.md).
• docstore.json — JSON-encoded document text, metadata, and id maps.

The similarity mode is recorded in docstore.json and restored by load. A store folder written before the mode field existed holds raw, unnormalized vectors, so it loads in "dot_product" mode — exactly the scoring it was written under — with no migration needed.

Document metadata must be JSON-serializable — the same constraint InMemoryVectorStore.dump imposes. If the docstore.json side-car is out of sync with its index.tvim (a partial copy, a stale backup, tampering), load raises a ValueError immediately rather than failing later with a KeyError at query time.

dump is atomic with respect to the destination: both files are written to sibling temp files and moved into place, so a failed dump (e.g. non-JSON-serializable metadata) leaves a store previously saved at the same path intact.

The store also supports pickle (e.g. for multiprocessing workers, provided the embedder is picklable) and copy.copy / copy.deepcopy — both copies return a fully independent store (there is no shallow copy that shares the underlying index).

Async Copy Chapter

Every read and write method has an a* counterpart: aadd_texts, aadd_documents, asimilarity_search*, aget_by_ids, adelete, afrom_texts.

They run the index work on a worker thread (asyncio.to_thread), so the event loop stays responsive while a large add or search is in flight — the same contract as VectorStore's own default async implementations, which offload via run_in_executor. The embedding step still awaits the embedder's own aembed_* coroutine.

Cancellation is only partial, and the distinction matters:

• asyncio.wait_for, task.cancel(), or a client disconnect returns control to the awaiting caller promptly — that part now works, where previously the coroutine ran to completion and the timeout never fired.
• It does not decide what happened to the write. If the worker thread had already started, it runs the call to completion — work inside the Rust core is not interruptible at all — and the write commits in full. If the executor was saturated, the call is cancelled before it ever starts and nothing is written. A cancelled write is "outcome unknown": it may have fully committed, or may never have begun. Neither "it happened" nor "it did not happen" is safe to assume; make retries idempotent by passing explicit ids, or read back to find out.
• What is guaranteed: the outcome is all-or-nothing. The store is never left in a torn state.
• Timing out does not make the work go away: the loop's shutdown (asyncio.run on the way out, or loop.shutdown_default_executor()) waits for the worker thread, so a process that exits right after a short timeout can still block for the rest of the in-flight call.

Thread safety Copy Chapter

The store is safe for concurrent multi-threaded use:

• Reads run concurrently and scale. similarity_search* and get_by_ids take no lock; the underlying index releases the GIL during scoring, so independent searches from multiple threads overlap and scale.
• Writes serialize. add_texts / add_documents, delete, and dump (and their async counterparts) serialize on a per-store lock.
• A read overlapping a write sees pre- or post-write state — never a torn one. Under heavy concurrent churn a search may transiently return fewer than k results (hits deleted mid-search are skipped).

What the contract does not cover:

• No cross-call atomicity. A caller-side check-then-act sequence (get_by_ids then delete, a count then a search) can interleave with other writers. Batch writes are not atomic with respect to readers: a search overlapping an upsert can briefly see a document id under both its old and new entry.
• dump serializes with writes (so it always snapshots a consistent store); reads may proceed during a dump.
• The embedder is invoked outside the store's lock and must be thread-safe itself.
• Two stores writing to the same path is safe. Concurrent dump calls to one destination from several threads each publish atomically and the last writer wins; a caller never sees a torn file, and never an error caused only by the other writer. Which writer wins is not defined.
• Multi-process access is not supported.

Known limitations Copy Chapter

• Max-marginal-relevance search is not supported. max_marginal_relevance_search and its variants raise NotImplementedError with an explanation. MMR requires the full-precision embedding of each candidate to compute pairwise diversity; turbovec discards full-precision vectors after quantization. If you need MMR, keep a parallel store with the raw embeddings and run MMR over that.
• Embeddings are not retained. search returns Document objects with page_content and metadata, but the original embedding is not recoverable.
• JSON-serializable metadata only. Non-JSON-serializable values (custom objects, sets, etc.) fail at save time — same constraint as the in-tree reference store.

File: docs/integrations/llama_index.md Copy Chapter

LlamaIndex integration

turbovec.llama_index.TurboQuantVectorStore is a LlamaIndex BasePydanticVectorStore backed by an IdMapIndex. It implements the same public surface as llama_index.core.vector_stores.simple.SimpleVectorStore and can be used as a drop-in replacement wherever the simple in-memory store is used.

Install Copy Chapter

pip install turbovec[llama-index]

Basic usage Copy Chapter

from llama_index.core import VectorStoreIndex, StorageContext
from turbovec.llama_index import TurboQuantVectorStore

vector_store = TurboQuantVectorStore()
storage_context = StorageContext.from_defaults(vector_store=vector_store)

index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
retriever = index.as_retriever(similarity_top_k=5)

The vector dimensionality is inferred from the embedding model on the first add() call.

Construction Copy Chapter

# No-arg: lazy. dim is inferred from the first add.
vector_store = TurboQuantVectorStore()

# from_params: same lazy behaviour, plus an explicit bit_width.
vector_store = TurboQuantVectorStore.from_params(bit_width=4)

# Pre-built index: bring your own IdMapIndex (e.g. one you loaded from disk).
from turbovec import IdMapIndex
vector_store = TurboQuantVectorStore(index=IdMapIndex(1536, 4))

bit_width is 2 or 4 and is fixed once the index is created.

Similarity modes Copy Chapter

The similarity keyword (on the constructor and from_params) selects how the similarities returned by query are computed. It is fixed for the lifetime of the store:

• "cosine" (default). Node embeddings are L2-normalized at add time and query embeddings at query time, so result.similarities are true cosine similarities in [-1, 1] and ranking matches SimpleVectorStore regardless of embedding magnitude — safe to feed into similarity-cutoff postprocessors. Zero vectors are kept as-is and score 0 against everything.
• "dot_product". Vectors are stored and queried raw: result.similarities are raw inner products and ranking is magnitude-aware.

vector_store = TurboQuantVectorStore(similarity="dot_product")

The similarity keyword is a turbovec extension: SimpleVectorStore computes cosine unconditionally, so code written against the reference behaves identically under the default.

The two `delete` signatures Copy Chapter

LlamaIndex's vector-store protocol has two distinct delete entry points:

`delete(ref_doc_id: str)` — remove an entire source document Copy Section

Removes every node whose ref_doc_id matches. Use this when you want to delete a whole parent document and its chunks in one call.

vector_store.delete("my-source-document-123")

Missing ref_doc_ids are silently ignored.

`delete_nodes(node_ids, filters)` — remove specific chunks Copy Section

Removes nodes matching either node_ids, filters, or both (intersected). Missing node_ids are silently ignored.

# By node_id
vector_store.delete_nodes(node_ids=["abc-123", "def-456"])

# By metadata filter
from llama_index.core.vector_stores.types import (
    MetadataFilter, MetadataFilters, FilterOperator,
)
filters = MetadataFilters(
    filters=[MetadataFilter(key="tier", value="archived", operator=FilterOperator.EQ)],
)
vector_store.delete_nodes(filters=filters)

# Both: intersect — delete only nodes in this list that ALSO match the filter
vector_store.delete_nodes(node_ids=["abc-123"], filters=filters)

`clear()` — drop everything Copy Section

vector_store.clear()

Resets the store while preserving the configured bit_width. The cleared store is immediately usable for new adds; dim is inferred again from the next batch.

Query Copy Chapter

LlamaIndex calls query(VectorStoreQuery) internally. If you've gone through VectorStoreIndex.from_documents(...), you won't call this directly — the retriever does. For direct use:

from llama_index.core.vector_stores.types import VectorStoreQuery

result = vector_store.query(VectorStoreQuery(
    query_embedding=[...],
    similarity_top_k=5,
))
# result.nodes, result.similarities, result.ids

query_embedding is required. turbovec doesn't embed query text itself; the calling component (retriever / query engine) is responsible for that.

Filtered query Copy Section

VectorStoreQuery accepts filters, node_ids, and doc_ids. All three intersect when more than one is supplied:

from llama_index.core.vector_stores.types import (
    MetadataFilter, MetadataFilters, FilterCondition, FilterOperator,
    VectorStoreQuery,
)

filters = MetadataFilters(
    filters=[
        MetadataFilter(key="tier", value="pro", operator=FilterOperator.EQ),
        MetadataFilter(key="year", value=2024, operator=FilterOperator.GTE),
    ],
    condition=FilterCondition.AND,
)

result = vector_store.query(VectorStoreQuery(
    query_embedding=[...],
    similarity_top_k=5,
    filters=filters,
    node_ids=["chunk-1", "chunk-2", "chunk-3"],   # restrict to these chunks
    doc_ids=["src-doc-42"],                        # restrict to chunks of this source doc
))

Supported operators on MetadataFilter: EQ, NE, GT, LT, GTE, LTE, IN, NIN, TEXT_MATCH, TEXT_MATCH_INSENSITIVE, CONTAINS, ANY, ALL, IS_EMPTY. Conditions: AND, OR, NOT. Nested MetadataFilters work.

Filter semantics match SimpleVectorStore's reference implementation — notably, every operator except IS_EMPTY returns False when the filter key is missing from the document's metadata, and TEXT_MATCH is case-insensitive (it lowercases both sides, as TEXT_MATCH_INSENSITIVE does).

Filters are resolved to a handle allowlist before scoring. Selective filters return up to similarity_top_k matches from the filtered set; you never get fewer just because the filter happened to exclude the top-scoring candidates.

An empty node_ids (or doc_ids) list restricts nothing — it behaves like omitting the argument. This follows the framework's own calling convention: VectorStoreIndex.as_retriever passes node_ids=list(index_struct.nodes_dict.values()), and for a stores_text=True store like this one that list is always empty — it means "unrestricted", and treating it as match-nothing would make every retriever query return zero results. get_nodes / delete_nodes are different: there node_ids is the selection, so an explicit empty list selects nothing.

Get nodes Copy Chapter

nodes = vector_store.get_nodes(node_ids=["chunk-1", "chunk-2"])
nodes = vector_store.get_nodes(filters=filters)
nodes = vector_store.get_nodes(node_ids=["chunk-1", "chunk-2"], filters=filters)  # intersect

Returns a List[BaseNode] reconstructed from the side-car. Missing node_ids are silently skipped. node_ids is the explicit selection: an empty list selects nothing and returns [] (same for delete_nodes, where an empty list is a no-op).

Upsert semantics Copy Chapter

Calling add() with a node whose node_id already exists replaces the existing entry. Matches LlamaIndex user expectation when re-indexing the same chunks.

A node_id repeated within a single add() batch raises ValueError — deduplicate before calling. (This differs from the LangChain and Haystack stores, which silently keep the last occurrence; here it's a hard error so an accidental duplicate doesn't quietly drop a node.)

node = TextNode(text="v1", embedding=[...])
vector_store.add([node])

# Same node_id, different text/embedding → replaces.
updated = TextNode(text="v2", id_=node.node_id, embedding=[...])
vector_store.add([updated])
assert len(vector_store._index) == 1

Async Copy Chapter

Every public method has an async counterpart, suitable for use in LlamaIndex's async retriever / query-engine paths:

await vector_store.async_add(nodes)
result = await vector_store.aquery(VectorStoreQuery(...))
fetched = await vector_store.aget_nodes(node_ids=[...])
await vector_store.adelete("ref-doc-id")
await vector_store.adelete_nodes(node_ids=[...])
await vector_store.aclear()

They run the index work on a worker thread (asyncio.to_thread), so the event loop stays responsive while a large add or query is in flight. BasePydanticVectorStore's defaults call straight into the sync body, which would block the loop for the operation's full duration.

Cancellation is only partial, and the distinction matters:

• asyncio.wait_for, task.cancel(), or a client disconnect returns control to the awaiting caller promptly — that part now works, where previously the coroutine ran to completion and the timeout never fired.
• It does not decide what happened to the add. If the worker thread had already started, it runs the call to completion — work inside the Rust core is not interruptible at all — and the add commits in full. If the executor was saturated, the call is cancelled before it ever starts and nothing is added. A cancelled async_add is "outcome unknown": it may have fully committed, or may never have begun. Re-adding the same node_id is an overwrite, so retrying is safe either way.
• What is guaranteed: the outcome is all-or-nothing. The store is never left in a torn state.
• Timing out does not make the work go away: the loop's shutdown (asyncio.run on the way out, or loop.shutdown_default_executor()) waits for the worker thread, so a process that exits right after a short timeout can still block for the rest of the in-flight call.

Persist / load Copy Chapter

Direct (file-stem) interface Copy Section

vector_store.persist("./store/vectors.json")
# ... later ...
vector_store = TurboQuantVectorStore.from_persist_path("./store/vectors.json")

persist_path is treated as a path stem — the binary index and JSON side-car are written next to each other as {stem}.tvim and {stem}.nodes.json. The extension on persist_path (e.g. .json, as LlamaIndex's StorageContext default uses) is replaced. Node metadata must be JSON-serializable. If the {stem}.nodes.json side-car is out of sync with its {stem}.tvim index (a partial copy, a stale backup, tampering), from_persist_path raises a ValueError immediately rather than failing later with a KeyError at query time.

persist is atomic with respect to the destination: both files are written to sibling temp files and moved into place, so a failed persist (e.g. non-JSON-serializable metadata) leaves a store previously persisted at the same stem intact.

The similarity mode is recorded in {stem}.nodes.json and restored by from_persist_path. A store persisted before the mode field existed holds raw, unnormalized vectors, so it loads in "dot_product" mode — exactly the scoring it was written under — with no migration needed.

Via `StorageContext` Copy Section

The store works with StorageContext.from_defaults(persist_dir=...) the same way SimpleVectorStore does:

# Persist
storage_context.persist(persist_dir="./store")

# Load
vector_store = TurboQuantVectorStore.from_persist_dir(persist_dir="./store")
storage_context = StorageContext.from_defaults(
    vector_store=vector_store,
    persist_dir="./store",
)

from_persist_dir(persist_dir, namespace="default", fs=None) constructs the namespaced filename ({persist_dir}/{namespace}__vector_store.json) and delegates to from_persist_path. Multiple namespaced stores can share a persist directory — including dotted namespaces (v1.2, v1.3), which map to distinct file pairs (v1.2__vector_store.tvim / v1.2__vector_store.nodes.json, and so on). namespace names a store within persist_dir, so it must be non-empty and must not contain path separators, .., or : (a Windows drive-relative name like C:foo would escape persist_dir); such a value raises ValueError. Any other string (alphanumerics, dash, underscore, dots) is accepted.

A store persisted by an older turbovec under a dotted namespace sits on disk under a truncated filename (v1.tvim for namespace v1.2). Loading finds it via a legacy-filename fallback — used only when the correct filename is absent — and the next persist writes the correct filenames.

Config-only round-trip Copy Section

config = vector_store.to_dict()                # {"bit_width": 4, "dim": 1536, "similarity": "cosine"}
fresh = TurboQuantVectorStore.from_dict(config)                   # empty store with the same config

to_dict / from_dict serialize only the store's configuration. Node data round-trips through persist / from_persist_path.

The store also supports pickle with full data fidelity (e.g. for multiprocessing workers) and copy.copy / copy.deepcopy — both copies return a fully independent store (there is no shallow copy that shares the underlying index).

Thread safety Copy Chapter

The store is safe for concurrent multi-threaded use:

• Reads run concurrently and scale. query and get_nodes take no lock; the underlying index releases the GIL during scoring, so independent queries from multiple threads overlap and scale.
• Writes serialize. add, delete, delete_nodes, clear, and persist serialize on a per-store lock. The async_add / a* variants delegate to the same locked bodies, so concurrent adds issue unique handles — no batch is ever rejected or lost to a handle collision.
• A read overlapping a write sees pre- or post-write state — never a torn one. Under heavy concurrent churn a query may transiently return fewer than similarity_top_k results (hits deleted mid-query are skipped).

What the contract does not cover:

• No cross-call atomicity. A caller-side check-then-act sequence (get_nodes then delete_nodes) can interleave with other writers. Batch writes are not atomic with respect to readers: a query overlapping a re-add of an existing node_id can briefly see that id under both its old and new entry.
• persist serializes with writes (so it always snapshots a consistent store); reads may proceed during a persist.
• Two stores writing to the same path is safe. Concurrent persist calls to one destination from several threads each publish atomically and the last writer wins; a caller never sees a torn file, and never an error caused only by the other writer. Which writer wins is not defined.
• Multi-process access is not supported.

Known limitations Copy Chapter

• MMR is not supported. Max-marginal-relevance retrieval requires the full-precision embedding of each candidate to compute pairwise diversity; turbovec discards full-precision vectors after quantization.
• get(text_id) raises rather than returning a vector — same reason. The full-precision embedding is not recoverable.
• fsspec filesystems are not supported. persist, from_persist_path, and from_persist_dir accept a local path. Pass fs=None (the default).
• JSON-serializable metadata only. Node metadata is stored as JSON in the side-car. Non-JSON-serializable values fail at persist time — same constraint as SimpleVectorStore.persist.
• stores_text = True. Unlike SimpleVectorStore, we keep node text in the side-car so query results return populated TextNodes without depending on a separate docstore. If you're swapping this in for SimpleVectorStore and your pipeline expects text to live elsewhere, the difference is harmless — the framework treats stores_text as informational.

File: docs/api.md Copy Chapter

API Reference

turbovec exposes two index types and one serialization format per type.

• TurboQuantIndex — positional index, O(1) swap_remove delete.
• IdMapIndex — stable external u64 ids on top of TurboQuantIndex.
• TQ+ calibration — the per-coordinate calibration lifecycle.
• File formats — .tv and .tvim, plus incremental saves.

All examples below are Python. The Rust API mirrors it closely (exceptions noted below) — see each type's rustdoc for the exact signatures.

`TurboQuantIndex` Copy Chapter

Positional index. Each vector is identified by its insertion slot (0..n). Fast and small, but external references to slots are invalidated by swap_remove. If you need stable ids, use IdMapIndex.

from turbovec import TurboQuantIndex

idx = TurboQuantIndex(dim=1536, bit_width=4)
idx.add(vectors)                        # np.ndarray of shape (n, dim), float32
scores, indices = idx.search(queries, k=10)

idx.swap_remove(5)                      # O(1); the previously-last vector moves into slot 5

idx.write("index.tv")                   # .tv format
loaded = TurboQuantIndex.load("index.tv")

dim is optional. Omit it to let the index pick up the dimensionality from the first batch of vectors:

idx = TurboQuantIndex(bit_width=4)      # dim inferred on first add
idx.add(vectors)                         # locks dim to vectors.shape[1]

Before the first add, idx.dim is None, len(idx) is 0, and search() returns empty results. Adding a zero-row batch is a no-op: dim is still checked against the batch, but a lazy index stays lazy and its serialized bytes are unchanged. (On the Rust API, dim_opt() is the equivalent of idx.dim and returns Option; dim() is deprecated — it returns usize with 0 for a lazy index, which is unsafe to do arithmetic with, so use dim_opt() on any path that can see one.)

Methods Copy Section

`swap_remove` semantics Copy Section

swap_remove(i) is named to match Rust's Vec::swap_remove: the last element moves into slot i, and the vector is truncated by one. It is not a shift — the slots after i do not move down by one. Order is not preserved; slot indices of vectors you didn't delete may now point at different vectors than before.

Use IdMapIndex if external references have to stay stable across deletes. Search masks are external references too — see A mask is invalidated by any mutation.

Low-level construction from raw parts (Rust) Copy Section

Rust embedders that hold an index payload already in memory — e.g. read out of a database page instead of a .tv file — can construct an index directly from its decoded fields with TurboQuantIndex::from_parts, skipping the file round-trip:

let index = TurboQuantIndex::from_parts(
    dim_opt,        // Option: Some(dim) committed, or None for lazy
    bit_width,      // 2, 3, or 4
    n_vectors,
    packed_codes,   // Vec
    scales,         // Vec<f32>
    tqplus_shift,   // Vec<f32> (length dim, or empty = identity)
    tqplus_scale,   // Vec<f32> (length dim, or empty = identity)
)?;

It is the single validated entry point for raw-part construction: every structural invariant is checked once and any violation returns a named FromPartsError (bit_width range, dim a positive multiple of 8 and ≤ 16384, packed_codes / scales / TQ+ lengths with overflow-checked size math, the lazy-state constraints, and the same value-level validation as the file loader — finite non-negative per-vector scales, finite TQ+ shifts, finite positive TQ+ scales) rather than panicking or reading out of bounds. An index accepted by from_parts therefore always survives its own write → load round-trip. The paired accessors packed_codes(), scales(), tqplus_shift(), tqplus_scale(), bit_width(), dim_opt() and len() return the fields it consumes, so an index round-trips through your own storage format. The per-coordinate encode / pack / search / codebook kernels are crate-internal — from_parts is the supported low-level API. (Rust only; the Python binding uses write / load.)

`IdMapIndex` Copy Chapter

Stable-id wrapper around TurboQuantIndex: a hash-table-backed u64 id ↔ slot mapping, with O(1) remove(id). Slot indices still move when a vector is removed, but ids do not.

import numpy as np
from turbovec import IdMapIndex

idx = IdMapIndex(dim=1536, bit_width=4)
idx.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

scores, ids = idx.search(queries, k=10)   # ids are uint64 external ids

idx.remove(1002)                           # O(1) by id
assert 1003 in idx                         # __contains__ sugar

idx.write("index.tvim")                    # .tvim format
loaded = IdMapIndex.load("index.tvim")

As with TurboQuantIndex, dim is optional and gets inferred from the first add_with_ids call:

idx = IdMapIndex(bit_width=4)            # dim inferred on first add
idx.add_with_ids(vectors, ids)           # locks dim to vectors.shape[1]

Methods Copy Section

When to use which Copy Section

• TurboQuantIndex — you never delete, or you're fine with positional ids.
• IdMapIndex — you need stable external ids (e.g. string-id → vector mapping maintained by the caller).

All the framework integrations (LangChain, LlamaIndex, Haystack) use IdMapIndex internally for exactly this reason.

TQ+ calibration Copy Chapter

TQ+ fits a per-coordinate (shift, scale) pair from the empirical quantiles of a sample of your vectors, and every stored vector is encoded in that one calibrated coordinate system. It is worth roughly +2.5 points of R@10 on average, and up to ~8.7 on the most anisotropic data measured.

The calibration comes from exactly one place: an explicit idx.calibrate(sample) call (calibrate / calibrate_2d in Rust). The index never fits one on its own — an index that is never calibrated is plain TurboQuant, with no fitted state anywhere, and its encoded bytes are independent of how adds were batched or ordered. idx.calibration_state reports which of the two states an index is in:

The sample is your responsibility. calibrate uses every row you give it. Around 1024 rows gets within half a point of R@10 of a fit on the entire corpus on most corpora measured, and 2048 does so everywhere — but it must be a representative, random sample of the vectors the index will hold. A sorted or clustered prefix of the same size fits quantiles that are shifted and far too narrow, and actively destroys recall. Passing the whole corpus is always safe.

calibrate may be called at any time and repeatedly. On a populated index it re-encodes every stored row from its stored codes — no original vectors needed. Know what that re-encode can and cannot do:

• Refitting with the same or a nearby pair is free: the codes reach an exact fixed point.
• Calibrating after a large uncalibrated ingest costs several points of recall versus calibrating first (the re-encode is a second quantization). Calibrate before adding when you can.
• A badly biased earlier calibration cannot be repaired by refitting: its too-narrow fit clipped coordinates to the outer centroids at encode time, and no later pair recovers what clipping destroyed. Rebuild from the source vectors.

The calibration round-trips exactly through write/load, to_bytes/from_bytes, pickling and copying, on both index types. Draining an index to zero vectors keeps its committed calibration.

Filtering Copy Chapter

Both index types support restricting the returned top-k to a caller-supplied subset of vectors. Unlike post-filtering (search then drop), the kernel never inserts disallowed vectors into the per-query heap, so you always get up to k results from the allowed set rather than fewer.

# IdMapIndex — allowlist of external ids (typical use)
allowed = np.array([1003, 1010, 1042], dtype=np.uint64)
scores, ids = idx.search(queries, k=10, allowlist=allowed)
# scores.shape == (nq, min(k, n_allowed)) == (nq, 3)   # 3 unique allowed ids

# TurboQuantIndex — bool mask over slots
mask = np.ones(len(idx), dtype=bool)
mask[disabled_slots] = False
scores, slots = idx.search(queries, k=10, mask=mask)

The output shape is (nq, min(k, n_allowed)), where n_allowed is the number of distinct allowed vectors — unique ids in the allowlist, or mask.sum() for a mask — the same shrinking behaviour you already see when k > len(idx). No -1 / NaN padding; pad on the caller side if you need a fixed-width batch.

A mask is invalidated by any mutation, not just a length-changing one Copy Section

A mask names slots, and swap_remove renumbers slots — so any mutation invalidates a mask, including one that leaves len(idx) unchanged. Rebuild the mask after every mutation.

The length check is not what protects you. It catches only a size difference (ValueError: mask length 100 does not match index size 99); a swap_remove(i) + add(...) pair restores the original length while leaving a different vector in slot i, so a mask built before that pair passes validation and then silently selects a different set of vectors than you intended. Nothing outside the index leaks and no error is raised — the selected set is simply the wrong one.

Allowlists on IdMapIndex do not have this failure mode, because they name external ids and the index never renumbers an id. An allowlist entry for an id that has since been removed raises KeyError (Rust: SearchError::UnknownId) instead of quietly resolving to some other vector. The one way an allowlist entry can come to name a different vector is if you re-add that same id yourself with different data.

Common use cases:

• Hybrid retrieval where a SQL/BM25 stage produces a candidate id set.
• Access control or multi-tenant queries (only return ids the caller can see).
• Time-windowed search (e.g. only documents from the last 7 days).

Two-stage 2-bit search Copy Chapter

A 2-bit index of 32,768 vectors or more searches in two stages. Nothing changes on disk: the file is the same whichever way it is searched. TURBOVEC_2BIT_PLANES=0 in the environment before the process first searches keeps the whole-index exact scan instead; it is read once per process.

A 2-bit code is a sign bit and a magnitude bit per coordinate. The in-memory search cache holds the two bits apart — the same bytes per vector, arranged differently — and a search

1. scans the sign bits alone (half the bytes of a full scan) for a shortlist of max(384, 16 × k) candidates — on a calibrated index the scan ranks against the query pulled halfway toward the coordinates' centre, which is what keeps the shortlist honest on corpora whose vectors share a strong common direction (see the MedCPT row below), and each candidate carries its plain score on,
2. ranks the shortlist with an estimate that adds the magnitude bits (a bit-count against the query rounded to 6 bits per coordinate), and
3. rescores the best max(64, 2 × k) with the exact scan's own arithmetic.

Scores are exact; the candidate set is approximate. Every returned score is bit-identical to what the whole-index scan returns for that id. What can differ is which ids are returned: a vector whose sign bits alone rank it outside the shortlist is not seen. How often that happens depends on the data:

Recall against float ground truth on the OpenAI corpora is unchanged at every k the benchmark suite reports. On random vectors — where a query has no real neighbours and the top-k is decided by noise-sized margins — the true nearest neighbour is in the top 10 for 74% of queries at d=768, against 86% with the whole-index scan. Check agreement on your own data if it is unlike the corpora above, and set TURBOVEC_2BIT_PLANES=0 if the whole-index scan's id set is what you need.

When it is faster. The first stage is cheaper than a full scan, and the later stages cost more as k grows, so the gain is largest for small k. Milliseconds per query on 100K OpenAI d=1536 vectors, 1,000 queries, whole-index scan → two-stage:

Through k=100 every cell is faster or within 4% of the whole-index scan; the cells at parity are multi-threaded single queries and x86 batches at the largest k.

What scans the whole index instead. Indexes below 32,768 vectors (an index that grows past the threshold switches then, and keeps the layout if it later shrinks); dimensions that are not a multiple of 32; x86 CPUs without AVX-512 VBMI and VNNI. Filtered searches use the two-stage path with a plain top-shortlist heap.

Staged 4-bit search Copy Chapter

A 4-bit index of 32,768 vectors or more searches in stages. Nothing changes on disk: the file is the same whichever way it is searched. TURBOVEC_4BIT_PLANES=0 in the environment before the process first searches keeps the whole-index exact scan instead; it is read once per process.

A 4-bit code is a sign bit and three lower bits per coordinate. The in-memory search cache holds the four bit planes apart — the same bytes per vector, arranged differently — and a search

1. scans the sign bits alone (a quarter of the bytes of a full scan) for a shortlist of max(256, 20 × k) candidates (16 × k from k = 64) — on a calibrated index the scan ranks against the query pulled halfway toward the coordinates' centre, which is what keeps the shortlist honest on corpora whose vectors share a strong common direction (see the MedCPT row below), and each candidate carries its plain score on,
2. ranks the shortlist with an estimate that adds the next bit plane and keeps the best max(256, 6 × k),
3. ranks those with all three lower planes, and
4. rescores the best max(64, 1.5 × k) with the exact scan's own arithmetic.

Scores are exact; the candidate set is approximate. Every returned score is bit-identical to what the whole-index scan returns for that id. What can differ is which ids are returned: a vector whose sign bits alone rank it outside the shortlist is not seen. On the corpora below the first stage keeps a wide margin:

The MedCPT corpus (#562) is the bunched case: mean pairwise cosine 0.65, a shared direction of norm 0.80, and a median gap between a query's first and second neighbour four times smaller than OpenAI's. Its ids differ from the whole-index scan's more often, but almost every difference is a swap at the list's edge: recall is within 0.1% of the scan's at every k and at both bit widths, and the true nearest neighbour is in the 4-bit top 10 for every query measured. Recall against float ground truth on the OpenAI corpora is unchanged at every k the benchmark suite reports. Structureless random vectors are a different matter: with no real neighbours, a shortlist of sign bits misses the exact top-k for most queries. Check agreement on your own data if it is unlike the corpora above, and set TURBOVEC_4BIT_PLANES=0 if the whole-index scan's id set is what you need.

How much faster. Milliseconds per query on 100K OpenAI d=1536 vectors, whole-index scan → staged; batches are 1,000 queries in one call:

Every cell is faster: one query per call 2.25x–4.57x, batches 1.09x–1.84x (harmonic mean over the 32 cells 1.87x). A single query is where the full 4-bit scan is dearest, so that is where the staged search gains most; a batch already shares each block's bytes across queries, so its gain is the quarter-width first stage alone, and it shrinks as k grows. On 100K OpenAI d=3072 the same cells read 1.10x–5.65x (harmonic mean 1.83x on ARM, 2.27x on x86).

Small indexes. The shortlist is sized by k, not by the index, so on an index of 40K vectors a k=100 shortlist is 4% of it and the ranking stages cost a batch more than the shorter scan saves: measured on 40K vectors (OpenAI d=1536 cut down, and all-mpnet-base-v2 d=768 at 41K), batches at k=100 run 0.80x–1.04x and at k=64 0.88x–1.15x of the whole-index scan, while every other cell is faster (one query per call 1.3x–3.7x; harmonic mean 1.3x–1.5x over the 16 cells). From about 64K vectors every cell is ahead.

What scans the whole index instead. Indexes below 32,768 vectors (an index that grows past the threshold switches then, and keeps the layout if it later shrinks); dimensions that are not a multiple of 32; x86 CPUs without AVX-512 VBMI and VNNI. Filtered searches use the staged path with a plain top-shortlist heap.

File formats Copy Chapter

`.tv` — `TurboQuantIndex` Copy Section

┌───────────────────────────────────────────┐
│ magic    "TVPI"  (4 bytes)                │
│ version  u8    = 6                        │
├───────────────────────────────────────────┤
│ core header                               │
│   bit_width    (u8)                       │
│   dim          (u32 LE)                   │
│   n_vectors    (u64 LE)                   │
├───────────────────────────────────────────┤
│ Lloyd-Max codebook                        │
│   boundaries  ((2^bit_width − 1) × f32 LE)│
│   centroids   (2^bit_width × f32 LE)      │
├───────────────────────────────────────────┤
│ codes — sequential blocked layout         │
│   n_byte_groups = dim / (8 / bit_width)   │
│   ceil(n_vectors / 32)                    │
│     × n_byte_groups × 32 bytes            │
├───────────────────────────────────────────┤
│ scales  (n_vectors × f32 LE)              │
│   per-vector length-renormalization       │
├───────────────────────────────────────────┤
│ TQ+ trailer                               │
│   n_calib  (u32 LE)  — 0 or dim           │
│   shift    (n_calib × f32 LE)             │
│   scale    (n_calib × f32 LE)             │
└───────────────────────────────────────────┘

The code payload is grouped into 32-vector blocks and padded up to a
whole block, so it is ceil(n_vectors / 32) * 32 vectors wide on disk.
At bit_width = 3 a byte holds only two codes rather than 8/3, making
the payload ~33% larger than dim * bit_width / 8 per vector would
suggest.

`.tvim` — `IdMapIndex` Copy Section

┌───────────────────────────────────────────┐
│ magic    "TVIM"  (4 bytes)                │
│ version  u8    = 6                        │
├───────────────────────────────────────────┤
│ core payload (same as .tv: header +       │
│   codebook + codes + scales + TQ+)        │
├───────────────────────────────────────────┤
│ slot_to_id  (n_vectors × u64 LE)          │
└───────────────────────────────────────────┘

On load, the reverse id → slot map is rebuilt in memory. Duplicate ids in the slot_to_id table are rejected as corrupt.

The v8 container Copy Section

What write(), to_bytes() and sync() produce today is the v8 container (magic TV8\0), the v7 container with one more superblock byte:

┌───────────────────────────────────────────┐
│ superblock                                │
│   magic     "TV8\0"                       │
│   revision  u8 = 1                        │
│   bit_width u8 · kind u8 (0 .tv, 1 .tvim) │
│   layout    u8 — 0 sequential, 1 planes   │
│   dim u32 · nonce u64 · ops capacity u32  │
│   codebook, TQ+ calibration, CRC          │
├───────────────────────────────────────────┤
│ header slot A │ header slot B  (alternate │
│   commits: generation, n, the partial     │
│   tail block's rows, pending redo ops,    │
│   delta digest, CRC)                      │
├───────────────────────────────────────────┤
│ block units — one per whole 32-row block  │
│   codes in `layout`, 32 scales, 32 ids    │
│   (.tvim)                                 │
└───────────────────────────────────────────┘

A unit's codes are the block's sequential-blocked rows (layout 0, as every v7 unit) or its bit planes (layout 1): the sign plane first, one byte per group of eight coordinates for each of the 32 rows, group-major, then the 32 rows' lower planes, least significant first. The two forms are the same number of bytes, so every offset in the file is the same whichever a unit holds. An index of 32,768 vectors or more keeps its search cache as bit planes (see the two-stage and staged searches above), and writes layout 1; smaller indexes, 3-bit indexes and dimensions that are not a multiple of 32 write layout 0. Either loads anywhere: a planes file on a host that scans the whole index (x86 without AVX-512, or under TURBOVEC_*BIT_PLANES=0) converts on load, as a sequential file converts to planes on a host that uses them. The planes form is arch-neutral; x86 permutes the sign bytes in place at load.

A sync() into a v7 file rewrites it as v8 once, as it would for a calibration change, and continues incrementally from there.

In-memory serialization Copy Section

Both index types (de)serialize their wire format in memory, without a filesystem round-trip:

payload = idx.to_bytes()                  # bytes, byte-identical to write(path)'s file
restored = IdMapIndex.from_bytes(payload) # same validation as load(path) on a write() file

to_bytes() returns exactly the bytes write(path) would put in the file (.tv for TurboQuantIndex, .tvim for IdMapIndex). from_bytes(data) accepts bytes or bytearray and applies exactly the same validation load applies to a write() file — version handling, structural and value-level checks, the embedded codebook check (a v6 file carries the Lloyd-Max codebook, and a file whose codebook is not a valid one for its (bit_width, dim) is rejected — the relevant case for anyone hand-writing files through the raw io::* writers), and the .tvim duplicate-id check — raising ValueError on a corrupt payload (there is no file to blame, so it is not an OSError). Both release the GIL. This is the path to use for caches and database columns, and it is what both index types' own pickle / copy support and the integration stores' are built on.

pickle.dumps(idx), copy.copy(idx) and copy.deepcopy(idx) work on both index types — they reduce to from_bytes(to_bytes()), so an index can cross a multiprocessing spawn boundary (the default start method on macOS and Windows) and a container holding one can be deep-copied. The copy is fully independent of the original. Everything true of to_bytes is therefore true of a pickle — in particular the calibration state round-trips exactly.

Equality and hashing stay identity-based, so idx == pickle.loads(pickle.dumps(idx)) is False even though the two hold the same vectors. Compare to_bytes() payloads to check that a saved and a loaded index agree.

An index defines no __bool__, so truthiness falls back to __len__: an index holding no vectors is falsy, and idx = idx or build_index() therefore discards a perfectly good empty index. Test for an index with idx is None, and for its contents with len(idx).

An index also takes no user attributes (idx.tag = "x" raises AttributeError) and is not subclassable (class Sub(IdMapIndex): ... raises TypeError). Both are deliberate. A per-instance __dict__ is not traversed by the garbage collector, so a reference cycle running through an attribute would leak the whole index rather than being collected, and the attributes would be silently dropped by pickle / copy, which reduce through from_bytes(to_bytes()) and carry the payload only. A subclass instance would likewise pickle and copy back to the base class, silently changing type. Attach per-index state by holding the index in an object of your own instead. Re-invoking idx.__init__(dim=...) on a built index does nothing at all — it neither resets nor re-shapes it — so build a new index instead of trying to reconfigure one in place.

On the Rust API the same pair exists as to_bytes() / from_bytes(&[u8]), alongside generic-sink forms write_to_writer<W: Write> / load_from_reader<R: Read> on both types. All four move the same v7 image the file writers produce: to_bytes() is byte-identical to the file write() leaves on disk, and write_to_writer streams it unit by unit rather than building a second copy in memory. The raw module-level io::write* / io::load* entry points were removed when v7 became the only format — build an index and use its own methods instead.

On the Rust API TurboQuantIndex::serialized_len() returns the exact byte count to_bytes() will return and write(path) will put in the file, computed from the index's geometry without serializing anything — for sizing a buffer, a database column or a quota check ahead of time. It is exact rather than an upper bound, and to_bytes() uses it to allocate its buffer once.

Incremental saves — `sync()` Copy Section

write(path) rewrites the whole file. sync(path) writes only what changed since the last sync to the same path, in a second container format (.tv / .tvim magic TV7\0) built for repeated small commits:

idx.sync("index.tv")     # first sync to a fresh path: writes the whole container
idx.add(more_vectors)
idx.swap_remove(3)
idx.sync("index.tv")     # writes the delta and commits
reloaded = TurboQuantIndex.load("index.tv")   # load() recognises both formats

A loaded index stays bound to the path it came from, so it keeps syncing forward incrementally rather than rewriting.

What it costs. An append writes the new 32-row blocks plus a commit header. A removal writes no block at all — it rides the commit header as a redo op, and a later sync folds it into its block. The whole-file events are an explicit calibrate() (a refit re-encodes every stored code) and enough accumulated removals to exceed the header's op capacity; both compact the file by rewriting it whole, through the same temp-file-and-rename path write() uses.

Durability. Every sync is durable when it returns — there is no fast mode, and the fsync is sync_all, not a data-only variant. A crash at any byte leaves the previous commit intact: the file carries two alternating commit headers, so a torn header fails its checksum and the other one is adopted, and each header names the blocks its own sync wrote along with their digest, so a commit that reached disk ahead of its data is detected rather than served. See Versioning and limits for what this does not cover — damage arriving after the write.

One writer per path. Each full write stamps the file with a random nonce, so if another process replaces the file underneath a bound index, the next sync reports it rather than writing over their commits. Two processes syncing one path concurrently is not supported; the check makes the unsupported case loud, not safe.

sync() files are path-only. from_bytes and load_from_reader read the write() format. A v7 container needs random access — two header slots, fixed-stride block units, redo ops — which a byte stream cannot serve, and to_bytes() only ever emits the write() format. Handing v7 bytes to from_bytes raises an error saying so and pointing at load(path).

Load performance Copy Section

The file stores the codes in the layout the search kernels consume — the arch-neutral sequential blocked rows, or, for an index of 32,768 vectors or more, its bit planes — plus the Lloyd-Max codebook, so a load seeds the search caches directly: there is no O(n·dim) repack and no codebook solve on first search. Non-x86 uses the stored layout as-is; x86 applies one cheap in-block permutation at load (a threaded pass, a few milliseconds for a 77 MB index). A v7 file, which holds the sequential rows only, converts to the planes on load (block-parallel; ~20 ms for 100K × 1536 at 4 bits with the pool, ~80 ms on one thread) and is rewritten as v8 by its next sync(). The rotation is deterministic and rebuilt from dim in well under a millisecond. A stored index survives cross-platform load → re-save byte-identically; the format itself adds no platform dependence.

Versioning and limits Copy Section

Both .tv and .tvim loads validate the header before allocating: bit_width must be 2/3/4, dim a positive multiple of 8 and ≤ 16384 (MAX_DIM — the same cap enforced at construction, so any index this build can create it can also load back), and every payload size is computed with checked arithmetic and read through a length-capped reader. A malformed or untrusted file therefore raises a clean error rather than panicking, dividing by zero, or driving an oversized allocation. Codebook, scale, and calibration values are additionally validated at the value level (finite, in-support), so a structurally valid file carrying out-of-range values in those fields is rejected rather than loaded.

What that validation does not give you is integrity checking of the payload. Neither format checksums its stored codes, and the value-level checks only reject values that leave the valid range — a flipped mantissa bit in a scale is still a finite positive float, and a flipped code byte is indistinguishable from a legitimate one. Damage arriving from outside the writer (a failing disk, a truncated copy, a bad transfer) therefore loads clean and changes search results silently.

Measured by flipping every one of the 32,912 bits of a 4114-byte .tv file in turn and loading each result: 1460 flips (4%) are rejected and 31,452 (96%) load and return a different index. By section, rejections are 144 of 144 header bits, 988 of 992 codebook bits, 169 of 3072 scale bits, 159 of 4128 calibration-trailer bits, and 0 of 24,576 code bits. The scale and trailer figures are the value-level checks doing exactly what they claim and no more: a flip that drives a float non-finite, negative, or out of support is caught, and every mantissa flip is not. The codes carry no validation of any kind, which is why that column is zero and why it is also the largest section.

This is a deliberate scope choice, not an oversight. A save is atomic and a crash mid-write leaves the previous file intact, so the writer cannot leave a torn index behind; what is out of scope is damage that arrives afterwards. If you need to detect that, checksum the file yourself or store it on a filesystem that does.

n_calib = 0 in the TQ+ trailer means an uncalibrated index; otherwise it equals dim. v7 and v8 are read: a v5 or v6 file is refused with an error naming its version, and turbovec::convert moves a file between v5, v6, v7 and v8 in any direction (cargo run --example convert -- <out> v8); converting a v8 planes file down re-containers the same codes, never re-quantizes. Versions 1 through 4 predate the v5 rotation change and cannot be decoded at all — their codes were encoded under a rotation this build cannot reproduce — so they must be rebuilt from the source vectors.

dim = 0 in the core header signals a lazy uncommitted index. It is only valid alongside n_vectors = 0; on load it produces an index whose dim is None until the first add / add_with_ids call.

Both formats carry a magic + version byte and are stable across minor versions. Breaking changes bump the version byte. write(), to_bytes() and sync() all produce v8: write() and to_bytes() an unclaimed snapshot, sync() a container it claims and then updates incrementally (see Incremental saves). v8 files are not readable by earlier turbovec releases, whose loaders reject the magic rather than misparse it; turbovec::convert writes a v7, v6 or v5 file for one.