{"owner":"microsoft","repo":"graphrag","hasSkills":true,"totalSkillsCount":12,"totalTokensCount":8142,"categories":["anthropic-skill","plugin-manifest"],"hasMcp":false,"mcpConfig":null,"found":[".agents/skills/update-deps/SKILL.md",".agents/skills/update-deps/references/migration-gotchas.md","packages/graphrag-cache/README.md","packages/graphrag-chunking/README.md","packages/graphrag-common/README.md","packages/graphrag-input/README.md","packages/graphrag-llm/README.md","packages/graphrag-llm/graphrag_llm/README.md","packages/graphrag-llm/notebooks/README.md","packages/graphrag-storage/README.md","packages/graphrag-vectors/README.md","packages/graphrag/README.md"],"skills":{".agents/skills/update-deps/SKILL.md":"---\nname: update-deps\ndescription: >\n Update this GraphRAG uv-workspace monorepo's dependencies and repair the resulting\n breakages so every check and test passes again. Use when asked to update, upgrade, or\n bump dependencies/packages/versions, refresh or regenerate the lockfile, migrate to a new\n major of pandas/numpy/pydantic/pyarrow, resolve dependency-related test or lint failures,\n or run a \"dependency sweep\" — even if the user only names a single package. Covers editing\n pyproject.toml version specifiers across the root dev-deps and every packages/* member,\n re-locking with uv, and fixing code and tests for library API changes (e.g. pandas 3.0).\n USE FOR: update dependencies, upgrade packages, bump versions, dependency sweep, uv lock,\n uv sync, refresh lockfile, migrate pandas/numpy, \"deps broke the tests\", pyproject bump.\nuser-invocable: true\n---\n\n# Update Dependencies (GraphRAG monorepo)\n\n## Goal\n\nRaise dependency versions across this uv workspace, re-lock, and fix any code or test\nfallout until `uv run poe check` and `uv run poe test_unit` are both green — without\ntouching the version machinery that the release process owns.\n\n## Hard rules (non-negotiable)\n\n1. **Only the Microsoft feed proxy.** All resolves and syncs MUST use\n `https://packagefeedproxy.microsoft.io/pypi/simple` (the `[[tool.uv.index]]` configured in\n the root `pyproject.toml`). Never resolve against public PyPI, never add\n `--index`/`--default-index`/`--index-url` overrides, and never set `UV_INDEX_URL` or\n `PIP_INDEX_URL` to pypi.org. Do not disable or reorder the configured index.\n2. **Target versions must be at least 7 days old.** Only bump to a version that was released\n more than 7 days ago. The proxy does not serve versions published within the last week —\n syncing to a too-new version WILL fail. Before pinning a specific version, verify its\n release date and pick the newest release older than 7 days; otherwise let the specifier\n float and let `uv lock` choose (it can only see eligible versions on the proxy anyway).\n\n## Layout facts\n\n- This is a **uv workspace monorepo**. The root [`pyproject.toml`](../../../pyproject.toml)\n holds the `dev` group under `[dependency-groups]` and the `[tool.poe.tasks]` commands.\n- Runtime dependencies live in each member's `packages/*/pyproject.toml` under\n `[project] dependencies`.\n- Workspace members are wired via `[tool.uv.sources]` (`{ workspace = true }`) and pinned to\n each other with `graphrag-*==X.Y.Z` lines.\n- Package resolution goes through a Microsoft-internal index (`[[tool.uv.index]]`); expect\n that feed to be used, not public PyPI directly.\n\n## Do NOT edit these (release-owned)\n\n1. **Cross-package pins** `graphrag-cache==...`, `graphrag-llm==...`, etc. in any package.\n These are rewritten automatically by\n [`scripts/update_workspace_dependency_versions.py`](../../../scripts/update_workspace_dependency_versions.py)\n from the semversioner version. Hand-editing them causes drift.\n2. **`[project] version` fields** — managed by semversioner (\"do not change the version\n here manually\").\n3. **`graspologic-native>=1.2,<1.3`** — held below 1.3 on purpose; 1.3.x changes Leiden\n clustering output and breaks golden regression data. Only bump with a deliberate\n golden-data refresh, and say so explicitly.\n\n## Process\n\n1. **Baseline first.** Confirm a clean working tree and that checks/tests already pass\n before changing anything, so later failures are attributable to the bump:\n - `uv run poe check`\n - `uv run poe test_unit`\n Prefer a dedicated branch (e.g. `dep-sweep`).\n\n2. **Decide the scope.** Either a targeted set of packages the user named, or a full sweep.\n Edit the `~=`/`>=`/`<` specifiers in the relevant `[project] dependencies`\n (`packages/*/pyproject.toml`) and the root `dev` group. Leave the release-owned lines\n above untouched.\n\n3. **Resolve and lock.** All of these use the configured Microsoft feed proxy — do not pass\n any index override (see Hard rules).\n - For a full \"get latest allowed\" pass: `uv lock --upgrade`.\n - For targeted bumps after editing specifiers: `uv lock`.\n - Then install: `uv sync --all-packages`.\n If resolution fails, read the conflict, relax/adjust the offending specifier, and re-lock.\n Do not delete `uv.lock` to force it.\n - If a sync fails to find a version you just pinned, it is almost certainly younger than 7\n days on the proxy — step down to the newest release older than one week.\n\n4. **Static checks.** Run `uv run poe check` (this is `ruff format --check` + `ruff check` +\n `pyright`). Apply safe autofixes with `uv run poe fix`; format with `uv run poe format`.\n Fix remaining lint/type errors by hand — see Gotchas and the migration reference.\n\n5. **Tests.** Run `uv run poe test_unit` (NOT `uv run poe test`, which runs the full coverage\n suite). Run `uv run poe test_verbs` and `uv run poe test_integration` when the change is\n broad or touches indexing/query. Investigate every new failure.\n\n6. **Repair breakages.** For test/type failures caused by a library's API change, load\n [`references/migration-gotchas.md`](references/migration-gotchas.md) and apply the\n documented pattern. Keep fixes minimal and consistent with sibling code; prefer a real\n fix over a `# noqa`.\n\n7. **Record the change.** Add a changelog entry:\n `uv run semversioner add-change -t patch -d \"\"` (use `minor`/`major`\n only if the user's intent warrants it).\n\n8. **Final verification.** Re-run `uv run poe check` and `uv run poe test_unit`; both must be\n green (see the known-flake note below before calling a failure a regression).\n\n## Gotchas (this repo)\n\n- **`test_unit`, not `test`.** `poe test` runs coverage over everything and is slow; use\n `test_unit` for the fast feedback loop.\n- **Ruff runs in preview mode** (`preview = true`, `target-version = \"py310\"`). Preview-only\n rules such as `RUF069` (float equality) and `ASYNC119` fire here even though they may not\n in other repos.\n- **Known pre-existing flake:**\n `tests/unit/indexing/test_profiling.py::TestWorkflowProfiler::test_handles_exception_in_context`\n is timing-sensitive and can fail intermittently — it is not a dependency regression.\n- **`uv sync --all-packages`** (not bare `uv sync`) to install every workspace member.\n- **pandas is on the 3.0 line and numpy on 2.x.** Their major-version API changes are the\n usual source of post-bump breakage — see the reference file.\n- Version-bump edits touch many `pyproject.toml` files; make sure you did not accidentally\n modify a `graphrag-*==` pin or a `version` field while editing nearby specifiers.\n\n## Load-on-demand reference\n\nWhen a bump breaks tests or type-checking with a library API change (especially pandas or\nnumpy), read [`references/migration-gotchas.md`](references/migration-gotchas.md) for\nverified, repo-specific fix patterns before improvising.\n\n## Completion checklist\n\n- [ ] All resolves/syncs used the `packagefeedproxy.microsoft.io` index; no public-PyPI or\n index-override was introduced.\n- [ ] No dependency was bumped to a version released within the last 7 days.\n- [ ] Only intended specifiers changed; no `graphrag-*==` pin or `version` field edited.\n- [ ] `uv.lock` regenerated via `uv lock`/`uv lock --upgrade` (not hand-edited or deleted).\n- [ ] `uv run poe check` passes (ruff format, ruff lint, pyright).\n- [ ] `uv run poe test_unit` passes (ignoring only the known profiling flake).\n- [ ] Broader suites run if the change was broad (`test_verbs`/`test_integration`).\n- [ ] A semversioner changelog entry was added.\n- [ ] Any risky/held pin (e.g. `graspologic-native`) left in place unless explicitly bumped.\n",".agents/skills/update-deps/references/migration-gotchas.md":"# Migration Gotchas — verified fixes for this repo\n\nLoad this when a dependency bump breaks tests or `pyright`/`ruff` with a library API change.\nEach entry is a pattern that was actually hit and fixed in this codebase. Apply the minimal\nfix and keep it consistent with sibling code.\n\n## pandas 3.0\n\n### `DataFrame.swapaxes` removed → `np.array_split(df, n)` returns ndarrays\n\n`np.array_split` internally calls `np.swapaxes`, which used to delegate to\n`DataFrame.swapaxes` and return DataFrames with column names intact. In pandas 3.0\n`swapaxes` was removed (deprecated in 2.1), so `np.array_split(df, n)` now yields plain\nnumpy arrays. Rebuilding with `pd.DataFrame(fold)` produces integer `RangeIndex` columns,\nso later `df[\"some_column\"]` raises `KeyError`.\n\nSymptom: `KeyError: ''` with a traceback ending in\n`pandas/core/indexes/range.py ... get_loc`.\n\nFix — split positional indices instead of the frame, then select rows with `iloc`:\n\n```python\n# Broken under pandas 3.0\nreturn [pd.DataFrame(fold) for fold in np.array_split(reports, n)]\n\n# Fixed — preserves columns, dtypes, and even fold sizes\nreturn [reports.iloc[indices] for indices in np.array_split(np.arange(len(reports)), n)]\n```\n\n### `copy=` keyword removed from `merge`/`concat`/`join`/`set_axis` etc.\n\npandas 3.0 makes Copy-on-Write the default and drops the `copy=` parameter. Calls like\n`df.merge(other, copy=False)` or `pd.concat([...], copy=False)` raise `TypeError`.\nFix: delete the `copy=` argument — CoW already avoids the unnecessary copy.\n\n### Chained-assignment / `inplace` under Copy-on-Write\n\nWith CoW, mutating a slice (`df[mask][\"col\"] = x`) no longer writes back and may warn/error.\nAssign through `.loc`: `df.loc[mask, \"col\"] = x`. Reassign results of `inplace=True`-style\noperations rather than relying on in-place mutation of a view.\n\n## numpy 2.x\n\n- Removed aliases (`np.float_`, `np.int0`, `np.bool8`, `np.object0`, etc.) — use the builtin\n or the explicit sized dtype (`np.float64`, `np.bool_`).\n- `np.array_split` on a DataFrame no longer preserves the frame (see the pandas entry above).\n- Some functions moved out of the top-level namespace; import from the documented submodule.\n\n## ruff (preview mode active in this repo)\n\n- **RUF069 (float-equality-comparison):** `x == 0.0` / `!= 0.0` is flagged. Prefer a\n non-equality guard when semantically valid (`x <= 0.0` for a non-positive divide guard) or\n `math.isclose(...)` for tolerance checks. Avoid a blanket `# noqa` when a real fix exists.\n- **ASYNC119 (yield in context manager in async generator):** do not `yield` while holding a\n `with`/`async with` in an async generator. Materialize inside the block, then yield after\n it closes — matching the sibling providers:\n\n ```python\n with Path.open(path, \"r\", encoding=enc) as f:\n rows = list(csv.DictReader(f))\n for row in rows:\n yield transform(row)\n ```\n\n## pyright\n\n- Type stubs travel with majors: the `dev` group pins `pandas-stubs~=3.0`. When bumping\n pandas, bump the matching stubs so `pyright` reflects the new API.\n- After dependency changes, `pyright` may surface new optional/overload errors from updated\n stubs; fix at the call site rather than suppressing, unless the stub is demonstrably wrong.\n\n## General approach\n\n1. Read the actual traceback/diagnostic to the leaf frame — the failing library call and the\n changed symbol are usually right there.\n2. Check how a sibling module in the same package already handles the pattern and match it.\n3. Prefer a real, minimal fix over suppression; re-run `uv run poe check` and\n `uv run poe test_unit` after each fix to confirm.\n","packages/graphrag-cache/README.md":"# GraphRAG Cache\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of utilities to handle GraphRAG caching implementation.\n\n### Basic\n\nThis example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system.\n\n[Open the notebook to explore the basic example code](example_notebooks/basic_cache_example.ipynb)\n\n### Custom Cache\n\nThis example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.\n\n[Open the notebook to explore the basic custom example code](example_notebooks/custom_cache_example.ipynb)\n\n#### Details\n\nBy default, the `create_cache` comes with the following cache providers registered that correspond to the entries in the `CacheType` enum.\n\n- `JsonCache`\n- `MemoryCache`\n- `NoopCache`\n- `SQLiteCache`\n\n`SQLiteCache` stores JSON-serializable values in a local SQLite database. It uses\nWAL mode and short-lived connections so multiple workers can safely share the\nsame database. Configure it with:\n\n```yaml\ncache:\n type: sqlite\n storage:\n type: file\n base_dir: cache\n database_name: cache.db\n```\n\nThe database is created within the configured file storage. SQLite requires\nlocal random-access file operations, so blob and Cosmos storage are not\nsupported. Child caches use isolated namespaces within the same database.\n\nAll cache providers expose `await cache.get_many(keys)` for bulk reads. The\nresult maps each existing, valid key to the same decoded value returned by\n`get()`. Missing and malformed entries are omitted, duplicate input keys appear\nat most once, and an empty input returns an empty dictionary. The default\nimplementation remains compatible with custom `Cache` subclasses, while\n`MemoryCache`, `NoopCache`, and `SQLiteCache` provide backend-specific\nimplementations. SQLite bulk reads use one worker dispatch and one connection,\nsplitting large key sets into bounded SQL batches.\n\nAn opt-in warm-cache benchmark compares bulk reads across the memory, JSON, and\nSQLite providers without enforcing a machine-dependent performance threshold:\n\n```bash\nuv run pytest -s tests/unit/cache/test_cache_benchmark.py \\\n --run-cache-benchmark\n```\n\nThe benchmark reports median latency, entries per second, and time relative to\nSQLite for 1,000 entries. Setup time is excluded from the measurements.\n\nThe preregistration happens dynamically, e.g., `JsonCache` is only imported and registered if you request a `JsonCache` with `create_cache(CacheType.Json, ...)`. There is no need to manually import and register builtin cache providers when using `create_cache`.\n\nIf you want a clean factory with no preregistered cache providers then directly import `cache_factory` and bypass using `create_cache`. The downside is that `cache_factory.create` uses a dict for init args instead of the strongly typed `CacheConfig` used with `create_cache`.\n\n```python\nfrom graphrag_cache.cache_factory import cache_factory\nfrom graphrag_cache.json_cache import JsonCache\n\n# cache_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\ncache_factory.register(\"my_cache_impl\", JsonCache)\n\ncache = cache_factory.create(\n strategy=\"my_cache_impl\", init_args={\"some_setting\": \"...\"}\n)\n\n...\n```\n","packages/graphrag-chunking/README.md":"# GraphRAG Chunking\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of text chunkers, a core config model, and a factory for acquiring instances.\n\n## Examples\n\n### Basic sentence chunking with nltk\n\nThe SentenceChunker class splits text into individual sentences by identifying sentence boundaries. It takes input text and returns a list where each element is a separate sentence, making it easy to process text at the sentence level.\n\n[Open the notebook to explore the basic sentence example code](example_notebooks/basic_sentence_example.ipynb)\n\n### Token chunking\n\nThe TokenChunker splits text into fixed-size chunks based on token count rather than sentence boundaries. It uses a tokenizer to encode text into tokens, then creates chunks of a specified size with configurable overlap between chunks.\n\n[Open the notebook to explore the token chunking example code](example_notebooks/token_chunking_example.ipynb)\n\n### Using the factory via helper util\n\nThe create_chunker factory function provides a configuration-driven approach to instantiate chunkers by accepting a ChunkingConfig object that specifies the chunking strategy and parameters. This allows for more flexible and maintainable code by separating chunker configuration from direct instantiation.\n\n[Open the notebook to explore the factory helper util example code](example_notebooks/factory_helper_util_example.ipynb)\n","packages/graphrag-common/README.md":"# GraphRAG Common\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides utility modules for GraphRAG, including a flexible factory system for dependency injection and service registration, and a comprehensive configuration loading system with Pydantic model support, environment variable substitution, and automatic file discovery.\n\n## Factory module\n\nThe Factory class provides a flexible dependency injection pattern that can register and create instances of classes implementing a common interface using string-based strategies. It supports both transient scope (creates new instances on each request) and singleton scope (returns the same instance after first creation).\n\n[Open the notebook to explore the factory module example code](example_notebooks/factory_module_example.ipynb)\n\n## Config module\n\nThe load_config function provides a comprehensive configuration loading system that automatically discovers and parses YAML/JSON config files into Pydantic models with support for environment variable substitution and .env file loading. It offers flexible features like config overrides, custom parsers for different file formats, and automatically sets the working directory to the config file location for relative path resolution.\n\n[Open the notebook to explore the config module example code](example_notebooks/config_module_example.ipynb)\n","packages/graphrag-input/README.md":"# GraphRAG Inputs\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides input document loading utilities for GraphRAG, supporting multiple file formats including CSV, JSON, JSON Lines, and plain text.\n\n## Supported File Types\n\nThe following four standard file formats are supported out of the box:\n\n- **CSV** - Tabular data with configurable column mappings\n- **JSON** - JSON files with configurable property paths\n- **JSON Lines** - Line-delimited JSON records\n- **Text** - Plain text files\n\n### Markitdown Support\n\nAdditionally, we support the `InputType.MarkItDown` format, which uses the [MarkItDown](https://github.com/microsoft/markitdown) library to import any supported file type. The MarkItDown converter can handle a wide variety of file formats including Office documents, PDFs, HTML, and more.\n\n**Note:** Additional optional dependencies may need to be installed depending on the file type you're processing. The choice of converter is determined by MarkItDowns's processing logic, which primarily uses the file extension to select the appropriate converter. Please refer to the [MarkItDown repository](https://github.com/microsoft/markitdown) for installation instructions and detailed information about supported formats.\n\n## Examples\n\nBasic usage with the factory:\n\n1. Import a pdf with MarkItDown:\n\n```bash\npip install 'markitdown[pdf]' # required dependency for pdf processing\n```\n\n2. YAML config example for above:\n\n```yaml\ninput:\n type: markitdown\n file_pattern: \".*\\\\.pdf$$\"\ninput_storage:\n type: file\n base_dir: \"input\"\n```\n\n[Open the notebook to explore the input example code](example_notebooks/input_example.ipynb)\n","packages/graphrag-llm/README.md":"# GraphRAG LLM\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n## Basic Completion\n\nThis example demonstrates basic usage of the LLM library to interact with Azure OpenAI. It loads environment variables for API configuration, creates a ModelConfig for Azure OpenAI, and sends a simple question to the model. The code handles both streaming and non-streaming responses (streaming responses are printed chunk by chunk in real-time, while non-streaming responses are printed all at once). It also shows how to use the gather_completion_response utility function as a simpler alternative that automatically handles both response types and returns the complete text.\n\n[Open the notebook to explore the basic completion example code](example_notebooks/basic_completion_example.ipynb)\n\n## Basic Embedding\n\nThis examples demonstrates how to generate text embeddings using the GraphRAG LLM library with Azure OpenAI's embedding service. It loads API credentials from environment variables, creates a ModelConfig for the Azure embedding model and configures authentication to use either API key or Azure Managed Identity. The script then creates an embedding client and processes a batch of two text strings (\"Hello world\" and \"How are you?\") to generate their vector embeddings.\n\n[Open the notebook to explore the basic embeddings example code](example_notebooks/basic_embedding_example.ipynb)\n\nView the [notebooks](notebooks/README.md) for more examples.\n","packages/graphrag-llm/graphrag_llm/README.md":"# GraphRAG LLM\n\nView the [notebooks](notebooks) for detailed examples.\n\n## Basic Completion\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.completion import LLMCompletion, create_completion\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMCompletionChunk, LLMCompletionResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nmodel_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_MODEL\"),\n azure_deployment_name=os.getenv(\"GRAPHRAG_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\nllm_completion: LLMCompletion = create_completion(model_config)\n\nresponse: LLMCompletionResponse | Iterator[LLMCompletionChunk] = (\n llm_completion.completion(\n messages=\"What is the capital of France?\",\n )\n)\n\nif isinstance(response, Iterator):\n # Streaming response\n for chunk in response:\n print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\nelse:\n # Non-streaming response\n print(response.choices[0].message.content)\n\n# Alternatively, you can use the utility function to gather the full response\n# The following is equivalent to the above logic. If all you care about is\n# the first choice response then you can use the gather_completion_response\n# utility function.\nresponse_text = gather_completion_response(response)\nprint(response_text)\n```\n\n## Basic Embedding\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.embedding import LLMEmbedding, create_embedding\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMEmbeddingResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nembedding_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"), # type: ignore\n azure_deployment_name=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\n\nllm_embedding: LLMEmbedding = create_embedding(embedding_config)\n\nembeddings: LLMEmbeddingResponse = llm_embedding.embedding(\n input=[\"Hello world\", \"How are you?\"]\n)\nfor data in embeddings.data:\n print(data.embedding[0:3])\n```","packages/graphrag-llm/notebooks/README.md":"To run the notebooks you need to add a `.env` file to the `notebooks` directory with the following information\n\n```\nGRAPHRAG_MODEL=\"...\"\nGRAPHRAG_EMBEDDING_MODEL=\"...\"\nGRAPHRAG_API_BASE=\"...\"\n# API Key and version are optional\n# If not provided, Azure managed identity will be used\nGRAPHRAG_API_KEY=\"...\"\nGRAPHRAG_API_VERSION=\"...\"\n```","packages/graphrag-storage/README.md":"# GraphRAG Storage\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides a unified storage abstraction layer with support for multiple backends including file system, Azure Blob, Azure Cosmos, and memory storage. It features a factory-based creation system with configuration-driven setup and extensible architecture for implementing custom storage providers.\n\n## Basic\n\nThis example creates a file storage system using the GraphRAG storage package's configuration system. The example shows setting up file storage in a specified directory and demonstrates basic storage operations like setting and getting key-value pairs.\n\n[Open the notebook to explore the basic storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Custom Storage\n\nHere we create a custom storage implementation by extending the base Storage class and registering it with the GraphRAG storage system. Once registered, the custom storage can be instantiated through the factory pattern using either StorageConfig or directly via storage_factory, enabling extensible storage solutions for different backends.\n\n[Open the notebook to explore the custom storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Details\n\nBy default, the `create_storage` comes with the following storage providers registered that correspond to the entries in the `StorageType` enum.\n\n- `FileStorage`\n- `AzureBlobStorage`\n- `AzureCosmosStorage`\n- `MemoryStorage`\n\nThe preregistration happens dynamically, e.g., `FileStorage` is only imported and registered if you request a `FileStorage` with `create_storage(StorageType.File, ...)`. There is no need to manually import and register builtin storage providers when using `create_storage`.\n\nIf you want a clean factory with no preregistered storage providers then directly import `storage_factory` and bypass using `create_storage`. The downside is that `storage_factory.create` uses a dict for init args instead of the strongly typed `StorageConfig` used with `create_storage`.\n\n```python\nfrom graphrag_storage.storage_factory import storage_factory\nfrom graphrag_storage.file_storage import FileStorage\n\n# storage_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\nstorage_factory.register(\"my_storage_key\", FileStorage)\n\nstorage = storage_factory.create(\n strategy=\"my_storage_key\", init_args={\"base_dir\": \"...\", \"other_settings\": \"...\"}\n)\n\n...\n```\n","packages/graphrag-vectors/README.md":"# GraphRAG Vectors\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides vector store implementations for GraphRAG with support for multiple backends including LanceDB, Azure AI Search, and Azure Cosmos DB. It offers both a convenient configuration-driven API and direct factory access for creating and managing vector stores with flexible index schema definitions.\n\n## Basic usage with the utility function (recommended)\n\nThis demonstrates the recommended approach to create a vector store using the create_vector_store convenience function with configuration objects that specify the store type and index schema. The example shows setting up a LanceDB vector store with a defined index configuration, then connecting to it and creating the index for vector operations.\n\n[Open the notebook to explore the basic usage with utility function example code](example_notebooks/basic_usage_with_utility_function_example.ipynb)\n\n## Basic usage implementing the factory directly\n\nThis example shows a different approach to create vector stores by directly using the vector_store_factory with enum types and dictionary-based initialization arguments. This method provides more direct control over the factory creation process while bypassing the convenience function layer.\n\n[Open the notebook to explore the basic usage using factory directly example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Supported Vector Stores\n\n- **LanceDB**: Local vector database\n- **Azure AI Search**: Azure's managed search service with vector capabilities\n- **Azure Cosmos DB**: Azure's NoSQL database with vector search support\n\n## Custom Vector Store\n\nYou can register custom vector store implementations:\n\n[Open the notebook to explore the custom vector example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Configuration\n\nVector stores are configured using:\n\n- `VectorStoreConfig`: baseline parameters for the store\n- `IndexSchema`: Schema configuration for the specific index to create/connect to (index name, field names, vector size)\n","packages/graphrag/README.md":"# GraphRAG\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n👉 [Microsoft Research Blog Post](https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/)
\n👉 [Read the docs](https://microsoft.github.io/graphrag)
\n👉 [GraphRAG Arxiv](https://arxiv.org/pdf/2404.16130)\n\n
\n \n \"PyPI\n \n \n \"PyPI\n \n \n \"GitHub\n \n \n \"GitHub\n \n
\n\n## Overview\n\nThe GraphRAG project is a data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs.\n\nTo learn more about GraphRAG and how it can be used to enhance your LLM's ability to reason about your private data, please visit the Microsoft Research Blog Post.\n\n## Quickstart\n\nTo get started with the GraphRAG system we recommend trying the [command line quickstart](https://microsoft.github.io/graphrag/get_started/).\n\n## Repository Guidance\n\nThis repository presents a methodology for using knowledge graph memory structures to enhance LLM outputs. Please note that the provided code serves as a demonstration and is not an officially supported Microsoft offering.\n\n⚠️ _Warning: GraphRAG indexing can be an expensive operation, please read all of the documentation to understand the process and costs involved, and start small._\n\n## Diving Deeper\n\n- To learn about our contribution guidelines, see [CONTRIBUTING.md](../../CONTRIBUTING.md)\n- To start developing _GraphRAG_, see [DEVELOPING.md](../../DEVELOPING.md)\n- Join the conversation and provide feedback in the [GitHub Discussions tab!](https://github.com/microsoft/graphrag/discussions)\n\n## Prompt Tuning\n\nUsing _GraphRAG_ with your data out of the box may not yield the best possible results.\nWe strongly recommend to fine-tune your prompts following the [Prompt Tuning Guide](https://microsoft.github.io/graphrag/prompt_tuning/overview/) in our documentation.\n\n## Versioning\n\nPlease see the [breaking changes](../../breaking-changes.md) document for notes on our approach to versioning the project.\n\n_Always run `uv run poe init --root [path] --force` between minor version bumps to ensure you have the latest config format. Run the provided migration notebook between major version bumps if you want to avoid re-indexing prior datasets. Note that this will overwrite your configuration and prompts, so back them up if necessary._\n\n## Responsible AI FAQ\n\nSee [RAI_TRANSPARENCY.md](../../RAI_TRANSPARENCY.md)\n\n- [What is GraphRAG?](../../RAI_TRANSPARENCY.md#what-is-graphrag)\n- [What can GraphRAG do?](../../RAI_TRANSPARENCY.md#what-can-graphrag-do)\n- [What are GraphRAG’s intended use(s)?](../../RAI_TRANSPARENCY.md#what-are-graphrags-intended-uses)\n- [How was GraphRAG evaluated? What metrics are used to measure performance?](../../RAI_TRANSPARENCY.md#how-was-graphrag-evaluated-what-metrics-are-used-to-measure-performance)\n- [What are the limitations of GraphRAG? How can users minimize the impact of GraphRAG’s limitations when using the system?](../../RAI_TRANSPARENCY.md#what-are-the-limitations-of-graphrag-how-can-users-minimize-the-impact-of-graphrags-limitations-when-using-the-system)\n- [What operational factors and settings allow for effective and responsible use of GraphRAG?](../../RAI_TRANSPARENCY.md#what-operational-factors-and-settings-allow-for-effective-and-responsible-use-of-graphrag)\n\n## Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft\ntrademarks or logos is subject to and must follow\n[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n\n## Privacy\n\n[Microsoft Privacy Statement](https://privacy.microsoft.com/en-us/privacystatement)\n"},"files":{".agents/skills/update-deps/SKILL.md":"---\nname: update-deps\ndescription: >\n Update this GraphRAG uv-workspace monorepo's dependencies and repair the resulting\n breakages so every check and test passes again. Use when asked to update, upgrade, or\n bump dependencies/packages/versions, refresh or regenerate the lockfile, migrate to a new\n major of pandas/numpy/pydantic/pyarrow, resolve dependency-related test or lint failures,\n or run a \"dependency sweep\" — even if the user only names a single package. Covers editing\n pyproject.toml version specifiers across the root dev-deps and every packages/* member,\n re-locking with uv, and fixing code and tests for library API changes (e.g. pandas 3.0).\n USE FOR: update dependencies, upgrade packages, bump versions, dependency sweep, uv lock,\n uv sync, refresh lockfile, migrate pandas/numpy, \"deps broke the tests\", pyproject bump.\nuser-invocable: true\n---\n\n# Update Dependencies (GraphRAG monorepo)\n\n## Goal\n\nRaise dependency versions across this uv workspace, re-lock, and fix any code or test\nfallout until `uv run poe check` and `uv run poe test_unit` are both green — without\ntouching the version machinery that the release process owns.\n\n## Hard rules (non-negotiable)\n\n1. **Only the Microsoft feed proxy.** All resolves and syncs MUST use\n `https://packagefeedproxy.microsoft.io/pypi/simple` (the `[[tool.uv.index]]` configured in\n the root `pyproject.toml`). Never resolve against public PyPI, never add\n `--index`/`--default-index`/`--index-url` overrides, and never set `UV_INDEX_URL` or\n `PIP_INDEX_URL` to pypi.org. Do not disable or reorder the configured index.\n2. **Target versions must be at least 7 days old.** Only bump to a version that was released\n more than 7 days ago. The proxy does not serve versions published within the last week —\n syncing to a too-new version WILL fail. Before pinning a specific version, verify its\n release date and pick the newest release older than 7 days; otherwise let the specifier\n float and let `uv lock` choose (it can only see eligible versions on the proxy anyway).\n\n## Layout facts\n\n- This is a **uv workspace monorepo**. The root [`pyproject.toml`](../../../pyproject.toml)\n holds the `dev` group under `[dependency-groups]` and the `[tool.poe.tasks]` commands.\n- Runtime dependencies live in each member's `packages/*/pyproject.toml` under\n `[project] dependencies`.\n- Workspace members are wired via `[tool.uv.sources]` (`{ workspace = true }`) and pinned to\n each other with `graphrag-*==X.Y.Z` lines.\n- Package resolution goes through a Microsoft-internal index (`[[tool.uv.index]]`); expect\n that feed to be used, not public PyPI directly.\n\n## Do NOT edit these (release-owned)\n\n1. **Cross-package pins** `graphrag-cache==...`, `graphrag-llm==...`, etc. in any package.\n These are rewritten automatically by\n [`scripts/update_workspace_dependency_versions.py`](../../../scripts/update_workspace_dependency_versions.py)\n from the semversioner version. Hand-editing them causes drift.\n2. **`[project] version` fields** — managed by semversioner (\"do not change the version\n here manually\").\n3. **`graspologic-native>=1.2,<1.3`** — held below 1.3 on purpose; 1.3.x changes Leiden\n clustering output and breaks golden regression data. Only bump with a deliberate\n golden-data refresh, and say so explicitly.\n\n## Process\n\n1. **Baseline first.** Confirm a clean working tree and that checks/tests already pass\n before changing anything, so later failures are attributable to the bump:\n - `uv run poe check`\n - `uv run poe test_unit`\n Prefer a dedicated branch (e.g. `dep-sweep`).\n\n2. **Decide the scope.** Either a targeted set of packages the user named, or a full sweep.\n Edit the `~=`/`>=`/`<` specifiers in the relevant `[project] dependencies`\n (`packages/*/pyproject.toml`) and the root `dev` group. Leave the release-owned lines\n above untouched.\n\n3. **Resolve and lock.** All of these use the configured Microsoft feed proxy — do not pass\n any index override (see Hard rules).\n - For a full \"get latest allowed\" pass: `uv lock --upgrade`.\n - For targeted bumps after editing specifiers: `uv lock`.\n - Then install: `uv sync --all-packages`.\n If resolution fails, read the conflict, relax/adjust the offending specifier, and re-lock.\n Do not delete `uv.lock` to force it.\n - If a sync fails to find a version you just pinned, it is almost certainly younger than 7\n days on the proxy — step down to the newest release older than one week.\n\n4. **Static checks.** Run `uv run poe check` (this is `ruff format --check` + `ruff check` +\n `pyright`). Apply safe autofixes with `uv run poe fix`; format with `uv run poe format`.\n Fix remaining lint/type errors by hand — see Gotchas and the migration reference.\n\n5. **Tests.** Run `uv run poe test_unit` (NOT `uv run poe test`, which runs the full coverage\n suite). Run `uv run poe test_verbs` and `uv run poe test_integration` when the change is\n broad or touches indexing/query. Investigate every new failure.\n\n6. **Repair breakages.** For test/type failures caused by a library's API change, load\n [`references/migration-gotchas.md`](references/migration-gotchas.md) and apply the\n documented pattern. Keep fixes minimal and consistent with sibling code; prefer a real\n fix over a `# noqa`.\n\n7. **Record the change.** Add a changelog entry:\n `uv run semversioner add-change -t patch -d \"\"` (use `minor`/`major`\n only if the user's intent warrants it).\n\n8. **Final verification.** Re-run `uv run poe check` and `uv run poe test_unit`; both must be\n green (see the known-flake note below before calling a failure a regression).\n\n## Gotchas (this repo)\n\n- **`test_unit`, not `test`.** `poe test` runs coverage over everything and is slow; use\n `test_unit` for the fast feedback loop.\n- **Ruff runs in preview mode** (`preview = true`, `target-version = \"py310\"`). Preview-only\n rules such as `RUF069` (float equality) and `ASYNC119` fire here even though they may not\n in other repos.\n- **Known pre-existing flake:**\n `tests/unit/indexing/test_profiling.py::TestWorkflowProfiler::test_handles_exception_in_context`\n is timing-sensitive and can fail intermittently — it is not a dependency regression.\n- **`uv sync --all-packages`** (not bare `uv sync`) to install every workspace member.\n- **pandas is on the 3.0 line and numpy on 2.x.** Their major-version API changes are the\n usual source of post-bump breakage — see the reference file.\n- Version-bump edits touch many `pyproject.toml` files; make sure you did not accidentally\n modify a `graphrag-*==` pin or a `version` field while editing nearby specifiers.\n\n## Load-on-demand reference\n\nWhen a bump breaks tests or type-checking with a library API change (especially pandas or\nnumpy), read [`references/migration-gotchas.md`](references/migration-gotchas.md) for\nverified, repo-specific fix patterns before improvising.\n\n## Completion checklist\n\n- [ ] All resolves/syncs used the `packagefeedproxy.microsoft.io` index; no public-PyPI or\n index-override was introduced.\n- [ ] No dependency was bumped to a version released within the last 7 days.\n- [ ] Only intended specifiers changed; no `graphrag-*==` pin or `version` field edited.\n- [ ] `uv.lock` regenerated via `uv lock`/`uv lock --upgrade` (not hand-edited or deleted).\n- [ ] `uv run poe check` passes (ruff format, ruff lint, pyright).\n- [ ] `uv run poe test_unit` passes (ignoring only the known profiling flake).\n- [ ] Broader suites run if the change was broad (`test_verbs`/`test_integration`).\n- [ ] A semversioner changelog entry was added.\n- [ ] Any risky/held pin (e.g. `graspologic-native`) left in place unless explicitly bumped.\n",".agents/skills/update-deps/references/migration-gotchas.md":"# Migration Gotchas — verified fixes for this repo\n\nLoad this when a dependency bump breaks tests or `pyright`/`ruff` with a library API change.\nEach entry is a pattern that was actually hit and fixed in this codebase. Apply the minimal\nfix and keep it consistent with sibling code.\n\n## pandas 3.0\n\n### `DataFrame.swapaxes` removed → `np.array_split(df, n)` returns ndarrays\n\n`np.array_split` internally calls `np.swapaxes`, which used to delegate to\n`DataFrame.swapaxes` and return DataFrames with column names intact. In pandas 3.0\n`swapaxes` was removed (deprecated in 2.1), so `np.array_split(df, n)` now yields plain\nnumpy arrays. Rebuilding with `pd.DataFrame(fold)` produces integer `RangeIndex` columns,\nso later `df[\"some_column\"]` raises `KeyError`.\n\nSymptom: `KeyError: ''` with a traceback ending in\n`pandas/core/indexes/range.py ... get_loc`.\n\nFix — split positional indices instead of the frame, then select rows with `iloc`:\n\n```python\n# Broken under pandas 3.0\nreturn [pd.DataFrame(fold) for fold in np.array_split(reports, n)]\n\n# Fixed — preserves columns, dtypes, and even fold sizes\nreturn [reports.iloc[indices] for indices in np.array_split(np.arange(len(reports)), n)]\n```\n\n### `copy=` keyword removed from `merge`/`concat`/`join`/`set_axis` etc.\n\npandas 3.0 makes Copy-on-Write the default and drops the `copy=` parameter. Calls like\n`df.merge(other, copy=False)` or `pd.concat([...], copy=False)` raise `TypeError`.\nFix: delete the `copy=` argument — CoW already avoids the unnecessary copy.\n\n### Chained-assignment / `inplace` under Copy-on-Write\n\nWith CoW, mutating a slice (`df[mask][\"col\"] = x`) no longer writes back and may warn/error.\nAssign through `.loc`: `df.loc[mask, \"col\"] = x`. Reassign results of `inplace=True`-style\noperations rather than relying on in-place mutation of a view.\n\n## numpy 2.x\n\n- Removed aliases (`np.float_`, `np.int0`, `np.bool8`, `np.object0`, etc.) — use the builtin\n or the explicit sized dtype (`np.float64`, `np.bool_`).\n- `np.array_split` on a DataFrame no longer preserves the frame (see the pandas entry above).\n- Some functions moved out of the top-level namespace; import from the documented submodule.\n\n## ruff (preview mode active in this repo)\n\n- **RUF069 (float-equality-comparison):** `x == 0.0` / `!= 0.0` is flagged. Prefer a\n non-equality guard when semantically valid (`x <= 0.0` for a non-positive divide guard) or\n `math.isclose(...)` for tolerance checks. Avoid a blanket `# noqa` when a real fix exists.\n- **ASYNC119 (yield in context manager in async generator):** do not `yield` while holding a\n `with`/`async with` in an async generator. Materialize inside the block, then yield after\n it closes — matching the sibling providers:\n\n ```python\n with Path.open(path, \"r\", encoding=enc) as f:\n rows = list(csv.DictReader(f))\n for row in rows:\n yield transform(row)\n ```\n\n## pyright\n\n- Type stubs travel with majors: the `dev` group pins `pandas-stubs~=3.0`. When bumping\n pandas, bump the matching stubs so `pyright` reflects the new API.\n- After dependency changes, `pyright` may surface new optional/overload errors from updated\n stubs; fix at the call site rather than suppressing, unless the stub is demonstrably wrong.\n\n## General approach\n\n1. Read the actual traceback/diagnostic to the leaf frame — the failing library call and the\n changed symbol are usually right there.\n2. Check how a sibling module in the same package already handles the pattern and match it.\n3. Prefer a real, minimal fix over suppression; re-run `uv run poe check` and\n `uv run poe test_unit` after each fix to confirm.\n","packages/graphrag-cache/README.md":"# GraphRAG Cache\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of utilities to handle GraphRAG caching implementation.\n\n### Basic\n\nThis example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system.\n\n[Open the notebook to explore the basic example code](example_notebooks/basic_cache_example.ipynb)\n\n### Custom Cache\n\nThis example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.\n\n[Open the notebook to explore the basic custom example code](example_notebooks/custom_cache_example.ipynb)\n\n#### Details\n\nBy default, the `create_cache` comes with the following cache providers registered that correspond to the entries in the `CacheType` enum.\n\n- `JsonCache`\n- `MemoryCache`\n- `NoopCache`\n- `SQLiteCache`\n\n`SQLiteCache` stores JSON-serializable values in a local SQLite database. It uses\nWAL mode and short-lived connections so multiple workers can safely share the\nsame database. Configure it with:\n\n```yaml\ncache:\n type: sqlite\n storage:\n type: file\n base_dir: cache\n database_name: cache.db\n```\n\nThe database is created within the configured file storage. SQLite requires\nlocal random-access file operations, so blob and Cosmos storage are not\nsupported. Child caches use isolated namespaces within the same database.\n\nAll cache providers expose `await cache.get_many(keys)` for bulk reads. The\nresult maps each existing, valid key to the same decoded value returned by\n`get()`. Missing and malformed entries are omitted, duplicate input keys appear\nat most once, and an empty input returns an empty dictionary. The default\nimplementation remains compatible with custom `Cache` subclasses, while\n`MemoryCache`, `NoopCache`, and `SQLiteCache` provide backend-specific\nimplementations. SQLite bulk reads use one worker dispatch and one connection,\nsplitting large key sets into bounded SQL batches.\n\nAn opt-in warm-cache benchmark compares bulk reads across the memory, JSON, and\nSQLite providers without enforcing a machine-dependent performance threshold:\n\n```bash\nuv run pytest -s tests/unit/cache/test_cache_benchmark.py \\\n --run-cache-benchmark\n```\n\nThe benchmark reports median latency, entries per second, and time relative to\nSQLite for 1,000 entries. Setup time is excluded from the measurements.\n\nThe preregistration happens dynamically, e.g., `JsonCache` is only imported and registered if you request a `JsonCache` with `create_cache(CacheType.Json, ...)`. There is no need to manually import and register builtin cache providers when using `create_cache`.\n\nIf you want a clean factory with no preregistered cache providers then directly import `cache_factory` and bypass using `create_cache`. The downside is that `cache_factory.create` uses a dict for init args instead of the strongly typed `CacheConfig` used with `create_cache`.\n\n```python\nfrom graphrag_cache.cache_factory import cache_factory\nfrom graphrag_cache.json_cache import JsonCache\n\n# cache_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\ncache_factory.register(\"my_cache_impl\", JsonCache)\n\ncache = cache_factory.create(\n strategy=\"my_cache_impl\", init_args={\"some_setting\": \"...\"}\n)\n\n...\n```\n","packages/graphrag-chunking/README.md":"# GraphRAG Chunking\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of text chunkers, a core config model, and a factory for acquiring instances.\n\n## Examples\n\n### Basic sentence chunking with nltk\n\nThe SentenceChunker class splits text into individual sentences by identifying sentence boundaries. It takes input text and returns a list where each element is a separate sentence, making it easy to process text at the sentence level.\n\n[Open the notebook to explore the basic sentence example code](example_notebooks/basic_sentence_example.ipynb)\n\n### Token chunking\n\nThe TokenChunker splits text into fixed-size chunks based on token count rather than sentence boundaries. It uses a tokenizer to encode text into tokens, then creates chunks of a specified size with configurable overlap between chunks.\n\n[Open the notebook to explore the token chunking example code](example_notebooks/token_chunking_example.ipynb)\n\n### Using the factory via helper util\n\nThe create_chunker factory function provides a configuration-driven approach to instantiate chunkers by accepting a ChunkingConfig object that specifies the chunking strategy and parameters. This allows for more flexible and maintainable code by separating chunker configuration from direct instantiation.\n\n[Open the notebook to explore the factory helper util example code](example_notebooks/factory_helper_util_example.ipynb)\n","packages/graphrag-common/README.md":"# GraphRAG Common\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides utility modules for GraphRAG, including a flexible factory system for dependency injection and service registration, and a comprehensive configuration loading system with Pydantic model support, environment variable substitution, and automatic file discovery.\n\n## Factory module\n\nThe Factory class provides a flexible dependency injection pattern that can register and create instances of classes implementing a common interface using string-based strategies. It supports both transient scope (creates new instances on each request) and singleton scope (returns the same instance after first creation).\n\n[Open the notebook to explore the factory module example code](example_notebooks/factory_module_example.ipynb)\n\n## Config module\n\nThe load_config function provides a comprehensive configuration loading system that automatically discovers and parses YAML/JSON config files into Pydantic models with support for environment variable substitution and .env file loading. It offers flexible features like config overrides, custom parsers for different file formats, and automatically sets the working directory to the config file location for relative path resolution.\n\n[Open the notebook to explore the config module example code](example_notebooks/config_module_example.ipynb)\n","packages/graphrag-input/README.md":"# GraphRAG Inputs\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides input document loading utilities for GraphRAG, supporting multiple file formats including CSV, JSON, JSON Lines, and plain text.\n\n## Supported File Types\n\nThe following four standard file formats are supported out of the box:\n\n- **CSV** - Tabular data with configurable column mappings\n- **JSON** - JSON files with configurable property paths\n- **JSON Lines** - Line-delimited JSON records\n- **Text** - Plain text files\n\n### Markitdown Support\n\nAdditionally, we support the `InputType.MarkItDown` format, which uses the [MarkItDown](https://github.com/microsoft/markitdown) library to import any supported file type. The MarkItDown converter can handle a wide variety of file formats including Office documents, PDFs, HTML, and more.\n\n**Note:** Additional optional dependencies may need to be installed depending on the file type you're processing. The choice of converter is determined by MarkItDowns's processing logic, which primarily uses the file extension to select the appropriate converter. Please refer to the [MarkItDown repository](https://github.com/microsoft/markitdown) for installation instructions and detailed information about supported formats.\n\n## Examples\n\nBasic usage with the factory:\n\n1. Import a pdf with MarkItDown:\n\n```bash\npip install 'markitdown[pdf]' # required dependency for pdf processing\n```\n\n2. YAML config example for above:\n\n```yaml\ninput:\n type: markitdown\n file_pattern: \".*\\\\.pdf$$\"\ninput_storage:\n type: file\n base_dir: \"input\"\n```\n\n[Open the notebook to explore the input example code](example_notebooks/input_example.ipynb)\n","packages/graphrag-llm/README.md":"# GraphRAG LLM\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n## Basic Completion\n\nThis example demonstrates basic usage of the LLM library to interact with Azure OpenAI. It loads environment variables for API configuration, creates a ModelConfig for Azure OpenAI, and sends a simple question to the model. The code handles both streaming and non-streaming responses (streaming responses are printed chunk by chunk in real-time, while non-streaming responses are printed all at once). It also shows how to use the gather_completion_response utility function as a simpler alternative that automatically handles both response types and returns the complete text.\n\n[Open the notebook to explore the basic completion example code](example_notebooks/basic_completion_example.ipynb)\n\n## Basic Embedding\n\nThis examples demonstrates how to generate text embeddings using the GraphRAG LLM library with Azure OpenAI's embedding service. It loads API credentials from environment variables, creates a ModelConfig for the Azure embedding model and configures authentication to use either API key or Azure Managed Identity. The script then creates an embedding client and processes a batch of two text strings (\"Hello world\" and \"How are you?\") to generate their vector embeddings.\n\n[Open the notebook to explore the basic embeddings example code](example_notebooks/basic_embedding_example.ipynb)\n\nView the [notebooks](notebooks/README.md) for more examples.\n","packages/graphrag-llm/graphrag_llm/README.md":"# GraphRAG LLM\n\nView the [notebooks](notebooks) for detailed examples.\n\n## Basic Completion\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.completion import LLMCompletion, create_completion\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMCompletionChunk, LLMCompletionResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nmodel_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_MODEL\"),\n azure_deployment_name=os.getenv(\"GRAPHRAG_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\nllm_completion: LLMCompletion = create_completion(model_config)\n\nresponse: LLMCompletionResponse | Iterator[LLMCompletionChunk] = (\n llm_completion.completion(\n messages=\"What is the capital of France?\",\n )\n)\n\nif isinstance(response, Iterator):\n # Streaming response\n for chunk in response:\n print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\nelse:\n # Non-streaming response\n print(response.choices[0].message.content)\n\n# Alternatively, you can use the utility function to gather the full response\n# The following is equivalent to the above logic. If all you care about is\n# the first choice response then you can use the gather_completion_response\n# utility function.\nresponse_text = gather_completion_response(response)\nprint(response_text)\n```\n\n## Basic Embedding\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.embedding import LLMEmbedding, create_embedding\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMEmbeddingResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nembedding_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"), # type: ignore\n azure_deployment_name=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\n\nllm_embedding: LLMEmbedding = create_embedding(embedding_config)\n\nembeddings: LLMEmbeddingResponse = llm_embedding.embedding(\n input=[\"Hello world\", \"How are you?\"]\n)\nfor data in embeddings.data:\n print(data.embedding[0:3])\n```","packages/graphrag-llm/notebooks/README.md":"To run the notebooks you need to add a `.env` file to the `notebooks` directory with the following information\n\n```\nGRAPHRAG_MODEL=\"...\"\nGRAPHRAG_EMBEDDING_MODEL=\"...\"\nGRAPHRAG_API_BASE=\"...\"\n# API Key and version are optional\n# If not provided, Azure managed identity will be used\nGRAPHRAG_API_KEY=\"...\"\nGRAPHRAG_API_VERSION=\"...\"\n```","packages/graphrag-storage/README.md":"# GraphRAG Storage\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides a unified storage abstraction layer with support for multiple backends including file system, Azure Blob, Azure Cosmos, and memory storage. It features a factory-based creation system with configuration-driven setup and extensible architecture for implementing custom storage providers.\n\n## Basic\n\nThis example creates a file storage system using the GraphRAG storage package's configuration system. The example shows setting up file storage in a specified directory and demonstrates basic storage operations like setting and getting key-value pairs.\n\n[Open the notebook to explore the basic storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Custom Storage\n\nHere we create a custom storage implementation by extending the base Storage class and registering it with the GraphRAG storage system. Once registered, the custom storage can be instantiated through the factory pattern using either StorageConfig or directly via storage_factory, enabling extensible storage solutions for different backends.\n\n[Open the notebook to explore the custom storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Details\n\nBy default, the `create_storage` comes with the following storage providers registered that correspond to the entries in the `StorageType` enum.\n\n- `FileStorage`\n- `AzureBlobStorage`\n- `AzureCosmosStorage`\n- `MemoryStorage`\n\nThe preregistration happens dynamically, e.g., `FileStorage` is only imported and registered if you request a `FileStorage` with `create_storage(StorageType.File, ...)`. There is no need to manually import and register builtin storage providers when using `create_storage`.\n\nIf you want a clean factory with no preregistered storage providers then directly import `storage_factory` and bypass using `create_storage`. The downside is that `storage_factory.create` uses a dict for init args instead of the strongly typed `StorageConfig` used with `create_storage`.\n\n```python\nfrom graphrag_storage.storage_factory import storage_factory\nfrom graphrag_storage.file_storage import FileStorage\n\n# storage_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\nstorage_factory.register(\"my_storage_key\", FileStorage)\n\nstorage = storage_factory.create(\n strategy=\"my_storage_key\", init_args={\"base_dir\": \"...\", \"other_settings\": \"...\"}\n)\n\n...\n```\n","packages/graphrag-vectors/README.md":"# GraphRAG Vectors\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides vector store implementations for GraphRAG with support for multiple backends including LanceDB, Azure AI Search, and Azure Cosmos DB. It offers both a convenient configuration-driven API and direct factory access for creating and managing vector stores with flexible index schema definitions.\n\n## Basic usage with the utility function (recommended)\n\nThis demonstrates the recommended approach to create a vector store using the create_vector_store convenience function with configuration objects that specify the store type and index schema. The example shows setting up a LanceDB vector store with a defined index configuration, then connecting to it and creating the index for vector operations.\n\n[Open the notebook to explore the basic usage with utility function example code](example_notebooks/basic_usage_with_utility_function_example.ipynb)\n\n## Basic usage implementing the factory directly\n\nThis example shows a different approach to create vector stores by directly using the vector_store_factory with enum types and dictionary-based initialization arguments. This method provides more direct control over the factory creation process while bypassing the convenience function layer.\n\n[Open the notebook to explore the basic usage using factory directly example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Supported Vector Stores\n\n- **LanceDB**: Local vector database\n- **Azure AI Search**: Azure's managed search service with vector capabilities\n- **Azure Cosmos DB**: Azure's NoSQL database with vector search support\n\n## Custom Vector Store\n\nYou can register custom vector store implementations:\n\n[Open the notebook to explore the custom vector example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Configuration\n\nVector stores are configured using:\n\n- `VectorStoreConfig`: baseline parameters for the store\n- `IndexSchema`: Schema configuration for the specific index to create/connect to (index name, field names, vector size)\n","packages/graphrag/README.md":"# GraphRAG\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n👉 [Microsoft Research Blog Post](https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/)
\n👉 [Read the docs](https://microsoft.github.io/graphrag)
\n👉 [GraphRAG Arxiv](https://arxiv.org/pdf/2404.16130)\n\n
\n \n \"PyPI\n \n \n \"PyPI\n \n \n \"GitHub\n \n \n \"GitHub\n \n
\n\n## Overview\n\nThe GraphRAG project is a data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs.\n\nTo learn more about GraphRAG and how it can be used to enhance your LLM's ability to reason about your private data, please visit the Microsoft Research Blog Post.\n\n## Quickstart\n\nTo get started with the GraphRAG system we recommend trying the [command line quickstart](https://microsoft.github.io/graphrag/get_started/).\n\n## Repository Guidance\n\nThis repository presents a methodology for using knowledge graph memory structures to enhance LLM outputs. Please note that the provided code serves as a demonstration and is not an officially supported Microsoft offering.\n\n⚠️ _Warning: GraphRAG indexing can be an expensive operation, please read all of the documentation to understand the process and costs involved, and start small._\n\n## Diving Deeper\n\n- To learn about our contribution guidelines, see [CONTRIBUTING.md](../../CONTRIBUTING.md)\n- To start developing _GraphRAG_, see [DEVELOPING.md](../../DEVELOPING.md)\n- Join the conversation and provide feedback in the [GitHub Discussions tab!](https://github.com/microsoft/graphrag/discussions)\n\n## Prompt Tuning\n\nUsing _GraphRAG_ with your data out of the box may not yield the best possible results.\nWe strongly recommend to fine-tune your prompts following the [Prompt Tuning Guide](https://microsoft.github.io/graphrag/prompt_tuning/overview/) in our documentation.\n\n## Versioning\n\nPlease see the [breaking changes](../../breaking-changes.md) document for notes on our approach to versioning the project.\n\n_Always run `uv run poe init --root [path] --force` between minor version bumps to ensure you have the latest config format. Run the provided migration notebook between major version bumps if you want to avoid re-indexing prior datasets. Note that this will overwrite your configuration and prompts, so back them up if necessary._\n\n## Responsible AI FAQ\n\nSee [RAI_TRANSPARENCY.md](../../RAI_TRANSPARENCY.md)\n\n- [What is GraphRAG?](../../RAI_TRANSPARENCY.md#what-is-graphrag)\n- [What can GraphRAG do?](../../RAI_TRANSPARENCY.md#what-can-graphrag-do)\n- [What are GraphRAG’s intended use(s)?](../../RAI_TRANSPARENCY.md#what-are-graphrags-intended-uses)\n- [How was GraphRAG evaluated? What metrics are used to measure performance?](../../RAI_TRANSPARENCY.md#how-was-graphrag-evaluated-what-metrics-are-used-to-measure-performance)\n- [What are the limitations of GraphRAG? How can users minimize the impact of GraphRAG’s limitations when using the system?](../../RAI_TRANSPARENCY.md#what-are-the-limitations-of-graphrag-how-can-users-minimize-the-impact-of-graphrags-limitations-when-using-the-system)\n- [What operational factors and settings allow for effective and responsible use of GraphRAG?](../../RAI_TRANSPARENCY.md#what-operational-factors-and-settings-allow-for-effective-and-responsible-use-of-graphrag)\n\n## Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft\ntrademarks or logos is subject to and must follow\n[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n\n## Privacy\n\n[Microsoft Privacy Statement](https://privacy.microsoft.com/en-us/privacystatement)\n"},"items":[{"name":"migration-gotchas.md","path":".agents/skills/update-deps/references/migration-gotchas.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/.agents/skills/update-deps/references/migration-gotchas.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Migration Gotchas — verified fixes for this repo\n\nLoad this when a dependency bump breaks tests or `pyright`/`ruff` with a library API change.\nEach entry is a pattern that was actually hit and fixed in this codebase. Apply the minimal\nfix and keep it consistent with sibling code.\n\n## pandas 3.0\n\n### `DataFrame.swapaxes` removed → `np.array_split(df, n)` returns ndarrays\n\n`np.array_split` internally calls `np.swapaxes`, which used to delegate to\n`DataFrame.swapaxes` and return DataFrames with column names intact. In pandas 3.0\n`swapaxes` was removed (deprecated in 2.1), so `np.array_split(df, n)` now yields plain\nnumpy arrays. Rebuilding with `pd.DataFrame(fold)` produces integer `RangeIndex` columns,\nso later `df[\"some_column\"]` raises `KeyError`.\n\nSymptom: `KeyError: ''` with a traceback ending in\n`pandas/core/indexes/range.py ... get_loc`.\n\nFix — split positional indices instead of the frame, then select rows with `iloc`:\n\n```python\n# Broken under pandas 3.0\nreturn [pd.DataFrame(fold) for fold in np.array_split(reports, n)]\n\n# Fixed — preserves columns, dtypes, and even fold sizes\nreturn [reports.iloc[indices] for indices in np.array_split(np.arange(len(reports)), n)]\n```\n\n### `copy=` keyword removed from `merge`/`concat`/`join`/`set_axis` etc.\n\npandas 3.0 makes Copy-on-Write the default and drops the `copy=` parameter. Calls like\n`df.merge(other, copy=False)` or `pd.concat([...], copy=False)` raise `TypeError`.\nFix: delete the `copy=` argument — CoW already avoids the unnecessary copy.\n\n### Chained-assignment / `inplace` under Copy-on-Write\n\nWith CoW, mutating a slice (`df[mask][\"col\"] = x`) no longer writes back and may warn/error.\nAssign through `.loc`: `df.loc[mask, \"col\"] = x`. Reassign results of `inplace=True`-style\noperations rather than relying on in-place mutation of a view.\n\n## numpy 2.x\n\n- Removed aliases (`np.float_`, `np.int0`, `np.bool8`, `np.object0`, etc.) — use the builtin\n or the explicit sized dtype (`np.float64`, `np.bool_`).\n- `np.array_split` on a DataFrame no longer preserves the frame (see the pandas entry above).\n- Some functions moved out of the top-level namespace; import from the documented submodule.\n\n## ruff (preview mode active in this repo)\n\n- **RUF069 (float-equality-comparison):** `x == 0.0` / `!= 0.0` is flagged. Prefer a\n non-equality guard when semantically valid (`x <= 0.0` for a non-positive divide guard) or\n `math.isclose(...)` for tolerance checks. Avoid a blanket `# noqa` when a real fix exists.\n- **ASYNC119 (yield in context manager in async generator):** do not `yield` while holding a\n `with`/`async with` in an async generator. Materialize inside the block, then yield after\n it closes — matching the sibling providers:\n\n ```python\n with Path.open(path, \"r\", encoding=enc) as f:\n rows = list(csv.DictReader(f))\n for row in rows:\n yield transform(row)\n ```\n\n## pyright\n\n- Type stubs travel with majors: the `dev` group pins `pandas-stubs~=3.0`. When bumping\n pandas, bump the matching stubs so `pyright` reflects the new API.\n- After dependency changes, `pyright` may surface new optional/overload errors from updated\n stubs; fix at the call site rather than suppressing, unless the stub is demonstrably wrong.\n\n## General approach\n\n1. Read the actual traceback/diagnostic to the leaf frame — the failing library call and the\n changed symbol are usually right there.\n2. Check how a sibling module in the same package already handles the pattern and match it.\n3. Prefer a real, minimal fix over suppression; re-run `uv run poe check` and\n `uv run poe test_unit` after each fix to confirm.\n","isInternal":false,"tokens":941,"sizeBytes":3631},{"name":"SKILL.md","path":".agents/skills/update-deps/SKILL.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/.agents/skills/update-deps/SKILL.md","title":"update-deps","category":"anthropic-skill","format":"markdown","content":"---\nname: update-deps\ndescription: >\n Update this GraphRAG uv-workspace monorepo's dependencies and repair the resulting\n breakages so every check and test passes again. Use when asked to update, upgrade, or\n bump dependencies/packages/versions, refresh or regenerate the lockfile, migrate to a new\n major of pandas/numpy/pydantic/pyarrow, resolve dependency-related test or lint failures,\n or run a \"dependency sweep\" — even if the user only names a single package. Covers editing\n pyproject.toml version specifiers across the root dev-deps and every packages/* member,\n re-locking with uv, and fixing code and tests for library API changes (e.g. pandas 3.0).\n USE FOR: update dependencies, upgrade packages, bump versions, dependency sweep, uv lock,\n uv sync, refresh lockfile, migrate pandas/numpy, \"deps broke the tests\", pyproject bump.\nuser-invocable: true\n---\n\n# Update Dependencies (GraphRAG monorepo)\n\n## Goal\n\nRaise dependency versions across this uv workspace, re-lock, and fix any code or test\nfallout until `uv run poe check` and `uv run poe test_unit` are both green — without\ntouching the version machinery that the release process owns.\n\n## Hard rules (non-negotiable)\n\n1. **Only the Microsoft feed proxy.** All resolves and syncs MUST use\n `https://packagefeedproxy.microsoft.io/pypi/simple` (the `[[tool.uv.index]]` configured in\n the root `pyproject.toml`). Never resolve against public PyPI, never add\n `--index`/`--default-index`/`--index-url` overrides, and never set `UV_INDEX_URL` or\n `PIP_INDEX_URL` to pypi.org. Do not disable or reorder the configured index.\n2. **Target versions must be at least 7 days old.** Only bump to a version that was released\n more than 7 days ago. The proxy does not serve versions published within the last week —\n syncing to a too-new version WILL fail. Before pinning a specific version, verify its\n release date and pick the newest release older than 7 days; otherwise let the specifier\n float and let `uv lock` choose (it can only see eligible versions on the proxy anyway).\n\n## Layout facts\n\n- This is a **uv workspace monorepo**. The root [`pyproject.toml`](../../../pyproject.toml)\n holds the `dev` group under `[dependency-groups]` and the `[tool.poe.tasks]` commands.\n- Runtime dependencies live in each member's `packages/*/pyproject.toml` under\n `[project] dependencies`.\n- Workspace members are wired via `[tool.uv.sources]` (`{ workspace = true }`) and pinned to\n each other with `graphrag-*==X.Y.Z` lines.\n- Package resolution goes through a Microsoft-internal index (`[[tool.uv.index]]`); expect\n that feed to be used, not public PyPI directly.\n\n## Do NOT edit these (release-owned)\n\n1. **Cross-package pins** `graphrag-cache==...`, `graphrag-llm==...`, etc. in any package.\n These are rewritten automatically by\n [`scripts/update_workspace_dependency_versions.py`](../../../scripts/update_workspace_dependency_versions.py)\n from the semversioner version. Hand-editing them causes drift.\n2. **`[project] version` fields** — managed by semversioner (\"do not change the version\n here manually\").\n3. **`graspologic-native>=1.2,<1.3`** — held below 1.3 on purpose; 1.3.x changes Leiden\n clustering output and breaks golden regression data. Only bump with a deliberate\n golden-data refresh, and say so explicitly.\n\n## Process\n\n1. **Baseline first.** Confirm a clean working tree and that checks/tests already pass\n before changing anything, so later failures are attributable to the bump:\n - `uv run poe check`\n - `uv run poe test_unit`\n Prefer a dedicated branch (e.g. `dep-sweep`).\n\n2. **Decide the scope.** Either a targeted set of packages the user named, or a full sweep.\n Edit the `~=`/`>=`/`<` specifiers in the relevant `[project] dependencies`\n (`packages/*/pyproject.toml`) and the root `dev` group. Leave the release-owned lines\n above untouched.\n\n3. **Resolve and lock.** All of these use the configured Microsoft feed proxy — do not pass\n any index override (see Hard rules).\n - For a full \"get latest allowed\" pass: `uv lock --upgrade`.\n - For targeted bumps after editing specifiers: `uv lock`.\n - Then install: `uv sync --all-packages`.\n If resolution fails, read the conflict, relax/adjust the offending specifier, and re-lock.\n Do not delete `uv.lock` to force it.\n - If a sync fails to find a version you just pinned, it is almost certainly younger than 7\n days on the proxy — step down to the newest release older than one week.\n\n4. **Static checks.** Run `uv run poe check` (this is `ruff format --check` + `ruff check` +\n `pyright`). Apply safe autofixes with `uv run poe fix`; format with `uv run poe format`.\n Fix remaining lint/type errors by hand — see Gotchas and the migration reference.\n\n5. **Tests.** Run `uv run poe test_unit` (NOT `uv run poe test`, which runs the full coverage\n suite). Run `uv run poe test_verbs` and `uv run poe test_integration` when the change is\n broad or touches indexing/query. Investigate every new failure.\n\n6. **Repair breakages.** For test/type failures caused by a library's API change, load\n [`references/migration-gotchas.md`](references/migration-gotchas.md) and apply the\n documented pattern. Keep fixes minimal and consistent with sibling code; prefer a real\n fix over a `# noqa`.\n\n7. **Record the change.** Add a changelog entry:\n `uv run semversioner add-change -t patch -d \"\"` (use `minor`/`major`\n only if the user's intent warrants it).\n\n8. **Final verification.** Re-run `uv run poe check` and `uv run poe test_unit`; both must be\n green (see the known-flake note below before calling a failure a regression).\n\n## Gotchas (this repo)\n\n- **`test_unit`, not `test`.** `poe test` runs coverage over everything and is slow; use\n `test_unit` for the fast feedback loop.\n- **Ruff runs in preview mode** (`preview = true`, `target-version = \"py310\"`). Preview-only\n rules such as `RUF069` (float equality) and `ASYNC119` fire here even though they may not\n in other repos.\n- **Known pre-existing flake:**\n `tests/unit/indexing/test_profiling.py::TestWorkflowProfiler::test_handles_exception_in_context`\n is timing-sensitive and can fail intermittently — it is not a dependency regression.\n- **`uv sync --all-packages`** (not bare `uv sync`) to install every workspace member.\n- **pandas is on the 3.0 line and numpy on 2.x.** Their major-version API changes are the\n usual source of post-bump breakage — see the reference file.\n- Version-bump edits touch many `pyproject.toml` files; make sure you did not accidentally\n modify a `graphrag-*==` pin or a `version` field while editing nearby specifiers.\n\n## Load-on-demand reference\n\nWhen a bump breaks tests or type-checking with a library API change (especially pandas or\nnumpy), read [`references/migration-gotchas.md`](references/migration-gotchas.md) for\nverified, repo-specific fix patterns before improvising.\n\n## Completion checklist\n\n- [ ] All resolves/syncs used the `packagefeedproxy.microsoft.io` index; no public-PyPI or\n index-override was introduced.\n- [ ] No dependency was bumped to a version released within the last 7 days.\n- [ ] Only intended specifiers changed; no `graphrag-*==` pin or `version` field edited.\n- [ ] `uv.lock` regenerated via `uv lock`/`uv lock --upgrade` (not hand-edited or deleted).\n- [ ] `uv run poe check` passes (ruff format, ruff lint, pyright).\n- [ ] `uv run poe test_unit` passes (ignoring only the known profiling flake).\n- [ ] Broader suites run if the change was broad (`test_verbs`/`test_integration`).\n- [ ] A semversioner changelog entry was added.\n- [ ] Any risky/held pin (e.g. `graspologic-native`) left in place unless explicitly bumped.\n","frontmatter":{"name":"update-deps","description":"Update this GraphRAG uv-workspace monorepo's dependencies and repair the resulting breakages so every check and test passes again. Use when asked to update, upgrade, or bump dependencies/packages/versions, refresh or regenerate the lockfile, migrate to a new major of pandas/numpy/pydantic/pyarrow, resolve dependency-related test or lint failures, or run a \"dependency sweep\" — even if the user only names a single package. Covers editing pyproject.toml version specifiers across the root dev-deps and every packages/* member, re-locking with uv, and fixing code and tests for library API changes (e.g. pandas 3.0). USE FOR: update dependencies, upgrade packages, bump versions, dependency sweep, uv lock, uv sync, refresh lockfile, migrate pandas/numpy, \"deps broke the tests\", pyproject bump.\n","user-invocable":true},"isInternal":false,"tokens":1953,"sizeBytes":7724},{"name":"README.md","path":"packages/graphrag-cache/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-cache/README.md","title":"graphrag-cache Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG Cache\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of utilities to handle GraphRAG caching implementation.\n\n### Basic\n\nThis example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system.\n\n[Open the notebook to explore the basic example code](example_notebooks/basic_cache_example.ipynb)\n\n### Custom Cache\n\nThis example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.\n\n[Open the notebook to explore the basic custom example code](example_notebooks/custom_cache_example.ipynb)\n\n#### Details\n\nBy default, the `create_cache` comes with the following cache providers registered that correspond to the entries in the `CacheType` enum.\n\n- `JsonCache`\n- `MemoryCache`\n- `NoopCache`\n- `SQLiteCache`\n\n`SQLiteCache` stores JSON-serializable values in a local SQLite database. It uses\nWAL mode and short-lived connections so multiple workers can safely share the\nsame database. Configure it with:\n\n```yaml\ncache:\n type: sqlite\n storage:\n type: file\n base_dir: cache\n database_name: cache.db\n```\n\nThe database is created within the configured file storage. SQLite requires\nlocal random-access file operations, so blob and Cosmos storage are not\nsupported. Child caches use isolated namespaces within the same database.\n\nAll cache providers expose `await cache.get_many(keys)` for bulk reads. The\nresult maps each existing, valid key to the same decoded value returned by\n`get()`. Missing and malformed entries are omitted, duplicate input keys appear\nat most once, and an empty input returns an empty dictionary. The default\nimplementation remains compatible with custom `Cache` subclasses, while\n`MemoryCache`, `NoopCache`, and `SQLiteCache` provide backend-specific\nimplementations. SQLite bulk reads use one worker dispatch and one connection,\nsplitting large key sets into bounded SQL batches.\n\nAn opt-in warm-cache benchmark compares bulk reads across the memory, JSON, and\nSQLite providers without enforcing a machine-dependent performance threshold:\n\n```bash\nuv run pytest -s tests/unit/cache/test_cache_benchmark.py \\\n --run-cache-benchmark\n```\n\nThe benchmark reports median latency, entries per second, and time relative to\nSQLite for 1,000 entries. Setup time is excluded from the measurements.\n\nThe preregistration happens dynamically, e.g., `JsonCache` is only imported and registered if you request a `JsonCache` with `create_cache(CacheType.Json, ...)`. There is no need to manually import and register builtin cache providers when using `create_cache`.\n\nIf you want a clean factory with no preregistered cache providers then directly import `cache_factory` and bypass using `create_cache`. The downside is that `cache_factory.create` uses a dict for init args instead of the strongly typed `CacheConfig` used with `create_cache`.\n\n```python\nfrom graphrag_cache.cache_factory import cache_factory\nfrom graphrag_cache.json_cache import JsonCache\n\n# cache_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\ncache_factory.register(\"my_cache_impl\", JsonCache)\n\ncache = cache_factory.create(\n strategy=\"my_cache_impl\", init_args={\"some_setting\": \"...\"}\n)\n\n...\n```\n","isInternal":false,"tokens":830,"sizeBytes":4061},{"name":"README.md","path":"packages/graphrag-chunking/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-chunking/README.md","title":"graphrag-chunking Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG Chunking\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of text chunkers, a core config model, and a factory for acquiring instances.\n\n## Examples\n\n### Basic sentence chunking with nltk\n\nThe SentenceChunker class splits text into individual sentences by identifying sentence boundaries. It takes input text and returns a list where each element is a separate sentence, making it easy to process text at the sentence level.\n\n[Open the notebook to explore the basic sentence example code](example_notebooks/basic_sentence_example.ipynb)\n\n### Token chunking\n\nThe TokenChunker splits text into fixed-size chunks based on token count rather than sentence boundaries. It uses a tokenizer to encode text into tokens, then creates chunks of a specified size with configurable overlap between chunks.\n\n[Open the notebook to explore the token chunking example code](example_notebooks/token_chunking_example.ipynb)\n\n### Using the factory via helper util\n\nThe create_chunker factory function provides a configuration-driven approach to instantiate chunkers by accepting a ChunkingConfig object that specifies the chunking strategy and parameters. This allows for more flexible and maintainable code by separating chunker configuration from direct instantiation.\n\n[Open the notebook to explore the factory helper util example code](example_notebooks/factory_helper_util_example.ipynb)\n","isInternal":false,"tokens":356,"sizeBytes":1891},{"name":"README.md","path":"packages/graphrag-common/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-common/README.md","title":"graphrag-common Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG Common\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides utility modules for GraphRAG, including a flexible factory system for dependency injection and service registration, and a comprehensive configuration loading system with Pydantic model support, environment variable substitution, and automatic file discovery.\n\n## Factory module\n\nThe Factory class provides a flexible dependency injection pattern that can register and create instances of classes implementing a common interface using string-based strategies. It supports both transient scope (creates new instances on each request) and singleton scope (returns the same instance after first creation).\n\n[Open the notebook to explore the factory module example code](example_notebooks/factory_module_example.ipynb)\n\n## Config module\n\nThe load_config function provides a comprehensive configuration loading system that automatically discovers and parses YAML/JSON config files into Pydantic models with support for environment variable substitution and .env file loading. It offers flexible features like config overrides, custom parsers for different file formats, and automatically sets the working directory to the config file location for relative path resolution.\n\n[Open the notebook to explore the config module example code](example_notebooks/config_module_example.ipynb)\n","isInternal":false,"tokens":321,"sizeBytes":1838},{"name":"README.md","path":"packages/graphrag-input/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-input/README.md","title":"graphrag-input Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG Inputs\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides input document loading utilities for GraphRAG, supporting multiple file formats including CSV, JSON, JSON Lines, and plain text.\n\n## Supported File Types\n\nThe following four standard file formats are supported out of the box:\n\n- **CSV** - Tabular data with configurable column mappings\n- **JSON** - JSON files with configurable property paths\n- **JSON Lines** - Line-delimited JSON records\n- **Text** - Plain text files\n\n### Markitdown Support\n\nAdditionally, we support the `InputType.MarkItDown` format, which uses the [MarkItDown](https://github.com/microsoft/markitdown) library to import any supported file type. The MarkItDown converter can handle a wide variety of file formats including Office documents, PDFs, HTML, and more.\n\n**Note:** Additional optional dependencies may need to be installed depending on the file type you're processing. The choice of converter is determined by MarkItDowns's processing logic, which primarily uses the file extension to select the appropriate converter. Please refer to the [MarkItDown repository](https://github.com/microsoft/markitdown) for installation instructions and detailed information about supported formats.\n\n## Examples\n\nBasic usage with the factory:\n\n1. Import a pdf with MarkItDown:\n\n```bash\npip install 'markitdown[pdf]' # required dependency for pdf processing\n```\n\n2. YAML config example for above:\n\n```yaml\ninput:\n type: markitdown\n file_pattern: \".*\\\\.pdf$$\"\ninput_storage:\n type: file\n base_dir: \"input\"\n```\n\n[Open the notebook to explore the input example code](example_notebooks/input_example.ipynb)\n","isInternal":false,"tokens":452,"sizeBytes":2131},{"name":"README.md","path":"packages/graphrag-llm/graphrag_llm/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-llm/graphrag_llm/README.md","title":"graphrag_llm Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG LLM\n\nView the [notebooks](notebooks) for detailed examples.\n\n## Basic Completion\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.completion import LLMCompletion, create_completion\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMCompletionChunk, LLMCompletionResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nmodel_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_MODEL\"),\n azure_deployment_name=os.getenv(\"GRAPHRAG_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\nllm_completion: LLMCompletion = create_completion(model_config)\n\nresponse: LLMCompletionResponse | Iterator[LLMCompletionChunk] = (\n llm_completion.completion(\n messages=\"What is the capital of France?\",\n )\n)\n\nif isinstance(response, Iterator):\n # Streaming response\n for chunk in response:\n print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\nelse:\n # Non-streaming response\n print(response.choices[0].message.content)\n\n# Alternatively, you can use the utility function to gather the full response\n# The following is equivalent to the above logic. If all you care about is\n# the first choice response then you can use the gather_completion_response\n# utility function.\nresponse_text = gather_completion_response(response)\nprint(response_text)\n```\n\n## Basic Embedding\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.embedding import LLMEmbedding, create_embedding\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMEmbeddingResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nembedding_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"), # type: ignore\n azure_deployment_name=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\n\nllm_embedding: LLMEmbedding = create_embedding(embedding_config)\n\nembeddings: LLMEmbeddingResponse = llm_embedding.embedding(\n input=[\"Hello world\", \"How are you?\"]\n)\nfor data in embeddings.data:\n print(data.embedding[0:3])\n```","isInternal":false,"tokens":592,"sizeBytes":2508},{"name":"README.md","path":"packages/graphrag-llm/notebooks/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-llm/notebooks/README.md","title":"notebooks Documentation","category":"plugin-manifest","format":"markdown","content":"To run the notebooks you need to add a `.env` file to the `notebooks` directory with the following information\n\n```\nGRAPHRAG_MODEL=\"...\"\nGRAPHRAG_EMBEDDING_MODEL=\"...\"\nGRAPHRAG_API_BASE=\"...\"\n# API Key and version are optional\n# If not provided, Azure managed identity will be used\nGRAPHRAG_API_KEY=\"...\"\nGRAPHRAG_API_VERSION=\"...\"\n```","isInternal":false,"tokens":90,"sizeBytes":335},{"name":"README.md","path":"packages/graphrag-llm/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-llm/README.md","title":"graphrag-llm Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG LLM\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n## Basic Completion\n\nThis example demonstrates basic usage of the LLM library to interact with Azure OpenAI. It loads environment variables for API configuration, creates a ModelConfig for Azure OpenAI, and sends a simple question to the model. The code handles both streaming and non-streaming responses (streaming responses are printed chunk by chunk in real-time, while non-streaming responses are printed all at once). It also shows how to use the gather_completion_response utility function as a simpler alternative that automatically handles both response types and returns the complete text.\n\n[Open the notebook to explore the basic completion example code](example_notebooks/basic_completion_example.ipynb)\n\n## Basic Embedding\n\nThis examples demonstrates how to generate text embeddings using the GraphRAG LLM library with Azure OpenAI's embedding service. It loads API credentials from environment variables, creates a ModelConfig for the Azure embedding model and configures authentication to use either API key or Azure Managed Identity. The script then creates an embedding client and processes a batch of two text strings (\"Hello world\" and \"How are you?\") to generate their vector embeddings.\n\n[Open the notebook to explore the basic embeddings example code](example_notebooks/basic_embedding_example.ipynb)\n\nView the [notebooks](notebooks/README.md) for more examples.\n","isInternal":false,"tokens":367,"sizeBytes":1919},{"name":"README.md","path":"packages/graphrag-storage/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-storage/README.md","title":"graphrag-storage Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG Storage\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides a unified storage abstraction layer with support for multiple backends including file system, Azure Blob, Azure Cosmos, and memory storage. It features a factory-based creation system with configuration-driven setup and extensible architecture for implementing custom storage providers.\n\n## Basic\n\nThis example creates a file storage system using the GraphRAG storage package's configuration system. The example shows setting up file storage in a specified directory and demonstrates basic storage operations like setting and getting key-value pairs.\n\n[Open the notebook to explore the basic storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Custom Storage\n\nHere we create a custom storage implementation by extending the base Storage class and registering it with the GraphRAG storage system. Once registered, the custom storage can be instantiated through the factory pattern using either StorageConfig or directly via storage_factory, enabling extensible storage solutions for different backends.\n\n[Open the notebook to explore the custom storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Details\n\nBy default, the `create_storage` comes with the following storage providers registered that correspond to the entries in the `StorageType` enum.\n\n- `FileStorage`\n- `AzureBlobStorage`\n- `AzureCosmosStorage`\n- `MemoryStorage`\n\nThe preregistration happens dynamically, e.g., `FileStorage` is only imported and registered if you request a `FileStorage` with `create_storage(StorageType.File, ...)`. There is no need to manually import and register builtin storage providers when using `create_storage`.\n\nIf you want a clean factory with no preregistered storage providers then directly import `storage_factory` and bypass using `create_storage`. The downside is that `storage_factory.create` uses a dict for init args instead of the strongly typed `StorageConfig` used with `create_storage`.\n\n```python\nfrom graphrag_storage.storage_factory import storage_factory\nfrom graphrag_storage.file_storage import FileStorage\n\n# storage_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\nstorage_factory.register(\"my_storage_key\", FileStorage)\n\nstorage = storage_factory.create(\n strategy=\"my_storage_key\", init_args={\"base_dir\": \"...\", \"other_settings\": \"...\"}\n)\n\n...\n```\n","isInternal":false,"tokens":573,"sizeBytes":2984},{"name":"README.md","path":"packages/graphrag-vectors/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag-vectors/README.md","title":"graphrag-vectors Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG Vectors\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides vector store implementations for GraphRAG with support for multiple backends including LanceDB, Azure AI Search, and Azure Cosmos DB. It offers both a convenient configuration-driven API and direct factory access for creating and managing vector stores with flexible index schema definitions.\n\n## Basic usage with the utility function (recommended)\n\nThis demonstrates the recommended approach to create a vector store using the create_vector_store convenience function with configuration objects that specify the store type and index schema. The example shows setting up a LanceDB vector store with a defined index configuration, then connecting to it and creating the index for vector operations.\n\n[Open the notebook to explore the basic usage with utility function example code](example_notebooks/basic_usage_with_utility_function_example.ipynb)\n\n## Basic usage implementing the factory directly\n\nThis example shows a different approach to create vector stores by directly using the vector_store_factory with enum types and dictionary-based initialization arguments. This method provides more direct control over the factory creation process while bypassing the convenience function layer.\n\n[Open the notebook to explore the basic usage using factory directly example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Supported Vector Stores\n\n- **LanceDB**: Local vector database\n- **Azure AI Search**: Azure's managed search service with vector capabilities\n- **Azure Cosmos DB**: Azure's NoSQL database with vector search support\n\n## Custom Vector Store\n\nYou can register custom vector store implementations:\n\n[Open the notebook to explore the custom vector example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Configuration\n\nVector stores are configured using:\n\n- `VectorStoreConfig`: baseline parameters for the store\n- `IndexSchema`: Schema configuration for the specific index to create/connect to (index name, field names, vector size)\n","isInternal":false,"tokens":462,"sizeBytes":2537},{"name":"README.md","path":"packages/graphrag/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/graphrag/HEAD/packages/graphrag/README.md","title":"graphrag Documentation","category":"plugin-manifest","format":"markdown","content":"# GraphRAG\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n👉 [Microsoft Research Blog Post](https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/)
\n👉 [Read the docs](https://microsoft.github.io/graphrag)
\n👉 [GraphRAG Arxiv](https://arxiv.org/pdf/2404.16130)\n\n\n\n## Overview\n\nThe GraphRAG project is a data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs.\n\nTo learn more about GraphRAG and how it can be used to enhance your LLM's ability to reason about your private data, please visit the Microsoft Research Blog Post.\n\n## Quickstart\n\nTo get started with the GraphRAG system we recommend trying the [command line quickstart](https://microsoft.github.io/graphrag/get_started/).\n\n## Repository Guidance\n\nThis repository presents a methodology for using knowledge graph memory structures to enhance LLM outputs. Please note that the provided code serves as a demonstration and is not an officially supported Microsoft offering.\n\n⚠️ _Warning: GraphRAG indexing can be an expensive operation, please read all of the documentation to understand the process and costs involved, and start small._\n\n## Diving Deeper\n\n- To learn about our contribution guidelines, see [CONTRIBUTING.md](../../CONTRIBUTING.md)\n- To start developing _GraphRAG_, see [DEVELOPING.md](../../DEVELOPING.md)\n- Join the conversation and provide feedback in the [GitHub Discussions tab!](https://github.com/microsoft/graphrag/discussions)\n\n## Prompt Tuning\n\nUsing _GraphRAG_ with your data out of the box may not yield the best possible results.\nWe strongly recommend to fine-tune your prompts following the [Prompt Tuning Guide](https://microsoft.github.io/graphrag/prompt_tuning/overview/) in our documentation.\n\n## Versioning\n\nPlease see the [breaking changes](../../breaking-changes.md) document for notes on our approach to versioning the project.\n\n_Always run `uv run poe init --root [path] --force` between minor version bumps to ensure you have the latest config format. Run the provided migration notebook between major version bumps if you want to avoid re-indexing prior datasets. Note that this will overwrite your configuration and prompts, so back them up if necessary._\n\n## Responsible AI FAQ\n\nSee [RAI_TRANSPARENCY.md](../../RAI_TRANSPARENCY.md)\n\n- [What is GraphRAG?](../../RAI_TRANSPARENCY.md#what-is-graphrag)\n- [What can GraphRAG do?](../../RAI_TRANSPARENCY.md#what-can-graphrag-do)\n- [What are GraphRAG’s intended use(s)?](../../RAI_TRANSPARENCY.md#what-are-graphrags-intended-uses)\n- [How was GraphRAG evaluated? What metrics are used to measure performance?](../../RAI_TRANSPARENCY.md#how-was-graphrag-evaluated-what-metrics-are-used-to-measure-performance)\n- [What are the limitations of GraphRAG? How can users minimize the impact of GraphRAG’s limitations when using the system?](../../RAI_TRANSPARENCY.md#what-are-the-limitations-of-graphrag-how-can-users-minimize-the-impact-of-graphrags-limitations-when-using-the-system)\n- [What operational factors and settings allow for effective and responsible use of GraphRAG?](../../RAI_TRANSPARENCY.md#what-operational-factors-and-settings-allow-for-effective-and-responsible-use-of-graphrag)\n\n## Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft\ntrademarks or logos is subject to and must follow\n[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n\n## Privacy\n\n[Microsoft Privacy Statement](https://privacy.microsoft.com/en-us/privacystatement)\n","isInternal":false,"tokens":1205,"sizeBytes":5141}],"systemPromptSnippet":"\n\n# Migration Gotchas — verified fixes for this repo\n\nLoad this when a dependency bump breaks tests or `pyright`/`ruff` with a library API change.\nEach entry is a pattern that was actually hit and fixed in this codebase. Apply the minimal\nfix and keep it consistent with sibling code.\n\n## pandas 3.0\n\n### `DataFrame.swapaxes` removed → `np.array_split(df, n)` returns ndarrays\n\n`np.array_split` internally calls `np.swapaxes`, which used to delegate to\n`DataFrame.swapaxes` and return DataFrames with column names intact. In pandas 3.0\n`swapaxes` was removed (deprecated in 2.1), so `np.array_split(df, n)` now yields plain\nnumpy arrays. Rebuilding with `pd.DataFrame(fold)` produces integer `RangeIndex` columns,\nso later `df[\"some_column\"]` raises `KeyError`.\n\nSymptom: `KeyError: ''` with a traceback ending in\n`pandas/core/indexes/range.py ... get_loc`.\n\nFix — split positional indices instead of the frame, then select rows with `iloc`:\n\n```python\n# Broken under pandas 3.0\nreturn [pd.DataFrame(fold) for fold in np.array_split(reports, n)]\n\n# Fixed — preserves columns, dtypes, and even fold sizes\nreturn [reports.iloc[indices] for indices in np.array_split(np.arange(len(reports)), n)]\n```\n\n### `copy=` keyword removed from `merge`/`concat`/`join`/`set_axis` etc.\n\npandas 3.0 makes Copy-on-Write the default and drops the `copy=` parameter. Calls like\n`df.merge(other, copy=False)` or `pd.concat([...], copy=False)` raise `TypeError`.\nFix: delete the `copy=` argument — CoW already avoids the unnecessary copy.\n\n### Chained-assignment / `inplace` under Copy-on-Write\n\nWith CoW, mutating a slice (`df[mask][\"col\"] = x`) no longer writes back and may warn/error.\nAssign through `.loc`: `df.loc[mask, \"col\"] = x`. Reassign results of `inplace=True`-style\noperations rather than relying on in-place mutation of a view.\n\n## numpy 2.x\n\n- Removed aliases (`np.float_`, `np.int0`, `np.bool8`, `np.object0`, etc.) — use the builtin\n or the explicit sized dtype (`np.float64`, `np.bool_`).\n- `np.array_split` on a DataFrame no longer preserves the frame (see the pandas entry above).\n- Some functions moved out of the top-level namespace; import from the documented submodule.\n\n## ruff (preview mode active in this repo)\n\n- **RUF069 (float-equality-comparison):** `x == 0.0` / `!= 0.0` is flagged. Prefer a\n non-equality guard when semantically valid (`x <= 0.0` for a non-positive divide guard) or\n `math.isclose(...)` for tolerance checks. Avoid a blanket `# noqa` when a real fix exists.\n- **ASYNC119 (yield in context manager in async generator):** do not `yield` while holding a\n `with`/`async with` in an async generator. Materialize inside the block, then yield after\n it closes — matching the sibling providers:\n\n ```python\n with Path.open(path, \"r\", encoding=enc) as f:\n rows = list(csv.DictReader(f))\n for row in rows:\n yield transform(row)\n ```\n\n## pyright\n\n- Type stubs travel with majors: the `dev` group pins `pandas-stubs~=3.0`. When bumping\n pandas, bump the matching stubs so `pyright` reflects the new API.\n- After dependency changes, `pyright` may surface new optional/overload errors from updated\n stubs; fix at the call site rather than suppressing, unless the stub is demonstrably wrong.\n\n## General approach\n\n1. Read the actual traceback/diagnostic to the leaf frame — the failing library call and the\n changed symbol are usually right there.\n2. Check how a sibling module in the same package already handles the pattern and match it.\n3. Prefer a real, minimal fix over suppression; re-run `uv run poe check` and\n `uv run poe test_unit` after each fix to confirm.\n\n\n---\nname: update-deps\ndescription: >\n Update this GraphRAG uv-workspace monorepo's dependencies and repair the resulting\n breakages so every check and test passes again. Use when asked to update, upgrade, or\n bump dependencies/packages/versions, refresh or regenerate the lockfile, migrate to a new\n major of pandas/numpy/pydantic/pyarrow, resolve dependency-related test or lint failures,\n or run a \"dependency sweep\" — even if the user only names a single package. Covers editing\n pyproject.toml version specifiers across the root dev-deps and every packages/* member,\n re-locking with uv, and fixing code and tests for library API changes (e.g. pandas 3.0).\n USE FOR: update dependencies, upgrade packages, bump versions, dependency sweep, uv lock,\n uv sync, refresh lockfile, migrate pandas/numpy, \"deps broke the tests\", pyproject bump.\nuser-invocable: true\n---\n\n# Update Dependencies (GraphRAG monorepo)\n\n## Goal\n\nRaise dependency versions across this uv workspace, re-lock, and fix any code or test\nfallout until `uv run poe check` and `uv run poe test_unit` are both green — without\ntouching the version machinery that the release process owns.\n\n## Hard rules (non-negotiable)\n\n1. **Only the Microsoft feed proxy.** All resolves and syncs MUST use\n `https://packagefeedproxy.microsoft.io/pypi/simple` (the `[[tool.uv.index]]` configured in\n the root `pyproject.toml`). Never resolve against public PyPI, never add\n `--index`/`--default-index`/`--index-url` overrides, and never set `UV_INDEX_URL` or\n `PIP_INDEX_URL` to pypi.org. Do not disable or reorder the configured index.\n2. **Target versions must be at least 7 days old.** Only bump to a version that was released\n more than 7 days ago. The proxy does not serve versions published within the last week —\n syncing to a too-new version WILL fail. Before pinning a specific version, verify its\n release date and pick the newest release older than 7 days; otherwise let the specifier\n float and let `uv lock` choose (it can only see eligible versions on the proxy anyway).\n\n## Layout facts\n\n- This is a **uv workspace monorepo**. The root [`pyproject.toml`](../../../pyproject.toml)\n holds the `dev` group under `[dependency-groups]` and the `[tool.poe.tasks]` commands.\n- Runtime dependencies live in each member's `packages/*/pyproject.toml` under\n `[project] dependencies`.\n- Workspace members are wired via `[tool.uv.sources]` (`{ workspace = true }`) and pinned to\n each other with `graphrag-*==X.Y.Z` lines.\n- Package resolution goes through a Microsoft-internal index (`[[tool.uv.index]]`); expect\n that feed to be used, not public PyPI directly.\n\n## Do NOT edit these (release-owned)\n\n1. **Cross-package pins** `graphrag-cache==...`, `graphrag-llm==...`, etc. in any package.\n These are rewritten automatically by\n [`scripts/update_workspace_dependency_versions.py`](../../../scripts/update_workspace_dependency_versions.py)\n from the semversioner version. Hand-editing them causes drift.\n2. **`[project] version` fields** — managed by semversioner (\"do not change the version\n here manually\").\n3. **`graspologic-native>=1.2,<1.3`** — held below 1.3 on purpose; 1.3.x changes Leiden\n clustering output and breaks golden regression data. Only bump with a deliberate\n golden-data refresh, and say so explicitly.\n\n## Process\n\n1. **Baseline first.** Confirm a clean working tree and that checks/tests already pass\n before changing anything, so later failures are attributable to the bump:\n - `uv run poe check`\n - `uv run poe test_unit`\n Prefer a dedicated branch (e.g. `dep-sweep`).\n\n2. **Decide the scope.** Either a targeted set of packages the user named, or a full sweep.\n Edit the `~=`/`>=`/`<` specifiers in the relevant `[project] dependencies`\n (`packages/*/pyproject.toml`) and the root `dev` group. Leave the release-owned lines\n above untouched.\n\n3. **Resolve and lock.** All of these use the configured Microsoft feed proxy — do not pass\n any index override (see Hard rules).\n - For a full \"get latest allowed\" pass: `uv lock --upgrade`.\n - For targeted bumps after editing specifiers: `uv lock`.\n - Then install: `uv sync --all-packages`.\n If resolution fails, read the conflict, relax/adjust the offending specifier, and re-lock.\n Do not delete `uv.lock` to force it.\n - If a sync fails to find a version you just pinned, it is almost certainly younger than 7\n days on the proxy — step down to the newest release older than one week.\n\n4. **Static checks.** Run `uv run poe check` (this is `ruff format --check` + `ruff check` +\n `pyright`). Apply safe autofixes with `uv run poe fix`; format with `uv run poe format`.\n Fix remaining lint/type errors by hand — see Gotchas and the migration reference.\n\n5. **Tests.** Run `uv run poe test_unit` (NOT `uv run poe test`, which runs the full coverage\n suite). Run `uv run poe test_verbs` and `uv run poe test_integration` when the change is\n broad or touches indexing/query. Investigate every new failure.\n\n6. **Repair breakages.** For test/type failures caused by a library's API change, load\n [`references/migration-gotchas.md`](references/migration-gotchas.md) and apply the\n documented pattern. Keep fixes minimal and consistent with sibling code; prefer a real\n fix over a `# noqa`.\n\n7. **Record the change.** Add a changelog entry:\n `uv run semversioner add-change -t patch -d \"\"` (use `minor`/`major`\n only if the user's intent warrants it).\n\n8. **Final verification.** Re-run `uv run poe check` and `uv run poe test_unit`; both must be\n green (see the known-flake note below before calling a failure a regression).\n\n## Gotchas (this repo)\n\n- **`test_unit`, not `test`.** `poe test` runs coverage over everything and is slow; use\n `test_unit` for the fast feedback loop.\n- **Ruff runs in preview mode** (`preview = true`, `target-version = \"py310\"`). Preview-only\n rules such as `RUF069` (float equality) and `ASYNC119` fire here even though they may not\n in other repos.\n- **Known pre-existing flake:**\n `tests/unit/indexing/test_profiling.py::TestWorkflowProfiler::test_handles_exception_in_context`\n is timing-sensitive and can fail intermittently — it is not a dependency regression.\n- **`uv sync --all-packages`** (not bare `uv sync`) to install every workspace member.\n- **pandas is on the 3.0 line and numpy on 2.x.** Their major-version API changes are the\n usual source of post-bump breakage — see the reference file.\n- Version-bump edits touch many `pyproject.toml` files; make sure you did not accidentally\n modify a `graphrag-*==` pin or a `version` field while editing nearby specifiers.\n\n## Load-on-demand reference\n\nWhen a bump breaks tests or type-checking with a library API change (especially pandas or\nnumpy), read [`references/migration-gotchas.md`](references/migration-gotchas.md) for\nverified, repo-specific fix patterns before improvising.\n\n## Completion checklist\n\n- [ ] All resolves/syncs used the `packagefeedproxy.microsoft.io` index; no public-PyPI or\n index-override was introduced.\n- [ ] No dependency was bumped to a version released within the last 7 days.\n- [ ] Only intended specifiers changed; no `graphrag-*==` pin or `version` field edited.\n- [ ] `uv.lock` regenerated via `uv lock`/`uv lock --upgrade` (not hand-edited or deleted).\n- [ ] `uv run poe check` passes (ruff format, ruff lint, pyright).\n- [ ] `uv run poe test_unit` passes (ignoring only the known profiling flake).\n- [ ] Broader suites run if the change was broad (`test_verbs`/`test_integration`).\n- [ ] A semversioner changelog entry was added.\n- [ ] Any risky/held pin (e.g. `graspologic-native`) left in place unless explicitly bumped.\n\n\n# GraphRAG Cache\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of utilities to handle GraphRAG caching implementation.\n\n### Basic\n\nThis example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system.\n\n[Open the notebook to explore the basic example code](example_notebooks/basic_cache_example.ipynb)\n\n### Custom Cache\n\nThis example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.\n\n[Open the notebook to explore the basic custom example code](example_notebooks/custom_cache_example.ipynb)\n\n#### Details\n\nBy default, the `create_cache` comes with the following cache providers registered that correspond to the entries in the `CacheType` enum.\n\n- `JsonCache`\n- `MemoryCache`\n- `NoopCache`\n- `SQLiteCache`\n\n`SQLiteCache` stores JSON-serializable values in a local SQLite database. It uses\nWAL mode and short-lived connections so multiple workers can safely share the\nsame database. Configure it with:\n\n```yaml\ncache:\n type: sqlite\n storage:\n type: file\n base_dir: cache\n database_name: cache.db\n```\n\nThe database is created within the configured file storage. SQLite requires\nlocal random-access file operations, so blob and Cosmos storage are not\nsupported. Child caches use isolated namespaces within the same database.\n\nAll cache providers expose `await cache.get_many(keys)` for bulk reads. The\nresult maps each existing, valid key to the same decoded value returned by\n`get()`. Missing and malformed entries are omitted, duplicate input keys appear\nat most once, and an empty input returns an empty dictionary. The default\nimplementation remains compatible with custom `Cache` subclasses, while\n`MemoryCache`, `NoopCache`, and `SQLiteCache` provide backend-specific\nimplementations. SQLite bulk reads use one worker dispatch and one connection,\nsplitting large key sets into bounded SQL batches.\n\nAn opt-in warm-cache benchmark compares bulk reads across the memory, JSON, and\nSQLite providers without enforcing a machine-dependent performance threshold:\n\n```bash\nuv run pytest -s tests/unit/cache/test_cache_benchmark.py \\\n --run-cache-benchmark\n```\n\nThe benchmark reports median latency, entries per second, and time relative to\nSQLite for 1,000 entries. Setup time is excluded from the measurements.\n\nThe preregistration happens dynamically, e.g., `JsonCache` is only imported and registered if you request a `JsonCache` with `create_cache(CacheType.Json, ...)`. There is no need to manually import and register builtin cache providers when using `create_cache`.\n\nIf you want a clean factory with no preregistered cache providers then directly import `cache_factory` and bypass using `create_cache`. The downside is that `cache_factory.create` uses a dict for init args instead of the strongly typed `CacheConfig` used with `create_cache`.\n\n```python\nfrom graphrag_cache.cache_factory import cache_factory\nfrom graphrag_cache.json_cache import JsonCache\n\n# cache_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\ncache_factory.register(\"my_cache_impl\", JsonCache)\n\ncache = cache_factory.create(\n strategy=\"my_cache_impl\", init_args={\"some_setting\": \"...\"}\n)\n\n...\n```\n\n\n# GraphRAG Chunking\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package contains a collection of text chunkers, a core config model, and a factory for acquiring instances.\n\n## Examples\n\n### Basic sentence chunking with nltk\n\nThe SentenceChunker class splits text into individual sentences by identifying sentence boundaries. It takes input text and returns a list where each element is a separate sentence, making it easy to process text at the sentence level.\n\n[Open the notebook to explore the basic sentence example code](example_notebooks/basic_sentence_example.ipynb)\n\n### Token chunking\n\nThe TokenChunker splits text into fixed-size chunks based on token count rather than sentence boundaries. It uses a tokenizer to encode text into tokens, then creates chunks of a specified size with configurable overlap between chunks.\n\n[Open the notebook to explore the token chunking example code](example_notebooks/token_chunking_example.ipynb)\n\n### Using the factory via helper util\n\nThe create_chunker factory function provides a configuration-driven approach to instantiate chunkers by accepting a ChunkingConfig object that specifies the chunking strategy and parameters. This allows for more flexible and maintainable code by separating chunker configuration from direct instantiation.\n\n[Open the notebook to explore the factory helper util example code](example_notebooks/factory_helper_util_example.ipynb)\n\n\n# GraphRAG Common\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides utility modules for GraphRAG, including a flexible factory system for dependency injection and service registration, and a comprehensive configuration loading system with Pydantic model support, environment variable substitution, and automatic file discovery.\n\n## Factory module\n\nThe Factory class provides a flexible dependency injection pattern that can register and create instances of classes implementing a common interface using string-based strategies. It supports both transient scope (creates new instances on each request) and singleton scope (returns the same instance after first creation).\n\n[Open the notebook to explore the factory module example code](example_notebooks/factory_module_example.ipynb)\n\n## Config module\n\nThe load_config function provides a comprehensive configuration loading system that automatically discovers and parses YAML/JSON config files into Pydantic models with support for environment variable substitution and .env file loading. It offers flexible features like config overrides, custom parsers for different file formats, and automatically sets the working directory to the config file location for relative path resolution.\n\n[Open the notebook to explore the config module example code](example_notebooks/config_module_example.ipynb)\n\n\n# GraphRAG Inputs\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides input document loading utilities for GraphRAG, supporting multiple file formats including CSV, JSON, JSON Lines, and plain text.\n\n## Supported File Types\n\nThe following four standard file formats are supported out of the box:\n\n- **CSV** - Tabular data with configurable column mappings\n- **JSON** - JSON files with configurable property paths\n- **JSON Lines** - Line-delimited JSON records\n- **Text** - Plain text files\n\n### Markitdown Support\n\nAdditionally, we support the `InputType.MarkItDown` format, which uses the [MarkItDown](https://github.com/microsoft/markitdown) library to import any supported file type. The MarkItDown converter can handle a wide variety of file formats including Office documents, PDFs, HTML, and more.\n\n**Note:** Additional optional dependencies may need to be installed depending on the file type you're processing. The choice of converter is determined by MarkItDowns's processing logic, which primarily uses the file extension to select the appropriate converter. Please refer to the [MarkItDown repository](https://github.com/microsoft/markitdown) for installation instructions and detailed information about supported formats.\n\n## Examples\n\nBasic usage with the factory:\n\n1. Import a pdf with MarkItDown:\n\n```bash\npip install 'markitdown[pdf]' # required dependency for pdf processing\n```\n\n2. YAML config example for above:\n\n```yaml\ninput:\n type: markitdown\n file_pattern: \".*\\\\.pdf$$\"\ninput_storage:\n type: file\n base_dir: \"input\"\n```\n\n[Open the notebook to explore the input example code](example_notebooks/input_example.ipynb)\n\n\n# GraphRAG LLM\n\nView the [notebooks](notebooks) for detailed examples.\n\n## Basic Completion\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.completion import LLMCompletion, create_completion\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMCompletionChunk, LLMCompletionResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nmodel_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_MODEL\"),\n azure_deployment_name=os.getenv(\"GRAPHRAG_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\nllm_completion: LLMCompletion = create_completion(model_config)\n\nresponse: LLMCompletionResponse | Iterator[LLMCompletionChunk] = (\n llm_completion.completion(\n messages=\"What is the capital of France?\",\n )\n)\n\nif isinstance(response, Iterator):\n # Streaming response\n for chunk in response:\n print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\nelse:\n # Non-streaming response\n print(response.choices[0].message.content)\n\n# Alternatively, you can use the utility function to gather the full response\n# The following is equivalent to the above logic. If all you care about is\n# the first choice response then you can use the gather_completion_response\n# utility function.\nresponse_text = gather_completion_response(response)\nprint(response_text)\n```\n\n## Basic Embedding\n\n```python\nimport os\nfrom collections.abc import AsyncIterator, Iterator\n\nfrom graphrag_llm.embedding import LLMEmbedding, create_embedding\nfrom graphrag_llm.config import ModelConfig\nfrom graphrag_llm.types import LLMEmbeddingResponse\nfrom graphrag_llm.utils import (\n gather_completion_response,\n)\n\napi_key = os.getenv(\"GRAPHRAG_API_KEY\")\nembedding_config = ModelConfig(\n model_provider=\"azure\",\n model=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"), # type: ignore\n azure_deployment_name=os.getenv(\"GRAPHRAG_EMBEDDING_MODEL\"),\n api_base=os.getenv(\"GRAPHRAG_API_BASE\"),\n api_version=os.getenv(\"GRAPHRAG_API_VERSION\"),\n api_key=api_key,\n azure_managed_identity=not api_key,\n)\n\nllm_embedding: LLMEmbedding = create_embedding(embedding_config)\n\nembeddings: LLMEmbeddingResponse = llm_embedding.embedding(\n input=[\"Hello world\", \"How are you?\"]\n)\nfor data in embeddings.data:\n print(data.embedding[0:3])\n```\n\n\nTo run the notebooks you need to add a `.env` file to the `notebooks` directory with the following information\n\n```\nGRAPHRAG_MODEL=\"...\"\nGRAPHRAG_EMBEDDING_MODEL=\"...\"\nGRAPHRAG_API_BASE=\"...\"\n# API Key and version are optional\n# If not provided, Azure managed identity will be used\nGRAPHRAG_API_KEY=\"...\"\nGRAPHRAG_API_VERSION=\"...\"\n```\n\n\n# GraphRAG LLM\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n## Basic Completion\n\nThis example demonstrates basic usage of the LLM library to interact with Azure OpenAI. It loads environment variables for API configuration, creates a ModelConfig for Azure OpenAI, and sends a simple question to the model. The code handles both streaming and non-streaming responses (streaming responses are printed chunk by chunk in real-time, while non-streaming responses are printed all at once). It also shows how to use the gather_completion_response utility function as a simpler alternative that automatically handles both response types and returns the complete text.\n\n[Open the notebook to explore the basic completion example code](example_notebooks/basic_completion_example.ipynb)\n\n## Basic Embedding\n\nThis examples demonstrates how to generate text embeddings using the GraphRAG LLM library with Azure OpenAI's embedding service. It loads API credentials from environment variables, creates a ModelConfig for the Azure embedding model and configures authentication to use either API key or Azure Managed Identity. The script then creates an embedding client and processes a batch of two text strings (\"Hello world\" and \"How are you?\") to generate their vector embeddings.\n\n[Open the notebook to explore the basic embeddings example code](example_notebooks/basic_embedding_example.ipynb)\n\nView the [notebooks](notebooks/README.md) for more examples.\n\n\n# GraphRAG Storage\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides a unified storage abstraction layer with support for multiple backends including file system, Azure Blob, Azure Cosmos, and memory storage. It features a factory-based creation system with configuration-driven setup and extensible architecture for implementing custom storage providers.\n\n## Basic\n\nThis example creates a file storage system using the GraphRAG storage package's configuration system. The example shows setting up file storage in a specified directory and demonstrates basic storage operations like setting and getting key-value pairs.\n\n[Open the notebook to explore the basic storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Custom Storage\n\nHere we create a custom storage implementation by extending the base Storage class and registering it with the GraphRAG storage system. Once registered, the custom storage can be instantiated through the factory pattern using either StorageConfig or directly via storage_factory, enabling extensible storage solutions for different backends.\n\n[Open the notebook to explore the custom storage example code](example_notebooks/basic_storage_example.ipynb)\n\n## Details\n\nBy default, the `create_storage` comes with the following storage providers registered that correspond to the entries in the `StorageType` enum.\n\n- `FileStorage`\n- `AzureBlobStorage`\n- `AzureCosmosStorage`\n- `MemoryStorage`\n\nThe preregistration happens dynamically, e.g., `FileStorage` is only imported and registered if you request a `FileStorage` with `create_storage(StorageType.File, ...)`. There is no need to manually import and register builtin storage providers when using `create_storage`.\n\nIf you want a clean factory with no preregistered storage providers then directly import `storage_factory` and bypass using `create_storage`. The downside is that `storage_factory.create` uses a dict for init args instead of the strongly typed `StorageConfig` used with `create_storage`.\n\n```python\nfrom graphrag_storage.storage_factory import storage_factory\nfrom graphrag_storage.file_storage import FileStorage\n\n# storage_factory has no preregistered providers so you must register any\n# providers you plan on using.\n# May also register a custom implementation, see above for example.\nstorage_factory.register(\"my_storage_key\", FileStorage)\n\nstorage = storage_factory.create(\n strategy=\"my_storage_key\", init_args={\"base_dir\": \"...\", \"other_settings\": \"...\"}\n)\n\n...\n```\n\n\n# GraphRAG Vectors\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\nThis package provides vector store implementations for GraphRAG with support for multiple backends including LanceDB, Azure AI Search, and Azure Cosmos DB. It offers both a convenient configuration-driven API and direct factory access for creating and managing vector stores with flexible index schema definitions.\n\n## Basic usage with the utility function (recommended)\n\nThis demonstrates the recommended approach to create a vector store using the create_vector_store convenience function with configuration objects that specify the store type and index schema. The example shows setting up a LanceDB vector store with a defined index configuration, then connecting to it and creating the index for vector operations.\n\n[Open the notebook to explore the basic usage with utility function example code](example_notebooks/basic_usage_with_utility_function_example.ipynb)\n\n## Basic usage implementing the factory directly\n\nThis example shows a different approach to create vector stores by directly using the vector_store_factory with enum types and dictionary-based initialization arguments. This method provides more direct control over the factory creation process while bypassing the convenience function layer.\n\n[Open the notebook to explore the basic usage using factory directly example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Supported Vector Stores\n\n- **LanceDB**: Local vector database\n- **Azure AI Search**: Azure's managed search service with vector capabilities\n- **Azure Cosmos DB**: Azure's NoSQL database with vector search support\n\n## Custom Vector Store\n\nYou can register custom vector store implementations:\n\n[Open the notebook to explore the custom vector example code](example_notebooks/basic_usage_factory_example.ipynb)\n\n## Configuration\n\nVector stores are configured using:\n\n- `VectorStoreConfig`: baseline parameters for the store\n- `IndexSchema`: Schema configuration for the specific index to create/connect to (index name, field names, vector size)\n\n\n# GraphRAG\n\n> [!WARNING]\n> GraphRAG is a research project that explores the functional use of graphs to form a targeted context for question answering. Since our first release in July 2024 the capabilities of frontier models have changed dramatically, and our portfolio of research projects has diversified to match. This project is largely in maintenance mode, and won't be accepting new PRs or implementing new features. We'll perform bug fixes and dependency updates as appropriate, particularly to address CVEs as they arise.\n\n👉 [Microsoft Research Blog Post](https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/)
\n👉 [Read the docs](https://microsoft.github.io/graphrag)
\n👉 [GraphRAG Arxiv](https://arxiv.org/pdf/2404.16130)\n\n\n\n## Overview\n\nThe GraphRAG project is a data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs.\n\nTo learn more about GraphRAG and how it can be used to enhance your LLM's ability to reason about your private data, please visit the Microsoft Research Blog Post.\n\n## Quickstart\n\nTo get started with the GraphRAG system we recommend trying the [command line quickstart](https://microsoft.github.io/graphrag/get_started/).\n\n## Repository Guidance\n\nThis repository presents a methodology for using knowledge graph memory structures to enhance LLM outputs. Please note that the provided code serves as a demonstration and is not an officially supported Microsoft offering.\n\n⚠️ _Warning: GraphRAG indexing can be an expensive operation, please read all of the documentation to understand the process and costs involved, and start small._\n\n## Diving Deeper\n\n- To learn about our contribution guidelines, see [CONTRIBUTING.md](../../CONTRIBUTING.md)\n- To start developing _GraphRAG_, see [DEVELOPING.md](../../DEVELOPING.md)\n- Join the conversation and provide feedback in the [GitHub Discussions tab!](https://github.com/microsoft/graphrag/discussions)\n\n## Prompt Tuning\n\nUsing _GraphRAG_ with your data out of the box may not yield the best possible results.\nWe strongly recommend to fine-tune your prompts following the [Prompt Tuning Guide](https://microsoft.github.io/graphrag/prompt_tuning/overview/) in our documentation.\n\n## Versioning\n\nPlease see the [breaking changes](../../breaking-changes.md) document for notes on our approach to versioning the project.\n\n_Always run `uv run poe init --root [path] --force` between minor version bumps to ensure you have the latest config format. Run the provided migration notebook between major version bumps if you want to avoid re-indexing prior datasets. Note that this will overwrite your configuration and prompts, so back them up if necessary._\n\n## Responsible AI FAQ\n\nSee [RAI_TRANSPARENCY.md](../../RAI_TRANSPARENCY.md)\n\n- [What is GraphRAG?](../../RAI_TRANSPARENCY.md#what-is-graphrag)\n- [What can GraphRAG do?](../../RAI_TRANSPARENCY.md#what-can-graphrag-do)\n- [What are GraphRAG’s intended use(s)?](../../RAI_TRANSPARENCY.md#what-are-graphrags-intended-uses)\n- [How was GraphRAG evaluated? What metrics are used to measure performance?](../../RAI_TRANSPARENCY.md#how-was-graphrag-evaluated-what-metrics-are-used-to-measure-performance)\n- [What are the limitations of GraphRAG? How can users minimize the impact of GraphRAG’s limitations when using the system?](../../RAI_TRANSPARENCY.md#what-are-the-limitations-of-graphrag-how-can-users-minimize-the-impact-of-graphrags-limitations-when-using-the-system)\n- [What operational factors and settings allow for effective and responsible use of GraphRAG?](../../RAI_TRANSPARENCY.md#what-operational-factors-and-settings-allow-for-effective-and-responsible-use-of-graphrag)\n\n## Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft\ntrademarks or logos is subject to and must follow\n[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n\n## Privacy\n\n[Microsoft Privacy Statement](https://privacy.microsoft.com/en-us/privacystatement)\n"}