{"owner":"topoteretes","repo":"cognee","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md","CLAUDE.md"],"skills":{"AGENTS.md":"## Repository Guidelines\n\nThis document summarizes how to work with the cognee repository: how it’s organized, how to build, test, lint, and contribute. It mirrors our actual tooling and CI while providing quick commands for local development.\n\n## Project Structure & Module Organization\n\n- `cognee/`: Core Python library and API.\n  - `api/`: FastAPI application and versioned routers (add, cognify, memify, search, delete, users, datasets, responses, visualize, settings, sync, update, checks).\n  - `cli/`: CLI entry points and subcommands invoked via `cognee` / `cognee-cli`.\n  - `infrastructure/`: Databases, LLM providers, embeddings, loaders, and storage adapters.\n  - `modules/`: Domain logic (graph, retrieval, ontology, users, processing, observability, etc.).\n  - `tasks/`: Reusable tasks (e.g., code graph, web scraping, storage). Extend with new tasks here.\n  - `eval_framework/`: Evaluation utilities and adapters.\n  - `shared/`: Cross-cutting helpers (logging, settings, utils).\n  - `tests/`: Unit, integration, CLI, and end-to-end tests organized by feature.\n  - `__main__.py`: Entrypoint to route to CLI.\n- `cognee-mcp/`: Model Context Protocol server exposing cognee as MCP tools (SSE/HTTP/stdio). Contains its own README and Dockerfile.\n- `cognee-frontend/`: Next.js UI for local development and demos.\n- `distributed/`: Utilities for distributed execution (Modal, workers, queues).\n- `examples/`: Example scripts demonstrating the public APIs and features (graph, code graph, multimodal, permissions, etc.).\n- `notebooks/`: Jupyter notebooks for demos and tutorials.\n- `alembic/`: Database migrations for relational backends.\n\nNotes:\n- Co-locate feature-specific helpers under their respective package (`modules/`, `infrastructure/`, or `tasks/`).\n- Extend the system by adding new tasks, loaders, or retrievers rather than modifying core pipeline mechanisms.\n\n## Build, Test, and Development Commands\n\nPython (root) – requires Python >= 3.10 and < 3.14. We recommend `uv` for speed and reproducibility.\n\n- Create/refresh env and install dev deps:\n```bash\nuv sync --dev --all-extras --reinstall\n```\n\n- Run the CLI (examples):\n```bash\nuv run cognee-cli add \"Cognee turns documents into AI memory.\"\nuv run cognee-cli cognify\nuv run cognee-cli search \"What does cognee do?\"\nuv run cognee-cli -ui   # Launches UI, backend API, and MCP server together\n```\n\n- Start the FastAPI server directly:\n```bash\nuv run python -m cognee.api.client\n```\n\n- Run tests (CI mirrors these commands):\n```bash\nuv run pytest cognee/tests/unit/ -v\nuv run pytest cognee/tests/integration/ -v\n```\n\n- Lint and format (ruff):\n```bash\nuv run ruff check .\nuv run ruff format .\n```\n\n- Optional static type checks (ty):\n```bash\nuv run ty check .\n```\n\nMCP Server (`cognee-mcp/`):\n\n- Install and run locally:\n```bash\ncd cognee-mcp\nuv sync --dev --all-extras --reinstall\nuv run python src/server.py               # stdio (default)\nuv run python src/server.py --transport sse\nuv run python src/server.py --transport http --host 127.0.0.1 --port 8000 --path /mcp\n```\n\n- API Mode (connect to a running Cognee API):\n```bash\nuv run python src/server.py --transport sse --api-url http://localhost:8000 --api-token YOUR_TOKEN\n```\n\n- Docker quickstart (examples): see `cognee-mcp/README.md` for full details\n```bash\ndocker run -e TRANSPORT_MODE=http --env-file ./.env -p 8000:8000 --rm -it cognee/cognee-mcp:main\n```\n\nFrontend (`cognee-frontend/`):\n```bash\ncd cognee-frontend\nnpm install\nnpm run dev     # Next.js dev server\nnpm run lint    # ESLint\nnpm run build && npm start\n```\n\n## Coding Style & Naming Conventions\n\nPython:\n- 4-space indentation, modules and functions in `snake_case`, classes in `PascalCase`.\n- Public APIs should be type-annotated where practical. Make sure type defined in API signature will be properly displayed in Swagger UI docs. For example this definition: content_type: Optional[str] = Form(default=None) maps to \"string\" as the default in Swagger docs for content_type, but it should be None/null instead.\n- Use `ruff format` before committing; `ruff check` enforces import hygiene and style (line-length 100 configured in `pyproject.toml`).\n- Prefer explicit, structured error handling. Use shared logging utilities in `cognee.shared.logging_utils`.\n\nMCP server and Frontend:\n- Follow the local `README.md` and ESLint/TypeScript configuration in `cognee-frontend/`.\n\n## Testing Guidelines\n\n- Place Python tests under `cognee/tests/`.\n  - Unit tests: `cognee/tests/unit/`\n  - Integration tests: `cognee/tests/integration/`\n  - CLI tests: `cognee/tests/cli_tests/`\n- Name test files `test_*.py`. Use `pytest.mark.asyncio` for async tests.\n- Avoid external state; rely on test fixtures and the CI-provided env vars when LLM/embedding providers are required. See CI workflows under `.github/workflows/` for expected environment variables.\n- When adding public APIs, provide/update targeted examples under `examples/python/`.\n\n## Commit & Pull Request Guidelines\n\n- Use clear, imperative subjects (≤ 72 chars) and conventional commit styling in PR titles. Our CI validates semantic PR titles (see `.github/workflows/pr_lint`). Examples:\n  - `feat(graph): add temporal edge weighting`\n  - `fix(api): handle missing auth cookie`\n  - `docs: update installation instructions`\n- Reference related issues/discussions in the PR body and provide brief context.\n- PRs should describe scope, list local test commands run, and mention any impacts on MCP server or UI if applicable.\n- Sign commits and affirm the DCO (see `CONTRIBUTING.md`).\n\n## CI Mirrors Local Commands\n\nOur GitHub Actions run the same ruff checks and pytest suites shown above (`.github/workflows/basic_tests.yml` and related workflows). Use the commands in this document locally to minimize CI surprises.\n","CLAUDE.md":"# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Project Overview\n\nCognee is an open-source AI memory platform that transforms raw data into persistent knowledge graphs for AI agents. It replaces traditional RAG (Retrieval-Augmented Generation) with an ECL (Extract, Cognify, Load) pipeline combining vector search, graph databases, and LLM-powered entity extraction.\n\n**Requirements**: Python 3.10 - 3.14\n\n## Development Commands\n\n### Setup\n```bash\n# Create virtual environment (recommended: uv)\nuv venv && source .venv/bin/activate\n\n# Install with pip, poetry, or uv\nuv pip install -e .\n\n# Install with dev dependencies\nuv pip install -e \".[dev]\"\n\n# Install with specific extras\nuv pip install -e \".[postgres,neo4j,docs,chromadb]\"\n\n# Set up pre-commit hooks\npre-commit install\n```\n\n### Available Installation Extras\n- **postgres** / **postgres-binary** - PostgreSQL + PGVector support (also enables the Postgres session-cache backend, `CACHE_BACKEND=postgres`)\n- **neo4j** - Neo4j graph database support\n- **neptune** - AWS Neptune support\n- **chromadb** - ChromaDB vector database\n- **docs** - Document processing (unstructured library)\n- **scraping** - Web scraping (Tavily, BeautifulSoup, Playwright)\n- **langchain** - LangChain integration\n- **llama-index** - LlamaIndex integration\n- **anthropic** - Anthropic Claude models\n- **gemini** - Google Gemini models\n- **ollama** - Ollama local models\n- **mistral** - Mistral AI models\n- **groq** - Groq API support\n- **llama-cpp** - Llama.cpp local inference\n- **huggingface** - HuggingFace transformers\n- **aws** - S3 storage backend\n- **redis** - Redis caching\n- **graphiti** - Graphiti-core integration\n- **baml** - BAML structured output\n- **dlt** - Data load tool (dlt) integration\n- **docling** - Docling document processing\n- **codegraph** - Code graph extraction\n- **evals** - Evaluation tools\n- **deepeval** - DeepEval testing framework\n- **posthog** - PostHog analytics\n- **tracing** - OpenTelemetry tracing\n- **distributed** - Modal distributed execution\n- **dev** - All development tools (pytest, ty, ruff, etc.)\n- **debug** - Debugpy for debugging\n\n### Testing\n```bash\n# Run all tests\npytest\n\n# Run with coverage\npytest --cov=cognee --cov-report=html\n\n# Run specific test file\npytest cognee/tests/test_custom_model.py\n\n# Run specific test function\npytest cognee/tests/test_custom_model.py::test_function_name\n\n# Run async tests\npytest -v cognee/tests/integration/\n\n# Run unit tests only\npytest cognee/tests/unit/\n\n# Run integration tests only\npytest cognee/tests/integration/\n```\n\n### Code Quality\n```bash\n# Run ruff linter\nruff check .\n\n# Run ruff formatter\nruff format .\n\n# Run both linting and formatting (pre-commit)\npre-commit run --all-files\n\n# Type checking with ty\nty check .\n```\n\n### Running Cognee\n```bash\n# Using Python SDK\nuv run python examples/demos/simple_cognee_example.py\n\n# Using CLI\ncognee-cli add \"Your text here\"\ncognee-cli cognify\ncognee-cli search \"Your query\"\ncognee-cli delete --all\n\n# Launch full stack with UI\ncognee-cli -ui\n```\n\n## Architecture Overview\n\n### Core Workflow: add → cognify → search/memify\n\n1. **add()** - Ingest data (files, URLs, text) into datasets\n2. **cognify()** - Extract entities/relationships and build knowledge graph\n3. **search()** - Query knowledge using various retrieval strategies\n4. **memify()** - Enrich graph with additional context and rules\n\n### Key Architectural Patterns\n\n#### 1. Pipeline-Based Processing\nAll data flows through task-based pipelines (`cognee/modules/pipelines/`). Tasks are composable units that can run sequentially or in parallel. Example pipeline tasks: `classify_documents`, `extract_graph_from_data`, `add_data_points`.\n\n#### 2. Interface-Based Database Adapters\nMultiple backends are supported through adapter interfaces:\n- **Graph**: Ladybug (default), Neo4j, Neptune, Postgres (demo) via `GraphDBInterface`\n- **Vector**: LanceDB (default), ChromaDB, PGVector via `VectorDBInterface`\n- **Relational**: SQLite (default), PostgreSQL\n\nKey files:\n- `cognee/infrastructure/databases/graph/graph_db_interface.py`\n- `cognee/infrastructure/databases/vector/vector_db_interface.py`\n\n#### 3. Multi-Tenant Access Control\nUser → Dataset → Data hierarchy with permission-based filtering. Enable with `ENABLE_BACKEND_ACCESS_CONTROL=True`. Each user+dataset combination can have isolated graph/vector databases (when using supported backends: Ladybug, LanceDB, SQLite, Postgres).\n\n### Layer Structure\n\n```\nAPI Layer (cognee/api/v1/)\n    ↓\nMain Functions (add, cognify, search, memify)\n    ↓\nPipeline Orchestrator (cognee/modules/pipelines/)\n    ↓\nTask Execution Layer (cognee/tasks/)\n    ↓\nDomain Modules (graph, retrieval, ingestion, etc.)\n    ↓\nInfrastructure Adapters (LLM, databases)\n    ↓\nExternal Services (OpenAI, Ladybug, LanceDB, etc.)\n```\n\n### Critical Data Flow Paths\n\n#### ADD: Data Ingestion\n`add()` → `resolve_data_directories` → `ingest_data` → `save_data_item_to_storage` → Create Dataset + Data records in relational DB\n\nKey files: `cognee/api/v1/add/add.py`, `cognee/tasks/ingestion/ingest_data.py`\n\n#### COGNIFY: Knowledge Graph Construction\n`cognify()` → `classify_documents` → `extract_chunks_from_documents` → `extract_graph_from_data` (LLM extracts entities/relationships using Instructor) → `summarize_text` → `add_data_points` (store in graph + vector DBs)\n\nKey files:\n- `cognee/api/v1/cognify/cognify.py`\n- `cognee/tasks/graph/extract_graph_from_data.py`\n- `cognee/tasks/storage/add_data_points.py`\n\n#### SEARCH: Retrieval\n`search(query_text, query_type)` → route to retriever type → filter by permissions → return results\n\nAvailable search types (from `cognee/modules/search/types/SearchType.py`):\n- **GRAPH_COMPLETION** (default) - Graph traversal + LLM completion\n- **GRAPH_SUMMARY_COMPLETION** - Uses pre-computed summaries with graph context\n- **GRAPH_COMPLETION_COT** - Chain-of-thought reasoning over graph\n- **GRAPH_COMPLETION_CONTEXT_EXTENSION** - Extended context graph retrieval\n- **TRIPLET_COMPLETION** - Triplet-based (subject-predicate-object) search\n- **RAG_COMPLETION** - Traditional RAG with chunks\n- **CHUNKS** - Vector similarity search over chunks\n- **CHUNKS_LEXICAL** - Lexical (keyword) search over chunks\n- **SUMMARIES** - Search pre-computed document summaries\n- **CYPHER** - Direct Cypher query execution (requires `ALLOW_CYPHER_QUERY=True`)\n- **NATURAL_LANGUAGE** - Natural language to structured query\n- **TEMPORAL** - Time-aware graph search\n- **FEELING_LUCKY** - Automatic search type selection\n- **CODING_RULES** - Code-specific search rules\n\nKey files:\n- `cognee/api/v1/search/search.py`\n- `cognee/modules/retrieval/context_providers/TripletSearchContextProvider.py`\n- `cognee/modules/search/types/SearchType.py`\n\n### Core Data Models\n\n#### Engine Models (`cognee/infrastructure/engine/models/`)\n- **DataPoint** - Base class for all graph nodes (versioned, with metadata)\n- **Edge** - Graph relationships (source, target, relationship type)\n- **Triplet** - (Subject, Predicate, Object) representation\n\n#### Graph Models (`cognee/shared/data_models.py`)\n- **KnowledgeGraph** - Container for nodes and edges\n- **Node** - Entity (id, name, type, description)\n- **Edge** - Relationship (source_node_id, target_node_id, relationship_name)\n\n### Key Infrastructure Components\n\n#### LLM Gateway (`cognee/infrastructure/llm/LLMGateway.py`)\nUnified interface for multiple LLM providers: OpenAI, Anthropic, Gemini, Ollama, Mistral, Bedrock. Uses Instructor for structured output extraction.\n\n#### Embedding Engines\nFactory pattern for embeddings: `cognee/infrastructure/databases/vector/embeddings/get_embedding_engine.py`\n\n#### Document Loaders\nSupport for PDF, DOCX, CSV, images, audio, code files in `cognee/infrastructure/files/`\n\n## Important Configuration\n\n### Environment Setup\nCopy `.env.template` to `.env` and configure:\n\n```bash\n# Minimal setup (defaults to OpenAI + local file-based databases)\nLLM_API_KEY=\"your_openai_api_key\"\nLLM_MODEL=\"openai/gpt-5-mini\"  # Default model\n```\n\n**Important**: If you configure only LLM or only embeddings, the other defaults to OpenAI. Ensure you have a working OpenAI API key, or configure both to avoid unexpected defaults.\n\nDefault databases (no extra setup needed):\n- **Relational**: SQLite (metadata and state storage)\n- **Vector**: LanceDB (embeddings for semantic search)\n- **Graph**: Ladybug (knowledge graph and relationships)\n\nAll stored in `.venv` by default. Override with `DATA_ROOT_DIRECTORY` and `SYSTEM_ROOT_DIRECTORY`.\n\n### Switching Databases\n\n#### Relational Databases\n```bash\n# PostgreSQL (requires postgres extra: pip install cognee[postgres])\nDB_PROVIDER=postgres\nDB_HOST=localhost\nDB_PORT=5432\nDB_USERNAME=cognee\nDB_PASSWORD=cognee\nDB_NAME=cognee_db\n```\n\n#### Vector Databases\nSupported: lancedb (default), pgvector, chromadb, qdrant, weaviate, milvus\n```bash\n# ChromaDB (requires chromadb extra)\nVECTOR_DB_PROVIDER=chromadb\n\n# PGVector (requires postgres extra)\nVECTOR_DB_PROVIDER=pgvector\nVECTOR_DB_URL=postgresql://cognee:cognee@localhost:5432/cognee_db\n```\n\n#### Graph Databases\nSupported: ladybug (default), neo4j, neptune, ladybug-remote, postgres (demo)\n```bash\n# Neo4j (requires neo4j extra: pip install cognee[neo4j])\nGRAPH_DATABASE_PROVIDER=neo4j\nGRAPH_DATABASE_URL=bolt://localhost:7687\nGRAPH_DATABASE_NAME=neo4j\nGRAPH_DATABASE_USERNAME=neo4j\nGRAPH_DATABASE_PASSWORD=yourpassword\n\n# Remote Ladybug\nGRAPH_DATABASE_PROVIDER=ladybug-remote\nGRAPH_DATABASE_URL=http://localhost:8000\nGRAPH_DATABASE_USERNAME=your_username\nGRAPH_DATABASE_PASSWORD=your_password\n\n# Postgres (requires postgres extra: pip install cognee[postgres])\n# DEMO, not production-ready — see the warning below.\n# Does not support raw Cypher queries, natural language search, or Graphiti.\nGRAPH_DATABASE_PROVIDER=postgres\nGRAPH_DATABASE_URL=postgresql+asyncpg://cognee:cognee@localhost:5432/cognee_db\n```\n\n> **⚠️ Warning:** Using Postgres as a graph store is currently a demo feature and is not\n> production-ready. Use it to demo keeping relational metadata, PGVector, and graph\n> state in a single Postgres service, but rely on a graph-native backend such as Kuzu or Neo4j\n> for production workloads.\n>\n> Interested in further development or production use of Postgres as a graph database? Write to\n> us at social@cognee.ai to explore the options.\n\n#### Session Cache\n```bash\n# Session/conversation cache backend: sqlite (default), postgres, redis, fs, tapes\nCACHE_BACKEND=sqlite\n# Optional explicit SQLAlchemy URL for sqlite/postgres cache backends (overrides defaults)\nCACHE_DB_URL=postgresql+asyncpg://cognee:cognee@localhost:5432/cognee_db\n```\n\n### LLM Provider Configuration\n\nSupported providers: OpenAI (default), Azure OpenAI, Google Gemini, Anthropic, AWS Bedrock, Ollama, LM Studio, Custom (OpenAI-compatible APIs)\n\n#### OpenAI (Recommended - Minimal Setup)\n```bash\nLLM_API_KEY=\"your_openai_api_key\"\nLLM_MODEL=\"openai/gpt-5-mini\"  # default; or gpt-5, gpt-4o, gpt-4o-mini, etc.\nLLM_PROVIDER=\"openai\"\n```\n\n#### Azure OpenAI\n```bash\nLLM_PROVIDER=\"azure\"\nLLM_MODEL=\"azure/gpt-4o-mini\"\nLLM_ENDPOINT=\"https://YOUR-RESOURCE.openai.azure.com/openai/deployments/gpt-4o-mini\"\nLLM_API_KEY=\"your_azure_api_key\"\nLLM_API_VERSION=\"2024-12-01-preview\"\n```\n\n#### Google Gemini (requires gemini extra)\n```bash\nLLM_PROVIDER=\"gemini\"\nLLM_MODEL=\"gemini/gemini-2.0-flash-exp\"\nLLM_API_KEY=\"your_gemini_api_key\"\n```\n\n#### Anthropic Claude (requires anthropic extra)\n```bash\nLLM_PROVIDER=\"anthropic\"\nLLM_MODEL=\"claude-3-5-sonnet-20241022\"\nLLM_API_KEY=\"your_anthropic_api_key\"\n```\n\n#### Ollama (Local - requires ollama extra)\n```bash\nLLM_PROVIDER=\"ollama\"\nLLM_MODEL=\"llama3.1:8b\"\nLLM_ENDPOINT=\"http://localhost:11434/v1\"\nLLM_API_KEY=\"ollama\"\nEMBEDDING_PROVIDER=\"ollama\"\nEMBEDDING_MODEL=\"nomic-embed-text:latest\"\nEMBEDDING_ENDPOINT=\"http://localhost:11434/api/embed\"\nHUGGINGFACE_TOKENIZER=\"nomic-ai/nomic-embed-text-v1.5\"\n```\n\n#### Custom / OpenRouter / vLLM\n```bash\nLLM_PROVIDER=\"custom\"\nLLM_MODEL=\"openrouter/google/gemini-2.0-flash-lite-preview-02-05:free\"\nLLM_ENDPOINT=\"https://openrouter.ai/api/v1\"\nLLM_API_KEY=\"your_api_key\"\n```\n\n#### AWS Bedrock (requires aws extra)\n```bash\nLLM_PROVIDER=\"bedrock\"\nLLM_MODEL=\"anthropic.claude-3-sonnet-20240229-v1:0\"\nAWS_REGION=\"us-east-1\"\nAWS_ACCESS_KEY_ID=\"your_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_secret_key\"\n# Optional for temporary credentials:\n# AWS_SESSION_TOKEN=\"your_session_token\"\n```\n\n#### LLM Rate Limiting\n```bash\nLLM_RATE_LIMIT_ENABLED=true\nLLM_RATE_LIMIT_REQUESTS=60  # Requests per interval\nLLM_RATE_LIMIT_INTERVAL=60  # Interval in seconds\n```\n\n#### Instructor Mode (Structured Output)\n```bash\n# LLM_INSTRUCTOR_MODE controls how structured data is extracted\n# Each LLM has its own default (e.g., gpt-4o models use \"json_schema_mode\")\n# Override if needed:\nLLM_INSTRUCTOR_MODE=\"json_schema_mode\"  # or \"tool_call\", \"md_json\", etc.\n```\n\n### Structured Output Framework\n```bash\n# Use Instructor (default, via litellm)\nSTRUCTURED_OUTPUT_FRAMEWORK=\"instructor\"\n\n# Or use BAML (requires baml extra: pip install cognee[baml])\nSTRUCTURED_OUTPUT_FRAMEWORK=\"baml\"\nBAML_LLM_PROVIDER=openai\nBAML_LLM_MODEL=\"gpt-4o-mini\"\nBAML_LLM_API_KEY=\"your_api_key\"\n```\n\n### Storage Backend\n```bash\n# Local filesystem (default)\nSTORAGE_BACKEND=\"local\"\n\n# S3 (requires aws extra: pip install cognee[aws])\nSTORAGE_BACKEND=\"s3\"\nSTORAGE_BUCKET_NAME=\"your-bucket-name\"\nAWS_REGION=\"us-east-1\"\nAWS_ACCESS_KEY_ID=\"your_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_secret_key\"\nDATA_ROOT_DIRECTORY=\"s3://your-bucket/cognee/data\"\nSYSTEM_ROOT_DIRECTORY=\"s3://your-bucket/cognee/system\"\n```\n\n## Extension Points\n\n### Adding New Functionality\n\n1. **New Task Type**: Create task function in `cognee/tasks/`, return Task object, register in pipeline\n2. **New Database Backend**: Implement `GraphDBInterface` or `VectorDBInterface` in `cognee/infrastructure/databases/`\n3. **New LLM Provider**: Add configuration in LLM config (uses litellm)\n4. **New Document Processor**: Extend loaders in `cognee/modules/data/processing/`\n5. **New Search Type**: Add to `SearchType` enum and implement retriever in `cognee/modules/retrieval/`\n6. **Custom Graph Models**: Define Pydantic models extending `DataPoint` in your code\n\n### Working with Ontologies\nCognee supports ontology-based entity extraction to ground knowledge graphs in standardized semantic frameworks (e.g., OWL ontologies).\n\nConfiguration:\n```bash\nONTOLOGY_RESOLVER=rdflib  # Default: uses rdflib and OWL files\nMATCHING_STRATEGY=fuzzy   # Default: fuzzy matching with 80% similarity\nONTOLOGY_FILE_PATH=/path/to/your/ontology.owl  # Full path to ontology file\n```\n\nImplementation: `cognee/modules/ontology/`\n\n## Branching Strategy\n\n**IMPORTANT**: Always branch from `dev`, not `main`. The `dev` branch is the active development branch.\n\n```bash\ngit checkout dev\ngit pull origin dev\ngit checkout -b feature/your-feature-name\n```\n\n**Core-team PRs must reference a Linear issue.** Put the issue key (e.g. `COG-123`)\nin the PR title or the branch name so Linear links the PR to its ticket. This is\nenforced by the `Require Linear issue` workflow (`linear-issue-check`), a required\nstatus check. Fork / external-contributor PRs are exempt (the check skips them), so\nthis rule applies only to internal PRs.\n\n## Code Style\n\n- **Formatter**: Ruff (configured in `pyproject.toml`)\n- **Line length**: 100 characters\n- **String quotes**: Use double quotes `\"` not single quotes `'` (enforced by ruff-format)\n- **Pre-commit hooks**: Run ruff linting and formatting automatically\n- **Type hints**: Encouraged (ty checks enabled)\n- **Important**: Always run `pre-commit run --all-files` before committing to catch formatting issues\n\n## Testing Strategy\n\nTests are organized in `cognee/tests/`:\n- `unit/` - Unit tests for individual modules\n- `integration/` - Full pipeline integration tests\n- `cli_tests/` - CLI command tests\n- `tasks/` - Task-specific tests\n\nWhen adding features, add corresponding tests. Integration tests should cover the full add → cognify → search flow.\n\n## API Structure\n\nFastAPI application with versioned routes under `cognee/api/v1/`:\n- `/add` - Data ingestion\n- `/cognify` - Knowledge graph processing\n- `/search` - Query interface\n- `/memify` - Graph enrichment\n- `/datasets` - Dataset management\n- `/users` - Authentication (when `REQUIRE_AUTHENTICATION` is effectively true; see auth posture below)\n- `/visualize` - Graph visualization server\n\n## Python SDK Entry Points\n\nMain functions exported from `cognee/__init__.py`:\n- `add(data, dataset_name)` - Ingest data\n- `cognify(datasets)` - Build knowledge graph\n- `search(query_text, query_type)` - Query knowledge\n- `memify(extraction_tasks, enrichment_tasks)` - Enrich graph\n- `delete(data_id)` - Remove data\n- `config()` - Configuration management\n- `datasets()` - Dataset operations\n\nAll functions are async - use `await` or `asyncio.run()`.\n\n## Security Considerations\n\nSeveral security environment variables in `.env`:\n- `ACCEPT_LOCAL_FILE_PATH` - Allow local file paths (default: True)\n- `ALLOW_HTTP_REQUESTS` - Allow HTTP requests from Cognee (default: True)\n- `ALLOW_CYPHER_QUERY` - Allow raw Cypher queries (default: True)\n- `ENABLE_BACKEND_ACCESS_CONTROL` - Multi-tenant isolation (default: True). When `true`, API auth is required and per-user/dataset DB isolation is enabled. When `false`, single-user mode: shared DBs and auth off unless overridden.\n- `REQUIRE_AUTHENTICATION` - Explicit auth override. Unset (default): follows `ENABLE_BACKEND_ACCESS_CONTROL`. `false` is ignored when `ENABLE_BACKEND_ACCESS_CONTROL=true`. For a single-user deployment with auth off, set `ENABLE_BACKEND_ACCESS_CONTROL=false` (and optionally `REQUIRE_AUTHENTICATION=false`).\n\nFor production deployments, review and tighten these settings.\n\n## Common Patterns\n\n### Creating a Custom Pipeline Task\n```python\nfrom cognee.modules.pipelines.tasks.Task import Task\n\nasync def my_custom_task(data):\n    # Your logic here\n    processed_data = process(data)\n    return processed_data\n\n# Use in pipeline\ntask = Task(my_custom_task)\n```\n\n### Accessing Databases Directly\n```python\nfrom cognee.infrastructure.databases.graph import get_graph_engine\nfrom cognee.infrastructure.databases.vector import get_vector_engine_async\n\ngraph_engine = await get_graph_engine()\nvector_engine = await get_vector_engine_async()\n```\n\n### Using LLM Gateway\n```python\nfrom cognee.infrastructure.llm.get_llm_client import get_llm_client\n\nllm_client = get_llm_client()\nresponse = await llm_client.acreate_structured_output(\n    text_input=\"Your prompt\",\n    system_prompt=\"System instructions\",\n    response_model=YourPydanticModel\n)\n```\n\n## Key Concepts\n\n### Datasets\nDatasets are project-level containers that support organization, permissions, and isolated processing workflows. Each user can have multiple datasets with different access permissions.\n\n```python\n# Create/use a dataset\nawait cognee.add(data, dataset_name=\"my_project\")\nawait cognee.cognify(datasets=[\"my_project\"])\n```\n\n### DataPoints\nAtomic knowledge units that form the foundation of graph structures. All graph nodes extend the `DataPoint` base class with versioning and metadata support.\n\n### Permissions System\nMulti-tenant architecture with users, roles, and Access Control Lists (ACLs):\n- Read, write, delete, and share permissions per dataset\n- Enable with `ENABLE_BACKEND_ACCESS_CONTROL=True`\n- Supports isolated databases per user+dataset (Ladybug, LanceDB, SQLite, Postgres)\n\n### Graph Visualization\nLaunch visualization server:\n```bash\n# Via CLI\ncognee-cli -ui  # Launches full stack with UI at http://localhost:3000\n\n# Via Python\nfrom cognee.api.v1.visualize import start_visualization_server\nawait start_visualization_server(port=8080)\n```\n\n## Debugging & Troubleshooting\n\n### Debug Configuration\n- Set `LITELLM_LOG=\"DEBUG\"` for verbose LLM logs (default: \"ERROR\")\n- Enable debug mode: `ENV=\"development\"` or `ENV=\"debug\"`\n- Disable telemetry: `TELEMETRY_DISABLED=1`\n- Check logs in structured format (uses structlog)\n- Use `debugpy` optional dependency for debugging: `pip install cognee[debug]`\n\n### Common Issues\n\n**Ollama + OpenAI Embeddings NoDataError**\n- Issue: Mixing Ollama with OpenAI embeddings can cause errors\n- Solution: Configure both LLM and embeddings to use the same provider, or ensure `HUGGINGFACE_TOKENIZER` is set when using Ollama\n\n**LM Studio Structured Output**\n- Issue: LM Studio requires explicit instructor mode\n- Solution: Set `LLM_INSTRUCTOR_MODE=\"json_schema_mode\"` (or appropriate mode)\n\n**Default Provider Fallback**\n- Issue: Configuring only LLM or only embeddings defaults the other to OpenAI\n- Solution: Always configure both LLM and embedding providers, or ensure valid OpenAI API key\n\n**Permission Denied on Search**\n- Behavior: Returns empty list rather than error (prevents information leakage)\n- Solution: Check dataset permissions and user access rights\n\n**Database Connection Issues**\n- Check: Verify database URLs, credentials, and that services are running\n- Docker users: Use `DB_HOST=host.docker.internal` for local databases\n\n**Rate Limiting Errors**\n- Enable client-side rate limiting: `LLM_RATE_LIMIT_ENABLED=true`\n- Adjust limits: `LLM_RATE_LIMIT_REQUESTS` and `LLM_RATE_LIMIT_INTERVAL`\n\n## Resources\n\n- [Documentation](https://docs.cognee.ai/)\n- [Discord Community](https://discord.gg/NQPKmU5CCg)\n- [GitHub Issues](https://github.com/topoteretes/cognee/issues)\n- [Example Notebooks](examples/python/)\n- [Research Paper](https://arxiv.org/abs/2505.24478) - Optimizing knowledge graphs for LLM reasoning\n"},"files":{"AGENTS.md":"## Repository Guidelines\n\nThis document summarizes how to work with the cognee repository: how it’s organized, how to build, test, lint, and contribute. It mirrors our actual tooling and CI while providing quick commands for local development.\n\n## Project Structure & Module Organization\n\n- `cognee/`: Core Python library and API.\n  - `api/`: FastAPI application and versioned routers (add, cognify, memify, search, delete, users, datasets, responses, visualize, settings, sync, update, checks).\n  - `cli/`: CLI entry points and subcommands invoked via `cognee` / `cognee-cli`.\n  - `infrastructure/`: Databases, LLM providers, embeddings, loaders, and storage adapters.\n  - `modules/`: Domain logic (graph, retrieval, ontology, users, processing, observability, etc.).\n  - `tasks/`: Reusable tasks (e.g., code graph, web scraping, storage). Extend with new tasks here.\n  - `eval_framework/`: Evaluation utilities and adapters.\n  - `shared/`: Cross-cutting helpers (logging, settings, utils).\n  - `tests/`: Unit, integration, CLI, and end-to-end tests organized by feature.\n  - `__main__.py`: Entrypoint to route to CLI.\n- `cognee-mcp/`: Model Context Protocol server exposing cognee as MCP tools (SSE/HTTP/stdio). Contains its own README and Dockerfile.\n- `cognee-frontend/`: Next.js UI for local development and demos.\n- `distributed/`: Utilities for distributed execution (Modal, workers, queues).\n- `examples/`: Example scripts demonstrating the public APIs and features (graph, code graph, multimodal, permissions, etc.).\n- `notebooks/`: Jupyter notebooks for demos and tutorials.\n- `alembic/`: Database migrations for relational backends.\n\nNotes:\n- Co-locate feature-specific helpers under their respective package (`modules/`, `infrastructure/`, or `tasks/`).\n- Extend the system by adding new tasks, loaders, or retrievers rather than modifying core pipeline mechanisms.\n\n## Build, Test, and Development Commands\n\nPython (root) – requires Python >= 3.10 and < 3.14. We recommend `uv` for speed and reproducibility.\n\n- Create/refresh env and install dev deps:\n```bash\nuv sync --dev --all-extras --reinstall\n```\n\n- Run the CLI (examples):\n```bash\nuv run cognee-cli add \"Cognee turns documents into AI memory.\"\nuv run cognee-cli cognify\nuv run cognee-cli search \"What does cognee do?\"\nuv run cognee-cli -ui   # Launches UI, backend API, and MCP server together\n```\n\n- Start the FastAPI server directly:\n```bash\nuv run python -m cognee.api.client\n```\n\n- Run tests (CI mirrors these commands):\n```bash\nuv run pytest cognee/tests/unit/ -v\nuv run pytest cognee/tests/integration/ -v\n```\n\n- Lint and format (ruff):\n```bash\nuv run ruff check .\nuv run ruff format .\n```\n\n- Optional static type checks (ty):\n```bash\nuv run ty check .\n```\n\nMCP Server (`cognee-mcp/`):\n\n- Install and run locally:\n```bash\ncd cognee-mcp\nuv sync --dev --all-extras --reinstall\nuv run python src/server.py               # stdio (default)\nuv run python src/server.py --transport sse\nuv run python src/server.py --transport http --host 127.0.0.1 --port 8000 --path /mcp\n```\n\n- API Mode (connect to a running Cognee API):\n```bash\nuv run python src/server.py --transport sse --api-url http://localhost:8000 --api-token YOUR_TOKEN\n```\n\n- Docker quickstart (examples): see `cognee-mcp/README.md` for full details\n```bash\ndocker run -e TRANSPORT_MODE=http --env-file ./.env -p 8000:8000 --rm -it cognee/cognee-mcp:main\n```\n\nFrontend (`cognee-frontend/`):\n```bash\ncd cognee-frontend\nnpm install\nnpm run dev     # Next.js dev server\nnpm run lint    # ESLint\nnpm run build && npm start\n```\n\n## Coding Style & Naming Conventions\n\nPython:\n- 4-space indentation, modules and functions in `snake_case`, classes in `PascalCase`.\n- Public APIs should be type-annotated where practical. Make sure type defined in API signature will be properly displayed in Swagger UI docs. For example this definition: content_type: Optional[str] = Form(default=None) maps to \"string\" as the default in Swagger docs for content_type, but it should be None/null instead.\n- Use `ruff format` before committing; `ruff check` enforces import hygiene and style (line-length 100 configured in `pyproject.toml`).\n- Prefer explicit, structured error handling. Use shared logging utilities in `cognee.shared.logging_utils`.\n\nMCP server and Frontend:\n- Follow the local `README.md` and ESLint/TypeScript configuration in `cognee-frontend/`.\n\n## Testing Guidelines\n\n- Place Python tests under `cognee/tests/`.\n  - Unit tests: `cognee/tests/unit/`\n  - Integration tests: `cognee/tests/integration/`\n  - CLI tests: `cognee/tests/cli_tests/`\n- Name test files `test_*.py`. Use `pytest.mark.asyncio` for async tests.\n- Avoid external state; rely on test fixtures and the CI-provided env vars when LLM/embedding providers are required. See CI workflows under `.github/workflows/` for expected environment variables.\n- When adding public APIs, provide/update targeted examples under `examples/python/`.\n\n## Commit & Pull Request Guidelines\n\n- Use clear, imperative subjects (≤ 72 chars) and conventional commit styling in PR titles. Our CI validates semantic PR titles (see `.github/workflows/pr_lint`). Examples:\n  - `feat(graph): add temporal edge weighting`\n  - `fix(api): handle missing auth cookie`\n  - `docs: update installation instructions`\n- Reference related issues/discussions in the PR body and provide brief context.\n- PRs should describe scope, list local test commands run, and mention any impacts on MCP server or UI if applicable.\n- Sign commits and affirm the DCO (see `CONTRIBUTING.md`).\n\n## CI Mirrors Local Commands\n\nOur GitHub Actions run the same ruff checks and pytest suites shown above (`.github/workflows/basic_tests.yml` and related workflows). Use the commands in this document locally to minimize CI surprises.\n","CLAUDE.md":"# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Project Overview\n\nCognee is an open-source AI memory platform that transforms raw data into persistent knowledge graphs for AI agents. It replaces traditional RAG (Retrieval-Augmented Generation) with an ECL (Extract, Cognify, Load) pipeline combining vector search, graph databases, and LLM-powered entity extraction.\n\n**Requirements**: Python 3.10 - 3.14\n\n## Development Commands\n\n### Setup\n```bash\n# Create virtual environment (recommended: uv)\nuv venv && source .venv/bin/activate\n\n# Install with pip, poetry, or uv\nuv pip install -e .\n\n# Install with dev dependencies\nuv pip install -e \".[dev]\"\n\n# Install with specific extras\nuv pip install -e \".[postgres,neo4j,docs,chromadb]\"\n\n# Set up pre-commit hooks\npre-commit install\n```\n\n### Available Installation Extras\n- **postgres** / **postgres-binary** - PostgreSQL + PGVector support (also enables the Postgres session-cache backend, `CACHE_BACKEND=postgres`)\n- **neo4j** - Neo4j graph database support\n- **neptune** - AWS Neptune support\n- **chromadb** - ChromaDB vector database\n- **docs** - Document processing (unstructured library)\n- **scraping** - Web scraping (Tavily, BeautifulSoup, Playwright)\n- **langchain** - LangChain integration\n- **llama-index** - LlamaIndex integration\n- **anthropic** - Anthropic Claude models\n- **gemini** - Google Gemini models\n- **ollama** - Ollama local models\n- **mistral** - Mistral AI models\n- **groq** - Groq API support\n- **llama-cpp** - Llama.cpp local inference\n- **huggingface** - HuggingFace transformers\n- **aws** - S3 storage backend\n- **redis** - Redis caching\n- **graphiti** - Graphiti-core integration\n- **baml** - BAML structured output\n- **dlt** - Data load tool (dlt) integration\n- **docling** - Docling document processing\n- **codegraph** - Code graph extraction\n- **evals** - Evaluation tools\n- **deepeval** - DeepEval testing framework\n- **posthog** - PostHog analytics\n- **tracing** - OpenTelemetry tracing\n- **distributed** - Modal distributed execution\n- **dev** - All development tools (pytest, ty, ruff, etc.)\n- **debug** - Debugpy for debugging\n\n### Testing\n```bash\n# Run all tests\npytest\n\n# Run with coverage\npytest --cov=cognee --cov-report=html\n\n# Run specific test file\npytest cognee/tests/test_custom_model.py\n\n# Run specific test function\npytest cognee/tests/test_custom_model.py::test_function_name\n\n# Run async tests\npytest -v cognee/tests/integration/\n\n# Run unit tests only\npytest cognee/tests/unit/\n\n# Run integration tests only\npytest cognee/tests/integration/\n```\n\n### Code Quality\n```bash\n# Run ruff linter\nruff check .\n\n# Run ruff formatter\nruff format .\n\n# Run both linting and formatting (pre-commit)\npre-commit run --all-files\n\n# Type checking with ty\nty check .\n```\n\n### Running Cognee\n```bash\n# Using Python SDK\nuv run python examples/demos/simple_cognee_example.py\n\n# Using CLI\ncognee-cli add \"Your text here\"\ncognee-cli cognify\ncognee-cli search \"Your query\"\ncognee-cli delete --all\n\n# Launch full stack with UI\ncognee-cli -ui\n```\n\n## Architecture Overview\n\n### Core Workflow: add → cognify → search/memify\n\n1. **add()** - Ingest data (files, URLs, text) into datasets\n2. **cognify()** - Extract entities/relationships and build knowledge graph\n3. **search()** - Query knowledge using various retrieval strategies\n4. **memify()** - Enrich graph with additional context and rules\n\n### Key Architectural Patterns\n\n#### 1. Pipeline-Based Processing\nAll data flows through task-based pipelines (`cognee/modules/pipelines/`). Tasks are composable units that can run sequentially or in parallel. Example pipeline tasks: `classify_documents`, `extract_graph_from_data`, `add_data_points`.\n\n#### 2. Interface-Based Database Adapters\nMultiple backends are supported through adapter interfaces:\n- **Graph**: Ladybug (default), Neo4j, Neptune, Postgres (demo) via `GraphDBInterface`\n- **Vector**: LanceDB (default), ChromaDB, PGVector via `VectorDBInterface`\n- **Relational**: SQLite (default), PostgreSQL\n\nKey files:\n- `cognee/infrastructure/databases/graph/graph_db_interface.py`\n- `cognee/infrastructure/databases/vector/vector_db_interface.py`\n\n#### 3. Multi-Tenant Access Control\nUser → Dataset → Data hierarchy with permission-based filtering. Enable with `ENABLE_BACKEND_ACCESS_CONTROL=True`. Each user+dataset combination can have isolated graph/vector databases (when using supported backends: Ladybug, LanceDB, SQLite, Postgres).\n\n### Layer Structure\n\n```\nAPI Layer (cognee/api/v1/)\n    ↓\nMain Functions (add, cognify, search, memify)\n    ↓\nPipeline Orchestrator (cognee/modules/pipelines/)\n    ↓\nTask Execution Layer (cognee/tasks/)\n    ↓\nDomain Modules (graph, retrieval, ingestion, etc.)\n    ↓\nInfrastructure Adapters (LLM, databases)\n    ↓\nExternal Services (OpenAI, Ladybug, LanceDB, etc.)\n```\n\n### Critical Data Flow Paths\n\n#### ADD: Data Ingestion\n`add()` → `resolve_data_directories` → `ingest_data` → `save_data_item_to_storage` → Create Dataset + Data records in relational DB\n\nKey files: `cognee/api/v1/add/add.py`, `cognee/tasks/ingestion/ingest_data.py`\n\n#### COGNIFY: Knowledge Graph Construction\n`cognify()` → `classify_documents` → `extract_chunks_from_documents` → `extract_graph_from_data` (LLM extracts entities/relationships using Instructor) → `summarize_text` → `add_data_points` (store in graph + vector DBs)\n\nKey files:\n- `cognee/api/v1/cognify/cognify.py`\n- `cognee/tasks/graph/extract_graph_from_data.py`\n- `cognee/tasks/storage/add_data_points.py`\n\n#### SEARCH: Retrieval\n`search(query_text, query_type)` → route to retriever type → filter by permissions → return results\n\nAvailable search types (from `cognee/modules/search/types/SearchType.py`):\n- **GRAPH_COMPLETION** (default) - Graph traversal + LLM completion\n- **GRAPH_SUMMARY_COMPLETION** - Uses pre-computed summaries with graph context\n- **GRAPH_COMPLETION_COT** - Chain-of-thought reasoning over graph\n- **GRAPH_COMPLETION_CONTEXT_EXTENSION** - Extended context graph retrieval\n- **TRIPLET_COMPLETION** - Triplet-based (subject-predicate-object) search\n- **RAG_COMPLETION** - Traditional RAG with chunks\n- **CHUNKS** - Vector similarity search over chunks\n- **CHUNKS_LEXICAL** - Lexical (keyword) search over chunks\n- **SUMMARIES** - Search pre-computed document summaries\n- **CYPHER** - Direct Cypher query execution (requires `ALLOW_CYPHER_QUERY=True`)\n- **NATURAL_LANGUAGE** - Natural language to structured query\n- **TEMPORAL** - Time-aware graph search\n- **FEELING_LUCKY** - Automatic search type selection\n- **CODING_RULES** - Code-specific search rules\n\nKey files:\n- `cognee/api/v1/search/search.py`\n- `cognee/modules/retrieval/context_providers/TripletSearchContextProvider.py`\n- `cognee/modules/search/types/SearchType.py`\n\n### Core Data Models\n\n#### Engine Models (`cognee/infrastructure/engine/models/`)\n- **DataPoint** - Base class for all graph nodes (versioned, with metadata)\n- **Edge** - Graph relationships (source, target, relationship type)\n- **Triplet** - (Subject, Predicate, Object) representation\n\n#### Graph Models (`cognee/shared/data_models.py`)\n- **KnowledgeGraph** - Container for nodes and edges\n- **Node** - Entity (id, name, type, description)\n- **Edge** - Relationship (source_node_id, target_node_id, relationship_name)\n\n### Key Infrastructure Components\n\n#### LLM Gateway (`cognee/infrastructure/llm/LLMGateway.py`)\nUnified interface for multiple LLM providers: OpenAI, Anthropic, Gemini, Ollama, Mistral, Bedrock. Uses Instructor for structured output extraction.\n\n#### Embedding Engines\nFactory pattern for embeddings: `cognee/infrastructure/databases/vector/embeddings/get_embedding_engine.py`\n\n#### Document Loaders\nSupport for PDF, DOCX, CSV, images, audio, code files in `cognee/infrastructure/files/`\n\n## Important Configuration\n\n### Environment Setup\nCopy `.env.template` to `.env` and configure:\n\n```bash\n# Minimal setup (defaults to OpenAI + local file-based databases)\nLLM_API_KEY=\"your_openai_api_key\"\nLLM_MODEL=\"openai/gpt-5-mini\"  # Default model\n```\n\n**Important**: If you configure only LLM or only embeddings, the other defaults to OpenAI. Ensure you have a working OpenAI API key, or configure both to avoid unexpected defaults.\n\nDefault databases (no extra setup needed):\n- **Relational**: SQLite (metadata and state storage)\n- **Vector**: LanceDB (embeddings for semantic search)\n- **Graph**: Ladybug (knowledge graph and relationships)\n\nAll stored in `.venv` by default. Override with `DATA_ROOT_DIRECTORY` and `SYSTEM_ROOT_DIRECTORY`.\n\n### Switching Databases\n\n#### Relational Databases\n```bash\n# PostgreSQL (requires postgres extra: pip install cognee[postgres])\nDB_PROVIDER=postgres\nDB_HOST=localhost\nDB_PORT=5432\nDB_USERNAME=cognee\nDB_PASSWORD=cognee\nDB_NAME=cognee_db\n```\n\n#### Vector Databases\nSupported: lancedb (default), pgvector, chromadb, qdrant, weaviate, milvus\n```bash\n# ChromaDB (requires chromadb extra)\nVECTOR_DB_PROVIDER=chromadb\n\n# PGVector (requires postgres extra)\nVECTOR_DB_PROVIDER=pgvector\nVECTOR_DB_URL=postgresql://cognee:cognee@localhost:5432/cognee_db\n```\n\n#### Graph Databases\nSupported: ladybug (default), neo4j, neptune, ladybug-remote, postgres (demo)\n```bash\n# Neo4j (requires neo4j extra: pip install cognee[neo4j])\nGRAPH_DATABASE_PROVIDER=neo4j\nGRAPH_DATABASE_URL=bolt://localhost:7687\nGRAPH_DATABASE_NAME=neo4j\nGRAPH_DATABASE_USERNAME=neo4j\nGRAPH_DATABASE_PASSWORD=yourpassword\n\n# Remote Ladybug\nGRAPH_DATABASE_PROVIDER=ladybug-remote\nGRAPH_DATABASE_URL=http://localhost:8000\nGRAPH_DATABASE_USERNAME=your_username\nGRAPH_DATABASE_PASSWORD=your_password\n\n# Postgres (requires postgres extra: pip install cognee[postgres])\n# DEMO, not production-ready — see the warning below.\n# Does not support raw Cypher queries, natural language search, or Graphiti.\nGRAPH_DATABASE_PROVIDER=postgres\nGRAPH_DATABASE_URL=postgresql+asyncpg://cognee:cognee@localhost:5432/cognee_db\n```\n\n> **⚠️ Warning:** Using Postgres as a graph store is currently a demo feature and is not\n> production-ready. Use it to demo keeping relational metadata, PGVector, and graph\n> state in a single Postgres service, but rely on a graph-native backend such as Kuzu or Neo4j\n> for production workloads.\n>\n> Interested in further development or production use of Postgres as a graph database? Write to\n> us at social@cognee.ai to explore the options.\n\n#### Session Cache\n```bash\n# Session/conversation cache backend: sqlite (default), postgres, redis, fs, tapes\nCACHE_BACKEND=sqlite\n# Optional explicit SQLAlchemy URL for sqlite/postgres cache backends (overrides defaults)\nCACHE_DB_URL=postgresql+asyncpg://cognee:cognee@localhost:5432/cognee_db\n```\n\n### LLM Provider Configuration\n\nSupported providers: OpenAI (default), Azure OpenAI, Google Gemini, Anthropic, AWS Bedrock, Ollama, LM Studio, Custom (OpenAI-compatible APIs)\n\n#### OpenAI (Recommended - Minimal Setup)\n```bash\nLLM_API_KEY=\"your_openai_api_key\"\nLLM_MODEL=\"openai/gpt-5-mini\"  # default; or gpt-5, gpt-4o, gpt-4o-mini, etc.\nLLM_PROVIDER=\"openai\"\n```\n\n#### Azure OpenAI\n```bash\nLLM_PROVIDER=\"azure\"\nLLM_MODEL=\"azure/gpt-4o-mini\"\nLLM_ENDPOINT=\"https://YOUR-RESOURCE.openai.azure.com/openai/deployments/gpt-4o-mini\"\nLLM_API_KEY=\"your_azure_api_key\"\nLLM_API_VERSION=\"2024-12-01-preview\"\n```\n\n#### Google Gemini (requires gemini extra)\n```bash\nLLM_PROVIDER=\"gemini\"\nLLM_MODEL=\"gemini/gemini-2.0-flash-exp\"\nLLM_API_KEY=\"your_gemini_api_key\"\n```\n\n#### Anthropic Claude (requires anthropic extra)\n```bash\nLLM_PROVIDER=\"anthropic\"\nLLM_MODEL=\"claude-3-5-sonnet-20241022\"\nLLM_API_KEY=\"your_anthropic_api_key\"\n```\n\n#### Ollama (Local - requires ollama extra)\n```bash\nLLM_PROVIDER=\"ollama\"\nLLM_MODEL=\"llama3.1:8b\"\nLLM_ENDPOINT=\"http://localhost:11434/v1\"\nLLM_API_KEY=\"ollama\"\nEMBEDDING_PROVIDER=\"ollama\"\nEMBEDDING_MODEL=\"nomic-embed-text:latest\"\nEMBEDDING_ENDPOINT=\"http://localhost:11434/api/embed\"\nHUGGINGFACE_TOKENIZER=\"nomic-ai/nomic-embed-text-v1.5\"\n```\n\n#### Custom / OpenRouter / vLLM\n```bash\nLLM_PROVIDER=\"custom\"\nLLM_MODEL=\"openrouter/google/gemini-2.0-flash-lite-preview-02-05:free\"\nLLM_ENDPOINT=\"https://openrouter.ai/api/v1\"\nLLM_API_KEY=\"your_api_key\"\n```\n\n#### AWS Bedrock (requires aws extra)\n```bash\nLLM_PROVIDER=\"bedrock\"\nLLM_MODEL=\"anthropic.claude-3-sonnet-20240229-v1:0\"\nAWS_REGION=\"us-east-1\"\nAWS_ACCESS_KEY_ID=\"your_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_secret_key\"\n# Optional for temporary credentials:\n# AWS_SESSION_TOKEN=\"your_session_token\"\n```\n\n#### LLM Rate Limiting\n```bash\nLLM_RATE_LIMIT_ENABLED=true\nLLM_RATE_LIMIT_REQUESTS=60  # Requests per interval\nLLM_RATE_LIMIT_INTERVAL=60  # Interval in seconds\n```\n\n#### Instructor Mode (Structured Output)\n```bash\n# LLM_INSTRUCTOR_MODE controls how structured data is extracted\n# Each LLM has its own default (e.g., gpt-4o models use \"json_schema_mode\")\n# Override if needed:\nLLM_INSTRUCTOR_MODE=\"json_schema_mode\"  # or \"tool_call\", \"md_json\", etc.\n```\n\n### Structured Output Framework\n```bash\n# Use Instructor (default, via litellm)\nSTRUCTURED_OUTPUT_FRAMEWORK=\"instructor\"\n\n# Or use BAML (requires baml extra: pip install cognee[baml])\nSTRUCTURED_OUTPUT_FRAMEWORK=\"baml\"\nBAML_LLM_PROVIDER=openai\nBAML_LLM_MODEL=\"gpt-4o-mini\"\nBAML_LLM_API_KEY=\"your_api_key\"\n```\n\n### Storage Backend\n```bash\n# Local filesystem (default)\nSTORAGE_BACKEND=\"local\"\n\n# S3 (requires aws extra: pip install cognee[aws])\nSTORAGE_BACKEND=\"s3\"\nSTORAGE_BUCKET_NAME=\"your-bucket-name\"\nAWS_REGION=\"us-east-1\"\nAWS_ACCESS_KEY_ID=\"your_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_secret_key\"\nDATA_ROOT_DIRECTORY=\"s3://your-bucket/cognee/data\"\nSYSTEM_ROOT_DIRECTORY=\"s3://your-bucket/cognee/system\"\n```\n\n## Extension Points\n\n### Adding New Functionality\n\n1. **New Task Type**: Create task function in `cognee/tasks/`, return Task object, register in pipeline\n2. **New Database Backend**: Implement `GraphDBInterface` or `VectorDBInterface` in `cognee/infrastructure/databases/`\n3. **New LLM Provider**: Add configuration in LLM config (uses litellm)\n4. **New Document Processor**: Extend loaders in `cognee/modules/data/processing/`\n5. **New Search Type**: Add to `SearchType` enum and implement retriever in `cognee/modules/retrieval/`\n6. **Custom Graph Models**: Define Pydantic models extending `DataPoint` in your code\n\n### Working with Ontologies\nCognee supports ontology-based entity extraction to ground knowledge graphs in standardized semantic frameworks (e.g., OWL ontologies).\n\nConfiguration:\n```bash\nONTOLOGY_RESOLVER=rdflib  # Default: uses rdflib and OWL files\nMATCHING_STRATEGY=fuzzy   # Default: fuzzy matching with 80% similarity\nONTOLOGY_FILE_PATH=/path/to/your/ontology.owl  # Full path to ontology file\n```\n\nImplementation: `cognee/modules/ontology/`\n\n## Branching Strategy\n\n**IMPORTANT**: Always branch from `dev`, not `main`. The `dev` branch is the active development branch.\n\n```bash\ngit checkout dev\ngit pull origin dev\ngit checkout -b feature/your-feature-name\n```\n\n**Core-team PRs must reference a Linear issue.** Put the issue key (e.g. `COG-123`)\nin the PR title or the branch name so Linear links the PR to its ticket. This is\nenforced by the `Require Linear issue` workflow (`linear-issue-check`), a required\nstatus check. Fork / external-contributor PRs are exempt (the check skips them), so\nthis rule applies only to internal PRs.\n\n## Code Style\n\n- **Formatter**: Ruff (configured in `pyproject.toml`)\n- **Line length**: 100 characters\n- **String quotes**: Use double quotes `\"` not single quotes `'` (enforced by ruff-format)\n- **Pre-commit hooks**: Run ruff linting and formatting automatically\n- **Type hints**: Encouraged (ty checks enabled)\n- **Important**: Always run `pre-commit run --all-files` before committing to catch formatting issues\n\n## Testing Strategy\n\nTests are organized in `cognee/tests/`:\n- `unit/` - Unit tests for individual modules\n- `integration/` - Full pipeline integration tests\n- `cli_tests/` - CLI command tests\n- `tasks/` - Task-specific tests\n\nWhen adding features, add corresponding tests. Integration tests should cover the full add → cognify → search flow.\n\n## API Structure\n\nFastAPI application with versioned routes under `cognee/api/v1/`:\n- `/add` - Data ingestion\n- `/cognify` - Knowledge graph processing\n- `/search` - Query interface\n- `/memify` - Graph enrichment\n- `/datasets` - Dataset management\n- `/users` - Authentication (when `REQUIRE_AUTHENTICATION` is effectively true; see auth posture below)\n- `/visualize` - Graph visualization server\n\n## Python SDK Entry Points\n\nMain functions exported from `cognee/__init__.py`:\n- `add(data, dataset_name)` - Ingest data\n- `cognify(datasets)` - Build knowledge graph\n- `search(query_text, query_type)` - Query knowledge\n- `memify(extraction_tasks, enrichment_tasks)` - Enrich graph\n- `delete(data_id)` - Remove data\n- `config()` - Configuration management\n- `datasets()` - Dataset operations\n\nAll functions are async - use `await` or `asyncio.run()`.\n\n## Security Considerations\n\nSeveral security environment variables in `.env`:\n- `ACCEPT_LOCAL_FILE_PATH` - Allow local file paths (default: True)\n- `ALLOW_HTTP_REQUESTS` - Allow HTTP requests from Cognee (default: True)\n- `ALLOW_CYPHER_QUERY` - Allow raw Cypher queries (default: True)\n- `ENABLE_BACKEND_ACCESS_CONTROL` - Multi-tenant isolation (default: True). When `true`, API auth is required and per-user/dataset DB isolation is enabled. When `false`, single-user mode: shared DBs and auth off unless overridden.\n- `REQUIRE_AUTHENTICATION` - Explicit auth override. Unset (default): follows `ENABLE_BACKEND_ACCESS_CONTROL`. `false` is ignored when `ENABLE_BACKEND_ACCESS_CONTROL=true`. For a single-user deployment with auth off, set `ENABLE_BACKEND_ACCESS_CONTROL=false` (and optionally `REQUIRE_AUTHENTICATION=false`).\n\nFor production deployments, review and tighten these settings.\n\n## Common Patterns\n\n### Creating a Custom Pipeline Task\n```python\nfrom cognee.modules.pipelines.tasks.Task import Task\n\nasync def my_custom_task(data):\n    # Your logic here\n    processed_data = process(data)\n    return processed_data\n\n# Use in pipeline\ntask = Task(my_custom_task)\n```\n\n### Accessing Databases Directly\n```python\nfrom cognee.infrastructure.databases.graph import get_graph_engine\nfrom cognee.infrastructure.databases.vector import get_vector_engine_async\n\ngraph_engine = await get_graph_engine()\nvector_engine = await get_vector_engine_async()\n```\n\n### Using LLM Gateway\n```python\nfrom cognee.infrastructure.llm.get_llm_client import get_llm_client\n\nllm_client = get_llm_client()\nresponse = await llm_client.acreate_structured_output(\n    text_input=\"Your prompt\",\n    system_prompt=\"System instructions\",\n    response_model=YourPydanticModel\n)\n```\n\n## Key Concepts\n\n### Datasets\nDatasets are project-level containers that support organization, permissions, and isolated processing workflows. Each user can have multiple datasets with different access permissions.\n\n```python\n# Create/use a dataset\nawait cognee.add(data, dataset_name=\"my_project\")\nawait cognee.cognify(datasets=[\"my_project\"])\n```\n\n### DataPoints\nAtomic knowledge units that form the foundation of graph structures. All graph nodes extend the `DataPoint` base class with versioning and metadata support.\n\n### Permissions System\nMulti-tenant architecture with users, roles, and Access Control Lists (ACLs):\n- Read, write, delete, and share permissions per dataset\n- Enable with `ENABLE_BACKEND_ACCESS_CONTROL=True`\n- Supports isolated databases per user+dataset (Ladybug, LanceDB, SQLite, Postgres)\n\n### Graph Visualization\nLaunch visualization server:\n```bash\n# Via CLI\ncognee-cli -ui  # Launches full stack with UI at http://localhost:3000\n\n# Via Python\nfrom cognee.api.v1.visualize import start_visualization_server\nawait start_visualization_server(port=8080)\n```\n\n## Debugging & Troubleshooting\n\n### Debug Configuration\n- Set `LITELLM_LOG=\"DEBUG\"` for verbose LLM logs (default: \"ERROR\")\n- Enable debug mode: `ENV=\"development\"` or `ENV=\"debug\"`\n- Disable telemetry: `TELEMETRY_DISABLED=1`\n- Check logs in structured format (uses structlog)\n- Use `debugpy` optional dependency for debugging: `pip install cognee[debug]`\n\n### Common Issues\n\n**Ollama + OpenAI Embeddings NoDataError**\n- Issue: Mixing Ollama with OpenAI embeddings can cause errors\n- Solution: Configure both LLM and embeddings to use the same provider, or ensure `HUGGINGFACE_TOKENIZER` is set when using Ollama\n\n**LM Studio Structured Output**\n- Issue: LM Studio requires explicit instructor mode\n- Solution: Set `LLM_INSTRUCTOR_MODE=\"json_schema_mode\"` (or appropriate mode)\n\n**Default Provider Fallback**\n- Issue: Configuring only LLM or only embeddings defaults the other to OpenAI\n- Solution: Always configure both LLM and embedding providers, or ensure valid OpenAI API key\n\n**Permission Denied on Search**\n- Behavior: Returns empty list rather than error (prevents information leakage)\n- Solution: Check dataset permissions and user access rights\n\n**Database Connection Issues**\n- Check: Verify database URLs, credentials, and that services are running\n- Docker users: Use `DB_HOST=host.docker.internal` for local databases\n\n**Rate Limiting Errors**\n- Enable client-side rate limiting: `LLM_RATE_LIMIT_ENABLED=true`\n- Adjust limits: `LLM_RATE_LIMIT_REQUESTS` and `LLM_RATE_LIMIT_INTERVAL`\n\n## Resources\n\n- [Documentation](https://docs.cognee.ai/)\n- [Discord Community](https://discord.gg/NQPKmU5CCg)\n- [GitHub Issues](https://github.com/topoteretes/cognee/issues)\n- [Example Notebooks](examples/python/)\n- [Research Paper](https://arxiv.org/abs/2505.24478) - Optimizing knowledge graphs for LLM reasoning\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"## Repository Guidelines\n\nThis document summarizes how to work with the cognee repository: how it’s organized, how to build, test, lint, and contribute. It mirrors our actual tooling and CI while providing quick commands for local development.\n\n## Project Structure & Module Organization\n\n- `cognee/`: Core Python library and API.\n  - `api/`: FastAPI application and versioned routers (add, cognify, memify, search, delete, users, datasets, responses, visualize, settings, sync, update, checks).\n  - `cli/`: CLI entry points and subcommands invoked via `cognee` / `cognee-cli`.\n  - `infrastructure/`: Databases, LLM providers, embeddings, loaders, and storage adapters.\n  - `modules/`: Domain logic (graph, retrieval, ontology, users, processing, observability, etc.).\n  - `tasks/`: Reusable tasks (e.g., code graph, web scraping, storage). Extend with new tasks here.\n  - `eval_framework/`: Evaluation utilities and adapters.\n  - `shared/`: Cross-cutting helpers (logging, settings, utils).\n  - `tests/`: Unit, integration, CLI, and end-to-end tests organized by feature.\n  - `__main__.py`: Entrypoint to route to CLI.\n- `cognee-mcp/`: Model Context Protocol server exposing cognee as MCP tools (SSE/HTTP/stdio). Contains its own README and Dockerfile.\n- `cognee-frontend/`: Next.js UI for local development and demos.\n- `distributed/`: Utilities for distributed execution (Modal, workers, queues).\n- `examples/`: Example scripts demonstrating the public APIs and features (graph, code graph, multimodal, permissions, etc.).\n- `notebooks/`: Jupyter notebooks for demos and tutorials.\n- `alembic/`: Database migrations for relational backends.\n\nNotes:\n- Co-locate feature-specific helpers under their respective package (`modules/`, `infrastructure/`, or `tasks/`).\n- Extend the system by adding new tasks, loaders, or retrievers rather than modifying core pipeline mechanisms.\n\n## Build, Test, and Development Commands\n\nPython (root) – requires Python >= 3.10 and < 3.14. We recommend `uv` for speed and reproducibility.\n\n- Create/refresh env and install dev deps:\n```bash\nuv sync --dev --all-extras --reinstall\n```\n\n- Run the CLI (examples):\n```bash\nuv run cognee-cli add \"Cognee turns documents into AI memory.\"\nuv run cognee-cli cognify\nuv run cognee-cli search \"What does cognee do?\"\nuv run cognee-cli -ui   # Launches UI, backend API, and MCP server together\n```\n\n- Start the FastAPI server directly:\n```bash\nuv run python -m cognee.api.client\n```\n\n- Run tests (CI mirrors these commands):\n```bash\nuv run pytest cognee/tests/unit/ -v\nuv run pytest cognee/tests/integration/ -v\n```\n\n- Lint and format (ruff):\n```bash\nuv run ruff check .\nuv run ruff format .\n```\n\n- Optional static type checks (ty):\n```bash\nuv run ty check .\n```\n\nMCP Server (`cognee-mcp/`):\n\n- Install and run locally:\n```bash\ncd cognee-mcp\nuv sync --dev --all-extras --reinstall\nuv run python src/server.py               # stdio (default)\nuv run python src/server.py --transport sse\nuv run python src/server.py --transport http --host 127.0.0.1 --port 8000 --path /mcp\n```\n\n- API Mode (connect to a running Cognee API):\n```bash\nuv run python src/server.py --transport sse --api-url http://localhost:8000 --api-token YOUR_TOKEN\n```\n\n- Docker quickstart (examples): see `cognee-mcp/README.md` for full details\n```bash\ndocker run -e TRANSPORT_MODE=http --env-file ./.env -p 8000:8000 --rm -it cognee/cognee-mcp:main\n```\n\nFrontend (`cognee-frontend/`):\n```bash\ncd cognee-frontend\nnpm install\nnpm run dev     # Next.js dev server\nnpm run lint    # ESLint\nnpm run build && npm start\n```\n\n## Coding Style & Naming Conventions\n\nPython:\n- 4-space indentation, modules and functions in `snake_case`, classes in `PascalCase`.\n- Public APIs should be type-annotated where practical. Make sure type defined in API signature will be properly displayed in Swagger UI docs. For example this definition: content_type: Optional[str] = Form(default=None) maps to \"string\" as the default in Swagger docs for content_type, but it should be None/null instead.\n- Use `ruff format` before committing; `ruff check` enforces import hygiene and style (line-length 100 configured in `pyproject.toml`).\n- Prefer explicit, structured error handling. Use shared logging utilities in `cognee.shared.logging_utils`.\n\nMCP server and Frontend:\n- Follow the local `README.md` and ESLint/TypeScript configuration in `cognee-frontend/`.\n\n## Testing Guidelines\n\n- Place Python tests under `cognee/tests/`.\n  - Unit tests: `cognee/tests/unit/`\n  - Integration tests: `cognee/tests/integration/`\n  - CLI tests: `cognee/tests/cli_tests/`\n- Name test files `test_*.py`. Use `pytest.mark.asyncio` for async tests.\n- Avoid external state; rely on test fixtures and the CI-provided env vars when LLM/embedding providers are required. See CI workflows under `.github/workflows/` for expected environment variables.\n- When adding public APIs, provide/update targeted examples under `examples/python/`.\n\n## Commit & Pull Request Guidelines\n\n- Use clear, imperative subjects (≤ 72 chars) and conventional commit styling in PR titles. Our CI validates semantic PR titles (see `.github/workflows/pr_lint`). Examples:\n  - `feat(graph): add temporal edge weighting`\n  - `fix(api): handle missing auth cookie`\n  - `docs: update installation instructions`\n- Reference related issues/discussions in the PR body and provide brief context.\n- PRs should describe scope, list local test commands run, and mention any impacts on MCP server or UI if applicable.\n- Sign commits and affirm the DCO (see `CONTRIBUTING.md`).\n\n## CI Mirrors Local Commands\n\nOur GitHub Actions run the same ruff checks and pytest suites shown above (`.github/workflows/basic_tests.yml` and related workflows). Use the commands in this document locally to minimize CI surprises.\n","category":"root","tokens":1442},{"name":"CLAUDE.md","path":"CLAUDE.md","title":"CLAUDE.md","content":"# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Project Overview\n\nCognee is an open-source AI memory platform that transforms raw data into persistent knowledge graphs for AI agents. It replaces traditional RAG (Retrieval-Augmented Generation) with an ECL (Extract, Cognify, Load) pipeline combining vector search, graph databases, and LLM-powered entity extraction.\n\n**Requirements**: Python 3.10 - 3.14\n\n## Development Commands\n\n### Setup\n```bash\n# Create virtual environment (recommended: uv)\nuv venv && source .venv/bin/activate\n\n# Install with pip, poetry, or uv\nuv pip install -e .\n\n# Install with dev dependencies\nuv pip install -e \".[dev]\"\n\n# Install with specific extras\nuv pip install -e \".[postgres,neo4j,docs,chromadb]\"\n\n# Set up pre-commit hooks\npre-commit install\n```\n\n### Available Installation Extras\n- **postgres** / **postgres-binary** - PostgreSQL + PGVector support (also enables the Postgres session-cache backend, `CACHE_BACKEND=postgres`)\n- **neo4j** - Neo4j graph database support\n- **neptune** - AWS Neptune support\n- **chromadb** - ChromaDB vector database\n- **docs** - Document processing (unstructured library)\n- **scraping** - Web scraping (Tavily, BeautifulSoup, Playwright)\n- **langchain** - LangChain integration\n- **llama-index** - LlamaIndex integration\n- **anthropic** - Anthropic Claude models\n- **gemini** - Google Gemini models\n- **ollama** - Ollama local models\n- **mistral** - Mistral AI models\n- **groq** - Groq API support\n- **llama-cpp** - Llama.cpp local inference\n- **huggingface** - HuggingFace transformers\n- **aws** - S3 storage backend\n- **redis** - Redis caching\n- **graphiti** - Graphiti-core integration\n- **baml** - BAML structured output\n- **dlt** - Data load tool (dlt) integration\n- **docling** - Docling document processing\n- **codegraph** - Code graph extraction\n- **evals** - Evaluation tools\n- **deepeval** - DeepEval testing framework\n- **posthog** - PostHog analytics\n- **tracing** - OpenTelemetry tracing\n- **distributed** - Modal distributed execution\n- **dev** - All development tools (pytest, ty, ruff, etc.)\n- **debug** - Debugpy for debugging\n\n### Testing\n```bash\n# Run all tests\npytest\n\n# Run with coverage\npytest --cov=cognee --cov-report=html\n\n# Run specific test file\npytest cognee/tests/test_custom_model.py\n\n# Run specific test function\npytest cognee/tests/test_custom_model.py::test_function_name\n\n# Run async tests\npytest -v cognee/tests/integration/\n\n# Run unit tests only\npytest cognee/tests/unit/\n\n# Run integration tests only\npytest cognee/tests/integration/\n```\n\n### Code Quality\n```bash\n# Run ruff linter\nruff check .\n\n# Run ruff formatter\nruff format .\n\n# Run both linting and formatting (pre-commit)\npre-commit run --all-files\n\n# Type checking with ty\nty check .\n```\n\n### Running Cognee\n```bash\n# Using Python SDK\nuv run python examples/demos/simple_cognee_example.py\n\n# Using CLI\ncognee-cli add \"Your text here\"\ncognee-cli cognify\ncognee-cli search \"Your query\"\ncognee-cli delete --all\n\n# Launch full stack with UI\ncognee-cli -ui\n```\n\n## Architecture Overview\n\n### Core Workflow: add → cognify → search/memify\n\n1. **add()** - Ingest data (files, URLs, text) into datasets\n2. **cognify()** - Extract entities/relationships and build knowledge graph\n3. **search()** - Query knowledge using various retrieval strategies\n4. **memify()** - Enrich graph with additional context and rules\n\n### Key Architectural Patterns\n\n#### 1. Pipeline-Based Processing\nAll data flows through task-based pipelines (`cognee/modules/pipelines/`). Tasks are composable units that can run sequentially or in parallel. Example pipeline tasks: `classify_documents`, `extract_graph_from_data`, `add_data_points`.\n\n#### 2. Interface-Based Database Adapters\nMultiple backends are supported through adapter interfaces:\n- **Graph**: Ladybug (default), Neo4j, Neptune, Postgres (demo) via `GraphDBInterface`\n- **Vector**: LanceDB (default), ChromaDB, PGVector via `VectorDBInterface`\n- **Relational**: SQLite (default), PostgreSQL\n\nKey files:\n- `cognee/infrastructure/databases/graph/graph_db_interface.py`\n- `cognee/infrastructure/databases/vector/vector_db_interface.py`\n\n#### 3. Multi-Tenant Access Control\nUser → Dataset → Data hierarchy with permission-based filtering. Enable with `ENABLE_BACKEND_ACCESS_CONTROL=True`. Each user+dataset combination can have isolated graph/vector databases (when using supported backends: Ladybug, LanceDB, SQLite, Postgres).\n\n### Layer Structure\n\n```\nAPI Layer (cognee/api/v1/)\n    ↓\nMain Functions (add, cognify, search, memify)\n    ↓\nPipeline Orchestrator (cognee/modules/pipelines/)\n    ↓\nTask Execution Layer (cognee/tasks/)\n    ↓\nDomain Modules (graph, retrieval, ingestion, etc.)\n    ↓\nInfrastructure Adapters (LLM, databases)\n    ↓\nExternal Services (OpenAI, Ladybug, LanceDB, etc.)\n```\n\n### Critical Data Flow Paths\n\n#### ADD: Data Ingestion\n`add()` → `resolve_data_directories` → `ingest_data` → `save_data_item_to_storage` → Create Dataset + Data records in relational DB\n\nKey files: `cognee/api/v1/add/add.py`, `cognee/tasks/ingestion/ingest_data.py`\n\n#### COGNIFY: Knowledge Graph Construction\n`cognify()` → `classify_documents` → `extract_chunks_from_documents` → `extract_graph_from_data` (LLM extracts entities/relationships using Instructor) → `summarize_text` → `add_data_points` (store in graph + vector DBs)\n\nKey files:\n- `cognee/api/v1/cognify/cognify.py`\n- `cognee/tasks/graph/extract_graph_from_data.py`\n- `cognee/tasks/storage/add_data_points.py`\n\n#### SEARCH: Retrieval\n`search(query_text, query_type)` → route to retriever type → filter by permissions → return results\n\nAvailable search types (from `cognee/modules/search/types/SearchType.py`):\n- **GRAPH_COMPLETION** (default) - Graph traversal + LLM completion\n- **GRAPH_SUMMARY_COMPLETION** - Uses pre-computed summaries with graph context\n- **GRAPH_COMPLETION_COT** - Chain-of-thought reasoning over graph\n- **GRAPH_COMPLETION_CONTEXT_EXTENSION** - Extended context graph retrieval\n- **TRIPLET_COMPLETION** - Triplet-based (subject-predicate-object) search\n- **RAG_COMPLETION** - Traditional RAG with chunks\n- **CHUNKS** - Vector similarity search over chunks\n- **CHUNKS_LEXICAL** - Lexical (keyword) search over chunks\n- **SUMMARIES** - Search pre-computed document summaries\n- **CYPHER** - Direct Cypher query execution (requires `ALLOW_CYPHER_QUERY=True`)\n- **NATURAL_LANGUAGE** - Natural language to structured query\n- **TEMPORAL** - Time-aware graph search\n- **FEELING_LUCKY** - Automatic search type selection\n- **CODING_RULES** - Code-specific search rules\n\nKey files:\n- `cognee/api/v1/search/search.py`\n- `cognee/modules/retrieval/context_providers/TripletSearchContextProvider.py`\n- `cognee/modules/search/types/SearchType.py`\n\n### Core Data Models\n\n#### Engine Models (`cognee/infrastructure/engine/models/`)\n- **DataPoint** - Base class for all graph nodes (versioned, with metadata)\n- **Edge** - Graph relationships (source, target, relationship type)\n- **Triplet** - (Subject, Predicate, Object) representation\n\n#### Graph Models (`cognee/shared/data_models.py`)\n- **KnowledgeGraph** - Container for nodes and edges\n- **Node** - Entity (id, name, type, description)\n- **Edge** - Relationship (source_node_id, target_node_id, relationship_name)\n\n### Key Infrastructure Components\n\n#### LLM Gateway (`cognee/infrastructure/llm/LLMGateway.py`)\nUnified interface for multiple LLM providers: OpenAI, Anthropic, Gemini, Ollama, Mistral, Bedrock. Uses Instructor for structured output extraction.\n\n#### Embedding Engines\nFactory pattern for embeddings: `cognee/infrastructure/databases/vector/embeddings/get_embedding_engine.py`\n\n#### Document Loaders\nSupport for PDF, DOCX, CSV, images, audio, code files in `cognee/infrastructure/files/`\n\n## Important Configuration\n\n### Environment Setup\nCopy `.env.template` to `.env` and configure:\n\n```bash\n# Minimal setup (defaults to OpenAI + local file-based databases)\nLLM_API_KEY=\"your_openai_api_key\"\nLLM_MODEL=\"openai/gpt-5-mini\"  # Default model\n```\n\n**Important**: If you configure only LLM or only embeddings, the other defaults to OpenAI. Ensure you have a working OpenAI API key, or configure both to avoid unexpected defaults.\n\nDefault databases (no extra setup needed):\n- **Relational**: SQLite (metadata and state storage)\n- **Vector**: LanceDB (embeddings for semantic search)\n- **Graph**: Ladybug (knowledge graph and relationships)\n\nAll stored in `.venv` by default. Override with `DATA_ROOT_DIRECTORY` and `SYSTEM_ROOT_DIRECTORY`.\n\n### Switching Databases\n\n#### Relational Databases\n```bash\n# PostgreSQL (requires postgres extra: pip install cognee[postgres])\nDB_PROVIDER=postgres\nDB_HOST=localhost\nDB_PORT=5432\nDB_USERNAME=cognee\nDB_PASSWORD=cognee\nDB_NAME=cognee_db\n```\n\n#### Vector Databases\nSupported: lancedb (default), pgvector, chromadb, qdrant, weaviate, milvus\n```bash\n# ChromaDB (requires chromadb extra)\nVECTOR_DB_PROVIDER=chromadb\n\n# PGVector (requires postgres extra)\nVECTOR_DB_PROVIDER=pgvector\nVECTOR_DB_URL=postgresql://cognee:cognee@localhost:5432/cognee_db\n```\n\n#### Graph Databases\nSupported: ladybug (default), neo4j, neptune, ladybug-remote, postgres (demo)\n```bash\n# Neo4j (requires neo4j extra: pip install cognee[neo4j])\nGRAPH_DATABASE_PROVIDER=neo4j\nGRAPH_DATABASE_URL=bolt://localhost:7687\nGRAPH_DATABASE_NAME=neo4j\nGRAPH_DATABASE_USERNAME=neo4j\nGRAPH_DATABASE_PASSWORD=yourpassword\n\n# Remote Ladybug\nGRAPH_DATABASE_PROVIDER=ladybug-remote\nGRAPH_DATABASE_URL=http://localhost:8000\nGRAPH_DATABASE_USERNAME=your_username\nGRAPH_DATABASE_PASSWORD=your_password\n\n# Postgres (requires postgres extra: pip install cognee[postgres])\n# DEMO, not production-ready — see the warning below.\n# Does not support raw Cypher queries, natural language search, or Graphiti.\nGRAPH_DATABASE_PROVIDER=postgres\nGRAPH_DATABASE_URL=postgresql+asyncpg://cognee:cognee@localhost:5432/cognee_db\n```\n\n> **⚠️ Warning:** Using Postgres as a graph store is currently a demo feature and is not\n> production-ready. Use it to demo keeping relational metadata, PGVector, and graph\n> state in a single Postgres service, but rely on a graph-native backend such as Kuzu or Neo4j\n> for production workloads.\n>\n> Interested in further development or production use of Postgres as a graph database? Write to\n> us at social@cognee.ai to explore the options.\n\n#### Session Cache\n```bash\n# Session/conversation cache backend: sqlite (default), postgres, redis, fs, tapes\nCACHE_BACKEND=sqlite\n# Optional explicit SQLAlchemy URL for sqlite/postgres cache backends (overrides defaults)\nCACHE_DB_URL=postgresql+asyncpg://cognee:cognee@localhost:5432/cognee_db\n```\n\n### LLM Provider Configuration\n\nSupported providers: OpenAI (default), Azure OpenAI, Google Gemini, Anthropic, AWS Bedrock, Ollama, LM Studio, Custom (OpenAI-compatible APIs)\n\n#### OpenAI (Recommended - Minimal Setup)\n```bash\nLLM_API_KEY=\"your_openai_api_key\"\nLLM_MODEL=\"openai/gpt-5-mini\"  # default; or gpt-5, gpt-4o, gpt-4o-mini, etc.\nLLM_PROVIDER=\"openai\"\n```\n\n#### Azure OpenAI\n```bash\nLLM_PROVIDER=\"azure\"\nLLM_MODEL=\"azure/gpt-4o-mini\"\nLLM_ENDPOINT=\"https://YOUR-RESOURCE.openai.azure.com/openai/deployments/gpt-4o-mini\"\nLLM_API_KEY=\"your_azure_api_key\"\nLLM_API_VERSION=\"2024-12-01-preview\"\n```\n\n#### Google Gemini (requires gemini extra)\n```bash\nLLM_PROVIDER=\"gemini\"\nLLM_MODEL=\"gemini/gemini-2.0-flash-exp\"\nLLM_API_KEY=\"your_gemini_api_key\"\n```\n\n#### Anthropic Claude (requires anthropic extra)\n```bash\nLLM_PROVIDER=\"anthropic\"\nLLM_MODEL=\"claude-3-5-sonnet-20241022\"\nLLM_API_KEY=\"your_anthropic_api_key\"\n```\n\n#### Ollama (Local - requires ollama extra)\n```bash\nLLM_PROVIDER=\"ollama\"\nLLM_MODEL=\"llama3.1:8b\"\nLLM_ENDPOINT=\"http://localhost:11434/v1\"\nLLM_API_KEY=\"ollama\"\nEMBEDDING_PROVIDER=\"ollama\"\nEMBEDDING_MODEL=\"nomic-embed-text:latest\"\nEMBEDDING_ENDPOINT=\"http://localhost:11434/api/embed\"\nHUGGINGFACE_TOKENIZER=\"nomic-ai/nomic-embed-text-v1.5\"\n```\n\n#### Custom / OpenRouter / vLLM\n```bash\nLLM_PROVIDER=\"custom\"\nLLM_MODEL=\"openrouter/google/gemini-2.0-flash-lite-preview-02-05:free\"\nLLM_ENDPOINT=\"https://openrouter.ai/api/v1\"\nLLM_API_KEY=\"your_api_key\"\n```\n\n#### AWS Bedrock (requires aws extra)\n```bash\nLLM_PROVIDER=\"bedrock\"\nLLM_MODEL=\"anthropic.claude-3-sonnet-20240229-v1:0\"\nAWS_REGION=\"us-east-1\"\nAWS_ACCESS_KEY_ID=\"your_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_secret_key\"\n# Optional for temporary credentials:\n# AWS_SESSION_TOKEN=\"your_session_token\"\n```\n\n#### LLM Rate Limiting\n```bash\nLLM_RATE_LIMIT_ENABLED=true\nLLM_RATE_LIMIT_REQUESTS=60  # Requests per interval\nLLM_RATE_LIMIT_INTERVAL=60  # Interval in seconds\n```\n\n#### Instructor Mode (Structured Output)\n```bash\n# LLM_INSTRUCTOR_MODE controls how structured data is extracted\n# Each LLM has its own default (e.g., gpt-4o models use \"json_schema_mode\")\n# Override if needed:\nLLM_INSTRUCTOR_MODE=\"json_schema_mode\"  # or \"tool_call\", \"md_json\", etc.\n```\n\n### Structured Output Framework\n```bash\n# Use Instructor (default, via litellm)\nSTRUCTURED_OUTPUT_FRAMEWORK=\"instructor\"\n\n# Or use BAML (requires baml extra: pip install cognee[baml])\nSTRUCTURED_OUTPUT_FRAMEWORK=\"baml\"\nBAML_LLM_PROVIDER=openai\nBAML_LLM_MODEL=\"gpt-4o-mini\"\nBAML_LLM_API_KEY=\"your_api_key\"\n```\n\n### Storage Backend\n```bash\n# Local filesystem (default)\nSTORAGE_BACKEND=\"local\"\n\n# S3 (requires aws extra: pip install cognee[aws])\nSTORAGE_BACKEND=\"s3\"\nSTORAGE_BUCKET_NAME=\"your-bucket-name\"\nAWS_REGION=\"us-east-1\"\nAWS_ACCESS_KEY_ID=\"your_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_secret_key\"\nDATA_ROOT_DIRECTORY=\"s3://your-bucket/cognee/data\"\nSYSTEM_ROOT_DIRECTORY=\"s3://your-bucket/cognee/system\"\n```\n\n## Extension Points\n\n### Adding New Functionality\n\n1. **New Task Type**: Create task function in `cognee/tasks/`, return Task object, register in pipeline\n2. **New Database Backend**: Implement `GraphDBInterface` or `VectorDBInterface` in `cognee/infrastructure/databases/`\n3. **New LLM Provider**: Add configuration in LLM config (uses litellm)\n4. **New Document Processor**: Extend loaders in `cognee/modules/data/processing/`\n5. **New Search Type**: Add to `SearchType` enum and implement retriever in `cognee/modules/retrieval/`\n6. **Custom Graph Models**: Define Pydantic models extending `DataPoint` in your code\n\n### Working with Ontologies\nCognee supports ontology-based entity extraction to ground knowledge graphs in standardized semantic frameworks (e.g., OWL ontologies).\n\nConfiguration:\n```bash\nONTOLOGY_RESOLVER=rdflib  # Default: uses rdflib and OWL files\nMATCHING_STRATEGY=fuzzy   # Default: fuzzy matching with 80% similarity\nONTOLOGY_FILE_PATH=/path/to/your/ontology.owl  # Full path to ontology file\n```\n\nImplementation: `cognee/modules/ontology/`\n\n## Branching Strategy\n\n**IMPORTANT**: Always branch from `dev`, not `main`. The `dev` branch is the active development branch.\n\n```bash\ngit checkout dev\ngit pull origin dev\ngit checkout -b feature/your-feature-name\n```\n\n**Core-team PRs must reference a Linear issue.** Put the issue key (e.g. `COG-123`)\nin the PR title or the branch name so Linear links the PR to its ticket. This is\nenforced by the `Require Linear issue` workflow (`linear-issue-check`), a required\nstatus check. Fork / external-contributor PRs are exempt (the check skips them), so\nthis rule applies only to internal PRs.\n\n## Code Style\n\n- **Formatter**: Ruff (configured in `pyproject.toml`)\n- **Line length**: 100 characters\n- **String quotes**: Use double quotes `\"` not single quotes `'` (enforced by ruff-format)\n- **Pre-commit hooks**: Run ruff linting and formatting automatically\n- **Type hints**: Encouraged (ty checks enabled)\n- **Important**: Always run `pre-commit run --all-files` before committing to catch formatting issues\n\n## Testing Strategy\n\nTests are organized in `cognee/tests/`:\n- `unit/` - Unit tests for individual modules\n- `integration/` - Full pipeline integration tests\n- `cli_tests/` - CLI command tests\n- `tasks/` - Task-specific tests\n\nWhen adding features, add corresponding tests. Integration tests should cover the full add → cognify → search flow.\n\n## API Structure\n\nFastAPI application with versioned routes under `cognee/api/v1/`:\n- `/add` - Data ingestion\n- `/cognify` - Knowledge graph processing\n- `/search` - Query interface\n- `/memify` - Graph enrichment\n- `/datasets` - Dataset management\n- `/users` - Authentication (when `REQUIRE_AUTHENTICATION` is effectively true; see auth posture below)\n- `/visualize` - Graph visualization server\n\n## Python SDK Entry Points\n\nMain functions exported from `cognee/__init__.py`:\n- `add(data, dataset_name)` - Ingest data\n- `cognify(datasets)` - Build knowledge graph\n- `search(query_text, query_type)` - Query knowledge\n- `memify(extraction_tasks, enrichment_tasks)` - Enrich graph\n- `delete(data_id)` - Remove data\n- `config()` - Configuration management\n- `datasets()` - Dataset operations\n\nAll functions are async - use `await` or `asyncio.run()`.\n\n## Security Considerations\n\nSeveral security environment variables in `.env`:\n- `ACCEPT_LOCAL_FILE_PATH` - Allow local file paths (default: True)\n- `ALLOW_HTTP_REQUESTS` - Allow HTTP requests from Cognee (default: True)\n- `ALLOW_CYPHER_QUERY` - Allow raw Cypher queries (default: True)\n- `ENABLE_BACKEND_ACCESS_CONTROL` - Multi-tenant isolation (default: True). When `true`, API auth is required and per-user/dataset DB isolation is enabled. When `false`, single-user mode: shared DBs and auth off unless overridden.\n- `REQUIRE_AUTHENTICATION` - Explicit auth override. Unset (default): follows `ENABLE_BACKEND_ACCESS_CONTROL`. `false` is ignored when `ENABLE_BACKEND_ACCESS_CONTROL=true`. For a single-user deployment with auth off, set `ENABLE_BACKEND_ACCESS_CONTROL=false` (and optionally `REQUIRE_AUTHENTICATION=false`).\n\nFor production deployments, review and tighten these settings.\n\n## Common Patterns\n\n### Creating a Custom Pipeline Task\n```python\nfrom cognee.modules.pipelines.tasks.Task import Task\n\nasync def my_custom_task(data):\n    # Your logic here\n    processed_data = process(data)\n    return processed_data\n\n# Use in pipeline\ntask = Task(my_custom_task)\n```\n\n### Accessing Databases Directly\n```python\nfrom cognee.infrastructure.databases.graph import get_graph_engine\nfrom cognee.infrastructure.databases.vector import get_vector_engine_async\n\ngraph_engine = await get_graph_engine()\nvector_engine = await get_vector_engine_async()\n```\n\n### Using LLM Gateway\n```python\nfrom cognee.infrastructure.llm.get_llm_client import get_llm_client\n\nllm_client = get_llm_client()\nresponse = await llm_client.acreate_structured_output(\n    text_input=\"Your prompt\",\n    system_prompt=\"System instructions\",\n    response_model=YourPydanticModel\n)\n```\n\n## Key Concepts\n\n### Datasets\nDatasets are project-level containers that support organization, permissions, and isolated processing workflows. Each user can have multiple datasets with different access permissions.\n\n```python\n# Create/use a dataset\nawait cognee.add(data, dataset_name=\"my_project\")\nawait cognee.cognify(datasets=[\"my_project\"])\n```\n\n### DataPoints\nAtomic knowledge units that form the foundation of graph structures. All graph nodes extend the `DataPoint` base class with versioning and metadata support.\n\n### Permissions System\nMulti-tenant architecture with users, roles, and Access Control Lists (ACLs):\n- Read, write, delete, and share permissions per dataset\n- Enable with `ENABLE_BACKEND_ACCESS_CONTROL=True`\n- Supports isolated databases per user+dataset (Ladybug, LanceDB, SQLite, Postgres)\n\n### Graph Visualization\nLaunch visualization server:\n```bash\n# Via CLI\ncognee-cli -ui  # Launches full stack with UI at http://localhost:3000\n\n# Via Python\nfrom cognee.api.v1.visualize import start_visualization_server\nawait start_visualization_server(port=8080)\n```\n\n## Debugging & Troubleshooting\n\n### Debug Configuration\n- Set `LITELLM_LOG=\"DEBUG\"` for verbose LLM logs (default: \"ERROR\")\n- Enable debug mode: `ENV=\"development\"` or `ENV=\"debug\"`\n- Disable telemetry: `TELEMETRY_DISABLED=1`\n- Check logs in structured format (uses structlog)\n- Use `debugpy` optional dependency for debugging: `pip install cognee[debug]`\n\n### Common Issues\n\n**Ollama + OpenAI Embeddings NoDataError**\n- Issue: Mixing Ollama with OpenAI embeddings can cause errors\n- Solution: Configure both LLM and embeddings to use the same provider, or ensure `HUGGINGFACE_TOKENIZER` is set when using Ollama\n\n**LM Studio Structured Output**\n- Issue: LM Studio requires explicit instructor mode\n- Solution: Set `LLM_INSTRUCTOR_MODE=\"json_schema_mode\"` (or appropriate mode)\n\n**Default Provider Fallback**\n- Issue: Configuring only LLM or only embeddings defaults the other to OpenAI\n- Solution: Always configure both LLM and embedding providers, or ensure valid OpenAI API key\n\n**Permission Denied on Search**\n- Behavior: Returns empty list rather than error (prevents information leakage)\n- Solution: Check dataset permissions and user access rights\n\n**Database Connection Issues**\n- Check: Verify database URLs, credentials, and that services are running\n- Docker users: Use `DB_HOST=host.docker.internal` for local databases\n\n**Rate Limiting Errors**\n- Enable client-side rate limiting: `LLM_RATE_LIMIT_ENABLED=true`\n- Adjust limits: `LLM_RATE_LIMIT_REQUESTS` and `LLM_RATE_LIMIT_INTERVAL`\n\n## Resources\n\n- [Documentation](https://docs.cognee.ai/)\n- [Discord Community](https://discord.gg/NQPKmU5CCg)\n- [GitHub Issues](https://github.com/topoteretes/cognee/issues)\n- [Example Notebooks](examples/python/)\n- [Research Paper](https://arxiv.org/abs/2505.24478) - Optimizing knowledge graphs for LLM reasoning\n","category":"root","tokens":5380}]}