{"owner":"infiniflow","repo":"ragflow","hasSkills":true,"totalSkillsCount":14,"totalTokensCount":41067,"categories":["anthropic-skill","copilot-instructions","root-instruction","plugin-manifest"],"hasMcp":false,"mcpConfig":null,"found":["docs/references/_category_.json",".github/copilot-instructions.md","web/AGENTS.md","AGENTS.md",".agents/skills/go-naming/SKILL.md",".agents/rules/named.md","tools/scripts/INSTALL_SCRIPTS_README.md","tools/firecrawl/README.md","tools/es-to-oceanbase-migration/README.md","tools/firecrawl/INSTALLATION.md","docs/references/python_api_reference.md","docs/references/http_api_reference.md","tools/chatgpt-on-wechat/plugins/README.md","tools/chatgpt-on-wechat/plugins/config.json"],"skills":{"docs/references/_category_.json":"{\n \"label\": \"References\",\n \"position\": 6,\n \"link\": {\n \"type\": \"generated-index\",\n \"description\": \"Miscellaneous References\"\n },\n \"customProps\": {\n \"sidebarIcon\": \"LucideScrollText\"\n }\n}\n",".github/copilot-instructions.md":"# Project instructions for Copilot\n\n## How to run (minimum)\n- Install:\n - python -m venv .venv && source .venv/bin/activate\n - pip install -r requirements.txt\n- Run:\n - (fill) e.g. uvicorn app.main:app --reload\n- Verify:\n - (fill) curl http://127.0.0.1:8000/health\n\n## Project layout (what matters)\n- app/: API entrypoints + routers\n- services/: business logic\n- configs/: config loading (.env)\n- docs/: documents\n- tests/: pytest\n\n## Conventions\n- Prefer small, incremental changes.\n- Add logging for new flows.\n- Add/adjust tests for behavior changes.\n","web/AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with the RAGFlow frontend (`web/`).\n\n## Project Overview\n\nRAGFlow frontend is a React/TypeScript application built with Vite:\n\n- **Components**: shadcn/ui\n- **Styling**: Tailwind CSS\n- **State**: Zustand\n- **Data Fetching**: TanStack Query (React Query)\n- **i18n**: react-i18next\n\n## Common Commands\n\n```bash\nnpm install\nnpm run dev # Development server\nnpm run build # Production build\nnpm run lint # oxlint\nnpm run format # oxfmt\nnpm run test # Jest tests\n```\n\n## Development Conventions\n\n### Pre-commit Format Validation (Mandatory)\n\nFrontend code changes **must pass the lefthook format checks before every commit**. Do not rely on CI (`web-oxfmt` / `web-oxlint` jobs) to catch formatting after pushing.\n\n- Before staging or committing changed `.ts`/`.tsx`/`.js`/etc. files, run the formatter and linter locally:\n - `npx oxfmt ` (or `npm run format`) to auto-fix formatting\n - `npx oxlint` (or `npm run lint`) for lint issues\n- Verify with `npx oxfmt --check ` before `git commit`.\n- Let the lefthook pre-commit hook run to completion and fix everything it reports; never bypass hooks with `--no-verify`.\n\n### Test File Placement\n\nWhen tests are grouped in a dedicated directory, that directory is always named `__tests__/` — never `tests/`. Example: `src/pages/agent/utils/__tests__/extractor-transform.test.ts` for `src/pages/agent/utils/extractor-transform.ts`. Co-located `foo.test.ts(x)` files next to the source file are also fine. When you find an existing test under a `tests/` directory, move it into `__tests__/` (keep git history via `git mv`) rather than adding new tests there.\n\n### Backend Conventions\n\nThe frontend serves only the **Go** backend. There is no runtime backend detection and no per-backend branching in app code; the dev server proxies `/api` and `/v1` to the Go API (`vite.config.ts`).\n\nDo not reintroduce backend-variant primitives or detection helpers (e.g. a `useIsGoBackend` hook, `pickByBackend`, ``, or an `/api/v1/language` probe) and do not create `go/`-vs-`python/` variant directories or files.\n\n### CSS and Layout Debugging\n\nWhen fixing CSS/layout issues (especially flex truncation, ellipsis, or element sizing), **always inspect the full parent hierarchy** for `flex-shrink`, `min-width`, and `overflow` constraints before applying fixes like `min-w-0`. Do not repeatedly apply the same fix without verifying the root cause.\n\n- Before editing, explain: (1) the full flex/container hierarchy from the target element up to the nearest non-flex ancestor, (2) what constraint is actually causing the bug, and (3) how the proposed fix addresses that root cause.\n\n### Color Tokens\n\nWhen writing or modifying styles, **use the project-defined color tokens from `src/tailwind.css`** (e.g., `bg-bg-base`, `text-text-primary`, `text-text-secondary`, `text-text-disabled`, `border-border-button`, `bg-bg-card`). Do not use arbitrary hex/RGB values or Tailwind's default palette colors (e.g., `emerald-500`, `blue-400`) directly in component class names. These tokens are defined for both light and dark modes and keep the UI consistent with the design system.\n\n### Scope and Boundaries\n\nRespect explicit boundaries from the user. If the user says **\"only fix the selected line\"** or **\"do not touch shared types/files\"**, follow that instruction exactly. Do not investigate unrelated errors, modify shared schemas (e.g., `LlmSettingFieldSchema`), or refactor other files without confirmation. If a change outside the described scope seems necessary, ask for permission first.\n\n### Internationalization (i18n)\n\nFor translation tasks, add keys **only to the explicitly requested language files** (commonly `src/locales/zh.ts` and `src/locales/en.ts`). Do not auto-propagate changes to all language files unless the user explicitly asks.\n\n- **Style for `en.ts`**: Sentence case — first word capitalized, rest lowercase (e.g., `referenceAnswer: 'Reference answer'`). Proper nouns remain as-is.\n\n### React Component Refactoring\n\nWhen refactoring or extracting components, **verify layout behavior after each structural change** (especially `flex-1`, conditional rendering, or flex direction changes). Check that existing buttons, alignment, and responsive behavior remain intact. After extraction, verify: (1) all original props and behavior are preserved, (2) layout in parent contexts is identical, and (3) no syntax or type errors were introduced.\n\n### State Management and Data Fetching\n\n#### Query Key Factory (Mandatory)\n\n**Never write raw `queryKey` arrays inline.** Always use a query key factory object that returns `as const` tuples. Raw arrays duplicated across `useQuery` and `invalidateQueries` are brittle, unreadable, and cause stale-cache bugs when key structures drift.\n\n```ts\n// ❌ Bad — raw array, hard to match with useQuery\nqueryClient.invalidateQueries({\n queryKey: [\n LLMApiAction.AddedProviders,\n params.provider_name,\n params.instance_name,\n 'models',\n ],\n});\n\n// ✅ Good — factory reference, self-documenting\nqueryClient.invalidateQueries({\n queryKey: LlmKeys.instanceModels(params.provider_name, params.instance_name),\n});\n```\n\n- Place the factory in the same file as the hooks, named `{Domain}Keys` (e.g., `LlmKeys`, `DatasetKeys`).\n- Every `useQuery` and every `invalidateQueries` must reference the same factory function.\n- Use `as const` on each factory return value for type-safe readonly tuples.\n\n#### Cache Debugging\n\nFor React Query / cache invalidation bugs, **carefully compare query keys across all consuming components and mutation hooks**. Mismatched keys (e.g., with/without `refreshCount`) are a common root cause of stale data or duplicate requests.\n\n- Systematically: (1) list every component/hook that calls `useQuery` for this data, (2) compare their query keys character-for-character, (3) check every mutation's `onSuccess` for cache invalidation, and (4) verify no parent re-renders are remounting the observer.\n\n#### Colocate Queries with the Consuming View\n\n**Fire a query in the component that renders its data — not in a parent page.** When a page switches between mutually exclusive views (tabs, view modes), extract each view into its own component that issues its own requests on mount. Conditional rendering then provides lazy loading for free.\n\n- Do not hoist child-view queries into the page component — it fires requests the user may never need (e.g., fetching the skill tree on page entry while the default view is the LLM wiki).\n- Do not thread `enabled` flags or view-mode props through hooks to gate a hoisted query; that is a sign the query lives at the wrong level. Split the view instead.\n- Remember the trade-off: with `gcTime: 0`, unmounting a view drops its cache, so switching back refetches. That is usually desirable for always-fresh data — do not reintroduce eager hoisting just to avoid the refetch.\n\n### Network Request Layering\n\nHTTP requests are organized in three layers. **Never import `@/utils/request`, `@/utils/next-request`, or `@/utils/api` directly inside a hook**:\n\n1. `src/hooks/use-xx-request.ts(x)` — React Query hooks; only call the service layer.\n2. `src/services/xx-service.ts` — Register endpoints via `registerNextServer`, all going through `@/utils/next-request`.\n3. `src/utils/next-request.ts` — The single axios instance; handles token, 401 redirects, and error notifications.\n\nInterface types are split between two folders:\n\n- Response/data shape → `src/interfaces/database/xx.ts`\n- Request params/body → `src/interfaces/request/xx.ts`\n\nModel-related endpoints (LLM provider / factory / my LLM, etc.) are consolidated in `src/services/llm-service.ts` rather than scattered across hooks. For GET endpoints, register with `method: 'get'` in the service, and on the call site pass `true` as the second argument to use the native axios config (e.g., `service.listProviders({ params: { available: true } }, true)`).\n\n### Shared UI Component Lock\n\nThe folder `src/components/ui/` is the project's **shared UI library** — it contains both official shadcn/ui primitives and project-authored common components built on top of shadcn. Both kinds are intended to be reused across the app and **must not be modified casually**.\n\n- **Do not modify, refactor, restyle, or \"improve\"** any file under `src/components/ui/` (including subfolders), even if it seems like the most direct fix.\n- If a component does not meet requirements, **wrap or compose it** in a new component **outside** `src/components/ui/` (e.g., under `src/components/` or a feature folder), and customize via `className`, `props`, or composition.\n- Exceptions require **explicit user approval** in the same conversation. When in doubt, ask first and propose a wrapper-based alternative.\n- Adding a new shared component to `src/components/ui/`, or upgrading a shadcn primitive via the official `shadcn` CLI, is allowed only when the user explicitly requests it.\n\n### React Patterns and Conventions\n\n- **Avoid inline event handlers.** Do not define arrow functions directly on JSX event props like `onClick={() => ...}` or `onChange={(e) => ...}`. Instead, extract the handler and reference it by name (e.g., `onClick={handleClick}`). Inline handlers are recreated on every render, which can break `React.memo` optimizations and make stack traces harder to read.\n- **Prefer `requestAnimationFrame` or `useLayoutEffect`** over `setTimeout(..., 0)` for focus or DOM measurement operations.\n- **Prefer `useTranslation` from `react-i18next`** over project-wrapped utilities like `useTranslate`.\n- Extract complex logic into hooks or utils; keep components lean.\n- Use **PascalCase** for constants and component names.\n - Components: `EditableTextarea`, `RAGFlowFormItem`\n - Constants: `InitialMockData`, `DefaultPlaceholder`\n- Avoid camelCase or SCREAMING_SNAKE_CASE for components and top-level constants.\n- **Place domain constant lists in the page's `constant/` directory** (e.g. `pages/agent/constant/pipeline.tsx`) when they belong to an existing family of constants there (like `ChunkerOperators`, `SingleOperators`), instead of declaring them locally next to their consumer. Keeping related lists in one file makes their overlaps and differences visible and gives new entries a single obvious home.\n- **Name things semantically, not generically.** A name should say what the value holds or what the callback does, so call sites read naturally. Don't leak a generic underlying-primitive API into your own component's public props — rename at the wrapper boundary (e.g., a tabs wrapper exposes `activeTab` / `onTabChange`, mapped internally to Radix's `value` / `onValueChange`), and align with what callers already name the state. Same for variables and functions: `hiddenNodeIds` over `nodeIdSet`, `pickByBackend` over `resolveVariant`.\n- Avoid duplicating component structures in JSX; favor render props or reusable components.\n\n### Utility Libraries and Reuse\n\n- **Time/date handling**: Use `dayjs` for all date/time formatting, parsing, and manipulation.\n- **Utility hooks**: Prefer `ahooks` for common reusable hooks (e.g., `useDebounce`, `useSetState`).\n- **General utilities**: Lodash is available for utility functions when needed.\n- **Project utilities first**: Before reaching for a third-party library, check if the project already has an existing utility or hook that covers the need.\n- **Extract and share**: If repeated logic cannot be satisfied by an existing project utility or a third-party library, extract it into an appropriate shared hook (`src/hooks/`) or utility file (`src/utils/`).\n- **Check for duplicate patterns before adding**: When asked to add logic (validation, existence checks, API calls, etc.), first search for existing hooks/functions that do the same or similar thing — especially in the same file or sibling hooks. If a hook already does X, call it instead of re-implementing X inline. This applies to mutations calling mutations, utility wrappers, and boilerplate around API calls.\n","AGENTS.md":"# RAGFlow Instructions\n\nUse this file as the local operating guide for the current codebase. Prefer the code and the current AGENTS.md over any older convention or remembered project shape.\n\n## Core Stance\n- Treat legacy code as liability, not as a compatibility target.\n- Prefer deletion over shims, deprecated branches, wrapper APIs, and dual-track migration notes.\n- If old and new implementations coexist, converge to one path unless an external contract forces compatibility.\n- Remove dead tests, commented-out code, stale docs, and \"move later\" notes instead of preserving them.\n- Reduce public surface area when a helper can be made private or internal.\n- Keep refactors centered on the owning abstraction, not on adjacent compatibility layers.\n\n## Current stack\n- Backend: Go implements the API, admin, ingestor, and syncer modes in `cmd/ragflow_server.go`; the CLI runs from `cmd/ragflow-cli.go`. The Docker entrypoint starts the Go services according to its command-line flags, and the frontend development server uses fixed Go backend proxy routes. The Python API, admin, and workers are still present pending removal.\n- Frontend: React + TypeScript + Vite in `web/`. When working under `web/`, read and follow `web/AGENTS.md` for frontend conventions.\n- The Go module contains the server, ingestion, parsing, agent runtime, CLI, and supporting services.\n- Runtime services commonly include MySQL/PostgreSQL, Kvrocks, NATS JetStream, MinIO, ClickHouse, and Elasticsearch/Infinity/OpenSearch depending on configuration.\n## Go-only target\n- Remove all Python code and Python-specific dependencies, scripts, containers, CI jobs, documentation, SDKs, and tests from the repository. Do not add new Python code while this removal is in progress. Replace Python-based preparation of Go native libraries, DeepDoc models, and the BPE table before removing the existing download scripts.\n- Move any still-required behavior and test coverage into the Go backend or the frontend before deleting the old implementation. Do not preserve a Python server, compatibility layer, fallback, or dual-backend path.\n- Before deleting `rag/`, move the Go DeepDoc `.ort` models and `ocr.res` out of `internal/rag/res/deepdoc/`, then update model discovery, dependency preparation, Docker packaging, and tests together.\n- Remove the frontend's Go/Python backend variants and update `web/CLAUDE.md` when the frontend no longer needs them. Until then, treat its dual-backend instructions as migration-only guidance; new frontend work should target the Go API.\n- Update Docker's default entrypoint, dependency image, and CI workflows when removing their remaining Python paths. Keep the frontend development proxy routes aligned with the Go API and Admin ports in `web/vite.config.ts`.\n- Keep this guide aligned with the checked-in tree. Once the Python directories are deleted, remove their migration inventory below.\n\n## Code Layout to Expect\n- `cmd/ragflow_server.go`: Go entrypoint for the API, admin, ingestor, and syncer modes (`--api`, `--admin`, `--ingestor`, `--syncer`).\n- `cmd/ragflow-cli.go`: Go CLI entrypoint.\n- `api/`, `rag/`, `deepdoc/`, and `agent/`: legacy Python implementation pending deletion. Do not add features or extend dependencies there.\n- `internal/`: main Go application code. Important subtrees:\n- `internal/agent/`: Go agent runtime, canvas execution, components, tool bindings, workflow helpers.\n- `internal/admin/`: Go admin routes, handlers, and services.\n- `internal/binding/cpp/`: C++ tokenizer binding built by `build.sh` for the Go server.\n- `internal/cli/`: CLI parsing, HTTP transport, command execution, response formatting.\n- `internal/dao/`: Go data-access layer and persistence-facing helpers.\n- `internal/deepdoc/`: Go DeepDOC integrations, especially native-backed PDF/DOCX parsing.\n- `internal/engine/`: search/index backends such as Elasticsearch and Infinity.\n- `internal/entity/`: shared Go entities and model definitions.\n- `internal/handler/`: HTTP handlers and route-facing request logic.\n- `internal/ingestion/`: Go ingestion pipeline, canvas adapter, components, wiring, service orchestration.\n- `internal/ingestion/component/`: stage implementations such as file/parser/chunker/tokenizer/extractor.\n- `internal/ingestion/pipeline/`: DSL translation, canvas-driven execution, checkpoints, resume/run logic.\n- `internal/parser/`: parser and chunk libraries used by ingestion and other Go paths.\n- `internal/parser/parser/`: typed parse-result parsers for markdown/html/pdf/docx/xlsx/text and related families.\n- `internal/parser/chunk/`: chunk operator library and DSL/typed execution helpers.\n- `internal/service/`: higher-level business services used by handlers and server flows.\n- `internal/storage/`: storage backends and in-memory test doubles.\n- `internal/router/`: HTTP route registration.\n- `internal/server/`: server bootstrap/config wiring.\n- `internal/syncer/`: Go file synchronization service and connectors.\n- `internal/rag/`: Go retrieval and RAG logic.\n- `internal/harness/`: Go runtime harness and supporting execution code.\n- `web/`: frontend application.\n- `docker/`: local and production compose files.\n- `sdk/python/`: legacy Python client SDK pending deletion.\n- `tools/scripts/download_dev_deps.py` and `deps/download_deps.py`: Python downloaders for the Go build's dependencies. `download_dev_deps.py` pulls the native static libraries (pdfium, pdf_oxide, office_oxide, onnxruntime) that `build.sh` links; `download_deps.py` pulls the runtime resources (cl100k BPE table, DeepDoc models, embedding tokenizer assets) and installs the stagehand server into the SDK cache. Versions and URLs are defined inline in each downloader (`tools/scripts/download_dev_deps.py` for native static libraries, `deps/download_deps.py` for runtime resources); the tokenizer assets are inlined as `TOKENIZER_ASSETS` in each downloader. Replace these with non-Python preparation before removing Python dependencies.\n- `test/`: contains Python tests, including tests that import the legacy Python implementation directly. Migrate needed coverage to Go tests or frontend tests, then remove the Python tests; keep only tests for supported code.\n\n## Go-Specific Rules\n- Treat `internal/ingestion`, `internal/parser`, and `internal/deepdoc` as actively refactored code. Prefer collapsing duplicate paths over preserving transitional wrappers.\n- Do not add or preserve deprecated Go APIs just to ease migration inside the repo.\n- Remove commented-out Go code instead of leaving recovery notes in place.\n- Keep package comments and doc comments aligned with the current runtime path, not with migration history.\n\n## GORM query traps\n- **Never pass a bare string ID as the second argument to `Take`/`First`** (e.g. `Take(&x, id)`). Under GORM v1.25.7 a non-numeric string is treated as a raw SQL condition and concatenated into the `WHERE` clause: `Take(&x, \"88b17c7a...\")` emits `WHERE 88b17c7a... LIMIT 1`, which MySQL rejects with `Error 1054 (42S22): Unknown column '...' in 'where clause'`. Only strings that parse via `strconv.Atoi` are treated as primary-key values, so this bug is invisible for integer IDs and catastrophic for UUID primary keys. Always bind explicitly: `Take(&x, \"id = ?\", id)` or `Where(\"id = ?\", id).Take(&x)`.\n- When you change a query form across many files, verify the generated SQL with a `Session(&gorm.Session{DryRun: true})` probe before relying on it. A passing unit test with no rows and a broken `WHERE` clause look identical unless the SQL is inspected.\n\n## Shared database schema (Go + Python)\nThe Go services and the Python API write to the same MySQL/PostgreSQL schema, and both own parts of it. Neither is authoritative over the whole.\n\nEach ORM derives index names differently:\n- Go (`internal/entity/`): GORM derives `idx__` for indexes and `uni_
_` for unique constraints. Tags in this repo name unique indexes explicitly, e.g. `uniqueIndex:idx_commit_file`.\n- Python (`api/db/db_models.py`): Peewee derives `__` (truncated with an md5 suffix past 64 chars, see `playhouse.migrate.make_index_name`). There is no `idx_` prefix.\n\nBoth sides reconcile by **column set, not by name**: Go's `hasUniqueIndex` (`internal/dao/migration.go`) accepts an existing index only when its full column set equals the requested one, and Python's `ensure_model_indexes` (`api/db/db_models.py`) keys `DB.get_indexes()` on the column tuple. Consequences:\n\n- Divergent names covering the same columns are expected, not a defect. Whichever runtime reaches the database first decides the name. Do not \"fix\" this by renaming inside a migration; it rewrites live databases for no functional gain.\n- Add a shared unique index to **both** sides (Go tag + Peewee `Meta.indexes` or `unique=True`), so each runtime recreates it when absent. Do not add a second index over a column set an existing one already covers.\n- In Go, declare single-column uniqueness with a named `uniqueIndex:`, not a bare `unique` tag. A bare `unique` becomes a table-level constraint in DDL, which SQLite materializes as an implicit index that cannot be dropped by name, and which GORM's `MigrateColumnUnique` then tries to drop under its own `uni_
_` name — MySQL answers 1091 every startup. `namedIndexMigrator` (`internal/dao/database.go`) suppresses exactly that phantom drop; see `TestNamedIndexMigratorSkipsPhantomUniqueDrop`.\n- A Python model existing does not mean Python owns that table's constraints. `pipeline_operation_log` has no `run_count` column and no unique index in `db_models.py`; both are added only by `internal/dao/migration.go`.\n\n## Go Test Tiers\nGo tests are classified by build tag. Tag a test file with `//go:build ` placed before the `package` clause. Run them through `build.sh` so the required CGO configuration and native libraries are available.\n\n| Tier | Build tag | Runs by default? | Needs |\n|---|---|---|---|\n| Unit | (none) | Yes (`bash build.sh --test`) | Native CGO static libs; no external services — uses in-memory SQLite, miniredis, or `httptest` stubs. |\n| Integration | `integration` | No (`-tags integration`) | A real service: MySQL/MinIO/Elasticsearch/Infinity/LLM. Single component, reasonably fast. |\n| E2E | `e2e` | No (`-tags e2e`) | Full cross-component pipeline (ingest → index → retrieve) against real services; heavy/slow. |\n| Manual | `manual` | No (`-tags manual`) | Very slow/expensive (deepdoc render/parity/snapshot/bench). **Local opt-in ONLY — never run in CI.** |\n| Native (orthogonal) | `cgo` / `!cgo` | `cgo` auto-satisfies under CGO_ENABLED=1 | Native static libs (`office_oxide`/`pdfium`/`pdf_oxide`). Combine with tiers, e.g. `//go:build cgo && integration`. |\n\nRun tiers locally via `build.sh`:\n```bash\nbash build.sh --test # unit tier (no tags)\nbash build.sh --test-integration ./... # integration tier\nbash build.sh --test-e2e # e2e tier\nbash build.sh --test-manual # manual tier (very slow)\nbash build.sh --test-all # integration + e2e (never includes manual)\n```\nRules:\n- New tests that touch a real external service MUST carry `integration`/`e2e`/`manual` — do not rely on `t.Skip` + env vars to soft-isolate them in the default unit run. Keep an env guard as a harmless secondary safety net if desired.\n- `manual` is never wired into CI or any automated pipeline.\n- `unit` (no tag) must stay free of external-service dependencies so `bash build.sh --test` runs without MySQL/MinIO/ES/Infinity/LLM. The native CGO static libraries (`office_oxide`/`pdfium`/`pdf_oxide`) are still required at build time and are wired by `build.sh --test`; that is expected, not an external service.\n\n## Working Rules\n- When reviewing documentation or code, inspect the full affected path and report all verifiable findings in one review; do not return after only a few findings and expose further issues in later rounds.\n- When handling review comments, independently verify each substantive claim against the current code or tests before accepting, rejecting, or acting on it.\n- Before editing, inspect the nearest code path that actually owns the behavior.\n- For server changes, work in the Go implementation. If a Python behavior is still needed, port it to Go and test the Go path before removing the Python code.\n- Keep changes small and local unless the task is explicitly a broader refactor.\n- Prefer one implementation path instead of preserving old and new versions side by side.\n- Preserve behavior with focused tests when the behavior is still valid; do not keep tests that protect obsolete behavior.\n- If a surface is only there for compatibility, remove it unless the user asks to keep it.\n- Do not add new compatibility wording in comments or docs.\n- When a maintainer takes over a community PR, a new commit generated by rewriting history (e.g. `merge`, `rebase -i`) must preserve the original author and add the maintainer as co-author (via a `Co-authored-by:` trailer) instead of overwriting the author with the maintainer alone.\n\n## Commands\n### Go backend\n```bash\nbash build.sh --all # build native code and Go server\nbash build.sh --go # build Go server with native library already built\nbash build.sh --test ./internal/handler/... # targeted unit tests\nbash build.sh --test # all unit tests\nbash build.sh --test-integration ./... # integration tests; requires real services\nbash build.sh --test-e2e # full-pipeline tests; requires real services\nbash build.sh --test-manual # local opt-in only\nbash build.sh --test-all # integration + e2e; excludes manual\n```\n\nNative libraries, Go DeepDoc models, tokenizer assets, and `deps/cl100k_base.tiktoken` come from `tools/scripts/download_dev_deps.py` (native static libs) and `deps/download_deps.py` (models, cl100k, tokenizer assets); run them with `uv run`. Build the Go resource image with `cd deps && docker build -f Dockerfile -t infiniflow/ragflow_deps:latest .` after downloading the resources.\n\n### Frontend\n```bash\ncd web\nnpm install\nnpm run dev\nnpm run build\nnpm run lint\nnpm run test\nnpm run type-check\n```\n\n## Validation Preference\n- Run the narrowest relevant test, lint, or build command after a change.\n- For backend changes, first run package-scoped `bash build.sh --test ./path/to/package/...`. Run the integration or e2e tier when the changed behavior needs real services or a full pipeline.\n- For frontend changes, prefer the touched-package lint, type-check, or test command.\n- Do not default to raw `go test`, `go build`, or IDE Run/Debug for Go in this repo. They often miss the required CGO flags and native static libraries (`office_oxide`, `pdfium-static`, `pdf_oxide`) that `build.sh` wires correctly.\n- After a Go build, execute the resulting binary's help command for each changed server mode; a successful link alone does not prove the binary starts. On this host, the original `build.sh` selected default LLD 18 and produced binaries that crashed before Go `main`; linking with LLD 20 started successfully. Verify the linker actually selected on Linux before trusting a new binary.\n- If Go native builds fail, inspect `build.sh` and `internal/development.md` before changing code. Common environment issues are missing downloaded native dependencies or an incompatible `ld.lld` on Linux.\n\n## Default review checklist\n- Remove instead of retaining `deprecated`, `legacy`, or compatibility-only code.\n- Collapse duplicate implementations to one path.\n- Drop stale comments and documentation that describe a superseded design.\n- Keep exported APIs only when the current code actually needs them.\n",".agents/skills/go-naming/SKILL.md":"---\nname: go-naming\ndescription: Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards.\n---\n\nStrictly follow the naming conventions in [rules/named.md](../../rules/named.md)\n",".agents/rules/named.md":"# Go Naming Best Practices\n\n## 1. Package Naming\n\n- **All lowercase, no underscores**: `package user`, not `package userService` or `package user_service`\n- **Short and meaningful**: `package http`, `package json`, `package dao`\n- **Avoid plurals**: `package user` not `package users`\n- **Avoid generic names**: Avoid `package util`, `package common`, `package base`\n\n```go\n// Recommended\npackage user\npackage handler\npackage service\n\n// Not recommended\npackage UserService\npackage user_service\npackage utils\n```\n\n## 2. File Naming\n\n- **All lowercase, underscore separated**: `user_handler.go`, `user_service.go`\n- **Test files**: `user_handler_test.go`\n- **Platform-specific**: `user_linux.go`, `user_windows.go`\n\n```\nuser/\n├── user_handler.go\n├── user_service.go\n├── user_dao.go\n└── user_test.go\n```\n\n## 3. Directory Naming\n\n- **All lowercase, no underscores or hyphens**: `internal/`, `pkg/`, `cmd/`\n- **Short and descriptive**: `handler/`, `service/`, `dao/`\n\n```\nproject/\n├── cmd/ # Main entry point\n│ └── server_main.go\n├── internal/ # Private code\n│ ├── handler/\n│ ├── service/\n│ ├── dao/\n│ ├── model/\n│ └── middleware/\n├── pkg/ # Public code\n└── api/ # API definitions\n```\n\n## 4. Interface Naming\n\n- **Single-method interfaces end with \"-er\"**: `Reader`, `Writer`, `Handler`\n- **Verb form**: `Reader`, `Executor`, `Validator`\n\n```go\n// Recommended\ntype Reader interface {\n Read(p []byte) (n int, err error)\n}\n\ntype UserService interface {\n Register(req *RegisterRequest) (*User, error)\n Login(req *LoginRequest) (*User, error)\n}\n\n// Not recommended\ntype UserInterface interface {}\ntype IUserService interface {}\n```\n\n## 5. Struct Naming\n\n- **CamelCase**: `UserService`, `UserHandler`\n- **Avoid redundant prefixes**: `User` not `UserModel`\n\n```go\n// Recommended\ntype UserService struct {}\ntype UserHandler struct {}\ntype RegisterRequest struct {}\n\n// Not recommended\ntype user_service struct {}\ntype SUserService struct {}\ntype UserModel struct {}\n```\n\n## 6. Method/Function Naming\n\n- **CamelCase**\n- **Start with verb**: `GetUser`, `CreateUser`, `DeleteUser`\n- **Boolean returns use Is/Has/Can prefix**: `IsValid`, `HasPermission`\n\n```go\n// Recommended\nfunc (s *UserService) Register(req *RegisterRequest) (*User, error)\nfunc (s *UserService) GetUserByID(id uint) (*User, error)\nfunc (s *UserService) IsEmailExists(email string) bool\n\n// Not recommended\nfunc (s *UserService) register_user()\nfunc (s *UserService) get_user_by_id()\nfunc (s *UserService) CheckEmailExists() // Should use Is/Has\n```\n\n## 7. Constant Naming\n\n- **CamelCase**: `const MaxRetryCount = 3`\n- **Enum constants**: `const StatusActive = \"active\"`\n\n```go\n// Recommended\nconst (\n StatusActive = \"1\"\n StatusInactive = \"0\"\n MaxRetryCount = 3\n)\n\n// Not recommended\nconst (\n STATUS_ACTIVE = \"1\" // Not all uppercase\n status_active = \"1\" // Not all lowercase\n)\n```\n\n## 8. Error Variable Naming\n\n- **Start with \"Err\"**: `ErrNotFound`, `ErrInvalidInput`\n\n```go\n// Recommended\nvar (\n ErrNotFound = errors.New(\"not found\")\n ErrInvalidInput = errors.New(\"invalid input\")\n ErrUnauthorized = errors.New(\"unauthorized\")\n)\n```\n\n## 9. Acronyms Keep Consistent Case\n\n```go\n// Recommended\ntype HTTPHandler struct {}\nvar URL string\nfunc GetHTTPClient() {}\nfunc ParseJSON() {}\n\n// Not recommended\ntype HttpHandler struct {}\nvar Url string\nfunc GetHttpClient() {}\n```\n\n## 10. Project Structure Naming\n\n```\nproject-name/\n├── cmd/ # Main programs\n│ └── app_name/\n│ └── main.go\n├── internal/ # Private code\n│ ├── handler/ # HTTP handlers\n│ ├── service/ # Business logic\n│ ├── repository/ # Data access\n│ ├── model/ # Data models\n│ └── config/ # Configuration\n├── pkg/ # Public code\n├── api/ # API definitions\n├── configs/ # Config files\n├── scripts/ # Scripts\n├── docs/ # Documentation\n├── go.mod\n└── go.sum\n```\n\n## Summary Table\n\n| Type | Rule | Example |\n| -------------- | ----------------------------------- | ------------------- |\n| Package | All lowercase, no underscores | `package user` |\n| File | All lowercase, underscore separated | `user_service.go` |\n| Directory | All lowercase, no separators | `internal/handler/` |\n| Struct | CamelCase, capitalized first letter | `UserService` |\n| Interface | CamelCase, -er suffix | `Reader`, `Writer` |\n| Method | CamelCase, verb prefix | `GetUserByID` |\n| Constant | CamelCase | `MaxRetryCount` |\n| Error Variable | Err prefix | `ErrNotFound` |\n","tools/scripts/INSTALL_SCRIPTS_README.md":"# RAGFlow CLI Installation Scripts\n\nRAGFlow publishes static Go CLI binaries as GitHub Release assets for:\n\n- Linux: `amd64`, `arm64`\n- macOS: `amd64`, `arm64`\n- Windows: `amd64`, `arm64`\n\nThe release workflow builds the binaries with `CGO_ENABLED=0`, uploads `SHA256SUMS`, and uploads `install.sh` and `install.ps1`.\n\n## Linux And macOS\n\nUse the script maintained in the RAGFlow repository:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | sh\n```\n\nInstall a specific version:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | VERSION=v1.0.0 sh\n```\n\nInstall to a user-writable directory:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | INSTALL_DIR=\"$HOME/.local/bin\" sh\n```\n\nThe Unix installer:\n\n- detects `linux` or `darwin`\n- detects `amd64` or `arm64`\n- downloads `ragflow-cli-{VERSION}-{OS}-{ARCH}`\n- verifies the file with the release `SHA256SUMS`\n- installs to `/usr/local/bin` by default\n\n## Windows PowerShell\n\nRun the repository script in PowerShell:\n\n```powershell\nirm https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.ps1 | iex\n```\nInstall a specific version:\n\n```powershell\n.\\install.ps1 -Version \"v1.0.0\"\n```\n\nInstall to a custom directory:\n\n```powershell\n.\\install.ps1 -InstallDir \"$env:USERPROFILE\\bin\"\n```\n\nThe Windows installer:\n\n- detects `windows/amd64` or `windows/arm64`\n- downloads `ragflow-cli-{VERSION}-windows-{ARCH}.exe`\n- verifies the file with the release `SHA256SUMS`\n- installs to `$env:LOCALAPPDATA\\Programs\\RAGFlow` by default\n- adds the install directory to the user `PATH`\n\n## Release Asset Names\n\nThe install scripts expect these names:\n\n```text\nragflow-cli-v1.0.0-linux-amd64\nragflow-cli-v1.0.0-linux-arm64\nragflow-cli-v1.0.0-darwin-amd64\nragflow-cli-v1.0.0-darwin-arm64\nragflow-cli-v1.0.0-windows-amd64.exe\nragflow-cli-v1.0.0-windows-arm64.exe\nSHA256SUMS\ninstall.sh\ninstall.ps1\n```\n\nBy default, both scripts install the latest stable GitHub Release. Set\n`VERSION` on Unix or `-Version` on Windows to pin a specific release.\n","tools/firecrawl/README.md":"# Firecrawl Integration for RAGFlow\n\nThis integration adds [Firecrawl](https://firecrawl.dev)'s powerful web scraping capabilities to [RAGFlow](https://github.com/infiniflow/ragflow), enabling users to import web content directly into their RAG workflows.\n\n## 🎯 **Integration Overview**\n\nThis integration implements the requirements from [Firecrawl Issue #2167](https://github.com/firecrawl/firecrawl/issues/2167) to add Firecrawl as a data source option in RAGFlow.\n\n### ✅ **Acceptance Criteria Met**\n\n- ✅ **Integration appears as selectable data source** in RAGFlow's UI\n- ✅ **Users can input Firecrawl API keys** through RAGFlow's configuration interface\n- ✅ **Successfully scrapes content** and imports into RAGFlow's document processing pipeline\n- ✅ **Handles edge cases** (rate limits, failed requests, malformed content)\n- ✅ **Includes documentation** and README updates\n- ✅ **Follows RAGFlow patterns** and coding standards\n- ✅ **Ready for engineering review**\n\n## 🚀 **Features**\n\n### Core Functionality\n- **Single URL Scraping** - Scrape individual web pages\n- **Website Crawling** - Crawl entire websites with job management\n- **Batch Processing** - Process multiple URLs simultaneously\n- **Multiple Output Formats** - Support for markdown, HTML, links, and screenshots\n\n### Integration Features\n- **RAGFlow Data Source** - Appears as selectable data source in RAGFlow UI\n- **API Configuration** - Secure API key management with validation\n- **Content Processing** - Converts Firecrawl output to RAGFlow document format\n- **Error Handling** - Comprehensive error handling and retry logic\n- **Rate Limiting** - Built-in rate limiting and request throttling\n\n### Quality Assurance\n- **Content Cleaning** - Intelligent content cleaning and normalization\n- **Metadata Extraction** - Rich metadata extraction and enrichment\n- **Document Chunking** - Automatic document chunking for RAG processing\n- **Language Detection** - Automatic language detection\n- **Validation** - Input validation and error checking\n\n## 📁 **File Structure**\n\n```\ntools/firecrawl/\n├── __init__.py # Package initialization\n├── firecrawl_connector.py # API communication with Firecrawl\n├── firecrawl_config.py # Configuration management\n├── firecrawl_processor.py # Content processing for RAGFlow\n├── firecrawl_ui.py # UI components for RAGFlow\n├── ragflow_integration.py # Main integration class\n├── example_usage.py # Usage examples\n├── requirements.txt # Python dependencies\n├── README.md # This file\n└── INSTALLATION.md # Installation guide\n```\n\n## 🔧 **Installation**\n\n### Prerequisites\n- RAGFlow instance running\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n### Setup\n1. **Get Firecrawl API Key**:\n - Visit [firecrawl.dev](https://firecrawl.dev)\n - Sign up for a free account\n - Copy your API key (starts with `fc-`)\n\n2. **Configure in RAGFlow**:\n - Go to RAGFlow UI → Data Sources → Add New Source\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify setup\n - You should see a success message\n\n## 🎮 **Usage**\n\n### Single URL Scraping\n1. Select \"Single URL\" as scrape type\n2. Enter the URL to scrape\n3. Choose output formats (markdown recommended for RAG)\n4. Start scraping\n\n### Website Crawling\n1. Select \"Crawl Website\" as scrape type\n2. Enter the starting URL\n3. Set crawl limit (maximum number of pages)\n4. Configure extraction options\n5. Start crawling\n\n### Batch Processing\n1. Select \"Batch URLs\" as scrape type\n2. Enter multiple URLs (one per line)\n3. Choose output formats\n4. Start batch processing\n\n## 🔧 **Configuration Options**\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## 📊 **API Reference**\n\n### RAGFlowFirecrawlIntegration\n\nMain integration class for Firecrawl with RAGFlow.\n\n#### Methods\n- `scrape_and_import(urls, formats, extract_options)` - Scrape URLs and convert to RAGFlow documents\n- `crawl_and_import(start_url, limit, scrape_options)` - Crawl website and convert to RAGFlow documents\n- `test_connection()` - Test connection to Firecrawl API\n- `validate_config(config_dict)` - Validate configuration settings\n\n### FirecrawlConnector\n\nHandles communication with the Firecrawl API.\n\n#### Methods\n- `scrape_url(url, formats, extract_options)` - Scrape single URL\n- `start_crawl(url, limit, scrape_options)` - Start crawl job\n- `get_crawl_status(job_id)` - Get crawl job status\n- `batch_scrape(urls, formats)` - Scrape multiple URLs concurrently\n\n### FirecrawlProcessor\n\nProcesses Firecrawl output for RAGFlow integration.\n\n#### Methods\n- `process_content(content)` - Process scraped content into RAGFlow document format\n- `process_batch(contents)` - Process multiple scraped contents\n- `chunk_content(document, chunk_size, chunk_overlap)` - Chunk document content for RAG processing\n\n## 🧪 **Testing**\n\nThe integration includes comprehensive testing:\n\n```bash\n# Run the test suite\ncd tools/firecrawl\npython3 -c \"\nimport sys\nsys.path.append('.')\nfrom ragflow_integration import create_firecrawl_integration\n\n# Test configuration\nconfig = {\n 'api_key': 'fc-test-key-123',\n 'api_url': 'https://api.firecrawl.dev'\n}\n\nintegration = create_firecrawl_integration(config)\nprint('✅ Integration working!')\n\"\n```\n\n## 🐛 **Error Handling**\n\nThe integration includes robust error handling for:\n\n- **Rate Limiting** - Automatic retry with exponential backoff\n- **Network Issues** - Retry logic with configurable timeouts\n- **Malformed Content** - Content validation and cleaning\n- **API Errors** - Detailed error messages and logging\n\n## 🔒 **Security**\n\n- API key validation and secure storage\n- Input sanitization and validation\n- Rate limiting to prevent abuse\n- Error handling without exposing sensitive information\n\n## 📈 **Performance**\n\n- Concurrent request processing\n- Configurable timeouts and retries\n- Efficient content processing\n- Memory-conscious document handling\n\n## 🤝 **Contributing**\n\nThis integration was created as part of the [Firecrawl bounty program](https://github.com/firecrawl/firecrawl/issues/2167).\n\n### Development\n1. Fork the RAGFlow repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## 📄 **License**\n\nThis integration is licensed under the same license as RAGFlow (Apache 2.0).\n\n## 🆘 **Support**\n\n- **Firecrawl Documentation**: [docs.firecrawl.dev](https://docs.firecrawl.dev)\n- **RAGFlow Documentation**: [RAGFlow GitHub](https://github.com/infiniflow/ragflow)\n- **Issues**: Report issues in the RAGFlow repository\n\n## 🎉 **Acknowledgments**\n\nThis integration was developed as part of the Firecrawl bounty program to bridge the gap between web content and RAG applications, making it easier for developers to build AI applications that can leverage real-time web data.\n\n---\n\n**Ready for RAGFlow Integration!** 🚀\n\nThis integration enables RAGFlow users to easily import web content into their knowledge retrieval systems, expanding the ecosystem for both Firecrawl and RAGFlow.\n","tools/es-to-oceanbase-migration/README.md":"# RAGFlow ES to OceanBase Migration Tool\n\nA CLI tool for migrating RAGFlow data from Elasticsearch to OceanBase. This tool is specifically designed for RAGFlow's data structure and handles schema conversion, vector data mapping, batch import, and resume capability.\n\n## Features\n\n- **RAGFlow-Specific**: Designed for RAGFlow's fixed data schema\n- **ES 8+ Support**: Uses `search_after` API for efficient data scrolling\n- **Vector Support**: Auto-detects vector field dimensions from ES mapping\n- **Batch Processing**: Configurable batch size for optimal performance\n- **Resume Capability**: Save and resume migration progress\n- **Data Consistency Validation**: Compare document counts and sample data\n- **Migration Report Generation**: Generate detailed migration reports\n\n## Quick Start\n\nThis section provides a complete guide to verify the migration works correctly with a real RAGFlow deployment.\n\n### Prerequisites\n\n- RAGFlow source code cloned\n- Docker and Docker Compose installed\n- This migration tool installed (`uv pip install -e .`)\n\n### Step 1: Start RAGFlow with Elasticsearch Backend\n\nFirst, start RAGFlow using Elasticsearch as the document storage backend (default configuration).\n\n```bash\n# Navigate to RAGFlow docker directory\ncd /path/to/ragflow/docker\n\n# Ensure DOC_ENGINE=elasticsearch in .env (this is the default)\n# DOC_ENGINE=elasticsearch\n\n# Start RAGFlow with Elasticsearch (--profile cpu for CPU, --profile gpu for GPU)\ndocker compose --profile elasticsearch --profile cpu up -d\n\n# Wait for services to be ready (this may take a few minutes)\ndocker compose ps\n\n# Check ES is running\ncurl -X GET \"http://localhost:9200/_cluster/health?pretty\"\n```\n\n### Step 2: Create Test Data in RAGFlow\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Create a new Knowledge Base\n3. Upload some test documents (PDF, TXT, DOCX, etc.)\n4. Wait for the documents to be parsed and indexed\n5. Test the knowledge base with some queries to ensure it works\n\n### Step 3: Verify ES Data (Optional)\n\nBefore migration, verify the data exists in Elasticsearch. This step is important to ensure you have a baseline for comparison after migration.\n\n```bash\n# Navigate to migration tool directory (from ragflow root)\ncd tools/es-to-oceanbase-migration\n\n# Activate the virtual environment if not already done\nsource .venv/bin/activate\n\n# Check connection and list indices\nes-ob-migrate status --es-host localhost --es-port 9200\n\n# First, find your actual index name (pattern: ragflow_{tenant_id})\ncurl -X GET \"http://localhost:9200/_cat/indices/ragflow_*?v\"\n\n# List all knowledge bases in the index\n# Replace ragflow_{tenant_id} with your actual index from the curl output above\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n\n# View sample documents\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_{tenant_id} --size 5\n\n# Check schema\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n```\n\n### Step 4: Start OceanBase for Migration\n\nStart RAGFlow's OceanBase service as the migration target:\n\n```bash\n# Navigate to ragflow docker directory (from ragflow root)\ncd ../docker\n\n# Start only OceanBase service from RAGFlow docker compose\ndocker compose --profile oceanbase up -d\n\n# Wait for OceanBase to be ready\ndocker compose logs -f oceanbase\n```\n\n### Step 5: Run Migration\n\nExecute the migration from Elasticsearch to OceanBase:\n\n```bash\ncd ../tools/es-to-oceanbase-migration\n\n# Option A: Migrate ALL ragflow_* indices (Recommended)\n# If --index and --table are omitted, the tool auto-discovers all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --batch-size 1000 \\\n --verify\n\n# Option B: Migrate a specific index\n# Use the SAME name for both --index and --table\n# The index name pattern is: ragflow_{tenant_id}\n# Find your tenant_id from Step 3's curl output\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --batch-size 1000 \\\n --verify\n```\n\nExpected output:\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_{tenant_id}\nTarget: localhost:2881/ragflow_doc.ragflow_{tenant_id}\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK (version: 4.3.5.1)\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Known RAGFlow fields: 25\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_{tenant_id}' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nStep 5: Verifying migration...\n✓ Document counts match: 1,234\n✓ Sample verification: 100/100 matched\n\nMigration completed successfully!\n Total: 1,234 documents\n Migrated: 1,234 documents\n Failed: 0 documents\n Duration: 45.2 seconds\n```\n\n### Step 6: Stop RAGFlow and Switch to OceanBase Backend\n\n```bash\n# Navigate to ragflow docker directory\ncd ../../docker\n\n# Stop only Elasticsearch and RAGFlow (but keep OceanBase running)\ndocker compose --profile elasticsearch --profile cpu down\n\n# Edit .env file, change:\n# DOC_ENGINE=elasticsearch -> DOC_ENGINE=oceanbase\n#\n# The OceanBase connection settings are already configured by default in .env\n```\n\n### Step 7: Start RAGFlow with OceanBase Backend\n\n```bash\n# OceanBase should still be running from Step 4\n# Start RAGFlow with OceanBase profile (OceanBase is already running)\ndocker compose --profile oceanbase --profile cpu up -d\n\n# Wait for services to start\ndocker compose ps\n\n# Check logs for any errors\ndocker compose logs -f ragflow-cpu\n```\n\n### Step 8: Data Integrity Verification (Optional)\n\nRun the verification command to compare ES and OceanBase data:\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --sample-size 100\n```\n\nExpected output:\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_{tenant_id} │\n│ OB Table: ragflow_{tenant_id} │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n### Step 9: Verify RAGFlow Works with OceanBase\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Navigate to your Knowledge Base\n3. Try the same queries you tested before migration\n\n## CLI Reference\n\n### `es-ob-migrate migrate`\n\nRun data migration from Elasticsearch to OceanBase.\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--es-host` | localhost | Elasticsearch host |\n| `--es-port` | 9200 | Elasticsearch port |\n| `--es-user` | None | ES username (if auth required) |\n| `--es-password` | None | ES password |\n| `--ob-host` | localhost | OceanBase host |\n| `--ob-port` | 2881 | OceanBase port |\n| `--ob-user` | root@test | OceanBase user (format: user@tenant) |\n| `--ob-password` | \"\" | OceanBase password |\n| `--ob-database` | test | OceanBase database name |\n| `-i, --index` | None | Source ES index (omit to migrate all ragflow_* indices) |\n| `-t, --table` | None | Target OB table (omit to use same name as index) |\n| `--batch-size` | 1000 | Documents per batch |\n| `--resume` | False | Resume from previous progress |\n| `--verify/--no-verify` | True | Verify after migration |\n\n**Example:**\n\n```bash\n# Migrate all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc\n\n# Migrate a specific index\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123\n\n# Resume interrupted migration\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --resume\n```\n\n**Resume Feature:**\n\nMigration progress is automatically saved to `.migration_progress/` directory. If migration is interrupted (network error, timeout, etc.), use `--resume` to continue from where it stopped:\n\n- Progress file: `.migration_progress/{index_name}_progress.json`\n- Contains: total count, migrated count, last document ID, timestamp\n- On resume: skips already migrated documents, continues from last position\n\n**Output:**\n\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_abc123\nTarget: localhost:2881/ragflow_doc.ragflow_abc123\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_abc123' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nMigration completed successfully!\n Total: 1,234 documents\n Duration: 45.2 seconds\n```\n\n---\n\n### `es-ob-migrate list-indices`\n\nList all RAGFlow indices (`ragflow_*`) in Elasticsearch.\n\n**Example:**\n\n```bash\nes-ob-migrate list-indices --es-host localhost --es-port 9200\n```\n\n**Output:**\n\n```\nRAGFlow Indices in Elasticsearch:\n\n Index Name Documents Type\n ragflow_abc123def456789 1234 Document Chunks\n ragflow_doc_meta_abc123def456789 56 Document Metadata\n\nTotal: 2 ragflow_* indices found\n```\n\n---\n\n### `es-ob-migrate schema`\n\nPreview schema analysis from ES mapping.\n\n**Example:**\n\n```bash\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nRAGFlow Schema Analysis for index: ragflow_abc123\n\nVector Fields:\n q_1024_vec: dense_vector (dim=1024)\n\nKnown RAGFlow Fields (25):\n id, kb_id, doc_id, docnm_kwd, content_with_weight, content_ltks,\n available_int, important_kwd, question_kwd, tag_kwd, page_num_int...\n\nUnknown Fields (stored in 'extra' column):\n custom_field_1, custom_field_2\n```\n\n---\n\n### `es-ob-migrate verify`\n\nVerify migration data consistency between ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --sample-size 100\n```\n\n**Output:**\n\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_abc123 │\n│ OB Table: ragflow_abc123 │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n---\n\n### `es-ob-migrate list-kb`\n\nList all knowledge bases in an ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nKnowledge Bases in index 'ragflow_abc123':\n\n KB ID Documents\n kb_001_finance_docs 456\n kb_002_technical_manual 321\n kb_003_product_faq 457\n\nTotal: 3 knowledge bases, 1234 documents\n```\n\n---\n\n### `es-ob-migrate sample`\n\nShow sample documents from ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_abc123 --size 2\n```\n\n**Output:**\n\n```\nSample Documents from 'ragflow_abc123':\n\nDocument 1:\n id: chunk_001_abc123\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: The company reported Q3 revenue of $1.2B...\n available_int: 1\n\nDocument 2:\n id: chunk_002_def456\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: Operating expenses decreased by 5%...\n available_int: 1\n```\n\n---\n\n### `es-ob-migrate status`\n\nCheck connection status to ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate status \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\"\n```\n\n**Output:**\n\n```\nConnection Status:\n\nElasticsearch:\n Host: localhost:9200\n Status: ✓ Connected\n Cluster: ragflow-cluster\n Version: 8.11.0\n Indices: 5\n\nOceanBase:\n Host: localhost:2881\n Status: ✓ Connected\n Version: 4.3.5.1\n```\n","tools/firecrawl/INSTALLATION.md":"# Installation Guide for Firecrawl RAGFlow Integration\n\nThis guide will help you install and configure the Firecrawl integration plugin for RAGFlow.\n\n## Prerequisites\n\n- RAGFlow instance running (version 0.20.5 or later)\n- Python 3.8 or higher\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n## Installation Methods\n\n### Method 1: Manual Installation\n\n1. **Download the plugin**:\n ```bash\n git clone https://github.com/firecrawl/firecrawl.git\n cd firecrawl/ragflow-firecrawl-integration\n ```\n\n2. **Install dependencies**:\n ```bash\n pip install -r plugin/firecrawl/requirements.txt\n ```\n\n3. **Copy plugin to RAGFlow**:\n ```bash\n # Assuming RAGFlow is installed in /opt/ragflow\n cp -r plugin/firecrawl /opt/ragflow/plugin/\n ```\n\n4. **Restart RAGFlow**:\n ```bash\n # Restart RAGFlow services\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n### Method 2: Using pip (if available)\n\n```bash\npip install ragflow-firecrawl-integration\n```\n\n### Method 3: Development Installation\n\n1. **Clone the repository**:\n ```bash\n git clone https://github.com/firecrawl/firecrawl.git\n cd firecrawl/ragflow-firecrawl-integration\n ```\n\n2. **Install in development mode**:\n ```bash\n pip install -e .\n ```\n\n## Configuration\n\n### 1. Get Firecrawl API Key\n\n1. Visit [firecrawl.dev](https://firecrawl.dev)\n2. Sign up for a free account\n3. Navigate to your dashboard\n4. Copy your API key (starts with `fc-`)\n\n### 2. Configure in RAGFlow\n\n1. **Access RAGFlow UI**:\n - Open your browser and go to your RAGFlow instance\n - Log in with your credentials\n\n2. **Add Firecrawl Data Source**:\n - Go to \"Data Sources\" → \"Add New Source\"\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify your setup\n - You should see a success message\n\n## Configuration Options\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## Environment Variables\n\nYou can also configure the plugin using environment variables:\n\n```bash\nexport FIRECRAWL_API_KEY=\"fc-your-api-key-here\"\nexport FIRECRAWL_API_URL=\"https://api.firecrawl.dev\"\nexport FIRECRAWL_MAX_RETRIES=\"3\"\nexport FIRECRAWL_TIMEOUT=\"30\"\nexport FIRECRAWL_RATE_LIMIT_DELAY=\"1.0\"\n```\n\n## Verification\n\n### 1. Check Plugin Installation\n\n```bash\n# Check if the plugin directory exists\nls -la /opt/ragflow/plugin/firecrawl/\n\n# Should show:\n# __init__.py\n# firecrawl_connector.py\n# firecrawl_config.py\n# firecrawl_processor.py\n# firecrawl_ui.py\n# ragflow_integration.py\n# requirements.txt\n```\n\n### 2. Test the Integration\n\n```bash\n# Run the example script\ncd /opt/ragflow/plugin/firecrawl/\npython example_usage.py\n```\n\n### 3. Check RAGFlow Logs\n\n```bash\n# Check RAGFlow server logs\ndocker logs docker-ragflow-cpu-1\n\n# Look for messages like:\n# \"Firecrawl plugin loaded successfully\"\n# \"Firecrawl data source registered\"\n```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Plugin not appearing in RAGFlow**:\n - Check if the plugin directory is in the correct location\n - Restart RAGFlow services\n - Check RAGFlow logs for errors\n\n2. **API Key Invalid**:\n - Ensure your API key starts with `fc-`\n - Verify the key is active in your Firecrawl dashboard\n - Check for typos in the configuration\n\n3. **Connection Timeout**:\n - Increase the timeout value in configuration\n - Check your network connection\n - Verify the API URL is correct\n\n4. **Rate Limiting**:\n - Increase the `rate_limit_delay` value\n - Reduce the number of concurrent requests\n - Check your Firecrawl usage limits\n\n### Debug Mode\n\nEnable debug logging to see detailed information:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.DEBUG)\n```\n\n### Check Dependencies\n\n```bash\n# Verify all dependencies are installed\npip list | grep -E \"(aiohttp|pydantic|requests)\"\n\n# Should show:\n# aiohttp>=3.8.0\n# pydantic>=2.0.0\n# requests>=2.28.0\n```\n\n## Uninstallation\n\nTo remove the plugin:\n\n1. **Remove plugin directory**:\n ```bash\n rm -rf /opt/ragflow/plugin/firecrawl/\n ```\n\n2. **Restart RAGFlow**:\n ```bash\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n3. **Remove dependencies** (optional):\n ```bash\n pip uninstall ragflow-firecrawl-integration\n ```\n\n## Support\n\nIf you encounter issues:\n\n1. Check the [troubleshooting section](#troubleshooting)\n2. Review RAGFlow logs for error messages\n3. Verify your Firecrawl API key and configuration\n4. Check the [Firecrawl documentation](https://docs.firecrawl.dev)\n5. Open an issue in the [Firecrawl repository](https://github.com/firecrawl/firecrawl/issues)\n\n## Next Steps\n\nAfter successful installation:\n\n1. Read the [README.md](README.md) for usage examples\n2. Try scraping a simple URL to test the integration\n3. Explore the different scraping options (single URL, crawl, batch)\n4. Configure your RAGFlow workflows to use the scraped content\n","docs/references/python_api_reference.md":"---\nsidebar_position: 5\nslug: /python_api_reference\nsidebar_custom_props: {\n categoryIcon: SiPython\n}\n---\n# Python API\n\nA complete reference for RAGFlow's Python APIs. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).\n\n:::tip NOTE\nRun the following command to download the Python SDK:\n\n```bash\npip install ragflow-sdk\n```\n\n:::\n\n---\n\n## ERROR CODES\n\nRAGFlow responses may contain both an HTTP status code and a business code in the JSON response body. These codes should be checked separately.\n\n### HTTP status codes\n\n| Code | Meaning |\n|------|---------|\n| 200 | The HTTP request was processed successfully. Check the response body `code` for the business result. |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Internal server error |\n\n### Response body codes\n\n| Code | Meaning |\n|------|---------|\n| 0 | Success |\n| 10 | Not effective |\n| 100 | Exception error |\n| 101 | Invalid request argument |\n| 102 | Invalid or missing data |\n| 103 | Operation error |\n| 105 | Connection error |\n| 106 | Operation still running |\n| 108 | Permission error |\n| 109 | Authentication error |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Server error |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\nCreates a model response for the given historical chat conversation via OpenAI's API.\n\n#### Parameters\n\n##### chat_id: `string`, *Required*\n\nExisting chat assistant ID. This value is part of the request path: `/api/v1/openai//chat/completions`.\n\n##### model: `string`, *Required*\n\nThe model used to generate the response. You may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n##### messages: `list[object]`, *Required*\n\nA list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n##### stream: `boolean`\n\nWhether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n#### Returns\n\n- Success: Response [message](https://platform.openai.com/docs/api-reference/chat/create) like OpenAI\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom openai import OpenAI\nimport json\n\nmodel = \"glm-4-flash@ZHIPU-AI\"\nclient = OpenAI(api_key=\"ragflow-api-key\", base_url=\"http://ragflow_address/api/v1/openai//chat\")\n\nstream = True\nreference = True\n\nrequest_kwargs = dict(\n model=model,\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Who are you?\"},\n {\"role\": \"assistant\", \"content\": \"I am an AI assistant named...\"},\n {\"role\": \"user\", \"content\": \"Can you tell me how to install neovim\"},\n ],\n extra_body={\n \"reference\": reference,\n \"reference_metadata\": {\n \"include\": True,\n \"fields\": [\"author\", \"year\", \"source\"],\n },\n },\n)\n\nif stream:\n completion = client.chat.completions.create(stream=True, **request_kwargs)\n for chunk in completion:\n print(chunk)\nelse:\n resp = client.chat.completions.with_raw_response.create(\n stream=False, **request_kwargs\n )\n print(\"status:\", resp.http_response.status_code)\n raw_text = resp.http_response.text\n print(\"raw:\", raw_text)\n\n data = json.loads(raw_text)\n print(\"assistant:\", data[\"choices\"][0][\"message\"].get(\"content\"))\n print(\"reference:\", data[\"choices\"][0][\"message\"].get(\"reference\"))\n```\n\nWhen `extra_body.reference` is `true`, the streamed final chunk may include `choices[0].delta.reference`, and the non-stream response may include `choices[0].message.reference`.\n\nWhen `extra_body.reference_metadata.include` is `true`, each reference chunk may include a `document_metadata` object in both streaming and non-streaming responses.\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n```python\nRAGFlow.create_dataset(\n name: str,\n parse_type: int = 1,\n parser_id: str | None = None,\n pipeline_id: str | None = None,\n embedding_model: Optional[str] = None,\n permission: str = \"me\",\n parser_config: Optional[DataSet.ParserConfig] = None,\n language: Optional[str] = None,\n) -> DataSet\n```\n\nCreates a dataset.\n\n`avatar`, `description`, `auto_metadata_config`, and `chunk_method` are not accepted by the RAGFlow 1.0 create endpoint. Set avatar and description through `DataSet.update()`, and metadata configuration through `DataSet.update_auto_metadata()`.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe unique name of the dataset to create. It must adhere to the following requirements:\n\n- Maximum 128 characters.\n- Case-insensitive.\n\n##### parse_type: `int`\n\nSelects the parser mode. Defaults to `1`.\n\n- `1`: BuiltIn. Use the builtin parser selected by `parser_id`.\n- `2`: Pipeline. Use the ingestion pipeline selected by `pipeline_id`.\n\n##### parser_id: `string | None`\n\nThe builtin parser ID. Defaults to `\"general\"` when `parse_type` is `1`.\n\nAvailable values include `\"general\"`, `\"qa\"`, `\"manual\"`, `\"table\"`, `\"paper\"`, `\"book\"`, `\"laws\"`, `\"presentation\"`, `\"picture\"`, `\"one\"`, `\"audio\"`, and `\"email\"`.\n\nMust be omitted when `parse_type` is `2`.\n\n##### pipeline_id: `string | None`\n\nThe ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n\n##### embedding_model: `string | None`\n\nThe embedding model to use for the dataset. Defaults to `None`.\n\n##### permission\n\nSpecifies who can access the dataset to create. Available options:\n\n- `\"me\"`: (Default) Only you can manage the dataset.\n- `\"team\"`: All team members can manage the dataset.\n\n##### language: `string | None`\n\nOptional document/dataset language, for example `\"English\"` or `\"Chinese\"`. Defaults to `None`.\n\n##### parser_config: `DataSet.ParserConfig | None`\n\nThe parser configuration of the dataset. Defaults to `None`.\n\nRAGFlow 1.0 stores parser settings per ingestion-pipeline component. Use component-scoped keys (keys containing `:`) as returned by the API; legacy flat keys are ignored.\n\n#### Returns\n\n- Success: A `dataset` object. Its `parser_id`, `pipeline_id`, and derived `parse_type` reflect the selected parser mode.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"kb_1\")\ndataset = rag_object.create_dataset(name=\"kb_table\", parse_type=1, parser_id=\"table\")\ndataset = rag_object.create_dataset(\n name=\"kb_pipeline\",\n parse_type=2,\n pipeline_id=\"\",\n)\n```\n\n---\n\n### Delete datasets\n\n```python\nRAGFlow.delete_datasets(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes datasets by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the datasets to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no datasets are deleted.\n- If an array of IDs is provided, only the datasets matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all datasets owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object.delete_datasets(ids=[\"d94a8dc02c9711f0930f7fbc369eab6d\",\"e94a8dc02c9711f0930f7fbc369eab6e\"])\nrag_object.delete_datasets(delete_all=True)\n```\n\n---\n\n### List datasets\n\n```python\nRAGFlow.list_datasets(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n) -> list[DataSet]\n```\n\nLists datasets.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the datasets will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of datasets on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which datasets should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved datasets should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the dataset to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the dataset to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the datasets to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n#### Returns\n\n- Success: A list of `DataSet` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n##### List all datasets\n\n```python\nfor dataset in rag_object.list_datasets():\n print(dataset)\n```\n\n##### Retrieve a dataset by ID\n\n```python\ndataset = rag_object.list_datasets(id = \"id_1\")\nprint(dataset[0])\n```\n\n##### Retrieve datasets by IDs\n\n```python\ndatasets = rag_object.list_datasets(ids=[\"id_1\", \"id_2\"])\nfor dataset in datasets:\n print(dataset)\n```\n\n---\n\n### Update dataset\n\n```python\nDataSet.update(update_message: dict)\n```\n\nUpdates configurations for the current dataset.\n\n#### Parameters\n\n##### update_message: `dict[str, str|int]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"name\"`: `string` The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"description\"`: (*Body parameter*), `string`\n The updated dataset description.\n - Maximum 65535 characters\n- `\"language\"`: (*Body parameter*), `string`\n The updated document/dataset language, for example `\"English\"` or `\"Chinese\"`.\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration)\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser mode: `1` for a builtin parser or `2` for an ingestion pipeline. Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`.\n\n`chunk_method` is not supported by the RAGFlow 1.0 SDK; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_name\")\ndataset = dataset[0]\ndataset.update({\"embedding_model\":\"BAAI/bge-zh-v1.5\", \"parse_type\":1, \"parser_id\":\"manual\"})\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n```python\nDataSet.upload_documents(document_list: list[dict])\n```\n\nUploads documents to the current dataset.\n\n#### Parameters\n\n##### document_list: `list[dict]`, *Required*\n\nA list of dictionaries representing the documents to upload, each containing the following keys:\n\n- `\"display_name\"`: `string`, *Required*\n The file name to display in the dataset.\n- `\"blob\"`: `bytes`, *Required*\n The binary content of the file to upload.\n\n#### Returns\n\n- Success: A list of uploaded `Document` objects.\n- Failure: `Exception`\n\n#### Examples\n\n```python\ndataset = rag_object.create_dataset(name=\"kb_name\")\n\nwith open(\"1.txt\", \"rb\") as file:\n documents = dataset.upload_documents([\n {\n \"display_name\": \"1.txt\",\n \"blob\": file.read(),\n }\n ])\n\nprint(documents[0].id)\n```\n\n---\n\n### Update document\n\n```python\nDocument.update(update_message: dict)\n```\n\nUpdates configurations for the current document.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"display_name\"`: `string` The name of the document to update.\n- `\"meta_fields\"`: `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: `string` The parsing method to apply to the document.\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: `dict[str, Any]` The parsing configuration for the document. Its attributes vary based on the selected `\"chunk_method\"`:\n - `\"chunk_method\"`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False},\"parent_child\":{\"use_parent_child\":False,\"children_delimiter\":\"\\n\"}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Returns\n\n- Success: The updated `Document` object.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id='id')\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\ndoc.update({\n \"parser_config\": {\n \"chunk_token_num\": 256,\n },\n \"chunk_method\": \"manual\",\n})\n```\n\n---\n\n### Download document\n\n```python\nDocument.download() -> bytes\n```\n\nDownloads the current document.\n\n#### Returns\n\nThe downloaded document in bytes.\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"https://\",\n)\ndataset = rag_object.list_datasets(id=\"id\")[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")[0]\n\nPath(\"~/ragflow.txt\").expanduser().write_bytes(doc.download())\n```\n\n---\n\n### List documents\n\n```python\nDataSet.list_documents(\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n keywords: str | None = None,\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n create_time_from: int = 0,\n create_time_to: int = 0\n)\n```\n\nLists documents in the current dataset.\n\n#### Parameters\n\n##### id: `string | None`\n\nThe ID of the document to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the documents to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n##### name: `string | None`\n\nThe exact name of the document to retrieve. Defaults to `None`.\n\n##### keywords: `string | None`\n\nThe keywords used to match document titles. Defaults to `None`.\n\n##### page: `int`\n\nSpecifies the page on which the documents will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of documents on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which documents should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved documents should be sorted in descending order. Defaults to `True`.\n\n##### create_time_from: `int`\nUnix timestamp for filtering documents created after this time. 0 means no filter. Defaults to 0.\n\n##### create_time_to: `int`\nUnix timestamp for filtering documents created before this time. 0 means no filter. Defaults to 0.\n\n#### Returns\n\n- Success: A list of `Document` objects.\n- Failure: `Exception`.\n\nA `Document` object contains the following attributes:\n\n- `id`: The document ID. Defaults to `\"\"`.\n- `name`: The document name. Defaults to `\"\"`.\n- `thumbnail`: The thumbnail image of the document. Defaults to `None`.\n- `dataset_id`: The dataset ID associated with the document. Defaults to `None`.\n- `chunk_method` The chunking method name. Defaults to `\"naive\"`.\n- `source_type`: The source type of the document. Defaults to `\"local\"`.\n- `type`: Type or category of the document. Defaults to `\"\"`. Reserved for future use.\n- `created_by`: `string` The creator of the document. Defaults to `\"\"`.\n- `size`: `int` The document size in bytes. Defaults to `0`.\n- `token_count`: `int` The number of tokens in the document. Defaults to `0`.\n- `chunk_count`: `int` The number of chunks in the document. Defaults to `0`.\n- `progress`: `float` The current processing progress as a percentage. Defaults to `0.0`.\n- `progress_msg`: `string` A message indicating the current progress status. Defaults to `\"\"`.\n- `process_begin_at`: `datetime` The start time of document processing. Defaults to `None`.\n- `process_duration`: `float` Duration of the processing in seconds. Defaults to `0.0`.\n- `run`: `string` The document's processing status:\n - `\"UNSTART\"` (default)\n - `\"RUNNING\"`\n - `\"CANCEL\"`\n - `\"DONE\"`\n - `\"FAIL\"`\n- `status`: `string` Reserved for future use.\n- `parser_config`: `ParserConfig` Configuration object for the parser. Its attributes vary based on the selected `chunk_method`:\n - `chunk_method`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"http://:9380\",\n)\ndataset = rag_object.create_dataset(name=\"kb_1\")\n\nfile_path = Path(\"~/ragflow.txt\").expanduser()\ndataset.upload_documents([\n {\n \"display_name\": file_path.name,\n \"blob\": file_path.read_bytes(),\n }\n])\n\nfor doc in dataset.list_documents(\n keywords=\"rag\",\n page=1,\n page_size=12,\n):\n print(doc)\n```\n\n---\n\n### Delete documents\n\n```python\nDataSet.delete_documents(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes documents by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the documents to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no documents are deleted.\n- If an array of IDs is provided, only the documents matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all documents in the current dataset when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_1\")\ndataset = dataset[0]\ndataset.delete_documents(ids=[\"id_1\",\"id_2\"])\ndataset.delete_documents(delete_all=True)\n```\n\n---\n\n### Parse documents\n\n```python\nDataSet.async_parse_documents(document_ids:list[str]) -> None\n```\n\nParses documents in the current dataset.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\n```\n\n---\n\n### Parse documents (with document status)\n\n```python\nDataSet.parse_documents(document_ids: list[str]) -> list[tuple[str, str, int, int]]\n```\n\nStarts parsing documents in the current dataset and synchronously waits for the results.\n\nThis method calls `async_parse_documents()` and blocks while polling until all requested documents reach a terminal state or report complete progress. It then returns the parsing status and statistics for each document. If a keyboard interruption occurs (e.g., `Ctrl+C`), it requests cancellation for the requested documents and continues polling for their final statuses. If a status request fails or a requested document is no longer found, the method raises an exception instead of continuing to poll.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\nA list of tuples with detailed parsing results:\n\n```python\n[\n (document_id: str, status: str, chunk_count: int, token_count: int),\n ...\n]\n```\n- `status`: The final parsing state (e.g., `DONE`, `FAIL`, `CANCEL`). If a document has not reached a terminal state but reports `progress >= 1.0`, its status is returned as `DONE`.\n- `chunk_count`: The number of content chunks created from the document.\n- `token_count`: The total number of tokens processed.\n\n---\n\n#### Example\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = dataset.list_documents(keywords=\"test\")\nids = [doc.id for doc in documents]\n\ntry:\n finished = dataset.parse_documents(ids)\n for doc_id, status, chunk_count, token_count in finished:\n print(f\"Document {doc_id} parsing finished with status: {status}, chunks: {chunk_count}, tokens: {token_count}\")\nexcept Exception as e:\n print(f\"Parsing failed: {e}\")\n```\n\n---\n\n### Stop parsing documents\n\n```python\nDataSet.async_cancel_parse_documents(document_ids:list[str])-> None\n```\n\nStops parsing specified documents.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents for which parsing should be stopped.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\ndataset.async_cancel_parse_documents(ids)\nprint(\"Async bulk parsing cancelled.\")\n```\n\n---\n\n## CHUNK MANAGEMENT WITHIN DATASET\n\n---\n\n### Add chunk\n\n```python\nDocument.add_chunk(content:str, important_keywords:list[str] = [], questions:list[str] = [], image_base64:str = None, *, tag_kwd:list[str] = []) -> Chunk\n```\n\nAdds a chunk to the current document.\n\n#### Parameters\n\n##### content: `string`, *Required*\n\nThe text content of the chunk.\n\n##### important_keywords: `list[str]`\n\nThe key terms or phrases to tag with the chunk.\n\n##### questions: `list[str]`\n\nOptional questions to use when embedding the chunk.\n\n##### image_base64: `string`\n\nA base64-encoded image to associate with the chunk. If the chunk already has an image, the new image will be vertically concatenated below the existing one.\n\n##### tag_kwd: `list[str]`\n\nTag keywords to associate with the chunk.\n\n#### Returns\n\n- Success: A `Chunk` object.\n- Failure: `Exception`.\n\nA `Chunk` object contains the following attributes:\n\n- `id`: `string`: The chunk ID.\n- `content`: `string` The text content of the chunk.\n- `important_keywords`: `list[str]` A list of key terms or phrases tagged with the chunk.\n- `tag_kwd`: `list[str]` A list of tag keywords associated with the chunk.\n- `questions`: `list[str]` A list of questions associated with the chunk.\n- `image_id`: `string` The image ID associated with the chunk (empty string if no image).\n- `create_time`: `string` The time when the chunk was created (added to the document).\n- `create_timestamp`: `float` The timestamp representing the creation time of the chunk, expressed in seconds since January 1, 1970.\n- `dataset_id`: `string` The ID of the associated dataset.\n- `document_name`: `string` The name of the associated document.\n- `document_id`: `string` The ID of the associated document.\n- `available`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(id=\"123\")\ndataset = datasets[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\n```\n\nAdding a chunk with an image:\n\n```python\nimport base64\n\nwith open(\"image.jpg\", \"rb\") as f:\n img_b64 = base64.b64encode(f.read()).decode()\nchunk = doc.add_chunk(content=\"description of image\", image_base64=img_b64)\n```\n\n---\n\n### List chunks\n\n```python\nDocument.list_chunks(keywords: str = None, page: int = 1, page_size: int = 30, id : str = None) -> list[Chunk]\n```\n\nLists chunks in the current document.\n\n#### Parameters\n\n##### keywords: `string`\n\nThe keywords used to match chunk content. Defaults to `None`\n\n##### page: `int`\n\nSpecifies the page on which the chunks will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks on each page. Defaults to `30`.\n\n##### id: `string`\n\nThe ID of the chunk to retrieve. Default: `None`\n\n#### Returns\n\n- Success: A list of `Chunk` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(\"123\")\ndataset = dataset[0]\ndocs = dataset.list_documents(keywords=\"test\", page=1, page_size=12)\nfor chunk in docs[0].list_chunks(keywords=\"rag\", page=0, page_size=12):\n print(chunk)\n```\n\n---\n\n### Delete chunks\n\n```python\nDocument.delete_chunks(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chunks by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chunks to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chunks are deleted.\n- If an array of IDs is provided, only the chunks matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chunks in the current document when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\ndoc.delete_chunks([\"id_1\",\"id_2\"])\ndoc.delete_chunks(delete_all=True)\n```\n\n---\n\n### Update chunk\n\n```python\nChunk.update(update_message: dict)\n```\n\nUpdates content or configurations for the current chunk.\n\n#### Parameters\n\n##### update_message: `dict[str, str|list[str]|bool]` *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"content\"`: `string` The text content of the chunk.\n- `\"important_keywords\"`: `list[str]` A list of key terms or phrases to tag with the chunk.\n- `\"questions\"`: `list[str]` A list of questions associated with the chunk.\n- `\"tag_kwd\"`: `list[str]` A list of tag keywords to associate with the chunk.\n- `\"positions\"`: `list` Updated source positions for the chunk.\n- `\"available\"`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n- `\"image_base64\"`: `string` Base64-encoded image content to associate with the chunk.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\nchunk.update({\"content\":\"sdfx...\"})\n```\n\n---\n\n### Retrieve chunks\n\n```python\nRAGFlow.retrieve(\n dataset_ids,\n document_ids=None,\n question='',\n page=1,\n page_size=30,\n similarity_threshold=0.2,\n vector_similarity_weight=0.3,\n top_k=1024,\n rerank_id: str | None = None,\n keyword: bool = False,\n cross_languages: list[str] | None = None,\n metadata_condition: dict | None = None)\n```\n\nRetrieves chunks from specified datasets.\n\n#### Parameters\n\n##### question: `string`\n\nThe user query or query keywords. Defaults to `\"\"`. When an empty string is provided, the API returns an empty retrieval result.\n\n##### dataset_ids: `list[str]`, *Required*\n\nThe IDs of the datasets to search. At least one dataset ID must be provided.\n\n##### document_ids: `list[str]`\n\nThe IDs of the documents to search. Defaults to `None`. You must ensure all selected documents use the same embedding model. Otherwise, an error will occur.\n\n##### page: `int`\n\nThe page number of the chunk retrieval results. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks returned on each page. Defaults to `30`.\n\n##### similarity_threshold: `float`\n\nThe minimum similarity score. Defaults to `0.2`.\n\n##### vector_similarity_weight: `float`\n\nThe weight of vector cosine similarity. Defaults to `0.3`. If x represents the vector cosine similarity, then (1 - x) is the term similarity weight.\n\n##### top_k: `int`\n\nThe number of chunks engaged in vector cosine computation. Defaults to `1024`.\n\n##### rerank_id: `string`\n\nThe ID of the rerank model. Defaults to `None`.\n\n##### keyword: `bool`\n\nIndicates whether to enable keyword-based matching:\n\n- `True`: Enable keyword-based matching.\n- `False`: Disable keyword-based matching (default).\n\n##### cross_languages: `list[string]`\n\nThe languages that should be translated into, in order to achieve keywords retrievals in different languages.\n\n##### metadata_condition: `dict`\n\nfilter condition for `meta_fields`.\n\n#### Returns\n\n- Success: A list of `Chunk` objects representing the document chunks.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"ragflow\")\ndataset = dataset[0]\nname = 'ragflow_test.txt'\npath = './test_data/ragflow_test.txt'\ndocuments =[{\"display_name\":\"test_retrieve_chunks.txt\",\"blob\":open(path, \"rb\").read()}]\ndocs = dataset.upload_documents(documents)\ndoc = docs[0]\ndoc.add_chunk(content=\"This is a chunk addition test\")\nfor c in rag_object.retrieve(dataset_ids=[dataset.id],document_ids=[doc.id]):\n print(c)\n```\n\n---\n\n## CHAT ASSISTANT MANAGEMENT\n\n---\n\n### Create chat assistant\n\n```python\nRAGFlow.create_chat(\n name: str,\n icon: str = \"\",\n dataset_ids: list[str] | None = None,\n llm_id: str | None = None,\n llm_setting: dict | None = None,\n prompt_config: dict | None = None,\n **kwargs\n) -> Chat\n```\n\nCreates a chat assistant.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe name of the chat assistant.\n\n##### icon: `string`\n\nBase64 encoding of the avatar. Defaults to `\"\"`.\n\n##### dataset_ids: `list[str]`\n\nThe IDs of the associated datasets. Defaults to `[]`. When omitted or empty, the SDK creates an empty chat assistant, and you can attach datasets later.\n\n##### llm_id: `str | None`\n\nThe LLM model name/ID to use. If `None`, the user’s default chat model is used. Defaults to `None`.\n\n##### llm_setting: `dict | None`\n\nConfiguration for LLM generation parameters. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"temperature\"`: `float` Controls the randomness of the model's output. Higher values increase creativity, while lower values make responses more deterministic. Defaults to `0.1`.\n- `\"top_p\"`: `float` Sets the nucleus sampling threshold. The model considers only the results of the tokens with `top_p` probability mass. Defaults to `0.3`.\n- `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have appeared in the text so far, increasing the likelihood of the model talking about new topics. Defaults to `0.4`.\n- `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text, decreasing the likelihood of repeating the same lines. Defaults to `0.7`.\n- `\"max_token\"`: `int` The maximum number of tokens to generate in the response. Defaults to `512`.\n\n##### prompt_config: `dict | None`\n\nInstructions and behavioral settings for the LLM. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n- `\"empty_response\"`: `string` The specific message returned when no relevant information is retrieved. If left blank, the LLM will generate its own response. Defaults to `None`.\n- `\"prologue\"`: `string` The initial greeting displayed to the user. Defaults to `\"Hi! I’m your assistant. What can I do for you?\"`.\n- `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references in its responses. Defaults to `True`.\n- `\"parameters\"`: `list[dict]` A list of variables utilized within the system prompt. Each entry must include a `\"key\"` (`string`) and an `\"optional\"` (`boolean`) status. The `knowledge` key is reserved for retrieved context chunks. Default: `[{\"key\": \"knowledge\", \"optional\": true}]`.\n\n#### Returns\n\n- Success: A `Chat` object representing the chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_ids = []\nfor dataset in datasets:\n dataset_ids.append(dataset.id)\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=dataset_ids)\n```\n\n---\n\n### Update chat assistant\n\n```python\nChat.update(update_message: dict)\n```\n\nPerforms a partial update to the configuration settings for the current chat assistant.\n\n`Chat.update()` utilizes the `PATCH /api/v1/chats/{chat_id}` endpoint. Only the specified keys are modified, while all other existing fields are preserved.\n\n#### Parameters\n\n##### update_message: `dict`, *Required*\n\nA dictionary containing the attributes to be updated. Supported keys include:\n\n- `\"name\"`: `string` The updated name of the chat assistant.\n- `\"icon\"`: `string` A Base64-encoded string representing the assistant's avatar.\n- `\"dataset_ids\"`: `list[string]` A list of unique identifiers for the datasets associated with the assistant.\n- `\"llm_id\"`: `string` The unique identifier or name of the LLM to be used.\n- `\"llm_setting\"`: `dict` Configuration for LLM generation parameters:\n - `\"temperature\"`: `float` Controls the randomness of the model's output.\n - `\"top_p\"`: `float` Sets the nucleus sampling threshold.\n - `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have already appeared in the text.\n - `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text.\n - `\"max_token\"`: `int` The maximum number of tokens to generate in the response.\n- `\"prompt_config\"`: `dict` Instructions and behavioral settings for the LLM:\n - `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n - `\"empty_response\"`: `string` The message returned when no relevant information is retrieved. Leave blank to allow the LLM to improvise.\n - `\"prologue\"`: `string` The initial greeting displayed to the user.\n - `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references.\n - `\"parameters\"`: `list[dict]` Variables used within the system prompt (e.g., the reserved `knowledge` key).\n- `\"similarity_threshold\"`: `float` The minimum similarity score required for retrieved context chunks. Defaults to `0.2`.\n- `\"vector_similarity_weight\"`: `float` The weight assigned to vector cosine similarity within the hybrid search score. Defaults to `0.3`.\n- `\"top_n\"`: `int` The number of top-ranked chunks provided to the LLM as context. Defaults to `6`.\n- `\"top_k\"`: `int` The size of the initial candidate pool retrieved for reranking. Defaults to `1024`.\n- `\"rerank_id\"`: `string` The unique identifier for the reranking model. If left empty, standard vector cosine similarity is used for ranking.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_id = datasets[0].id\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=[dataset_id])\nassistant.update({\"name\": \"Stefan\", \"llm_setting\": {\"temperature\": 0.8}, \"top_n\": 8})\n```\n\n---\n\n### Delete chat assistants\n\n```python\nRAGFlow.delete_chats(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chat assistants by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chat assistants to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chat assistants are deleted.\n- If an array of IDs is provided, only the chat assistants matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chat assistants owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nrag_object.delete_chats(ids=[\"id_1\",\"id_2\"])\nrag_object.delete_chats(delete_all=True)\n```\n\n---\n\n### List chat assistants\n\n```python\nRAGFlow.list_chats(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n name: str | None = None,\n keywords: str | None = None,\n owner_ids: str | list[str] | None = None,\n) -> list[Chat]\n```\n\nLists chat assistants.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the chat assistants will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of chat assistants on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe attribute by which the results are sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved chat assistants should be sorted in descending order. Defaults to `True`.\n\n##### id: `string | None`\n\nExact match on chat assistant ID. Defaults to `None`.\n\n##### name: `string | None`\n\nFilters results by the exact name of the chat assistant. Defaults to `None`.\n\n##### keywords: `string | None`\n\nPerforms a case-insensitive fuzzy search against chat assistant names. Defaults to `None`.\n\n##### owner_ids: `string | list[string] | None`\n\nFilters results by one or more owner tenant IDs. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Chat` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nfor assistant in rag_object.list_chats():\n print(assistant)\n```\n\n---\n\n## SESSION MANAGEMENT\n\n---\n\n### Create session with chat assistant\n\n```python\nChat.create_session(name: str = \"New session\") -> Session\n```\n\nCreates a session with the current chat assistant.\n\n#### Parameters\n\n##### name: `string`\n\nThe name of the chat session to create.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `name`: `string` The name of the created session.\n - `message`: `list[Message]` The opening message of the created session. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `chat_id`: `string` The ID of the associated chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session()\n```\n\n---\n\n### Update chat assistant's session\n\n```python\nSession.update(update_message: dict)\n```\n\nUpdates the current session of the current chat assistant.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with only one key:\n\n- `\"name\"`: `string` The revised name of the session.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session(\"session_name\")\nsession.update({\"name\": \"updated_name\"})\n```\n\n---\n\n### List chat assistant's sessions\n\n```python\nChat.list_sessions(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str = None,\n name: str = None,\n user_id: str = None\n) -> list[Session]\n```\n\nLists sessions associated with the current chat assistant.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the sessions will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of sessions on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which sessions should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved sessions should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the chat session to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the chat session to retrieve. Defaults to `None`.\n\n##### user_id: `str`\n\nThe optional user-defined ID to filter sessions by. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Session` objects associated with the current chat assistant.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nfor session in assistant.list_sessions():\n print(session)\n```\n\n---\n\n### Delete chat assistant's sessions\n\n```python\nChat.delete_sessions(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes sessions of the current chat assistant by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the sessions to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no sessions are deleted.\n- If an array of IDs is provided, only the sessions matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all sessions of the current chat assistant when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nassistant.delete_sessions(ids=[\"id_1\",\"id_2\"])\nassistant.delete_sessions(delete_all=True)\n```\n\n---\n\n### Create session with agent\n\n```python\nAgent.create_session(**kwargs) -> Session\n```\n\nCreates a session with the current agent.\n\n#### Parameters\n\n##### **kwargs\n\nThe parameters in `begin` component.\n\nAlso supports:\n\n- `release` (`bool | str`, optional): When set to `True` (or `\"true\"`), creates a session with the published agent app only.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `message`: `list[Message]` The messages of the created session assistant. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `agent_id`: `string` The ID of the associated agent.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nagent_id = \"AGENT_ID\"\nagent = rag_object.get_agent(agent_id)\nsession = agent.create_session()\n# Or create in release mode:\n# session = agent.create_session(release=True)\n```\n\n---\n\n### Converse with agent\n\n```python\nSession.ask(question: str = \"\", stream: bool = False, **kwargs) -> Optional[Message | iter[Message]]\n```\n\nAsks a specified agent through the unified completion endpoint.\n\n:::tip NOTE\nIn streaming mode, not all responses include a reference, as this depends on the system's judgment.\n:::\n\n#### Parameters\n\n##### question: `string`\n\nThe user message sent to the agent. If the **Begin** component takes parameters, `question` can be an empty string.\n\n##### stream: `bool`\n\nIndicates whether to output responses in a streaming way:\n\n- `True`: Enable streaming.\n- `False`: Disable streaming.\n\n##### kwargs: `dict`\n\nAdditional request parameters forwarded to the completion API. Common options:\n\n- `inputs`: Variables defined in the **Begin** component.\n- `session_id`: Continue an existing session instead of creating a new one.\n- `release`: Use the latest published version of the agent.\n- `return_trace`: Include execution trace information in the response.\n- Other custom Begin component parameters supported by the current workflow.\n\n#### Returns\n\n- A `Message` object containing the response to the question if `stream` is set to `False`\n- An iterator containing multiple `message` objects (`iter[Message]`) if `stream` is set to `True`\n\nThe following shows the attributes of a `Message` object:\n\n##### id: `string`\n\nThe auto-generated message ID.\n\n##### content: `string`\n\nThe content of the message. Defaults to `\"Hi! I am your assistant, can I help you?\"`.\n\n##### reference: `list[Chunk]`\n\nA list of `Chunk` objects representing references to the message, each containing the following attributes:\n\n- `id` `string`\n The chunk ID.\n- `content` `string`\n The content of the chunk.\n- `image_id` `string`\n The ID of the snapshot of the chunk. Applicable only when the source of the chunk is an image, PPT, PPTX, or PDF file.\n- `document_id` `string`\n The ID of the referenced document.\n- `document_name` `string`\n The name of the referenced document.\n- `document_metadata` `dict`\n Optional document metadata, returned only when `extra_body.reference_metadata.include` is `true`.\n- `position` `list[str]`\n The location information of the chunk within the referenced document.\n- `dataset_id` `string`\n The ID of the dataset to which the referenced document belongs.\n- `similarity` `float`\n A composite similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity. It is the weighted sum of `vector_similarity` and `term_similarity`.\n- `vector_similarity` `float`\n A vector similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between vector embeddings.\n- `term_similarity` `float`\n A keyword similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between keywords.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nAGENT_id = \"AGENT_ID\"\nagent = rag_object.get_agent(AGENT_id)\nsession = agent.create_session()\n\nprint(\"\\n===== Miss R ====\\n\")\nprint(\"Hello. What can I do for you?\")\n\nwhile True:\n question = input(\"\\n===== User ====\\n> \")\n print(\"\\n==== Miss R ====\\n\")\n\n cont = \"\"\n for ans in session.ask(question, stream=True):\n print(ans.content[len(cont):], end='', flush=True)\n cont = ans.content\n```\n\nUse Begin inputs and request trace output:\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://\"`. For the general parser, send `\"parser_id\": \"general\"`. |\n| Dataset parser configuration | Go stores and returns component-scoped parser settings. Most legacy top-level `parser_config` fields are not mapped into that component structure. | Read the returned component map and send component-scoped keys when changing parser settings. |\n| Errors and authentication | Some validation business codes, messages, and authentication HTTP statuses differ from Python. | Branch on documented codes rather than exact message text. |\n\n:::note\nGo responses include `X-API-Source: go`, which can be used to verify the serving backend during rollout.\n:::\n\n### Removed API aliases\n\nThe following paths remain available only on the Python backend. They are not registered by the RAGFlow 1.0 Go backend.\n\n| Removed endpoint | Replacement endpoint |\n|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|\n| **POST** `/api/v1/chats_openai/{chat_id}/chat/completions` | **POST** `/api/v1/openai/{chat_id}/chat/completions` |\n| **PUT** `/api/v1/chats/{chat_id}/sessions/{session_id}` | **PATCH** `/api/v1/chats/{chat_id}/sessions/{session_id}` |\n| **POST** `/api/v1/chats/{chat_id}/completions` | **POST** `/api/v1/chat/completions` |\n| **POST** `/api/v1/sessions/related_questions` | **POST** `/api/v1/chat/recommendation` |\n| **PUT** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` | **PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` |\n| **GET** `/v1/system/healthz` | **GET** `/api/v1/system/healthz` |\n| **POST** `/v1/document/upload_info` | **POST** `/api/v1/documents/upload` |\n| **POST** `/api/v1/file/upload` | **POST** `/api/v1/files` |\n| **POST** `/api/v1/file/create` | **POST** `/api/v1/files` |\n| **GET** `/api/v1/file/list` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/root_folder` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/parent_folder` | **GET** `/api/v1/files/{file_id}/parent` |\n| **GET** `/api/v1/file/all_parent_folder` | **GET** `/api/v1/files/{file_id}/ancestors` |\n| **POST** `/api/v1/file/rm` | **DELETE** `/api/v1/files` |\n| **POST** `/api/v1/file/rename` | **POST** `/api/v1/files/move` |\n| **GET** `/api/v1/file/get/{file_id}` | **GET** `/api/v1/files/{file_id}` |\n| **POST** `/api/v1/file/mv` | **POST** `/api/v1/files/move` |\n| **POST** `/api/v1/file/convert` | **POST** `/api/v1/files/link-to-datasets` |\n| **POST** `/api/v1/agents_openai/{agent_id}/chat/completions` | **POST** `/api/v1/agents/chat/completions` with `\"openai-compatible\": true` |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\n**POST** `/api/v1/openai/{chat_id}/chat/completions`\n\nCreates a model response for a given chat conversation.\n\n:::caution REMOVED ALIAS\n`POST /api/v1/chats_openai/{chat_id}/chat/completions` is not available in 1.0. Use this endpoint instead.\n:::\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/openai/{chat_id}/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"model\"`: `string` (optional)\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"extra_body\"`: `object` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/openai/{chat_id}/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"model\": \"glm-4-flash@ZHIPU-AI\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true,\n \"extra_body\": {\n \"reference\": true,\n \"reference_metadata\": {\n \"include\": true,\n \"fields\": [\"author\", \"year\", \"source\"]\n },\n \"metadata_condition\": {\n \"logic\": \"and\",\n \"conditions\": [\n {\n \"name\": \"author\",\n \"comparison_operator\": \"is\",\n \"value\": \"bob\"\n }\n ]\n }\n }\n }'\n```\n\n##### Request parameters\n\n- `chat_id` (*Path parameter*) `string`, *Required*\n Existing chat assistant ID. The request will use that chat assistant's knowledge and settings.\n\n- `model` (*Body parameter*) `string`\n The model used to generate the response. When `chat_id` is provided, you may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `extra_body` (*Body parameter*) `object`\n Extra request parameters:\n - `reference`: `boolean` - include reference in the final chunk (stream) or in the final message (non-stream).\n - `reference_metadata`: `object` - include document metadata in each reference chunk.\n - `include`: `boolean` - enable document metadata in reference chunks.\n - `fields`: `list[string]` - optional allowlist of metadata keys. Omit to include all. Use an empty list to include none.\n - `metadata_condition`: `object` - metadata filter conditions applied to retrieval results.\n\n#### Response\n\nStream:\n\n```json\ndata:{\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"Hello! It seems like you're just greeting me. If you have a specific\",\n \"role\": \"assistant\",\n \"function_call\": null,\n \"tool_calls\": null,\n \"reasoning_content\": null\n },\n \"finish_reason\": null,\n \"index\": 0,\n \"logprobs\": null\n }\n ],\n \"created\": 1755084508,\n \"model\": \"model\",\n \"object\": \"chat.completion.chunk\",\n \"system_fingerprint\": \"\",\n \"usage\": null\n}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" question or need information, feel free to ask, and I'll do my best\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" to assist you based on the knowledge base provided.\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": null, \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": \"stop\", \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": {\"prompt_tokens\": 5, \"completion_tokens\": 188, \"total_tokens\": 193}}\n\ndata:[DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"Hello! I'm your smart assistant. What can I do for you?\",\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": 1755084403,\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"model\": \"model\",\n \"object\": \"chat.completion\",\n \"usage\": {\n \"completion_tokens\": 55,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 55,\n \"reasoning_tokens\": 5,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 5,\n \"total_tokens\": 60\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n---\n\n### Create OpenAI-compatible agent completion\n\n:::caution REMOVED ALIAS\nRAGFlow 1.0 Go does not serve `POST /api/v1/agents_openai/{agent_id}/chat/completions`. Use the endpoint below and move `agent_id` into the request body.\n:::\n\n**POST** `/api/v1/agents/chat/completions`\n\nCreates a model response for a given chat conversation.\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/agents/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"agent_id\"`: `string`\n - `\"openai-compatible\"`: `boolean`\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"session_id\"`: `string` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/agents/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"agent_id\": \"{agent_id}\",\n \"openai-compatible\": true,\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true\n }'\n```\n\n##### Body parameters\n\n- `agent_id` (*Body parameter*) `string`, *Required*\n The ID of the agent to run.\n\n- `openai-compatible` (*Body parameter*) `boolean`, *Required*\n Set this field to `true` to use the OpenAI-compatible request and response format.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `session_id` (*Body parameter*) `string`\n Agent session id.\n\n#### Response\n\nStream:\n\n```json\n...\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \" terminal\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \".\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"id\": \"4b8935ac0a22deb1\",\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"url\": null,\n \"similarity\": 0.5697155305154673,\n \"vector_similarity\": 0.7323851005515574,\n \"term_similarity\": 0.5000000005,\n \"doc_type\": \"\"\n }\n },\n \"doc_aggs\": {\n \"INSTALL22.md\": {\n \"doc_name\": \"INSTALL22.md\",\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"count\": 3\n },\n \"INSTALL.md\": {\n \"doc_name\": \"INSTALL.md\",\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL(1).md\": {\n \"doc_name\": \"INSTALL(1).md\",\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL3.md\": {\n \"doc_name\": \"INSTALL3.md\",\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"count\": 1\n }\n }\n }\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: [DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"\\nTo install Neovim, the process varies depending on your operating system:\\n\\n### For Windows:\\n1. **Download from GitHub**: \\n - Visit the [Neovim releases page](https://github.com/neovim/neovim/releases)\\n - Download the latest Windows installer (nvim-win64.msi)\\n - Run the installer and follow the prompts\\n\\n2. **Using winget** (Windows Package Manager):\\n...\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"doc_type\": \"\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"id\": \"4b8935ac0a22deb1\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"similarity\": 0.5697155305154673,\n \"term_similarity\": 0.5000000005,\n \"url\": null,\n \"vector_similarity\": 0.7323851005515574\n }\n },\n \"doc_aggs\": {\n \"INSTALL(1).md\": {\n \"count\": 2,\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL(1).md\"\n },\n \"INSTALL.md\": {\n \"count\": 2,\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL.md\"\n },\n \"INSTALL22.md\": {\n \"count\": 3,\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL22.md\"\n },\n \"INSTALL3.md\": {\n \"count\": 1,\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL3.md\"\n }\n }\n },\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": null,\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"object\": \"chat.completion\",\n \"param\": null,\n \"usage\": {\n \"completion_tokens\": 415,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 0,\n \"reasoning_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 6,\n \"total_tokens\": 421\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n**POST** `/api/v1/datasets`\n\nCreates a dataset.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### A basic request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test_1\",\n \"language\": \"English\",\n \"parse_type\": 1,\n \"parser_id\": \"general\"\n }'\n```\n\n##### A request example specifying ingestion pipeline\n\n:::caution WARNING\nDo not include `\"parser_id\"` when specifying an ingestion pipeline.\n:::\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test-sdk\",\n \"language\": \"English\",\n \"parse_type\": 2,\n \"pipeline_id\": \"\"\n }'\n```\n\n##### Body parameters\n\n- `\"name\"`: `string`, *Required*\n The unique name of the dataset to create. It must adhere to the following requirements:\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n If the name is already taken, the dataset is created with a numbered suffix such as `test_1(1)`, `test_1(2)`, and so on.\n\n- `\"language\"`: `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the server/database default is used.\n\n- `\"embedding_model\"`: `string`\n The name of the embedding model to use. For example: `\"BAAI/bge-large-zh-v1.5@BAAI\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n\n- `\"permission\"`: `string`\n Specifies who can access the dataset to create. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n\n- `\"parser_id\"`: `enum`\n The built-in parser to use. Required when `\"parse_type\"` is `1`.\n - `\"general\"`: General (default)\n - `\"qa\"`: Q&A\n - `\"manual\"`: Manual\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"audio\"`: Audio\n - `\"email\"`: Email\n - Must be omitted when `\"parse_type\"` is `2`.\n\n- `\"parser_config\"`: `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n- `\"parse_type\"`: `int`, *Required*\n The parser selection mode:\n - `1`: BuiltIn. Requires `\"parser_id\"`.\n - `2`: Pipeline. Requires `\"pipeline_id\"`.\n\n- `\"pipeline_id\"`: `string`\n The ingestion pipeline ID. Can be found in the corresponding URL in the RAGFlow UI.\n - Required when `\"parse_type\"` is `2`.\n - Must be a 32-character lowercase hexadecimal string, e.g., `\"d0bebe30ae2211f0970942010a8e0005\"`.\n - Must be omitted when `\"parse_type\"` is `1`.\n\n:::caution WARNING\n`\"parse_type\"` is required when creating a dataset. Choose exactly one mode:\n\n- `\"parse_type\": 1` with `\"parser_id\"` for a builtin parser.\n- `\"parse_type\": 2` with `\"pipeline_id\"` for an ingestion pipeline.\n\n`\"chunk_method\"`, `\"avatar\"`, `\"description\"`, and `\"auto_metadata_config\"` are not accepted by the RAGFlow 1.0 create endpoint. Use `\"parser_id\"` instead of `\"chunk_method\"`; set avatar and description with the update endpoint, and metadata configuration with the metadata endpoint.\n:::\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"chunk_count\": 0,\n \"create_time\": 1745836841611,\n \"created_by\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"document_count\": 0,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5@BAAI\",\n \"id\": \"3b4de7d4241d11f0a6a79f24fc270c7f\",\n \"language\": \"English\",\n \"name\": \"RAGFlow example\",\n \"pagerank\": 0,\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 128,\n \"delimiters\": [\"\\n\", \"!?;。；！？\"],\n \"parent_child\": {\n \"use_parent_child\": false,\n \"children_delimiter\": \"\\n\"\n }\n }\n },\n \"parser_id\": \"general\",\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"tenant_id\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"token_num\": 0,\n \"keywords_similarity_weight\": 0.7\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"Field: - Message: - Value: <>\"\n}\n```\n\n---\n\n### Delete datasets\n\n**DELETE** `/api/v1/datasets`\n\nDeletes datasets by ID.\n\n#### Request\n\n- Method: DELETE\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"ids\"`: `list[string]` or `null`\n - `\"delete_all\"`: `boolean`\n\n##### Request example\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"ids\": [\"d94a8dc02c9711f0930f7fbc369eab6d\", \"e94a8dc02c9711f0930f7fbc369eab6e\"]\n }'\n```\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"delete_all\": true\n }'\n```\n\n##### Body parameters\n\n- `\"ids\"`: `list[string]` or `null`\n Specifies the datasets to delete:\n - If omitted, or set to `null` or an empty array, no datasets are deleted.\n - If an array of IDs is provided, only the datasets matching those IDs are deleted.\n- `\"delete_all\"`: `boolean`\n Whether to delete all datasets owned by the current user when `\"ids\"` is omitted, or set to `null` or an empty array. Defaults to `false`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\":108,\n \"message\":\"User '' lacks permission for datasets: ''\"\n}\n\n```\n\n---\n\n### Update dataset\n\n**PUT** `/api/v1/datasets/{dataset_id}`\n\nUpdates configurations for a specified dataset.\n\n#### Request\n\n- Method: PUT\n- URL: `/api/v1/datasets/{dataset_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"avatar\"`: `string`\n - `\"description\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"pagerank\"`: `int`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### Request example\n\n```bash\ncurl --request PUT \\\n --url http://{address}/api/v1/datasets/{dataset_id} \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '\n {\n \"name\": \"updated_dataset\"\n }'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to update.\n- `\"name\"`: (*Body parameter*), `string`\n The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"language\"`: (*Body parameter*), `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the existing dataset language remains unchanged.\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n Refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration).\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser selection mode: `1` for a builtin parser (`parser_id` required) or `2` for an ingestion pipeline (`pipeline_id` required). Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`; must be omitted when `parse_type` is `2`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n`chunk_method` is not supported by RAGFlow 1.0; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"Can't change tenant_id.\"\n}\n```\n\n---\n\n### List datasets\n\n**GET** `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n\nLists datasets.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n- Headers:\n - `'Authorization: Bearer '`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id} \\\n --header 'Authorization: Bearer '\n```\n\n```bash\n# List datasets with parsing status\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets?include_parsing_status=true' \\\n --header 'Authorization: Bearer '\n```\n\n##### Query parameters\n\n- `page`\n Specifies the page on which the datasets will be displayed. Defaults to `1`.\n- `page_size`\n The number of datasets on each page. Defaults to `30`.\n- `orderby`\n The field by which datasets should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`\n Indicates whether the retrieved datasets should be sorted in descending order. Defaults to `true`.\n- `sort`\n Orders by several fields at once, written as `column:direction` terms separated by commas, such as `name:asc,create_time:desc`. A term with no direction is ascending. `sort` takes precedence over `orderby` and `desc`, which keep working on their own. A term naming a field this list does not sort by is skipped. When `sort` yields no terms at all, `orderby` and `desc` decide the order. When it yields terms whose fields are all skipped, the result is ordered by `create_time` in the direction of the first term, and the older pair is not read.\n- `name`\n The name of the dataset to retrieve.\n- `id`\n The ID of the dataset to retrieve.\n- `include_parsing_status`\n Whether to include document parsing status counts in the response. Defaults to `false`. When set to `true`, each dataset object in the response will include the following additional fields:\n - `unstart_count`: Number of documents not yet started parsing.\n - `running_count`: Number of documents currently being parsed.\n - `cancel_count`: Number of documents whose parsing was canceled.\n - `done_count`: Number of documents that have been successfully parsed.\n - `fail_count`: Number of documents whose parsing failed.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": \"\",\n \"chunk_count\": 59,\n \"create_date\": \"Sat, 14 Sep 2024 01:12:37 GMT\",\n \"create_time\": 1726276357324,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"description\": null,\n \"document_count\": 1,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5\",\n \"id\": \"6e211ee0723611efa10a0242ac120007\",\n \"language\": \"English\",\n \"name\": \"mysql\",\n \"parser_id\": \"general\",\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 8192,\n \"delimiters\": [\"\\n\"]\n }\n },\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"69736c5e723611efb51b0242ac120007\",\n \"token_num\": 12744,\n \"update_date\": \"Thu, 10 Oct 2024 04:07:23 GMT\",\n \"update_time\": 1728533243536,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nSuccess (with `include_parsing_status=true`):\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": null,\n \"cancel_count\": 0,\n \"chunk_count\": 30,\n \"parser_id\": \"qa\",\n \"create_date\": \"2026-03-09T18:57:13\",\n \"create_time\": 1773053833094,\n \"created_by\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"description\": null,\n \"document_count\": 1,\n \"done_count\": 1,\n \"embedding_model\": \"text-embedding-v2@Tongyi-Qianwen\",\n \"fail_count\": 0,\n \"id\": \"ba6586c21ba611f1a3dc476f0709e75e\",\n \"language\": \"English\",\n \"name\": \"Test Dataset\",\n \"parser_config\": {\n \"Extractor:AutoExtractDefault\": {\n \"llm_id\": \"deepseek-chat@DeepSeek\"\n }\n },\n \"permission\": \"me\",\n \"running_count\": 0,\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"token_num\": 1746,\n \"unstart_count\": 0,\n \"update_date\": \"2026-03-09T18:59:32\",\n \"update_time\": 1773053972723,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset doesn't exist\"\n}\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n**POST** `/api/v1/datasets/{dataset_id}/documents`\n\nUploads documents to a specified dataset.\n\nThis endpoint supports three creation modes via the optional `type` query parameter:\n\n- `type=local` or omitted: Upload one or more local files using `multipart/form-data`.\n- `type=web`: Crawl a web page and save it as a document.\n- `type=empty`: Create an empty virtual document by name.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets/{dataset_id}/documents`\n- Query:\n - `type`: Optional. One of `local`, `web`, or `empty`. Defaults to `local`.\n- Headers:\n - `'Content-Type: multipart/form-data'` for `type=local` and `type=web`\n - `'Content-Type: application/json'` for `type=empty`\n - `'Authorization: Bearer '`\n- Body:\n - For `type=local`: form field `'file=@{FILE_PATH}'`\n - For `type=web`: form fields `'name'` and `'url'`\n - For `type=empty`: JSON body with `'name'`\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'file=@./test1.txt' \\\n --form 'file=@./test2.pdf'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=web' \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'name=example-page' \\\n --form 'url=https://example.com'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=empty' \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\"name\":\"blank.txt\"}'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to which the documents will be uploaded.\n- `type`: (*Query parameter*)\n Controls how the document is created:\n - `local`: Upload files.\n - `web`: Crawl a URL into a document.\n - `empty`: Create an empty document without file upload.\n- `'file'`: (*Body parameter*)\n A document to upload. Required when `type=local`.\n- `'name'`: (*Body parameter*)\n The document name. Required when `type=web` or `type=empty`.\n- `'url'`: (*Body parameter*)\n The source URL to crawl. Required when `type=web`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"chunk_method\": \"naive\",\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"dataset_id\": \"527fa74891e811ef9c650242ac120006\",\n \"id\": \"b330ec2e91ec11efbc510242ac120004\",\n \"location\": \"1.txt\",\n \"name\": \"1.txt\",\n \"parser_config\": {\n \"chunk_token_num\": 128,\n \"delimiter\": \"\\\\n\",\n \"html4excel\": false,\n \"layout_recognize\": \"DeepDOC\"\n },\n \"run\": \"UNSTART\",\n \"size\": 17966,\n \"thumbnail\": \"\",\n \"type\": \"doc\"\n }\n ]\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"No file part!\"\n}\n```\n\n---\n\n### Update document\n\n**PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nUpdates configurations for a specified document.\n\nSaving parser settings preserves existing chunks, chunk images, counters, and parsing progress. The saved settings take effect on the next parse; this update does not start parsing or delete existing results. Use the document parse action to clear existing chunks and re-parse with the saved settings.\n\n#### Request\n\n- Method: PATCH\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`:`string`\n - `\"meta_fields\"`:`object`\n - `\"chunk_method\"`:`string`\n - `\"parser_config\"`:`object`\n\n##### Request example\n\n```bash\ncurl --request PATCH \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --header 'Content-Type: application/json' \\\n --data '\n {\n \"name\": \"manual.txt\",\n \"chunk_method\": \"manual\",\n \"parser_config\": {\"chunk_token_num\": 128}\n }'\n\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the associated dataset.\n- `document_id`: (*Path parameter*)\n The ID of the document to update.\n- `\"name\"`: (*Body parameter*), `string`\n- `\"meta_fields\"`: (*Body parameter*), `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: (*Body parameter*), `string`\n The parsing method to apply to the document:\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. The attributes in this JSON object vary with the selected `\"chunk_method\"`:\n - If `\"chunk_method\"` is `\"naive\"`, the `\"parser_config\"` object contains the following attributes:\n - `\"chunk_token_num\"`: Defaults to `256`.\n - `\"layout_recognize\"`: Defaults to `\"DeepDOC\"`.\n - `\"html4excel\"`: Indicates whether to convert Excel documents into HTML format. Defaults to `false`.\n - `\"delimiter\"`: Defaults to `\"\\n\"`.\n - `\"task_page_size\"`: Defaults to `12`. For PDF only.\n- `\"enabled\"`: (*Body parameter*), `integer`\n Whether the document should be **available** in the knowledge base.\n - `1` → （available）\n - `0` → （unavailable）\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"id\": \"cd38dd72d4a611f0af9c71de94a988ef\",\n \"name\": \"large.md\",\n \"type\": \"doc\",\n \"suffix\": \"md\",\n \"size\": 2306906,\n \"location\": \"large.md\",\n \"source_type\": \"local\",\n \"status\": \"1\",\n \"run\": \"DONE\",\n \"dataset_id\": \"5f546a1ad4a611f0af9c71de94a988ef\",\n\n \"chunk_method\": \"naive\",\n \"chunk_count\": 2,\n \"token_count\": 8126,\n\n \"created_by\": \"eab7f446cb5a11f0ab334fbc3aa38f35\",\n \"create_date\": \"Tue, 09 Dec 2025 10:28:52 GMT\",\n \"create_time\": 1765247332122,\n \"update_date\": \"Wed, 17 Dec 2025 10:51:16 GMT\",\n \"update_time\": 1765939876819,\n\n \"process_begin_at\": \"Wed, 17 Dec 2025 10:33:55 GMT\",\n \"process_duration\": 14.8615,\n \"progress\": 1.0,\n\n \"progress_msg\": [\n \"10:33:58 Task has been received.\",\n \"10:33:59 Page(1~100000001): Start to parse.\",\n \"10:33:59 Page(1~100000001): Finish parsing.\",\n \"10:34:07 Page(1~100000001): Generate 2 chunks\",\n \"10:34:09 Page(1~100000001): Embedding chunks (2.13s)\",\n \"10:34:09 Page(1~100000001): Indexing done (0.31s).\",\n \"10:34:09 Page(1~100000001): Task done (11.68s)\"\n ],\n\n \"parser_config\": {\n \"chunk_token_num\": 512,\n \"delimiter\": \"\\n\",\n \"auto_keywords\": 0,\n \"auto_questions\": 0,\n \"topn_tags\": 3,\n \"layout_recognize\": \"DeepDOC\",\n \"html4excel\": false,\n \"image_context_size\": 0,\n \"table_context_size\": 0\n },\n\n \"meta_fields\": {},\n \"pipeline_id\": \"\",\n \"thumbnail\": \"\"\n }\n}\n\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset does not have the document.\"\n}\n```\n\n---\n\n### Download document\n\n**GET** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nDownloads a document from a specified dataset.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Authorization: Bearer '`\n- Output:\n - `'{PATH_TO_THE_FILE}'`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --output ./ragflow.txt\n```\n\n##### Path parameters\n\n- `dataset_id`\n The associated dataset ID.\n- `document_id`\n The ID of the document to download.\n\n#### Response\n\nSuccess:\n\n```json\nThis is a test to verify the file download feature.\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"You do not own the dataset 7898da028a0511efbf750242ac1220005.\"\n}\n```\n\n---\n\n### List documents\n\n**GET** `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n\nLists documents in a specified dataset.\n\nTo retrieve a specific document's settings and metadata, pass its document ID in the `id` query parameter: `/api/v1/datasets/{dataset_id}/documents?id={document_id}`.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n\n##### Request examples\n\n**A basic request with pagination:**\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents?page=1&page_size=10 \\\n --header 'Authorization: Bearer '\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The associated dataset ID.\n- `keywords`: (*Query parameter*), `string`\n The keywords used to match document titles.\n- `page`: (*Query parameter*), `integer`\n Specifies the page on which the documents will be displayed. Defaults to `1`.\n- `page_size`: (*Query parameter*), `integer`\n The maximum number of documents on each page. Defaults to `30`.\n- `orderby`: (*Query parameter*), `string`\n The field by which documents should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`: (*Query parameter*), `boolean`\n Indicates whether the retrieved documents should be sorted in descending order. Defaults to `true`.\n- `id`: (*Query parameter*), `string`\n The ID of the document to retrieve.\n- `create_time_from`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created after this time. 0 means no filter. Defaults to `0`.\n- `create_time_to`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created before this time. 0 means no filter. Defaults to `0`.\n- `suffix`: (*Query parameter*), `array[string]`\n Filter by file suffix. Supports multiple values, e.g., `pdf`, `txt`, and `docx`. Defaults to all suffixes.\n- `run`: (*Query parameter*), `array[string]`\n Filter by document processing status. Supports numeric, text, and mixed formats:\n - Numeric format: `[\"0\", \"1\", \"2\", \"3\", \"4\", \"5\"]`\n - Text format: `[UNSTART, RUNNING, CANCEL, DONE, FAIL, SCHEDULE]`\n - Mixed format: `[UNSTART, 1, DONE]` (mixing numeric and text formats)\n - Status mapping:\n - `0` / `UNSTART`: Document not yet processed\n - `1` / `RUNNING`: Document is currently being processed\n - `2` / `CANCEL`: Document processing was canceled\n - `3` / `DONE`: Document processing completed successfully\n - `4` / `FAIL`: Document processing failed\n - `5` / `SCHEDULE`: Document is scheduled and waiting to be processed\n Defaults to all statuses.\n- `metadata_condition`: (*Query parameter*), `object` (JSON in query)\n Optional metadata filter applied to documents when `document_ids` is not provided. Uses the same structure as retrieval:\n - `logic`: `\"and\"` (default) or `\"or\"`\n - `conditions`: array of `{ \"name\": string, \"comparison_operator\": string, \"value\": string }`\n - `comparison_operator` supports: `is`, `not is`, `contains`, `not contains`, `in`, `not in`, `start with`, `end with`, `>`, `<`, `≥`, `≤`, `empty`, `not empty`\n\n##### Usage examples\n\n**A request with multiple filtering parameters**\n\n```bash\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?suffix=pdf&run=DONE&page=1&page_size=10' \\\n --header 'Authorization: Bearer '\n```\n\n**Filter by metadata (query JSON):**\n\n```bash\ncurl -G \\\n --url \"http://{address}/api/v1/datasets/{dataset_id}/documents\" \\\n --header 'Authorization: Bearer ' \\\n --data-urlencode 'metadata_condition={\"logic\":\"and\",\"conditions\":[{\"name\":\"tags\",\"comparison_operator\":\"is\",\"value\":\"bar\"},{\"name\":\"author\",\"comparison_operator\":\"is\",\"value\":\"alice\"}]}'\n```\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"docs\": [\n {\n \"chunk_count\": 0,\n \"create_date\": \"Mon, 14 Oct 2024 09:11:01 GMT\",\n \"create_time\": 1728897061948,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"id\": \"3bcfbf8a8a0c11ef8aba0242ac120006\",\n \"knowledgebase_id\": \"7898da028a0511efbf750242ac120005\",\n \"location\": \"Test_2.txt\",\n \"name\": \"Test_2.txt\",\n \"parser_config\": {\n \"chunk_token_count\": 128,\n \"delimiter\": \"\\n\",\n \"layout_recognize\": \"DeepDOC\",\n \"task_page_size\": 12\n },\n ","tools/chatgpt-on-wechat/plugins/README.md":"# plugins Documentation\n\nPath: `tools/chatgpt-on-wechat/plugins/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/README.md)","tools/chatgpt-on-wechat/plugins/config.json":"# Plugins Skill\n\nPath: `tools/chatgpt-on-wechat/plugins/config.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/config.json)"},"files":{"docs/references/_category_.json":"{\n \"label\": \"References\",\n \"position\": 6,\n \"link\": {\n \"type\": \"generated-index\",\n \"description\": \"Miscellaneous References\"\n },\n \"customProps\": {\n \"sidebarIcon\": \"LucideScrollText\"\n }\n}\n",".github/copilot-instructions.md":"# Project instructions for Copilot\n\n## How to run (minimum)\n- Install:\n - python -m venv .venv && source .venv/bin/activate\n - pip install -r requirements.txt\n- Run:\n - (fill) e.g. uvicorn app.main:app --reload\n- Verify:\n - (fill) curl http://127.0.0.1:8000/health\n\n## Project layout (what matters)\n- app/: API entrypoints + routers\n- services/: business logic\n- configs/: config loading (.env)\n- docs/: documents\n- tests/: pytest\n\n## Conventions\n- Prefer small, incremental changes.\n- Add logging for new flows.\n- Add/adjust tests for behavior changes.\n","web/AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with the RAGFlow frontend (`web/`).\n\n## Project Overview\n\nRAGFlow frontend is a React/TypeScript application built with Vite:\n\n- **Components**: shadcn/ui\n- **Styling**: Tailwind CSS\n- **State**: Zustand\n- **Data Fetching**: TanStack Query (React Query)\n- **i18n**: react-i18next\n\n## Common Commands\n\n```bash\nnpm install\nnpm run dev # Development server\nnpm run build # Production build\nnpm run lint # oxlint\nnpm run format # oxfmt\nnpm run test # Jest tests\n```\n\n## Development Conventions\n\n### Pre-commit Format Validation (Mandatory)\n\nFrontend code changes **must pass the lefthook format checks before every commit**. Do not rely on CI (`web-oxfmt` / `web-oxlint` jobs) to catch formatting after pushing.\n\n- Before staging or committing changed `.ts`/`.tsx`/`.js`/etc. files, run the formatter and linter locally:\n - `npx oxfmt ` (or `npm run format`) to auto-fix formatting\n - `npx oxlint` (or `npm run lint`) for lint issues\n- Verify with `npx oxfmt --check ` before `git commit`.\n- Let the lefthook pre-commit hook run to completion and fix everything it reports; never bypass hooks with `--no-verify`.\n\n### Test File Placement\n\nWhen tests are grouped in a dedicated directory, that directory is always named `__tests__/` — never `tests/`. Example: `src/pages/agent/utils/__tests__/extractor-transform.test.ts` for `src/pages/agent/utils/extractor-transform.ts`. Co-located `foo.test.ts(x)` files next to the source file are also fine. When you find an existing test under a `tests/` directory, move it into `__tests__/` (keep git history via `git mv`) rather than adding new tests there.\n\n### Backend Conventions\n\nThe frontend serves only the **Go** backend. There is no runtime backend detection and no per-backend branching in app code; the dev server proxies `/api` and `/v1` to the Go API (`vite.config.ts`).\n\nDo not reintroduce backend-variant primitives or detection helpers (e.g. a `useIsGoBackend` hook, `pickByBackend`, ``, or an `/api/v1/language` probe) and do not create `go/`-vs-`python/` variant directories or files.\n\n### CSS and Layout Debugging\n\nWhen fixing CSS/layout issues (especially flex truncation, ellipsis, or element sizing), **always inspect the full parent hierarchy** for `flex-shrink`, `min-width`, and `overflow` constraints before applying fixes like `min-w-0`. Do not repeatedly apply the same fix without verifying the root cause.\n\n- Before editing, explain: (1) the full flex/container hierarchy from the target element up to the nearest non-flex ancestor, (2) what constraint is actually causing the bug, and (3) how the proposed fix addresses that root cause.\n\n### Color Tokens\n\nWhen writing or modifying styles, **use the project-defined color tokens from `src/tailwind.css`** (e.g., `bg-bg-base`, `text-text-primary`, `text-text-secondary`, `text-text-disabled`, `border-border-button`, `bg-bg-card`). Do not use arbitrary hex/RGB values or Tailwind's default palette colors (e.g., `emerald-500`, `blue-400`) directly in component class names. These tokens are defined for both light and dark modes and keep the UI consistent with the design system.\n\n### Scope and Boundaries\n\nRespect explicit boundaries from the user. If the user says **\"only fix the selected line\"** or **\"do not touch shared types/files\"**, follow that instruction exactly. Do not investigate unrelated errors, modify shared schemas (e.g., `LlmSettingFieldSchema`), or refactor other files without confirmation. If a change outside the described scope seems necessary, ask for permission first.\n\n### Internationalization (i18n)\n\nFor translation tasks, add keys **only to the explicitly requested language files** (commonly `src/locales/zh.ts` and `src/locales/en.ts`). Do not auto-propagate changes to all language files unless the user explicitly asks.\n\n- **Style for `en.ts`**: Sentence case — first word capitalized, rest lowercase (e.g., `referenceAnswer: 'Reference answer'`). Proper nouns remain as-is.\n\n### React Component Refactoring\n\nWhen refactoring or extracting components, **verify layout behavior after each structural change** (especially `flex-1`, conditional rendering, or flex direction changes). Check that existing buttons, alignment, and responsive behavior remain intact. After extraction, verify: (1) all original props and behavior are preserved, (2) layout in parent contexts is identical, and (3) no syntax or type errors were introduced.\n\n### State Management and Data Fetching\n\n#### Query Key Factory (Mandatory)\n\n**Never write raw `queryKey` arrays inline.** Always use a query key factory object that returns `as const` tuples. Raw arrays duplicated across `useQuery` and `invalidateQueries` are brittle, unreadable, and cause stale-cache bugs when key structures drift.\n\n```ts\n// ❌ Bad — raw array, hard to match with useQuery\nqueryClient.invalidateQueries({\n queryKey: [\n LLMApiAction.AddedProviders,\n params.provider_name,\n params.instance_name,\n 'models',\n ],\n});\n\n// ✅ Good — factory reference, self-documenting\nqueryClient.invalidateQueries({\n queryKey: LlmKeys.instanceModels(params.provider_name, params.instance_name),\n});\n```\n\n- Place the factory in the same file as the hooks, named `{Domain}Keys` (e.g., `LlmKeys`, `DatasetKeys`).\n- Every `useQuery` and every `invalidateQueries` must reference the same factory function.\n- Use `as const` on each factory return value for type-safe readonly tuples.\n\n#### Cache Debugging\n\nFor React Query / cache invalidation bugs, **carefully compare query keys across all consuming components and mutation hooks**. Mismatched keys (e.g., with/without `refreshCount`) are a common root cause of stale data or duplicate requests.\n\n- Systematically: (1) list every component/hook that calls `useQuery` for this data, (2) compare their query keys character-for-character, (3) check every mutation's `onSuccess` for cache invalidation, and (4) verify no parent re-renders are remounting the observer.\n\n#### Colocate Queries with the Consuming View\n\n**Fire a query in the component that renders its data — not in a parent page.** When a page switches between mutually exclusive views (tabs, view modes), extract each view into its own component that issues its own requests on mount. Conditional rendering then provides lazy loading for free.\n\n- Do not hoist child-view queries into the page component — it fires requests the user may never need (e.g., fetching the skill tree on page entry while the default view is the LLM wiki).\n- Do not thread `enabled` flags or view-mode props through hooks to gate a hoisted query; that is a sign the query lives at the wrong level. Split the view instead.\n- Remember the trade-off: with `gcTime: 0`, unmounting a view drops its cache, so switching back refetches. That is usually desirable for always-fresh data — do not reintroduce eager hoisting just to avoid the refetch.\n\n### Network Request Layering\n\nHTTP requests are organized in three layers. **Never import `@/utils/request`, `@/utils/next-request`, or `@/utils/api` directly inside a hook**:\n\n1. `src/hooks/use-xx-request.ts(x)` — React Query hooks; only call the service layer.\n2. `src/services/xx-service.ts` — Register endpoints via `registerNextServer`, all going through `@/utils/next-request`.\n3. `src/utils/next-request.ts` — The single axios instance; handles token, 401 redirects, and error notifications.\n\nInterface types are split between two folders:\n\n- Response/data shape → `src/interfaces/database/xx.ts`\n- Request params/body → `src/interfaces/request/xx.ts`\n\nModel-related endpoints (LLM provider / factory / my LLM, etc.) are consolidated in `src/services/llm-service.ts` rather than scattered across hooks. For GET endpoints, register with `method: 'get'` in the service, and on the call site pass `true` as the second argument to use the native axios config (e.g., `service.listProviders({ params: { available: true } }, true)`).\n\n### Shared UI Component Lock\n\nThe folder `src/components/ui/` is the project's **shared UI library** — it contains both official shadcn/ui primitives and project-authored common components built on top of shadcn. Both kinds are intended to be reused across the app and **must not be modified casually**.\n\n- **Do not modify, refactor, restyle, or \"improve\"** any file under `src/components/ui/` (including subfolders), even if it seems like the most direct fix.\n- If a component does not meet requirements, **wrap or compose it** in a new component **outside** `src/components/ui/` (e.g., under `src/components/` or a feature folder), and customize via `className`, `props`, or composition.\n- Exceptions require **explicit user approval** in the same conversation. When in doubt, ask first and propose a wrapper-based alternative.\n- Adding a new shared component to `src/components/ui/`, or upgrading a shadcn primitive via the official `shadcn` CLI, is allowed only when the user explicitly requests it.\n\n### React Patterns and Conventions\n\n- **Avoid inline event handlers.** Do not define arrow functions directly on JSX event props like `onClick={() => ...}` or `onChange={(e) => ...}`. Instead, extract the handler and reference it by name (e.g., `onClick={handleClick}`). Inline handlers are recreated on every render, which can break `React.memo` optimizations and make stack traces harder to read.\n- **Prefer `requestAnimationFrame` or `useLayoutEffect`** over `setTimeout(..., 0)` for focus or DOM measurement operations.\n- **Prefer `useTranslation` from `react-i18next`** over project-wrapped utilities like `useTranslate`.\n- Extract complex logic into hooks or utils; keep components lean.\n- Use **PascalCase** for constants and component names.\n - Components: `EditableTextarea`, `RAGFlowFormItem`\n - Constants: `InitialMockData`, `DefaultPlaceholder`\n- Avoid camelCase or SCREAMING_SNAKE_CASE for components and top-level constants.\n- **Place domain constant lists in the page's `constant/` directory** (e.g. `pages/agent/constant/pipeline.tsx`) when they belong to an existing family of constants there (like `ChunkerOperators`, `SingleOperators`), instead of declaring them locally next to their consumer. Keeping related lists in one file makes their overlaps and differences visible and gives new entries a single obvious home.\n- **Name things semantically, not generically.** A name should say what the value holds or what the callback does, so call sites read naturally. Don't leak a generic underlying-primitive API into your own component's public props — rename at the wrapper boundary (e.g., a tabs wrapper exposes `activeTab` / `onTabChange`, mapped internally to Radix's `value` / `onValueChange`), and align with what callers already name the state. Same for variables and functions: `hiddenNodeIds` over `nodeIdSet`, `pickByBackend` over `resolveVariant`.\n- Avoid duplicating component structures in JSX; favor render props or reusable components.\n\n### Utility Libraries and Reuse\n\n- **Time/date handling**: Use `dayjs` for all date/time formatting, parsing, and manipulation.\n- **Utility hooks**: Prefer `ahooks` for common reusable hooks (e.g., `useDebounce`, `useSetState`).\n- **General utilities**: Lodash is available for utility functions when needed.\n- **Project utilities first**: Before reaching for a third-party library, check if the project already has an existing utility or hook that covers the need.\n- **Extract and share**: If repeated logic cannot be satisfied by an existing project utility or a third-party library, extract it into an appropriate shared hook (`src/hooks/`) or utility file (`src/utils/`).\n- **Check for duplicate patterns before adding**: When asked to add logic (validation, existence checks, API calls, etc.), first search for existing hooks/functions that do the same or similar thing — especially in the same file or sibling hooks. If a hook already does X, call it instead of re-implementing X inline. This applies to mutations calling mutations, utility wrappers, and boilerplate around API calls.\n","AGENTS.md":"# RAGFlow Instructions\n\nUse this file as the local operating guide for the current codebase. Prefer the code and the current AGENTS.md over any older convention or remembered project shape.\n\n## Core Stance\n- Treat legacy code as liability, not as a compatibility target.\n- Prefer deletion over shims, deprecated branches, wrapper APIs, and dual-track migration notes.\n- If old and new implementations coexist, converge to one path unless an external contract forces compatibility.\n- Remove dead tests, commented-out code, stale docs, and \"move later\" notes instead of preserving them.\n- Reduce public surface area when a helper can be made private or internal.\n- Keep refactors centered on the owning abstraction, not on adjacent compatibility layers.\n\n## Current stack\n- Backend: Go implements the API, admin, ingestor, and syncer modes in `cmd/ragflow_server.go`; the CLI runs from `cmd/ragflow-cli.go`. The Docker entrypoint starts the Go services according to its command-line flags, and the frontend development server uses fixed Go backend proxy routes. The Python API, admin, and workers are still present pending removal.\n- Frontend: React + TypeScript + Vite in `web/`. When working under `web/`, read and follow `web/AGENTS.md` for frontend conventions.\n- The Go module contains the server, ingestion, parsing, agent runtime, CLI, and supporting services.\n- Runtime services commonly include MySQL/PostgreSQL, Kvrocks, NATS JetStream, MinIO, ClickHouse, and Elasticsearch/Infinity/OpenSearch depending on configuration.\n## Go-only target\n- Remove all Python code and Python-specific dependencies, scripts, containers, CI jobs, documentation, SDKs, and tests from the repository. Do not add new Python code while this removal is in progress. Replace Python-based preparation of Go native libraries, DeepDoc models, and the BPE table before removing the existing download scripts.\n- Move any still-required behavior and test coverage into the Go backend or the frontend before deleting the old implementation. Do not preserve a Python server, compatibility layer, fallback, or dual-backend path.\n- Before deleting `rag/`, move the Go DeepDoc `.ort` models and `ocr.res` out of `internal/rag/res/deepdoc/`, then update model discovery, dependency preparation, Docker packaging, and tests together.\n- Remove the frontend's Go/Python backend variants and update `web/CLAUDE.md` when the frontend no longer needs them. Until then, treat its dual-backend instructions as migration-only guidance; new frontend work should target the Go API.\n- Update Docker's default entrypoint, dependency image, and CI workflows when removing their remaining Python paths. Keep the frontend development proxy routes aligned with the Go API and Admin ports in `web/vite.config.ts`.\n- Keep this guide aligned with the checked-in tree. Once the Python directories are deleted, remove their migration inventory below.\n\n## Code Layout to Expect\n- `cmd/ragflow_server.go`: Go entrypoint for the API, admin, ingestor, and syncer modes (`--api`, `--admin`, `--ingestor`, `--syncer`).\n- `cmd/ragflow-cli.go`: Go CLI entrypoint.\n- `api/`, `rag/`, `deepdoc/`, and `agent/`: legacy Python implementation pending deletion. Do not add features or extend dependencies there.\n- `internal/`: main Go application code. Important subtrees:\n- `internal/agent/`: Go agent runtime, canvas execution, components, tool bindings, workflow helpers.\n- `internal/admin/`: Go admin routes, handlers, and services.\n- `internal/binding/cpp/`: C++ tokenizer binding built by `build.sh` for the Go server.\n- `internal/cli/`: CLI parsing, HTTP transport, command execution, response formatting.\n- `internal/dao/`: Go data-access layer and persistence-facing helpers.\n- `internal/deepdoc/`: Go DeepDOC integrations, especially native-backed PDF/DOCX parsing.\n- `internal/engine/`: search/index backends such as Elasticsearch and Infinity.\n- `internal/entity/`: shared Go entities and model definitions.\n- `internal/handler/`: HTTP handlers and route-facing request logic.\n- `internal/ingestion/`: Go ingestion pipeline, canvas adapter, components, wiring, service orchestration.\n- `internal/ingestion/component/`: stage implementations such as file/parser/chunker/tokenizer/extractor.\n- `internal/ingestion/pipeline/`: DSL translation, canvas-driven execution, checkpoints, resume/run logic.\n- `internal/parser/`: parser and chunk libraries used by ingestion and other Go paths.\n- `internal/parser/parser/`: typed parse-result parsers for markdown/html/pdf/docx/xlsx/text and related families.\n- `internal/parser/chunk/`: chunk operator library and DSL/typed execution helpers.\n- `internal/service/`: higher-level business services used by handlers and server flows.\n- `internal/storage/`: storage backends and in-memory test doubles.\n- `internal/router/`: HTTP route registration.\n- `internal/server/`: server bootstrap/config wiring.\n- `internal/syncer/`: Go file synchronization service and connectors.\n- `internal/rag/`: Go retrieval and RAG logic.\n- `internal/harness/`: Go runtime harness and supporting execution code.\n- `web/`: frontend application.\n- `docker/`: local and production compose files.\n- `sdk/python/`: legacy Python client SDK pending deletion.\n- `tools/scripts/download_dev_deps.py` and `deps/download_deps.py`: Python downloaders for the Go build's dependencies. `download_dev_deps.py` pulls the native static libraries (pdfium, pdf_oxide, office_oxide, onnxruntime) that `build.sh` links; `download_deps.py` pulls the runtime resources (cl100k BPE table, DeepDoc models, embedding tokenizer assets) and installs the stagehand server into the SDK cache. Versions and URLs are defined inline in each downloader (`tools/scripts/download_dev_deps.py` for native static libraries, `deps/download_deps.py` for runtime resources); the tokenizer assets are inlined as `TOKENIZER_ASSETS` in each downloader. Replace these with non-Python preparation before removing Python dependencies.\n- `test/`: contains Python tests, including tests that import the legacy Python implementation directly. Migrate needed coverage to Go tests or frontend tests, then remove the Python tests; keep only tests for supported code.\n\n## Go-Specific Rules\n- Treat `internal/ingestion`, `internal/parser`, and `internal/deepdoc` as actively refactored code. Prefer collapsing duplicate paths over preserving transitional wrappers.\n- Do not add or preserve deprecated Go APIs just to ease migration inside the repo.\n- Remove commented-out Go code instead of leaving recovery notes in place.\n- Keep package comments and doc comments aligned with the current runtime path, not with migration history.\n\n## GORM query traps\n- **Never pass a bare string ID as the second argument to `Take`/`First`** (e.g. `Take(&x, id)`). Under GORM v1.25.7 a non-numeric string is treated as a raw SQL condition and concatenated into the `WHERE` clause: `Take(&x, \"88b17c7a...\")` emits `WHERE 88b17c7a... LIMIT 1`, which MySQL rejects with `Error 1054 (42S22): Unknown column '...' in 'where clause'`. Only strings that parse via `strconv.Atoi` are treated as primary-key values, so this bug is invisible for integer IDs and catastrophic for UUID primary keys. Always bind explicitly: `Take(&x, \"id = ?\", id)` or `Where(\"id = ?\", id).Take(&x)`.\n- When you change a query form across many files, verify the generated SQL with a `Session(&gorm.Session{DryRun: true})` probe before relying on it. A passing unit test with no rows and a broken `WHERE` clause look identical unless the SQL is inspected.\n\n## Shared database schema (Go + Python)\nThe Go services and the Python API write to the same MySQL/PostgreSQL schema, and both own parts of it. Neither is authoritative over the whole.\n\nEach ORM derives index names differently:\n- Go (`internal/entity/`): GORM derives `idx_
_` for indexes and `uni_
_` for unique constraints. Tags in this repo name unique indexes explicitly, e.g. `uniqueIndex:idx_commit_file`.\n- Python (`api/db/db_models.py`): Peewee derives `__` (truncated with an md5 suffix past 64 chars, see `playhouse.migrate.make_index_name`). There is no `idx_` prefix.\n\nBoth sides reconcile by **column set, not by name**: Go's `hasUniqueIndex` (`internal/dao/migration.go`) accepts an existing index only when its full column set equals the requested one, and Python's `ensure_model_indexes` (`api/db/db_models.py`) keys `DB.get_indexes()` on the column tuple. Consequences:\n\n- Divergent names covering the same columns are expected, not a defect. Whichever runtime reaches the database first decides the name. Do not \"fix\" this by renaming inside a migration; it rewrites live databases for no functional gain.\n- Add a shared unique index to **both** sides (Go tag + Peewee `Meta.indexes` or `unique=True`), so each runtime recreates it when absent. Do not add a second index over a column set an existing one already covers.\n- In Go, declare single-column uniqueness with a named `uniqueIndex:`, not a bare `unique` tag. A bare `unique` becomes a table-level constraint in DDL, which SQLite materializes as an implicit index that cannot be dropped by name, and which GORM's `MigrateColumnUnique` then tries to drop under its own `uni_
_` name — MySQL answers 1091 every startup. `namedIndexMigrator` (`internal/dao/database.go`) suppresses exactly that phantom drop; see `TestNamedIndexMigratorSkipsPhantomUniqueDrop`.\n- A Python model existing does not mean Python owns that table's constraints. `pipeline_operation_log` has no `run_count` column and no unique index in `db_models.py`; both are added only by `internal/dao/migration.go`.\n\n## Go Test Tiers\nGo tests are classified by build tag. Tag a test file with `//go:build ` placed before the `package` clause. Run them through `build.sh` so the required CGO configuration and native libraries are available.\n\n| Tier | Build tag | Runs by default? | Needs |\n|---|---|---|---|\n| Unit | (none) | Yes (`bash build.sh --test`) | Native CGO static libs; no external services — uses in-memory SQLite, miniredis, or `httptest` stubs. |\n| Integration | `integration` | No (`-tags integration`) | A real service: MySQL/MinIO/Elasticsearch/Infinity/LLM. Single component, reasonably fast. |\n| E2E | `e2e` | No (`-tags e2e`) | Full cross-component pipeline (ingest → index → retrieve) against real services; heavy/slow. |\n| Manual | `manual` | No (`-tags manual`) | Very slow/expensive (deepdoc render/parity/snapshot/bench). **Local opt-in ONLY — never run in CI.** |\n| Native (orthogonal) | `cgo` / `!cgo` | `cgo` auto-satisfies under CGO_ENABLED=1 | Native static libs (`office_oxide`/`pdfium`/`pdf_oxide`). Combine with tiers, e.g. `//go:build cgo && integration`. |\n\nRun tiers locally via `build.sh`:\n```bash\nbash build.sh --test # unit tier (no tags)\nbash build.sh --test-integration ./... # integration tier\nbash build.sh --test-e2e # e2e tier\nbash build.sh --test-manual # manual tier (very slow)\nbash build.sh --test-all # integration + e2e (never includes manual)\n```\nRules:\n- New tests that touch a real external service MUST carry `integration`/`e2e`/`manual` — do not rely on `t.Skip` + env vars to soft-isolate them in the default unit run. Keep an env guard as a harmless secondary safety net if desired.\n- `manual` is never wired into CI or any automated pipeline.\n- `unit` (no tag) must stay free of external-service dependencies so `bash build.sh --test` runs without MySQL/MinIO/ES/Infinity/LLM. The native CGO static libraries (`office_oxide`/`pdfium`/`pdf_oxide`) are still required at build time and are wired by `build.sh --test`; that is expected, not an external service.\n\n## Working Rules\n- When reviewing documentation or code, inspect the full affected path and report all verifiable findings in one review; do not return after only a few findings and expose further issues in later rounds.\n- When handling review comments, independently verify each substantive claim against the current code or tests before accepting, rejecting, or acting on it.\n- Before editing, inspect the nearest code path that actually owns the behavior.\n- For server changes, work in the Go implementation. If a Python behavior is still needed, port it to Go and test the Go path before removing the Python code.\n- Keep changes small and local unless the task is explicitly a broader refactor.\n- Prefer one implementation path instead of preserving old and new versions side by side.\n- Preserve behavior with focused tests when the behavior is still valid; do not keep tests that protect obsolete behavior.\n- If a surface is only there for compatibility, remove it unless the user asks to keep it.\n- Do not add new compatibility wording in comments or docs.\n- When a maintainer takes over a community PR, a new commit generated by rewriting history (e.g. `merge`, `rebase -i`) must preserve the original author and add the maintainer as co-author (via a `Co-authored-by:` trailer) instead of overwriting the author with the maintainer alone.\n\n## Commands\n### Go backend\n```bash\nbash build.sh --all # build native code and Go server\nbash build.sh --go # build Go server with native library already built\nbash build.sh --test ./internal/handler/... # targeted unit tests\nbash build.sh --test # all unit tests\nbash build.sh --test-integration ./... # integration tests; requires real services\nbash build.sh --test-e2e # full-pipeline tests; requires real services\nbash build.sh --test-manual # local opt-in only\nbash build.sh --test-all # integration + e2e; excludes manual\n```\n\nNative libraries, Go DeepDoc models, tokenizer assets, and `deps/cl100k_base.tiktoken` come from `tools/scripts/download_dev_deps.py` (native static libs) and `deps/download_deps.py` (models, cl100k, tokenizer assets); run them with `uv run`. Build the Go resource image with `cd deps && docker build -f Dockerfile -t infiniflow/ragflow_deps:latest .` after downloading the resources.\n\n### Frontend\n```bash\ncd web\nnpm install\nnpm run dev\nnpm run build\nnpm run lint\nnpm run test\nnpm run type-check\n```\n\n## Validation Preference\n- Run the narrowest relevant test, lint, or build command after a change.\n- For backend changes, first run package-scoped `bash build.sh --test ./path/to/package/...`. Run the integration or e2e tier when the changed behavior needs real services or a full pipeline.\n- For frontend changes, prefer the touched-package lint, type-check, or test command.\n- Do not default to raw `go test`, `go build`, or IDE Run/Debug for Go in this repo. They often miss the required CGO flags and native static libraries (`office_oxide`, `pdfium-static`, `pdf_oxide`) that `build.sh` wires correctly.\n- After a Go build, execute the resulting binary's help command for each changed server mode; a successful link alone does not prove the binary starts. On this host, the original `build.sh` selected default LLD 18 and produced binaries that crashed before Go `main`; linking with LLD 20 started successfully. Verify the linker actually selected on Linux before trusting a new binary.\n- If Go native builds fail, inspect `build.sh` and `internal/development.md` before changing code. Common environment issues are missing downloaded native dependencies or an incompatible `ld.lld` on Linux.\n\n## Default review checklist\n- Remove instead of retaining `deprecated`, `legacy`, or compatibility-only code.\n- Collapse duplicate implementations to one path.\n- Drop stale comments and documentation that describe a superseded design.\n- Keep exported APIs only when the current code actually needs them.\n",".agents/skills/go-naming/SKILL.md":"---\nname: go-naming\ndescription: Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards.\n---\n\nStrictly follow the naming conventions in [rules/named.md](../../rules/named.md)\n",".agents/rules/named.md":"# Go Naming Best Practices\n\n## 1. Package Naming\n\n- **All lowercase, no underscores**: `package user`, not `package userService` or `package user_service`\n- **Short and meaningful**: `package http`, `package json`, `package dao`\n- **Avoid plurals**: `package user` not `package users`\n- **Avoid generic names**: Avoid `package util`, `package common`, `package base`\n\n```go\n// Recommended\npackage user\npackage handler\npackage service\n\n// Not recommended\npackage UserService\npackage user_service\npackage utils\n```\n\n## 2. File Naming\n\n- **All lowercase, underscore separated**: `user_handler.go`, `user_service.go`\n- **Test files**: `user_handler_test.go`\n- **Platform-specific**: `user_linux.go`, `user_windows.go`\n\n```\nuser/\n├── user_handler.go\n├── user_service.go\n├── user_dao.go\n└── user_test.go\n```\n\n## 3. Directory Naming\n\n- **All lowercase, no underscores or hyphens**: `internal/`, `pkg/`, `cmd/`\n- **Short and descriptive**: `handler/`, `service/`, `dao/`\n\n```\nproject/\n├── cmd/ # Main entry point\n│ └── server_main.go\n├── internal/ # Private code\n│ ├── handler/\n│ ├── service/\n│ ├── dao/\n│ ├── model/\n│ └── middleware/\n├── pkg/ # Public code\n└── api/ # API definitions\n```\n\n## 4. Interface Naming\n\n- **Single-method interfaces end with \"-er\"**: `Reader`, `Writer`, `Handler`\n- **Verb form**: `Reader`, `Executor`, `Validator`\n\n```go\n// Recommended\ntype Reader interface {\n Read(p []byte) (n int, err error)\n}\n\ntype UserService interface {\n Register(req *RegisterRequest) (*User, error)\n Login(req *LoginRequest) (*User, error)\n}\n\n// Not recommended\ntype UserInterface interface {}\ntype IUserService interface {}\n```\n\n## 5. Struct Naming\n\n- **CamelCase**: `UserService`, `UserHandler`\n- **Avoid redundant prefixes**: `User` not `UserModel`\n\n```go\n// Recommended\ntype UserService struct {}\ntype UserHandler struct {}\ntype RegisterRequest struct {}\n\n// Not recommended\ntype user_service struct {}\ntype SUserService struct {}\ntype UserModel struct {}\n```\n\n## 6. Method/Function Naming\n\n- **CamelCase**\n- **Start with verb**: `GetUser`, `CreateUser`, `DeleteUser`\n- **Boolean returns use Is/Has/Can prefix**: `IsValid`, `HasPermission`\n\n```go\n// Recommended\nfunc (s *UserService) Register(req *RegisterRequest) (*User, error)\nfunc (s *UserService) GetUserByID(id uint) (*User, error)\nfunc (s *UserService) IsEmailExists(email string) bool\n\n// Not recommended\nfunc (s *UserService) register_user()\nfunc (s *UserService) get_user_by_id()\nfunc (s *UserService) CheckEmailExists() // Should use Is/Has\n```\n\n## 7. Constant Naming\n\n- **CamelCase**: `const MaxRetryCount = 3`\n- **Enum constants**: `const StatusActive = \"active\"`\n\n```go\n// Recommended\nconst (\n StatusActive = \"1\"\n StatusInactive = \"0\"\n MaxRetryCount = 3\n)\n\n// Not recommended\nconst (\n STATUS_ACTIVE = \"1\" // Not all uppercase\n status_active = \"1\" // Not all lowercase\n)\n```\n\n## 8. Error Variable Naming\n\n- **Start with \"Err\"**: `ErrNotFound`, `ErrInvalidInput`\n\n```go\n// Recommended\nvar (\n ErrNotFound = errors.New(\"not found\")\n ErrInvalidInput = errors.New(\"invalid input\")\n ErrUnauthorized = errors.New(\"unauthorized\")\n)\n```\n\n## 9. Acronyms Keep Consistent Case\n\n```go\n// Recommended\ntype HTTPHandler struct {}\nvar URL string\nfunc GetHTTPClient() {}\nfunc ParseJSON() {}\n\n// Not recommended\ntype HttpHandler struct {}\nvar Url string\nfunc GetHttpClient() {}\n```\n\n## 10. Project Structure Naming\n\n```\nproject-name/\n├── cmd/ # Main programs\n│ └── app_name/\n│ └── main.go\n├── internal/ # Private code\n│ ├── handler/ # HTTP handlers\n│ ├── service/ # Business logic\n│ ├── repository/ # Data access\n│ ├── model/ # Data models\n│ └── config/ # Configuration\n├── pkg/ # Public code\n├── api/ # API definitions\n├── configs/ # Config files\n├── scripts/ # Scripts\n├── docs/ # Documentation\n├── go.mod\n└── go.sum\n```\n\n## Summary Table\n\n| Type | Rule | Example |\n| -------------- | ----------------------------------- | ------------------- |\n| Package | All lowercase, no underscores | `package user` |\n| File | All lowercase, underscore separated | `user_service.go` |\n| Directory | All lowercase, no separators | `internal/handler/` |\n| Struct | CamelCase, capitalized first letter | `UserService` |\n| Interface | CamelCase, -er suffix | `Reader`, `Writer` |\n| Method | CamelCase, verb prefix | `GetUserByID` |\n| Constant | CamelCase | `MaxRetryCount` |\n| Error Variable | Err prefix | `ErrNotFound` |\n","tools/scripts/INSTALL_SCRIPTS_README.md":"# RAGFlow CLI Installation Scripts\n\nRAGFlow publishes static Go CLI binaries as GitHub Release assets for:\n\n- Linux: `amd64`, `arm64`\n- macOS: `amd64`, `arm64`\n- Windows: `amd64`, `arm64`\n\nThe release workflow builds the binaries with `CGO_ENABLED=0`, uploads `SHA256SUMS`, and uploads `install.sh` and `install.ps1`.\n\n## Linux And macOS\n\nUse the script maintained in the RAGFlow repository:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | sh\n```\n\nInstall a specific version:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | VERSION=v1.0.0 sh\n```\n\nInstall to a user-writable directory:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | INSTALL_DIR=\"$HOME/.local/bin\" sh\n```\n\nThe Unix installer:\n\n- detects `linux` or `darwin`\n- detects `amd64` or `arm64`\n- downloads `ragflow-cli-{VERSION}-{OS}-{ARCH}`\n- verifies the file with the release `SHA256SUMS`\n- installs to `/usr/local/bin` by default\n\n## Windows PowerShell\n\nRun the repository script in PowerShell:\n\n```powershell\nirm https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.ps1 | iex\n```\nInstall a specific version:\n\n```powershell\n.\\install.ps1 -Version \"v1.0.0\"\n```\n\nInstall to a custom directory:\n\n```powershell\n.\\install.ps1 -InstallDir \"$env:USERPROFILE\\bin\"\n```\n\nThe Windows installer:\n\n- detects `windows/amd64` or `windows/arm64`\n- downloads `ragflow-cli-{VERSION}-windows-{ARCH}.exe`\n- verifies the file with the release `SHA256SUMS`\n- installs to `$env:LOCALAPPDATA\\Programs\\RAGFlow` by default\n- adds the install directory to the user `PATH`\n\n## Release Asset Names\n\nThe install scripts expect these names:\n\n```text\nragflow-cli-v1.0.0-linux-amd64\nragflow-cli-v1.0.0-linux-arm64\nragflow-cli-v1.0.0-darwin-amd64\nragflow-cli-v1.0.0-darwin-arm64\nragflow-cli-v1.0.0-windows-amd64.exe\nragflow-cli-v1.0.0-windows-arm64.exe\nSHA256SUMS\ninstall.sh\ninstall.ps1\n```\n\nBy default, both scripts install the latest stable GitHub Release. Set\n`VERSION` on Unix or `-Version` on Windows to pin a specific release.\n","tools/firecrawl/README.md":"# Firecrawl Integration for RAGFlow\n\nThis integration adds [Firecrawl](https://firecrawl.dev)'s powerful web scraping capabilities to [RAGFlow](https://github.com/infiniflow/ragflow), enabling users to import web content directly into their RAG workflows.\n\n## 🎯 **Integration Overview**\n\nThis integration implements the requirements from [Firecrawl Issue #2167](https://github.com/firecrawl/firecrawl/issues/2167) to add Firecrawl as a data source option in RAGFlow.\n\n### ✅ **Acceptance Criteria Met**\n\n- ✅ **Integration appears as selectable data source** in RAGFlow's UI\n- ✅ **Users can input Firecrawl API keys** through RAGFlow's configuration interface\n- ✅ **Successfully scrapes content** and imports into RAGFlow's document processing pipeline\n- ✅ **Handles edge cases** (rate limits, failed requests, malformed content)\n- ✅ **Includes documentation** and README updates\n- ✅ **Follows RAGFlow patterns** and coding standards\n- ✅ **Ready for engineering review**\n\n## 🚀 **Features**\n\n### Core Functionality\n- **Single URL Scraping** - Scrape individual web pages\n- **Website Crawling** - Crawl entire websites with job management\n- **Batch Processing** - Process multiple URLs simultaneously\n- **Multiple Output Formats** - Support for markdown, HTML, links, and screenshots\n\n### Integration Features\n- **RAGFlow Data Source** - Appears as selectable data source in RAGFlow UI\n- **API Configuration** - Secure API key management with validation\n- **Content Processing** - Converts Firecrawl output to RAGFlow document format\n- **Error Handling** - Comprehensive error handling and retry logic\n- **Rate Limiting** - Built-in rate limiting and request throttling\n\n### Quality Assurance\n- **Content Cleaning** - Intelligent content cleaning and normalization\n- **Metadata Extraction** - Rich metadata extraction and enrichment\n- **Document Chunking** - Automatic document chunking for RAG processing\n- **Language Detection** - Automatic language detection\n- **Validation** - Input validation and error checking\n\n## 📁 **File Structure**\n\n```\ntools/firecrawl/\n├── __init__.py # Package initialization\n├── firecrawl_connector.py # API communication with Firecrawl\n├── firecrawl_config.py # Configuration management\n├── firecrawl_processor.py # Content processing for RAGFlow\n├── firecrawl_ui.py # UI components for RAGFlow\n├── ragflow_integration.py # Main integration class\n├── example_usage.py # Usage examples\n├── requirements.txt # Python dependencies\n├── README.md # This file\n└── INSTALLATION.md # Installation guide\n```\n\n## 🔧 **Installation**\n\n### Prerequisites\n- RAGFlow instance running\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n### Setup\n1. **Get Firecrawl API Key**:\n - Visit [firecrawl.dev](https://firecrawl.dev)\n - Sign up for a free account\n - Copy your API key (starts with `fc-`)\n\n2. **Configure in RAGFlow**:\n - Go to RAGFlow UI → Data Sources → Add New Source\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify setup\n - You should see a success message\n\n## 🎮 **Usage**\n\n### Single URL Scraping\n1. Select \"Single URL\" as scrape type\n2. Enter the URL to scrape\n3. Choose output formats (markdown recommended for RAG)\n4. Start scraping\n\n### Website Crawling\n1. Select \"Crawl Website\" as scrape type\n2. Enter the starting URL\n3. Set crawl limit (maximum number of pages)\n4. Configure extraction options\n5. Start crawling\n\n### Batch Processing\n1. Select \"Batch URLs\" as scrape type\n2. Enter multiple URLs (one per line)\n3. Choose output formats\n4. Start batch processing\n\n## 🔧 **Configuration Options**\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## 📊 **API Reference**\n\n### RAGFlowFirecrawlIntegration\n\nMain integration class for Firecrawl with RAGFlow.\n\n#### Methods\n- `scrape_and_import(urls, formats, extract_options)` - Scrape URLs and convert to RAGFlow documents\n- `crawl_and_import(start_url, limit, scrape_options)` - Crawl website and convert to RAGFlow documents\n- `test_connection()` - Test connection to Firecrawl API\n- `validate_config(config_dict)` - Validate configuration settings\n\n### FirecrawlConnector\n\nHandles communication with the Firecrawl API.\n\n#### Methods\n- `scrape_url(url, formats, extract_options)` - Scrape single URL\n- `start_crawl(url, limit, scrape_options)` - Start crawl job\n- `get_crawl_status(job_id)` - Get crawl job status\n- `batch_scrape(urls, formats)` - Scrape multiple URLs concurrently\n\n### FirecrawlProcessor\n\nProcesses Firecrawl output for RAGFlow integration.\n\n#### Methods\n- `process_content(content)` - Process scraped content into RAGFlow document format\n- `process_batch(contents)` - Process multiple scraped contents\n- `chunk_content(document, chunk_size, chunk_overlap)` - Chunk document content for RAG processing\n\n## 🧪 **Testing**\n\nThe integration includes comprehensive testing:\n\n```bash\n# Run the test suite\ncd tools/firecrawl\npython3 -c \"\nimport sys\nsys.path.append('.')\nfrom ragflow_integration import create_firecrawl_integration\n\n# Test configuration\nconfig = {\n 'api_key': 'fc-test-key-123',\n 'api_url': 'https://api.firecrawl.dev'\n}\n\nintegration = create_firecrawl_integration(config)\nprint('✅ Integration working!')\n\"\n```\n\n## 🐛 **Error Handling**\n\nThe integration includes robust error handling for:\n\n- **Rate Limiting** - Automatic retry with exponential backoff\n- **Network Issues** - Retry logic with configurable timeouts\n- **Malformed Content** - Content validation and cleaning\n- **API Errors** - Detailed error messages and logging\n\n## 🔒 **Security**\n\n- API key validation and secure storage\n- Input sanitization and validation\n- Rate limiting to prevent abuse\n- Error handling without exposing sensitive information\n\n## 📈 **Performance**\n\n- Concurrent request processing\n- Configurable timeouts and retries\n- Efficient content processing\n- Memory-conscious document handling\n\n## 🤝 **Contributing**\n\nThis integration was created as part of the [Firecrawl bounty program](https://github.com/firecrawl/firecrawl/issues/2167).\n\n### Development\n1. Fork the RAGFlow repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## 📄 **License**\n\nThis integration is licensed under the same license as RAGFlow (Apache 2.0).\n\n## 🆘 **Support**\n\n- **Firecrawl Documentation**: [docs.firecrawl.dev](https://docs.firecrawl.dev)\n- **RAGFlow Documentation**: [RAGFlow GitHub](https://github.com/infiniflow/ragflow)\n- **Issues**: Report issues in the RAGFlow repository\n\n## 🎉 **Acknowledgments**\n\nThis integration was developed as part of the Firecrawl bounty program to bridge the gap between web content and RAG applications, making it easier for developers to build AI applications that can leverage real-time web data.\n\n---\n\n**Ready for RAGFlow Integration!** 🚀\n\nThis integration enables RAGFlow users to easily import web content into their knowledge retrieval systems, expanding the ecosystem for both Firecrawl and RAGFlow.\n","tools/es-to-oceanbase-migration/README.md":"# RAGFlow ES to OceanBase Migration Tool\n\nA CLI tool for migrating RAGFlow data from Elasticsearch to OceanBase. This tool is specifically designed for RAGFlow's data structure and handles schema conversion, vector data mapping, batch import, and resume capability.\n\n## Features\n\n- **RAGFlow-Specific**: Designed for RAGFlow's fixed data schema\n- **ES 8+ Support**: Uses `search_after` API for efficient data scrolling\n- **Vector Support**: Auto-detects vector field dimensions from ES mapping\n- **Batch Processing**: Configurable batch size for optimal performance\n- **Resume Capability**: Save and resume migration progress\n- **Data Consistency Validation**: Compare document counts and sample data\n- **Migration Report Generation**: Generate detailed migration reports\n\n## Quick Start\n\nThis section provides a complete guide to verify the migration works correctly with a real RAGFlow deployment.\n\n### Prerequisites\n\n- RAGFlow source code cloned\n- Docker and Docker Compose installed\n- This migration tool installed (`uv pip install -e .`)\n\n### Step 1: Start RAGFlow with Elasticsearch Backend\n\nFirst, start RAGFlow using Elasticsearch as the document storage backend (default configuration).\n\n```bash\n# Navigate to RAGFlow docker directory\ncd /path/to/ragflow/docker\n\n# Ensure DOC_ENGINE=elasticsearch in .env (this is the default)\n# DOC_ENGINE=elasticsearch\n\n# Start RAGFlow with Elasticsearch (--profile cpu for CPU, --profile gpu for GPU)\ndocker compose --profile elasticsearch --profile cpu up -d\n\n# Wait for services to be ready (this may take a few minutes)\ndocker compose ps\n\n# Check ES is running\ncurl -X GET \"http://localhost:9200/_cluster/health?pretty\"\n```\n\n### Step 2: Create Test Data in RAGFlow\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Create a new Knowledge Base\n3. Upload some test documents (PDF, TXT, DOCX, etc.)\n4. Wait for the documents to be parsed and indexed\n5. Test the knowledge base with some queries to ensure it works\n\n### Step 3: Verify ES Data (Optional)\n\nBefore migration, verify the data exists in Elasticsearch. This step is important to ensure you have a baseline for comparison after migration.\n\n```bash\n# Navigate to migration tool directory (from ragflow root)\ncd tools/es-to-oceanbase-migration\n\n# Activate the virtual environment if not already done\nsource .venv/bin/activate\n\n# Check connection and list indices\nes-ob-migrate status --es-host localhost --es-port 9200\n\n# First, find your actual index name (pattern: ragflow_{tenant_id})\ncurl -X GET \"http://localhost:9200/_cat/indices/ragflow_*?v\"\n\n# List all knowledge bases in the index\n# Replace ragflow_{tenant_id} with your actual index from the curl output above\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n\n# View sample documents\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_{tenant_id} --size 5\n\n# Check schema\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n```\n\n### Step 4: Start OceanBase for Migration\n\nStart RAGFlow's OceanBase service as the migration target:\n\n```bash\n# Navigate to ragflow docker directory (from ragflow root)\ncd ../docker\n\n# Start only OceanBase service from RAGFlow docker compose\ndocker compose --profile oceanbase up -d\n\n# Wait for OceanBase to be ready\ndocker compose logs -f oceanbase\n```\n\n### Step 5: Run Migration\n\nExecute the migration from Elasticsearch to OceanBase:\n\n```bash\ncd ../tools/es-to-oceanbase-migration\n\n# Option A: Migrate ALL ragflow_* indices (Recommended)\n# If --index and --table are omitted, the tool auto-discovers all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --batch-size 1000 \\\n --verify\n\n# Option B: Migrate a specific index\n# Use the SAME name for both --index and --table\n# The index name pattern is: ragflow_{tenant_id}\n# Find your tenant_id from Step 3's curl output\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --batch-size 1000 \\\n --verify\n```\n\nExpected output:\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_{tenant_id}\nTarget: localhost:2881/ragflow_doc.ragflow_{tenant_id}\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK (version: 4.3.5.1)\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Known RAGFlow fields: 25\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_{tenant_id}' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nStep 5: Verifying migration...\n✓ Document counts match: 1,234\n✓ Sample verification: 100/100 matched\n\nMigration completed successfully!\n Total: 1,234 documents\n Migrated: 1,234 documents\n Failed: 0 documents\n Duration: 45.2 seconds\n```\n\n### Step 6: Stop RAGFlow and Switch to OceanBase Backend\n\n```bash\n# Navigate to ragflow docker directory\ncd ../../docker\n\n# Stop only Elasticsearch and RAGFlow (but keep OceanBase running)\ndocker compose --profile elasticsearch --profile cpu down\n\n# Edit .env file, change:\n# DOC_ENGINE=elasticsearch -> DOC_ENGINE=oceanbase\n#\n# The OceanBase connection settings are already configured by default in .env\n```\n\n### Step 7: Start RAGFlow with OceanBase Backend\n\n```bash\n# OceanBase should still be running from Step 4\n# Start RAGFlow with OceanBase profile (OceanBase is already running)\ndocker compose --profile oceanbase --profile cpu up -d\n\n# Wait for services to start\ndocker compose ps\n\n# Check logs for any errors\ndocker compose logs -f ragflow-cpu\n```\n\n### Step 8: Data Integrity Verification (Optional)\n\nRun the verification command to compare ES and OceanBase data:\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --sample-size 100\n```\n\nExpected output:\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_{tenant_id} │\n│ OB Table: ragflow_{tenant_id} │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n### Step 9: Verify RAGFlow Works with OceanBase\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Navigate to your Knowledge Base\n3. Try the same queries you tested before migration\n\n## CLI Reference\n\n### `es-ob-migrate migrate`\n\nRun data migration from Elasticsearch to OceanBase.\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--es-host` | localhost | Elasticsearch host |\n| `--es-port` | 9200 | Elasticsearch port |\n| `--es-user` | None | ES username (if auth required) |\n| `--es-password` | None | ES password |\n| `--ob-host` | localhost | OceanBase host |\n| `--ob-port` | 2881 | OceanBase port |\n| `--ob-user` | root@test | OceanBase user (format: user@tenant) |\n| `--ob-password` | \"\" | OceanBase password |\n| `--ob-database` | test | OceanBase database name |\n| `-i, --index` | None | Source ES index (omit to migrate all ragflow_* indices) |\n| `-t, --table` | None | Target OB table (omit to use same name as index) |\n| `--batch-size` | 1000 | Documents per batch |\n| `--resume` | False | Resume from previous progress |\n| `--verify/--no-verify` | True | Verify after migration |\n\n**Example:**\n\n```bash\n# Migrate all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc\n\n# Migrate a specific index\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123\n\n# Resume interrupted migration\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --resume\n```\n\n**Resume Feature:**\n\nMigration progress is automatically saved to `.migration_progress/` directory. If migration is interrupted (network error, timeout, etc.), use `--resume` to continue from where it stopped:\n\n- Progress file: `.migration_progress/{index_name}_progress.json`\n- Contains: total count, migrated count, last document ID, timestamp\n- On resume: skips already migrated documents, continues from last position\n\n**Output:**\n\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_abc123\nTarget: localhost:2881/ragflow_doc.ragflow_abc123\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_abc123' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nMigration completed successfully!\n Total: 1,234 documents\n Duration: 45.2 seconds\n```\n\n---\n\n### `es-ob-migrate list-indices`\n\nList all RAGFlow indices (`ragflow_*`) in Elasticsearch.\n\n**Example:**\n\n```bash\nes-ob-migrate list-indices --es-host localhost --es-port 9200\n```\n\n**Output:**\n\n```\nRAGFlow Indices in Elasticsearch:\n\n Index Name Documents Type\n ragflow_abc123def456789 1234 Document Chunks\n ragflow_doc_meta_abc123def456789 56 Document Metadata\n\nTotal: 2 ragflow_* indices found\n```\n\n---\n\n### `es-ob-migrate schema`\n\nPreview schema analysis from ES mapping.\n\n**Example:**\n\n```bash\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nRAGFlow Schema Analysis for index: ragflow_abc123\n\nVector Fields:\n q_1024_vec: dense_vector (dim=1024)\n\nKnown RAGFlow Fields (25):\n id, kb_id, doc_id, docnm_kwd, content_with_weight, content_ltks,\n available_int, important_kwd, question_kwd, tag_kwd, page_num_int...\n\nUnknown Fields (stored in 'extra' column):\n custom_field_1, custom_field_2\n```\n\n---\n\n### `es-ob-migrate verify`\n\nVerify migration data consistency between ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --sample-size 100\n```\n\n**Output:**\n\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_abc123 │\n│ OB Table: ragflow_abc123 │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n---\n\n### `es-ob-migrate list-kb`\n\nList all knowledge bases in an ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nKnowledge Bases in index 'ragflow_abc123':\n\n KB ID Documents\n kb_001_finance_docs 456\n kb_002_technical_manual 321\n kb_003_product_faq 457\n\nTotal: 3 knowledge bases, 1234 documents\n```\n\n---\n\n### `es-ob-migrate sample`\n\nShow sample documents from ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_abc123 --size 2\n```\n\n**Output:**\n\n```\nSample Documents from 'ragflow_abc123':\n\nDocument 1:\n id: chunk_001_abc123\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: The company reported Q3 revenue of $1.2B...\n available_int: 1\n\nDocument 2:\n id: chunk_002_def456\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: Operating expenses decreased by 5%...\n available_int: 1\n```\n\n---\n\n### `es-ob-migrate status`\n\nCheck connection status to ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate status \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\"\n```\n\n**Output:**\n\n```\nConnection Status:\n\nElasticsearch:\n Host: localhost:9200\n Status: ✓ Connected\n Cluster: ragflow-cluster\n Version: 8.11.0\n Indices: 5\n\nOceanBase:\n Host: localhost:2881\n Status: ✓ Connected\n Version: 4.3.5.1\n```\n","tools/firecrawl/INSTALLATION.md":"# Installation Guide for Firecrawl RAGFlow Integration\n\nThis guide will help you install and configure the Firecrawl integration plugin for RAGFlow.\n\n## Prerequisites\n\n- RAGFlow instance running (version 0.20.5 or later)\n- Python 3.8 or higher\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n## Installation Methods\n\n### Method 1: Manual Installation\n\n1. **Download the plugin**:\n ```bash\n git clone https://github.com/firecrawl/firecrawl.git\n cd firecrawl/ragflow-firecrawl-integration\n ```\n\n2. **Install dependencies**:\n ```bash\n pip install -r plugin/firecrawl/requirements.txt\n ```\n\n3. **Copy plugin to RAGFlow**:\n ```bash\n # Assuming RAGFlow is installed in /opt/ragflow\n cp -r plugin/firecrawl /opt/ragflow/plugin/\n ```\n\n4. **Restart RAGFlow**:\n ```bash\n # Restart RAGFlow services\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n### Method 2: Using pip (if available)\n\n```bash\npip install ragflow-firecrawl-integration\n```\n\n### Method 3: Development Installation\n\n1. **Clone the repository**:\n ```bash\n git clone https://github.com/firecrawl/firecrawl.git\n cd firecrawl/ragflow-firecrawl-integration\n ```\n\n2. **Install in development mode**:\n ```bash\n pip install -e .\n ```\n\n## Configuration\n\n### 1. Get Firecrawl API Key\n\n1. Visit [firecrawl.dev](https://firecrawl.dev)\n2. Sign up for a free account\n3. Navigate to your dashboard\n4. Copy your API key (starts with `fc-`)\n\n### 2. Configure in RAGFlow\n\n1. **Access RAGFlow UI**:\n - Open your browser and go to your RAGFlow instance\n - Log in with your credentials\n\n2. **Add Firecrawl Data Source**:\n - Go to \"Data Sources\" → \"Add New Source\"\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify your setup\n - You should see a success message\n\n## Configuration Options\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## Environment Variables\n\nYou can also configure the plugin using environment variables:\n\n```bash\nexport FIRECRAWL_API_KEY=\"fc-your-api-key-here\"\nexport FIRECRAWL_API_URL=\"https://api.firecrawl.dev\"\nexport FIRECRAWL_MAX_RETRIES=\"3\"\nexport FIRECRAWL_TIMEOUT=\"30\"\nexport FIRECRAWL_RATE_LIMIT_DELAY=\"1.0\"\n```\n\n## Verification\n\n### 1. Check Plugin Installation\n\n```bash\n# Check if the plugin directory exists\nls -la /opt/ragflow/plugin/firecrawl/\n\n# Should show:\n# __init__.py\n# firecrawl_connector.py\n# firecrawl_config.py\n# firecrawl_processor.py\n# firecrawl_ui.py\n# ragflow_integration.py\n# requirements.txt\n```\n\n### 2. Test the Integration\n\n```bash\n# Run the example script\ncd /opt/ragflow/plugin/firecrawl/\npython example_usage.py\n```\n\n### 3. Check RAGFlow Logs\n\n```bash\n# Check RAGFlow server logs\ndocker logs docker-ragflow-cpu-1\n\n# Look for messages like:\n# \"Firecrawl plugin loaded successfully\"\n# \"Firecrawl data source registered\"\n```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Plugin not appearing in RAGFlow**:\n - Check if the plugin directory is in the correct location\n - Restart RAGFlow services\n - Check RAGFlow logs for errors\n\n2. **API Key Invalid**:\n - Ensure your API key starts with `fc-`\n - Verify the key is active in your Firecrawl dashboard\n - Check for typos in the configuration\n\n3. **Connection Timeout**:\n - Increase the timeout value in configuration\n - Check your network connection\n - Verify the API URL is correct\n\n4. **Rate Limiting**:\n - Increase the `rate_limit_delay` value\n - Reduce the number of concurrent requests\n - Check your Firecrawl usage limits\n\n### Debug Mode\n\nEnable debug logging to see detailed information:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.DEBUG)\n```\n\n### Check Dependencies\n\n```bash\n# Verify all dependencies are installed\npip list | grep -E \"(aiohttp|pydantic|requests)\"\n\n# Should show:\n# aiohttp>=3.8.0\n# pydantic>=2.0.0\n# requests>=2.28.0\n```\n\n## Uninstallation\n\nTo remove the plugin:\n\n1. **Remove plugin directory**:\n ```bash\n rm -rf /opt/ragflow/plugin/firecrawl/\n ```\n\n2. **Restart RAGFlow**:\n ```bash\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n3. **Remove dependencies** (optional):\n ```bash\n pip uninstall ragflow-firecrawl-integration\n ```\n\n## Support\n\nIf you encounter issues:\n\n1. Check the [troubleshooting section](#troubleshooting)\n2. Review RAGFlow logs for error messages\n3. Verify your Firecrawl API key and configuration\n4. Check the [Firecrawl documentation](https://docs.firecrawl.dev)\n5. Open an issue in the [Firecrawl repository](https://github.com/firecrawl/firecrawl/issues)\n\n## Next Steps\n\nAfter successful installation:\n\n1. Read the [README.md](README.md) for usage examples\n2. Try scraping a simple URL to test the integration\n3. Explore the different scraping options (single URL, crawl, batch)\n4. Configure your RAGFlow workflows to use the scraped content\n","docs/references/python_api_reference.md":"---\nsidebar_position: 5\nslug: /python_api_reference\nsidebar_custom_props: {\n categoryIcon: SiPython\n}\n---\n# Python API\n\nA complete reference for RAGFlow's Python APIs. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).\n\n:::tip NOTE\nRun the following command to download the Python SDK:\n\n```bash\npip install ragflow-sdk\n```\n\n:::\n\n---\n\n## ERROR CODES\n\nRAGFlow responses may contain both an HTTP status code and a business code in the JSON response body. These codes should be checked separately.\n\n### HTTP status codes\n\n| Code | Meaning |\n|------|---------|\n| 200 | The HTTP request was processed successfully. Check the response body `code` for the business result. |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Internal server error |\n\n### Response body codes\n\n| Code | Meaning |\n|------|---------|\n| 0 | Success |\n| 10 | Not effective |\n| 100 | Exception error |\n| 101 | Invalid request argument |\n| 102 | Invalid or missing data |\n| 103 | Operation error |\n| 105 | Connection error |\n| 106 | Operation still running |\n| 108 | Permission error |\n| 109 | Authentication error |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Server error |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\nCreates a model response for the given historical chat conversation via OpenAI's API.\n\n#### Parameters\n\n##### chat_id: `string`, *Required*\n\nExisting chat assistant ID. This value is part of the request path: `/api/v1/openai//chat/completions`.\n\n##### model: `string`, *Required*\n\nThe model used to generate the response. You may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n##### messages: `list[object]`, *Required*\n\nA list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n##### stream: `boolean`\n\nWhether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n#### Returns\n\n- Success: Response [message](https://platform.openai.com/docs/api-reference/chat/create) like OpenAI\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom openai import OpenAI\nimport json\n\nmodel = \"glm-4-flash@ZHIPU-AI\"\nclient = OpenAI(api_key=\"ragflow-api-key\", base_url=\"http://ragflow_address/api/v1/openai//chat\")\n\nstream = True\nreference = True\n\nrequest_kwargs = dict(\n model=model,\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Who are you?\"},\n {\"role\": \"assistant\", \"content\": \"I am an AI assistant named...\"},\n {\"role\": \"user\", \"content\": \"Can you tell me how to install neovim\"},\n ],\n extra_body={\n \"reference\": reference,\n \"reference_metadata\": {\n \"include\": True,\n \"fields\": [\"author\", \"year\", \"source\"],\n },\n },\n)\n\nif stream:\n completion = client.chat.completions.create(stream=True, **request_kwargs)\n for chunk in completion:\n print(chunk)\nelse:\n resp = client.chat.completions.with_raw_response.create(\n stream=False, **request_kwargs\n )\n print(\"status:\", resp.http_response.status_code)\n raw_text = resp.http_response.text\n print(\"raw:\", raw_text)\n\n data = json.loads(raw_text)\n print(\"assistant:\", data[\"choices\"][0][\"message\"].get(\"content\"))\n print(\"reference:\", data[\"choices\"][0][\"message\"].get(\"reference\"))\n```\n\nWhen `extra_body.reference` is `true`, the streamed final chunk may include `choices[0].delta.reference`, and the non-stream response may include `choices[0].message.reference`.\n\nWhen `extra_body.reference_metadata.include` is `true`, each reference chunk may include a `document_metadata` object in both streaming and non-streaming responses.\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n```python\nRAGFlow.create_dataset(\n name: str,\n parse_type: int = 1,\n parser_id: str | None = None,\n pipeline_id: str | None = None,\n embedding_model: Optional[str] = None,\n permission: str = \"me\",\n parser_config: Optional[DataSet.ParserConfig] = None,\n language: Optional[str] = None,\n) -> DataSet\n```\n\nCreates a dataset.\n\n`avatar`, `description`, `auto_metadata_config`, and `chunk_method` are not accepted by the RAGFlow 1.0 create endpoint. Set avatar and description through `DataSet.update()`, and metadata configuration through `DataSet.update_auto_metadata()`.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe unique name of the dataset to create. It must adhere to the following requirements:\n\n- Maximum 128 characters.\n- Case-insensitive.\n\n##### parse_type: `int`\n\nSelects the parser mode. Defaults to `1`.\n\n- `1`: BuiltIn. Use the builtin parser selected by `parser_id`.\n- `2`: Pipeline. Use the ingestion pipeline selected by `pipeline_id`.\n\n##### parser_id: `string | None`\n\nThe builtin parser ID. Defaults to `\"general\"` when `parse_type` is `1`.\n\nAvailable values include `\"general\"`, `\"qa\"`, `\"manual\"`, `\"table\"`, `\"paper\"`, `\"book\"`, `\"laws\"`, `\"presentation\"`, `\"picture\"`, `\"one\"`, `\"audio\"`, and `\"email\"`.\n\nMust be omitted when `parse_type` is `2`.\n\n##### pipeline_id: `string | None`\n\nThe ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n\n##### embedding_model: `string | None`\n\nThe embedding model to use for the dataset. Defaults to `None`.\n\n##### permission\n\nSpecifies who can access the dataset to create. Available options:\n\n- `\"me\"`: (Default) Only you can manage the dataset.\n- `\"team\"`: All team members can manage the dataset.\n\n##### language: `string | None`\n\nOptional document/dataset language, for example `\"English\"` or `\"Chinese\"`. Defaults to `None`.\n\n##### parser_config: `DataSet.ParserConfig | None`\n\nThe parser configuration of the dataset. Defaults to `None`.\n\nRAGFlow 1.0 stores parser settings per ingestion-pipeline component. Use component-scoped keys (keys containing `:`) as returned by the API; legacy flat keys are ignored.\n\n#### Returns\n\n- Success: A `dataset` object. Its `parser_id`, `pipeline_id`, and derived `parse_type` reflect the selected parser mode.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"kb_1\")\ndataset = rag_object.create_dataset(name=\"kb_table\", parse_type=1, parser_id=\"table\")\ndataset = rag_object.create_dataset(\n name=\"kb_pipeline\",\n parse_type=2,\n pipeline_id=\"\",\n)\n```\n\n---\n\n### Delete datasets\n\n```python\nRAGFlow.delete_datasets(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes datasets by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the datasets to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no datasets are deleted.\n- If an array of IDs is provided, only the datasets matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all datasets owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object.delete_datasets(ids=[\"d94a8dc02c9711f0930f7fbc369eab6d\",\"e94a8dc02c9711f0930f7fbc369eab6e\"])\nrag_object.delete_datasets(delete_all=True)\n```\n\n---\n\n### List datasets\n\n```python\nRAGFlow.list_datasets(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n) -> list[DataSet]\n```\n\nLists datasets.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the datasets will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of datasets on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which datasets should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved datasets should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the dataset to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the dataset to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the datasets to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n#### Returns\n\n- Success: A list of `DataSet` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n##### List all datasets\n\n```python\nfor dataset in rag_object.list_datasets():\n print(dataset)\n```\n\n##### Retrieve a dataset by ID\n\n```python\ndataset = rag_object.list_datasets(id = \"id_1\")\nprint(dataset[0])\n```\n\n##### Retrieve datasets by IDs\n\n```python\ndatasets = rag_object.list_datasets(ids=[\"id_1\", \"id_2\"])\nfor dataset in datasets:\n print(dataset)\n```\n\n---\n\n### Update dataset\n\n```python\nDataSet.update(update_message: dict)\n```\n\nUpdates configurations for the current dataset.\n\n#### Parameters\n\n##### update_message: `dict[str, str|int]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"name\"`: `string` The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"description\"`: (*Body parameter*), `string`\n The updated dataset description.\n - Maximum 65535 characters\n- `\"language\"`: (*Body parameter*), `string`\n The updated document/dataset language, for example `\"English\"` or `\"Chinese\"`.\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration)\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser mode: `1` for a builtin parser or `2` for an ingestion pipeline. Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`.\n\n`chunk_method` is not supported by the RAGFlow 1.0 SDK; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_name\")\ndataset = dataset[0]\ndataset.update({\"embedding_model\":\"BAAI/bge-zh-v1.5\", \"parse_type\":1, \"parser_id\":\"manual\"})\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n```python\nDataSet.upload_documents(document_list: list[dict])\n```\n\nUploads documents to the current dataset.\n\n#### Parameters\n\n##### document_list: `list[dict]`, *Required*\n\nA list of dictionaries representing the documents to upload, each containing the following keys:\n\n- `\"display_name\"`: `string`, *Required*\n The file name to display in the dataset.\n- `\"blob\"`: `bytes`, *Required*\n The binary content of the file to upload.\n\n#### Returns\n\n- Success: A list of uploaded `Document` objects.\n- Failure: `Exception`\n\n#### Examples\n\n```python\ndataset = rag_object.create_dataset(name=\"kb_name\")\n\nwith open(\"1.txt\", \"rb\") as file:\n documents = dataset.upload_documents([\n {\n \"display_name\": \"1.txt\",\n \"blob\": file.read(),\n }\n ])\n\nprint(documents[0].id)\n```\n\n---\n\n### Update document\n\n```python\nDocument.update(update_message: dict)\n```\n\nUpdates configurations for the current document.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"display_name\"`: `string` The name of the document to update.\n- `\"meta_fields\"`: `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: `string` The parsing method to apply to the document.\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: `dict[str, Any]` The parsing configuration for the document. Its attributes vary based on the selected `\"chunk_method\"`:\n - `\"chunk_method\"`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False},\"parent_child\":{\"use_parent_child\":False,\"children_delimiter\":\"\\n\"}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Returns\n\n- Success: The updated `Document` object.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id='id')\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\ndoc.update({\n \"parser_config\": {\n \"chunk_token_num\": 256,\n },\n \"chunk_method\": \"manual\",\n})\n```\n\n---\n\n### Download document\n\n```python\nDocument.download() -> bytes\n```\n\nDownloads the current document.\n\n#### Returns\n\nThe downloaded document in bytes.\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"https://\",\n)\ndataset = rag_object.list_datasets(id=\"id\")[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")[0]\n\nPath(\"~/ragflow.txt\").expanduser().write_bytes(doc.download())\n```\n\n---\n\n### List documents\n\n```python\nDataSet.list_documents(\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n keywords: str | None = None,\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n create_time_from: int = 0,\n create_time_to: int = 0\n)\n```\n\nLists documents in the current dataset.\n\n#### Parameters\n\n##### id: `string | None`\n\nThe ID of the document to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the documents to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n##### name: `string | None`\n\nThe exact name of the document to retrieve. Defaults to `None`.\n\n##### keywords: `string | None`\n\nThe keywords used to match document titles. Defaults to `None`.\n\n##### page: `int`\n\nSpecifies the page on which the documents will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of documents on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which documents should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved documents should be sorted in descending order. Defaults to `True`.\n\n##### create_time_from: `int`\nUnix timestamp for filtering documents created after this time. 0 means no filter. Defaults to 0.\n\n##### create_time_to: `int`\nUnix timestamp for filtering documents created before this time. 0 means no filter. Defaults to 0.\n\n#### Returns\n\n- Success: A list of `Document` objects.\n- Failure: `Exception`.\n\nA `Document` object contains the following attributes:\n\n- `id`: The document ID. Defaults to `\"\"`.\n- `name`: The document name. Defaults to `\"\"`.\n- `thumbnail`: The thumbnail image of the document. Defaults to `None`.\n- `dataset_id`: The dataset ID associated with the document. Defaults to `None`.\n- `chunk_method` The chunking method name. Defaults to `\"naive\"`.\n- `source_type`: The source type of the document. Defaults to `\"local\"`.\n- `type`: Type or category of the document. Defaults to `\"\"`. Reserved for future use.\n- `created_by`: `string` The creator of the document. Defaults to `\"\"`.\n- `size`: `int` The document size in bytes. Defaults to `0`.\n- `token_count`: `int` The number of tokens in the document. Defaults to `0`.\n- `chunk_count`: `int` The number of chunks in the document. Defaults to `0`.\n- `progress`: `float` The current processing progress as a percentage. Defaults to `0.0`.\n- `progress_msg`: `string` A message indicating the current progress status. Defaults to `\"\"`.\n- `process_begin_at`: `datetime` The start time of document processing. Defaults to `None`.\n- `process_duration`: `float` Duration of the processing in seconds. Defaults to `0.0`.\n- `run`: `string` The document's processing status:\n - `\"UNSTART\"` (default)\n - `\"RUNNING\"`\n - `\"CANCEL\"`\n - `\"DONE\"`\n - `\"FAIL\"`\n- `status`: `string` Reserved for future use.\n- `parser_config`: `ParserConfig` Configuration object for the parser. Its attributes vary based on the selected `chunk_method`:\n - `chunk_method`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"http://:9380\",\n)\ndataset = rag_object.create_dataset(name=\"kb_1\")\n\nfile_path = Path(\"~/ragflow.txt\").expanduser()\ndataset.upload_documents([\n {\n \"display_name\": file_path.name,\n \"blob\": file_path.read_bytes(),\n }\n])\n\nfor doc in dataset.list_documents(\n keywords=\"rag\",\n page=1,\n page_size=12,\n):\n print(doc)\n```\n\n---\n\n### Delete documents\n\n```python\nDataSet.delete_documents(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes documents by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the documents to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no documents are deleted.\n- If an array of IDs is provided, only the documents matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all documents in the current dataset when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_1\")\ndataset = dataset[0]\ndataset.delete_documents(ids=[\"id_1\",\"id_2\"])\ndataset.delete_documents(delete_all=True)\n```\n\n---\n\n### Parse documents\n\n```python\nDataSet.async_parse_documents(document_ids:list[str]) -> None\n```\n\nParses documents in the current dataset.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\n```\n\n---\n\n### Parse documents (with document status)\n\n```python\nDataSet.parse_documents(document_ids: list[str]) -> list[tuple[str, str, int, int]]\n```\n\nStarts parsing documents in the current dataset and synchronously waits for the results.\n\nThis method calls `async_parse_documents()` and blocks while polling until all requested documents reach a terminal state or report complete progress. It then returns the parsing status and statistics for each document. If a keyboard interruption occurs (e.g., `Ctrl+C`), it requests cancellation for the requested documents and continues polling for their final statuses. If a status request fails or a requested document is no longer found, the method raises an exception instead of continuing to poll.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\nA list of tuples with detailed parsing results:\n\n```python\n[\n (document_id: str, status: str, chunk_count: int, token_count: int),\n ...\n]\n```\n- `status`: The final parsing state (e.g., `DONE`, `FAIL`, `CANCEL`). If a document has not reached a terminal state but reports `progress >= 1.0`, its status is returned as `DONE`.\n- `chunk_count`: The number of content chunks created from the document.\n- `token_count`: The total number of tokens processed.\n\n---\n\n#### Example\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = dataset.list_documents(keywords=\"test\")\nids = [doc.id for doc in documents]\n\ntry:\n finished = dataset.parse_documents(ids)\n for doc_id, status, chunk_count, token_count in finished:\n print(f\"Document {doc_id} parsing finished with status: {status}, chunks: {chunk_count}, tokens: {token_count}\")\nexcept Exception as e:\n print(f\"Parsing failed: {e}\")\n```\n\n---\n\n### Stop parsing documents\n\n```python\nDataSet.async_cancel_parse_documents(document_ids:list[str])-> None\n```\n\nStops parsing specified documents.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents for which parsing should be stopped.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\ndataset.async_cancel_parse_documents(ids)\nprint(\"Async bulk parsing cancelled.\")\n```\n\n---\n\n## CHUNK MANAGEMENT WITHIN DATASET\n\n---\n\n### Add chunk\n\n```python\nDocument.add_chunk(content:str, important_keywords:list[str] = [], questions:list[str] = [], image_base64:str = None, *, tag_kwd:list[str] = []) -> Chunk\n```\n\nAdds a chunk to the current document.\n\n#### Parameters\n\n##### content: `string`, *Required*\n\nThe text content of the chunk.\n\n##### important_keywords: `list[str]`\n\nThe key terms or phrases to tag with the chunk.\n\n##### questions: `list[str]`\n\nOptional questions to use when embedding the chunk.\n\n##### image_base64: `string`\n\nA base64-encoded image to associate with the chunk. If the chunk already has an image, the new image will be vertically concatenated below the existing one.\n\n##### tag_kwd: `list[str]`\n\nTag keywords to associate with the chunk.\n\n#### Returns\n\n- Success: A `Chunk` object.\n- Failure: `Exception`.\n\nA `Chunk` object contains the following attributes:\n\n- `id`: `string`: The chunk ID.\n- `content`: `string` The text content of the chunk.\n- `important_keywords`: `list[str]` A list of key terms or phrases tagged with the chunk.\n- `tag_kwd`: `list[str]` A list of tag keywords associated with the chunk.\n- `questions`: `list[str]` A list of questions associated with the chunk.\n- `image_id`: `string` The image ID associated with the chunk (empty string if no image).\n- `create_time`: `string` The time when the chunk was created (added to the document).\n- `create_timestamp`: `float` The timestamp representing the creation time of the chunk, expressed in seconds since January 1, 1970.\n- `dataset_id`: `string` The ID of the associated dataset.\n- `document_name`: `string` The name of the associated document.\n- `document_id`: `string` The ID of the associated document.\n- `available`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(id=\"123\")\ndataset = datasets[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\n```\n\nAdding a chunk with an image:\n\n```python\nimport base64\n\nwith open(\"image.jpg\", \"rb\") as f:\n img_b64 = base64.b64encode(f.read()).decode()\nchunk = doc.add_chunk(content=\"description of image\", image_base64=img_b64)\n```\n\n---\n\n### List chunks\n\n```python\nDocument.list_chunks(keywords: str = None, page: int = 1, page_size: int = 30, id : str = None) -> list[Chunk]\n```\n\nLists chunks in the current document.\n\n#### Parameters\n\n##### keywords: `string`\n\nThe keywords used to match chunk content. Defaults to `None`\n\n##### page: `int`\n\nSpecifies the page on which the chunks will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks on each page. Defaults to `30`.\n\n##### id: `string`\n\nThe ID of the chunk to retrieve. Default: `None`\n\n#### Returns\n\n- Success: A list of `Chunk` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(\"123\")\ndataset = dataset[0]\ndocs = dataset.list_documents(keywords=\"test\", page=1, page_size=12)\nfor chunk in docs[0].list_chunks(keywords=\"rag\", page=0, page_size=12):\n print(chunk)\n```\n\n---\n\n### Delete chunks\n\n```python\nDocument.delete_chunks(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chunks by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chunks to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chunks are deleted.\n- If an array of IDs is provided, only the chunks matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chunks in the current document when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\ndoc.delete_chunks([\"id_1\",\"id_2\"])\ndoc.delete_chunks(delete_all=True)\n```\n\n---\n\n### Update chunk\n\n```python\nChunk.update(update_message: dict)\n```\n\nUpdates content or configurations for the current chunk.\n\n#### Parameters\n\n##### update_message: `dict[str, str|list[str]|bool]` *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"content\"`: `string` The text content of the chunk.\n- `\"important_keywords\"`: `list[str]` A list of key terms or phrases to tag with the chunk.\n- `\"questions\"`: `list[str]` A list of questions associated with the chunk.\n- `\"tag_kwd\"`: `list[str]` A list of tag keywords to associate with the chunk.\n- `\"positions\"`: `list` Updated source positions for the chunk.\n- `\"available\"`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n- `\"image_base64\"`: `string` Base64-encoded image content to associate with the chunk.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\nchunk.update({\"content\":\"sdfx...\"})\n```\n\n---\n\n### Retrieve chunks\n\n```python\nRAGFlow.retrieve(\n dataset_ids,\n document_ids=None,\n question='',\n page=1,\n page_size=30,\n similarity_threshold=0.2,\n vector_similarity_weight=0.3,\n top_k=1024,\n rerank_id: str | None = None,\n keyword: bool = False,\n cross_languages: list[str] | None = None,\n metadata_condition: dict | None = None)\n```\n\nRetrieves chunks from specified datasets.\n\n#### Parameters\n\n##### question: `string`\n\nThe user query or query keywords. Defaults to `\"\"`. When an empty string is provided, the API returns an empty retrieval result.\n\n##### dataset_ids: `list[str]`, *Required*\n\nThe IDs of the datasets to search. At least one dataset ID must be provided.\n\n##### document_ids: `list[str]`\n\nThe IDs of the documents to search. Defaults to `None`. You must ensure all selected documents use the same embedding model. Otherwise, an error will occur.\n\n##### page: `int`\n\nThe page number of the chunk retrieval results. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks returned on each page. Defaults to `30`.\n\n##### similarity_threshold: `float`\n\nThe minimum similarity score. Defaults to `0.2`.\n\n##### vector_similarity_weight: `float`\n\nThe weight of vector cosine similarity. Defaults to `0.3`. If x represents the vector cosine similarity, then (1 - x) is the term similarity weight.\n\n##### top_k: `int`\n\nThe number of chunks engaged in vector cosine computation. Defaults to `1024`.\n\n##### rerank_id: `string`\n\nThe ID of the rerank model. Defaults to `None`.\n\n##### keyword: `bool`\n\nIndicates whether to enable keyword-based matching:\n\n- `True`: Enable keyword-based matching.\n- `False`: Disable keyword-based matching (default).\n\n##### cross_languages: `list[string]`\n\nThe languages that should be translated into, in order to achieve keywords retrievals in different languages.\n\n##### metadata_condition: `dict`\n\nfilter condition for `meta_fields`.\n\n#### Returns\n\n- Success: A list of `Chunk` objects representing the document chunks.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"ragflow\")\ndataset = dataset[0]\nname = 'ragflow_test.txt'\npath = './test_data/ragflow_test.txt'\ndocuments =[{\"display_name\":\"test_retrieve_chunks.txt\",\"blob\":open(path, \"rb\").read()}]\ndocs = dataset.upload_documents(documents)\ndoc = docs[0]\ndoc.add_chunk(content=\"This is a chunk addition test\")\nfor c in rag_object.retrieve(dataset_ids=[dataset.id],document_ids=[doc.id]):\n print(c)\n```\n\n---\n\n## CHAT ASSISTANT MANAGEMENT\n\n---\n\n### Create chat assistant\n\n```python\nRAGFlow.create_chat(\n name: str,\n icon: str = \"\",\n dataset_ids: list[str] | None = None,\n llm_id: str | None = None,\n llm_setting: dict | None = None,\n prompt_config: dict | None = None,\n **kwargs\n) -> Chat\n```\n\nCreates a chat assistant.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe name of the chat assistant.\n\n##### icon: `string`\n\nBase64 encoding of the avatar. Defaults to `\"\"`.\n\n##### dataset_ids: `list[str]`\n\nThe IDs of the associated datasets. Defaults to `[]`. When omitted or empty, the SDK creates an empty chat assistant, and you can attach datasets later.\n\n##### llm_id: `str | None`\n\nThe LLM model name/ID to use. If `None`, the user’s default chat model is used. Defaults to `None`.\n\n##### llm_setting: `dict | None`\n\nConfiguration for LLM generation parameters. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"temperature\"`: `float` Controls the randomness of the model's output. Higher values increase creativity, while lower values make responses more deterministic. Defaults to `0.1`.\n- `\"top_p\"`: `float` Sets the nucleus sampling threshold. The model considers only the results of the tokens with `top_p` probability mass. Defaults to `0.3`.\n- `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have appeared in the text so far, increasing the likelihood of the model talking about new topics. Defaults to `0.4`.\n- `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text, decreasing the likelihood of repeating the same lines. Defaults to `0.7`.\n- `\"max_token\"`: `int` The maximum number of tokens to generate in the response. Defaults to `512`.\n\n##### prompt_config: `dict | None`\n\nInstructions and behavioral settings for the LLM. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n- `\"empty_response\"`: `string` The specific message returned when no relevant information is retrieved. If left blank, the LLM will generate its own response. Defaults to `None`.\n- `\"prologue\"`: `string` The initial greeting displayed to the user. Defaults to `\"Hi! I’m your assistant. What can I do for you?\"`.\n- `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references in its responses. Defaults to `True`.\n- `\"parameters\"`: `list[dict]` A list of variables utilized within the system prompt. Each entry must include a `\"key\"` (`string`) and an `\"optional\"` (`boolean`) status. The `knowledge` key is reserved for retrieved context chunks. Default: `[{\"key\": \"knowledge\", \"optional\": true}]`.\n\n#### Returns\n\n- Success: A `Chat` object representing the chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_ids = []\nfor dataset in datasets:\n dataset_ids.append(dataset.id)\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=dataset_ids)\n```\n\n---\n\n### Update chat assistant\n\n```python\nChat.update(update_message: dict)\n```\n\nPerforms a partial update to the configuration settings for the current chat assistant.\n\n`Chat.update()` utilizes the `PATCH /api/v1/chats/{chat_id}` endpoint. Only the specified keys are modified, while all other existing fields are preserved.\n\n#### Parameters\n\n##### update_message: `dict`, *Required*\n\nA dictionary containing the attributes to be updated. Supported keys include:\n\n- `\"name\"`: `string` The updated name of the chat assistant.\n- `\"icon\"`: `string` A Base64-encoded string representing the assistant's avatar.\n- `\"dataset_ids\"`: `list[string]` A list of unique identifiers for the datasets associated with the assistant.\n- `\"llm_id\"`: `string` The unique identifier or name of the LLM to be used.\n- `\"llm_setting\"`: `dict` Configuration for LLM generation parameters:\n - `\"temperature\"`: `float` Controls the randomness of the model's output.\n - `\"top_p\"`: `float` Sets the nucleus sampling threshold.\n - `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have already appeared in the text.\n - `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text.\n - `\"max_token\"`: `int` The maximum number of tokens to generate in the response.\n- `\"prompt_config\"`: `dict` Instructions and behavioral settings for the LLM:\n - `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n - `\"empty_response\"`: `string` The message returned when no relevant information is retrieved. Leave blank to allow the LLM to improvise.\n - `\"prologue\"`: `string` The initial greeting displayed to the user.\n - `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references.\n - `\"parameters\"`: `list[dict]` Variables used within the system prompt (e.g., the reserved `knowledge` key).\n- `\"similarity_threshold\"`: `float` The minimum similarity score required for retrieved context chunks. Defaults to `0.2`.\n- `\"vector_similarity_weight\"`: `float` The weight assigned to vector cosine similarity within the hybrid search score. Defaults to `0.3`.\n- `\"top_n\"`: `int` The number of top-ranked chunks provided to the LLM as context. Defaults to `6`.\n- `\"top_k\"`: `int` The size of the initial candidate pool retrieved for reranking. Defaults to `1024`.\n- `\"rerank_id\"`: `string` The unique identifier for the reranking model. If left empty, standard vector cosine similarity is used for ranking.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_id = datasets[0].id\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=[dataset_id])\nassistant.update({\"name\": \"Stefan\", \"llm_setting\": {\"temperature\": 0.8}, \"top_n\": 8})\n```\n\n---\n\n### Delete chat assistants\n\n```python\nRAGFlow.delete_chats(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chat assistants by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chat assistants to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chat assistants are deleted.\n- If an array of IDs is provided, only the chat assistants matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chat assistants owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nrag_object.delete_chats(ids=[\"id_1\",\"id_2\"])\nrag_object.delete_chats(delete_all=True)\n```\n\n---\n\n### List chat assistants\n\n```python\nRAGFlow.list_chats(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n name: str | None = None,\n keywords: str | None = None,\n owner_ids: str | list[str] | None = None,\n) -> list[Chat]\n```\n\nLists chat assistants.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the chat assistants will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of chat assistants on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe attribute by which the results are sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved chat assistants should be sorted in descending order. Defaults to `True`.\n\n##### id: `string | None`\n\nExact match on chat assistant ID. Defaults to `None`.\n\n##### name: `string | None`\n\nFilters results by the exact name of the chat assistant. Defaults to `None`.\n\n##### keywords: `string | None`\n\nPerforms a case-insensitive fuzzy search against chat assistant names. Defaults to `None`.\n\n##### owner_ids: `string | list[string] | None`\n\nFilters results by one or more owner tenant IDs. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Chat` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nfor assistant in rag_object.list_chats():\n print(assistant)\n```\n\n---\n\n## SESSION MANAGEMENT\n\n---\n\n### Create session with chat assistant\n\n```python\nChat.create_session(name: str = \"New session\") -> Session\n```\n\nCreates a session with the current chat assistant.\n\n#### Parameters\n\n##### name: `string`\n\nThe name of the chat session to create.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `name`: `string` The name of the created session.\n - `message`: `list[Message]` The opening message of the created session. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `chat_id`: `string` The ID of the associated chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session()\n```\n\n---\n\n### Update chat assistant's session\n\n```python\nSession.update(update_message: dict)\n```\n\nUpdates the current session of the current chat assistant.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with only one key:\n\n- `\"name\"`: `string` The revised name of the session.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session(\"session_name\")\nsession.update({\"name\": \"updated_name\"})\n```\n\n---\n\n### List chat assistant's sessions\n\n```python\nChat.list_sessions(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str = None,\n name: str = None,\n user_id: str = None\n) -> list[Session]\n```\n\nLists sessions associated with the current chat assistant.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the sessions will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of sessions on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which sessions should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved sessions should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the chat session to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the chat session to retrieve. Defaults to `None`.\n\n##### user_id: `str`\n\nThe optional user-defined ID to filter sessions by. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Session` objects associated with the current chat assistant.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nfor session in assistant.list_sessions():\n print(session)\n```\n\n---\n\n### Delete chat assistant's sessions\n\n```python\nChat.delete_sessions(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes sessions of the current chat assistant by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the sessions to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no sessions are deleted.\n- If an array of IDs is provided, only the sessions matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all sessions of the current chat assistant when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nassistant.delete_sessions(ids=[\"id_1\",\"id_2\"])\nassistant.delete_sessions(delete_all=True)\n```\n\n---\n\n### Create session with agent\n\n```python\nAgent.create_session(**kwargs) -> Session\n```\n\nCreates a session with the current agent.\n\n#### Parameters\n\n##### **kwargs\n\nThe parameters in `begin` component.\n\nAlso supports:\n\n- `release` (`bool | str`, optional): When set to `True` (or `\"true\"`), creates a session with the published agent app only.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `message`: `list[Message]` The messages of the created session assistant. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `agent_id`: `string` The ID of the associated agent.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nagent_id = \"AGENT_ID\"\nagent = rag_object.get_agent(agent_id)\nsession = agent.create_session()\n# Or create in release mode:\n# session = agent.create_session(release=True)\n```\n\n---\n\n### Converse with agent\n\n```python\nSession.ask(question: str = \"\", stream: bool = False, **kwargs) -> Optional[Message | iter[Message]]\n```\n\nAsks a specified agent through the unified completion endpoint.\n\n:::tip NOTE\nIn streaming mode, not all responses include a reference, as this depends on the system's judgment.\n:::\n\n#### Parameters\n\n##### question: `string`\n\nThe user message sent to the agent. If the **Begin** component takes parameters, `question` can be an empty string.\n\n##### stream: `bool`\n\nIndicates whether to output responses in a streaming way:\n\n- `True`: Enable streaming.\n- `False`: Disable streaming.\n\n##### kwargs: `dict`\n\nAdditional request parameters forwarded to the completion API. Common options:\n\n- `inputs`: Variables defined in the **Begin** component.\n- `session_id`: Continue an existing session instead of creating a new one.\n- `release`: Use the latest published version of the agent.\n- `return_trace`: Include execution trace information in the response.\n- Other custom Begin component parameters supported by the current workflow.\n\n#### Returns\n\n- A `Message` object containing the response to the question if `stream` is set to `False`\n- An iterator containing multiple `message` objects (`iter[Message]`) if `stream` is set to `True`\n\nThe following shows the attributes of a `Message` object:\n\n##### id: `string`\n\nThe auto-generated message ID.\n\n##### content: `string`\n\nThe content of the message. Defaults to `\"Hi! I am your assistant, can I help you?\"`.\n\n##### reference: `list[Chunk]`\n\nA list of `Chunk` objects representing references to the message, each containing the following attributes:\n\n- `id` `string`\n The chunk ID.\n- `content` `string`\n The content of the chunk.\n- `image_id` `string`\n The ID of the snapshot of the chunk. Applicable only when the source of the chunk is an image, PPT, PPTX, or PDF file.\n- `document_id` `string`\n The ID of the referenced document.\n- `document_name` `string`\n The name of the referenced document.\n- `document_metadata` `dict`\n Optional document metadata, returned only when `extra_body.reference_metadata.include` is `true`.\n- `position` `list[str]`\n The location information of the chunk within the referenced document.\n- `dataset_id` `string`\n The ID of the dataset to which the referenced document belongs.\n- `similarity` `float`\n A composite similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity. It is the weighted sum of `vector_similarity` and `term_similarity`.\n- `vector_similarity` `float`\n A vector similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between vector embeddings.\n- `term_similarity` `float`\n A keyword similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between keywords.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nAGENT_id = \"AGENT_ID\"\nagent = rag_object.get_agent(AGENT_id)\nsession = agent.create_session()\n\nprint(\"\\n===== Miss R ====\\n\")\nprint(\"Hello. What can I do for you?\")\n\nwhile True:\n question = input(\"\\n===== User ====\\n> \")\n print(\"\\n==== Miss R ====\\n\")\n\n cont = \"\"\n for ans in session.ask(question, stream=True):\n print(ans.content[len(cont):], end='', flush=True)\n cont = ans.content\n```\n\nUse Begin inputs and request trace output:\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://\"`. For the general parser, send `\"parser_id\": \"general\"`. |\n| Dataset parser configuration | Go stores and returns component-scoped parser settings. Most legacy top-level `parser_config` fields are not mapped into that component structure. | Read the returned component map and send component-scoped keys when changing parser settings. |\n| Errors and authentication | Some validation business codes, messages, and authentication HTTP statuses differ from Python. | Branch on documented codes rather than exact message text. |\n\n:::note\nGo responses include `X-API-Source: go`, which can be used to verify the serving backend during rollout.\n:::\n\n### Removed API aliases\n\nThe following paths remain available only on the Python backend. They are not registered by the RAGFlow 1.0 Go backend.\n\n| Removed endpoint | Replacement endpoint |\n|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|\n| **POST** `/api/v1/chats_openai/{chat_id}/chat/completions` | **POST** `/api/v1/openai/{chat_id}/chat/completions` |\n| **PUT** `/api/v1/chats/{chat_id}/sessions/{session_id}` | **PATCH** `/api/v1/chats/{chat_id}/sessions/{session_id}` |\n| **POST** `/api/v1/chats/{chat_id}/completions` | **POST** `/api/v1/chat/completions` |\n| **POST** `/api/v1/sessions/related_questions` | **POST** `/api/v1/chat/recommendation` |\n| **PUT** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` | **PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` |\n| **GET** `/v1/system/healthz` | **GET** `/api/v1/system/healthz` |\n| **POST** `/v1/document/upload_info` | **POST** `/api/v1/documents/upload` |\n| **POST** `/api/v1/file/upload` | **POST** `/api/v1/files` |\n| **POST** `/api/v1/file/create` | **POST** `/api/v1/files` |\n| **GET** `/api/v1/file/list` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/root_folder` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/parent_folder` | **GET** `/api/v1/files/{file_id}/parent` |\n| **GET** `/api/v1/file/all_parent_folder` | **GET** `/api/v1/files/{file_id}/ancestors` |\n| **POST** `/api/v1/file/rm` | **DELETE** `/api/v1/files` |\n| **POST** `/api/v1/file/rename` | **POST** `/api/v1/files/move` |\n| **GET** `/api/v1/file/get/{file_id}` | **GET** `/api/v1/files/{file_id}` |\n| **POST** `/api/v1/file/mv` | **POST** `/api/v1/files/move` |\n| **POST** `/api/v1/file/convert` | **POST** `/api/v1/files/link-to-datasets` |\n| **POST** `/api/v1/agents_openai/{agent_id}/chat/completions` | **POST** `/api/v1/agents/chat/completions` with `\"openai-compatible\": true` |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\n**POST** `/api/v1/openai/{chat_id}/chat/completions`\n\nCreates a model response for a given chat conversation.\n\n:::caution REMOVED ALIAS\n`POST /api/v1/chats_openai/{chat_id}/chat/completions` is not available in 1.0. Use this endpoint instead.\n:::\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/openai/{chat_id}/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"model\"`: `string` (optional)\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"extra_body\"`: `object` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/openai/{chat_id}/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"model\": \"glm-4-flash@ZHIPU-AI\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true,\n \"extra_body\": {\n \"reference\": true,\n \"reference_metadata\": {\n \"include\": true,\n \"fields\": [\"author\", \"year\", \"source\"]\n },\n \"metadata_condition\": {\n \"logic\": \"and\",\n \"conditions\": [\n {\n \"name\": \"author\",\n \"comparison_operator\": \"is\",\n \"value\": \"bob\"\n }\n ]\n }\n }\n }'\n```\n\n##### Request parameters\n\n- `chat_id` (*Path parameter*) `string`, *Required*\n Existing chat assistant ID. The request will use that chat assistant's knowledge and settings.\n\n- `model` (*Body parameter*) `string`\n The model used to generate the response. When `chat_id` is provided, you may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `extra_body` (*Body parameter*) `object`\n Extra request parameters:\n - `reference`: `boolean` - include reference in the final chunk (stream) or in the final message (non-stream).\n - `reference_metadata`: `object` - include document metadata in each reference chunk.\n - `include`: `boolean` - enable document metadata in reference chunks.\n - `fields`: `list[string]` - optional allowlist of metadata keys. Omit to include all. Use an empty list to include none.\n - `metadata_condition`: `object` - metadata filter conditions applied to retrieval results.\n\n#### Response\n\nStream:\n\n```json\ndata:{\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"Hello! It seems like you're just greeting me. If you have a specific\",\n \"role\": \"assistant\",\n \"function_call\": null,\n \"tool_calls\": null,\n \"reasoning_content\": null\n },\n \"finish_reason\": null,\n \"index\": 0,\n \"logprobs\": null\n }\n ],\n \"created\": 1755084508,\n \"model\": \"model\",\n \"object\": \"chat.completion.chunk\",\n \"system_fingerprint\": \"\",\n \"usage\": null\n}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" question or need information, feel free to ask, and I'll do my best\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" to assist you based on the knowledge base provided.\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": null, \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": \"stop\", \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": {\"prompt_tokens\": 5, \"completion_tokens\": 188, \"total_tokens\": 193}}\n\ndata:[DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"Hello! I'm your smart assistant. What can I do for you?\",\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": 1755084403,\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"model\": \"model\",\n \"object\": \"chat.completion\",\n \"usage\": {\n \"completion_tokens\": 55,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 55,\n \"reasoning_tokens\": 5,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 5,\n \"total_tokens\": 60\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n---\n\n### Create OpenAI-compatible agent completion\n\n:::caution REMOVED ALIAS\nRAGFlow 1.0 Go does not serve `POST /api/v1/agents_openai/{agent_id}/chat/completions`. Use the endpoint below and move `agent_id` into the request body.\n:::\n\n**POST** `/api/v1/agents/chat/completions`\n\nCreates a model response for a given chat conversation.\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/agents/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"agent_id\"`: `string`\n - `\"openai-compatible\"`: `boolean`\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"session_id\"`: `string` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/agents/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"agent_id\": \"{agent_id}\",\n \"openai-compatible\": true,\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true\n }'\n```\n\n##### Body parameters\n\n- `agent_id` (*Body parameter*) `string`, *Required*\n The ID of the agent to run.\n\n- `openai-compatible` (*Body parameter*) `boolean`, *Required*\n Set this field to `true` to use the OpenAI-compatible request and response format.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `session_id` (*Body parameter*) `string`\n Agent session id.\n\n#### Response\n\nStream:\n\n```json\n...\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \" terminal\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \".\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"id\": \"4b8935ac0a22deb1\",\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"url\": null,\n \"similarity\": 0.5697155305154673,\n \"vector_similarity\": 0.7323851005515574,\n \"term_similarity\": 0.5000000005,\n \"doc_type\": \"\"\n }\n },\n \"doc_aggs\": {\n \"INSTALL22.md\": {\n \"doc_name\": \"INSTALL22.md\",\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"count\": 3\n },\n \"INSTALL.md\": {\n \"doc_name\": \"INSTALL.md\",\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL(1).md\": {\n \"doc_name\": \"INSTALL(1).md\",\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL3.md\": {\n \"doc_name\": \"INSTALL3.md\",\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"count\": 1\n }\n }\n }\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: [DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"\\nTo install Neovim, the process varies depending on your operating system:\\n\\n### For Windows:\\n1. **Download from GitHub**: \\n - Visit the [Neovim releases page](https://github.com/neovim/neovim/releases)\\n - Download the latest Windows installer (nvim-win64.msi)\\n - Run the installer and follow the prompts\\n\\n2. **Using winget** (Windows Package Manager):\\n...\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"doc_type\": \"\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"id\": \"4b8935ac0a22deb1\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"similarity\": 0.5697155305154673,\n \"term_similarity\": 0.5000000005,\n \"url\": null,\n \"vector_similarity\": 0.7323851005515574\n }\n },\n \"doc_aggs\": {\n \"INSTALL(1).md\": {\n \"count\": 2,\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL(1).md\"\n },\n \"INSTALL.md\": {\n \"count\": 2,\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL.md\"\n },\n \"INSTALL22.md\": {\n \"count\": 3,\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL22.md\"\n },\n \"INSTALL3.md\": {\n \"count\": 1,\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL3.md\"\n }\n }\n },\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": null,\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"object\": \"chat.completion\",\n \"param\": null,\n \"usage\": {\n \"completion_tokens\": 415,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 0,\n \"reasoning_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 6,\n \"total_tokens\": 421\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n**POST** `/api/v1/datasets`\n\nCreates a dataset.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### A basic request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test_1\",\n \"language\": \"English\",\n \"parse_type\": 1,\n \"parser_id\": \"general\"\n }'\n```\n\n##### A request example specifying ingestion pipeline\n\n:::caution WARNING\nDo not include `\"parser_id\"` when specifying an ingestion pipeline.\n:::\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test-sdk\",\n \"language\": \"English\",\n \"parse_type\": 2,\n \"pipeline_id\": \"\"\n }'\n```\n\n##### Body parameters\n\n- `\"name\"`: `string`, *Required*\n The unique name of the dataset to create. It must adhere to the following requirements:\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n If the name is already taken, the dataset is created with a numbered suffix such as `test_1(1)`, `test_1(2)`, and so on.\n\n- `\"language\"`: `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the server/database default is used.\n\n- `\"embedding_model\"`: `string`\n The name of the embedding model to use. For example: `\"BAAI/bge-large-zh-v1.5@BAAI\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n\n- `\"permission\"`: `string`\n Specifies who can access the dataset to create. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n\n- `\"parser_id\"`: `enum`\n The built-in parser to use. Required when `\"parse_type\"` is `1`.\n - `\"general\"`: General (default)\n - `\"qa\"`: Q&A\n - `\"manual\"`: Manual\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"audio\"`: Audio\n - `\"email\"`: Email\n - Must be omitted when `\"parse_type\"` is `2`.\n\n- `\"parser_config\"`: `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n- `\"parse_type\"`: `int`, *Required*\n The parser selection mode:\n - `1`: BuiltIn. Requires `\"parser_id\"`.\n - `2`: Pipeline. Requires `\"pipeline_id\"`.\n\n- `\"pipeline_id\"`: `string`\n The ingestion pipeline ID. Can be found in the corresponding URL in the RAGFlow UI.\n - Required when `\"parse_type\"` is `2`.\n - Must be a 32-character lowercase hexadecimal string, e.g., `\"d0bebe30ae2211f0970942010a8e0005\"`.\n - Must be omitted when `\"parse_type\"` is `1`.\n\n:::caution WARNING\n`\"parse_type\"` is required when creating a dataset. Choose exactly one mode:\n\n- `\"parse_type\": 1` with `\"parser_id\"` for a builtin parser.\n- `\"parse_type\": 2` with `\"pipeline_id\"` for an ingestion pipeline.\n\n`\"chunk_method\"`, `\"avatar\"`, `\"description\"`, and `\"auto_metadata_config\"` are not accepted by the RAGFlow 1.0 create endpoint. Use `\"parser_id\"` instead of `\"chunk_method\"`; set avatar and description with the update endpoint, and metadata configuration with the metadata endpoint.\n:::\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"chunk_count\": 0,\n \"create_time\": 1745836841611,\n \"created_by\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"document_count\": 0,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5@BAAI\",\n \"id\": \"3b4de7d4241d11f0a6a79f24fc270c7f\",\n \"language\": \"English\",\n \"name\": \"RAGFlow example\",\n \"pagerank\": 0,\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 128,\n \"delimiters\": [\"\\n\", \"!?;。；！？\"],\n \"parent_child\": {\n \"use_parent_child\": false,\n \"children_delimiter\": \"\\n\"\n }\n }\n },\n \"parser_id\": \"general\",\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"tenant_id\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"token_num\": 0,\n \"keywords_similarity_weight\": 0.7\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"Field: - Message: - Value: <>\"\n}\n```\n\n---\n\n### Delete datasets\n\n**DELETE** `/api/v1/datasets`\n\nDeletes datasets by ID.\n\n#### Request\n\n- Method: DELETE\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"ids\"`: `list[string]` or `null`\n - `\"delete_all\"`: `boolean`\n\n##### Request example\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"ids\": [\"d94a8dc02c9711f0930f7fbc369eab6d\", \"e94a8dc02c9711f0930f7fbc369eab6e\"]\n }'\n```\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"delete_all\": true\n }'\n```\n\n##### Body parameters\n\n- `\"ids\"`: `list[string]` or `null`\n Specifies the datasets to delete:\n - If omitted, or set to `null` or an empty array, no datasets are deleted.\n - If an array of IDs is provided, only the datasets matching those IDs are deleted.\n- `\"delete_all\"`: `boolean`\n Whether to delete all datasets owned by the current user when `\"ids\"` is omitted, or set to `null` or an empty array. Defaults to `false`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\":108,\n \"message\":\"User '' lacks permission for datasets: ''\"\n}\n\n```\n\n---\n\n### Update dataset\n\n**PUT** `/api/v1/datasets/{dataset_id}`\n\nUpdates configurations for a specified dataset.\n\n#### Request\n\n- Method: PUT\n- URL: `/api/v1/datasets/{dataset_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"avatar\"`: `string`\n - `\"description\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"pagerank\"`: `int`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### Request example\n\n```bash\ncurl --request PUT \\\n --url http://{address}/api/v1/datasets/{dataset_id} \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '\n {\n \"name\": \"updated_dataset\"\n }'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to update.\n- `\"name\"`: (*Body parameter*), `string`\n The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"language\"`: (*Body parameter*), `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the existing dataset language remains unchanged.\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n Refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration).\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser selection mode: `1` for a builtin parser (`parser_id` required) or `2` for an ingestion pipeline (`pipeline_id` required). Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`; must be omitted when `parse_type` is `2`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n`chunk_method` is not supported by RAGFlow 1.0; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"Can't change tenant_id.\"\n}\n```\n\n---\n\n### List datasets\n\n**GET** `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n\nLists datasets.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n- Headers:\n - `'Authorization: Bearer '`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id} \\\n --header 'Authorization: Bearer '\n```\n\n```bash\n# List datasets with parsing status\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets?include_parsing_status=true' \\\n --header 'Authorization: Bearer '\n```\n\n##### Query parameters\n\n- `page`\n Specifies the page on which the datasets will be displayed. Defaults to `1`.\n- `page_size`\n The number of datasets on each page. Defaults to `30`.\n- `orderby`\n The field by which datasets should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`\n Indicates whether the retrieved datasets should be sorted in descending order. Defaults to `true`.\n- `sort`\n Orders by several fields at once, written as `column:direction` terms separated by commas, such as `name:asc,create_time:desc`. A term with no direction is ascending. `sort` takes precedence over `orderby` and `desc`, which keep working on their own. A term naming a field this list does not sort by is skipped. When `sort` yields no terms at all, `orderby` and `desc` decide the order. When it yields terms whose fields are all skipped, the result is ordered by `create_time` in the direction of the first term, and the older pair is not read.\n- `name`\n The name of the dataset to retrieve.\n- `id`\n The ID of the dataset to retrieve.\n- `include_parsing_status`\n Whether to include document parsing status counts in the response. Defaults to `false`. When set to `true`, each dataset object in the response will include the following additional fields:\n - `unstart_count`: Number of documents not yet started parsing.\n - `running_count`: Number of documents currently being parsed.\n - `cancel_count`: Number of documents whose parsing was canceled.\n - `done_count`: Number of documents that have been successfully parsed.\n - `fail_count`: Number of documents whose parsing failed.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": \"\",\n \"chunk_count\": 59,\n \"create_date\": \"Sat, 14 Sep 2024 01:12:37 GMT\",\n \"create_time\": 1726276357324,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"description\": null,\n \"document_count\": 1,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5\",\n \"id\": \"6e211ee0723611efa10a0242ac120007\",\n \"language\": \"English\",\n \"name\": \"mysql\",\n \"parser_id\": \"general\",\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 8192,\n \"delimiters\": [\"\\n\"]\n }\n },\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"69736c5e723611efb51b0242ac120007\",\n \"token_num\": 12744,\n \"update_date\": \"Thu, 10 Oct 2024 04:07:23 GMT\",\n \"update_time\": 1728533243536,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nSuccess (with `include_parsing_status=true`):\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": null,\n \"cancel_count\": 0,\n \"chunk_count\": 30,\n \"parser_id\": \"qa\",\n \"create_date\": \"2026-03-09T18:57:13\",\n \"create_time\": 1773053833094,\n \"created_by\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"description\": null,\n \"document_count\": 1,\n \"done_count\": 1,\n \"embedding_model\": \"text-embedding-v2@Tongyi-Qianwen\",\n \"fail_count\": 0,\n \"id\": \"ba6586c21ba611f1a3dc476f0709e75e\",\n \"language\": \"English\",\n \"name\": \"Test Dataset\",\n \"parser_config\": {\n \"Extractor:AutoExtractDefault\": {\n \"llm_id\": \"deepseek-chat@DeepSeek\"\n }\n },\n \"permission\": \"me\",\n \"running_count\": 0,\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"token_num\": 1746,\n \"unstart_count\": 0,\n \"update_date\": \"2026-03-09T18:59:32\",\n \"update_time\": 1773053972723,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset doesn't exist\"\n}\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n**POST** `/api/v1/datasets/{dataset_id}/documents`\n\nUploads documents to a specified dataset.\n\nThis endpoint supports three creation modes via the optional `type` query parameter:\n\n- `type=local` or omitted: Upload one or more local files using `multipart/form-data`.\n- `type=web`: Crawl a web page and save it as a document.\n- `type=empty`: Create an empty virtual document by name.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets/{dataset_id}/documents`\n- Query:\n - `type`: Optional. One of `local`, `web`, or `empty`. Defaults to `local`.\n- Headers:\n - `'Content-Type: multipart/form-data'` for `type=local` and `type=web`\n - `'Content-Type: application/json'` for `type=empty`\n - `'Authorization: Bearer '`\n- Body:\n - For `type=local`: form field `'file=@{FILE_PATH}'`\n - For `type=web`: form fields `'name'` and `'url'`\n - For `type=empty`: JSON body with `'name'`\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'file=@./test1.txt' \\\n --form 'file=@./test2.pdf'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=web' \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'name=example-page' \\\n --form 'url=https://example.com'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=empty' \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\"name\":\"blank.txt\"}'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to which the documents will be uploaded.\n- `type`: (*Query parameter*)\n Controls how the document is created:\n - `local`: Upload files.\n - `web`: Crawl a URL into a document.\n - `empty`: Create an empty document without file upload.\n- `'file'`: (*Body parameter*)\n A document to upload. Required when `type=local`.\n- `'name'`: (*Body parameter*)\n The document name. Required when `type=web` or `type=empty`.\n- `'url'`: (*Body parameter*)\n The source URL to crawl. Required when `type=web`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"chunk_method\": \"naive\",\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"dataset_id\": \"527fa74891e811ef9c650242ac120006\",\n \"id\": \"b330ec2e91ec11efbc510242ac120004\",\n \"location\": \"1.txt\",\n \"name\": \"1.txt\",\n \"parser_config\": {\n \"chunk_token_num\": 128,\n \"delimiter\": \"\\\\n\",\n \"html4excel\": false,\n \"layout_recognize\": \"DeepDOC\"\n },\n \"run\": \"UNSTART\",\n \"size\": 17966,\n \"thumbnail\": \"\",\n \"type\": \"doc\"\n }\n ]\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"No file part!\"\n}\n```\n\n---\n\n### Update document\n\n**PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nUpdates configurations for a specified document.\n\nSaving parser settings preserves existing chunks, chunk images, counters, and parsing progress. The saved settings take effect on the next parse; this update does not start parsing or delete existing results. Use the document parse action to clear existing chunks and re-parse with the saved settings.\n\n#### Request\n\n- Method: PATCH\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`:`string`\n - `\"meta_fields\"`:`object`\n - `\"chunk_method\"`:`string`\n - `\"parser_config\"`:`object`\n\n##### Request example\n\n```bash\ncurl --request PATCH \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --header 'Content-Type: application/json' \\\n --data '\n {\n \"name\": \"manual.txt\",\n \"chunk_method\": \"manual\",\n \"parser_config\": {\"chunk_token_num\": 128}\n }'\n\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the associated dataset.\n- `document_id`: (*Path parameter*)\n The ID of the document to update.\n- `\"name\"`: (*Body parameter*), `string`\n- `\"meta_fields\"`: (*Body parameter*), `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: (*Body parameter*), `string`\n The parsing method to apply to the document:\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. The attributes in this JSON object vary with the selected `\"chunk_method\"`:\n - If `\"chunk_method\"` is `\"naive\"`, the `\"parser_config\"` object contains the following attributes:\n - `\"chunk_token_num\"`: Defaults to `256`.\n - `\"layout_recognize\"`: Defaults to `\"DeepDOC\"`.\n - `\"html4excel\"`: Indicates whether to convert Excel documents into HTML format. Defaults to `false`.\n - `\"delimiter\"`: Defaults to `\"\\n\"`.\n - `\"task_page_size\"`: Defaults to `12`. For PDF only.\n- `\"enabled\"`: (*Body parameter*), `integer`\n Whether the document should be **available** in the knowledge base.\n - `1` → （available）\n - `0` → （unavailable）\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"id\": \"cd38dd72d4a611f0af9c71de94a988ef\",\n \"name\": \"large.md\",\n \"type\": \"doc\",\n \"suffix\": \"md\",\n \"size\": 2306906,\n \"location\": \"large.md\",\n \"source_type\": \"local\",\n \"status\": \"1\",\n \"run\": \"DONE\",\n \"dataset_id\": \"5f546a1ad4a611f0af9c71de94a988ef\",\n\n \"chunk_method\": \"naive\",\n \"chunk_count\": 2,\n \"token_count\": 8126,\n\n \"created_by\": \"eab7f446cb5a11f0ab334fbc3aa38f35\",\n \"create_date\": \"Tue, 09 Dec 2025 10:28:52 GMT\",\n \"create_time\": 1765247332122,\n \"update_date\": \"Wed, 17 Dec 2025 10:51:16 GMT\",\n \"update_time\": 1765939876819,\n\n \"process_begin_at\": \"Wed, 17 Dec 2025 10:33:55 GMT\",\n \"process_duration\": 14.8615,\n \"progress\": 1.0,\n\n \"progress_msg\": [\n \"10:33:58 Task has been received.\",\n \"10:33:59 Page(1~100000001): Start to parse.\",\n \"10:33:59 Page(1~100000001): Finish parsing.\",\n \"10:34:07 Page(1~100000001): Generate 2 chunks\",\n \"10:34:09 Page(1~100000001): Embedding chunks (2.13s)\",\n \"10:34:09 Page(1~100000001): Indexing done (0.31s).\",\n \"10:34:09 Page(1~100000001): Task done (11.68s)\"\n ],\n\n \"parser_config\": {\n \"chunk_token_num\": 512,\n \"delimiter\": \"\\n\",\n \"auto_keywords\": 0,\n \"auto_questions\": 0,\n \"topn_tags\": 3,\n \"layout_recognize\": \"DeepDOC\",\n \"html4excel\": false,\n \"image_context_size\": 0,\n \"table_context_size\": 0\n },\n\n \"meta_fields\": {},\n \"pipeline_id\": \"\",\n \"thumbnail\": \"\"\n }\n}\n\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset does not have the document.\"\n}\n```\n\n---\n\n### Download document\n\n**GET** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nDownloads a document from a specified dataset.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Authorization: Bearer '`\n- Output:\n - `'{PATH_TO_THE_FILE}'`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --output ./ragflow.txt\n```\n\n##### Path parameters\n\n- `dataset_id`\n The associated dataset ID.\n- `document_id`\n The ID of the document to download.\n\n#### Response\n\nSuccess:\n\n```json\nThis is a test to verify the file download feature.\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"You do not own the dataset 7898da028a0511efbf750242ac1220005.\"\n}\n```\n\n---\n\n### List documents\n\n**GET** `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n\nLists documents in a specified dataset.\n\nTo retrieve a specific document's settings and metadata, pass its document ID in the `id` query parameter: `/api/v1/datasets/{dataset_id}/documents?id={document_id}`.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n\n##### Request examples\n\n**A basic request with pagination:**\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents?page=1&page_size=10 \\\n --header 'Authorization: Bearer '\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The associated dataset ID.\n- `keywords`: (*Query parameter*), `string`\n The keywords used to match document titles.\n- `page`: (*Query parameter*), `integer`\n Specifies the page on which the documents will be displayed. Defaults to `1`.\n- `page_size`: (*Query parameter*), `integer`\n The maximum number of documents on each page. Defaults to `30`.\n- `orderby`: (*Query parameter*), `string`\n The field by which documents should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`: (*Query parameter*), `boolean`\n Indicates whether the retrieved documents should be sorted in descending order. Defaults to `true`.\n- `id`: (*Query parameter*), `string`\n The ID of the document to retrieve.\n- `create_time_from`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created after this time. 0 means no filter. Defaults to `0`.\n- `create_time_to`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created before this time. 0 means no filter. Defaults to `0`.\n- `suffix`: (*Query parameter*), `array[string]`\n Filter by file suffix. Supports multiple values, e.g., `pdf`, `txt`, and `docx`. Defaults to all suffixes.\n- `run`: (*Query parameter*), `array[string]`\n Filter by document processing status. Supports numeric, text, and mixed formats:\n - Numeric format: `[\"0\", \"1\", \"2\", \"3\", \"4\", \"5\"]`\n - Text format: `[UNSTART, RUNNING, CANCEL, DONE, FAIL, SCHEDULE]`\n - Mixed format: `[UNSTART, 1, DONE]` (mixing numeric and text formats)\n - Status mapping:\n - `0` / `UNSTART`: Document not yet processed\n - `1` / `RUNNING`: Document is currently being processed\n - `2` / `CANCEL`: Document processing was canceled\n - `3` / `DONE`: Document processing completed successfully\n - `4` / `FAIL`: Document processing failed\n - `5` / `SCHEDULE`: Document is scheduled and waiting to be processed\n Defaults to all statuses.\n- `metadata_condition`: (*Query parameter*), `object` (JSON in query)\n Optional metadata filter applied to documents when `document_ids` is not provided. Uses the same structure as retrieval:\n - `logic`: `\"and\"` (default) or `\"or\"`\n - `conditions`: array of `{ \"name\": string, \"comparison_operator\": string, \"value\": string }`\n - `comparison_operator` supports: `is`, `not is`, `contains`, `not contains`, `in`, `not in`, `start with`, `end with`, `>`, `<`, `≥`, `≤`, `empty`, `not empty`\n\n##### Usage examples\n\n**A request with multiple filtering parameters**\n\n```bash\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?suffix=pdf&run=DONE&page=1&page_size=10' \\\n --header 'Authorization: Bearer '\n```\n\n**Filter by metadata (query JSON):**\n\n```bash\ncurl -G \\\n --url \"http://{address}/api/v1/datasets/{dataset_id}/documents\" \\\n --header 'Authorization: Bearer ' \\\n --data-urlencode 'metadata_condition={\"logic\":\"and\",\"conditions\":[{\"name\":\"tags\",\"comparison_operator\":\"is\",\"value\":\"bar\"},{\"name\":\"author\",\"comparison_operator\":\"is\",\"value\":\"alice\"}]}'\n```\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"docs\": [\n {\n \"chunk_count\": 0,\n \"create_date\": \"Mon, 14 Oct 2024 09:11:01 GMT\",\n \"create_time\": 1728897061948,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"id\": \"3bcfbf8a8a0c11ef8aba0242ac120006\",\n \"knowledgebase_id\": \"7898da028a0511efbf750242ac120005\",\n \"location\": \"Test_2.txt\",\n \"name\": \"Test_2.txt\",\n \"parser_config\": {\n \"chunk_token_count\": 128,\n \"delimiter\": \"\\n\",\n \"layout_recognize\": \"DeepDOC\",\n \"task_page_size\": 12\n },\n ","tools/chatgpt-on-wechat/plugins/README.md":"# plugins Documentation\n\nPath: `tools/chatgpt-on-wechat/plugins/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/README.md)","tools/chatgpt-on-wechat/plugins/config.json":"# Plugins Skill\n\nPath: `tools/chatgpt-on-wechat/plugins/config.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/config.json)"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# RAGFlow Instructions\n\nUse this file as the local operating guide for the current codebase. Prefer the code and the current AGENTS.md over any older convention or remembered project shape.\n\n## Core Stance\n- Treat legacy code as liability, not as a compatibility target.\n- Prefer deletion over shims, deprecated branches, wrapper APIs, and dual-track migration notes.\n- If old and new implementations coexist, converge to one path unless an external contract forces compatibility.\n- Remove dead tests, commented-out code, stale docs, and \"move later\" notes instead of preserving them.\n- Reduce public surface area when a helper can be made private or internal.\n- Keep refactors centered on the owning abstraction, not on adjacent compatibility layers.\n\n## Current stack\n- Backend: Go implements the API, admin, ingestor, and syncer modes in `cmd/ragflow_server.go`; the CLI runs from `cmd/ragflow-cli.go`. The Docker entrypoint starts the Go services according to its command-line flags, and the frontend development server uses fixed Go backend proxy routes. The Python API, admin, and workers are still present pending removal.\n- Frontend: React + TypeScript + Vite in `web/`. When working under `web/`, read and follow `web/AGENTS.md` for frontend conventions.\n- The Go module contains the server, ingestion, parsing, agent runtime, CLI, and supporting services.\n- Runtime services commonly include MySQL/PostgreSQL, Kvrocks, NATS JetStream, MinIO, ClickHouse, and Elasticsearch/Infinity/OpenSearch depending on configuration.\n## Go-only target\n- Remove all Python code and Python-specific dependencies, scripts, containers, CI jobs, documentation, SDKs, and tests from the repository. Do not add new Python code while this removal is in progress. Replace Python-based preparation of Go native libraries, DeepDoc models, and the BPE table before removing the existing download scripts.\n- Move any still-required behavior and test coverage into the Go backend or the frontend before deleting the old implementation. Do not preserve a Python server, compatibility layer, fallback, or dual-backend path.\n- Before deleting `rag/`, move the Go DeepDoc `.ort` models and `ocr.res` out of `internal/rag/res/deepdoc/`, then update model discovery, dependency preparation, Docker packaging, and tests together.\n- Remove the frontend's Go/Python backend variants and update `web/CLAUDE.md` when the frontend no longer needs them. Until then, treat its dual-backend instructions as migration-only guidance; new frontend work should target the Go API.\n- Update Docker's default entrypoint, dependency image, and CI workflows when removing their remaining Python paths. Keep the frontend development proxy routes aligned with the Go API and Admin ports in `web/vite.config.ts`.\n- Keep this guide aligned with the checked-in tree. Once the Python directories are deleted, remove their migration inventory below.\n\n## Code Layout to Expect\n- `cmd/ragflow_server.go`: Go entrypoint for the API, admin, ingestor, and syncer modes (`--api`, `--admin`, `--ingestor`, `--syncer`).\n- `cmd/ragflow-cli.go`: Go CLI entrypoint.\n- `api/`, `rag/`, `deepdoc/`, and `agent/`: legacy Python implementation pending deletion. Do not add features or extend dependencies there.\n- `internal/`: main Go application code. Important subtrees:\n- `internal/agent/`: Go agent runtime, canvas execution, components, tool bindings, workflow helpers.\n- `internal/admin/`: Go admin routes, handlers, and services.\n- `internal/binding/cpp/`: C++ tokenizer binding built by `build.sh` for the Go server.\n- `internal/cli/`: CLI parsing, HTTP transport, command execution, response formatting.\n- `internal/dao/`: Go data-access layer and persistence-facing helpers.\n- `internal/deepdoc/`: Go DeepDOC integrations, especially native-backed PDF/DOCX parsing.\n- `internal/engine/`: search/index backends such as Elasticsearch and Infinity.\n- `internal/entity/`: shared Go entities and model definitions.\n- `internal/handler/`: HTTP handlers and route-facing request logic.\n- `internal/ingestion/`: Go ingestion pipeline, canvas adapter, components, wiring, service orchestration.\n- `internal/ingestion/component/`: stage implementations such as file/parser/chunker/tokenizer/extractor.\n- `internal/ingestion/pipeline/`: DSL translation, canvas-driven execution, checkpoints, resume/run logic.\n- `internal/parser/`: parser and chunk libraries used by ingestion and other Go paths.\n- `internal/parser/parser/`: typed parse-result parsers for markdown/html/pdf/docx/xlsx/text and related families.\n- `internal/parser/chunk/`: chunk operator library and DSL/typed execution helpers.\n- `internal/service/`: higher-level business services used by handlers and server flows.\n- `internal/storage/`: storage backends and in-memory test doubles.\n- `internal/router/`: HTTP route registration.\n- `internal/server/`: server bootstrap/config wiring.\n- `internal/syncer/`: Go file synchronization service and connectors.\n- `internal/rag/`: Go retrieval and RAG logic.\n- `internal/harness/`: Go runtime harness and supporting execution code.\n- `web/`: frontend application.\n- `docker/`: local and production compose files.\n- `sdk/python/`: legacy Python client SDK pending deletion.\n- `tools/scripts/download_dev_deps.py` and `deps/download_deps.py`: Python downloaders for the Go build's dependencies. `download_dev_deps.py` pulls the native static libraries (pdfium, pdf_oxide, office_oxide, onnxruntime) that `build.sh` links; `download_deps.py` pulls the runtime resources (cl100k BPE table, DeepDoc models, embedding tokenizer assets) and installs the stagehand server into the SDK cache. Versions and URLs are defined inline in each downloader (`tools/scripts/download_dev_deps.py` for native static libraries, `deps/download_deps.py` for runtime resources); the tokenizer assets are inlined as `TOKENIZER_ASSETS` in each downloader. Replace these with non-Python preparation before removing Python dependencies.\n- `test/`: contains Python tests, including tests that import the legacy Python implementation directly. Migrate needed coverage to Go tests or frontend tests, then remove the Python tests; keep only tests for supported code.\n\n## Go-Specific Rules\n- Treat `internal/ingestion`, `internal/parser`, and `internal/deepdoc` as actively refactored code. Prefer collapsing duplicate paths over preserving transitional wrappers.\n- Do not add or preserve deprecated Go APIs just to ease migration inside the repo.\n- Remove commented-out Go code instead of leaving recovery notes in place.\n- Keep package comments and doc comments aligned with the current runtime path, not with migration history.\n\n## GORM query traps\n- **Never pass a bare string ID as the second argument to `Take`/`First`** (e.g. `Take(&x, id)`). Under GORM v1.25.7 a non-numeric string is treated as a raw SQL condition and concatenated into the `WHERE` clause: `Take(&x, \"88b17c7a...\")` emits `WHERE 88b17c7a... LIMIT 1`, which MySQL rejects with `Error 1054 (42S22): Unknown column '...' in 'where clause'`. Only strings that parse via `strconv.Atoi` are treated as primary-key values, so this bug is invisible for integer IDs and catastrophic for UUID primary keys. Always bind explicitly: `Take(&x, \"id = ?\", id)` or `Where(\"id = ?\", id).Take(&x)`.\n- When you change a query form across many files, verify the generated SQL with a `Session(&gorm.Session{DryRun: true})` probe before relying on it. A passing unit test with no rows and a broken `WHERE` clause look identical unless the SQL is inspected.\n\n## Shared database schema (Go + Python)\nThe Go services and the Python API write to the same MySQL/PostgreSQL schema, and both own parts of it. Neither is authoritative over the whole.\n\nEach ORM derives index names differently:\n- Go (`internal/entity/`): GORM derives `idx_
_` for indexes and `uni_
_` for unique constraints. Tags in this repo name unique indexes explicitly, e.g. `uniqueIndex:idx_commit_file`.\n- Python (`api/db/db_models.py`): Peewee derives `__` (truncated with an md5 suffix past 64 chars, see `playhouse.migrate.make_index_name`). There is no `idx_` prefix.\n\nBoth sides reconcile by **column set, not by name**: Go's `hasUniqueIndex` (`internal/dao/migration.go`) accepts an existing index only when its full column set equals the requested one, and Python's `ensure_model_indexes` (`api/db/db_models.py`) keys `DB.get_indexes()` on the column tuple. Consequences:\n\n- Divergent names covering the same columns are expected, not a defect. Whichever runtime reaches the database first decides the name. Do not \"fix\" this by renaming inside a migration; it rewrites live databases for no functional gain.\n- Add a shared unique index to **both** sides (Go tag + Peewee `Meta.indexes` or `unique=True`), so each runtime recreates it when absent. Do not add a second index over a column set an existing one already covers.\n- In Go, declare single-column uniqueness with a named `uniqueIndex:`, not a bare `unique` tag. A bare `unique` becomes a table-level constraint in DDL, which SQLite materializes as an implicit index that cannot be dropped by name, and which GORM's `MigrateColumnUnique` then tries to drop under its own `uni_
_` name — MySQL answers 1091 every startup. `namedIndexMigrator` (`internal/dao/database.go`) suppresses exactly that phantom drop; see `TestNamedIndexMigratorSkipsPhantomUniqueDrop`.\n- A Python model existing does not mean Python owns that table's constraints. `pipeline_operation_log` has no `run_count` column and no unique index in `db_models.py`; both are added only by `internal/dao/migration.go`.\n\n## Go Test Tiers\nGo tests are classified by build tag. Tag a test file with `//go:build ` placed before the `package` clause. Run them through `build.sh` so the required CGO configuration and native libraries are available.\n\n| Tier | Build tag | Runs by default? | Needs |\n|---|---|---|---|\n| Unit | (none) | Yes (`bash build.sh --test`) | Native CGO static libs; no external services — uses in-memory SQLite, miniredis, or `httptest` stubs. |\n| Integration | `integration` | No (`-tags integration`) | A real service: MySQL/MinIO/Elasticsearch/Infinity/LLM. Single component, reasonably fast. |\n| E2E | `e2e` | No (`-tags e2e`) | Full cross-component pipeline (ingest → index → retrieve) against real services; heavy/slow. |\n| Manual | `manual` | No (`-tags manual`) | Very slow/expensive (deepdoc render/parity/snapshot/bench). **Local opt-in ONLY — never run in CI.** |\n| Native (orthogonal) | `cgo` / `!cgo` | `cgo` auto-satisfies under CGO_ENABLED=1 | Native static libs (`office_oxide`/`pdfium`/`pdf_oxide`). Combine with tiers, e.g. `//go:build cgo && integration`. |\n\nRun tiers locally via `build.sh`:\n```bash\nbash build.sh --test # unit tier (no tags)\nbash build.sh --test-integration ./... # integration tier\nbash build.sh --test-e2e # e2e tier\nbash build.sh --test-manual # manual tier (very slow)\nbash build.sh --test-all # integration + e2e (never includes manual)\n```\nRules:\n- New tests that touch a real external service MUST carry `integration`/`e2e`/`manual` — do not rely on `t.Skip` + env vars to soft-isolate them in the default unit run. Keep an env guard as a harmless secondary safety net if desired.\n- `manual` is never wired into CI or any automated pipeline.\n- `unit` (no tag) must stay free of external-service dependencies so `bash build.sh --test` runs without MySQL/MinIO/ES/Infinity/LLM. The native CGO static libraries (`office_oxide`/`pdfium`/`pdf_oxide`) are still required at build time and are wired by `build.sh --test`; that is expected, not an external service.\n\n## Working Rules\n- When reviewing documentation or code, inspect the full affected path and report all verifiable findings in one review; do not return after only a few findings and expose further issues in later rounds.\n- When handling review comments, independently verify each substantive claim against the current code or tests before accepting, rejecting, or acting on it.\n- Before editing, inspect the nearest code path that actually owns the behavior.\n- For server changes, work in the Go implementation. If a Python behavior is still needed, port it to Go and test the Go path before removing the Python code.\n- Keep changes small and local unless the task is explicitly a broader refactor.\n- Prefer one implementation path instead of preserving old and new versions side by side.\n- Preserve behavior with focused tests when the behavior is still valid; do not keep tests that protect obsolete behavior.\n- If a surface is only there for compatibility, remove it unless the user asks to keep it.\n- Do not add new compatibility wording in comments or docs.\n- When a maintainer takes over a community PR, a new commit generated by rewriting history (e.g. `merge`, `rebase -i`) must preserve the original author and add the maintainer as co-author (via a `Co-authored-by:` trailer) instead of overwriting the author with the maintainer alone.\n\n## Commands\n### Go backend\n```bash\nbash build.sh --all # build native code and Go server\nbash build.sh --go # build Go server with native library already built\nbash build.sh --test ./internal/handler/... # targeted unit tests\nbash build.sh --test # all unit tests\nbash build.sh --test-integration ./... # integration tests; requires real services\nbash build.sh --test-e2e # full-pipeline tests; requires real services\nbash build.sh --test-manual # local opt-in only\nbash build.sh --test-all # integration + e2e; excludes manual\n```\n\nNative libraries, Go DeepDoc models, tokenizer assets, and `deps/cl100k_base.tiktoken` come from `tools/scripts/download_dev_deps.py` (native static libs) and `deps/download_deps.py` (models, cl100k, tokenizer assets); run them with `uv run`. Build the Go resource image with `cd deps && docker build -f Dockerfile -t infiniflow/ragflow_deps:latest .` after downloading the resources.\n\n### Frontend\n```bash\ncd web\nnpm install\nnpm run dev\nnpm run build\nnpm run lint\nnpm run test\nnpm run type-check\n```\n\n## Validation Preference\n- Run the narrowest relevant test, lint, or build command after a change.\n- For backend changes, first run package-scoped `bash build.sh --test ./path/to/package/...`. Run the integration or e2e tier when the changed behavior needs real services or a full pipeline.\n- For frontend changes, prefer the touched-package lint, type-check, or test command.\n- Do not default to raw `go test`, `go build`, or IDE Run/Debug for Go in this repo. They often miss the required CGO flags and native static libraries (`office_oxide`, `pdfium-static`, `pdf_oxide`) that `build.sh` wires correctly.\n- After a Go build, execute the resulting binary's help command for each changed server mode; a successful link alone does not prove the binary starts. On this host, the original `build.sh` selected default LLD 18 and produced binaries that crashed before Go `main`; linking with LLD 20 started successfully. Verify the linker actually selected on Linux before trusting a new binary.\n- If Go native builds fail, inspect `build.sh` and `internal/development.md` before changing code. Common environment issues are missing downloaded native dependencies or an incompatible `ld.lld` on Linux.\n\n## Default review checklist\n- Remove instead of retaining `deprecated`, `legacy`, or compatibility-only code.\n- Collapse duplicate implementations to one path.\n- Drop stale comments and documentation that describe a superseded design.\n- Keep exported APIs only when the current code actually needs them.\n","isInternal":false,"tokens":3585,"sizeBytes":15753},{"name":"AGENTS.md","path":"web/AGENTS.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/web/AGENTS.md","title":"web - AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# AGENTS.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with the RAGFlow frontend (`web/`).\n\n## Project Overview\n\nRAGFlow frontend is a React/TypeScript application built with Vite:\n\n- **Components**: shadcn/ui\n- **Styling**: Tailwind CSS\n- **State**: Zustand\n- **Data Fetching**: TanStack Query (React Query)\n- **i18n**: react-i18next\n\n## Common Commands\n\n```bash\nnpm install\nnpm run dev # Development server\nnpm run build # Production build\nnpm run lint # oxlint\nnpm run format # oxfmt\nnpm run test # Jest tests\n```\n\n## Development Conventions\n\n### Pre-commit Format Validation (Mandatory)\n\nFrontend code changes **must pass the lefthook format checks before every commit**. Do not rely on CI (`web-oxfmt` / `web-oxlint` jobs) to catch formatting after pushing.\n\n- Before staging or committing changed `.ts`/`.tsx`/`.js`/etc. files, run the formatter and linter locally:\n - `npx oxfmt ` (or `npm run format`) to auto-fix formatting\n - `npx oxlint` (or `npm run lint`) for lint issues\n- Verify with `npx oxfmt --check ` before `git commit`.\n- Let the lefthook pre-commit hook run to completion and fix everything it reports; never bypass hooks with `--no-verify`.\n\n### Test File Placement\n\nWhen tests are grouped in a dedicated directory, that directory is always named `__tests__/` — never `tests/`. Example: `src/pages/agent/utils/__tests__/extractor-transform.test.ts` for `src/pages/agent/utils/extractor-transform.ts`. Co-located `foo.test.ts(x)` files next to the source file are also fine. When you find an existing test under a `tests/` directory, move it into `__tests__/` (keep git history via `git mv`) rather than adding new tests there.\n\n### Backend Conventions\n\nThe frontend serves only the **Go** backend. There is no runtime backend detection and no per-backend branching in app code; the dev server proxies `/api` and `/v1` to the Go API (`vite.config.ts`).\n\nDo not reintroduce backend-variant primitives or detection helpers (e.g. a `useIsGoBackend` hook, `pickByBackend`, ``, or an `/api/v1/language` probe) and do not create `go/`-vs-`python/` variant directories or files.\n\n### CSS and Layout Debugging\n\nWhen fixing CSS/layout issues (especially flex truncation, ellipsis, or element sizing), **always inspect the full parent hierarchy** for `flex-shrink`, `min-width`, and `overflow` constraints before applying fixes like `min-w-0`. Do not repeatedly apply the same fix without verifying the root cause.\n\n- Before editing, explain: (1) the full flex/container hierarchy from the target element up to the nearest non-flex ancestor, (2) what constraint is actually causing the bug, and (3) how the proposed fix addresses that root cause.\n\n### Color Tokens\n\nWhen writing or modifying styles, **use the project-defined color tokens from `src/tailwind.css`** (e.g., `bg-bg-base`, `text-text-primary`, `text-text-secondary`, `text-text-disabled`, `border-border-button`, `bg-bg-card`). Do not use arbitrary hex/RGB values or Tailwind's default palette colors (e.g., `emerald-500`, `blue-400`) directly in component class names. These tokens are defined for both light and dark modes and keep the UI consistent with the design system.\n\n### Scope and Boundaries\n\nRespect explicit boundaries from the user. If the user says **\"only fix the selected line\"** or **\"do not touch shared types/files\"**, follow that instruction exactly. Do not investigate unrelated errors, modify shared schemas (e.g., `LlmSettingFieldSchema`), or refactor other files without confirmation. If a change outside the described scope seems necessary, ask for permission first.\n\n### Internationalization (i18n)\n\nFor translation tasks, add keys **only to the explicitly requested language files** (commonly `src/locales/zh.ts` and `src/locales/en.ts`). Do not auto-propagate changes to all language files unless the user explicitly asks.\n\n- **Style for `en.ts`**: Sentence case — first word capitalized, rest lowercase (e.g., `referenceAnswer: 'Reference answer'`). Proper nouns remain as-is.\n\n### React Component Refactoring\n\nWhen refactoring or extracting components, **verify layout behavior after each structural change** (especially `flex-1`, conditional rendering, or flex direction changes). Check that existing buttons, alignment, and responsive behavior remain intact. After extraction, verify: (1) all original props and behavior are preserved, (2) layout in parent contexts is identical, and (3) no syntax or type errors were introduced.\n\n### State Management and Data Fetching\n\n#### Query Key Factory (Mandatory)\n\n**Never write raw `queryKey` arrays inline.** Always use a query key factory object that returns `as const` tuples. Raw arrays duplicated across `useQuery` and `invalidateQueries` are brittle, unreadable, and cause stale-cache bugs when key structures drift.\n\n```ts\n// ❌ Bad — raw array, hard to match with useQuery\nqueryClient.invalidateQueries({\n queryKey: [\n LLMApiAction.AddedProviders,\n params.provider_name,\n params.instance_name,\n 'models',\n ],\n});\n\n// ✅ Good — factory reference, self-documenting\nqueryClient.invalidateQueries({\n queryKey: LlmKeys.instanceModels(params.provider_name, params.instance_name),\n});\n```\n\n- Place the factory in the same file as the hooks, named `{Domain}Keys` (e.g., `LlmKeys`, `DatasetKeys`).\n- Every `useQuery` and every `invalidateQueries` must reference the same factory function.\n- Use `as const` on each factory return value for type-safe readonly tuples.\n\n#### Cache Debugging\n\nFor React Query / cache invalidation bugs, **carefully compare query keys across all consuming components and mutation hooks**. Mismatched keys (e.g., with/without `refreshCount`) are a common root cause of stale data or duplicate requests.\n\n- Systematically: (1) list every component/hook that calls `useQuery` for this data, (2) compare their query keys character-for-character, (3) check every mutation's `onSuccess` for cache invalidation, and (4) verify no parent re-renders are remounting the observer.\n\n#### Colocate Queries with the Consuming View\n\n**Fire a query in the component that renders its data — not in a parent page.** When a page switches between mutually exclusive views (tabs, view modes), extract each view into its own component that issues its own requests on mount. Conditional rendering then provides lazy loading for free.\n\n- Do not hoist child-view queries into the page component — it fires requests the user may never need (e.g., fetching the skill tree on page entry while the default view is the LLM wiki).\n- Do not thread `enabled` flags or view-mode props through hooks to gate a hoisted query; that is a sign the query lives at the wrong level. Split the view instead.\n- Remember the trade-off: with `gcTime: 0`, unmounting a view drops its cache, so switching back refetches. That is usually desirable for always-fresh data — do not reintroduce eager hoisting just to avoid the refetch.\n\n### Network Request Layering\n\nHTTP requests are organized in three layers. **Never import `@/utils/request`, `@/utils/next-request`, or `@/utils/api` directly inside a hook**:\n\n1. `src/hooks/use-xx-request.ts(x)` — React Query hooks; only call the service layer.\n2. `src/services/xx-service.ts` — Register endpoints via `registerNextServer`, all going through `@/utils/next-request`.\n3. `src/utils/next-request.ts` — The single axios instance; handles token, 401 redirects, and error notifications.\n\nInterface types are split between two folders:\n\n- Response/data shape → `src/interfaces/database/xx.ts`\n- Request params/body → `src/interfaces/request/xx.ts`\n\nModel-related endpoints (LLM provider / factory / my LLM, etc.) are consolidated in `src/services/llm-service.ts` rather than scattered across hooks. For GET endpoints, register with `method: 'get'` in the service, and on the call site pass `true` as the second argument to use the native axios config (e.g., `service.listProviders({ params: { available: true } }, true)`).\n\n### Shared UI Component Lock\n\nThe folder `src/components/ui/` is the project's **shared UI library** — it contains both official shadcn/ui primitives and project-authored common components built on top of shadcn. Both kinds are intended to be reused across the app and **must not be modified casually**.\n\n- **Do not modify, refactor, restyle, or \"improve\"** any file under `src/components/ui/` (including subfolders), even if it seems like the most direct fix.\n- If a component does not meet requirements, **wrap or compose it** in a new component **outside** `src/components/ui/` (e.g., under `src/components/` or a feature folder), and customize via `className`, `props`, or composition.\n- Exceptions require **explicit user approval** in the same conversation. When in doubt, ask first and propose a wrapper-based alternative.\n- Adding a new shared component to `src/components/ui/`, or upgrading a shadcn primitive via the official `shadcn` CLI, is allowed only when the user explicitly requests it.\n\n### React Patterns and Conventions\n\n- **Avoid inline event handlers.** Do not define arrow functions directly on JSX event props like `onClick={() => ...}` or `onChange={(e) => ...}`. Instead, extract the handler and reference it by name (e.g., `onClick={handleClick}`). Inline handlers are recreated on every render, which can break `React.memo` optimizations and make stack traces harder to read.\n- **Prefer `requestAnimationFrame` or `useLayoutEffect`** over `setTimeout(..., 0)` for focus or DOM measurement operations.\n- **Prefer `useTranslation` from `react-i18next`** over project-wrapped utilities like `useTranslate`.\n- Extract complex logic into hooks or utils; keep components lean.\n- Use **PascalCase** for constants and component names.\n - Components: `EditableTextarea`, `RAGFlowFormItem`\n - Constants: `InitialMockData`, `DefaultPlaceholder`\n- Avoid camelCase or SCREAMING_SNAKE_CASE for components and top-level constants.\n- **Place domain constant lists in the page's `constant/` directory** (e.g. `pages/agent/constant/pipeline.tsx`) when they belong to an existing family of constants there (like `ChunkerOperators`, `SingleOperators`), instead of declaring them locally next to their consumer. Keeping related lists in one file makes their overlaps and differences visible and gives new entries a single obvious home.\n- **Name things semantically, not generically.** A name should say what the value holds or what the callback does, so call sites read naturally. Don't leak a generic underlying-primitive API into your own component's public props — rename at the wrapper boundary (e.g., a tabs wrapper exposes `activeTab` / `onTabChange`, mapped internally to Radix's `value` / `onValueChange`), and align with what callers already name the state. Same for variables and functions: `hiddenNodeIds` over `nodeIdSet`, `pickByBackend` over `resolveVariant`.\n- Avoid duplicating component structures in JSX; favor render props or reusable components.\n\n### Utility Libraries and Reuse\n\n- **Time/date handling**: Use `dayjs` for all date/time formatting, parsing, and manipulation.\n- **Utility hooks**: Prefer `ahooks` for common reusable hooks (e.g., `useDebounce`, `useSetState`).\n- **General utilities**: Lodash is available for utility functions when needed.\n- **Project utilities first**: Before reaching for a third-party library, check if the project already has an existing utility or hook that covers the need.\n- **Extract and share**: If repeated logic cannot be satisfied by an existing project utility or a third-party library, extract it into an appropriate shared hook (`src/hooks/`) or utility file (`src/utils/`).\n- **Check for duplicate patterns before adding**: When asked to add logic (validation, existence checks, API calls, etc.), first search for existing hooks/functions that do the same or similar thing — especially in the same file or sibling hooks. If a hook already does X, call it instead of re-implementing X inline. This applies to mutations calling mutations, utility wrappers, and boilerplate around API calls.\n","isInternal":false,"tokens":2773,"sizeBytes":12118},{"name":"named.md","path":".agents/rules/named.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/.agents/rules/named.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"# Go Naming Best Practices\n\n## 1. Package Naming\n\n- **All lowercase, no underscores**: `package user`, not `package userService` or `package user_service`\n- **Short and meaningful**: `package http`, `package json`, `package dao`\n- **Avoid plurals**: `package user` not `package users`\n- **Avoid generic names**: Avoid `package util`, `package common`, `package base`\n\n```go\n// Recommended\npackage user\npackage handler\npackage service\n\n// Not recommended\npackage UserService\npackage user_service\npackage utils\n```\n\n## 2. File Naming\n\n- **All lowercase, underscore separated**: `user_handler.go`, `user_service.go`\n- **Test files**: `user_handler_test.go`\n- **Platform-specific**: `user_linux.go`, `user_windows.go`\n\n```\nuser/\n├── user_handler.go\n├── user_service.go\n├── user_dao.go\n└── user_test.go\n```\n\n## 3. Directory Naming\n\n- **All lowercase, no underscores or hyphens**: `internal/`, `pkg/`, `cmd/`\n- **Short and descriptive**: `handler/`, `service/`, `dao/`\n\n```\nproject/\n├── cmd/ # Main entry point\n│ └── server_main.go\n├── internal/ # Private code\n│ ├── handler/\n│ ├── service/\n│ ├── dao/\n│ ├── model/\n│ └── middleware/\n├── pkg/ # Public code\n└── api/ # API definitions\n```\n\n## 4. Interface Naming\n\n- **Single-method interfaces end with \"-er\"**: `Reader`, `Writer`, `Handler`\n- **Verb form**: `Reader`, `Executor`, `Validator`\n\n```go\n// Recommended\ntype Reader interface {\n Read(p []byte) (n int, err error)\n}\n\ntype UserService interface {\n Register(req *RegisterRequest) (*User, error)\n Login(req *LoginRequest) (*User, error)\n}\n\n// Not recommended\ntype UserInterface interface {}\ntype IUserService interface {}\n```\n\n## 5. Struct Naming\n\n- **CamelCase**: `UserService`, `UserHandler`\n- **Avoid redundant prefixes**: `User` not `UserModel`\n\n```go\n// Recommended\ntype UserService struct {}\ntype UserHandler struct {}\ntype RegisterRequest struct {}\n\n// Not recommended\ntype user_service struct {}\ntype SUserService struct {}\ntype UserModel struct {}\n```\n\n## 6. Method/Function Naming\n\n- **CamelCase**\n- **Start with verb**: `GetUser`, `CreateUser`, `DeleteUser`\n- **Boolean returns use Is/Has/Can prefix**: `IsValid`, `HasPermission`\n\n```go\n// Recommended\nfunc (s *UserService) Register(req *RegisterRequest) (*User, error)\nfunc (s *UserService) GetUserByID(id uint) (*User, error)\nfunc (s *UserService) IsEmailExists(email string) bool\n\n// Not recommended\nfunc (s *UserService) register_user()\nfunc (s *UserService) get_user_by_id()\nfunc (s *UserService) CheckEmailExists() // Should use Is/Has\n```\n\n## 7. Constant Naming\n\n- **CamelCase**: `const MaxRetryCount = 3`\n- **Enum constants**: `const StatusActive = \"active\"`\n\n```go\n// Recommended\nconst (\n StatusActive = \"1\"\n StatusInactive = \"0\"\n MaxRetryCount = 3\n)\n\n// Not recommended\nconst (\n STATUS_ACTIVE = \"1\" // Not all uppercase\n status_active = \"1\" // Not all lowercase\n)\n```\n\n## 8. Error Variable Naming\n\n- **Start with \"Err\"**: `ErrNotFound`, `ErrInvalidInput`\n\n```go\n// Recommended\nvar (\n ErrNotFound = errors.New(\"not found\")\n ErrInvalidInput = errors.New(\"invalid input\")\n ErrUnauthorized = errors.New(\"unauthorized\")\n)\n```\n\n## 9. Acronyms Keep Consistent Case\n\n```go\n// Recommended\ntype HTTPHandler struct {}\nvar URL string\nfunc GetHTTPClient() {}\nfunc ParseJSON() {}\n\n// Not recommended\ntype HttpHandler struct {}\nvar Url string\nfunc GetHttpClient() {}\n```\n\n## 10. Project Structure Naming\n\n```\nproject-name/\n├── cmd/ # Main programs\n│ └── app_name/\n│ └── main.go\n├── internal/ # Private code\n│ ├── handler/ # HTTP handlers\n│ ├── service/ # Business logic\n│ ├── repository/ # Data access\n│ ├── model/ # Data models\n│ └── config/ # Configuration\n├── pkg/ # Public code\n├── api/ # API definitions\n├── configs/ # Config files\n├── scripts/ # Scripts\n├── docs/ # Documentation\n├── go.mod\n└── go.sum\n```\n\n## Summary Table\n\n| Type | Rule | Example |\n| -------------- | ----------------------------------- | ------------------- |\n| Package | All lowercase, no underscores | `package user` |\n| File | All lowercase, underscore separated | `user_service.go` |\n| Directory | All lowercase, no separators | `internal/handler/` |\n| Struct | CamelCase, capitalized first letter | `UserService` |\n| Interface | CamelCase, -er suffix | `Reader`, `Writer` |\n| Method | CamelCase, verb prefix | `GetUserByID` |\n| Constant | CamelCase | `MaxRetryCount` |\n| Error Variable | Err prefix | `ErrNotFound` |\n","isInternal":false,"tokens":1221,"sizeBytes":5093},{"name":"SKILL.md","path":".agents/skills/go-naming/SKILL.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/.agents/skills/go-naming/SKILL.md","title":"go-naming","category":"anthropic-skill","format":"markdown","content":"---\nname: go-naming\ndescription: Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards.\n---\n\nStrictly follow the naming conventions in [rules/named.md](../../rules/named.md)\n","frontmatter":{"name":"go-naming","description":"Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards."},"isInternal":false,"tokens":74,"sizeBytes":381},{"name":"_category_.json","path":"docs/references/_category_.json","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/docs/references/_category_.json","title":"References Skill","category":"anthropic-skill","format":"json","content":"{\n \"label\": \"References\",\n \"position\": 6,\n \"link\": {\n \"type\": \"generated-index\",\n \"description\": \"Miscellaneous References\"\n },\n \"customProps\": {\n \"sidebarIcon\": \"LucideScrollText\"\n }\n}\n","frontmatter":{"label":"References","position":6,"link":{"type":"generated-index","description":"Miscellaneous References"},"customProps":{"sidebarIcon":"LucideScrollText"}},"isInternal":false,"tokens":59,"sizeBytes":201},{"name":"http_api_reference.md","path":"docs/references/http_api_reference.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/docs/references/http_api_reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"---\nsidebar_position: 4\nslug: /http_api_reference\nsidebar_custom_props: {\n categoryIcon: LucideGlobe\n}\n---\n# HTTP API\n\nA complete reference for RAGFlow's RESTful API, including migration notes for the 1.0 Go backend. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).\n\nIn the standard response envelope, `message` is always a non-empty string. Successful responses default to `\"success\"`; error responses contain a description of the error.\n\n---\n\n## ERROR CODES\n\nRAGFlow responses may contain both an HTTP status code and a business code in the JSON response body. These codes should be checked separately.\n\n### HTTP status codes\n\n| Code | Meaning |\n|------|---------|\n| 200 | The HTTP request was processed successfully. Check the response body `code` for the business result. |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Internal server error |\n\n### Response body codes\n\n| Code | Meaning |\n|------|---------|\n| 0 | Success |\n| 10 | Not effective |\n| 100 | Exception error |\n| 101 | Invalid request argument |\n| 102 | Invalid or missing data |\n| 103 | Operation error |\n| 105 | Connection error |\n| 106 | Operation still running |\n| 108 | Permission error |\n| 109 | Authentication error |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Server error |\n---\n\n## Migrate to 1.0\n\nRAGFlow 1.0 moves the public REST API from the Python backend to Go. The canonical endpoints documented below remain available.\n\nThe following changes require client updates:\n\n| Area | RAGFlow 1.0 Go change | Client action |\n|------|------------------------|---------------|\n| Dataset parser selection | Dataset create and update use `parse_type` with `parser_id` (BuiltIn) or `pipeline_id` (Pipeline); responses use the canonical value `general` instead of the legacy `chunk_method: \"naive\"`. | Replace `chunk_method` with `\"parse_type\": 1` and `\"parser_id\": \"\"`. For the general parser, send `\"parser_id\": \"general\"`. |\n| Dataset parser configuration | Go stores and returns component-scoped parser settings. Most legacy top-level `parser_config` fields are not mapped into that component structure. | Read the returned component map and send component-scoped keys when changing parser settings. |\n| Errors and authentication | Some validation business codes, messages, and authentication HTTP statuses differ from Python. | Branch on documented codes rather than exact message text. |\n\n:::note\nGo responses include `X-API-Source: go`, which can be used to verify the serving backend during rollout.\n:::\n\n### Removed API aliases\n\nThe following paths remain available only on the Python backend. They are not registered by the RAGFlow 1.0 Go backend.\n\n| Removed endpoint | Replacement endpoint |\n|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|\n| **POST** `/api/v1/chats_openai/{chat_id}/chat/completions` | **POST** `/api/v1/openai/{chat_id}/chat/completions` |\n| **PUT** `/api/v1/chats/{chat_id}/sessions/{session_id}` | **PATCH** `/api/v1/chats/{chat_id}/sessions/{session_id}` |\n| **POST** `/api/v1/chats/{chat_id}/completions` | **POST** `/api/v1/chat/completions` |\n| **POST** `/api/v1/sessions/related_questions` | **POST** `/api/v1/chat/recommendation` |\n| **PUT** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` | **PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` |\n| **GET** `/v1/system/healthz` | **GET** `/api/v1/system/healthz` |\n| **POST** `/v1/document/upload_info` | **POST** `/api/v1/documents/upload` |\n| **POST** `/api/v1/file/upload` | **POST** `/api/v1/files` |\n| **POST** `/api/v1/file/create` | **POST** `/api/v1/files` |\n| **GET** `/api/v1/file/list` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/root_folder` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/parent_folder` | **GET** `/api/v1/files/{file_id}/parent` |\n| **GET** `/api/v1/file/all_parent_folder` | **GET** `/api/v1/files/{file_id}/ancestors` |\n| **POST** `/api/v1/file/rm` | **DELETE** `/api/v1/files` |\n| **POST** `/api/v1/file/rename` | **POST** `/api/v1/files/move` |\n| **GET** `/api/v1/file/get/{file_id}` | **GET** `/api/v1/files/{file_id}` |\n| **POST** `/api/v1/file/mv` | **POST** `/api/v1/files/move` |\n| **POST** `/api/v1/file/convert` | **POST** `/api/v1/files/link-to-datasets` |\n| **POST** `/api/v1/agents_openai/{agent_id}/chat/completions` | **POST** `/api/v1/agents/chat/completions` with `\"openai-compatible\": true` |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\n**POST** `/api/v1/openai/{chat_id}/chat/completions`\n\nCreates a model response for a given chat conversation.\n\n:::caution REMOVED ALIAS\n`POST /api/v1/chats_openai/{chat_id}/chat/completions` is not available in 1.0. Use this endpoint instead.\n:::\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/openai/{chat_id}/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"model\"`: `string` (optional)\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"extra_body\"`: `object` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/openai/{chat_id}/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"model\": \"glm-4-flash@ZHIPU-AI\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true,\n \"extra_body\": {\n \"reference\": true,\n \"reference_metadata\": {\n \"include\": true,\n \"fields\": [\"author\", \"year\", \"source\"]\n },\n \"metadata_condition\": {\n \"logic\": \"and\",\n \"conditions\": [\n {\n \"name\": \"author\",\n \"comparison_operator\": \"is\",\n \"value\": \"bob\"\n }\n ]\n }\n }\n }'\n```\n\n##### Request parameters\n\n- `chat_id` (*Path parameter*) `string`, *Required*\n Existing chat assistant ID. The request will use that chat assistant's knowledge and settings.\n\n- `model` (*Body parameter*) `string`\n The model used to generate the response. When `chat_id` is provided, you may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `extra_body` (*Body parameter*) `object`\n Extra request parameters:\n - `reference`: `boolean` - include reference in the final chunk (stream) or in the final message (non-stream).\n - `reference_metadata`: `object` - include document metadata in each reference chunk.\n - `include`: `boolean` - enable document metadata in reference chunks.\n - `fields`: `list[string]` - optional allowlist of metadata keys. Omit to include all. Use an empty list to include none.\n - `metadata_condition`: `object` - metadata filter conditions applied to retrieval results.\n\n#### Response\n\nStream:\n\n```json\ndata:{\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"Hello! It seems like you're just greeting me. If you have a specific\",\n \"role\": \"assistant\",\n \"function_call\": null,\n \"tool_calls\": null,\n \"reasoning_content\": null\n },\n \"finish_reason\": null,\n \"index\": 0,\n \"logprobs\": null\n }\n ],\n \"created\": 1755084508,\n \"model\": \"model\",\n \"object\": \"chat.completion.chunk\",\n \"system_fingerprint\": \"\",\n \"usage\": null\n}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" question or need information, feel free to ask, and I'll do my best\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" to assist you based on the knowledge base provided.\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": null, \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": \"stop\", \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": {\"prompt_tokens\": 5, \"completion_tokens\": 188, \"total_tokens\": 193}}\n\ndata:[DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"Hello! I'm your smart assistant. What can I do for you?\",\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": 1755084403,\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"model\": \"model\",\n \"object\": \"chat.completion\",\n \"usage\": {\n \"completion_tokens\": 55,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 55,\n \"reasoning_tokens\": 5,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 5,\n \"total_tokens\": 60\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n---\n\n### Create OpenAI-compatible agent completion\n\n:::caution REMOVED ALIAS\nRAGFlow 1.0 Go does not serve `POST /api/v1/agents_openai/{agent_id}/chat/completions`. Use the endpoint below and move `agent_id` into the request body.\n:::\n\n**POST** `/api/v1/agents/chat/completions`\n\nCreates a model response for a given chat conversation.\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/agents/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"agent_id\"`: `string`\n - `\"openai-compatible\"`: `boolean`\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"session_id\"`: `string` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/agents/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"agent_id\": \"{agent_id}\",\n \"openai-compatible\": true,\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true\n }'\n```\n\n##### Body parameters\n\n- `agent_id` (*Body parameter*) `string`, *Required*\n The ID of the agent to run.\n\n- `openai-compatible` (*Body parameter*) `boolean`, *Required*\n Set this field to `true` to use the OpenAI-compatible request and response format.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `session_id` (*Body parameter*) `string`\n Agent session id.\n\n#### Response\n\nStream:\n\n```json\n...\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \" terminal\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \".\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"id\": \"4b8935ac0a22deb1\",\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"url\": null,\n \"similarity\": 0.5697155305154673,\n \"vector_similarity\": 0.7323851005515574,\n \"term_similarity\": 0.5000000005,\n \"doc_type\": \"\"\n }\n },\n \"doc_aggs\": {\n \"INSTALL22.md\": {\n \"doc_name\": \"INSTALL22.md\",\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"count\": 3\n },\n \"INSTALL.md\": {\n \"doc_name\": \"INSTALL.md\",\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL(1).md\": {\n \"doc_name\": \"INSTALL(1).md\",\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL3.md\": {\n \"doc_name\": \"INSTALL3.md\",\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"count\": 1\n }\n }\n }\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: [DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"\\nTo install Neovim, the process varies depending on your operating system:\\n\\n### For Windows:\\n1. **Download from GitHub**: \\n - Visit the [Neovim releases page](https://github.com/neovim/neovim/releases)\\n - Download the latest Windows installer (nvim-win64.msi)\\n - Run the installer and follow the prompts\\n\\n2. **Using winget** (Windows Package Manager):\\n...\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"doc_type\": \"\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"id\": \"4b8935ac0a22deb1\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"similarity\": 0.5697155305154673,\n \"term_similarity\": 0.5000000005,\n \"url\": null,\n \"vector_similarity\": 0.7323851005515574\n }\n },\n \"doc_aggs\": {\n \"INSTALL(1).md\": {\n \"count\": 2,\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL(1).md\"\n },\n \"INSTALL.md\": {\n \"count\": 2,\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL.md\"\n },\n \"INSTALL22.md\": {\n \"count\": 3,\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL22.md\"\n },\n \"INSTALL3.md\": {\n \"count\": 1,\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL3.md\"\n }\n }\n },\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": null,\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"object\": \"chat.completion\",\n \"param\": null,\n \"usage\": {\n \"completion_tokens\": 415,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 0,\n \"reasoning_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 6,\n \"total_tokens\": 421\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n**POST** `/api/v1/datasets`\n\nCreates a dataset.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### A basic request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test_1\",\n \"language\": \"English\",\n \"parse_type\": 1,\n \"parser_id\": \"general\"\n }'\n```\n\n##### A request example specifying ingestion pipeline\n\n:::caution WARNING\nDo not include `\"parser_id\"` when specifying an ingestion pipeline.\n:::\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test-sdk\",\n \"language\": \"English\",\n \"parse_type\": 2,\n \"pipeline_id\": \"\"\n }'\n```\n\n##### Body parameters\n\n- `\"name\"`: `string`, *Required*\n The unique name of the dataset to create. It must adhere to the following requirements:\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n If the name is already taken, the dataset is created with a numbered suffix such as `test_1(1)`, `test_1(2)`, and so on.\n\n- `\"language\"`: `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the server/database default is used.\n\n- `\"embedding_model\"`: `string`\n The name of the embedding model to use. For example: `\"BAAI/bge-large-zh-v1.5@BAAI\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n\n- `\"permission\"`: `string`\n Specifies who can access the dataset to create. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n\n- `\"parser_id\"`: `enum`\n The built-in parser to use. Required when `\"parse_type\"` is `1`.\n - `\"general\"`: General (default)\n - `\"qa\"`: Q&A\n - `\"manual\"`: Manual\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"audio\"`: Audio\n - `\"email\"`: Email\n - Must be omitted when `\"parse_type\"` is `2`.\n\n- `\"parser_config\"`: `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n- `\"parse_type\"`: `int`, *Required*\n The parser selection mode:\n - `1`: BuiltIn. Requires `\"parser_id\"`.\n - `2`: Pipeline. Requires `\"pipeline_id\"`.\n\n- `\"pipeline_id\"`: `string`\n The ingestion pipeline ID. Can be found in the corresponding URL in the RAGFlow UI.\n - Required when `\"parse_type\"` is `2`.\n - Must be a 32-character lowercase hexadecimal string, e.g., `\"d0bebe30ae2211f0970942010a8e0005\"`.\n - Must be omitted when `\"parse_type\"` is `1`.\n\n:::caution WARNING\n`\"parse_type\"` is required when creating a dataset. Choose exactly one mode:\n\n- `\"parse_type\": 1` with `\"parser_id\"` for a builtin parser.\n- `\"parse_type\": 2` with `\"pipeline_id\"` for an ingestion pipeline.\n\n`\"chunk_method\"`, `\"avatar\"`, `\"description\"`, and `\"auto_metadata_config\"` are not accepted by the RAGFlow 1.0 create endpoint. Use `\"parser_id\"` instead of `\"chunk_method\"`; set avatar and description with the update endpoint, and metadata configuration with the metadata endpoint.\n:::\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"chunk_count\": 0,\n \"create_time\": 1745836841611,\n \"created_by\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"document_count\": 0,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5@BAAI\",\n \"id\": \"3b4de7d4241d11f0a6a79f24fc270c7f\",\n \"language\": \"English\",\n \"name\": \"RAGFlow example\",\n \"pagerank\": 0,\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 128,\n \"delimiters\": [\"\\n\", \"!?;。；！？\"],\n \"parent_child\": {\n \"use_parent_child\": false,\n \"children_delimiter\": \"\\n\"\n }\n }\n },\n \"parser_id\": \"general\",\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"tenant_id\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"token_num\": 0,\n \"keywords_similarity_weight\": 0.7\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"Field: - Message: - Value: <>\"\n}\n```\n\n---\n\n### Delete datasets\n\n**DELETE** `/api/v1/datasets`\n\nDeletes datasets by ID.\n\n#### Request\n\n- Method: DELETE\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"ids\"`: `list[string]` or `null`\n - `\"delete_all\"`: `boolean`\n\n##### Request example\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"ids\": [\"d94a8dc02c9711f0930f7fbc369eab6d\", \"e94a8dc02c9711f0930f7fbc369eab6e\"]\n }'\n```\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"delete_all\": true\n }'\n```\n\n##### Body parameters\n\n- `\"ids\"`: `list[string]` or `null`\n Specifies the datasets to delete:\n - If omitted, or set to `null` or an empty array, no datasets are deleted.\n - If an array of IDs is provided, only the datasets matching those IDs are deleted.\n- `\"delete_all\"`: `boolean`\n Whether to delete all datasets owned by the current user when `\"ids\"` is omitted, or set to `null` or an empty array. Defaults to `false`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\":108,\n \"message\":\"User '' lacks permission for datasets: ''\"\n}\n\n```\n\n---\n\n### Update dataset\n\n**PUT** `/api/v1/datasets/{dataset_id}`\n\nUpdates configurations for a specified dataset.\n\n#### Request\n\n- Method: PUT\n- URL: `/api/v1/datasets/{dataset_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"avatar\"`: `string`\n - `\"description\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"pagerank\"`: `int`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### Request example\n\n```bash\ncurl --request PUT \\\n --url http://{address}/api/v1/datasets/{dataset_id} \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '\n {\n \"name\": \"updated_dataset\"\n }'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to update.\n- `\"name\"`: (*Body parameter*), `string`\n The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"language\"`: (*Body parameter*), `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the existing dataset language remains unchanged.\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n Refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration).\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser selection mode: `1` for a builtin parser (`parser_id` required) or `2` for an ingestion pipeline (`pipeline_id` required). Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`; must be omitted when `parse_type` is `2`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n`chunk_method` is not supported by RAGFlow 1.0; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"Can't change tenant_id.\"\n}\n```\n\n---\n\n### List datasets\n\n**GET** `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n\nLists datasets.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n- Headers:\n - `'Authorization: Bearer '`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id} \\\n --header 'Authorization: Bearer '\n```\n\n```bash\n# List datasets with parsing status\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets?include_parsing_status=true' \\\n --header 'Authorization: Bearer '\n```\n\n##### Query parameters\n\n- `page`\n Specifies the page on which the datasets will be displayed. Defaults to `1`.\n- `page_size`\n The number of datasets on each page. Defaults to `30`.\n- `orderby`\n The field by which datasets should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`\n Indicates whether the retrieved datasets should be sorted in descending order. Defaults to `true`.\n- `sort`\n Orders by several fields at once, written as `column:direction` terms separated by commas, such as `name:asc,create_time:desc`. A term with no direction is ascending. `sort` takes precedence over `orderby` and `desc`, which keep working on their own. A term naming a field this list does not sort by is skipped. When `sort` yields no terms at all, `orderby` and `desc` decide the order. When it yields terms whose fields are all skipped, the result is ordered by `create_time` in the direction of the first term, and the older pair is not read.\n- `name`\n The name of the dataset to retrieve.\n- `id`\n The ID of the dataset to retrieve.\n- `include_parsing_status`\n Whether to include document parsing status counts in the response. Defaults to `false`. When set to `true`, each dataset object in the response will include the following additional fields:\n - `unstart_count`: Number of documents not yet started parsing.\n - `running_count`: Number of documents currently being parsed.\n - `cancel_count`: Number of documents whose parsing was canceled.\n - `done_count`: Number of documents that have been successfully parsed.\n - `fail_count`: Number of documents whose parsing failed.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": \"\",\n \"chunk_count\": 59,\n \"create_date\": \"Sat, 14 Sep 2024 01:12:37 GMT\",\n \"create_time\": 1726276357324,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"description\": null,\n \"document_count\": 1,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5\",\n \"id\": \"6e211ee0723611efa10a0242ac120007\",\n \"language\": \"English\",\n \"name\": \"mysql\",\n \"parser_id\": \"general\",\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 8192,\n \"delimiters\": [\"\\n\"]\n }\n },\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"69736c5e723611efb51b0242ac120007\",\n \"token_num\": 12744,\n \"update_date\": \"Thu, 10 Oct 2024 04:07:23 GMT\",\n \"update_time\": 1728533243536,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nSuccess (with `include_parsing_status=true`):\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": null,\n \"cancel_count\": 0,\n \"chunk_count\": 30,\n \"parser_id\": \"qa\",\n \"create_date\": \"2026-03-09T18:57:13\",\n \"create_time\": 1773053833094,\n \"created_by\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"description\": null,\n \"document_count\": 1,\n \"done_count\": 1,\n \"embedding_model\": \"text-embedding-v2@Tongyi-Qianwen\",\n \"fail_count\": 0,\n \"id\": \"ba6586c21ba611f1a3dc476f0709e75e\",\n \"language\": \"English\",\n \"name\": \"Test Dataset\",\n \"parser_config\": {\n \"Extractor:AutoExtractDefault\": {\n \"llm_id\": \"deepseek-chat@DeepSeek\"\n }\n },\n \"permission\": \"me\",\n \"running_count\": 0,\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"token_num\": 1746,\n \"unstart_count\": 0,\n \"update_date\": \"2026-03-09T18:59:32\",\n \"update_time\": 1773053972723,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset doesn't exist\"\n}\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n**POST** `/api/v1/datasets/{dataset_id}/documents`\n\nUploads documents to a specified dataset.\n\nThis endpoint supports three creation modes via the optional `type` query parameter:\n\n- `type=local` or omitted: Upload one or more local files using `multipart/form-data`.\n- `type=web`: Crawl a web page and save it as a document.\n- `type=empty`: Create an empty virtual document by name.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets/{dataset_id}/documents`\n- Query:\n - `type`: Optional. One of `local`, `web`, or `empty`. Defaults to `local`.\n- Headers:\n - `'Content-Type: multipart/form-data'` for `type=local` and `type=web`\n - `'Content-Type: application/json'` for `type=empty`\n - `'Authorization: Bearer '`\n- Body:\n - For `type=local`: form field `'file=@{FILE_PATH}'`\n - For `type=web`: form fields `'name'` and `'url'`\n - For `type=empty`: JSON body with `'name'`\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'file=@./test1.txt' \\\n --form 'file=@./test2.pdf'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=web' \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'name=example-page' \\\n --form 'url=https://example.com'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=empty' \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\"name\":\"blank.txt\"}'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to which the documents will be uploaded.\n- `type`: (*Query parameter*)\n Controls how the document is created:\n - `local`: Upload files.\n - `web`: Crawl a URL into a document.\n - `empty`: Create an empty document without file upload.\n- `'file'`: (*Body parameter*)\n A document to upload. Required when `type=local`.\n- `'name'`: (*Body parameter*)\n The document name. Required when `type=web` or `type=empty`.\n- `'url'`: (*Body parameter*)\n The source URL to crawl. Required when `type=web`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"chunk_method\": \"naive\",\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"dataset_id\": \"527fa74891e811ef9c650242ac120006\",\n \"id\": \"b330ec2e91ec11efbc510242ac120004\",\n \"location\": \"1.txt\",\n \"name\": \"1.txt\",\n \"parser_config\": {\n \"chunk_token_num\": 128,\n \"delimiter\": \"\\\\n\",\n \"html4excel\": false,\n \"layout_recognize\": \"DeepDOC\"\n },\n \"run\": \"UNSTART\",\n \"size\": 17966,\n \"thumbnail\": \"\",\n \"type\": \"doc\"\n }\n ]\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"No file part!\"\n}\n```\n\n---\n\n### Update document\n\n**PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nUpdates configurations for a specified document.\n\nSaving parser settings preserves existing chunks, chunk images, counters, and parsing progress. The saved settings take effect on the next parse; this update does not start parsing or delete existing results. Use the document parse action to clear existing chunks and re-parse with the saved settings.\n\n#### Request\n\n- Method: PATCH\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`:`string`\n - `\"meta_fields\"`:`object`\n - `\"chunk_method\"`:`string`\n - `\"parser_config\"`:`object`\n\n##### Request example\n\n```bash\ncurl --request PATCH \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --header 'Content-Type: application/json' \\\n --data '\n {\n \"name\": \"manual.txt\",\n \"chunk_method\": \"manual\",\n \"parser_config\": {\"chunk_token_num\": 128}\n }'\n\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the associated dataset.\n- `document_id`: (*Path parameter*)\n The ID of the document to update.\n- `\"name\"`: (*Body parameter*), `string`\n- `\"meta_fields\"`: (*Body parameter*), `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: (*Body parameter*), `string`\n The parsing method to apply to the document:\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. The attributes in this JSON object vary with the selected `\"chunk_method\"`:\n - If `\"chunk_method\"` is `\"naive\"`, the `\"parser_config\"` object contains the following attributes:\n - `\"chunk_token_num\"`: Defaults to `256`.\n - `\"layout_recognize\"`: Defaults to `\"DeepDOC\"`.\n - `\"html4excel\"`: Indicates whether to convert Excel documents into HTML format. Defaults to `false`.\n - `\"delimiter\"`: Defaults to `\"\\n\"`.\n - `\"task_page_size\"`: Defaults to `12`. For PDF only.\n- `\"enabled\"`: (*Body parameter*), `integer`\n Whether the document should be **available** in the knowledge base.\n - `1` → （available）\n - `0` → （unavailable）\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"id\": \"cd38dd72d4a611f0af9c71de94a988ef\",\n \"name\": \"large.md\",\n \"type\": \"doc\",\n \"suffix\": \"md\",\n \"size\": 2306906,\n \"location\": \"large.md\",\n \"source_type\": \"local\",\n \"status\": \"1\",\n \"run\": \"DONE\",\n \"dataset_id\": \"5f546a1ad4a611f0af9c71de94a988ef\",\n\n \"chunk_method\": \"naive\",\n \"chunk_count\": 2,\n \"token_count\": 8126,\n\n \"created_by\": \"eab7f446cb5a11f0ab334fbc3aa38f35\",\n \"create_date\": \"Tue, 09 Dec 2025 10:28:52 GMT\",\n \"create_time\": 1765247332122,\n \"update_date\": \"Wed, 17 Dec 2025 10:51:16 GMT\",\n \"update_time\": 1765939876819,\n\n \"process_begin_at\": \"Wed, 17 Dec 2025 10:33:55 GMT\",\n \"process_duration\": 14.8615,\n \"progress\": 1.0,\n\n \"progress_msg\": [\n \"10:33:58 Task has been received.\",\n \"10:33:59 Page(1~100000001): Start to parse.\",\n \"10:33:59 Page(1~100000001): Finish parsing.\",\n \"10:34:07 Page(1~100000001): Generate 2 chunks\",\n \"10:34:09 Page(1~100000001): Embedding chunks (2.13s)\",\n \"10:34:09 Page(1~100000001): Indexing done (0.31s).\",\n \"10:34:09 Page(1~100000001): Task done (11.68s)\"\n ],\n\n \"parser_config\": {\n \"chunk_token_num\": 512,\n \"delimiter\": \"\\n\",\n \"auto_keywords\": 0,\n \"auto_questions\": 0,\n \"topn_tags\": 3,\n \"layout_recognize\": \"DeepDOC\",\n \"html4excel\": false,\n \"image_context_size\": 0,\n \"table_context_size\": 0\n },\n\n \"meta_fields\": {},\n \"pipeline_id\": \"\",\n \"thumbnail\": \"\"\n }\n}\n\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset does not have the document.\"\n}\n```\n\n---\n\n### Download document\n\n**GET** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nDownloads a document from a specified dataset.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Authorization: Bearer '`\n- Output:\n - `'{PATH_TO_THE_FILE}'`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --output ./ragflow.txt\n```\n\n##### Path parameters\n\n- `dataset_id`\n The associated dataset ID.\n- `document_id`\n The ID of the document to download.\n\n#### Response\n\nSuccess:\n\n```json\nThis is a test to verify the file download feature.\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"You do not own the dataset 7898da028a0511efbf750242ac1220005.\"\n}\n```\n\n---\n\n### List documents\n\n**GET** `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n\nLists documents in a specified dataset.\n\nTo retrieve a specific document's settings and metadata, pass its document ID in the `id` query parameter: `/api/v1/datasets/{dataset_id}/documents?id={document_id}`.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n\n##### Request examples\n\n**A basic request with pagination:**\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents?page=1&page_size=10 \\\n --header 'Authorization: Bearer '\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The associated dataset ID.\n- `keywords`: (*Query parameter*), `string`\n The keywords used to match document titles.\n- `page`: (*Query parameter*), `integer`\n Specifies the page on which the documents will be displayed. Defaults to `1`.\n- `page_size`: (*Query parameter*), `integer`\n The maximum number of documents on each page. Defaults to `30`.\n- `orderby`: (*Query parameter*), `string`\n The field by which documents should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`: (*Query parameter*), `boolean`\n Indicates whether the retrieved documents should be sorted in descending order. Defaults to `true`.\n- `id`: (*Query parameter*), `string`\n The ID of the document to retrieve.\n- `create_time_from`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created after this time. 0 means no filter. Defaults to `0`.\n- `create_time_to`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created before this time. 0 means no filter. Defaults to `0`.\n- `suffix`: (*Query parameter*), `array[string]`\n Filter by file suffix. Supports multiple values, e.g., `pdf`, `txt`, and `docx`. Defaults to all suffixes.\n- `run`: (*Query parameter*), `array[string]`\n Filter by document processing status. Supports numeric, text, and mixed formats:\n - Numeric format: `[\"0\", \"1\", \"2\", \"3\", \"4\", \"5\"]`\n - Text format: `[UNSTART, RUNNING, CANCEL, DONE, FAIL, SCHEDULE]`\n - Mixed format: `[UNSTART, 1, DONE]` (mixing numeric and text formats)\n - Status mapping:\n - `0` / `UNSTART`: Document not yet processed\n - `1` / `RUNNING`: Document is currently being processed\n - `2` / `CANCEL`: Document processing was canceled\n - `3` / `DONE`: Document processing completed successfully\n - `4` / `FAIL`: Document processing failed\n - `5` / `SCHEDULE`: Document is scheduled and waiting to be processed\n Defaults to all statuses.\n- `metadata_condition`: (*Query parameter*), `object` (JSON in query)\n Optional metadata filter applied to documents when `document_ids` is not provided. Uses the same structure as retrieval:\n - `logic`: `\"and\"` (default) or `\"or\"`\n - `conditions`: array of `{ \"name\": string, \"comparison_operator\": string, \"value\": string }`\n - `comparison_operator` supports: `is`, `not is`, `contains`, `not contains`, `in`, `not in`, `start with`, `end with`, `>`, `<`, `≥`, `≤`, `empty`, `not empty`\n\n##### Usage examples\n\n**A request with multiple filtering parameters**\n\n```bash\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?suffix=pdf&run=DONE&page=1&page_size=10' \\\n --header 'Authorization: Bearer '\n```\n\n**Filter by metadata (query JSON):**\n\n```bash\ncurl -G \\\n --url \"http://{address}/api/v1/datasets/{dataset_id}/documents\" \\\n --header 'Authorization: Bearer ' \\\n --data-urlencode 'metadata_condition={\"logic\":\"and\",\"conditions\":[{\"name\":\"tags\",\"comparison_operator\":\"is\",\"value\":\"bar\"},{\"name\":\"author\",\"comparison_operator\":\"is\",\"value\":\"alice\"}]}'\n```\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"docs\": [\n {\n \"chunk_count\": 0,\n \"create_date\": \"Mon, 14 Oct 2024 09:11:01 GMT\",\n \"create_time\": 1728897061948,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"id\": \"3bcfbf8a8a0c11ef8aba0242ac120006\",\n \"knowledgebase_id\": \"7898da028a0511efbf750242ac120005\",\n \"location\": \"Test_2.txt\",\n \"name\": \"Test_2.txt\",\n \"parser_config\": {\n \"chunk_token_count\": 128,\n \"delimiter\": \"\\n\",\n \"layout_recognize\": \"DeepDOC\",\n \"task_page_size\": 12\n },\n ","frontmatter":{"sidebar_position":4,"slug":"/http_api_reference","sidebar_custom_props":{"categoryIcon":"LucideGlobe"}},"isInternal":false,"tokens":12743,"sizeBytes":50024},{"name":"python_api_reference.md","path":"docs/references/python_api_reference.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/docs/references/python_api_reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"---\nsidebar_position: 5\nslug: /python_api_reference\nsidebar_custom_props: {\n categoryIcon: SiPython\n}\n---\n# Python API\n\nA complete reference for RAGFlow's Python APIs. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).\n\n:::tip NOTE\nRun the following command to download the Python SDK:\n\n```bash\npip install ragflow-sdk\n```\n\n:::\n\n---\n\n## ERROR CODES\n\nRAGFlow responses may contain both an HTTP status code and a business code in the JSON response body. These codes should be checked separately.\n\n### HTTP status codes\n\n| Code | Meaning |\n|------|---------|\n| 200 | The HTTP request was processed successfully. Check the response body `code` for the business result. |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Internal server error |\n\n### Response body codes\n\n| Code | Meaning |\n|------|---------|\n| 0 | Success |\n| 10 | Not effective |\n| 100 | Exception error |\n| 101 | Invalid request argument |\n| 102 | Invalid or missing data |\n| 103 | Operation error |\n| 105 | Connection error |\n| 106 | Operation still running |\n| 108 | Permission error |\n| 109 | Authentication error |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Server error |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\nCreates a model response for the given historical chat conversation via OpenAI's API.\n\n#### Parameters\n\n##### chat_id: `string`, *Required*\n\nExisting chat assistant ID. This value is part of the request path: `/api/v1/openai//chat/completions`.\n\n##### model: `string`, *Required*\n\nThe model used to generate the response. You may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n##### messages: `list[object]`, *Required*\n\nA list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n##### stream: `boolean`\n\nWhether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n#### Returns\n\n- Success: Response [message](https://platform.openai.com/docs/api-reference/chat/create) like OpenAI\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom openai import OpenAI\nimport json\n\nmodel = \"glm-4-flash@ZHIPU-AI\"\nclient = OpenAI(api_key=\"ragflow-api-key\", base_url=\"http://ragflow_address/api/v1/openai//chat\")\n\nstream = True\nreference = True\n\nrequest_kwargs = dict(\n model=model,\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Who are you?\"},\n {\"role\": \"assistant\", \"content\": \"I am an AI assistant named...\"},\n {\"role\": \"user\", \"content\": \"Can you tell me how to install neovim\"},\n ],\n extra_body={\n \"reference\": reference,\n \"reference_metadata\": {\n \"include\": True,\n \"fields\": [\"author\", \"year\", \"source\"],\n },\n },\n)\n\nif stream:\n completion = client.chat.completions.create(stream=True, **request_kwargs)\n for chunk in completion:\n print(chunk)\nelse:\n resp = client.chat.completions.with_raw_response.create(\n stream=False, **request_kwargs\n )\n print(\"status:\", resp.http_response.status_code)\n raw_text = resp.http_response.text\n print(\"raw:\", raw_text)\n\n data = json.loads(raw_text)\n print(\"assistant:\", data[\"choices\"][0][\"message\"].get(\"content\"))\n print(\"reference:\", data[\"choices\"][0][\"message\"].get(\"reference\"))\n```\n\nWhen `extra_body.reference` is `true`, the streamed final chunk may include `choices[0].delta.reference`, and the non-stream response may include `choices[0].message.reference`.\n\nWhen `extra_body.reference_metadata.include` is `true`, each reference chunk may include a `document_metadata` object in both streaming and non-streaming responses.\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n```python\nRAGFlow.create_dataset(\n name: str,\n parse_type: int = 1,\n parser_id: str | None = None,\n pipeline_id: str | None = None,\n embedding_model: Optional[str] = None,\n permission: str = \"me\",\n parser_config: Optional[DataSet.ParserConfig] = None,\n language: Optional[str] = None,\n) -> DataSet\n```\n\nCreates a dataset.\n\n`avatar`, `description`, `auto_metadata_config`, and `chunk_method` are not accepted by the RAGFlow 1.0 create endpoint. Set avatar and description through `DataSet.update()`, and metadata configuration through `DataSet.update_auto_metadata()`.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe unique name of the dataset to create. It must adhere to the following requirements:\n\n- Maximum 128 characters.\n- Case-insensitive.\n\n##### parse_type: `int`\n\nSelects the parser mode. Defaults to `1`.\n\n- `1`: BuiltIn. Use the builtin parser selected by `parser_id`.\n- `2`: Pipeline. Use the ingestion pipeline selected by `pipeline_id`.\n\n##### parser_id: `string | None`\n\nThe builtin parser ID. Defaults to `\"general\"` when `parse_type` is `1`.\n\nAvailable values include `\"general\"`, `\"qa\"`, `\"manual\"`, `\"table\"`, `\"paper\"`, `\"book\"`, `\"laws\"`, `\"presentation\"`, `\"picture\"`, `\"one\"`, `\"audio\"`, and `\"email\"`.\n\nMust be omitted when `parse_type` is `2`.\n\n##### pipeline_id: `string | None`\n\nThe ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n\n##### embedding_model: `string | None`\n\nThe embedding model to use for the dataset. Defaults to `None`.\n\n##### permission\n\nSpecifies who can access the dataset to create. Available options:\n\n- `\"me\"`: (Default) Only you can manage the dataset.\n- `\"team\"`: All team members can manage the dataset.\n\n##### language: `string | None`\n\nOptional document/dataset language, for example `\"English\"` or `\"Chinese\"`. Defaults to `None`.\n\n##### parser_config: `DataSet.ParserConfig | None`\n\nThe parser configuration of the dataset. Defaults to `None`.\n\nRAGFlow 1.0 stores parser settings per ingestion-pipeline component. Use component-scoped keys (keys containing `:`) as returned by the API; legacy flat keys are ignored.\n\n#### Returns\n\n- Success: A `dataset` object. Its `parser_id`, `pipeline_id`, and derived `parse_type` reflect the selected parser mode.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"kb_1\")\ndataset = rag_object.create_dataset(name=\"kb_table\", parse_type=1, parser_id=\"table\")\ndataset = rag_object.create_dataset(\n name=\"kb_pipeline\",\n parse_type=2,\n pipeline_id=\"\",\n)\n```\n\n---\n\n### Delete datasets\n\n```python\nRAGFlow.delete_datasets(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes datasets by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the datasets to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no datasets are deleted.\n- If an array of IDs is provided, only the datasets matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all datasets owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object.delete_datasets(ids=[\"d94a8dc02c9711f0930f7fbc369eab6d\",\"e94a8dc02c9711f0930f7fbc369eab6e\"])\nrag_object.delete_datasets(delete_all=True)\n```\n\n---\n\n### List datasets\n\n```python\nRAGFlow.list_datasets(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n) -> list[DataSet]\n```\n\nLists datasets.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the datasets will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of datasets on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which datasets should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved datasets should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the dataset to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the dataset to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the datasets to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n#### Returns\n\n- Success: A list of `DataSet` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n##### List all datasets\n\n```python\nfor dataset in rag_object.list_datasets():\n print(dataset)\n```\n\n##### Retrieve a dataset by ID\n\n```python\ndataset = rag_object.list_datasets(id = \"id_1\")\nprint(dataset[0])\n```\n\n##### Retrieve datasets by IDs\n\n```python\ndatasets = rag_object.list_datasets(ids=[\"id_1\", \"id_2\"])\nfor dataset in datasets:\n print(dataset)\n```\n\n---\n\n### Update dataset\n\n```python\nDataSet.update(update_message: dict)\n```\n\nUpdates configurations for the current dataset.\n\n#### Parameters\n\n##### update_message: `dict[str, str|int]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"name\"`: `string` The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"description\"`: (*Body parameter*), `string`\n The updated dataset description.\n - Maximum 65535 characters\n- `\"language\"`: (*Body parameter*), `string`\n The updated document/dataset language, for example `\"English\"` or `\"Chinese\"`.\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration)\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser mode: `1` for a builtin parser or `2` for an ingestion pipeline. Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`.\n\n`chunk_method` is not supported by the RAGFlow 1.0 SDK; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_name\")\ndataset = dataset[0]\ndataset.update({\"embedding_model\":\"BAAI/bge-zh-v1.5\", \"parse_type\":1, \"parser_id\":\"manual\"})\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n```python\nDataSet.upload_documents(document_list: list[dict])\n```\n\nUploads documents to the current dataset.\n\n#### Parameters\n\n##### document_list: `list[dict]`, *Required*\n\nA list of dictionaries representing the documents to upload, each containing the following keys:\n\n- `\"display_name\"`: `string`, *Required*\n The file name to display in the dataset.\n- `\"blob\"`: `bytes`, *Required*\n The binary content of the file to upload.\n\n#### Returns\n\n- Success: A list of uploaded `Document` objects.\n- Failure: `Exception`\n\n#### Examples\n\n```python\ndataset = rag_object.create_dataset(name=\"kb_name\")\n\nwith open(\"1.txt\", \"rb\") as file:\n documents = dataset.upload_documents([\n {\n \"display_name\": \"1.txt\",\n \"blob\": file.read(),\n }\n ])\n\nprint(documents[0].id)\n```\n\n---\n\n### Update document\n\n```python\nDocument.update(update_message: dict)\n```\n\nUpdates configurations for the current document.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"display_name\"`: `string` The name of the document to update.\n- `\"meta_fields\"`: `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: `string` The parsing method to apply to the document.\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: `dict[str, Any]` The parsing configuration for the document. Its attributes vary based on the selected `\"chunk_method\"`:\n - `\"chunk_method\"`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False},\"parent_child\":{\"use_parent_child\":False,\"children_delimiter\":\"\\n\"}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Returns\n\n- Success: The updated `Document` object.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id='id')\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\ndoc.update({\n \"parser_config\": {\n \"chunk_token_num\": 256,\n },\n \"chunk_method\": \"manual\",\n})\n```\n\n---\n\n### Download document\n\n```python\nDocument.download() -> bytes\n```\n\nDownloads the current document.\n\n#### Returns\n\nThe downloaded document in bytes.\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"https://\",\n)\ndataset = rag_object.list_datasets(id=\"id\")[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")[0]\n\nPath(\"~/ragflow.txt\").expanduser().write_bytes(doc.download())\n```\n\n---\n\n### List documents\n\n```python\nDataSet.list_documents(\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n keywords: str | None = None,\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n create_time_from: int = 0,\n create_time_to: int = 0\n)\n```\n\nLists documents in the current dataset.\n\n#### Parameters\n\n##### id: `string | None`\n\nThe ID of the document to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the documents to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n##### name: `string | None`\n\nThe exact name of the document to retrieve. Defaults to `None`.\n\n##### keywords: `string | None`\n\nThe keywords used to match document titles. Defaults to `None`.\n\n##### page: `int`\n\nSpecifies the page on which the documents will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of documents on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which documents should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved documents should be sorted in descending order. Defaults to `True`.\n\n##### create_time_from: `int`\nUnix timestamp for filtering documents created after this time. 0 means no filter. Defaults to 0.\n\n##### create_time_to: `int`\nUnix timestamp for filtering documents created before this time. 0 means no filter. Defaults to 0.\n\n#### Returns\n\n- Success: A list of `Document` objects.\n- Failure: `Exception`.\n\nA `Document` object contains the following attributes:\n\n- `id`: The document ID. Defaults to `\"\"`.\n- `name`: The document name. Defaults to `\"\"`.\n- `thumbnail`: The thumbnail image of the document. Defaults to `None`.\n- `dataset_id`: The dataset ID associated with the document. Defaults to `None`.\n- `chunk_method` The chunking method name. Defaults to `\"naive\"`.\n- `source_type`: The source type of the document. Defaults to `\"local\"`.\n- `type`: Type or category of the document. Defaults to `\"\"`. Reserved for future use.\n- `created_by`: `string` The creator of the document. Defaults to `\"\"`.\n- `size`: `int` The document size in bytes. Defaults to `0`.\n- `token_count`: `int` The number of tokens in the document. Defaults to `0`.\n- `chunk_count`: `int` The number of chunks in the document. Defaults to `0`.\n- `progress`: `float` The current processing progress as a percentage. Defaults to `0.0`.\n- `progress_msg`: `string` A message indicating the current progress status. Defaults to `\"\"`.\n- `process_begin_at`: `datetime` The start time of document processing. Defaults to `None`.\n- `process_duration`: `float` Duration of the processing in seconds. Defaults to `0.0`.\n- `run`: `string` The document's processing status:\n - `\"UNSTART\"` (default)\n - `\"RUNNING\"`\n - `\"CANCEL\"`\n - `\"DONE\"`\n - `\"FAIL\"`\n- `status`: `string` Reserved for future use.\n- `parser_config`: `ParserConfig` Configuration object for the parser. Its attributes vary based on the selected `chunk_method`:\n - `chunk_method`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"http://:9380\",\n)\ndataset = rag_object.create_dataset(name=\"kb_1\")\n\nfile_path = Path(\"~/ragflow.txt\").expanduser()\ndataset.upload_documents([\n {\n \"display_name\": file_path.name,\n \"blob\": file_path.read_bytes(),\n }\n])\n\nfor doc in dataset.list_documents(\n keywords=\"rag\",\n page=1,\n page_size=12,\n):\n print(doc)\n```\n\n---\n\n### Delete documents\n\n```python\nDataSet.delete_documents(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes documents by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the documents to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no documents are deleted.\n- If an array of IDs is provided, only the documents matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all documents in the current dataset when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_1\")\ndataset = dataset[0]\ndataset.delete_documents(ids=[\"id_1\",\"id_2\"])\ndataset.delete_documents(delete_all=True)\n```\n\n---\n\n### Parse documents\n\n```python\nDataSet.async_parse_documents(document_ids:list[str]) -> None\n```\n\nParses documents in the current dataset.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\n```\n\n---\n\n### Parse documents (with document status)\n\n```python\nDataSet.parse_documents(document_ids: list[str]) -> list[tuple[str, str, int, int]]\n```\n\nStarts parsing documents in the current dataset and synchronously waits for the results.\n\nThis method calls `async_parse_documents()` and blocks while polling until all requested documents reach a terminal state or report complete progress. It then returns the parsing status and statistics for each document. If a keyboard interruption occurs (e.g., `Ctrl+C`), it requests cancellation for the requested documents and continues polling for their final statuses. If a status request fails or a requested document is no longer found, the method raises an exception instead of continuing to poll.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\nA list of tuples with detailed parsing results:\n\n```python\n[\n (document_id: str, status: str, chunk_count: int, token_count: int),\n ...\n]\n```\n- `status`: The final parsing state (e.g., `DONE`, `FAIL`, `CANCEL`). If a document has not reached a terminal state but reports `progress >= 1.0`, its status is returned as `DONE`.\n- `chunk_count`: The number of content chunks created from the document.\n- `token_count`: The total number of tokens processed.\n\n---\n\n#### Example\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = dataset.list_documents(keywords=\"test\")\nids = [doc.id for doc in documents]\n\ntry:\n finished = dataset.parse_documents(ids)\n for doc_id, status, chunk_count, token_count in finished:\n print(f\"Document {doc_id} parsing finished with status: {status}, chunks: {chunk_count}, tokens: {token_count}\")\nexcept Exception as e:\n print(f\"Parsing failed: {e}\")\n```\n\n---\n\n### Stop parsing documents\n\n```python\nDataSet.async_cancel_parse_documents(document_ids:list[str])-> None\n```\n\nStops parsing specified documents.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents for which parsing should be stopped.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\ndataset.async_cancel_parse_documents(ids)\nprint(\"Async bulk parsing cancelled.\")\n```\n\n---\n\n## CHUNK MANAGEMENT WITHIN DATASET\n\n---\n\n### Add chunk\n\n```python\nDocument.add_chunk(content:str, important_keywords:list[str] = [], questions:list[str] = [], image_base64:str = None, *, tag_kwd:list[str] = []) -> Chunk\n```\n\nAdds a chunk to the current document.\n\n#### Parameters\n\n##### content: `string`, *Required*\n\nThe text content of the chunk.\n\n##### important_keywords: `list[str]`\n\nThe key terms or phrases to tag with the chunk.\n\n##### questions: `list[str]`\n\nOptional questions to use when embedding the chunk.\n\n##### image_base64: `string`\n\nA base64-encoded image to associate with the chunk. If the chunk already has an image, the new image will be vertically concatenated below the existing one.\n\n##### tag_kwd: `list[str]`\n\nTag keywords to associate with the chunk.\n\n#### Returns\n\n- Success: A `Chunk` object.\n- Failure: `Exception`.\n\nA `Chunk` object contains the following attributes:\n\n- `id`: `string`: The chunk ID.\n- `content`: `string` The text content of the chunk.\n- `important_keywords`: `list[str]` A list of key terms or phrases tagged with the chunk.\n- `tag_kwd`: `list[str]` A list of tag keywords associated with the chunk.\n- `questions`: `list[str]` A list of questions associated with the chunk.\n- `image_id`: `string` The image ID associated with the chunk (empty string if no image).\n- `create_time`: `string` The time when the chunk was created (added to the document).\n- `create_timestamp`: `float` The timestamp representing the creation time of the chunk, expressed in seconds since January 1, 1970.\n- `dataset_id`: `string` The ID of the associated dataset.\n- `document_name`: `string` The name of the associated document.\n- `document_id`: `string` The ID of the associated document.\n- `available`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(id=\"123\")\ndataset = datasets[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\n```\n\nAdding a chunk with an image:\n\n```python\nimport base64\n\nwith open(\"image.jpg\", \"rb\") as f:\n img_b64 = base64.b64encode(f.read()).decode()\nchunk = doc.add_chunk(content=\"description of image\", image_base64=img_b64)\n```\n\n---\n\n### List chunks\n\n```python\nDocument.list_chunks(keywords: str = None, page: int = 1, page_size: int = 30, id : str = None) -> list[Chunk]\n```\n\nLists chunks in the current document.\n\n#### Parameters\n\n##### keywords: `string`\n\nThe keywords used to match chunk content. Defaults to `None`\n\n##### page: `int`\n\nSpecifies the page on which the chunks will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks on each page. Defaults to `30`.\n\n##### id: `string`\n\nThe ID of the chunk to retrieve. Default: `None`\n\n#### Returns\n\n- Success: A list of `Chunk` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(\"123\")\ndataset = dataset[0]\ndocs = dataset.list_documents(keywords=\"test\", page=1, page_size=12)\nfor chunk in docs[0].list_chunks(keywords=\"rag\", page=0, page_size=12):\n print(chunk)\n```\n\n---\n\n### Delete chunks\n\n```python\nDocument.delete_chunks(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chunks by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chunks to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chunks are deleted.\n- If an array of IDs is provided, only the chunks matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chunks in the current document when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\ndoc.delete_chunks([\"id_1\",\"id_2\"])\ndoc.delete_chunks(delete_all=True)\n```\n\n---\n\n### Update chunk\n\n```python\nChunk.update(update_message: dict)\n```\n\nUpdates content or configurations for the current chunk.\n\n#### Parameters\n\n##### update_message: `dict[str, str|list[str]|bool]` *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"content\"`: `string` The text content of the chunk.\n- `\"important_keywords\"`: `list[str]` A list of key terms or phrases to tag with the chunk.\n- `\"questions\"`: `list[str]` A list of questions associated with the chunk.\n- `\"tag_kwd\"`: `list[str]` A list of tag keywords to associate with the chunk.\n- `\"positions\"`: `list` Updated source positions for the chunk.\n- `\"available\"`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n- `\"image_base64\"`: `string` Base64-encoded image content to associate with the chunk.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\nchunk.update({\"content\":\"sdfx...\"})\n```\n\n---\n\n### Retrieve chunks\n\n```python\nRAGFlow.retrieve(\n dataset_ids,\n document_ids=None,\n question='',\n page=1,\n page_size=30,\n similarity_threshold=0.2,\n vector_similarity_weight=0.3,\n top_k=1024,\n rerank_id: str | None = None,\n keyword: bool = False,\n cross_languages: list[str] | None = None,\n metadata_condition: dict | None = None)\n```\n\nRetrieves chunks from specified datasets.\n\n#### Parameters\n\n##### question: `string`\n\nThe user query or query keywords. Defaults to `\"\"`. When an empty string is provided, the API returns an empty retrieval result.\n\n##### dataset_ids: `list[str]`, *Required*\n\nThe IDs of the datasets to search. At least one dataset ID must be provided.\n\n##### document_ids: `list[str]`\n\nThe IDs of the documents to search. Defaults to `None`. You must ensure all selected documents use the same embedding model. Otherwise, an error will occur.\n\n##### page: `int`\n\nThe page number of the chunk retrieval results. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks returned on each page. Defaults to `30`.\n\n##### similarity_threshold: `float`\n\nThe minimum similarity score. Defaults to `0.2`.\n\n##### vector_similarity_weight: `float`\n\nThe weight of vector cosine similarity. Defaults to `0.3`. If x represents the vector cosine similarity, then (1 - x) is the term similarity weight.\n\n##### top_k: `int`\n\nThe number of chunks engaged in vector cosine computation. Defaults to `1024`.\n\n##### rerank_id: `string`\n\nThe ID of the rerank model. Defaults to `None`.\n\n##### keyword: `bool`\n\nIndicates whether to enable keyword-based matching:\n\n- `True`: Enable keyword-based matching.\n- `False`: Disable keyword-based matching (default).\n\n##### cross_languages: `list[string]`\n\nThe languages that should be translated into, in order to achieve keywords retrievals in different languages.\n\n##### metadata_condition: `dict`\n\nfilter condition for `meta_fields`.\n\n#### Returns\n\n- Success: A list of `Chunk` objects representing the document chunks.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"ragflow\")\ndataset = dataset[0]\nname = 'ragflow_test.txt'\npath = './test_data/ragflow_test.txt'\ndocuments =[{\"display_name\":\"test_retrieve_chunks.txt\",\"blob\":open(path, \"rb\").read()}]\ndocs = dataset.upload_documents(documents)\ndoc = docs[0]\ndoc.add_chunk(content=\"This is a chunk addition test\")\nfor c in rag_object.retrieve(dataset_ids=[dataset.id],document_ids=[doc.id]):\n print(c)\n```\n\n---\n\n## CHAT ASSISTANT MANAGEMENT\n\n---\n\n### Create chat assistant\n\n```python\nRAGFlow.create_chat(\n name: str,\n icon: str = \"\",\n dataset_ids: list[str] | None = None,\n llm_id: str | None = None,\n llm_setting: dict | None = None,\n prompt_config: dict | None = None,\n **kwargs\n) -> Chat\n```\n\nCreates a chat assistant.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe name of the chat assistant.\n\n##### icon: `string`\n\nBase64 encoding of the avatar. Defaults to `\"\"`.\n\n##### dataset_ids: `list[str]`\n\nThe IDs of the associated datasets. Defaults to `[]`. When omitted or empty, the SDK creates an empty chat assistant, and you can attach datasets later.\n\n##### llm_id: `str | None`\n\nThe LLM model name/ID to use. If `None`, the user’s default chat model is used. Defaults to `None`.\n\n##### llm_setting: `dict | None`\n\nConfiguration for LLM generation parameters. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"temperature\"`: `float` Controls the randomness of the model's output. Higher values increase creativity, while lower values make responses more deterministic. Defaults to `0.1`.\n- `\"top_p\"`: `float` Sets the nucleus sampling threshold. The model considers only the results of the tokens with `top_p` probability mass. Defaults to `0.3`.\n- `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have appeared in the text so far, increasing the likelihood of the model talking about new topics. Defaults to `0.4`.\n- `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text, decreasing the likelihood of repeating the same lines. Defaults to `0.7`.\n- `\"max_token\"`: `int` The maximum number of tokens to generate in the response. Defaults to `512`.\n\n##### prompt_config: `dict | None`\n\nInstructions and behavioral settings for the LLM. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n- `\"empty_response\"`: `string` The specific message returned when no relevant information is retrieved. If left blank, the LLM will generate its own response. Defaults to `None`.\n- `\"prologue\"`: `string` The initial greeting displayed to the user. Defaults to `\"Hi! I’m your assistant. What can I do for you?\"`.\n- `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references in its responses. Defaults to `True`.\n- `\"parameters\"`: `list[dict]` A list of variables utilized within the system prompt. Each entry must include a `\"key\"` (`string`) and an `\"optional\"` (`boolean`) status. The `knowledge` key is reserved for retrieved context chunks. Default: `[{\"key\": \"knowledge\", \"optional\": true}]`.\n\n#### Returns\n\n- Success: A `Chat` object representing the chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_ids = []\nfor dataset in datasets:\n dataset_ids.append(dataset.id)\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=dataset_ids)\n```\n\n---\n\n### Update chat assistant\n\n```python\nChat.update(update_message: dict)\n```\n\nPerforms a partial update to the configuration settings for the current chat assistant.\n\n`Chat.update()` utilizes the `PATCH /api/v1/chats/{chat_id}` endpoint. Only the specified keys are modified, while all other existing fields are preserved.\n\n#### Parameters\n\n##### update_message: `dict`, *Required*\n\nA dictionary containing the attributes to be updated. Supported keys include:\n\n- `\"name\"`: `string` The updated name of the chat assistant.\n- `\"icon\"`: `string` A Base64-encoded string representing the assistant's avatar.\n- `\"dataset_ids\"`: `list[string]` A list of unique identifiers for the datasets associated with the assistant.\n- `\"llm_id\"`: `string` The unique identifier or name of the LLM to be used.\n- `\"llm_setting\"`: `dict` Configuration for LLM generation parameters:\n - `\"temperature\"`: `float` Controls the randomness of the model's output.\n - `\"top_p\"`: `float` Sets the nucleus sampling threshold.\n - `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have already appeared in the text.\n - `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text.\n - `\"max_token\"`: `int` The maximum number of tokens to generate in the response.\n- `\"prompt_config\"`: `dict` Instructions and behavioral settings for the LLM:\n - `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n - `\"empty_response\"`: `string` The message returned when no relevant information is retrieved. Leave blank to allow the LLM to improvise.\n - `\"prologue\"`: `string` The initial greeting displayed to the user.\n - `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references.\n - `\"parameters\"`: `list[dict]` Variables used within the system prompt (e.g., the reserved `knowledge` key).\n- `\"similarity_threshold\"`: `float` The minimum similarity score required for retrieved context chunks. Defaults to `0.2`.\n- `\"vector_similarity_weight\"`: `float` The weight assigned to vector cosine similarity within the hybrid search score. Defaults to `0.3`.\n- `\"top_n\"`: `int` The number of top-ranked chunks provided to the LLM as context. Defaults to `6`.\n- `\"top_k\"`: `int` The size of the initial candidate pool retrieved for reranking. Defaults to `1024`.\n- `\"rerank_id\"`: `string` The unique identifier for the reranking model. If left empty, standard vector cosine similarity is used for ranking.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_id = datasets[0].id\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=[dataset_id])\nassistant.update({\"name\": \"Stefan\", \"llm_setting\": {\"temperature\": 0.8}, \"top_n\": 8})\n```\n\n---\n\n### Delete chat assistants\n\n```python\nRAGFlow.delete_chats(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chat assistants by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chat assistants to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chat assistants are deleted.\n- If an array of IDs is provided, only the chat assistants matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chat assistants owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nrag_object.delete_chats(ids=[\"id_1\",\"id_2\"])\nrag_object.delete_chats(delete_all=True)\n```\n\n---\n\n### List chat assistants\n\n```python\nRAGFlow.list_chats(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n name: str | None = None,\n keywords: str | None = None,\n owner_ids: str | list[str] | None = None,\n) -> list[Chat]\n```\n\nLists chat assistants.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the chat assistants will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of chat assistants on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe attribute by which the results are sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved chat assistants should be sorted in descending order. Defaults to `True`.\n\n##### id: `string | None`\n\nExact match on chat assistant ID. Defaults to `None`.\n\n##### name: `string | None`\n\nFilters results by the exact name of the chat assistant. Defaults to `None`.\n\n##### keywords: `string | None`\n\nPerforms a case-insensitive fuzzy search against chat assistant names. Defaults to `None`.\n\n##### owner_ids: `string | list[string] | None`\n\nFilters results by one or more owner tenant IDs. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Chat` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nfor assistant in rag_object.list_chats():\n print(assistant)\n```\n\n---\n\n## SESSION MANAGEMENT\n\n---\n\n### Create session with chat assistant\n\n```python\nChat.create_session(name: str = \"New session\") -> Session\n```\n\nCreates a session with the current chat assistant.\n\n#### Parameters\n\n##### name: `string`\n\nThe name of the chat session to create.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `name`: `string` The name of the created session.\n - `message`: `list[Message]` The opening message of the created session. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `chat_id`: `string` The ID of the associated chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session()\n```\n\n---\n\n### Update chat assistant's session\n\n```python\nSession.update(update_message: dict)\n```\n\nUpdates the current session of the current chat assistant.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with only one key:\n\n- `\"name\"`: `string` The revised name of the session.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session(\"session_name\")\nsession.update({\"name\": \"updated_name\"})\n```\n\n---\n\n### List chat assistant's sessions\n\n```python\nChat.list_sessions(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str = None,\n name: str = None,\n user_id: str = None\n) -> list[Session]\n```\n\nLists sessions associated with the current chat assistant.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the sessions will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of sessions on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which sessions should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved sessions should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the chat session to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the chat session to retrieve. Defaults to `None`.\n\n##### user_id: `str`\n\nThe optional user-defined ID to filter sessions by. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Session` objects associated with the current chat assistant.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nfor session in assistant.list_sessions():\n print(session)\n```\n\n---\n\n### Delete chat assistant's sessions\n\n```python\nChat.delete_sessions(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes sessions of the current chat assistant by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the sessions to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no sessions are deleted.\n- If an array of IDs is provided, only the sessions matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all sessions of the current chat assistant when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nassistant.delete_sessions(ids=[\"id_1\",\"id_2\"])\nassistant.delete_sessions(delete_all=True)\n```\n\n---\n\n### Create session with agent\n\n```python\nAgent.create_session(**kwargs) -> Session\n```\n\nCreates a session with the current agent.\n\n#### Parameters\n\n##### **kwargs\n\nThe parameters in `begin` component.\n\nAlso supports:\n\n- `release` (`bool | str`, optional): When set to `True` (or `\"true\"`), creates a session with the published agent app only.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `message`: `list[Message]` The messages of the created session assistant. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `agent_id`: `string` The ID of the associated agent.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nagent_id = \"AGENT_ID\"\nagent = rag_object.get_agent(agent_id)\nsession = agent.create_session()\n# Or create in release mode:\n# session = agent.create_session(release=True)\n```\n\n---\n\n### Converse with agent\n\n```python\nSession.ask(question: str = \"\", stream: bool = False, **kwargs) -> Optional[Message | iter[Message]]\n```\n\nAsks a specified agent through the unified completion endpoint.\n\n:::tip NOTE\nIn streaming mode, not all responses include a reference, as this depends on the system's judgment.\n:::\n\n#### Parameters\n\n##### question: `string`\n\nThe user message sent to the agent. If the **Begin** component takes parameters, `question` can be an empty string.\n\n##### stream: `bool`\n\nIndicates whether to output responses in a streaming way:\n\n- `True`: Enable streaming.\n- `False`: Disable streaming.\n\n##### kwargs: `dict`\n\nAdditional request parameters forwarded to the completion API. Common options:\n\n- `inputs`: Variables defined in the **Begin** component.\n- `session_id`: Continue an existing session instead of creating a new one.\n- `release`: Use the latest published version of the agent.\n- `return_trace`: Include execution trace information in the response.\n- Other custom Begin component parameters supported by the current workflow.\n\n#### Returns\n\n- A `Message` object containing the response to the question if `stream` is set to `False`\n- An iterator containing multiple `message` objects (`iter[Message]`) if `stream` is set to `True`\n\nThe following shows the attributes of a `Message` object:\n\n##### id: `string`\n\nThe auto-generated message ID.\n\n##### content: `string`\n\nThe content of the message. Defaults to `\"Hi! I am your assistant, can I help you?\"`.\n\n##### reference: `list[Chunk]`\n\nA list of `Chunk` objects representing references to the message, each containing the following attributes:\n\n- `id` `string`\n The chunk ID.\n- `content` `string`\n The content of the chunk.\n- `image_id` `string`\n The ID of the snapshot of the chunk. Applicable only when the source of the chunk is an image, PPT, PPTX, or PDF file.\n- `document_id` `string`\n The ID of the referenced document.\n- `document_name` `string`\n The name of the referenced document.\n- `document_metadata` `dict`\n Optional document metadata, returned only when `extra_body.reference_metadata.include` is `true`.\n- `position` `list[str]`\n The location information of the chunk within the referenced document.\n- `dataset_id` `string`\n The ID of the dataset to which the referenced document belongs.\n- `similarity` `float`\n A composite similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity. It is the weighted sum of `vector_similarity` and `term_similarity`.\n- `vector_similarity` `float`\n A vector similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between vector embeddings.\n- `term_similarity` `float`\n A keyword similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between keywords.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nAGENT_id = \"AGENT_ID\"\nagent = rag_object.get_agent(AGENT_id)\nsession = agent.create_session()\n\nprint(\"\\n===== Miss R ====\\n\")\nprint(\"Hello. What can I do for you?\")\n\nwhile True:\n question = input(\"\\n===== User ====\\n> \")\n print(\"\\n==== Miss R ====\\n\")\n\n cont = \"\"\n for ans in session.ask(question, stream=True):\n print(ans.content[len(cont):], end='', flush=True)\n cont = ans.content\n```\n\nUse Begin inputs and request trace output:\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://=3.8.0\n# pydantic>=2.0.0\n# requests>=2.28.0\n```\n\n## Uninstallation\n\nTo remove the plugin:\n\n1. **Remove plugin directory**:\n ```bash\n rm -rf /opt/ragflow/plugin/firecrawl/\n ```\n\n2. **Restart RAGFlow**:\n ```bash\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n3. **Remove dependencies** (optional):\n ```bash\n pip uninstall ragflow-firecrawl-integration\n ```\n\n## Support\n\nIf you encounter issues:\n\n1. Check the [troubleshooting section](#troubleshooting)\n2. Review RAGFlow logs for error messages\n3. Verify your Firecrawl API key and configuration\n4. Check the [Firecrawl documentation](https://docs.firecrawl.dev)\n5. Open an issue in the [Firecrawl repository](https://github.com/firecrawl/firecrawl/issues)\n\n## Next Steps\n\nAfter successful installation:\n\n1. Read the [README.md](README.md) for usage examples\n2. Try scraping a simple URL to test the integration\n3. Explore the different scraping options (single URL, crawl, batch)\n4. Configure your RAGFlow workflows to use the scraped content\n","isInternal":false,"tokens":1401,"sizeBytes":5573},{"name":"INSTALL_SCRIPTS_README.md","path":"tools/scripts/INSTALL_SCRIPTS_README.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/scripts/INSTALL_SCRIPTS_README.md","title":"Scripts Skill","category":"anthropic-skill","format":"markdown","content":"# RAGFlow CLI Installation Scripts\n\nRAGFlow publishes static Go CLI binaries as GitHub Release assets for:\n\n- Linux: `amd64`, `arm64`\n- macOS: `amd64`, `arm64`\n- Windows: `amd64`, `arm64`\n\nThe release workflow builds the binaries with `CGO_ENABLED=0`, uploads `SHA256SUMS`, and uploads `install.sh` and `install.ps1`.\n\n## Linux And macOS\n\nUse the script maintained in the RAGFlow repository:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | sh\n```\n\nInstall a specific version:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | VERSION=v1.0.0 sh\n```\n\nInstall to a user-writable directory:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | INSTALL_DIR=\"$HOME/.local/bin\" sh\n```\n\nThe Unix installer:\n\n- detects `linux` or `darwin`\n- detects `amd64` or `arm64`\n- downloads `ragflow-cli-{VERSION}-{OS}-{ARCH}`\n- verifies the file with the release `SHA256SUMS`\n- installs to `/usr/local/bin` by default\n\n## Windows PowerShell\n\nRun the repository script in PowerShell:\n\n```powershell\nirm https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.ps1 | iex\n```\nInstall a specific version:\n\n```powershell\n.\\install.ps1 -Version \"v1.0.0\"\n```\n\nInstall to a custom directory:\n\n```powershell\n.\\install.ps1 -InstallDir \"$env:USERPROFILE\\bin\"\n```\n\nThe Windows installer:\n\n- detects `windows/amd64` or `windows/arm64`\n- downloads `ragflow-cli-{VERSION}-windows-{ARCH}.exe`\n- verifies the file with the release `SHA256SUMS`\n- installs to `$env:LOCALAPPDATA\\Programs\\RAGFlow` by default\n- adds the install directory to the user `PATH`\n\n## Release Asset Names\n\nThe install scripts expect these names:\n\n```text\nragflow-cli-v1.0.0-linux-amd64\nragflow-cli-v1.0.0-linux-arm64\nragflow-cli-v1.0.0-darwin-amd64\nragflow-cli-v1.0.0-darwin-arm64\nragflow-cli-v1.0.0-windows-amd64.exe\nragflow-cli-v1.0.0-windows-arm64.exe\nSHA256SUMS\ninstall.sh\ninstall.ps1\n```\n\nBy default, both scripts install the latest stable GitHub Release. Set\n`VERSION` on Unix or `-Version` on Windows to pin a specific release.\n","isInternal":false,"tokens":595,"sizeBytes":2153},{"name":"copilot-instructions.md","path":".github/copilot-instructions.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/.github/copilot-instructions.md","title":"GitHub Copilot Instructions","category":"copilot-instructions","format":"markdown","content":"# Project instructions for Copilot\n\n## How to run (minimum)\n- Install:\n - python -m venv .venv && source .venv/bin/activate\n - pip install -r requirements.txt\n- Run:\n - (fill) e.g. uvicorn app.main:app --reload\n- Verify:\n - (fill) curl http://127.0.0.1:8000/health\n\n## Project layout (what matters)\n- app/: API entrypoints + routers\n- services/: business logic\n- configs/: config loading (.env)\n- docs/: documents\n- tests/: pytest\n\n## Conventions\n- Prefer small, incremental changes.\n- Add logging for new flows.\n- Add/adjust tests for behavior changes.\n","isInternal":false,"tokens":156,"sizeBytes":558},{"name":"README.md","path":"tools/chatgpt-on-wechat/plugins/README.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/README.md","title":"plugins Documentation","category":"plugin-manifest","format":"markdown","content":"# plugins Documentation\n\nPath: `tools/chatgpt-on-wechat/plugins/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/README.md)","tokens":300,"sizeBytes":205},{"name":"README.md","path":"tools/es-to-oceanbase-migration/README.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/es-to-oceanbase-migration/README.md","title":"es-to-oceanbase-migration Documentation","category":"plugin-manifest","format":"markdown","content":"# RAGFlow ES to OceanBase Migration Tool\n\nA CLI tool for migrating RAGFlow data from Elasticsearch to OceanBase. This tool is specifically designed for RAGFlow's data structure and handles schema conversion, vector data mapping, batch import, and resume capability.\n\n## Features\n\n- **RAGFlow-Specific**: Designed for RAGFlow's fixed data schema\n- **ES 8+ Support**: Uses `search_after` API for efficient data scrolling\n- **Vector Support**: Auto-detects vector field dimensions from ES mapping\n- **Batch Processing**: Configurable batch size for optimal performance\n- **Resume Capability**: Save and resume migration progress\n- **Data Consistency Validation**: Compare document counts and sample data\n- **Migration Report Generation**: Generate detailed migration reports\n\n## Quick Start\n\nThis section provides a complete guide to verify the migration works correctly with a real RAGFlow deployment.\n\n### Prerequisites\n\n- RAGFlow source code cloned\n- Docker and Docker Compose installed\n- This migration tool installed (`uv pip install -e .`)\n\n### Step 1: Start RAGFlow with Elasticsearch Backend\n\nFirst, start RAGFlow using Elasticsearch as the document storage backend (default configuration).\n\n```bash\n# Navigate to RAGFlow docker directory\ncd /path/to/ragflow/docker\n\n# Ensure DOC_ENGINE=elasticsearch in .env (this is the default)\n# DOC_ENGINE=elasticsearch\n\n# Start RAGFlow with Elasticsearch (--profile cpu for CPU, --profile gpu for GPU)\ndocker compose --profile elasticsearch --profile cpu up -d\n\n# Wait for services to be ready (this may take a few minutes)\ndocker compose ps\n\n# Check ES is running\ncurl -X GET \"http://localhost:9200/_cluster/health?pretty\"\n```\n\n### Step 2: Create Test Data in RAGFlow\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Create a new Knowledge Base\n3. Upload some test documents (PDF, TXT, DOCX, etc.)\n4. Wait for the documents to be parsed and indexed\n5. Test the knowledge base with some queries to ensure it works\n\n### Step 3: Verify ES Data (Optional)\n\nBefore migration, verify the data exists in Elasticsearch. This step is important to ensure you have a baseline for comparison after migration.\n\n```bash\n# Navigate to migration tool directory (from ragflow root)\ncd tools/es-to-oceanbase-migration\n\n# Activate the virtual environment if not already done\nsource .venv/bin/activate\n\n# Check connection and list indices\nes-ob-migrate status --es-host localhost --es-port 9200\n\n# First, find your actual index name (pattern: ragflow_{tenant_id})\ncurl -X GET \"http://localhost:9200/_cat/indices/ragflow_*?v\"\n\n# List all knowledge bases in the index\n# Replace ragflow_{tenant_id} with your actual index from the curl output above\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n\n# View sample documents\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_{tenant_id} --size 5\n\n# Check schema\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n```\n\n### Step 4: Start OceanBase for Migration\n\nStart RAGFlow's OceanBase service as the migration target:\n\n```bash\n# Navigate to ragflow docker directory (from ragflow root)\ncd ../docker\n\n# Start only OceanBase service from RAGFlow docker compose\ndocker compose --profile oceanbase up -d\n\n# Wait for OceanBase to be ready\ndocker compose logs -f oceanbase\n```\n\n### Step 5: Run Migration\n\nExecute the migration from Elasticsearch to OceanBase:\n\n```bash\ncd ../tools/es-to-oceanbase-migration\n\n# Option A: Migrate ALL ragflow_* indices (Recommended)\n# If --index and --table are omitted, the tool auto-discovers all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --batch-size 1000 \\\n --verify\n\n# Option B: Migrate a specific index\n# Use the SAME name for both --index and --table\n# The index name pattern is: ragflow_{tenant_id}\n# Find your tenant_id from Step 3's curl output\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --batch-size 1000 \\\n --verify\n```\n\nExpected output:\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_{tenant_id}\nTarget: localhost:2881/ragflow_doc.ragflow_{tenant_id}\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK (version: 4.3.5.1)\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Known RAGFlow fields: 25\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_{tenant_id}' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nStep 5: Verifying migration...\n✓ Document counts match: 1,234\n✓ Sample verification: 100/100 matched\n\nMigration completed successfully!\n Total: 1,234 documents\n Migrated: 1,234 documents\n Failed: 0 documents\n Duration: 45.2 seconds\n```\n\n### Step 6: Stop RAGFlow and Switch to OceanBase Backend\n\n```bash\n# Navigate to ragflow docker directory\ncd ../../docker\n\n# Stop only Elasticsearch and RAGFlow (but keep OceanBase running)\ndocker compose --profile elasticsearch --profile cpu down\n\n# Edit .env file, change:\n# DOC_ENGINE=elasticsearch -> DOC_ENGINE=oceanbase\n#\n# The OceanBase connection settings are already configured by default in .env\n```\n\n### Step 7: Start RAGFlow with OceanBase Backend\n\n```bash\n# OceanBase should still be running from Step 4\n# Start RAGFlow with OceanBase profile (OceanBase is already running)\ndocker compose --profile oceanbase --profile cpu up -d\n\n# Wait for services to start\ndocker compose ps\n\n# Check logs for any errors\ndocker compose logs -f ragflow-cpu\n```\n\n### Step 8: Data Integrity Verification (Optional)\n\nRun the verification command to compare ES and OceanBase data:\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --sample-size 100\n```\n\nExpected output:\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_{tenant_id} │\n│ OB Table: ragflow_{tenant_id} │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n### Step 9: Verify RAGFlow Works with OceanBase\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Navigate to your Knowledge Base\n3. Try the same queries you tested before migration\n\n## CLI Reference\n\n### `es-ob-migrate migrate`\n\nRun data migration from Elasticsearch to OceanBase.\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--es-host` | localhost | Elasticsearch host |\n| `--es-port` | 9200 | Elasticsearch port |\n| `--es-user` | None | ES username (if auth required) |\n| `--es-password` | None | ES password |\n| `--ob-host` | localhost | OceanBase host |\n| `--ob-port` | 2881 | OceanBase port |\n| `--ob-user` | root@test | OceanBase user (format: user@tenant) |\n| `--ob-password` | \"\" | OceanBase password |\n| `--ob-database` | test | OceanBase database name |\n| `-i, --index` | None | Source ES index (omit to migrate all ragflow_* indices) |\n| `-t, --table` | None | Target OB table (omit to use same name as index) |\n| `--batch-size` | 1000 | Documents per batch |\n| `--resume` | False | Resume from previous progress |\n| `--verify/--no-verify` | True | Verify after migration |\n\n**Example:**\n\n```bash\n# Migrate all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc\n\n# Migrate a specific index\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123\n\n# Resume interrupted migration\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --resume\n```\n\n**Resume Feature:**\n\nMigration progress is automatically saved to `.migration_progress/` directory. If migration is interrupted (network error, timeout, etc.), use `--resume` to continue from where it stopped:\n\n- Progress file: `.migration_progress/{index_name}_progress.json`\n- Contains: total count, migrated count, last document ID, timestamp\n- On resume: skips already migrated documents, continues from last position\n\n**Output:**\n\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_abc123\nTarget: localhost:2881/ragflow_doc.ragflow_abc123\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_abc123' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nMigration completed successfully!\n Total: 1,234 documents\n Duration: 45.2 seconds\n```\n\n---\n\n### `es-ob-migrate list-indices`\n\nList all RAGFlow indices (`ragflow_*`) in Elasticsearch.\n\n**Example:**\n\n```bash\nes-ob-migrate list-indices --es-host localhost --es-port 9200\n```\n\n**Output:**\n\n```\nRAGFlow Indices in Elasticsearch:\n\n Index Name Documents Type\n ragflow_abc123def456789 1234 Document Chunks\n ragflow_doc_meta_abc123def456789 56 Document Metadata\n\nTotal: 2 ragflow_* indices found\n```\n\n---\n\n### `es-ob-migrate schema`\n\nPreview schema analysis from ES mapping.\n\n**Example:**\n\n```bash\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nRAGFlow Schema Analysis for index: ragflow_abc123\n\nVector Fields:\n q_1024_vec: dense_vector (dim=1024)\n\nKnown RAGFlow Fields (25):\n id, kb_id, doc_id, docnm_kwd, content_with_weight, content_ltks,\n available_int, important_kwd, question_kwd, tag_kwd, page_num_int...\n\nUnknown Fields (stored in 'extra' column):\n custom_field_1, custom_field_2\n```\n\n---\n\n### `es-ob-migrate verify`\n\nVerify migration data consistency between ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --sample-size 100\n```\n\n**Output:**\n\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_abc123 │\n│ OB Table: ragflow_abc123 │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n---\n\n### `es-ob-migrate list-kb`\n\nList all knowledge bases in an ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nKnowledge Bases in index 'ragflow_abc123':\n\n KB ID Documents\n kb_001_finance_docs 456\n kb_002_technical_manual 321\n kb_003_product_faq 457\n\nTotal: 3 knowledge bases, 1234 documents\n```\n\n---\n\n### `es-ob-migrate sample`\n\nShow sample documents from ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_abc123 --size 2\n```\n\n**Output:**\n\n```\nSample Documents from 'ragflow_abc123':\n\nDocument 1:\n id: chunk_001_abc123\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: The company reported Q3 revenue of $1.2B...\n available_int: 1\n\nDocument 2:\n id: chunk_002_def456\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: Operating expenses decreased by 5%...\n available_int: 1\n```\n\n---\n\n### `es-ob-migrate status`\n\nCheck connection status to ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate status \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\"\n```\n\n**Output:**\n\n```\nConnection Status:\n\nElasticsearch:\n Host: localhost:9200\n Status: ✓ Connected\n Cluster: ragflow-cluster\n Version: 8.11.0\n Indices: 5\n\nOceanBase:\n Host: localhost:2881\n Status: ✓ Connected\n Version: 4.3.5.1\n```\n","isInternal":false,"tokens":3748,"sizeBytes":16183},{"name":"README.md","path":"tools/firecrawl/README.md","rawUrl":"https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/firecrawl/README.md","title":"firecrawl Documentation","category":"plugin-manifest","format":"markdown","content":"# Firecrawl Integration for RAGFlow\n\nThis integration adds [Firecrawl](https://firecrawl.dev)'s powerful web scraping capabilities to [RAGFlow](https://github.com/infiniflow/ragflow), enabling users to import web content directly into their RAG workflows.\n\n## 🎯 **Integration Overview**\n\nThis integration implements the requirements from [Firecrawl Issue #2167](https://github.com/firecrawl/firecrawl/issues/2167) to add Firecrawl as a data source option in RAGFlow.\n\n### ✅ **Acceptance Criteria Met**\n\n- ✅ **Integration appears as selectable data source** in RAGFlow's UI\n- ✅ **Users can input Firecrawl API keys** through RAGFlow's configuration interface\n- ✅ **Successfully scrapes content** and imports into RAGFlow's document processing pipeline\n- ✅ **Handles edge cases** (rate limits, failed requests, malformed content)\n- ✅ **Includes documentation** and README updates\n- ✅ **Follows RAGFlow patterns** and coding standards\n- ✅ **Ready for engineering review**\n\n## 🚀 **Features**\n\n### Core Functionality\n- **Single URL Scraping** - Scrape individual web pages\n- **Website Crawling** - Crawl entire websites with job management\n- **Batch Processing** - Process multiple URLs simultaneously\n- **Multiple Output Formats** - Support for markdown, HTML, links, and screenshots\n\n### Integration Features\n- **RAGFlow Data Source** - Appears as selectable data source in RAGFlow UI\n- **API Configuration** - Secure API key management with validation\n- **Content Processing** - Converts Firecrawl output to RAGFlow document format\n- **Error Handling** - Comprehensive error handling and retry logic\n- **Rate Limiting** - Built-in rate limiting and request throttling\n\n### Quality Assurance\n- **Content Cleaning** - Intelligent content cleaning and normalization\n- **Metadata Extraction** - Rich metadata extraction and enrichment\n- **Document Chunking** - Automatic document chunking for RAG processing\n- **Language Detection** - Automatic language detection\n- **Validation** - Input validation and error checking\n\n## 📁 **File Structure**\n\n```\ntools/firecrawl/\n├── __init__.py # Package initialization\n├── firecrawl_connector.py # API communication with Firecrawl\n├── firecrawl_config.py # Configuration management\n├── firecrawl_processor.py # Content processing for RAGFlow\n├── firecrawl_ui.py # UI components for RAGFlow\n├── ragflow_integration.py # Main integration class\n├── example_usage.py # Usage examples\n├── requirements.txt # Python dependencies\n├── README.md # This file\n└── INSTALLATION.md # Installation guide\n```\n\n## 🔧 **Installation**\n\n### Prerequisites\n- RAGFlow instance running\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n### Setup\n1. **Get Firecrawl API Key**:\n - Visit [firecrawl.dev](https://firecrawl.dev)\n - Sign up for a free account\n - Copy your API key (starts with `fc-`)\n\n2. **Configure in RAGFlow**:\n - Go to RAGFlow UI → Data Sources → Add New Source\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify setup\n - You should see a success message\n\n## 🎮 **Usage**\n\n### Single URL Scraping\n1. Select \"Single URL\" as scrape type\n2. Enter the URL to scrape\n3. Choose output formats (markdown recommended for RAG)\n4. Start scraping\n\n### Website Crawling\n1. Select \"Crawl Website\" as scrape type\n2. Enter the starting URL\n3. Set crawl limit (maximum number of pages)\n4. Configure extraction options\n5. Start crawling\n\n### Batch Processing\n1. Select \"Batch URLs\" as scrape type\n2. Enter multiple URLs (one per line)\n3. Choose output formats\n4. Start batch processing\n\n## 🔧 **Configuration Options**\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## 📊 **API Reference**\n\n### RAGFlowFirecrawlIntegration\n\nMain integration class for Firecrawl with RAGFlow.\n\n#### Methods\n- `scrape_and_import(urls, formats, extract_options)` - Scrape URLs and convert to RAGFlow documents\n- `crawl_and_import(start_url, limit, scrape_options)` - Crawl website and convert to RAGFlow documents\n- `test_connection()` - Test connection to Firecrawl API\n- `validate_config(config_dict)` - Validate configuration settings\n\n### FirecrawlConnector\n\nHandles communication with the Firecrawl API.\n\n#### Methods\n- `scrape_url(url, formats, extract_options)` - Scrape single URL\n- `start_crawl(url, limit, scrape_options)` - Start crawl job\n- `get_crawl_status(job_id)` - Get crawl job status\n- `batch_scrape(urls, formats)` - Scrape multiple URLs concurrently\n\n### FirecrawlProcessor\n\nProcesses Firecrawl output for RAGFlow integration.\n\n#### Methods\n- `process_content(content)` - Process scraped content into RAGFlow document format\n- `process_batch(contents)` - Process multiple scraped contents\n- `chunk_content(document, chunk_size, chunk_overlap)` - Chunk document content for RAG processing\n\n## 🧪 **Testing**\n\nThe integration includes comprehensive testing:\n\n```bash\n# Run the test suite\ncd tools/firecrawl\npython3 -c \"\nimport sys\nsys.path.append('.')\nfrom ragflow_integration import create_firecrawl_integration\n\n# Test configuration\nconfig = {\n 'api_key': 'fc-test-key-123',\n 'api_url': 'https://api.firecrawl.dev'\n}\n\nintegration = create_firecrawl_integration(config)\nprint('✅ Integration working!')\n\"\n```\n\n## 🐛 **Error Handling**\n\nThe integration includes robust error handling for:\n\n- **Rate Limiting** - Automatic retry with exponential backoff\n- **Network Issues** - Retry logic with configurable timeouts\n- **Malformed Content** - Content validation and cleaning\n- **API Errors** - Detailed error messages and logging\n\n## 🔒 **Security**\n\n- API key validation and secure storage\n- Input sanitization and validation\n- Rate limiting to prevent abuse\n- Error handling without exposing sensitive information\n\n## 📈 **Performance**\n\n- Concurrent request processing\n- Configurable timeouts and retries\n- Efficient content processing\n- Memory-conscious document handling\n\n## 🤝 **Contributing**\n\nThis integration was created as part of the [Firecrawl bounty program](https://github.com/firecrawl/firecrawl/issues/2167).\n\n### Development\n1. Fork the RAGFlow repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## 📄 **License**\n\nThis integration is licensed under the same license as RAGFlow (Apache 2.0).\n\n## 🆘 **Support**\n\n- **Firecrawl Documentation**: [docs.firecrawl.dev](https://docs.firecrawl.dev)\n- **RAGFlow Documentation**: [RAGFlow GitHub](https://github.com/infiniflow/ragflow)\n- **Issues**: Report issues in the RAGFlow repository\n\n## 🎉 **Acknowledgments**\n\nThis integration was developed as part of the Firecrawl bounty program to bridge the gap between web content and RAG applications, making it easier for developers to build AI applications that can leverage real-time web data.\n\n---\n\n**Ready for RAGFlow Integration!** 🚀\n\nThis integration enables RAGFlow users to easily import web content into their knowledge retrieval systems, expanding the ecosystem for both Firecrawl and RAGFlow.\n","isInternal":false,"tokens":1766,"sizeBytes":7894}],"systemPromptSnippet":"\n\n# RAGFlow Instructions\n\nUse this file as the local operating guide for the current codebase. Prefer the code and the current AGENTS.md over any older convention or remembered project shape.\n\n## Core Stance\n- Treat legacy code as liability, not as a compatibility target.\n- Prefer deletion over shims, deprecated branches, wrapper APIs, and dual-track migration notes.\n- If old and new implementations coexist, converge to one path unless an external contract forces compatibility.\n- Remove dead tests, commented-out code, stale docs, and \"move later\" notes instead of preserving them.\n- Reduce public surface area when a helper can be made private or internal.\n- Keep refactors centered on the owning abstraction, not on adjacent compatibility layers.\n\n## Current stack\n- Backend: Go implements the API, admin, ingestor, and syncer modes in `cmd/ragflow_server.go`; the CLI runs from `cmd/ragflow-cli.go`. The Docker entrypoint starts the Go services according to its command-line flags, and the frontend development server uses fixed Go backend proxy routes. The Python API, admin, and workers are still present pending removal.\n- Frontend: React + TypeScript + Vite in `web/`. When working under `web/`, read and follow `web/AGENTS.md` for frontend conventions.\n- The Go module contains the server, ingestion, parsing, agent runtime, CLI, and supporting services.\n- Runtime services commonly include MySQL/PostgreSQL, Kvrocks, NATS JetStream, MinIO, ClickHouse, and Elasticsearch/Infinity/OpenSearch depending on configuration.\n## Go-only target\n- Remove all Python code and Python-specific dependencies, scripts, containers, CI jobs, documentation, SDKs, and tests from the repository. Do not add new Python code while this removal is in progress. Replace Python-based preparation of Go native libraries, DeepDoc models, and the BPE table before removing the existing download scripts.\n- Move any still-required behavior and test coverage into the Go backend or the frontend before deleting the old implementation. Do not preserve a Python server, compatibility layer, fallback, or dual-backend path.\n- Before deleting `rag/`, move the Go DeepDoc `.ort` models and `ocr.res` out of `internal/rag/res/deepdoc/`, then update model discovery, dependency preparation, Docker packaging, and tests together.\n- Remove the frontend's Go/Python backend variants and update `web/CLAUDE.md` when the frontend no longer needs them. Until then, treat its dual-backend instructions as migration-only guidance; new frontend work should target the Go API.\n- Update Docker's default entrypoint, dependency image, and CI workflows when removing their remaining Python paths. Keep the frontend development proxy routes aligned with the Go API and Admin ports in `web/vite.config.ts`.\n- Keep this guide aligned with the checked-in tree. Once the Python directories are deleted, remove their migration inventory below.\n\n## Code Layout to Expect\n- `cmd/ragflow_server.go`: Go entrypoint for the API, admin, ingestor, and syncer modes (`--api`, `--admin`, `--ingestor`, `--syncer`).\n- `cmd/ragflow-cli.go`: Go CLI entrypoint.\n- `api/`, `rag/`, `deepdoc/`, and `agent/`: legacy Python implementation pending deletion. Do not add features or extend dependencies there.\n- `internal/`: main Go application code. Important subtrees:\n- `internal/agent/`: Go agent runtime, canvas execution, components, tool bindings, workflow helpers.\n- `internal/admin/`: Go admin routes, handlers, and services.\n- `internal/binding/cpp/`: C++ tokenizer binding built by `build.sh` for the Go server.\n- `internal/cli/`: CLI parsing, HTTP transport, command execution, response formatting.\n- `internal/dao/`: Go data-access layer and persistence-facing helpers.\n- `internal/deepdoc/`: Go DeepDOC integrations, especially native-backed PDF/DOCX parsing.\n- `internal/engine/`: search/index backends such as Elasticsearch and Infinity.\n- `internal/entity/`: shared Go entities and model definitions.\n- `internal/handler/`: HTTP handlers and route-facing request logic.\n- `internal/ingestion/`: Go ingestion pipeline, canvas adapter, components, wiring, service orchestration.\n- `internal/ingestion/component/`: stage implementations such as file/parser/chunker/tokenizer/extractor.\n- `internal/ingestion/pipeline/`: DSL translation, canvas-driven execution, checkpoints, resume/run logic.\n- `internal/parser/`: parser and chunk libraries used by ingestion and other Go paths.\n- `internal/parser/parser/`: typed parse-result parsers for markdown/html/pdf/docx/xlsx/text and related families.\n- `internal/parser/chunk/`: chunk operator library and DSL/typed execution helpers.\n- `internal/service/`: higher-level business services used by handlers and server flows.\n- `internal/storage/`: storage backends and in-memory test doubles.\n- `internal/router/`: HTTP route registration.\n- `internal/server/`: server bootstrap/config wiring.\n- `internal/syncer/`: Go file synchronization service and connectors.\n- `internal/rag/`: Go retrieval and RAG logic.\n- `internal/harness/`: Go runtime harness and supporting execution code.\n- `web/`: frontend application.\n- `docker/`: local and production compose files.\n- `sdk/python/`: legacy Python client SDK pending deletion.\n- `tools/scripts/download_dev_deps.py` and `deps/download_deps.py`: Python downloaders for the Go build's dependencies. `download_dev_deps.py` pulls the native static libraries (pdfium, pdf_oxide, office_oxide, onnxruntime) that `build.sh` links; `download_deps.py` pulls the runtime resources (cl100k BPE table, DeepDoc models, embedding tokenizer assets) and installs the stagehand server into the SDK cache. Versions and URLs are defined inline in each downloader (`tools/scripts/download_dev_deps.py` for native static libraries, `deps/download_deps.py` for runtime resources); the tokenizer assets are inlined as `TOKENIZER_ASSETS` in each downloader. Replace these with non-Python preparation before removing Python dependencies.\n- `test/`: contains Python tests, including tests that import the legacy Python implementation directly. Migrate needed coverage to Go tests or frontend tests, then remove the Python tests; keep only tests for supported code.\n\n## Go-Specific Rules\n- Treat `internal/ingestion`, `internal/parser`, and `internal/deepdoc` as actively refactored code. Prefer collapsing duplicate paths over preserving transitional wrappers.\n- Do not add or preserve deprecated Go APIs just to ease migration inside the repo.\n- Remove commented-out Go code instead of leaving recovery notes in place.\n- Keep package comments and doc comments aligned with the current runtime path, not with migration history.\n\n## GORM query traps\n- **Never pass a bare string ID as the second argument to `Take`/`First`** (e.g. `Take(&x, id)`). Under GORM v1.25.7 a non-numeric string is treated as a raw SQL condition and concatenated into the `WHERE` clause: `Take(&x, \"88b17c7a...\")` emits `WHERE 88b17c7a... LIMIT 1`, which MySQL rejects with `Error 1054 (42S22): Unknown column '...' in 'where clause'`. Only strings that parse via `strconv.Atoi` are treated as primary-key values, so this bug is invisible for integer IDs and catastrophic for UUID primary keys. Always bind explicitly: `Take(&x, \"id = ?\", id)` or `Where(\"id = ?\", id).Take(&x)`.\n- When you change a query form across many files, verify the generated SQL with a `Session(&gorm.Session{DryRun: true})` probe before relying on it. A passing unit test with no rows and a broken `WHERE` clause look identical unless the SQL is inspected.\n\n## Shared database schema (Go + Python)\nThe Go services and the Python API write to the same MySQL/PostgreSQL schema, and both own parts of it. Neither is authoritative over the whole.\n\nEach ORM derives index names differently:\n- Go (`internal/entity/`): GORM derives `idx_
_` for indexes and `uni_
_` for unique constraints. Tags in this repo name unique indexes explicitly, e.g. `uniqueIndex:idx_commit_file`.\n- Python (`api/db/db_models.py`): Peewee derives `__` (truncated with an md5 suffix past 64 chars, see `playhouse.migrate.make_index_name`). There is no `idx_` prefix.\n\nBoth sides reconcile by **column set, not by name**: Go's `hasUniqueIndex` (`internal/dao/migration.go`) accepts an existing index only when its full column set equals the requested one, and Python's `ensure_model_indexes` (`api/db/db_models.py`) keys `DB.get_indexes()` on the column tuple. Consequences:\n\n- Divergent names covering the same columns are expected, not a defect. Whichever runtime reaches the database first decides the name. Do not \"fix\" this by renaming inside a migration; it rewrites live databases for no functional gain.\n- Add a shared unique index to **both** sides (Go tag + Peewee `Meta.indexes` or `unique=True`), so each runtime recreates it when absent. Do not add a second index over a column set an existing one already covers.\n- In Go, declare single-column uniqueness with a named `uniqueIndex:`, not a bare `unique` tag. A bare `unique` becomes a table-level constraint in DDL, which SQLite materializes as an implicit index that cannot be dropped by name, and which GORM's `MigrateColumnUnique` then tries to drop under its own `uni_
_` name — MySQL answers 1091 every startup. `namedIndexMigrator` (`internal/dao/database.go`) suppresses exactly that phantom drop; see `TestNamedIndexMigratorSkipsPhantomUniqueDrop`.\n- A Python model existing does not mean Python owns that table's constraints. `pipeline_operation_log` has no `run_count` column and no unique index in `db_models.py`; both are added only by `internal/dao/migration.go`.\n\n## Go Test Tiers\nGo tests are classified by build tag. Tag a test file with `//go:build ` placed before the `package` clause. Run them through `build.sh` so the required CGO configuration and native libraries are available.\n\n| Tier | Build tag | Runs by default? | Needs |\n|---|---|---|---|\n| Unit | (none) | Yes (`bash build.sh --test`) | Native CGO static libs; no external services — uses in-memory SQLite, miniredis, or `httptest` stubs. |\n| Integration | `integration` | No (`-tags integration`) | A real service: MySQL/MinIO/Elasticsearch/Infinity/LLM. Single component, reasonably fast. |\n| E2E | `e2e` | No (`-tags e2e`) | Full cross-component pipeline (ingest → index → retrieve) against real services; heavy/slow. |\n| Manual | `manual` | No (`-tags manual`) | Very slow/expensive (deepdoc render/parity/snapshot/bench). **Local opt-in ONLY — never run in CI.** |\n| Native (orthogonal) | `cgo` / `!cgo` | `cgo` auto-satisfies under CGO_ENABLED=1 | Native static libs (`office_oxide`/`pdfium`/`pdf_oxide`). Combine with tiers, e.g. `//go:build cgo && integration`. |\n\nRun tiers locally via `build.sh`:\n```bash\nbash build.sh --test # unit tier (no tags)\nbash build.sh --test-integration ./... # integration tier\nbash build.sh --test-e2e # e2e tier\nbash build.sh --test-manual # manual tier (very slow)\nbash build.sh --test-all # integration + e2e (never includes manual)\n```\nRules:\n- New tests that touch a real external service MUST carry `integration`/`e2e`/`manual` — do not rely on `t.Skip` + env vars to soft-isolate them in the default unit run. Keep an env guard as a harmless secondary safety net if desired.\n- `manual` is never wired into CI or any automated pipeline.\n- `unit` (no tag) must stay free of external-service dependencies so `bash build.sh --test` runs without MySQL/MinIO/ES/Infinity/LLM. The native CGO static libraries (`office_oxide`/`pdfium`/`pdf_oxide`) are still required at build time and are wired by `build.sh --test`; that is expected, not an external service.\n\n## Working Rules\n- When reviewing documentation or code, inspect the full affected path and report all verifiable findings in one review; do not return after only a few findings and expose further issues in later rounds.\n- When handling review comments, independently verify each substantive claim against the current code or tests before accepting, rejecting, or acting on it.\n- Before editing, inspect the nearest code path that actually owns the behavior.\n- For server changes, work in the Go implementation. If a Python behavior is still needed, port it to Go and test the Go path before removing the Python code.\n- Keep changes small and local unless the task is explicitly a broader refactor.\n- Prefer one implementation path instead of preserving old and new versions side by side.\n- Preserve behavior with focused tests when the behavior is still valid; do not keep tests that protect obsolete behavior.\n- If a surface is only there for compatibility, remove it unless the user asks to keep it.\n- Do not add new compatibility wording in comments or docs.\n- When a maintainer takes over a community PR, a new commit generated by rewriting history (e.g. `merge`, `rebase -i`) must preserve the original author and add the maintainer as co-author (via a `Co-authored-by:` trailer) instead of overwriting the author with the maintainer alone.\n\n## Commands\n### Go backend\n```bash\nbash build.sh --all # build native code and Go server\nbash build.sh --go # build Go server with native library already built\nbash build.sh --test ./internal/handler/... # targeted unit tests\nbash build.sh --test # all unit tests\nbash build.sh --test-integration ./... # integration tests; requires real services\nbash build.sh --test-e2e # full-pipeline tests; requires real services\nbash build.sh --test-manual # local opt-in only\nbash build.sh --test-all # integration + e2e; excludes manual\n```\n\nNative libraries, Go DeepDoc models, tokenizer assets, and `deps/cl100k_base.tiktoken` come from `tools/scripts/download_dev_deps.py` (native static libs) and `deps/download_deps.py` (models, cl100k, tokenizer assets); run them with `uv run`. Build the Go resource image with `cd deps && docker build -f Dockerfile -t infiniflow/ragflow_deps:latest .` after downloading the resources.\n\n### Frontend\n```bash\ncd web\nnpm install\nnpm run dev\nnpm run build\nnpm run lint\nnpm run test\nnpm run type-check\n```\n\n## Validation Preference\n- Run the narrowest relevant test, lint, or build command after a change.\n- For backend changes, first run package-scoped `bash build.sh --test ./path/to/package/...`. Run the integration or e2e tier when the changed behavior needs real services or a full pipeline.\n- For frontend changes, prefer the touched-package lint, type-check, or test command.\n- Do not default to raw `go test`, `go build`, or IDE Run/Debug for Go in this repo. They often miss the required CGO flags and native static libraries (`office_oxide`, `pdfium-static`, `pdf_oxide`) that `build.sh` wires correctly.\n- After a Go build, execute the resulting binary's help command for each changed server mode; a successful link alone does not prove the binary starts. On this host, the original `build.sh` selected default LLD 18 and produced binaries that crashed before Go `main`; linking with LLD 20 started successfully. Verify the linker actually selected on Linux before trusting a new binary.\n- If Go native builds fail, inspect `build.sh` and `internal/development.md` before changing code. Common environment issues are missing downloaded native dependencies or an incompatible `ld.lld` on Linux.\n\n## Default review checklist\n- Remove instead of retaining `deprecated`, `legacy`, or compatibility-only code.\n- Collapse duplicate implementations to one path.\n- Drop stale comments and documentation that describe a superseded design.\n- Keep exported APIs only when the current code actually needs them.\n\n\n# AGENTS.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with the RAGFlow frontend (`web/`).\n\n## Project Overview\n\nRAGFlow frontend is a React/TypeScript application built with Vite:\n\n- **Components**: shadcn/ui\n- **Styling**: Tailwind CSS\n- **State**: Zustand\n- **Data Fetching**: TanStack Query (React Query)\n- **i18n**: react-i18next\n\n## Common Commands\n\n```bash\nnpm install\nnpm run dev # Development server\nnpm run build # Production build\nnpm run lint # oxlint\nnpm run format # oxfmt\nnpm run test # Jest tests\n```\n\n## Development Conventions\n\n### Pre-commit Format Validation (Mandatory)\n\nFrontend code changes **must pass the lefthook format checks before every commit**. Do not rely on CI (`web-oxfmt` / `web-oxlint` jobs) to catch formatting after pushing.\n\n- Before staging or committing changed `.ts`/`.tsx`/`.js`/etc. files, run the formatter and linter locally:\n - `npx oxfmt ` (or `npm run format`) to auto-fix formatting\n - `npx oxlint` (or `npm run lint`) for lint issues\n- Verify with `npx oxfmt --check ` before `git commit`.\n- Let the lefthook pre-commit hook run to completion and fix everything it reports; never bypass hooks with `--no-verify`.\n\n### Test File Placement\n\nWhen tests are grouped in a dedicated directory, that directory is always named `__tests__/` — never `tests/`. Example: `src/pages/agent/utils/__tests__/extractor-transform.test.ts` for `src/pages/agent/utils/extractor-transform.ts`. Co-located `foo.test.ts(x)` files next to the source file are also fine. When you find an existing test under a `tests/` directory, move it into `__tests__/` (keep git history via `git mv`) rather than adding new tests there.\n\n### Backend Conventions\n\nThe frontend serves only the **Go** backend. There is no runtime backend detection and no per-backend branching in app code; the dev server proxies `/api` and `/v1` to the Go API (`vite.config.ts`).\n\nDo not reintroduce backend-variant primitives or detection helpers (e.g. a `useIsGoBackend` hook, `pickByBackend`, ``, or an `/api/v1/language` probe) and do not create `go/`-vs-`python/` variant directories or files.\n\n### CSS and Layout Debugging\n\nWhen fixing CSS/layout issues (especially flex truncation, ellipsis, or element sizing), **always inspect the full parent hierarchy** for `flex-shrink`, `min-width`, and `overflow` constraints before applying fixes like `min-w-0`. Do not repeatedly apply the same fix without verifying the root cause.\n\n- Before editing, explain: (1) the full flex/container hierarchy from the target element up to the nearest non-flex ancestor, (2) what constraint is actually causing the bug, and (3) how the proposed fix addresses that root cause.\n\n### Color Tokens\n\nWhen writing or modifying styles, **use the project-defined color tokens from `src/tailwind.css`** (e.g., `bg-bg-base`, `text-text-primary`, `text-text-secondary`, `text-text-disabled`, `border-border-button`, `bg-bg-card`). Do not use arbitrary hex/RGB values or Tailwind's default palette colors (e.g., `emerald-500`, `blue-400`) directly in component class names. These tokens are defined for both light and dark modes and keep the UI consistent with the design system.\n\n### Scope and Boundaries\n\nRespect explicit boundaries from the user. If the user says **\"only fix the selected line\"** or **\"do not touch shared types/files\"**, follow that instruction exactly. Do not investigate unrelated errors, modify shared schemas (e.g., `LlmSettingFieldSchema`), or refactor other files without confirmation. If a change outside the described scope seems necessary, ask for permission first.\n\n### Internationalization (i18n)\n\nFor translation tasks, add keys **only to the explicitly requested language files** (commonly `src/locales/zh.ts` and `src/locales/en.ts`). Do not auto-propagate changes to all language files unless the user explicitly asks.\n\n- **Style for `en.ts`**: Sentence case — first word capitalized, rest lowercase (e.g., `referenceAnswer: 'Reference answer'`). Proper nouns remain as-is.\n\n### React Component Refactoring\n\nWhen refactoring or extracting components, **verify layout behavior after each structural change** (especially `flex-1`, conditional rendering, or flex direction changes). Check that existing buttons, alignment, and responsive behavior remain intact. After extraction, verify: (1) all original props and behavior are preserved, (2) layout in parent contexts is identical, and (3) no syntax or type errors were introduced.\n\n### State Management and Data Fetching\n\n#### Query Key Factory (Mandatory)\n\n**Never write raw `queryKey` arrays inline.** Always use a query key factory object that returns `as const` tuples. Raw arrays duplicated across `useQuery` and `invalidateQueries` are brittle, unreadable, and cause stale-cache bugs when key structures drift.\n\n```ts\n// ❌ Bad — raw array, hard to match with useQuery\nqueryClient.invalidateQueries({\n queryKey: [\n LLMApiAction.AddedProviders,\n params.provider_name,\n params.instance_name,\n 'models',\n ],\n});\n\n// ✅ Good — factory reference, self-documenting\nqueryClient.invalidateQueries({\n queryKey: LlmKeys.instanceModels(params.provider_name, params.instance_name),\n});\n```\n\n- Place the factory in the same file as the hooks, named `{Domain}Keys` (e.g., `LlmKeys`, `DatasetKeys`).\n- Every `useQuery` and every `invalidateQueries` must reference the same factory function.\n- Use `as const` on each factory return value for type-safe readonly tuples.\n\n#### Cache Debugging\n\nFor React Query / cache invalidation bugs, **carefully compare query keys across all consuming components and mutation hooks**. Mismatched keys (e.g., with/without `refreshCount`) are a common root cause of stale data or duplicate requests.\n\n- Systematically: (1) list every component/hook that calls `useQuery` for this data, (2) compare their query keys character-for-character, (3) check every mutation's `onSuccess` for cache invalidation, and (4) verify no parent re-renders are remounting the observer.\n\n#### Colocate Queries with the Consuming View\n\n**Fire a query in the component that renders its data — not in a parent page.** When a page switches between mutually exclusive views (tabs, view modes), extract each view into its own component that issues its own requests on mount. Conditional rendering then provides lazy loading for free.\n\n- Do not hoist child-view queries into the page component — it fires requests the user may never need (e.g., fetching the skill tree on page entry while the default view is the LLM wiki).\n- Do not thread `enabled` flags or view-mode props through hooks to gate a hoisted query; that is a sign the query lives at the wrong level. Split the view instead.\n- Remember the trade-off: with `gcTime: 0`, unmounting a view drops its cache, so switching back refetches. That is usually desirable for always-fresh data — do not reintroduce eager hoisting just to avoid the refetch.\n\n### Network Request Layering\n\nHTTP requests are organized in three layers. **Never import `@/utils/request`, `@/utils/next-request`, or `@/utils/api` directly inside a hook**:\n\n1. `src/hooks/use-xx-request.ts(x)` — React Query hooks; only call the service layer.\n2. `src/services/xx-service.ts` — Register endpoints via `registerNextServer`, all going through `@/utils/next-request`.\n3. `src/utils/next-request.ts` — The single axios instance; handles token, 401 redirects, and error notifications.\n\nInterface types are split between two folders:\n\n- Response/data shape → `src/interfaces/database/xx.ts`\n- Request params/body → `src/interfaces/request/xx.ts`\n\nModel-related endpoints (LLM provider / factory / my LLM, etc.) are consolidated in `src/services/llm-service.ts` rather than scattered across hooks. For GET endpoints, register with `method: 'get'` in the service, and on the call site pass `true` as the second argument to use the native axios config (e.g., `service.listProviders({ params: { available: true } }, true)`).\n\n### Shared UI Component Lock\n\nThe folder `src/components/ui/` is the project's **shared UI library** — it contains both official shadcn/ui primitives and project-authored common components built on top of shadcn. Both kinds are intended to be reused across the app and **must not be modified casually**.\n\n- **Do not modify, refactor, restyle, or \"improve\"** any file under `src/components/ui/` (including subfolders), even if it seems like the most direct fix.\n- If a component does not meet requirements, **wrap or compose it** in a new component **outside** `src/components/ui/` (e.g., under `src/components/` or a feature folder), and customize via `className`, `props`, or composition.\n- Exceptions require **explicit user approval** in the same conversation. When in doubt, ask first and propose a wrapper-based alternative.\n- Adding a new shared component to `src/components/ui/`, or upgrading a shadcn primitive via the official `shadcn` CLI, is allowed only when the user explicitly requests it.\n\n### React Patterns and Conventions\n\n- **Avoid inline event handlers.** Do not define arrow functions directly on JSX event props like `onClick={() => ...}` or `onChange={(e) => ...}`. Instead, extract the handler and reference it by name (e.g., `onClick={handleClick}`). Inline handlers are recreated on every render, which can break `React.memo` optimizations and make stack traces harder to read.\n- **Prefer `requestAnimationFrame` or `useLayoutEffect`** over `setTimeout(..., 0)` for focus or DOM measurement operations.\n- **Prefer `useTranslation` from `react-i18next`** over project-wrapped utilities like `useTranslate`.\n- Extract complex logic into hooks or utils; keep components lean.\n- Use **PascalCase** for constants and component names.\n - Components: `EditableTextarea`, `RAGFlowFormItem`\n - Constants: `InitialMockData`, `DefaultPlaceholder`\n- Avoid camelCase or SCREAMING_SNAKE_CASE for components and top-level constants.\n- **Place domain constant lists in the page's `constant/` directory** (e.g. `pages/agent/constant/pipeline.tsx`) when they belong to an existing family of constants there (like `ChunkerOperators`, `SingleOperators`), instead of declaring them locally next to their consumer. Keeping related lists in one file makes their overlaps and differences visible and gives new entries a single obvious home.\n- **Name things semantically, not generically.** A name should say what the value holds or what the callback does, so call sites read naturally. Don't leak a generic underlying-primitive API into your own component's public props — rename at the wrapper boundary (e.g., a tabs wrapper exposes `activeTab` / `onTabChange`, mapped internally to Radix's `value` / `onValueChange`), and align with what callers already name the state. Same for variables and functions: `hiddenNodeIds` over `nodeIdSet`, `pickByBackend` over `resolveVariant`.\n- Avoid duplicating component structures in JSX; favor render props or reusable components.\n\n### Utility Libraries and Reuse\n\n- **Time/date handling**: Use `dayjs` for all date/time formatting, parsing, and manipulation.\n- **Utility hooks**: Prefer `ahooks` for common reusable hooks (e.g., `useDebounce`, `useSetState`).\n- **General utilities**: Lodash is available for utility functions when needed.\n- **Project utilities first**: Before reaching for a third-party library, check if the project already has an existing utility or hook that covers the need.\n- **Extract and share**: If repeated logic cannot be satisfied by an existing project utility or a third-party library, extract it into an appropriate shared hook (`src/hooks/`) or utility file (`src/utils/`).\n- **Check for duplicate patterns before adding**: When asked to add logic (validation, existence checks, API calls, etc.), first search for existing hooks/functions that do the same or similar thing — especially in the same file or sibling hooks. If a hook already does X, call it instead of re-implementing X inline. This applies to mutations calling mutations, utility wrappers, and boilerplate around API calls.\n\n\n# Go Naming Best Practices\n\n## 1. Package Naming\n\n- **All lowercase, no underscores**: `package user`, not `package userService` or `package user_service`\n- **Short and meaningful**: `package http`, `package json`, `package dao`\n- **Avoid plurals**: `package user` not `package users`\n- **Avoid generic names**: Avoid `package util`, `package common`, `package base`\n\n```go\n// Recommended\npackage user\npackage handler\npackage service\n\n// Not recommended\npackage UserService\npackage user_service\npackage utils\n```\n\n## 2. File Naming\n\n- **All lowercase, underscore separated**: `user_handler.go`, `user_service.go`\n- **Test files**: `user_handler_test.go`\n- **Platform-specific**: `user_linux.go`, `user_windows.go`\n\n```\nuser/\n├── user_handler.go\n├── user_service.go\n├── user_dao.go\n└── user_test.go\n```\n\n## 3. Directory Naming\n\n- **All lowercase, no underscores or hyphens**: `internal/`, `pkg/`, `cmd/`\n- **Short and descriptive**: `handler/`, `service/`, `dao/`\n\n```\nproject/\n├── cmd/ # Main entry point\n│ └── server_main.go\n├── internal/ # Private code\n│ ├── handler/\n│ ├── service/\n│ ├── dao/\n│ ├── model/\n│ └── middleware/\n├── pkg/ # Public code\n└── api/ # API definitions\n```\n\n## 4. Interface Naming\n\n- **Single-method interfaces end with \"-er\"**: `Reader`, `Writer`, `Handler`\n- **Verb form**: `Reader`, `Executor`, `Validator`\n\n```go\n// Recommended\ntype Reader interface {\n Read(p []byte) (n int, err error)\n}\n\ntype UserService interface {\n Register(req *RegisterRequest) (*User, error)\n Login(req *LoginRequest) (*User, error)\n}\n\n// Not recommended\ntype UserInterface interface {}\ntype IUserService interface {}\n```\n\n## 5. Struct Naming\n\n- **CamelCase**: `UserService`, `UserHandler`\n- **Avoid redundant prefixes**: `User` not `UserModel`\n\n```go\n// Recommended\ntype UserService struct {}\ntype UserHandler struct {}\ntype RegisterRequest struct {}\n\n// Not recommended\ntype user_service struct {}\ntype SUserService struct {}\ntype UserModel struct {}\n```\n\n## 6. Method/Function Naming\n\n- **CamelCase**\n- **Start with verb**: `GetUser`, `CreateUser`, `DeleteUser`\n- **Boolean returns use Is/Has/Can prefix**: `IsValid`, `HasPermission`\n\n```go\n// Recommended\nfunc (s *UserService) Register(req *RegisterRequest) (*User, error)\nfunc (s *UserService) GetUserByID(id uint) (*User, error)\nfunc (s *UserService) IsEmailExists(email string) bool\n\n// Not recommended\nfunc (s *UserService) register_user()\nfunc (s *UserService) get_user_by_id()\nfunc (s *UserService) CheckEmailExists() // Should use Is/Has\n```\n\n## 7. Constant Naming\n\n- **CamelCase**: `const MaxRetryCount = 3`\n- **Enum constants**: `const StatusActive = \"active\"`\n\n```go\n// Recommended\nconst (\n StatusActive = \"1\"\n StatusInactive = \"0\"\n MaxRetryCount = 3\n)\n\n// Not recommended\nconst (\n STATUS_ACTIVE = \"1\" // Not all uppercase\n status_active = \"1\" // Not all lowercase\n)\n```\n\n## 8. Error Variable Naming\n\n- **Start with \"Err\"**: `ErrNotFound`, `ErrInvalidInput`\n\n```go\n// Recommended\nvar (\n ErrNotFound = errors.New(\"not found\")\n ErrInvalidInput = errors.New(\"invalid input\")\n ErrUnauthorized = errors.New(\"unauthorized\")\n)\n```\n\n## 9. Acronyms Keep Consistent Case\n\n```go\n// Recommended\ntype HTTPHandler struct {}\nvar URL string\nfunc GetHTTPClient() {}\nfunc ParseJSON() {}\n\n// Not recommended\ntype HttpHandler struct {}\nvar Url string\nfunc GetHttpClient() {}\n```\n\n## 10. Project Structure Naming\n\n```\nproject-name/\n├── cmd/ # Main programs\n│ └── app_name/\n│ └── main.go\n├── internal/ # Private code\n│ ├── handler/ # HTTP handlers\n│ ├── service/ # Business logic\n│ ├── repository/ # Data access\n│ ├── model/ # Data models\n│ └── config/ # Configuration\n├── pkg/ # Public code\n├── api/ # API definitions\n├── configs/ # Config files\n├── scripts/ # Scripts\n├── docs/ # Documentation\n├── go.mod\n└── go.sum\n```\n\n## Summary Table\n\n| Type | Rule | Example |\n| -------------- | ----------------------------------- | ------------------- |\n| Package | All lowercase, no underscores | `package user` |\n| File | All lowercase, underscore separated | `user_service.go` |\n| Directory | All lowercase, no separators | `internal/handler/` |\n| Struct | CamelCase, capitalized first letter | `UserService` |\n| Interface | CamelCase, -er suffix | `Reader`, `Writer` |\n| Method | CamelCase, verb prefix | `GetUserByID` |\n| Constant | CamelCase | `MaxRetryCount` |\n| Error Variable | Err prefix | `ErrNotFound` |\n\n\n---\nname: go-naming\ndescription: Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards.\n---\n\nStrictly follow the naming conventions in [rules/named.md](../../rules/named.md)\n\n\n{\n \"label\": \"References\",\n \"position\": 6,\n \"link\": {\n \"type\": \"generated-index\",\n \"description\": \"Miscellaneous References\"\n },\n \"customProps\": {\n \"sidebarIcon\": \"LucideScrollText\"\n }\n}\n\n\n---\nsidebar_position: 4\nslug: /http_api_reference\nsidebar_custom_props: {\n categoryIcon: LucideGlobe\n}\n---\n# HTTP API\n\nA complete reference for RAGFlow's RESTful API, including migration notes for the 1.0 Go backend. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).\n\nIn the standard response envelope, `message` is always a non-empty string. Successful responses default to `\"success\"`; error responses contain a description of the error.\n\n---\n\n## ERROR CODES\n\nRAGFlow responses may contain both an HTTP status code and a business code in the JSON response body. These codes should be checked separately.\n\n### HTTP status codes\n\n| Code | Meaning |\n|------|---------|\n| 200 | The HTTP request was processed successfully. Check the response body `code` for the business result. |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Internal server error |\n\n### Response body codes\n\n| Code | Meaning |\n|------|---------|\n| 0 | Success |\n| 10 | Not effective |\n| 100 | Exception error |\n| 101 | Invalid request argument |\n| 102 | Invalid or missing data |\n| 103 | Operation error |\n| 105 | Connection error |\n| 106 | Operation still running |\n| 108 | Permission error |\n| 109 | Authentication error |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Server error |\n---\n\n## Migrate to 1.0\n\nRAGFlow 1.0 moves the public REST API from the Python backend to Go. The canonical endpoints documented below remain available.\n\nThe following changes require client updates:\n\n| Area | RAGFlow 1.0 Go change | Client action |\n|------|------------------------|---------------|\n| Dataset parser selection | Dataset create and update use `parse_type` with `parser_id` (BuiltIn) or `pipeline_id` (Pipeline); responses use the canonical value `general` instead of the legacy `chunk_method: \"naive\"`. | Replace `chunk_method` with `\"parse_type\": 1` and `\"parser_id\": \"\"`. For the general parser, send `\"parser_id\": \"general\"`. |\n| Dataset parser configuration | Go stores and returns component-scoped parser settings. Most legacy top-level `parser_config` fields are not mapped into that component structure. | Read the returned component map and send component-scoped keys when changing parser settings. |\n| Errors and authentication | Some validation business codes, messages, and authentication HTTP statuses differ from Python. | Branch on documented codes rather than exact message text. |\n\n:::note\nGo responses include `X-API-Source: go`, which can be used to verify the serving backend during rollout.\n:::\n\n### Removed API aliases\n\nThe following paths remain available only on the Python backend. They are not registered by the RAGFlow 1.0 Go backend.\n\n| Removed endpoint | Replacement endpoint |\n|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|\n| **POST** `/api/v1/chats_openai/{chat_id}/chat/completions` | **POST** `/api/v1/openai/{chat_id}/chat/completions` |\n| **PUT** `/api/v1/chats/{chat_id}/sessions/{session_id}` | **PATCH** `/api/v1/chats/{chat_id}/sessions/{session_id}` |\n| **POST** `/api/v1/chats/{chat_id}/completions` | **POST** `/api/v1/chat/completions` |\n| **POST** `/api/v1/sessions/related_questions` | **POST** `/api/v1/chat/recommendation` |\n| **PUT** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` | **PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks/{chunk_id}` |\n| **GET** `/v1/system/healthz` | **GET** `/api/v1/system/healthz` |\n| **POST** `/v1/document/upload_info` | **POST** `/api/v1/documents/upload` |\n| **POST** `/api/v1/file/upload` | **POST** `/api/v1/files` |\n| **POST** `/api/v1/file/create` | **POST** `/api/v1/files` |\n| **GET** `/api/v1/file/list` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/root_folder` | **GET** `/api/v1/files` |\n| **GET** `/api/v1/file/parent_folder` | **GET** `/api/v1/files/{file_id}/parent` |\n| **GET** `/api/v1/file/all_parent_folder` | **GET** `/api/v1/files/{file_id}/ancestors` |\n| **POST** `/api/v1/file/rm` | **DELETE** `/api/v1/files` |\n| **POST** `/api/v1/file/rename` | **POST** `/api/v1/files/move` |\n| **GET** `/api/v1/file/get/{file_id}` | **GET** `/api/v1/files/{file_id}` |\n| **POST** `/api/v1/file/mv` | **POST** `/api/v1/files/move` |\n| **POST** `/api/v1/file/convert` | **POST** `/api/v1/files/link-to-datasets` |\n| **POST** `/api/v1/agents_openai/{agent_id}/chat/completions` | **POST** `/api/v1/agents/chat/completions` with `\"openai-compatible\": true` |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\n**POST** `/api/v1/openai/{chat_id}/chat/completions`\n\nCreates a model response for a given chat conversation.\n\n:::caution REMOVED ALIAS\n`POST /api/v1/chats_openai/{chat_id}/chat/completions` is not available in 1.0. Use this endpoint instead.\n:::\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/openai/{chat_id}/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"model\"`: `string` (optional)\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"extra_body\"`: `object` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/openai/{chat_id}/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"model\": \"glm-4-flash@ZHIPU-AI\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true,\n \"extra_body\": {\n \"reference\": true,\n \"reference_metadata\": {\n \"include\": true,\n \"fields\": [\"author\", \"year\", \"source\"]\n },\n \"metadata_condition\": {\n \"logic\": \"and\",\n \"conditions\": [\n {\n \"name\": \"author\",\n \"comparison_operator\": \"is\",\n \"value\": \"bob\"\n }\n ]\n }\n }\n }'\n```\n\n##### Request parameters\n\n- `chat_id` (*Path parameter*) `string`, *Required*\n Existing chat assistant ID. The request will use that chat assistant's knowledge and settings.\n\n- `model` (*Body parameter*) `string`\n The model used to generate the response. When `chat_id` is provided, you may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `extra_body` (*Body parameter*) `object`\n Extra request parameters:\n - `reference`: `boolean` - include reference in the final chunk (stream) or in the final message (non-stream).\n - `reference_metadata`: `object` - include document metadata in each reference chunk.\n - `include`: `boolean` - enable document metadata in reference chunks.\n - `fields`: `list[string]` - optional allowlist of metadata keys. Omit to include all. Use an empty list to include none.\n - `metadata_condition`: `object` - metadata filter conditions applied to retrieval results.\n\n#### Response\n\nStream:\n\n```json\ndata:{\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"Hello! It seems like you're just greeting me. If you have a specific\",\n \"role\": \"assistant\",\n \"function_call\": null,\n \"tool_calls\": null,\n \"reasoning_content\": null\n },\n \"finish_reason\": null,\n \"index\": 0,\n \"logprobs\": null\n }\n ],\n \"created\": 1755084508,\n \"model\": \"model\",\n \"object\": \"chat.completion.chunk\",\n \"system_fingerprint\": \"\",\n \"usage\": null\n}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" question or need information, feel free to ask, and I'll do my best\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": \" to assist you based on the knowledge base provided.\", \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": null, \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": null}\n\ndata:{\"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\", \"choices\": [{\"delta\": {\"content\": null, \"role\": \"assistant\", \"function_call\": null, \"tool_calls\": null, \"reasoning_content\": null}, \"finish_reason\": \"stop\", \"index\": 0, \"logprobs\": null}], \"created\": 1755084508, \"model\": \"model\", \"object\": \"chat.completion.chunk\", \"system_fingerprint\": \"\", \"usage\": {\"prompt_tokens\": 5, \"completion_tokens\": 188, \"total_tokens\": 193}}\n\ndata:[DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"Hello! I'm your smart assistant. What can I do for you?\",\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": 1755084403,\n \"id\": \"chatcmpl-3b0397f277f511f0b47f729e3aa55728\",\n \"model\": \"model\",\n \"object\": \"chat.completion\",\n \"usage\": {\n \"completion_tokens\": 55,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 55,\n \"reasoning_tokens\": 5,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 5,\n \"total_tokens\": 60\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n---\n\n### Create OpenAI-compatible agent completion\n\n:::caution REMOVED ALIAS\nRAGFlow 1.0 Go does not serve `POST /api/v1/agents_openai/{agent_id}/chat/completions`. Use the endpoint below and move `agent_id` into the request body.\n:::\n\n**POST** `/api/v1/agents/chat/completions`\n\nCreates a model response for a given chat conversation.\n\nThis API follows the same request and response format as OpenAI's API. It allows you to interact with the model in a manner similar to how you would with [OpenAI's API](https://platform.openai.com/docs/api-reference/chat/create).\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/agents/chat/completions`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"agent_id\"`: `string`\n - `\"openai-compatible\"`: `boolean`\n - `\"messages\"`: `object list`\n - `\"stream\"`: `boolean`\n - `\"session_id\"`: `string` (optional)\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/agents/chat/completions \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"agent_id\": \"{agent_id}\",\n \"openai-compatible\": true,\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"stream\": true\n }'\n```\n\n##### Body parameters\n\n- `agent_id` (*Body parameter*) `string`, *Required*\n The ID of the agent to run.\n\n- `openai-compatible` (*Body parameter*) `boolean`, *Required*\n Set this field to `true` to use the OpenAI-compatible request and response format.\n\n- `messages` (*Body parameter*) `list[object]`, *Required*\n A list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n- `stream` (*Body parameter*) `boolean`\n Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n- `session_id` (*Body parameter*) `string`\n Agent session id.\n\n#### Response\n\nStream:\n\n```json\n...\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \" terminal\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \".\"\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: {\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"object\": \"chat.completion.chunk\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"choices\": [\n {\n \"delta\": {\n \"content\": \"\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"id\": \"4b8935ac0a22deb1\",\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"url\": null,\n \"similarity\": 0.5697155305154673,\n \"vector_similarity\": 0.7323851005515574,\n \"term_similarity\": 0.5000000005,\n \"doc_type\": \"\"\n }\n },\n \"doc_aggs\": {\n \"INSTALL22.md\": {\n \"doc_name\": \"INSTALL22.md\",\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"count\": 3\n },\n \"INSTALL.md\": {\n \"doc_name\": \"INSTALL.md\",\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL(1).md\": {\n \"doc_name\": \"INSTALL(1).md\",\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"count\": 2\n },\n \"INSTALL3.md\": {\n \"doc_name\": \"INSTALL3.md\",\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"count\": 1\n }\n }\n }\n },\n \"finish_reason\": null,\n \"index\": 0\n }\n ]\n}\n\ndata: [DONE]\n```\n\nNon-stream:\n\n```json\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"\\nTo install Neovim, the process varies depending on your operating system:\\n\\n### For Windows:\\n1. **Download from GitHub**: \\n - Visit the [Neovim releases page](https://github.com/neovim/neovim/releases)\\n - Download the latest Windows installer (nvim-win64.msi)\\n - Run the installer and follow the prompts\\n\\n2. **Using winget** (Windows Package Manager):\\n...\",\n \"reference\": {\n \"chunks\": {\n \"20\": {\n \"content\": \"```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.\",\n \"dataset_id\": \"456ce60c5e1511f0907f09f583941b45\",\n \"doc_type\": \"\",\n \"document_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"document_name\": \"INSTALL22.md\",\n \"document_metadata\": {\n \"author\": \"bob\",\n \"year\": \"2023\",\n \"source\": \"internal\"\n },\n \"id\": \"4b8935ac0a22deb1\",\n \"image_id\": \"\",\n \"positions\": [\n [\n 12,\n 11,\n 11,\n 11,\n 11\n ]\n ],\n \"similarity\": 0.5697155305154673,\n \"term_similarity\": 0.5000000005,\n \"url\": null,\n \"vector_similarity\": 0.7323851005515574\n }\n },\n \"doc_aggs\": {\n \"INSTALL(1).md\": {\n \"count\": 2,\n \"doc_id\": \"4bdfb42e5e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL(1).md\"\n },\n \"INSTALL.md\": {\n \"count\": 2,\n \"doc_id\": \"4bd7fdd85e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL.md\"\n },\n \"INSTALL22.md\": {\n \"count\": 3,\n \"doc_id\": \"4bdd2ff65e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL22.md\"\n },\n \"INSTALL3.md\": {\n \"count\": 1,\n \"doc_id\": \"4bdab5825e1511f0907f09f583941b45\",\n \"doc_name\": \"INSTALL3.md\"\n }\n }\n },\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": null,\n \"id\": \"c39f6f9c83d911f0858253708ecb6573\",\n \"model\": \"d1f79142831f11f09cc51795b9eb07c0\",\n \"object\": \"chat.completion\",\n \"param\": null,\n \"usage\": {\n \"completion_tokens\": 415,\n \"completion_tokens_details\": {\n \"accepted_prediction_tokens\": 0,\n \"reasoning_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n },\n \"prompt_tokens\": 6,\n \"total_tokens\": 421\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The last content of this conversation is not from user.\"\n}\n```\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n**POST** `/api/v1/datasets`\n\nCreates a dataset.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### A basic request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test_1\",\n \"language\": \"English\",\n \"parse_type\": 1,\n \"parser_id\": \"general\"\n }'\n```\n\n##### A request example specifying ingestion pipeline\n\n:::caution WARNING\nDo not include `\"parser_id\"` when specifying an ingestion pipeline.\n:::\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"name\": \"test-sdk\",\n \"language\": \"English\",\n \"parse_type\": 2,\n \"pipeline_id\": \"\"\n }'\n```\n\n##### Body parameters\n\n- `\"name\"`: `string`, *Required*\n The unique name of the dataset to create. It must adhere to the following requirements:\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n If the name is already taken, the dataset is created with a numbered suffix such as `test_1(1)`, `test_1(2)`, and so on.\n\n- `\"language\"`: `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the server/database default is used.\n\n- `\"embedding_model\"`: `string`\n The name of the embedding model to use. For example: `\"BAAI/bge-large-zh-v1.5@BAAI\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n\n- `\"permission\"`: `string`\n Specifies who can access the dataset to create. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n\n- `\"parser_id\"`: `enum`\n The built-in parser to use. Required when `\"parse_type\"` is `1`.\n - `\"general\"`: General (default)\n - `\"qa\"`: Q&A\n - `\"manual\"`: Manual\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"audio\"`: Audio\n - `\"email\"`: Email\n - Must be omitted when `\"parse_type\"` is `2`.\n\n- `\"parser_config\"`: `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n- `\"parse_type\"`: `int`, *Required*\n The parser selection mode:\n - `1`: BuiltIn. Requires `\"parser_id\"`.\n - `2`: Pipeline. Requires `\"pipeline_id\"`.\n\n- `\"pipeline_id\"`: `string`\n The ingestion pipeline ID. Can be found in the corresponding URL in the RAGFlow UI.\n - Required when `\"parse_type\"` is `2`.\n - Must be a 32-character lowercase hexadecimal string, e.g., `\"d0bebe30ae2211f0970942010a8e0005\"`.\n - Must be omitted when `\"parse_type\"` is `1`.\n\n:::caution WARNING\n`\"parse_type\"` is required when creating a dataset. Choose exactly one mode:\n\n- `\"parse_type\": 1` with `\"parser_id\"` for a builtin parser.\n- `\"parse_type\": 2` with `\"pipeline_id\"` for an ingestion pipeline.\n\n`\"chunk_method\"`, `\"avatar\"`, `\"description\"`, and `\"auto_metadata_config\"` are not accepted by the RAGFlow 1.0 create endpoint. Use `\"parser_id\"` instead of `\"chunk_method\"`; set avatar and description with the update endpoint, and metadata configuration with the metadata endpoint.\n:::\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"chunk_count\": 0,\n \"create_time\": 1745836841611,\n \"created_by\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"document_count\": 0,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5@BAAI\",\n \"id\": \"3b4de7d4241d11f0a6a79f24fc270c7f\",\n \"language\": \"English\",\n \"name\": \"RAGFlow example\",\n \"pagerank\": 0,\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 128,\n \"delimiters\": [\"\\n\", \"!?;。；！？\"],\n \"parent_child\": {\n \"use_parent_child\": false,\n \"children_delimiter\": \"\\n\"\n }\n }\n },\n \"parser_id\": \"general\",\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"tenant_id\": \"3af81804241d11f0a6a79f24fc270c7f\",\n \"token_num\": 0,\n \"keywords_similarity_weight\": 0.7\n }\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"Field: - Message: - Value: <>\"\n}\n```\n\n---\n\n### Delete datasets\n\n**DELETE** `/api/v1/datasets`\n\nDeletes datasets by ID.\n\n#### Request\n\n- Method: DELETE\n- URL: `/api/v1/datasets`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"ids\"`: `list[string]` or `null`\n - `\"delete_all\"`: `boolean`\n\n##### Request example\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"ids\": [\"d94a8dc02c9711f0930f7fbc369eab6d\", \"e94a8dc02c9711f0930f7fbc369eab6e\"]\n }'\n```\n\n```bash\ncurl --request DELETE \\\n --url http://{address}/api/v1/datasets \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\n \"delete_all\": true\n }'\n```\n\n##### Body parameters\n\n- `\"ids\"`: `list[string]` or `null`\n Specifies the datasets to delete:\n - If omitted, or set to `null` or an empty array, no datasets are deleted.\n - If an array of IDs is provided, only the datasets matching those IDs are deleted.\n- `\"delete_all\"`: `boolean`\n Whether to delete all datasets owned by the current user when `\"ids\"` is omitted, or set to `null` or an empty array. Defaults to `false`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\":108,\n \"message\":\"User '' lacks permission for datasets: ''\"\n}\n\n```\n\n---\n\n### Update dataset\n\n**PUT** `/api/v1/datasets/{dataset_id}`\n\nUpdates configurations for a specified dataset.\n\n#### Request\n\n- Method: PUT\n- URL: `/api/v1/datasets/{dataset_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`: `string`\n - `\"avatar\"`: `string`\n - `\"description\"`: `string`\n - `\"language\"`: `string`\n - `\"embedding_model\"`: `string`\n - `\"permission\"`: `string`\n - `\"pagerank\"`: `int`\n - `\"parse_type\"`: `int`\n - `\"parser_id\"`: `string`\n - `\"pipeline_id\"`: `string`\n - `\"parser_config\"`: `object`\n\n##### Request example\n\n```bash\ncurl --request PUT \\\n --url http://{address}/api/v1/datasets/{dataset_id} \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '\n {\n \"name\": \"updated_dataset\"\n }'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to update.\n- `\"name\"`: (*Body parameter*), `string`\n The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"language\"`: (*Body parameter*), `string`\n Optional document/dataset language, for example: `\"English\"` or `\"Chinese\"`. Leading and trailing whitespace are stripped. After trimming, it must contain at least 1 character and at most 32 characters. The limit counts characters, not UTF-8 bytes. No restricted list of language values is enforced. If omitted, the existing dataset language remains unchanged.\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n Refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration).\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser selection mode: `1` for a builtin parser (`parser_id` required) or `2` for an ingestion pipeline (`pipeline_id` required). Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`; must be omitted when `parse_type` is `2`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. RAGFlow 1.0 stores parser settings per ingestion-pipeline component; use component-scoped keys (keys containing `:`) as returned by the API. Legacy flat keys are ignored.\n\n`chunk_method` is not supported by RAGFlow 1.0; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"Can't change tenant_id.\"\n}\n```\n\n---\n\n### List datasets\n\n**GET** `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n\nLists datasets.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id}&include_parsing_status={include_parsing_status}`\n- Headers:\n - `'Authorization: Bearer '`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&sort={sort}&name={dataset_name}&id={dataset_id} \\\n --header 'Authorization: Bearer '\n```\n\n```bash\n# List datasets with parsing status\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets?include_parsing_status=true' \\\n --header 'Authorization: Bearer '\n```\n\n##### Query parameters\n\n- `page`\n Specifies the page on which the datasets will be displayed. Defaults to `1`.\n- `page_size`\n The number of datasets on each page. Defaults to `30`.\n- `orderby`\n The field by which datasets should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`\n Indicates whether the retrieved datasets should be sorted in descending order. Defaults to `true`.\n- `sort`\n Orders by several fields at once, written as `column:direction` terms separated by commas, such as `name:asc,create_time:desc`. A term with no direction is ascending. `sort` takes precedence over `orderby` and `desc`, which keep working on their own. A term naming a field this list does not sort by is skipped. When `sort` yields no terms at all, `orderby` and `desc` decide the order. When it yields terms whose fields are all skipped, the result is ordered by `create_time` in the direction of the first term, and the older pair is not read.\n- `name`\n The name of the dataset to retrieve.\n- `id`\n The ID of the dataset to retrieve.\n- `include_parsing_status`\n Whether to include document parsing status counts in the response. Defaults to `false`. When set to `true`, each dataset object in the response will include the following additional fields:\n - `unstart_count`: Number of documents not yet started parsing.\n - `running_count`: Number of documents currently being parsed.\n - `cancel_count`: Number of documents whose parsing was canceled.\n - `done_count`: Number of documents that have been successfully parsed.\n - `fail_count`: Number of documents whose parsing failed.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": \"\",\n \"chunk_count\": 59,\n \"create_date\": \"Sat, 14 Sep 2024 01:12:37 GMT\",\n \"create_time\": 1726276357324,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"description\": null,\n \"document_count\": 1,\n \"embedding_model\": \"BAAI/bge-large-zh-v1.5\",\n \"id\": \"6e211ee0723611efa10a0242ac120007\",\n \"language\": \"English\",\n \"name\": \"mysql\",\n \"parser_id\": \"general\",\n \"parser_config\": {\n \"GeneralChunker:SixApplesFall\": {\n \"chunk_token_size\": 8192,\n \"delimiters\": [\"\\n\"]\n }\n },\n \"permission\": \"me\",\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"69736c5e723611efb51b0242ac120007\",\n \"token_num\": 12744,\n \"update_date\": \"Thu, 10 Oct 2024 04:07:23 GMT\",\n \"update_time\": 1728533243536,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nSuccess (with `include_parsing_status=true`):\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"avatar\": null,\n \"cancel_count\": 0,\n \"chunk_count\": 30,\n \"parser_id\": \"qa\",\n \"create_date\": \"2026-03-09T18:57:13\",\n \"create_time\": 1773053833094,\n \"created_by\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"description\": null,\n \"document_count\": 1,\n \"done_count\": 1,\n \"embedding_model\": \"text-embedding-v2@Tongyi-Qianwen\",\n \"fail_count\": 0,\n \"id\": \"ba6586c21ba611f1a3dc476f0709e75e\",\n \"language\": \"English\",\n \"name\": \"Test Dataset\",\n \"parser_config\": {\n \"Extractor:AutoExtractDefault\": {\n \"llm_id\": \"deepseek-chat@DeepSeek\"\n }\n },\n \"permission\": \"me\",\n \"running_count\": 0,\n \"similarity_threshold\": 0.2,\n \"status\": \"1\",\n \"tenant_id\": \"928f92a210b911f1ac4cc39e0b8fa3ad\",\n \"token_num\": 1746,\n \"unstart_count\": 0,\n \"update_date\": \"2026-03-09T18:59:32\",\n \"update_time\": 1773053972723,\n \"keywords_similarity_weight\": 0.7\n }\n ],\n \"total_datasets\": 1\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset doesn't exist\"\n}\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n**POST** `/api/v1/datasets/{dataset_id}/documents`\n\nUploads documents to a specified dataset.\n\nThis endpoint supports three creation modes via the optional `type` query parameter:\n\n- `type=local` or omitted: Upload one or more local files using `multipart/form-data`.\n- `type=web`: Crawl a web page and save it as a document.\n- `type=empty`: Create an empty virtual document by name.\n\n#### Request\n\n- Method: POST\n- URL: `/api/v1/datasets/{dataset_id}/documents`\n- Query:\n - `type`: Optional. One of `local`, `web`, or `empty`. Defaults to `local`.\n- Headers:\n - `'Content-Type: multipart/form-data'` for `type=local` and `type=web`\n - `'Content-Type: application/json'` for `type=empty`\n - `'Authorization: Bearer '`\n- Body:\n - For `type=local`: form field `'file=@{FILE_PATH}'`\n - For `type=web`: form fields `'name'` and `'url'`\n - For `type=empty`: JSON body with `'name'`\n\n##### Request example\n\n```bash\ncurl --request POST \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'file=@./test1.txt' \\\n --form 'file=@./test2.pdf'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=web' \\\n --header 'Content-Type: multipart/form-data' \\\n --header 'Authorization: Bearer ' \\\n --form 'name=example-page' \\\n --form 'url=https://example.com'\n```\n\n```bash\ncurl --request POST \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?type=empty' \\\n --header 'Content-Type: application/json' \\\n --header 'Authorization: Bearer ' \\\n --data '{\"name\":\"blank.txt\"}'\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the dataset to which the documents will be uploaded.\n- `type`: (*Query parameter*)\n Controls how the document is created:\n - `local`: Upload files.\n - `web`: Crawl a URL into a document.\n - `empty`: Create an empty document without file upload.\n- `'file'`: (*Body parameter*)\n A document to upload. Required when `type=local`.\n- `'name'`: (*Body parameter*)\n The document name. Required when `type=web` or `type=empty`.\n- `'url'`: (*Body parameter*)\n The source URL to crawl. Required when `type=web`.\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": [\n {\n \"chunk_method\": \"naive\",\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"dataset_id\": \"527fa74891e811ef9c650242ac120006\",\n \"id\": \"b330ec2e91ec11efbc510242ac120004\",\n \"location\": \"1.txt\",\n \"name\": \"1.txt\",\n \"parser_config\": {\n \"chunk_token_num\": 128,\n \"delimiter\": \"\\\\n\",\n \"html4excel\": false,\n \"layout_recognize\": \"DeepDOC\"\n },\n \"run\": \"UNSTART\",\n \"size\": 17966,\n \"thumbnail\": \"\",\n \"type\": \"doc\"\n }\n ]\n}\n```\n\nFailure:\n\n```json\n{\n \"code\": 101,\n \"message\": \"No file part!\"\n}\n```\n\n---\n\n### Update document\n\n**PATCH** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nUpdates configurations for a specified document.\n\nSaving parser settings preserves existing chunks, chunk images, counters, and parsing progress. The saved settings take effect on the next parse; this update does not start parsing or delete existing results. Use the document parse action to clear existing chunks and re-parse with the saved settings.\n\n#### Request\n\n- Method: PATCH\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n- Body:\n - `\"name\"`:`string`\n - `\"meta_fields\"`:`object`\n - `\"chunk_method\"`:`string`\n - `\"parser_config\"`:`object`\n\n##### Request example\n\n```bash\ncurl --request PATCH \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --header 'Content-Type: application/json' \\\n --data '\n {\n \"name\": \"manual.txt\",\n \"chunk_method\": \"manual\",\n \"parser_config\": {\"chunk_token_num\": 128}\n }'\n\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The ID of the associated dataset.\n- `document_id`: (*Path parameter*)\n The ID of the document to update.\n- `\"name\"`: (*Body parameter*), `string`\n- `\"meta_fields\"`: (*Body parameter*), `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: (*Body parameter*), `string`\n The parsing method to apply to the document:\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: (*Body parameter*), `object`\n The configuration settings for the dataset parser. The attributes in this JSON object vary with the selected `\"chunk_method\"`:\n - If `\"chunk_method\"` is `\"naive\"`, the `\"parser_config\"` object contains the following attributes:\n - `\"chunk_token_num\"`: Defaults to `256`.\n - `\"layout_recognize\"`: Defaults to `\"DeepDOC\"`.\n - `\"html4excel\"`: Indicates whether to convert Excel documents into HTML format. Defaults to `false`.\n - `\"delimiter\"`: Defaults to `\"\\n\"`.\n - `\"task_page_size\"`: Defaults to `12`. For PDF only.\n- `\"enabled\"`: (*Body parameter*), `integer`\n Whether the document should be **available** in the knowledge base.\n - `1` → （available）\n - `0` → （unavailable）\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"id\": \"cd38dd72d4a611f0af9c71de94a988ef\",\n \"name\": \"large.md\",\n \"type\": \"doc\",\n \"suffix\": \"md\",\n \"size\": 2306906,\n \"location\": \"large.md\",\n \"source_type\": \"local\",\n \"status\": \"1\",\n \"run\": \"DONE\",\n \"dataset_id\": \"5f546a1ad4a611f0af9c71de94a988ef\",\n\n \"chunk_method\": \"naive\",\n \"chunk_count\": 2,\n \"token_count\": 8126,\n\n \"created_by\": \"eab7f446cb5a11f0ab334fbc3aa38f35\",\n \"create_date\": \"Tue, 09 Dec 2025 10:28:52 GMT\",\n \"create_time\": 1765247332122,\n \"update_date\": \"Wed, 17 Dec 2025 10:51:16 GMT\",\n \"update_time\": 1765939876819,\n\n \"process_begin_at\": \"Wed, 17 Dec 2025 10:33:55 GMT\",\n \"process_duration\": 14.8615,\n \"progress\": 1.0,\n\n \"progress_msg\": [\n \"10:33:58 Task has been received.\",\n \"10:33:59 Page(1~100000001): Start to parse.\",\n \"10:33:59 Page(1~100000001): Finish parsing.\",\n \"10:34:07 Page(1~100000001): Generate 2 chunks\",\n \"10:34:09 Page(1~100000001): Embedding chunks (2.13s)\",\n \"10:34:09 Page(1~100000001): Indexing done (0.31s).\",\n \"10:34:09 Page(1~100000001): Task done (11.68s)\"\n ],\n\n \"parser_config\": {\n \"chunk_token_num\": 512,\n \"delimiter\": \"\\n\",\n \"auto_keywords\": 0,\n \"auto_questions\": 0,\n \"topn_tags\": 3,\n \"layout_recognize\": \"DeepDOC\",\n \"html4excel\": false,\n \"image_context_size\": 0,\n \"table_context_size\": 0\n },\n\n \"meta_fields\": {},\n \"pipeline_id\": \"\",\n \"thumbnail\": \"\"\n }\n}\n\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"The dataset does not have the document.\"\n}\n```\n\n---\n\n### Download document\n\n**GET** `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n\nDownloads a document from a specified dataset.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents/{document_id}`\n- Headers:\n - `'Authorization: Bearer '`\n- Output:\n - `'{PATH_TO_THE_FILE}'`\n\n##### Request example\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents/{document_id} \\\n --header 'Authorization: Bearer ' \\\n --output ./ragflow.txt\n```\n\n##### Path parameters\n\n- `dataset_id`\n The associated dataset ID.\n- `document_id`\n The ID of the document to download.\n\n#### Response\n\nSuccess:\n\n```json\nThis is a test to verify the file download feature.\n```\n\nFailure:\n\n```json\n{\n \"code\": 102,\n \"message\": \"You do not own the dataset 7898da028a0511efbf750242ac1220005.\"\n}\n```\n\n---\n\n### List documents\n\n**GET** `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n\nLists documents in a specified dataset.\n\nTo retrieve a specific document's settings and metadata, pass its document ID in the `id` query parameter: `/api/v1/datasets/{dataset_id}/documents?id={document_id}`.\n\n#### Request\n\n- Method: GET\n- URL: `/api/v1/datasets/{dataset_id}/documents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&keywords={keywords}&id={document_id}&name={document_name}&create_time_from={timestamp}&create_time_to={timestamp}&suffix={file_suffix}&run={run_status}&metadata_condition={json}`\n- Headers:\n - `'Content-Type: application/json'`\n - `'Authorization: Bearer '`\n\n##### Request examples\n\n**A basic request with pagination:**\n\n```bash\ncurl --request GET \\\n --url http://{address}/api/v1/datasets/{dataset_id}/documents?page=1&page_size=10 \\\n --header 'Authorization: Bearer '\n```\n\n##### Request parameters\n\n- `dataset_id`: (*Path parameter*)\n The associated dataset ID.\n- `keywords`: (*Query parameter*), `string`\n The keywords used to match document titles.\n- `page`: (*Query parameter*), `integer`\n Specifies the page on which the documents will be displayed. Defaults to `1`.\n- `page_size`: (*Query parameter*), `integer`\n The maximum number of documents on each page. Defaults to `30`.\n- `orderby`: (*Query parameter*), `string`\n The field by which documents should be sorted. Available options:\n - `create_time` (default)\n - `update_time`\n- `desc`: (*Query parameter*), `boolean`\n Indicates whether the retrieved documents should be sorted in descending order. Defaults to `true`.\n- `id`: (*Query parameter*), `string`\n The ID of the document to retrieve.\n- `create_time_from`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created after this time. 0 means no filter. Defaults to `0`.\n- `create_time_to`: (*Query parameter*), `integer`\n Unix timestamp for filtering documents created before this time. 0 means no filter. Defaults to `0`.\n- `suffix`: (*Query parameter*), `array[string]`\n Filter by file suffix. Supports multiple values, e.g., `pdf`, `txt`, and `docx`. Defaults to all suffixes.\n- `run`: (*Query parameter*), `array[string]`\n Filter by document processing status. Supports numeric, text, and mixed formats:\n - Numeric format: `[\"0\", \"1\", \"2\", \"3\", \"4\", \"5\"]`\n - Text format: `[UNSTART, RUNNING, CANCEL, DONE, FAIL, SCHEDULE]`\n - Mixed format: `[UNSTART, 1, DONE]` (mixing numeric and text formats)\n - Status mapping:\n - `0` / `UNSTART`: Document not yet processed\n - `1` / `RUNNING`: Document is currently being processed\n - `2` / `CANCEL`: Document processing was canceled\n - `3` / `DONE`: Document processing completed successfully\n - `4` / `FAIL`: Document processing failed\n - `5` / `SCHEDULE`: Document is scheduled and waiting to be processed\n Defaults to all statuses.\n- `metadata_condition`: (*Query parameter*), `object` (JSON in query)\n Optional metadata filter applied to documents when `document_ids` is not provided. Uses the same structure as retrieval:\n - `logic`: `\"and\"` (default) or `\"or\"`\n - `conditions`: array of `{ \"name\": string, \"comparison_operator\": string, \"value\": string }`\n - `comparison_operator` supports: `is`, `not is`, `contains`, `not contains`, `in`, `not in`, `start with`, `end with`, `>`, `<`, `≥`, `≤`, `empty`, `not empty`\n\n##### Usage examples\n\n**A request with multiple filtering parameters**\n\n```bash\ncurl --request GET \\\n --url 'http://{address}/api/v1/datasets/{dataset_id}/documents?suffix=pdf&run=DONE&page=1&page_size=10' \\\n --header 'Authorization: Bearer '\n```\n\n**Filter by metadata (query JSON):**\n\n```bash\ncurl -G \\\n --url \"http://{address}/api/v1/datasets/{dataset_id}/documents\" \\\n --header 'Authorization: Bearer ' \\\n --data-urlencode 'metadata_condition={\"logic\":\"and\",\"conditions\":[{\"name\":\"tags\",\"comparison_operator\":\"is\",\"value\":\"bar\"},{\"name\":\"author\",\"comparison_operator\":\"is\",\"value\":\"alice\"}]}'\n```\n\n#### Response\n\nSuccess:\n\n```json\n{\n \"code\": 0,\n \"data\": {\n \"docs\": [\n {\n \"chunk_count\": 0,\n \"create_date\": \"Mon, 14 Oct 2024 09:11:01 GMT\",\n \"create_time\": 1728897061948,\n \"created_by\": \"69736c5e723611efb51b0242ac120007\",\n \"id\": \"3bcfbf8a8a0c11ef8aba0242ac120006\",\n \"knowledgebase_id\": \"7898da028a0511efbf750242ac120005\",\n \"location\": \"Test_2.txt\",\n \"name\": \"Test_2.txt\",\n \"parser_config\": {\n \"chunk_token_count\": 128,\n \"delimiter\": \"\\n\",\n \"layout_recognize\": \"DeepDOC\",\n \"task_page_size\": 12\n },\n\n\n---\nsidebar_position: 5\nslug: /python_api_reference\nsidebar_custom_props: {\n categoryIcon: SiPython\n}\n---\n# Python API\n\nA complete reference for RAGFlow's Python APIs. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).\n\n:::tip NOTE\nRun the following command to download the Python SDK:\n\n```bash\npip install ragflow-sdk\n```\n\n:::\n\n---\n\n## ERROR CODES\n\nRAGFlow responses may contain both an HTTP status code and a business code in the JSON response body. These codes should be checked separately.\n\n### HTTP status codes\n\n| Code | Meaning |\n|------|---------|\n| 200 | The HTTP request was processed successfully. Check the response body `code` for the business result. |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Internal server error |\n\n### Response body codes\n\n| Code | Meaning |\n|------|---------|\n| 0 | Success |\n| 10 | Not effective |\n| 100 | Exception error |\n| 101 | Invalid request argument |\n| 102 | Invalid or missing data |\n| 103 | Operation error |\n| 105 | Connection error |\n| 106 | Operation still running |\n| 108 | Permission error |\n| 109 | Authentication error |\n| 400 | Bad request |\n| 401 | Unauthorized |\n| 403 | Forbidden |\n| 404 | Not found |\n| 409 | Conflict |\n| 500 | Server error |\n\n---\n\n## OpenAI-Compatible API\n\n---\n\n### Create chat completion\n\nCreates a model response for the given historical chat conversation via OpenAI's API.\n\n#### Parameters\n\n##### chat_id: `string`, *Required*\n\nExisting chat assistant ID. This value is part of the request path: `/api/v1/openai//chat/completions`.\n\n##### model: `string`, *Required*\n\nThe model used to generate the response. You may also use the legacy placeholder value `\"model\"` to keep using the chat assistant's configured model.\n\n##### messages: `list[object]`, *Required*\n\nA list of historical chat messages used to generate the response. This must contain at least one message with the `user` role.\n\n##### stream: `boolean`\n\nWhether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.\n\n#### Returns\n\n- Success: Response [message](https://platform.openai.com/docs/api-reference/chat/create) like OpenAI\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom openai import OpenAI\nimport json\n\nmodel = \"glm-4-flash@ZHIPU-AI\"\nclient = OpenAI(api_key=\"ragflow-api-key\", base_url=\"http://ragflow_address/api/v1/openai//chat\")\n\nstream = True\nreference = True\n\nrequest_kwargs = dict(\n model=model,\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Who are you?\"},\n {\"role\": \"assistant\", \"content\": \"I am an AI assistant named...\"},\n {\"role\": \"user\", \"content\": \"Can you tell me how to install neovim\"},\n ],\n extra_body={\n \"reference\": reference,\n \"reference_metadata\": {\n \"include\": True,\n \"fields\": [\"author\", \"year\", \"source\"],\n },\n },\n)\n\nif stream:\n completion = client.chat.completions.create(stream=True, **request_kwargs)\n for chunk in completion:\n print(chunk)\nelse:\n resp = client.chat.completions.with_raw_response.create(\n stream=False, **request_kwargs\n )\n print(\"status:\", resp.http_response.status_code)\n raw_text = resp.http_response.text\n print(\"raw:\", raw_text)\n\n data = json.loads(raw_text)\n print(\"assistant:\", data[\"choices\"][0][\"message\"].get(\"content\"))\n print(\"reference:\", data[\"choices\"][0][\"message\"].get(\"reference\"))\n```\n\nWhen `extra_body.reference` is `true`, the streamed final chunk may include `choices[0].delta.reference`, and the non-stream response may include `choices[0].message.reference`.\n\nWhen `extra_body.reference_metadata.include` is `true`, each reference chunk may include a `document_metadata` object in both streaming and non-streaming responses.\n\n## DATASET MANAGEMENT\n\n---\n\n### Create dataset\n\n```python\nRAGFlow.create_dataset(\n name: str,\n parse_type: int = 1,\n parser_id: str | None = None,\n pipeline_id: str | None = None,\n embedding_model: Optional[str] = None,\n permission: str = \"me\",\n parser_config: Optional[DataSet.ParserConfig] = None,\n language: Optional[str] = None,\n) -> DataSet\n```\n\nCreates a dataset.\n\n`avatar`, `description`, `auto_metadata_config`, and `chunk_method` are not accepted by the RAGFlow 1.0 create endpoint. Set avatar and description through `DataSet.update()`, and metadata configuration through `DataSet.update_auto_metadata()`.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe unique name of the dataset to create. It must adhere to the following requirements:\n\n- Maximum 128 characters.\n- Case-insensitive.\n\n##### parse_type: `int`\n\nSelects the parser mode. Defaults to `1`.\n\n- `1`: BuiltIn. Use the builtin parser selected by `parser_id`.\n- `2`: Pipeline. Use the ingestion pipeline selected by `pipeline_id`.\n\n##### parser_id: `string | None`\n\nThe builtin parser ID. Defaults to `\"general\"` when `parse_type` is `1`.\n\nAvailable values include `\"general\"`, `\"qa\"`, `\"manual\"`, `\"table\"`, `\"paper\"`, `\"book\"`, `\"laws\"`, `\"presentation\"`, `\"picture\"`, `\"one\"`, `\"audio\"`, and `\"email\"`.\n\nMust be omitted when `parse_type` is `2`.\n\n##### pipeline_id: `string | None`\n\nThe ingestion pipeline ID. Required when `parse_type` is `2`; must be omitted when `parse_type` is `1`.\n\n##### embedding_model: `string | None`\n\nThe embedding model to use for the dataset. Defaults to `None`.\n\n##### permission\n\nSpecifies who can access the dataset to create. Available options:\n\n- `\"me\"`: (Default) Only you can manage the dataset.\n- `\"team\"`: All team members can manage the dataset.\n\n##### language: `string | None`\n\nOptional document/dataset language, for example `\"English\"` or `\"Chinese\"`. Defaults to `None`.\n\n##### parser_config: `DataSet.ParserConfig | None`\n\nThe parser configuration of the dataset. Defaults to `None`.\n\nRAGFlow 1.0 stores parser settings per ingestion-pipeline component. Use component-scoped keys (keys containing `:`) as returned by the API; legacy flat keys are ignored.\n\n#### Returns\n\n- Success: A `dataset` object. Its `parser_id`, `pipeline_id`, and derived `parse_type` reflect the selected parser mode.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"kb_1\")\ndataset = rag_object.create_dataset(name=\"kb_table\", parse_type=1, parser_id=\"table\")\ndataset = rag_object.create_dataset(\n name=\"kb_pipeline\",\n parse_type=2,\n pipeline_id=\"\",\n)\n```\n\n---\n\n### Delete datasets\n\n```python\nRAGFlow.delete_datasets(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes datasets by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the datasets to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no datasets are deleted.\n- If an array of IDs is provided, only the datasets matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all datasets owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object.delete_datasets(ids=[\"d94a8dc02c9711f0930f7fbc369eab6d\",\"e94a8dc02c9711f0930f7fbc369eab6e\"])\nrag_object.delete_datasets(delete_all=True)\n```\n\n---\n\n### List datasets\n\n```python\nRAGFlow.list_datasets(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n) -> list[DataSet]\n```\n\nLists datasets.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the datasets will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of datasets on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which datasets should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved datasets should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the dataset to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the dataset to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the datasets to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n#### Returns\n\n- Success: A list of `DataSet` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n##### List all datasets\n\n```python\nfor dataset in rag_object.list_datasets():\n print(dataset)\n```\n\n##### Retrieve a dataset by ID\n\n```python\ndataset = rag_object.list_datasets(id = \"id_1\")\nprint(dataset[0])\n```\n\n##### Retrieve datasets by IDs\n\n```python\ndatasets = rag_object.list_datasets(ids=[\"id_1\", \"id_2\"])\nfor dataset in datasets:\n print(dataset)\n```\n\n---\n\n### Update dataset\n\n```python\nDataSet.update(update_message: dict)\n```\n\nUpdates configurations for the current dataset.\n\n#### Parameters\n\n##### update_message: `dict[str, str|int]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"name\"`: `string` The revised name of the dataset.\n - Basic Multilingual Plane (BMP) only\n - Maximum 128 characters\n - Case-insensitive\n- `\"avatar\"`: (*Body parameter*), `string`\n The updated base64 encoding of the avatar.\n - Maximum 65535 characters\n- `\"description\"`: (*Body parameter*), `string`\n The updated dataset description.\n - Maximum 65535 characters\n- `\"language\"`: (*Body parameter*), `string`\n The updated document/dataset language, for example `\"English\"` or `\"Chinese\"`.\n- `\"embedding_model\"`: (*Body parameter*), `string`\n The updated embedding model name.\n - Ensure that `\"chunk_count\"` is `0` before updating `\"embedding_model\"`.\n - Maximum 255 characters\n - Must follow `model_name@model_factory` format\n- `\"permission\"`: (*Body parameter*), `string`\n The updated dataset permission. Available options:\n - `\"me\"`: (Default) Only you can manage the dataset.\n - `\"team\"`: All team members can manage the dataset.\n- `\"pagerank\"`: (*Body parameter*), `int`\n refer to [Set page rank](https://ragflow.io/docs/dev/dataset_configuration)\n - Default: `0`\n - Minimum: `0`\n - Maximum: `100`\n- `\"parse_type\"`: (*Body parameter*), `int`\n The parser mode: `1` for a builtin parser or `2` for an ingestion pipeline. Required whenever `parser_id` or `pipeline_id` is provided.\n- `\"parser_id\"`: (*Body parameter*), `string`\n The builtin parser ID. Required when `parse_type` is `1`.\n- `\"pipeline_id\"`: (*Body parameter*), `string`\n The ingestion pipeline ID. Required when `parse_type` is `2`.\n\n`chunk_method` is not supported by the RAGFlow 1.0 SDK; use `parse_type` together with `parser_id` or `pipeline_id`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_name\")\ndataset = dataset[0]\ndataset.update({\"embedding_model\":\"BAAI/bge-zh-v1.5\", \"parse_type\":1, \"parser_id\":\"manual\"})\n```\n\n---\n\n## FILE MANAGEMENT WITHIN DATASET\n\n---\n\n### Upload documents\n\n```python\nDataSet.upload_documents(document_list: list[dict])\n```\n\nUploads documents to the current dataset.\n\n#### Parameters\n\n##### document_list: `list[dict]`, *Required*\n\nA list of dictionaries representing the documents to upload, each containing the following keys:\n\n- `\"display_name\"`: `string`, *Required*\n The file name to display in the dataset.\n- `\"blob\"`: `bytes`, *Required*\n The binary content of the file to upload.\n\n#### Returns\n\n- Success: A list of uploaded `Document` objects.\n- Failure: `Exception`\n\n#### Examples\n\n```python\ndataset = rag_object.create_dataset(name=\"kb_name\")\n\nwith open(\"1.txt\", \"rb\") as file:\n documents = dataset.upload_documents([\n {\n \"display_name\": \"1.txt\",\n \"blob\": file.read(),\n }\n ])\n\nprint(documents[0].id)\n```\n\n---\n\n### Update document\n\n```python\nDocument.update(update_message: dict)\n```\n\nUpdates configurations for the current document.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"display_name\"`: `string` The name of the document to update.\n- `\"meta_fields\"`: `dict[str, Any]` The meta fields of the document.\n- `\"chunk_method\"`: `string` The parsing method to apply to the document.\n - `\"naive\"`: General\n - `\"manual\"`: Manual\n - `\"qa\"`: Q&A\n - `\"table\"`: Table\n - `\"paper\"`: Paper\n - `\"book\"`: Book\n - `\"laws\"`: Laws\n - `\"presentation\"`: Presentation\n - `\"picture\"`: Picture\n - `\"one\"`: One\n - `\"email\"`: Email\n- `\"parser_config\"`: `dict[str, Any]` The parsing configuration for the document. Its attributes vary based on the selected `\"chunk_method\"`:\n - `\"chunk_method\"`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False},\"parent_child\":{\"use_parent_child\":False,\"children_delimiter\":\"\\n\"}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Returns\n\n- Success: The updated `Document` object.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id='id')\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\ndoc.update({\n \"parser_config\": {\n \"chunk_token_num\": 256,\n },\n \"chunk_method\": \"manual\",\n})\n```\n\n---\n\n### Download document\n\n```python\nDocument.download() -> bytes\n```\n\nDownloads the current document.\n\n#### Returns\n\nThe downloaded document in bytes.\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"https://\",\n)\ndataset = rag_object.list_datasets(id=\"id\")[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")[0]\n\nPath(\"~/ragflow.txt\").expanduser().write_bytes(doc.download())\n```\n\n---\n\n### List documents\n\n```python\nDataSet.list_documents(\n id: str | None = None,\n ids: list[str] | None = None,\n name: str | None = None,\n keywords: str | None = None,\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n create_time_from: int = 0,\n create_time_to: int = 0\n)\n```\n\nLists documents in the current dataset.\n\n#### Parameters\n\n##### id: `string | None`\n\nThe ID of the document to retrieve. Defaults to `None`.\n\n##### ids: `list[str] | None`\n\nThe IDs of the documents to retrieve. Defaults to `None`. The `id` and `ids` parameters cannot be used together.\n\n##### name: `string | None`\n\nThe exact name of the document to retrieve. Defaults to `None`.\n\n##### keywords: `string | None`\n\nThe keywords used to match document titles. Defaults to `None`.\n\n##### page: `int`\n\nSpecifies the page on which the documents will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of documents on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which documents should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved documents should be sorted in descending order. Defaults to `True`.\n\n##### create_time_from: `int`\nUnix timestamp for filtering documents created after this time. 0 means no filter. Defaults to 0.\n\n##### create_time_to: `int`\nUnix timestamp for filtering documents created before this time. 0 means no filter. Defaults to 0.\n\n#### Returns\n\n- Success: A list of `Document` objects.\n- Failure: `Exception`.\n\nA `Document` object contains the following attributes:\n\n- `id`: The document ID. Defaults to `\"\"`.\n- `name`: The document name. Defaults to `\"\"`.\n- `thumbnail`: The thumbnail image of the document. Defaults to `None`.\n- `dataset_id`: The dataset ID associated with the document. Defaults to `None`.\n- `chunk_method` The chunking method name. Defaults to `\"naive\"`.\n- `source_type`: The source type of the document. Defaults to `\"local\"`.\n- `type`: Type or category of the document. Defaults to `\"\"`. Reserved for future use.\n- `created_by`: `string` The creator of the document. Defaults to `\"\"`.\n- `size`: `int` The document size in bytes. Defaults to `0`.\n- `token_count`: `int` The number of tokens in the document. Defaults to `0`.\n- `chunk_count`: `int` The number of chunks in the document. Defaults to `0`.\n- `progress`: `float` The current processing progress as a percentage. Defaults to `0.0`.\n- `progress_msg`: `string` A message indicating the current progress status. Defaults to `\"\"`.\n- `process_begin_at`: `datetime` The start time of document processing. Defaults to `None`.\n- `process_duration`: `float` Duration of the processing in seconds. Defaults to `0.0`.\n- `run`: `string` The document's processing status:\n - `\"UNSTART\"` (default)\n - `\"RUNNING\"`\n - `\"CANCEL\"`\n - `\"DONE\"`\n - `\"FAIL\"`\n- `status`: `string` Reserved for future use.\n- `parser_config`: `ParserConfig` Configuration object for the parser. Its attributes vary based on the selected `chunk_method`:\n - `chunk_method`=`\"naive\"`:\n `{\"chunk_token_num\":128,\"delimiter\":\"\\n\",\"html4excel\":False,\"layout_recognize\":\"DeepDOC\",\"raptor\":{\"use_raptor\":False}}`.\n - `chunk_method`=`\"qa\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"manual\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"table\"`:\n `None`\n - `chunk_method`=`\"paper\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"book\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"laws\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"presentation\"`:\n `{\"raptor\": {\"use_raptor\": False}}`\n - `chunk_method`=`\"picture\"`:\n `None`\n - `chunk_method`=`\"one\"`:\n `None`\n - `chunk_method`=`\"email\"`:\n `None`\n\n#### Examples\n\n```python\nfrom pathlib import Path\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(\n api_key=\"\",\n base_url=\"http://:9380\",\n)\ndataset = rag_object.create_dataset(name=\"kb_1\")\n\nfile_path = Path(\"~/ragflow.txt\").expanduser()\ndataset.upload_documents([\n {\n \"display_name\": file_path.name,\n \"blob\": file_path.read_bytes(),\n }\n])\n\nfor doc in dataset.list_documents(\n keywords=\"rag\",\n page=1,\n page_size=12,\n):\n print(doc)\n```\n\n---\n\n### Delete documents\n\n```python\nDataSet.delete_documents(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes documents by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the documents to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no documents are deleted.\n- If an array of IDs is provided, only the documents matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all documents in the current dataset when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"kb_1\")\ndataset = dataset[0]\ndataset.delete_documents(ids=[\"id_1\",\"id_2\"])\ndataset.delete_documents(delete_all=True)\n```\n\n---\n\n### Parse documents\n\n```python\nDataSet.async_parse_documents(document_ids:list[str]) -> None\n```\n\nParses documents in the current dataset.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\n```\n\n---\n\n### Parse documents (with document status)\n\n```python\nDataSet.parse_documents(document_ids: list[str]) -> list[tuple[str, str, int, int]]\n```\n\nStarts parsing documents in the current dataset and synchronously waits for the results.\n\nThis method calls `async_parse_documents()` and blocks while polling until all requested documents reach a terminal state or report complete progress. It then returns the parsing status and statistics for each document. If a keyboard interruption occurs (e.g., `Ctrl+C`), it requests cancellation for the requested documents and continues polling for their final statuses. If a status request fails or a requested document is no longer found, the method raises an exception instead of continuing to poll.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents to parse.\n\n#### Returns\n\nA list of tuples with detailed parsing results:\n\n```python\n[\n (document_id: str, status: str, chunk_count: int, token_count: int),\n ...\n]\n```\n- `status`: The final parsing state (e.g., `DONE`, `FAIL`, `CANCEL`). If a document has not reached a terminal state but reports `progress >= 1.0`, its status is returned as `DONE`.\n- `chunk_count`: The number of content chunks created from the document.\n- `token_count`: The total number of tokens processed.\n\n---\n\n#### Example\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = dataset.list_documents(keywords=\"test\")\nids = [doc.id for doc in documents]\n\ntry:\n finished = dataset.parse_documents(ids)\n for doc_id, status, chunk_count, token_count in finished:\n print(f\"Document {doc_id} parsing finished with status: {status}, chunks: {chunk_count}, tokens: {token_count}\")\nexcept Exception as e:\n print(f\"Parsing failed: {e}\")\n```\n\n---\n\n### Stop parsing documents\n\n```python\nDataSet.async_cancel_parse_documents(document_ids:list[str])-> None\n```\n\nStops parsing specified documents.\n\n#### Parameters\n\n##### document_ids: `list[str]`, *Required*\n\nThe IDs of the documents for which parsing should be stopped.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.create_dataset(name=\"dataset_name\")\ndocuments = [\n {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',\"rb\").read()},\n {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',\"rb\").read()},\n {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',\"rb\").read()}\n]\ndataset.upload_documents(documents)\ndocuments = dataset.list_documents(keywords=\"test\")\nids = []\nfor document in documents:\n ids.append(document.id)\ndataset.async_parse_documents(ids)\nprint(\"Async bulk parsing initiated.\")\ndataset.async_cancel_parse_documents(ids)\nprint(\"Async bulk parsing cancelled.\")\n```\n\n---\n\n## CHUNK MANAGEMENT WITHIN DATASET\n\n---\n\n### Add chunk\n\n```python\nDocument.add_chunk(content:str, important_keywords:list[str] = [], questions:list[str] = [], image_base64:str = None, *, tag_kwd:list[str] = []) -> Chunk\n```\n\nAdds a chunk to the current document.\n\n#### Parameters\n\n##### content: `string`, *Required*\n\nThe text content of the chunk.\n\n##### important_keywords: `list[str]`\n\nThe key terms or phrases to tag with the chunk.\n\n##### questions: `list[str]`\n\nOptional questions to use when embedding the chunk.\n\n##### image_base64: `string`\n\nA base64-encoded image to associate with the chunk. If the chunk already has an image, the new image will be vertically concatenated below the existing one.\n\n##### tag_kwd: `list[str]`\n\nTag keywords to associate with the chunk.\n\n#### Returns\n\n- Success: A `Chunk` object.\n- Failure: `Exception`.\n\nA `Chunk` object contains the following attributes:\n\n- `id`: `string`: The chunk ID.\n- `content`: `string` The text content of the chunk.\n- `important_keywords`: `list[str]` A list of key terms or phrases tagged with the chunk.\n- `tag_kwd`: `list[str]` A list of tag keywords associated with the chunk.\n- `questions`: `list[str]` A list of questions associated with the chunk.\n- `image_id`: `string` The image ID associated with the chunk (empty string if no image).\n- `create_time`: `string` The time when the chunk was created (added to the document).\n- `create_timestamp`: `float` The timestamp representing the creation time of the chunk, expressed in seconds since January 1, 1970.\n- `dataset_id`: `string` The ID of the associated dataset.\n- `document_name`: `string` The name of the associated document.\n- `document_id`: `string` The ID of the associated document.\n- `available`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(id=\"123\")\ndataset = datasets[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\n```\n\nAdding a chunk with an image:\n\n```python\nimport base64\n\nwith open(\"image.jpg\", \"rb\") as f:\n img_b64 = base64.b64encode(f.read()).decode()\nchunk = doc.add_chunk(content=\"description of image\", image_base64=img_b64)\n```\n\n---\n\n### List chunks\n\n```python\nDocument.list_chunks(keywords: str = None, page: int = 1, page_size: int = 30, id : str = None) -> list[Chunk]\n```\n\nLists chunks in the current document.\n\n#### Parameters\n\n##### keywords: `string`\n\nThe keywords used to match chunk content. Defaults to `None`\n\n##### page: `int`\n\nSpecifies the page on which the chunks will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks on each page. Defaults to `30`.\n\n##### id: `string`\n\nThe ID of the chunk to retrieve. Default: `None`\n\n#### Returns\n\n- Success: A list of `Chunk` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(\"123\")\ndataset = dataset[0]\ndocs = dataset.list_documents(keywords=\"test\", page=1, page_size=12)\nfor chunk in docs[0].list_chunks(keywords=\"rag\", page=0, page_size=12):\n print(chunk)\n```\n\n---\n\n### Delete chunks\n\n```python\nDocument.delete_chunks(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chunks by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chunks to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chunks are deleted.\n- If an array of IDs is provided, only the chunks matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chunks in the current document when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\ndoc.delete_chunks([\"id_1\",\"id_2\"])\ndoc.delete_chunks(delete_all=True)\n```\n\n---\n\n### Update chunk\n\n```python\nChunk.update(update_message: dict)\n```\n\nUpdates content or configurations for the current chunk.\n\n#### Parameters\n\n##### update_message: `dict[str, str|list[str]|bool]` *Required*\n\nA dictionary representing the attributes to update, with the following keys:\n\n- `\"content\"`: `string` The text content of the chunk.\n- `\"important_keywords\"`: `list[str]` A list of key terms or phrases to tag with the chunk.\n- `\"questions\"`: `list[str]` A list of questions associated with the chunk.\n- `\"tag_kwd\"`: `list[str]` A list of tag keywords to associate with the chunk.\n- `\"positions\"`: `list` Updated source positions for the chunk.\n- `\"available\"`: `bool` The chunk's availability status in the dataset. Value options:\n - `False`: Unavailable\n - `True`: Available (default)\n- `\"image_base64\"`: `string` Base64-encoded image content to associate with the chunk.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(id=\"123\")\ndataset = dataset[0]\ndoc = dataset.list_documents(id=\"wdfxb5t547d\")\ndoc = doc[0]\nchunk = doc.add_chunk(content=\"xxxxxxx\")\nchunk.update({\"content\":\"sdfx...\"})\n```\n\n---\n\n### Retrieve chunks\n\n```python\nRAGFlow.retrieve(\n dataset_ids,\n document_ids=None,\n question='',\n page=1,\n page_size=30,\n similarity_threshold=0.2,\n vector_similarity_weight=0.3,\n top_k=1024,\n rerank_id: str | None = None,\n keyword: bool = False,\n cross_languages: list[str] | None = None,\n metadata_condition: dict | None = None)\n```\n\nRetrieves chunks from specified datasets.\n\n#### Parameters\n\n##### question: `string`\n\nThe user query or query keywords. Defaults to `\"\"`. When an empty string is provided, the API returns an empty retrieval result.\n\n##### dataset_ids: `list[str]`, *Required*\n\nThe IDs of the datasets to search. At least one dataset ID must be provided.\n\n##### document_ids: `list[str]`\n\nThe IDs of the documents to search. Defaults to `None`. You must ensure all selected documents use the same embedding model. Otherwise, an error will occur.\n\n##### page: `int`\n\nThe page number of the chunk retrieval results. Defaults to `1`.\n\n##### page_size: `int`\n\nThe maximum number of chunks returned on each page. Defaults to `30`.\n\n##### similarity_threshold: `float`\n\nThe minimum similarity score. Defaults to `0.2`.\n\n##### vector_similarity_weight: `float`\n\nThe weight of vector cosine similarity. Defaults to `0.3`. If x represents the vector cosine similarity, then (1 - x) is the term similarity weight.\n\n##### top_k: `int`\n\nThe number of chunks engaged in vector cosine computation. Defaults to `1024`.\n\n##### rerank_id: `string`\n\nThe ID of the rerank model. Defaults to `None`.\n\n##### keyword: `bool`\n\nIndicates whether to enable keyword-based matching:\n\n- `True`: Enable keyword-based matching.\n- `False`: Disable keyword-based matching (default).\n\n##### cross_languages: `list[string]`\n\nThe languages that should be translated into, in order to achieve keywords retrievals in different languages.\n\n##### metadata_condition: `dict`\n\nfilter condition for `meta_fields`.\n\n#### Returns\n\n- Success: A list of `Chunk` objects representing the document chunks.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndataset = rag_object.list_datasets(name=\"ragflow\")\ndataset = dataset[0]\nname = 'ragflow_test.txt'\npath = './test_data/ragflow_test.txt'\ndocuments =[{\"display_name\":\"test_retrieve_chunks.txt\",\"blob\":open(path, \"rb\").read()}]\ndocs = dataset.upload_documents(documents)\ndoc = docs[0]\ndoc.add_chunk(content=\"This is a chunk addition test\")\nfor c in rag_object.retrieve(dataset_ids=[dataset.id],document_ids=[doc.id]):\n print(c)\n```\n\n---\n\n## CHAT ASSISTANT MANAGEMENT\n\n---\n\n### Create chat assistant\n\n```python\nRAGFlow.create_chat(\n name: str,\n icon: str = \"\",\n dataset_ids: list[str] | None = None,\n llm_id: str | None = None,\n llm_setting: dict | None = None,\n prompt_config: dict | None = None,\n **kwargs\n) -> Chat\n```\n\nCreates a chat assistant.\n\n#### Parameters\n\n##### name: `string`, *Required*\n\nThe name of the chat assistant.\n\n##### icon: `string`\n\nBase64 encoding of the avatar. Defaults to `\"\"`.\n\n##### dataset_ids: `list[str]`\n\nThe IDs of the associated datasets. Defaults to `[]`. When omitted or empty, the SDK creates an empty chat assistant, and you can attach datasets later.\n\n##### llm_id: `str | None`\n\nThe LLM model name/ID to use. If `None`, the user’s default chat model is used. Defaults to `None`.\n\n##### llm_setting: `dict | None`\n\nConfiguration for LLM generation parameters. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"temperature\"`: `float` Controls the randomness of the model's output. Higher values increase creativity, while lower values make responses more deterministic. Defaults to `0.1`.\n- `\"top_p\"`: `float` Sets the nucleus sampling threshold. The model considers only the results of the tokens with `top_p` probability mass. Defaults to `0.3`.\n- `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have appeared in the text so far, increasing the likelihood of the model talking about new topics. Defaults to `0.4`.\n- `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text, decreasing the likelihood of repeating the same lines. Defaults to `0.7`.\n- `\"max_token\"`: `int` The maximum number of tokens to generate in the response. Defaults to `512`.\n\n##### prompt_config: `dict | None`\n\nInstructions and behavioral settings for the LLM. Defaults to `None` (server-side defaults apply). Supported keys:\n\n- `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n- `\"empty_response\"`: `string` The specific message returned when no relevant information is retrieved. If left blank, the LLM will generate its own response. Defaults to `None`.\n- `\"prologue\"`: `string` The initial greeting displayed to the user. Defaults to `\"Hi! I’m your assistant. What can I do for you?\"`.\n- `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references in its responses. Defaults to `True`.\n- `\"parameters\"`: `list[dict]` A list of variables utilized within the system prompt. Each entry must include a `\"key\"` (`string`) and an `\"optional\"` (`boolean`) status. The `knowledge` key is reserved for retrieved context chunks. Default: `[{\"key\": \"knowledge\", \"optional\": true}]`.\n\n#### Returns\n\n- Success: A `Chat` object representing the chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_ids = []\nfor dataset in datasets:\n dataset_ids.append(dataset.id)\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=dataset_ids)\n```\n\n---\n\n### Update chat assistant\n\n```python\nChat.update(update_message: dict)\n```\n\nPerforms a partial update to the configuration settings for the current chat assistant.\n\n`Chat.update()` utilizes the `PATCH /api/v1/chats/{chat_id}` endpoint. Only the specified keys are modified, while all other existing fields are preserved.\n\n#### Parameters\n\n##### update_message: `dict`, *Required*\n\nA dictionary containing the attributes to be updated. Supported keys include:\n\n- `\"name\"`: `string` The updated name of the chat assistant.\n- `\"icon\"`: `string` A Base64-encoded string representing the assistant's avatar.\n- `\"dataset_ids\"`: `list[string]` A list of unique identifiers for the datasets associated with the assistant.\n- `\"llm_id\"`: `string` The unique identifier or name of the LLM to be used.\n- `\"llm_setting\"`: `dict` Configuration for LLM generation parameters:\n - `\"temperature\"`: `float` Controls the randomness of the model's output.\n - `\"top_p\"`: `float` Sets the nucleus sampling threshold.\n - `\"presence_penalty\"`: `float` Penalizes tokens based on whether they have already appeared in the text.\n - `\"frequency_penalty\"`: `float` Penalizes tokens based on their existing frequency in the text.\n - `\"max_token\"`: `int` The maximum number of tokens to generate in the response.\n- `\"prompt_config\"`: `dict` Instructions and behavioral settings for the LLM:\n - `\"system\"`: `string` The core system prompt or instructions defining the assistant's persona.\n - `\"empty_response\"`: `string` The message returned when no relevant information is retrieved. Leave blank to allow the LLM to improvise.\n - `\"prologue\"`: `string` The initial greeting displayed to the user.\n - `\"quote\"`: `boolean` Determines whether the assistant should include citations or source references.\n - `\"parameters\"`: `list[dict]` Variables used within the system prompt (e.g., the reserved `knowledge` key).\n- `\"similarity_threshold\"`: `float` The minimum similarity score required for retrieved context chunks. Defaults to `0.2`.\n- `\"vector_similarity_weight\"`: `float` The weight assigned to vector cosine similarity within the hybrid search score. Defaults to `0.3`.\n- `\"top_n\"`: `int` The number of top-ranked chunks provided to the LLM as context. Defaults to `6`.\n- `\"top_k\"`: `int` The size of the initial candidate pool retrieved for reranking. Defaults to `1024`.\n- `\"rerank_id\"`: `string` The unique identifier for the reranking model. If left empty, standard vector cosine similarity is used for ranking.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\ndatasets = rag_object.list_datasets(name=\"kb_1\")\ndataset_id = datasets[0].id\nassistant = rag_object.create_chat(\"Miss R\", dataset_ids=[dataset_id])\nassistant.update({\"name\": \"Stefan\", \"llm_setting\": {\"temperature\": 0.8}, \"top_n\": 8})\n```\n\n---\n\n### Delete chat assistants\n\n```python\nRAGFlow.delete_chats(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes chat assistants by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the chat assistants to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no chat assistants are deleted.\n- If an array of IDs is provided, only the chat assistants matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all chat assistants owned by the current user when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nrag_object.delete_chats(ids=[\"id_1\",\"id_2\"])\nrag_object.delete_chats(delete_all=True)\n```\n\n---\n\n### List chat assistants\n\n```python\nRAGFlow.list_chats(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str | None = None,\n name: str | None = None,\n keywords: str | None = None,\n owner_ids: str | list[str] | None = None,\n) -> list[Chat]\n```\n\nLists chat assistants.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the chat assistants will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of chat assistants on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe attribute by which the results are sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved chat assistants should be sorted in descending order. Defaults to `True`.\n\n##### id: `string | None`\n\nExact match on chat assistant ID. Defaults to `None`.\n\n##### name: `string | None`\n\nFilters results by the exact name of the chat assistant. Defaults to `None`.\n\n##### keywords: `string | None`\n\nPerforms a case-insensitive fuzzy search against chat assistant names. Defaults to `None`.\n\n##### owner_ids: `string | list[string] | None`\n\nFilters results by one or more owner tenant IDs. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Chat` objects.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nfor assistant in rag_object.list_chats():\n print(assistant)\n```\n\n---\n\n## SESSION MANAGEMENT\n\n---\n\n### Create session with chat assistant\n\n```python\nChat.create_session(name: str = \"New session\") -> Session\n```\n\nCreates a session with the current chat assistant.\n\n#### Parameters\n\n##### name: `string`\n\nThe name of the chat session to create.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `name`: `string` The name of the created session.\n - `message`: `list[Message]` The opening message of the created session. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `chat_id`: `string` The ID of the associated chat assistant.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session()\n```\n\n---\n\n### Update chat assistant's session\n\n```python\nSession.update(update_message: dict)\n```\n\nUpdates the current session of the current chat assistant.\n\n#### Parameters\n\n##### update_message: `dict[str, Any]`, *Required*\n\nA dictionary representing the attributes to update, with only one key:\n\n- `\"name\"`: `string` The revised name of the session.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nsession = assistant.create_session(\"session_name\")\nsession.update({\"name\": \"updated_name\"})\n```\n\n---\n\n### List chat assistant's sessions\n\n```python\nChat.list_sessions(\n page: int = 1,\n page_size: int = 30,\n orderby: str = \"create_time\",\n desc: bool = True,\n id: str = None,\n name: str = None,\n user_id: str = None\n) -> list[Session]\n```\n\nLists sessions associated with the current chat assistant.\n\n#### Parameters\n\n##### page: `int`\n\nSpecifies the page on which the sessions will be displayed. Defaults to `1`.\n\n##### page_size: `int`\n\nThe number of sessions on each page. Defaults to `30`.\n\n##### orderby: `string`\n\nThe field by which sessions should be sorted. Available options:\n\n- `\"create_time\"` (default)\n- `\"update_time\"`\n\n##### desc: `bool`\n\nIndicates whether the retrieved sessions should be sorted in descending order. Defaults to `True`.\n\n##### id: `string`\n\nThe ID of the chat session to retrieve. Defaults to `None`.\n\n##### name: `string`\n\nThe name of the chat session to retrieve. Defaults to `None`.\n\n##### user_id: `str`\n\nThe optional user-defined ID to filter sessions by. Defaults to `None`.\n\n#### Returns\n\n- Success: A list of `Session` objects associated with the current chat assistant.\n- Failure: `Exception`.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nfor session in assistant.list_sessions():\n print(session)\n```\n\n---\n\n### Delete chat assistant's sessions\n\n```python\nChat.delete_sessions(ids: list[str] | None = None, delete_all: bool = False)\n```\n\nDeletes sessions of the current chat assistant by ID.\n\n#### Parameters\n\n##### ids: `list[str]` or `None`\n\nThe IDs of the sessions to delete. Defaults to `None`.\n\n- If omitted, or set to `null` or an empty array, no sessions are deleted.\n- If an array of IDs is provided, only the sessions matching those IDs are deleted.\n\n##### delete_all: `bool`\n\nWhether to delete all sessions of the current chat assistant when `ids` is omitted, or set to `None` or an empty list. Defaults to `False`.\n\n#### Returns\n\n- Success: No value is returned.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nassistant = rag_object.list_chats(name=\"Miss R\")\nassistant = assistant[0]\nassistant.delete_sessions(ids=[\"id_1\",\"id_2\"])\nassistant.delete_sessions(delete_all=True)\n```\n\n---\n\n### Create session with agent\n\n```python\nAgent.create_session(**kwargs) -> Session\n```\n\nCreates a session with the current agent.\n\n#### Parameters\n\n##### **kwargs\n\nThe parameters in `begin` component.\n\nAlso supports:\n\n- `release` (`bool | str`, optional): When set to `True` (or `\"true\"`), creates a session with the published agent app only.\n\n#### Returns\n\n- Success: A `Session` object containing the following attributes:\n - `id`: `string` The auto-generated unique identifier of the created session.\n - `message`: `list[Message]` The messages of the created session assistant. Default: `[{\"role\": \"assistant\", \"content\": \"Hi! I am your assistant, can I help you?\"}]`\n - `agent_id`: `string` The ID of the associated agent.\n- Failure: `Exception`\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nagent_id = \"AGENT_ID\"\nagent = rag_object.get_agent(agent_id)\nsession = agent.create_session()\n# Or create in release mode:\n# session = agent.create_session(release=True)\n```\n\n---\n\n### Converse with agent\n\n```python\nSession.ask(question: str = \"\", stream: bool = False, **kwargs) -> Optional[Message | iter[Message]]\n```\n\nAsks a specified agent through the unified completion endpoint.\n\n:::tip NOTE\nIn streaming mode, not all responses include a reference, as this depends on the system's judgment.\n:::\n\n#### Parameters\n\n##### question: `string`\n\nThe user message sent to the agent. If the **Begin** component takes parameters, `question` can be an empty string.\n\n##### stream: `bool`\n\nIndicates whether to output responses in a streaming way:\n\n- `True`: Enable streaming.\n- `False`: Disable streaming.\n\n##### kwargs: `dict`\n\nAdditional request parameters forwarded to the completion API. Common options:\n\n- `inputs`: Variables defined in the **Begin** component.\n- `session_id`: Continue an existing session instead of creating a new one.\n- `release`: Use the latest published version of the agent.\n- `return_trace`: Include execution trace information in the response.\n- Other custom Begin component parameters supported by the current workflow.\n\n#### Returns\n\n- A `Message` object containing the response to the question if `stream` is set to `False`\n- An iterator containing multiple `message` objects (`iter[Message]`) if `stream` is set to `True`\n\nThe following shows the attributes of a `Message` object:\n\n##### id: `string`\n\nThe auto-generated message ID.\n\n##### content: `string`\n\nThe content of the message. Defaults to `\"Hi! I am your assistant, can I help you?\"`.\n\n##### reference: `list[Chunk]`\n\nA list of `Chunk` objects representing references to the message, each containing the following attributes:\n\n- `id` `string`\n The chunk ID.\n- `content` `string`\n The content of the chunk.\n- `image_id` `string`\n The ID of the snapshot of the chunk. Applicable only when the source of the chunk is an image, PPT, PPTX, or PDF file.\n- `document_id` `string`\n The ID of the referenced document.\n- `document_name` `string`\n The name of the referenced document.\n- `document_metadata` `dict`\n Optional document metadata, returned only when `extra_body.reference_metadata.include` is `true`.\n- `position` `list[str]`\n The location information of the chunk within the referenced document.\n- `dataset_id` `string`\n The ID of the dataset to which the referenced document belongs.\n- `similarity` `float`\n A composite similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity. It is the weighted sum of `vector_similarity` and `term_similarity`.\n- `vector_similarity` `float`\n A vector similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between vector embeddings.\n- `term_similarity` `float`\n A keyword similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between keywords.\n\n#### Examples\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://:9380\")\nAGENT_id = \"AGENT_ID\"\nagent = rag_object.get_agent(AGENT_id)\nsession = agent.create_session()\n\nprint(\"\\n===== Miss R ====\\n\")\nprint(\"Hello. What can I do for you?\")\n\nwhile True:\n question = input(\"\\n===== User ====\\n> \")\n print(\"\\n==== Miss R ====\\n\")\n\n cont = \"\"\n for ans in session.ask(question, stream=True):\n print(ans.content[len(cont):], end='', flush=True)\n cont = ans.content\n```\n\nUse Begin inputs and request trace output:\n\n```python\nfrom ragflow_sdk import RAGFlow, Agent\n\nrag_object = RAGFlow(api_key=\"\", base_url=\"http://\n# Plugins Skill\n\nPath: `tools/chatgpt-on-wechat/plugins/config.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/config.json)\n\n\n# Installation Guide for Firecrawl RAGFlow Integration\n\nThis guide will help you install and configure the Firecrawl integration plugin for RAGFlow.\n\n## Prerequisites\n\n- RAGFlow instance running (version 0.20.5 or later)\n- Python 3.8 or higher\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n## Installation Methods\n\n### Method 1: Manual Installation\n\n1. **Download the plugin**:\n ```bash\n git clone https://github.com/firecrawl/firecrawl.git\n cd firecrawl/ragflow-firecrawl-integration\n ```\n\n2. **Install dependencies**:\n ```bash\n pip install -r plugin/firecrawl/requirements.txt\n ```\n\n3. **Copy plugin to RAGFlow**:\n ```bash\n # Assuming RAGFlow is installed in /opt/ragflow\n cp -r plugin/firecrawl /opt/ragflow/plugin/\n ```\n\n4. **Restart RAGFlow**:\n ```bash\n # Restart RAGFlow services\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n### Method 2: Using pip (if available)\n\n```bash\npip install ragflow-firecrawl-integration\n```\n\n### Method 3: Development Installation\n\n1. **Clone the repository**:\n ```bash\n git clone https://github.com/firecrawl/firecrawl.git\n cd firecrawl/ragflow-firecrawl-integration\n ```\n\n2. **Install in development mode**:\n ```bash\n pip install -e .\n ```\n\n## Configuration\n\n### 1. Get Firecrawl API Key\n\n1. Visit [firecrawl.dev](https://firecrawl.dev)\n2. Sign up for a free account\n3. Navigate to your dashboard\n4. Copy your API key (starts with `fc-`)\n\n### 2. Configure in RAGFlow\n\n1. **Access RAGFlow UI**:\n - Open your browser and go to your RAGFlow instance\n - Log in with your credentials\n\n2. **Add Firecrawl Data Source**:\n - Go to \"Data Sources\" → \"Add New Source\"\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify your setup\n - You should see a success message\n\n## Configuration Options\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## Environment Variables\n\nYou can also configure the plugin using environment variables:\n\n```bash\nexport FIRECRAWL_API_KEY=\"fc-your-api-key-here\"\nexport FIRECRAWL_API_URL=\"https://api.firecrawl.dev\"\nexport FIRECRAWL_MAX_RETRIES=\"3\"\nexport FIRECRAWL_TIMEOUT=\"30\"\nexport FIRECRAWL_RATE_LIMIT_DELAY=\"1.0\"\n```\n\n## Verification\n\n### 1. Check Plugin Installation\n\n```bash\n# Check if the plugin directory exists\nls -la /opt/ragflow/plugin/firecrawl/\n\n# Should show:\n# __init__.py\n# firecrawl_connector.py\n# firecrawl_config.py\n# firecrawl_processor.py\n# firecrawl_ui.py\n# ragflow_integration.py\n# requirements.txt\n```\n\n### 2. Test the Integration\n\n```bash\n# Run the example script\ncd /opt/ragflow/plugin/firecrawl/\npython example_usage.py\n```\n\n### 3. Check RAGFlow Logs\n\n```bash\n# Check RAGFlow server logs\ndocker logs docker-ragflow-cpu-1\n\n# Look for messages like:\n# \"Firecrawl plugin loaded successfully\"\n# \"Firecrawl data source registered\"\n```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Plugin not appearing in RAGFlow**:\n - Check if the plugin directory is in the correct location\n - Restart RAGFlow services\n - Check RAGFlow logs for errors\n\n2. **API Key Invalid**:\n - Ensure your API key starts with `fc-`\n - Verify the key is active in your Firecrawl dashboard\n - Check for typos in the configuration\n\n3. **Connection Timeout**:\n - Increase the timeout value in configuration\n - Check your network connection\n - Verify the API URL is correct\n\n4. **Rate Limiting**:\n - Increase the `rate_limit_delay` value\n - Reduce the number of concurrent requests\n - Check your Firecrawl usage limits\n\n### Debug Mode\n\nEnable debug logging to see detailed information:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.DEBUG)\n```\n\n### Check Dependencies\n\n```bash\n# Verify all dependencies are installed\npip list | grep -E \"(aiohttp|pydantic|requests)\"\n\n# Should show:\n# aiohttp>=3.8.0\n# pydantic>=2.0.0\n# requests>=2.28.0\n```\n\n## Uninstallation\n\nTo remove the plugin:\n\n1. **Remove plugin directory**:\n ```bash\n rm -rf /opt/ragflow/plugin/firecrawl/\n ```\n\n2. **Restart RAGFlow**:\n ```bash\n docker compose -f /opt/ragflow/docker/docker-compose.yml restart\n ```\n\n3. **Remove dependencies** (optional):\n ```bash\n pip uninstall ragflow-firecrawl-integration\n ```\n\n## Support\n\nIf you encounter issues:\n\n1. Check the [troubleshooting section](#troubleshooting)\n2. Review RAGFlow logs for error messages\n3. Verify your Firecrawl API key and configuration\n4. Check the [Firecrawl documentation](https://docs.firecrawl.dev)\n5. Open an issue in the [Firecrawl repository](https://github.com/firecrawl/firecrawl/issues)\n\n## Next Steps\n\nAfter successful installation:\n\n1. Read the [README.md](README.md) for usage examples\n2. Try scraping a simple URL to test the integration\n3. Explore the different scraping options (single URL, crawl, batch)\n4. Configure your RAGFlow workflows to use the scraped content\n\n\n# RAGFlow CLI Installation Scripts\n\nRAGFlow publishes static Go CLI binaries as GitHub Release assets for:\n\n- Linux: `amd64`, `arm64`\n- macOS: `amd64`, `arm64`\n- Windows: `amd64`, `arm64`\n\nThe release workflow builds the binaries with `CGO_ENABLED=0`, uploads `SHA256SUMS`, and uploads `install.sh` and `install.ps1`.\n\n## Linux And macOS\n\nUse the script maintained in the RAGFlow repository:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | sh\n```\n\nInstall a specific version:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | VERSION=v1.0.0 sh\n```\n\nInstall to a user-writable directory:\n\n```sh\ncurl -fsSL https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.sh | INSTALL_DIR=\"$HOME/.local/bin\" sh\n```\n\nThe Unix installer:\n\n- detects `linux` or `darwin`\n- detects `amd64` or `arm64`\n- downloads `ragflow-cli-{VERSION}-{OS}-{ARCH}`\n- verifies the file with the release `SHA256SUMS`\n- installs to `/usr/local/bin` by default\n\n## Windows PowerShell\n\nRun the repository script in PowerShell:\n\n```powershell\nirm https://raw.githubusercontent.com/infiniflow/ragflow/main/tools/scripts/install.ps1 | iex\n```\nInstall a specific version:\n\n```powershell\n.\\install.ps1 -Version \"v1.0.0\"\n```\n\nInstall to a custom directory:\n\n```powershell\n.\\install.ps1 -InstallDir \"$env:USERPROFILE\\bin\"\n```\n\nThe Windows installer:\n\n- detects `windows/amd64` or `windows/arm64`\n- downloads `ragflow-cli-{VERSION}-windows-{ARCH}.exe`\n- verifies the file with the release `SHA256SUMS`\n- installs to `$env:LOCALAPPDATA\\Programs\\RAGFlow` by default\n- adds the install directory to the user `PATH`\n\n## Release Asset Names\n\nThe install scripts expect these names:\n\n```text\nragflow-cli-v1.0.0-linux-amd64\nragflow-cli-v1.0.0-linux-arm64\nragflow-cli-v1.0.0-darwin-amd64\nragflow-cli-v1.0.0-darwin-arm64\nragflow-cli-v1.0.0-windows-amd64.exe\nragflow-cli-v1.0.0-windows-arm64.exe\nSHA256SUMS\ninstall.sh\ninstall.ps1\n```\n\nBy default, both scripts install the latest stable GitHub Release. Set\n`VERSION` on Unix or `-Version` on Windows to pin a specific release.\n\n\n# Project instructions for Copilot\n\n## How to run (minimum)\n- Install:\n - python -m venv .venv && source .venv/bin/activate\n - pip install -r requirements.txt\n- Run:\n - (fill) e.g. uvicorn app.main:app --reload\n- Verify:\n - (fill) curl http://127.0.0.1:8000/health\n\n## Project layout (what matters)\n- app/: API entrypoints + routers\n- services/: business logic\n- configs/: config loading (.env)\n- docs/: documents\n- tests/: pytest\n\n## Conventions\n- Prefer small, incremental changes.\n- Add logging for new flows.\n- Add/adjust tests for behavior changes.\n\n\n# plugins Documentation\n\nPath: `tools/chatgpt-on-wechat/plugins/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/infiniflow/ragflow/HEAD/tools/chatgpt-on-wechat/plugins/README.md)\n\n\n# RAGFlow ES to OceanBase Migration Tool\n\nA CLI tool for migrating RAGFlow data from Elasticsearch to OceanBase. This tool is specifically designed for RAGFlow's data structure and handles schema conversion, vector data mapping, batch import, and resume capability.\n\n## Features\n\n- **RAGFlow-Specific**: Designed for RAGFlow's fixed data schema\n- **ES 8+ Support**: Uses `search_after` API for efficient data scrolling\n- **Vector Support**: Auto-detects vector field dimensions from ES mapping\n- **Batch Processing**: Configurable batch size for optimal performance\n- **Resume Capability**: Save and resume migration progress\n- **Data Consistency Validation**: Compare document counts and sample data\n- **Migration Report Generation**: Generate detailed migration reports\n\n## Quick Start\n\nThis section provides a complete guide to verify the migration works correctly with a real RAGFlow deployment.\n\n### Prerequisites\n\n- RAGFlow source code cloned\n- Docker and Docker Compose installed\n- This migration tool installed (`uv pip install -e .`)\n\n### Step 1: Start RAGFlow with Elasticsearch Backend\n\nFirst, start RAGFlow using Elasticsearch as the document storage backend (default configuration).\n\n```bash\n# Navigate to RAGFlow docker directory\ncd /path/to/ragflow/docker\n\n# Ensure DOC_ENGINE=elasticsearch in .env (this is the default)\n# DOC_ENGINE=elasticsearch\n\n# Start RAGFlow with Elasticsearch (--profile cpu for CPU, --profile gpu for GPU)\ndocker compose --profile elasticsearch --profile cpu up -d\n\n# Wait for services to be ready (this may take a few minutes)\ndocker compose ps\n\n# Check ES is running\ncurl -X GET \"http://localhost:9200/_cluster/health?pretty\"\n```\n\n### Step 2: Create Test Data in RAGFlow\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Create a new Knowledge Base\n3. Upload some test documents (PDF, TXT, DOCX, etc.)\n4. Wait for the documents to be parsed and indexed\n5. Test the knowledge base with some queries to ensure it works\n\n### Step 3: Verify ES Data (Optional)\n\nBefore migration, verify the data exists in Elasticsearch. This step is important to ensure you have a baseline for comparison after migration.\n\n```bash\n# Navigate to migration tool directory (from ragflow root)\ncd tools/es-to-oceanbase-migration\n\n# Activate the virtual environment if not already done\nsource .venv/bin/activate\n\n# Check connection and list indices\nes-ob-migrate status --es-host localhost --es-port 9200\n\n# First, find your actual index name (pattern: ragflow_{tenant_id})\ncurl -X GET \"http://localhost:9200/_cat/indices/ragflow_*?v\"\n\n# List all knowledge bases in the index\n# Replace ragflow_{tenant_id} with your actual index from the curl output above\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n\n# View sample documents\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_{tenant_id} --size 5\n\n# Check schema\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_{tenant_id}\n```\n\n### Step 4: Start OceanBase for Migration\n\nStart RAGFlow's OceanBase service as the migration target:\n\n```bash\n# Navigate to ragflow docker directory (from ragflow root)\ncd ../docker\n\n# Start only OceanBase service from RAGFlow docker compose\ndocker compose --profile oceanbase up -d\n\n# Wait for OceanBase to be ready\ndocker compose logs -f oceanbase\n```\n\n### Step 5: Run Migration\n\nExecute the migration from Elasticsearch to OceanBase:\n\n```bash\ncd ../tools/es-to-oceanbase-migration\n\n# Option A: Migrate ALL ragflow_* indices (Recommended)\n# If --index and --table are omitted, the tool auto-discovers all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --batch-size 1000 \\\n --verify\n\n# Option B: Migrate a specific index\n# Use the SAME name for both --index and --table\n# The index name pattern is: ragflow_{tenant_id}\n# Find your tenant_id from Step 3's curl output\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --batch-size 1000 \\\n --verify\n```\n\nExpected output:\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_{tenant_id}\nTarget: localhost:2881/ragflow_doc.ragflow_{tenant_id}\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK (version: 4.3.5.1)\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Known RAGFlow fields: 25\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_{tenant_id}' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nStep 5: Verifying migration...\n✓ Document counts match: 1,234\n✓ Sample verification: 100/100 matched\n\nMigration completed successfully!\n Total: 1,234 documents\n Migrated: 1,234 documents\n Failed: 0 documents\n Duration: 45.2 seconds\n```\n\n### Step 6: Stop RAGFlow and Switch to OceanBase Backend\n\n```bash\n# Navigate to ragflow docker directory\ncd ../../docker\n\n# Stop only Elasticsearch and RAGFlow (but keep OceanBase running)\ndocker compose --profile elasticsearch --profile cpu down\n\n# Edit .env file, change:\n# DOC_ENGINE=elasticsearch -> DOC_ENGINE=oceanbase\n#\n# The OceanBase connection settings are already configured by default in .env\n```\n\n### Step 7: Start RAGFlow with OceanBase Backend\n\n```bash\n# OceanBase should still be running from Step 4\n# Start RAGFlow with OceanBase profile (OceanBase is already running)\ndocker compose --profile oceanbase --profile cpu up -d\n\n# Wait for services to start\ndocker compose ps\n\n# Check logs for any errors\ndocker compose logs -f ragflow-cpu\n```\n\n### Step 8: Data Integrity Verification (Optional)\n\nRun the verification command to compare ES and OceanBase data:\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_{tenant_id} \\\n --table ragflow_{tenant_id} \\\n --sample-size 100\n```\n\nExpected output:\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_{tenant_id} │\n│ OB Table: ragflow_{tenant_id} │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n### Step 9: Verify RAGFlow Works with OceanBase\n\n1. Open RAGFlow Web UI: http://localhost:9380\n2. Navigate to your Knowledge Base\n3. Try the same queries you tested before migration\n\n## CLI Reference\n\n### `es-ob-migrate migrate`\n\nRun data migration from Elasticsearch to OceanBase.\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--es-host` | localhost | Elasticsearch host |\n| `--es-port` | 9200 | Elasticsearch port |\n| `--es-user` | None | ES username (if auth required) |\n| `--es-password` | None | ES password |\n| `--ob-host` | localhost | OceanBase host |\n| `--ob-port` | 2881 | OceanBase port |\n| `--ob-user` | root@test | OceanBase user (format: user@tenant) |\n| `--ob-password` | \"\" | OceanBase password |\n| `--ob-database` | test | OceanBase database name |\n| `-i, --index` | None | Source ES index (omit to migrate all ragflow_* indices) |\n| `-t, --table` | None | Target OB table (omit to use same name as index) |\n| `--batch-size` | 1000 | Documents per batch |\n| `--resume` | False | Resume from previous progress |\n| `--verify/--no-verify` | True | Verify after migration |\n\n**Example:**\n\n```bash\n# Migrate all ragflow_* indices\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc\n\n# Migrate a specific index\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123\n\n# Resume interrupted migration\nes-ob-migrate migrate \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --resume\n```\n\n**Resume Feature:**\n\nMigration progress is automatically saved to `.migration_progress/` directory. If migration is interrupted (network error, timeout, etc.), use `--resume` to continue from where it stopped:\n\n- Progress file: `.migration_progress/{index_name}_progress.json`\n- Contains: total count, migrated count, last document ID, timestamp\n- On resume: skips already migrated documents, continues from last position\n\n**Output:**\n\n```\nRAGFlow ES to OceanBase Migration\nSource: localhost:9200/ragflow_abc123\nTarget: localhost:2881/ragflow_doc.ragflow_abc123\n\nStep 1: Checking connections...\n ES cluster status: green\n OceanBase connection: OK\n\nStep 2: Analyzing ES index...\n Auto-detected vector dimension: 1024\n Total documents: 1,234\n\nStep 3: Creating OceanBase table...\n Created table 'ragflow_abc123' with RAGFlow schema\n\nStep 4: Migrating data...\nMigrating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1,234/1,234\n\nMigration completed successfully!\n Total: 1,234 documents\n Duration: 45.2 seconds\n```\n\n---\n\n### `es-ob-migrate list-indices`\n\nList all RAGFlow indices (`ragflow_*`) in Elasticsearch.\n\n**Example:**\n\n```bash\nes-ob-migrate list-indices --es-host localhost --es-port 9200\n```\n\n**Output:**\n\n```\nRAGFlow Indices in Elasticsearch:\n\n Index Name Documents Type\n ragflow_abc123def456789 1234 Document Chunks\n ragflow_doc_meta_abc123def456789 56 Document Metadata\n\nTotal: 2 ragflow_* indices found\n```\n\n---\n\n### `es-ob-migrate schema`\n\nPreview schema analysis from ES mapping.\n\n**Example:**\n\n```bash\nes-ob-migrate schema --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nRAGFlow Schema Analysis for index: ragflow_abc123\n\nVector Fields:\n q_1024_vec: dense_vector (dim=1024)\n\nKnown RAGFlow Fields (25):\n id, kb_id, doc_id, docnm_kwd, content_with_weight, content_ltks,\n available_int, important_kwd, question_kwd, tag_kwd, page_num_int...\n\nUnknown Fields (stored in 'extra' column):\n custom_field_1, custom_field_2\n```\n\n---\n\n### `es-ob-migrate verify`\n\nVerify migration data consistency between ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate verify \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\" \\\n --ob-database ragflow_doc \\\n --index ragflow_abc123 --table ragflow_abc123 \\\n --sample-size 100\n```\n\n**Output:**\n\n```\n╭─────────────────────────────────────────────────────────────╮\n│ Migration Verification Report │\n├─────────────────────────────────────────────────────────────┤\n│ ES Index: ragflow_abc123 │\n│ OB Table: ragflow_abc123 │\n├─────────────────────────────────────────────────────────────┤\n│ Document Counts │\n│ ES: 1,234 │\n│ OB: 1,234 │\n│ Match: ✓ Yes │\n├─────────────────────────────────────────────────────────────┤\n│ Sample Verification (100 documents) │\n│ Matched: 100 │\n│ Match Rate: 100.0% │\n├─────────────────────────────────────────────────────────────┤\n│ Result: ✓ PASSED │\n╰─────────────────────────────────────────────────────────────╯\n```\n\n---\n\n### `es-ob-migrate list-kb`\n\nList all knowledge bases in an ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate list-kb --es-host localhost --es-port 9200 --index ragflow_abc123\n```\n\n**Output:**\n\n```\nKnowledge Bases in index 'ragflow_abc123':\n\n KB ID Documents\n kb_001_finance_docs 456\n kb_002_technical_manual 321\n kb_003_product_faq 457\n\nTotal: 3 knowledge bases, 1234 documents\n```\n\n---\n\n### `es-ob-migrate sample`\n\nShow sample documents from ES index.\n\n**Example:**\n\n```bash\nes-ob-migrate sample --es-host localhost --es-port 9200 --index ragflow_abc123 --size 2\n```\n\n**Output:**\n\n```\nSample Documents from 'ragflow_abc123':\n\nDocument 1:\n id: chunk_001_abc123\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: The company reported Q3 revenue of $1.2B...\n available_int: 1\n\nDocument 2:\n id: chunk_002_def456\n kb_id: kb_001_finance_docs\n doc_id: doc_001\n docnm_kwd: quarterly_report.pdf\n content_with_weight: Operating expenses decreased by 5%...\n available_int: 1\n```\n\n---\n\n### `es-ob-migrate status`\n\nCheck connection status to ES and OceanBase.\n\n**Example:**\n\n```bash\nes-ob-migrate status \\\n --es-host localhost --es-port 9200 \\\n --ob-host localhost --ob-port 2881 \\\n --ob-user \"root@ragflow\" --ob-password \"infini_rag_flow\"\n```\n\n**Output:**\n\n```\nConnection Status:\n\nElasticsearch:\n Host: localhost:9200\n Status: ✓ Connected\n Cluster: ragflow-cluster\n Version: 8.11.0\n Indices: 5\n\nOceanBase:\n Host: localhost:2881\n Status: ✓ Connected\n Version: 4.3.5.1\n```\n\n\n# Firecrawl Integration for RAGFlow\n\nThis integration adds [Firecrawl](https://firecrawl.dev)'s powerful web scraping capabilities to [RAGFlow](https://github.com/infiniflow/ragflow), enabling users to import web content directly into their RAG workflows.\n\n## 🎯 **Integration Overview**\n\nThis integration implements the requirements from [Firecrawl Issue #2167](https://github.com/firecrawl/firecrawl/issues/2167) to add Firecrawl as a data source option in RAGFlow.\n\n### ✅ **Acceptance Criteria Met**\n\n- ✅ **Integration appears as selectable data source** in RAGFlow's UI\n- ✅ **Users can input Firecrawl API keys** through RAGFlow's configuration interface\n- ✅ **Successfully scrapes content** and imports into RAGFlow's document processing pipeline\n- ✅ **Handles edge cases** (rate limits, failed requests, malformed content)\n- ✅ **Includes documentation** and README updates\n- ✅ **Follows RAGFlow patterns** and coding standards\n- ✅ **Ready for engineering review**\n\n## 🚀 **Features**\n\n### Core Functionality\n- **Single URL Scraping** - Scrape individual web pages\n- **Website Crawling** - Crawl entire websites with job management\n- **Batch Processing** - Process multiple URLs simultaneously\n- **Multiple Output Formats** - Support for markdown, HTML, links, and screenshots\n\n### Integration Features\n- **RAGFlow Data Source** - Appears as selectable data source in RAGFlow UI\n- **API Configuration** - Secure API key management with validation\n- **Content Processing** - Converts Firecrawl output to RAGFlow document format\n- **Error Handling** - Comprehensive error handling and retry logic\n- **Rate Limiting** - Built-in rate limiting and request throttling\n\n### Quality Assurance\n- **Content Cleaning** - Intelligent content cleaning and normalization\n- **Metadata Extraction** - Rich metadata extraction and enrichment\n- **Document Chunking** - Automatic document chunking for RAG processing\n- **Language Detection** - Automatic language detection\n- **Validation** - Input validation and error checking\n\n## 📁 **File Structure**\n\n```\ntools/firecrawl/\n├── __init__.py # Package initialization\n├── firecrawl_connector.py # API communication with Firecrawl\n├── firecrawl_config.py # Configuration management\n├── firecrawl_processor.py # Content processing for RAGFlow\n├── firecrawl_ui.py # UI components for RAGFlow\n├── ragflow_integration.py # Main integration class\n├── example_usage.py # Usage examples\n├── requirements.txt # Python dependencies\n├── README.md # This file\n└── INSTALLATION.md # Installation guide\n```\n\n## 🔧 **Installation**\n\n### Prerequisites\n- RAGFlow instance running\n- Firecrawl API key (get one at [firecrawl.dev](https://firecrawl.dev))\n\n### Setup\n1. **Get Firecrawl API Key**:\n - Visit [firecrawl.dev](https://firecrawl.dev)\n - Sign up for a free account\n - Copy your API key (starts with `fc-`)\n\n2. **Configure in RAGFlow**:\n - Go to RAGFlow UI → Data Sources → Add New Source\n - Select \"Firecrawl Web Scraper\"\n - Enter your API key\n - Configure additional options if needed\n\n3. **Test Connection**:\n - Click \"Test Connection\" to verify setup\n - You should see a success message\n\n## 🎮 **Usage**\n\n### Single URL Scraping\n1. Select \"Single URL\" as scrape type\n2. Enter the URL to scrape\n3. Choose output formats (markdown recommended for RAG)\n4. Start scraping\n\n### Website Crawling\n1. Select \"Crawl Website\" as scrape type\n2. Enter the starting URL\n3. Set crawl limit (maximum number of pages)\n4. Configure extraction options\n5. Start crawling\n\n### Batch Processing\n1. Select \"Batch URLs\" as scrape type\n2. Enter multiple URLs (one per line)\n3. Choose output formats\n4. Start batch processing\n\n## 🔧 **Configuration Options**\n\n| Option | Description | Default | Required |\n|--------------------|----------------------------------|-----------------------------|----------|\n| `api_key` | Your Firecrawl API key | - | Yes |\n| `api_url` | Firecrawl API endpoint | `https://api.firecrawl.dev` | No |\n| `max_retries` | Maximum retry attempts | 3 | No |\n| `timeout` | Request timeout (seconds) | 30 | No |\n| `rate_limit_delay` | Delay between requests (seconds) | 1.0 | No |\n\n## 📊 **API Reference**\n\n### RAGFlowFirecrawlIntegration\n\nMain integration class for Firecrawl with RAGFlow.\n\n#### Methods\n- `scrape_and_import(urls, formats, extract_options)` - Scrape URLs and convert to RAGFlow documents\n- `crawl_and_import(start_url, limit, scrape_options)` - Crawl website and convert to RAGFlow documents\n- `test_connection()` - Test connection to Firecrawl API\n- `validate_config(config_dict)` - Validate configuration settings\n\n### FirecrawlConnector\n\nHandles communication with the Firecrawl API.\n\n#### Methods\n- `scrape_url(url, formats, extract_options)` - Scrape single URL\n- `start_crawl(url, limit, scrape_options)` - Start crawl job\n- `get_crawl_status(job_id)` - Get crawl job status\n- `batch_scrape(urls, formats)` - Scrape multiple URLs concurrently\n\n### FirecrawlProcessor\n\nProcesses Firecrawl output for RAGFlow integration.\n\n#### Methods\n- `process_content(content)` - Process scraped content into RAGFlow document format\n- `process_batch(contents)` - Process multiple scraped contents\n- `chunk_content(document, chunk_size, chunk_overlap)` - Chunk document content for RAG processing\n\n## 🧪 **Testing**\n\nThe integration includes comprehensive testing:\n\n```bash\n# Run the test suite\ncd tools/firecrawl\npython3 -c \"\nimport sys\nsys.path.append('.')\nfrom ragflow_integration import create_firecrawl_integration\n\n# Test configuration\nconfig = {\n 'api_key': 'fc-test-key-123',\n 'api_url': 'https://api.firecrawl.dev'\n}\n\nintegration = create_firecrawl_integration(config)\nprint('✅ Integration working!')\n\"\n```\n\n## 🐛 **Error Handling**\n\nThe integration includes robust error handling for:\n\n- **Rate Limiting** - Automatic retry with exponential backoff\n- **Network Issues** - Retry logic with configurable timeouts\n- **Malformed Content** - Content validation and cleaning\n- **API Errors** - Detailed error messages and logging\n\n## 🔒 **Security**\n\n- API key validation and secure storage\n- Input sanitization and validation\n- Rate limiting to prevent abuse\n- Error handling without exposing sensitive information\n\n## 📈 **Performance**\n\n- Concurrent request processing\n- Configurable timeouts and retries\n- Efficient content processing\n- Memory-conscious document handling\n\n## 🤝 **Contributing**\n\nThis integration was created as part of the [Firecrawl bounty program](https://github.com/firecrawl/firecrawl/issues/2167).\n\n### Development\n1. Fork the RAGFlow repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## 📄 **License**\n\nThis integration is licensed under the same license as RAGFlow (Apache 2.0).\n\n## 🆘 **Support**\n\n- **Firecrawl Documentation**: [docs.firecrawl.dev](https://docs.firecrawl.dev)\n- **RAGFlow Documentation**: [RAGFlow GitHub](https://github.com/infiniflow/ragflow)\n- **Issues**: Report issues in the RAGFlow repository\n\n## 🎉 **Acknowledgments**\n\nThis integration was developed as part of the Firecrawl bounty program to bridge the gap between web content and RAG applications, making it easier for developers to build AI applications that can leverage real-time web data.\n\n---\n\n**Ready for RAGFlow Integration!** 🚀\n\nThis integration enables RAGFlow users to easily import web content into their knowledge retrieval systems, expanding the ecosystem for both Firecrawl and RAGFlow.\n"}