WeKnora Documentation

Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

19,779 stars Go Go 1 file · ~30,045 tokens #agent#agentic#ai#chatbot#embeddings#evaluation#generative-ai#golang
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy WeKnora on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run) 
docker run -d --name weknora -p 5432:5432 --restart always weknora-odl-hybrid:local
Configured Services & Ports
⚙️ postgres
image: paradedb/paradedb:v0.25.11-pg17
🔌 Ports: ${DB_PORT:-5432}:5432
⚙️ redis
image: redis:7.0-alpine
🔌 Ports: ${REDIS_PORT:-6379}:6379
⚙️ searxng-init
image: busybox:1.36
⚙️ searxng
image: searxng/searxng:latest
🔌 Ports: ${SEARXNG_BIND:-127.0.0.1}:${SEARXNG_PORT:-8888}:8080
⚙️ minio
image: pgsty/minio:latest
🔌 Ports: ${MINIO_PORT:-9000}:9000, ${MINIO_CONSOLE_PORT:-9001}:9001
⚙️ qdrant
image: qdrant/qdrant:v1.16.2
🔌 Ports: ${QDRANT_REST_PORT:-6333}:6333, ${QDRANT_PORT:-6334}:6334
⚙️ opensearch
image: opensearchproject/opensearch:3.3.2
🔌 Ports: ${OPENSEARCH_PORT:-9200}:9200
⚙️ opensearch-dashboards
image: opensearchproject/opensearch-dashboards:3.3.0
🔌 Ports: ${OPENSEARCH_DASHBOARDS_PORT:-5601}:5601
⚙️ milvus
image: milvusdb/milvus:v2.6.11
🔌 Ports: ${MILVUS_PORT:-19530}:19530, ${MILVUS_HEALTH_PORT:-9091}:9091
⚙️ neo4j
image: neo4j:latest
🔌 Ports: 7474:7474, 7687:7687
⚙️ sandbox
image: wechatopenai/weknora-sandbox:${WEKNORA_VERSION:-latest}
⚙️ docreader
image: wechatopenai/weknora-docreader:${WEKNORA_VERSION:-latest}
🔌 Ports: ${DOCREADER_PORT:-50051}:50051
⚙️ odl-hybrid
image: weknora-odl-hybrid:local
🔌 Ports: ${ODL_HYBRID_PORT:-5002}:5002
⚙️ dex
image: dexidp/dex:latest
🔌 Ports: 5556:5556
⚙️ langfuse-db-init
image: paradedb/paradedb:v0.25.11-pg17
⚙️ langfuse-clickhouse
image: clickhouse/clickhouse-server:24.8
⚙️ langfuse-minio
image: pgsty/minio:RELEASE.2026-08-04T00-00-00Z
🔌 Ports: ${LANGFUSE_MINIO_S3_PORT:-9100}:9000, ${LANGFUSE_MINIO_CONSOLE_PORT:-9101}:9001
⚙️ langfuse-worker
image: langfuse/langfuse-worker:3
⚙️ langfuse-web
image: langfuse/langfuse:3
🔌 Ports: ${LANGFUSE_WEB_PORT:-3000}:3000
docker-compose.yml
yaml (docker-compose.yml) 
# 开发环境配置 - 只启动基础设施服务，app 和 frontend 在本地运行
services:
  # 只启动依赖的基础设施服务
  postgres:
    image: paradedb/paradedb:v0.25.11-pg17
    container_name: WeKnora-postgres-dev
    ports:
      - "${DB_PORT:-5432}:5432"
    environment:
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}
    volumes:
      - postgres-data-dev:/var/lib/postgresql/data
    networks:
      - WeKnora-network-dev
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USER} -d ${DB_NAME}"]
      interval: 10s
      timeout: 10s
      retries: 3
      start_period: 30s
    restart: unless-stopped
    stop_grace_period: 1m

  redis:
    image: redis:7.0-alpine
    container_name: WeKnora-redis-dev
    ports:
      - "${REDIS_PORT:-6379}:6379"
    volumes:
      - redis_data_dev:/data
    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD}
    restart: always
    networks:
      - WeKnora-network-dev

  # See docker-compose.yml searxng-init for rationale (avoid in-place rewrite of
  # the repo-tracked settings.yml by the searxng entrypoint).
  searxng-init:
    image: busybox:1.36
    container_name: WeKnora-searxng-init-dev
    command: ["sh", "-c", "cp /template/settings.yml /etc/searxng/settings.yml && chmod 0644 /etc/searxng/settings.yml"]
    volumes:
      - ./docker/searxng/settings.yml:/template/settings.yml:ro
      - searxng_config_dev:/etc/searxng
    restart: "no"
    networks:
      - WeKnora-network-dev
    profiles:
      - searxng
      - full

  searxng:
    image: searxng/searxng:latest
    container_name: WeKnora-searxng-dev
    # Local dev: bind to loopback by default; the host-side WeKnora process
    # reaches it at http://127.0.0.1:${SEARXNG_PORT}.
    ports:
      - "${SEARXNG_BIND:-127.0.0.1}:${SEARXNG_PORT:-8888}:8080"
    volumes:
      - searxng_config_dev:/etc/searxng
    environment:
      - SEARXNG_BASE_URL=http://localhost:${SEARXNG_PORT:-8888}/
      - INSTANCE_NAME=weknora-searxng-dev
      # See docker-compose.yml for rationale on the default secret.
      - SEARXNG_SECRET=${SEARXNG_SECRET:-weknora-default-searxng-secret-rotate-before-exposing-publicly}
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    restart: unless-stopped
    depends_on:
      searxng-init:
        condition: service_completed_successfully
    networks:
      - WeKnora-network-dev
    profiles:
      - searxng
      - full

  minio:
    image: pgsty/minio:latest
    container_name: WeKnora-minio-dev
    ports:
      - "${MINIO_PORT:-9000}:9000"
      - "${MINIO_CONSOLE_PORT:-9001}:9001"
    environment:
      - MINIO_ROOT_USER=${MINIO_ACCESS_KEY_ID:-minioadmin}
      - MINIO_ROOT_PASSWORD=${MINIO_SECRET_ACCESS_KEY:-minioadmin}
    command: server --console-address ":9001" /data
    volumes:
      - minio_data_dev:/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    networks:
      - WeKnora-network-dev
    profiles:
      - minio
      - full

  qdrant:
    image: qdrant/qdrant:v1.16.2
    container_name: WeKnora-qdrant-dev
    ports:
      - "${QDRANT_REST_PORT:-6333}:6333"
      - "${QDRANT_PORT:-6334}:6334"
    volumes:
      - qdrant_data_dev:/qdrant/storage
    networks:
      - WeKnora-network-dev
    restart: unless-stopped
    profiles:
      - qdrant
      - full

  # OpenSearch k-NN (Phase 3 driver). Single-node dev profile with the
  # security plugin disabled → plain HTTP on :9200, no auth/TLS. The image
  # bundles the opensearch-knn plugin. For production use a secured,
  # multi-node cluster. See website-docs/03-features/05-retrieval-engines.md.
  opensearch:
    image: opensearchproject/opensearch:3.3.2
    container_name: WeKnora-opensearch-dev
    environment:
      - discovery.type=single-node
      # dev only: plain HTTP on :9200, no TLS/auth. The entrypoint script
      # honours DISABLE_SECURITY_PLUGIN (env var) to skip both the demo
      # install and the OPENSEARCH_INITIAL_ADMIN_PASSWORD requirement.
      - DISABLE_SECURITY_PLUGIN=true
      - DISABLE_INSTALL_DEMO_CONFIG=true
      - OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m
      - bootstrap.memory_lock=true
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "${OPENSEARCH_PORT:-9200}:9200"
    volumes:
      - opensearch_data_dev:/usr/share/opensearch/data
    networks:
      - WeKnora-network-dev
    restart: unless-stopped
    profiles:
      - opensearch
      - full
      # Also a member of opensearch-ui so the Dashboards depends_on resolves
      # when only that profile is active (`--profile opensearch-ui up`).
      - opensearch-ui

  # Optional UI for visual index/mapping/query inspection. Decoupled from the
  # "opensearch" / "full" profiles so the heavy Dashboards container is never
  # forced up alongside the cluster — the driver e2e is fully curl-verifiable
  # against :9200. Start it on demand with `--profile opensearch-ui up -d`
  # (depends_on pulls the cluster in automatically).
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:3.3.0
    container_name: WeKnora-opensearch-dashboards-dev
    environment:
      - OPENSEARCH_HOSTS=["http://opensearch:9200"]
      - DISABLE_SECURITY_DASHBOARDS_PLUGIN=true
    ports:
      - "${OPENSEARCH_DASHBOARDS_PORT:-5601}:5601"
    networks:
      - WeKnora-network-dev
    depends_on:
      - opensearch
    restart: unless-stopped
    profiles:
      - opensearch-ui

  milvus:
    image: milvusdb/milvus:v2.6.11
    container_name: WeKnora-milvus-dev
    security_opt:
      - seccomp:unconfined
    command: ["milvus", "run", "standalone"]
    environment:
      - ETCD_USE_EMBED=true
      - ETCD_DATA_DIR=/var/lib/milvus/etcd
      - COMMON_STORAGETYPE=local
      - DEPLOY_MODE=STANDALONE
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
      interval: 30s
      start_period: 90s
      timeout: 20s
      retries: 3
    ports:
      - "${MILVUS_PORT:-19530}:19530"
      - "${MILVUS_HEALTH_PORT:-9091}:9091"
    volumes:
      - milvus_data_dev:/var/lib/milvus
    networks:
      - WeKnora-network-dev
    restart: unless-stopped
    profiles:
      - milvus
      - full

  neo4j:
    image: neo4j:latest
    container_name: WeKnora-neo4j-dev
    volumes:
      - neo4j-data-dev:/data
    environment:
      - NEO4J_AUTH=${NEO4J_USERNAME:-neo4j}/${NEO4J_PASSWORD:-password}
      - NEO4J_apoc_export_file_enabled=true
      - NEO4J_apoc_import_file_enabled=true
      - NEO4J_apoc_import_file_use__neo4j__config=true
      - NEO4JLABS_PLUGINS=["apoc"]
    ports:
      - "7474:7474"
      - "7687:7687"
    restart: always
    networks:
      - WeKnora-network-dev
    profiles:
      - neo4j
      - full

  # Sandbox 镜像：仅用于 build/pull，非常驻服务；本地 app 执行 Skills 时按需 docker run 该镜像，用毕即释
  sandbox:
    image: wechatopenai/weknora-sandbox:${WEKNORA_VERSION:-latest}
    container_name: WeKnora-sandbox-dev
    build:
      context: .
      dockerfile: docker/Dockerfile.sandbox
    profiles:
      - full
    command: ["true"]
    restart: "no"

  docreader:
    build:
      context: .
      dockerfile: docker/Dockerfile.docreader
    image: wechatopenai/weknora-docreader:${WEKNORA_VERSION:-latest}
    container_name: WeKnora-docreader-dev
    ports:
      - "${DOCREADER_PORT:-50051}:50051"
    volumes:
      - docreader-tmp-dev:/tmp/docreader
    environment:
      # ========== 基础 ==========
      - TZ=${TZ:-Asia/Shanghai}
      - LOG_LEVEL=${LOG_LEVEL:-}
      - DOCREADER_IMAGE_OUTPUT_DIR=/tmp/docreader
      - MAX_FILE_SIZE_MB=${MAX_FILE_SIZE_MB:-}
      # docreader 对外 HTTP 抓取的 SSRF 白名单（逗号分隔；与 app 共用同名变量）
      - SSRF_WHITELIST=${SSRF_WHITELIST:-}
      - SSRF_WHITELIST_EXTRA=${SSRF_WHITELIST_EXTRA:-}
      - SSRF_DNS_WHITELIST_ONLY=${SSRF_DNS_WHITELIST_ONLY:-}
      # ========== gRPC 服务端 TLS / 认证 ==========
      - GRPC_TLS_ENABLED=${GRPC_TLS_ENABLED:-false}
      - GRPC_TLS_CERT=${GRPC_TLS_CERT:-}
      - GRPC_TLS_KEY=${GRPC_TLS_KEY:-}
      - GRPC_TLS_CA=${GRPC_TLS_CA:-}
      - GRPC_TLS_SERVER_NAME=${GRPC_TLS_SERVER_NAME:-}
      # true 时强制要求客户端证书（mTLS）
      - GRPC_MTLS_REQUIRE_CLIENT_CERT=${GRPC_MTLS_REQUIRE_CLIENT_CERT:-false}
      - GRPC_AUTH_TOKEN=${GRPC_AUTH_TOKEN:-}
      # ========== gRPC 服务参数 ==========
      - DOCREADER_GRPC_MAX_WORKERS=${DOCREADER_GRPC_MAX_WORKERS:-4}
      - DOCREADER_GRPC_MAX_FILE_SIZE_MB=${DOCREADER_GRPC_MAX_FILE_SIZE_MB:-${MAX_FILE_SIZE_MB:-50}}
      - DOCREADER_GRPC_PORT=${DOCREADER_GRPC_PORT:-50051}
      # Optional RapidOCR subprocesses, independent of ordinary parser RPCs.
      - DOCREADER_PREPROCESS_MAX_WORKERS=${DOCREADER_PREPROCESS_MAX_WORKERS:-1}
      - DOCREADER_PREPROCESS_TIMEOUT_SECONDS=${DOCREADER_PREPROCESS_TIMEOUT_SECONDS:-30}
      - DOCREADER_PREPROCESS_THREADS=${DOCREADER_PREPROCESS_THREADS:-2}
      - DOCREADER_PREPROCESS_MEMORY_MB=${DOCREADER_PREPROCESS_MEMORY_MB:-2048}
      - DOCREADER_PREPROCESS_MAX_PIXELS=${DOCREADER_PREPROCESS_MAX_PIXELS:-16000000}
      - DOCREADER_PREPROCESS_MAX_EDGE=${DOCREADER_PREPROCESS_MAX_EDGE:-2000}
      # ========== 解析通用 ==========
      - DOCREADER_DOCX_MAX_PAGES=${DOCREADER_DOCX_MAX_PAGES:-0}
      - DOCREADER_MARKITDOWN_MAX_WORKERS=${DOCREADER_MARKITDOWN_MAX_WORKERS:-1}
      # 出站 HTTP/HTTPS 代理（抓取网页 / 远程图片时）
      - DOCREADER_EXTERNAL_HTTP_PROXY=${DOCREADER_EXTERNAL_HTTP_PROXY:-}
      - DOCREADER_EXTERNAL_HTTPS_PROXY=${DOCREADER_EXTERNAL_HTTPS_PROXY:-}
      # ========== OpenDataLoader hybrid ==========
      - DOCREADER_ODL_MAX_WORKERS=${DOCREADER_ODL_MAX_WORKERS:-1}
      - DOCREADER_ODL_HYBRID=${DOCREADER_ODL_HYBRID:-off}
      - DOCREADER_ODL_HYBRID_URL=${DOCREADER_ODL_HYBRID_URL:-http://odl-hybrid:5002}
      - DOCREADER_ODL_HYBRID_MODE=${DOCREADER_ODL_HYBRID_MODE:-auto}
      - DOCREADER_ODL_HYBRID_FALLBACK=${DOCREADER_ODL_HYBRID_FALLBACK:-false}
      - DOCREADER_ODL_MARKDOWN_WITH_HTML=${DOCREADER_ODL_MARKDOWN_WITH_HTML:-false}
      # ========== PDF 渲染 ==========
      - DOCREADER_PDF_RENDER_MAX_WORKERS=${DOCREADER_PDF_RENDER_MAX_WORKERS:-1}
      # 单个 PDF 内部扫描页渲染并行进程数（pdfium 非线程安全）。留空走代码默认 min(4,cpu)，
      # 避免在低核容器上过度并行（写死数值会覆盖 CPU 感知逻辑）。
      - DOCREADER_PDF_RENDER_PARALLELISM=${DOCREADER_PDF_RENDER_PARALLELISM:-}
      - DOCREADER_PDF_RENDER_DPI=${DOCREADER_PDF_RENDER_DPI:-200}
      # JPEG 质量（代码默认 85；compose 此前误写 90，已对齐）
      - DOCREADER_PDF_JPEG_QUALITY=${DOCREADER_PDF_JPEG_QUALITY:-85}
      # 渲染页图最大长边像素（防超大页撑爆 gRPC 消息上限；密集中文建议 >=1600）
      - DOCREADER_PDF_RENDER_MAX_EDGE=${DOCREADER_PDF_RENDER_MAX_EDGE:-2000}
      # 强制 PDF 全页按扫描件解析（文本层差但可 OCR 时；速度/成本上升）
      - DOCREADER_PDF_FORCE_SCANNED=${DOCREADER_PDF_FORCE_SCANNED:-false}
      # ========== PDF 版式 / 文本调优 ==========
      # glyph 间距超过 median 字宽的多少倍时插空格（默认 0.4）
      - DOCREADER_PDF_WORD_GAP_WIDTH_RATIO=${DOCREADER_PDF_WORD_GAP_WIDTH_RATIO:-0.4}
      # 丢弃窄边栏（arXiv 侧栏）的页宽比例阈值（默认 0.12）
      - DOCREADER_PDF_MARGIN_COL_WIDTH_RATIO=${DOCREADER_PDF_MARGIN_COL_WIDTH_RATIO:-0.12}
      - DOCREADER_PDF_MIN_HEADING_LINE_CHARS=${DOCREADER_PDF_MIN_HEADING_LINE_CHARS:-8}
      # 清理 U+FFFE/软连字符、剥离矢量图轴文字、图表区按 JPEG 渲染
      - DOCREADER_PDF_SANITIZE_TEXT=${DOCREADER_PDF_SANITIZE_TEXT:-true}
      - DOCREADER_PDF_STRIP_CHART_DEBRIS=${DOCREADER_PDF_STRIP_CHART_DEBRIS:-true}
      - DOCREADER_PDF_RENDER_VECTOR_FIGURES=${DOCREADER_PDF_RENDER_VECTOR_FIGURES:-true}
      - DOCREADER_PDF_DETECT_HEADINGS=${DOCREADER_PDF_DETECT_HEADINGS:-}
      - DOCREADER_PDF_LAYOUT_ORDERING=${DOCREADER_PDF_LAYOUT_ORDERING:-}
      - DOCREADER_PDF_FILTER_HIDDEN_TEXT=${DOCREADER_PDF_FILTER_HIDDEN_TEXT:-}
      # 扫描件判定：图片面积占比 >= 此值视为扫描页（默认留空走代码默认）
      - DOCREADER_PDF_SCAN_IMAGE_RATIO=${DOCREADER_PDF_SCAN_IMAGE_RATIO:-}
      - DOCREADER_PDF_SCAN_MIN_CHARS=${DOCREADER_PDF_SCAN_MIN_CHARS:-}
      # 嵌入图片抽取
      - DOCREADER_PDF_EXTRACT_EMBEDDED_IMAGES=${DOCREADER_PDF_EXTRACT_EMBEDDED_IMAGES:-}
      - DOCREADER_PDF_EMBED_MIN_PIXELS=${DOCREADER_PDF_EMBED_MIN_PIXELS:-}
      - DOCREADER_PDF_EMBED_MIN_AREA_RATIO=${DOCREADER_PDF_EMBED_MIN_AREA_RATIO:-}
      - DOCREADER_PDF_EMBED_REPEAT_PAGE_FRAC=${DOCREADER_PDF_EMBED_REPEAT_PAGE_FRAC:-}
      - DOCREADER_PDF_EMBED_MAX_IMAGES=${DOCREADER_PDF_EMBED_MAX_IMAGES:-}
      # 图表区识别
      - DOCREADER_PDF_MIN_CHART_REGION_AREA=${DOCREADER_PDF_MIN_CHART_REGION_AREA:-}
      - DOCREADER_PDF_MIN_CHART_REGION_CHARS=${DOCREADER_PDF_MIN_CHART_REGION_CHARS:-}
      - DOCREADER_PDF_MAX_CHART_REGION_AREA=${DOCREADER_PDF_MAX_CHART_REGION_AREA:-}
      - DOCREADER_PDF_MAX_FIGURE_HEIGHT_RATIO=${DOCREADER_PDF_MAX_FIGURE_HEIGHT_RATIO:-}
    healthcheck:
      test: ["CMD", "grpc_health_probe", "-addr=localhost:50051"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    networks:
      - WeKnora-network-dev
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"

  # OpenDataLoader hybrid backend (optional). Enable profile "odl-hybrid" and set
  # DOCREADER_ODL_HYBRID=docling-fast on docreader. Default --no-ocr (no EasyOCR).
  # Local build only — not published to Docker Hub.
  odl-hybrid:
    build:
      context: .
      dockerfile: docker/Dockerfile.odl-hybrid
    image: weknora-odl-hybrid:local
    container_name: WeKnora-odl-hybrid
    profiles:
      - odl-hybrid
    ports:
      - "${ODL_HYBRID_PORT:-5002}:5002"
    environment:
      # Default --no-ocr (digital PDFs). Scanned PDFs: use builtin OCR / MinerU, or
      # ODL_HYBRID_EXTRA_ARGS="--force-ocr" (needs EasyOCR + libGL in image).
      - ODL_HYBRID_EXTRA_ARGS=${ODL_HYBRID_EXTRA_ARGS:---no-ocr}
    networks:
      - WeKnora-network-dev
    restart: unless-stopped

  dex:
    image: dexidp/dex:latest
    container_name: WeKnora-dex-dev
    ports:
      - "5556:5556"
    volumes:
      - ./misc/dex-config.yaml:/etc/dex/config.yaml
    command: ["dex", "serve", "/etc/dex/config.yaml"]
    profiles:
      - dex
      - full

  # ---------------------------------------------------------------------------
  # Langfuse 自建栈（dev 对称版）
  #
  # 用法：
  #   docker compose -f docker-compose.dev.yml --profile langfuse up -d
  #
  # 本地 app (go run) 需要的环境变量：
  #   export LANGFUSE_HOST=http://localhost:3000
  #   export LANGFUSE_PUBLIC_KEY=pk-lf-xxx
  #   export LANGFUSE_SECRET_KEY=sk-lf-xxx
  #
  # 复用 dev 已有的 postgres（独立 langfuse 数据库）和 redis（DB 1），
  # 新增：clickhouse、minio、web、worker + 一次性 db-init，和生产版结构一致。
  # ---------------------------------------------------------------------------
  # 复用 dev 已有的 ParadeDB 镜像，不额外拉 postgres 镜像
  langfuse-db-init:
    image: paradedb/paradedb:v0.25.11-pg17
    container_name: WeKnora-langfuse-db-init-dev
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      PGPASSWORD: ${DB_PASSWORD}
    # ${LANGFUSE_DB_NAME:-langfuse} / ${DB_USER} 由 compose 解析成字面量后再传给 shell。
    entrypoint: ["sh", "-c"]
    command:
      - |
        set -e
        echo "[langfuse-db-init] ensuring database '${LANGFUSE_DB_NAME:-langfuse}' exists..."
        # 先刷新现有库的 collation（镜像 ICU 2.36 与宿主 2.41 不匹配时必须做），否则 CREATE DATABASE 会失败
        psql -h postgres -U ${DB_USER} -d postgres -v ON_ERROR_STOP=0 -c "ALTER DATABASE template1 REFRESH COLLATION VERSION;" >/dev/null 2>&1 || true
        psql -h postgres -U ${DB_USER} -d postgres -v ON_ERROR_STOP=0 -c "ALTER DATABASE postgres REFRESH COLLATION VERSION;" >/dev/null 2>&1 || true
        # 幂等创建：已存在则跳过；不存在则从 template0 克隆（template0 永远不会有 collation 漂移）
        if psql -h postgres -U ${DB_USER} -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname='${LANGFUSE_DB_NAME:-langfuse}'" | grep -q 1; then
          echo "[langfuse-db-init] database '${LANGFUSE_DB_NAME:-langfuse}' already exists, skipping."
        else
          psql -h postgres -U ${DB_USER} -d postgres -v ON_ERROR_STOP=1 -c "CREATE DATABASE \"${LANGFUSE_DB_NAME:-langfuse}\" TEMPLATE template0;"
          echo "[langfuse-db-init] database '${LANGFUSE_DB_NAME:-langfuse}' created."
        fi
        echo "[langfuse-db-init] done."
    networks:
      - WeKnora-network-dev
    restart: "no"
    profiles:
      - langfuse
      - full

  langfuse-clickhouse:
    image: clickhouse/clickhouse-server:24.8
    container_name: WeKnora-langfuse-clickhouse-dev
    restart: unless-stopped
    user: "101:101"
    environment:
      CLICKHOUSE_DB: default
      CLICKHOUSE_USER: ${LANGFUSE_CLICKHOUSE_USER:-clickhouse}
      CLICKHOUSE_PASSWORD: ${LANGFUSE_CLICKHOUSE_PASSWORD:-clickhouse}
    volumes:
      - langfuse_clickhouse_data_dev:/var/lib/clickhouse
      - langfuse_clickhouse_logs_dev:/var/log/clickhouse-server
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://localhost:8123/ping || exit 1
      interval: 5s
      timeout: 5s
      retries: 10
      start_period: 10s
    networks:
      - WeKnora-network-dev
    profiles:
      - langfuse
      - full

  langfuse-minio:
    image: pgsty/minio:RELEASE.2026-08-04T00-00-00Z
    container_name: WeKnora-langfuse-minio-dev
    restart: unless-stopped
    entrypoint: sh
    command: -c 'mkdir -p /data/langfuse && minio server --address ":9000" --console-address ":9001" /data'
    environment:
      MINIO_ROOT_USER: ${LANGFUSE_MINIO_USER:-langfuseminio}
      MINIO_ROOT_PASSWORD: ${LANGFUSE_MINIO_PASSWORD:-langfuseminiosecret}
    ports:
      - "${LANGFUSE_MINIO_S3_PORT:-9100}:9000"
      - "${LANGFUSE_MINIO_CONSOLE_PORT:-9101}:9001"
    volumes:
      - langfuse_minio_data_dev:/data
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 5s
      timeout: 10s
      retries: 5
    networks:
      - WeKnora-network-dev
    profiles:
      - langfuse
      - full

  langfuse-worker:
    image: langfuse/langfuse-worker:3
    container_name: WeKnora-langfuse-worker-dev
    restart: unless-stopped
    depends_on: &langfuse-dev-depends-on
      langfuse-db-init:
        condition: service_completed_successfully
      redis:
        condition: service_started
      langfuse-clickhouse:
        condition: service_healthy
      langfuse-minio:
        condition: service_healthy
    # wrapper entrypoint 把 DB_PASSWORD / REDIS_PASSWORD URL 编码后再拼 URL，
    # 避免密码含 '@' / '#' 等字符导致 Prisma P1013 解析失败。
    # 注意：compose 覆盖 entrypoint 会清空镜像默认 CMD，所以末尾写死原始命令。
    entrypoint:
      - /bin/sh
      - -ec
      - |
        _enc() { node -e 'process.stdout.write(encodeURIComponent(process.argv[1]))' "$$1"; }
        DU=$$(_enc "$$_LF_DB_USER")
        DP=$$(_enc "$$_LF_DB_PASSWORD")
        RP=$$(_enc "$$_LF_REDIS_PASSWORD")
        export DATABASE_URL="postgresql://$$DU:$$DP@postgres:5432/$$_LF_DB_NAME"
        export REDIS_CONNECTION_STRING="redis://:$$RP@redis:6379/$$_LF_REDIS_DB"
        unset _LF_DB_USER _LF_DB_PASSWORD _LF_REDIS_PASSWORD
        exec dumb-init -- ./worker/entrypoint.sh node worker/dist/index.js
    environment: &langfuse-dev-env
      # 原始凭证（未 URL 编码），由 entrypoint wrapper 读取并组装
      _LF_DB_USER: ${DB_USER}
      _LF_DB_PASSWORD: ${DB_PASSWORD}
      _LF_DB_NAME: ${LANGFUSE_DB_NAME:-langfuse}
      _LF_REDIS_PASSWORD: ${REDIS_PASSWORD}
      _LF_REDIS_DB: ${LANGFUSE_REDIS_DB:-1}
      SALT: ${LANGFUSE_SALT:-weknora-langfuse-dev-salt-change-me}
      ENCRYPTION_KEY: ${LANGFUSE_ENCRYPTION_KEY:-0000000000000000000000000000000000000000000000000000000000000000}
      NEXTAUTH_URL: ${LANGFUSE_NEXTAUTH_URL:-http://localhost:3000}
      NEXTAUTH_SECRET: ${LANGFUSE_NEXTAUTH_SECRET:-weknora-langfuse-dev-nextauth-secret-change-me}
      TELEMETRY_ENABLED: ${LANGFUSE_TELEMETRY_ENABLED:-false}
      LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES: "false"
      CLICKHOUSE_URL: http://langfuse-clickhouse:8123
      CLICKHOUSE_MIGRATION_URL: clickhouse://langfuse-clickhouse:9000
      CLICKHOUSE_USER: ${LANGFUSE_CLICKHOUSE_USER:-clickhouse}
      CLICKHOUSE_PASSWORD: ${LANGFUSE_CLICKHOUSE_PASSWORD:-clickhouse}
      CLICKHOUSE_CLUSTER_ENABLED: "false"
      LANGFUSE_S3_EVENT_UPLOAD_BUCKET: langfuse
      LANGFUSE_S3_EVENT_UPLOAD_REGION: auto
      LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID: ${LANGFUSE_MINIO_USER:-langfuseminio}
      LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY: ${LANGFUSE_MINIO_PASSWORD:-langfuseminiosecret}
      LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT: http://langfuse-minio:9000
      LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE: "true"
      LANGFUSE_S3_EVENT_UPLOAD_PREFIX: events/
      LANGFUSE_S3_MEDIA_UPLOAD_BUCKET: langfuse
      LANGFUSE_S3_MEDIA_UPLOAD_REGION: auto
      LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID: ${LANGFUSE_MINIO_USER:-langfuseminio}
      LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY: ${LANGFUSE_MINIO_PASSWORD:-langfuseminiosecret}
      LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT: ${LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT:-http://localhost:9100}
      LANGFUSE_S3_MEDIA_UPLOAD_FORCE_PATH_STYLE: "true"
      LANGFUSE_S3_MEDIA_UPLOAD_PREFIX: media/
    networks:
      - WeKnora-network-dev
    profiles:
      - langfuse
      - full

  langfuse-web:
    image: langfuse/langfuse:3
    container_name: WeKnora-langfuse-web-dev
    restart: unless-stopped
    depends_on: *langfuse-dev-depends-on
    ports:
      - "${LANGFUSE_WEB_PORT:-3000}:3000"
    entrypoint:
      - /bin/sh
      - -ec
      - |
        _enc() { node -e 'process.stdout.write(encodeURIComponent(process.argv[1]))' "$$1"; }
        DU=$$(_enc "$$_LF_DB_USER")
        DP=$$(_enc "$$_LF_DB_PASSWORD")
        RP=$$(_enc "$$_LF_REDIS_PASSWORD")
        export DATABASE_URL="postgresql://$$DU:$$DP@postgres:5432/$$_LF_DB_NAME"
        export REDIS_CONNECTION_STRING="redis://:$$RP@redis:6379/$$_LF_REDIS_DB"
        unset _LF_DB_USER _LF_DB_PASSWORD _LF_REDIS_PASSWORD
        if [ -n "$$NEXT_PUBLIC_LANGFUSE_CLOUD_REGION" ]; then
          exec dumb-init -- ./web/entrypoint.sh node --import dd-trace/initialize.mjs ./web/server.js --keepAliveTimeout 110000
        else
          exec dumb-init -- ./web/entrypoint.sh node ./web/server.js --keepAliveTimeout 110000
        fi
    environment:
      <<: *langfuse-dev-env
      LANGFUSE_INIT_ORG_ID: ${LANGFUSE_INIT_ORG_ID:-}
      LANGFUSE_INIT_ORG_NAME: ${LANGFUSE_INIT_ORG_NAME:-}
      LANGFUSE_INIT_PROJECT_ID: ${LANGFUSE_INIT_PROJECT_ID:-}
      LANGFUSE_INIT_PROJECT_NAME: ${LANGFUSE_INIT_PROJECT_NAME:-}
      LANGFUSE_INIT_PROJECT_PUBLIC_KEY: ${LANGFUSE_INIT_PROJECT_PUBLIC_KEY:-}
      LANGFUSE_INIT_PROJECT_SECRET_KEY: ${LANGFUSE_INIT_PROJECT_SECRET_KEY:-}
      LANGFUSE_INIT_USER_EMAIL: ${LANGFUSE_INIT_USER_EMAIL:-}
      LANGFUSE_INIT_USER_NAME: ${LANGFUSE_INIT_USER_NAME:-}
      LANGFUSE_INIT_USER_PASSWORD: ${LANGFUSE_INIT_USER_PASSWORD:-}
    networks:
      - WeKnora-network-dev
    profiles:
      - langfuse
      - full


networks:
  WeKnora-network-dev:
    driver: bridge

volumes:
  postgres-data-dev:
  redis_data_dev:
  minio_data_dev:
  neo4j-data-dev:
  qdrant_data_dev:
  opensearch_data_dev:
  milvus_data_dev:
  docreader-tmp-dev:
  langfuse_clickhouse_data_dev:
  langfuse_clickhouse_logs_dev:
  langfuse_minio_data_dev:
  searxng_config_dev:

1. Project Overview & Quickstart (Tencent/WeKnora)

File: README.md

💡 WeKnora — Turn Documents into Living Knowledge with RAG, Agents and Auto-Wiki

📌 Overview

WeKnora is an open-source, LLM-powered knowledge framework built for enterprise-grade document understanding, semantic retrieval, and autonomous reasoning.

It is organized around three core capabilities: RAG-based Quick Q&A for everyday lookups, a ReAct Agent that autonomously orchestrates retrieval, MCP tools and web search to handle complex multi-step tasks, and a brand-new Wiki Mode in which agents distill raw documents into a self-maintaining, interlinked markdown knowledge base with an interactive knowledge graph, complete with manual editing, revision history and one-click rollback. Knowledge curation is equally hands-on: a tree-structured folder view preserves the directory layout of uploads, and chunk editing with revision history lets retrieval chunks be edited, diffed and reverted like documents. Combined with multi-source ingestion (Feishu wiki / Feishu Drive / Notion / Yuque / RSS, and growing), website embed widgets for publishing agents to external sites, scoped API keys with a principal model for programmatic integrations, multi-instance storage backends per workspace for flexible data placement, 20+ LLM provider integrations, full Langfuse observability plus a runtime task-queue dashboard with worker-pool governance, enterprise-ready multi-workspace RBAC (4-tier role matrix + per-resource ownership + per-workspace audit log), and a fully self-hostable modular architecture, WeKnora turns scattered documents into a queryable, reasoning-capable, continuously evolving knowledge asset.

The framework supports auto-syncing knowledge from Feishu, Notion, and Yuque (more data sources coming soon), handles 10+ document formats including PDF, Word, images, and Excel, and can serve Q&A directly through IM channels like WeCom, Feishu, Slack, and Telegram. It is compatible with major LLM providers including OpenAI, DeepSeek, Qwen (Alibaba Cloud), Zhipu, Hunyuan, Gemini, MiniMax, NVIDIA, and Ollama. Its fully modular design allows swapping LLMs, vector databases, and storage backends, with support for local and private cloud deployment ensuring complete data sovereignty. WeKnora also integrates with Langfuse for comprehensive observability into agent reasoning, token usage, and pipeline tracing.

✨ Latest Updates

  • v0.7.2 — Launched the official product documentation site (VitePress; six sections, ~50 pages covering ~360 API endpoints and ~150 environment variables, with standalone Docker/Nginx deployment, quickstart sample data and a local MCP demo); knowledge base folder tree (upload paths stored as first-class data, browse/rename/re-file documents like a file manager); chunk editing with revision history (edit retrieval chunks in the UI, per-version diff and rollback, automatic reindexing, plus custom document metadata); Wiki page revision history (snapshots + line-level diff + one-click rollback + in-browser manual editing); directly loadable file URLs via resource_urls=public / RESOURCE_URL_MODE (third-party apps render images and files without a second authenticated proxy call); Feishu Drive data source and docx sync through the blocks API; batch document tagging; MCP Server 1.1.x (migrated to the mcp 2.x high-level API, official PyPI package tencent-weknora-mcp, new create_knowledge_from_text and list_shared_knowledge_bases for 29 tools total); AWS S3 default credential chain (IAM Role / IRSA); local HTML upload parsing; QQBot markdown replies; new PR CI checks for app / frontend / docreader / mcp-server. Plus large-scale router and modelcontext refactors, rerank and chunking quality work, and broad stability fixes. See CHANGELOG.md.
  • v0.7.1 — New Yunzhijia (云之家) IM integration (WebSocket + image messages + markdown replies); Volcengine rerank provider (with request batching) and Zhipu AI web search provider; platform-scoped API keys for control-plane automation (tenant management, system settings, runtime queues, audit logs); per-KB activity audit trail; FAQ management enhancements (filtering, tagging, export, import tracking); Langfuse OTLP/OTel tracing migration with W3C traceparent propagation; chat header actions with one-click Markdown export and wiki tool results in the references drawer; prompt-cache observability; session channel governance (admin-scoped IM/embed/API sessions); resilient Feishu large-wiki sync; and removal of the legacy Neo4j conversation-memory dependency. Plus broad slug-integrity, SSRF-transport, and state-sync hardening. See CHANGELOG.md.
  • v0.7.0 — Fine-grained scoped API keys & principal model (capability-level grants + per-KB restriction + API integration playground); runtime task-queue observability dashboard & worker-pool governance (per-stage pools + per-model concurrency governors + failed-task inspection/retry); multi-instance storage backends (multiple storage instances per workspace, per-KB binding, default instance); session-scoped temporary attachments (async image/doc parsing + combined limits); question & follow-up suggestions; stable resource registry with LLM-context alias compaction; @Skill / @MCP mentions with scoped agent runtime; mid-conversation MCP OAuth; QQBot & Lark (Feishu International) IM integration; Redis TLS; Requesty model provider + Keenable web search; tenantless provisioning & gated self-service workspaces; admin password reset; knowledge base duplicate flow; weknora CLI v0.10. Plus broad security hardening (SSRF, secret redaction, SQL validation, IDOR). See CHANGELOG.md.
  • v0.6.3 — Website embed widget & Integrations Center (secure-mode token exchange + rate limits); chat experience overhaul (citation popovers, RAG pipeline progress, streaming markdown); document multi-tag & batch reparse; Wiki folders & hierarchy navigation; RSS data source; MCP OAuth2; EPUB / MHTML parsing; agent model-readiness checks; model test debugger; session source filter; workspace deletion UI. See CHANGELOG.md.
  • v0.6.2 — Per-upload process configuration with upload-confirm dialog; document reparse with process_config; weknora CLI v0.9 (bundled Agent Skills, session stop, auth/profile harmonization); KB marquee multi-select; HNSW index for 1024-dim pgvector embeddings; chat resources store refactor; Langfuse-only tracing (Jaeger removed). See CHANGELOG.md.
  • v0.6.1 — Document parsing trace timeline (Langfuse-style span tree with stage-by-stage progress + stop-parse); OpenSearch vector store driver; declarative built-in models via YAML; system admin & consolidated platform settings + audit log; new-user onboarding guide; settings UI redesign; weknora CLI v0.7 / v0.8 (agent-first wire contract, NDJSON, --dry-run); OpenDataLoader + PaddleOCR-VL parsers; MCP server multi-transport (stdio / SSE / HTTP); per-model thinking-mode config; Tencent LKEAP rerank + native Gemini embeddings + MiniMax-M3. See CHANGELOG.md.
  • v0.6.0 — Workspace RBAC (4-tier role matrix Owner / Admin / Contributor / Viewer + per-KB ownership + per-workspace audit log), workspace member management & multi-workspace UX, self-service workspaces; weknora CLI v0.4 GA with mcp serve; KB retrieval fan-out across vector stores; AES-256-GCM credential encryption + docreader gRPC TLS + Token; Zhipu embedder + Huawei OBS; server-side user preferences; Go 1.26.0. See docs/RBAC说明.md and CHANGELOG.md.
  • v0.5.2 — Wiki ingest scales to 40k-document KBs (task queue + DLQ); MCP human-in-the-loop tool approval; Anthropic / Apache Doris / Tencent VectorDB / KS3 / SearXNG backends; adaptive 3-tier chunking with live preview; global ⌘K command palette; Yuque connector + WeChat Mini Program; weknora CLI preview.
  • v0.5.1 — Knowledge-base batch management; workspace-wide IM channels overview; session search + user-scoped pinning; unified Model / Web Search / MCP settings cards; per-agent LLM timeout; desktop workspace switching.
  • v0.5.0 — Wiki Mode GA — agents auto-generate structured, interlinked Markdown wiki pages with a knowledge graph; wiki browser + visual graph in the UI.
  • v0.4.0 — WeKnora Cloud (hosted LLM + parsing); Chrome Extension; ClawHub Skill; WeChat IM; attachment processing; Azure OpenAI / Alibaba OSS; Notion connector; Baidu + Ollama web search; VectorStore management.
  • v0.3.6 — ASR (audio); Feishu data-source auto-sync; OIDC; IM quote-reply context + thread-based sessions; document summarization; Tavily search; parallel tool calling; agent @mention scope restriction.
  • v0.3.5 — Telegram / DingTalk / Mattermost IM; IM slash commands + QA queue; suggested questions; VLM auto-describe MCP tool images; Novita AI; channel tracking.
  • v0.3.4 — WeCom / Feishu / Slack IM; multimodal image support; NVIDIA model API; Weaviate; AWS S3; AES-256-GCM API-key encryption; built-in MCP service; hybrid-search optimization; final_answer tool.
  • v0.3.3 — Parent-child chunking; KB pinning; fallback response; passage cleaning for rerank; storage auto-creation; Milvus.
  • v0.3.2 — Knowledge Search entry; per-source parser & storage engine config; image rendering in local storage; document preview; Volcengine TOS; Mermaid rendering; batch session management; memory graph preview.
  • v0.3.0 — Shared Space; Agent Skills + sandboxed execution; custom agents; Data Analyst agent; thinking mode; Bing / Google web search; API Key auth; Helm chart; Korean i18n; Qdrant.
  • v0.2.0 — Agent Mode (ReACT); multi-type knowledge bases (FAQ + document); conversation strategy config; DuckDuckGo web search; MCP tool integration; new UI with agent mode switching; MQ async task management.

📱 Interface Showcase

💬 Intelligent Q&A Conversation
📖 Wiki Browser 🕸️ Wiki Knowledge Graph
🕘 Wiki Page Revision History & Rollback ✂️ Chunk Editing & Revision History
📁 Folder Tree & Batch Operations 🤖 Agent Mode · Tool Call Process
🔭 Observability · Langfuse Tracing

🏗️ Architecture

Fully modular pipeline from document parsing, vectorization, and retrieval to LLM inference — every component is swappable and extensible. Supports local / private cloud deployment with full data sovereignty and a zero-barrier Web UI for quick onboarding.

🧩 Feature Overview

Intelligent Conversation

Capability Details
Intelligent Reasoning ReACT progressive multi-step reasoning, autonomously orchestrating knowledge retrieval, MCP tools, and web search
Quick Q&A RAG-based Q&A over knowledge bases for fast and accurate answers
Wiki Mode Agent-driven auto-generation of structured, interlinked markdown Wiki pages from raw documents; in-browser manual editing, page revision history, line-level diff and one-click rollback
Tool Calling Built-in tools, MCP tools (incl. OAuth2 remote services, mid-conversation OAuth), web search; @Skill / @MCP mentions to scope the agent runtime per turn
Conversation Strategy Online Prompt editing, retrieval threshold tuning, multi-turn context awareness, per-agent citation output toggle
Suggested Questions Auto-generated question suggestions and after-answer follow-ups based on knowledge base content
Temporary Attachments Session-scoped image / document uploads with async parsing for one-off Q&A, with a combined image + attachment limit
Citations & RAG Progress Inline citation popovers and a references drawer (web / KB source distinction), shared markdown rendering, and stage-by-stage RAG pipeline progress in chat
Session Management Filter and group sidebar sessions by source (Web / IM / Embed), with inline session-title rename

Knowledge Management

Capability Details
Knowledge Base Types FAQ / Document / Wiki with folder import, URL import, multi-tag management, and online entry
Folder Tree Folder uploads keep their original directory structure, with a sidebar tree for browsing, folder rename, and re-filing documents into another folder
Chunk Editing & Revisions Edit retrieval chunks directly in the UI with per-version snapshots, diff and one-click rollback, and automatic reindexing after an edit; generated questions can be added, edited, deleted and regenerated; custom document metadata supported
Per-Upload Process Config Override parser, chunking, multimodal (VLM / ASR), graph extraction, and question generation per upload batch via upload-confirm dialog or process_config API; reparse with new settings
Batch Reparse Re-queue parsing for multiple documents at once with optional per-batch process_config
Data Source Import Auto-sync from Feishu wiki / Feishu Drive / Lark / Notion / Yuque / RSS feeds (more data sources coming soon); incremental and full sync
Document Formats PDF / Word / Txt / Markdown / HTML / EPUB / MHTML / Images / CSV / Excel / PPT / JSON
Retrieval Strategies BM25 sparse / Dense retrieval / GraphRAG / parent-child chunking / HNSW-accelerated pgvector (1024-dim) / multi-dimensional indexing
Batch Selection & Tagging Marquee drag-select multiple documents in the KB list for batch reparse and batch tagging (common tags pre-selected)
E2E Testing Full-pipeline visualization with recall hit rate, BLEU / ROUGE metric evaluation

Integrations & Extensions

Capability Details
LLMs OpenAI / Azure OpenAI / Anthropic (Claude) / DeepSeek / Qwen (Alibaba Cloud) / Zhipu / Hunyuan / Doubao (Volcengine) / Gemini / MiniMax / NVIDIA / Novita AI / SiliconFlow / OpenRouter / Requesty / Ollama
Embeddings Ollama / BGE / GTE / Zhipu / OpenAI-compatible APIs
Vector DBs PostgreSQL (pgvector) / Elasticsearch / OpenSearch / Milvus / Weaviate / Qdrant / Apache Doris / Tencent VectorDB
Object Storage Local / MinIO / AWS S3 (IAM Role / IRSA default credential chain) / Volcengine TOS / Alibaba Cloud OSS / Kingsoft Cloud KS3 / Huawei Cloud OBS; multiple storage instances per workspace with per-KB binding and a default instance
IM Channels WeCom / Feishu / Lark (Feishu International) / QQBot / Slack / Telegram / DingTalk / Mattermost / WeChat / Yunzhijia
Website Embed Publish agents via embed widget with domain allowlists, rate limits, and secure-mode token exchange
Web Search DuckDuckGo / Bing / Google / Tavily / Baidu / Ollama / SearXNG / Keenable / Zhipu AI
API Integration Scoped API keys (capability-level grants + per-KB restriction + throttled last-used tracking) with an API integration playground; MCP OAuth and embed sessions isolated per principal; resource_urls=public returns directly loadable file/image URLs, removing the second authenticated proxy call
MCP Server Official PyPI package tencent-weknora-mcp with 29 tools over stdio / SSE / HTTP transports

Platform

Capability Details
Deployment Local / Docker / Kubernetes (Helm) with private and offline support
UI Web UI / RESTful API / CLI (weknora) / Chrome Extension / Website Embed Widget / WeChat Mini Program
Access Control Workspace RBAC with 4-tier role matrix (Owner / Admin / Contributor / Viewer), per-KB resource ownership, per-workspace audit log, invite-only workspaces, tenantless provisioning & gated self-service workspace creation, admin password reset (session revocation), cross-workspace superuser, scoped API keys
Security AES-256-GCM at-rest encryption for API keys and MCP / data-source credentials with graceful key rotation; gRPC TLS + Token between app and docreader; Redis TLS; SSRF-safe HTTP client (data sources, URL import, redirect chains); secret redaction in responses; sandbox isolation for agent skills
Observability Integrated Langfuse (sole tracing backend) for ReAct loops, token tracking, tool calls, and pipeline tracing; built-in Langfuse-style document parsing trace timeline with stage-by-stage progress; system-admin runtime task-queue dashboard (queue depth, per-model concurrency, failed-task inspection & manual retry)
Task Management MQ async tasks with per-stage worker-pool governance (core / post-process / enrichment / maintenance + elastic shared pool, plus an independent Wiki pool) and per-model background concurrency governors; automatic database migration on version upgrade
Model Management Centralized config, declarative built-in models via YAML, per-knowledge-base model selection, per-model thinking-mode and embedding-dimension overrides, interactive model test debugger, multi-workspace built-in model sharing, WeKnora Cloud hosted models and parsing

🧩 Chrome Extension

WeKnora Chrome Extension lets you capture web content directly into your WeKnora knowledge base. Select text, images, or entire pages in the browser and save them as knowledge entries with one click — no copy-paste or file upload needed.

📱 WeChat Mini Program

The WeKnora Mini Program provides a lightweight mobile client for configuring WeKnora API access, selecting knowledge bases, importing URLs, and asking knowledge chat from WeChat.

🦞 ClawHub Skill

WeKnora ClawHub Skill is a WeKnora skill published on the ClawHub platform. Once installed, it enables document import (file / URL / Markdown), hybrid search (vector + keyword) across knowledge bases, and knowledge entry management — all through the WeKnora REST API.

  • Document Import — Upload files, import web pages, or write Markdown knowledge via the agent
  • Hybrid Search — Search within or across knowledge bases with vector + keyword retrieval
  • Knowledge Management — List, browse, edit, and delete knowledge entries programmatically

⌨️ Command-Line Interface

weknora is the official CLI for driving the API from a terminal or an AI
agent. It is agent-first: every command emits a stable JSON envelope by
default (with typed error codes mapped to exit codes), and --format text
renders for humans. It also serves a curated MCP tool surface
(weknora mcp serve) and ships bundled Agent Skills.

bash
weknora profile add prod --host https://kb.example.com --use
weknora auth login
weknora kb list
weknora link --kb my-knowledge-base    # bind the current directory
weknora doc upload notes.md
weknora chat "summarise the design doc"

For headless / CI use, set WEKNORA_API_KEY + WEKNORA_HOST and skip
auth login entirely — no credentials written to disk.

See cli/README.md for install + 5-minute quickstart and
cli/AGENTS.md for the operational contract AI agents rely on.

🚀 Getting Started

🛠 Prerequisites

📦 Installation & Launch

bash
git clone https://github.com/Tencent/WeKnora.git
cd WeKnora
cp .env.example .env   # Edit .env as needed, see comments in the file
docker compose pull     # Pull the latest images
docker compose up -d    # Start core services

Once started, visit http://localhost to get started.

To use a local Ollama model, run ollama serve > /dev/null 2>&1 & first.

🔄 Upgrading

If you already have WeKnora running and downloaded a newer release:

bash
# Set WEKNORA_VERSION in .env to the target release (e.g. 0.7.0), or keep latest
docker compose pull     # Pull images matching WEKNORA_VERSION
docker compose up -d    # Recreate containers with new images

docker compose up -d alone reuses locally cached images and may leave the UI version out of sync with the release you downloaded.

🔧 Optional Services (Docker Compose Profiles)

Add --profile flags to enable additional components. Multiple profiles can be combined:

Profile Description Command
(default) Core services docker compose pull && docker compose up -d
full All features docker compose --profile full pull && docker compose --profile full up -d
neo4j Knowledge Graph (Neo4j) docker compose --profile neo4j pull && docker compose --profile neo4j up -d
minio Object Storage (MinIO) docker compose --profile minio pull && docker compose --profile minio up -d
langfuse Tracing (Langfuse) docker compose --profile langfuse pull && docker compose --profile langfuse up -d

Combine profiles: docker compose --profile neo4j --profile minio pull && docker compose --profile neo4j --profile minio up -d

Stop services: docker compose down

🌐 Service URLs

Service URL
Web UI http://localhost
Backend API http://localhost:8080
Langfuse Tracing http://localhost:3000

MCP Server

Please refer to the MCP Configuration Guide for the necessary setup.

🔌 Using WeChat Dialog Open Platform

WeKnora serves as the core technology framework for the WeChat Dialog Open Platform, providing a more convenient usage approach:

  • Zero-code Deployment: Simply upload knowledge to quickly deploy intelligent Q&A services within the WeChat ecosystem, achieving an "ask and answer" experience
  • Efficient Question Management: Support for categorized management of high-frequency questions, with rich data tools to ensure accurate, reliable, and easily maintainable answers
  • WeChat Ecosystem Integration: Through the WeChat Dialog Open Platform, WeKnora's intelligent Q&A capabilities can be seamlessly integrated into WeChat Official Accounts, Mini Programs, and other WeChat scenarios, enhancing user interaction experiences

📘 API Reference

Official product documentation: website-docs/ — the complete documentation set organized as Getting Started → Architecture → Features → API → Clients → Development, covering ~360 API endpoints, ~150 environment variables, and 9 extension points. The directory is also a VitePress site: run cd website-docs && npm install && npm run dev to preview locally, or deploy it standalone with the Dockerfile inside.

Troubleshooting FAQ: Troubleshooting FAQ

Detailed API documentation is available at: API Docs

Product plans and upcoming features: Roadmap

🧭 Developer Guide

If you need to frequently modify code, you don't need to rebuild Docker images every time! Use fast development mode:

bash
# Start infrastructure
make dev-start

# Start backend (new terminal)
make dev-app

# Start frontend (new terminal)
make dev-frontend

Development Advantages:

  • ✅ Frontend modifications auto hot-reload (no restart needed)
  • ✅ Backend modifications quick restart (5-10 seconds, supports Air hot-reload)
  • ✅ No need to rebuild Docker images
  • ✅ Support IDE breakpoint debugging

Detailed Documentation: Development Environment Quick Start

🤝 Contributing

Welcome to submit Issues or Pull Requests.

Process: Fork → Create branch → Commit changes → Open PR

Standards: Format code with gofmt, follow Conventional Commits (feat: / fix: / docs: / test: / refactor:)

Validation

For a focused PR, validate the changed scope first:

bash
git fetch origin main
git diff --check origin/main...HEAD
golangci-lint run --new-from-rev=origin/main ./...
go test ./path/to/changed/package -count=1

Run gofmt on changed Go files before committing. For frontend changes, run the relevant tests from frontend/ and use npm run type-check when the change affects TypeScript or Vue components.

The full maintainer gate remains:

bash
make fmt
make lint
make test

make fmt formats the entire Go repository, so run it only with a clean worktree and review the resulting diff. Some full-suite tests require local infrastructure or service configuration. If a full check fails for an unrelated baseline or environment reason, include the exact command and failure in the PR while still providing passing targeted tests for your change.

🔒 Security Notice

Important: Starting from v0.1.3, WeKnora includes login authentication functionality to enhance system security. For production deployments, we strongly recommend:

  • Deploy WeKnora services in internal/private network environments rather than public internet
  • Avoid exposing the service directly to public networks to prevent potential information leakage
  • Configure proper firewall rules and access controls for your deployment environment
  • Regularly update to the latest version for security patches and improvements

👥 Contributors

Thanks to these excellent contributors:

📄 License

This project is licensed under the MIT License.
You are free to use, modify, and distribute the code with proper attribution.

📈 Project Statistics

File: cli/README.md

weknora — WeKnora CLI

A command-line interface for the WeKnora RAG knowledge-base server. Lets you
authenticate, manage knowledge bases and documents, run hybrid search, and
ask streaming RAG questions from your terminal or from an AI agent.

bash
$ weknora --help
Command-line client for the WeKnora RAG server. Manage knowledge bases
and documents, run hybrid search, chat with grounded answers, or expose
a curated read-only MCP tool surface for AI agents.

Available Commands:
  agent       Manage custom agents (CRUD + status/check)
  api         Make a raw API request to the WeKnora server
  auth        Manage authentication credentials and profiles
  chat        Ask a streaming RAG question against a knowledge base
  chunk       Manage document chunks (RAG retrieval debug)
  completion  Generate the autocompletion script for the specified shell
  config      Inspect the CLI's resolved configuration
  doc         Manage documents in a knowledge base
  doctor      Run 4 self-checks: base URL, auth, server version, credential storage
  exit-codes  Exit code matrix and the agent action for each
  help        Help about any command
  kb          Manage knowledge bases
  link        Bind the current directory to a knowledge base
  mcp         Run weknora as a Model Context Protocol server
  message     Inspect and manage messages inside chat sessions
  model       Manage models (list / view / create / update / delete)
  profile     Manage CLI profiles (named connection targets)
  schema      Machine-readable contract for a command (or the whole surface)
  search      Search across chunks, knowledge bases, documents, or sessions
  session     Manage chat sessions
  skills      List and install the bundled Agent Skills
  unlink      Remove the directory's knowledge-base binding
  version     Show CLI build metadata

The wire contract for AI agents is documented below.
For contributing to the CLI source, see AGENTS.md.

Install

From source

Requires Go 1.26+.

bash
git clone https://github.com/Tencent/WeKnora.git
cd WeKnora/cli
go build -o weknora .
sudo mv weknora /usr/local/bin/   # or anywhere on $PATH

Pre-built binaries

Building from source is the supported install method today. Pre-built binaries,
go install, and a Homebrew formula are planned to accompany a tagged release;
until then, use the from-source build above.

5-minute quickstart

text
/* Detailed source-code truncated for AI context efficiency. */

Agent quick start

For AI agents (any MCP-capable host) integrating WeKnora:

  1. Install: build from source (see Install)
  2. Authenticate. In a sandbox / CI, the stateless path needs no auth login
    and writes nothing to disk — just set two env vars:
    bash
    export WEKNORA_API_KEY="sk-…"   # or WEKNORA_TOKEN for a bearer JWT
export WEKNORA_HOST="https://kb.example.com"
weknora kb list                 # already authenticated
    Or, for a persisted local profile:
    bash
    weknora profile add prod --host <server-url> --use
weknora auth login
  3. Register MCP in the host's MCP config:
    json
    {"mcpServers": {"weknora": {"command": "weknora", "args": ["mcp", "serve"]}}}
  4. Read the wire contract before
    parsing --format json output.
  5. Read the exit-10 anti-patterns before
    any destructive call.

Bundled Agent Skills. This CLI ships Agent Skills
under skills/ that teach an agent to drive WeKnora without trial and error:

  • weknora-shared — read first: auth/profile
    sequence, --kb resolution, the JSON-envelope + exit-code contract, the exit-10
    protocol, --dry-run, and CLI-vs-MCP selection.
  • weknora-rag-search — when to use chat
    vs session ask vs search chunks, plus retrieval gotchas.

Install them with the CLI (the skills are embedded in the binary, no checkout
needed):

bash
weknora skills install                      # writes to ~/.claude/skills
weknora skills install --dir <agent-skills-dir> --force   # other agents / overwrite
weknora skills list --format json           # what would be installed

Existing files are left untouched without --force; --dry-run previews the
file list. Each skill's frontmatter records the CLI version it was
tested_against; a CI parity test (internal/skillparity) fails if a skill
ever references a command, flag, or MCP tool the CLI no longer has.

Multi-profile

profile.* manages profile records (positional <name>); auth.* operates
on the active profile (override per-invocation with the global --profile
flag). Create a profile first, then authenticate it:

bash
weknora profile add prod    --host https://prod.example.com --use     # add + switch
weknora auth login                                                    # authenticate active (prod)

weknora profile add staging --host https://staging.example.com        # add (stays inactive)
echo "sk-..." | weknora --profile staging auth login --with-token     # authenticate staging

weknora auth list
weknora profile use prod                                              # switch back

Credentials are persisted to your OS keyring (Keychain on macOS, libsecret on
Linux, Wincred on Windows) when available, otherwise to a 0600-mode file
under $XDG_CONFIG_HOME/weknora/secrets/. The active profile lives in
~/.config/weknora/config.yaml.

To remove a profile's stored credentials:

bash
weknora auth logout                       # active profile
weknora --profile staging auth logout     # specific profile
weknora auth logout --all

Wire contract

Designed to be AI-agent-first. Stable across minor releases; breaking
changes announced in the changelog and the corresponding
weknora --version bump. This section is the human overview; the complete,
authoritative contract (envelope field stability, error taxonomy, streaming,
confirmation and dry-run protocols) lives in AGENTS.md.

Streams

  • stdout is the data channel: bare JSON with --format json, or
    human-formatted output. Never carries error text.
  • stderr is logs, progress, warnings, and errors. A non-empty
    stderr does not mean failure — read the exit code.

JSON output

Every command supports --format json, wrapping the resource in the
symmetric envelope {ok, data, meta?} — data is an array for list /
search, a single object for view and write outcomes. --jq runs
against the whole envelope, so reach into list items with .data[]:

bash
weknora kb list --format json                              # {"ok":true,"data":[{"id":"kb_x",…}],"meta":{"count":1,…}}
weknora kb view kb_x --format json                         # {"ok":true,"data":{"id":"kb_x","name":"Eng",…}}
weknora kb list --format json --jq '.data[] | {id, name}'  # project listed fields out of each item
weknora kb list --format json --jq '.data[].id'            # ids only
weknora kb list --format json --jq '.meta.count'           # number returned

--format ndjson is also accepted for streaming list commands; each
element is emitted as its own JSON line. --format json is the default
regardless of TTY — running weknora kb list | jq works without an
explicit flag. Use --format text for human-readable output.

Errors

On failure, stdout stays empty and the typed error goes to stderr in
this format:

text
<code.namespace>: <message>[: <wrapped cause>]
hint: <actionable next-step>

Example:

text
auth.unauthenticated: fetch current user: HTTP error 401: ...
hint: run `weknora auth login`

Under --format json the same failure is the typed error envelope on stderr
({ok:false, error:{type, exit_code, hint?, retry_argv?, …}}) — see
AGENTS.md §1.4 for the field-by-field contract and the full code
taxonomy.

Exit codes

Code Meaning Agent action
0 success continue
1 typed local.* / operation.failed / unclassified read stderr, decide retry/abort
2 flag / argument validation error re-check weknora <cmd> --help
3 auth.* (token missing / expired / forbidden) re-auth, then retry
4 resource.not_found verify the resource id
5 input.* (other than confirmation_required) adjust args, retry
6 server.rate_limited back off, retry
7 server.* / network.* transient — retry with backoff
10 input.confirmation_required (high-risk write) ask the human, retry with -y only after explicit approval
124 operation.timeout (e.g. doc wait --timeout reached) raise --timeout or check the underlying job
130 cancelled by signal — typed operation.cancelled errors exit 1; main.go promotes the process exit to 130 when the root context was signal-cancelled (SIGINT / SIGTERM) stop, do not retry

Run weknora exit-codes for the machine-readable matrix (JSON); weknora help exit-codes for the human-readable table.

Exit 10 is the wire-level signal for "destructive write needs
explicit confirmation". Pass -y/--yes on kb delete /
doc delete (including --all --kb=) / session delete /
profile remove (on the current profile) / agent delete /
chunk delete when running headless.
Never auto-add -y without the user's explicit go-ahead — exit 10
is the guard against unintended writes.

Other AI-agent ergonomics

  • For chat / session ask in AI-agent contexts, pass --format json for a
    bounded answer-event envelope. Add --reference for indexed citations,
    --verbose for reasoning/tools/lifecycle events, or --format ndjson for
    the unmodified raw stream.
  • --format json composes with the global --profile <name> for
    single-shot profile overrides without disk writes.
  • weknora mcp serve exposes a curated read-only tool surface over
    stdio MCP for any MCP-compatible client.
  • weknora schema enumerates every command with its used_for, and
    weknora schema <cmd path> (e.g. weknora schema doc update) prints that
    command's full contract — used_for, required_flags, examples,
    output, warnings, risk, and local flags — as the standard envelope,
    so an agent can discover the surface without scraping --help prose.

Advanced operations not exposed as flags

WeKnora CLI exposes top use cases as polished commands; deep
configuration goes through the raw HTTP passthrough. CLI flag coverage
targets common workflows, not 1:1 API parity. Examples of deep
operations that intentionally go through weknora api:

  • Tuning a KB's nested config — chunking strategy, summary model,
    multimodal extraction defaults, FAQ thresholds, VLM model. Use
    weknora api PUT /api/v1/knowledge-bases/ --input - with a JSON
    body matching the server's UpdateKnowledgeBaseRequest. (Note: the
    storage provider is set once at create time via
    kb create --storage-provider <name> and is not updatable.)
  • Per-request chat parameters — multi-KB scope, summary model
    override, image attachments, web search toggle. Use weknora api POST /api/v1/knowledge-chat/<session-id> --input -.
  • Per-request session ask --agent overrides — same shape via
    weknora api POST /api/v1/agent-chat/<session-id> --input -.
  • Operations without a CLI verb — register / change-password /
    OIDC flows, organization / sharing endpoints, tenant management.

weknora api --help documents the raw passthrough. Run
weknora doctor first to verify auth and base URL.

Dry-run preview

Add --dry-run to any mutation command to preview the would-be action without executing it. Useful for verifying flag/arg parsing before committing to a destructive operation, or for agent-side action planning.

bash
# Preview a kb create without actually creating
weknora kb create --name "test-kb" --description "for review" --dry-run

# Output (single line; pretty-printed here for readability):
# {
#   "ok": true,
#   "meta": {
#     "dry_run": true,
#     "plan": {
#       "action": "kb.create",
#       "args": {"name": "test-kb", "description": "for review"}
#     }
#   }
# }
# Exit code: 0

dry-run is offline: no network calls, no file IO, no credential touches. Works without an active profile.

For destructive commands, dry-run does NOT trigger the exit-10 confirmation flow:

bash
weknora kb delete kb_xxxx --dry-run   # exit 0, no prompt
weknora kb delete kb_xxxx             # exit 10, prompts for -y

For the api command, dry-run requires explicit write method (POST/PUT/PATCH/DELETE); GET returns FlagError:

bash
echo '{"name":"foo"}' | weknora api -X POST /api/v1/knowledge-bases --input - --dry-run   # OK
weknora api /api/v1/knowledge-bases --dry-run                                              # exit 2: requires explicit -X

Resuming streams

The weknora session resume command resumes an SSE event stream for an existing assistant message. Useful for network-blip recovery or polling long-running agent invocations:

bash
# Original streaming call captures session_id + message_id from init event:
weknora session ask "..." --agent ag_xxxx --format ndjson | tee /tmp/stream.ndjson
# {"type":"init","session_id":"sess_abc","message_id":"msg_xyz"}
# ... events flow ...
# [network blip]

# Resume the same stream:
weknora session resume sess_abc --message msg_xyz
# Server REPLAYS all stored events from the start, then tails new ones.
# Agent must dedupe (by message_id or event hash) to avoid double-processing.

Tool-approval unlock chain

An agent run may pause the stream on a tool-approval event until a human approves or rejects the pending tool call. The unlock sequence:

bash
# 1. Stream pauses with a tool-approval event carrying a pending_id.
# 2. Surface the pending tool call to the user; get explicit go-ahead.
weknora session tool-approval resolve pend_xxx -y                      # approve
# weknora session tool-approval resolve pend_xxx --reject --reason "..." -y  # reject
# 3. Resume the stream — server replays + tails from where the run was blocked.
weknora session resume sess_abc --message msg_xyz

Pass --modified-args '{"key":"value"}' to replace tool arguments on approve (must be a non-empty JSON object). Never auto-pass -y — the approval is the exit-10 human-in-the-loop gate.

Server-side buffer TTL: 1 hour for redis mode; process lifetime for memory mode (default). After TTL, expect local.sse_stream_aborted typed error.

See cli/AGENTS.md "Stream recovery" section for the full agent contract.

Health check

Run weknora doctor for a 4-status diagnostic (OK / warn / fail /
skip) covering base URL reachability, authentication, server-CLI
version skew, and credential storage backend. Add --format json for
machine-readable output, --offline to skip network checks.

For per-resource verification, the status / check verb pair gives
a fast vs deep choice:

Verb Cost Use
weknora kb status <kb-id> 1 HTTP live counts / processing flag
weknora kb check <kb-id> 1+N HTTP adds failed_count via doc-list page-walk
weknora agent status <agent-id> 1 HTTP reachable / model_id
weknora agent check <agent-id> 1+N HTTP also probes every KB in the agent's scope

weknora doc wait <doc-id> [<doc-id>...] blocks until each document
reaches a terminal parse_status (completed or failed). Exit codes:
0 (all completed), 1 (any failed), 124 (--timeout reached), 130
(Ctrl-C / SIGTERM). Multi-target is polled concurrently (max 5 in
flight; pipe through xargs -P for more).

Development

bash
# Run unit + contract tests
go test ./...

# Run the real-server e2e suite (requires WEKNORA_E2E_HOST + token env vars)
go test -tags acceptance_e2e ./acceptance/e2e/...

# Static analysis
go vet ./...

CI (.github/workflows/cli.yml) runs build + unit + contract tests on Linux /
macOS / Windows × Go 1.26, path-filtered to changes under cli/.

Contributing / Reporting issues

  • Bugs and feature requests: file an issue at
    github.com/Tencent/WeKnora/issues.
  • Security disclosures: see the repository-level
    SECURITY.md. Do not file public issues for
    security findings.
  • Pull requests: the developer guide for editing the CLI lives in
    AGENTS.md (build / test / command-surface design SOP /
    CRUD flag conventions). Run go test ./... -race -count=1 and go vet ./...
    before submitting.

License

MIT — see the repository LICENSE.

File: client/README.md

WeKnora HTTP 客户端

这个包提供了与WeKnora服务进行交互的客户端库，支持所有基于HTTP的接口调用，使其他模块更方便地集成WeKnora服务，无需直接编写HTTP请求代码。

主要功能

该客户端包含以下主要功能模块：

  1. 会话管理：创建、获取、更新和删除会话
  2. 知识库管理：创建、获取、更新和删除知识库
  3. 知识管理：添加、获取和删除知识内容
  4. 空间管理：空间的CRUD操作
  5. 知识问答：支持普通问答和流式问答
  6. Agent问答：支持基于Agent的智能问答，包含思考过程、工具调用和反思
  7. 分块管理：查询、更新和删除知识分块
  8. 消息管理：获取和删除会话消息
  9. 模型管理：创建、获取、更新和删除模型

使用方法

创建客户端实例

go
import (
    "context"
    "github.com/Tencent/WeKnora/client"
    "time"
)

// 创建客户端实例
apiClient := client.NewClient(
    "http://api.example.com", 
    client.WithToken("your-auth-token"),
    client.WithTimeout(30*time.Second),
)

空间配置

客户端支持通过 WithTenantID 设置默认空间，请求时会自动携带 X-Tenant-ID 请求头：

go
tenantID := uint64(10000)
apiClient := client.NewClient(
    "http://api.example.com",
    client.WithToken("your-auth-token"),
    client.WithTenantID(tenantID),
)

如果某个请求需要临时切换空间，可以在 context 中设置 TenantID，值可以是 uint64、*uint64 或字符串形式的数字，客户端会优先使用该值：

go
ctx := context.WithValue(context.Background(), "TenantID", uint64(10000))
// 调用任意客户端方法时传入 ctx，即可切换到空间 10000

示例：创建知识库并上传文件

go
// 创建知识库
kb := &client.KnowledgeBase{
    Name:        "测试知识库",
    ChunkingConfig: client.ChunkingConfig{
        ChunkSize:    500,
        ChunkOverlap: 50,
        Separators:   []string{"\n\n", "\n", ". ", "? ", "! "},
    },
    ImageProcessingConfig: client.ImageProcessingConfig{
        ModelID: "image_model_id",
    },
    EmbeddingModelID: "embedding_model_id",
    SummaryModelID:   "summary_model_id",
}

kb, err := apiClient.CreateKnowledgeBase(context.Background(), kb)
if err != nil {
    // 处理错误
}

// 上传知识文件并添加元数据
metadata := map[string]string{
    "source": "local",
    "type":   "document",
}
knowledge, err := apiClient.CreateKnowledgeFromFile(context.Background(), kb.ID, "path/to/file.pdf", metadata)
if err != nil {
    // 处理错误
}

示例：创建会话并进行问答

go
// 创建会话
sessionRequest := &client.CreateSessionRequest{
    KnowledgeBaseID: knowledgeBaseID,
    SessionStrategy: &client.SessionStrategy{
        MaxRounds:        10,
        EnableRewrite:    true,
        FallbackStrategy: "fixed_answer",
        FallbackResponse: "抱歉，我无法回答这个问题",
        EmbeddingTopK:    5,
        KeywordThreshold: 0.5,
        VectorThreshold:  0.7,
        RerankModelID:    "rerank_model_id",
        RerankTopK:       3,
        RerankThreshold:  0.8,
        SummaryModelID:   "summary_model_id",
    },
}

session, err := apiClient.CreateSession(context.Background(), sessionRequest)
if err != nil {
    // 处理错误
}

// 普通问答
answer, err := apiClient.KnowledgeQA(context.Background(), session.ID, &client.KnowledgeQARequest{
    Query: "什么是人工智能?",
})
if err != nil {
    // 处理错误
}

// 流式问答
err = apiClient.KnowledgeQAStream(context.Background(), session.ID, &client.KnowledgeQARequest{
    Query:            "什么是机器学习?",
    KnowledgeBaseIDs: []string{knowledgeBaseID}, // 可选：指定知识库
    WebSearchEnabled: false,                      // 可选：是否启用网络搜索
}, func(response *client.StreamResponse) error {
    // 处理每个响应片段
    fmt.Print(response.Content)
    return nil
})
if err != nil {
    // 处理错误
}

示例：Agent智能问答

Agent问答提供更强大的智能对话能力，支持工具调用、思考过程展示和自我反思。

text
/* Detailed source-code truncated for AI context efficiency. */

Agent事件类型说明

事件类型 说明 何时触发
AgentResponseTypeThinking Agent思考过程 Agent分析问题和制定计划时
AgentResponseTypeToolCall 工具调用 Agent决定使用某个工具时
AgentResponseTypeToolResult 工具执行结果 工具执行完成后
AgentResponseTypeReferences 知识引用 检索到相关知识时
AgentResponseTypeAnswer 最终答案 Agent生成回答时（流式）
AgentResponseTypeReflection 自我反思 Agent评估自己的回答时
AgentResponseTypeError 错误 发生错误时

Agent问答测试工具

我们提供了一个交互式命令行工具用于测试Agent功能：

bash
cd client/cmd/agent_test
go build -o agent_test
./agent_test -url http://localhost:8080 -kb <knowledge_base_id>

该工具支持：

  • 创建和管理会话
  • 交互式Agent问答
  • 实时显示所有Agent事件
  • 性能统计和调试信息

详细使用说明请参考 client/cmd/agent_test/README.md。

Agent问答的高级用法

更多高级用法示例，请参考 agent_example.go 文件，包括：

  • 基础Agent问答
  • 工具调用跟踪
  • 知识引用捕获
  • 完整事件跟踪
  • 自定义错误处理
  • 流取消控制
  • 多会话管理
text
### 示例：管理模型

```go
// 创建模型
modelRequest := &client.CreateModelRequest{
    Name:        "测试模型",
    Type:        client.ModelTypeChat,
    Source:      client.ModelSourceInternal,
    Parameters: client.ModelParameters{
        "temperature": 0.7,
        "top_p":       0.9,
    },
    IsDefault: true,
}
model, err := apiClient.CreateModel(context.Background(), modelRequest)
if err != nil {
    // 处理错误
}

// 列出所有模型
models, err := apiClient.ListModels(context.Background())
if err != nil {
    // 处理错误
}

示例：管理知识分块

go
// 列出知识分块
chunks, total, err := apiClient.ListKnowledgeChunks(context.Background(), knowledgeID, 1, 10)
if err != nil {
    // 处理错误
}

// 更新分块
updateRequest := &client.UpdateChunkRequest{
    Content:   "更新后的分块内容",
    IsEnabled: true,
}
updatedChunk, err := apiClient.UpdateChunk(context.Background(), knowledgeID, chunkID, updateRequest)
if err != nil {
    // 处理错误
}

示例：重新解析知识

go
// 重新解析知识（删除现有内容并重新解析）
// 适用场景：
// 1. 原始解析失败，需要重试
// 2. 更新了解析配置（如分块策略、多模态设置等），需要重新解析
// 3. 知识内容已更新，需要刷新解析结果

knowledge, err := apiClient.ReparseKnowledge(context.Background(), knowledgeID)
if err != nil {
    // 处理错误
}

// 知识将进入 "pending" 状态，异步重新解析
fmt.Printf("Knowledge ID: %s\n", knowledge.ID)
fmt.Printf("Parse Status: %s\n", knowledge.ParseStatus)      // "pending"
fmt.Printf("Enable Status: %s\n", knowledge.EnableStatus)    // "disabled"

// 可以轮询检查解析状态
for {
    time.Sleep(5 * time.Second)
    knowledge, err := apiClient.GetKnowledge(context.Background(), knowledgeID)
    if err != nil {
        // 处理错误
    }
    
    if knowledge.ParseStatus == "completed" {
        fmt.Println("Knowledge re-parsing completed!")
        break
    } else if knowledge.ParseStatus == "failed" {
        fmt.Printf("Knowledge re-parsing failed: %s\n", knowledge.ErrorMessage)
        break
    }
}

示例：取消解析

go
// 取消正在进行的解析任务（资源紧张 / 上传错误文件时使用）
// - 已经 completed / failed 的知识不能取消
// - 已写入的分块/索引会保留，可后续调用 ReparseKnowledge 重新解析

knowledge, err := apiClient.CancelKnowledgeParse(context.Background(), knowledgeID)
if err != nil {
    // 处理错误
}
fmt.Printf("Parse Status: %s\n", knowledge.ParseStatus) // "cancelled"

示例：查看文档解析追踪（Span 树）

go
// 获取文档解析流水线的 Span 树（root → stage → subspan）
// - attempt 传 0 表示获取最新一次解析尝试
// - 始终返回 5 个标准阶段：docreader / chunking / embedding / multimodal / postprocess
trace, err := apiClient.GetKnowledgeProcessingSpans(context.Background(), knowledgeID, 0)
if err != nil {
    // 处理错误
}
fmt.Printf("ParseStatus=%s CurrentStage=%s\n", trace.ParseStatus, trace.CurrentStage)
for _, stage := range trace.Trace.Children {
    fmt.Printf("- %s: %s (%dms)\n", stage.Name, stage.Status, stage.DurationMs)
}

示例：获取会话消息

go
// 获取最近消息
messages, err := apiClient.GetRecentMessages(context.Background(), sessionID, 10)
if err != nil {
    // 处理错误
}

// 获取指定时间之前的消息
beforeTime := time.Now().Add(-24 * time.Hour)
olderMessages, err := apiClient.GetMessagesBefore(context.Background(), sessionID, beforeTime, 10)
if err != nil {
    // 处理错误
}

完整示例

请参考 example.go 文件中的 ExampleUsage 函数，其中展示了客户端的完整使用流程。

File: client/README_EN.md

WeKnora HTTP Client

This package provides a client library for interacting with WeKnora services, supporting all HTTP-based interface calls, making it easier for other modules to integrate with WeKnora services without having to write HTTP request code directly.

Main Features

The client includes the following main functional modules:

  1. Session Management: Create, retrieve, update, and delete sessions
  2. Knowledge Base Management: Create, retrieve, update, and delete knowledge bases
  3. Knowledge Management: Add, retrieve, and delete knowledge content
  4. Workspace Management: CRUD operations for workspaces
  5. Knowledge Q&A: Supports regular Q&A and streaming Q&A
  6. Chunk Management: Query, update, and delete knowledge chunks
  7. Message Management: Retrieve and delete session messages
  8. Model Management: Create, retrieve, update, and delete models
  9. Evaluation Function: Start evaluation tasks and get evaluation results

Usage

Creating Client Instance

go
import (
    "context"
    "github.com/Tencent/WeKnora/client"
    "time"
)

// Create client instance
apiClient := client.NewClient(
    "http://api.example.com", 
    client.WithToken("your-auth-token"),
    client.WithTimeout(30*time.Second),
)

Workspace Configuration

You can set a default workspace with WithTenantID; the client will automatically send the X-Tenant-ID header:

go
tenantID := uint64(10000)
apiClient := client.NewClient(
    "http://api.example.com",
    client.WithToken("your-auth-token"),
    client.WithTenantID(tenantID),
)

If a single request needs a different workspace, set TenantID in the request context. The value can be a uint64, *uint64, or a numeric string, and it will take precedence over the client default:

go
ctx := context.WithValue(context.Background(), "TenantID", uint64(10000))
// Pass ctx into any client method to switch to workspace 10000 for that request

Example: Create Knowledge Base and Upload File

go
// Create knowledge base
kb := &client.KnowledgeBase{
    Name:        "Test Knowledge Base",
    ChunkingConfig: client.ChunkingConfig{
        ChunkSize:    500,
        ChunkOverlap: 50,
        Separators:   []string{"\n\n", "\n", ". ", "? ", "! "},
    },
    ImageProcessingConfig: client.ImageProcessingConfig{
        ModelID: "image_model_id",
    },
    EmbeddingModelID: "embedding_model_id",
    SummaryModelID:   "summary_model_id",
}

kb, err := apiClient.CreateKnowledgeBase(context.Background(), kb)
if err != nil {
    // Handle error
}

// Upload knowledge file with metadata
metadata := map[string]string{
    "source": "local",
    "type":   "document",
}
knowledge, err := apiClient.CreateKnowledgeFromFile(context.Background(), kb.ID, "path/to/file.pdf", metadata)
if err != nil {
    // Handle error
}

Example: Create Session and Chat

go
// Create session
sessionRequest := &client.CreateSessionRequest{
    KnowledgeBaseID: knowledgeBaseID,
    SessionStrategy: &client.SessionStrategy{
        MaxRounds:        10,
        EnableRewrite:    true,
        FallbackStrategy: "fixed_answer",
        FallbackResponse: "Sorry, I cannot answer this question",
        EmbeddingTopK:    5,
        KeywordThreshold: 0.5,
        VectorThreshold:  0.7,
        RerankModelID:    "rerank_model_id",
        RerankTopK:       3,
        RerankThreshold:  0.8,
        SummaryModelID:   "summary_model_id",
    },
}

session, err := apiClient.CreateSession(context.Background(), sessionRequest)
if err != nil {
    // Handle error
}

// Regular Q&A
answer, err := apiClient.KnowledgeQA(context.Background(), session.ID, &client.KnowledgeQARequest{
    Query: "What is artificial intelligence?",
})
if err != nil {
    // Handle error
}

// Streaming Q&A
err = apiClient.KnowledgeQAStream(context.Background(), session.ID, "What is machine learning?", func(response *client.StreamResponse) error {
    // Handle each response chunk
    fmt.Print(response.Content)
    return nil
})
if err != nil {
    // Handle error
}

Example: Managing Models

go
// Create model
modelRequest := &client.CreateModelRequest{
    Name:        "Test Model",
    Type:        client.ModelTypeChat,
    Source:      client.ModelSourceInternal,
    Parameters: client.ModelParameters{
        "temperature": 0.7,
        "top_p":       0.9,
    },
    IsDefault: true,
}
model, err := apiClient.CreateModel(context.Background(), modelRequest)
if err != nil {
    // Handle error
}

// List all models
models, err := apiClient.ListModels(context.Background())
if err != nil {
    // Handle error
}

Example: Managing Knowledge Chunks

go
// List knowledge chunks
chunks, total, err := apiClient.ListKnowledgeChunks(context.Background(), knowledgeID, 1, 10)
if err != nil {
    // Handle error
}

// Update chunk
updateRequest := &client.UpdateChunkRequest{
    Content:   "Updated chunk content",
    IsEnabled: true,
}
updatedChunk, err := apiClient.UpdateChunk(context.Background(), knowledgeID, chunkID, updateRequest)
if err != nil {
    // Handle error
}

Example: Getting Session Messages

go
// Get recent messages
messages, err := apiClient.GetRecentMessages(context.Background(), sessionID, 10)
if err != nil {
    // Handle error
}

// Get messages before a specific time
beforeTime := time.Now().Add(-24 * time.Hour)
olderMessages, err := apiClient.GetMessagesBefore(context.Background(), sessionID, beforeTime, 10)
if err != nil {
    // Handle error
}

Complete Example

Please refer to the ExampleUsage function in the example.go file, which demonstrates the complete usage flow of the client.

File: dataset/README_zh.md

QA数据集采样工具

一个全面的QA数据集采样工具，使用OpenAI的GPT模型生成答案。该工具帮助您从大规模数据集（如MS MARCO）创建高质量的问答数据集。

功能特性

  • 智能采样：智能地从大型数据集中采样查询、文档和相关性判断
  • 答案生成：使用OpenAI的GPT模型自动生成高质量答案
  • 断点续传：支持中断后继续生成，从上次位置开始
  • 进度跟踪：实时进度更新和统计信息
  • 结果可视化：易于阅读的问答对展示，包含完整上下文

安装指南

系统要求

  • Python 3.7+
  • OpenAI API密钥

安装依赖

bash
pip install pandas pyarrow openai

设置环境变量

bash
export OPENAI_API_KEY="你的openai-api-key"
# 可选：使用自定义OpenAI端点
export OPENAI_BASE_URL="https://api.openai.com/v1"

准备数据集

您可以使用任何符合格式要求的QA数据集，或下载预处理好的样本：

使用HuggingFace/ModelScope样本
我们提供了来自流行QA数据集的预处理样本：

  • MarkrAI/eli5_sample_autorag
  • MarkrAI/msmarco_sample_autorag
  • MarkrAI/triviaqa_sample_autorag
  • gnekt/hotpotqa_small_sample_autorag

使用您自己的数据集
确保您的数据集包含以下文件：

  • queries.parquet（列：id, text）
  • corpus.parquet（列：id, text）
  • qrels.parquet（列：qid, pid）

快速开始

1. 从大型数据集采样

首先，从完整数据集中采样查询、文档和相关性判断的子集：

bash
python dataset/qa_dataset.py sample \
  --queries ~/dataset/mmarco-queries.parquet \
  --corpus ~/dataset/mmarco-corpus.parquet \
  --qrels ~/dataset/mmarco-qrels.parquet \
  --nq 100 \
  --output_dir ./dataset/samples

2. 生成答案

使用OpenAI的GPT模型为采样的问答生成答案：

bash
python dataset/qa_dataset.py generate \
  --input_dir ./dataset/samples \
  --output_dir ./dataset/samples

3. 查看结果

展示生成的问答对及其上下文：

bash
python dataset/qa_dataset.py show \
  --input_dir ./dataset/samples \
  -n 5

详细使用说明

采样命令

从完整数据集中创建代表性样本。

bash
python dataset/qa_dataset.py sample [选项]

必需参数：

  • --queries：查询parquet文件路径（列：id, text）
  • --corpus：语料库parquet文件路径（列：id, text）
  • --qrels：相关性判断parquet文件路径（列：qid, pid）

可选参数：

  • --nq：要采样的查询数量（默认：1000）
  • --output_dir：采样数据输出目录（默认：./save）

示例：

bash
python dataset/qa_dataset.py sample \
  --queries data/queries.parquet \
  --corpus data/corpus.parquet \
  --qrels data/qrels.parquet \
  --nq 500 \
  --output_dir ./my_sample

生成命令

使用OpenAI API为采样问题生成答案。

bash
python dataset/qa_dataset.py generate [选项]

必需参数：

  • --input_dir：包含采样数据的目录（queries.parquet, corpus.parquet, qrels.parquet）

可选参数：

  • --output_dir：生成答案的输出目录（默认：./save）

特性：

  • 断点续传：中断后自动从上次位置继续
  • 错误处理：API调用失败自动重试3次
  • 进度保存：每成功生成一个答案就保存进度

示例：

bash
python dataset/qa_dataset.py generate \
  --input_dir ./my_sample \
  --output_dir ./my_sample

展示命令

展示生成的问答对及完整上下文。

bash
python dataset/qa_dataset.py show [选项]

必需参数：

  • --input_dir：包含QA数据的目录（queries.parquet, corpus.parquet, qrels.parquet, qas.parquet, answers.parquet）

可选参数：

  • -n：要展示的结果数量（默认：5）

示例：

bash
python dataset/qa_dataset.py show \
  --input_dir ./my_sample \
  -n 3

输入数据格式

查询文件 (queries.parquet)

列名 类型 描述
id string 唯一查询标识符
text string 实际的问题文本

语料库文件 (corpus.parquet)

列名 类型 描述
id string 唯一段落/文档标识符
text string 段落/文档内容

相关性判断文件 (qrels.parquet)

列名 类型 描述
qid string 查询ID（匹配queries.id）
pid string 段落ID（匹配corpus.id）

输出文件

运行所有命令后，输出目录将包含：

采样数据

  • queries.parquet：采样的查询子集
  • corpus.parquet：采样的文档子集
  • qrels.parquet：采样的相关性判断

生成的答案

  • answers.parquet：生成的答案（含唯一ID）
  • qas.parquet：问答映射（qid → aid）

高级用法

自定义OpenAI配置

您可以使用不同的OpenAI模型或端点：

bash
# 使用GPT-4 Turbo
export OPENAI_API_KEY="你的密钥"
python dataset/qa_dataset.py generate --input_dir ./samples

# 使用Azure OpenAI
export OPENAI_API_KEY="azure密钥"
export OPENAI_BASE_URL="https://你的资源.openai.azure.com/openai/deployments/gpt-4"
python dataset/qa_dataset.py generate --input_dir ./samples

大型数据集采样

对于非常大的数据集，建议分批采样：

bash
# 第一批
python dataset/qa_dataset.py sample --nq 1000 --output_dir ./batch1
python dataset/qa_dataset.py generate --input_dir ./batch1

# 第二批
python dataset/qa_dataset.py sample --nq 1000 --output_dir ./batch2
python dataset/qa_dataset.py generate --input_dir ./batch2

故障排除

常见问题

1. OpenAI API错误

  • 确保API密钥设置正确：echo $OPENAI_API_KEY
  • 检查API配额和账单状态
  • 验证与OpenAI的网络连接

2. 大数据集内存问题

  • 减小--nq参数以获得更小的样本
  • 确保pandas操作有足够的RAM
  • 考虑使用更小的parquet文件

3. 文件未找到错误

  • 验证所有输入文件路径是否正确
  • 确保parquet文件有正确的列名
  • 检查文件权限

调试模式

通过添加打印语句或使用Python调试器启用详细输出：

bash
python -m pdb dataset/qa_dataset.py sample --queries ...

示例工作流

bash
# 1. 设置环境
export OPENAI_API_KEY="sk-..."

# 2. 从MS MARCO采样200个查询
python dataset/qa_dataset.py sample \
  --queries ~/mmarco/queries.parquet \
  --corpus ~/mmarco/corpus.parquet \
  --qrels ~/mmarco/qrels.parquet \
  --nq 200 \
  --output_dir ./marco_sample

# 3. 生成答案（根据API速率限制可能需要一些时间）
python dataset/qa_dataset.py generate \
  --input_dir ./marco_sample \
  --output_dir ./marco_sample

# 4. 查看结果
python dataset/qa_dataset.py show \
  --input_dir ./marco_sample \
  -n 10

贡献

欢迎提交问题和功能增强请求！

许可证

MIT许可证 - 可自由用于研究和项目。

File: docreader/README.md

DocReader Service

DocReader 是 WeKnora 项目中负责文档解析和处理的 gRPC 服务。它支持多种文档格式的读取、OCR 识别、多模态处理等功能。

Docker Compose 环境变量配置

在 docker-compose.yml 文件中，docreader 服务配置了以下环境变量：

yaml
docreader:
  image: wechatopenai/weknora-docreader:${WEKNORA_VERSION:-latest}
  environment:
    - MINIO_ENDPOINT=minio:9000
    - MINIO_PUBLIC_ENDPOINT=http://localhost:${MINIO_PORT:-9000}
    - MINERU_ENDPOINT=${MINERU_ENDPOINT:-}
    - MAX_FILE_SIZE_MB=${MAX_FILE_SIZE_MB:-}

环境变量说明

1. MINIO_ENDPOINT

  • 说明: MinIO 服务的内部访问地址（容器间通信）
  • 默认值: minio:9000
  • 用途: DocReader 服务使用此地址连接到 MinIO 对象存储服务，用于读取和存储文档处理过程中的文件
  • 配置示例:
    yaml
    - MINIO_ENDPOINT=minio:9000  # Docker 网络内部地址

2. MINIO_PUBLIC_ENDPOINT

  • 说明: MinIO 服务的公开访问地址（外部访问）
  • 默认值: http://localhost:9000
  • 用途: 用于生成可从外部访问的文件 URL，例如在文档解析后返回图片链接时使用
  • 重要提示:
    • 如果需要从其他设备或容器访问，需要将 localhost 替换为实际的 IP 地址
    • 可以在 .env 文件中配置 MINIO_PORT 来自定义端口
  • 配置示例:
    bash
    # .env 文件
MINIO_PORT=9000
    或直接在 docker-compose.yml 中修改：
    yaml
    - MINIO_PUBLIC_ENDPOINT=http://192.168.1.100:9000  # 使用实际 IP

3. MINERU_ENDPOINT

  • 说明: MinerU 服务的访问地址（可选）
  • 默认值: 空（不使用 MinerU）
  • 用途: MinerU 是一个高级文档解析服务，支持更复杂的文档结构识别和处理。配置此变量后，DocReader 可以调用 MinerU 进行文档解析
  • 配置示例:
    bash
    # .env 文件
MINERU_ENDPOINT=http://mineru-service:8080

4. MAX_FILE_SIZE_MB

  • 说明: 允许上传的最大文件大小（单位：MB）
  • 默认值: 50 MB
  • 用途: 限制 gRPC 服务接收的文件大小，防止过大的文件导致服务崩溃或性能问题
  • 配置示例:
    bash
    # .env 文件
MAX_FILE_SIZE_MB=100  # 允许最大 100MB 的文件

其他可配置的环境变量

除了 docker-compose.yml 中已配置的变量外，DocReader 还支持以下环境变量（可根据需要添加）：

gRPC 配置

  • DOCREADER_GRPC_MAX_WORKERS: gRPC 服务的最大工作线程数（默认：4）
  • DOCREADER_GRPC_PORT: gRPC 服务监听端口（默认：50051）

解析器资源控制

  • DOCREADER_MARKITDOWN_MAX_WORKERS: MarkItDown 解析的最大并发数（默认：1，设为 0 可关闭限流）
  • DOCREADER_PDF_RENDER_MAX_WORKERS: 扫描 PDF 渲染为图片的最大并发数（默认：1，设为 0 可关闭限流）
  • DOCREADER_PDF_RENDER_DPI: 扫描 PDF 渲染 DPI（默认：200）
  • DOCREADER_PDF_JPEG_QUALITY: 扫描 PDF 输出 JPEG 质量（默认：85，范围会自动限制在 1-95）

OCR / VLM

DocReader 自身不再内置 OCR 与 VLM 后端。扫描 PDF 会被渲染为 JPEG 图片后交由 Go App 侧调用 OCR/VLM 服务处理，相关配置请参考主项目文档。

存储配置

DocReader 支持多种存储后端：

MinIO/S3 存储（推荐）

  • STORAGE_TYPE: 设置为 minio
  • MINIO_ACCESS_KEY_ID: MinIO 访问密钥 ID（默认：minioadmin）
  • MINIO_SECRET_ACCESS_KEY: MinIO 访问密钥（默认：minioadmin）
  • MINIO_BUCKET_NAME: MinIO 存储桶名称（默认：WeKnora）
  • MINIO_PATH_PREFIX: 文件路径前缀
  • MINIO_USE_SSL: 是否使用 SSL（默认：false）

腾讯云 COS 存储

  • STORAGE_TYPE: 设置为 cos
  • COS_SECRET_ID: COS 访问密钥 ID
  • COS_SECRET_KEY: COS 访问密钥
  • COS_REGION: COS 区域
  • COS_BUCKET_NAME: COS 存储桶名称
  • COS_APP_ID: COS 应用 ID
  • COS_PATH_PREFIX: 文件路径前缀
  • COS_ENABLE_OLD_DOMAIN: 是否使用旧域名（默认：true）

阿里云 OSS 存储

  • STORAGE_TYPE: 设置为 oss
  • OSS_ACCESS_KEY_ID: OSS 访问密钥 ID
  • OSS_ACCESS_KEY_SECRET: OSS 访问密钥
  • OSS_ENDPOINT: OSS 端点（如 oss-cn-hangzhou.aliyuncs.com）
  • OSS_BUCKET_NAME: OSS 存储桶名称
  • OSS_REGION: OSS 区域（如 cn-hangzhou）
  • OSS_PATH_PREFIX: 文件路径前缀

代理配置

如果需要通过代理访问外部服务：

  • EXTERNAL_HTTP_PROXY: HTTP 代理地址
  • EXTERNAL_HTTPS_PROXY: HTTPS 代理地址

图像处理配置

扫描 PDF 会被渲染为 JPEG 图片后交给 Go App 侧 OCR 处理。如果在导入多个大 PDF 时出现资源占用过高，
可以优先调低 DOCREADER_PDF_RENDER_MAX_WORKERS 或 DOCREADER_MARKITDOWN_MAX_WORKERS。

配置示例

基础配置（使用 MinIO）

yaml
docreader:
  environment:
    - MINIO_ENDPOINT=minio:9000
    - MINIO_PUBLIC_ENDPOINT=http://localhost:9000
    - MAX_FILE_SIZE_MB=50

高级配置（启用 MinerU）

yaml
docreader:
  environment:
    - MINIO_ENDPOINT=minio:9000
    - MINIO_PUBLIC_ENDPOINT=http://192.168.1.100:9000
    - MINERU_ENDPOINT=http://mineru:8080
    - MAX_FILE_SIZE_MB=100

使用腾讯云 COS

yaml
docreader:
  environment:
    - STORAGE_TYPE=cos
    - COS_SECRET_ID=your_secret_id
    - COS_SECRET_KEY=your_secret_key
    - COS_REGION=ap-guangzhou
    - COS_BUCKET_NAME=your-bucket
    - COS_APP_ID=your_app_id
    - MAX_FILE_SIZE_MB=50

使用阿里云 OSS

yaml
docreader:
  environment:
    - STORAGE_TYPE=oss
    - OSS_ACCESS_KEY_ID=your_access_key_id
    - OSS_ACCESS_KEY_SECRET=your_access_key_secret
    - OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
    - OSS_BUCKET_NAME=your-bucket
    - OSS_REGION=cn-hangzhou
    - MAX_FILE_SIZE_MB=50

常见问题

1. DocReader 服务无法启动？

检查容器日志中是否存在依赖缺失或权限相关错误，必要时确认 MINIO_ENDPOINT / 存储相关环境变量是否正确配置。

2. 图片无法显示？

检查 MINIO_PUBLIC_ENDPOINT 配置：

  • 确保使用的是可从浏览器访问的地址
  • 如果从其他设备访问，不要使用 localhost，应使用实际 IP 地址

3. 文件上传失败？

检查 MAX_FILE_SIZE_MB 配置，确保限制足够大。同时需要确保前端和后端服务的文件大小限制保持一致。

服务健康检查

DocReader 服务配置了健康检查：

yaml
healthcheck:
  test: ["CMD", "grpc_health_probe", "-addr=localhost:50051"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 60s

可以通过以下命令检查服务状态：

bash
docker ps | grep docreader
docker logs WeKnora-docreader

更多信息

  • 服务端口：50051（gRPC）
  • 容器名称：WeKnora-docreader
  • 网络：WeKnora-network
  • 重启策略：unless-stopped

File: docs/api/agent.md

智能体（Agent）管理 API

返回目录

概述

智能体 API 用于管理自定义智能体（Custom Agent）。系统提供了内置智能体，同时支持用户创建自定义智能体来满足不同的业务场景需求。

智能体的共享与跨组织分发（/agents/:id/shares 等）属于组织协作能力，文档见 组织管理 API。本文件只覆盖智能体自身的 CRUD、复制、占位符、类型预设以及推荐问题接口。

内置智能体

系统默认提供以下内置智能体：

ID 名称 描述 模式
builtin-quick-answer 快速问答 基于知识库的 RAG 问答，快速准确地回答问题 quick-answer
builtin-smart-reasoning 智能推理 ReAct 推理框架，支持多步思考和工具调用 smart-reasoning
builtin-data-analyst 数据分析师 专业数据分析智能体，支持 CSV/Excel 文件的 SQL 查询与统计分析 smart-reasoning

智能体模式

模式 说明
quick-answer RAG 模式，快速问答，直接基于知识库检索结果生成回答
smart-reasoning ReAct 模式，支持多步推理和工具调用

API 列表

方法 路径 描述
POST /agents 创建智能体
GET /agents 获取智能体列表
GET /agents/:id 获取智能体详情
PUT /agents/:id 更新智能体
DELETE /agents/:id 删除智能体
POST /agents/:id/copy 复制智能体
GET /agents/placeholders 获取占位符定义

POST `/agents` - 创建智能体

创建新的自定义智能体。成功返回 HTTP 201。

请求体参数:

参数 类型 必填 说明
name string 是 智能体名称
description string 否 智能体描述
avatar string 否 头像（emoji 或图标名称）
config object 否 智能体配置，详见 配置参数

请求:

curl
curl --location 'http://localhost:8080/api/v1/agents' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "name": "我的智能体",
    "description": "自定义智能体描述",
    "avatar": "🤖",
    "config": {
        "agent_mode": "smart-reasoning",
        "system_prompt": "你是一个专业的助手...",
        "temperature": 0.7,
        "max_iterations": 10,
        "kb_selection_mode": "all",
        "web_search_enabled": true,
        "multi_turn_enabled": true,
        "history_turns": 5
    }
}'

响应:

json
{
    "success": true,
    "data": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "我的智能体",
        "description": "自定义智能体描述",
        "avatar": "🤖",
        "is_builtin": false,
        "tenant_id": 1,
        "created_by": "user-123",
        "config": {
            "agent_mode": "smart-reasoning",
            "system_prompt": "你是一个专业的助手...",
            "temperature": 0.7,
            "max_iterations": 10
        },
        "created_at": "2025-01-19T10:00:00Z",
        "updated_at": "2025-01-19T10:00:00Z"
    }
}

错误响应:

状态码 错误码 错误 说明
400 1000 Bad Request 请求参数错误或智能体名称为空
500 1007 Internal Server Error 服务器内部错误

GET `/agents` - 获取智能体列表

获取当前空间的所有智能体，包括内置智能体和自定义智能体。响应中额外返回 disabled_own_agent_ids，指示当前空间在前端对话下拉框中主动隐藏的本空间自有智能体 ID 列表（不影响其他空间）。

请求:

curl
curl --location 'http://localhost:8080/api/v1/agents' \
--header 'X-API-Key: sk-xxxxx'

响应:

json
{
    "success": true,
    "data": [
        {
            "id": "builtin-quick-answer",
            "name": "快速问答",
            "description": "基于知识库的 RAG 问答，快速准确地回答问题",
            "avatar": "💬",
            "is_builtin": true,
            "tenant_id": 10000,
            "created_by": "",
            "config": {
                "agent_mode": "quick-answer",
                "temperature": 0.3,
                "max_completion_tokens": 2048,
                "kb_selection_mode": "all",
                "web_search_enabled": false,
                "multi_turn_enabled": true,
                "history_turns": 5
            },
            "created_at": "2025-12-29T20:06:01.696308+08:00",
            "updated_at": "2025-12-29T20:06:01.696308+08:00",
            "deleted_at": null
        },
        {
            "id": "550e8400-e29b-41d4-a716-446655440000",
            "name": "我的智能体",
            "is_builtin": false,
            "config": {
                "agent_mode": "smart-reasoning"
            }
        }
    ],
    "disabled_own_agent_ids": []
}

错误响应:

状态码 错误码 错误 说明
401 1001 Unauthorized 缺少空间上下文
500 1007 Internal Server Error 服务器内部错误

GET `/agents/:id` - 获取智能体详情

根据 ID 获取智能体的详细信息。

路径参数:

参数 类型 说明
id string 智能体 ID

请求:

curl
curl --location 'http://localhost:8080/api/v1/agents/builtin-quick-answer' \
--header 'X-API-Key: sk-xxxxx'

响应:

json
{
    "success": true,
    "data": {
        "id": "builtin-quick-answer",
        "name": "快速问答",
        "description": "基于知识库的 RAG 问答，快速准确地回答问题",
        "is_builtin": true,
        "tenant_id": 1,
        "config": {
            "agent_mode": "quick-answer",
            "system_prompt": "",
            "context_template": "请根据以下参考资料回答用户问题...",
            "temperature": 0.7,
            "max_completion_tokens": 2048,
            "kb_selection_mode": "all",
            "web_search_enabled": true,
            "multi_turn_enabled": true,
            "history_turns": 5
        },
        "created_at": "2025-01-01T00:00:00Z",
        "updated_at": "2025-01-01T00:00:00Z"
    }
}

错误响应:

状态码 错误码 错误 说明
400 1000 Bad Request 智能体 ID 为空
404 1003 Not Found 智能体不存在
500 1007 Internal Server Error 服务器内部错误

PUT `/agents/:id` - 更新智能体

更新智能体的名称、描述、头像和配置。内置智能体不可修改。

路径参数:

参数 类型 说明
id string 智能体 ID

请求体参数:

参数 类型 必填 说明
name string 否 智能体名称
description string 否 智能体描述
avatar string 否 智能体头像
config object 否 智能体配置

请求:

curl
curl --location --request PUT 'http://localhost:8080/api/v1/agents/550e8400-e29b-41d4-a716-446655440000' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "name": "更新后的智能体",
    "description": "更新后的描述",
    "config": {
        "agent_mode": "smart-reasoning",
        "temperature": 0.8,
        "max_iterations": 20
    }
}'

响应:

json
{
    "success": true,
    "data": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "更新后的智能体",
        "description": "更新后的描述",
        "config": {
            "agent_mode": "smart-reasoning",
            "temperature": 0.8,
            "max_iterations": 20
        },
        "updated_at": "2025-01-19T11:00:00Z"
    }
}

错误响应:

状态码 错误码 错误 说明
400 1000 Bad Request 请求参数错误或智能体名称为空
403 1002 Forbidden 无法修改内置智能体
404 1003 Not Found 智能体不存在
500 1007 Internal Server Error 服务器内部错误

DELETE `/agents/:id` - 删除智能体

删除指定的自定义智能体。内置智能体不可删除。

路径参数:

参数 类型 说明
id string 智能体 ID

请求:

curl
curl --location --request DELETE 'http://localhost:8080/api/v1/agents/550e8400-e29b-41d4-a716-446655440000' \
--header 'X-API-Key: sk-xxxxx'

响应:

json
{
    "success": true,
    "message": "Agent deleted successfully"
}

错误响应:

状态码 错误码 错误 说明
400 1000 Bad Request 智能体 ID 为空
403 1002 Forbidden 无法删除内置智能体
404 1003 Not Found 智能体不存在
500 1007 Internal Server Error 服务器内部错误

POST `/agents/:id/copy` - 复制智能体

复制指定的智能体，创建一个新的副本，副本始终为自定义智能体。支持复制内置智能体。成功返回 HTTP 201。

路径参数:

参数 类型 说明
id string 源智能体 ID

请求:

curl
curl --location --request POST 'http://localhost:8080/api/v1/agents/builtin-smart-reasoning/copy' \
--header 'X-API-Key: sk-xxxxx'

响应:

json
{
    "success": true,
    "data": {
        "id": "660e8400-e29b-41d4-a716-446655440001",
        "name": "智能推理 (副本)",
        "description": "ReAct 推理框架，支持多步思考和工具调用",
        "is_builtin": false,
        "config": {
            "agent_mode": "smart-reasoning",
            "max_iterations": 50
        },
        "created_at": "2025-01-19T12:00:00Z",
        "updated_at": "2025-01-19T12:00:00Z"
    }
}

错误响应:

状态码 错误码 错误 说明
400 1000 Bad Request 智能体 ID 为空
404 1003 Not Found 智能体不存在
500 1007 Internal Server Error 服务器内部错误

GET `/agents/placeholders` - 获取占位符定义

获取所有可用的提示词占位符定义，按字段类型分组。这些占位符可用于系统提示词和上下文模板中。

请求:

curl
curl --location 'http://localhost:8080/api/v1/agents/placeholders' \
--header 'X-API-Key: your_api_key'

响应:

json
{
    "success": true,
    "data": {
        "all": [...],
        "system_prompt": [...],
        "agent_system_prompt": [...],
        "context_template": [...],
        "rewrite_system_prompt": [...],
        "rewrite_prompt": [...],
        "fallback_prompt": [...]
    }
}

配置参数

智能体的 config 对象支持以下配置项：

基础设置

参数 类型 默认值 说明
agent_mode string - 智能体模式：quick-answer（RAG）或 smart-reasoning（ReAct）
system_prompt string - 系统提示词，支持使用占位符
system_prompt_id string - 系统提示词模板 ID（引用 prompt_templates/ YAML 文件中的模板）
context_template string - 上下文模板（仅 quick-answer 模式使用）
context_template_id string - 上下文模板 ID（引用 prompt_templates/ YAML 文件中的模板）

模型设置

参数 类型 默认值 说明
model_id string - 对话模型 ID
rerank_model_id string - 重排序模型 ID
temperature float 0.7 温度参数（0-1）
max_completion_tokens int 2048 最大生成 token 数
thinking *bool nil 是否启用思考模式（适用于支持扩展思考的模型）

Agent 模式设置

参数 类型 默认值 说明
max_iterations int 10 ReAct 最大迭代次数
allowed_tools []string - 允许使用的工具列表
mcp_selection_mode string - MCP 服务选择模式：all/selected/none
mcp_services []string - 选中的 MCP 服务 ID 列表
skills_selection_mode string - Skills 选择模式：all/selected/none
selected_skills []string - 选中的 Skill 名称列表（mode 为 selected 时）

知识库设置

参数 类型 默认值 说明
kb_selection_mode string - 知识库选择模式：all/selected/none
knowledge_bases []string - 关联的知识库 ID 列表
retrieve_kb_only_when_mentioned bool false 仅在用户通过 @ 显式提及时才检索知识库
supported_file_types []string - 支持的文件类型（如 ["csv", "xlsx"]）

图片上传 / 多模态设置

参数 类型 默认值 说明
image_upload_enabled bool false 是否允许上传图片
vlm_model_id string - 图片分析所用的 VLM 模型 ID
image_storage_provider string - 图片存储提供者：local/minio/cos/tos/oss，为空使用全局默认

FAQ 策略设置

参数 类型 默认值 说明
faq_priority_enabled bool true FAQ 优先策略开关
faq_direct_answer_threshold float 0.9 FAQ 直接回答阈值
faq_score_boost float 1.2 FAQ 分数加成系数

网络搜索设置

参数 类型 默认值 说明
web_search_enabled bool true 是否启用网络搜索
web_search_max_results int 5 网络搜索最大结果数
web_search_provider_id string - 网络搜索提供者 ID，为空使用空间默认提供者
web_fetch_enabled bool false 是否自动获取重排后的搜索结果页面全文
web_fetch_top_n int 3 重排后获取全文的最大页面数

多轮对话设置

参数 类型 默认值 说明
multi_turn_enabled bool true 是否启用多轮对话
history_turns int 5 保留的历史轮次数

检索策略设置

参数 类型 默认值 说明
embedding_top_k int 10 向量检索 TopK
keyword_threshold float 0.3 关键词检索阈值
vector_threshold float 0.5 向量检索阈值
rerank_top_k int 5 重排序 TopK
rerank_threshold float 0.5 重排序阈值

推荐问题设置

question_suggestions 是智能体拥有的统一策略。网页嵌入等渠道只能关闭展示，不能覆盖内容或生成规则。

参数 类型 默认值 说明
question_suggestions.starters.enabled bool true 是否在首次提问前展示开场问题
question_suggestions.starters.mode string hybrid curated、knowledge 或 hybrid
question_suggestions.starters.items []string [] 运营配置的开场问题
question_suggestions.starters.count int 6 展示数量，范围 1-8
question_suggestions.follow_ups.enabled bool false 是否在每次完整回答后异步生成追问
question_suggestions.follow_ups.mode string hybrid generated、knowledge 或 hybrid
question_suggestions.follow_ups.count int 3 生成数量，范围 1-5
question_suggestions.follow_ups.model_id string - 独立生成模型；为空使用本轮对话模型
question_suggestions.follow_ups.categories []string clarify,deepen,action 允许的问题类型
question_suggestions.follow_ups.max_context_turns int 2 生成时使用的最近对话轮数，范围 1-5
question_suggestions.follow_ups.additional_instruction string - 智能体作者的附加生成要求
question_suggestions.follow_ups.suppress_on_fallback bool true 兜底回答后不展示
question_suggestions.follow_ups.suppress_when_answer_asks_question bool true 回答本身以问题结尾时不展示
question_suggestions.follow_ups.knowledge_fallback bool true 模型失败时使用知识库候选补位
question_suggestions.follow_ups.allow_regenerate bool false 是否允许用户换一批

旧 suggested_prompts 会在数据库迁移时一次性写入 starters.items，API 不再接受该字段。

高级设置

参数 类型 默认值 说明
enable_query_expansion bool true 是否启用查询扩展
enable_rewrite bool true 是否启用多轮对话查询改写
rewrite_prompt_system string - 改写系统提示词
rewrite_prompt_user string - 改写用户提示词模板
fallback_strategy string model 回退策略：fixed（固定回复）或 model（模型生成）；未设置时在服务端默认为 model
fallback_response string - 固定回退回复（fallback_strategy 为 fixed 时使用）
fallback_prompt string - 回退提示词（fallback_strategy 为 model 时使用）

使用 Agent 进行问答

创建或获取智能体后，可以通过 /agent-chat/:session_id 接口使用智能体进行问答。详情请参考 聊天功能 API。

在问答请求中使用 agent_id 参数指定要使用的智能体：

curl
curl --location 'http://localhost:8080/api/v1/agent-chat/session-123' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "query": "帮我分析一下这份数据",
    "agent_enabled": true,
    "agent_id": "builtin-data-analyst"
}'

相关文档

  • 智能体的组织共享、跨空间分发与禁用（/agents/:id/shares、/shared-agents 等）：见 组织管理 API
  • 智能体绑定 IM 渠道（/agents/:id/im-channels）：见组织/IM 渠道相关文档
  • 网络搜索提供者配置（被 web_search_provider_id 引用）：见 Web Search API

File: docs/api/auth.md

认证管理 API

返回目录

OIDC 完整调用流程见 ../OIDC认证调用流程.md。本文档作为端点参考。

说明

WeKnora 的 /auth/* 端点本身不需要 X-API-Key，但部分端点需要在 Authorization: Bearer <token> 头中携带由 /auth/login 或 /auth/oidc/callback 返回的 JWT：

端点 鉴权方式
/auth/register /auth/login 无
/auth/oidc/config /auth/oidc/url /auth/oidc/callback 无
/auth/refresh refresh_token（请求体携带）
/auth/validate /auth/me /auth/logout /auth/change-password Bearer JWT

注册接口可通过环境变量 DISABLE_REGISTRATION=true 关闭。

端点一览

方法 路径 描述
POST /auth/register 用户注册
POST /auth/login 用户登录
GET /auth/oidc/config 获取 OIDC 配置元数据
GET /auth/oidc/url 获取 OIDC 授权链接
GET /auth/oidc/callback OIDC 授权回调（由 IdP 重定向触发）
POST /auth/refresh 用 refresh_token 换新的 access_token
GET /auth/validate 验证 JWT 有效性
POST /auth/logout 退出登录
GET /auth/me 获取当前用户信息
POST /auth/change-password 修改密码

POST `/auth/register` - 用户注册

参数说明（请求体）:

字段 类型 必填 校验 说明
username string 是 长度 2-50 用户名
email string 是 邮箱格式 邮箱
password string 是 最少 6 位 密码

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/register' \
--header 'Content-Type: application/json' \
--data '{
    "username": "alice",
    "email": "[email protected]",
    "password": "secret123"
}'

响应（201 Created）:

json
{
    "success": true,
    "message": "Registration successful",
    "user": {
        "id": "usr-...",
        "username": "alice",
        "email": "[email protected]",
        "tenant_id": 1,
        "is_active": true,
        "created_at": "2026-05-11T10:00:00+08:00",
        "updated_at": "2026-05-11T10:00:00+08:00"
    },
    "tenant": {
        "id": 1,
        "name": "alice's workspace",
        "api_key": "sk-..."
    }
}

错误: 注册被禁用 → 403；参数校验失败 → 400。

POST `/auth/login` - 用户登录

参数说明（请求体）:

字段 类型 必填 说明
email string 是 注册邮箱
password string 是 密码

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/login' \
--header 'Content-Type: application/json' \
--data '{
    "email": "[email protected]",
    "password": "secret123"
}'

响应:

json
{
    "success": true,
    "message": "Login successful",
    "user": { "id": "usr-...", "username": "alice", "email": "[email protected]" },
    "tenant": { "id": 1, "name": "alice's workspace", "api_key": "sk-..." },
    "token": "eyJhbGciOi...",
    "refresh_token": "eyJhbGciOi..."
}

错误: 邮箱或密码错误 → 401；账号被禁用 → 403。

GET `/auth/oidc/config` - 获取 OIDC 配置元数据

返回 OIDC 是否启用以及 Provider 显示名，前端登录页据此决定是否展示 OIDC 登录按钮。

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/oidc/config'

响应:

json
{
    "success": true,
    "enabled": true,
    "provider_display_name": "WeKnora SSO"
}

GET `/auth/oidc/url` - 获取 OIDC 授权链接

返回前端应跳转的 OIDC IdP 授权页 URL 与状态码。

查询参数:

字段 类型 必填 说明
redirect string 否 登录成功后前端期望落地的路径（如 /dashboard），透传到 state

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/oidc/url?redirect=%2Fdashboard'

响应:

json
{
    "success": true,
    "provider_display_name": "WeKnora SSO",
    "authorization_url": "https://idp.example.com/oauth/authorize?client_id=...&state=...",
    "state": "abcdef..."
}

GET `/auth/oidc/callback` - OIDC 授权回调

由 IdP 在用户授权后重定向到此端点。一般不需要客户端代码直接调用——它的作用是把登录结果通过浏览器 hash 传回前端首页。

查询参数:

字段 类型 必填 说明
code string 是 IdP 颁发的 authorization code
state string 是 与 /auth/oidc/url 返回值一致
error string 否 IdP 返回的错误标识
error_description string 否 IdP 返回的错误详情

响应：始终返回 302 Found，跳转到 /，并把结果编码进 URL hash：

  • 成功：/#oidc_result=，其中 payload 包含 success / user / tenant / token / refresh_token / is_new_user，与登录响应一致。
  • 失败：/#oidc_error=<reason>[&oidc_error_description=<message>]，常见 reason 包括 invalid_state、missing_code、login_failed、payload_encode_failed。

GET `/auth/oidc/start` - 发起 OIDC 登录（直接 302）

与 /auth/oidc/url 不同，此端点直接 302 重定向到 OIDC Provider 的授权页，不返回 JSON，因此无需前端 JS 介入。适用于外部平台（如企业门户 / Nexus）直接给出一个链接即可触发 OIDC 授权码流程，借助 IdP 的 SSO session 实现免再次输密码。

回调地址由后端根据请求自身的 origin（<scheme>://<host>/api/v1/auth/oidc/callback）自动构造，无需调用方提供。

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/oidc/start'

响应：302 Found，Location 指向 IdP 授权页（含 client_id / state / redirect_uri / scope）。

登录成功后的回调行为与 /auth/oidc/callback 一致：302 回前端首页并把登录结果编码进 URL hash。当前登录后固定落到默认首页 /platform/knowledge-bases（直达指定业务页的 next 参数为未来扩展）。

POST `/auth/refresh` - 刷新令牌

参数说明（请求体）:

字段 类型 必填 说明
refreshToken string 是 登录时颁发的 refresh_token

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/refresh' \
--header 'Content-Type: application/json' \
--data '{
    "refreshToken": "eyJhbGciOi..."
}'

响应:

json
{
    "success": true,
    "message": "Token refreshed successfully",
    "access_token": "eyJhbGciOi...",
    "refresh_token": "eyJhbGciOi..."
}

错误: refresh_token 无效或过期 → 401。

GET `/auth/validate` - 验证 JWT

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/validate' \
--header 'Authorization: Bearer eyJhbGciOi...'

响应:

json
{
    "success": true,
    "valid": true,
    "user_id": "usr-...",
    "tenant_id": 1
}

无效 token 返回 401。

POST `/auth/logout` - 退出登录

请求:

curl
curl --location --request POST 'http://localhost:8080/api/v1/auth/logout' \
--header 'Authorization: Bearer eyJhbGciOi...'

响应: { "success": true, "message": "Logged out successfully" }

GET `/auth/me` - 获取当前用户信息

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/me' \
--header 'Authorization: Bearer eyJhbGciOi...'

响应:

json
{
    "success": true,
    "data": {
        "user": {
            "id": "usr-...",
            "username": "alice",
            "email": "[email protected]",
            "avatar": "",
            "tenant_id": 1,
            "is_active": true,
            "can_access_all_tenants": false,
            "created_at": "2026-05-11T10:00:00+08:00",
            "updated_at": "2026-05-11T10:00:00+08:00"
        },
        "tenant": {
            "id": 1,
            "name": "My Workspace"
        },
        "memberships": [
            {
                "tenant_id": 1,
                "tenant_name": "My Workspace",
                "role": "owner"
            }
        ],
        "tenant_required": false,
        "capabilities": {
            "can_create_tenant": false,
            "auto_accept_invitation": false
        }
    }
}

capabilities 供 SPA 读取部署级开关，无需调用超管设置 API：

字段 说明
can_create_tenant 当前用户是否可自助创建空间
auto_accept_invitation 全局 tenant.auto_accept_invitation：邮箱邀请已注册用户时是否直接加入（无需收件箱确认）

POST `/auth/change-password` - 修改密码

修改当前用户的登录密码。新密码须满足 8–32 位且同时包含字母与数字；不能与当前密码相同。成功后所有会话被撤销，需使用新密码重新登录。

参数说明（请求体）:

字段 类型 必填 校验 说明
old_password string 是 当前密码
new_password string 是 8–32 位，须含字母与数字，且不同于旧密码 新密码

请求:

curl
curl --location 'http://localhost:8080/api/v1/auth/change-password' \
--header 'Authorization: Bearer eyJhbGciOi...' \
--header 'Content-Type: application/json' \
--data '{
    "old_password": "secret123",
    "new_password": "newsecret456"
}'

响应: { "success": true, "message": "Password changed successfully" }

错误（400）:

error.details 含义
invalid_old_password 当前密码不正确
password_policy 新密码不满足长度/复杂度要求
same_password 新密码与当前密码相同

File: docs/api/chat.md

聊天功能 API

返回目录

方法 路径 描述
POST /knowledge-chat/:session_id 基于知识库的问答
POST /agent-chat/:session_id 基于 Agent 的智能问答
POST /knowledge-search 基于知识库的搜索知识
GET /sessions/:session_id/messages/:message_id/suggestions 获取已生成的回答后推荐
POST /sessions/:session_id/messages/:message_id/suggestions 确保生成或换一批推荐
POST /sessions/:session_id/suggestion-events 上报曝光、点击、关闭事件

POST `/knowledge-chat/:session_id` - 基于知识库的问答

基于知识库的 RAG 问答，支持 SSE 流式响应。

查询参数：

参数 取值 说明
resource_urls handle（默认）/ public public 让答案与引用里的图片直接返回可加载的 http(s) 链接，省去逐个调用 /files 代理。详见文件与图片引用

同样适用于下面的 /agent-chat/:session_id、/knowledge-search 与 /sessions/continue-stream/:session_id。

请求参数：

参数 类型 必填 说明
query string 是 查询文本
knowledge_base_ids string[] 否 知识库 ID 列表
knowledge_ids string[] 否 知识文件 ID 列表，指定具体文件进行检索
agent_id string 否 自定义 Agent ID，指定使用的智能体
summary_model_id string 否 覆盖默认的摘要模型 ID
mentioned_items object[] 否 @提及的知识库和文件列表
disable_title bool 否 是否禁用自动标题生成（默认 false）
images object[] 否 附带的图片（base64 格式），需要 Agent 启用图片上传
channel string 否 来源渠道标识：web、api、im、browser_extension
suggestion_attribution object 否 用户从推荐问题发起本轮时传入 {suggestion_set_id, question_id}；服务端会校验归属

请求:

curl
curl --location 'http://localhost:8080/api/v1/knowledge-chat/ceb9babb-1e30-41d7-817d-fd584954304b' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "query": "彗尾的形状",
    "knowledge_base_ids": ["kb-00000001"],
    "agent_id": "builtin-quick-answer"
}'

响应格式:
服务器端事件流（Server-Sent Events，Content-Type: text/event-stream）

响应:

text
event: message
data: {"id":"3475c004-0ada-4306-9d30-d7f5efce50d2","response_type":"references","content":"","done":false,"knowledge_references":[{"id":"c8347bef-...","content":"彗星xxx。","knowledge_id":"a6790b93-...","chunk_index":0,"knowledge_title":"彗星.txt","score":4.04,"match_type":3,"chunk_type":"text","knowledge_filename":"彗星.txt"}]}

event: message
data: {"id":"3475c004-0ada-4306-9d30-d7f5efce50d2","response_type":"answer","content":"彗尾的形状主要表现为...","done":false,"knowledge_references":null}

event: message
data: {"id":"3475c004-0ada-4306-9d30-d7f5efce50d2","response_type":"answer","content":"","done":true,"knowledge_references":null}

POST `/agent-chat/:session_id` - 基于 Agent 的智能问答

Agent 模式支持更智能的问答，包括工具调用、网络搜索、多知识库检索等能力。

请求参数：

参数 类型 必填 说明
query string 是 查询文本
knowledge_base_ids string[] 否 知识库 ID 列表，可动态指定本次查询使用的知识库
knowledge_ids string[] 否 知识文件 ID 列表，可动态指定本次查询使用的具体文件
agent_enabled bool 否 是否启用 Agent 模式（默认 false，优先使用 Agent 配置）
agent_id string 否 自定义 Agent ID，指定使用的智能体（支持共享 Agent）
web_search_enabled bool 否 是否启用网络搜索（默认 false）
summary_model_id string 否 覆盖默认的摘要模型 ID
mentioned_items object[] 否 @提及的知识库和文件列表
disable_title bool 否 是否禁用自动标题生成（默认 false）
images object[] 否 附带的图片（base64 格式），需要 Agent 启用图片上传
channel string 否 来源渠道标识：web、api、im、browser_extension
suggestion_attribution object 否 用户从推荐问题发起本轮时传入 {suggestion_set_id, question_id}；服务端会校验归属

回答后推荐问题

回答主消息完成后，服务端会异步生成推荐问题，不阻塞 SSE 的 complete/done 事件。生成结果按“空间、助手消息、位置、配置快照、语言”持久化并去重。

http
POST /api/v1/sessions/{session_id}/messages/{message_id}/suggestions
Content-Type: application/json

{"regenerate": false}

状态包括 generating、ready、suppressed、failed。ready 时的每个问题都有稳定 id，点击后应先上报事件，并在下一次聊天请求中携带 suggestion_attribution。

http
POST /api/v1/sessions/{session_id}/suggestion-events
Content-Type: application/json

{
  "suggestion_set_id": "...",
  "question_id": "...",
  "event_type": "click"
}

网页嵌入提供同构接口：/api/v1/embed/{channel_id}/sessions/{session_id}/...，继续使用嵌入令牌和 X-Embed-Session。

mentioned_items 结构：

字段 类型 说明
id string 知识库或文件 ID
name string 显示名称
type string 类型：kb（知识库）或 file（文件）
kb_type string 知识库类型：document 或 faq（仅 type=kb 时）

images 结构：

字段 类型 说明
data string base64 编码的图片数据（data:image/png;base64,...）

请求:

curl
curl --location 'http://localhost:8080/api/v1/agent-chat/ceb9babb-1e30-41d7-817d-fd584954304b' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "query": "帮我查询今天的天气",
    "agent_enabled": true,
    "web_search_enabled": true,
    "knowledge_base_ids": ["kb-00000001"],
    "agent_id": "builtin-smart-reasoning",
    "mentioned_items": [
        {
            "id": "kb-00000001",
            "name": "天气知识库",
            "type": "kb",
            "kb_type": "document"
        }
    ]
}'

响应格式:
服务器端事件流（Server-Sent Events，Content-Type: text/event-stream）

响应类型说明：

response_type 描述
agent_query Agent 开始处理查询
thinking Agent 思考过程
tool_call 工具调用信息
tool_result 工具调用结果
references 知识库检索引用
answer 最终回答内容
reflection Agent 反思内容
session_title 自动生成的会话标题
error 错误信息

响应示例:

text
event: message
data: {"id":"req-001","response_type":"thinking","content":"用户想查询天气，我需要使用网络搜索工具...","done":false}

event: message
data: {"id":"req-001","response_type":"tool_call","content":"","done":false,"data":{"tool_name":"web_search","arguments":{"query":"今天天气"}}}

event: message
data: {"id":"req-001","response_type":"tool_result","content":"搜索结果：今天晴，气温25°C...","done":false}

event: message
data: {"id":"req-001","response_type":"answer","content":"根据查询结果，今天天气晴朗，气温约25°C。","done":false}

event: message
data: {"id":"req-001","response_type":"answer","content":"","done":true}

File: docs/api/chunk.md

分块管理 API

返回目录

方法 路径 描述
GET /chunks/:knowledge_id 获取知识的分块列表
PUT /chunks/:knowledge_id/:id 更新分块
DELETE /chunks/:knowledge_id/:id 删除单个分块
DELETE /chunks/:knowledge_id 删除知识下的所有分块
GET /chunks/by-id/:id 根据分块 ID 直接获取分块
DELETE /chunks/by-id/:id/questions 删除分块下的某个生成问题

GET `/chunks/:knowledge_id` - 获取知识的分块列表

路径参数:

字段 类型 说明
knowledge_id string 知识 ID

查询参数:

字段 类型 默认 说明
page int 1 页码
page_size int 20 每页条数

请求:

curl
curl --location 'http://localhost:8080/api/v1/chunks/4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5?page=1&page_size=1' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json'

响应:

json
{
    "data": [
        {
            "id": "df10b37d-cd05-4b14-ba8a-e1bd0eb3bbd7",
            "tenant_id": 1,
            "knowledge_id": "4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5",
            "knowledge_base_id": "kb-00000001",
            "tag_id": "",
            "content": "彗星xxxx",
            "chunk_index": 0,
            "is_enabled": true,
            "status": 2,
            "start_at": 0,
            "end_at": 964,
            "pre_chunk_id": "",
            "next_chunk_id": "",
            "chunk_type": "text",
            "parent_chunk_id": "",
            "relation_chunks": null,
            "indirect_relation_chunks": null,
            "metadata": null,
            "content_hash": "",
            "image_info": "",
            "created_at": "2025-08-12T11:52:36.168632+08:00",
            "updated_at": "2025-08-12T11:52:53.376871+08:00",
            "deleted_at": null
        }
    ],
    "page": 1,
    "page_size": 1,
    "success": true,
    "total": 5
}

PUT `/chunks/:knowledge_id/:id` - 更新分块

更新指定分块的内容和属性。所有字段均可选，未传则保留原值。

路径参数:

字段 类型 说明
knowledge_id string 知识 ID
id string 分块 ID

参数说明（请求体）:

字段 类型 必填 说明
content string 否 分块内容
chunk_index int 否 分块在知识中的序号
is_enabled boolean 否 是否启用
start_at int 否 起始位置（字符偏移）
end_at int 否 结束位置（字符偏移）
image_info string 否 图像分块的元信息（JSON 字符串）

请求:

curl
curl --location --request PUT 'http://localhost:8080/api/v1/chunks/4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5/df10b37d-cd05-4b14-ba8a-e1bd0eb3bbd7' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "content": "更新后的分块内容",
    "is_enabled": true
}'

响应:

json
{
    "data": {
        "id": "df10b37d-cd05-4b14-ba8a-e1bd0eb3bbd7",
        "content": "更新后的分块内容",
        "is_enabled": true,
        "...": "其他字段同 GET 响应"
    },
    "success": true
}

DELETE `/chunks/:knowledge_id/:id` - 删除单个分块

路径参数: 同 PUT。

请求:

curl
curl --location --request DELETE 'http://localhost:8080/api/v1/chunks/4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5/df10b37d-cd05-4b14-ba8a-e1bd0eb3bbd7' \
--header 'X-API-Key: sk-xxxxx'

响应:

json
{
    "message": "Chunk deleted",
    "success": true
}

DELETE `/chunks/:knowledge_id` - 删除知识下的所有分块

路径参数:

字段 类型 说明
knowledge_id string 知识 ID

请求:

curl
curl --location --request DELETE 'http://localhost:8080/api/v1/chunks/4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5' \
--header 'X-API-Key: sk-xxxxx'

响应:

json
{
    "message": "All chunks under knowledge deleted",
    "success": true
}

GET `/chunks/by-id/:id` - 根据 ID 直接获取分块

无需提供 knowledge_id 即可获取分块。常用于跨知识库的引用展示。

路径参数:

字段 类型 说明
id string 分块 ID

请求:

curl
curl --location 'http://localhost:8080/api/v1/chunks/by-id/df10b37d-cd05-4b14-ba8a-e1bd0eb3bbd7' \
--header 'X-API-Key: sk-xxxxx'

响应: 同 GET /chunks/:knowledge_id 列表中的单条 data。

DELETE `/chunks/by-id/:id/questions` - 删除分块下的某个生成问题

删除指定分块关联的某条生成问题。

路径参数:

字段 类型 说明
id string 分块 ID

参数说明（请求体）:

字段 类型 必填 说明
question_id string 是 问题 ID

请求:

curl
curl --location --request DELETE 'http://localhost:8080/api/v1/chunks/by-id/df10b37d-cd05-4b14-ba8a-e1bd0eb3bbd7/questions' \
--header 'X-API-Key: sk-xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "question_id": "q-00000001"
}'

响应:

json
{
    "message": "Question deleted successfully",
    "success": true
}

2. Official Technical Reference & Guides (Tencent/Tencent.github.io)

File: konajdk/sm/README.md

Tencent Kona JDK ShangMi Reference Guide

Overview of ShangMi

ShangMi is a set of cryptographic techniques and tools in commercial or business applications, including the encryption of sensitive business data, secure communication channels, digital signatures for document verification, and even more.

Generally, the below ShangMi cryptographic algorithms and secure communication protocols are widely applied.

ShangMi cryptographic algorithms

  • SM2, which is a public key algorithm based on Elliptic Curve Cryptography (ECC). It complies with China's specifications:
    • GB/T 32918.1-2016 Part 1：General
    • GB/T 32918.2-2016 Part 2：Digital signature algorithm
    • GB/T 32918.3-2016 Part 3：Key exchange protocol
    • GB/T 32918.4-2016 Part 4：Public key encryption algorithm
    • GB/T 32918.5-2017 Part 5：Parameter definition
  • SM3, which is a cryptographic hash algorithm. It complies with China's specification:
    • GB/T 32905-2016 SM3 cryptographic hash algorithm
  • SM4, which is a block encryption algorithm. It complies with China's specification:
    • GB/T 32907-2016 SM4 block cipher algorithm

ShangMi secure communication protocols

  • Transport Layer Cryptography Protocol (TLCP), which is a TLS liked protocol. This protocol requires an endpoint must have two certificates, one for signature and the other for encryption. It complies China's specification:
    • GB/T 38636-2020 Information security technology-Transport layer cryptography protocol (TLCP)
  • ShangMi (SM) Cipher Suites for TLS 1.3, which defines elements for applying ShangMi algorithms over TLS 1.3. It complies IETF's Informational specification:
    • RFC 8998 ShangMi (SM) Cipher Suites for TLS 1.3

Providers and implementations

Tencent Kona JDK extends Java Cryptography Architecture (JCA) and Java Cryptography Extension (JCE) providers, exactly SUN, SunEC and SunJCE, to support ShangMi cryptographic algorithms, and extends Java Secure Socket Extension (JSSE) provider, exactly SunJSSE, to support ShangMi secure communication protocols.

The algorithms are implemented by the different security providers in the JDK. The followings list the security providers and the corresponding supported ShangMi algorithms and protocols.

  • SUN
    • MessageDigest
      • SM3
  • SunEC
    • KeyAgreement
      • SM2
    • KeyFactory
      • SM2
    • Signature
      • SM3withSM2
  • SunJCE
    • Cipher
      • SM4
    • KeyGenerator
      • HmacSM3
      • SM4
    • Mac
      • HmacSM3
  • SunJSSE
    • SSLContext
      • TLCP 1.1
        • ECC_SM4_GCM_SM3
        • ECC_SM4_CBC_SM3
        • ECDHE_SM4_GCM_SM3
        • ECDHE_SM4_CBC_SM3
      • TLS 1.3/RFC 8998
        • TLS_SM4_GCM_SM3

Since Tencent Kona JDK just uses the existing OpenJDK security providers to accommodate the ShangMi stuff, and these providers are already installed with the highest priorities, so the application developers can use these out-of-the-box features as those JDK built-in algorithms (e.g. EC and AES) and protocols (e.g. TLS 1.3).

The following sections will unveil the details on using the above algorithms and protocols.

SM2

Key pair

Generating SM2 key pair is the same as generating the key pairs on other EC curves. It just needs to invoke the standard JDK APIs. The JDK builtin ECKeyPairGenerator. In the key pair generated by this implementation, the format of private key is PKCS#8 and the format of public key is X.509.

Create KeyPairGenerator implementation for EC algorithm.

text
KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("EC);
keyPairGenerator.initialize(spec);

spec can be SM2ParameterSpec (use SM2ParameterSpec.instance() to create the instance) or ECGenParameterSpec (use new ECGenParameterSpec("curveSM2") to create the instance).

Generate key pair.

text
KeyPair keyPair = keyPairGenerator.generateKeyPair();
ECPublicKey publicKey = (ECPublicKey) keyPair.getPublic();
ECPrivateKey privateKey = (ECPrivateKey) keyPair.getPrivate();

SM2 key pair is also EC key pair, so the public key and private key are also ECPublicKey and ECPrivateKey respectively.

For more information on key pair generation, please refer to the official KeyPairGenerator docs.

Generally, in the signing and encrypting operations, the key pairs are already generated. They are not generated on the runtime. It just reads the existing public key and private key to create PublicKey and PrivateKey instances respectively.

text
byte[] encodedPublicKey = ;
byte[] encodedPrivateKey = ;

KeyFactory keyFactory = KeyFactory.getInstance("SM2");

SM2PublicKeySpec publicKeySpec = new SM2PublicKeySpec(encodedPublicKey);
PublicKey publicKey = keyFactory.generatePublic(publicKeySpec);

SM2PrivateKeySpec privateKeySpec = new SM2PrivateKeySpec(encodedPrivateKey);
PrivateKey privateKey = keyFactory.generatePrivate(privateKeySpec);

Signature

Using SM2 signature is the same as using other existing signatures, like ECDSA, though the parameters are different.

Create Signature instance.

text
Signature signature = Signature.getInstance("SM3withSM2);

Initialize the Signature instance with the private key for signing.

text
signature.initSign(privateKey);

Behind the above initialization, it uses the default SM2 ID, exactly 1234567812345678. It also derives the public key from the private key. According to the associated specification, public key must be used for generating SM2 signature data. This is a significant difference between the international algorithms, like ECDSA, and SM2 signature algorithm. Note that calculating the public key leads to some overhead. That is harmful for the performance.

If not using the default ID or expect to eliminate the cost on calculating public key, it would provide an AlgorithmParameterSpec, exactly SM2SignatureParameterSpec, instance.

text
byte[] altID = <custom ID>;
ECPublicKey publicKey = ;
SM2SignatureParameterSpec paramSpec = new SM2SignatureParameterSpec(altID, publicKey);
signature.setParameter(paramSpec);
signature.initSign(privateKey);

After parameter setup and initialization, it is time to pass the message data in.

text
byte[] message = ;
signature.update(message);

Generate signature data.

text
byte[] sign = signature.sign();

Initialize the Signature instance with public key for verifying.

text
signature.initVerify(publicKey);

Pass the message in.

text
byte[] message = ;
signature.update(message);

Pass the signature data in.

text
boolean verified = signature.verify(sign);

If the verification is successful, the result is true, otherwise false.

It has to know that the private key is used for signing, however the public key is used for verifying. For detailed information about Signature APIs, please refer to the official Signature docs.

Encryption

Because of the performance concern, public key encryption is only used for encrypting short but critical data. The same to SM2 encryption.

Create Cipher instance.

text
Cipher cipher = Cipher.getInstance("SM2");

Initialize the Cipher instance with public key and set the mode to encryption.

text
cipher.init(Cipher.ENCRYPT_MODE, publicKey);

Pass the message in and generate the ciphertext.

text
byte[] message = ;
byte[] ciphertext = cipher.doFinal(message);

Initialize the Cipher instance with private key and set the mode to decryption.

text
cipher.init(Cipher.DECRYPT_MODE, privateKey);

Pass the ciphertext in and generate the plaintext.

text
byte[] cleartext = cipher.doFinal(ciphertext);

It has to know that public key is used for encryption, however the private key is used for decryption. For detailed information on Cipher APIs, please refer to the official Cipher docs.

SM3

Using SM3 hash algorithm is the same as using other existing hash algorithm, like SHA-256. It only invokes JDK APIs to generate the message digest (hash).

Create MessageDigest instance.

text
MessageDigest md = MessageDigest.getInstance("SM3");

It can pass all the message in and generate the digest.

text
byte[] message = ;
byte[] digest = md.digest(message);

It also can pass the message segments, and finally generate the digest.

text
byte[] messageSegment1 = ;
byte[] messageSegment2 = ;

// Pass the message segments in
md.update(messageSegment1);
md.update(messageSegment2);

// Finally generate the digest
byte[] digest = md.digest();

For detailed information about MessageDigest APIs, please refer to the official MessageDigest docs.

HmacSM3

Using HmacSM3 algorithm is the same as using other existing MAC algorithm, like HmacSHA256. It only invokes JDK APIs to generate the message authentication code.

Prepare a 16-bytes key.

text
byte[] key = ;
SecretKey secretKey = new SecretKeySpec(key, "SM4");

Create Mac instance.

text
Mac hmac = Mac.getInstance("HmacSM3");

Initialize the Mac instance with the key.

text
hmac.init(secretKey);

Pass all the message in and generate message authentication code. The code is 32-bytes length.

text
byte[] message = ;
byte[] mac = hmac.doFinal(message);

It also can pass the message segments, and finally generate the message authentication code.

text
byte[] messageSegment1 = ;
byte[] messageSegment2 = ;

// Pass the message segments in
hmac.update(messageSegment1);
hmac.update(messageSegment2);

// Finally generate the message authentication code
byte[] mac = hmac.doFinal();

For the detailed information about Mac APIs, please refer to the official Mac docs.

SM4

Using SM4 cipher algorithm is the same as using other existing block cipher algorithm, like AES. It only invokes JDK APIs to do encryption and decryption. Tencent Kona JDK supports four operation modes, including CBC, CTR, ECB and GCM. It also supports PKCS#7 padding specification.

Prepare a 16-bytes key.

text
byte[] key = ;
SecretKey secretKey = new SecretKeySpec(key, "SM4");

Create Cipher instance.

text
Cipher cipher = Cipher.getInstance(transformation);

The transformation consists of algorithm name, operation mode and padding type. A symbol / is used as separator between the parts.

The following transformations are supported:

  • SM4/CBC/NoPadding: The operation mode is CBC. No padding is needed. It requires the input must be 16x bytes.
  • SM4/CBC/PKCS7Padding: The operation mode is CBC. PKCS#7 padding is used. It doesn't require that the input must be 16x bytes.
  • SM4/CTR/NoPadding: The operation mode is CBC. No padding is needed. It doesn't require that the input must be 16x bytes.
  • SM4/ECB/NoPadding: The operation mode is ECB. No padding is needed. It requires the input must be 16x bytes.
  • SM4/ECB/PKCS7Padding: The operation mode is ECB. PKCS#7 padding is used. It doesn't require that the input must be 16x bytes.
  • SM4/GCM/NoPadding: The operation mode is GCM. No padding is needed. It doesn't require that the input must be 16x bytes.

Create the algorithm parameters.

text
AlgorithmParameterSpec paramSpec = ;

Different operation modes may need different AlgorithmParameterSpec types,

  • CBC and CTR modes use IvParameterSpec. This parameter type needs a 16-bytes as Initialization Vector (IV).
  • GCM mode uses GCMParameterSpec. This parameter type needs a 12-bytes as Initialization Vector (IV) and specifies the tag length to 12-bytes.

Initialize the Cipher instance with the key and parameter type and set the mode to encryption.

text
cipher.init(Cipher.ENCRYPT_MODE, secretKey, paramSpec);

Pass all the message in and generate the ciphertext.

text
byte[] ciphertext = cipher.doFinal(message);

Initialize the Cipher instance with the key and parameter type and set the mode to decryption.

text
cipher.init(Cipher.DECRYPT_MODE, secretKey);

Pass all the ciphertext in and generate the plaintext.

text
byte[] cleartext = cipher.doFinal(ciphertext);

It also can receive the plaintext/ciphertext segments in, and then generate the ciphertext/plaintext segments.

text
byte[] input1 = <segment1>;
byte[] input2 = <segment2>;

// Pass the plaintext/ciphertext segments in
byte[] output1 = cipher.update(input1);
byte[] output2 = cipher.update(input2);

// Generate the ciphertext/plaintext segments
byte[] outputFinal = cipher.doFinal();

For the detailed information about Cipher APIs, please refer to the official [Cipher] docs.

Certificate

The so-called ShangMi certificate also complies to X.509 specification, though the public key algorithm is EC with SM2 curve and the signature algorithm is SM3withSM2.

text
String certPEM = <certificate in PEM format>;
CertificateFactory cf = CertificateFactory.getInstance("X.509");
X509Certificate certificate = (X509Certificate) cf.generateCertificate(
        new ByteArrayInputStream(certPEM.getBytes(StandardCharsets.UTF_8));

For more information on certificate parsing, please read the official reference on CertificateFactory.

It can parse public key from the ShangMi certificate.

text
PublicKey publicKey = certificate.getPublicKey();

publicKey can be cast to ECPublicKey.

It can parse private key from PKCS#8 format.

text
String peyPEM = ;
PKCS8EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(
        Base64.getMimeDecoder().decode(keyPem));
KeyFactory keyFactory = KeyFactory.getInstance("EC");
PrivateKey privateKey = keyFactory.generatePrivate(privateKeySpec);

privateKey can be cast to ECPrivateKey.

It can sign data with PKCS#8 private key and verify the signature with X.509 certificate.

Create Signature instance and use SM3withSM2 signature algorithm.

text
Signature signature = Signature.getInstance("SM3withSM2");

Sign the message with the private key and generate the signature data.

text
signature.initSign(privateKey);
signature.update(message);
byte[] sign = signature.sign();

Verify the signature data with the certificate and determine whether the verification passes or fails.

text
signature.initVerify(certificate);
signature.update(message);
boolean verified = signature.verify(sign);

The certificate has to contain digitalSignature key usage for verifying signature data. But it also can verify the signature with the public key directly.

text
signature.initVerify(publicKey);
signature.update(message);
boolean verified = signature.verify(sign);

TLCP 1.1 and TLS 1.3/RFC 8998

In order to take advantage of the implementations on TLCP 1.1 and TLS 1.3/RFC 8998, the most important point is taking SSLSocket or SSLEngine to apply the SSLContext implementation that supports the protocols.

text
KeyStore trustStore = <a trust store carring the CAs>;
TrustManagerFactory tmf = TrustManagerFactory.getInstance("PKIX");
tmf.init(trustStore);

KeyStore keyStore = <a key store carring the end entity certificates>;
KeyManagerFactory kmf = KeyManagerFactory.getInstance("NewSunX509");
kmf.init(keyStore, keyStorePassword);

SSLContext context = SSLContext.getInstance("TLCPv1.1");
context.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new SecureRandom());

When create SSLContext instances, it allows to specify the following context protocols:

  • TLCPv1.1, which indicates only TLCP 1.1 is supported.
  • TLSv1.3, which indicates only TLS 1.3 is supported.
  • TLCP, which indicates TLCP 1.1, TLS 1.3 and TLS 1.2 are supported.

Please read the official JSSE reference for understanding it deeply.

File: konajdk/README.md

Tencent Kona

Tencent Kona is a no-cost, production-ready distribution of the Open Java Development Kit (OpenJDK), Long-Term Support (LTS) with quarterly updates.

Kona serves as the default JDK at Tencent internally, optimized for extreme-scale of Big Data, Machine Learning and Cloud Computing workloads. It is also used to build and maintain a Tencent supported version of OpenJDK for Tencent customers and partners who wish to use OpenJDK to run their applications.

Kona also implements China's cryptographic algorithms, including SM2, SM3 and SM4, and secure communication protocol TLCP (or SMSSL). For more details on these features, please refer to the reference guide.

Supported JDK Releases

Tencent Kona currently supports the JDK Long-Term Support (LTS) releases 8, 11, 17, 21 and 25.
For more details, please go through the individual repository for each of these releases:

Supported System Platforms

Tencent Kona currently supports Linux/x86_64, Linux/aarch64, Windows/x86_64, macOS/x86_64 and macOS/aarch64.

License

Tencent Kona is under the same licensing terms as the upstream OpenJDK project. It is clearly a "friendly fork". Please find the license text for more details.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#agent Browse agent catalog → #agentic Browse agentic catalog → #ai Browse ai catalog → #chatbot Browse chatbot catalog → #embeddings Browse embeddings catalog → #evaluation Browse evaluation catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →