kotaemon Documentation

File: README.md Copy Chapter

Introduction Copy Chapter

This project serves as a functional RAG UI for both end users who want to do QA on their
documents and developers who want to build their own RAG pipeline.

+----------------------------------------------------------------------------+
| End users: Those who use apps built with `kotaemon`.                       |
| (You use an app like the one in the demo above)                            |
|     +----------------------------------------------------------------+     |
|     | Developers: Those who built with `kotaemon`.                   |     |
|     | (You have `import kotaemon` somewhere in your project)         |     |
|     |     +----------------------------------------------------+     |     |
|     |     | Contributors: Those who make `kotaemon` better.    |     |     |
|     |     | (You make PR to this repo)                         |     |     |
|     |     +----------------------------------------------------+     |     |
|     +----------------------------------------------------------------+     |
+----------------------------------------------------------------------------+

For end users Copy Section

• Clean & Minimalistic UI: A user-friendly interface for RAG-based QA.
• Support for Various LLMs: Compatible with LLM API providers (OpenAI, AzureOpenAI, Cohere, etc.) and local LLMs (via ollama and llama-cpp-python).
• Easy Installation: Simple scripts to get you started quickly.

For developers Copy Section

• Framework for RAG Pipelines: Tools to build your own RAG-based document QA pipeline.
• Customizable UI: See your RAG pipeline in action with the provided UI, built with Gradio .
• Gradio Theme: If you use Gradio for development, check out our theme here: kotaemon-gradio-theme.

Key Features Copy Chapter

• Host your own document QA (RAG) web-UI: Support multi-user login, organize your files in private/public collections, collaborate and share your favorite chat with others.

• Organize your LLM & Embedding models: Support both local LLMs & popular API providers (OpenAI, Azure, Ollama, Groq).

• Hybrid RAG pipeline: Sane default RAG pipeline with hybrid (full-text & vector) retriever and re-ranking to ensure best retrieval quality.

• Multi-modal QA support: Perform Question Answering on multiple documents with figures and tables support. Support multi-modal document parsing (selectable options on UI).

• Advanced citations with document preview: By default the system will provide detailed citations to ensure the correctness of LLM answers. View your citations (incl. relevant score) directly in the in-browser PDF viewer with highlights. Warning when retrieval pipeline return low relevant articles.

• Support complex reasoning methods: Use question decomposition to answer your complex/multi-hop question. Support agent-based reasoning with ReAct, ReWOO and other agents.

• Configurable settings UI: You can adjust most important aspects of retrieval & generation process on the UI (incl. prompts).

• Extensible: Being built on Gradio, you are free to customize or add any UI elements as you like. Also, we aim to support multiple strategies for document indexing & retrieval. GraphRAG indexing pipeline is provided as an example.

Installation Copy Chapter

If you are not a developer and just want to use the app, please check out our easy-to-follow User Guide. Download the .zip file from the latest release to get all the newest features and bug fixes.

System requirements Copy Section

1. Python >= 3.10
2. Docker: optional, if you install with Docker
3. Unstructured if you want to process files other than .pdf, .html, .mhtml, and .xlsx documents. Installation steps differ depending on your operating system. Please visit the link and follow the specific instructions provided there.

With Docker (recommended) Copy Section

1. We support both lite & full version of Docker images. With full version, the extra packages of unstructured will be installed, which can support additional file types (.doc, .docx, ...) but the cost is larger docker image size. For most users, the lite image should work well in most cases.

   • To use the full version.

     shell Copy

     docker run \
     -e GRADIO_SERVER_NAME=0.0.0.0 \
     -e GRADIO_SERVER_PORT=7860 \
     -v ./ktem_app_data:/app/ktem_app_data \
     -p 7860:7860 -it --rm \
     ghcr.io/cinnamon/kotaemon:main-full

   • To use the full version with bundled Ollama for local / private RAG.

     shell Copy

     # change image name to
     docker run <...> ghcr.io/cinnamon/kotaemon:main-ollama

   • To use the lite version.

   shell Copy

   # change image name to
    docker run <...> ghcr.io/cinnamon/kotaemon:main-lite

2. We currently support and test two platforms: linux/amd64 and linux/arm64 (for newer Mac). You can specify the platform by passing --platform in the docker run command. For example:

   shell Copy

   # To run docker with platform linux/arm64
   docker run \
   -e GRADIO_SERVER_NAME=0.0.0.0 \
   -e GRADIO_SERVER_PORT=7860 \
   -v ./ktem_app_data:/app/ktem_app_data \
   -p 7860:7860 -it --rm \
   --platform linux/arm64 \
   ghcr.io/cinnamon/kotaemon:main-lite

3. Once everything is set up correctly, you can go to http://localhost:7860/ to access the WebUI.

4. We use GHCR to store docker images, all images can be found here.

Without Docker Copy Section

1. Clone the repository:

   shell Copy

   git clone https://github.com/Cinnamon/kotaemon
   cd kotaemon

2. Setup the environment:

• Option 1: Using uv (recommended)

  shell Copy

  uv sync --python 3.10
  source .venv/bin/activate

• Option 2: Using conda

  shell Copy

  conda create -n kotaemon python=3.10
  conda activate kotaemon
  
  pip install -e "libs/kotaemon[all]"
  pip install -e "libs/ktem"

3. Create a .env file in the root of this project. Use .env.example as a template.

   The .env file is there to serve use cases where users want to pre-config the models before starting up the app (e.g. deploy the app on HF hub). The file will only be used to populate the db once upon the first run, it will no longer be used in consequent runs.

4. (Optional) To enable in-browser PDF_JS viewer, download PDF_JS_DIST then extract it to libs/ktem/ktem/assets/prebuilt.

5. Start the web server:

   shell Copy

   python app.py

   • The app will be automatically launched in your browser.
   • Default username and password are both admin. You can set up additional users directly through the UI.

6. Check the Resources tab and LLMs and Embeddings and ensure that your api_key value is set correctly from your .env file. If it is not set, you can set it there.

Setup GraphRAG Copy Section

[!NOTE]
Official MS GraphRAG indexing only works with OpenAI or Ollama API.
We recommend most users to use NanoGraphRAG implementation for straightforward integration with Kotaemon.

Setup Nano GRAPHRAG

• Install nano-GraphRAG: pip install nano-graphrag
• nano-graphrag install might introduce version conflicts, see this issue
  • To quickly fix: pip uninstall hnswlib chroma-hnswlib && pip install chroma-hnswlib
• Launch Kotaemon with USE_NANO_GRAPHRAG=true environment variable.
• Set your default LLM & Embedding models in Resources setting and it will be recognized automatically from NanoGraphRAG.

Setup LIGHTRAG

• Install LightRAG: pip install git+https://github.com/HKUDS/LightRAG.git
• LightRAG install might introduce version conflicts, see this issue
  • To quickly fix: pip uninstall hnswlib chroma-hnswlib && pip install chroma-hnswlib
• Launch Kotaemon with USE_LIGHTRAG=true environment variable.
• Set your default LLM & Embedding models in Resources setting and it will be recognized automatically from LightRAG.

Setup MS GRAPHRAG

• Non-Docker Installation: If you are not using Docker, install GraphRAG with the following command:

  shell Copy

  pip install "graphrag<=0.3.6" future

• Setting Up API KEY: To use the GraphRAG retriever feature, ensure you set the GRAPHRAG_API_KEY environment variable. You can do this directly in your environment or by adding it to a .env file.

• Using Local Models and Custom Settings: If you want to use GraphRAG with local models (like Ollama) or customize the default LLM and other configurations, set the USE_CUSTOMIZED_GRAPHRAG_SETTING environment variable to true. Then, adjust your settings in the settings.yaml.example file.

Setup Local Models (for local/private RAG) Copy Section

See Local model setup.

Setup multimodal document parsing (OCR, table parsing, figure extraction) Copy Section

These options are available:

• Azure Document Intelligence (API)
• Adobe PDF Extract (API)
• Docling (local, open-source) – see integrations/docling.md for Kotaemon-specific setup.
• PaddleOCR (local, open-source) – see integrations/paddle_ocr.md for Kotaemon-specific setup.

Select corresponding loaders in Settings -> Retrieval Settings -> File loader

Customize your application Copy Section

• By default, all application data is stored in the ./ktem_app_data folder. You can back up or copy this folder to transfer your installation to a new machine.

• For advanced users or specific use cases, you can customize these files:

  • flowsettings.py
  • .env

`flowsettings.py`

This file contains the configuration of your application. You can use the example
here as the starting point.

Notable settings

# setup your preferred document store (with full-text search capabilities)
KH_DOCSTORE=(Elasticsearch | LanceDB | SimpleFileDocumentStore)

# setup your preferred vectorstore (for vector-based search)
KH_VECTORSTORE=(ChromaDB | LanceDB | InMemory | Milvus | Qdrant)

# Enable / disable multimodal QA
KH_REASONINGS_USE_MULTIMODAL=True

# Setup your new reasoning pipeline or modify existing one.
KH_REASONINGS = [
    "ktem.reasoning.simple.FullQAPipeline",
    "ktem.reasoning.simple.FullDecomposeQAPipeline",
    "ktem.reasoning.react.ReactAgentPipeline",
    "ktem.reasoning.rewoo.RewooAgentPipeline",
]

`.env`

This file provides another way to configure your models and credentials.

Configure model via the .env file

• Alternatively, you can configure the models via the .env file with the information needed to connect to the LLMs. This file is located in the folder of the application. If you don't see it, you can create one.

• Currently, the following providers are supported:

  • OpenAI

    In the .env file, set the OPENAI_API_KEY variable with your OpenAI API key in order
    to enable access to OpenAI's models. There are other variables that can be modified,
    please feel free to edit them to fit your case. Otherwise, the default parameter should
    work for most people.

    shell Copy

    OPENAI_API_BASE=https://api.openai.com/v1
    OPENAI_API_KEY=<your OpenAI API key here>
    OPENAI_CHAT_MODEL=gpt-3.5-turbo
    OPENAI_EMBEDDINGS_MODEL=text-embedding-ada-002

  • Azure OpenAI

    For OpenAI models via Azure platform, you need to provide your Azure endpoint and API
    key. You might also need to provide your developments' name for the chat model and the
    embedding model depending on how you set up Azure development.

    shell Copy

    AZURE_OPENAI_ENDPOINT=
    AZURE_OPENAI_API_KEY=
    OPENAI_API_VERSION=2024-02-15-preview
    AZURE_OPENAI_CHAT_DEPLOYMENT=gpt-35-turbo
    AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT=text-embedding-ada-002

  • Local Models

    • Using ollama OpenAI compatible server:

      • Install ollama and start the application.

      • Pull your model, for example:

        shell Copy

        ollama pull llama3.1:8b
        ollama pull nomic-embed-text

      • Set the model names on web UI and make it as default:

    • Using GGUF with llama-cpp-python

      You can search and download a LLM to be ran locally from the Hugging Face Hub. Currently, these model formats are supported:

      • GGUF

        You should choose a model whose size is less than your device's memory and should leave
        about 2 GB. For example, if you have 16 GB of RAM in total, of which 12 GB is available,
        then you should choose a model that takes up at most 10 GB of RAM. Bigger models tend to
        give better generation but also take more processing time.

        Here are some recommendations and their size in memory:

      • Qwen1.5-1.8B-Chat-GGUF: around 2 GB

        Add a new LlamaCpp model with the provided model name on the web UI.

Adding your own RAG pipeline Copy Section

Custom Reasoning Pipeline

1. Check the default pipeline implementation in here. You can make quick adjustment to how the default QA pipeline work.
2. Add new .py implementation in libs/ktem/ktem/reasoning/ and later include it in flowssettings to enable it on the UI.

Custom Indexing Pipeline

• Check sample implementation in libs/ktem/ktem/index/file/graph

(more instruction WIP).

Citation Copy Chapter

Please cite this project as

@misc{kotaemon2024,
    title = {Kotaemon - An open-source RAG-based tool for chatting with any content.},
    author = {The Kotaemon Team},
    year = {2024},
    howpublished = {\url{https://github.com/Cinnamon/kotaemon}},
}

Star History Copy Chapter

Contribution Copy Chapter

Since our project is actively being developed, we greatly value your feedback and contributions. Please see our Contributing Guide to get started. Thank you to all our contributors!

File: docs/development/contributing.md Copy Chapter

Contributing

Setting up Copy Chapter

• Clone the repo

  shell Copy

  git clone [email protected]:Cinnamon/kotaemon.git
  cd kotaemon

• Install the environment

  • Create a conda environment (python >= 3.10 is recommended)

    shell Copy

    conda create -n kotaemon python=3.10
    conda activate kotaemon
    
    # install dependencies
    cd libs/kotaemon
    pip install -e ".[all]"

  • Or run the installer (one of the scripts/run_* scripts depends on your OS), then
    you will have all the dependencies installed as a conda environment at
    install_dir/env.

    shell Copy

    conda activate install_dir/env

• Pre-commit

  shell Copy

  pre-commit install

• Test

  shell Copy

  pytest tests

Package overview Copy Chapter

kotaemon library focuses on the AI building blocks to implement a RAG-based QA application. It consists of base interfaces, core components and a list of utilities:

• Base interfaces: kotaemon defines the base interface of a component in a pipeline. A pipeline is also a component. By clearly define this interface, a pipeline of steps can be easily constructed and orchestrated.
• Core components: kotaemon implements (or wraps 3rd-party libraries
  like Langchain, llama-index,... when possible) commonly used components in
  kotaemon use cases. Some of these components are: LLM, vector store,
  document store, retriever... For a detailed list and description of these
  components, please refer to the API Reference section.
• List of utilities: kotaemon provides utilities and tools that are
  usually needed in client project. For example, it provides a prompt
  engineering UI for AI developers in a project to quickly create a prompt
  engineering tool for DMs and QALs. It also provides a command to quickly spin
  up a project code base. For a full list and description of these utilities,
  please refer to the Utilities section.

mindmap
  root((kotaemon))
    Base Interfaces
      Document
      LLMInterface
      RetrievedDocument
      BaseEmbeddings
      BaseChat
      BaseCompletion
      ...
    Core Components
      LLMs
        AzureOpenAI
        OpenAI
      Embeddings
        AzureOpenAI
        OpenAI
        HuggingFaceEmbedding
      VectorStore
        InMemoryVectorstore
        ChromaVectorstore
      Agent
      Tool
      DocumentStore
      ...
    Utilities
      Scaffold project
      PromptUI
      Documentation Support

Common conventions Copy Chapter

• PR title: One-line description (example: Feat: Declare BaseComponent and decide LLM call interface).
• [Encouraged] Provide a quick description in the PR, so that:
  • Reviewers can quickly understand the direction of the PR.
  • It will be included in the commit message when the PR is merged.

Environment caching on PR Copy Chapter

• To speed up CI, environments are cached based on the version specified in __init__.py.
• Since dependencies versions in setup.py are not pinned, you need to pump the version in order to use a new environment. That environment will then be cached and used by your subsequence commits within the PR, until you pump the version again
• The new environment created during your PR is cached and will be available to others once the PR is merged.
• If you are experimenting with new dependencies and want a fresh environment every time, add [ignore cache] in your commit message. The CI will create a fresh environment to run your commit and then discard it.
• If your PR include updated dependencies, the recommended workflow would be:
  • Doing development as usual.
  • When you want to run the CI, push a commit with the message containing [ignore cache].
  • Once the PR is final, pump the version in __init__.py and push a final commit not containing [ignore cache].

Merge PR guideline Copy Chapter

• Use squash and merge option
• 1st line message is the PR title.
• The text area is the PR description.

File: docs/development/create-a-component.md Copy Chapter

Creating a component

A fundamental concept in kotaemon is "component".

Anything that isn't data or data structure is a "component". A component can be
thought of as a step within a pipeline. It takes in some input, processes it,
and returns an output, just the same as a Python function! The output will then
become an input for the next component in a pipeline. In fact, a pipeline is just
a component. More appropriately, a nested component: a component that makes use of one or more other components in
the processing step. So in reality, there isn't a difference between a pipeline
and a component! Because of that, in kotaemon, we will consider them the
same as "component".

To define a component, you will:

1. Create a class that subclasses from kotaemon.base.BaseComponent
2. Declare init params with type annotation
3. Declare nodes (nodes are just other components!) with type annotation
4. Implement the processing logic in run.

The syntax of a component is as follow:

from kotaemon.base import BaseComponent
from kotaemon.llms import LCAzureChatOpenAI
from kotaemon.parsers import RegexExtractor


class FancyPipeline(BaseComponent):
    param1: str = "This is param1"
    param2: int = 10
    param3: float

    node1: BaseComponent    # this is a node because of BaseComponent type annotation
    node2: LCAzureChatOpenAI  # this is also a node because LCAzureChatOpenAI subclasses BaseComponent
    node3: RegexExtractor   # this is also a node bceause RegexExtractor subclasses BaseComponent

    def run(self, some_text: str):
        prompt = (self.param1 + some_text) * int(self.param2 + self.param3)
        llm_pred = self.node2(prompt).text
        matches = self.node3(llm_pred)
        return matches

Then this component can be used as follow:

llm = LCAzureChatOpenAI(endpoint="some-endpont")
extractor = RegexExtractor(pattern=["yes", "Yes"])

component = FancyPipeline(
    param1="Hello"
    param3=1.5
    node1=llm,
    node2=llm,
    node3=extractor
)
component("goodbye")

This way, we can define each operation as a reusable component, and use them to
compose larger reusable components!

Benefits of component Copy Chapter

By defining a component as above, we formally encapsulate all the necessary
information inside a single class. This introduces several benefits:

1. Allow tools like promptui to inspect the inner working of a component in
   order to automatically generate the promptui.
2. Allow visualizing a pipeline for debugging purpose.

File: docs/development/data-components.md Copy Chapter

Data & Data Structure Components

The data & data structure components include:

• The Document class.
• The document store.
• The vector store.

Data Loader Copy Chapter

• PdfLoader

• Layout-aware with table parsing PdfLoader

  • MathPixLoader: To use this loader, you need MathPix API key, refer to mathpix docs for more information

  • OCRLoader: This loader uses lib-table and Flax pipeline to perform OCR and read table structure from PDF file (TODO: add more info about deployment of this module).

  • Output:

    • Document: text + metadata to identify whether it is table or not

      text Copy

      - "source": source file name
      - "type": "table" or "text"
      - "table_origin": original table in markdown format (to be feed to LLM or visualize using external tools)
      - "page_label": page number in the original PDF document

Document Store Copy Chapter

• InMemoryDocumentStore

Vector Store Copy Chapter

• ChromaVectorStore
• InMemoryVectorStore

File: docs/development/index.md Copy Chapter

{%
include-markdown "../../README.md"
start=""
end=""
%}

File: docs/development/utilities.md Copy Chapter

Utilities

Prompt engineering UI Copy Chapter

Important: despite the name prompt engineering UI, this tool allows testers to test any kind of parameters that are exposed by developers. Prompt is one kind of param. There can be other type of params that testers can tweak (e.g. top_k, temperature...).

In the development process, developers typically build the pipeline. However, for use
cases requiring expertise in prompt creation, non-technical members (testers, domain experts) can be more
effective. To facilitate this, kotaemon offers a user-friendly prompt engineering UI
that developers integrate into their pipelines. This enables non-technical members to
adjust prompts and parameters, run experiments, and export results for optimization.

As of Sept 2023, there are 2 kinds of prompt engineering UI:

• Simple pipeline: run one-way from start to finish.
• Chat pipeline: interactive back-and-forth.

Simple pipeline Copy Section

For simple pipeline, the supported client project workflow looks as follow:

1. [tech] Build pipeline
2. [tech] Export pipeline to config: $ kotaemon promptui export <module.path.piplineclass> --output
3. [tech] Customize the config
4. [tech] Spin up prompt engineering UI: $ kotaemon promptui run
5. [non-tech] Change params, run inference
6. [non-tech] Export to Excel
7. [non-tech] Select the set of params that achieve the best output

The prompt engineering UI prominently involves from step 2 to step 7 (step 1 is normally
done by the developers, while step 7 happens exclusively in Excel file).

Step 2 - Export pipeline to config

Command:

$ kotaemon promptui export <module.path.piplineclass> --output

where:

• <module.path.pipelineclass> is a dot-separated path to the pipeline. For example, if your pipeline can be accessed with from projectA.pipelines import AnsweringPipeline, then this value is projectA.pipelines.AnswerPipeline.
• `` is the target file path that the config will be exported to. If the config file already exists, and contains information of other pipelines, the config of current pipeline will additionally be added. If it contains information of the current pipeline (in the past), the old information will be replaced.

By default, all params in a pipeline (including nested params) will be export to the configuration file. For params that you do not wish to expose to the UI, you can directly remove them from the config YAML file. You can also annotate those param with ignore_ui=True, and they will be ignored in the config generation process. Example:

class Pipeline(BaseComponent):
    param1: str = Param(default="hello")
    param2: str = Param(default="goodbye", ignore_ui=True)

Declared as above, and param1 will show up in the config YAML file, while param2 will not.

Step 3 - Customize the config

developers can further edit the config file in this step to get the most suitable UI (step 4) with their tasks. The exported config will have this overall schema:

<module.path.pipelineclass1>:
  params: ... (Detail param information to initiate a pipeline. This corresponds to the pipeline init parameters.)
  inputs: ... (Detail the input of the pipeline e.g. a text prompt. This corresponds to the params of `run(...)` method.)
  outputs: ... (Detail the output of the pipeline e.g. prediction, accuracy... This is the output information we wish to see in the UI.)
  logs: ... (Detail what information should show up in the log.)

Input and params

The inputs section have the overall schema as follow:

inputs:
  :
    component: <supported-UI-component>
    params: # this section is optional)
      value: <default-value>
  : ... # similar to above
params:
  : ... # similar to those in the inputs

The list of supported prompt UI and their corresponding gradio UI components:

COMPONENTS_CLASS = {
    "text": gr.components.Textbox,
    "checkbox": gr.components.CheckboxGroup,
    "dropdown": gr.components.Dropdown,
    "file": gr.components.File,
    "image": gr.components.Image,
    "number": gr.components.Number,
    "radio": gr.components.Radio,
    "slider": gr.components.Slider,
}

Outputs

The outputs are a list of variables that we wish to show in the UI. Since in Python, the function output doesn't have variable name, so output declaration is a little bit different than input and param declaration:

outputs:
  - component: <supported-UI-component>
    step: <name-of-pipeline-step>
    item: <jsonpath way to retrieve the info>
  - ... # similar to above

where:

• component: the same text string and corresponding Gradio UI as in inputs & params
• step: the pipeline step that we wish to look fetch and show output on the UI
• item: the jsonpath mechanism to get the targeted variable from the step above

Logs

The logs show a list of sheetname and how to retrieve the desired information.

logs:
  <logname>:
    inputs:
      - name: <column name>
        step: 
        variable: 
      - ...
    outputs:
      - name: <column name>
        step: 
        item: <how to retrieve the output of that step>

Step 4 + 5 - Spin up prompt engineering UI + Perform prompt engineering

Command:

$ kotaemon promptui run

This will generate an UI as follow:

where:

• The tabs at the top of the UI corresponds to the pipeline to do prompt engineering.
• The inputs and params tabs allow users to edit (these corresponds to the inputs and params in the config file).
• The outputs panel holds the UI elements to show the outputs defined in config file.
• The Run button: will execute pipeline with the supplied inputs and params, and render result in the outputs panel.
• The Export button: will export the logs of all the run to an Excel files users to inspect for best set of params.

Step 6 - Export to Excel

Upon clicking export, the users can download Excel file.

Chat pipeline Copy Section

Chat pipeline workflow is different from simple pipeline workflow. In simple pipeline, each Run creates a set of output, input and params for users to compare. In chat pipeline, each Run is not a one-off run, but a long interactive session. Hence, the workflow is as follow:

1. Set the desired parameters.
2. Click "New chat" to start a chat session with the supplied parameters. This set of parameters will persist until the end of the chat session. During an ongoing chat session, changing the parameters will not take any effect.
3. Chat and interact with the chat bot on the right panel. You can add any additional input (if any), and they will be supplied to the chatbot.
4. During chat, the log of the chat will show up in the "Output" tabs. This is empty by default, so if you want to show the log here, tell the AI developers to configure the UI settings.
5. When finishing chat, select your preference in the radio box. Click "End chat". This will save the chat log and the preference to disk.
6. To compare the result of different run, click "Export" to get an Excel spreadsheet summary of different run.

File: docs/pages/app/ext/user-management.md Copy Chapter

ktem provides user management as an extension. To enable user management, in
your flowsettings.py, set the following variables:

• KH_FEATURE_USER_MANAGEMENT: True to enable.
• KH_FEATURE_USER_MANAGEMENT_ADMIN: the admin username. This user will be
  created when the app 1st start.
• KH_FEATURE_USER_MANAGEMENT_PASSWORD: the admin password. This value
  accompanies the admin username.

Once enabled, you have access to the following features:

• User login/logout (located in Settings Tab)
• User changing password (located in Settings Tab)
• Create / List / Edit / Delete user (located in Resources > Users Tab)

File: docs/pages/app/index/file.md Copy Chapter

The file index stores files in a local folder and index them for retrieval.
This file index provides the following infrastructure to support the indexing:

• SQL table Source: store the list of files that are indexed by the system
• Vector store: contain the embedding of segments of the files
• Document store: contain the text of segments of the files. Each text stored
  in this document store is associated with a vector in the vector store.
• SQL table Index: store the relationship between (1) the source and the
  docstore, and (2) the source and the vector store.

The indexing and retrieval pipelines are encouraged to use the above software
infrastructure.

Indexing pipeline Copy Chapter

The ktem has default indexing pipeline: ktem.index.file.pipelines.IndexDocumentPipeline.

This default pipeline works as follow:

• Input: list of file paths
• Output: list of nodes that are indexed into database
• Process:
  • Read files into texts. Different file types has different ways to read texts.
  • Split text files into smaller segments
  • Run each segments into embeddings.
  • Store the embeddings into vector store. Store the texts of each segment
    into docstore. Store the list of files in Source. Store the linking
    between Sources and docstore + vectorstore in Index table.

You can customize this default pipeline if your indexing process is close to the
default pipeline. You can create your own indexing pipeline if there are too
much different logic.

Customize the default pipeline Copy Section

The default pipeline provides the contact points in flowsettings.py.

1. FILE_INDEX_PIPELINE_FILE_EXTRACTORS. Supply overriding file extractor,
   based on file extension. Example: {".pdf": "path.to.PDFReader", ".xlsx": "path.to.ExcelReader"}
2. FILE_INDEX_PIPELINE_SPLITTER_CHUNK_SIZE. The expected number of characters
   of each text segment. Example: 1024.
3. FILE_INDEX_PIPELINE_SPLITTER_CHUNK_OVERLAP. The expected number of
   characters that consecutive text segments should overlap with each other.
   Example: 256.

Create your own indexing pipeline Copy Section

Your indexing pipeline will subclass BaseFileIndexIndexing.

You should define the following methods:

• run(self, file_paths): run the indexing given the pipeline
• get_pipeline(cls, user_settings, index_settings): return the
  fully-initialized pipeline, ready to be used by ktem.
  • user_settings: is a dictionary contains user settings (e.g. {"pdf_mode": True, "num_retrieval": 5}). You can declare these settings in the get_user_settings classmethod. ktem will collect these settings into the app Settings page, and will supply these user settings to your get_pipeline method.
  • index_settings: is a dictionary. Currently it's empty for File Index.
• get_user_settings: to declare user settings, return a dictionary.

By subclassing BaseFileIndexIndexing, You will have access to the following resources:

• self._Source: the source table
• self._Index: the index table
• self._VS: the vector store
• self._DS: the docstore

Once you have prepared your pipeline, register it in flowsettings.py: FILE_INDEX_PIPELINE = "".

Retrieval pipeline Copy Chapter

The ktem has default retrieval pipeline:
ktem.index.file.pipelines.DocumentRetrievalPipeline. This pipeline works as
follow:

• Input: user text query & optionally a list of source file ids
• Output: the output segments that match the user text query
• Process:
  • If a list of source file ids is given, get the list of vector ids that
    associate with those file ids.
  • Embed the user text query.
  • Query the vector store. Provide a list of vector ids to limit query scope
    if the user restrict.
  • Return the matched text segments

Create your own retrieval pipeline Copy Section

Your retrieval pipeline will subclass BaseFileIndexRetriever. The retriever
has the same database, vectorstore and docstore accesses like the indexing
pipeline.

You should define the following methods:

• run(self, query, file_ids): retrieve relevant documents relating to the
  query. If file_ids is given, you should restrict your search within these
  file_ids.
• get_pipeline(cls, user_settings, index_settings, selected): return the
  fully-initialized pipeline, ready to be used by ktem.
  • user_settings: is a dictionary contains user settings (e.g. {"pdf_mode": True, "num_retrieval": 5}). You can declare these settings in the get_user_settings classmethod. ktem will collect these settings into the app Settings page, and will supply these user settings to your get_pipeline method.
    • index_settings: is a dictionary. Currently it's empty for File Index.
    • selected: a list of file ids selected by user. If user doesn't select
      anything, this variable will be None.
• get_user_settings: to declare user settings, return a dictionary.

Once you build the retrieval pipeline class, you can register it in
flowsettings.py: FILE_INDEXING_RETRIEVER_PIPELIENS = ["path.to.retrieval.pipelie"]. Because there can be
multiple parallel pipelines within an index, this variable takes a list of
string rather than a string.

Software infrastructure Copy Chapter

File: docs/pages/app/settings/overview.md Copy Chapter

Overview

There are 3 kinds of settings in ktem, geared towards different stakeholders
for different use cases:

• Developer settings. These settings are meant for very basic app customization, such as database URL, cloud config, logging config, which features to enable... You will be interested in the developer settings if you deploy ktem to your customers, or if you build extension for ktem for developers. These settings are declared inside flowsettings.py.
• Admin settings. These settings show up in the Admin page, and are meant to allow admin-level user to customize low level features, such as which credentials to connect to data sources, which keys to use for LLM...
• User settings. These settings are meant for run-time users to tweak ktem to their personal needs, such as which output languages the chatbot should generate, which reasoning type to use...

File: docs/pages/app/settings/user-settings.md Copy Chapter

User settings

ktem allows developers to extend the index and the reasoning pipeline. In
many cases, these components can have settings that should be modified by
users at run-time, (e.g. topk, chunksize...). These are the user settings.

ktem allows developers to declare such user settings in their code. Once
declared, ktem will render them in a Settings page.

There are 2 places that ktem looks for declared user settings. You can
refer to the respective pages.

• In the index.
• In the reasoning pipeline.

Syntax of a settings Copy Chapter

A collection of settings is a dictionary of type dict[str, dict], where the
key is a setting id, and the value is the description of the setting.

settings = {
    "topk": {
        "name": "Top-k chunks",
        "value": 10,
        "component": "number",
    },
    "lang": {
        "name": "Languages",
        "value": "en",
        "component": "dropdown",
        "choices": [("en", "English"), ("cn", "Chinese")],
    }
}

Each setting description must have:

• name: the human-understandable name of the settings.

• value: the default value of the settings.

• component: the UI component to render such setting on the UI. Available:

  • "text": single-value
  • "number": single-value
  • "checkbox": single-value
  • "dropdown": choices
  • "radio": choices
  • "checkboxgroup": choices

• choices: the list of choices, if the component type allows.

Settings page structure Copy Chapter