anything-llm Documentation

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy anything-llm on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run)
docker run -d --name anything-llm -p 3001:3001 --restart always mintplex-labs/anything-llm:latest
Configured Services & Ports
βš™οΈ anything-llm
image: build locally
πŸ”Œ Ports: 3001:3001
docker-compose.yml
yaml (docker-compose.yml)
name: anythingllm

networks:
  anything-llm:
    driver: bridge

services:
  anything-llm:
    container_name: anythingllm
    build:
      context: ../.
      dockerfile: ./docker/Dockerfile
      args:
        ARG_UID: ${UID:-1000}
        ARG_GID: ${GID:-1000}
    cap_add:
      - SYS_ADMIN
    volumes:
      - "./.env:/app/server/.env"
      - "../server/storage:/app/server/storage"
      - "../collector/hotdir/:/app/collector/hotdir"
      - "../collector/outputs/:/app/collector/outputs"
    user: "${UID:-1000}:${GID:-1000}"
    ports:
      - "3001:3001"
    env_file:
      - .env
    networks:
      - anything-llm
    extra_hosts:
      - "host.docker.internal:host-gateway"

Technical Documentation: Mintplex-Labs/anything-llm

ℹ️ Provenance: Multi-Tier Ingestion Β· CodeWiki Reference

Overview (`README.md`)

Chat with your docs. Automate complex workflows with AI Agents. Hyper-configurable, multi-user ready, battle-testedβ€”and runs locally by default with zero setup friction.

Watch the demo!

Product Overview

AnythingLLM is the all-in-one AI application that lets you build a private, fully-featured ChatGPTβ€”without compromises. Connect your favorite local or cloud LLM, ingest your documents, and start chatting in minutes. Out of the box you get built-in agents, multi-user support, vector databases, and document pipelines β€” no extra configuration required.

AnythingLLM supports multiple users as well where you can control the access and experience per user without compromising the security or privacy of the instance or your intellectual property.

Cool Features of AnythingLLM

Supported LLMs, Embedder Models, Speech models, and Vector Databases

Large Language Models (LLMs):

Embedder models:

Audio Transcription models:

TTS (text-to-speech) support:

STT (speech-to-text) support:

  • Native Browser Built-in (default)

Vector Databases:

Technical Overview

This monorepo consists of six main sections:

  • frontend: A viteJS + React frontend that you can run to easily create and manage all your content the LLM can use.
  • server: A NodeJS express server to handle all the interactions and do all the vectorDB management and LLM interactions.
  • collector: NodeJS express server that processes and parses documents from the UI.
  • docker: Docker instructions and build process + information for building from source.
  • embed: Submodule for generation & creation of the web embed widget.
  • browser-extension: Submodule for the chrome browser extension.

πŸ›³ Self-Hosting

Mintplex Labs & the community maintain a number of deployment methods, scripts, and templates that you can use to run AnythingLLM locally. Refer to the table below to read how to deploy on your preferred environment or to automatically deploy.

Docker AWS GCP Digital Ocean Render.com
Deploy on Docker Deploy on AWS Deploy on GCP Deploy on DigitalOcean Deploy on Render.com
Railway RepoCloud Elestio Northflank Sealos
Deploy on Railway Deploy on RepoCloud Deploy on Elestio Deploy on Northflank Deploy on Sealos
Easypanel
Deploy on Easypanel

or set up a production AnythingLLM instance without Docker β†’

How to setup for development

  • yarn setup To fill in the required .env files you'll need in each of the application sections (from root of repo).
    • Go fill those out before proceeding. Ensure server/.env.development is filled or else things won't work right.
  • yarn dev:server To boot the server locally (from root of repo).
  • yarn dev:frontend To boot the frontend locally (from root of repo).
  • yarn dev:collector To then run the document collector (from root of repo).

Learn about documents

Telemetry & Privacy

AnythingLLM by Mintplex Labs Inc contains a telemetry feature that collects anonymous usage information.

More about Telemetry & Privacy for AnythingLLM

Why?

We use this information to help us understand how AnythingLLM is used, to help us prioritize work on new features and bug fixes, and to help us improve AnythingLLM's performance and stability.

Opting out

Set DISABLE_TELEMETRY in your server or docker .env settings to "true" to opt out of telemetry. You can also do this in-app by going to the sidebar > Privacy and disabling telemetry.

What do you explicitly track?

We will only track usage details that help us make product and roadmap decisions, specifically:

  • Type of your installation (Docker or Desktop)

  • When a document is added or removed. No information about the document. Just that the event occurred. This gives us an idea of use.

  • Type of vector database in use. This helps us prioritize changes when updates arrive for that provider.

  • Type of LLM provider & model tag in use. This helps us prioritize changes when updates arrive for that provider or model, or combination thereof. eg: reasoning vs regular, multi-modal models, etc.

  • When a chat is sent. This is the most regular "event" and gives us an idea of the daily-activity of this project across all installations. Again, only the event is sent - we have no information on the nature or content of the chat itself.

You can verify these claims by finding all locations Telemetry.sendTelemetry is called. Additionally these events are written to the output log so you can also see the specific data which was sent - if enabled. No IP or other identifying information is collected. The Telemetry provider is PostHog - an open-source telemetry collection service.

We take privacy very seriously, and we hope you understand that we want to learn how our tool is used, without using annoying popup surveys, so we can build something worth using. The anonymous data is never shared with third parties, ever.

View all telemetry events in source code

Other outbound connections

If you disable telemetry, you would still see outbound connections to the following services:

  • If using an external tool, LLM, Embedding models, or Vector databases, you will still see outbound connections to the respective service provider.
  • cdn.anythingllm.com for pulling models from our mirror CDN. This is not tracked by telemetry and is actually useful for those in VPN restricted regions.
  • github/githubusercontent.com There are some various flat files that are downloaded from these domains for context window caching.

Basically, if telemetry is disabled we don't collect anything. However, depending on your setup you may still see outbound connections and would be subject to the terms of service of the respective service provider.

πŸ‘‹ Contributing

πŸ’– Sponsors

All Sponsors

🌟 Contributors

πŸ”— More Products

[


Copyright Β© 2026 Mintplex Labs.
This project is MIT licensed.


Overview (`.devcontainer/README.md`)

AnythingLLM Development Container Setup

Welcome to the AnythingLLM development container configuration, designed to create a seamless and feature-rich development environment for this project.

**PLEASE READ THIS**

Prerequisites

Features

  • Base Image: Built on mcr.microsoft.com/devcontainers/javascript-node:1-18-bookworm, thus Node.JS LTS v18.
  • Additional Tools: Includes hadolint, and essential apt-packages such as curl, gnupg, and more.
  • Ports: Configured to auto-forward ports 3000 (Frontend) and 3001 (Backend).
  • Environment Variables: Sets NODE_ENV to development and ESLINT_USE_FLAT_CONFIG to true.
  • VS Code Extensions: A suite of extensions such as Prettier, Docker, ESLint, and more are automatically installed. Please revise if you do not agree with any of these extensions. AI-powered extensions and time trackers are (for now) not included to avoid any privacy concerns, but you can install them later in your own environment.

Getting Started

  1. Using GitHub Codespaces. Just select to create a new workspace, and the devcontainer will be created for you.

  2. Using your Local VSCode (Release or Insiders). We suggest you first make a fork of the repo and then clone it to your local machine using VSCode tools. Then open the project folder in VSCode, which will prompt you to open the project in a devcontainer. Select yes, and the devcontainer will be created for you. If this does not happen, you can open the command palette and select "Remote-Containers: Reopen in Container".

On Creation:

When the container is built for the first time, it will automatically run yarn setup to ensure everything is in place for the Collector, Server and Frontend. This command is expected to be automatically re-run if there is a content change on next reboot.

Work in the Container:

Once the container is up, be patient. Some extensions may complain because dependencies are still being installed, and in the Extensions tab, some may ask you to "Reload" the project. Don't do that yet. First, wait until all settle down for the first time. We suggest you create a new VSCode profile for this devcontainer, so any configuration and extensions you change, won't affect your default profile.

Checklist:

  • The usual message asking you to start the Server and Frontend in different windows are now "hidden" in the building process of the devcontainer. Don't forget to do as suggested.
  • Open a JavaScript file, for example "server/index.js" and check if eslint is working. It will complain that 'err' is defined but never used.. This means it is working.
  • Open a React File, for example, "frontend/src/main.jsx," and check if eslint complains about Fast refresh only works when a file has exports. Move your component(s) to a separate file.. Again, it means eslint is working. Now check at the status bar if the Prettier has a double checkmark :heavy_check_mark: (double). It means Prettier is working. You will see a nice extension Formatting::heavy_check_mark: that can be used to disable the Format on Save feature temporarily.
  • Check if, on the left pane, you have the NPM Scripts (this may be disabled; look at the "Explorer" tree-dots up-right). There will be scripts inside the package.json files. You will basically need to run the dev:collector, dev:server and the dev:frontend in this order. When the frontend finishes starting, a window browser will open inside the VSCode. Still, you can open it outside.

:warning: Important for all developers :warning:

  • When you are using the NODE_ENV=development the server will not store the configurations you set for security reasons. Please set the proper config on file .env.development. The side-effect if you don't, everytime you restart the server, you will be sent to the "Onboarding" page again.

Note when using GitHub Codespaces

  • When running the "Server" for the first time, it will automatically configure its port to be publicly accessible by default, as this is required for the front end to reach the server backend. To know more, read the content of the .env file on the frontend folder about this, and if any issues occur, make sure to manually set the port "Visibility" of the "Server" is set to "Public" if needed. Again, this is only needed for developing on GitHub Codespaces.

For the Collector:

  • In the past, the Collector dwelled within the Python domain, but now it has journeyed to the splendid realm of Node.JS. Consequently, the configuration complexities of bygone versions are no longer a concern.

Now it is ready to start

In the status bar you will see three shortcuts names Collector, Server and Frontend. Just click-and-wait on that order (don't forget to set the Server port 3001 to Public if you are using GH Codespaces before starting the Frontend).

Now you can enjoy your time developing instead of reconfiguring everything.

Debugging with the devcontainers

For debugging the collector, server and frontend

First, make sure the built-in extension (ms-vscode.js-debug) is active (I don't know why it would not be, but just in case). If you want, you can install the nightly version (ms-vscode.js-debug-nightly)

Then, in the "Run and Debug" tab (Ctrl+shift+D), you can select on the menu:

  • Collector debug. This will start the collector in debug mode and attach the debugger. Works very well.
  • Server debug. This will start the server in debug mode and attach the debugger. Works very well.
  • Frontend debug. This will start the frontend in debug mode and attach the debugger. I am still struggling with this one. I don't know if VSCode can handle the .jsx files seamlessly as the pure .js on the server. Maybe there is a need for a particular configuration for Vite or React. Anyway, it starts. Another two configurations launch Chrome and Edge, and I think we could add breakpoints on .jsx files somehow. The best scenario would be always to use the embedded browser. WIP.

Please leave comments on the Issues tab or the


Overview (`cloud-deployments/openshift/README.md`)

[!IMPORTANT]
This is a community-maintained template and is not officially supported by the AnythingLLM team. You could encounter issues or even deployment failures in future versions of AnythingLLM. We do our best to keep this template and all community contributions backwards compatible, but we cannot guarantee it.

OpenShift Deployment Template for AnythingLLM

This directory contains a specialized Dockerfile and entrypoint script for deploying AnythingLLM on Red Hat OpenShift clusters.

Why This Template Exists

OpenShift has a unique security model that differs from standard Docker/Kubernetes deployments:

  1. Arbitrary UIDs: OpenShift runs containers with randomly assigned user IDs (UIDs) that don't exist in /etc/passwd
  2. GID 0 Requirement: All containers run with GID 0 (root group) as the primary group
  3. Restricted SCCs: The default Security Context Constraints (SCCs) prevent containers from running as specific users

These requirements are incompatible with the standard AnythingLLM Docker image, which uses a fixed anythingllm user with UID/GID 1000.

Key Differences from Standard Dockerfile

Feature Standard Docker OpenShift Template
File ownership anythingllm:anythingllm anythingllm:0 (root group)
File permissions Standard Group-writable (g+w)
/etc/passwd Read-only Group-writable for UID injection
Supplementary groups None Added to group 0
Entrypoint Standard Handles arbitrary UID scenarios

When to Use This Template

Use this template only if you are deploying to:

  • Red Hat OpenShift (any version)
  • OKD (OpenShift Origin)
  • Any Kubernetes cluster with OpenShift-style restricted SCCs

Do NOT use this for:

  • Standard Docker deployments
  • Docker Compose
  • Generic Kubernetes (use the standard image with appropriate securityContext)
  • Cloud container services (AWS ECS, Google Cloud Run, Azure Container Instances)

Building the Image

From the repository root:

bash
docker build -f cloud-deployments/openshift/Dockerfile -t anythingllm:openshift .

For multi-architecture builds:

bash
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -f cloud-deployments/openshift/Dockerfile \
  -t your-registry/anythingllm:openshift \
  --push .

Deploying to OpenShift

Using `oc` CLI

bash
# Create a new project (namespace)
oc new-project anythingllm

# Create a deployment
oc new-app your-registry/anythingllm:openshift

# Expose the service
oc expose svc/anythingllm --port=3001

# Set required environment variables
oc set env deployment/anythingllm \
  STORAGE_DIR=/app/server/storage \
  JWT_SECRET=$(openssl rand -hex 32)

Using a DeploymentConfig YAML

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: anythingllm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: anythingllm
  template:
    metadata:
      labels:
        app: anythingllm
    spec:
      containers:
      - name: anythingllm
        image: your-registry/anythingllm:openshift
        ports:
        - containerPort: 3001
        env:
        - name: STORAGE_DIR
          value: /app/server/storage
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: anythingllm-secrets
              key: jwt-secret
        volumeMounts:
        - name: storage
          mountPath: /app/server/storage
      volumes:
      - name: storage
        persistentVolumeClaim:
          claimName: anythingllm-storage

Persistent Storage

OpenShift PersistentVolumeClaims work with this image. Ensure the PVC is created before deployment:

yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: anythingllm-storage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

Troubleshooting

Permission Denied Errors

If you see permission errors, verify:

  1. You're using this OpenShift-specific image, not the standard one
  2. The PVC has correct access modes
  3. No custom SCCs are overriding the default behavior

User Not Found in passwd

The entrypoint script automatically handles this by injecting a passwd entry at runtime. If issues persist, check that /etc/passwd is group-writable in your image.


Overview (`extras/translator/README.md`)

AnythingLLM Auto-translater

The AnythingLLM Auto-translator is a way for us to translate our locales but at no cost or overhead. However these are expected to be run manually and while they may not be 100% accurate improvement in models have made this work trivial without the need for an api.

Getting started

  • Install Ollama w/Docker Model Runner
  • ollama pull translategemma:4b
  • cd extras/translator && cp .env.example .env

Run the script

All translations are based on english dictionary. So the English dictionary must have an entry for it to be supported.

cd extras/translator

Target a specific language
node index.mjs <lang-code> eg: node index.mjs es for Spanish.

Do all languages
node index.mjs --all this is NOT recommended

Gotchas

  • Sometimes translations go on for a while until the token window is exceeded - you can see this by the massive lines extending beyond the page.
  • Some languages operate different with single words or special symbols like "@" and will go off the rails. If the English text is one word and the translated text is 100 words, you dont need to be linguist to know that it is probably wrong.
  • You should always review every line for discrepencies or removal of {{}} inputs or brand name hallunications eg: AnyLLM instead of AnythingLLM

Overview (`server/storage/README.md`)

AnythingLLM Storage

This folder is for the local or disk storage of ready-to-embed documents, vector-cached embeddings, and the disk-storage of LanceDB and the local SQLite database.

This folder should contain the following folders.
documents
lancedb (if using lancedb)
vector-cache
and a file named exactly anythingllm.db

Common issues

SQLITE_FILE_CANNOT_BE_OPENED in the server log = The DB file does not exist probably because the node instance does not have the correct permissions to write a file to the disk. To solve this..

  • Local dev

    • Create a anythingllm.db empty file in this directory. Thats all. No need to reboot the server or anything. If your permissions are correct this should not ever occur since the server will create the file if it does not exist automatically.
  • Docker Instance

    • Get your AnythingLLM docker container id with docker ps -a. The container must be running to execute the next commands.

    • Run docker container exec -u 0 -t <ANYTHINGLLM DOCKER CONTAINER ID> mkdir -p /app/server/storage /app/server/storage/documents /app/server/storage/vector-cache /app/server/storage/lancedb

    • Run docker container exec -u 0 -t <ANYTHINGLLM DOCKER CONTAINER ID> touch /app/server/storage/anythingllm.db

    • Run docker container exec -u 0 -t <ANYTHINGLLM DOCKER CONTAINER ID> chown -R anythingllm:anythingllm /app/collector /app/server

    • The above commands will create the appropriate folders inside of the docker container and will persist as long as you do not destroy the container and volume. This will also fix any ownership issues of folder files in the collector and the server.


Overview (`server/storage/models/README.md`)

Native models used by AnythingLLM

This folder is specifically created as a local cache and storage folder that is used for native models that can run on a CPU.

Currently, AnythingLLM uses this folder for the following parts of the application.

Embedding

When your embedding engine preference is native we will use the ONNX all-MiniLM-L6-v2 model built by Xenova on HuggingFace.co. This model is a quantized and WASM version of the popular all-MiniLM-L6-v2 which produces a 384-dimension vector.

If you are using the native embedding engine your vector database should be configured to accept 384-dimension models if that parameter is directly editable (Pinecone only).

Audio/Video transcription

AnythingLLM allows you to upload various audio and video formats as source documents. In all cases the audio tracks will be transcribed by a locally running ONNX model whisper-small built by Xenova on HuggingFace.co. The model is a smaller version of the OpenAI Whisper model. Given the model runs locally on CPU, larger files will result in longer transcription times.

Once transcribed you can embed these transcriptions into your workspace like you would any other file!

Other external model/transcription providers are also live.

Text generation (LLM selection)

[!IMPORTANT]
Use of a locally running LLM model is experimental and may behave unexpectedly, crash, or not function at all.
We suggest for production-use of a local LLM model to use a purpose-built inference server like LocalAI or LMStudio.

[!TIP]
We recommend at least using a 4-bit or 5-bit quantized model for your LLM. Lower quantization models tend to
just output unreadable garbage.

If you would like to use a local Llama compatible LLM model for chatting you can select any model from this HuggingFace search filter

Requirements

  • Model must be in the latest GGUF format
  • Model should be compatible with latest llama.cpp
  • You should have the proper RAM to run such a model. Requirement depends on model size.

Where do I put my GGUF model?

[!IMPORTANT]
If running in Docker you should be running the container to a mounted storage location on the host machine so you
can update the storage files directly without having to re-download or re-build your docker container. See suggested Docker config

[!NOTE]
/server/storage/models/downloaded is the default location that your model files should be at.
Your storage directory may differ if you changed the STORAGE_DIR environment variable.

All local models you want to have available for LLM selection should be placed in the server/storage/models/downloaded folder. Only .gguf files will be allowed to be selected from the UI.


Overview (`cloud-deployments/helm/charts/anythingllm/README.md`)

anythingllm

AnythingLLM

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.

Configuration & Usage

  • Config vs Secrets: This chart exposes application configuration via two mechanisms:

    • config (in values.yaml) β€” rendered into a ConfigMap and injected using envFrom in the pod. Do NOT place sensitive values (API keys, secrets) in config because ConfigMaps are not encrypted.
    • env / envFrom β€” the preferred way to inject secrets. Use Kubernetes Secret objects and reference them from env (with valueFrom.secretKeyRef) or envFrom.secretRef.
  • Storage & STORAGE_DIR mapping: The chart creates (or mounts) a PersistentVolumeClaim using the persistentVolume.* settings. The container mount path is set from persistentVolume.mountPath. Ensure the container STORAGE_DIR config key matches that path (defaults are set in values.yaml).

Providing API keys & secrets (recommended)

Use Kubernetes Secrets. Below are example workflows and values.yaml snippets.

  1. Create a Kubernetes Secret with API keys:
text
kubectl create secret generic openai-secret --from-literal=OPENAI_KEY="sk-..."
# or from a file
# kubectl create secret generic openai-secret --from-file=OPENAI_KEY=/path/to/keyfile
  1. Reference the Secret from values.yaml using envFrom (recommended when your secret contains multiple env keys):
yaml
envFrom:
  - secretRef:
      name: openai-secret

This will inject all key/value pairs from the openai-secret Secret as environment variables in the container.

  1. Or reference a single secret key via env (explicit mapping):
yaml
env:
  - name: OPENAI_KEY
    valueFrom:
      secretKeyRef:
        name: openai-secret
        key: OPENAI_KEY

Notes:

  • Avoid placing secret values into config: (the chart's ConfigMap) β€” ConfigMaps are visible to anyone who can read the namespace. Use Secret objects for any credentials/tokens.
  • If you use a GitOps workflow, consider integrating an external secret operator (ExternalSecrets, SealedSecrets, etc.) so you don't store raw secrets in Git.

Example values-secret.yaml to pass during helm install

yaml
image:
  repository: mintplexlabs/anythingllm
  tag: "1.18.0"

service:
  type: ClusterIP
  port: 3001

# Reference secret containing API keys
envFrom:
  - secretRef:
      name: openai-secret

# Optionally override other values
persistentVolume:
  size: 16Gi
  mountPath: /storage

Install with:

text
helm install my-anythingllm ./anythingllm -f values-secret.yaml

Best practices & tips

  • Use envFrom for convenience when many environment variables are stored in a single Secret and use env/valueFrom for explicit single-key mappings.
  • Use kubectl create secret generic or your secrets management solution. If you need to reference multiple different provider keys (OpenAI, Anthropic, etc.), create a single Secret with multiple keys or multiple Secrets and add multiple envFrom entries.
  • Keep probe paths and service.port aligned. If your probes fail after deployment, check that the probe port matches the container port (or named port http) and that the path is valid.
  • For storage, if you have a pre-existing PVC set persistentVolume.existingClaim to the PVC name; the chart will mount that claim (and will not attempt to create a new PVC).
  • For production, provide resource requests and limits in values.yaml to prevent scheduler starvation and to control cost.

Values

Key Type Default Description
affinity object {}
config.DISABLE_TELEMETRY string "true"
config.GID string "1000"
config.NODE_ENV string "production"
config.STORAGE_DIR string "/storage"
config.UID string "1000"
env object {}
envFrom object {}
fullnameOverride string ""
image.pullPolicy string "IfNotPresent"
image.repository string "mintplexlabs/anythingllm"
image.tag string "1.18.0"
imagePullSecrets list []
ingress.annotations object {}
ingress.className string ""
ingress.enabled bool false
ingress.hosts[0].host string "chart-example.local"
ingress.hosts[0].paths[0].path string "/"
ingress.hosts[0].paths[0].pathType string "ImplementationSpecific"
ingress.tls list []
initContainers list []
livenessProbe.failureThreshold int 3
livenessProbe.httpGet.path string "/v1/api/health"
livenessProbe.httpGet.port int 8888
livenessProbe.initialDelaySeconds int 15
livenessProbe.periodSeconds int 5
nameOverride string ""
nodeSelector object {}
persistentVolume.accessModes[0] string "ReadWriteOnce"
persistentVolume.annotations object {}
persistentVolume.existingClaim string ""
persistentVolume.labels object {}
persistentVolume.mountPath string "/storage"
persistentVolume.size string "8Gi"
podAnnotations object {}
podLabels object {}
podSecurityContext.fsGroup int 1000
readinessProbe.httpGet.path string "/v1/api/health"
readinessProbe.httpGet.port int 8888
readinessProbe.initialDelaySeconds int 15
readinessProbe.periodSeconds int 5
readinessProbe.successThreshold int 2
replicaCount int 1
resources object {}
securityContext object {}
service.port int 3001
service.type string "ClusterIP"
serviceAccount.annotations object {}
serviceAccount.automount bool true
serviceAccount.create bool true
serviceAccount.name string ""
tolerations list []
volumeMounts list []
volumes list []

--- METRICS ---