# Technical Documentation: Mintplex-Labs/anything-llm > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/Mintplex-Labs/anything-llm) ### Overview (`README.md`)

AnythingLLM logo

Mintplex-Labs%2Fanything-llm | Trendshift

AnythingLLM: The all-in-one AI app you were looking for.
Chat with your docs, use AI Agents, hyper-configurable, multi-user, & no frustrating setup required.

Discord | License | Docs | Hosted Instance

English · 简体中文 · 日本語

👉 AnythingLLM for desktop (Mac, Windows, & Linux)! Download Now

📱 AnythingLLM Mobile (Android) is open source! View the repo

Chat with your docs. Automate complex workflows with AI Agents. Hyper-configurable, multi-user ready, battle-tested—and runs locally by default with zero setup friction.
Watch the demo! [](https://youtu.be/f95rGD9trL0)
### Product Overview AnythingLLM is the all-in-one AI application that lets you build a private, fully-featured ChatGPT—without compromises. Connect your favorite local or cloud LLM, ingest your documents, and start chatting in minutes. Out of the box you get built-in agents, multi-user support, vector databases, and document pipelines — no extra configuration required. AnythingLLM supports multiple users as well where you can control the access and experience per user without compromising the security or privacy of the instance or your intellectual property. ## Cool Features of AnythingLLM - [Built-in local LLM engine](/docker/HOW_TO_USE_DOCKER.md#built-in-llm-engine-optional) - Run local models on your CPU or GPU (NVIDIA via CUDA, AMD and Intel via Vulkan) with no extra setup, powered by [Lemonade](https://lemonade-server.ai) and [llama.cpp](https://github.com/ggml-org/llama.cpp). - [Dynamic Model Routing](https://docs.anythingllm.com/model-router/overview) - Automatically route chats to the best provider & model for the conversation based on rules you define. - [Automatic & User Managed Memories](https://docs.anythingllm.com/features/memories) - Have your LLM remember important information about you or your workspace. - [Scheduled Tasks](https://docs.anythingllm.com/scheduled-jobs/overview) - Run recurring tasks or prompts on a cron schedule with full agent capabilities. - [Intelligent Skill Selection](https://docs.anythingllm.com/agent/intelligent-tool-selection) Enable **unlimited** tools for your models while reducing token usage by up to 80% per query - [No-code AI Agent builder](https://docs.anythingllm.com/agent-flows/overview) - [MCP-compatibility](https://docs.anythingllm.com/mcp-compatibility/overview) - [Multi-modal support (both closed and open-source LLMs!)](https://docs.anythingllm.com/features/language-models) - [Custom AI Agents](https://docs.anythingllm.com/agent/custom/introduction) - 👤 Multi-user instance support and permissioning _Docker version only_ - 🎨 [Custom color themes & white-labeling](https://docs.anythingllm.com/features/branding) - Pick a built-in palette like True Dark or create your own from your brand colors, and set it for everyone on your instance. _Docker version only_ - 🦾 Agents inside your workspace (browse the web, etc) - 💬 [Custom Embeddable Chat widget for your website](https://github.com/Mintplex-Labs/anythingllm-embed/blob/main/README.md) _Docker version only_ - 📖 Multiple document type support (PDF, TXT, DOCX, etc) - Intuitive chat UI with drag-and-drop uploads and source citations. - Production-ready for any cloud deployment. - Works with all popular [closed and open-source LLM providers](#supported-llms-embedder-models-speech-models-and-vector-databases). - Built-in optimizations for large document sets—lower costs and faster responses than other chat UIs. - Full Developer API for custom integrations! - ...and much more—install in minutes and see for yourself. ### Supported LLMs, Embedder Models, Speech models, and Vector Databases **Large Language Models (LLMs):** - [Any open-source llama.cpp compatible model](/server/storage/models/README.md#text-generation-llm-selection) - [OpenAI](https://openai.com) - [OpenAI (Generic)](https://openai.com) - [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) - [AWS Bedrock](https://aws.amazon.com/bedrock/) - [Anthropic](https://www.anthropic.com/) - [NVIDIA NIM (chat models)](https://build.nvidia.com/explore/discover) - [Google Gemini Pro](https://ai.google.dev/) - [Ollama (chat models)](https://ollama.ai/) - [LM Studio (all models)](https://lmstudio.ai) - [LocalAI (all models)](https://localai.io/) - [Together AI (chat models)](https://www.together.ai/) - [Fireworks AI (chat models)](https://fireworks.ai/) - [Perplexity (chat models)](https://www.perplexity.ai/) - [OpenRouter (chat models)](https://openrouter.ai/) - [DeepSeek (chat models)](https://deepseek.com/) - [Mistral](https://mistral.ai/) - [Groq](https://groq.com/) - [Cohere](https://cohere.com/) - [KoboldCPP](https://github.com/LostRuins/koboldcpp) - [LiteLLM](https://github.com/BerriAI/litellm) - [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui) - [Apipie](https://apipie.ai/) - [xAI](https://x.ai/) - [Z.AI (chat models)](https://z.ai/model-api) - [Novita AI (chat models)](https://novita.ai/model-api/product/llm-api?utm_source=github_anything-llm&utm_medium=github_readme&utm_campaign=link) - [PPIO](https://ppinfra.com?utm_source=github_anything-llm) - [Gitee AI](https://ai.gitee.com/) - [Moonshot AI](https://www.moonshot.ai/) - [Microsoft Foundry Local](https://github.com/microsoft/Foundry-Local) - [CometAPI (chat models)](https://www.cometapi.com/) - [llmman](https://github.com/llmmanorg/llmman) - [PrivateModeAI (chat models)](https://privatemode.ai/) - [SambaNova Cloud (chat models)](https://cloud.sambanova.ai/) - [Lemonade by AMD](https://lemonade-server.ai) - [Minimax](https://platform.minimax.io) - [Cerebras (chat models)](https://www.cerebras.ai/) - [oMLX](https://github.com/jundot/omlx) **Embedder models:** - [AnythingLLM Native Embedder](/server/storage/models/README.md) (default) - [OpenAI](https://openai.com) - [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) - [Gemini](https://ai.google.dev/) - [LocalAI (all)](https://localai.io/) - [Ollama (all)](https://ollama.ai/) - [LM Studio (all)](https://lmstudio.ai) - [Lemonade](https://lemonade-server.ai) - [OpenRouter](https://openrouter.ai/) - [LiteLLM](https://github.com/BerriAI/litellm) - [Cohere](https://cohere.com/) - [Voyage AI](https://www.voyageai.com/) - [Mistral](https://mistral.ai/) - Generic OpenAI-compatible embedding APIs **Audio Transcription models:** - [AnythingLLM Built-in](https://github.com/Mintplex-Labs/anything-llm/tree/master/server/storage/models#audiovideo-transcription) (default) - [OpenAI](https://openai.com/) **TTS (text-to-speech) support:** - Native Browser Built-in (default) - [PiperTTSLocal - runs in browser](https://github.com/rhasspy/piper) - [OpenAI TTS](https://platform.openai.com/docs/guides/text-to-speech#voice-options) - [ElevenLabs](https://elevenlabs.io/) - Any OpenAI Compatible TTS service. **STT (speech-to-text) support:** - Native Browser Built-in (default) **Vector Databases:** - [LanceDB](https://github.com/lancedb/lancedb) (default) - [PGVector](https://github.com/pgvector/pgvector) - [Astra DB](https://www.datastax.com/products/datastax-astra) - [Pinecone](https://pinecone.io) - [Chroma & ChromaCloud](https://trychroma.com) - [Weaviate](https://weaviate.io) - [Qdrant](https://qdrant.tech) - [Milvus](https://milvus.io) - [Zilliz](https://zilliz.com) ### Technical Overview This monorepo consists of six main sections: - `frontend`: A viteJS + React frontend that you can run to easily create and manage all your content the LLM can use. - `server`: A NodeJS express server to handle all the interactions and do all the vectorDB management and LLM interactions. - `collector`: NodeJS express server that processes and parses documents from the UI. - `docker`: Docker instructions and build process + information for building from source. - `embed`: Submodule for generation & creation of the [web embed widget](https://github.com/Mintplex-Labs/anythingllm-embed). - `browser-extension`: Submodule for the [chrome browser extension](https://github.com/Mintplex-Labs/anythingllm-extension). ## 🛳 Self-Hosting Mintplex Labs & the community maintain a number of deployment methods, scripts, and templates that you can use to run AnythingLLM locally. Refer to the table below to read how to deploy on your preferred environment or to automatically deploy. | Docker | AWS | GCP | Digital Ocean | Render.com | |----------------------------------------|----|-----|---------------|------------| | [![Deploy on Docker][docker-btn]][docker-deploy] | [![Deploy on AWS][aws-btn]][aws-deploy] | [![Deploy on GCP][gcp-btn]][gcp-deploy] | [![Deploy on DigitalOcean][do-btn]][do-deploy] | [![Deploy on Render.com][render-btn]][render-deploy] | | Railway | RepoCloud | Elestio | Northflank | Sealos | | --------------------------------------------------- | --------------------------------------------------------- | --------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------ | | [![Deploy on Railway][railway-btn]][railway-deploy] | [![Deploy on RepoCloud][repocloud-btn]][repocloud-deploy] | [![Deploy on Elestio][elestio-btn]][elestio-deploy] | [![Deploy on Northflank][northflank-btn]][northflank-deploy] | [![Deploy on Sealos][sealos-btn]][sealos-deploy] | | Easypanel | | --------------------------------------------------------- | | [![Deploy on Easypanel][easypanel-btn]][easypanel-deploy] | [or set up a production AnythingLLM instance without Docker →](./BARE_METAL.md) ## How to setup for development - `yarn setup` To fill in the required `.env` files you'll need in each of the application sections (from root of repo). - Go fill those out before proceeding. Ensure `server/.env.development` is filled or else things won't work right. - `yarn dev:server` To boot the server locally (from root of repo). - `yarn dev:frontend` To boot the frontend locally (from root of repo). - `yarn dev:collector` To then run the document collector (from root of repo). [Learn about documents](./server/storage/documents/DOCUMENTS.md) ## Telemetry & Privacy AnythingLLM by Mintplex Labs Inc contains a telemetry feature that collects anonymous usage information.
More about Telemetry & Privacy for AnythingLLM ### Why? We use this information to help us understand how AnythingLLM is used, to help us prioritize work on new features and bug fixes, and to help us improve AnythingLLM's performance and stability. ### Opting out Set `DISABLE_TELEMETRY` in your server or docker .env settings to "true" to opt out of telemetry. You can also do this in-app by going to the sidebar > `Privacy` and disabling telemetry. ### What do you explicitly track? We will only track usage details that help us make product and roadmap decisions, specifically: - Type of your installation (Docker or Desktop) - When a document is added or removed. No information _about_ the document. Just that the event occurred. This gives us an idea of use. - Type of vector database in use. This helps us prioritize changes when updates arrive for that provider. - Type of LLM provider & model tag in use. This helps us prioritize changes when updates arrive for that provider or model, or combination thereof. eg: reasoning vs regular, multi-modal models, etc. - When a chat is sent. This is the most regular "event" and gives us an idea of the daily-activity of this project across all installations. Again, only the **event** is sent - we have no information on the nature or content of the chat itself. You can verify these claims by finding all locations `Telemetry.sendTelemetry` is called. Additionally these events are written to the output log so you can also see the specific data which was sent - if enabled. **No IP or other identifying information is collected**. The Telemetry provider is [PostHog](https://posthog.com/) - an open-source telemetry collection service. We take privacy very seriously, and we hope you understand that we want to learn how our tool is used, without using annoying popup surveys, so we can build something worth using. The anonymous data is _never_ shared with third parties, ever. [View all telemetry events in source code](https://github.com/search?q=repo%3AMintplex-Labs%2Fanything-llm%20.sendTelemetry(&type=code) ### Other outbound connections If you disable telemetry, you would still see outbound connections to the following services: - If using an external tool, LLM, Embedding models, or Vector databases, you will still see outbound connections to the respective service provider. - `cdn.anythingllm.com` for pulling models from our mirror CDN. This is not tracked by telemetry and is actually useful for those in VPN restricted regions. - `github/githubusercontent.com` There are some various flat files that are downloaded from these domains for context window caching. Basically, if telemetry is disabled we don't collect anything. However, depending on your setup you may still see outbound connections and would be subject to the terms of service of the respective service provider.
## 👋 Contributing - [Contributing to AnythingLLM](./CONTRIBUTING.md) - How to contribute to AnythingLLM. ## 💖 Sponsors ### All Sponsors User avatar: JaschaUser avatar: KickAssUser avatar: ShadowArcanistUser avatar: AtlasUser avatar: Predrag StojadinovićUser avatar: Diego SpinolaUser avatar: KyleUser avatar: Giulio De PasqualeUser avatar: User avatar: MacStadiumUser avatar: User avatar: User avatar: User avatar: User avatar: DennisUser avatar: Michael Hamilton, Ph.D.User avatar: User avatar: TernaryLabsUser avatar: Daniel CelaUser avatar: AlessoUser avatar: Rune MathisenUser avatar: User avatar: User avatar: AlanUser avatar: Damien PetersUser avatar: DCS DigitalUser avatar: Paul McilreavyUser avatar: Til WolfUser avatar: Leopoldo Crhistian Riverin GomezUser avatar: AJEsauUser avatar: Steven VanOmmerenUser avatar: Casey BoettcherUser avatar: zebbernUser avatar: AvineetUser avatar: ChrisUser avatar: User avatar: Tim ChampUser avatar: Peter MathisenUser avatar: Ed di GirolamoUser avatar: Wojciech MiłkowskiUser avatar: ADS FundUser avatar: arc46 GmbHUser avatar: Li YinUser avatar: AdaLUser avatar: User avatar: MikUser avatar: ## 🌟 Contributors [](https://github.com/mintplex-labs/anything-llm/graphs/contributors) [](https://star-history.com/#mintplex-labs/anything-llm&Date) ## 🔗 More Products - **[AnythingLLM Mobile (Open-Source Licensed)][anythingllm-mobile]:** The same AnythingLLM on-device expierence, but on your phone. - **[AnythingLLM Browser Extension][anythingllm-extension]:** A browser extension that allows you to use AnythingLLM in your browser. - **[AnythingLLM Embed][anythingllm-embed]:** A widget that allows you to embed AnythingLLM in your website. - **[Open Computer][open-computer]:** A full computer environment for AI agents to use.
[
--- Copyright © 2026 [Mintplex Labs][profile-link].
This project is [MIT](./LICENSE) licensed. [back-to-top]: https://img.shields.io/badge/-BACK_TO_TOP-222628?style=flat-square [profile-link]: https://github.com/mintplex-labs [anythingllm-mobile]: https://github.com/Mintplex-Labs/anythingllm-mobile [anythingllm-extension]: https://github.com/Mintplex-Labs/anythingllm-extension [anythingllm-embed]: https://github.com/Mintplex-Labs/anythingllm-embed [open-computer]: https://github.com/Mintplex-Labs/open-computer [docker-btn]: ./images/deployBtns/docker.png [docker-deploy]: ./docker/HOW_TO_USE_DOCKER.md [aws-btn]: ./images/deployBtns/aws.png [aws-deploy]: ./cloud-deployments/aws/cloudformation/DEPLOY.md [gcp-btn]: https://deploy.cloud.run/button.svg [gcp-deploy]: ./cloud-deployments/gcp/deployment/DEPLOY.md [do-btn]: https://www.deploytodo.com/do-btn-blue.svg [do-deploy]: ./cloud-deployments/digitalocean/terraform/DEPLOY.md [render-btn]: https://render.com/images/deploy-to-render-button.svg [render-deploy]: https://render.com/deploy?repo=https://github.com/Mintplex-Labs/anything-llm&branch=render [render-btn]: https://render.com/images/deploy-to-render-button.svg [render-deploy]: https://render.com/deploy?repo=https://github.com/Mintplex-Labs/anything-llm&branch=render [railway-btn]: https://railway.app/button.svg [railway-deploy]: https://railway.app/template/HNSCS1?referralCode=WFgJkn [repocloud-btn]: https://d16t0pc4846x52.cloudfront.net/deploylobe.svg [repocloud-deploy]: https://repocloud.io/details/?app_id=276 [elestio-btn]: https://elest.io/images/logos/deploy-to-elestio-btn.png [elestio-deploy]: https://elest.io/open-source/anythingllm [northflank-btn]: https://assets.northflank.com/deploy_to_northflank_smm_36700fb050.svg [northflank-deploy]: https://northflank.com/stacks/deploy-anythingllm [sealos-btn]: https://sealos.io/Deploy-on-Sealos.svg [sealos-deploy]: https://sealos.io/products/app-store/anything-llm [easypanel-btn]: https://easypanel.io/img/deploy-on-easypanel-40.svg [easypanel-deploy]: https://easypanel.io/templates/anythingllm --- ### Overview (`.devcontainer/README.md`) # AnythingLLM Development Container Setup Welcome to the AnythingLLM development container configuration, designed to create a seamless and feature-rich development environment for this project.

PLEASE READ THIS

## Prerequisites - [Docker](https://www.docker.com/get-started) - [Visual Studio Code](https://code.visualstudio.com/) - [Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) VS Code extension ## Features - **Base Image**: Built on `mcr.microsoft.com/devcontainers/javascript-node:1-18-bookworm`, thus Node.JS LTS v18. - **Additional Tools**: Includes `hadolint`, and essential apt-packages such as `curl`, `gnupg`, and more. - **Ports**: Configured to auto-forward ports `3000` (Frontend) and `3001` (Backend). - **Environment Variables**: Sets `NODE_ENV` to `development` and `ESLINT_USE_FLAT_CONFIG` to `true`. - **VS Code Extensions**: A suite of extensions such as `Prettier`, `Docker`, `ESLint`, and more are automatically installed. Please revise if you do not agree with any of these extensions. AI-powered extensions and time trackers are (for now) not included to avoid any privacy concerns, but you can install them later in your own environment. ## Getting Started 1. Using GitHub Codespaces. Just select to create a new workspace, and the devcontainer will be created for you. 2. Using your Local VSCode (Release or Insiders). We suggest you first make a fork of the repo and then clone it to your local machine using VSCode tools. Then open the project folder in VSCode, which will prompt you to open the project in a devcontainer. Select yes, and the devcontainer will be created for you. If this does not happen, you can open the command palette and select "Remote-Containers: Reopen in Container". ## On Creation: When the container is built for the first time, it will automatically run `yarn setup` to ensure everything is in place for the Collector, Server and Frontend. This command is expected to be automatically re-run if there is a content change on next reboot. ## Work in the Container: Once the container is up, be patient. Some extensions may complain because dependencies are still being installed, and in the Extensions tab, some may ask you to "Reload" the project. Don't do that yet. First, wait until all settle down for the first time. We suggest you create a new VSCode profile for this devcontainer, so any configuration and extensions you change, won't affect your default profile. Checklist: - [ ] The usual message asking you to start the Server and Frontend in different windows are now "hidden" in the building process of the devcontainer. Don't forget to do as suggested. - [ ] Open a JavaScript file, for example "server/index.js" and check if `eslint` is working. It will complain that `'err' is defined but never used.`. This means it is working. - [ ] Open a React File, for example, "frontend/src/main.jsx," and check if `eslint` complains about `Fast refresh only works when a file has exports. Move your component(s) to a separate file.`. Again, it means `eslint` is working. Now check at the status bar if the `Prettier` has a double checkmark :heavy_check_mark: (double). It means Prettier is working. You will see a nice extension `Formatting:`:heavy_check_mark: that can be used to disable the `Format on Save` feature temporarily. - [ ] Check if, on the left pane, you have the NPM Scripts (this may be disabled; look at the "Explorer" tree-dots up-right). There will be scripts inside the `package.json` files. You will basically need to run the `dev:collector`, `dev:server` and the `dev:frontend` in this order. When the frontend finishes starting, a window browser will open **inside** the VSCode. Still, you can open it outside. :warning: **Important for all developers** :warning: - [ ] When you are using the `NODE_ENV=development` the server will not store the configurations you set for security reasons. Please set the proper config on file `.env.development`. The side-effect if you don't, everytime you restart the server, you will be sent to the "Onboarding" page again. **Note when using GitHub Codespaces** - [ ] When running the "Server" for the first time, it will automatically configure its port to be publicly accessible by default, as this is required for the front end to reach the server backend. To know more, read the content of the `.env` file on the frontend folder about this, and if any issues occur, make sure to manually set the port "Visibility" of the "Server" is set to "Public" if needed. Again, this is only needed for developing on GitHub Codespaces. **For the Collector:** - [x] In the past, the Collector dwelled within the Python domain, but now it has journeyed to the splendid realm of Node.JS. Consequently, the configuration complexities of bygone versions are no longer a concern. ### Now it is ready to start In the status bar you will see three shortcuts names `Collector`, `Server` and `Frontend`. Just click-and-wait on that order (don't forget to set the Server port 3001 to Public if you are using GH Codespaces **_before_** starting the Frontend). Now you can enjoy your time developing instead of reconfiguring everything. ## Debugging with the devcontainers ### For debugging the collector, server and frontend First, make sure the built-in extension (ms-vscode.js-debug) is active (I don't know why it would not be, but just in case). If you want, you can install the nightly version (ms-vscode.js-debug-nightly) Then, in the "Run and Debug" tab (Ctrl+shift+D), you can select on the menu: - Collector debug. This will start the collector in debug mode and attach the debugger. Works very well. - Server debug. This will start the server in debug mode and attach the debugger. Works very well. - Frontend debug. This will start the frontend in debug mode and attach the debugger. I am still struggling with this one. I don't know if VSCode can handle the .jsx files seamlessly as the pure .js on the server. Maybe there is a need for a particular configuration for Vite or React. Anyway, it starts. Another two configurations launch Chrome and Edge, and I think we could add breakpoints on .jsx files somehow. The best scenario would be always to use the embedded browser. WIP. Please leave comments on the Issues tab or the []("https://discord.gg/6UyHPeGZAC") --- ### Overview (`cloud-deployments/openshift/README.md`) > [!IMPORTANT] > This is a community-maintained template and is not officially supported by the AnythingLLM team. You could encounter issues or even deployment failures in future versions of AnythingLLM. We do our best to keep this template and all community contributions backwards compatible, but we cannot guarantee it. # OpenShift Deployment Template for AnythingLLM This directory contains a specialized Dockerfile and entrypoint script for deploying AnythingLLM on **Red Hat OpenShift** clusters. ## Why This Template Exists OpenShift has a unique security model that differs from standard Docker/Kubernetes deployments: 1. **Arbitrary UIDs**: OpenShift runs containers with randomly assigned user IDs (UIDs) that don't exist in `/etc/passwd` 2. **GID 0 Requirement**: All containers run with GID 0 (root group) as the primary group 3. **Restricted SCCs**: The default Security Context Constraints (SCCs) prevent containers from running as specific users These requirements are incompatible with the standard AnythingLLM Docker image, which uses a fixed `anythingllm` user with UID/GID 1000. ## Key Differences from Standard Dockerfile | Feature | Standard Docker | OpenShift Template | |---------|-----------------|-------------------| | File ownership | `anythingllm:anythingllm` | `anythingllm:0` (root group) | | File permissions | Standard | Group-writable (`g+w`) | | `/etc/passwd` | Read-only | Group-writable for UID injection | | Supplementary groups | None | Added to group 0 | | Entrypoint | Standard | Handles arbitrary UID scenarios | ## When to Use This Template Use this template **only** if you are deploying to: - Red Hat OpenShift (any version) - OKD (OpenShift Origin) - Any Kubernetes cluster with OpenShift-style restricted SCCs **Do NOT use this for:** - Standard Docker deployments - Docker Compose - Generic Kubernetes (use the standard image with appropriate `securityContext`) - Cloud container services (AWS ECS, Google Cloud Run, Azure Container Instances) ## Building the Image From the repository root: ```bash docker build -f cloud-deployments/openshift/Dockerfile -t anythingllm:openshift . ``` For multi-architecture builds: ```bash docker buildx build \ --platform linux/amd64,linux/arm64 \ -f cloud-deployments/openshift/Dockerfile \ -t your-registry/anythingllm:openshift \ --push . ``` ## Deploying to OpenShift ### Using `oc` CLI ```bash # Create a new project (namespace) oc new-project anythingllm # Create a deployment oc new-app your-registry/anythingllm:openshift # Expose the service oc expose svc/anythingllm --port=3001 # Set required environment variables oc set env deployment/anythingllm \ STORAGE_DIR=/app/server/storage \ JWT_SECRET=$(openssl rand -hex 32) ``` ### Using a DeploymentConfig YAML ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: anythingllm spec: replicas: 1 selector: matchLabels: app: anythingllm template: metadata: labels: app: anythingllm spec: containers: - name: anythingllm image: your-registry/anythingllm:openshift ports: - containerPort: 3001 env: - name: STORAGE_DIR value: /app/server/storage - name: JWT_SECRET valueFrom: secretKeyRef: name: anythingllm-secrets key: jwt-secret volumeMounts: - name: storage mountPath: /app/server/storage volumes: - name: storage persistentVolumeClaim: claimName: anythingllm-storage ``` ## Persistent Storage OpenShift PersistentVolumeClaims work with this image. Ensure the PVC is created before deployment: ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: anythingllm-storage spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi ``` ## Troubleshooting ### Permission Denied Errors If you see permission errors, verify: 1. You're using this OpenShift-specific image, not the standard one 2. The PVC has correct access modes 3. No custom SCCs are overriding the default behavior ### User Not Found in passwd The entrypoint script automatically handles this by injecting a passwd entry at runtime. If issues persist, check that `/etc/passwd` is group-writable in your image. --- ### Overview (`extras/translator/README.md`) # AnythingLLM Auto-translater The AnythingLLM Auto-translator is a way for us to translate our locales but at no cost or overhead. However these are expected to be run manually and while they may not be 100% accurate improvement in models have made this work trivial without the need for an api. ## Getting started - Install Ollama [w/Docker Model Runner](https://ollama.com) - `ollama pull translategemma:4b` - `cd extras/translator && cp .env.example .env` ## Run the script All translations are based on english dictionary. So the English dictionary must have an entry for it to be supported. `cd extras/translator` **Target a specific language** `node index.mjs ` eg: `node index.mjs es` for Spanish. **Do all languages** `node index.mjs --all` _this is NOT recommended_ ## Gotchas - Sometimes translations go on for a while until the token window is exceeded - you can see this by the massive lines extending beyond the page. - Some languages operate different with single words or special symbols like "@" and will go off the rails. If the English text is one word and the translated text is 100 words, you dont need to be linguist to know that it is probably wrong. - You should always review every line for discrepencies or removal of `{{}}` inputs or brand name hallunications eg: `AnyLLM` instead of `AnythingLLM` --- ### Overview (`server/storage/README.md`) # AnythingLLM Storage This folder is for the local or disk storage of ready-to-embed documents, vector-cached embeddings, and the disk-storage of LanceDB and the local SQLite database. This folder should contain the following folders. `documents` `lancedb` (if using lancedb) `vector-cache` and a file named exactly `anythingllm.db` ### Common issues **SQLITE_FILE_CANNOT_BE_OPENED** in the server log = The DB file does not exist probably because the node instance does not have the correct permissions to write a file to the disk. To solve this.. - Local dev - Create a `anythingllm.db` empty file in this directory. Thats all. No need to reboot the server or anything. If your permissions are correct this should not ever occur since the server will create the file if it does not exist automatically. - Docker Instance - Get your AnythingLLM docker container id with `docker ps -a`. The container must be running to execute the next commands. - Run `docker container exec -u 0 -t mkdir -p /app/server/storage /app/server/storage/documents /app/server/storage/vector-cache /app/server/storage/lancedb` - Run `docker container exec -u 0 -t touch /app/server/storage/anythingllm.db` - Run `docker container exec -u 0 -t chown -R anythingllm:anythingllm /app/collector /app/server` - The above commands will create the appropriate folders inside of the docker container and will persist as long as you do not destroy the container and volume. This will also fix any ownership issues of folder files in the collector and the server. --- ### Overview (`server/storage/models/README.md`) # Native models used by AnythingLLM This folder is specifically created as a local cache and storage folder that is used for native models that can run on a CPU. Currently, AnythingLLM uses this folder for the following parts of the application. ## Embedding When your embedding engine preference is `native` we will use the ONNX **all-MiniLM-L6-v2** model built by [Xenova on HuggingFace.co](https://huggingface.co/Xenova/all-MiniLM-L6-v2). This model is a quantized and WASM version of the popular [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) which produces a 384-dimension vector. If you are using the `native` embedding engine your vector database should be configured to accept 384-dimension models if that parameter is directly editable (Pinecone only). ## Audio/Video transcription AnythingLLM allows you to upload various audio and video formats as source documents. In all cases the audio tracks will be transcribed by a locally running ONNX model **whisper-small** built by [Xenova on HuggingFace.co](https://huggingface.co/Xenova/whisper-small). The model is a smaller version of the OpenAI Whisper model. Given the model runs locally on CPU, larger files will result in longer transcription times. Once transcribed you can embed these transcriptions into your workspace like you would any other file! **Other external model/transcription providers are also live.** - [OpenAI Whisper via API key.](https://openai.com/research/whisper) ## Text generation (LLM selection) > [!IMPORTANT] > Use of a locally running LLM model is **experimental** and may behave unexpectedly, crash, or not function at all. > We suggest for production-use of a local LLM model to use a purpose-built inference server like [LocalAI](https://localai.io) or [LMStudio](https://lmstudio.ai). > [!TIP] > We recommend at _least_ using a 4-bit or 5-bit quantized model for your LLM. Lower quantization models tend to > just output unreadable garbage. If you would like to use a local Llama compatible LLM model for chatting you can select any model from this [HuggingFace search filter](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf&other=text-generation-inference&sort=trending) **Requirements** - Model must be in the latest `GGUF` format - Model should be compatible with latest `llama.cpp` - You should have the proper RAM to run such a model. Requirement depends on model size. ### Where do I put my GGUF model? > [!IMPORTANT] > If running in Docker you should be running the container to a mounted storage location on the host machine so you > can update the storage files directly without having to re-download or re-build your docker container. [See suggested Docker config](../../../docker/HOW_TO_USE_DOCKER.md#recommend-way-to-run-dockerized-anythingllm) > [!NOTE] > `/server/storage/models/downloaded` is the default location that your model files should be at. > Your storage directory may differ if you changed the STORAGE_DIR environment variable. All local models you want to have available for LLM selection should be placed in the `server/storage/models/downloaded` folder. Only `.gguf` files will be allowed to be selected from the UI. --- ### Overview (`cloud-deployments/helm/charts/anythingllm/README.md`) # anythingllm [AnythingLLM](https://github.com/Mintplex-Labs/anything-llm) The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more. **Configuration & Usage** - **Config vs Secrets:** This chart exposes application configuration via two mechanisms: - `config` (in `values.yaml`) — rendered into a `ConfigMap` and injected using `envFrom` in the pod. Do NOT place sensitive values (API keys, secrets) in `config` because `ConfigMap`s are not encrypted. - `env` / `envFrom` — the preferred way to inject secrets. Use Kubernetes `Secret` objects and reference them from `env` (with `valueFrom.secretKeyRef`) or `envFrom.secretRef`. - **Storage & STORAGE_DIR mapping:** The chart creates (or mounts) a `PersistentVolumeClaim` using the `persistentVolume.*` settings. The container mount path is set from `persistentVolume.mountPath`. Ensure the container `STORAGE_DIR` config key matches that path (defaults are set in `values.yaml`). **Providing API keys & secrets (recommended)** Use Kubernetes Secrets. Below are example workflows and `values.yaml` snippets. 1. Create a Kubernetes Secret with API keys: ``` kubectl create secret generic openai-secret --from-literal=OPENAI_KEY="sk-..." # or from a file # kubectl create secret generic openai-secret --from-file=OPENAI_KEY=/path/to/keyfile ``` 2. Reference the Secret from `values.yaml` using `envFrom` (recommended when your secret contains multiple env keys): ```yaml envFrom: - secretRef: name: openai-secret ``` This will inject all key/value pairs from the `openai-secret` Secret as environment variables in the container. 3. Or reference a single secret key via `env` (explicit mapping): ```yaml env: - name: OPENAI_KEY valueFrom: secretKeyRef: name: openai-secret key: OPENAI_KEY ``` Notes: - Avoid placing secret values into `config:` (the chart's `ConfigMap`) — `ConfigMap`s are visible to anyone who can read the namespace. Use `Secret` objects for any credentials/tokens. - If you use a GitOps workflow, consider integrating an external secret operator (ExternalSecrets, SealedSecrets, etc.) so you don't store raw secrets in Git. **Example `values-secret.yaml` to pass during `helm install`** ```yaml image: repository: mintplexlabs/anythingllm tag: "1.18.0" service: type: ClusterIP port: 3001 # Reference secret containing API keys envFrom: - secretRef: name: openai-secret # Optionally override other values persistentVolume: size: 16Gi mountPath: /storage ``` Install with: ``` helm install my-anythingllm ./anythingllm -f values-secret.yaml ``` **Best practices & tips** - Use `envFrom` for convenience when many environment variables are stored in a single `Secret` and use `env`/`valueFrom` for explicit single-key mappings. - Use `kubectl create secret generic` or your secrets management solution. If you need to reference multiple different provider keys (OpenAI, Anthropic, etc.), create a single `Secret` with multiple keys or multiple Secrets and add multiple `envFrom` entries. - Keep probe paths and `service.port` aligned. If your probes fail after deployment, check that the probe `port` matches the container port (or named port `http`) and that the `path` is valid. - For storage, if you have a pre-existing PVC set `persistentVolume.existingClaim` to the PVC name; the chart will mount that claim (and will not attempt to create a new PVC). - For production, provide resource `requests` and `limits` in `values.yaml` to prevent scheduler starvation and to control cost. ## Values | Key | Type | Default | Description | | ---------------------------------- | ------ | ---------------------------- | ----------- | | affinity | object | `{}` | | | config.DISABLE_TELEMETRY | string | `"true"` | | | config.GID | string | `"1000"` | | | config.NODE_ENV | string | `"production"` | | | config.STORAGE_DIR | string | `"/storage"` | | | config.UID | string | `"1000"` | | | env | object | `{}` | | | envFrom | object | `{}` | | | fullnameOverride | string | `""` | | | image.pullPolicy | string | `"IfNotPresent"` | | | image.repository | string | `"mintplexlabs/anythingllm"` | | | image.tag | string | `"1.18.0"` | | | imagePullSecrets | list | `[]` | | | ingress.annotations | object | `{}` | | | ingress.className | string | `""` | | | ingress.enabled | bool | `false` | | | ingress.hosts[0].host | string | `"chart-example.local"` | | | ingress.hosts[0].paths[0].path | string | `"/"` | | | ingress.hosts[0].paths[0].pathType | string | `"ImplementationSpecific"` | | | ingress.tls | list | `[]` | | | initContainers | list | `[]` | | | livenessProbe.failureThreshold | int | `3` | | | livenessProbe.httpGet.path | string | `"/v1/api/health"` | | | livenessProbe.httpGet.port | int | `8888` | | | livenessProbe.initialDelaySeconds | int | `15` | | | livenessProbe.periodSeconds | int | `5` | | | nameOverride | string | `""` | | | nodeSelector | object | `{}` | | | persistentVolume.accessModes[0] | string | `"ReadWriteOnce"` | | | persistentVolume.annotations | object | `{}` | | | persistentVolume.existingClaim | string | `""` | | | persistentVolume.labels | object | `{}` | | | persistentVolume.mountPath | string | `"/storage"` | | | persistentVolume.size | string | `"8Gi"` | | | podAnnotations | object | `{}` | | | podLabels | object | `{}` | | | podSecurityContext.fsGroup | int | `1000` | | | readinessProbe.httpGet.path | string | `"/v1/api/health"` | | | readinessProbe.httpGet.port | int | `8888` | | | readinessProbe.initialDelaySeconds | int | `15` | | | readinessProbe.periodSeconds | int | `5` | | | readinessProbe.successThreshold | int | `2` | | | replicaCount | int | `1` | | | resources | object | `{}` | | | securityContext | object | `{}` | | | service.port | int | `3001` | | | service.type | string | `"ClusterIP"` | | | serviceAccount.annotations | object | `{}` | | | serviceAccount.automount | bool | `true` | | | serviceAccount.create | bool | `true` | | | serviceAccount.name | string | `""` | | | tolerations | list | `[]` | | | volumeMounts | list | `[]` | | | volumes | list | `[]` | | --- --- METRICS --- - Files Extracted: 7 - Estimated Token Budget: ~13169 tokens - Canonical Reference: https://codewiki.google/github.com/Mintplex-Labs/anything-llm