{"owner":"TabbyML","repo":"tabby","hasSkills":true,"totalSkillsCount":28,"totalTokensCount":10452,"categories":["anthropic-skill"],"hasMcp":false,"mcpConfig":null,"found":["website/docs/references/programming-languages.md","website/docs/references/models-http-api/anthropic-claude.md","website/docs/references/_category_.yaml","website/docs/references/models-http-api/amazon-bedrock.md","website/docs/extensions/configurations.md","website/docs/extensions/troubleshooting/index.md","website/docs/references/models-http-api/deepinfra.md","website/docs/references/models-http-api/deepseek.md","website/docs/references/models-http-api/azure-openai.md","website/docs/references/models-http-api/huggingface.md","website/docs/references/models-http-api/fireworks.md","website/docs/references/models-http-api/jan.ai.md","website/docs/references/models-http-api/llamafile.md","website/docs/references/models-http-api/lm-studio.md","website/docs/references/models-http-api/mistral-ai.md","website/docs/references/models-http-api/ollama.md","website/docs/references/models-http-api/openai.md","website/docs/references/models-http-api/openrouter.md","website/docs/references/models-http-api/perplexity.md","website/docs/references/models-http-api/vllm.md","website/docs/references/models-http-api/voyage-ai.md","website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml","website/docs/references/cloud-deployment/bentoml/bentofile.yaml","website/docs/references/cloud-deployment/bentoml/index.md","website/docs/references/cloud-deployment/hugging-face/index.md","website/docs/references/cloud-deployment/modal/index.md","website/docs/references/cloud-deployment/skypilot/index.md","website/docs/references/cloud-deployment/skypilot/tabby.yaml"],"skills":{"website/docs/references/programming-languages.md":"---\nsidebar_position: 6\n---\n\n# Programming Languages\n\nMost models nowadays support a large number of programming languages (thanks to [The Stack](https://huggingface.co/datasets/bigcode/the-stack), which has collected 358 programming languages).\nIn Tabby, we need to add configuration for each language to maximize performance and completion quality.\n\nCurrently, there are two aspects of support that need to be added for each language.\n\n**Stop Words**\n\nStop words determine when the language model can early stop its decoding steps, resulting in better latency and affecting the quality of completion. We suggest adding all top-level keywords as part of the stop words.\n\n**Repository Context**\n\nWe parse languages into chunks and compute a token-based index for serving time Retrieval Augmented Code Completion. In Tabby, we define these repository contexts as [treesitter queries](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax), and the query results will be indexed.\n\nFor an actual example of an issue or pull request adding the above support, please check out https://github.com/TabbyML/tabby/issues/553 as a reference.\n\n## Supported Languages\n\n* [Rust](https://www.rust-lang.org/)\n* [Python](https://www.python.org/)\n* [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)\n* [TypeScript](https://www.typescriptlang.org/)\n* [Golang](https://go.dev/)\n* [Ruby](https://www.ruby-lang.org/)\n* [Java](https://www.java.com/)\n* [Kotlin](https://www.kotlinlang.org/)\n* [C/C++](https://cplusplus.com/)\n* [PHP](https://www.php.net/)\n* [C#](https://learn.microsoft.com/en-us/dotnet/csharp/)\n* [Solidity](https://soliditylang.org/)\n* [R](https://www.r-project.org/)\n* [Dart](https://dart.dev/)\n* [Lua](https://www.lua.org)\n* [Elixir](https://elixir-lang.org)\n* [OCaml](https://ocaml.org/)\n* [GDScript](https://gdscript.com/)\n* [Scala](https://www.scala-lang.org/)\n\n## Languages Missing Certain Support\n\n| Language | Stop Words (time to contribute: ~5 min) | Repository Context (time to contribute: ~1 hr) |\n| :------: | :-------------------------------------: | :--------------------------------------------: |\n| CSS | 🚫 | 🚫 |\n| Haskell | 🚫 | 🚫 |\n| Julia | 🚫 | 🚫 |\n| Perl | 🚫 | 🚫 |\n","website/docs/references/models-http-api/anthropic-claude.md":"# Anthropic Claude\n\n[Anthropic](https://www.anthropic.com/) is an AI research company that develops large language models, including the Claude family of models. While Tabby doesn't natively support Claude's API, you can access Claude models through an OpenAI-compatible API interface using [claude2openai](https://github.com/missuo/claude2openai) as a middleware.\n\n## Chat model\n\nAfter deploying the claude2openai middleware, you can access all Claude family models through an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"claude-3-sonnet-20240229\"\n# Middleware endpoint (adjust host and port according to your deployment)\napi_endpoint = \"http://127.0.0.1:6600/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAnthropic currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nAnthropic currently does not provide embedding model APIs.\n","website/docs/references/_category_.yaml":"label: 📚 References\nposition: 100","website/docs/references/models-http-api/amazon-bedrock.md":"# Amazon Bedrock\n\nAmazon Bedrock is a fully managed service on AWS that provides access to foundation models from various AI companies through a single API. With [Amazon Bedrock Access Gateway](https://github.com/aws-samples/bedrock-access-gateway), you can access Anthropic's Claude models through an OpenAI-compatible interface, enabling seamless integration with tools and applications designed for OpenAI's API structure.\n\nFollow the Amazon Bedrock Access Gateway setup guide to deploy your own OpenAI-compatible API endpoint for Claude models.\n\n## Chat model\n\nAmazon Bedrock Access Gateway provides an OpenAI-compatible chat API interface for Claude models. Here we use the `us.anthropic.claude-3-5-sonnet-20241022-v2:0` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"us.anthropic.claude-3-5-sonnet-20241022-v2:0\"\napi_endpoint = \"http://Bedrock-Proxy-xxxxx.{Region}.elb.amazonaws.com/api/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAmazon Bedrock does not provide completion models.\n\n## Embeddings model\n\nWhile Amazon Bedrock supports embeddings models, Tabby does not currently support the embeddings API interface for Amazon models.\n","website/docs/extensions/configurations.md":"---\nsidebar_position: 98\n---\n\n# Configuration\n\nThis document describes the available configurations for [Tabby Agent](https://github.com/TabbyML/tabby/tree/main/clients/tabby-agent).\n\n## Config File\n\nThe Tabby agent, which is the core component of Tabby IDE extensions, reads configurations from the `~/.tabby-client/agent/config.toml` file. This file is automatically created when you first run the Tabby IDE extensions. You can edit this file to modify the configurations. The Tabby IDE extensions will automatically reload the config file when it detects changes.\n\n:::tip\n- The config file is written in [TOML](https://toml.io/en/). When you edit the config template and want to enable a configuration, make sure to uncomment the full section including the leading line, not just the line with the configuration value.\n- Configurations set via the IDE settings page take precedence over the config file. If you want to use a configuration from the config file, make sure that the IDE setting is empty.\n- If you are using the Tabby VSCode extension in a web browser, this config file is not available. You can use the VSCode settings page to configure the extension.\n:::\n\n## Server\n\nThe `server` section contains configurations related to the Tabby server.\n\n**NOTE**: If your Tabby server requires an authentication token, remember to set it here.\n\n```toml\n# Server\n# You can set the server endpoint here and an optional authentication token if required.\n[server]\nendpoint = \"http://localhost:8080\" # http or https URL\ntoken = \"your-token-here\" # if token is set, request header Authorization = \"Bearer $token\" will be added automatically\n\n# You can add custom request headers.\n[server.requestHeaders]\nHeader1 = \"Value1\" # list your custom headers here\nHeader2 = \"Value2\" # values can be strings, numbers or booleans\n```\n\n## Logs\n\nIf you encounter any issues with the Tabby IDE extensions and need to report a bug, you can enable debug logs to help us investigate the issue.\n\n```toml\n# Logs\n# You can set the log level here. The log file is located at ~/.tabby-client/agent/logs/.\n[logs]\nlevel = \"silent\" # \"silent\" or \"error\" or \"debug\"\n```\n\n## Usage Collection\n\nTabby IDE extensions collect aggregated anonymous usage data and sends it to the Tabby team to help improve our products.\n\n**Do not worry, your code, generated completions, or any identifying information is never tracked or transmitted.** \n\nThe data we collect, as of the latest update on November 6, 2023, contains following major parts:\n\n- System info and extension version info\n- Completions statistics\n - Completion count\n - Completion accepted count\n - Completion HTTP request latency\n\nWe sincerely appreciate your contribution in sending anonymous usage data. However, if you prefer not to participate, you can disable anonymous usage tracking here:\n\n```toml\n# Anonymous usage tracking\n[anonymousUsageTracking]\ndisable = false # set to true to disable\n```\n","website/docs/extensions/troubleshooting/index.md":"---\nsidebar_position: 99\n---\n\n# Troubleshooting\n\nThis document aims to assist you in troubleshooting issues with the Tabby extensions \nfor various IDEs such as VSCode, IntelliJ Platform IDEs, and Vim / NeoVim.\n\n## Tabby Initialization Failed?\n\nThis problem may occur when you first install the Tabby in IntelliJ Platform IDEs \nor Vim/NeoVim. \n\nThe Tabby IDE extension runs its core logic in the Tabby agent. In the case of VSCode, \nthe agent runs within the VSCode Extension Host, while for IntelliJ Platform IDEs \nand Vim/NeoVim, the agent runs as a separate Node.js process.\n\n### Install Node.js\n\nYou can follow the instructions on the [Node.js website](https://nodejs.org/en/download/)\nto install Node.js. Alternatively, you can use a version manager such as [nvm](https://github.com/nvm-sh/nvm). \n**Note**: Tabby IDE extension requires Node.js version 18.0.0 or higher.\n\n### Specify Node Binary Path\n\nIf the node binary is already accessible via your `PATH` environment variable, \nyou can skip this step. Otherwise, you will need to specify the path to the node \nbinary in the IDE settings.\n\nFor IntelliJ Platform IDEs (Tabby plugin version 0.6.0 or higher):\n* Click on Tabby plugin status bar item and select `Open Settings...`.\n* Enter the path to the node binary on your system in the `Node binary` field,\n e.g. `/usr/local/bin/node`, `C:\\Program Files\\nodejs\\node.exe`. \n If you are using a version manager such as `nvm`, you can enter the path to the\n node binary installed by the version manager, e.g. `~/.nvm/versions/node/v18.18.0/bin/node`.\n* Restart the IDE\n\nIf you installed Node.js via snap, please use `/snap/node/current/bin/node` rather than \n`/snap/bin/node` as the node binary path.\n\n## Cannot Connect to Tabby Server?\n\nIf you have setup the endpoint for the Tabby server but the status bar item of \nthe Tabby IDE extension still displays a mark indicating \"Disconnected\", \nfollow the steps below to troubleshoot the issue.\n\n### Check Endpoint Settings\n\nVerify that the endpoint setting is correct. You can set the endpoint in the \nIDE settings page (except for Vim/NeoVim, which do not have this option) or by \nediting the [config file](https://tabby.tabbyml.com/docs/extensions/configurations). \nKeep in mind that the IDE settings take priority over the config file. \nIf you wish to use the setting from the config file, ensure that the IDE setting\nis empty.\n\n### Authentication Token\n\nIf you have enabled authentication on your Tabby server, you will need to set\nthe authentication token in the IDE or the [config file](https://tabby.tabbyml.com/docs/extensions/configurations).\n\n### Verify Tabby Server Status\n\nOnce the Tabby server is running, it should display a log message such as \n`Listening at 0.0.0.0:8080`. \nOpen your browser and navigate to `http://localhost:8080/swagger-ui/`\n(Replace `localhost:8080` with the correct IP/domain and port if you have setup your \nTabby server on a remote machine). The browser should display a web page with Swagger UI. \n\nTo test the server, expand the `/v1/completions` section, click on `Try it out`, \nand then click `Execute`. \nIf you receive a response, it indicates that the Tabby server is running properly.\n\nYou can also use `curl` to send a completion request to Tabby server. For example:\n\n```shell\ncurl -X 'POST' \\\n 'http://localhost:8080/v1/completions' \\\n -H 'accept: application/json' \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"language\": \"python\",\n \"segments\": {\n \"prefix\": \"def fib(n):\\n \",\n \"suffix\": \"\\n return fib(n - 1) + fib(n - 2)\"\n }\n}'\n```\n\nIf you can not see Swagger UI page, or can not get response of completion request, \nplease check the server log to see if there is any error.\n\n### Proxy Settings\n\nPlease note that Tabby extensions for IDEs currently do not support proxy settings. \nIf you need to access your Tabby server through a proxy, consider setting up \na reverse proxy and using the reverse proxy URL as the endpoint for the Tabby IDE extension.\n\n## Cannot Get Any Completions?\n\nIf you are able to connect the Tabby extension to the Tabby server but are unable to \nreceive any completions, you can follow the steps below to troubleshoot the issue.\n\n### Check Trigger Mode Settings\n\nTabby is set to automatic trigger mode by default. In this mode, you should receive \ncompletions after a short delay when you stop typing. The delay may vary depending \non your server's performance and settings. \nIf you are using manual trigger mode, you need to press `Alt + \\` (`Ctrl + \\` for IntelliJ and Vim plugins) \nto trigger a completion request. The status bar item of Tabby IDE extension should show a loading \nindicator for a brief period before displaying the completions. \nKeep in mind that Tabby may not provide any suggestions if there is no necessary\nfor the current code context.\n\n### Check Request Timeouts\n\nIf your completion requests are timing out, Tabby may display a warning message. \nThis could be due to network issues or poor server performance, especially when \nrunning a large model on a CPU. To improve performance, consider running the model \non a GPU with CUDA or ROCm support or on Apple M1/M2 with Metal support. When running \nthe server, make sure to specify the device in the arguments using `--device cuda`, `--device rocm` or\n`--device metal`. You can also try using a smaller model from the available [models](https://tabby.tabbyml.com/docs/models/). \n\n## Problems with the Chat Panel?\n\n### Check Server Enabling Chat Feature\n\nThe chat panel is a feature that requires the Tabby server to enable the chat feature. Please make sure that the chat feature is properly configured in the server configuration file or that the server is started with the `--chat-model MODEL_NAME` flag. \nYou can access the server management page at `https://demo.tabbyml.com/system` to check if the chat feature is enabled.\n\n### Check Browser Compatibility in IntelliJ Platform IDEs\n\nThe chat panel in IntelliJ Platform IDEs is implemented using an embedded JCEF browser. The support is included in the default Java Runtime for JetBrains IDEs, but it may not be by default in Android Studio. \nPlease follow the steps below to ensure that the JCEF browser is enabled in your IDE:\n\n1. Open `Search Everywhere` (Double `Shift`) or `Find Action` (`Ctrl + Shift + A`), and search for the action `Choose Boot Java Runtime for the IDE...`\n2. Select a Java Runtime that includes JCEF support \n ![Choose Boot Java Runtime for the IDE](./intellij-choose-boot-runtime-for-the-ide.png)\n3. Restart the IDE\n\n## Want to Deep Dive via Logs?\n\nIf you cannot solve the issue using the previous steps, you may want to \ninvestigate further by checking the logs of Tabby extensions. \nIf you want help from the community, it is also recommended to share the logs \nof the Tabby extensions.\n\n### VSCode\n\nFor VSCode, you can check the `Output` window and select the `Tabby` channel or\n`Tabby Agent` channel.\n![VSCode Output View](./vscode-output-view.png)\nThe default log level is `info`. To enable debug logs, you can use the command \n`Developer: Set Log Level...` to set the log level to `debug`.\n\n### IntelliJ Platform IDEs\n\nFor IntelliJ Platform IDEs, you can check the logs for the IDE using `Help -> Show Log in Explorer|Finder|Files...`, \nor follow [this document](https://intellij-support.jetbrains.com/hc/en-us/articles/207241085-Locating-IDE-log-files) to locate the log file.\nThis log file contains all the logs for the IDE, and you can filter them by searching for the \nkeyword `com.tabbyml.intellijtabby`.\nYou can also enable all level logs by editing `Help -> Diagnostic Tools -> Debug Log Settings...` and\nadd `com.tabbyml.intellijtabby:all` to the list.\n\n### Tabby Agent Logs\n\nAs the agent runs as a separate Node.js process for IntelliJ Platform IDEs plugin \nand Vim/NeoVim plugin, its logs are written separately.\n\nBy default, the agent logs are set to `\"silent\"`, which means the agent logs are \nnot written to disk. To enable Tabby agent debug logs, editing the \n[config file](https://tabby.tabbyml.com/docs/extensions/configurations), \nuncomment the `logs` section and set `level` to `\"debug\"`, then save the file \nto apply the changes.\n \nYou can find the agent logs in the `~/.tabby-client/agent/logs` directory. These \nlogs are written using [pino](https://github.com/pinojs/pino), and you can \nuse `pino-pretty` to format the log file for easier readability.\n\n```shell\ntail -f ~/.tabby-client/agent/logs/20240101.0.log | npx pino-pretty\n```\n\n## Still Have Issues?\n\nIf you still have any issues, please feel free to [open an issue on github](https://github.com/TabbyML/tabby/issues/new), \nor join our [slack community](https://links.tabbyml.com/join-slack)\nfor further support.\n\n","website/docs/references/models-http-api/deepinfra.md":"# DeepInfra\n\n[DeepInfra](https://deepinfra.com/) is a cloud platform providing efficient and scalable model inference services, offering access to various open-source models like [Llama 3](https://deepinfra.com/meta-llama/Llama-3.3-70B-Instruct), [Mixtral](https://deepinfra.com/mistralai/Mixtral-8x7B-Instruct-v0.1), and [Qwen](https://deepinfra.com/Qwen/Qwen2.5-Coder-32B-Instruct).\n\n## Chat model\n\nDeepInfra provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"meta-llama/Llama-3.3-70B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepInfra provides an OpenAI-compatible completion API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"openai/completion\"\nmodel_name = \"Qwen/Qwen2.5-Coder-32B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepInfra also provides an OpenAI-compatible embeddings API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"openai/embedding\"\nmodel_name = \"BAAI/bge-base-en-v1.5\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n","website/docs/references/models-http-api/deepseek.md":"# DeepSeek\n\n[DeepSeek](https://www.deepseek.com/) is an AI company that develops large language models specialized in coding and general tasks. Their models include [DeepSeek V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) for general tasks and [DeepSeek Coder](https://huggingface.co/collections/deepseek-ai/deepseekcoder-v2-666bf4b274a5f556827ceeca) specifically optimized for programming tasks.\n\n## Chat model\n\nDeepSeek provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepSeek offers a specialized completion API interface for code completion tasks.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"deepseek/completion\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/beta\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepSeek currently does not provide embedding model APIs.\n","website/docs/references/models-http-api/azure-openai.md":"# Azure OpenAI\n\n[Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) is a cloud-based service that provides Azure customers with access to OpenAI's powerful language models including GPT-4, GPT-3.5, and various embedding models.\n\nPlease be aware that azure will be supported starting with version 0.24, which is scheduled for release by end of 01/2025\n\n## Chat model\n\nIt supports various GPT series chat models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"azure/chat\"\nmodel_name = \"gpt-4o-mini\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAzure OpenAI currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nIt supports text-embedding-3-small, text-embedding-3-large and other embedding models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"azure/embedding\"\nmodel_name = \"text-embedding-3-large\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n","website/docs/references/models-http-api/huggingface.md":"# Hugging Face Inference Providers\n\n[Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers) offers access to frontier open models from multiple providers through a unified API. \n\nYou'll need a [Hugging Face account](https://huggingface.co/join) and an [access token](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained).\n\n## Chat model\n\nHugging Face Inference Providers provides an OpenAI-compatible chat API interface. Here we use the `MiniMaxAI/MiniMax-M2` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"MiniMaxAI/MiniMax-M2\" # specify the model you want to use\napi_endpoint = \"https://router.huggingface.co/v1\"\napi_key = \"your-hf-token\"\n```\n\n### Available models\n\nYou can find a complete list of models supported by at least one provider [on the Hub](https://huggingface.co/models?inference_provider=all). You can also access these programmatically, see this [guide](https://huggingface.co/docs/inference-providers/hub-api) for more details.\n\n## Completion model\n\nHugging Face Inference Providers does not offer completion models (FIM) through their OpenAI-compatible API. For code completion, use a local model with Tabby.\n\n## Embeddings model\n\nWhile Hugging Face Inference Providers supports embeddings models, Tabby does not currently support the embeddings API interface for Hugging Face Inference Providers.\n","website/docs/references/models-http-api/fireworks.md":"# Fireworks\n\n[Fireworks](https://app.fireworks.ai/) is a cloud platform that offers efficient model inference and deployment services,\nproviding cost-effective access to a variety of AI models through their API service,\nincluding [Llama 2](https://fireworks.ai/models/fireworks/llama-v2-70b-chat),\n[DeepSeek V3](https://fireworks.ai/models/fireworks/deepseek-v3),\n[DeepSeek Coder](https://fireworks.ai/models/fireworks/deepseek-coder-v2-instruct) and other open-source models.\n\n## Chat model\n\nFireworks provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"accounts/fireworks/models/deepseek-v3\"\napi_endpoint = \"https://api.fireworks.ai/inference/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nFireworks does not offer completion models (FIM) through their API.\n\n## Embeddings model\n\nWhile Fireworks provides embedding model APIs, Tabby has not yet implemented a compatible client to interface with these APIs. Therefore, embedding functionality is currently not available through Tabby's integration with Fireworks.\n","website/docs/references/models-http-api/jan.ai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/jan.ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/jan.ai.md)","website/docs/references/models-http-api/llamafile.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/llamafile.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/llamafile.md)","website/docs/references/models-http-api/lm-studio.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/lm-studio.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/lm-studio.md)","website/docs/references/models-http-api/mistral-ai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/mistral-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/mistral-ai.md)","website/docs/references/models-http-api/ollama.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/ollama.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/ollama.md)","website/docs/references/models-http-api/openai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openai.md)","website/docs/references/models-http-api/openrouter.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openrouter.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openrouter.md)","website/docs/references/models-http-api/perplexity.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/perplexity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/perplexity.md)","website/docs/references/models-http-api/vllm.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/vllm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/vllm.md)","website/docs/references/models-http-api/voyage-ai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/voyage-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/voyage-ai.md)","website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml)","website/docs/references/cloud-deployment/bentoml/bentofile.yaml":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentofile.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentofile.yaml)","website/docs/references/cloud-deployment/bentoml/index.md":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/index.md)","website/docs/references/cloud-deployment/hugging-face/index.md":"# Hugging-face Skill\n\nPath: `website/docs/references/cloud-deployment/hugging-face/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/hugging-face/index.md)","website/docs/references/cloud-deployment/modal/index.md":"# Modal Skill\n\nPath: `website/docs/references/cloud-deployment/modal/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/modal/index.md)","website/docs/references/cloud-deployment/skypilot/index.md":"# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/index.md)","website/docs/references/cloud-deployment/skypilot/tabby.yaml":"# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/tabby.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/tabby.yaml)"},"files":{"website/docs/references/programming-languages.md":"---\nsidebar_position: 6\n---\n\n# Programming Languages\n\nMost models nowadays support a large number of programming languages (thanks to [The Stack](https://huggingface.co/datasets/bigcode/the-stack), which has collected 358 programming languages).\nIn Tabby, we need to add configuration for each language to maximize performance and completion quality.\n\nCurrently, there are two aspects of support that need to be added for each language.\n\n**Stop Words**\n\nStop words determine when the language model can early stop its decoding steps, resulting in better latency and affecting the quality of completion. We suggest adding all top-level keywords as part of the stop words.\n\n**Repository Context**\n\nWe parse languages into chunks and compute a token-based index for serving time Retrieval Augmented Code Completion. In Tabby, we define these repository contexts as [treesitter queries](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax), and the query results will be indexed.\n\nFor an actual example of an issue or pull request adding the above support, please check out https://github.com/TabbyML/tabby/issues/553 as a reference.\n\n## Supported Languages\n\n* [Rust](https://www.rust-lang.org/)\n* [Python](https://www.python.org/)\n* [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)\n* [TypeScript](https://www.typescriptlang.org/)\n* [Golang](https://go.dev/)\n* [Ruby](https://www.ruby-lang.org/)\n* [Java](https://www.java.com/)\n* [Kotlin](https://www.kotlinlang.org/)\n* [C/C++](https://cplusplus.com/)\n* [PHP](https://www.php.net/)\n* [C#](https://learn.microsoft.com/en-us/dotnet/csharp/)\n* [Solidity](https://soliditylang.org/)\n* [R](https://www.r-project.org/)\n* [Dart](https://dart.dev/)\n* [Lua](https://www.lua.org)\n* [Elixir](https://elixir-lang.org)\n* [OCaml](https://ocaml.org/)\n* [GDScript](https://gdscript.com/)\n* [Scala](https://www.scala-lang.org/)\n\n## Languages Missing Certain Support\n\n| Language | Stop Words (time to contribute: ~5 min) | Repository Context (time to contribute: ~1 hr) |\n| :------: | :-------------------------------------: | :--------------------------------------------: |\n| CSS | 🚫 | 🚫 |\n| Haskell | 🚫 | 🚫 |\n| Julia | 🚫 | 🚫 |\n| Perl | 🚫 | 🚫 |\n","website/docs/references/models-http-api/anthropic-claude.md":"# Anthropic Claude\n\n[Anthropic](https://www.anthropic.com/) is an AI research company that develops large language models, including the Claude family of models. While Tabby doesn't natively support Claude's API, you can access Claude models through an OpenAI-compatible API interface using [claude2openai](https://github.com/missuo/claude2openai) as a middleware.\n\n## Chat model\n\nAfter deploying the claude2openai middleware, you can access all Claude family models through an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"claude-3-sonnet-20240229\"\n# Middleware endpoint (adjust host and port according to your deployment)\napi_endpoint = \"http://127.0.0.1:6600/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAnthropic currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nAnthropic currently does not provide embedding model APIs.\n","website/docs/references/_category_.yaml":"label: 📚 References\nposition: 100","website/docs/references/models-http-api/amazon-bedrock.md":"# Amazon Bedrock\n\nAmazon Bedrock is a fully managed service on AWS that provides access to foundation models from various AI companies through a single API. With [Amazon Bedrock Access Gateway](https://github.com/aws-samples/bedrock-access-gateway), you can access Anthropic's Claude models through an OpenAI-compatible interface, enabling seamless integration with tools and applications designed for OpenAI's API structure.\n\nFollow the Amazon Bedrock Access Gateway setup guide to deploy your own OpenAI-compatible API endpoint for Claude models.\n\n## Chat model\n\nAmazon Bedrock Access Gateway provides an OpenAI-compatible chat API interface for Claude models. Here we use the `us.anthropic.claude-3-5-sonnet-20241022-v2:0` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"us.anthropic.claude-3-5-sonnet-20241022-v2:0\"\napi_endpoint = \"http://Bedrock-Proxy-xxxxx.{Region}.elb.amazonaws.com/api/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAmazon Bedrock does not provide completion models.\n\n## Embeddings model\n\nWhile Amazon Bedrock supports embeddings models, Tabby does not currently support the embeddings API interface for Amazon models.\n","website/docs/extensions/configurations.md":"---\nsidebar_position: 98\n---\n\n# Configuration\n\nThis document describes the available configurations for [Tabby Agent](https://github.com/TabbyML/tabby/tree/main/clients/tabby-agent).\n\n## Config File\n\nThe Tabby agent, which is the core component of Tabby IDE extensions, reads configurations from the `~/.tabby-client/agent/config.toml` file. This file is automatically created when you first run the Tabby IDE extensions. You can edit this file to modify the configurations. The Tabby IDE extensions will automatically reload the config file when it detects changes.\n\n:::tip\n- The config file is written in [TOML](https://toml.io/en/). When you edit the config template and want to enable a configuration, make sure to uncomment the full section including the leading line, not just the line with the configuration value.\n- Configurations set via the IDE settings page take precedence over the config file. If you want to use a configuration from the config file, make sure that the IDE setting is empty.\n- If you are using the Tabby VSCode extension in a web browser, this config file is not available. You can use the VSCode settings page to configure the extension.\n:::\n\n## Server\n\nThe `server` section contains configurations related to the Tabby server.\n\n**NOTE**: If your Tabby server requires an authentication token, remember to set it here.\n\n```toml\n# Server\n# You can set the server endpoint here and an optional authentication token if required.\n[server]\nendpoint = \"http://localhost:8080\" # http or https URL\ntoken = \"your-token-here\" # if token is set, request header Authorization = \"Bearer $token\" will be added automatically\n\n# You can add custom request headers.\n[server.requestHeaders]\nHeader1 = \"Value1\" # list your custom headers here\nHeader2 = \"Value2\" # values can be strings, numbers or booleans\n```\n\n## Logs\n\nIf you encounter any issues with the Tabby IDE extensions and need to report a bug, you can enable debug logs to help us investigate the issue.\n\n```toml\n# Logs\n# You can set the log level here. The log file is located at ~/.tabby-client/agent/logs/.\n[logs]\nlevel = \"silent\" # \"silent\" or \"error\" or \"debug\"\n```\n\n## Usage Collection\n\nTabby IDE extensions collect aggregated anonymous usage data and sends it to the Tabby team to help improve our products.\n\n**Do not worry, your code, generated completions, or any identifying information is never tracked or transmitted.** \n\nThe data we collect, as of the latest update on November 6, 2023, contains following major parts:\n\n- System info and extension version info\n- Completions statistics\n - Completion count\n - Completion accepted count\n - Completion HTTP request latency\n\nWe sincerely appreciate your contribution in sending anonymous usage data. However, if you prefer not to participate, you can disable anonymous usage tracking here:\n\n```toml\n# Anonymous usage tracking\n[anonymousUsageTracking]\ndisable = false # set to true to disable\n```\n","website/docs/extensions/troubleshooting/index.md":"---\nsidebar_position: 99\n---\n\n# Troubleshooting\n\nThis document aims to assist you in troubleshooting issues with the Tabby extensions \nfor various IDEs such as VSCode, IntelliJ Platform IDEs, and Vim / NeoVim.\n\n## Tabby Initialization Failed?\n\nThis problem may occur when you first install the Tabby in IntelliJ Platform IDEs \nor Vim/NeoVim. \n\nThe Tabby IDE extension runs its core logic in the Tabby agent. In the case of VSCode, \nthe agent runs within the VSCode Extension Host, while for IntelliJ Platform IDEs \nand Vim/NeoVim, the agent runs as a separate Node.js process.\n\n### Install Node.js\n\nYou can follow the instructions on the [Node.js website](https://nodejs.org/en/download/)\nto install Node.js. Alternatively, you can use a version manager such as [nvm](https://github.com/nvm-sh/nvm). \n**Note**: Tabby IDE extension requires Node.js version 18.0.0 or higher.\n\n### Specify Node Binary Path\n\nIf the node binary is already accessible via your `PATH` environment variable, \nyou can skip this step. Otherwise, you will need to specify the path to the node \nbinary in the IDE settings.\n\nFor IntelliJ Platform IDEs (Tabby plugin version 0.6.0 or higher):\n* Click on Tabby plugin status bar item and select `Open Settings...`.\n* Enter the path to the node binary on your system in the `Node binary` field,\n e.g. `/usr/local/bin/node`, `C:\\Program Files\\nodejs\\node.exe`. \n If you are using a version manager such as `nvm`, you can enter the path to the\n node binary installed by the version manager, e.g. `~/.nvm/versions/node/v18.18.0/bin/node`.\n* Restart the IDE\n\nIf you installed Node.js via snap, please use `/snap/node/current/bin/node` rather than \n`/snap/bin/node` as the node binary path.\n\n## Cannot Connect to Tabby Server?\n\nIf you have setup the endpoint for the Tabby server but the status bar item of \nthe Tabby IDE extension still displays a mark indicating \"Disconnected\", \nfollow the steps below to troubleshoot the issue.\n\n### Check Endpoint Settings\n\nVerify that the endpoint setting is correct. You can set the endpoint in the \nIDE settings page (except for Vim/NeoVim, which do not have this option) or by \nediting the [config file](https://tabby.tabbyml.com/docs/extensions/configurations). \nKeep in mind that the IDE settings take priority over the config file. \nIf you wish to use the setting from the config file, ensure that the IDE setting\nis empty.\n\n### Authentication Token\n\nIf you have enabled authentication on your Tabby server, you will need to set\nthe authentication token in the IDE or the [config file](https://tabby.tabbyml.com/docs/extensions/configurations).\n\n### Verify Tabby Server Status\n\nOnce the Tabby server is running, it should display a log message such as \n`Listening at 0.0.0.0:8080`. \nOpen your browser and navigate to `http://localhost:8080/swagger-ui/`\n(Replace `localhost:8080` with the correct IP/domain and port if you have setup your \nTabby server on a remote machine). The browser should display a web page with Swagger UI. \n\nTo test the server, expand the `/v1/completions` section, click on `Try it out`, \nand then click `Execute`. \nIf you receive a response, it indicates that the Tabby server is running properly.\n\nYou can also use `curl` to send a completion request to Tabby server. For example:\n\n```shell\ncurl -X 'POST' \\\n 'http://localhost:8080/v1/completions' \\\n -H 'accept: application/json' \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"language\": \"python\",\n \"segments\": {\n \"prefix\": \"def fib(n):\\n \",\n \"suffix\": \"\\n return fib(n - 1) + fib(n - 2)\"\n }\n}'\n```\n\nIf you can not see Swagger UI page, or can not get response of completion request, \nplease check the server log to see if there is any error.\n\n### Proxy Settings\n\nPlease note that Tabby extensions for IDEs currently do not support proxy settings. \nIf you need to access your Tabby server through a proxy, consider setting up \na reverse proxy and using the reverse proxy URL as the endpoint for the Tabby IDE extension.\n\n## Cannot Get Any Completions?\n\nIf you are able to connect the Tabby extension to the Tabby server but are unable to \nreceive any completions, you can follow the steps below to troubleshoot the issue.\n\n### Check Trigger Mode Settings\n\nTabby is set to automatic trigger mode by default. In this mode, you should receive \ncompletions after a short delay when you stop typing. The delay may vary depending \non your server's performance and settings. \nIf you are using manual trigger mode, you need to press `Alt + \\` (`Ctrl + \\` for IntelliJ and Vim plugins) \nto trigger a completion request. The status bar item of Tabby IDE extension should show a loading \nindicator for a brief period before displaying the completions. \nKeep in mind that Tabby may not provide any suggestions if there is no necessary\nfor the current code context.\n\n### Check Request Timeouts\n\nIf your completion requests are timing out, Tabby may display a warning message. \nThis could be due to network issues or poor server performance, especially when \nrunning a large model on a CPU. To improve performance, consider running the model \non a GPU with CUDA or ROCm support or on Apple M1/M2 with Metal support. When running \nthe server, make sure to specify the device in the arguments using `--device cuda`, `--device rocm` or\n`--device metal`. You can also try using a smaller model from the available [models](https://tabby.tabbyml.com/docs/models/). \n\n## Problems with the Chat Panel?\n\n### Check Server Enabling Chat Feature\n\nThe chat panel is a feature that requires the Tabby server to enable the chat feature. Please make sure that the chat feature is properly configured in the server configuration file or that the server is started with the `--chat-model MODEL_NAME` flag. \nYou can access the server management page at `https://demo.tabbyml.com/system` to check if the chat feature is enabled.\n\n### Check Browser Compatibility in IntelliJ Platform IDEs\n\nThe chat panel in IntelliJ Platform IDEs is implemented using an embedded JCEF browser. The support is included in the default Java Runtime for JetBrains IDEs, but it may not be by default in Android Studio. \nPlease follow the steps below to ensure that the JCEF browser is enabled in your IDE:\n\n1. Open `Search Everywhere` (Double `Shift`) or `Find Action` (`Ctrl + Shift + A`), and search for the action `Choose Boot Java Runtime for the IDE...`\n2. Select a Java Runtime that includes JCEF support \n ![Choose Boot Java Runtime for the IDE](./intellij-choose-boot-runtime-for-the-ide.png)\n3. Restart the IDE\n\n## Want to Deep Dive via Logs?\n\nIf you cannot solve the issue using the previous steps, you may want to \ninvestigate further by checking the logs of Tabby extensions. \nIf you want help from the community, it is also recommended to share the logs \nof the Tabby extensions.\n\n### VSCode\n\nFor VSCode, you can check the `Output` window and select the `Tabby` channel or\n`Tabby Agent` channel.\n![VSCode Output View](./vscode-output-view.png)\nThe default log level is `info`. To enable debug logs, you can use the command \n`Developer: Set Log Level...` to set the log level to `debug`.\n\n### IntelliJ Platform IDEs\n\nFor IntelliJ Platform IDEs, you can check the logs for the IDE using `Help -> Show Log in Explorer|Finder|Files...`, \nor follow [this document](https://intellij-support.jetbrains.com/hc/en-us/articles/207241085-Locating-IDE-log-files) to locate the log file.\nThis log file contains all the logs for the IDE, and you can filter them by searching for the \nkeyword `com.tabbyml.intellijtabby`.\nYou can also enable all level logs by editing `Help -> Diagnostic Tools -> Debug Log Settings...` and\nadd `com.tabbyml.intellijtabby:all` to the list.\n\n### Tabby Agent Logs\n\nAs the agent runs as a separate Node.js process for IntelliJ Platform IDEs plugin \nand Vim/NeoVim plugin, its logs are written separately.\n\nBy default, the agent logs are set to `\"silent\"`, which means the agent logs are \nnot written to disk. To enable Tabby agent debug logs, editing the \n[config file](https://tabby.tabbyml.com/docs/extensions/configurations), \nuncomment the `logs` section and set `level` to `\"debug\"`, then save the file \nto apply the changes.\n \nYou can find the agent logs in the `~/.tabby-client/agent/logs` directory. These \nlogs are written using [pino](https://github.com/pinojs/pino), and you can \nuse `pino-pretty` to format the log file for easier readability.\n\n```shell\ntail -f ~/.tabby-client/agent/logs/20240101.0.log | npx pino-pretty\n```\n\n## Still Have Issues?\n\nIf you still have any issues, please feel free to [open an issue on github](https://github.com/TabbyML/tabby/issues/new), \nor join our [slack community](https://links.tabbyml.com/join-slack)\nfor further support.\n\n","website/docs/references/models-http-api/deepinfra.md":"# DeepInfra\n\n[DeepInfra](https://deepinfra.com/) is a cloud platform providing efficient and scalable model inference services, offering access to various open-source models like [Llama 3](https://deepinfra.com/meta-llama/Llama-3.3-70B-Instruct), [Mixtral](https://deepinfra.com/mistralai/Mixtral-8x7B-Instruct-v0.1), and [Qwen](https://deepinfra.com/Qwen/Qwen2.5-Coder-32B-Instruct).\n\n## Chat model\n\nDeepInfra provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"meta-llama/Llama-3.3-70B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepInfra provides an OpenAI-compatible completion API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"openai/completion\"\nmodel_name = \"Qwen/Qwen2.5-Coder-32B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepInfra also provides an OpenAI-compatible embeddings API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"openai/embedding\"\nmodel_name = \"BAAI/bge-base-en-v1.5\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n","website/docs/references/models-http-api/deepseek.md":"# DeepSeek\n\n[DeepSeek](https://www.deepseek.com/) is an AI company that develops large language models specialized in coding and general tasks. Their models include [DeepSeek V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) for general tasks and [DeepSeek Coder](https://huggingface.co/collections/deepseek-ai/deepseekcoder-v2-666bf4b274a5f556827ceeca) specifically optimized for programming tasks.\n\n## Chat model\n\nDeepSeek provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepSeek offers a specialized completion API interface for code completion tasks.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"deepseek/completion\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/beta\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepSeek currently does not provide embedding model APIs.\n","website/docs/references/models-http-api/azure-openai.md":"# Azure OpenAI\n\n[Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) is a cloud-based service that provides Azure customers with access to OpenAI's powerful language models including GPT-4, GPT-3.5, and various embedding models.\n\nPlease be aware that azure will be supported starting with version 0.24, which is scheduled for release by end of 01/2025\n\n## Chat model\n\nIt supports various GPT series chat models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"azure/chat\"\nmodel_name = \"gpt-4o-mini\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAzure OpenAI currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nIt supports text-embedding-3-small, text-embedding-3-large and other embedding models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"azure/embedding\"\nmodel_name = \"text-embedding-3-large\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n","website/docs/references/models-http-api/huggingface.md":"# Hugging Face Inference Providers\n\n[Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers) offers access to frontier open models from multiple providers through a unified API. \n\nYou'll need a [Hugging Face account](https://huggingface.co/join) and an [access token](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained).\n\n## Chat model\n\nHugging Face Inference Providers provides an OpenAI-compatible chat API interface. Here we use the `MiniMaxAI/MiniMax-M2` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"MiniMaxAI/MiniMax-M2\" # specify the model you want to use\napi_endpoint = \"https://router.huggingface.co/v1\"\napi_key = \"your-hf-token\"\n```\n\n### Available models\n\nYou can find a complete list of models supported by at least one provider [on the Hub](https://huggingface.co/models?inference_provider=all). You can also access these programmatically, see this [guide](https://huggingface.co/docs/inference-providers/hub-api) for more details.\n\n## Completion model\n\nHugging Face Inference Providers does not offer completion models (FIM) through their OpenAI-compatible API. For code completion, use a local model with Tabby.\n\n## Embeddings model\n\nWhile Hugging Face Inference Providers supports embeddings models, Tabby does not currently support the embeddings API interface for Hugging Face Inference Providers.\n","website/docs/references/models-http-api/fireworks.md":"# Fireworks\n\n[Fireworks](https://app.fireworks.ai/) is a cloud platform that offers efficient model inference and deployment services,\nproviding cost-effective access to a variety of AI models through their API service,\nincluding [Llama 2](https://fireworks.ai/models/fireworks/llama-v2-70b-chat),\n[DeepSeek V3](https://fireworks.ai/models/fireworks/deepseek-v3),\n[DeepSeek Coder](https://fireworks.ai/models/fireworks/deepseek-coder-v2-instruct) and other open-source models.\n\n## Chat model\n\nFireworks provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"accounts/fireworks/models/deepseek-v3\"\napi_endpoint = \"https://api.fireworks.ai/inference/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nFireworks does not offer completion models (FIM) through their API.\n\n## Embeddings model\n\nWhile Fireworks provides embedding model APIs, Tabby has not yet implemented a compatible client to interface with these APIs. Therefore, embedding functionality is currently not available through Tabby's integration with Fireworks.\n","website/docs/references/models-http-api/jan.ai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/jan.ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/jan.ai.md)","website/docs/references/models-http-api/llamafile.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/llamafile.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/llamafile.md)","website/docs/references/models-http-api/lm-studio.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/lm-studio.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/lm-studio.md)","website/docs/references/models-http-api/mistral-ai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/mistral-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/mistral-ai.md)","website/docs/references/models-http-api/ollama.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/ollama.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/ollama.md)","website/docs/references/models-http-api/openai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openai.md)","website/docs/references/models-http-api/openrouter.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openrouter.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openrouter.md)","website/docs/references/models-http-api/perplexity.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/perplexity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/perplexity.md)","website/docs/references/models-http-api/vllm.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/vllm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/vllm.md)","website/docs/references/models-http-api/voyage-ai.md":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/voyage-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/voyage-ai.md)","website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml)","website/docs/references/cloud-deployment/bentoml/bentofile.yaml":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentofile.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentofile.yaml)","website/docs/references/cloud-deployment/bentoml/index.md":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/index.md)","website/docs/references/cloud-deployment/hugging-face/index.md":"# Hugging-face Skill\n\nPath: `website/docs/references/cloud-deployment/hugging-face/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/hugging-face/index.md)","website/docs/references/cloud-deployment/modal/index.md":"# Modal Skill\n\nPath: `website/docs/references/cloud-deployment/modal/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/modal/index.md)","website/docs/references/cloud-deployment/skypilot/index.md":"# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/index.md)","website/docs/references/cloud-deployment/skypilot/tabby.yaml":"# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/tabby.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/tabby.yaml)"},"items":[{"name":"configurations.md","path":"website/docs/extensions/configurations.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/extensions/configurations.md","title":"Extensions Skill","category":"anthropic-skill","format":"markdown","content":"---\nsidebar_position: 98\n---\n\n# Configuration\n\nThis document describes the available configurations for [Tabby Agent](https://github.com/TabbyML/tabby/tree/main/clients/tabby-agent).\n\n## Config File\n\nThe Tabby agent, which is the core component of Tabby IDE extensions, reads configurations from the `~/.tabby-client/agent/config.toml` file. This file is automatically created when you first run the Tabby IDE extensions. You can edit this file to modify the configurations. The Tabby IDE extensions will automatically reload the config file when it detects changes.\n\n:::tip\n- The config file is written in [TOML](https://toml.io/en/). When you edit the config template and want to enable a configuration, make sure to uncomment the full section including the leading line, not just the line with the configuration value.\n- Configurations set via the IDE settings page take precedence over the config file. If you want to use a configuration from the config file, make sure that the IDE setting is empty.\n- If you are using the Tabby VSCode extension in a web browser, this config file is not available. You can use the VSCode settings page to configure the extension.\n:::\n\n## Server\n\nThe `server` section contains configurations related to the Tabby server.\n\n**NOTE**: If your Tabby server requires an authentication token, remember to set it here.\n\n```toml\n# Server\n# You can set the server endpoint here and an optional authentication token if required.\n[server]\nendpoint = \"http://localhost:8080\" # http or https URL\ntoken = \"your-token-here\" # if token is set, request header Authorization = \"Bearer $token\" will be added automatically\n\n# You can add custom request headers.\n[server.requestHeaders]\nHeader1 = \"Value1\" # list your custom headers here\nHeader2 = \"Value2\" # values can be strings, numbers or booleans\n```\n\n## Logs\n\nIf you encounter any issues with the Tabby IDE extensions and need to report a bug, you can enable debug logs to help us investigate the issue.\n\n```toml\n# Logs\n# You can set the log level here. The log file is located at ~/.tabby-client/agent/logs/.\n[logs]\nlevel = \"silent\" # \"silent\" or \"error\" or \"debug\"\n```\n\n## Usage Collection\n\nTabby IDE extensions collect aggregated anonymous usage data and sends it to the Tabby team to help improve our products.\n\n**Do not worry, your code, generated completions, or any identifying information is never tracked or transmitted.** \n\nThe data we collect, as of the latest update on November 6, 2023, contains following major parts:\n\n- System info and extension version info\n- Completions statistics\n - Completion count\n - Completion accepted count\n - Completion HTTP request latency\n\nWe sincerely appreciate your contribution in sending anonymous usage data. However, if you prefer not to participate, you can disable anonymous usage tracking here:\n\n```toml\n# Anonymous usage tracking\n[anonymousUsageTracking]\ndisable = false # set to true to disable\n```\n","frontmatter":{"sidebar_position":98},"isInternal":false,"tokens":652,"sizeBytes":2932},{"name":"index.md","path":"website/docs/extensions/troubleshooting/index.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/extensions/troubleshooting/index.md","title":"Troubleshooting Skill","category":"anthropic-skill","format":"markdown","content":"---\nsidebar_position: 99\n---\n\n# Troubleshooting\n\nThis document aims to assist you in troubleshooting issues with the Tabby extensions \nfor various IDEs such as VSCode, IntelliJ Platform IDEs, and Vim / NeoVim.\n\n## Tabby Initialization Failed?\n\nThis problem may occur when you first install the Tabby in IntelliJ Platform IDEs \nor Vim/NeoVim. \n\nThe Tabby IDE extension runs its core logic in the Tabby agent. In the case of VSCode, \nthe agent runs within the VSCode Extension Host, while for IntelliJ Platform IDEs \nand Vim/NeoVim, the agent runs as a separate Node.js process.\n\n### Install Node.js\n\nYou can follow the instructions on the [Node.js website](https://nodejs.org/en/download/)\nto install Node.js. Alternatively, you can use a version manager such as [nvm](https://github.com/nvm-sh/nvm). \n**Note**: Tabby IDE extension requires Node.js version 18.0.0 or higher.\n\n### Specify Node Binary Path\n\nIf the node binary is already accessible via your `PATH` environment variable, \nyou can skip this step. Otherwise, you will need to specify the path to the node \nbinary in the IDE settings.\n\nFor IntelliJ Platform IDEs (Tabby plugin version 0.6.0 or higher):\n* Click on Tabby plugin status bar item and select `Open Settings...`.\n* Enter the path to the node binary on your system in the `Node binary` field,\n e.g. `/usr/local/bin/node`, `C:\\Program Files\\nodejs\\node.exe`. \n If you are using a version manager such as `nvm`, you can enter the path to the\n node binary installed by the version manager, e.g. `~/.nvm/versions/node/v18.18.0/bin/node`.\n* Restart the IDE\n\nIf you installed Node.js via snap, please use `/snap/node/current/bin/node` rather than \n`/snap/bin/node` as the node binary path.\n\n## Cannot Connect to Tabby Server?\n\nIf you have setup the endpoint for the Tabby server but the status bar item of \nthe Tabby IDE extension still displays a mark indicating \"Disconnected\", \nfollow the steps below to troubleshoot the issue.\n\n### Check Endpoint Settings\n\nVerify that the endpoint setting is correct. You can set the endpoint in the \nIDE settings page (except for Vim/NeoVim, which do not have this option) or by \nediting the [config file](https://tabby.tabbyml.com/docs/extensions/configurations). \nKeep in mind that the IDE settings take priority over the config file. \nIf you wish to use the setting from the config file, ensure that the IDE setting\nis empty.\n\n### Authentication Token\n\nIf you have enabled authentication on your Tabby server, you will need to set\nthe authentication token in the IDE or the [config file](https://tabby.tabbyml.com/docs/extensions/configurations).\n\n### Verify Tabby Server Status\n\nOnce the Tabby server is running, it should display a log message such as \n`Listening at 0.0.0.0:8080`. \nOpen your browser and navigate to `http://localhost:8080/swagger-ui/`\n(Replace `localhost:8080` with the correct IP/domain and port if you have setup your \nTabby server on a remote machine). The browser should display a web page with Swagger UI. \n\nTo test the server, expand the `/v1/completions` section, click on `Try it out`, \nand then click `Execute`. \nIf you receive a response, it indicates that the Tabby server is running properly.\n\nYou can also use `curl` to send a completion request to Tabby server. For example:\n\n```shell\ncurl -X 'POST' \\\n 'http://localhost:8080/v1/completions' \\\n -H 'accept: application/json' \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"language\": \"python\",\n \"segments\": {\n \"prefix\": \"def fib(n):\\n \",\n \"suffix\": \"\\n return fib(n - 1) + fib(n - 2)\"\n }\n}'\n```\n\nIf you can not see Swagger UI page, or can not get response of completion request, \nplease check the server log to see if there is any error.\n\n### Proxy Settings\n\nPlease note that Tabby extensions for IDEs currently do not support proxy settings. \nIf you need to access your Tabby server through a proxy, consider setting up \na reverse proxy and using the reverse proxy URL as the endpoint for the Tabby IDE extension.\n\n## Cannot Get Any Completions?\n\nIf you are able to connect the Tabby extension to the Tabby server but are unable to \nreceive any completions, you can follow the steps below to troubleshoot the issue.\n\n### Check Trigger Mode Settings\n\nTabby is set to automatic trigger mode by default. In this mode, you should receive \ncompletions after a short delay when you stop typing. The delay may vary depending \non your server's performance and settings. \nIf you are using manual trigger mode, you need to press `Alt + \\` (`Ctrl + \\` for IntelliJ and Vim plugins) \nto trigger a completion request. The status bar item of Tabby IDE extension should show a loading \nindicator for a brief period before displaying the completions. \nKeep in mind that Tabby may not provide any suggestions if there is no necessary\nfor the current code context.\n\n### Check Request Timeouts\n\nIf your completion requests are timing out, Tabby may display a warning message. \nThis could be due to network issues or poor server performance, especially when \nrunning a large model on a CPU. To improve performance, consider running the model \non a GPU with CUDA or ROCm support or on Apple M1/M2 with Metal support. When running \nthe server, make sure to specify the device in the arguments using `--device cuda`, `--device rocm` or\n`--device metal`. You can also try using a smaller model from the available [models](https://tabby.tabbyml.com/docs/models/). \n\n## Problems with the Chat Panel?\n\n### Check Server Enabling Chat Feature\n\nThe chat panel is a feature that requires the Tabby server to enable the chat feature. Please make sure that the chat feature is properly configured in the server configuration file or that the server is started with the `--chat-model MODEL_NAME` flag. \nYou can access the server management page at `https://demo.tabbyml.com/system` to check if the chat feature is enabled.\n\n### Check Browser Compatibility in IntelliJ Platform IDEs\n\nThe chat panel in IntelliJ Platform IDEs is implemented using an embedded JCEF browser. The support is included in the default Java Runtime for JetBrains IDEs, but it may not be by default in Android Studio. \nPlease follow the steps below to ensure that the JCEF browser is enabled in your IDE:\n\n1. Open `Search Everywhere` (Double `Shift`) or `Find Action` (`Ctrl + Shift + A`), and search for the action `Choose Boot Java Runtime for the IDE...`\n2. Select a Java Runtime that includes JCEF support \n ![Choose Boot Java Runtime for the IDE](./intellij-choose-boot-runtime-for-the-ide.png)\n3. Restart the IDE\n\n## Want to Deep Dive via Logs?\n\nIf you cannot solve the issue using the previous steps, you may want to \ninvestigate further by checking the logs of Tabby extensions. \nIf you want help from the community, it is also recommended to share the logs \nof the Tabby extensions.\n\n### VSCode\n\nFor VSCode, you can check the `Output` window and select the `Tabby` channel or\n`Tabby Agent` channel.\n![VSCode Output View](./vscode-output-view.png)\nThe default log level is `info`. To enable debug logs, you can use the command \n`Developer: Set Log Level...` to set the log level to `debug`.\n\n### IntelliJ Platform IDEs\n\nFor IntelliJ Platform IDEs, you can check the logs for the IDE using `Help -> Show Log in Explorer|Finder|Files...`, \nor follow [this document](https://intellij-support.jetbrains.com/hc/en-us/articles/207241085-Locating-IDE-log-files) to locate the log file.\nThis log file contains all the logs for the IDE, and you can filter them by searching for the \nkeyword `com.tabbyml.intellijtabby`.\nYou can also enable all level logs by editing `Help -> Diagnostic Tools -> Debug Log Settings...` and\nadd `com.tabbyml.intellijtabby:all` to the list.\n\n### Tabby Agent Logs\n\nAs the agent runs as a separate Node.js process for IntelliJ Platform IDEs plugin \nand Vim/NeoVim plugin, its logs are written separately.\n\nBy default, the agent logs are set to `\"silent\"`, which means the agent logs are \nnot written to disk. To enable Tabby agent debug logs, editing the \n[config file](https://tabby.tabbyml.com/docs/extensions/configurations), \nuncomment the `logs` section and set `level` to `\"debug\"`, then save the file \nto apply the changes.\n \nYou can find the agent logs in the `~/.tabby-client/agent/logs` directory. These \nlogs are written using [pino](https://github.com/pinojs/pino), and you can \nuse `pino-pretty` to format the log file for easier readability.\n\n```shell\ntail -f ~/.tabby-client/agent/logs/20240101.0.log | npx pino-pretty\n```\n\n## Still Have Issues?\n\nIf you still have any issues, please feel free to [open an issue on github](https://github.com/TabbyML/tabby/issues/new), \nor join our [slack community](https://links.tabbyml.com/join-slack)\nfor further support.\n\n","frontmatter":{"sidebar_position":99},"isInternal":false,"tokens":2087,"sizeBytes":8725},{"name":"_category_.yaml","path":"website/docs/references/_category_.yaml","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/_category_.yaml","title":"References Skill","category":"anthropic-skill","format":"yaml","content":"label: 📚 References\nposition: 100","isInternal":false,"tokens":11,"sizeBytes":36},{"name":"bentodeploy.yaml","path":"website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml","title":"Bentoml Skill","category":"anthropic-skill","format":"yaml","content":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml)","tokens":300,"sizeBytes":240},{"name":"bentofile.yaml","path":"website/docs/references/cloud-deployment/bentoml/bentofile.yaml","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentofile.yaml","title":"Bentoml Skill","category":"anthropic-skill","format":"yaml","content":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentofile.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentofile.yaml)","tokens":300,"sizeBytes":236},{"name":"index.md","path":"website/docs/references/cloud-deployment/bentoml/index.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/index.md","title":"Bentoml Skill","category":"anthropic-skill","format":"markdown","content":"# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/index.md)","tokens":300,"sizeBytes":224},{"name":"index.md","path":"website/docs/references/cloud-deployment/hugging-face/index.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/hugging-face/index.md","title":"Hugging-face Skill","category":"anthropic-skill","format":"markdown","content":"# Hugging-face Skill\n\nPath: `website/docs/references/cloud-deployment/hugging-face/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/hugging-face/index.md)","tokens":300,"sizeBytes":239},{"name":"index.md","path":"website/docs/references/cloud-deployment/modal/index.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/modal/index.md","title":"Modal Skill","category":"anthropic-skill","format":"markdown","content":"# Modal Skill\n\nPath: `website/docs/references/cloud-deployment/modal/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/modal/index.md)","tokens":300,"sizeBytes":218},{"name":"index.md","path":"website/docs/references/cloud-deployment/skypilot/index.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/index.md","title":"Skypilot Skill","category":"anthropic-skill","format":"markdown","content":"# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/index.md)","tokens":300,"sizeBytes":227},{"name":"tabby.yaml","path":"website/docs/references/cloud-deployment/skypilot/tabby.yaml","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/tabby.yaml","title":"Skypilot Skill","category":"anthropic-skill","format":"yaml","content":"# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/tabby.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/tabby.yaml)","tokens":300,"sizeBytes":231},{"name":"amazon-bedrock.md","path":"website/docs/references/models-http-api/amazon-bedrock.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/amazon-bedrock.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Amazon Bedrock\n\nAmazon Bedrock is a fully managed service on AWS that provides access to foundation models from various AI companies through a single API. With [Amazon Bedrock Access Gateway](https://github.com/aws-samples/bedrock-access-gateway), you can access Anthropic's Claude models through an OpenAI-compatible interface, enabling seamless integration with tools and applications designed for OpenAI's API structure.\n\nFollow the Amazon Bedrock Access Gateway setup guide to deploy your own OpenAI-compatible API endpoint for Claude models.\n\n## Chat model\n\nAmazon Bedrock Access Gateway provides an OpenAI-compatible chat API interface for Claude models. Here we use the `us.anthropic.claude-3-5-sonnet-20241022-v2:0` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"us.anthropic.claude-3-5-sonnet-20241022-v2:0\"\napi_endpoint = \"http://Bedrock-Proxy-xxxxx.{Region}.elb.amazonaws.com/api/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAmazon Bedrock does not provide completion models.\n\n## Embeddings model\n\nWhile Amazon Bedrock supports embeddings models, Tabby does not currently support the embeddings API interface for Amazon models.\n","isInternal":false,"tokens":283,"sizeBytes":1216},{"name":"anthropic-claude.md","path":"website/docs/references/models-http-api/anthropic-claude.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/anthropic-claude.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Anthropic Claude\n\n[Anthropic](https://www.anthropic.com/) is an AI research company that develops large language models, including the Claude family of models. While Tabby doesn't natively support Claude's API, you can access Claude models through an OpenAI-compatible API interface using [claude2openai](https://github.com/missuo/claude2openai) as a middleware.\n\n## Chat model\n\nAfter deploying the claude2openai middleware, you can access all Claude family models through an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"claude-3-sonnet-20240229\"\n# Middleware endpoint (adjust host and port according to your deployment)\napi_endpoint = \"http://127.0.0.1:6600/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAnthropic currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nAnthropic currently does not provide embedding model APIs.\n","isInternal":false,"tokens":227,"sizeBytes":951},{"name":"azure-openai.md","path":"website/docs/references/models-http-api/azure-openai.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/azure-openai.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Azure OpenAI\n\n[Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) is a cloud-based service that provides Azure customers with access to OpenAI's powerful language models including GPT-4, GPT-3.5, and various embedding models.\n\nPlease be aware that azure will be supported starting with version 0.24, which is scheduled for release by end of 01/2025\n\n## Chat model\n\nIt supports various GPT series chat models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"azure/chat\"\nmodel_name = \"gpt-4o-mini\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAzure OpenAI currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nIt supports text-embedding-3-small, text-embedding-3-large and other embedding models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"azure/embedding\"\nmodel_name = \"text-embedding-3-large\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n","isInternal":false,"tokens":278,"sizeBytes":1142},{"name":"deepinfra.md","path":"website/docs/references/models-http-api/deepinfra.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/deepinfra.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# DeepInfra\n\n[DeepInfra](https://deepinfra.com/) is a cloud platform providing efficient and scalable model inference services, offering access to various open-source models like [Llama 3](https://deepinfra.com/meta-llama/Llama-3.3-70B-Instruct), [Mixtral](https://deepinfra.com/mistralai/Mixtral-8x7B-Instruct-v0.1), and [Qwen](https://deepinfra.com/Qwen/Qwen2.5-Coder-32B-Instruct).\n\n## Chat model\n\nDeepInfra provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"meta-llama/Llama-3.3-70B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepInfra provides an OpenAI-compatible completion API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"openai/completion\"\nmodel_name = \"Qwen/Qwen2.5-Coder-32B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepInfra also provides an OpenAI-compatible embeddings API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"openai/embedding\"\nmodel_name = \"BAAI/bge-base-en-v1.5\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n","isInternal":false,"tokens":373,"sizeBytes":1274},{"name":"deepseek.md","path":"website/docs/references/models-http-api/deepseek.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/deepseek.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# DeepSeek\n\n[DeepSeek](https://www.deepseek.com/) is an AI company that develops large language models specialized in coding and general tasks. Their models include [DeepSeek V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) for general tasks and [DeepSeek Coder](https://huggingface.co/collections/deepseek-ai/deepseekcoder-v2-666bf4b274a5f556827ceeca) specifically optimized for programming tasks.\n\n## Chat model\n\nDeepSeek provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepSeek offers a specialized completion API interface for code completion tasks.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"deepseek/completion\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/beta\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepSeek currently does not provide embedding model APIs.\n","isInternal":false,"tokens":263,"sizeBytes":1031},{"name":"fireworks.md","path":"website/docs/references/models-http-api/fireworks.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/fireworks.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Fireworks\n\n[Fireworks](https://app.fireworks.ai/) is a cloud platform that offers efficient model inference and deployment services,\nproviding cost-effective access to a variety of AI models through their API service,\nincluding [Llama 2](https://fireworks.ai/models/fireworks/llama-v2-70b-chat),\n[DeepSeek V3](https://fireworks.ai/models/fireworks/deepseek-v3),\n[DeepSeek Coder](https://fireworks.ai/models/fireworks/deepseek-coder-v2-instruct) and other open-source models.\n\n## Chat model\n\nFireworks provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"accounts/fireworks/models/deepseek-v3\"\napi_endpoint = \"https://api.fireworks.ai/inference/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nFireworks does not offer completion models (FIM) through their API.\n\n## Embeddings model\n\nWhile Fireworks provides embedding model APIs, Tabby has not yet implemented a compatible client to interface with these APIs. Therefore, embedding functionality is currently not available through Tabby's integration with Fireworks.\n","isInternal":false,"tokens":262,"sizeBytes":1112},{"name":"huggingface.md","path":"website/docs/references/models-http-api/huggingface.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/huggingface.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Hugging Face Inference Providers\n\n[Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers) offers access to frontier open models from multiple providers through a unified API. \n\nYou'll need a [Hugging Face account](https://huggingface.co/join) and an [access token](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained).\n\n## Chat model\n\nHugging Face Inference Providers provides an OpenAI-compatible chat API interface. Here we use the `MiniMaxAI/MiniMax-M2` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"MiniMaxAI/MiniMax-M2\" # specify the model you want to use\napi_endpoint = \"https://router.huggingface.co/v1\"\napi_key = \"your-hf-token\"\n```\n\n### Available models\n\nYou can find a complete list of models supported by at least one provider [on the Hub](https://huggingface.co/models?inference_provider=all). You can also access these programmatically, see this [guide](https://huggingface.co/docs/inference-providers/hub-api) for more details.\n\n## Completion model\n\nHugging Face Inference Providers does not offer completion models (FIM) through their OpenAI-compatible API. For code completion, use a local model with Tabby.\n\n## Embeddings model\n\nWhile Hugging Face Inference Providers supports embeddings models, Tabby does not currently support the embeddings API interface for Hugging Face Inference Providers.\n","isInternal":false,"tokens":343,"sizeBytes":1468},{"name":"jan.ai.md","path":"website/docs/references/models-http-api/jan.ai.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/jan.ai.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/jan.ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/jan.ai.md)","tokens":300,"sizeBytes":216},{"name":"llamafile.md","path":"website/docs/references/models-http-api/llamafile.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/llamafile.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/llamafile.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/llamafile.md)","tokens":300,"sizeBytes":222},{"name":"lm-studio.md","path":"website/docs/references/models-http-api/lm-studio.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/lm-studio.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/lm-studio.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/lm-studio.md)","tokens":300,"sizeBytes":222},{"name":"mistral-ai.md","path":"website/docs/references/models-http-api/mistral-ai.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/mistral-ai.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/mistral-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/mistral-ai.md)","tokens":300,"sizeBytes":224},{"name":"ollama.md","path":"website/docs/references/models-http-api/ollama.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/ollama.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/ollama.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/ollama.md)","tokens":300,"sizeBytes":216},{"name":"openai.md","path":"website/docs/references/models-http-api/openai.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openai.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openai.md)","tokens":300,"sizeBytes":216},{"name":"openrouter.md","path":"website/docs/references/models-http-api/openrouter.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openrouter.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openrouter.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openrouter.md)","tokens":300,"sizeBytes":224},{"name":"perplexity.md","path":"website/docs/references/models-http-api/perplexity.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/perplexity.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/perplexity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/perplexity.md)","tokens":300,"sizeBytes":224},{"name":"vllm.md","path":"website/docs/references/models-http-api/vllm.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/vllm.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/vllm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/vllm.md)","tokens":300,"sizeBytes":212},{"name":"voyage-ai.md","path":"website/docs/references/models-http-api/voyage-ai.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/voyage-ai.md","title":"Models-http-api Skill","category":"anthropic-skill","format":"markdown","content":"# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/voyage-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/voyage-ai.md)","tokens":300,"sizeBytes":222},{"name":"programming-languages.md","path":"website/docs/references/programming-languages.md","rawUrl":"https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/programming-languages.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"---\nsidebar_position: 6\n---\n\n# Programming Languages\n\nMost models nowadays support a large number of programming languages (thanks to [The Stack](https://huggingface.co/datasets/bigcode/the-stack), which has collected 358 programming languages).\nIn Tabby, we need to add configuration for each language to maximize performance and completion quality.\n\nCurrently, there are two aspects of support that need to be added for each language.\n\n**Stop Words**\n\nStop words determine when the language model can early stop its decoding steps, resulting in better latency and affecting the quality of completion. We suggest adding all top-level keywords as part of the stop words.\n\n**Repository Context**\n\nWe parse languages into chunks and compute a token-based index for serving time Retrieval Augmented Code Completion. In Tabby, we define these repository contexts as [treesitter queries](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax), and the query results will be indexed.\n\nFor an actual example of an issue or pull request adding the above support, please check out https://github.com/TabbyML/tabby/issues/553 as a reference.\n\n## Supported Languages\n\n* [Rust](https://www.rust-lang.org/)\n* [Python](https://www.python.org/)\n* [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)\n* [TypeScript](https://www.typescriptlang.org/)\n* [Golang](https://go.dev/)\n* [Ruby](https://www.ruby-lang.org/)\n* [Java](https://www.java.com/)\n* [Kotlin](https://www.kotlinlang.org/)\n* [C/C++](https://cplusplus.com/)\n* [PHP](https://www.php.net/)\n* [C#](https://learn.microsoft.com/en-us/dotnet/csharp/)\n* [Solidity](https://soliditylang.org/)\n* [R](https://www.r-project.org/)\n* [Dart](https://dart.dev/)\n* [Lua](https://www.lua.org)\n* [Elixir](https://elixir-lang.org)\n* [OCaml](https://ocaml.org/)\n* [GDScript](https://gdscript.com/)\n* [Scala](https://www.scala-lang.org/)\n\n## Languages Missing Certain Support\n\n| Language | Stop Words (time to contribute: ~5 min) | Repository Context (time to contribute: ~1 hr) |\n| :------: | :-------------------------------------: | :--------------------------------------------: |\n| CSS | 🚫 | 🚫 |\n| Haskell | 🚫 | 🚫 |\n| Julia | 🚫 | 🚫 |\n| Perl | 🚫 | 🚫 |\n","frontmatter":{"sidebar_position":6},"isInternal":false,"tokens":573,"sizeBytes":2582}],"systemPromptSnippet":"\n\n---\nsidebar_position: 98\n---\n\n# Configuration\n\nThis document describes the available configurations for [Tabby Agent](https://github.com/TabbyML/tabby/tree/main/clients/tabby-agent).\n\n## Config File\n\nThe Tabby agent, which is the core component of Tabby IDE extensions, reads configurations from the `~/.tabby-client/agent/config.toml` file. This file is automatically created when you first run the Tabby IDE extensions. You can edit this file to modify the configurations. The Tabby IDE extensions will automatically reload the config file when it detects changes.\n\n:::tip\n- The config file is written in [TOML](https://toml.io/en/). When you edit the config template and want to enable a configuration, make sure to uncomment the full section including the leading line, not just the line with the configuration value.\n- Configurations set via the IDE settings page take precedence over the config file. If you want to use a configuration from the config file, make sure that the IDE setting is empty.\n- If you are using the Tabby VSCode extension in a web browser, this config file is not available. You can use the VSCode settings page to configure the extension.\n:::\n\n## Server\n\nThe `server` section contains configurations related to the Tabby server.\n\n**NOTE**: If your Tabby server requires an authentication token, remember to set it here.\n\n```toml\n# Server\n# You can set the server endpoint here and an optional authentication token if required.\n[server]\nendpoint = \"http://localhost:8080\" # http or https URL\ntoken = \"your-token-here\" # if token is set, request header Authorization = \"Bearer $token\" will be added automatically\n\n# You can add custom request headers.\n[server.requestHeaders]\nHeader1 = \"Value1\" # list your custom headers here\nHeader2 = \"Value2\" # values can be strings, numbers or booleans\n```\n\n## Logs\n\nIf you encounter any issues with the Tabby IDE extensions and need to report a bug, you can enable debug logs to help us investigate the issue.\n\n```toml\n# Logs\n# You can set the log level here. The log file is located at ~/.tabby-client/agent/logs/.\n[logs]\nlevel = \"silent\" # \"silent\" or \"error\" or \"debug\"\n```\n\n## Usage Collection\n\nTabby IDE extensions collect aggregated anonymous usage data and sends it to the Tabby team to help improve our products.\n\n**Do not worry, your code, generated completions, or any identifying information is never tracked or transmitted.** \n\nThe data we collect, as of the latest update on November 6, 2023, contains following major parts:\n\n- System info and extension version info\n- Completions statistics\n - Completion count\n - Completion accepted count\n - Completion HTTP request latency\n\nWe sincerely appreciate your contribution in sending anonymous usage data. However, if you prefer not to participate, you can disable anonymous usage tracking here:\n\n```toml\n# Anonymous usage tracking\n[anonymousUsageTracking]\ndisable = false # set to true to disable\n```\n\n\n---\nsidebar_position: 99\n---\n\n# Troubleshooting\n\nThis document aims to assist you in troubleshooting issues with the Tabby extensions \nfor various IDEs such as VSCode, IntelliJ Platform IDEs, and Vim / NeoVim.\n\n## Tabby Initialization Failed?\n\nThis problem may occur when you first install the Tabby in IntelliJ Platform IDEs \nor Vim/NeoVim. \n\nThe Tabby IDE extension runs its core logic in the Tabby agent. In the case of VSCode, \nthe agent runs within the VSCode Extension Host, while for IntelliJ Platform IDEs \nand Vim/NeoVim, the agent runs as a separate Node.js process.\n\n### Install Node.js\n\nYou can follow the instructions on the [Node.js website](https://nodejs.org/en/download/)\nto install Node.js. Alternatively, you can use a version manager such as [nvm](https://github.com/nvm-sh/nvm). \n**Note**: Tabby IDE extension requires Node.js version 18.0.0 or higher.\n\n### Specify Node Binary Path\n\nIf the node binary is already accessible via your `PATH` environment variable, \nyou can skip this step. Otherwise, you will need to specify the path to the node \nbinary in the IDE settings.\n\nFor IntelliJ Platform IDEs (Tabby plugin version 0.6.0 or higher):\n* Click on Tabby plugin status bar item and select `Open Settings...`.\n* Enter the path to the node binary on your system in the `Node binary` field,\n e.g. `/usr/local/bin/node`, `C:\\Program Files\\nodejs\\node.exe`. \n If you are using a version manager such as `nvm`, you can enter the path to the\n node binary installed by the version manager, e.g. `~/.nvm/versions/node/v18.18.0/bin/node`.\n* Restart the IDE\n\nIf you installed Node.js via snap, please use `/snap/node/current/bin/node` rather than \n`/snap/bin/node` as the node binary path.\n\n## Cannot Connect to Tabby Server?\n\nIf you have setup the endpoint for the Tabby server but the status bar item of \nthe Tabby IDE extension still displays a mark indicating \"Disconnected\", \nfollow the steps below to troubleshoot the issue.\n\n### Check Endpoint Settings\n\nVerify that the endpoint setting is correct. You can set the endpoint in the \nIDE settings page (except for Vim/NeoVim, which do not have this option) or by \nediting the [config file](https://tabby.tabbyml.com/docs/extensions/configurations). \nKeep in mind that the IDE settings take priority over the config file. \nIf you wish to use the setting from the config file, ensure that the IDE setting\nis empty.\n\n### Authentication Token\n\nIf you have enabled authentication on your Tabby server, you will need to set\nthe authentication token in the IDE or the [config file](https://tabby.tabbyml.com/docs/extensions/configurations).\n\n### Verify Tabby Server Status\n\nOnce the Tabby server is running, it should display a log message such as \n`Listening at 0.0.0.0:8080`. \nOpen your browser and navigate to `http://localhost:8080/swagger-ui/`\n(Replace `localhost:8080` with the correct IP/domain and port if you have setup your \nTabby server on a remote machine). The browser should display a web page with Swagger UI. \n\nTo test the server, expand the `/v1/completions` section, click on `Try it out`, \nand then click `Execute`. \nIf you receive a response, it indicates that the Tabby server is running properly.\n\nYou can also use `curl` to send a completion request to Tabby server. For example:\n\n```shell\ncurl -X 'POST' \\\n 'http://localhost:8080/v1/completions' \\\n -H 'accept: application/json' \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"language\": \"python\",\n \"segments\": {\n \"prefix\": \"def fib(n):\\n \",\n \"suffix\": \"\\n return fib(n - 1) + fib(n - 2)\"\n }\n}'\n```\n\nIf you can not see Swagger UI page, or can not get response of completion request, \nplease check the server log to see if there is any error.\n\n### Proxy Settings\n\nPlease note that Tabby extensions for IDEs currently do not support proxy settings. \nIf you need to access your Tabby server through a proxy, consider setting up \na reverse proxy and using the reverse proxy URL as the endpoint for the Tabby IDE extension.\n\n## Cannot Get Any Completions?\n\nIf you are able to connect the Tabby extension to the Tabby server but are unable to \nreceive any completions, you can follow the steps below to troubleshoot the issue.\n\n### Check Trigger Mode Settings\n\nTabby is set to automatic trigger mode by default. In this mode, you should receive \ncompletions after a short delay when you stop typing. The delay may vary depending \non your server's performance and settings. \nIf you are using manual trigger mode, you need to press `Alt + \\` (`Ctrl + \\` for IntelliJ and Vim plugins) \nto trigger a completion request. The status bar item of Tabby IDE extension should show a loading \nindicator for a brief period before displaying the completions. \nKeep in mind that Tabby may not provide any suggestions if there is no necessary\nfor the current code context.\n\n### Check Request Timeouts\n\nIf your completion requests are timing out, Tabby may display a warning message. \nThis could be due to network issues or poor server performance, especially when \nrunning a large model on a CPU. To improve performance, consider running the model \non a GPU with CUDA or ROCm support or on Apple M1/M2 with Metal support. When running \nthe server, make sure to specify the device in the arguments using `--device cuda`, `--device rocm` or\n`--device metal`. You can also try using a smaller model from the available [models](https://tabby.tabbyml.com/docs/models/). \n\n## Problems with the Chat Panel?\n\n### Check Server Enabling Chat Feature\n\nThe chat panel is a feature that requires the Tabby server to enable the chat feature. Please make sure that the chat feature is properly configured in the server configuration file or that the server is started with the `--chat-model MODEL_NAME` flag. \nYou can access the server management page at `https://demo.tabbyml.com/system` to check if the chat feature is enabled.\n\n### Check Browser Compatibility in IntelliJ Platform IDEs\n\nThe chat panel in IntelliJ Platform IDEs is implemented using an embedded JCEF browser. The support is included in the default Java Runtime for JetBrains IDEs, but it may not be by default in Android Studio. \nPlease follow the steps below to ensure that the JCEF browser is enabled in your IDE:\n\n1. Open `Search Everywhere` (Double `Shift`) or `Find Action` (`Ctrl + Shift + A`), and search for the action `Choose Boot Java Runtime for the IDE...`\n2. Select a Java Runtime that includes JCEF support \n ![Choose Boot Java Runtime for the IDE](./intellij-choose-boot-runtime-for-the-ide.png)\n3. Restart the IDE\n\n## Want to Deep Dive via Logs?\n\nIf you cannot solve the issue using the previous steps, you may want to \ninvestigate further by checking the logs of Tabby extensions. \nIf you want help from the community, it is also recommended to share the logs \nof the Tabby extensions.\n\n### VSCode\n\nFor VSCode, you can check the `Output` window and select the `Tabby` channel or\n`Tabby Agent` channel.\n![VSCode Output View](./vscode-output-view.png)\nThe default log level is `info`. To enable debug logs, you can use the command \n`Developer: Set Log Level...` to set the log level to `debug`.\n\n### IntelliJ Platform IDEs\n\nFor IntelliJ Platform IDEs, you can check the logs for the IDE using `Help -> Show Log in Explorer|Finder|Files...`, \nor follow [this document](https://intellij-support.jetbrains.com/hc/en-us/articles/207241085-Locating-IDE-log-files) to locate the log file.\nThis log file contains all the logs for the IDE, and you can filter them by searching for the \nkeyword `com.tabbyml.intellijtabby`.\nYou can also enable all level logs by editing `Help -> Diagnostic Tools -> Debug Log Settings...` and\nadd `com.tabbyml.intellijtabby:all` to the list.\n\n### Tabby Agent Logs\n\nAs the agent runs as a separate Node.js process for IntelliJ Platform IDEs plugin \nand Vim/NeoVim plugin, its logs are written separately.\n\nBy default, the agent logs are set to `\"silent\"`, which means the agent logs are \nnot written to disk. To enable Tabby agent debug logs, editing the \n[config file](https://tabby.tabbyml.com/docs/extensions/configurations), \nuncomment the `logs` section and set `level` to `\"debug\"`, then save the file \nto apply the changes.\n \nYou can find the agent logs in the `~/.tabby-client/agent/logs` directory. These \nlogs are written using [pino](https://github.com/pinojs/pino), and you can \nuse `pino-pretty` to format the log file for easier readability.\n\n```shell\ntail -f ~/.tabby-client/agent/logs/20240101.0.log | npx pino-pretty\n```\n\n## Still Have Issues?\n\nIf you still have any issues, please feel free to [open an issue on github](https://github.com/TabbyML/tabby/issues/new), \nor join our [slack community](https://links.tabbyml.com/join-slack)\nfor further support.\n\n\nlabel: 📚 References\nposition: 100\n\n\n# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentodeploy.yaml)\n\n\n# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/bentofile.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/bentofile.yaml)\n\n\n# Bentoml Skill\n\nPath: `website/docs/references/cloud-deployment/bentoml/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/bentoml/index.md)\n\n\n# Hugging-face Skill\n\nPath: `website/docs/references/cloud-deployment/hugging-face/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/hugging-face/index.md)\n\n\n# Modal Skill\n\nPath: `website/docs/references/cloud-deployment/modal/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/modal/index.md)\n\n\n# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/index.md)\n\n\n# Skypilot Skill\n\nPath: `website/docs/references/cloud-deployment/skypilot/tabby.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/cloud-deployment/skypilot/tabby.yaml)\n\n\n# Amazon Bedrock\n\nAmazon Bedrock is a fully managed service on AWS that provides access to foundation models from various AI companies through a single API. With [Amazon Bedrock Access Gateway](https://github.com/aws-samples/bedrock-access-gateway), you can access Anthropic's Claude models through an OpenAI-compatible interface, enabling seamless integration with tools and applications designed for OpenAI's API structure.\n\nFollow the Amazon Bedrock Access Gateway setup guide to deploy your own OpenAI-compatible API endpoint for Claude models.\n\n## Chat model\n\nAmazon Bedrock Access Gateway provides an OpenAI-compatible chat API interface for Claude models. Here we use the `us.anthropic.claude-3-5-sonnet-20241022-v2:0` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"us.anthropic.claude-3-5-sonnet-20241022-v2:0\"\napi_endpoint = \"http://Bedrock-Proxy-xxxxx.{Region}.elb.amazonaws.com/api/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAmazon Bedrock does not provide completion models.\n\n## Embeddings model\n\nWhile Amazon Bedrock supports embeddings models, Tabby does not currently support the embeddings API interface for Amazon models.\n\n\n# Anthropic Claude\n\n[Anthropic](https://www.anthropic.com/) is an AI research company that develops large language models, including the Claude family of models. While Tabby doesn't natively support Claude's API, you can access Claude models through an OpenAI-compatible API interface using [claude2openai](https://github.com/missuo/claude2openai) as a middleware.\n\n## Chat model\n\nAfter deploying the claude2openai middleware, you can access all Claude family models through an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"claude-3-sonnet-20240229\"\n# Middleware endpoint (adjust host and port according to your deployment)\napi_endpoint = \"http://127.0.0.1:6600/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAnthropic currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nAnthropic currently does not provide embedding model APIs.\n\n\n# Azure OpenAI\n\n[Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) is a cloud-based service that provides Azure customers with access to OpenAI's powerful language models including GPT-4, GPT-3.5, and various embedding models.\n\nPlease be aware that azure will be supported starting with version 0.24, which is scheduled for release by end of 01/2025\n\n## Chat model\n\nIt supports various GPT series chat models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"azure/chat\"\nmodel_name = \"gpt-4o-mini\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nAzure OpenAI currently does not offer completion-specific API endpoints.\n\n## Embeddings model\n\nIt supports text-embedding-3-small, text-embedding-3-large and other embedding models through an Azure OpenAI-compatible API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"azure/embedding\"\nmodel_name = \"text-embedding-3-large\"\napi_endpoint = \"https://.openai.azure.com\"\napi_key = \"your-api-key\"\n```\n\n\n# DeepInfra\n\n[DeepInfra](https://deepinfra.com/) is a cloud platform providing efficient and scalable model inference services, offering access to various open-source models like [Llama 3](https://deepinfra.com/meta-llama/Llama-3.3-70B-Instruct), [Mixtral](https://deepinfra.com/mistralai/Mixtral-8x7B-Instruct-v0.1), and [Qwen](https://deepinfra.com/Qwen/Qwen2.5-Coder-32B-Instruct).\n\n## Chat model\n\nDeepInfra provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"meta-llama/Llama-3.3-70B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepInfra provides an OpenAI-compatible completion API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"openai/completion\"\nmodel_name = \"Qwen/Qwen2.5-Coder-32B-Instruct\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepInfra also provides an OpenAI-compatible embeddings API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.embedding.http]\nkind = \"openai/embedding\"\nmodel_name = \"BAAI/bge-base-en-v1.5\"\napi_endpoint = \"https://api.deepinfra.com/v1/openai\"\napi_key = \"your-api-key\"\n```\n\n\n# DeepSeek\n\n[DeepSeek](https://www.deepseek.com/) is an AI company that develops large language models specialized in coding and general tasks. Their models include [DeepSeek V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) for general tasks and [DeepSeek Coder](https://huggingface.co/collections/deepseek-ai/deepseekcoder-v2-666bf4b274a5f556827ceeca) specifically optimized for programming tasks.\n\n## Chat model\n\nDeepSeek provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nDeepSeek offers a specialized completion API interface for code completion tasks.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.completion.http]\nkind = \"deepseek/completion\"\nmodel_name = \"your_model\"\napi_endpoint = \"https://api.deepseek.com/beta\"\napi_key = \"your-api-key\"\n```\n\n## Embeddings model\n\nDeepSeek currently does not provide embedding model APIs.\n\n\n# Fireworks\n\n[Fireworks](https://app.fireworks.ai/) is a cloud platform that offers efficient model inference and deployment services,\nproviding cost-effective access to a variety of AI models through their API service,\nincluding [Llama 2](https://fireworks.ai/models/fireworks/llama-v2-70b-chat),\n[DeepSeek V3](https://fireworks.ai/models/fireworks/deepseek-v3),\n[DeepSeek Coder](https://fireworks.ai/models/fireworks/deepseek-coder-v2-instruct) and other open-source models.\n\n## Chat model\n\nFireworks provides an OpenAI-compatible chat API interface.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"accounts/fireworks/models/deepseek-v3\"\napi_endpoint = \"https://api.fireworks.ai/inference/v1\"\napi_key = \"your-api-key\"\n```\n\n## Completion model\n\nFireworks does not offer completion models (FIM) through their API.\n\n## Embeddings model\n\nWhile Fireworks provides embedding model APIs, Tabby has not yet implemented a compatible client to interface with these APIs. Therefore, embedding functionality is currently not available through Tabby's integration with Fireworks.\n\n\n# Hugging Face Inference Providers\n\n[Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers) offers access to frontier open models from multiple providers through a unified API. \n\nYou'll need a [Hugging Face account](https://huggingface.co/join) and an [access token](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained).\n\n## Chat model\n\nHugging Face Inference Providers provides an OpenAI-compatible chat API interface. Here we use the `MiniMaxAI/MiniMax-M2` model as an example.\n\n```toml title=\"~/.tabby/config.toml\"\n[model.chat.http]\nkind = \"openai/chat\"\nmodel_name = \"MiniMaxAI/MiniMax-M2\" # specify the model you want to use\napi_endpoint = \"https://router.huggingface.co/v1\"\napi_key = \"your-hf-token\"\n```\n\n### Available models\n\nYou can find a complete list of models supported by at least one provider [on the Hub](https://huggingface.co/models?inference_provider=all). You can also access these programmatically, see this [guide](https://huggingface.co/docs/inference-providers/hub-api) for more details.\n\n## Completion model\n\nHugging Face Inference Providers does not offer completion models (FIM) through their OpenAI-compatible API. For code completion, use a local model with Tabby.\n\n## Embeddings model\n\nWhile Hugging Face Inference Providers supports embeddings models, Tabby does not currently support the embeddings API interface for Hugging Face Inference Providers.\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/jan.ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/jan.ai.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/llamafile.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/llamafile.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/lm-studio.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/lm-studio.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/mistral-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/mistral-ai.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/ollama.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/ollama.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openai.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/openrouter.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/openrouter.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/perplexity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/perplexity.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/vllm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/vllm.md)\n\n\n# Models-http-api Skill\n\nPath: `website/docs/references/models-http-api/voyage-ai.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/TabbyML/tabby/HEAD/website/docs/references/models-http-api/voyage-ai.md)\n\n\n---\nsidebar_position: 6\n---\n\n# Programming Languages\n\nMost models nowadays support a large number of programming languages (thanks to [The Stack](https://huggingface.co/datasets/bigcode/the-stack), which has collected 358 programming languages).\nIn Tabby, we need to add configuration for each language to maximize performance and completion quality.\n\nCurrently, there are two aspects of support that need to be added for each language.\n\n**Stop Words**\n\nStop words determine when the language model can early stop its decoding steps, resulting in better latency and affecting the quality of completion. We suggest adding all top-level keywords as part of the stop words.\n\n**Repository Context**\n\nWe parse languages into chunks and compute a token-based index for serving time Retrieval Augmented Code Completion. In Tabby, we define these repository contexts as [treesitter queries](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax), and the query results will be indexed.\n\nFor an actual example of an issue or pull request adding the above support, please check out https://github.com/TabbyML/tabby/issues/553 as a reference.\n\n## Supported Languages\n\n* [Rust](https://www.rust-lang.org/)\n* [Python](https://www.python.org/)\n* [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)\n* [TypeScript](https://www.typescriptlang.org/)\n* [Golang](https://go.dev/)\n* [Ruby](https://www.ruby-lang.org/)\n* [Java](https://www.java.com/)\n* [Kotlin](https://www.kotlinlang.org/)\n* [C/C++](https://cplusplus.com/)\n* [PHP](https://www.php.net/)\n* [C#](https://learn.microsoft.com/en-us/dotnet/csharp/)\n* [Solidity](https://soliditylang.org/)\n* [R](https://www.r-project.org/)\n* [Dart](https://dart.dev/)\n* [Lua](https://www.lua.org)\n* [Elixir](https://elixir-lang.org)\n* [OCaml](https://ocaml.org/)\n* [GDScript](https://gdscript.com/)\n* [Scala](https://www.scala-lang.org/)\n\n## Languages Missing Certain Support\n\n| Language | Stop Words (time to contribute: ~5 min) | Repository Context (time to contribute: ~1 hr) |\n| :------: | :-------------------------------------: | :--------------------------------------------: |\n| CSS | 🚫 | 🚫 |\n| Haskell | 🚫 | 🚫 |\n| Julia | 🚫 | 🚫 |\n| Perl | 🚫 | 🚫 |\n"}