TensorRT-LLM Model Context Protocol (MCP) Server

GitHub

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform...

14,346 stars

Python 1 file · ~6,758 tokens #blackwell #cuda #llm-serving #moe #pytorch

TensorRT-LLM MCP Server

MCP server for NVIDIA/TensorRT-LLM

Ready-to-Use Client Configurations

{
  "mcpServers": {
    "TensorRT-LLM": {
      "command": "npx",
      "args": [
        "-y",
        "@NVIDIA/TensorRT-LLM"
      ]
    }
  }
}

{
  "mcpServers": {
    "TensorRT-LLM": {
      "command": "npx",
      "args": [
        "-y",
        "@NVIDIA/TensorRT-LLM"
      ]
    }
  }
}

{
  "mcpServers": {
    "TensorRT-LLM": {
      "command": "npx",
      "args": [
        "-y",
        "@NVIDIA/TensorRT-LLM"
      ]
    }
  }
}

{
  "mcpServers": {
    "TensorRT-LLM": {
      "command": "npx",
      "args": [
        "-y",
        "@NVIDIA/TensorRT-LLM"
      ],
      "disabled": false,
      "autoApprove": []
    }
  }
}

{
  "mcpServers": {
    "TensorRT-LLM": {
      "command": "npx",
      "args": [
        "-y",
        "@NVIDIA/TensorRT-LLM"
      ]
    }
  }
}

docker run -i --rm mcp/TensorRT-LLM