helicone Documentation

Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

5,630 stars TypeScript TypeScript 1 file · ~3,370 tokens #agent-monitoring#analytics#evaluation#gpt#langchain#large-language-models#llama-index#llm
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy helicone on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run) 
docker run -d --name helicone -p 54388:54388 --restart always postgres:17.4
Configured Services & Ports
⚙️ db
image: postgres:17.4
🔌 Ports: 54388:5432
⚙️ clickhouse
image: clickhouse/clickhouse-server:24.3.13.40
🔌 Ports: 18123:8123, 19000:9000
⚙️ minio
image: minio/minio
🔌 Ports: 9000:9000, 9001:9001
⚙️ minio-setup
image: minio/mc:RELEASE.2025-04-08T15-39-49Z
⚙️ mailhog
image: mailhog/mailhog
🔌 Ports: 1025:1025, 8025:8025
⚙️ redis
image: redis:8.0.2-alpine
🔌 Ports: 6379:6379
⚙️ migrations
image: build locally
⚙️ jawn
image: build locally
🔌 Ports: ${JAWN_PORT:-8585}:${JAWN_PORT:-8585}
⚙️ web
image: build locally
🔌 Ports: 3000:3000
⚙️ jawn-dev
image: build locally
🔌 Ports: ${JAWN_PORT:-8585}:${JAWN_PORT:-8585}
⚙️ web-dev
image: build locally
🔌 Ports: 3000:3000
⚙️ worker-openai-proxy
image: worker-local
🔌 Ports: 8787:8787
⚙️ worker-helicone-api
image: worker-local
🔌 Ports: 8788:8788
⚙️ worker-gateway-api
image: worker-local
🔌 Ports: 8789:8789
⚙️ worker-anthropic-proxy
image: worker-local
🔌 Ports: 8790:8790
⚙️ worker-generate-api
image: worker-local
🔌 Ports: 8791:8791
⚙️ zookeeper
image: confluentinc/cp-zookeeper:7.8.0
🔌 Ports: 2181:2181
⚙️ kafka
image: confluentinc/cp-kafka:7.8.0
🔌 Ports: 9092:9092, 29092:29092, 9999:9999
docker-compose.yml
yaml (docker-compose.yml) 
name: helicone-self-host

services:
  # =============================================================================
  # INFRASTRUCTURE SERVICES (Default profile - always runs)
  # =============================================================================

  db:
    image: postgres:17.4
    container_name: helicone-postgres-flyway-test
    environment:
      POSTGRES_DB: helicone_test
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: testpassword
    ports:
      - "54388:5432"
    volumes:
      - postgres_flyway_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

  clickhouse:
    image: clickhouse/clickhouse-server:24.3.13.40
    container_name: helicone-clickhouse
    ports:
      - "18123:8123" # HTTP interface
      - "19000:9000" # Native interface
    volumes:
      - clickhouse_data:/var/lib/clickhouse
    ulimits:
      nofile:
        soft: 262144
        hard: 262144
    environment:
      CLICKHOUSE_DEFAULT_USER: default
    restart: unless-stopped
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "wget --no-verbose --tries=1 --spider http://localhost:8123/ping || exit 1",
        ]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  minio:
    image: minio/minio
    container_name: helicone-minio
    ports:
      - "9000:9000" # MinIO API port
      - "9001:9001" # MinIO Console port
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    volumes:
      - minio_data:/data
    command: server /data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 5s

  minio-setup:
    image: minio/mc:RELEASE.2025-04-08T15-39-49Z
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc alias set localminio http://minio:9000 minioadmin minioadmin;
      /usr/bin/mc mb --ignore-existing localminio/request-response-storage;
      /usr/bin/mc mb --ignore-existing localminio/prompt-body-storage;
      /usr/bin/mc mb --ignore-existing localminio/hql-store;
      exit 0;
      "

  mailhog:
    image: mailhog/mailhog
    container_name: mailhog
    ports:
      - "1025:1025" # SMTP port
      - "8025:8025" # Web UI port

  redis:
    image: redis:8.0.2-alpine
    restart: unless-stopped
    ports:
      - "6379:6379"
    environment:
      - REDIS_PASSWORD=helicone-redis-pw
      - REDIS_PORT=6379
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 5s

  migrations:
    build:
      context: ../
      dockerfile: docker/dockerfiles/dockerfile_migrations
    depends_on:
      db:
        condition: service_healthy
      clickhouse:
        condition: service_healthy
    environment:
      FLYWAY_URL: jdbc:postgresql://db:5432/helicone_test
      FLYWAY_USER: postgres
      FLYWAY_PASSWORD: testpassword
      CLICKHOUSE_HOST: http://clickhouse:8123
      CLICKHOUSE_USER: default
    volumes:
      - ../supabase:/app/supabase
      - ../clickhouse:/app/clickhouse

  # =============================================================================
  # HELICONE CORE SERVICES (include-helicone profile)
  # =============================================================================

  jawn:
    container_name: helicone-jawn
    build:
      context: ../
      dockerfile: valhalla/dockerfile
    restart: unless-stopped
    ports:
      - ${JAWN_PORT:-8585}:${JAWN_PORT:-8585}
    profiles: ["include-helicone"]
    depends_on:
      db:
        condition: service_healthy
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      migrations:
        condition: service_completed_successfully
    environment:
      PORT: ${JAWN_PORT:-8585}
      NEXT_PUBLIC_BETTER_AUTH: "true"
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin
      S3_BUCKET_NAME: "request-response-storage"
      S3_PROMPT_BUCKET_NAME: "prompt-body-storage"
      S3_ENDPOINT_PUBLIC: http://localhost:9000
      CLICKHOUSE_CREDS: |
        {
          "CLICKHOUSE_HOST": "http://clickhouse:8123",
          "CLICKHOUSE_USER": "default",
          "CLICKHOUSE_PASSWORD": "",
          "CLICKHOUSE_HQL_USER": "hql_user",
          "CLICKHOUSE_HQL_PASSWORD": ""
        }
      VERCEL_ENV: ${VERCEL_ENV:-development}
      BETTER_AUTH_SECRET: ${BETTER_AUTH_SECRET:-your-secret-key}
      SUPABASE_DATABASE_URL: postgres://postgres:testpassword@db:5432/helicone_test

  web:
    container_name: helicone-web
    build:
      context: ../
      dockerfile: docker/dockerfiles/dockerfile_web
      args:
        NEXT_PUBLIC_HELICONE_JAWN_SERVICE: ${JAWN_PUBLIC_URL:-http://localhost:8585}
        NEXT_PUBLIC_BETTER_AUTH: "true"
        NODE_ENV: development
    ports:
      - "3000:3000"
    profiles: ["include-helicone"]
    depends_on:
      jawn:
        condition: service_started
      db:
        condition: service_healthy
    environment:
      NEXT_PUBLIC_HELICONE_JAWN_SERVICE: ${JAWN_PUBLIC_URL:-http://localhost:8585}
      DATABASE_URL: postgres://postgres:testpassword@db:5432/helicone_test
      CLICKHOUSE_HOST: http://clickhouse:8123
      CLICKHOUSE_USER: default
      CLICKHOUSE_PASSWORD: ""
      VERCEL_ENV: ${VERCEL_ENV:-development}
      SMTP_HOST: mailhog
      SMTP_PORT: 1025
      NEXT_PUBLIC_BETTER_AUTH: "true"
      BETTER_AUTH_SECRET: ${BETTER_AUTH_SECRET:-your-secret-key}
      SMTP_SECURE: false
      NODE_ENV: development

  # =============================================================================
  # DEVELOPMENT SERVICES (dev profile)
  # =============================================================================

  jawn-dev:
    container_name: helicone-jawn-dev
    build:
      context: ../
      dockerfile: valhalla/dockerfile_dev
    restart: unless-stopped
    ports:
      - ${JAWN_PORT:-8585}:${JAWN_PORT:-8585}
    profiles: ["dev"]
    depends_on:
      db:
        condition: service_healthy
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      migrations:
        condition: service_completed_successfully
    environment:
      PORT: ${JAWN_PORT:-8585}
      NEXT_PUBLIC_BETTER_AUTH: "true"
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin
      S3_BUCKET_NAME: "request-response-storage"
      S3_PROMPT_BUCKET_NAME: "prompt-body-storage"
      S3_ENDPOINT_PUBLIC: http://localhost:9000
      CLICKHOUSE_CREDS: |
        {
          "CLICKHOUSE_HOST": "http://clickhouse:8123",
          "CLICKHOUSE_USER": "default",
          "CLICKHOUSE_PASSWORD": "",
          "CLICKHOUSE_HQL_USER": "hql_user",
          "CLICKHOUSE_HQL_PASSWORD": ""
        }
      VERCEL_ENV: ${VERCEL_ENV:-development}
      BETTER_AUTH_SECRET: ${BETTER_AUTH_SECRET:-your-secret-key}
      SUPABASE_DATABASE_URL: postgres://postgres:testpassword@db:5432/helicone_test
    volumes:
      - ../valhalla:/app/valhalla
      - ../shared:/app/shared
      - /app/valhalla/node_modules
    working_dir: /app/valhalla/jawn
    command: yarn dev

  web-dev:
    container_name: helicone-web-dev
    build:
      context: ../web
      dockerfile: docker/dockerfiles/dockerfile_web_dev
    ports:
      - "3000:3000"
    profiles: ["dev"]
    depends_on:
      jawn-dev:
        condition: service_started
      db:
        condition: service_healthy
    volumes:
      - ../web:/app/web
      - /app/web/node_modules
      - /app/web/.next
    environment:
      NEXT_PUBLIC_HELICONE_JAWN_SERVICE: ${JAWN_PUBLIC_URL:-http://localhost:8585}
      DATABASE_URL: postgres://postgres:testpassword@db:5432/helicone_test
      CLICKHOUSE_HOST: http://clickhouse:8123
      CLICKHOUSE_USER: default
      CLICKHOUSE_PASSWORD: ""
      VERCEL_ENV: ${VERCEL_ENV:-development}
      SMTP_HOST: mailhog
      SMTP_PORT: 1025
      NEXT_PUBLIC_BETTER_AUTH: "true"
      BETTER_AUTH_SECRET: ${BETTER_AUTH_SECRET:-your-secret-key}
      SMTP_SECURE: false
      NODE_ENV: development
    command: yarn dev:local -p 3000

  # =============================================================================
  # WORKER SERVICES (workers profile)
  # =============================================================================

  worker-openai-proxy:
    image: worker-local
    build:
      context: ../worker
      dockerfile: docker/dockerfiles/dockerfile_worker
    container_name: helicone-worker-openai-proxy
    ports:
      - "8787:8787"
    profiles: ["workers"]
    depends_on:
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      db:
        condition: service_healthy
    environment:
      PORT: 8787
      WORKER_TYPE: OPENAI_PROXY
      WORKER_PORT: 8787
      CLICKHOUSE_HOST: http://clickhouse:8123
      VALHALLA_URL: http://jawn:${JAWN_PORT:-8585}
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin
      S3_BUCKET_NAME: request-response-storage
      S3_PROMPT_BUCKET_NAME: prompt-body-storage

  worker-helicone-api:
    image: worker-local
    build:
      context: ../worker
      dockerfile: docker/dockerfiles/dockerfile_worker
    container_name: helicone-worker-api
    ports:
      - "8788:8788"
    profiles: ["workers"]
    depends_on:
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      db:
        condition: service_healthy
    environment:
      PORT: 8788
      WORKER_TYPE: HELICONE_API
      WORKER_PORT: 8788
      CLICKHOUSE_HOST: http://clickhouse:8123
      VALHALLA_URL: http://jawn:${JAWN_PORT:-8585}
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin

  worker-gateway-api:
    image: worker-local
    build:
      context: ../worker
      dockerfile: docker/dockerfiles/dockerfile_worker
    container_name: helicone-worker-gateway
    ports:
      - "8789:8789"
    profiles: ["workers"]
    depends_on:
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      db:
        condition: service_healthy
    environment:
      PORT: 8789
      WORKER_TYPE: GATEWAY_API
      WORKER_PORT: 8789
      GATEWAY_TARGET: https://openrouter.ai
      CLICKHOUSE_HOST: http://clickhouse:8123
      VALHALLA_URL: http://jawn:${JAWN_PORT:-8585}
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin

  worker-anthropic-proxy:
    image: worker-local
    build:
      context: ../worker
      dockerfile: docker/dockerfiles/dockerfile_worker
    container_name: helicone-worker-anthropic
    ports:
      - "8790:8790"
    profiles: ["workers"]
    depends_on:
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      db:
        condition: service_healthy
    environment:
      PORT: 8790
      WORKER_TYPE: ANTHROPIC_PROXY
      WORKER_PORT: 8790
      CLICKHOUSE_HOST: http://clickhouse:8123
      VALHALLA_URL: http://jawn:${JAWN_PORT:-8585}
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin

  worker-generate-api:
    image: worker-local
    build:
      context: ../worker
      dockerfile: docker/dockerfiles/dockerfile_worker
    container_name: helicone-worker-generate
    ports:
      - "8791:8791"
    profiles: ["workers"]
    depends_on:
      clickhouse:
        condition: service_healthy
      minio:
        condition: service_healthy
      db:
        condition: service_healthy
    environment:
      PORT: 8791
      WORKER_TYPE: GENERATE_API
      WORKER_PORT: 8791
      CLICKHOUSE_HOST: http://clickhouse:8123
      VALHALLA_URL: http://jawn:${JAWN_PORT:-8585}
      S3_ENDPOINT: http://minio:9000
      S3_ACCESS_KEY: minioadmin
      S3_SECRET_KEY: minioadmin

  # =============================================================================
  # KAFKA SERVICES (kafka profile - experimental)
  # =============================================================================

  zookeeper:
    image: confluentinc/cp-zookeeper:7.8.0
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    profiles: ["kafka"]
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_SERVERS: zookeeper:2888:3888

  kafka:
    image: confluentinc/cp-kafka:7.8.0
    hostname: kafka
    container_name: kafka
    ports:
      - "9092:9092"
      - "29092:29092"
      - "9999:9999"
    profiles: ["kafka"]
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:19092,EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092,DOCKER://host.docker.internal:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,DOCKER:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_BROKER_ID: 1
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_JMX_PORT: 9999
      KAFKA_JMX_HOSTNAME: ${DOCKER_HOST_IP:-127.0.0.1}
      KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
      KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"

volumes:
  postgres_flyway_data:
    driver: local
  minio_data:
    driver: local
  clickhouse_data:
    driver: local

networks:
  traces:
    driver: bridge

Helicone is an AI Gateway & LLM Observability Platform for AI Engineers

  • 🌐 AI Gateway: Access 100+ AI models with 1 API key through the OpenAI API with intelligent routing and automatic fallbacks. Get started in 2 minutes.
  • 🔌 Quick integration: One-line of code to log all your requests from OpenAI, Anthropic, LangChain, Gemini, Vercel AI SDK, and more.
  • 📊 Observe: Inspect and debug traces & sessions for agents, chatbots, document processing pipelines, and more
  • 📈 Analyze: Track metrics like cost, latency, quality, and more. Export to PostHog in one-line for custom dashboards
  • 🎮 Playground: Rapidly test and iterate on prompts, sessions and traces in our UI.
  • 🧠 Prompt Management: Version prompts using production data. Deploy prompts through the AI Gateway without code changes. Your prompts remain under your control, always accessible.
  • 🎛️ Fine-tune: Fine-tune with one of our fine-tuning partners: OpenPipe or Autonomi (more coming soon)
  • 🛡️ Enterprise Ready: SOC 2 and GDPR compliant

🎁 Generous monthly free tier (10k requests/month) - No credit card required!

Quick Start ⚡️

  1. Get your API key by signing up here and add credits at helicone.ai/credits

  2. Update the baseURL in your code and add your API key.

    typescript
    import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai",
  apiKey: process.env.HELICONE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",  // claude-sonnet-4, gemini-2.0-flash or any model from https://www.helicone.ai/models
  messages: [{ role: "user", content: "Hello!" }]
});

  3. 🎉 You're all set! View your logs at Helicone and access 100+ models through one API.

Self-Hosting Open Source LLM Observability

Docker

Helicone is simple to self-host and update. To get started locally, just use our docker-compose file.

bash
# Clone the repository
git clone https://github.com/Helicone/helicone.git
cd docker
cp .env.example .env

# Start the services
./helicone-compose.sh helicone up

Helm

For Enterprise workloads, we also have a production-ready Helm chart available. To access, contact us at [email protected].

Manual deployment is not recommended. Please use Docker or Helm. If you must, follow the instructions here.

Architecture

Helicone is comprised of five services:

  • Web: Frontend Platform (NextJS)
  • Worker: Proxy Logging (Cloudflare Workers)
  • Jawn: Dedicated Server for serving collecting logs (Express + Tsoa)
  • Supabase: Application Database and Auth
  • ClickHouse: Analytics Database
  • Minio: Object Storage for logs.

Integrations 🔌

Inference Providers

Integration Supports Description
AI Gateway JS/TS, Python, cURL Unified API for 100+ providers with intelligent routing, automatic fallbacks, and unified observability
Async Logging (OpenLLMetry) JS/TS, Python Asynchronous logging for multiple LLM platforms
OpenAI JS/TS, Python Inference provider
Azure OpenAI JS/TS, Python Inference provider
Anthropic JS/TS, Python Inference provider
Ollama JS/TS Run and use large language models locally
AWS Bedrock JS/TS Inference provider
Gemini API JS/TS Inference provider
Gemini Vertex AI JS/TS Gemini models on Google Cloud's Vertex AI
Vercel AI JS/TS AI SDK for building AI-powered applications
Anyscale JS/TS, Python Inference provider
TogetherAI JS/TS, Python Inference provider
Hyperbolic JS/TS, Python Inference provider
Groq JS/TS, Python High-performance models
DeepInfra JS/TS, Python Serverless AI inference for various models
Fireworks AI JS/TS, Python Fast inference API for open-source LLMs

Frameworks

Framework Supports Description
LangChain JS/TS, Python Use AI Gateway with LangChain for unified provider access
LlamaIndex Python Framework for building LLM-powered data applications
LangGraph Python Build stateful, multi-actor applications with LLMs
Vercel AI SDK JS/TS AI SDK for building AI-powered applications
Semantic Kernel C#, Python Microsoft's AI orchestration framework
CrewAI Python Framework for orchestrating role-playing AI agents
ModelFusion JS/TS Abstraction layer for integrating AI models into JavaScript and TypeScript applications
PostHog JS/TS, Python, cURL Product analytics platform. Build custom dashboards.
RAGAS Python Evaluation framework for retrieval-augmented generation
Open WebUI JS/TS Web interface for interacting with local LLMs
MetaGPT YAML Multi-agent framework
Open Devin Docker AI software engineer
Mem0 EmbedChain Python Framework for building RAG applications
Dify No code required LLMOps platform for AI-native application development

This list may be out of date. Don't see your provider or framework? Check out the latest integrations in our docs. If not found there, request a new integration by contacting [email protected].

Contributing

We ❤️ our contributors! We warmly welcome contributions for documentation, integrations, costs, and feature requests.

If you have an idea for how Helicone can be better, create a GitHub issue.

License

Helicone is licensed under the Apache v2.0 License.

Additional Resources

For more information, visit our documentation.

Contributors

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#agent-monitoring Browse agent-monitoring catalog → #analytics Browse analytics catalog → #evaluation Browse evaluation catalog → #gpt Browse gpt catalog → #langchain Browse langchain catalog → #large-language-models Browse large-language-models catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →