BentoML Docker & Self-Hosted Deployment

GitHub

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

8,634 stars Python Python 1 file · ~26,512 tokens #ai-inference#deep-learning#generative-ai#inference-platform#llm#llm-inference#llm-serving#llmops
RAW Compose
1. Pull Image (DOCKERHUB) docker pull jaegertracing/all-in-one
2. Docker Run (1-Click Launch) docker run -d --name bentoml -p 6831:6831 --restart always jaegertracing/all-in-one:1.38
3. Podman Run podman run -d --name bentoml -p 6831:6831 --restart always jaegertracing/all-in-one:1.38
4. Skopeo Inspect (Zero-Pull Manifest) skopeo inspect docker://docker.io/jaegertracing/all-in-one:1.38
5. ORAS Pull (OCI Artifact) oras pull jaegertracing/all-in-one:1.38
6. Compose Deployment docker compose up -d
docs/source/build-with-bentoml/snippets/tracing/docker-compose.yml
Download .yml 
version: "3.7"
services:
  jaeger:
    image: jaegertracing/all-in-one:1.38
    ports:
      - "6831:6831/udp"
      - "16686:16686"
      - "14268:14268"
      - "5778:5778"
      - "4317:4317"
      - "4318:4318"
    networks:
      - jaeger-network
    environment:
      - COLLECTOR_OTLP_ENABLED=true
  iris_classifier:
    image: iris_classifier:klncyjcfqwldtgxi
    volumes:
      - ./bentoml_configuration.yaml:/home/bentoml/bentoml_configuration.yaml
    ports:
      - "3000:3000"
      - "3001:3001"
    command: ["serve-grpc"]
    environment:
      - OTEL_EXPORTER_JAEGER_AGENT_HOST=jaeger
      - OTEL_EXPORTER_JAEGER_AGENT_PORT=6831
      - BENTOML_CONFIG=/home/bentoml/bentoml_configuration.yaml
    networks:
      - jaeger-network
    depends_on:
      - jaeger

networks:
  jaeger-network: