shimmy Docker & Self-Hosted Deployment

GitHub

Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

5,721 stars Rust Rust 1 file · ~20,439 tokens #api-server#command-line-tool#developer-tools#gguf#huggingface#huggingface-models#huggingface-transformers#inference-server
RAW Compose
1. Pull Image (DOCKERHUB) docker pull library/nginx
2. Docker Run (1-Click Launch) docker run -d --name shimmy -p 11434:11434 --restart always nginx:alpine
3. Podman Run podman run -d --name shimmy -p 11434:11434 --restart always nginx:alpine
4. Skopeo Inspect (Zero-Pull Manifest) skopeo inspect docker://docker.io/library/nginx:alpine
5. ORAS Pull (OCI Artifact) oras pull nginx:alpine
6. Compose Deployment docker compose up -d
deploy/docker-compose.yml
Download .yml 
# Docker Compose configuration for local development and cloud deployment
version: '3.8'

services:
  shimmy:
    build:
      context: ..
      dockerfile: deploy/Dockerfile
    ports:
      - "11434:11434"
    environment:
      - RUST_LOG=info
    volumes:
      # Mount models directory if you have local models
      - ./models:/app/models:ro
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:11434/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 10s

  # Optional: Add Nginx reverse proxy for production
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      # Add SSL certificates if needed
      # - ./ssl:/etc/nginx/ssl:ro
    depends_on:
      - shimmy
    restart: unless-stopped
    profiles:
      - production