LlamaFactory Docker & Self-Hosted Deployment

GitHub

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

74,006 stars Python Python 1 file · ~22,913 tokens #agent#ai#deepseek#fine-tuning#gemma#gpt#instruction-tuning#large-language-models
RAW Compose
1. Pull Image (DOCKERHUB) docker pull hiyouga/llamafactory
2. Docker Run (1-Click Launch) docker run -d --name llamafactory -p 7860:7860 --restart always hiyouga/llamafactory:latest
3. Podman Run podman run -d --name llamafactory -p 7860:7860 --restart always hiyouga/llamafactory:latest
4. Skopeo Inspect (Zero-Pull Manifest) skopeo inspect docker://docker.io/hiyouga/llamafactory:latest
5. ORAS Pull (OCI Artifact) oras pull hiyouga/llamafactory:latest
6. Compose Deployment docker compose up -d
docker/docker-rocm/docker-compose.yml
Download .yml 
services:
  llamafactory:
    build:
      dockerfile: ./docker/docker-rocm/Dockerfile
      context: ../..
      args:
        PIP_INDEX: https://pypi.org/simple
    container_name: llamafactory
    ports:
      - "7860:7860"
      - "8000:8000"
    ipc: host
    tty: true
    # shm_size: "16gb"  # ipc: host is set
    stdin_open: true
    command: bash
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    restart: unless-stopped