{"owner":"AlexsJones","repo":"llmfit","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"files":{"AGENTS.md":"# AGENTS.md\n\nInstructions for AI agents contributing to this codebase.\n\n---\n\n## Project overview\n\n`llmfit` is a Rust CLI/TUI tool that matches LLM models against local system hardware (RAM, CPU, GPU). It detects system specs, loads a model database from embedded JSON, scores each model's fit, and presents results in an interactive terminal UI or classic table output.\n\n## Language and toolchain\n\n- Rust, edition 2024.\n- Build with `cargo build`. Run with `cargo run`.\n- No nightly features required. Stable toolchain only.\n- Minimum supported Rust version: whatever edition 2024 requires (1.85+).\n\n## Architecture\n\n```\nllmfit-core/      Shared Rust library. It owns hardware detection, model data,\n                 fit analysis, planning, providers, benchmarks, quality checks,\n                 model updates, diagnostics, claims, and result sharing.\n\nllmfit-tui/       Main `llmfit` binary. It provides the CLI, ratatui TUI,\n                 Axum HTTP API, embedded Web dashboard, and stdio MCP server.\n                 `main.rs` parses all clap flags and selects an interface.\n\nllmfit-desktop/   Tauri desktop application. Tauri commands call llmfit-core\n                 for hardware detection, fit analysis, and Ollama downloads.\n\nllmfit-web/       React 18 and Vite dashboard. It calls `/api/v1/*` endpoints\n                 from llmfit-tui. The llmfit-tui build script embeds `dist/`.\n                 This directory is not a Cargo workspace member.\n\nllmfit-python/    Python package wrapper. Its wheel includes the compiled Rust\n                 binary. `python -m llmfit` forwards arguments to that binary.\n                 It does not expose llmfit-core through a native Python API.\n```\n\nThe Cargo workspace contains `llmfit-core`, `llmfit-tui`, and\n`llmfit-desktop`. The default members are `llmfit-core` and `llmfit-tui`.\n\nSource modules in `llmfit-core/src/`:\n\n- `analysis.rs`: Builds model-fit results. It marks installed models and applies\n  local, community, and measured benchmark calibration.\n- `bench.rs`: Runs throughput benchmarks against Ollama and OpenAI-compatible\n  endpoints. It also discovers available benchmark targets.\n- `benchmarks.rs`: Loads embedded and remote benchmark data. It builds measured\n  throughput indexes and hardware leaderboard queries.\n- `claim.rs`: Calculates model resource bounds. It renders Kubernetes DRA\n  `ResourceClaim` and `ResourceClaimTemplate` manifests.\n- `doctor.rs`: Collects installation, hardware, runtime, and model diagnostics.\n- `fit.rs`: Calculates fit level, run mode, runtime, quantization, score, and\n  estimated throughput.\n- `hardware.rs`: Detects RAM, CPU, GPUs, unified memory, clusters, and memory\n  bandwidth.\n- `models.rs`: Defines model metadata. It loads embedded HF and ONNX catalogs,\n  custom models, and the update cache.\n- `plan.rs`: Estimates memory, throughput, run paths, and hardware upgrade needs\n  for a requested model setup.\n- `providers.rs`: Integrates Ollama, MLX, llama.cpp, Docker Model Runner,\n  LM Studio, vLLM, and RamaLama.\n- `quality.rs`: Runs response quality tests. It scores roles, builds routing\n  recommendations, and compares results with baselines.\n- `share.rs`: Stores local benchmark results. It handles GitHub authentication\n  and submits community benchmark data.\n- `task_bench.rs`: Provides task benchmark scores for model and task pairs.\n- `update.rs`: Fetches model metadata and manages the local model update cache.\n\nSource modules in `llmfit-tui/src/`:\n\n- `main.rs`: Owns CLI parsing, hardware overrides, command execution, and\n  interface dispatch.\n- `display.rs`: Renders classic CLI tables, model plans, JSON, and CSV output.\n- `download_history.rs`: Stores persistent model download records.\n- `events.rs`: Publishes optional NATS system events and periodic snapshots.\n- `filter_config.rs`: Loads and saves persistent TUI filter settings.\n- `mcp_server.rs`: Exposes hardware, model, runtime, and planning MCP tools.\n- `serve_api.rs`: Serves the embedded Web dashboard and JSON API with Axum.\n- `serve_shared.rs`: Converts shared core types into API and MCP JSON values.\n- `theme.rs`: Defines TUI color themes and stores the selected theme.\n- `tui_app.rs`: Owns TUI state, model results, filters, downloads, and selection.\n- `tui_events.rs`: Handles crossterm input and mutates TUI state.\n- `tui_ui.rs`: Renders TUI views, tables, details, plans, and popups with\n  ratatui.\n\n## Data flow\n\nAll interfaces use the same core analysis flow:\n\n1. `SystemSpecs::detect()` detects CPU, RAM, GPU, unified-memory, and cluster\n   information. CLI hardware overrides can replace detected values.\n2. `ModelDatabase::new()` loads the embedded HF and ONNX catalogs.\n3. Custom models replace matching embedded models. The update cache appends\n   models that are not already present.\n4. `build_model_fits()` removes backend-incompatible models. It calls\n   `ModelFit::analyze_with_forced_runtime()` for each remaining model.\n5. Fit analysis selects a runtime, quantization, and run mode. It calculates\n   memory use, throughput, fit level, score components, and notes.\n6. Local benchmark results, community results, and measured presets can replace\n   or calibrate formula-based throughput estimates.\n7. Each interface applies its own filters, sorting, limits, and presentation.\n\n`ModelFit::analyze()` is the default analysis wrapper. Use\n`analyze_with_context_limit()` for a context cap. Use\n`analyze_with_forced_runtime()` for runtime selection. Use\n`analyze_with_config()` for custom calculation parameters. These methods share\nthe private `analyze_inner()` implementation.\n\nInterface-specific flow:\n\n- CLI: `main.rs` dispatches a subcommand. The command calls llmfit-core and\n  writes a table, JSON, or CSV result.\n- TUI: `App` owns model and filter state. `tui_events` changes that state.\n  `apply_filters()` updates visible indices. `tui_ui` renders the current state.\n- Web: React calls `/api/v1/*`. Axum handlers in `serve_api.rs` call llmfit-core\n  and return JSON. The same server returns the embedded React assets.\n- MCP: `LlmfitMcpServer` receives stdio tool calls. Each tool calls shared core\n  analysis or planning logic and returns JSON text.\n- Desktop: Tauri commands call llmfit-core and serialize results for the desktop\n  UI. Ollama pull state stays in the Tauri application state.\n- Python: The Python entry point locates the installed `llmfit` binary. It then\n  replaces the process on Unix or starts a subprocess on Windows.\n\n## Model database\n\n- Source: `llmfit-core/data/hf_models.json` (33 models).\n- Generated by `scripts/scrape_hf_models.py` (Python, stdlib only, no pip deps).\n- Embedded at compile time via `include_str!(\"../data/hf_models.json\")`.\n- Schema per entry: name, provider, parameter_count, min_ram_gb, recommended_ram_gb, min_vram_gb, quantization, context_length, use_case.\n- `min_vram_gb` is VRAM needed for GPU inference. `min_ram_gb` is system RAM needed for CPU inference. Both are derived from the same parameter count.\n- RAM formula: `params * 0.5 bytes (Q4_K_M) / 1024^3 * 1.2 overhead`.\n- VRAM formula: `params * 0.5 bytes (Q4_K_M) / 1024^3 * 1.1 activation overhead`.\n- Recommended RAM: `model_size * 2.0`.\n\nDo not manually edit `hf_models.json`. Regenerate it by running the scraper:\n\n```sh\npython3 scripts/scrape_hf_models.py\n```\n\nThe scraper has hardcoded fallback entries for gated models that require authentication.\n\n## Conventions\n\n- No `unsafe` code.\n- No `.unwrap()` on user-facing paths. Use proper error handling or `expect()` with a descriptive message for internal invariants only.\n- Fit levels are ordered: Perfect > Good > Marginal > TooTight. Do not add levels without updating `rank_models_by_fit()` sort logic.\n- Fit is VRAM-first. `RunMode` has five execution paths: `Gpu`, `MoeOffload`,\n  `CpuOffload`, `CpuOnly`, and `TensorParallel`.\n- `Gpu` keeps the model in VRAM. `MoeOffload` keeps active experts in VRAM and\n  inactive experts in RAM. `CpuOffload` splits work between VRAM and RAM.\n  `CpuOnly` uses system RAM. `TensorParallel` distributes work across nodes.\n- `min_vram_gb` is the VRAM needed to load model weights on GPU. `min_ram_gb` is the system RAM needed for CPU-only inference (same weights, loaded into RAM instead). They represent the same workload on different hardware paths.\n- On Apple Silicon (unified memory), VRAM = system RAM. The `CpuOffload` path is skipped because there is no separate RAM pool to spill to. `SystemSpecs::unified_memory` tracks this.\n- TUI rendering is stateless. `tui_ui::draw()` must not mutate `App`. Pass `&mut App` only for `TableState` widget requirements -- do not use it to change application state.\n- Event handling in `tui_events.rs` is the sole place that mutates `App` in the TUI loop.\n- Keep `display.rs` and `tui_*.rs` independent. The CLI path must work without initializing any TUI state.\n\n## Adding a new model to the database\n\n1. Add the model's HuggingFace repo ID to `TARGET_MODELS` in `scripts/scrape_hf_models.py`.\n2. If the model is gated (requires HF auth), add a fallback entry to the `FALLBACK` dict in the same script.\n3. Run `python3 scripts/scrape_hf_models.py`.\n4. Verify the output in `llmfit-core/data/hf_models.json`.\n5. Run `cargo build` to verify compilation.\n\n## Adding a new filter\n\n1. Add the filter state to `App` in `tui_app.rs`.\n2. Add filtering logic inside `apply_filters()`.\n3. Add the keybinding in `tui_events.rs` (Normal mode handler).\n4. Add the UI widget in `tui_ui.rs` (`draw_search_and_filters()` function).\n5. Update the status bar help text in `draw_status_bar()`.\n\n## Adding a new CLI subcommand\n\n1. Add a variant to the `Commands` enum in `main.rs`.\n2. Add the match arm in the `main()` function's command dispatch.\n3. Use `display.rs` functions for output, or add new ones as needed.\n\n## Testing\n\nThe project has Rust, Web, and Python test suites.\n\n- Rust unit tests live beside code in `llmfit-core/src/` and\n  `llmfit-tui/src/`.\n- Core integration tests in `llmfit-core/tests/` validate catalog schemas and\n  ONNX model data.\n- CLI integration tests in `llmfit-tui/tests/` use `assert_cmd` against the\n  compiled `llmfit` binary.\n- HTTP API tests exercise Axum routers and JSON responses in `serve_api.rs`.\n- TUI tests focus on state transitions, filters, event handling, and render\n  output. Keep production rendering stateless.\n- Web tests use Vitest, jsdom, and Testing Library. They cover API query\n  construction, localization, filtering, and dashboard interactions.\n- Python tests use pytest. They cover binary discovery, package versioning, and\n  invocation of the packaged Rust binary.\n\nRun the default Rust test set:\n\n```sh\ncargo test\n```\n\nRun all Rust workspace members, including the desktop crate:\n\n```sh\ncargo test --workspace\n```\n\nRun one Rust package:\n\n```sh\ncargo test -p llmfit-core\ncargo test -p llmfit\n```\n\nRun the Web tests:\n\n```sh\nnpm --prefix llmfit-web test\n```\n\nRun the Python tests and quality checks:\n\n```sh\nuv run --project llmfit-python pytest llmfit-python/tests\nmake -C llmfit-python check\n```\n\n## Dependencies policy\n\n- Prefer crates that are well-maintained and have minimal transitive dependencies.\n- `sysinfo` is the system detection crate. Do not replace it with raw platform calls.\n- `ureq` is the blocking HTTP client for providers, benchmarks, updates, quality\n  tests, and sharing. Do not add a second core HTTP client without a concrete need.\n- `which` locates installed runtime binaries. Keep runtime discovery in\n  `providers.rs` instead of adding manual `PATH` parsing.\n- `regex` supports response scoring and text parsing. `serde_yml` parses quality\n  test configuration. `base64` encodes benchmark submissions for GitHub.\n- `objc2-metal` reads the effective Metal working-set limit on macOS. Keep it a\n  macOS-only dependency. Do not replace it with raw platform calls.\n- `ratatui` + `crossterm` is the TUI stack. Do not mix in `termion` or `ncurses`.\n- `clap` with derive feature for CLI parsing. Do not use manual arg parsing.\n- The Python scraper uses only stdlib (`urllib`, `json`). Do not add pip dependencies.\n\n## Common tasks\n\n```sh\n# Build\ncargo build\n\n# Run TUI\ncargo run\n\n# Run CLI mode\ncargo run -- --cli\n\n# Run specific subcommand\ncargo run -- system\ncargo run -- fit --perfect -n 5\ncargo run -- search \"llama\"\n\n# Refresh model database\npython3 scripts/scrape_hf_models.py && cargo build\n\n# Check for compilation issues\ncargo check\n\n# Format code\ncargo fmt\n\n# Lint\ncargo clippy\n```\n\n## Platform notes\n\n- GPU detection shells out to `nvidia-smi` (NVIDIA) and `rocm-smi` (AMD). These are best-effort and fail silently if unavailable.\n- Apple Silicon detection uses `system_profiler SPDisplaysDataType`. On unified memory Macs, VRAM is reported as available system RAM (same pool).\n- `sysinfo` handles cross-platform RAM/CPU. No conditional compilation needed.\n- The TUI uses crossterm which works on Linux, macOS, and Windows terminals.\n"}}