Open-Source Evaluation & Testing library for LLM Agents
# AGENTS.md
giskard-oss β behavioral config for interactive coding assistants with a human in the loop.
## Workflow Orchestration
### 1. Plan Mode Default
β Enter plan mode for ANY non-trivial task (3+ steps or touches multiple libs)
β Brainstorm first (design β spec), then write an implementation plan
β Save specs and plans in the repo's established planning docs. Prefer `docs/specs/` and `docs/plans/`; if the repo already uses another planning directory, follow that convention.
β If something goes sideways, STOP and re-plan immediately
### 2. Subagent Strategy
β Use subagents or equivalent isolated work sessions when the assistant supports them
β Prefer a fresh subagent/session per task; controller curates exactly the context each one needs
β Two-stage review after each task: spec compliance first, then code quality
### 3. Self-Improvement Loop
β After ANY correction: self-document the rule (naming, file headers, examples); only add to AGENTS.md if it cannot be self-documented
β When a correction changes what was implemented vs. what the plan says: update the plan file immediately so reviewers never see a mismatch
### 4. Verification Before Done
β Never mark a task complete without proving it works
β Run: `make format && make check && make test-unit PACKAGE=<affected-lib>`
β Show actual output. Never assume tests pass.
β Ask yourself: "Would a staff engineer approve this?"
### 5. Demand Elegance
β Pause and ask "is there a more elegant solution?"
β Skip for simple fixes β don't over-engineer
### 6. Autonomous Bug Fixing
β When given a bug report with clear scope: just fix it
β No `# type: ignore`, no patched test assertions β fix the root cause
### 7. Commit Hygiene
β Use Conventional Commits for every commit message (for example, `fix(llm): update google interactions translator`)
## Task Management
1. Brainstorm β Spec β Plan β save to the repo's established planning docs, preferably `docs/specs/` and `docs/plans/`
2. Verify Plan β check in before starting
3. Track Progress β mark items complete as you go
4. Explain Changes β high-level summary at each step
5. Document Results β add review section to the plan file
6. Capture Lessons β update AGENTS.md directly after corrections
## Core Principles
β Simplicity First: make every change as simple as possible; prefer deleting lines over adding them
β No Laziness: find root causes; no band-aids, no temporary fixes; senior developer standards
β Minimal Impact: only touch what's necessary; no side effects; no reformatting untouched lines
## Cursor Cloud specific instructions
Non-obvious context for cloud agents (startup refresh is `make install` then `make install-tools`; `uv` is on `PATH` for login shells via `~/.bashrc`).
β This repo is a `uv` workspace monorepo of pure Python libraries (`libs/giskard-*`). There is no server, daemon, or CLI to start β "running" the product means importing the libraries from Python or running `pytest`. Prefer Makefile targets over raw `uv`/`pytest` (`make help`).
β Import namespace is `giskard.<sublib>` (e.g. `import giskard.checks`), not `giskard_checks`.
β Do not run full `make setup` in this VM: its `pre-commit-install` step fails with `Cowardly refusing to install hooks with core.hooksPath set`. That is expected (global `core.hooksPath`) and harmless β use `make install` + `make install-tools` instead; git hooks are not needed to validate work.
β Lint/test/build/run all work without secrets: `make lint`, `make test-unit [PACKAGE=<lib>]`, and importing the libs. Unit tests exclude anything marked `functional`.
β Functional/integration tests are excluded by default and require live LLM provider API keys (e.g. `OPENAI_API_KEY`, `GOOGLE_API_KEY`, `ANTHROPIC_API_KEY`); `giskard-scan` may also reach Hugging Face Hub. Do not expect these to run without those secrets and network access.
β `make check` runs `security` (`pip-audit`) and license checks (`check-licenses`/`check-notices`) that reach the network (PyPI / license metadata); if offline these steps can fail even when the code is fine.