File: README.md
oh — OpenHarness & ohmo
OpenHarness delivers core lightweight agent infrastructure: tool-use, skills, memory, and multi-agent coordination.
ohmo is a personal AI agent built on OpenHarness — not another chatbot, but an assistant that actually works for you over long sessions. Chat with ohmo in Feishu / Slack / Telegram / Discord, and it forks branches, writes code, runs tests, and opens PRs on its own. ohmo runs on your existing Claude Code or Codex subscription — no extra API key needed.
Join the community: contribute Harness for open agent development.
One Command (oh) to Launch OpenHarness and Unlock All Agent Harnesses.
Supports CLI agent integration including OpenClaw, nanobot, Cursor, and more.
✨ OpenHarness's Key Harness Features
| 🔄 Agent Loop | 🔧 Harness Toolkit | 🧠 Context & Memory | 🛡️ Governance | 🤝 Swarm Coordination |
|---|
🤔 What is an Agent Harness?
An Agent Harness is the complete infrastructure that wraps around an LLM to make it a functional agent. The model provides intelligence; the harness provides hands, eyes, memory, and safety boundaries.
OpenHarness is an open-source Python implementation designed for researchers, builders, and the community:
- Understand how production AI agents work under the hood
- Experiment with cutting-edge tools, skills, and agent coordination patterns
- Extend the harness with custom plugins, providers, and domain knowledge
- Build specialized agents on top of proven architecture
📰 What's New
- Unreleased 🔍 Dry-run safe preview:
oh --dry-runpreviews resolved runtime settings, auth state, skills, commands, tools, and configured MCP servers without executing the model, tools, or subagents.- Dry-run now reports a
ready/warning/blockedreadiness verdict with concrete next-step suggestions such as fixing auth, fixing MCP config, or running the prompt directly. - Prompt previews include likely matching skills and tools, while slash-command previews show whether the command is mostly read-only or stateful.
- 2026-04-18 ⚙️ v0.1.7 — Packaging & TUI polish:
- Install script now links
oh,ohmo, andopenharnessinto~/.local/bininstead of prepending the virtualenvbindirectory toPATH, which avoids clobbering Conda-managed shells. - React TUI now supports
Shift+Enterto insert a newline while keeping plainEnteras submit. - Busy-state animation in the React TUI is quieter and less error-prone on Windows terminals, with conservative spinner frames and reduced flashing.
- Install script now links
- 2026-04-10 🧠 v0.1.6 — Auto-Compaction & Markdown TUI:
- Auto-Compaction preserves task state and channel logs across context compression — agents can run multi-day sessions without manual compact/clear
- Subprocess teammates run in headless worker mode; agent team creation stabilized
- Assistant messages now render full Markdown in the React TUI
ohmogains channel slash commands and multimodal attachment support
- 2026-04-08 🔌 v0.1.5 — MCP HTTP transport & Swarm polling:
- MCP protocol adds HTTP transport, auto-reconnect on disconnect, and tool-only server compatibility
- JSON Schema types inferred for MCP tool inputs — no manual type mapping needed
ohmochannels support file attachments and multimodal gateway messages- Subprocess agents are now pollable in real runs; permission modals serialized to prevent input swallowing
- 2026-04-08 🌙 v0.1.4 — Multi-provider auth & Moonshot/Kimi:
- Native Moonshot/Kimi provider with
reasoning_contentsupport for thinking models - Auth overhaul: fixed provider-switching key mismatch,
OPENAI_BASE_URLenv override, profile-scoped credential priority - MCP gracefully handles disconnected servers in
call_tool/read_resource - Security: built-in sensitive-path protection in PermissionChecker, hardened
web_fetchURL validation - Stability: EIO crash recovery in Ink TUI,
--debuglogging, Windows cmd flash fix
- Native Moonshot/Kimi provider with
- 2026-04-06 🚀 v0.1.2 — Unified setup flows and
ohmopersonal-agent app:oh setupnow guides provider selection as workflows instead of exposing raw auth/provider internals- Compatible API setup is now profile-scoped, so Anthropic/OpenAI-compatible endpoints can keep separate keys
ohmoships as a packaged app with~/.ohmoworkspace, gateway, bootstrap prompts, and channel config flow
- 2026-04-01 🎨 v0.1.0 — Initial OpenHarness open-source release featuring complete Harness architecture:
🚀 Quick Start
1. Install
Linux / macOS / WSL
# One-click install
curl -fsSL https://raw.githubusercontent.com/HKUDS/OpenHarness/main/scripts/install.sh | bash
# Or via pip
pip install openharness-aiWindows (Native)
# One-click install (PowerShell)
iex (Invoke-WebRequest -Uri 'https://raw.githubusercontent.com/HKUDS/OpenHarness/main/scripts/install.ps1')
# Or via pip
pip install openharness-aiNote: Windows support is now native. In PowerShell, use openh instead of oh because oh can resolve to the built-in Out-Host alias.
2. Configure
oh setup # interactive wizard — pick a provider, authenticate, done
# On Windows PowerShell, use: openh setupSupports Claude / OpenAI / Copilot / Codex / Moonshot(Kimi) / GLM / MiniMax / NVIDIA NIM and any compatible endpoint.
3. Run
oh
# On Windows PowerShell, use: openh4. Set up ohmo (Personal Agent)
Want an AI agent that works for you from Feishu / Slack / Telegram / Discord?
ohmo init # initialize ~/.ohmo workspace
ohmo config # configure channels and provider
ohmo gateway start # start the gateway — ohmo is now live in your chat appohmo runs on your existing Claude Code subscription or Codex subscription — no extra API key needed.
Non-Interactive Mode (Pipes & Scripts)
# Single prompt → stdout
oh -p "Explain this codebase"
# JSON output for programmatic use
oh -p "List all functions in main.py" --output-format json
# Stream JSON events in real-time
oh -p "Fix the bug" --output-format stream-jsonDry Run (Safe Preview)
Use --dry-run when you want to inspect what OpenHarness would use before any live execution starts.
# Preview an interactive session setup
oh --dry-run
# Preview one prompt without executing the model or tools
oh --dry-run -p "Review this bug fix and grep for failing tests"
# Preview a slash command path
oh --dry-run -p "/plugin list"
# Get structured output for scripts or channels
oh --dry-run -p "Explain this repository" --output-format jsonDry-run is intentionally static:
- It does not call the model
- It does not execute tools or spawn subagents
- It does not connect to MCP servers
- It does resolve settings, auth status, prompt assembly, skills, commands, tools, and obvious MCP config problems
Readiness levels:
ready: configuration looks usable; the next suggested action is usually to run the prompt directlywarning: OpenHarness can resolve the session, but something important still looks wrong, such as broken MCP config or missing auth for later model workblocked: the requested path will not run successfully as-is, for example an unknown slash command or a prompt that cannot resolve a runtime client
next actions in the dry-run output tell you the shortest fix or follow-up step, such as:
- run
oh auth login - fix or disable broken MCP configuration
- run the prompt directly with
oh -p "..."or open the interactive UI withoh
🔌 Provider Compatibility
OpenHarness treats providers as workflows backed by named profiles. In day-to-day use, prefer:
oh setup
oh provider list
oh provider use <profile>Built-in Workflows
| Workflow | What it is | Typical backends |
|---|---|---|
| Anthropic-Compatible API | Anthropic-style request format | Claude official, Kimi, GLM, MiniMax, internal Anthropic-compatible gateways |
| Claude Subscription | Claude CLI subscription bridge | Local ~/.claude/.credentials.json |
| OpenAI-Compatible API | OpenAI-style request format | OpenAI official, OpenRouter, DashScope, DeepSeek, SiliconFlow, Groq, Ollama, GitHub Models |
| Codex Subscription | Codex CLI subscription bridge | Local ~/.codex/auth.json |
| GitHub Copilot | Copilot OAuth workflow | GitHub Copilot device-flow login |
Compatible API Families
Anthropic-Compatible API
Typical examples:
| Backend | Base URL | Example models |
|---|---|---|
| Claude official | https://api.anthropic.com |
claude-sonnet-4-6, claude-opus-4-6 |
| Moonshot / Kimi | https://api.moonshot.cn/anthropic |
kimi-k2.5 |
| Zhipu / GLM | custom Anthropic-compatible endpoint | glm-4.5 |
| MiniMax | custom Anthropic-compatible endpoint | minimax-m1 |
OpenAI-Compatible API
Any provider implementing the OpenAI /v1/chat/completions style API works:
| Backend | Base URL | Example models |
|---|---|---|
| OpenAI | https://api.openai.com/v1 |
gpt-5.4, gpt-4.1 |
| OpenRouter | https://openrouter.ai/api/v1 |
provider-specific |
| Alibaba DashScope | https://dashscope.aliyuncs.com/compatible-mode/v1 |
qwen3.5-flash, qwen3-max, deepseek-r1 |
| DeepSeek | https://api.deepseek.com |
deepseek-chat, deepseek-reasoner |
| GitHub Models | https://models.inference.ai.azure.com |
gpt-4o, Meta-Llama-3.1-405B-Instruct |
| SiliconFlow | https://api.siliconflow.cn/v1 |
deepseek-ai/DeepSeek-V3 |
| NVIDIA NIM | https://integrate.api.nvidia.com/v1 |
openai/gpt-oss-120b, nvidia/llama-3.3-nemotron-super-49b-v1 |
| Google Gemini | https://generativelanguage.googleapis.com/v1beta/openai |
gemini-2.5-flash, gemini-2.5-pro |
| Groq | https://api.groq.com/openai/v1 |
llama-3.3-70b-versatile |
| Ollama (local) | http://localhost:11434/v1 |
any local model |
Advanced Profile Management
# List saved workflows
oh provider list
# Switch the active workflow
oh provider use codex
# Add your own compatible endpoint
oh provider add my-endpoint \
--label "My Endpoint" \
--provider openai \
--api-format openai \
--auth-source openai_api_key \
--model my-model \
--base-url https://example.com/v1For custom compatible endpoints, OpenHarness can bind credentials per profile instead of forcing every Anthropic-compatible or OpenAI-compatible backend to share the same API key.
Ollama (Local Models)
Run local models through Ollama's OpenAI-compatible endpoint:
# Add an Ollama provider profile
oh provider add ollama \
--label "Ollama" \
--provider Ollama \
--api-format openai \
--auth-source openai_api_key \
--model glm-4.7-flash:q8_0 \
--base-url http://localhost:11434/v1Saved provider profile: ollama# Activate and verify
oh provider use ollamaActivated provider profile: ollamaoh provider listclaude-api: Anthropic-Compatible API [ready]
...
moonshot: Moonshot (Kimi) [missing auth]
auth=moonshot_api_key model=kimi-k2.5 base_url=https://api.moonshot.cn/v1
* ollama: Ollama [ready]
auth=openai_api_key model=glm-4.7-flash:q8_0 base_url=http://localhost:11434/v1GitHub Copilot Format (`--api-format copilot`)
Use your existing GitHub Copilot subscription as the LLM backend. Authentication uses GitHub's OAuth device flow — no API keys needed.
# One-time login (opens browser for GitHub authorization)
oh auth copilot-login
# Then launch with Copilot as the provider
uv run oh --api-format copilot
# Or via environment variable
export OPENHARNESS_API_FORMAT=copilot
uv run oh
# Check auth status
oh auth status
# Remove stored credentials
oh auth copilot-logout| Feature | Details |
|---|---|
| Auth method | GitHub OAuth device flow (no API key needed) |
| Token management | Automatic refresh of short-lived session tokens |
| Enterprise | Supports GitHub Enterprise via --github-domain flag |
| Models | Uses Copilot's default model selection |
| API | OpenAI-compatible chat completions under the hood |
🏗️ Harness Architecture
OpenHarness implements the core Agent Harness pattern with 10 subsystems:
openharness/
engine/ # 🧠 Agent Loop — query → stream → tool-call → loop
tools/ # 🔧 43 Tools — file I/O, shell, search, web, MCP
skills/ # 📚 Knowledge — on-demand skill loading (.md files)
plugins/ # 🔌 Extensions — commands, hooks, agents, MCP servers
permissions/ # 🛡️ Safety — multi-level modes, path rules, command deny
hooks/ # ⚡ Lifecycle — PreToolUse/PostToolUse event hooks
commands/ # 💬 54 Commands — /help, /commit, /plan, /resume, ...
mcp/ # 🌐 MCP — Model Context Protocol client
memory/ # 🧠 Memory — persistent cross-session knowledge
tasks/ # 📋 Tasks — background task management
coordinator/ # 🤝 Multi-Agent — subagent spawning, team coordination
prompts/ # 📝 Context — system prompt assembly, CLAUDE.md, skills
config/ # ⚙️ Settings — multi-layer config, migrations
ui/ # 🖥️ React TUI — backend protocol + frontendThe Agent Loop
The heart of the harness. One loop, endlessly composable:
while True:
response = await api.stream(messages, tools)
if response.stop_reason != "tool_use":
break # Model is done
for tool_call in response.tool_uses:
# Permission check → Hook → Execute → Hook → Result
result = await harness.execute_tool(tool_call)
messages.append(tool_results)
# Loop continues — model sees results, decides next actionThe model decides what to do. The harness handles how — safely, efficiently, with full observability.
Harness Flow
flowchart LR
U[User Prompt] --> C[CLI or React TUI]
C --> R[RuntimeBundle]
R --> Q[QueryEngine]
Q --> A[Anthropic-compatible API Client]
A -->|tool_use| T[Tool Registry]
T --> P[Permissions + Hooks]
P --> X[Files Shell Web MCP Tasks]
X --> Q✨ Features
🔧 Tools (43+)
| Category | Tools | Description |
|---|---|---|
| File I/O | Bash, Read, Write, Edit, Glob, Grep | Core file operations with permission checks |
| Search | WebFetch, WebSearch, ToolSearch, LSP | Web and code search capabilities |
| Notebook | NotebookEdit | Jupyter notebook cell editing |
| Agent | Agent, SendMessage, TeamCreate/Delete | Subagent spawning and coordination |
| Task | TaskCreate/Get/List/Update/Stop/Output | Background task management |
| MCP | MCPTool, ListMcpResources, ReadMcpResource | Model Context Protocol integration |
| Mode | EnterPlanMode, ExitPlanMode, Worktree | Workflow mode switching |
| Schedule | CronCreate/List/Delete, RemoteTrigger | Scheduled and remote execution |
| Meta | Skill, Config, Brief, Sleep, AskUser | Knowledge loading, configuration, interaction |
Every tool has:
- Pydantic input validation — structured, type-safe inputs
- Self-describing JSON Schema — models understand tools automatically
- Permission integration — checked before every execution
- Hook support — PreToolUse/PostToolUse lifecycle events
📚 Skills System
Skills are on-demand knowledge — loaded only when the model needs them:
Available Skills:
- commit: Create clean, well-structured git commits
- review: Review code for bugs, security issues, and quality
- debug: Diagnose and fix bugs systematically
- plan: Design an implementation plan before coding
- test: Write and run tests for code
- simplify: Refactor code to be simpler and more maintainable
- pdf: PDF processing with pypdf (from anthropics/skills)
- xlsx: Excel operations (from anthropics/skills)
- ... 40+ moreSkills can live in bundled, user, ohmo, project, or plugin locations. User-level skills are loaded from:
~/.openharness/skills/<skill>/SKILL.md
~/.claude/skills/<skill>/SKILL.md
~/.agents/skills/<skill>/SKILL.mdProject-level skills are enabled by default and are discovered from the current working directory up to the git root:
<project>/.openharness/skills/<skill>/SKILL.md
<project>/.agents/skills/<skill>/SKILL.md
<project>/.claude/skills/<skill>/SKILL.mdDisable project skills for untrusted repositories with:
oh config set allow_project_skills falseUse /skills to list loaded skills with their source and path. User-invocable skills can be run directly as slash commands, for example /deploy staging.
Compatible with anthropics/skills — use the SKILL.md directory layout above.
🌐 Web search and proxy settings
Built-in web_search uses DuckDuckGo HTML search by default. In regions where that endpoint is unreachable, point OpenHarness at a trusted public HTML search endpoint or your own SearXNG instance:
export OPENHARNESS_WEB_SEARCH_URL="https://your-searxng.example/search"web_search and web_fetch keep trust_env=False for SSRF safety, so they do not automatically inherit HTTP_PROXY / HTTPS_PROXY. If you need a proxy, opt in with an OpenHarness-specific variable:
export OPENHARNESS_WEB_PROXY="http://127.0.0.1:7890"The proxy URL must be HTTP/HTTPS and cannot contain embedded credentials.
🔌 Plugin System
Compatible with claude-code plugins. Tested with 12 official plugins:
| Plugin | Type | What it does |
|---|---|---|
commit-commands |
Commands | Git commit, push, PR workflows |
security-guidance |
Hooks | Security warnings on file edits |
hookify |
Commands + Agents | Create custom behavior hooks |
feature-dev |
Commands | Feature development workflow |
code-review |
Agents | Multi-agent PR review |
pr-review-toolkit |
Agents | Specialized PR review agents |
# Manage plugins
oh plugin list
oh plugin install <source>
oh plugin enable <name>🤝 Ecosystem Workflows
OpenHarness is useful as a lightweight harness layer around Claude-style tooling conventions:
- OpenClaw-oriented workflows can reuse Markdown-first knowledge and command-driven collaboration patterns.
- Claude-style plugins and skills stay portable because OpenHarness keeps those formats familiar.
- ClawTeam-style multi-agent work maps well onto the built-in team, task, and background execution primitives.
For concrete usage ideas instead of generic claims, see docs/SHOWCASE.md.
🛡️ Permissions
Multi-level safety with fine-grained control:
| Mode | Behavior | Use Case |
|---|---|---|
| Default | Ask before write/execute | Daily development |
| Auto | Allow everything | Sandboxed environments |
| Plan Mode | Block all writes | Large refactors, review first |
Path-level rules in settings.json:
{
"permission": {
"mode": "default",
"path_rules": [{"pattern": "/etc/*", "allow": false}],
"denied_commands": ["rm -rf /", "DROP TABLE *"]
}
}🖥️ Terminal UI
React/Ink TUI with full interactive experience:
- Command picker: Type
/→ arrow keys to select → Enter - Permission dialog: Interactive y/n with tool details
- Mode switcher:
/permissions→ select from list - Session resume:
/resume→ pick from history - Animated spinner: Real-time feedback during tool execution
- Keyboard shortcuts: Shown at the bottom, context-aware
📡 CLI
oh [OPTIONS] COMMAND [ARGS]
Session: -c/--continue, -r/--resume, -n/--name
Model: -m/--model, --effort, --max-turns
Output: -p/--print, --output-format text|json|stream-json
Permissions: --permission-mode, --dangerously-skip-permissions
Context: -s/--system-prompt, --append-system-prompt, --settings
Advanced: -d/--debug, --mcp-config, --bare
Subcommands: oh setup | oh provider | oh auth | oh mcp | oh plugin🧑💼 ohmo Personal Agent
ohmo is a personal-agent app built on top of OpenHarness. It is packaged alongside oh, with its own workspace and gateway:
# Initialize personal workspace
ohmo init
# Configure gateway channels and pick a provider profile
ohmo config
# Run the personal agent
ohmo
# Run the gateway in foreground
ohmo gateway run
# Check or restart the gateway
ohmo gateway status
ohmo gateway restartKey concepts:
~/.ohmo/- personal workspace root
soul.md- long-term agent personality and behavior
identity.md- who
ohmois
- who
user.md- user profile and preferences
BOOTSTRAP.md- first-run landing ritual
memory/- personal memory
gateway.json- selected provider profile and channel configuration
ohmo config uses the same workflow language as oh setup, so you can point the personal-agent gateway at:
Anthropic-Compatible APIClaude SubscriptionOpenAI-Compatible APICodex SubscriptionGitHub Copilot
ohmo init creates the home workspace once. After that, use ohmo config to update provider and channel settings; if the gateway is already running, the config flow can restart it for you.
Currently ohmo init / ohmo config can guide channel setup for:
- Telegram
- Slack
- Discord
- Feishu
📊 Test Results
| Suite | Tests | Status |
|---|---|---|
| Unit + Integration | 114 | ✅ All passing |
| CLI Flags E2E | 6 | ✅ Real model calls |
| Harness Features E2E | 9 | ✅ Retry, skills, parallel, permissions |
| React TUI E2E | 3 | ✅ Welcome, conversation, status |
| TUI Interactions E2E | 4 | ✅ Commands, permissions, shortcuts |
| Real Skills + Plugins | 12 | ✅ anthropics/skills + claude-code/plugins |
# Run all tests
uv run pytest -q # 114 unit/integration
python scripts/test_harness_features.py # Harness E2E
python scripts/test_real_skills_plugins.py # Real plugins E2E🔧 Extending OpenHarness
Add a Custom Tool
from pydantic import BaseModel, Field
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
class MyToolInput(BaseModel):
query: str = Field(description="Search query")
class MyTool(BaseTool):
name = "my_tool"
description = "Does something useful"
input_model = MyToolInput
async def execute(self, arguments: MyToolInput, context: ToolExecutionContext) -> ToolResult:
return ToolResult(output=f"Result for: {arguments.query}")Add a Custom Skill
Create ~/.openharness/skills/my-skill.md:
---
name: my-skill
description: Expert guidance for my specific domain
---
# My Skill
## When to use
Use when the user asks about [your domain].
## Workflow
1. Step one
2. Step two
...Add a Plugin
Create .openharness/plugins/my-plugin/.claude-plugin/plugin.json:
{
"name": "my-plugin",
"version": "1.0.0",
"description": "My custom plugin"
}Add commands in commands/*.md, hooks in hooks/hooks.json, agents in agents/*.md.
🌍 Showcase
OpenHarness is most useful when treated as a small, inspectable harness you can adapt to a real workflow:
- Repo coding assistant for reading code, patching files, and running checks locally.
- Headless scripting tool for
jsonandstream-jsonoutput in automation flows. - Plugin and skill testbed for experimenting with Claude-style extensions.
- Multi-agent prototype harness for task delegation and background execution.
- Provider comparison sandbox across Anthropic-compatible backends.
See docs/SHOWCASE.md for short, reproducible examples.
🤝 Contributing
OpenHarness is a community-driven research project. We welcome contributions in:
| Area | Examples |
|---|---|
| Tools | New tool implementations for specific domains |
| Skills | Domain knowledge .md files (finance, science, DevOps...) |
| Plugins | Workflow plugins with commands, hooks, agents |
| Providers | Support for more LLM backends (OpenAI, Ollama, etc.) |
| Multi-Agent | Coordination protocols, team patterns |
| Testing | E2E scenarios, edge cases, benchmarks |
| Documentation | Architecture guides, tutorials, translations |
# Development setup
git clone https://github.com/HKUDS/OpenHarness.git
cd OpenHarness
uv sync --extra dev
uv run pytest -q # Verify everything worksUseful contributor entry points:
CONTRIBUTING.mdfor setup, checks, and PR expectationsCHANGELOG.mdfor user-visible changesdocs/SHOWCASE.mdfor real-world usage patterns worth documenting
🔧 Troubleshooting
Backspace key in macOS Terminal.app
OpenHarness handles both common terminal delete sequences, including the raw DEL byte (0x7f) that macOS Terminal.app sends for Backspace. If Backspace inserts spaces or visible control characters instead of deleting text, upgrade OpenHarness first.
For older versions that do not include this fix, use a terminal that sends a standard Backspace sequence or adjust your terminal keyboard profile as a temporary workaround.
📄 License
MIT — see LICENSE.
File: README.zh-CN.md
[Binary file]
File: CHANGELOG.md
Changelog
All notable changes to OpenHarness should be recorded in this file.
The format is based on Keep a Changelog, and this project currently tracks changes in a lightweight, repository-oriented way.
[Unreleased]
Added
- Hooks now support a
priorityfield (default0). Within an event, hooks run highest-priority first, and hooks sharing a priority keep their registration order. This lets users order, for example, a security-check hook ahead of a logging hook regardless of where each is declared in settings or contributed by plugins. edit_fileandwrite_filein the React TUI now preview a unified diff before applying file changes, let users approve once or for the rest of the session, and skip the extra prompt automatically infull_automode.
Fixed
- Codex subscription requests now pass reasoning effort separately, enabling
gpt-5.5withxhigheffort instead of treatinggpt-5.5 xhighas an unsupported model name. - Telegram channel now delivers replies again under
ohmo init --no-interactiveand other configs that do not write areply_to_messagefield.TelegramConfigdeclaresreply_to_message: bool = Trueso the attribute access inTelegramChannel.sendno longer raisesAttributeErrorand outbound progress/tool-hint/final messages are sent as expected. See issue #243.
[0.1.9] - 2026-05-07
Added
- Added a bundled
skill-creatorskill for creating, improving, and verifying OpenHarness/ohmo skills. - User-invocable skills can now be triggered directly as slash commands, with support for skill-specific arguments and model override metadata.
Fixed
oh setupcan now update the API key for an already-configured API-key provider profile instead of only changing the model.oh provider edit --api-key <key>can now replace a saved profile API key, andoh provider add ... --api-key <key>can store one during profile creation.
[0.1.8] - 2026-05-06
Added
- Built-in
nvidiaprovider profile sooh setupoffers NVIDIA NIM as a first-class OpenAI-compatible provider choice, withNVIDIA_API_KEYauth source,openai/gpt-oss-120bas the default model, and the NVIDIA NIM endpoint. - Built-in
qwenprovider profile sooh setupoffers Qwen (DashScope) as a first-class provider choice, withdashscope_api_keyauth source,qwen-plusas the default model, and the DashScope OpenAI-compatible endpoint. - Plugin tool discovery: plugins can now provide
BaseToolsubclasses in a/tools/directory and they are auto-discovered, instantiated, and registered in the tool registry at runtime. Addtools_dirtoplugin.json(defaults to"tools"). oh --dry-runsafe preview mode for inspecting resolved runtime settings, auth state, prompt assembly, commands, skills, tools, and configured MCP servers without executing the model or tools.- Built-in
minimaxprovider profile sooh setupoffers MiniMax as a first-class provider choice, withMINIMAX_API_KEYauth source,MiniMax-M2.7as the default model, andMiniMax-M2.7-highspeedin the model picker. - Docker as an alternative sandbox backend (
sandbox.backend = "docker") for stronger execution isolation with configurable resource limits, network isolation, and automatic image management. - Built-in
geminiprovider profile sooh setupoffers Google Gemini as a first-class provider choice, withgemini_api_keyauth source andgemini-2.5-flashas the default model. diagnoseskill: trace agent run failures and regressions using structured evidence from run artifacts.- OpenAI-compatible API client (
--api-format openai) supporting any provider that implements the OpenAI/v1/chat/completionsformat, including Alibaba DashScope, DeepSeek, GitHub Models, Groq, Together AI, Ollama, and more. OPENHARNESS_API_FORMATenvironment variable for selecting the API format.OPENAI_API_KEYfallback when using OpenAI-format providers.- GitHub Actions CI workflow for Python linting, tests, and frontend TypeScript checks.
CONTRIBUTING.mdwith local setup, validation commands, and PR expectations.docs/SHOWCASE.mdwith concrete OpenHarness usage patterns and demo commands.- GitHub issue templates and a pull request template.
- React TUI assistant messages now render structured Markdown blocks, including headings, lists, code fences, blockquotes, links, and tables.
- Built-in
codexoutput style for compact, low-noise transcript rendering in React TUI.
Fixed
Subprocess teammate spawn (
agenttool,task_create) now works on Windows under Git Bash.subprocess_backend.spawnbuilds a direct-execargvlist and passes it through newargv=andenv=kwargs onBackgroundTaskManager.create_agent_task/create_shell_task;_start_processthen runs the executable viaasyncio.create_subprocess_exec(*argv)with no shell in between. Previously the spawn command was a single string interpreted bybash -lc, which on Windows could not reliably exec a Windows-pathed Python interpreter (e.g.C:\Users\...\python.exe) — Git Bash's escape parser consumed the backslashes from the embedded env-prefix and, even with proper quoting, bash launched viaasyncio.create_subprocess_execreturnedcommand not foundfor Windows-pathed binaries that worked perfectly when invoked interactively. Bypassing the shell sidesteps the entire class of cross-platform quoting and path-translation hazard. The legacy shell-evaluatedcommand=path is preserved for callers (e.g.BashTool) that legitimately want shell semantics. See issue #230.Bundled skill loader now uses
yaml.safe_loadfor SKILL.md frontmatter, matching the user-skill loader. The shared parser is extracted toopenharness.skills._frontmatterso bundled and user skills handle YAML block scalars (>,|), quoted values, and other standard YAML constructs the same way.Compaction now detects llama.cpp/OpenAI-compatible context overflow errors, accounts for image blocks in auto-compact token estimates, and strips image payloads from summarizer-only compaction requests.
Large tool results are now bounded in conversation history: oversized outputs are saved under
tool_artifacts, old MCP results become microcompactable, and context collapse trims stale tool-result payloads.ohmo now keeps personal memory isolated from OpenHarness project memory:
/memoryin ohmo sessions targets the ohmo workspace memory store, and ohmo runtime prompt refreshes no longer inject project memory unless explicitly requested.Fixed
globandgreptools hanging indefinitely when thergsubprocess produced enough stderr output to fill the OS pipe buffer.stderris now redirected toDEVNULLso it is discarded rather than blocking the child process.Fixed
bash_toolhanging after a timed-out command when the subprocess stdout stream stayed open._read_remaining_outputnow applies a 2-secondasyncio.wait_fortimeout so the tool always returns promptly.Fixed
session_runnerbackground task deadlock caused by an unreadstderr=PIPEstream. The subprocess now usesstderr=STDOUTso all output merges into the single readable stdout pipe.React TUI prompt input now treats the raw DEL byte (
0x7f) as backward delete while preserving true forward-delete escape sequences, fixing backspace failures seen in some macOS terminal environments.todo_writetool now updates an existing unchecked item in-place whenchecked=Trueinstead of appending a duplicate[x]line.Built-in
Exploreandclaude-code-guideagents no longer hard-codemodel="haiku", which caused them to fail for users on non-Anthropic providers (OpenAI, Bedrock, custom base URLs, etc.). Both agents now usemodel="inherit"so they run with whatever model the parent session is using.build_inherited_cli_flagsis also fixed to skip the--modelflag entirely when the value is"inherit", letting the subprocess correctly inherit the parent model via theOPENHARNESS_MODELenvironment variable instead of receiving the literal string"inherit"as a model name.React TUI spinner now stays visible throughout the entire agent turn:
assistant_completeno longer resetsbusystate prematurely, andtool_startedexplicitly setsbusy=trueso the status bar remains active even when tool calls follow an assistant message.line_completeis the sole signal that ends the turn and clears the spinner.Skill loader now uses
yaml.safe_loadto parse SKILL.md frontmatter, correctly handling YAML block scalars (>,|), quoted values, and other standard YAML constructs instead of naive line-by-line splitting.BackendHostConfigwas missing thecwdfield, causingAttributeError: 'BackendHostConfig' object has no attribute 'cwd'on startup whenohwas run after the runtime refactor that addedcwdsupport tobuild_runtime.Shell-escape
$ARGUMENTSsubstitution in command hooks to prevent shell injection from payload values containing metacharacters like$(...)or backticks.Swarm
_READ_ONLY_TOOLSnow uses actual registered tool names (snake_case) instead of PascalCase, fixing read-only auto-approval inhandle_permission_request.Memory scanner now parses YAML frontmatter (
name,description,type) instead of returning raw---as description.Memory search matches against body content in addition to metadata, with metadata weighted higher for relevance.
Memory search tokenizer handles Han characters for multilingual queries.
Fixed duplicate response in React TUI caused by double Enter key submission in the input handler.
Fixed concurrent permission modals overwriting each other in TUI default mode when the LLM returns multiple tool calls in one response;
_ask_permissionnow serialises callers via anasyncio.Lockso each modal is shown and resolved before the next one is emitted.Fixed React TUI Markdown tables to size columns from rendered cell text so inline formatting like code spans and bold text no longer breaks alignment.
Fixed grep tool crashing with
ValueError/LimitOverrunErrorwhen ripgrep outputs a line longer than 64 KB (e.g. minified assets or lock files). The asyncio subprocess stream limit is now 8 MB and oversized lines are skipped rather than terminating the session.Fixed React TUI exit leaving the shell prompt concatenated with the last TUI line. The terminal cleanup handler now writes a trailing newline (
\n) alongside the cursor-show escape sequence so the shell prompt always starts on a fresh line.Reduced React TUI redraw pressure when
output_style=codexby avoiding token-level assistant buffer flushes during streaming.
Changed
- ohmo Feishu group routing now supports managed group creation, gateway-scoped provider/model commands, and stricter group mention handling so group conversations only wake ohmo when explicitly addressed.
- Dry-run output now reports a
ready/warning/blockedreadiness verdict, concretenext_actions, likely matching skills/tools for normal prompts, and richer slash-command previews for read-only vs stateful command paths. - React TUI now groups consecutive
tool+tool_resulttranscript rows into a single compound row: success shows the result line count inline (e.g.→ 24L), errors show a red icon and up to 5 lines of error detail beneath the tool row. Standalone successful tool results are suppressed to reduce transcript noise; standalone errors are still surfaced. - README now links to contribution docs, changelog, showcase material, and provider compatibility guidance.
- README quick start now includes a one-command demo and clearer provider compatibility notes.
- README provider compatibility section updated to include OpenAI-format providers.
[0.1.7] - 2026-04-18
Fixed
- Install script now links
oh,ohmo, andopenharnessinto~/.local/bininstead of prepending the virtualenvbindirectory toPATH, which avoids overriding Conda-managed shells while preserving global command discovery. - React TUI prompt now supports
Shift+Enterfor inserting a newline without submitting the current prompt. - React TUI busy-state animation is less error-prone on Windows terminals: the extra pseudo-animation line was removed, Windows now uses conservative ASCII spinner frames, and the spinner interval was slightly slowed to reduce flashing.
[0.1.0] - 2026-04-01
Added
- Initial public release of OpenHarness.
- Core agent loop, tool registry, permission system, hooks, skills, plugins, MCP support, and terminal UI.
File: CONTRIBUTING.md
Contributing to OpenHarness
OpenHarness is an open-source agent harness focused on clarity, hackability, and compatibility with Claude-style workflows.
Ways to contribute
- Fix bugs or tighten edge-case handling in the harness runtime.
- Improve docs, onboarding, examples, and architecture notes.
- Add tests for tools, permissions, plugins, MCP, or multi-agent flows.
- Contribute new skills, plugins, or provider compatibility improvements.
- Share real usage patterns that can be added to
docs/SHOWCASE.md.
Development setup
git clone https://github.com/HKUDS/OpenHarness.git
cd OpenHarness
uv sync --extra devIf you want to work on the React terminal UI as well:
cd frontend/terminal
npm ci
cd ../..Local checks
Run the same core checks that CI runs before opening a PR:
uv run ruff check src tests scripts
uv run pytest -qFrontend sanity check:
cd frontend/terminal
npx tsc --noEmitPull request expectations
- Keep PRs scoped. Small, reviewable changes merge faster than broad rewrites.
- Include the problem, the change, and how you verified it.
- Add or update tests when behavior changes.
- Update docs when CLI flags, workflows, or compatibility claims change.
- Add a short entry under
UnreleasedinCHANGELOG.mdfor user-visible changes. - If you are improving type coverage, feel free to run
uv run mypy src/openharness, but it is not yet a required green check for the whole repo.
Documentation and community contributions
Issue #7 surfaced several high-value docs needs. Useful contributions in that area include:
- README accuracy improvements and compatibility notes.
- Short, reproducible examples for common workflows.
- Showcase entries based on real usage rather than generic marketing claims.
- Contribution and maintenance docs that make the repo easier to navigate.
Reporting bugs and proposing features
- Use the GitHub issue templates when possible.
- Include environment details, exact commands, and error output for bugs.
- For features, explain the concrete workflow gap and expected behavior.
- If the request is mostly documentation or maintenance related, say that explicitly so it can be scoped as a docs PR.
File: RELEASE_NOTES_v0.1.8.md
v0.1.8 — Providers, ohmo Feishu Groups, and Safer Remotes
OpenHarness v0.1.8 is a stabilization and provider-expansion release. It adds more first-class provider workflows, improves ohmo's Feishu group experience, hardens remote-channel security, and fixes several Windows/MCP reliability issues.
Highlights
New provider workflows
- Added NVIDIA NIM as a built-in OpenAI-compatible provider using
NVIDIA_API_KEY. - Added ModelScope Inference API support.
- Added Qwen (DashScope), MiniMax, and Gemini provider profiles.
- Improved OpenAI-compatible API behavior, including explicit bearer authorization headers and `` block filtering for compatible streaming responses.
- Added NVIDIA NIM as a built-in OpenAI-compatible provider using
ohmo Feishu group support
- Added Feishu managed group creation flow for ohmo.
- Added gateway-scoped provider/model commands for chat-based operation.
- Improved group routing and mention policy so ohmo responds in shared Feishu groups only when explicitly addressed.
- Hardened Feishu attachment filename handling.
Security and remote-channel hardening
- Kept sensitive config/auth/provider/model/ship commands local-only by default in remote channels.
- Kept bridge commands local-only by default.
- Added coverage for bridge spawn blocking and remote gateway security behavior.
- Rejected path traversal names during plugin uninstall.
Windows and shell reliability
- Fixed Windows agent/subagent spawning by direct-executing teammate argv instead of shell-wrapping Python paths.
- Windows shell resolution now skips discovered
bash.exebinaries that cannot actually run commands, falling back to PowerShell/cmd. - Improved Windows gateway process lifecycle handling.
- Avoided shell execution when opening browsers on Windows.
MCP, tools, and stability
- MCP startup now isolates failed servers instead of aborting the whole OpenHarness startup.
- Fixed subprocess stderr pipe deadlocks in grep/glob/bash/session runner paths.
- Bounded large tool results in conversation history and improved compaction under large/vision contexts.
- Improved skill frontmatter parsing with YAML
safe_loadfor bundled and user skills.
TUI and UX fixes
- Restored raw
DELbackspace handling for macOS Terminal-style environments. - Added better slash-command completion, markdown table rendering, spinner behavior, and escape interruption.
- Added image-to-text fallback support for text-only models and
--vision-modeloverride.
- Restored raw
External contributors
Thanks to the external contributors whose PRs are included in this release:
- @Litianhui888 — MCP startup isolation (#237)
- @Hinotoi-agent — remote command security hardening (#232, #209, #208, #198, #197)
- @nsxdavid — Windows agent spawn fix (#231)
- @voidborne-d — bundled skill frontmatter parsing (#229)
- @Mcy0618 — image-to-text fallback and ModelScope support (#227, #224)
- @WANG-Guangxin — invalid regex fallback fix (#219)
- @glitch-ux — Windows browser auth safety, async coordinator draining, autopilot shell safety, hook events (#217, #200, #188, #170)
- @Escapingbug — Windows gateway lifecycle and Telegram proxy config fixes (#193, #192)
- @ZevGit — Qwen provider profile (#207)
- @yl-jiang — subprocess stderr deadlock fixes and OpenAI-compatible think-block filtering (#205, #174)
- @he-yufeng — TUI slash-command completion polish (#185)
- @powAu3 — raw DEL backspace fix (#182)
- @flobo3 — shell subprocess stdin default fix (#179)
Install
pip install --upgrade openharness-ai==0.1.8Or run the installer from the repository docs.
File: RELEASE_NOTES_v0.1.9.md
v0.1.9 — Skill Workflows and Provider Key Updates
OpenHarness v0.1.9 is a small follow-up release after v0.1.8 focused on making skills easier to create and invoke, plus fixing provider API key updates.
Highlights
Bundled skill creator
- Added a bundled
skill-creatorskill for creating, improving, and verifying OpenHarness/ohmo skills. - This makes repeatable workflows easier to capture as first-class skills.
- Added a bundled
User-invocable skill slash commands
- Skills marked as user-invocable can now be triggered directly with slash commands.
- Slash-invoked skills support user arguments and can request a model override through skill metadata.
Provider API key update fix
oh setupnow lets users update the API key for an already-configured API-key provider profile.oh provider edit --api-key <key>can replace a saved profile key directly.oh provider add ... --api-key <key>can store a key while creating a provider profile.
Fixes
- Fixed issue #238, where users could change a configured provider model but had no supported path to update the saved API key.
Install
pip install --upgrade openharness-ai==0.1.9Contributors
This release is primarily a maintainer follow-up release. Thanks to the users and contributors who reported and validated the provider key update workflow, especially the reporter of #238.
File: autopilot-dashboard/package.json
{
"name": "autopilot-dashboard",
"private": true,
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^19.1.0",
"react-dom": "^19.1.0"
},
"devDependencies": {
"@types/react": "^19.1.2",
"@types/react-dom": "^19.1.2",
"@vitejs/plugin-react": "^4.4.1",
"typescript": "~5.8.3",
"vite": "^6.3.2"
}
}
File: autopilot-dashboard/tsconfig.app.json
{
"compilerOptions": {
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsBuildInfo",
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"noUncheckedSideEffectImports": true
},
"include": ["src"]
}
File: autopilot-dashboard/tsconfig.json
{
"files": [],
"references": [{ "path": "./tsconfig.app.json" }]
}
File: autopilot-dashboard/public/snapshot.json
{
"generated_at": 1776338766.0050209,
"repo_name": "OpenHarness-new",
"repo_path": "/home/tangjiabin/OpenHarness-new",
"focus": null,
"counts": {
"queued": 0,
"accepted": 0,
"preparing": 0,
"running": 0,
"verifying": 0,
"pr_open": 0,
"waiting_ci": 0,
"repairing": 0,
"completed": 0,
"merged": 0,
"failed": 0,
"rejected": 0,
"superseded": 0
},
"status_order": [
"queued",
"accepted",
"preparing",
"running",
"verifying",
"pr_open",
"waiting_ci",
"repairing",
"completed",
"merged",
"failed",
"rejected",
"superseded"
],
"columns": {
"queued": [],
"accepted": [],
"preparing": [],
"running": [],
"verifying": [],
"pr_open": [],
"waiting_ci": [],
"repairing": [],
"completed": [],
"merged": [],
"failed": [],
"rejected": [],
"superseded": []
},
"cards": [],
"journal": [],
"policies": {
"autopilot": "/home/tangjiabin/OpenHarness-new/.openharness/autopilot/autopilot_policy.yaml",
"verification": "/home/tangjiabin/OpenHarness-new/.openharness/autopilot/verification_policy.yaml",
"release": "/home/tangjiabin/OpenHarness-new/.openharness/autopilot/release_policy.yaml"
},
"active_context": "# Active Repo Context\n\nGenerated at: 2026-04-16 11:05:49 UTC\n\n## Current Task Focus\n- No active repo task focus yet.\n\n## In Progress\n- None.\n\n## Next Up\n- No queued items.\n\n## Recently Completed\n- None yet.\n\n## Recent Failures\n- None.\n\n## Recent Repo Journal\n- Journal is empty.\n\n## Policies\n- Autopilot: /home/tangjiabin/OpenHarness-new/.openharness/autopilot/autopilot_policy.yaml\n- Verification: /home/tangjiabin/OpenHarness-new/.openharness/autopilot/verification_policy.yaml\n- Release: /home/tangjiabin/OpenHarness-new/.openharness/autopilot/release_policy.yaml"
}
File: docs/SHOWCASE.md
OpenHarness Showcase
This page collects concrete ways to use OpenHarness without overselling the project. Each example is intended to be small, reproducible, and easy to extend.
1. Repository-aware coding assistant
Use OpenHarness as a lightweight local coding agent for reading code, making edits, and running validation commands.
uv run ohExample prompt:
Review this repo, identify the highest-risk bug, patch it, and run the relevant tests.2. Headless automation for scripts and CI
The print mode is useful when you want structured output in shell pipelines or automation jobs.
uv run oh -p "Summarize the purpose of this repository" --output-format json
uv run oh -p "List files that define the permission system" --output-format stream-json3. Skill and plugin playground
OpenHarness can load Markdown skills and Claude-style plugin layouts, which makes it useful for experimentation with custom workflows.
Examples:
- Put a custom skill in
~/.openharness/skills/. - Install a plugin into
~/.openharness/plugins/. - Use the same workflow conventions across multiple local projects.
4. Multi-agent and background task experiments
The repo includes team coordination primitives, background task management, and task inspection tools.
Example prompts:
Spawn a worker to audit the test suite while you inspect the CLI command registry.Create a background task that runs the slow integration script and report back when it finishes.5. Provider compatibility testbed
OpenHarness is useful when you need to compare Anthropic-compatible backends behind one harness.
Typical scenarios:
- Default Anthropic setup.
- Moonshot/Kimi through an Anthropic-compatible endpoint.
- Vertex-compatible and Bedrock-compatible gateways.
- Internal proxies that expose an Anthropic-style API surface.
See the provider compatibility table in README.md.
6. Documentation-first onboarding
If you are evaluating the project rather than contributing code, start here:
README.mdfor install, usage, and architecture.CONTRIBUTING.mdfor contributor workflow.CHANGELOG.mdfor visible repo changes.
How to contribute a showcase entry
Good showcase additions are:
- Based on a real workflow you ran.
- Short enough to reproduce locally.
- Honest about prerequisites and limitations.
- Focused on what OpenHarness makes easier, not on generic LLM claims.
File: docs/autopilot/snapshot.json
{
"generated_at": 1776338766.0050209,
"repo_name": "OpenHarness-new",
"repo_path": "/home/tangjiabin/OpenHarness-new",
"focus": null,
"counts": {
"queued": 0,
"accepted": 0,
"preparing": 0,
"running": 0,
"verifying": 0,
"pr_open": 0,
"waiting_ci": 0,
"repairing": 0,
"completed": 0,
"merged": 0,
"failed": 0,
"rejected": 0,
"superseded": 0
},
"status_order": [
"queued",
"accepted",
"preparing",
"running",
"verifying",
"pr_open",
"waiting_ci",
"repairing",
"completed",
"merged",
"failed",
"rejected",
"superseded"
],
"columns": {
"queued": [],
"accepted": [],
"preparing": [],
"running": [],
"verifying": [],
"pr_open": [],
"waiting_ci": [],
"repairing": [],
"completed": [],
"merged": [],
"failed": [],
"rejected": [],
"superseded": []
},
"cards": [],
"journal": [],
"policies": {
"autopilot": "/home/tangjiabin/OpenHarness-new/.openharness/autopilot/autopilot_policy.yaml",
"verification": "/home/tangjiabin/OpenHarness-new/.openharness/autopilot/verification_policy.yaml",
"release": "/home/tangjiabin/OpenHarness-new/.openharness/autopilot/release_policy.yaml"
},
"active_context": "# Active Repo Context\n\nGenerated at: 2026-04-16 11:05:49 UTC\n\n## Current Task Focus\n- No active repo task focus yet.\n\n## In Progress\n- None.\n\n## Next Up\n- No queued items.\n\n## Recently Completed\n- None yet.\n\n## Recent Failures\n- None.\n\n## Recent Repo Journal\n- Journal is empty.\n\n## Policies\n- Autopilot: /home/tangjiabin/OpenHarness-new/.openharness/autopilot/autopilot_policy.yaml\n- Verification: /home/tangjiabin/OpenHarness-new/.openharness/autopilot/verification_policy.yaml\n- Release: /home/tangjiabin/OpenHarness-new/.openharness/autopilot/release_policy.yaml"
}
File: frontend/terminal/package.json
{
"name": "@openharness/terminal",
"private": true,
"type": "module",
"scripts": {
"start": "tsx src/index.tsx"
},
"dependencies": {
"ink": "^5.1.0",
"ink-text-input": "^6.0.0",
"marked": "^18.0.0",
"react": "^18.3.1",
"string-width": "^7.2.0"
},
"devDependencies": {
"@types/node": "^22.13.10",
"@types/react": "^18.3.12",
"tsx": "^4.19.2",
"typescript": "^5.7.3"
}
}
File: frontend/terminal/tsconfig.json
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"jsx": "react-jsx",
"esModuleInterop": true,
"strict": false,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src//*.ts", "src//*.tsx"]
}
File: src/openharness/skills/bundled/content/commit.md
commit
Create clean, well-structured git commits.
When to use
Use when the user asks to commit changes, create a PR, or prepare code for review.
Workflow
- Run
git statusandgit diffto understand all changes - Analyze changes: categorize as feature, fix, refactor, docs, test, etc.
- Draft a concise commit message:
- First line: imperative mood, under 72 chars, describes the "why"
- Body (if needed): explain context, trade-offs, or breaking changes
- Stage only relevant files — never stage .env, credentials, or large binaries
- Create the commit
Rules
- Prefer specific
git add <file>overgit add -A - Never use
--no-verifyunless explicitly asked - Never amend published commits unless explicitly asked
- If a pre-commit hook fails, fix the issue and create a NEW commit (don't --amend)
- Include
Co-Authored-Byif pair programming
File: src/openharness/skills/bundled/content/debug.md
debug
Diagnose and fix bugs systematically.
When to use
Use when the user reports a bug, error, or unexpected behavior.
Workflow
- Reproduce: understand the exact steps that trigger the issue
- Read the error: stack traces, log messages, error codes
- Locate: use Grep/Read to find the relevant code path
- Hypothesize: form a theory about the root cause
- Verify: add logging or read surrounding code to confirm
- Fix: make the minimal change that addresses the root cause
- Test: verify the fix works and doesn't break other things
Rules
- Read the error message carefully before searching code
- Don't guess — verify your hypothesis before changing code
- Fix the root cause, not the symptom
- Don't retry the same approach if it failed — investigate why
- If stuck after 3 attempts, explain what you've tried and ask for help
File: src/openharness/skills/bundled/content/diagnose.md
diagnose
Diagnose why an agent run failed, regressed, or produced unexpected output — using structured evidence instead of intuition.
When to use
Use when the user asks:
- "Why did this run fail?"
- "What changed between this run and the last one?"
- "The output looks wrong — what happened?"
- "Why is this run worse than before?"
Workflow
- Locate the run artifacts — check
artifacts/runs/<run_id>/or the latest run directory for:manifest.json— run metadata, task type, input hash, timestampsexecution_trace.jsonl— the full tool-call chain, one event per lineverification_report.json— what was verified and whether it passedfailure_signature.json— which stage failed and why (if present)
- Read the failure signature first — if it exists, it already localizes the failure to a specific stage (routing, execution, verification, or governance)
- Trace the execution chain — walk
execution_trace.jsonlforward to find where output diverged from expectation - Compare with a passing run — if a previous run succeeded, diff the two manifests and traces to identify what changed
- Identify the failure layer:
- Routing: wrong task type or profile selected
- Execution: tool call error, permission block, or unexpected result
- Verification: output produced but did not match expected artifact
- Governance: run was halted by a safety or boundary check
- Report with evidence — cite specific file paths, line numbers in the trace, or field values from the manifest; never summarize without pointing to the source
Rules
- Evidence before intuition: always read the trace before forming a hypothesis
- Localize before explaining: identify which stage failed before describing why
- Compare, don't guess: if a previous run succeeded, diff it — don't speculate about what changed
- If no artifacts exist, say so clearly and ask the user to re-run with archive mode enabled
File: src/openharness/skills/bundled/content/plan.md
plan
Design an implementation plan before coding.
When to use
Use when the user asks to plan, design, or architect a feature before implementing it.
Workflow
- Understand the requirement:
- What problem does this solve?
- What are the constraints?
- What's the expected outcome?
- Explore the codebase to find:
- Existing functions/utilities that can be reused
- Patterns used elsewhere in the project
- Files that will need modification
- Design the approach:
- Break into discrete steps
- Identify dependencies between steps
- Consider edge cases and error handling
- Present the plan:
- Start with a Context section (why this change)
- List concrete steps with file paths
- Include a verification section (how to test)
Rules
- Read before suggesting — never propose changes to code you haven't read
- Prefer editing existing files over creating new ones
- Reuse existing patterns and utilities
- Include file paths and line numbers when referencing code
- Don't over-plan: match complexity to the task
File: src/openharness/skills/bundled/content/review.md
review
Review code for bugs, security issues, and quality.
When to use
Use when the user asks to review code, a PR, or a diff.
Workflow
- Read the changed files or diff thoroughly
- Check for:
- Bugs: logic errors, off-by-one, null/undefined access, race conditions
- Security: injection, XSS, hardcoded secrets, path traversal
- Performance: N+1 queries, unnecessary allocations, missing indexes
- Tests: are new code paths covered? Are edge cases tested?
- Style: naming consistency, dead code, unnecessary complexity
- Provide concrete, actionable feedback with file:line references
- Prioritize findings by severity (critical > major > minor > nit)
Rules
- Be specific: "line 42 may throw if
useris null" not "check for null" - Suggest fixes, don't just point out problems
- Acknowledge good patterns too
- Don't nitpick formatting if there's a linter
File: src/openharness/skills/bundled/content/simplify.md
simplify
Refactor code to be simpler and more maintainable.
When to use
Use when the user asks to simplify, clean up, or refactor code.
Workflow
- Read the target code completely
- Identify complexity sources:
- Unnecessary abstractions or indirection
- Duplicated logic that should be consolidated
- Over-engineered patterns (factories, builders, strategies for simple cases)
- Dead code paths or unused variables
- Propose changes that reduce line count while preserving behavior
- Verify the refactored code passes existing tests
Rules
- Don't add features or change behavior — simplification only
- Prefer deleting code over refactoring it
- Three similar lines are better than a premature abstraction
- Remove backwards-compatibility shims for removed features
- Don't add comments to explain simple code
File: src/openharness/skills/bundled/content/skill-creator.md
name: skill-creator Create, improve, and verify OpenHarness skills. Use this whenever the user asks to create a new skill, convert a workflow into a skill, update an existing SKILL.md, add skills for oh/ohmo, design skill trigger behavior, or test whether a skill loads and works correctly.
skill-creator
Create or improve OpenHarness skills in the directory-based SKILL.md format.
When to use
Use this skill when the user wants to:
- create a new skill from a workflow, repeated task, domain guide, or tool process
- convert existing notes, prompts, or instructions into a reusable skill
- modify or harden an existing skill
- make a skill available to
oh,ohmo, or a plugin - debug why the
skilltool cannot find or load a skill - test trigger wording, skill metadata, or skill behavior
OpenHarness skill locations
Choose the target deliberately:
- Built-in OpenHarness skills live in
src/openharness/skills/bundled/content/*.md. - User skills for
ohlive in~/.openharness/skills/<skill-dir>/SKILL.md. - Private
ohmoskills live in~/.ohmo/skills/<skill-dir>/SKILL.md. - Plugin skills live in
/skills/<skill-dir>/SKILL.md.
OpenHarness user and plugin skills use a directory layout. Do not create flat*.md user skills under skills/; they will not be loaded by the normal loader.
Skill anatomy
Use this structure for user, ohmo, and plugin skills:
my-skill/
├── SKILL.md
├── scripts/ # optional deterministic helpers
├── references/ # optional long docs loaded only when relevant
└── assets/ # optional templates or static filesUse this structure inside SKILL.md:
---
name: my-skill
description: >
What the skill does and exactly when to use it.
---
# my-skill
Short purpose statement.
## When to use
Trigger contexts in plain language.
## Workflow
Concrete steps the agent should follow.
## Rules
Important boundaries, verification, and failure handling.Workflow
Capture intent before writing.
Ask or infer what the skill should help with, when it should trigger, what
output it should produce, and what success looks like. If the conversation
already contains the workflow, extract the sequence from history before
asking more questions.Inspect existing examples.
Read nearby skills and loader behavior before choosing a layout. For
OpenHarness, confirm whether the target is bundled, user, ohmo-private, or
plugin-provided.Write the metadata first.
Thedescriptionis the primary trigger hint shown in the available skills
list. Make it specific and slightly assertive: include both what the skill
does and the phrases or situations that should trigger it. Avoid vague
descriptions like "Helpful notes for X".Keep the body procedural.
Use concise imperative steps. Explain why non-obvious constraints matter.
Prefer general principles over overfitted examples. If a skill is getting
long, move details intoreferences/and point to the relevant file.Add deterministic resources only when they pay for themselves.
If the skill repeatedly needs the same script, template, schema, or checklist,
place it underscripts/,assets/, orreferences/rather than making the
model regenerate it every time.Verify loading.
Run a local loader check for the chosen target. For bundled skills, use:bashPYTHONPATH=src python - <<'PY' from openharness.skills import load_skill_registry for skill in load_skill_registry().list_skills(): print(skill.name, skill.source, skill.path) PYFor ohmo-private skills, include the extra skill directory:
bashPYTHONPATH=src python - <<'PY' from openharness.skills import load_skill_registry from ohmo.workspace import get_skills_dir for skill in load_skill_registry(extra_skill_dirs=[get_skills_dir()]).list_skills(): print(skill.name, skill.source, skill.path) PYTest behavior.
Create two or three realistic prompts that should use the skill. If the skill
changes code or files, add regression tests around loading or command
behavior. Run targeted tests first, then broader tests if the loader or
runtime path changed.Iterate from evidence.
If a skill under-triggers, improve the frontmatter description. If it
triggers but produces poor work, improve the body workflow. If agents repeat
deterministic work, add a helper script or reference file.
Writing rules
- Preserve the skill name when updating an existing skill unless the user
explicitly asks to rename it. - Do not overwrite user-created skills without reading the existing
SKILL.md. - Keep
SKILL.mdunder roughly 500 lines; usereferences/for long material. - Keep trigger guidance in frontmatter
description, not buried only in the
body. - Make skills safe and unsurprising. Do not create skills that hide behavior,
bypass permissions, exfiltrate data, or encourage unauthorized access. - Prefer directory names that are lowercase kebab-case, even if the displayed
skillnameis more human-readable. - Mention exact install or verification paths in the final response.
Test prompts
After drafting a skill, propose a small eval set:
{
"skill_name": "my-skill",
"evals": [
{
"id": "happy-path",
"prompt": "A realistic user request that should trigger the skill.",
"expected_output": "What good behavior looks like."
},
{
"id": "near-miss",
"prompt": "A similar request that should not trigger this skill.",
"expected_output": "The agent should choose a different path."
}
]
}For objective skills, add assertions or tests. For subjective skills, collect
human feedback and revise the workflow from concrete complaints rather than
adding rigid rules.
File: src/openharness/skills/bundled/content/test.md
test
Write and run tests for code.
When to use
Use when the user asks to write tests, verify behavior, or improve test coverage.
Workflow
- Understand what needs testing:
- New feature? Write unit tests for the happy path and edge cases
- Bug fix? Write a regression test that would have caught the bug
- Refactor? Ensure existing tests still pass
- Follow the project's testing patterns:
- Check existing tests for framework (pytest, jest, go test, etc.)
- Match naming conventions and file organization
- Use the same fixtures and helpers
- Write tests that are:
- Independent: each test can run alone
- Deterministic: same result every time
- Fast: mock external services, use in-memory databases
- Run the tests and verify they pass
Rules
- Test behavior, not implementation details
- One assertion per test when possible
- Use descriptive test names that explain the scenario
- Don't test framework or library code
- Mock at system boundaries (external APIs, filesystem, network)
File: .claude/skills/harness-eval/SKILL.md
name: harness-eval version: 0.2.0
Harness Eval — End-to-End Feature Validation
Validate OpenHarness features by running real agent loops against an unfamiliar codebase with actual LLM API calls. Every test exercises the full stack: API client → model → tool calls → execution → result.
Core Principles
- Test on an unfamiliar project — never test on OpenHarness itself (the agent modifies its own code). Clone a real project as the workspace.
- Use real API calls — no mocks. Configure a real LLM endpoint.
- Multi-turn conversations — always test 2+ turns where the model needs prior context.
- Combine features — test hooks+skills+agent loop together, not in isolation.
- Verify tool execution — inspect tool call lists and output files, not just model text.
Workflow
1. Prepare Workspace
Clone an unfamiliar project (do not use OpenHarness):
git clone https://github.com/HKUDS/AutoAgent /tmp/eval-workspace2. Configure Environment
export ANTHROPIC_API_KEY=sk-xxx
export ANTHROPIC_BASE_URL=https://api.moonshot.cn/anthropic # or any provider
export ANTHROPIC_MODEL=kimi-k2.5For long-running real evals, do not artificially lower max_turns. Use the product default (200) unless the user explicitly wants a tighter bound.
3. Prepare Real Sandbox Runtime When Relevant
If the task is validating sandbox behavior, install and verify the actual runtime before running agent loops:
npm install -g @anthropic-ai/sandbox-runtime
sudo apt-get update
sudo apt-get install -y bubblewrap ripgrep
which srt
which bwrap
which rg
srt --versionThen run a minimal smoke check through OpenHarness, not just raw srt, so you verify the real adapter path:
from pathlib import Path
from openharness.config.settings import Settings, SandboxSettings, save_settings
from openharness.tools.bash_tool import BashTool
cfg = Path("/tmp/openharness-sandbox-settings.json")
save_settings(Settings(sandbox=SandboxSettings(enabled=True, fail_if_unavailable=True)), cfg)
# Point config loader at this file, then run BashTool on a tiny command such as `pwd`.If sandbox dependencies are missing, treat that as an environment/setup failure, not a feature regression.
4. Design Tests
Each test follows this pattern:
engine = make_engine(system_prompt="...", cwd=UNFAMILIAR_PROJECT)
evs1 = [ev async for ev in engine.submit_message("Read X, analyze Y")]
r1 = collect(evs1) # text, tools, turns, tokens
evs2 = [ev async for ev in engine.submit_message("Based on what you found...")]
r2 = collect(evs2)
assert "grep" in r1["tools"] # verify tools ranFor detailed code templates and the make_engine/collect helpers, consult references/test-patterns.md.
5. Prefer Long-Horizon, Real Agent Loops
For meaningful end-to-end validation, prefer unfamiliar-repo tasks that force multiple turns, context reuse, and mixed tool usage.
Recommended pattern:
- Use a real external workspace such as
AutoAgent - Use real provider credentials and the actual target model
- Keep
max_turns=200 - Use per-prompt timeouts large enough for real exploration, such as
240-600s - Require at least 2 turns per scenario
- Verify both text quality and tool traces
- Keep polling long-running sessions until they finish; do not abandon a run after the first long pause
Recommended long-horizon scenarios:
architecture_multiturn- Turn 1: map architecture, shell/subprocess surfaces, and test entrypoints
- Turn 2: identify top risks and propose refactors
- Turn 3: condense into onboarding or remediation actions
- Success:
bash,glob,grep,read_fileall appear; no timeout; noMaxTurnsExceeded
hook_block_and_recover- Force the model to try
bash - Block it with a real pre-tool hook
- Verify the model adapts with
glob/grep/read_file
- Force the model to try
sandbox_multiturn- Enable real sandbox settings with
fail_if_unavailable=true - First prompt must start with exactly one shell command such as
pwd && ls -la - Second prompt must explicitly reuse the prior shell findings
- Success:
bashexecutes via sandbox, non-shell tools continue the task, and the agent recovers from incidental repo errors
- Enable real sandbox settings with
When a scenario fails, classify it before changing code:
MaxTurnsExceeded: likely eval harness misconfiguration ifmax_turnswas manually loweredtimeout: task is too broad or per-prompt timeout is too small- sandbox unavailable: environment missing
srt,bwrap, orrg - tool error with task still completed: feature may still be healthy; inspect recovery behavior
6. Run Tests
python tests/test_merged_prs_on_autoagent.py # PR feature tests
python tests/test_real_large_tasks.py # large multi-step tasks
python tests/test_hooks_skills_plugins_real.py # hooks/skills/plugins
python -m pytest tests/ -q -k "not autoagent" # unit tests (no API)For ad hoc long-horizon validation, it is acceptable to run a temporary Python driver script as long as it:
- uses real OpenHarness engine/tool objects
- targets an unfamiliar repository
- prints per-scenario JSON summaries
- records tools, errors, turns, and token usage
- stays attached until completion
7. Interpret Results
| Result | Meaning | Action |
|---|---|---|
| PASS with tool calls | Feature works end-to-end | Done |
| PASS without tool calls | Model answered from knowledge | Rewrite prompt to force tool use |
| FAIL with exception | Code bug | Read traceback |
| FAIL with wrong output | Model behavior issue | Check system prompt and tool schemas |
| Timeout | Task too complex | Increase max_turns or simplify prompt |
For long-running real evals, refine the timeout guidance:
- First check whether
max_turnswas manually set too low - If
max_turns=200and the run still fails, the next suspect is wall-clock timeout, not turn count - Distinguish environment failures from product failures
- Example: missing dependency in the unfamiliar target repo is not automatically an OpenHarness regression
- Example: missing
srt/bwrap/rgis an eval environment issue
Feature Coverage Checklist
- Engine: multi-turn memory, tool chaining, parallel tools, error recovery, auto-compaction
- Swarm: InProcessBackend lifecycle, concurrent teammates, coordinator+notifications
- Hooks: pre_tool_use blocking → model adapts, post_tool_use firing
- Skills: skill tool invocation → model follows instructions
- Plugins: plugin-provided skill loaded and used in agent loop
- Memory: YAML frontmatter parsing, body content search, context injection
- Session: save → load → resume with context preserved
- Providers: Anthropic client, OpenAI client (with reasoning_content), multi-turn
- Cost: token accumulation across turns
Common Pitfalls
- Testing on OpenHarness itself — agent modifies its own running code
- Using mocks — misses serialization and API compatibility bugs
- Single-turn only — misses context accumulation and compaction bugs
- Artificially lowering
max_turnsduring real evals — can create false failures that do not reflect product defaults - Not checking tool call list — model may claim tool use without calling it
- Hardcoding paths — use
WORKSPACEvariable, skip in CI withpytest.mark.skipif - Declaring sandbox “tested” after only checking raw
srt— verify the OpenHarness adapter path too - Abandoning long tasks too early — some real tasks pause for minutes before the next event arrives
Additional Resources
Reference Files
references/test-patterns.md— Complete code templates formake_engine,collect, and each feature categoryreferences/feature-matrix.md— Detailed test cases for every OpenHarness module
Existing Test Files
Working test suites in the repo:
tests/test_merged_prs_on_autoagent.py— PR feature validationtests/test_real_large_tasks.py— Large multi-step taskstests/test_hooks_skills_plugins_real.py— Hooks/skills/plugins in agent loopstests/test_untested_features.py— Module-level integration tests
File: .claude/skills/harness-eval/references/feature-matrix.md
Feature Test Matrix — Detailed Test Cases
Engine & Tools
| Test | What to Verify | Key Assertion |
|---|---|---|
| Multi-turn memory | Set fact turn 1, recall turn 3 | Fact appears in turn 3 response |
| Tool chaining | glob → grep → read in one task | All 3 tools in tool list |
| Write→Edit→Read | Create, modify, verify file | File content matches expected |
| Parallel tools | 3+ tool calls in one response | 3+ tools in single turn |
| Error recovery | Tool fails, model adapts | Alternative tool used after error |
| Auto-compaction | 5+ tasks on shared engine | No context overflow crash |
Swarm & Coordinator
| Test | What to Verify | Key Assertion |
|---|---|---|
| InProcessBackend | spawn → active → status → shutdown | All states transition correctly |
| Concurrent teammates | 2+ agents running simultaneously | Both complete, total time < 2x single |
| Coordinator + notifications | Multi-turn delegation with XML | Coordinator synthesizes worker results |
| Permission sync | request → pending → resolve | Pending count goes to 0 after resolve |
Hooks, Skills, Plugins
| Test | What to Verify | Key Assertion |
|---|---|---|
| Hook blocks → adapt | pre_tool_use blocks bash | "bash" in errors, "glob" in tools |
| Skill invocation | Model calls skill tool | "skill" in tools, content drives next action |
| Plugin skill | Plugin provides skill | Loaded via skill tool, model follows it |
| Hook + skill combined | Hook gates writes, skill guides | Protected file untouched, new file created |
Memory, Session, Config
| Test | What to Verify | Key Assertion |
|---|---|---|
| Memory frontmatter | YAML parsed, not "---" | description != "---", body searchable |
| Session resume | Save → load → continue | Model remembers prior context |
| Cost tracking | Tokens accumulate | in_tokens strictly increasing |
| Cron CRUD | Create, toggle, mark_run, delete | Job count correct at each step |
Provider Compatibility
| Test | What to Verify | Key Assertion |
|---|---|---|
| Anthropic client | Standard tool calling | Tools execute, response coherent |
| OpenAI client | Tool calling + reasoning_content | No 400 error on tool call round-trip |
| OpenAI multi-turn | reasoning_content persists | 3+ turns without API error |
File: .claude/skills/harness-eval/references/test-patterns.md
Test Patterns — Code Templates for Harness Eval
Engine Setup Helpers
import asyncio, sys, os
from pathlib import Path
sys.path.insert(0, "src")
API_KEY = os.environ.get("ANTHROPIC_API_KEY", "your-key")
BASE_URL = os.environ.get("ANTHROPIC_BASE_URL", "https://api.moonshot.cn/anthropic")
OPENAI_BASE = os.environ.get("OPENAI_BASE_URL", "https://api.moonshot.cn/v1")
MODEL = os.environ.get("ANTHROPIC_MODEL", "kimi-k2.5")
WORKSPACE = Path("/tmp/eval-workspace") # unfamiliar project
def make_anthropic_engine(system_prompt, cwd=None, extra_tools=None):
from openharness.api.client import AnthropicApiClient
from openharness.config.settings import PermissionSettings
from openharness.engine.query_engine import QueryEngine
from openharness.permissions.checker import PermissionChecker
from openharness.permissions.modes import PermissionMode
from openharness.tools.base import ToolRegistry
from openharness.tools.bash_tool import BashTool
from openharness.tools.file_read_tool import FileReadTool
from openharness.tools.file_write_tool import FileWriteTool
from openharness.tools.file_edit_tool import FileEditTool
from openharness.tools.glob_tool import GlobTool
from openharness.tools.grep_tool import GrepTool
api = AnthropicApiClient(api_key=API_KEY, base_url=BASE_URL)
reg = ToolRegistry()
for t in [BashTool(), FileReadTool(), FileWriteTool(), FileEditTool(), GlobTool(), GrepTool()]:
reg.register(t)
for t in (extra_tools or []):
reg.register(t)
checker = PermissionChecker(PermissionSettings(mode=PermissionMode.FULL_AUTO))
return QueryEngine(
api_client=api, tool_registry=reg, permission_checker=checker,
cwd=Path(cwd or WORKSPACE), model=MODEL, system_prompt=system_prompt, max_tokens=4096,
)
def make_openai_engine(system_prompt, cwd=None, extra_tools=None):
from openharness.api.openai_client import OpenAICompatibleClient
# Same structure as above, but with:
api = OpenAICompatibleClient(api_key=API_KEY, base_url=OPENAI_BASE)
# ... rest identical
def collect(events):
from openharness.engine.stream_events import (
AssistantTextDelta, AssistantTurnComplete,
ToolExecutionStarted, ToolExecutionCompleted,
)
r = {"text": "", "tools": [], "turns": 0, "in_tok": 0, "out_tok": 0}
for ev in events:
if isinstance(ev, AssistantTextDelta):
r["text"] += ev.text
elif isinstance(ev, ToolExecutionStarted):
r["tools"].append(ev.tool_name)
elif isinstance(ev, AssistantTurnComplete):
r["turns"] += 1
r["in_tok"] += ev.usage.input_tokens
r["out_tok"] += ev.usage.output_tokens
return rMulti-Turn Memory Test
async def test_multi_turn_memory():
engine = make_anthropic_engine("Remember what the user tells you.")
[ev async for ev in engine.submit_message("My favorite number is 42.")]
[ev async for ev in engine.submit_message("What is 2+2?")]
evs = [ev async for ev in engine.submit_message("What is my favorite number?")]
r = collect(evs)
assert "42" in r["text"]Hook Blocks Tool → Model Adapts
async def test_hook_blocks():
from openharness.hooks.events import HookEvent
from openharness.hooks.loader import HookRegistry
from openharness.hooks.schemas import CommandHookDefinition
from openharness.hooks.executor import HookExecutor, HookExecutionContext
hook_reg = HookRegistry()
hook_reg.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(
type="command", command="exit 1",
matcher="bash", block_on_failure=True, timeout_seconds=5,
))
# ... create engine with hook_executor, model tries bash, gets blocked, adapts to globSkill Tool Invocation
async def test_skill_invocation():
from openharness.tools.skill_tool import SkillTool
engine = make_anthropic_engine(
"Use the 'skill' tool to load instructions before working.",
extra_tools=[SkillTool()],
)
evs = [ev async for ev in engine.submit_message(
"Load the 'diagnose' skill, then investigate the codebase."
)]
r = collect(evs)
assert "skill" in r["tools"]InProcess Concurrent Teammates
async def test_concurrent_teammates():
from openharness.swarm.in_process import start_in_process_teammate, TeammateAbortController
from openharness.swarm.types import TeammateSpawnConfig
from openharness.engine.query import QueryContext
async def run_one(name, prompt):
ctx = QueryContext(api_client=api, tool_registry=reg, ...)
config = TeammateSpawnConfig(name=name, team="test", prompt=prompt, ...)
abort = TeammateAbortController()
await start_in_process_teammate(config=config, agent_id=f"{name}@test", abort_controller=abort, query_context=ctx)
await asyncio.gather(
asyncio.wait_for(run_one("worker-a", "Count .py files"), timeout=30),
asyncio.wait_for(run_one("worker-b", "Find main class"), timeout=30),
)Session Save → Resume
async def test_session_resume():
from openharness.services.session_storage import save_session_snapshot, load_session_snapshot
from openharness.engine.messages import ConversationMessage
engine1 = make_anthropic_engine("Remember context.")
[ev async for ev in engine1.submit_message("Project uses FastAPI + React.")]
save_session_snapshot(cwd=tmpdir, model=MODEL, system_prompt="...", messages=engine1.messages, usage=engine1.total_usage)
loaded = load_session_snapshot(tmpdir)
engine2 = make_anthropic_engine("Continue analysis.")
engine2.load_messages([ConversationMessage.model_validate(m) for m in loaded["messages"]])
evs = [ev async for ev in engine2.submit_message("What tech stack did I mention?")]
assert "fastapi" in collect(evs)["text"].lower()File: .claude/skills/pr-merge/SKILL.md
name: pr-merge version: 0.1.0
PR Merge — Contributor-First Pull Request Integration
Merge external pull requests while maximizing original author attribution. Core principle: merge first, resolve conflicts after — never rewrite a contributor's work from scratch.
Core Principles
- Preserve authorship — use
gh pr merge --squashfor clean PRs. For manual merges, use--author="Name ". - Merge first, fix after — accept the PR's approach even if it differs from local style. Fix conflicts in a separate commit.
- Selective merge is OK — exclude files with
--excludewhen a PR contains features already implemented locally. Document what was excluded. - Never
git apply+ self-commit — this loses author attribution entirely.
Workflow
1. Triage Open PRs
gh pr list --repo OWNER/REPO --state open \
--json number,title,author,additions,deletions,mergeableClassify each PR: merge directly, merge with conflict resolution, selective merge, or close.
2. Merge Clean PRs via GitHub
Prefer gh pr merge — preserves author automatically:
gh pr merge NUMBER --repo OWNER/REPO --squash \
--subject "feat: description (#NUMBER)"3. Handle Conflicting PRs Locally
git fetch origin pull/NUMBER/head:pr-NUMBER
git merge pr-NUMBER --no-edit
# Resolve conflicts keeping both sides where possible
git add -A && git commit --no-edit4. Selective Merge (Skip Some Files)
When a PR contains features already implemented locally:
gh pr diff NUMBER --repo OWNER/REPO | \
git apply --exclude='path/to/skip.py' --exclude='CHANGELOG.md'
git commit --author="Author Name <author@email>" \
-m "feat: description (#NUMBER)
Cherry-picked from PR #NUMBER. Excluded: file.py (already implemented)."5. Close Duplicate/Superseded PRs
gh pr close NUMBER --repo OWNER/REPO \
--comment "Fixed via PR #OTHER. Thank you for the contribution!"6. Post-Merge Verification
python -m ruff check src/ tests/ # lint
python -m pytest tests/ -q # unit tests
git push origin main # pushThen run harness-eval for end-to-end verification on an unfamiliar codebase.
Attribution Checklist
Before pushing a merged PR:
- Original author appears in
git log(via--authoror GitHub squash merge) - Commit message references PR number (
#NUMBER) - If selectively merged, commit body explains exclusions
- Closed PRs have a comment thanking the contributor
- Duplicate PRs acknowledge the contributor's investigation
Common Pitfalls
git apply+ self-commit loses author- Rewriting from scratch instead of merging — merge their code, fix style after
- Force-pushing main after merge — may remove contributor commits
- Forgetting CHANGELOG conflicts — always exclude and handle manually
- Not testing after merge — clean merge doesn't mean working code
Additional Resources
Reference Files
references/merge-scenarios.md— Detailed examples for each merge scenario (clean, conflicting, selective, duplicate)
File: .claude/skills/pr-merge/references/merge-scenarios.md
Merge Scenarios — Detailed Examples
Scenario 1: Clean Merge (No Conflicts)
PR adds a new feature, no files overlap with local changes.
# Preferred: squash merge via GitHub (preserves author)
gh pr merge 17 --repo HKUDS/OpenHarness --squash \
--subject "feat(skills): add diagnose skill (#17)"
# Result: author "Qu Zhi" appears in git logScenario 2: CHANGELOG Conflict Only
Almost every PR touches CHANGELOG.md. Handle by excluding it:
gh pr diff 14 --repo OWNER/REPO | \
git apply --exclude='CHANGELOG.md'
git add -A
# Manually merge CHANGELOG sections (keep both)
# Then commit with original author
git commit --author="washi4 <[email protected]>" \
-m "feat: add OpenAI-compatible client (#14)"Scenario 3: Conflicting UI Files
PR #14 modified ui/app.py and ui/backend_host.py which were also changed by PR #13.
# Apply excluding conflicting files
gh pr diff 14 --repo OWNER/REPO | \
git apply --exclude='src/ui/app.py' --exclude='src/ui/backend_host.py' --exclude='CHANGELOG.md'
# Manually add the PR's changes to conflicting files
# Read the PR diff to understand what they added, apply by hand
# Commit with author
git commit --author="Author <email>" -m "feat: description (#14)"Scenario 4: Selective Merge (Skip Features)
PR #16 has auto-compact (already implemented locally) + --resume (wanted) + cron (wanted).
# Exclude the file containing the unwanted feature
gh pr diff 16 --repo OWNER/REPO | git apply \
--exclude='src/engine/query_engine.py' \ # skip auto-compact
--exclude='CHANGELOG.md' \
--exclude='README.md'
# Commit with explanation
git commit --author="Chao Qin <[email protected]>" \
-m "feat: wire --resume/--continue and cron scheduler (#16)
Cherry-picked from PR #16. Excluded auto-compact (already implemented
with LLM-based approach from reference source)."Scenario 5: Duplicate PRs (Same Bug, Different Fix)
PR #11 and #13 both fix the double-Enter bug. #13 is smaller and cleaner.
# Merge #13 (the better fix)
gh pr merge 13 --repo OWNER/REPO --squash
# Close #11 with acknowledgment
gh pr close 11 --repo OWNER/REPO --comment \
"Fixed via PR #13 (smaller patch for the same bug). \
Thank you for the detailed investigation!"Scenario 6: PR From a Fork
The contributor pushed to their fork, not a branch on the repo.
# Fetch via PR ref (works for any PR regardless of source)
git fetch origin pull/14/head:pr-14
git merge pr-14 --no-edit
# If merge conflict:
git mergetool # or manually resolve
git add -A && git commit --no-edit
git push origin mainScenario 7: Batch Merge (Multiple PRs)
Merge in dependency order, test after each:
# 1. Small fixes first (least risk)
gh pr merge 17 --squash # skill file
gh pr merge 13 --squash # 1-file bug fix
# 2. Medium changes
gh pr merge 12 --squash # memory improvement
# 3. Large features last (most conflict risk)
# PR #14 needs manual conflict resolution
git fetch origin pull/14/head:pr-14
git merge pr-14
# resolve CHANGELOG conflict
git push origin main
# 4. Test after all merges
python -m ruff check src/ tests/
python -m pytest tests/ -qPost-Merge Fix Commit Pattern
When the merged PR introduces a compatibility issue (e.g., API mismatch with a provider):
# Fix in a SEPARATE commit (don't amend the author's commit)
git commit -m "fix(api): handle Kimi reasoning_content in OpenAI client
Kimi k2.5 requires reasoning_content on assistant tool_call messages.
Fix: capture during streaming, replay when converting back.
Found during post-merge testing with harness-eval."This preserves the original author's commit intact while documenting the fix separately.
File: .github/PULL_REQUEST_TEMPLATE.md
Summary
- What problem does this PR solve?
- What changed?
Validation
-
uv run ruff check src tests scripts -
uv run pytest -q -
cd frontend/terminal && npx tsc --noEmit(if frontend touched)
Notes
- Related issue:
- Follow-up work:
File: .github/ISSUE_TEMPLATE/bug_report.yml
name: Bug report
title: "[Bug]: "
labels:
- bug
body: - type: markdown
attributes:
value: |
Thanks for reporting a bug. Please include concrete steps, environment details, and error output. - type: textarea
id: summary
attributes:
label: What happened?
validations:
required: true - type: textarea
id: repro
attributes:
label: Steps to reproduce
placeholder: |
1. Runuv run oh ...
2. Trigger ...
3. Observe ...
validations:
required: true - type: textarea
id: environment
attributes:
label: Environment
validations:
required: true - type: textarea
id: logs
attributes:
label: Relevant logs or screenshots
render: shell
File: .github/ISSUE_TEMPLATE/config.yml
blank_issues_enabled: true
contact_links:
- name: Contribution guide
url: https://github.com/HKUDS/OpenHarness/blob/main/CONTRIBUTING.md
about: Read local setup, validation, and PR expectations before contributing. - name: Showcase ideas
url: https://github.com/HKUDS/OpenHarness/blob/main/docs/SHOWCASE.md
about: See example workflows and suggest new real-world use cases.
File: .github/ISSUE_TEMPLATE/feature_request.yml
name: Feature request
title: "[Feature]: "
labels:
- enhancement
body: - type: markdown
attributes:
value: |
Please describe the workflow gap as concretely as possible. Small, scoped requests are easier to prioritize. - type: textarea
id: problem
attributes:
label: What workflow is missing or painful today?
validations:
required: true - type: textarea
id: proposal
attributes:
label: Proposed change
validations:
required: true - type: textarea
id: alternatives
attributes:
label: Alternatives considered - type: textarea
id: extra
attributes:
label: Additional context
File: .github/workflows/autopilot-pages.yml
name: Autopilot Pages
on:
push:
branches:
- main
paths:
- "docs/autopilot/"
- "autopilot-dashboard/"
- ".github/workflows/autopilot-pages.yml"
workflow_dispatch:
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: autopilot-pages
cancel-in-progress: true
jobs:
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
cache-dependency-path: autopilot-dashboard/package-lock.json
- name: Install dependencies
working-directory: autopilot-dashboard
run: npm ci
- name: Build dashboard
working-directory: autopilot-dashboard
run: npm run build
- name: Configure Pages
uses: actions/configure-pages@v5
with:
enablement: true
- name: Upload autopilot dashboard artifact
uses: actions/upload-pages-artifact@v3
with:
path: docs/autopilot
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4File: .github/workflows/autopilot-run-next.yml
name: Autopilot Run Next
on:
schedule:
- cron: "0 */2 * * *"
workflow_dispatch:
permissions:
contents: write
issues: write
pull-requests: write
concurrency:
group: autopilot-run-next
cancel-in-progress: true
jobs:
run-next:
runs-on: self-hosted
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
clean: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Set up uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev
- name: Execute autopilot tick
run: uv run python -m openharness autopilot tick --cwd "$GITHUB_WORKSPACE"
- name: Export dashboard
run: uv run python -m openharness autopilot export-dashboard --cwd "$GITHUB_WORKSPACE"
- name: Commit dashboard updates
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add docs/autopilot
git diff --cached --quiet && exit 0
git commit -m "autopilot: refresh dashboard after run"
git pushFile: .github/workflows/autopilot-scan.yml
name: Autopilot Scan
on:
schedule:
- cron: "*/30 * * * *"
workflow_dispatch:
permissions:
contents: write
issues: write
pull-requests: write
concurrency:
group: autopilot-scan
cancel-in-progress: true
jobs:
scan:
runs-on: self-hosted
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
clean: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Set up uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev
- name: Scan autopilot intake
run: uv run python -m openharness autopilot scan all --cwd "$GITHUB_WORKSPACE"
- name: Export dashboard
run: uv run python -m openharness autopilot export-dashboard --cwd "$GITHUB_WORKSPACE"
- name: Commit dashboard updates
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add docs/autopilot
git diff --cached --quiet && exit 0
git commit -m "autopilot: refresh dashboard after scan"
git pushFile: .github/workflows/ci.yml
name: CI
on:
push:
branches:
- main
pull_request:
jobs:
python-tests:
name: Python tests (${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version:
- "3.10"
- "3.11"
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Set up uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev
- name: Run test suite
run: uv run pytest -q python-quality:
name: Python quality
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Set up uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- name: Install dependencies
run: uv sync --extra dev
- name: Ruff
run: uv run ruff check src tests scripts frontend-typecheck:
name: Frontend typecheck
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontend/terminal
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: npm
cache-dependency-path: frontend/terminal/package-lock.json
- name: Install frontend dependencies
run: npm ci
- name: TypeScript check
run: npx tsc --noEmit