Technical Documentation: tailcallhq/forgecode

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`) Copy Section

⚒️ Forge: AI-Enhanced Terminal Development Environment

Table of Contents

• Quickstart
• Usage Examples
• Why Forge?
• How Forge Works: Three Modes
  • Interactive Mode (TUI)
  • One-Shot CLI Mode
  • ZSH Plugin Mode (: prefix)
• ZSH Plugin: The : Prefix System
  • Agents
  • Sending Prompts
  • Attaching Files
  • Conversation Management
  • Git Integration
  • Shell Command Tools
  • Session & Configuration
  • Skills
  • Customizing Agent Behavior
  • Semantic Search (Workspace)
  • Quick Reference: All : Commands
• Command-Line Options
• Advanced Configuration
  • Provider Configuration
    • Managing Provider Credentials
    • Deprecated: Environment Variables
  • forge.yaml Configuration Options
  • Environment Variables
  • MCP Configuration
  • Example Use Cases
  • Usage in Multi-Agent Workflows
• Documentation
• Community
• Support Us

Quickstart Copy Chapter

To get started with Forge, run the command below:

curl -fsSL https://forgecode.dev/cli | sh

On first run, Forge will guide you through setting up your AI provider credentials using the interactive login flow. Alternatively, you can configure providers beforehand:

# Configure your provider credentials interactively
forge provider login

# Then start Forge
forge

That's it! Forge is now ready to assist you with your development tasks.

Usage Examples Copy Chapter

Forge can be used in different ways depending on your needs. Here are some common usage patterns:

Code Understanding

> Can you explain how the authentication system works in this codebase?

Forge will analyze your project's structure, identify authentication-related files, and provide a detailed explanation of the authentication flow, including the relationships between different components.

Implementing New Features

> I need to add a dark mode toggle to our React application. How should I approach this?

Forge will suggest the best approach based on your current codebase, explain the steps needed, and even scaffold the necessary components and styles for you.

Debugging Assistance

> I'm getting this error: "TypeError: Cannot read property 'map' of undefined". What might be causing it?

Forge will analyze the error, suggest potential causes based on your code, and propose different solutions to fix the issue.

Code Reviews

> Please review the code in src/components/UserProfile.js and suggest improvements

Forge will analyze the code, identify potential issues, and suggest improvements for readability, performance, security, and maintainability.

Learning New Technologies

> I want to integrate GraphQL into this Express application. Can you explain how to get started?

Forge will provide a tailored tutorial on integrating GraphQL with Express, using your specific project structure as context.

Database Schema Design

> I need to design a database schema for a blog with users, posts, comments, and categories

Forge will suggest an appropriate schema design, including tables/collections, relationships, indexes, and constraints based on your project's existing database technology.

Refactoring Legacy Code

> Help me refactor this class-based component to use React Hooks

Forge can help modernize your codebase by walking you through refactoring steps and implementing them with your approval.

Git Operations

> I need to merge branch 'feature/user-profile' into main but there are conflicts

Forge can guide you through resolving git conflicts, explaining the differences and suggesting the best way to reconcile them.

Why Forge? Copy Chapter

Forge is designed for developers who want to enhance their workflow with AI assistance while maintaining full control over their development environment.

• Zero configuration - Just add your API key and you're ready to go
• Seamless integration - Works right in your terminal, where you already work
• Multi-provider support - Use OpenAI, Anthropic, or other LLM providers
• Secure by design - Restricted shell mode limits file system access and prevents unintended changes
• Open-source - Transparent, extensible, and community-driven

Forge helps you code faster, solve complex problems, and learn new technologies without leaving your terminal.

How Forge Works: Three Modes Copy Chapter

Forge has three distinct ways to use it. Understanding this distinction upfront will save you confusion.

Interactive Mode (TUI) Copy Section

Running forge with no arguments starts the interactive terminal UI, a persistent session where you type prompts and the AI responds in a conversational loop. This is the primary way to do multi-step work.

forge                              # Start a new interactive session
forge conversation resume <id>     # Resume a specific saved conversation in interactive mode
forge --conversation-id <id>       # Same: resume conversation by ID
forge --agent <agent-id>           # Start interactive session with a specific agent
forge -C /path/to/project          # Start in a specific directory
forge --sandbox experiment-name    # Create an isolated git worktree + branch, then start there

Once inside interactive mode, type your prompt and press Enter. Forge reads files, writes patches, runs commands, and maintains context across the whole session.

One-Shot CLI Mode Copy Section

Pass -p (or --prompt) to run a single prompt and exit. Forge does the work and returns to your shell. Useful for scripts, piping output, or quick tasks.

forge -p "Explain the purpose of src/main.rs"
forge -p "Add error handling to the parse() function in lib.rs"
echo "What does this do?" | forge    # Pipe input as the prompt
forge commit                         # Generate an AI commit message and commit (exits when done)
forge commit --preview               # Generate commit message, print it, then exit
forge suggest "find large log files" # Translate natural language to a shell command, then exit

Note: forge conversation resume opens the interactive TUI. It does not just print a message and exit. If you run it and see the cursor waiting, you are inside the interactive session. Type your prompt or press Ctrl+C to exit.

ZSH Plugin Mode (`:` prefix) Copy Section

Install the ZSH plugin once with forge setup, then use : commands directly at your shell prompt without ever typing forge. This is the fastest mode for day-to-day development: send prompts, switch conversations, commit, and suggest commands without leaving your shell.

: refactor the auth module      # Send a prompt to the active agent
:commit                         # AI-powered git commit
:suggest "find large log files" # Translate description → shell command in your buffer
:conversation                   # Browse saved conversations with interactive picker

See the full ZSH Plugin reference below for all commands and aliases.

ZSH Plugin: The `:` Prefix System Copy Chapter

When you install the ZSH plugin (forge setup), you get a : prefix command system at your shell prompt. This is the fastest way to use Forge during normal development; you never leave your shell.

How it works: Lines starting with : are intercepted before the shell sees them and routed to Forge. Everything else runs normally.

: <prompt>         # Send a prompt to the active agent
:sage <prompt>     # Send a prompt to a specific agent by name (sage, muse, forge, or any custom agent)
:agent <name>      # Switch the active agent; opens interactive picker if no name given

Agents Copy Section

Forge ships with three built-in agents, each with a different role:

Sending Prompts Copy Section

: refactor the auth module to use the new middleware
:sage how does the caching layer work?    # sage = read-only research agent
:muse design a deployment strategy        # muse = planning agent (writes to plans/)
:ask how does X work?                     # alias for :sage
:plan create a migration plan             # alias for :muse

The agent context persists. Typing :sage alone (no prompt text) switches the active agent to sage for all subsequent : commands.

Attaching Files Copy Section

Type @ in a prompt, then press Tab to fuzzy-search and select files. The path is inserted as @[filename] and attached as context to the AI.

: review this code @[src/auth.rs] @[tests/auth_test.rs]

Conversation Management Copy Section

Forge saves every conversation. You can switch between them like switching directories.

:new                      # Start a fresh conversation (saves current for :conversation -)
:new <initial prompt>     # Start a new conversation and immediately send a prompt
:conversation             # Open interactive picker: browse and switch conversations with preview
:conversation <id>        # Switch directly to a conversation by ID
:conversation -           # Toggle between current and previous conversation (like cd -)
:clone                    # Branch the current conversation (try a different direction)
:clone <id>               # Clone a specific conversation by ID
:rename <name>            # Rename the current conversation
:conversation-rename      # Rename a conversation via interactive picker
:retry                    # Retry the last prompt (useful if the AI misunderstood)
:copy                     # Copy the last AI response to clipboard as markdown
:dump                     # Export conversation as JSON
:dump html                # Export conversation as formatted HTML
:compact                  # Manually compact context to free up token budget

Git Integration Copy Section

:commit                   # AI reads your diff, writes a commit message, and commits immediately
:commit <context>         # Same, but pass extra context: :commit fix typo in readme
:commit-preview           # AI generates the message and puts "git commit -m '...'" in your buffer
                          # so you can review/edit the message before pressing Enter

Shell Command Tools Copy Section

:suggest <description>    # Translate natural language to a shell command and put it in your buffer
:edit                     # Open $EDITOR to compose a complex multi-line prompt, then send it

Session & Configuration Copy Section

Some commands change settings for the current session only. Others persist to your config file (~/forge/.forge.toml). The distinction matters:

# Session-only (reset when you close the terminal; not saved to config)
:model <model-id>              # Change model for this session only
:reasoning-effort <level>      # Set reasoning effort: none/minimal/low/medium/high/xhigh/max
:agent <id>                    # Switch active agent for this session

# Persistent (saved to config file)
:config-model <model-id>       # Set default model globally  (alias: :cm)
:config-provider               # Switch provider globally    (alias: :provider, :p)
:config-reasoning-effort <lvl> # Set default reasoning effort globally (alias: :cre)
:config-commit-model <id>      # Set model used for :commit  (alias: :ccm)
:config-suggest-model <id>     # Set model used for :suggest (alias: :csm)
:config-reload                 # Reset session overrides back to global config (alias: :cr)

# View & edit config
:info                          # Show current session info (model, agent, conversation ID)
:config                        # Display effective resolved configuration in TOML format
:config-edit                   # Open config file in $EDITOR (alias: :ce)
:tools                         # List available tools for the current agent
:skill                         # List available skills

Skills Copy Section

Skills are reusable workflows the AI can invoke as tools. Forge ships three built-in skills:

• create-skill: scaffold a new custom skill
• execute-plan: execute a plan file from plans/
• github-pr-description: generate a PR description from your diff

Use :skill to list available skills. The AI invokes them automatically when relevant, or you can ask explicitly: : generate a PR description using the github-pr-description skill.

Custom skills live in SKILL.md files with YAML front-matter. Precedence (highest first):

Project-local skills override global ones, which override built-in ones. To scaffold a new skill, ask: : create a new skill.

Customizing Agent Behavior Copy Section

AGENTS.md: Create this file in your project root (or ~/forge/AGENTS.md globally) to give all agents persistent instructions such as coding conventions, commit message style, and things to avoid. Forge reads it automatically at the start of every conversation.

Custom agents: Place a .md file with YAML front-matter in .forge/agents/ (project) or ~/forge/agents/ (global) to define additional agents with their own models, tools, and system prompts. Project-local agents override global ones. The built-in agent files in crates/forge_repo/src/agents/ are good examples of the format.

Custom commands: Place YAML files in .forge/commands/ (project) or ~/forge/commands/ (global) to define shortcut commands available via :commandname. Commands can also be defined inline in forge.yaml under the commands: key.

Semantic Search (Workspace) Copy Section

:sync                     # Index your codebase for semantic search
:workspace-init           # Initialize workspace for indexing
:workspace-status         # Show indexing status
:workspace-info           # Show workspace details

After running :sync, the AI can search your codebase by meaning rather than exact text matches. Indexing sends file content to the workspace server, which defaults to https://api.forgecode.dev. Set FORGE_WORKSPACE_SERVER_URL to override this if self-hosting.

Quick Reference: All `:` Commands Copy Section

Command-Line Options Copy Chapter

Here's a quick reference of Forge's command-line options:

Subcommands Copy Section

/* Detailed source-code truncated for AI context efficiency. */

Advanced Configuration Copy Chapter

Provider Configuration Copy Section

Forge supports multiple AI providers. The recommended way to configure providers is using the interactive login command:

forge provider login

This will:

1. Show you a list of available providers
2. Guide you through entering the required credentials

Managing Provider Credentials

# Login to a provider (add or update credentials)
forge provider login

# Remove provider credentials
forge provider logout

# List supported providers
forge provider list

Deprecated: Environment Variables

⚠️ DEPRECATED: Using .env files for provider configuration is deprecated and will be removed in a future version. Please use forge provider login instead.

For backward compatibility, Forge still supports environment variables. On first run, any credentials found in environment variables will be automatically migrated to file-based storage.

Legacy Environment Variable Setup (Deprecated)

OpenRouter

# .env
OPENROUTER_API_KEY=<your_openrouter_api_key>

Requesty

# .env
REQUESTY_API_KEY=<your_requesty_api_key>

x-ai

# .env
XAI_API_KEY=<your_xai_api_key>

z.ai

# .env
ZAI_API_KEY=<your_zai_api_key>

# Or for coding plan subscription
ZAI_CODING_API_KEY=<your_zai_coding_api_key>

Cerebras

# .env
CEREBRAS_API_KEY=<your_cerebras_api_key>

Neuralwatt

# .env
NEURALWATT_API_KEY=<your_neuralwatt_api_key>

OrcaRouter

# .env
ORCAROUTER_API_KEY=<your_orcarouter_api_key>

Meta

# .env
META_API_KEY=<your_meta_model_api_key>

# forge.yaml
model: muse-spark-1.1

IO Intelligence

# .env
IO_INTELLIGENCE_API_KEY=<your_io_intelligence_api_key>

# forge.yaml
model: meta-llama/Llama-3.3-70B-Instruct

OpenAI

# .env
OPENAI_API_KEY=<your_openai_api_key>

# forge.yaml
model: o3-mini-high

Anthropic

# .env
ANTHROPIC_API_KEY=<your_anthropic_api_key>

# forge.yaml
model: claude-3.7-sonnet

Google Vertex AI

Setup Instructions:

1. Install Google Cloud CLI and authenticate:

   bash Copy

   gcloud auth login
   gcloud config set project YOUR_PROJECT_ID

2. Get your authentication token:

   bash Copy

   gcloud auth print-access-token

3. Use the token when logging in via Forge:

   bash Copy

   forge provider login
   # Select Google Vertex AI and enter your credentials

Legacy .env setup:

# .env
PROJECT_ID=<your_project_id>
LOCATION=<your_location>
VERTEX_AI_AUTH_TOKEN=<your_auth_token>

# forge.yaml
model: google/gemini-2.5-pro

Available Models:

• Claude models: claude-sonnet-4@20250514
• Gemini models: gemini-2.5-pro, gemini-2.0-flash

Use the /model command in Forge CLI to see all available models.

OpenAI-Compatible Providers

# .env
OPENAI_API_KEY=<your_provider_api_key>
OPENAI_URL=<your_provider_url>

# forge.yaml
model: <provider-specific-model>

Groq

# .env
OPENAI_API_KEY=<your_groq_api_key>
OPENAI_URL=https://api.groq.com/openai/v1

# forge.yaml
model: deepseek-r1-distill-llama-70b

Amazon Bedrock

To use Amazon Bedrock models with Forge, you'll need to first set up the Bedrock Access Gateway:

1. Set up Bedrock Access Gateway:

   • Follow the deployment steps in the Bedrock Access Gateway repo
   • Create your own API key in Secrets Manager
   • Deploy the CloudFormation stack
   • Note your API Base URL from the CloudFormation outputs

2. Configure in Forge:

   bash Copy

   forge provider login
   # Select OpenAI-compatible provider and enter your Bedrock Gateway details

Legacy .env setup:

# .env
OPENAI_API_KEY=<your_bedrock_gateway_api_key>
OPENAI_URL=<your_bedrock_gateway_base_url>

# forge.yaml
model: anthropic.claude-3-opus

ForgeCode Services

# .env
FORGE_API_KEY=<your_forge_api_key>

# forge.yaml
model: claude-3.7-sonnet

forge.yaml Configuration Options Copy Section

Environment Variables Copy Section

Forge supports several environment variables for advanced configuration and fine-tuning. These can be set in your .env file or system environment.

Retry Configuration

Control how Forge handles retry logic for failed requests:

# .env
FORGE_RETRY_INITIAL_BACKOFF_MS=1000    # Initial backoff time in milliseconds (default: 1000)
FORGE_RETRY_BACKOFF_FACTOR=2           # Multiplier for backoff time (default: 2)
FORGE_RETRY_MAX_ATTEMPTS=3             # Maximum retry attempts (default: 3)
FORGE_SUPPRESS_RETRY_ERRORS=false      # Suppress retry error messages (default: false)
FORGE_RETRY_STATUS_CODES=429,500,502   # HTTP status codes to retry (default: 429,500,502,503,504)

HTTP Configuration

Fine-tune HTTP client behavior for API requests:

# .env
FORGE_HTTP_CONNECT_TIMEOUT=30              # Connection timeout in seconds (default: 30)
FORGE_HTTP_READ_TIMEOUT=900                # Read timeout in seconds (default: 900)
FORGE_HTTP_POOL_IDLE_TIMEOUT=90            # Pool idle timeout in seconds (default: 90)
FORGE_HTTP_POOL_MAX_IDLE_PER_HOST=5        # Max idle connections per host (default: 5)
FORGE_HTTP_MAX_REDIRECTS=10                # Maximum redirects to follow (default: 10)
FORGE_HTTP_USE_HICKORY=false               # Use Hickory DNS resolver (default: false)
FORGE_HTTP_TLS_BACKEND=default             # TLS backend: "default" or "rustls" (default: "default")
FORGE_HTTP_MIN_TLS_VERSION=1.2             # Minimum TLS version: "1.0", "1.1", "1.2", "1.3"
FORGE_HTTP_MAX_TLS_VERSION=1.3             # Maximum TLS version: "1.0", "1.1", "1.2", "1.3"
FORGE_HTTP_ADAPTIVE_WINDOW=true            # Enable HTTP/2 adaptive window (default: true)
FORGE_HTTP_KEEP_ALIVE_INTERVAL=60          # Keep-alive interval in seconds (default: 60, use "none"/"disabled" to disable)
FORGE_HTTP_KEEP_ALIVE_TIMEOUT=10           # Keep-alive timeout in seconds (default: 10)
FORGE_HTTP_KEEP_ALIVE_WHILE_IDLE=true      # Keep-alive while idle (default: true)
FORGE_HTTP_ACCEPT_INVALID_CERTS=false      # Accept invalid certificates (default: false) - USE WITH CAUTION
FORGE_HTTP_ROOT_CERT_PATHS=/path/to/cert1.pem,/path/to/cert2.crt  # Paths to root certificate files (PEM, CRT, CER format), multiple paths separated by commas

⚠️ Security Warning: Setting FORGE_HTTP_ACCEPT_INVALID_CERTS=true disables SSL/TLS certificate verification, which can expose you to man-in-the-middle attacks. Only use this in development environments or when you fully trust the network and endpoints.

API Configuration

Override default API endpoints and provider/model settings:

# .env
FORGE_API_URL=https://api.forgecode.dev  # Custom Forge API URL (default: https://api.forgecode.dev)
FORGE_WORKSPACE_SERVER_URL=http://localhost:8080  # URL for the indexing server (default: https://api.forgecode.dev/)

Tool Configuration

Configuring the tool calls settings:

# .env
FORGE_TOOL_TIMEOUT=300         # Maximum execution time in seconds for a tool before it is terminated to prevent hanging the session. (default: 300)
FORGE_MAX_IMAGE_SIZE=10485760  # Maximum image file size in bytes for read_image operations (default: 10485760 - 10 MB)
FORGE_DUMP_AUTO_OPEN=false     # Automatically open dump files in browser (default: false)
FORGE_DEBUG_REQUESTS=/path/to/debug/requests.json  # Write debug HTTP request files to specified path (supports absolute and relative paths)

ZSH Plugin Configuration

Configure the ZSH plugin behavior:

# .env
FORGE_BIN=forge                    # Command to use for forge operations (default: "forge")

The FORGE_BIN environment variable allows you to customize the command used by the ZSH plugin when transforming : prefixed commands. If not set, it defaults to "forge".

Display Configuration

Configure display options for the Forge UI and ZSH theme:

# .env
FORGE_CURRENCY_SYMBOL="$"         # Currency symbol for cost display in ZSH theme (default: "$")
FORGE_CURRENCY_CONVERSION_RATE=1.0  # Conversion rate for currency display (default: 1.0)
NERD_FONT=1                       # Enable Nerd Font icons in ZSH theme (default: auto-detected, set to "1" or "true" to enable, "0" or "false" to disable)
USE_NERD_FONT=1                   # Alternative variable for enabling Nerd Font icons (same behavior as NERD_FONT)

The FORGE_CURRENCY_SYMBOL and FORGE_CURRENCY_CONVERSION_RATE variables control how costs are displayed in the ZSH theme right prompt. Use these to customize the currency display for your region or preferred currency.

System Configuration

System-level environment variables (usually set automatically):

# .env
FORGE_CONFIG=/custom/config/dir        # Base directory for all Forge config files (default: ~/.forge)
FORGE_MAX_SEARCH_RESULT_BYTES=10240   # Maximum bytes for search results (default: 10240 - 10 KB)
FORGE_HISTORY_FILE=/path/to/history    # Custom path for Forge history file (default: uses system default location)
FORGE_BANNER="Your custom banner text" # Custom banner text to display on startup (default: Forge ASCII art)
FORGE_MAX_CONVERSATIONS=100            # Maximum number of conversations to show in list (default: 100)
FORGE_MAX_LINE_LENGTH=2000             # Maximum characters per line for file read operations (default: 2000)
FORGE_STDOUT_MAX_LINE_LENGTH=2000      # Maximum characters per line for shell output (default: 2000)
SHELL=/bin/zsh                         # Shell to use for command execution (Unix/Linux/macOS)
COMSPEC=cmd.exe                        # Command processor to use (Windows)

Semantic Search Configuration

Configure semantic search behavior for code understanding:

# .env
FORGE_SEM_SEARCH_LIMIT=200            # Maximum number of results to return from initial vector search (default: 200)
FORGE_SEM_SEARCH_TOP_K=20             # Top-k parameter for relevance filtering during semantic search (default: 20)

Logging Configuration

Configure logging verbosity and output:

# .env
FORGE_LOG=forge=info                  # Log filter level (default: forge=debug when tracking disabled, forge=info when tracking enabled)

The FORGE_LOG variable controls the logging level for Forge's internal operations using the standard tracing filter syntax. Common values:

• forge=error - Only errors
• forge=warn - Warnings and errors
• forge=info - Informational messages (default when tracking enabled)
• forge=debug - Debug information (default when tracking disabled)
• forge=trace - Detailed tracing

Tracking Configuration

Control tracking of user-identifying metadata in telemetry events:

# .env
FORGE_TRACKER=false                   # Disable tracking enrichment metadata (default: true)

The FORGE_TRACKER variable controls whether tracking enrichment metadata is included in telemetry events.

The forge.yaml file supports several advanced configuration options that let you customize Forge's behavior.

Custom Rules

Add your own guidelines that all agents should follow when generating responses.

# forge.yaml
custom_rules: |
  1. Always add comprehensive error handling to any code you write.
  2. Include unit tests for all new functions.
  3. Follow our team's naming convention: camelCase for variables, PascalCase for classes.

Commands

Define custom commands as shortcuts for repetitive prompts:

# forge.yaml
commands:
  - name: "refactor"
    description: "Refactor selected code"
    prompt: "Please refactor this code to improve readability and performance"

Model

Specify the default AI model to use for all agents in the workflow.

# forge.yaml
model: "claude-3.7-sonnet"

Max Walker Depth

Control how deeply Forge traverses your project directory structure when gathering context.

# forge.yaml
max_walker_depth: 3 # Limit directory traversal to 3 levels deep

Temperature

Adjust the creativity and randomness in AI responses. Lower values (0.0-0.3) produce more focused, deterministic outputs, while higher values (0.7-2.0) generate more diverse and creative results.

# forge.yaml
temperature: 0.7 # Balanced creativity and focus

Tool Max Failure Limit

Control how many times a tool can fail before Forge forces completion to prevent infinite retry loops. This helps avoid situations where an agent gets stuck repeatedly trying the same failing operation.

# forge.yaml
max_tool_failure_per_turn: 3 # Allow up to 3 failures per tool before forcing completion

Set to a higher value if you want more retry attempts, or lower if you want faster failure detection.

Max Requests Per Turn

Limit the maximum number of requests an agent can make in a single conversation turn. This prevents runaway conversations and helps control API usage and costs.

# forge.yaml
max_requests_per_turn: 50 # Allow up to 50 requests per turn

When this limit is reached, Forge will:

• Ask you if you wish to continue
• If you respond with 'Yes', it will continue the conversation
• If you respond with 'No', it will end the conversation

Model Context Protocol (MCP)

The MCP feature allows AI agents to communicate with external tools and services. This implementation follows Anthropic's Model Context Protocol design.

MCP Configuration Copy Section

Configure MCP servers using the CLI:

# List all MCP servers
forge mcp list

# Import a server from JSON
forge mcp import

# Show server configuration details
forge mcp show

# Remove a server
forge mcp remove

# Reload servers and rebuild caches
forge mcp reload

Or manually create a .mcp.json file with the following structure:

{
  "mcpServers": {
    "server_name": {
      "command": "command_to_execute",
      "args": ["arg1", "arg2"],
      "env": { "ENV_VAR": "value" }
    },
    "another_server": {
      "url": "http://localhost:3000/events"
    }
  }
}

MCP configurations are read from two locations (project-local takes precedence):

1. Project-local: .mcp.json in your project directory
2. Global: ~/forge/.mcp.json

Example Use Cases Copy Section

MCP can be used for various integrations:

• Web browser automation
• External API interactions
• Tool integration
• Custom service connections

Usage in Multi-Agent Workflows Copy Section

MCP tools can be used as part of multi-agent workflows, allowing specialized agents to interact with external systems as part of a collaborative problem-solving approach.

Documentation Copy Chapter

For comprehensive documentation on all features and capabilities, please visit the documentation site.

Installation Copy Chapter

# YOLO
curl -fsSL https://forgecode.dev/cli | sh

# Package managers
nix run github:tailcallhq/forgecode # for latest dev branch

Community Copy Chapter

Join our vibrant Discord community to connect with other Forge users and contributors, get help with your projects, share ideas, and provide feedback!

Support Us Copy Chapter

Your support drives Forge's continued evolution! By starring our GitHub repository, you:

• Help others discover this powerful tool 🔍
• Motivate our development team 💪
• Enable us to prioritize new features 🛠️
• Strengthen our open-source community 🌱

Overview (`benchmarks/README.md`) Copy Section

Forge Code Evaluations

A flexible evaluation framework for running automated tests and benchmarks against Forge Code commands.

Quick Start Copy Chapter

Setup Copy Section

Before running evaluations, create a forgee symlink to the debug binary:

# Create symlink in your PATH (e.g., ~/bin or /usr/local/bin)
ln -sf /path/to/code-forge/target/debug/forge ~/forgee

# Or if ~/bin is in your PATH
ln -sf $(pwd)/target/debug/forge ~/bin/forgee

Why is this needed? Tasks execute in temporary directories, so relative paths like ../../target/debug/forge won't work. The forgee symlink provides a stable absolute path that works from any directory.

Running Evaluations Copy Section

# Run an evaluation
npm run eval ./evals/create_skill/task.yml

# Set custom log level
LOG_LEVEL=debug npm run eval ./evals/create_skill/task.yml

How It Works Copy Chapter

The evaluation system executes commands based on task definitions and validates their output. It supports:

• Parallel execution with configurable concurrency
• Timeout handling for long-running tasks
• Data-driven testing using CSV files
• Output validation using regex patterns
• Debug artifacts stored with timestamps

Project Structure Copy Chapter

benchmarks/
├── cli.ts                    # Main CLI entry point
├── command-generator.ts      # Command template rendering
├── task-executor.ts          # Task execution with timeout support
├── model.ts                  # TypeScript types for tasks
├── parse.ts                  # CLI argument parsing
└── evals/                    # Evaluation definitions
    └── create_skill/
        ├── task.yml          # Task definition
        ├── create_skill_tasks.csv  # Test data
        └── debug/            # Debug outputs (timestamped)

Creating an Evaluation Copy Chapter

1. Create an Evaluation Directory Copy Section

mkdir -p evals/my_eval

2. Create a `task.yml` File Copy Section

The task file defines how to run your evaluation:

# Optional: Commands to run before evaluation starts
before_run:
  - cargo build
  - npm install

# Required: Command(s) to execute for each test case
# Single command
run: forgee -p '{{prompt}}'

# Or multiple commands (executed sequentially)
run:
  - echo "Step 1: {{task}}"
  - forgee -p '{{prompt}}'
  - echo "Step 2: Complete"

# Execution configuration
parallelism: 10  # Number of tasks to run in parallel (default: 1)
timeout: 60      # Timeout in seconds (optional)
early_exit: true # Stop execution when validations pass (optional)

# Optional: Validations to run on output
validations:
  - name: "Check success message"
    type: regex
    regex: \[[0-9:]*\] Skill create-skill

# Required: Data sources for test cases
sources:
  - csv: my_tasks.csv

Task File Schema

before_run (optional): Array of shell commands to execute before running tasks

• Runs sequentially before the main command execution
• Executes in a temporary directory created for the evaluation run
• Useful for building binaries or setting up dependencies

run (required): Command(s) to execute for each test case

• Can be a single string or an array of strings
• Commands support template placeholders (e.g., {{variable}})
• Multiple commands are executed sequentially
• If any command fails, subsequent commands are skipped

parallelism (optional): Number of tasks to run concurrently (default: 1)

timeout (optional): Maximum execution time in seconds per task

early_exit (optional): Stop command execution when all validations pass

validations (optional): Array of validation rules

• name: Human-readable description
• type: Validation type. Supported values:
  • regex: Match output against a regular expression pattern
  • shell: Execute a shell command with output as stdin
• For regex type:
  • regex: Regular expression pattern to match in output
• For shell type:
  • command: Shell command to execute (receives task output via stdin)
  • exit_code: Expected exit code (default: 0)

sources (required): Array of data sources

• Currently supports CSV files: - csv: filename.csv
• Future: Command output: - cmd: command

3. Create Test Data (CSV) Copy Section

Create a CSV file with columns matching your template variables:

prompt,expected_output
"Create a backup script","backup.sh"
"Generate API client","api_client.py"

The column names (e.g., prompt, expected_output) become template variables you can use in your command with {{column_name}}.

4. Run the Evaluation Copy Section

npm run eval ./evals/my_eval/task.yml

Template Variables Copy Chapter

Commands support Handlebars template syntax. Variables are populated from CSV columns:

run:
  command: ./tool --input '{{input_file}}' --format {{format}} --verbose

With CSV:

input_file,format
data1.txt,json
data2.txt,yaml

Output and Debugging Copy Chapter

Debug Artifacts Copy Section

Each run creates a timestamped debug directory:

evals/my_eval/debug/2025-11-23T10-30-45-123Z/
├── task-1.log
├── task-2.log
└── task-3.log

Each log file contains the full output (stdout + stderr) from the command execution.

Task Status Copy Section

Tasks can have four statuses:

• passed: Task completed successfully and passed all validations
• validation_failed: Task completed but failed one or more validations
• timeout: Task exceeded the timeout limit
• failed: Task execution failed (non-zero exit code)

Logging Copy Section

Human-readable output (default):

npm run eval ./evals/my_eval/task.yml

Machine-readable JSON output:

LOG_JSON=1 npm run eval ./evals/my_eval/task.yml | jq .

Debug logging:

LOG_LEVEL=debug npm run eval ./evals/my_eval/task.yml

Examples Copy Chapter

Example 1: Simple Sequential Execution Copy Section

run: echo "Processing {{name}}"
sources:
  - csv: names.csv

name
Alice
Bob
Charlie

Example 2: Parallel Execution with Timeout Copy Section

run: ./slow_task --id {{task_id}}
parallelism: 5
timeout: 30
sources:
  - csv: tasks.csv

Example 3: Multiple Commands Copy Section

run:
  - echo "Starting task {{id}}"
  - ./process --input {{file}}
  - echo "Task {{id}} complete"
parallelism: 3
timeout: 120
sources:
  - csv: tasks.csv

Example 4: Shell Command Validation Copy Section

run: echo "{{message}}"
parallelism: 3
validations:
  # Using grep to check if output contains specific text
  - name: "Contains 'test' word"
    type: shell
    command: grep -q "test"
    exit_code: 0
  
  # Count words and ensure it's greater than 2
  - name: "More than 2 words"
    type: shell
    command: test $(wc -w | awk '{print $1}') -gt 2
    exit_code: 0
  
  # Traditional regex validation (for comparison)
  - name: "Contains test or validation"
    type: regex
    regex: "test|validation"
sources:
  - csv: messages.csv

Example 5: Regex Validation Copy Section

run: cargo test {{test_name}}
validations:
  - name: "All tests passed"
    type: regex
    regex: test result:\s+ok
sources:
  - csv: tests.csv

Tips Copy Chapter

1. Use quotes in commands: When passing CSV values with spaces, wrap them in quotes:

   yaml Copy

   command: forge -p '{{prompt}}'

2. Build before running: Use before_run to ensure binaries are up-to-date:

   yaml Copy

   before_run:
     - cargo build

3. Start with low parallelism: Test with parallelism: 1 first, then increase:

   yaml Copy

   parallelism: 1  # Start here

4. Set appropriate timeouts: Add timeouts to prevent hanging:

   yaml Copy

   timeout: 60  # seconds

5. Check debug logs: When tasks fail, check the debug directory for full output:

   bash Copy

   cat evals/my_eval/debug/*/task-1.log

Exit Codes Copy Chapter

The CLI exits with:

• 0: All tasks passed
• 1: One or more tasks failed (excluding timeouts and validation failures)

Overview (`shell-plugin/README.md`) Copy Section

Forge ZSH Plugin

A powerful ZSH plugin that provides intelligent command transformation, file tagging, and conversation management for the Forge AI assistant.

Features Copy Chapter

• Smart Command Transformation: Convert :command syntax into forge executions
• Agent Selection: Tab completion for available agents using :agent_name
• File Tagging: Interactive file selection with @[filename] syntax
• Syntax Highlighting: Visual feedback for commands and tagged files
• Conversation Continuity: Automatic session management across commands
• Interactive Completion: Fuzzy finding for files and agents via built-in picker

Prerequisites Copy Chapter

Before using this plugin, ensure you have the following tools installed:

• fd - Fast file finder (alternative to find)
• forge - The Forge CLI tool

Installation of Prerequisites Copy Section

# macOS (using Homebrew)
brew install fd

# Ubuntu/Debian
sudo apt install fd-find

# Arch Linux
sudo pacman -S fd

Usage Copy Chapter

Starting a Conversation Copy Section

Begin any command with : followed by your prompt:

: Get the current time

This automatically starts a new conversation with the default Forge agent.

Using Specific Agents Copy Section

Specify an agent by name after the colon:

:sage How does caching work in this system?
:muse Create a deployment strategy for my app

Tab Completion: Type : followed by partial agent name and press TAB for interactive selection.

File Tagging Copy Section

Tag files in your commands using the @[filename] syntax:

: Review this code @[src/main.rs]
: Explain the configuration in @[config.yaml]

Interactive Selection: Type @ and press TAB to search and select files interactively using fuzzy finder.

Conversation Continuity Copy Section

Commands within the same session maintain context:

# First command
: My project uses React and TypeScript

# Second command (remembers previous context)
: How can I optimize the build process?

The plugin automatically manages conversation IDs to maintain context across related commands.

Command Naming Copy Section

Shell commands should follow the Object-Action format.

Examples:

• :provider-login
• :sync-status

For backward compatibility, :login remains available as an alias for :provider-login.

Session Management Copy Section

Starting New Sessions

Clear the current conversation context and start fresh:

:new
# or use the alias
:n

This will:

• Clear the current conversation ID
• Show the banner with helpful information
• Reset the session state
• Display a confirmation message with timestamp

System Information

View system and project information:

:info
# or use the alias
:i

This displays:

• System information

• Project details

• Current configuration

• Current configuration

Switching Conversations

Browse and switch between conversations interactively:

:conversation
# or use the alias
:c

This will display an interactive list of all conversations with preview, allowing you to select and switch.

Switch to a specific conversation by ID:

:conversation <conversation_id>

Toggle between current and previous conversation (like cd -):

:conversation -
# or
:c -

The plugin remembers your previous conversation, allowing you to quickly toggle back and forth. This works just like cd - in your shell, and also works with :new - when you start a new conversation, you can toggle back to your previous one.

If there's no previous conversation tracked (e.g., first time using the plugin), :c - will show the conversation list popup, allowing you to select a conversation.

This is useful when:

• You need to temporarily check another conversation and come back
• You're comparing or referencing information between two conversations
• You want to quickly switch context between related tasks
• You started a new conversation but want to reference the previous one

Cloning Conversations

Create a copy of an existing conversation with interactive selection:

:clone

This will:

• Display an interactive list of all conversations with preview
• Allow you to select a conversation to clone
• Create a new conversation with the same content
• Automatically switch to the cloned conversation
• Show the cloned conversation content and details

You can also clone a specific conversation by providing its ID:

:clone <conversation_id>

This is useful when you want to:

• Create a backup before making significant changes
• Start a new conversation branch from an existing context
• Experiment with different approaches while preserving the original

Session Status

The plugin automatically displays session information including:

• Conversation ID when starting new sessions
• Active agent information
• New session confirmations with timestamps

Syntax Highlighting Copy Chapter

The plugin provides visual feedback through syntax highlighting:

• Tagged Files (@[filename]): Displayed in green bold
• Agent Commands (:agent): Agent names in yellow bold
• Command Text: Remaining text in white bold

Configuration Copy Chapter

Customize the plugin behavior by setting these variables before loading the plugin:

# Custom forge binary location
export FORGE_BIN="/path/to/custom/forge"

Available Configuration Variables Copy Section

• FORGE_BIN: Path to the forge executable (default: forge)
• FORGE_EDITOR: Editor command to use for :edit command (default: $EDITOR or nano)
• FORGE_SYNC_ENABLED: Enable/disable automatic workspace sync (default: true)
• FORGE_MAX_COMMIT_DIFF: Maximum diff size for commit message generation in bytes (default: 100000)
• FORGE_SKIP_INTERACTIVE: Skip interactive prompts (internal use)
• FORGE_CURRENCY_SYMBOL: Currency symbol for cost display in ZSH theme (default: "$")
• FORGE_CURRENCY_CONVERSION_RATE: Conversion rate for currency display (default: 1.0)
• NERD_FONT: Enable Nerd Font icons in ZSH theme (default: auto-detected, set to "1" or "true" to enable, "0" or "false" to disable)
• USE_NERD_FONT: Alternative variable for enabling Nerd Font icons (same behavior as NERD_FONT)
• Internal pattern matching for conversation syntax (:)
• New session command keyword: :new or :n

Codebase Indexing Copy Section

Sync your codebase for semantic search:

:sync

This will index the current directory for semantic code search.

Environment Diagnostics Copy Section

Run comprehensive environment diagnostics to check your Forge setup:

:doctor

This will check:

• ZSH version and terminal information
• Forge installation and version
• Plugin and theme loading status
• Completions availability
• Dependencies (fd, bat)
• ZSH plugins (autosuggestions, syntax-highlighting)
• Editor configuration and PATH setup
• Nerd Font support for icons

.forge Directory Copy Section

The plugin creates a .forge directory in your current working directory (similar to .git) for temporary files:

• FORGE_EDITMSG.md: Temporary file used when opening an external editor with :edit

Advanced Features Copy Chapter

Command History Copy Section

All transformed commands are properly saved to ZSH history, allowing you to:

• Navigate command history with arrow keys
• Search previous forge commands with Ctrl+R
• Reuse complex commands with file tags

Keyboard Shortcuts Copy Section

• Tab: Interactive completion for files (@) and agents (:)
• Enter: Transform and execute :commands
• Ctrl+C: Interrupt running forge commands

Examples Copy Chapter

Basic Usage Copy Section

: What's the weather like?
:sage Explain the MVC pattern
:planner Help me structure this project

With File Tagging Copy Section

: Review this implementation @[src/auth.rs]
: Debug the issue in @[logs/error.log] @[config/app.yml]

Session Flow Copy Section

: I'm working on a Rust web API
: What are the best practices for error handling?
: Show me an example with @[src/errors.rs]
:info
:new
: New conversation starts here

Codebase Indexing Copy Section

# Sync current directory for semantic search
:sync

Overview (`crates/forge_select/README.md`) Copy Section

forge_select

A centralized crate for user interaction prompts using dialoguer.

Purpose Copy Chapter

This crate provides a unified interface for terminal user interactions across the forge codebase. It encapsulates all direct dependencies on dialoguer, ensuring no other crates need to depend on it directly.

Features Copy Chapter

• Select prompts: Choose from a list of options
• Confirm prompts: Yes/no questions
• Input prompts: Text input from user
• Multi-select prompts: Choose multiple options from a list
• Consistent theming: All prompts use a unified color scheme
• Error handling: Graceful handling of user interruptions

Usage Copy Chapter

Select from options Copy Section

use forge_select::ForgeSelect;

let options = vec!["Option 1", "Option 2", "Option 3"];
let selected = ForgeSelect::select("Choose an option:", options)
    .with_starting_cursor(1)
    .prompt()?;

Confirm (yes/no) Copy Section

use forge_select::ForgeSelect;

let confirmed = ForgeSelect::confirm("Are you sure?")
    .with_default(true)
    .prompt()?;

Text input Copy Section

use forge_select::ForgeSelect;

let name = ForgeSelect::input("Enter your name:")
    .allow_empty(false)
    .with_default("John")
    .prompt()?;

Multi-select Copy Section

use forge_select::ForgeSelect;

let options = vec!["Red", "Green", "Blue"];
let selected = ForgeSelect::multi_select("Choose colors:", options)
    .prompt()?;

Design Copy Chapter

Builder Pattern Copy Section

All prompt types use a builder pattern for configuration:

• Create the builder with ForgeSelect::select(), ForgeSelect::confirm(), etc.
• Configure options with .with_*() methods
• Execute with .prompt()

Ownership vs Clone Copy Section

Two variants for select operations:

• select(): Requires Clone for options, useful when you need the list after selection
• select_owned(): Takes ownership, no Clone required, more efficient

Theme Copy Section

All prompts use a consistent ColorfulTheme from dialoguer, providing a unified look across the application.

Integration Copy Chapter

This crate is used by:

• forge_main: For CLI user interactions
• forge_infra: For implementing the UserInfra trait

No other crates should depend on dialoguer directly - use this crate instead.

Overview (`.forge/skills/create-plan/README.md`) Copy Section

Create Plan Skill

Tools and scripts for creating and validating implementation plans.

Files Copy Chapter

• SKILL.md - Main skill instructions for AI agents
• validate-plan.sh - Validates a single plan file
• validate-all-plans.sh - Validates all plans in a directory

Validation Scripts Copy Chapter

validate-plan.sh Copy Section

Validates the structure and content of a single plan file.

Usage:

./.forge/skills/create-plan/validate-plan.sh plans/2025-11-27-example-v1.md

Checks:

• ✓ Filename follows convention: YYYY-MM-DD-task-name-vN.md
  • Year is reasonable (2020 to current year + 1)
  • Month is valid (01-12)
  • Day is valid (01-31)
  • Task name is meaningful (not generic like "test", "task", "temp")
  • Task name length is reasonable (5-60 characters)
  • Version number is reasonable (not > 50)
  • No uppercase letters or underscores (use lowercase and hyphens only)
• ✓ File is in plans/ directory
• ✓ All required sections present:
  • Main heading (# Title)
  • ## Objective
  • ## Implementation Plan
  • ## Verification Criteria
  • ## Potential Risks and Mitigations
  • ## Alternative Approaches
• ✓ Implementation Plan uses checkbox format (- [ ])
• ✓ No numbered lists or plain bullets in Implementation Plan
• ✓ No code blocks (```) in the plan
• ✓ No code snippets (detects suspicious patterns)
• ✓ No placeholder tasks (TODO, TBD, etc.)
• ✓ Task quality and density:
  • Task descriptions are descriptive (≥ 20 characters recommended)
  • Average task length is substantial (30-200 chars recommended)
  • No generic/vague descriptions ("implement feature", "fix bug", etc.)
  • No duplicate or highly similar tasks
  • Consistent and sequential numbering if tasks are numbered
• ✓ Verification criteria have content
• ✓ Risks include mitigations
• ✓ Reasonable number of tasks (3-20)

Exit Codes:

• 0 - Validation passed
• 1 - Validation failed (errors found)

validate-all-plans.sh Copy Section

Validates all plan files in a directory.

Usage:

# Validate all plans in default directory (plans/)
./.forge/skills/create-plan/validate-all-plans.sh

# Validate plans in custom directory
./.forge/skills/create-plan/validate-all-plans.sh path/to/plans

Exit Codes:

• 0 - All plans passed validation
• 1 - One or more plans failed validation

Integration Copy Chapter

Pre-commit Hook Copy Section

Add to .git/hooks/pre-commit:

#!/bin/bash
# Validate plans before committing

if git diff --cached --name-only | grep -q "^plans/.*\.md$"; then
    echo "Validating modified plans..."
    ./.forge/skills/create-plan/validate-all-plans.sh plans/
    exit $?
fi

CI/CD Copy Section

Add to your CI pipeline:

- name: Validate Plans
  run: ./.forge/skills/create-plan/validate-all-plans.sh plans/

Example Valid Plan Copy Chapter

See SKILL.md for the complete plan template structure.

Common Validation Errors Copy Chapter

1. Invalid filename format: Must follow YYYY-MM-DD-task-name-vN.md pattern
   • Use lowercase letters only
   • Use hyphens (not underscores) to separate words
   • Use valid date (month 01-12, day 01-31, year 2020+)
   • Avoid generic names like "test", "task", "temp"
2. Missing checkboxes: Use - [ ] not 1. or -
3. Code blocks: Plans should use natural language, not code
4. Missing sections: All required sections must be present
5. Empty sections: Sections should have meaningful content
6. Poor task quality: Tasks should be descriptive and specific
   • Avoid short descriptions like "Do this", "Fix that", "Update code"
   • Avoid generic descriptions like "implement feature", "add functionality"
   • Include rationale and context in task descriptions
   • Aim for 30-150 characters per task description

Overview (`.github/scripts/bounty/README.md`) Copy Section

Bounty Automation

Automates the full lifecycle of issue bounties — from label propagation when a PR is opened, through claiming when work begins, to rewarding when a PR is merged.

Flow Copy Chapter

Issue created
└── maintainer adds  bounty: $N  label
        │
        ├── parse-sync-generic-bounty.ts  →  adds generic  bounty  label
        │
        ▼
Issue assigned to contributor
└── parse-sync-claimed.ts  →  adds  bounty: claimed
        │
        ▼
Contributor opens PR with "Closes #N" / "Fixes #N" / "Resolves #N"
└── parse-propagate-label.ts  →  copies  bounty: $N  to PR
                              →  posts comment on issue: "PR #X opened by @author"
        │
        ▼
PR is merged
└── parse-mark-rewarded.ts  →  adds  bounty: rewarded  to PR
                            →  adds  bounty: rewarded  to linked issue(s)
                            →  removes  bounty: claimed  from linked issue(s)
        │
        ▼
Bounty lifecycle complete

Removal path:
maintainer removes last  bounty: $N  label
└── parse-sync-generic-bounty.ts  →  removes generic  bounty  label

Labels Copy Chapter

Bounty values follow the Fibonacci sequence: $100, $200, $300, $500, $800, $1300, $2100, $3400, $5500.

Pipeline Copy Chapter

Each workflow job runs a single Node process (run.ts) that executes all three stages in memory — no temp files, no shell piping:

parse  →  ParsedIntent (in memory)  →  plan  →  BatchPlan (in memory)  →  execute

Stage 1 — parse (parse-*.ts): Reads the GitHub Actions event payload from GITHUB_EVENT_PATH and returns a ParsedIntent object. Pure — makes no API calls.

Stage 2 — plan (plan.ts): Receives the ParsedIntent, fetches current labels only for targets not already known from the event payload, diffs desired vs actual state, and returns a BatchPlan. Minimises API calls by reusing label data already present in the event.

Stage 3 — execute (execute.ts): Receives the BatchPlan and applies all mutations. Label additions per target are sent as a single batched POST /labels call. Each removal is a separate DELETE (GitHub has no bulk-remove endpoint). Comments are posted last.

This design means:

• The parse stage is trivially unit-testable (pure function, no mocks needed).
• The plan stage is testable with a minimal mock that only needs getLabels.
• The execute stage is testable with a mock that tracks calls — no HTTP.
• API calls are minimised: additions are batched per target; label state already in the event payload is never re-fetched.

Scripts Copy Chapter

The workflow invokes a single orchestrator script (run.ts) per job. The parse, plan, and execute modules are called directly in the same Node process, passing objects in memory.

`run.ts` Copy Section

The entrypoint used by bounty.yml. Accepts a --script flag to select which parse module to run, then calls plan and execute in sequence — all in-process.

npx tsx .github/scripts/bounty/run.ts \
  --script  \
  --repo <owner/repo> \
  --token <github-token> \
  [--pr <number> | --issue <number>]

`parse-propagate-label.ts` Copy Section

Triggered by: pull_request — opened, edited, reopened.

1. Parses the PR body for closing keywords (closes, fixes, resolves, case-insensitive).
2. Returns a ParsedIntent with a labelCopies field — the plan stage fetches each linked issue's labels and copies any bounty: $N ones onto the PR.
3. Includes a comment mutation per linked issue (the plan stage drops it if the issue has no bounty labels).

`parse-sync-claimed.ts` Copy Section

Triggered by: issues — assigned, unassigned.

• assigned: returns add: ["bounty: claimed"] if the issue has a bounty: $N label.
• unassigned: returns remove: ["bounty: claimed"] only when no assignees remain.
• Issue labels are already in the event payload and supplied as knownLabels — no extra fetch.

`parse-sync-generic-bounty.ts` Copy Section

Triggered by: issues — labeled, unlabeled.

Keeps the generic bounty label in sync with value labels. Inspects event.label (the label that just changed) and only acts when it matches bounty: $.

• labeled: returns add: ["bounty"].
• unlabeled: returns remove: ["bounty"] only when no value labels remain (guards against mid-tier-swap removal when a maintainer swaps one value label for another).
• Issue labels from the event are supplied as knownLabels — no extra fetch.

`parse-mark-rewarded.ts` Copy Section

Triggered by: pull_request_target — closed (merged only).

1. Returns empty intent if the PR was not merged or has no bounty: $N label.
2. Returns add: ["bounty: rewarded"] for the PR (labels known from event, no fetch).
3. Parses the PR body for linked issues; returns add: ["bounty: rewarded"], remove: ["bounty: claimed"] for each (plan stage fetches their labels).

Uses pull_request_target so the job has write access to issues and PRs from forks.

`plan.ts` Copy Section

Receives a ParsedIntent from the parse stage. Resolves labelCopies by fetching source issue labels. Fetches current labels for any target not already in knownLabels. Filters out no-op adds and removes. Returns a BatchPlan.

`execute.ts` Copy Section

Receives a BatchPlan from the plan stage. For each mutation: one batched POST for all additions, one DELETE per removal, one POST per comment.

Shared Module Copy Chapter

github-api.ts defines:

• Event payload types (PullRequestEvent, IssuesEvent)
• Pipeline types (ParsedIntent, BatchPlan, TargetMutation)
• The GitHubApi interface (injectable for testing)
• GitHubRestApi — the production implementation using node:https

All scripts import types from github-api.ts and accept a GitHubApi instance in their run() / plan() / execute() function signature, making every step independently mockable.

Tests Copy Chapter

Unit tests live alongside each script (*.test.ts) and use Node's built-in node:test runner.

npm run test:bounty

• Parse tests: pure — no mock needed, just call parse() with a synthetic event.
• Plan and execute tests: use a mock GitHubApi that tracks calls and returns preset label lists.
• The CLI entrypoint in each script (yargs parsing + GITHUB_EVENT_PATH read) is guarded behind an import.meta.url check so it does not execute on import.

Workflow Source Copy Chapter

bounty.yml is auto-generated from Rust source in crates/forge_ci. Do not edit it by hand — modify crates/forge_ci/src/workflows/bounty.rs and regenerate with:

cargo test -p forge_ci

Overview (`benchmarks/evals/sem_search/README.md`) Copy Section

Semantic Search Tool Selection Evaluation

This evaluation validates that the Forge agent correctly identifies and uses the sem_search (Codebase Search) tool when presented with conceptual, functionality-based code location queries.

What We're Testing Copy Chapter

The agent's ability to recognize when a user query requires semantic understanding rather than exact pattern matching. Specifically, we test whether the agent:

1. Correctly invokes sem_search for conceptual queries about code functionality
2. Understands the distinction between semantic search (conceptual) vs regex search (exact patterns)
3. Recognizes appropriate use cases such as:
   • Finding code by its purpose/behavior (e.g., "retry logic with exponential backoff")
   • Locating implementation patterns (e.g., "authentication token validation")
   • Discovering architectural components (e.g., "message transformation between AI providers")
   • Identifying system behaviors (e.g., "rate limiting for API requests")

Test Scenarios Copy Chapter

The evaluation uses real-world queries that describe:

• Message transformation between different AI provider formats
• Retry mechanisms with exponential backoff patterns
• Authentication flows including token validation and refresh
• Error handling for network failures
• Caching strategies for API responses
• Streaming response handling from LLM APIs
• File validation including upload and size checks
• Rate limiting implementations for API requests
• Tool registration and availability management
• Context management including conversation history truncation

Expected Behavior Copy Chapter

For all test queries, the agent should invoke the sem_search tool (displayed as "Codebase Search" in logs) rather than falling back to regex-based search tool, demonstrating understanding that these are conceptual queries requiring semantic code comprehension.

Overview (`benchmarks/evals/semantic_search_quality/README.md`) Copy Section

Semantic Search Quality Evaluation

This evaluation measures the quality of semantic search queries generated by Forge using an LLM-as-judge approach.

Quick Start Copy Chapter

Test Your Queries Manually Copy Section

Before running the full eval, test your queries to get immediate feedback:

# Interactive mode
npx tsx benchmarks/evals/semantic_search_quality/test_queries.ts

# Non-interactive mode
npx tsx benchmarks/evals/semantic_search_quality/test_queries.ts \
  "retry mechanism with exponential backoff" \
  "I need to find the implementation code for retry logic"

The query tester provides:

• Detailed scores for embedding and reranking queries
• Specific strengths and weaknesses
• Actionable suggestions for improvement
• Differentiation analysis

Run the Full Evaluation Copy Section

# Run the entire semantic search quality eval
npm run eval semantic_search_quality

# Run with specific parallelism
npm run eval semantic_search_quality -- --parallelism 10

# Run with specific model
npm run eval semantic_search_quality -- --model "anthropic/claude-sonnet-4.5"

Purpose Copy Chapter

The semantic search feature uses two types of queries:

1. Embedding Query (query): Converted to vector embeddings for semantic similarity search
2. Reranking Query (use_case): Used by the reranker to filter results based on intent

This eval verifies that:

• Queries are well-formed and contain sufficient context
• The two queries are different and serve distinct purposes
• Results match the user's intent (implementation vs docs vs tests, etc.)
• Correct file types are returned based on intent
• Unwanted file types are avoided

How It Works Copy Chapter

1. Test Execution Copy Section

The eval runs various semantic search tasks covering different intents:

• Implementation: Find actual code, not documentation
• Flow Understanding: Understand how systems work
• Architecture: Understand structural design
• Tests: Find test examples and fixtures
• Documentation: Find docs and guides
• Debugging: Investigate issues
• Modification: Find code to modify
• Configuration: Find config files and schemas

2. LLM Judge Evaluation Copy Section

After each task, an LLM judge (Gemini 3 Pro) evaluates:

Query Quality (40 points)

• Embedding Query (15pts): Domain terms, technical context, behavior description
• Reranking Query (15pts): Intent clarity, context about WHY, specificity
• Query Differentiation (10pts): Queries serve different purposes

Result Relevance (60 points)

• Intent Matching (25pts): Results match the stated intent
• File Type Accuracy (20pts): Correct file types returned
• Avoidance Compliance (15pts): Unwanted file types avoided

Pass Criteria

• Total score >= 70/100
• No critical issues (e.g., identical queries, severe intent mismatch)

3. Structured Output Copy Section

The judge uses structured output (Zod schema) to provide:

• Detailed scores for each dimension
• Specific feedback on what's good/bad
• List of critical issues
• Overall pass/fail determination

Environment Variables Copy Chapter

Required:

• GOOGLE_APPLICATION_CREDENTIALS: Path to Google Cloud service account JSON credentials file

Or authenticate using gcloud auth application-default login.

The LLM judge uses Vertex AI to run Gemini 3 Pro. It supports Google's Application Default Credentials through the google-auth-library. The most common authentication method is to set the path to a JSON credentials file in the GOOGLE_APPLICATION_CREDENTIALS environment variable.

Test Cases Copy Chapter

The eval includes diverse test cases:

Implementation Queries Copy Section

- task: "Find the retry mechanism with exponential backoff implementation"
  intent: "implementation"
  expected_file_types: "rust,typescript,javascript"
  should_avoid: "markdown,txt,documentation"

Flow Understanding Copy Section

- task: "How does the authentication flow work from request to response?"
  intent: "flow_understanding"
  expected_file_types: "rust,typescript"
  should_avoid: "test,markdown"

Documentation Copy Section

- task: "Find documentation on how to configure semantic search"
  intent: "documentation"
  expected_file_types: "markdown,txt"
  should_avoid: "rust,typescript,javascript"

Interpreting Results Copy Chapter

High Scores (>80) Copy Section

Queries are excellent - domain-specific, well-differentiated, and would return highly relevant results.

Medium Scores (60-80) Copy Section

Queries are functional but have room for improvement. Check feedback for specific suggestions.

Low Scores (<60) Copy Section

Queries have significant issues:

• Too generic or vague
• Embedding and reranking queries too similar
• Intent mismatch
• Would return wrong file types

Common Issues Copy Chapter

1. Identical Queries: Embedding and reranking queries are the same or too similar

   • Fix: Make reranking query focus on intent, not just rephrasing

2. Generic Queries: Lack domain-specific context

   • Fix: Add technical terms, describe behavior, include framework names

3. Intent Mismatch: Queries would return docs when implementation is needed

   • Fix: Be explicit in reranking query about what type of code is needed

4. Poor Differentiation: Both queries try to do the same thing

   • Fix: Embedding = WHAT code does, Reranking = WHY you need it

Architecture Copy Chapter

task.yml
├── Runs Forge with semantic search task
├── Captures debug output (context.json)
└── Validates with llm_judge.ts
    ├── Extracts sem_search calls
    ├── Calls Gemini 3 Pro with structured output
    └── Returns evaluation with scores

Extending the Eval Copy Chapter

To add new test cases:

1. Add to task.yml under sources:

- task: "Your task description"
  intent: "implementation|flow_understanding|tests|documentation|debugging|modification|architecture|configuration"
  expected_file_types: "comma,separated,extensions"
  should_avoid: "comma,separated,extensions"

2. Choose appropriate intent:
   • implementation: Actual code implementation
   • flow_understanding: Understanding how things work
   • architecture: Structural/design understanding
   • tests: Test files and fixtures
   • documentation: Docs, README, guides
   • debugging: Finding bugs/issues
   • modification: Code to modify/extend
   • configuration: Config files, schemas

Future Improvements Copy Chapter

• Add actual result verification (not just query quality)
• Measure embedding quality directly
• Test reranking effectiveness with real results
• Add precision/recall metrics
• Compare against human-labeled ground truth
• Test edge cases (very short queries, multi-lingual code, etc.)

Overview (`.forge/skills/debug-cli/scripts/README.md`) Copy Section

Scripts

This directory contains helper scripts for CLI debugging and testing.

test_cli.sh Copy Chapter

Basic smoke test script that:

• Builds the forge CLI
• Tests main help command
• Tests version command
• Tests help for various subcommands

Usage:

./scripts/test_cli.sh

Tool Guidelines (`docs/tool-guidelines.md`) Copy Section

Claude Tool Description Best Practices

Key Principles Copy Chapter

When defining tools for Claude to use, the quality of your tool descriptions dramatically affects performance. Here are the most important best practices extracted from Anthropic's documentation:

1. Provide Extremely Detailed Descriptions Copy Section

This is by far the most important factor in tool performance. Your descriptions should explain every detail about the tool, including:

• What the tool does - Be explicit about the tool's primary function and purpose
• When it should be used - Specify the scenarios where this tool is appropriate
• When it should NOT be used - Clarify scenarios where the tool is not suitable
• What each parameter means - Explain how each parameter affects the tool's behavior
• Limitations and caveats - Note what information the tool does not return or handle
• Additional context - If the tool name is unclear, provide extra clarification

2. Use Comprehensive But Concise Descriptions Copy Section

The more context you can give Claude about your tools, the better it will be at deciding when and how to use them. Aim for 3-4 sentences per tool description, or more if the tool is complex.

IMPORTANT: Tool descriptions must never exceed 1024 characters. This is enforced by tests to ensure compatibility with LLM API constraints.

3. Prioritize Descriptions Over Examples Copy Section

While you can include examples of how to use a tool in its description or in the accompanying prompt, this is less important than having a clear and comprehensive explanation of the tool's purpose and parameters. Only add examples after you've fully fleshed out the description.

4. Register All Tools in the Registry Copy Section

Every tool must be registered in the crates/forge_services/src/tools/registry.rs file to be available for use. The ToolRegistry::tools() method returns all available tools configured with the given infrastructure.

Example Comparison Copy Chapter

Good Tool Description Example: Copy Section

{
  "name": "get_stock_price",
  "description": "Retrieves the current stock price for a given ticker symbol. The ticker symbol must be a valid symbol for a publicly traded company on a major US stock exchange like NYSE or NASDAQ. The tool will return the latest trade price in USD. It should be used when the user asks about the current or most recent price of a specific stock. It will not provide any other information about the stock or company.",
  "input_schema": {
    "type": "object",
    "properties": {
      "ticker": {
        "type": "string",
        "description": "The stock ticker symbol, e.g. AAPL for Apple Inc."
      }
    },
    "required": ["ticker"]
  }
}

Poor Tool Description Example: Copy Section

{
  "name": "get_stock_price",
  "description": "Gets the stock price for a ticker.",
  "input_schema": {
    "type": "object",
    "properties": {
      "ticker": {
        "type": "string"
      }
    },
    "required": ["ticker"]
  }
}

Why the Good Description Works Better Copy Chapter

The good description clearly explains:

• What the tool does (retrieves the current stock price)
• What data format it returns (latest trade price in USD)
• When to use it (when the user asks about current/recent price)
• What limitations it has (won't provide other company info)
• What the parameter means (ticker symbol for publicly traded companies)

The poor description is too brief and leaves Claude with many open questions about the tool's behavior and usage. It doesn't explain what kind of data is returned, what format it's in, when to use it, or what parameters are expected.

Practical Tips Copy Chapter

1. Be specific about data formats - Explain what format the data will be returned in
2. Clarify parameter constraints - Note any restrictions on parameter values
3. Explain tool selection criteria - Help Claude understand when to pick this tool over others
4. Describe error handling - Note how the tool behaves with invalid inputs
5. Include domain-specific details - Add contextual information related to the tool's domain
6. Keep descriptions under 1024 characters - Ensure compatibility with LLM API constraints
7. Register your tool in the registry - Add your tool to the registry.rs file for availability

Thorough yet concise tool descriptions lead to more accurate tool selection, fewer clarification questions, and better overall performance when using Claude with tools.

Source: Anthropic Documentation on Tool Use

--- METRICS ---

• Files Extracted: 10
• Estimated Token Budget: ~19987 tokens
• Canonical Reference: https://codewiki.google/github.com/tailcallhq/forgecode