eigent Documentation

Eigent: The Open Source Cowork Desktop - Local and Free Alternative to Claude Cowork and Codex

RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy eigent on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run)
docker run -d --name eigent -p 5432:5432 --restart always postgres:15
Configured Services & Ports
βš™οΈ postgres
image: postgres:15
πŸ”Œ Ports: 5432:5432
βš™οΈ redis
image: redis:7-alpine
πŸ”Œ Ports: 6379:6379
docker-compose.yml
yaml (docker-compose.yml)
services:
  # PostgreSQL Database
  postgres:
    image: postgres:15
    container_name: eigent_postgres
    restart: unless-stopped
    environment:
      POSTGRES_DB: eigent
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: 123456
      POSTGRES_INITDB_ARGS: '--encoding=UTF-8 --lc-collate=C --lc-ctype=C'
    ports:
      - '5432:5432'
    volumes:
      - postgres_data:/var/lib/postgresql/data
    networks:
      - eigent_network
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U postgres -d eigent']
      interval: 10s
      timeout: 5s
      retries: 5

  # Redis
  redis:
    image: redis:7-alpine
    container_name: eigent-redis-dev
    restart: unless-stopped
    ports:
      - '6379:6379'
    volumes:
      - eigent_redis_data:/data
    networks:
      - eigent_network
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  postgres_data:
    driver: local
  eigent_redis_data:

networks:
  eigent_network:
    driver: bridge

File: README.md


EigentΒ is the open source Cowork desktop application, empowering you to build, manage, and deploy a custom AI workforce that can turn your most complex workflows into automated tasks. As a leading open-source Cowork product, Eigent brings together the best of open-source collaboration and AI-powered automation.

Built on CAMEL-AI's acclaimed open-source project, our system introduces a Multi-Agent Workforce that boosts productivity through parallel execution, customization, and privacy protection.

⭐ 100% Open Source - πŸ₯‡ Local Deployment - πŸ† MCP Integration

  • βœ… Zero Setup - No technical configuration required
  • βœ… Multi-Agent Coordination - Handle complex multi-agent workflows
  • βœ… Single-Agent Harness - Run focused tasks with one meta agent
  • βœ… Local Deployment
  • βœ… Open Source
  • βœ… Model Agnostic - Support any model of your choice
  • βœ… MCP Integration
  • βœ… Skill Integration
  • βœ… Built-in Browser & Terminal Toolkits
  • βœ… Enterprise Features - SSO, access control, and custom enquiries

Table of contents

TOC

**πŸš€ Getting Started with Open Source Cowork**

πŸ”“ Build in Public β€” Eigent is 100% open source from day one. Every feature, every commit, every decision is transparent. We believe the best AI tools should be built openly with the community, not behind closed doors.

The recommended way to run Eigent β€” fully standalone with complete control over your data, no cloud account required.

πŸ‘‰ Full Local Deployment Guide

This setup includes:

  • Local backend server with full API
  • Local model integration (vLLM, Ollama, LM Studio, etc.)
  • Complete isolation from cloud services
  • Zero external dependencies

⚑ Quick Start (Cloud-Connected)

For a quick preview using our cloud backend β€” get started in seconds:

Prerequisites

  • Node.js (version 18-22) and npm

Steps

bash
git clone https://github.com/eigent-ai/eigent.git
cd eigent
npm install
npm run dev

Note: This mode connects to Eigent cloud services and requires account registration. For a fully standalone experience, use Local Deployment instead.

Updating Dependencies

After pulling new code (git pull), update both frontend and backend dependencies:

bash
# 1. Update frontend dependencies (in project root)
npm install

# 2. Update backend/Python dependencies (in backend directory)
cd backend
uv sync

🏒 Enterprise

For organizations requiring maximum security, customization, and control:

  • Exclusive Features (like SSO & custom development)
  • Scalable Enterprise Deployment
  • Negotiated SLAs & implementation services

πŸ“§ For further details, contact our sales team.

☁️ Cloud Version

For teams who prefer managed infrastructure, we also offer a cloud platform. The fastest way to experience Eigent's multi-agent AI capabilities without setup complexity. We'll host the models, APIs, and cloud storage, ensuring Eigent runs flawlessly.

  • Instant Access - Start building multi-agent workflows in minutes.
  • Managed Infrastructure - We handle scaling, updates, and maintenance.
  • Premium Support - Subscribe and get priority assistance from our engineering team.

image-public-beta

Get started at Eigent.ai β†’

**✨ Key features - Open Source Cowork**

πŸ§‘β€πŸ’» Cowork with Single Agent

Start with one focused agent for direct tasks. Research, write, debug, and operate alongside it in your desktop workspace.

🏭 Cowork with Workforce

Scale to multiple specialized agents that divide work, collaborate in parallel, and execute complex multi-step workflows together.

⏰ Automation

Schedule recurring workflows and let agents run tasks at the right timeβ€”so work continues even when you step away.

πŸ”’ Local & Secure

Run agents on your machine with local-first execution. Your files, credentials, and context stay under your control.

🧠 Model Agnostic

Connect the models you already useβ€”cloud APIs, enterprise gateways, or local inferenceβ€”without locking into one vendor.

πŸ‘ 100% Open Source

Eigent is completely open-sourced. You can download, inspect, and modify the code, ensuring transparency and fostering a community-driven ecosystem for multi-agent innovation.

🧩 Use Cases - Open Source Cowork

Explore how Eigent turns complex desktop work into repeatable agent workflows.

For Developers

[Build 10 Chinese New Year HTML5 Games with Eigent](https://www.eigent.ai/use-cases/build-10-cny-horse-themed-html5-games)

Coordinate parallel agents to build ten polished, mobile-friendly browser games across genres, complete with scoring, increasing difficulty, and restart flows.

View demo β†’

View guide β†’

[Build a 3D Snow Bros Platformer with Gemini 3.1 Pro](https://www.eigent.ai/use-cases/build-3d-snow-bros-platformer-gemini)

Create a complete browser-based 3D platformer with snowball combat, enemy chains, scoring, lives, scaling difficulty, and layered environments.

View demo β†’

View guide β†’

[Automate Monthly Dev Reports with DeepSeek via Ollama](https://www.eigent.ai/use-cases/monthly-dev-reports-automated-eigent-with-deepseek-v4-pro-via-ollama)

Review a month of GitHub pull requests with a locally hosted model, generate a Word summary, and prepare the corresponding Slack release update.

View demo β†’

View guide β†’

[Organize Desktop Files](https://www.eigent.ai/use-cases/organize-desktop-files)

Ask Eigent to inspect a cluttered desktop and organize files into a cleaner, more useful structure directly on your machine.

View demo β†’

View guide β†’

[Audit ML CI Failures with Gemini 3.5 Flash on Eigent](https://www.eigent.ai/use-cases/eigent-gemini-managed-agents)

Orchestrate a multi-agent CI investigation that fetches logs, compares golden values, traces evidence, delegates deep reasoning, and produces structured audit reports.

View demo β†’

View guide β†’

[Ticket Management System Integration and Reporting](https://www.eigent.ai/use-cases/ticket-management-system-integration-and-reporting)

Import local ticket data into a browser-based management system, then generate a statistical report with charts and visual summaries.

View demo β†’

View guide β†’

Explore more use cases β†’

πŸ› οΈ Tech Stack

Eigent open-source Cowork desktop is built on modern, reliable technologies that ensure scalability, performance, and extensibility.

Backend

  • Framework:Β FastAPI
  • Package Manager:Β uv
  • Async Server:Β Uvicorn
  • Authentication:Β OAuth 2.0, Passlib.
  • Multi-agent framework: CAMEL

Frontend

  • Framework:Β React
  • Desktop App Framework:Β Electron
  • Language:Β TypeScript
  • UI:Β Tailwind CSS, Radix UI, Lucide React, Framer Motion
  • State Management:Β Zustand
  • Flow Editor:Β React Flow

🌟 Staying ahead - Open Source Cowork

[!IMPORTANT]

Star Eigent, You will receive all release notifications from GitHub without any delay ~ ⭐️

πŸ—ΊοΈ Roadmap - Open Source Cowork

Our open-source Cowork continues to evolve with input from the community. Here's what's coming next:

Topics Issues Discord Channel
Context Engineering - Prompt caching - System prompt optimize - Toolkit docstring optimize - Context compression Join Discord β†’
Multi-modal Enhancement - More accurate image understanding when using browser - Advanced video generation Join Discord β†’
Multi-agent system - Workforce support fixed workflow - Workforce support multi-round conversion Join Discord β†’
Browser Toolkit - BrowseComp integration - Benchmark improvement - Forbid repeated page visiting - Automatic cache button clicking Join Discord β†’
Document Toolkit - Support dynamic file editing Join Discord β†’
Terminal Toolkit - Benchmark improvement - Terminal-Bench integration Join Discord β†’
Environment & RL - Environment design - Data-generation - RL framework integration (VERL, TRL, OpenRLHF) Join Discord β†’

We believe in building trust and embracing all forms of open-source collaborations. Your creative contributions help drive the innovation of Eigent. Explore our GitHub issues and projects to dive in and show us what you’ve got 🀝❀️ Contribution Guideline

Contributors

Made with contrib.rocks.

Eigent is built on top of CAMEL-AI.org's research and infrastructures. Sponsoring CAMEL-AI.org will make Eigent better.

**πŸ“„Β Open Source License**

This repository is licensed under the Apache License 2.0.

🌐 Community & Contact

For more information please contact [email protected]

  • GitHub Issues: Report bugs, request features, and track development. Submit an issue

  • Discord: Get real-time support, chat with the community, and stay updated. Join us

  • X (Twitter): Follow for updates, AI insights, and key announcements. Follow us

  • WeChat Community: Scan the QR code below to add our WeChat assistant, and join our WeChat community group.


File: backend/app/agent/README.md

Agent Module

This module provides the agent infrastructure for eigent, built on top of the CAMEL framework.

Architecture Overview

mermaid
graph TB
    subgraph "Core"
        AM["agent_model.py(Core Factory)"]
        LCA["listen_chat_agent.py(Base Agent Class)"]
        TOOLS["tools.py(Toolkit Loaders)"]
        PROMPT["prompt.py(System Prompts)"]
    end

    subgraph "Factory"
        BROWSER["browser.py"]
        DEV["developer.py"]
        DOC["document.py"]
        MODAL["multi_modal.py"]
        MCP["mcp.py"]
        QC["question_confirm.py"]
        SOCIAL["social_media.py"]
        SUMMARY["task_summary.py"]
    end

    subgraph "External"
        CAMEL["CAMEL Framework"]
        TASK["Task Service"]
        TOOLKITS["Toolkits"]
    end

    AM --> LCA
    AM --> CAMEL
    AM --> TASK

    LCA --> CAMEL
    LCA --> TASK

    BROWSER --> AM
    BROWSER --> PROMPT
    BROWSER --> TOOLKITS

    DEV --> AM
    DEV --> PROMPT
    DEV --> TOOLKITS

    DOC --> AM
    DOC --> PROMPT
    DOC --> TOOLKITS

    MODAL --> AM
    MODAL --> PROMPT
    MODAL --> TOOLKITS

    MCP --> LCA
    MCP --> TOOLS
    MCP --> PROMPT

    QC --> AM
    QC --> PROMPT

    SOCIAL --> AM
    SOCIAL --> PROMPT
    SOCIAL --> TOOLKITS

    SUMMARY --> AM
    SUMMARY --> PROMPT

File Descriptions

File Purpose
agent_model.py Core factory function for creating agents with event loop management
listen_chat_agent.py Base agent class extending CAMEL's ChatAgent with task tracking
tools.py Toolkit and MCP tools loader utilities
prompt.py System prompts for all 8 agent types

Factory Files

File Agent Type
browser.py Senior Research Analyst
developer.py Lead Software Engineer
document.py Documentation Specialist
multi_modal.py Creative Content Specialist
mcp.py MCP Server Agent
question_confirm.py Question Confirmation
social_media.py Social Media Manager
task_summary.py Task Summarizer

File: backend/app/utils/telemetry/README.md

Workforce Telemetry

OpenTelemetry-based telemetry for CAMEL workforce events, sent to Langfuse for observability.

Configuration

Add the following environment variables to ~/.eigent/.env:

bash
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_BASE_URL=https://us.cloud.langfuse.com  # Optional, defaults to US cloud

If these keys are not specified, telemetry will be disabled.

Langfuse Setup

Privacy

Only metadata is captured (task IDs, timings, model names, token counts, quality scores). No PII or detailed task content is sent to Langfuse.

Architecture

Singleton TracerProvider

The TracerProvider is initialized once during FastAPI startup (main.py) to ensure only one BatchSpanProcessor is running, regardless of how many WorkforceMetricsCallback instances are created. This prevents:

  • Resource leaks from multiple background export threads
  • OOM issues from unbounded span queuing (max queue: 4096 spans)
  • Excessive memory usage across multiple workforce sessions

The initialization happens in the startup event:

python
@api.on_event("startup")
async def startup_event():
    from app.utils.telemetry.workforce_metrics import initialize_tracer_provider
    initialize_tracer_provider()

Batch Processing Configuration

  • max_queue_size: 4096 spans (drops oldest when full)
  • export_timeout_millis: 30000 (30s timeout for exports)
  • schedule_delay_millis: 3000 (exports every 3s)
  • max_export_batch_size: 1024 (max spans per export)

Span Structure

All spans share common resource attributes and scope information:

json
{
  "resourceAttributes": {
    "service.name": "eigent-workforce",
    "eigent.project.id": "1768815931733-6575",
    "eigent.task.id": "1768815944094-9806"
  },
  "scope": {
    "name": "eigent.workforce",
    "version": "0.2.83a9"
  }
}

worker.created

Emitted when a worker is created.

json
{
  "attributes": {
    "eigent.worker.id": "73d20286-2c17-467a-8153-2a6ea8cbb6c2",
    "eigent.worker.type": "SingleAgentWorker",
    "eigent.worker.role": "Developer Agent: A master-level coding...",
    "eigent.worker.agent": "developer_agent",
    "eigent.worker.model.type": "gpt-4.1-mini"
  }
}

task.created

Emitted when a task is created.

json
{
  "attributes": {
    "eigent.task.id": "1768815944094-9806.1",
    "eigent.task.description": "Task description",
    "eigent.project.id": "1768815931733-6575",
    "eigent.task.parent_id": "1768815944094-9806",
    "eigent.task.type": "task_type"
  }
}

task.assigned

Emitted when a task is assigned to a worker.

json
{
  "attributes": {
    "eigent.task.id": "1768815944094-9806.1",
    "eigent.worker.id": "0fae2d3d-7c0a-4b50-b09d-da35ae61786d",
    "eigent.project.id": "1768815931733-6575",
    "eigent.task.queue_time_seconds": "1.5",
    "eigent.task.dependencies": "[\"dep_1\", \"dep_2\"]"
  }
}

task.execution:{task_id}

Long-running span tracking task execution from start to completion.

json
{
  "attributes": {
    "eigent.task.id": "1768815944094-9806.1",
    "eigent.project.id": "1768815931733-6575",
    "eigent.task.status": "completed",
    "eigent.worker.id": "0fae2d3d-7c0a-4b50-b09d-da35ae61786d",
    "eigent.task.timestamp": "2026-01-19T09:46:40.045077+00:00",
    "eigent.task.parent_id": "1768815944094-9806",
    "eigent.task.processing_time_seconds": "10.926168203353882",
    "eigent.task.quality_score": "80",
    "eigent.task.token_usage.total_tokens": "37284"
  }
}

workforce.all_tasks_completed

Emitted when all tasks in the workforce are completed.

json
{
  "attributes": {
    "eigent.project.id": "1768815931733-6575",
    "eigent.task.id": "1768815944094-9806",
    "eigent.task.timestamp": "2026-01-19T09:46:44.901068+00:00",
    "workforce.total_tasks": "5"
  }
}

log.message

Emitted for error and critical log messages.

json
{
  "attributes": {
    "log.level": "error",
    "log.message": "Error message",
    "eigent.project.id": "1768815931733-6575"
  }
}

Captured Attributes Reference

Project & Task

  • eigent.project.id - Workforce/project identifier
  • eigent.task.id - Task identifier
  • eigent.task.description - Task description
  • eigent.task.parent_id - Parent task ID
  • eigent.task.type - Task type
  • eigent.task.status - Task status (started, completed, failed)
  • eigent.task.timestamp - ISO 8601 timestamp
  • eigent.task.dependencies - JSON array of dependency task IDs
  • eigent.task.queue_time_seconds - Time in queue before assignment
  • eigent.task.processing_time_seconds - Task execution duration
  • eigent.task.quality_score - Quality score (0-100)

Worker

  • eigent.worker.id - Worker UUID
  • eigent.worker.type - Worker class type
  • eigent.worker.role - Worker role description
  • eigent.worker.agent - Agent type (developer_agent, browser_agent, etc.)
  • eigent.worker.model.type - Model name (gpt-4, claude-3, etc.)

Token Usage

  • eigent.task.token_usage.total_tokens - Total tokens used
  • eigent.task.token_usage.* - Additional token usage metrics

Langfuse

  • langfuse.session.id - Set to project ID for grouping
  • langfuse.tags - ["workforce", "camel", "eigent"]

Workforce

  • workforce.total_tasks - Total number of tasks completed

File: backend/benchmark/harbor/README.md

Eigent Benchmark β€” Harbor Adapter

Converts eigent benchmark tasks into Harbor task format for standardized agent evaluation.

Prerequisites

bash
# Install Harbor CLI
uv tool install harbor

# Set your API key
export ANTHROPIC_API_KEY=your-key-here

Quick Start

bash
cd backend/benchmark/harbor

# 1. Generate Harbor tasks
python run_adapter.py

# 2. Verify with oracle (should score 1.0)
harbor run \
    -p datasets/eigent-bench \
    -a oracle \
    --env docker

# 3. Run with an agent
harbor run \
    -p datasets/eigent-bench \
    -a claude-code \
    -m anthropic/claude-sonnet-4-20250514 \
    --env docker

Generating Tasks

bash
cd backend/benchmark/harbor

# Default (all tasks, 50/50 weights)
python run_adapter.py

# Custom weights (70% checker, 30% grader)
python run_adapter.py --checker-weight 0.7 --grader-weight 0.3

# Limit number of tasks
python run_adapter.py --limit 2

Running Benchmarks

Harbor builds a Docker image from each task's environment/Dockerfile automatically on first run. Subsequent runs reuse the cached image.

Run a specific task

bash
# Run only task 0 (hello world)
harbor run \
    -p datasets/eigent-bench \
    -a claude-code \
    -m anthropic/claude-haiku-4-5-20251001 \
    -t "eigent-bench-0000" \
    --env docker

Run all tasks

bash
harbor run \
    -p datasets/eigent-bench \
    -a claude-code \
    -m anthropic/claude-sonnet-4-20250514 \
    --env docker

Run with different models

bash
# Claude Haiku
harbor run -p datasets/eigent-bench -a claude-code \
    -m anthropic/claude-haiku-4-5-20251001 --env docker

Run with concurrency

bash
harbor run \
    -p datasets/eigent-bench \
    -a claude-code \
    -m anthropic/claude-sonnet-4-20250514 \
    --env docker \
    --n-concurrent 3

Rebuild Docker Image

Harbor caches Docker images. To force a rebuild (e.g., after changing the Dockerfile):

bash
# Clear Harbor's Docker build cache
docker builder prune -f

# Then rerun β€” Harbor will rebuild the image
harbor run -p datasets/eigent-bench -a oracle --env docker

Reward Computation

text
reward = checker_weight Γ— checker_score + grader_weight Γ— grader_score
  • checker_score: 1.0 if checker passes, 0.0 if fails
  • grader_score: completed_milestones / total_milestones (0.0–1.0)
  • Default weights: 0.5 / 0.5 (configurable via CLI flags)

Results

Results are written to jobs/<timestamp>/result.json. Each trial directory contains:

text
jobs/<timestamp>/
β”œβ”€β”€ result.json                    # Overall results
β”œβ”€β”€ job.log                        # Build + run logs
└── eigent-bench-NNNN__/
    β”œβ”€β”€ agent/                     # Agent logs and trajectory
    β”‚   β”œβ”€β”€ claude-code.txt        # Raw agent output
    β”‚   └── trajectory.json        # ATIF trajectory
    └── verifier/
        β”œβ”€β”€ metrics.json           # Detailed checker/grader results
        └── test-stdout.txt        # Verifier output

Generated Task Structure

text
eigent-bench-NNNN/
β”œβ”€β”€ task.toml           # Harbor metadata
β”œβ”€β”€ instruction.md      # Task question
β”œβ”€β”€ environment/
β”‚   β”œβ”€β”€ Dockerfile      # Eigent stack + evaluation deps
β”‚   └── workspace/
β”‚       └── .env.development
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ test.sh         # Entry point β†’ evaluate.py
β”‚   β”œβ”€β”€ evaluate.py     # Checker+grader β†’ reward.txt
β”‚   β”œβ”€β”€ checker_N.py    # Copied from benchmark/checker/
β”‚   β”œβ”€β”€ grader_N.py     # Copied from benchmark/grader/
β”‚   └── config.json     # Task config + weights
└── solution/
    └── solve.sh        # Oracle solution

File: backend/benchmark/README.md

Benchmark

Run workforce benchmarks against the Eigent API and grade results.

Setup

  1. Start the backend and frontend in the main project directory:
bash
npm run dev
  1. Set your API key:
bash
export OPENAI_API_KEY=sk-...

Usage

From the backend/ directory:

bash
# Run all benchmarks
python3 -m benchmark.main

# Run a specific benchmark
python3 -m benchmark.main benchmark/dataset/0.json

Structure

text
benchmark/
  main.py           # Entry point
  client.py         # API client (SSE streaming, auto task start, auto human reply)
  environment.py    # BenchmarkConfig, BenchmarkData, Env, Tests models
  dataset/          # Benchmark JSON configs
    0.json
  checker/        # Checker scripts (pass/fail per benchmark)
    0.py
  grader/         # Grading scripts (milestone completeness per benchmark)
    0.py

Adding a benchmark

  1. Create benchmark/dataset/<n>.json:
json
{
  "data": {
    "name": "<n>",
    "question": "Your task description",
    "env": {}
  },
  "metadata": {
    "difficulty": "easy|medium|hard",
    "description": "Brief description of what this benchmark tests",
    "tags": ["tag1", "tag2"]
  },
  "model_kwargs": {
    "model_platform": "openai",
    "model_type": "gpt-4o"
  },
  "tests": {
    "grader": ["benchmark/grader/<n>.py"],
    "checker": ["benchmark/checker/<n>.py"]
  }
}

The metadata field (optional) provides information about the benchmark:

  • difficulty: Indicates complexity level: "easy", "medium", or "hard"
  • description: Brief explanation of what skills or capabilities the benchmark tests
  • tags: Array of keywords for filtering and organization

The model_kwargs field is optional. Defaults come from BENCHMARK_* environment variables (see below), falling back to openai / gpt-5.2 / $OPENAI_API_KEY. Per-benchmark JSON values override the environment defaults.

Custom model providers

You can override the model for all benchmarks via environment variables (see .env.example):

bash
export BENCHMARK_MODEL_PLATFORM="openai-compatible-model"
export BENCHMARK_MODEL_TYPE=""
export BENCHMARK_API_KEY=""
export BENCHMARK_API_URL=""
Variable Default Description
BENCHMARK_MODEL_PLATFORM openai Provider name. Use openai-compatible-model for any OpenAI-compatible API.
BENCHMARK_MODEL_TYPE gpt-5.2 Model identifier passed to the provider.
BENCHMARK_API_KEY $OPENAI_API_KEY API key for the provider.
BENCHMARK_API_URL https://api.openai.com/v1 Base URL for the provider's API.

Important: If the model is served through an OpenAI-compatible API (e.g. DeepSeek, MiniMax, Ollama, vLLM, LiteLLM, or any other non-OpenAI provider), set BENCHMARK_MODEL_PLATFORM to openai-compatible-model β€” not openai. The openai platform value is reserved for the official OpenAI API only.

To override a single benchmark, add model_kwargs to its JSON config β€” these take priority over environment variables.

  1. Create benchmark/checker/<n>.py with a check(working_directory: str) -> bool function.

  2. Create benchmark/grader/<n>.py with a grade(working_directory: str) -> tuple[int, int] function.

Checker

Each checker is a Python script that checks whether a benchmark task succeeded. It must export a check(working_directory: str) -> bool function that:

  • Returns True and prints PASS if the task succeeded.
  • Returns False and prints FAIL: <reason> if the task failed.
python
def check(working_directory: str) -> bool:
    # check files, run scripts, inspect output, etc.
    if success:
        print("PASS")
        return True
    else:
        print("FAIL: expected X, got Y")
        return False

Grader

Each grader is a Python script that evaluates milestone completeness. It must export a grade(working_directory: str) -> tuple[int, int] function that returns (completed, total):

python
def grade(working_directory: str) -> tuple[int, int]:
    total = 3
    completed = 0
    if milestone_1_done:
        completed += 1
    if milestone_2_done:
        completed += 1
    if milestone_3_done:
        completed += 1
    return completed, total  # e.g. 2/3

Results

After all benchmarks finish, results are saved to benchmark/{timestamp}_results.csv:

csv
benchmark,model,type,script,result
0,openai/gpt-5.2,checker,benchmark/checker/0.py,FAIL
0,openai/gpt-5.2,grader,benchmark/grader/0.py,7/7

Result CSV files are gitignored.

TODO: With MCP servers

To provide MCP servers to the workforce, add installed_mcp to env.
Use env_file to store credentials separately (.env files are gitignored).
The credentials are injected into each MCP server's env before sending the payload to the backend.

text
# benchmark/envs/1.env
NOTION_API_KEY=ntn_xxxxx
json
{
  "data": {
    "name": "1",
    "question": "List all Notion pages",
    "env": {
      "env_file": "benchmark/envs/1.env",
      "installed_mcp": {
        "mcpServers": {
          "notion": {
            "command": "npx",
            "args": ["@modelcontextprotocol/server-notion"]
          }
        }
      }
    }
  },
  "tests": {
    "checker": ["benchmark/checker/1.py"]
  }
}

File: backend/README.md

bash
# Option 1: Start with uvicorn directly
uv run uvicorn main:api --port 5001

# Option 2: Standalone mode (no Electron dependency)
uv run python main.py

# Option 3: If uv run hangs, delete lock files and retry, or use venv directly:
.venv/bin/python main.py
# or
.venv/bin/uvicorn main:api --port 5001 --host 0.0.0.0

# If uv hangs, delete lock files first: rm -f uv_installing.lock uv_installed.lock

Environment Variables (Standalone)

Variable Default Description
EIGENT_BRAIN_PORT 5001 Listening port
EIGENT_BRAIN_HOST 0.0.0.0 Listening address
EIGENT_DEBUG - Set to 1/true to enable reload
EIGENT_WORKSPACE ~/.eigent/workspace Working directory
EIGENT_DEPLOYMENT_TYPE (auto) local / cloud_vm / sandbox / docker; determines Hands capabilities (see ADR-0006)
EIGENT_HANDS_MODE - Set to remote to enable RemoteHands (remote cluster resource mode)
EIGENT_HANDS_CLUSTER_CONFIG_FILE - Path to RemoteHands config file (TOML); recommended
EIGENT_HANDS_TERMINAL - Override terminal hand: 1/true/yes or 0/false/no
EIGENT_HANDS_BROWSER - Override browser hand
EIGENT_HANDS_FILESYSTEM - Override filesystem scope: full / workspace_only
EIGENT_HANDS_MCP - Override MCP mode: all / allowlist

RemoteHands config file example:

bash
cp backend/config/hands_clusters.example.toml ~/.eigent/hands_clusters.toml
export EIGENT_HANDS_MODE=remote
export EIGENT_HANDS_CLUSTER_CONFIG_FILE=~/.eigent/hands_clusters.toml

i18n operation process: https://github.com/Anbarryprojects/fastapi-babel

bash
pybabel extract -F babel.cfg -o messages.pot . --ignore-pot-creation-date  # Extract multilingual strings from code to messages.pot file
pybabel init -i messages.pot -d lang -l zh_CN   # Generate Chinese language pack, can only be generated initially, subsequent execution will cause overwrite
pybabel compile -d lang -l zh_CN                # Compile language pack


pybabel update -i messages.pot -d lang
# -i messages.pot: Specify the input file as the generated .pot file
# -d translations: Specify the translation directory, which typically contains .po files for each language
# -l zh: Specify the language code
bash
# regular search
\berror\b(?!\])