## File: README.md
**Eigent**Β is the open source Cowork desktop application, empowering you to build, manage, and deploy a custom AI workforce that can turn your most complex workflows into automated tasks. As a leading open-source Cowork product, Eigent brings together the best of open-source collaboration and AI-powered automation. Built on [CAMEL-AI][camel-site]'s acclaimed open-source project, our system introduces a **Multi-Agent Workforce** that **boosts productivity** through parallel execution, customization, and privacy protection. ### ⭐ 100% Open Source - πŸ₯‡ Local Deployment - πŸ† MCP Integration - βœ… **Zero Setup** - No technical configuration required - βœ… **Multi-Agent Coordination** - Handle complex multi-agent workflows - βœ… **Single-Agent Harness** - Run focused tasks with one meta agent - βœ… **Local Deployment** - βœ… **Open Source** - βœ… **Model Agnostic** - Support any model of your choice - βœ… **MCP Integration** - βœ… **Skill Integration** - βœ… **Built-in Browser & Terminal Toolkits** - βœ… **Enterprise Features** - SSO, access control, and custom enquiries [![][image-join-us]][join-us] Table of contents #### TOC - [πŸš€ Getting Started with Open Source Cowork](#-getting-started-with-open-source-Cowork) - [🏠 Local Deployment (Recommended)](#-local-deployment-recommended) - [⚑ Quick Start (Cloud-Connected)](#-quick-start-cloud-connected) - [🏒 Enterprise](#-enterprise) - [☁️ Cloud Version](#%EF%B8%8F-cloud-version) - [✨ Key features - Open Source Cowork](#-key-features---open-source-Cowork) - [πŸ§‘β€πŸ’» Cowork with Single Agent](#-cowork-with-single-agent) - [🏭 Cowork with Workforce](#-cowork-with-workforce) - [⏰ Automation](#-automation) - [πŸ”’ Local & Secure](#-local--secure) - [🧠 Model Agnostic](#-model-agnostic) - [πŸ‘ 100% Open Source](#-100-open-source) - [🧩 Use Cases - Open Source Cowork](#-use-cases---open-source-Cowork) - [For Developers](#for-developers) - [Featured](#featured) - [πŸ› οΈ Tech Stack](#-tech-stack) - [Backend](#backend) - [Frontend](#frontend) - [🌟 Staying ahead - Open Source Cowork](#-staying-ahead---open-source-Cowork) - [πŸ—ΊοΈ Roadmap - Open Source Cowork](#%EF%B8%8F-roadmap---open-source-Cowork) - [πŸ“–Β Contributing](#-contributing) - [Main Contributors](#main-contributors) - [Distinguished ambassador](#distinguished-ambassador) - [Ecosystem](#ecosystem) - [πŸ“„Β Open Source License](#-open-source-license) - [🌐 Community & contact](#-community--contact) #### ## **πŸš€ Getting Started with Open Source Cowork** > **πŸ”“ Build in Public** β€” Eigent is **100% open source** from day one. Every feature, every commit, every decision is transparent. We believe the best AI tools should be built openly with the community, not behind closed doors. ### 🏠 Local Deployment (Recommended) The recommended way to run Eigent β€” fully standalone with complete control over your data, no cloud account required. πŸ‘‰ **[Full Local Deployment Guide](./server/README_EN.md)** This setup includes: - Local backend server with full API - Local model integration (vLLM, Ollama, LM Studio, etc.) - Complete isolation from cloud services - Zero external dependencies ### ⚑ Quick Start (Cloud-Connected) For a quick preview using our cloud backend β€” get started in seconds: #### Prerequisites - Node.js (version 18-22) and npm #### Steps ```bash git clone https://github.com/eigent-ai/eigent.git cd eigent npm install npm run dev ``` > Note: This mode connects to Eigent cloud services and requires account registration. For a fully standalone experience, use [Local Deployment](#-local-deployment-recommended) instead. #### Updating Dependencies After pulling new code (`git pull`), update both frontend and backend dependencies: ```bash # 1. Update frontend dependencies (in project root) npm install # 2. Update backend/Python dependencies (in backend directory) cd backend uv sync ``` ### 🏒 Enterprise For organizations requiring maximum security, customization, and control: - **Exclusive Features** (like SSO & custom development) - **Scalable Enterprise Deployment** - **Negotiated SLAs** & implementation services πŸ“§ For further details, [contact our sales team](https://www.eigent.ai/contact-sales). ### ☁️ Cloud Version For teams who prefer managed infrastructure, we also offer a cloud platform. The fastest way to experience Eigent's multi-agent AI capabilities without setup complexity. We'll host the models, APIs, and cloud storage, ensuring Eigent runs flawlessly. - **Instant Access** - Start building multi-agent workflows in minutes. - **Managed Infrastructure** - We handle scaling, updates, and maintenance. - **Premium Support** - Subscribe and get priority assistance from our engineering team. [![image-public-beta]][eigent-download] [Get started at Eigent.ai β†’](https://www.eigent.ai/download) ## **✨ Key features - Open Source Cowork** ### πŸ§‘β€πŸ’» Cowork with Single Agent Start with one focused agent for direct tasks. Research, write, debug, and operate alongside it in your desktop workspace. ### 🏭 Cowork with Workforce Scale to multiple specialized agents that divide work, collaborate in parallel, and execute complex multi-step workflows together. ### ⏰ Automation Schedule recurring workflows and let agents run tasks at the right timeβ€”so work continues even when you step away. ### πŸ”’ Local & Secure Run agents on your machine with local-first execution. Your files, credentials, and context stay under your control. ### 🧠 Model Agnostic Connect the models you already useβ€”cloud APIs, enterprise gateways, or local inferenceβ€”without locking into one vendor. ### πŸ‘ 100% Open Source Eigent is completely open-sourced. You can download, inspect, and modify the code, ensuring transparency and fostering a community-driven ecosystem for multi-agent innovation. ## 🧩 Use Cases - Open Source Cowork Explore how Eigent turns complex desktop work into repeatable agent workflows. ### For Developers #### [Build 10 Chinese New Year HTML5 Games with Eigent](https://www.eigent.ai/use-cases/build-10-cny-horse-themed-html5-games) Coordinate parallel agents to build ten polished, mobile-friendly browser games across genres, complete with scoring, increasing difficulty, and restart flows. [View demo β†’](https://www.eigent.ai/use-cases/build-10-cny-horse-themed-html5-games/video) [View guide β†’](https://www.eigent.ai/use-cases/build-10-cny-horse-themed-html5-games) #### [Build a 3D Snow Bros Platformer with Gemini 3.1 Pro](https://www.eigent.ai/use-cases/build-3d-snow-bros-platformer-gemini) Create a complete browser-based 3D platformer with snowball combat, enemy chains, scoring, lives, scaling difficulty, and layered environments. [View demo β†’](https://www.eigent.ai/use-cases/build-3d-snow-bros-platformer-gemini/video) [View guide β†’](https://www.eigent.ai/use-cases/build-3d-snow-bros-platformer-gemini) #### [Automate Monthly Dev Reports with DeepSeek via Ollama](https://www.eigent.ai/use-cases/monthly-dev-reports-automated-eigent-with-deepseek-v4-pro-via-ollama) Review a month of GitHub pull requests with a locally hosted model, generate a Word summary, and prepare the corresponding Slack release update. [View demo β†’](https://www.eigent.ai/use-cases/monthly-dev-reports-automated-eigent-with-deepseek-v4-pro-via-ollama/video) [View guide β†’](https://www.eigent.ai/use-cases/monthly-dev-reports-automated-eigent-with-deepseek-v4-pro-via-ollama) ### Featured #### [Organize Desktop Files](https://www.eigent.ai/use-cases/organize-desktop-files) Ask Eigent to inspect a cluttered desktop and organize files into a cleaner, more useful structure directly on your machine. [View demo β†’](https://www.eigent.ai/use-cases/organize-desktop-files/video) [View guide β†’](https://www.eigent.ai/use-cases/organize-desktop-files) #### [Audit ML CI Failures with Gemini 3.5 Flash on Eigent](https://www.eigent.ai/use-cases/eigent-gemini-managed-agents) Orchestrate a multi-agent CI investigation that fetches logs, compares golden values, traces evidence, delegates deep reasoning, and produces structured audit reports. [View demo β†’](https://www.eigent.ai/use-cases/eigent-gemini-managed-agents/video) [View guide β†’](https://www.eigent.ai/use-cases/eigent-gemini-managed-agents) #### [Ticket Management System Integration and Reporting](https://www.eigent.ai/use-cases/ticket-management-system-integration-and-reporting) Import local ticket data into a browser-based management system, then generate a statistical report with charts and visual summaries. [View demo β†’](https://www.eigent.ai/use-cases/ticket-management-system-integration-and-reporting/video) [View guide β†’](https://www.eigent.ai/use-cases/ticket-management-system-integration-and-reporting) [Explore more use cases β†’](https://www.eigent.ai/use-cases) ## πŸ› οΈ Tech Stack Eigent open-source Cowork desktop is built on modern, reliable technologies that ensure scalability, performance, and extensibility. ### Backend - **Framework:**Β FastAPI - **Package Manager:**Β uv - **Async Server:**Β Uvicorn - **Authentication:**Β OAuth 2.0, Passlib. - **Multi-agent framework:** CAMEL ### Frontend - **Framework:**Β React - **Desktop App Framework:**Β Electron - **Language:**Β TypeScript - **UI:**Β Tailwind CSS, Radix UI, Lucide React, Framer Motion - **State Management:**Β Zustand - **Flow Editor:**Β React Flow ## 🌟 Staying ahead - Open Source Cowork > [!IMPORTANT] > > **Star Eigent**, You will receive all release notifications from GitHub without any delay ~ ⭐️ ![][image-star-us] ## πŸ—ΊοΈ Roadmap - Open Source Cowork Our open-source Cowork continues to evolve with input from the community. Here's what's coming next: | Topics | Issues | Discord Channel | | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------- | | **Context Engineering** | - Prompt caching - System prompt optimize - Toolkit docstring optimize - Context compression | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | | **Multi-modal Enhancement** | - More accurate image understanding when using browser - Advanced video generation | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | | **Multi-agent system** | - Workforce support fixed workflow - Workforce support multi-round conversion | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | | **Browser Toolkit** | - BrowseComp integration - Benchmark improvement - Forbid repeated page visiting - Automatic cache button clicking | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | | **Document Toolkit** | - Support dynamic file editing | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | | **Terminal Toolkit** | - Benchmark improvement - Terminal-Bench integration | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | | **Environment & RL** | - Environment design - Data-generation - RL framework integration (VERL, TRL, OpenRLHF) | [**Join Discord β†’**](https://discord.com/invite/CNcNpquyDc) | ## [🀝 Contributing][contribution-link] We believe in building trust and embracing all forms of open-source collaborations. Your creative contributions help drive the innovation of `Eigent`. Explore our GitHub issues and projects to dive in and show us what you’ve got 🀝❀️ [Contribution Guideline][contribution-link] ## Contributors Made with [contrib.rocks](https://contrib.rocks). ## [❀️ Sponsor][sponsor-link] Eigent is built on top of [CAMEL-AI.org][camel-ai-org-github]'s research and infrastructures. [Sponsoring CAMEL-AI.org][sponsor-link] will make `Eigent` better. ## **πŸ“„Β Open Source License** This repository is licensed under the [Apache License 2.0](LICENSE). ## 🌐 Community & Contact For more information please contact info@eigent.ai - **GitHub Issues:** Report bugs, request features, and track development. [Submit an issue][github-issue-link] - **Discord:** Get real-time support, chat with the community, and stay updated. [Join us](https://discord.com/invite/CNcNpquyDc) - **X (Twitter):** Follow for updates, AI insights, and key announcements. [Follow us][social-x-link] - **WeChat Community:** Scan the QR code below to add our WeChat assistant, and join our WeChat community group. [built-with-camel]: https://img.shields.io/badge/-Built--with--CAMEL-4C19E8.svg?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQ4IiBoZWlnaHQ9IjI3MiIgdmlld0JveD0iMCAwIDI0OCAyNzIiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CjxwYXRoIGQ9Ik04LjgzMTE3IDE4LjU4NjVMMCAzMC44MjY3QzUuNDY2OTIgMzUuMDQzMiAxNS4xMzkxIDM4LjgyNTggMjQuODExNCAzNi4yOTU5QzMwLjY5ODggNDAuOTM0MSAzOS42NzAyIDQwLjIzMTMgNDQuMTU1OSA0MC4wOTA4QzQzLjQ1NSA0Ny4zOTk0IDQyLjQ3MzcgNzAuOTU1OCA0NC4xNTU5IDEwNi43MTJDNDUuODM4IDE0Mi40NjggNzEuNzcwOCAxNjYuODY4IDg0LjUyNjkgMTc0LjU5OEw3Ni4wMDAyIDIyMEw4NC41MjY5IDI3MkgxMDguOTE4TDk4LjAwMDIgMjIwTDEwOC45MTggMTc0LjU5OEwxMjkuOTQ0IDI3MkgxNTQuNzU2TDEzNC4xNSAxNzQuNTk4SDE4Ny4xMzdMMTY2LjUzMSAyNzJIMTkxLjc2M0wyMTIuMzY5IDE3NC41OThMMjI2IDIyMEwyMTIuMzY5IDI3MkgyMzcuNjAxTDI0OC4wMDEgMjIwTDIzNy4xOCAxNzQuNTk4QzIzOS4yODMgMTY5LjExNyAyNDAuNDAxIDE2Ni45NzYgMjQxLjgwNiAxNjEuMTA1QzI0OS4zNzUgMTI5LjQ4MSAyMzUuMDc3IDEwMy45MDEgMjI2LjY2NyA5NC40ODRMMjA2LjQ4MSA3My44MjNDMTk3LjY1IDY0Ljk2ODMgMTgyLjUxMSA2NC41NDY3IDE3Mi44MzkgNzIuNTU4MUMxNjUuNzI4IDc4LjQ0NzcgMTYxLjcwMSA3OC43NzI3IDE1NC43NTYgNzIuNTU4MUMxNTEuODEyIDcwLjAyODEgMTQ0LjUzNSA2MS40ODg5IDEzNC45OTEgNTMuNTgzN0MxMjUuMzE5IDQ1LjU3MjMgMTA4LjQ5NyA0OC45NDU1IDEwMi4xODkgNTUuNjkxOUw3My41OTMxIDg0LjM2NDRWNy42MjM0OUw3OS4xMjczIDBDNjAuOTA0MiAzLjY1NDMzIDIzLjgwMjEgOS41NjMwOSAxOS43NjUgMTAuNTc1MUMxNS43Mjc5IDExLjU4NyAxMC43OTM3IDE2LjMzNzcgOC44MzExNyAxOC41ODY1WiIgZmlsbD0id2hpdGUiLz4KPHBhdGggZD0iTTQzLjIwMzggMTguNzE4N0w0OS4wOTEyIDEzLjA0OTNMNTQuOTc4NyAxOC43MTg3TDQ5LjA5MTIgMjQuODI0Mkw0My4yMDM4IDE4LjcxODdaIiBmaWxsPSIjNEMxOUU4Ii8+Cjwvc3ZnPgo= [camel-ai-org-github]: https://github.com/camel-ai [camel-github]: https://github.com/camel-ai/camel [camel-site]: https://www.camel-ai.org [contribution-link]: https://github.com/eigent-ai/eigent/blob/main/CONTRIBUTING.md [discord-image]: https://img.shields.io/discord/1082486657678311454?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb [discord-url]: https://discord.com/invite/CNcNpquyDc [docs-site]: https://docs.eigent.ai [download-shield]: https://img.shields.io/badge/Download%20Eigent-363AF5?style=plastic [eigent-download]: https://www.eigent.ai/download [eigent-github]: https://github.com/eigent-ai/eigent [eigent-site]: https://www.eigent.ai [github-issue-link]: https://github.com/eigent-ai/eigent/issues [github-star]: https://img.shields.io/github/stars/eigent-ai?color=F5F4F0&labelColor=gray&style=plastic&logo=github [image-head]: https://eigent-ai.github.io/.github/assets/head.png [image-join-us]: https://camel-ai.github.io/camel_asset/graphics/join_us.png [image-opensource]: https://eigent-ai.github.io/.github/assets/opensource.png [image-public-beta]: https://eigent-ai.github.io/.github/assets/banner.png [image-seperator]: https://eigent-ai.github.io/.github/assets/seperator.png [image-star-us]: https://eigent-ai.github.io/.github/assets/star-us.gif [join-us]: https://www.eigent.ai/careers [join-us-image]: https://img.shields.io/badge/Join%20Us-yellow?style=plastic [reddit-image]: https://img.shields.io/reddit/subreddit-subscribers/CamelAI?style=plastic&logo=reddit&label=r%2FCAMEL&labelColor=white [reddit-url]: https://www.reddit.com/r/CamelAI/ [social-x-link]: https://x.com/Eigent_AI [social-x-shield]: https://img.shields.io/badge/-%40Eigent_AI-white?labelColor=gray&logo=x&logoColor=white&style=plastic [sponsor-link]: https://github.com/sponsors/camel-ai [sponsor-shield]: https://img.shields.io/badge/-Sponsor%20CAMEL--AI-1d1d1d?logo=github&logoColor=white&style=plastic [wechat-image]: https://img.shields.io/badge/WeChat-CamelAIOrg-brightgreen?logo=wechat&logoColor=white [wechat-url]: https://ghli.org/camel/wechat.png --- ## File: backend/app/agent/README.md # Agent Module This module provides the agent infrastructure for eigent, built on top of the CAMEL framework. ## Architecture Overview ```mermaid graph TB subgraph "Core" AM["agent_model.py(Core Factory)"] LCA["listen_chat_agent.py(Base Agent Class)"] TOOLS["tools.py(Toolkit Loaders)"] PROMPT["prompt.py(System Prompts)"] end subgraph "Factory" BROWSER["browser.py"] DEV["developer.py"] DOC["document.py"] MODAL["multi_modal.py"] MCP["mcp.py"] QC["question_confirm.py"] SOCIAL["social_media.py"] SUMMARY["task_summary.py"] end subgraph "External" CAMEL["CAMEL Framework"] TASK["Task Service"] TOOLKITS["Toolkits"] end AM --> LCA AM --> CAMEL AM --> TASK LCA --> CAMEL LCA --> TASK BROWSER --> AM BROWSER --> PROMPT BROWSER --> TOOLKITS DEV --> AM DEV --> PROMPT DEV --> TOOLKITS DOC --> AM DOC --> PROMPT DOC --> TOOLKITS MODAL --> AM MODAL --> PROMPT MODAL --> TOOLKITS MCP --> LCA MCP --> TOOLS MCP --> PROMPT QC --> AM QC --> PROMPT SOCIAL --> AM SOCIAL --> PROMPT SOCIAL --> TOOLKITS SUMMARY --> AM SUMMARY --> PROMPT ``` ## File Descriptions | File | Purpose | | ---------------------- | -------------------------------------------------------------------- | | `agent_model.py` | Core factory function for creating agents with event loop management | | `listen_chat_agent.py` | Base agent class extending CAMEL's ChatAgent with task tracking | | `tools.py` | Toolkit and MCP tools loader utilities | | `prompt.py` | System prompts for all 8 agent types | ### Factory Files | File | Agent Type | | --------------------- | --------------------------- | | `browser.py` | Senior Research Analyst | | `developer.py` | Lead Software Engineer | | `document.py` | Documentation Specialist | | `multi_modal.py` | Creative Content Specialist | | `mcp.py` | MCP Server Agent | | `question_confirm.py` | Question Confirmation | | `social_media.py` | Social Media Manager | | `task_summary.py` | Task Summarizer | --- ## File: backend/app/utils/telemetry/README.md # Workforce Telemetry OpenTelemetry-based telemetry for CAMEL workforce events, sent to Langfuse for observability. ## Configuration Add the following environment variables to `~/.eigent/.env`: ```bash LANGFUSE_PUBLIC_KEY=pk-lf-... LANGFUSE_SECRET_KEY=sk-lf-... LANGFUSE_BASE_URL=https://us.cloud.langfuse.com # Optional, defaults to US cloud ``` **If these keys are not specified, telemetry will be disabled.** ## Langfuse Setup - **Cloud**: Sign up at [Langfuse Cloud](https://cloud.langfuse.com) - **Self-hosted**: Use the [open-source version](https://langfuse.com/self-hosting) - **Documentation**: [https://langfuse.com/docs](https://langfuse.com/docs) ## Privacy Only **metadata** is captured (task IDs, timings, model names, token counts, quality scores). **No PII or detailed task content** is sent to Langfuse. ## Architecture ### Singleton TracerProvider The `TracerProvider` is initialized once during FastAPI startup (`main.py`) to ensure only one `BatchSpanProcessor` is running, regardless of how many `WorkforceMetricsCallback` instances are created. This prevents: - Resource leaks from multiple background export threads - OOM issues from unbounded span queuing (max queue: 4096 spans) - Excessive memory usage across multiple workforce sessions The initialization happens in the startup event: ```python @api.on_event("startup") async def startup_event(): from app.utils.telemetry.workforce_metrics import initialize_tracer_provider initialize_tracer_provider() ``` ### Batch Processing Configuration - `max_queue_size`: 4096 spans (drops oldest when full) - `export_timeout_millis`: 30000 (30s timeout for exports) - `schedule_delay_millis`: 3000 (exports every 3s) - `max_export_batch_size`: 1024 (max spans per export) ## Span Structure All spans share common resource attributes and scope information: ```json { "resourceAttributes": { "service.name": "eigent-workforce", "eigent.project.id": "1768815931733-6575", "eigent.task.id": "1768815944094-9806" }, "scope": { "name": "eigent.workforce", "version": "0.2.83a9" } } ``` ### worker.created Emitted when a worker is created. ```json { "attributes": { "eigent.worker.id": "73d20286-2c17-467a-8153-2a6ea8cbb6c2", "eigent.worker.type": "SingleAgentWorker", "eigent.worker.role": "Developer Agent: A master-level coding...", "eigent.worker.agent": "developer_agent", "eigent.worker.model.type": "gpt-4.1-mini" } } ``` ### task.created Emitted when a task is created. ```json { "attributes": { "eigent.task.id": "1768815944094-9806.1", "eigent.task.description": "Task description", "eigent.project.id": "1768815931733-6575", "eigent.task.parent_id": "1768815944094-9806", "eigent.task.type": "task_type" } } ``` ### task.assigned Emitted when a task is assigned to a worker. ```json { "attributes": { "eigent.task.id": "1768815944094-9806.1", "eigent.worker.id": "0fae2d3d-7c0a-4b50-b09d-da35ae61786d", "eigent.project.id": "1768815931733-6575", "eigent.task.queue_time_seconds": "1.5", "eigent.task.dependencies": "[\"dep_1\", \"dep_2\"]" } } ``` ### task.execution:{task_id} Long-running span tracking task execution from start to completion. ```json { "attributes": { "eigent.task.id": "1768815944094-9806.1", "eigent.project.id": "1768815931733-6575", "eigent.task.status": "completed", "eigent.worker.id": "0fae2d3d-7c0a-4b50-b09d-da35ae61786d", "eigent.task.timestamp": "2026-01-19T09:46:40.045077+00:00", "eigent.task.parent_id": "1768815944094-9806", "eigent.task.processing_time_seconds": "10.926168203353882", "eigent.task.quality_score": "80", "eigent.task.token_usage.total_tokens": "37284" } } ``` ### workforce.all_tasks_completed Emitted when all tasks in the workforce are completed. ```json { "attributes": { "eigent.project.id": "1768815931733-6575", "eigent.task.id": "1768815944094-9806", "eigent.task.timestamp": "2026-01-19T09:46:44.901068+00:00", "workforce.total_tasks": "5" } } ``` ### log.message Emitted for error and critical log messages. ```json { "attributes": { "log.level": "error", "log.message": "Error message", "eigent.project.id": "1768815931733-6575" } } ``` ## Captured Attributes Reference ### Project & Task - `eigent.project.id` - Workforce/project identifier - `eigent.task.id` - Task identifier - `eigent.task.description` - Task description - `eigent.task.parent_id` - Parent task ID - `eigent.task.type` - Task type - `eigent.task.status` - Task status (started, completed, failed) - `eigent.task.timestamp` - ISO 8601 timestamp - `eigent.task.dependencies` - JSON array of dependency task IDs - `eigent.task.queue_time_seconds` - Time in queue before assignment - `eigent.task.processing_time_seconds` - Task execution duration - `eigent.task.quality_score` - Quality score (0-100) ### Worker - `eigent.worker.id` - Worker UUID - `eigent.worker.type` - Worker class type - `eigent.worker.role` - Worker role description - `eigent.worker.agent` - Agent type (developer_agent, browser_agent, etc.) - `eigent.worker.model.type` - Model name (gpt-4, claude-3, etc.) ### Token Usage - `eigent.task.token_usage.total_tokens` - Total tokens used - `eigent.task.token_usage.*` - Additional token usage metrics ### Langfuse - `langfuse.session.id` - Set to project ID for grouping - `langfuse.tags` - ["workforce", "camel", "eigent"] ### Workforce - `workforce.total_tasks` - Total number of tasks completed --- ## File: backend/benchmark/harbor/README.md # Eigent Benchmark β€” Harbor Adapter Converts eigent benchmark tasks into [Harbor](https://harborframework.com/) task format for standardized agent evaluation. ## Prerequisites ```bash # Install Harbor CLI uv tool install harbor # Set your API key export ANTHROPIC_API_KEY=your-key-here ``` ## Quick Start ```bash cd backend/benchmark/harbor # 1. Generate Harbor tasks python run_adapter.py # 2. Verify with oracle (should score 1.0) harbor run \ -p datasets/eigent-bench \ -a oracle \ --env docker # 3. Run with an agent harbor run \ -p datasets/eigent-bench \ -a claude-code \ -m anthropic/claude-sonnet-4-20250514 \ --env docker ``` ## Generating Tasks ```bash cd backend/benchmark/harbor # Default (all tasks, 50/50 weights) python run_adapter.py # Custom weights (70% checker, 30% grader) python run_adapter.py --checker-weight 0.7 --grader-weight 0.3 # Limit number of tasks python run_adapter.py --limit 2 ``` ## Running Benchmarks Harbor builds a Docker image from each task's `environment/Dockerfile` automatically on first run. Subsequent runs reuse the cached image. ### Run a specific task ```bash # Run only task 0 (hello world) harbor run \ -p datasets/eigent-bench \ -a claude-code \ -m anthropic/claude-haiku-4-5-20251001 \ -t "eigent-bench-0000" \ --env docker ``` ### Run all tasks ```bash harbor run \ -p datasets/eigent-bench \ -a claude-code \ -m anthropic/claude-sonnet-4-20250514 \ --env docker ``` ### Run with different models ```bash # Claude Haiku harbor run -p datasets/eigent-bench -a claude-code \ -m anthropic/claude-haiku-4-5-20251001 --env docker ``` ### Run with concurrency ```bash harbor run \ -p datasets/eigent-bench \ -a claude-code \ -m anthropic/claude-sonnet-4-20250514 \ --env docker \ --n-concurrent 3 ``` ## Rebuild Docker Image Harbor caches Docker images. To force a rebuild (e.g., after changing the Dockerfile): ```bash # Clear Harbor's Docker build cache docker builder prune -f # Then rerun β€” Harbor will rebuild the image harbor run -p datasets/eigent-bench -a oracle --env docker ``` ## Reward Computation ``` reward = checker_weight Γ— checker_score + grader_weight Γ— grader_score ``` - `checker_score`: 1.0 if checker passes, 0.0 if fails - `grader_score`: completed_milestones / total_milestones (0.0–1.0) - Default weights: 0.5 / 0.5 (configurable via CLI flags) ## Results Results are written to `jobs//result.json`. Each trial directory contains: ``` jobs// β”œβ”€β”€ result.json # Overall results β”œβ”€β”€ job.log # Build + run logs └── eigent-bench-NNNN__/ β”œβ”€β”€ agent/ # Agent logs and trajectory β”‚ β”œβ”€β”€ claude-code.txt # Raw agent output β”‚ └── trajectory.json # ATIF trajectory └── verifier/ β”œβ”€β”€ metrics.json # Detailed checker/grader results └── test-stdout.txt # Verifier output ``` ## Generated Task Structure ``` eigent-bench-NNNN/ β”œβ”€β”€ task.toml # Harbor metadata β”œβ”€β”€ instruction.md # Task question β”œβ”€β”€ environment/ β”‚ β”œβ”€β”€ Dockerfile # Eigent stack + evaluation deps β”‚ └── workspace/ β”‚ └── .env.development β”œβ”€β”€ tests/ β”‚ β”œβ”€β”€ test.sh # Entry point β†’ evaluate.py β”‚ β”œβ”€β”€ evaluate.py # Checker+grader β†’ reward.txt β”‚ β”œβ”€β”€ checker_N.py # Copied from benchmark/checker/ β”‚ β”œβ”€β”€ grader_N.py # Copied from benchmark/grader/ β”‚ └── config.json # Task config + weights └── solution/ └── solve.sh # Oracle solution ``` --- ## File: backend/benchmark/README.md # Benchmark Run workforce benchmarks against the Eigent API and grade results. ## Setup 1. Start the backend and frontend in the main project directory: ```bash npm run dev ``` 2. Set your API key: ```bash export OPENAI_API_KEY=sk-... ``` ## Usage From the `backend/` directory: ```bash # Run all benchmarks python3 -m benchmark.main # Run a specific benchmark python3 -m benchmark.main benchmark/dataset/0.json ``` ## Structure ``` benchmark/ main.py # Entry point client.py # API client (SSE streaming, auto task start, auto human reply) environment.py # BenchmarkConfig, BenchmarkData, Env, Tests models dataset/ # Benchmark JSON configs 0.json checker/ # Checker scripts (pass/fail per benchmark) 0.py grader/ # Grading scripts (milestone completeness per benchmark) 0.py ``` ## Adding a benchmark 1. Create `benchmark/dataset/.json`: ```json { "data": { "name": "", "question": "Your task description", "env": {} }, "metadata": { "difficulty": "easy|medium|hard", "description": "Brief description of what this benchmark tests", "tags": ["tag1", "tag2"] }, "model_kwargs": { "model_platform": "openai", "model_type": "gpt-4o" }, "tests": { "grader": ["benchmark/grader/.py"], "checker": ["benchmark/checker/.py"] } } ``` The `metadata` field (optional) provides information about the benchmark: - `difficulty`: Indicates complexity level: "easy", "medium", or "hard" - `description`: Brief explanation of what skills or capabilities the benchmark tests - `tags`: Array of keywords for filtering and organization The `model_kwargs` field is optional. Defaults come from `BENCHMARK_*` environment variables (see below), falling back to `openai` / `gpt-5.2` / `$OPENAI_API_KEY`. Per-benchmark JSON values override the environment defaults. ### Custom model providers You can override the model for all benchmarks via environment variables (see `.env.example`): ```bash export BENCHMARK_MODEL_PLATFORM="openai-compatible-model" export BENCHMARK_MODEL_TYPE="" export BENCHMARK_API_KEY="" export BENCHMARK_API_URL="" ``` | Variable | Default | Description | | -------------------------- | --------------------------- | --------------------------------------------------------------------------- | | `BENCHMARK_MODEL_PLATFORM` | `openai` | Provider name. Use `openai-compatible-model` for any OpenAI-compatible API. | | `BENCHMARK_MODEL_TYPE` | `gpt-5.2` | Model identifier passed to the provider. | | `BENCHMARK_API_KEY` | `$OPENAI_API_KEY` | API key for the provider. | | `BENCHMARK_API_URL` | `https://api.openai.com/v1` | Base URL for the provider's API. | > **Important:** If the model is served through an OpenAI-compatible API (e.g. DeepSeek, MiniMax, Ollama, vLLM, LiteLLM, or any other non-OpenAI provider), set `BENCHMARK_MODEL_PLATFORM` to `openai-compatible-model` β€” **not** `openai`. The `openai` platform value is reserved for the official OpenAI API only. To override a single benchmark, add `model_kwargs` to its JSON config β€” these take priority over environment variables. 2. Create `benchmark/checker/.py` with a `check(working_directory: str) -> bool` function. 1. Create `benchmark/grader/.py` with a `grade(working_directory: str) -> tuple[int, int]` function. ## Checker Each checker is a Python script that checks whether a benchmark task succeeded. It must export a `check(working_directory: str) -> bool` function that: - Returns `True` and prints `PASS` if the task succeeded. - Returns `False` and prints `FAIL: ` if the task failed. ```python def check(working_directory: str) -> bool: # check files, run scripts, inspect output, etc. if success: print("PASS") return True else: print("FAIL: expected X, got Y") return False ``` ## Grader Each grader is a Python script that evaluates milestone completeness. It must export a `grade(working_directory: str) -> tuple[int, int]` function that returns `(completed, total)`: ```python def grade(working_directory: str) -> tuple[int, int]: total = 3 completed = 0 if milestone_1_done: completed += 1 if milestone_2_done: completed += 1 if milestone_3_done: completed += 1 return completed, total # e.g. 2/3 ``` ## Results After all benchmarks finish, results are saved to `benchmark/{timestamp}_results.csv`: ```csv benchmark,model,type,script,result 0,openai/gpt-5.2,checker,benchmark/checker/0.py,FAIL 0,openai/gpt-5.2,grader,benchmark/grader/0.py,7/7 ``` Result CSV files are gitignored. ## TODO: With MCP servers To provide MCP servers to the workforce, add `installed_mcp` to `env`. Use `env_file` to store credentials separately (`.env` files are gitignored). The credentials are injected into each MCP server's `env` before sending the payload to the backend. ``` # benchmark/envs/1.env NOTION_API_KEY=ntn_xxxxx ``` ```json { "data": { "name": "1", "question": "List all Notion pages", "env": { "env_file": "benchmark/envs/1.env", "installed_mcp": { "mcpServers": { "notion": { "command": "npx", "args": ["@modelcontextprotocol/server-notion"] } } } } }, "tests": { "checker": ["benchmark/checker/1.py"] } } ``` --- ## File: backend/README.md ```bash # Option 1: Start with uvicorn directly uv run uvicorn main:api --port 5001 # Option 2: Standalone mode (no Electron dependency) uv run python main.py # Option 3: If uv run hangs, delete lock files and retry, or use venv directly: .venv/bin/python main.py # or .venv/bin/uvicorn main:api --port 5001 --host 0.0.0.0 # If uv hangs, delete lock files first: rm -f uv_installing.lock uv_installed.lock ``` ### Environment Variables (Standalone) | Variable | Default | Description | | ---------------------------------- | ------------------- | ----------------------------------------------------------------------------------------- | | `EIGENT_BRAIN_PORT` | 5001 | Listening port | | `EIGENT_BRAIN_HOST` | 0.0.0.0 | Listening address | | `EIGENT_DEBUG` | - | Set to 1/true to enable reload | | `EIGENT_WORKSPACE` | ~/.eigent/workspace | Working directory | | `EIGENT_DEPLOYMENT_TYPE` | (auto) | `local` / `cloud_vm` / `sandbox` / `docker`; determines Hands capabilities (see ADR-0006) | | `EIGENT_HANDS_MODE` | - | Set to `remote` to enable `RemoteHands` (remote cluster resource mode) | | `EIGENT_HANDS_CLUSTER_CONFIG_FILE` | - | Path to `RemoteHands` config file (TOML); **recommended** | | `EIGENT_HANDS_TERMINAL` | - | Override terminal hand: `1`/`true`/`yes` or `0`/`false`/`no` | | `EIGENT_HANDS_BROWSER` | - | Override browser hand | | `EIGENT_HANDS_FILESYSTEM` | - | Override filesystem scope: `full` / `workspace_only` | | `EIGENT_HANDS_MCP` | - | Override MCP mode: `all` / `allowlist` | RemoteHands config file example: ```bash cp backend/config/hands_clusters.example.toml ~/.eigent/hands_clusters.toml export EIGENT_HANDS_MODE=remote export EIGENT_HANDS_CLUSTER_CONFIG_FILE=~/.eigent/hands_clusters.toml ``` i18n operation process: https://github.com/Anbarryprojects/fastapi-babel ```bash pybabel extract -F babel.cfg -o messages.pot . --ignore-pot-creation-date # Extract multilingual strings from code to messages.pot file pybabel init -i messages.pot -d lang -l zh_CN # Generate Chinese language pack, can only be generated initially, subsequent execution will cause overwrite pybabel compile -d lang -l zh_CN # Compile language pack pybabel update -i messages.pot -d lang # -i messages.pot: Specify the input file as the generated .pot file # -d translations: Specify the translation directory, which typically contains .po files for each language # -l zh: Specify the language code ``` ```bash # regular search \berror\b(?!\]) ```