{"owner":"microsoft","repo":"autogen","hasSkills":true,"totalSkillsCount":43,"totalTokensCount":19240,"categories":["plugin-manifest","copilot-instructions","anthropic-skill"],"hasMcp":true,"mcpConfig":{"mcpServers":{"autogen":{"command":"npx","args":["-y","@modelcontextprotocol/server-autogen"]}}},"found":["python/packages/autogen-test-utils/README.md",".github/copilot-instructions.md","python/packages/magentic-one-cli/README.md","python/packages/autogen-magentic-one/README.md","python/packages/autogen-ext/README.md","python/packages/autogen-core/README.md","python/packages/pyautogen/README.md","python/packages/component-schema-gen/README.md","python/packages/autogen-studio/README.md","python/packages/agbench/README.md","python/packages/autogen-agentchat/README.md","python/packages/agbench/CONTRIBUTING.md","python/packages/autogen-studio/.devcontainer/devcontainer.json","python/packages/autogen-studio/frontend/package.json","python/packages/autogen-studio/frontend/README.md","python/packages/autogen-studio/frontend/tsconfig.json","python/packages/autogen-studio/notebooks/team.json","python/packages/autogen-studio/notebooks/travel_team.json","python/packages/agbench/benchmarks/README.md","python/packages/agbench/benchmarks/GAIA/config.yaml","python/packages/agbench/benchmarks/GAIA/ENV.yaml","python/packages/agbench/benchmarks/GAIA/README.md","python/packages/agbench/benchmarks/HumanEval/config.yaml","python/packages/agbench/benchmarks/HumanEval/ENV.yaml","python/packages/agbench/benchmarks/HumanEval/README.md","python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml","python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml","python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml","python/packages/autogen-studio/frontend/src/components/shared/README.md","python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md","python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json","python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json","python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml","python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json","python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md"],"skills":{"python/packages/autogen-test-utils/README.md":"# test-utils\n",".github/copilot-instructions.md":"# AutoGen Multi-Agent AI Framework\n\nAutoGen is a multi-language framework for creating AI agents that can act autonomously or work alongside humans. The project has separate Python and .NET implementations with their own development workflows.\n\nAlways reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.\n\n## Working Effectively\n\n### Prerequisites and Environment Setup\n\n**CRITICAL**: Install both .NET 8.0 and 9.0 for full compatibility:\n- Install uv package manager: `python3 -m pip install uv` \n- Install .NET 9.0 SDK: `wget https://dot.net/v1/dotnet-install.sh && chmod +x dotnet-install.sh && ./dotnet-install.sh --channel 9.0`\n- Install .NET 8.0 runtime: `./dotnet-install.sh --channel 8.0 --runtime dotnet && ./dotnet-install.sh --channel 8.0 --runtime aspnetcore`\n- Update PATH: `export PATH=\"$HOME/.dotnet:$PATH\"`\n\n### Python Development Workflow\n\n**Bootstrap and build Python environment:**\n```bash\ncd /home/runner/work/autogen/autogen/python\nuv sync --all-extras  # NEVER CANCEL: Takes 2 minutes. Set timeout to 300+ seconds.\nsource .venv/bin/activate\n```\n\n**Validate Python development:**\n```bash\n# Quick validation (under 1 second each)\npoe format  # Code formatting\npoe lint    # Linting with ruff\n\n# Type checking - NEVER CANCEL these commands\npoe mypy     # Takes 6 minutes. Set timeout to 600+ seconds.\npoe pyright  # Takes 41 seconds. Set timeout to 120+ seconds.\n\n# Individual package testing (core package example)\npoe --directory ./packages/autogen-core test  # Takes 10 seconds. Set timeout to 60+ seconds.\n\n# Documentation - NEVER CANCEL\npoe docs-build  # Takes 1 minute 16 seconds. Set timeout to 300+ seconds.\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: Python environment setup takes 2 minutes minimum\n- **NEVER CANCEL**: mypy type checking takes 6 minutes \n- **NEVER CANCEL**: Documentation build takes 1+ minutes\n- Format/lint tasks complete in under 1 second\n- Individual package tests typically complete in 10-60 seconds\n\n### .NET Development Workflow\n\n**Bootstrap and build .NET environment:**\n```bash\ncd /home/runner/work/autogen/autogen/dotnet\nexport PATH=\"$HOME/.dotnet:$PATH\"\ndotnet restore  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\ndotnet build --configuration Release  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\n```\n\n**Validate .NET development:**\n```bash\n# Unit tests - NEVER CANCEL\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Takes 25 seconds. Set timeout to 120+ seconds.\n\n# Format check (if build fails) \ndotnet format --verify-no-changes\n\n# Run samples\ncd samples/Hello\ndotnet run\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: .NET restore takes 53 seconds minimum\n- **NEVER CANCEL**: .NET build takes 53 seconds minimum  \n- **NEVER CANCEL**: .NET unit tests take 25 seconds minimum\n- All build and test commands require appropriate timeouts\n\n### Complete Validation Workflow\n\n**Run full check suite (Python):**\n```bash\ncd /home/runner/work/autogen/autogen/python\nsource .venv/bin/activate\npoe check  # NEVER CANCEL: Runs all checks. Takes 7+ minutes total. Set timeout to 900+ seconds.\n```\n\n## Validation Scenarios\n\n### Manual Validation Requirements\nAlways manually validate changes by running complete user scenarios after making modifications:\n\n**Python validation scenarios:**\n1. **Import test**: Verify core imports work:\n   ```python\n   from autogen_agentchat.agents import AssistantAgent\n   from autogen_core import AgentRuntime\n   from autogen_ext.models.openai import OpenAIChatCompletionClient\n   ```\n\n2. **AutoGen Studio test**: Verify web interface can start:\n   ```bash\n   autogenstudio ui --help  # Should show help without errors\n   ```\n\n3. **Documentation test**: Build and verify docs generate without errors:\n   ```bash\n   poe docs-build && ls docs/build/index.html\n   ```\n\n**.NET validation scenarios:**\n1. **Sample execution**: Run Hello sample to verify runtime works:\n   ```bash\n   cd dotnet/samples/Hello && dotnet run --help\n   ```\n\n2. **Build validation**: Ensure all projects compile:\n   ```bash\n   dotnet build --configuration Release --no-restore\n   ```\n\n3. **Test execution**: Run unit tests to verify functionality:\n   ```bash\n   dotnet test --filter \"Category=UnitV2\" --configuration Release --no-build\n   ```\n\n## Common Issues and Workarounds\n\n### Network-Related Issues\n- **Python tests may fail** with network errors (tiktoken downloads, Playwright browser downloads) in sandboxed environments - this is expected\n- **Documentation intersphinx warnings** due to inability to reach external documentation sites - this is expected\n- **Individual package tests work better** than full test suite in network-restricted environments\n\n### .NET Runtime Issues  \n- **Requires both .NET 8.0 and 9.0**: Build uses 9.0 SDK but tests need 8.0 runtime\n- **Global.json specifies 9.0.100**: Must install exact .NET 9.0 version or later\n- **Path configuration critical**: Ensure `$HOME/.dotnet` is in PATH before system .NET\n\n### Python Package Issues\n- **Use uv exclusively**: Do not use pip/conda for dependency management\n- **Virtual environment required**: Always activate `.venv` before running commands\n- **Package workspace structure**: Project uses uv workspace with multiple packages\n\n## Timing Reference\n\n### Python Commands\n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `uv sync --all-extras` | 2 minutes | 300+ seconds | NEVER CANCEL |\n| `poe mypy` | 6 minutes | 600+ seconds | NEVER CANCEL |\n| `poe pyright` | 41 seconds | 120+ seconds | NEVER CANCEL |\n| `poe docs-build` | 1 min 16 sec | 300+ seconds | NEVER CANCEL |\n| `poe format` | <1 second | 30 seconds | Quick |\n| `poe lint` | <1 second | 30 seconds | Quick |\n| Individual package test | 10 seconds | 60+ seconds | May have network failures |\n\n### .NET Commands  \n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `dotnet restore` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet build --configuration Release` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet test --filter \"Category=UnitV2\"` | 25 seconds | 120+ seconds | NEVER CANCEL |\n| `dotnet format --verify-no-changes` | 5-10 seconds | 60 seconds | Quick validation |\n\n## Repository Structure\n\n### Python Packages (`python/packages/`)\n- `autogen-core`: Core agent runtime, model interfaces, and base components\n- `autogen-agentchat`: High-level multi-agent conversation APIs  \n- `autogen-ext`: Extensions for specific model providers and tools\n- `autogen-studio`: Web-based IDE for agent workflows\n- `agbench`: Benchmarking suite for agent performance\n- `magentic-one-cli`: Multi-agent team CLI application\n\n### .NET Projects (`dotnet/src/`)\n- `AutoGen`: Legacy 0.2-style .NET packages (being deprecated)\n- `Microsoft.AutoGen.*`: New event-driven .NET packages\n- `AutoGen.Core`: Core .NET agent functionality\n- Multiple provider packages: OpenAI, Anthropic, Ollama, etc.\n\n### Key Configuration Files\n- `python/pyproject.toml`: Python workspace and tool configuration\n- `dotnet/global.json`: .NET SDK version requirements  \n- `dotnet/AutoGen.sln`: .NET solution file\n- `python/uv.lock`: Locked Python dependencies\n\n## Development Best Practices\n\n### Before Committing Changes\n**ALWAYS run these validation steps:**\n\n**Python:**\n```bash\ncd python && source .venv/bin/activate\npoe format    # Fix formatting\npoe lint      # Check code quality  \npoe mypy      # Type checking (6 minutes)\npoe docs-build # Verify docs build (1+ minutes)\n```\n\n**.NET:**\n```bash  \ncd dotnet && export PATH=\"$HOME/.dotnet:$PATH\"\ndotnet format --verify-no-changes  # Check formatting\ndotnet build --configuration Release --no-restore  # Build (53 seconds)\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Test (25 seconds)\n```\n\n### Key Directories Reference\n```\nautogen/\n├── python/                    # Python implementation\n│   ├── packages/             # Individual Python packages\n│   ├── docs/                 # Sphinx documentation\n│   ├── samples/              # Example code\n│   └── pyproject.toml        # Workspace configuration\n├── dotnet/                   # .NET implementation  \n│   ├── src/                  # Source projects\n│   ├── test/                 # Test projects\n│   ├── samples/              # Sample applications\n│   └── AutoGen.sln          # Solution file\n├── .github/workflows/        # CI/CD pipelines\n└── docs/                     # Additional documentation\n```\n\nThis framework supports creating both simple single-agent applications and complex multi-agent workflows with support for various LLM providers, tools, and deployment patterns.","python/packages/magentic-one-cli/README.md":"# magentic-one-cli\n","python/packages/autogen-magentic-one/README.md":"# Magentic-One\n\n> Magentic-One is now available as part of the `autogen-agentchat` library.\n> Please see the [user guide](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html) for information.\n\n> Looking for the original implementation of Magentic-One? It is available [here](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one).\n\n[Magentic-One](https://aka.ms/magentic-one-blog) is a generalist multi-agent system for solving open-ended web and file-based tasks across a variety of domains. It represents a significant step forward for multi-agent systems, achieving competitive performance on a number of agentic benchmarks (see the [technical report](https://arxiv.org/abs/2411.04468) for full details).\n\nWhen originally released in [November 2024](https://aka.ms/magentic-one-blog) Magentic-One was [implemented directly on the `autogen-core` library](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one). We have now ported Magentic-One to use `autogen-agentchat`, providing a more modular and easier to use interface. To this end, the older implementation is deprecated, but can be accessed at [https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one).\n\nMoving forward, the Magentic-One orchestrator [MagenticOneGroupChat](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.teams.html#autogen_agentchat.teams.MagenticOneGroupChat) is now simply an AgentChat team, supporting all standard AgentChat agents and features. Likewise, Magentic-One's [MultimodalWebSurfer](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.web_surfer.html#autogen_ext.agents.web_surfer.MultimodalWebSurfer), [FileSurfer](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.file_surfer.html#autogen_ext.agents.file_surfer.FileSurfer), and [MagenticOneCoderAgent](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.teams.magentic_one.html) agents are now broadly available as AgentChat agents, to be used in any AgentChat workflows.\n\nLastly, there is a helper class, [MagenticOne](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.teams.magentic_one.html#autogen_ext.teams.magentic_one.MagenticOne), which bundles all of this together as it was in the paper with minimal configuration\n\n## Citation\n\n```\n@misc{fourney2024magenticonegeneralistmultiagentsolving,\n      title={Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks},\n      author={Adam Fourney and Gagan Bansal and Hussein Mozannar and Cheng Tan and Eduardo Salinas and Erkang and Zhu and Friederike Niedtner and Grace Proebsting and Griffin Bassman and Jack Gerrits and Jacob Alber and Peter Chang and Ricky Loynd and Robert West and Victor Dibia and Ahmed Awadallah and Ece Kamar and Rafah Hosn and Saleema Amershi},\n      year={2024},\n      eprint={2411.04468},\n      archivePrefix={arXiv},\n      primaryClass={cs.AI},\n      url={https://arxiv.org/abs/2411.04468},\n}\n```\n","python/packages/autogen-ext/README.md":"# AutoGen Extensions\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/extensions-user-guide/index.html)\n\nAutoGen is designed to be extensible. The `autogen-ext` package contains many different component implementations maintained by the AutoGen project. However, we strongly encourage others to build their own components and publish them as part of the ecosytem.\n","python/packages/autogen-core/README.md":"# AutoGen Core\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html)\n\nAutoGen core offers an easy way to quickly build event-driven, distributed, scalable, resilient AI agent systems. Agents are developed by using the [Actor model](https://en.wikipedia.org/wiki/Actor_model). You can build and run your agent system locally and easily move to a distributed system in the cloud when you are ready.\n","python/packages/pyautogen/README.md":"# pyautogen\n\n> **NOTE:** This is a proxy package for the latest version of [`autogen-agentchat`](https://pypi.org/project/autogen-agentchat/). If you are looking for the 0.2.x version, please pin to `pyautogen~=0.2.0`.\n> To migrate from 0.2.x to the latest version, please refer to the [migration guide](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/migration-guide.html).\n> Read our [previous clarification regarding to forks](https://github.com/microsoft/autogen/discussions/4217).\n> We have regained admin access to this package.\n\nAutoGen is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans.\n\n- [Project homepage](https://github.com/microsoft/autogen)\n- [Documentation](https://microsoft.github.io/autogen/)\n- [Discord](https://aka.ms/autogen-discord)\n- [Contact](mailto:autogen@microsoft.com)\n","python/packages/component-schema-gen/README.md":"# component-schema-gen\n\nThis is a tool to generate schema for built in components.\n\nSimply run `gen-component-schema` and it will print the schema to be used.\n","python/packages/autogen-studio/README.md":"# AutoGen Studio\n\n[![PyPI version](https://badge.fury.io/py/autogenstudio.svg)](https://badge.fury.io/py/autogenstudio)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/autogenstudio)\n\n![ARA](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)\n\nAutoGen Studio is an AutoGen-powered AI app (user interface) to help you rapidly prototype AI agents, enhance them with skills, compose them into workflows and interact with them to accomplish tasks. It is built on top of the [AutoGen](https://microsoft.github.io/autogen) framework, which is a toolkit for building AI agents.\n\nCode for AutoGen Studio is on GitHub at [microsoft/autogen](https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio)\n\n> [!CAUTION]\n> AutoGen Studio is meant to help you rapidly prototype multi-agent workflows and demonstrate an example of end user interfaces built with AutoGen. It is **not meant to be a production-ready app**. Developers are encouraged to use the [AutoGen framework](https://microsoft.github.io/autogen) to build their own applications, implementing authentication, security and other features required for deployed applications.\n\n> [!WARNING]\n> AutoGen Studio is under active development. Expect breaking changes in upcoming releases.\n\n## A Note on Security\n\nAutoGen Studio is a research prototype and is **not meant to be used** in a production environment. Some baseline practices are encouraged e.g., using Docker code execution environment for your agents.\n\nHowever, other considerations such as rigorous tests related to jailbreaking, ensuring LLMs only have access to the right keys of data given the end user's permissions, and other security features are not implemented in AutoGen Studio.\n\nIf you are building a production application, please use the [AutoGen framework](https://microsoft.github.io/autogen) and implement the necessary security features.\n\n## Updates\n\n- **2024-11-14:** AutoGen Studio is being rewritten to use the updated AutoGen 0.4.0 api AgentChat api.\n- **2024-04-17:** April 17: AutoGen Studio database layer is now rewritten to use [SQLModel](https://sqlmodel.tiangolo.com/) (Pydantic + SQLAlchemy). This provides entity linking (skills, models, agents and workflows are linked via association tables) and supports multiple [database backend dialects](https://docs.sqlalchemy.org/en/20/dialects/) supported in SQLAlchemy (SQLite, PostgreSQL, MySQL, Oracle, Microsoft SQL Server). The backend database can be specified a `--database-uri` argument when running the application. For example, `autogenstudio ui --database-uri sqlite:///database.sqlite` for SQLite and `autogenstudio ui --database-uri postgresql+psycopg://user:password@localhost/dbname` for PostgreSQL.\n- **2024-03-12:** Default directory for AutoGen Studio is now /home/\\<USER\\>/.autogenstudio. You can also specify this directory using the `--appdir` argument when running the application. For example, `autogenstudio ui --appdir /path/to/folder`. This will store the database and other files in the specified directory e.g. `/path/to/folder/database.sqlite`. `.env` files in that directory will be used to set environment variables for the app.\n\n## Project Structure:\n\n- `autogenstudio/` contains code for the backend classes and web api (FastAPI)\n- `frontend/` contains code for the webui, built with Gatsby and TailwindCSS\n\n## Installation\n\nThere are two ways to install AutoGen Studio - from PyPi or from the source. We **recommend installing from PyPi** unless you plan to modify the source code.\n\n### Install from PyPi (Recommended)\n\nWe recommend using a virtual environment (e.g., venv) to avoid conflicts with existing Python packages. With Python 3.10 or newer active in your virtual environment, use pip to install AutoGen Studio:\n\n```bash\npip install -U autogenstudio\n```\n\n### Install from source\n\n_Note: This approach requires some familiarity with building interfaces in React._\n\n### Important: Git LFS Requirement\n\nAutoGen Studio uses Git Large File Storage (LFS) for managing image and other large files. If you clone the repository without git-lfs, you'll encounter build errors related to image formats.\n\n**Before cloning the repository:**\n\n1. Install git-lfs:\n\n   ```bash\n   # On Debian/Ubuntu\n   apt-get install git-lfs\n\n   # On macOS with Homebrew\n   brew install git-lfs\n\n   # On Windows with Chocolatey\n   choco install git-lfs\n   ```\n\n2. Set up git-lfs:\n   ```bash\n   git lfs install\n   ```\n\n**If you've already cloned the repository:**\n\n```bash\ngit lfs install\ngit lfs fetch --all\ngit lfs checkout  # downloads all missing image files to the working directory\n```\n\nThis setup is handled automatically if you use the dev container method of installation.\n\nYou have two options for installing from source: manually or using a dev container.\n\n#### A) Install from source manually\n\n1. Ensure you have Python 3.10+ and Node.js (version above 14.15.0) installed.\n2. Clone the AutoGen Studio repository and install its Python dependencies using `pip install -e .`\n3. Navigate to the `python/packages/autogen-studio/frontend` directory, install the dependencies, and build the UI:\n\n   ```bash\n   npm install -g gatsby-cli\n   npm install --global yarn\n   cd frontend\n   yarn install\n   yarn build\n   # Windows users may need alternative commands to build the frontend:\n   gatsby clean && rmdir /s /q ..\\\\autogenstudio\\\\web\\\\ui 2>nul & (set \\\"PREFIX_PATH_VALUE=\\\" || ver>nul) && gatsby build --prefix-paths && xcopy /E /I /Y public ..\\\\autogenstudio\\\\web\\\\ui\n   ```\n\n#### B) Install from source using a dev container\n\n1. Follow the [Dev Containers tutorial](https://code.visualstudio.com/docs/devcontainers/tutorial) to install VS Code, Docker and relevant extensions.\n2. Clone the AutoGen Studio repository.\n3. Open `python/packages/autogen-studio/`in VS Code. Click the blue button in bottom the corner or press F1 and select _\"Dev Containers: Reopen in Container\"_.\n4. Build the UI:\n\n   ```bash\n   cd frontend\n   yarn build\n   ```\n\n### Running the Application\n\nOnce installed, run the web UI by entering the following in your terminal:\n\n```bash\nautogenstudio ui --port 8081\n```\n\nThis command will start the application on the specified port. Open your web browser and go to <http://localhost:8081/> to use AutoGen Studio.\n\nAutoGen Studio also takes several parameters to customize the application:\n\n- `--host <host>` argument to specify the host address. By default, it is set to `localhost`.\n- `--appdir <appdir>` argument to specify the directory where the app files (e.g., database and generated user files) are stored. By default, it is set to the `.autogenstudio` directory in the user's home directory.\n- `--port <port>` argument to specify the port number. By default, it is set to `8080`.\n- `--reload` argument to enable auto-reloading of the server when changes are made to the code. By default, it is set to `False`.\n- `--database-uri` argument to specify the database URI. Example values include `sqlite:///database.sqlite` for SQLite and `postgresql+psycopg://user:password@localhost/dbname` for PostgreSQL. If this is not specified, the database URL defaults to a `database.sqlite` file in the `--appdir` directory.\n- `--upgrade-database` argument to upgrade the database schema to the latest version. By default, it is set to `False`.\n\nNow that you have AutoGen Studio installed and running, you are ready to explore its capabilities, including defining and modifying agent workflows, interacting with agents and sessions, and expanding agent skills.\n\n## AutoGen Studio Lite\n\nAutoGen Studio Lite provides a lightweight way to quickly prototype and experiment with AI agent teams. It's designed for rapid experimentation without the full database setup.\n\n### CLI Usage\n\nLaunch Studio Lite from the command line:\n\n```bash\n# Quick start with default team\nautogenstudio lite\n\n# Use custom team file\nautogenstudio lite --team ./my_team.json --port 8080\n\n# Custom session name\nautogenstudio lite --session-name \"My Experiment\" --auto-open\n```\n\n### Programmatic Usage\n\nUse Studio Lite directly in your Python code:\n\n```python\nfrom autogenstudio.lite import LiteStudio\n\n# Quick start with default team\nstudio = LiteStudio()\n# Use with AutoGen team objects\nfrom autogen_agentchat.teams import RoundRobinGroupChat\nteam = RoundRobinGroupChat([agent1, agent2], termination_condition=...)\n\n# Context manager usage\nwith LiteStudio(team=team) as studio:\n    # Studio runs in background\n    # Do other work here\n    pass\n```\n\n#### Local frontend development server\n\nSee `./frontend/README.md`\n\n## Contribution Guide\n\nWe welcome contributions to AutoGen Studio. We recommend the following general steps to contribute to the project:\n\n- Review the overall AutoGen project [contribution guide](https://github.com/microsoft/autogen?tab=readme-ov-file#contributing)\n- Please review the AutoGen Studio [roadmap](https://github.com/microsoft/autogen/issues/4006) to get a sense of the current priorities for the project. Help is appreciated especially with Studio issues tagged with `help-wanted`\n- Please initiate a discussion on the roadmap issue or a new issue to discuss your proposed contribution.\n- Submit a pull request with your contribution!\n- If you are modifying AutoGen Studio, it has its own devcontainer. See instructions in `.devcontainer/README.md` to use it\n- Please use the tag `proj-studio` for any issues, questions, and PRs related to Studio\n\n## FAQ\n\nPlease refer to the AutoGen Studio [FAQs](https://microsoft.github.io/autogen/docs/autogen-studio/faqs) page for more information.\n\n## Acknowledgements\n\nAutoGen Studio is Based on the [AutoGen](https://microsoft.github.io/autogen) project. It was adapted from a research prototype built in October 2023 (original credits: Gagan Bansal, Adam Fourney, Victor Dibia, Piali Choudhury, Saleema Amershi, Ahmed Awadallah, Chi Wang).\n","python/packages/agbench/README.md":"# AutoGenBench\n\nAutoGenBench (agbench) is a tool for repeatedly running a set of pre-defined AutoGen tasks in a setting with tightly-controlled initial conditions. With each run, AutoGenBench will start from a blank slate. The agents being evaluated will need to work out what code needs to be written, and what libraries or dependencies to install, to solve tasks. The results of each run are logged, and can be ingested by analysis or metrics scripts (such as `agbench tabulate`). By default, all runs are conducted in freshly-initialized docker containers, providing the recommended level of consistency and safety.\n\nAutoGenBench works with all AutoGen 0.1.*, and 0.2.* versions.\n\n## Technical Specifications\n\nIf you are already an AutoGenBench pro, and want the full technical specifications, please review the [contributor&#39;s guide](CONTRIBUTING.md).\n\n## Docker Requirement\n\nAutoGenBench also requires Docker (Desktop or Engine). **It will not run in GitHub codespaces**, unless you opt for native execution (which is strongly discouraged). To install Docker Desktop see [https://www.docker.com/products/docker-desktop/](https://www.docker.com/products/docker-desktop/).\n\nIf you are working in WSL, you can follow the instructions below to set up your environment:\n\n1. Install Docker Desktop. After installation, restart is needed, then open Docker Desktop, in Settings, Ressources, WSL Integration, Enable integration with additional distros – Ubuntu\n2. Clone autogen and export `AUTOGEN_REPO_BASE`. This environment variable enables the Docker containers to use the correct version agents.\n    ```bash\n    git clone git@github.com:microsoft/autogen.git\n    export AUTOGEN_REPO_BASE=<path_to_autogen>\n    ```\n\n## Installation and Setup\n\n[Deprecated currently] **To get the most out of AutoGenBench, the `agbench` package should be installed**. At present, the easiest way to do this is to install it via `pip`.\n\n\nIf you would prefer working from source code (e.g., for development, or to utilize an alternate branch), simply clone the [AutoGen](https://github.com/microsoft/autogen) repository, then install `agbench` via:\n\n```\npip install -e autogen/python/packages/agbench\n```\n\nAfter installation, you must configure your API keys. As with other AutoGen applications, AutoGenBench will look for the OpenAI keys in the OAI_CONFIG_LIST file in the current working directory, or the OAI_CONFIG_LIST environment variable. This behavior can be overridden using a command-line parameter described later.\n\nIf you will be running multiple benchmarks, it is often most convenient to leverage the environment variable option. You can load your keys into the environment variable by executing:\n\n```\nexport OAI_CONFIG_LIST=$(cat ./OAI_CONFIG_LIST)\n```\n\nIf an OAI_CONFIG_LIST is *not* provided (by means of file or environment variable), AutoGenBench will use the OPENAI_API_KEY environment variable instead.\n\nFor some benchmark scenarios, additional keys may be required (e.g., keys for the Bing Search API). These can be added to an `ENV.json` file in the current working folder. An example `ENV.json` file is provided below:\n\n```\n{\n    \"BING_API_KEY\": \"xxxyyyzzz\"\n}\n```\n\n## A Typical Session\n\nOnce AutoGenBench and necessary keys are installed, a typical session will look as follows:\n\n\n\nNavigate to HumanEval\n\n```bash\ncd autogen/python/packages/agbench/benchmarks/HumanEval\n```\n**Note:** The following instructions are specific to the HumanEval benchmark. For other benchmarks, please refer to the README in the respective benchmark folder, e.g.,: [AssistantBench](benchmarks/AssistantBench/README.md).\n\n\nCreate a file called ENV.json with the following (required) contents (If you're using MagenticOne), if using Azure:\n\n```json\n{\n    \"CHAT_COMPLETION_KWARGS_JSON\": \"{}\",\n    \"CHAT_COMPLETION_PROVIDER\": \"azure\"\n}\n```\n\nYou can also use the openai client by replacing the last two entries in the ENV file by:\n\n- `CHAT_COMPLETION_PROVIDER='openai'`\n- `CHAT_COMPLETION_KWARGS_JSON` with the following JSON structure:\n\n```json\n{\n  \"api_key\": \"REPLACE_WITH_YOUR_API\",\n  \"model\": \"REPLACE_WITH_YOUR_MODEL\"\n}\n```\n\nNow initialize the tasks.\n\n```bash\npython Scripts/init_tasks.py\n```\n\nNote: This will attempt to download HumanEval\n\n\nOnce the script completes, you should now see a folder in your current directory called `Tasks` that contains one JSONL file per template in `Templates`.\n\nNow to run a specific subset of HumanEval use:\n\n```bash\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nYou should see the command line print the raw logs that shows the agents in action To see a summary of the results (e.g., task completion rates), in a new terminal run the following:\n\n```bash\nagbench tabulate Results/human_eval_MagenticOne\n```\n\nWhere:\n\n- `agbench run Tasks/human_eval_MagenticOne.jsonl` runs the tasks defined in `Tasks/human_eval_MagenticOne.jsonl`\n- `agbench tablue results/human_eval_MagenticOne` tabulates the results of the run\n\nEach of these commands has extensive in-line help via:\n\n- `agbench --help`\n- `agbench run --help`\n- `agbench tabulate --help`\n- `agbench remove_missing --help`\n\n**NOTE:** If you are running `agbench` from within the repository, you need to navigate to the appropriate scenario folder (e.g., `scenarios/HumanEval`) and run the `Scripts/init_tasks.py` file.\n\nMore details of each command are provided in the sections that follow.\n\n\n## Running AutoGenBench\n\nTo run a benchmark (which executes the tasks, but does not compute metrics), simply execute:\n\n```\ncd [BENCHMARK]\nagbench run Tasks/*.jsonl\n```\n\nFor example,\n\n```\ncd HumanEval\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe default is to run each task once. To run each scenario 10 times, use:\n\n```\nagbench run --repeat 10 Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe `agbench` command-line tool allows a number of command-line arguments to control various parameters of execution. Type ``agbench -h`` to explore these options:\n\n```\n'agbench run' will run the specified autogen scenarios for a given number of repetitions and record all logs and trace information. When running in a Docker environment (default), each run will begin from a common, tightly controlled, environment. The resultant logs can then be further processed by other scripts to produce metrics.\n\npositional arguments:\n  scenario      The JSONL scenario file to run. If a directory is specified,\n                then all JSONL scenarios in the directory are run. (default:\n                ./scenarios)\n\noptions:\n  -h, --help            show this help message and exit\n  -c CONFIG, --config CONFIG\n                        The environment variable name or path to the OAI_CONFIG_LIST (default: OAI_CONFIG_LIST).\n  -r REPEAT, --repeat REPEAT\n                        The number of repetitions to run for each scenario (default: 1).\n  -s SUBSAMPLE, --subsample SUBSAMPLE\n                        Run on a subsample of the tasks in the JSONL file(s). If a decimal value is specified, then run on\n                        the given proportion of tasks in each file. For example \"0.7\" would run on 70% of tasks, and \"1.0\"\n                        would run on 100% of tasks. If an integer value is specified, then randomly select *that* number of\n                        tasks from each specified JSONL file. For example \"7\" would run tasks, while \"1\" would run only 1\n                        task from each specified JSONL file. (default: 1.0; which is 100%)\n  -m MODEL, --model MODEL\n                        Filters the config_list to include only models matching the provided model name (default: None, which\n                        is all models).\n  --requirements REQUIREMENTS\n                        The requirements file to pip install before running the scenario.\n  -d DOCKER_IMAGE, --docker-image DOCKER_IMAGE\n                        The Docker image to use when running scenarios. Can not be used together with --native. (default:\n                        'agbench:default', which will be created if not present)\n  --native              Run the scenarios natively rather than in docker. NOTE: This is not advisable, and should be done\n                        with great caution.\n```\n\n## Results\n\nBy default, the AutoGenBench stores results in a folder hierarchy with the following template:\n\n``./results/[scenario]/[task_id]/[instance_id]``\n\nFor example, consider the following folders:\n\n``./results/default_two_agents/two_agent_stocks/0``\n``./results/default_two_agents/two_agent_stocks/1``\n\n...\n\n``./results/default_two_agents/two_agent_stocks/9``\n\nThis folder holds the results for the ``two_agent_stocks`` task of the ``default_two_agents`` tasks file. The ``0`` folder contains the results of the first instance / run. The ``1`` folder contains the results of the second run, and so on. You can think of the _task_id_ as mapping to a prompt, or a unique set of parameters, while the _instance_id_ defines a specific attempt or run.\n\nWithin each folder, you will find the following files:\n\n- *timestamp.txt*: records the date and time of the run, along with the version of the autogen-agentchat library installed\n- *console_log.txt*: all console output produced by Docker when running AutoGen. Read this like you would a regular console.\n- *[agent]_messages.json*: for each Agent, a log of their messages dictionaries\n- *./coding*: A directory containing all code written by AutoGen, and all artifacts produced by that code.\n\n## Contributing or Defining New Tasks or Benchmarks\n\nIf you would like to develop -- or even contribute -- your own tasks or benchmarks, please review the [contributor&#39;s guide](CONTRIBUTING.md) for complete technical details.\n","python/packages/autogen-agentchat/README.md":"# AutoGen AgentChat\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/index.html)\n\nAgentChat is a high-level API for building multi-agent applications.\nIt is built on top of the [`autogen-core`](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html) package.\nFor beginner users, AgentChat is the recommended starting point.\nFor advanced users, [`autogen-core`](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html)'s event-driven\nprogramming model provides more flexibility and control over the underlying components.\n\nAgentChat provides intuitive defaults, such as **Agents** with preset\nbehaviors and **Teams** with predefined [multi-agent design patterns](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/design-patterns/intro.html).\n","python/packages/agbench/CONTRIBUTING.md":"# Contributing to AutoGenBench\n\nAs part of the broader AutoGen project, AutoGenBench welcomes community contributions. Contributions are subject to AutoGen's [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute), as well as a few additional AutoGenBench-specific requirements outlined here. You may also wish to develop your own private benchmark scenarios and the guidance in this document will help with such efforts as well. Below you will find the general requirements, followed by a detailed technical description.\n\n## General Contribution Requirements\nWe ask that all contributions to AutoGenBench adhere to the following:\n\n- Follow AutoGen's broader [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute)\n- All AutoGenBench benchmarks should live in a subfolder of `/benchmarks` alongside `HumanEval`, `GAIA`, etc.\n- Benchmark scenarios should include a detailed README.md, in the root of their folder, describing the benchmark and providing citations where warranted.\n- Benchmark data (tasks, ground truth, etc.) should be downloaded from their original sources rather than hosted in the AutoGen repository (unless the benchmark is original, and the repository *is* the original source)\n    - You can use the `Scripts/init_tasks.py` file to automate this download.\n- Basic scoring should be compatible with the `agbench tabulate` command (e.g., by outputting logs compatible with the default tabulation mechanism, or by providing a `Scripts/custom_tabulate.py` file)\n\nThese requirements are further detailed below, but if you simply copy the `HumanEval` folder, you will already be off to a great start.\n\n## Implementing and Running Benchmark Tasks\nAt the core of any benchmark is a set of tasks. To implement tasks that are runnable by AutoGenBench, you must adhere to AutoGenBench's templating and scenario expansion algorithms, as outlined below.\n\n### Task Definitions\n\nAll tasks are stored in JSONL files (in subdirectories under `./Tasks`). Each line of a tasks file is a JSON object with the following schema:\n\n```\n{\n   \"id\": string,\n   \"template\": dirname,\n   \"substitutions\" {\n       \"filename1\": {\n       \t   \"find_string1_1\": replace_string1_1,\n           \"find_string1_2\": replace_string1_2,\n           ...\n           \"find_string1_M\": replace_string1_N\n       }\n       \"filename2\": {\n       \t   \"find_string2_1\": replace_string2_1,\n           \"find_string2_2\": replace_string2_2,\n           ...\n           \"find_string2_N\": replace_string2_N\n       }\n   }\n}\n```\n\nFor example:\n\n```\n{\n    \"id\": \"two_agent_stocks_gpt4\",\n    \"template\": \"default_two_agents\",\n    \"substitutions\": {\n\t\"scenario.py\": {\n            \"__MODEL__\": \"gpt-4\",\n\t},\n\t\"prompt.txt\": {\n            \"__PROMPT__\": \"Plot and save to disk a chart of NVDA and TESLA stock price YTD.\"\n        }\n    }\n}\n```\n\nIn this example, the string `__MODEL__` will be replaced in the file `scenarios.py`, while the string `__PROMPT__` will be replaced in the `prompt.txt` file.\n\nThe `template` field can also take on a list value, but this usage is considered advanced and is not described here. See the `agbench/run_cmd.py` code, or the `GAIA` benchmark tasks files for additional information about this option.\n\n\n## Task Instance Expansion Algorithm\n\nOnce the tasks have been defined, as per above, they must be \"instantiated\" before they can be run. This instantiation happens automatically when the user issues the `agbench run` command and involves creating a local folder to share with Docker. Each instance and repetition gets its own folder along the path: `./results/[scenario]/[task_id]/[instance_id]`. For the sake of brevity we will refer to this folder as the `DEST_FOLDER`.\n\nThe algorithm for populating the `DEST_FOLDER` is as follows:\n\n1. Pre-populate DEST_FOLDER with all the basic starter files for running a scenario (found in `agbench/template`).\n2. Recursively copy the template folder specified in the JSONL line to DEST_FOLDER (if the JSON `template` attribute points to a folder) If the JSONs `template` attribute instead points to a file, copy the file, but rename it to `scenario.py`\n3. Apply any string replacements, as outlined in the prior section.\n4. Write a run.sh file to DEST_FOLDER that will be executed by Docker when it is loaded. The `run.sh` is described below.\n\n## Scenario Execution Algorithm\n\nOnce the task has been instantiated it is run (via run.sh). This script will execute the following steps:\n\n1. If a file named `global_init.sh` is present, run it.\n2. If a file named `scenario_init.sh` is present, run it.\n3. Install the requirements.txt file (if running in Docker)\n4. Run the task via `python scenario.py`\n5. If the scenario.py exited cleanly (exit code 0), then print \"SCENARIO.PY COMPLETE !#!#\"\n6. Clean up (delete cache, etc.)\n7. If a file named `scenario_finalize.sh` is present, run it.\n8. If a file named `global_finalize.sh` is present, run it.\n9. echo \"RUN COMPLETE !#!#\", signaling that all steps completed.\n\nNotably, this means that scenarios can add custom init and teardown logic by including `scenario_init.sh` and `scenario_finalize.sh` files.\n\nAt the time of this writing, the run.sh file is as follows:\n\n```sh\nexport AUTOGEN_TESTBED_SETTING=\"Docker\"\numask 000\n\n# Run the global init script if it exists\nif [ -f global_init.sh ] ; then\n    . ./global_init.sh\nfi\n\n# Run the scenario init script if it exists\nif [ -f scenario_init.sh ] ; then\n    . ./scenario_init.sh\nfi\n\n# Run the scenario\npip install -r requirements.txt\npython scenario.py\nEXIT_CODE=$?\nif [ $EXIT_CODE -ne 0 ]; then\n    echo SCENARIO.PY EXITED WITH CODE: $EXIT_CODE !#!#\nelse\n    echo SCENARIO.PY COMPLETE !#!#\nfi\n\n# Clean up\nif [ -d .cache ] ; then\n    rm -Rf .cache\nfi\n\n# Run the scenario finalize script if it exists\nif [ -f scenario_finalize.sh ] ; then\n    . ./scenario_finalize.sh\nfi\n\n# Run the global finalize script if it exists\nif [ -f global_finalize.sh ] ; then\n    . ./global_finalize.sh\nfi\n\necho RUN.SH COMPLETE !#!#\n```\n\nBe warned that this listing is provided here for illustration purposes, and may vary over time. The source of truth are the `run.sh` files found in the ``./results/[taskset]/[task_id]/[instance_id]`` folders.\n\n\n## Integrating with the `tabulate` \nThe above details are sufficient for defining and running tasks, but if you wish to support the `agbench tabulate`  commands, a few additional steps are required.\n\n### Tabulations\n\nIf you wish to leverage the default tabulation logic, it is as simple as arranging your `scenario.py` file to output the string \"ALL TESTS PASSED !#!#\" to the console in the event that a task was solved correctly.\n\nIf you wish to implement your own tabulation logic, simply create the file `Scripts/custom_tabulate.py` and include a `main(args)` method. Here, the `args` parameter will be provided by AutoGenBench, and is a drop-in replacement for `sys.argv`. In particular, `args[0]` will be the invocation command (similar to the executable or script name in `sys.argv`), and the remaining values (`args[1:]`) are the command line parameters.\n\nShould you provide a custom tabulation script, please implement `--help` and `-h` options for documenting your interface.\n\nThe `scenarios/GAIA/Scripts/custom_tabulate.py` is a great example of custom tabulation. It also shows how you can reuse some components of the default tabulator to speed up development.\n\n\n\n## Scripts/init_tasks.py\nFinally, you should provide an `Scripts/init_tasks.py` file, in your benchmark folder, and include a `main()` method therein. \n\nThis `init_tasks.py` script is a great place to download benchmarks from their original sources and convert them to the JSONL format required by AutoGenBench:\n- See `HumanEval/Scripts/init_tasks.py` for an example of how to expand a benchmark from an original GitHub repository.\n- See `GAIA/Scripts/init_tasks.py` for an example of how to expand a benchmark from `Hugging Face Hub`.\n","python/packages/autogen-studio/.devcontainer/devcontainer.json":"# .devcontainer Skill\n\nPath: `python/packages/autogen-studio/.devcontainer/devcontainer.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/.devcontainer/devcontainer.json)","python/packages/autogen-studio/frontend/package.json":"# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/package.json)","python/packages/autogen-studio/frontend/README.md":"# frontend Documentation\n\nPath: `python/packages/autogen-studio/frontend/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/README.md)","python/packages/autogen-studio/frontend/tsconfig.json":"# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/tsconfig.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/tsconfig.json)","python/packages/autogen-studio/notebooks/team.json":"# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/team.json)","python/packages/autogen-studio/notebooks/travel_team.json":"# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/travel_team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/travel_team.json)","python/packages/agbench/benchmarks/README.md":"# benchmarks Documentation\n\nPath: `python/packages/agbench/benchmarks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/README.md)","python/packages/agbench/benchmarks/GAIA/config.yaml":"# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/config.yaml)","python/packages/agbench/benchmarks/GAIA/ENV.yaml":"# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/ENV.yaml)","python/packages/agbench/benchmarks/GAIA/README.md":"# GAIA Documentation\n\nPath: `python/packages/agbench/benchmarks/GAIA/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/README.md)","python/packages/agbench/benchmarks/HumanEval/config.yaml":"# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/config.yaml)","python/packages/agbench/benchmarks/HumanEval/ENV.yaml":"# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/ENV.yaml)","python/packages/agbench/benchmarks/HumanEval/README.md":"# HumanEval Documentation\n\nPath: `python/packages/agbench/benchmarks/HumanEval/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/README.md)","python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml)","python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml)","python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml)","python/packages/autogen-studio/frontend/src/components/shared/README.md":"# shared Documentation\n\nPath: `python/packages/autogen-studio/frontend/src/components/shared/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/shared/README.md)","python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md":"# task_centric_memory Documentation\n\nPath: `python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md)","python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json":"# Teachability Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json)","python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json":"# Self_teaching Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json)","python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json":"# Demonstration Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml)","python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json":"# Gallery Skill\n\nPath: `python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json)","python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md":"# _host Documentation\n\nPath: `python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md)"},"files":{"python/packages/autogen-test-utils/README.md":"# test-utils\n",".github/copilot-instructions.md":"# AutoGen Multi-Agent AI Framework\n\nAutoGen is a multi-language framework for creating AI agents that can act autonomously or work alongside humans. The project has separate Python and .NET implementations with their own development workflows.\n\nAlways reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.\n\n## Working Effectively\n\n### Prerequisites and Environment Setup\n\n**CRITICAL**: Install both .NET 8.0 and 9.0 for full compatibility:\n- Install uv package manager: `python3 -m pip install uv` \n- Install .NET 9.0 SDK: `wget https://dot.net/v1/dotnet-install.sh && chmod +x dotnet-install.sh && ./dotnet-install.sh --channel 9.0`\n- Install .NET 8.0 runtime: `./dotnet-install.sh --channel 8.0 --runtime dotnet && ./dotnet-install.sh --channel 8.0 --runtime aspnetcore`\n- Update PATH: `export PATH=\"$HOME/.dotnet:$PATH\"`\n\n### Python Development Workflow\n\n**Bootstrap and build Python environment:**\n```bash\ncd /home/runner/work/autogen/autogen/python\nuv sync --all-extras  # NEVER CANCEL: Takes 2 minutes. Set timeout to 300+ seconds.\nsource .venv/bin/activate\n```\n\n**Validate Python development:**\n```bash\n# Quick validation (under 1 second each)\npoe format  # Code formatting\npoe lint    # Linting with ruff\n\n# Type checking - NEVER CANCEL these commands\npoe mypy     # Takes 6 minutes. Set timeout to 600+ seconds.\npoe pyright  # Takes 41 seconds. Set timeout to 120+ seconds.\n\n# Individual package testing (core package example)\npoe --directory ./packages/autogen-core test  # Takes 10 seconds. Set timeout to 60+ seconds.\n\n# Documentation - NEVER CANCEL\npoe docs-build  # Takes 1 minute 16 seconds. Set timeout to 300+ seconds.\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: Python environment setup takes 2 minutes minimum\n- **NEVER CANCEL**: mypy type checking takes 6 minutes \n- **NEVER CANCEL**: Documentation build takes 1+ minutes\n- Format/lint tasks complete in under 1 second\n- Individual package tests typically complete in 10-60 seconds\n\n### .NET Development Workflow\n\n**Bootstrap and build .NET environment:**\n```bash\ncd /home/runner/work/autogen/autogen/dotnet\nexport PATH=\"$HOME/.dotnet:$PATH\"\ndotnet restore  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\ndotnet build --configuration Release  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\n```\n\n**Validate .NET development:**\n```bash\n# Unit tests - NEVER CANCEL\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Takes 25 seconds. Set timeout to 120+ seconds.\n\n# Format check (if build fails) \ndotnet format --verify-no-changes\n\n# Run samples\ncd samples/Hello\ndotnet run\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: .NET restore takes 53 seconds minimum\n- **NEVER CANCEL**: .NET build takes 53 seconds minimum  \n- **NEVER CANCEL**: .NET unit tests take 25 seconds minimum\n- All build and test commands require appropriate timeouts\n\n### Complete Validation Workflow\n\n**Run full check suite (Python):**\n```bash\ncd /home/runner/work/autogen/autogen/python\nsource .venv/bin/activate\npoe check  # NEVER CANCEL: Runs all checks. Takes 7+ minutes total. Set timeout to 900+ seconds.\n```\n\n## Validation Scenarios\n\n### Manual Validation Requirements\nAlways manually validate changes by running complete user scenarios after making modifications:\n\n**Python validation scenarios:**\n1. **Import test**: Verify core imports work:\n   ```python\n   from autogen_agentchat.agents import AssistantAgent\n   from autogen_core import AgentRuntime\n   from autogen_ext.models.openai import OpenAIChatCompletionClient\n   ```\n\n2. **AutoGen Studio test**: Verify web interface can start:\n   ```bash\n   autogenstudio ui --help  # Should show help without errors\n   ```\n\n3. **Documentation test**: Build and verify docs generate without errors:\n   ```bash\n   poe docs-build && ls docs/build/index.html\n   ```\n\n**.NET validation scenarios:**\n1. **Sample execution**: Run Hello sample to verify runtime works:\n   ```bash\n   cd dotnet/samples/Hello && dotnet run --help\n   ```\n\n2. **Build validation**: Ensure all projects compile:\n   ```bash\n   dotnet build --configuration Release --no-restore\n   ```\n\n3. **Test execution**: Run unit tests to verify functionality:\n   ```bash\n   dotnet test --filter \"Category=UnitV2\" --configuration Release --no-build\n   ```\n\n## Common Issues and Workarounds\n\n### Network-Related Issues\n- **Python tests may fail** with network errors (tiktoken downloads, Playwright browser downloads) in sandboxed environments - this is expected\n- **Documentation intersphinx warnings** due to inability to reach external documentation sites - this is expected\n- **Individual package tests work better** than full test suite in network-restricted environments\n\n### .NET Runtime Issues  \n- **Requires both .NET 8.0 and 9.0**: Build uses 9.0 SDK but tests need 8.0 runtime\n- **Global.json specifies 9.0.100**: Must install exact .NET 9.0 version or later\n- **Path configuration critical**: Ensure `$HOME/.dotnet` is in PATH before system .NET\n\n### Python Package Issues\n- **Use uv exclusively**: Do not use pip/conda for dependency management\n- **Virtual environment required**: Always activate `.venv` before running commands\n- **Package workspace structure**: Project uses uv workspace with multiple packages\n\n## Timing Reference\n\n### Python Commands\n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `uv sync --all-extras` | 2 minutes | 300+ seconds | NEVER CANCEL |\n| `poe mypy` | 6 minutes | 600+ seconds | NEVER CANCEL |\n| `poe pyright` | 41 seconds | 120+ seconds | NEVER CANCEL |\n| `poe docs-build` | 1 min 16 sec | 300+ seconds | NEVER CANCEL |\n| `poe format` | <1 second | 30 seconds | Quick |\n| `poe lint` | <1 second | 30 seconds | Quick |\n| Individual package test | 10 seconds | 60+ seconds | May have network failures |\n\n### .NET Commands  \n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `dotnet restore` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet build --configuration Release` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet test --filter \"Category=UnitV2\"` | 25 seconds | 120+ seconds | NEVER CANCEL |\n| `dotnet format --verify-no-changes` | 5-10 seconds | 60 seconds | Quick validation |\n\n## Repository Structure\n\n### Python Packages (`python/packages/`)\n- `autogen-core`: Core agent runtime, model interfaces, and base components\n- `autogen-agentchat`: High-level multi-agent conversation APIs  \n- `autogen-ext`: Extensions for specific model providers and tools\n- `autogen-studio`: Web-based IDE for agent workflows\n- `agbench`: Benchmarking suite for agent performance\n- `magentic-one-cli`: Multi-agent team CLI application\n\n### .NET Projects (`dotnet/src/`)\n- `AutoGen`: Legacy 0.2-style .NET packages (being deprecated)\n- `Microsoft.AutoGen.*`: New event-driven .NET packages\n- `AutoGen.Core`: Core .NET agent functionality\n- Multiple provider packages: OpenAI, Anthropic, Ollama, etc.\n\n### Key Configuration Files\n- `python/pyproject.toml`: Python workspace and tool configuration\n- `dotnet/global.json`: .NET SDK version requirements  \n- `dotnet/AutoGen.sln`: .NET solution file\n- `python/uv.lock`: Locked Python dependencies\n\n## Development Best Practices\n\n### Before Committing Changes\n**ALWAYS run these validation steps:**\n\n**Python:**\n```bash\ncd python && source .venv/bin/activate\npoe format    # Fix formatting\npoe lint      # Check code quality  \npoe mypy      # Type checking (6 minutes)\npoe docs-build # Verify docs build (1+ minutes)\n```\n\n**.NET:**\n```bash  \ncd dotnet && export PATH=\"$HOME/.dotnet:$PATH\"\ndotnet format --verify-no-changes  # Check formatting\ndotnet build --configuration Release --no-restore  # Build (53 seconds)\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Test (25 seconds)\n```\n\n### Key Directories Reference\n```\nautogen/\n├── python/                    # Python implementation\n│   ├── packages/             # Individual Python packages\n│   ├── docs/                 # Sphinx documentation\n│   ├── samples/              # Example code\n│   └── pyproject.toml        # Workspace configuration\n├── dotnet/                   # .NET implementation  \n│   ├── src/                  # Source projects\n│   ├── test/                 # Test projects\n│   ├── samples/              # Sample applications\n│   └── AutoGen.sln          # Solution file\n├── .github/workflows/        # CI/CD pipelines\n└── docs/                     # Additional documentation\n```\n\nThis framework supports creating both simple single-agent applications and complex multi-agent workflows with support for various LLM providers, tools, and deployment patterns.","python/packages/magentic-one-cli/README.md":"# magentic-one-cli\n","python/packages/autogen-magentic-one/README.md":"# Magentic-One\n\n> Magentic-One is now available as part of the `autogen-agentchat` library.\n> Please see the [user guide](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html) for information.\n\n> Looking for the original implementation of Magentic-One? It is available [here](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one).\n\n[Magentic-One](https://aka.ms/magentic-one-blog) is a generalist multi-agent system for solving open-ended web and file-based tasks across a variety of domains. It represents a significant step forward for multi-agent systems, achieving competitive performance on a number of agentic benchmarks (see the [technical report](https://arxiv.org/abs/2411.04468) for full details).\n\nWhen originally released in [November 2024](https://aka.ms/magentic-one-blog) Magentic-One was [implemented directly on the `autogen-core` library](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one). We have now ported Magentic-One to use `autogen-agentchat`, providing a more modular and easier to use interface. To this end, the older implementation is deprecated, but can be accessed at [https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one).\n\nMoving forward, the Magentic-One orchestrator [MagenticOneGroupChat](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.teams.html#autogen_agentchat.teams.MagenticOneGroupChat) is now simply an AgentChat team, supporting all standard AgentChat agents and features. Likewise, Magentic-One's [MultimodalWebSurfer](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.web_surfer.html#autogen_ext.agents.web_surfer.MultimodalWebSurfer), [FileSurfer](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.file_surfer.html#autogen_ext.agents.file_surfer.FileSurfer), and [MagenticOneCoderAgent](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.teams.magentic_one.html) agents are now broadly available as AgentChat agents, to be used in any AgentChat workflows.\n\nLastly, there is a helper class, [MagenticOne](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.teams.magentic_one.html#autogen_ext.teams.magentic_one.MagenticOne), which bundles all of this together as it was in the paper with minimal configuration\n\n## Citation\n\n```\n@misc{fourney2024magenticonegeneralistmultiagentsolving,\n      title={Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks},\n      author={Adam Fourney and Gagan Bansal and Hussein Mozannar and Cheng Tan and Eduardo Salinas and Erkang and Zhu and Friederike Niedtner and Grace Proebsting and Griffin Bassman and Jack Gerrits and Jacob Alber and Peter Chang and Ricky Loynd and Robert West and Victor Dibia and Ahmed Awadallah and Ece Kamar and Rafah Hosn and Saleema Amershi},\n      year={2024},\n      eprint={2411.04468},\n      archivePrefix={arXiv},\n      primaryClass={cs.AI},\n      url={https://arxiv.org/abs/2411.04468},\n}\n```\n","python/packages/autogen-ext/README.md":"# AutoGen Extensions\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/extensions-user-guide/index.html)\n\nAutoGen is designed to be extensible. The `autogen-ext` package contains many different component implementations maintained by the AutoGen project. However, we strongly encourage others to build their own components and publish them as part of the ecosytem.\n","python/packages/autogen-core/README.md":"# AutoGen Core\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html)\n\nAutoGen core offers an easy way to quickly build event-driven, distributed, scalable, resilient AI agent systems. Agents are developed by using the [Actor model](https://en.wikipedia.org/wiki/Actor_model). You can build and run your agent system locally and easily move to a distributed system in the cloud when you are ready.\n","python/packages/pyautogen/README.md":"# pyautogen\n\n> **NOTE:** This is a proxy package for the latest version of [`autogen-agentchat`](https://pypi.org/project/autogen-agentchat/). If you are looking for the 0.2.x version, please pin to `pyautogen~=0.2.0`.\n> To migrate from 0.2.x to the latest version, please refer to the [migration guide](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/migration-guide.html).\n> Read our [previous clarification regarding to forks](https://github.com/microsoft/autogen/discussions/4217).\n> We have regained admin access to this package.\n\nAutoGen is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans.\n\n- [Project homepage](https://github.com/microsoft/autogen)\n- [Documentation](https://microsoft.github.io/autogen/)\n- [Discord](https://aka.ms/autogen-discord)\n- [Contact](mailto:autogen@microsoft.com)\n","python/packages/component-schema-gen/README.md":"# component-schema-gen\n\nThis is a tool to generate schema for built in components.\n\nSimply run `gen-component-schema` and it will print the schema to be used.\n","python/packages/autogen-studio/README.md":"# AutoGen Studio\n\n[![PyPI version](https://badge.fury.io/py/autogenstudio.svg)](https://badge.fury.io/py/autogenstudio)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/autogenstudio)\n\n![ARA](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)\n\nAutoGen Studio is an AutoGen-powered AI app (user interface) to help you rapidly prototype AI agents, enhance them with skills, compose them into workflows and interact with them to accomplish tasks. It is built on top of the [AutoGen](https://microsoft.github.io/autogen) framework, which is a toolkit for building AI agents.\n\nCode for AutoGen Studio is on GitHub at [microsoft/autogen](https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio)\n\n> [!CAUTION]\n> AutoGen Studio is meant to help you rapidly prototype multi-agent workflows and demonstrate an example of end user interfaces built with AutoGen. It is **not meant to be a production-ready app**. Developers are encouraged to use the [AutoGen framework](https://microsoft.github.io/autogen) to build their own applications, implementing authentication, security and other features required for deployed applications.\n\n> [!WARNING]\n> AutoGen Studio is under active development. Expect breaking changes in upcoming releases.\n\n## A Note on Security\n\nAutoGen Studio is a research prototype and is **not meant to be used** in a production environment. Some baseline practices are encouraged e.g., using Docker code execution environment for your agents.\n\nHowever, other considerations such as rigorous tests related to jailbreaking, ensuring LLMs only have access to the right keys of data given the end user's permissions, and other security features are not implemented in AutoGen Studio.\n\nIf you are building a production application, please use the [AutoGen framework](https://microsoft.github.io/autogen) and implement the necessary security features.\n\n## Updates\n\n- **2024-11-14:** AutoGen Studio is being rewritten to use the updated AutoGen 0.4.0 api AgentChat api.\n- **2024-04-17:** April 17: AutoGen Studio database layer is now rewritten to use [SQLModel](https://sqlmodel.tiangolo.com/) (Pydantic + SQLAlchemy). This provides entity linking (skills, models, agents and workflows are linked via association tables) and supports multiple [database backend dialects](https://docs.sqlalchemy.org/en/20/dialects/) supported in SQLAlchemy (SQLite, PostgreSQL, MySQL, Oracle, Microsoft SQL Server). The backend database can be specified a `--database-uri` argument when running the application. For example, `autogenstudio ui --database-uri sqlite:///database.sqlite` for SQLite and `autogenstudio ui --database-uri postgresql+psycopg://user:password@localhost/dbname` for PostgreSQL.\n- **2024-03-12:** Default directory for AutoGen Studio is now /home/\\<USER\\>/.autogenstudio. You can also specify this directory using the `--appdir` argument when running the application. For example, `autogenstudio ui --appdir /path/to/folder`. This will store the database and other files in the specified directory e.g. `/path/to/folder/database.sqlite`. `.env` files in that directory will be used to set environment variables for the app.\n\n## Project Structure:\n\n- `autogenstudio/` contains code for the backend classes and web api (FastAPI)\n- `frontend/` contains code for the webui, built with Gatsby and TailwindCSS\n\n## Installation\n\nThere are two ways to install AutoGen Studio - from PyPi or from the source. We **recommend installing from PyPi** unless you plan to modify the source code.\n\n### Install from PyPi (Recommended)\n\nWe recommend using a virtual environment (e.g., venv) to avoid conflicts with existing Python packages. With Python 3.10 or newer active in your virtual environment, use pip to install AutoGen Studio:\n\n```bash\npip install -U autogenstudio\n```\n\n### Install from source\n\n_Note: This approach requires some familiarity with building interfaces in React._\n\n### Important: Git LFS Requirement\n\nAutoGen Studio uses Git Large File Storage (LFS) for managing image and other large files. If you clone the repository without git-lfs, you'll encounter build errors related to image formats.\n\n**Before cloning the repository:**\n\n1. Install git-lfs:\n\n   ```bash\n   # On Debian/Ubuntu\n   apt-get install git-lfs\n\n   # On macOS with Homebrew\n   brew install git-lfs\n\n   # On Windows with Chocolatey\n   choco install git-lfs\n   ```\n\n2. Set up git-lfs:\n   ```bash\n   git lfs install\n   ```\n\n**If you've already cloned the repository:**\n\n```bash\ngit lfs install\ngit lfs fetch --all\ngit lfs checkout  # downloads all missing image files to the working directory\n```\n\nThis setup is handled automatically if you use the dev container method of installation.\n\nYou have two options for installing from source: manually or using a dev container.\n\n#### A) Install from source manually\n\n1. Ensure you have Python 3.10+ and Node.js (version above 14.15.0) installed.\n2. Clone the AutoGen Studio repository and install its Python dependencies using `pip install -e .`\n3. Navigate to the `python/packages/autogen-studio/frontend` directory, install the dependencies, and build the UI:\n\n   ```bash\n   npm install -g gatsby-cli\n   npm install --global yarn\n   cd frontend\n   yarn install\n   yarn build\n   # Windows users may need alternative commands to build the frontend:\n   gatsby clean && rmdir /s /q ..\\\\autogenstudio\\\\web\\\\ui 2>nul & (set \\\"PREFIX_PATH_VALUE=\\\" || ver>nul) && gatsby build --prefix-paths && xcopy /E /I /Y public ..\\\\autogenstudio\\\\web\\\\ui\n   ```\n\n#### B) Install from source using a dev container\n\n1. Follow the [Dev Containers tutorial](https://code.visualstudio.com/docs/devcontainers/tutorial) to install VS Code, Docker and relevant extensions.\n2. Clone the AutoGen Studio repository.\n3. Open `python/packages/autogen-studio/`in VS Code. Click the blue button in bottom the corner or press F1 and select _\"Dev Containers: Reopen in Container\"_.\n4. Build the UI:\n\n   ```bash\n   cd frontend\n   yarn build\n   ```\n\n### Running the Application\n\nOnce installed, run the web UI by entering the following in your terminal:\n\n```bash\nautogenstudio ui --port 8081\n```\n\nThis command will start the application on the specified port. Open your web browser and go to <http://localhost:8081/> to use AutoGen Studio.\n\nAutoGen Studio also takes several parameters to customize the application:\n\n- `--host <host>` argument to specify the host address. By default, it is set to `localhost`.\n- `--appdir <appdir>` argument to specify the directory where the app files (e.g., database and generated user files) are stored. By default, it is set to the `.autogenstudio` directory in the user's home directory.\n- `--port <port>` argument to specify the port number. By default, it is set to `8080`.\n- `--reload` argument to enable auto-reloading of the server when changes are made to the code. By default, it is set to `False`.\n- `--database-uri` argument to specify the database URI. Example values include `sqlite:///database.sqlite` for SQLite and `postgresql+psycopg://user:password@localhost/dbname` for PostgreSQL. If this is not specified, the database URL defaults to a `database.sqlite` file in the `--appdir` directory.\n- `--upgrade-database` argument to upgrade the database schema to the latest version. By default, it is set to `False`.\n\nNow that you have AutoGen Studio installed and running, you are ready to explore its capabilities, including defining and modifying agent workflows, interacting with agents and sessions, and expanding agent skills.\n\n## AutoGen Studio Lite\n\nAutoGen Studio Lite provides a lightweight way to quickly prototype and experiment with AI agent teams. It's designed for rapid experimentation without the full database setup.\n\n### CLI Usage\n\nLaunch Studio Lite from the command line:\n\n```bash\n# Quick start with default team\nautogenstudio lite\n\n# Use custom team file\nautogenstudio lite --team ./my_team.json --port 8080\n\n# Custom session name\nautogenstudio lite --session-name \"My Experiment\" --auto-open\n```\n\n### Programmatic Usage\n\nUse Studio Lite directly in your Python code:\n\n```python\nfrom autogenstudio.lite import LiteStudio\n\n# Quick start with default team\nstudio = LiteStudio()\n# Use with AutoGen team objects\nfrom autogen_agentchat.teams import RoundRobinGroupChat\nteam = RoundRobinGroupChat([agent1, agent2], termination_condition=...)\n\n# Context manager usage\nwith LiteStudio(team=team) as studio:\n    # Studio runs in background\n    # Do other work here\n    pass\n```\n\n#### Local frontend development server\n\nSee `./frontend/README.md`\n\n## Contribution Guide\n\nWe welcome contributions to AutoGen Studio. We recommend the following general steps to contribute to the project:\n\n- Review the overall AutoGen project [contribution guide](https://github.com/microsoft/autogen?tab=readme-ov-file#contributing)\n- Please review the AutoGen Studio [roadmap](https://github.com/microsoft/autogen/issues/4006) to get a sense of the current priorities for the project. Help is appreciated especially with Studio issues tagged with `help-wanted`\n- Please initiate a discussion on the roadmap issue or a new issue to discuss your proposed contribution.\n- Submit a pull request with your contribution!\n- If you are modifying AutoGen Studio, it has its own devcontainer. See instructions in `.devcontainer/README.md` to use it\n- Please use the tag `proj-studio` for any issues, questions, and PRs related to Studio\n\n## FAQ\n\nPlease refer to the AutoGen Studio [FAQs](https://microsoft.github.io/autogen/docs/autogen-studio/faqs) page for more information.\n\n## Acknowledgements\n\nAutoGen Studio is Based on the [AutoGen](https://microsoft.github.io/autogen) project. It was adapted from a research prototype built in October 2023 (original credits: Gagan Bansal, Adam Fourney, Victor Dibia, Piali Choudhury, Saleema Amershi, Ahmed Awadallah, Chi Wang).\n","python/packages/agbench/README.md":"# AutoGenBench\n\nAutoGenBench (agbench) is a tool for repeatedly running a set of pre-defined AutoGen tasks in a setting with tightly-controlled initial conditions. With each run, AutoGenBench will start from a blank slate. The agents being evaluated will need to work out what code needs to be written, and what libraries or dependencies to install, to solve tasks. The results of each run are logged, and can be ingested by analysis or metrics scripts (such as `agbench tabulate`). By default, all runs are conducted in freshly-initialized docker containers, providing the recommended level of consistency and safety.\n\nAutoGenBench works with all AutoGen 0.1.*, and 0.2.* versions.\n\n## Technical Specifications\n\nIf you are already an AutoGenBench pro, and want the full technical specifications, please review the [contributor&#39;s guide](CONTRIBUTING.md).\n\n## Docker Requirement\n\nAutoGenBench also requires Docker (Desktop or Engine). **It will not run in GitHub codespaces**, unless you opt for native execution (which is strongly discouraged). To install Docker Desktop see [https://www.docker.com/products/docker-desktop/](https://www.docker.com/products/docker-desktop/).\n\nIf you are working in WSL, you can follow the instructions below to set up your environment:\n\n1. Install Docker Desktop. After installation, restart is needed, then open Docker Desktop, in Settings, Ressources, WSL Integration, Enable integration with additional distros – Ubuntu\n2. Clone autogen and export `AUTOGEN_REPO_BASE`. This environment variable enables the Docker containers to use the correct version agents.\n    ```bash\n    git clone git@github.com:microsoft/autogen.git\n    export AUTOGEN_REPO_BASE=<path_to_autogen>\n    ```\n\n## Installation and Setup\n\n[Deprecated currently] **To get the most out of AutoGenBench, the `agbench` package should be installed**. At present, the easiest way to do this is to install it via `pip`.\n\n\nIf you would prefer working from source code (e.g., for development, or to utilize an alternate branch), simply clone the [AutoGen](https://github.com/microsoft/autogen) repository, then install `agbench` via:\n\n```\npip install -e autogen/python/packages/agbench\n```\n\nAfter installation, you must configure your API keys. As with other AutoGen applications, AutoGenBench will look for the OpenAI keys in the OAI_CONFIG_LIST file in the current working directory, or the OAI_CONFIG_LIST environment variable. This behavior can be overridden using a command-line parameter described later.\n\nIf you will be running multiple benchmarks, it is often most convenient to leverage the environment variable option. You can load your keys into the environment variable by executing:\n\n```\nexport OAI_CONFIG_LIST=$(cat ./OAI_CONFIG_LIST)\n```\n\nIf an OAI_CONFIG_LIST is *not* provided (by means of file or environment variable), AutoGenBench will use the OPENAI_API_KEY environment variable instead.\n\nFor some benchmark scenarios, additional keys may be required (e.g., keys for the Bing Search API). These can be added to an `ENV.json` file in the current working folder. An example `ENV.json` file is provided below:\n\n```\n{\n    \"BING_API_KEY\": \"xxxyyyzzz\"\n}\n```\n\n## A Typical Session\n\nOnce AutoGenBench and necessary keys are installed, a typical session will look as follows:\n\n\n\nNavigate to HumanEval\n\n```bash\ncd autogen/python/packages/agbench/benchmarks/HumanEval\n```\n**Note:** The following instructions are specific to the HumanEval benchmark. For other benchmarks, please refer to the README in the respective benchmark folder, e.g.,: [AssistantBench](benchmarks/AssistantBench/README.md).\n\n\nCreate a file called ENV.json with the following (required) contents (If you're using MagenticOne), if using Azure:\n\n```json\n{\n    \"CHAT_COMPLETION_KWARGS_JSON\": \"{}\",\n    \"CHAT_COMPLETION_PROVIDER\": \"azure\"\n}\n```\n\nYou can also use the openai client by replacing the last two entries in the ENV file by:\n\n- `CHAT_COMPLETION_PROVIDER='openai'`\n- `CHAT_COMPLETION_KWARGS_JSON` with the following JSON structure:\n\n```json\n{\n  \"api_key\": \"REPLACE_WITH_YOUR_API\",\n  \"model\": \"REPLACE_WITH_YOUR_MODEL\"\n}\n```\n\nNow initialize the tasks.\n\n```bash\npython Scripts/init_tasks.py\n```\n\nNote: This will attempt to download HumanEval\n\n\nOnce the script completes, you should now see a folder in your current directory called `Tasks` that contains one JSONL file per template in `Templates`.\n\nNow to run a specific subset of HumanEval use:\n\n```bash\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nYou should see the command line print the raw logs that shows the agents in action To see a summary of the results (e.g., task completion rates), in a new terminal run the following:\n\n```bash\nagbench tabulate Results/human_eval_MagenticOne\n```\n\nWhere:\n\n- `agbench run Tasks/human_eval_MagenticOne.jsonl` runs the tasks defined in `Tasks/human_eval_MagenticOne.jsonl`\n- `agbench tablue results/human_eval_MagenticOne` tabulates the results of the run\n\nEach of these commands has extensive in-line help via:\n\n- `agbench --help`\n- `agbench run --help`\n- `agbench tabulate --help`\n- `agbench remove_missing --help`\n\n**NOTE:** If you are running `agbench` from within the repository, you need to navigate to the appropriate scenario folder (e.g., `scenarios/HumanEval`) and run the `Scripts/init_tasks.py` file.\n\nMore details of each command are provided in the sections that follow.\n\n\n## Running AutoGenBench\n\nTo run a benchmark (which executes the tasks, but does not compute metrics), simply execute:\n\n```\ncd [BENCHMARK]\nagbench run Tasks/*.jsonl\n```\n\nFor example,\n\n```\ncd HumanEval\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe default is to run each task once. To run each scenario 10 times, use:\n\n```\nagbench run --repeat 10 Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe `agbench` command-line tool allows a number of command-line arguments to control various parameters of execution. Type ``agbench -h`` to explore these options:\n\n```\n'agbench run' will run the specified autogen scenarios for a given number of repetitions and record all logs and trace information. When running in a Docker environment (default), each run will begin from a common, tightly controlled, environment. The resultant logs can then be further processed by other scripts to produce metrics.\n\npositional arguments:\n  scenario      The JSONL scenario file to run. If a directory is specified,\n                then all JSONL scenarios in the directory are run. (default:\n                ./scenarios)\n\noptions:\n  -h, --help            show this help message and exit\n  -c CONFIG, --config CONFIG\n                        The environment variable name or path to the OAI_CONFIG_LIST (default: OAI_CONFIG_LIST).\n  -r REPEAT, --repeat REPEAT\n                        The number of repetitions to run for each scenario (default: 1).\n  -s SUBSAMPLE, --subsample SUBSAMPLE\n                        Run on a subsample of the tasks in the JSONL file(s). If a decimal value is specified, then run on\n                        the given proportion of tasks in each file. For example \"0.7\" would run on 70% of tasks, and \"1.0\"\n                        would run on 100% of tasks. If an integer value is specified, then randomly select *that* number of\n                        tasks from each specified JSONL file. For example \"7\" would run tasks, while \"1\" would run only 1\n                        task from each specified JSONL file. (default: 1.0; which is 100%)\n  -m MODEL, --model MODEL\n                        Filters the config_list to include only models matching the provided model name (default: None, which\n                        is all models).\n  --requirements REQUIREMENTS\n                        The requirements file to pip install before running the scenario.\n  -d DOCKER_IMAGE, --docker-image DOCKER_IMAGE\n                        The Docker image to use when running scenarios. Can not be used together with --native. (default:\n                        'agbench:default', which will be created if not present)\n  --native              Run the scenarios natively rather than in docker. NOTE: This is not advisable, and should be done\n                        with great caution.\n```\n\n## Results\n\nBy default, the AutoGenBench stores results in a folder hierarchy with the following template:\n\n``./results/[scenario]/[task_id]/[instance_id]``\n\nFor example, consider the following folders:\n\n``./results/default_two_agents/two_agent_stocks/0``\n``./results/default_two_agents/two_agent_stocks/1``\n\n...\n\n``./results/default_two_agents/two_agent_stocks/9``\n\nThis folder holds the results for the ``two_agent_stocks`` task of the ``default_two_agents`` tasks file. The ``0`` folder contains the results of the first instance / run. The ``1`` folder contains the results of the second run, and so on. You can think of the _task_id_ as mapping to a prompt, or a unique set of parameters, while the _instance_id_ defines a specific attempt or run.\n\nWithin each folder, you will find the following files:\n\n- *timestamp.txt*: records the date and time of the run, along with the version of the autogen-agentchat library installed\n- *console_log.txt*: all console output produced by Docker when running AutoGen. Read this like you would a regular console.\n- *[agent]_messages.json*: for each Agent, a log of their messages dictionaries\n- *./coding*: A directory containing all code written by AutoGen, and all artifacts produced by that code.\n\n## Contributing or Defining New Tasks or Benchmarks\n\nIf you would like to develop -- or even contribute -- your own tasks or benchmarks, please review the [contributor&#39;s guide](CONTRIBUTING.md) for complete technical details.\n","python/packages/autogen-agentchat/README.md":"# AutoGen AgentChat\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/index.html)\n\nAgentChat is a high-level API for building multi-agent applications.\nIt is built on top of the [`autogen-core`](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html) package.\nFor beginner users, AgentChat is the recommended starting point.\nFor advanced users, [`autogen-core`](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html)'s event-driven\nprogramming model provides more flexibility and control over the underlying components.\n\nAgentChat provides intuitive defaults, such as **Agents** with preset\nbehaviors and **Teams** with predefined [multi-agent design patterns](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/design-patterns/intro.html).\n","python/packages/agbench/CONTRIBUTING.md":"# Contributing to AutoGenBench\n\nAs part of the broader AutoGen project, AutoGenBench welcomes community contributions. Contributions are subject to AutoGen's [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute), as well as a few additional AutoGenBench-specific requirements outlined here. You may also wish to develop your own private benchmark scenarios and the guidance in this document will help with such efforts as well. Below you will find the general requirements, followed by a detailed technical description.\n\n## General Contribution Requirements\nWe ask that all contributions to AutoGenBench adhere to the following:\n\n- Follow AutoGen's broader [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute)\n- All AutoGenBench benchmarks should live in a subfolder of `/benchmarks` alongside `HumanEval`, `GAIA`, etc.\n- Benchmark scenarios should include a detailed README.md, in the root of their folder, describing the benchmark and providing citations where warranted.\n- Benchmark data (tasks, ground truth, etc.) should be downloaded from their original sources rather than hosted in the AutoGen repository (unless the benchmark is original, and the repository *is* the original source)\n    - You can use the `Scripts/init_tasks.py` file to automate this download.\n- Basic scoring should be compatible with the `agbench tabulate` command (e.g., by outputting logs compatible with the default tabulation mechanism, or by providing a `Scripts/custom_tabulate.py` file)\n\nThese requirements are further detailed below, but if you simply copy the `HumanEval` folder, you will already be off to a great start.\n\n## Implementing and Running Benchmark Tasks\nAt the core of any benchmark is a set of tasks. To implement tasks that are runnable by AutoGenBench, you must adhere to AutoGenBench's templating and scenario expansion algorithms, as outlined below.\n\n### Task Definitions\n\nAll tasks are stored in JSONL files (in subdirectories under `./Tasks`). Each line of a tasks file is a JSON object with the following schema:\n\n```\n{\n   \"id\": string,\n   \"template\": dirname,\n   \"substitutions\" {\n       \"filename1\": {\n       \t   \"find_string1_1\": replace_string1_1,\n           \"find_string1_2\": replace_string1_2,\n           ...\n           \"find_string1_M\": replace_string1_N\n       }\n       \"filename2\": {\n       \t   \"find_string2_1\": replace_string2_1,\n           \"find_string2_2\": replace_string2_2,\n           ...\n           \"find_string2_N\": replace_string2_N\n       }\n   }\n}\n```\n\nFor example:\n\n```\n{\n    \"id\": \"two_agent_stocks_gpt4\",\n    \"template\": \"default_two_agents\",\n    \"substitutions\": {\n\t\"scenario.py\": {\n            \"__MODEL__\": \"gpt-4\",\n\t},\n\t\"prompt.txt\": {\n            \"__PROMPT__\": \"Plot and save to disk a chart of NVDA and TESLA stock price YTD.\"\n        }\n    }\n}\n```\n\nIn this example, the string `__MODEL__` will be replaced in the file `scenarios.py`, while the string `__PROMPT__` will be replaced in the `prompt.txt` file.\n\nThe `template` field can also take on a list value, but this usage is considered advanced and is not described here. See the `agbench/run_cmd.py` code, or the `GAIA` benchmark tasks files for additional information about this option.\n\n\n## Task Instance Expansion Algorithm\n\nOnce the tasks have been defined, as per above, they must be \"instantiated\" before they can be run. This instantiation happens automatically when the user issues the `agbench run` command and involves creating a local folder to share with Docker. Each instance and repetition gets its own folder along the path: `./results/[scenario]/[task_id]/[instance_id]`. For the sake of brevity we will refer to this folder as the `DEST_FOLDER`.\n\nThe algorithm for populating the `DEST_FOLDER` is as follows:\n\n1. Pre-populate DEST_FOLDER with all the basic starter files for running a scenario (found in `agbench/template`).\n2. Recursively copy the template folder specified in the JSONL line to DEST_FOLDER (if the JSON `template` attribute points to a folder) If the JSONs `template` attribute instead points to a file, copy the file, but rename it to `scenario.py`\n3. Apply any string replacements, as outlined in the prior section.\n4. Write a run.sh file to DEST_FOLDER that will be executed by Docker when it is loaded. The `run.sh` is described below.\n\n## Scenario Execution Algorithm\n\nOnce the task has been instantiated it is run (via run.sh). This script will execute the following steps:\n\n1. If a file named `global_init.sh` is present, run it.\n2. If a file named `scenario_init.sh` is present, run it.\n3. Install the requirements.txt file (if running in Docker)\n4. Run the task via `python scenario.py`\n5. If the scenario.py exited cleanly (exit code 0), then print \"SCENARIO.PY COMPLETE !#!#\"\n6. Clean up (delete cache, etc.)\n7. If a file named `scenario_finalize.sh` is present, run it.\n8. If a file named `global_finalize.sh` is present, run it.\n9. echo \"RUN COMPLETE !#!#\", signaling that all steps completed.\n\nNotably, this means that scenarios can add custom init and teardown logic by including `scenario_init.sh` and `scenario_finalize.sh` files.\n\nAt the time of this writing, the run.sh file is as follows:\n\n```sh\nexport AUTOGEN_TESTBED_SETTING=\"Docker\"\numask 000\n\n# Run the global init script if it exists\nif [ -f global_init.sh ] ; then\n    . ./global_init.sh\nfi\n\n# Run the scenario init script if it exists\nif [ -f scenario_init.sh ] ; then\n    . ./scenario_init.sh\nfi\n\n# Run the scenario\npip install -r requirements.txt\npython scenario.py\nEXIT_CODE=$?\nif [ $EXIT_CODE -ne 0 ]; then\n    echo SCENARIO.PY EXITED WITH CODE: $EXIT_CODE !#!#\nelse\n    echo SCENARIO.PY COMPLETE !#!#\nfi\n\n# Clean up\nif [ -d .cache ] ; then\n    rm -Rf .cache\nfi\n\n# Run the scenario finalize script if it exists\nif [ -f scenario_finalize.sh ] ; then\n    . ./scenario_finalize.sh\nfi\n\n# Run the global finalize script if it exists\nif [ -f global_finalize.sh ] ; then\n    . ./global_finalize.sh\nfi\n\necho RUN.SH COMPLETE !#!#\n```\n\nBe warned that this listing is provided here for illustration purposes, and may vary over time. The source of truth are the `run.sh` files found in the ``./results/[taskset]/[task_id]/[instance_id]`` folders.\n\n\n## Integrating with the `tabulate` \nThe above details are sufficient for defining and running tasks, but if you wish to support the `agbench tabulate`  commands, a few additional steps are required.\n\n### Tabulations\n\nIf you wish to leverage the default tabulation logic, it is as simple as arranging your `scenario.py` file to output the string \"ALL TESTS PASSED !#!#\" to the console in the event that a task was solved correctly.\n\nIf you wish to implement your own tabulation logic, simply create the file `Scripts/custom_tabulate.py` and include a `main(args)` method. Here, the `args` parameter will be provided by AutoGenBench, and is a drop-in replacement for `sys.argv`. In particular, `args[0]` will be the invocation command (similar to the executable or script name in `sys.argv`), and the remaining values (`args[1:]`) are the command line parameters.\n\nShould you provide a custom tabulation script, please implement `--help` and `-h` options for documenting your interface.\n\nThe `scenarios/GAIA/Scripts/custom_tabulate.py` is a great example of custom tabulation. It also shows how you can reuse some components of the default tabulator to speed up development.\n\n\n\n## Scripts/init_tasks.py\nFinally, you should provide an `Scripts/init_tasks.py` file, in your benchmark folder, and include a `main()` method therein. \n\nThis `init_tasks.py` script is a great place to download benchmarks from their original sources and convert them to the JSONL format required by AutoGenBench:\n- See `HumanEval/Scripts/init_tasks.py` for an example of how to expand a benchmark from an original GitHub repository.\n- See `GAIA/Scripts/init_tasks.py` for an example of how to expand a benchmark from `Hugging Face Hub`.\n","python/packages/autogen-studio/.devcontainer/devcontainer.json":"# .devcontainer Skill\n\nPath: `python/packages/autogen-studio/.devcontainer/devcontainer.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/.devcontainer/devcontainer.json)","python/packages/autogen-studio/frontend/package.json":"# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/package.json)","python/packages/autogen-studio/frontend/README.md":"# frontend Documentation\n\nPath: `python/packages/autogen-studio/frontend/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/README.md)","python/packages/autogen-studio/frontend/tsconfig.json":"# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/tsconfig.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/tsconfig.json)","python/packages/autogen-studio/notebooks/team.json":"# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/team.json)","python/packages/autogen-studio/notebooks/travel_team.json":"# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/travel_team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/travel_team.json)","python/packages/agbench/benchmarks/README.md":"# benchmarks Documentation\n\nPath: `python/packages/agbench/benchmarks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/README.md)","python/packages/agbench/benchmarks/GAIA/config.yaml":"# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/config.yaml)","python/packages/agbench/benchmarks/GAIA/ENV.yaml":"# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/ENV.yaml)","python/packages/agbench/benchmarks/GAIA/README.md":"# GAIA Documentation\n\nPath: `python/packages/agbench/benchmarks/GAIA/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/README.md)","python/packages/agbench/benchmarks/HumanEval/config.yaml":"# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/config.yaml)","python/packages/agbench/benchmarks/HumanEval/ENV.yaml":"# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/ENV.yaml)","python/packages/agbench/benchmarks/HumanEval/README.md":"# HumanEval Documentation\n\nPath: `python/packages/agbench/benchmarks/HumanEval/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/README.md)","python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml)","python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml)","python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml)","python/packages/autogen-studio/frontend/src/components/shared/README.md":"# shared Documentation\n\nPath: `python/packages/autogen-studio/frontend/src/components/shared/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/shared/README.md)","python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md":"# task_centric_memory Documentation\n\nPath: `python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md)","python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json":"# Teachability Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json)","python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json":"# Self_teaching Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json)","python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json":"# Demonstration Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml)","python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml)","python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json":"# Gallery Skill\n\nPath: `python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json)","python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md":"# _host Documentation\n\nPath: `python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md)"},"items":[{"name":"config.yaml","path":"python/packages/agbench/benchmarks/GAIA/config.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/config.yaml","title":"GAIA Skill","category":"anthropic-skill","format":"yaml","content":"# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/config.yaml)","tokens":300,"sizeBytes":213},{"name":"ENV.yaml","path":"python/packages/agbench/benchmarks/GAIA/ENV.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/ENV.yaml","title":"GAIA Skill","category":"anthropic-skill","format":"yaml","content":"# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/ENV.yaml)","tokens":300,"sizeBytes":207},{"name":"config.yaml","path":"python/packages/agbench/benchmarks/HumanEval/config.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/config.yaml","title":"HumanEval Skill","category":"anthropic-skill","format":"yaml","content":"# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/config.yaml)","tokens":300,"sizeBytes":228},{"name":"ENV.yaml","path":"python/packages/agbench/benchmarks/HumanEval/ENV.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/ENV.yaml","title":"HumanEval Skill","category":"anthropic-skill","format":"yaml","content":"# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/ENV.yaml)","tokens":300,"sizeBytes":222},{"name":"CONTRIBUTING.md","path":"python/packages/agbench/CONTRIBUTING.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/CONTRIBUTING.md","title":"Agbench Skill","category":"anthropic-skill","format":"markdown","content":"# Contributing to AutoGenBench\n\nAs part of the broader AutoGen project, AutoGenBench welcomes community contributions. Contributions are subject to AutoGen's [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute), as well as a few additional AutoGenBench-specific requirements outlined here. You may also wish to develop your own private benchmark scenarios and the guidance in this document will help with such efforts as well. Below you will find the general requirements, followed by a detailed technical description.\n\n## General Contribution Requirements\nWe ask that all contributions to AutoGenBench adhere to the following:\n\n- Follow AutoGen's broader [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute)\n- All AutoGenBench benchmarks should live in a subfolder of `/benchmarks` alongside `HumanEval`, `GAIA`, etc.\n- Benchmark scenarios should include a detailed README.md, in the root of their folder, describing the benchmark and providing citations where warranted.\n- Benchmark data (tasks, ground truth, etc.) should be downloaded from their original sources rather than hosted in the AutoGen repository (unless the benchmark is original, and the repository *is* the original source)\n    - You can use the `Scripts/init_tasks.py` file to automate this download.\n- Basic scoring should be compatible with the `agbench tabulate` command (e.g., by outputting logs compatible with the default tabulation mechanism, or by providing a `Scripts/custom_tabulate.py` file)\n\nThese requirements are further detailed below, but if you simply copy the `HumanEval` folder, you will already be off to a great start.\n\n## Implementing and Running Benchmark Tasks\nAt the core of any benchmark is a set of tasks. To implement tasks that are runnable by AutoGenBench, you must adhere to AutoGenBench's templating and scenario expansion algorithms, as outlined below.\n\n### Task Definitions\n\nAll tasks are stored in JSONL files (in subdirectories under `./Tasks`). Each line of a tasks file is a JSON object with the following schema:\n\n```\n{\n   \"id\": string,\n   \"template\": dirname,\n   \"substitutions\" {\n       \"filename1\": {\n       \t   \"find_string1_1\": replace_string1_1,\n           \"find_string1_2\": replace_string1_2,\n           ...\n           \"find_string1_M\": replace_string1_N\n       }\n       \"filename2\": {\n       \t   \"find_string2_1\": replace_string2_1,\n           \"find_string2_2\": replace_string2_2,\n           ...\n           \"find_string2_N\": replace_string2_N\n       }\n   }\n}\n```\n\nFor example:\n\n```\n{\n    \"id\": \"two_agent_stocks_gpt4\",\n    \"template\": \"default_two_agents\",\n    \"substitutions\": {\n\t\"scenario.py\": {\n            \"__MODEL__\": \"gpt-4\",\n\t},\n\t\"prompt.txt\": {\n            \"__PROMPT__\": \"Plot and save to disk a chart of NVDA and TESLA stock price YTD.\"\n        }\n    }\n}\n```\n\nIn this example, the string `__MODEL__` will be replaced in the file `scenarios.py`, while the string `__PROMPT__` will be replaced in the `prompt.txt` file.\n\nThe `template` field can also take on a list value, but this usage is considered advanced and is not described here. See the `agbench/run_cmd.py` code, or the `GAIA` benchmark tasks files for additional information about this option.\n\n\n## Task Instance Expansion Algorithm\n\nOnce the tasks have been defined, as per above, they must be \"instantiated\" before they can be run. This instantiation happens automatically when the user issues the `agbench run` command and involves creating a local folder to share with Docker. Each instance and repetition gets its own folder along the path: `./results/[scenario]/[task_id]/[instance_id]`. For the sake of brevity we will refer to this folder as the `DEST_FOLDER`.\n\nThe algorithm for populating the `DEST_FOLDER` is as follows:\n\n1. Pre-populate DEST_FOLDER with all the basic starter files for running a scenario (found in `agbench/template`).\n2. Recursively copy the template folder specified in the JSONL line to DEST_FOLDER (if the JSON `template` attribute points to a folder) If the JSONs `template` attribute instead points to a file, copy the file, but rename it to `scenario.py`\n3. Apply any string replacements, as outlined in the prior section.\n4. Write a run.sh file to DEST_FOLDER that will be executed by Docker when it is loaded. The `run.sh` is described below.\n\n## Scenario Execution Algorithm\n\nOnce the task has been instantiated it is run (via run.sh). This script will execute the following steps:\n\n1. If a file named `global_init.sh` is present, run it.\n2. If a file named `scenario_init.sh` is present, run it.\n3. Install the requirements.txt file (if running in Docker)\n4. Run the task via `python scenario.py`\n5. If the scenario.py exited cleanly (exit code 0), then print \"SCENARIO.PY COMPLETE !#!#\"\n6. Clean up (delete cache, etc.)\n7. If a file named `scenario_finalize.sh` is present, run it.\n8. If a file named `global_finalize.sh` is present, run it.\n9. echo \"RUN COMPLETE !#!#\", signaling that all steps completed.\n\nNotably, this means that scenarios can add custom init and teardown logic by including `scenario_init.sh` and `scenario_finalize.sh` files.\n\nAt the time of this writing, the run.sh file is as follows:\n\n```sh\nexport AUTOGEN_TESTBED_SETTING=\"Docker\"\numask 000\n\n# Run the global init script if it exists\nif [ -f global_init.sh ] ; then\n    . ./global_init.sh\nfi\n\n# Run the scenario init script if it exists\nif [ -f scenario_init.sh ] ; then\n    . ./scenario_init.sh\nfi\n\n# Run the scenario\npip install -r requirements.txt\npython scenario.py\nEXIT_CODE=$?\nif [ $EXIT_CODE -ne 0 ]; then\n    echo SCENARIO.PY EXITED WITH CODE: $EXIT_CODE !#!#\nelse\n    echo SCENARIO.PY COMPLETE !#!#\nfi\n\n# Clean up\nif [ -d .cache ] ; then\n    rm -Rf .cache\nfi\n\n# Run the scenario finalize script if it exists\nif [ -f scenario_finalize.sh ] ; then\n    . ./scenario_finalize.sh\nfi\n\n# Run the global finalize script if it exists\nif [ -f global_finalize.sh ] ; then\n    . ./global_finalize.sh\nfi\n\necho RUN.SH COMPLETE !#!#\n```\n\nBe warned that this listing is provided here for illustration purposes, and may vary over time. The source of truth are the `run.sh` files found in the ``./results/[taskset]/[task_id]/[instance_id]`` folders.\n\n\n## Integrating with the `tabulate` \nThe above details are sufficient for defining and running tasks, but if you wish to support the `agbench tabulate`  commands, a few additional steps are required.\n\n### Tabulations\n\nIf you wish to leverage the default tabulation logic, it is as simple as arranging your `scenario.py` file to output the string \"ALL TESTS PASSED !#!#\" to the console in the event that a task was solved correctly.\n\nIf you wish to implement your own tabulation logic, simply create the file `Scripts/custom_tabulate.py` and include a `main(args)` method. Here, the `args` parameter will be provided by AutoGenBench, and is a drop-in replacement for `sys.argv`. In particular, `args[0]` will be the invocation command (similar to the executable or script name in `sys.argv`), and the remaining values (`args[1:]`) are the command line parameters.\n\nShould you provide a custom tabulation script, please implement `--help` and `-h` options for documenting your interface.\n\nThe `scenarios/GAIA/Scripts/custom_tabulate.py` is a great example of custom tabulation. It also shows how you can reuse some components of the default tabulator to speed up development.\n\n\n\n## Scripts/init_tasks.py\nFinally, you should provide an `Scripts/init_tasks.py` file, in your benchmark folder, and include a `main()` method therein. \n\nThis `init_tasks.py` script is a great place to download benchmarks from their original sources and convert them to the JSONL format required by AutoGenBench:\n- See `HumanEval/Scripts/init_tasks.py` for an example of how to expand a benchmark from an original GitHub repository.\n- See `GAIA/Scripts/init_tasks.py` for an example of how to expand a benchmark from `Hugging Face Hub`.\n","isInternal":false,"tokens":1872,"sizeBytes":7907},{"name":"demonstration.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml","title":"Configs Skill","category":"anthropic-skill","format":"yaml","content":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml)","tokens":300,"sizeBytes":274},{"name":"self_teaching.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml","title":"Configs Skill","category":"anthropic-skill","format":"yaml","content":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml)","tokens":300,"sizeBytes":274},{"name":"teachability.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml","title":"Configs Skill","category":"anthropic-skill","format":"yaml","content":"# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml)","tokens":300,"sizeBytes":272},{"name":"add_topic.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml","title":"Insights Skill","category":"anthropic-skill","format":"yaml","content":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml)","tokens":300,"sizeBytes":291},{"name":"cell_towers_2_demo.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml","title":"Insights Skill","category":"anthropic-skill","format":"yaml","content":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml)","tokens":300,"sizeBytes":309},{"name":"liar_advice.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml","title":"Insights Skill","category":"anthropic-skill","format":"yaml","content":"# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml)","tokens":300,"sizeBytes":295},{"name":"10_liars.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml","title":"Tasks Skill","category":"anthropic-skill","format":"yaml","content":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml)","tokens":300,"sizeBytes":280},{"name":"100_vampires.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml","title":"Tasks Skill","category":"anthropic-skill","format":"yaml","content":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml)","tokens":300,"sizeBytes":288},{"name":"autogen_package.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml","title":"Tasks Skill","category":"anthropic-skill","format":"yaml","content":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml)","tokens":300,"sizeBytes":294},{"name":"cell_towers_1.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml","title":"Tasks Skill","category":"anthropic-skill","format":"yaml","content":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml)","tokens":300,"sizeBytes":290},{"name":"cell_towers_2.yaml","path":"python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml","title":"Tasks Skill","category":"anthropic-skill","format":"yaml","content":"# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml)","tokens":300,"sizeBytes":290},{"name":"session.json","path":"python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json","title":"Demonstration Skill","category":"anthropic-skill","format":"json","content":"# Demonstration Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json)","tokens":300,"sizeBytes":298},{"name":"session.json","path":"python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json","title":"Self_teaching Skill","category":"anthropic-skill","format":"json","content":"# Self_teaching Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json)","tokens":300,"sizeBytes":298},{"name":"session.json","path":"python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json","title":"Teachability Skill","category":"anthropic-skill","format":"json","content":"# Teachability Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json)","tokens":300,"sizeBytes":295},{"name":"devcontainer.json","path":"python/packages/autogen-studio/.devcontainer/devcontainer.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/.devcontainer/devcontainer.json","title":".devcontainer Skill","category":"anthropic-skill","format":"json","content":"# .devcontainer Skill\n\nPath: `python/packages/autogen-studio/.devcontainer/devcontainer.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/.devcontainer/devcontainer.json)","tokens":300,"sizeBytes":244},{"name":"package.json","path":"python/packages/autogen-studio/frontend/package.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/package.json","title":"Frontend Skill","category":"anthropic-skill","format":"json","content":"# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/package.json)","tokens":300,"sizeBytes":219},{"name":"default_gallery.json","path":"python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json","title":"Gallery Skill","category":"anthropic-skill","format":"json","content":"# Gallery Skill\n\nPath: `python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json)","tokens":300,"sizeBytes":292},{"name":"tsconfig.json","path":"python/packages/autogen-studio/frontend/tsconfig.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/tsconfig.json","title":"Frontend Skill","category":"anthropic-skill","format":"json","content":"# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/tsconfig.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/tsconfig.json)","tokens":300,"sizeBytes":221},{"name":"team.json","path":"python/packages/autogen-studio/notebooks/team.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/team.json","title":"Notebooks Skill","category":"anthropic-skill","format":"json","content":"# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/team.json)","tokens":300,"sizeBytes":216},{"name":"travel_team.json","path":"python/packages/autogen-studio/notebooks/travel_team.json","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/travel_team.json","title":"Notebooks Skill","category":"anthropic-skill","format":"json","content":"# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/travel_team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/travel_team.json)","tokens":300,"sizeBytes":230},{"name":"copilot-instructions.md","path":".github/copilot-instructions.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/.github/copilot-instructions.md","title":"GitHub Copilot Instructions","category":"copilot-instructions","format":"markdown","content":"# AutoGen Multi-Agent AI Framework\n\nAutoGen is a multi-language framework for creating AI agents that can act autonomously or work alongside humans. The project has separate Python and .NET implementations with their own development workflows.\n\nAlways reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.\n\n## Working Effectively\n\n### Prerequisites and Environment Setup\n\n**CRITICAL**: Install both .NET 8.0 and 9.0 for full compatibility:\n- Install uv package manager: `python3 -m pip install uv` \n- Install .NET 9.0 SDK: `wget https://dot.net/v1/dotnet-install.sh && chmod +x dotnet-install.sh && ./dotnet-install.sh --channel 9.0`\n- Install .NET 8.0 runtime: `./dotnet-install.sh --channel 8.0 --runtime dotnet && ./dotnet-install.sh --channel 8.0 --runtime aspnetcore`\n- Update PATH: `export PATH=\"$HOME/.dotnet:$PATH\"`\n\n### Python Development Workflow\n\n**Bootstrap and build Python environment:**\n```bash\ncd /home/runner/work/autogen/autogen/python\nuv sync --all-extras  # NEVER CANCEL: Takes 2 minutes. Set timeout to 300+ seconds.\nsource .venv/bin/activate\n```\n\n**Validate Python development:**\n```bash\n# Quick validation (under 1 second each)\npoe format  # Code formatting\npoe lint    # Linting with ruff\n\n# Type checking - NEVER CANCEL these commands\npoe mypy     # Takes 6 minutes. Set timeout to 600+ seconds.\npoe pyright  # Takes 41 seconds. Set timeout to 120+ seconds.\n\n# Individual package testing (core package example)\npoe --directory ./packages/autogen-core test  # Takes 10 seconds. Set timeout to 60+ seconds.\n\n# Documentation - NEVER CANCEL\npoe docs-build  # Takes 1 minute 16 seconds. Set timeout to 300+ seconds.\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: Python environment setup takes 2 minutes minimum\n- **NEVER CANCEL**: mypy type checking takes 6 minutes \n- **NEVER CANCEL**: Documentation build takes 1+ minutes\n- Format/lint tasks complete in under 1 second\n- Individual package tests typically complete in 10-60 seconds\n\n### .NET Development Workflow\n\n**Bootstrap and build .NET environment:**\n```bash\ncd /home/runner/work/autogen/autogen/dotnet\nexport PATH=\"$HOME/.dotnet:$PATH\"\ndotnet restore  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\ndotnet build --configuration Release  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\n```\n\n**Validate .NET development:**\n```bash\n# Unit tests - NEVER CANCEL\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Takes 25 seconds. Set timeout to 120+ seconds.\n\n# Format check (if build fails) \ndotnet format --verify-no-changes\n\n# Run samples\ncd samples/Hello\ndotnet run\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: .NET restore takes 53 seconds minimum\n- **NEVER CANCEL**: .NET build takes 53 seconds minimum  \n- **NEVER CANCEL**: .NET unit tests take 25 seconds minimum\n- All build and test commands require appropriate timeouts\n\n### Complete Validation Workflow\n\n**Run full check suite (Python):**\n```bash\ncd /home/runner/work/autogen/autogen/python\nsource .venv/bin/activate\npoe check  # NEVER CANCEL: Runs all checks. Takes 7+ minutes total. Set timeout to 900+ seconds.\n```\n\n## Validation Scenarios\n\n### Manual Validation Requirements\nAlways manually validate changes by running complete user scenarios after making modifications:\n\n**Python validation scenarios:**\n1. **Import test**: Verify core imports work:\n   ```python\n   from autogen_agentchat.agents import AssistantAgent\n   from autogen_core import AgentRuntime\n   from autogen_ext.models.openai import OpenAIChatCompletionClient\n   ```\n\n2. **AutoGen Studio test**: Verify web interface can start:\n   ```bash\n   autogenstudio ui --help  # Should show help without errors\n   ```\n\n3. **Documentation test**: Build and verify docs generate without errors:\n   ```bash\n   poe docs-build && ls docs/build/index.html\n   ```\n\n**.NET validation scenarios:**\n1. **Sample execution**: Run Hello sample to verify runtime works:\n   ```bash\n   cd dotnet/samples/Hello && dotnet run --help\n   ```\n\n2. **Build validation**: Ensure all projects compile:\n   ```bash\n   dotnet build --configuration Release --no-restore\n   ```\n\n3. **Test execution**: Run unit tests to verify functionality:\n   ```bash\n   dotnet test --filter \"Category=UnitV2\" --configuration Release --no-build\n   ```\n\n## Common Issues and Workarounds\n\n### Network-Related Issues\n- **Python tests may fail** with network errors (tiktoken downloads, Playwright browser downloads) in sandboxed environments - this is expected\n- **Documentation intersphinx warnings** due to inability to reach external documentation sites - this is expected\n- **Individual package tests work better** than full test suite in network-restricted environments\n\n### .NET Runtime Issues  \n- **Requires both .NET 8.0 and 9.0**: Build uses 9.0 SDK but tests need 8.0 runtime\n- **Global.json specifies 9.0.100**: Must install exact .NET 9.0 version or later\n- **Path configuration critical**: Ensure `$HOME/.dotnet` is in PATH before system .NET\n\n### Python Package Issues\n- **Use uv exclusively**: Do not use pip/conda for dependency management\n- **Virtual environment required**: Always activate `.venv` before running commands\n- **Package workspace structure**: Project uses uv workspace with multiple packages\n\n## Timing Reference\n\n### Python Commands\n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `uv sync --all-extras` | 2 minutes | 300+ seconds | NEVER CANCEL |\n| `poe mypy` | 6 minutes | 600+ seconds | NEVER CANCEL |\n| `poe pyright` | 41 seconds | 120+ seconds | NEVER CANCEL |\n| `poe docs-build` | 1 min 16 sec | 300+ seconds | NEVER CANCEL |\n| `poe format` | <1 second | 30 seconds | Quick |\n| `poe lint` | <1 second | 30 seconds | Quick |\n| Individual package test | 10 seconds | 60+ seconds | May have network failures |\n\n### .NET Commands  \n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `dotnet restore` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet build --configuration Release` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet test --filter \"Category=UnitV2\"` | 25 seconds | 120+ seconds | NEVER CANCEL |\n| `dotnet format --verify-no-changes` | 5-10 seconds | 60 seconds | Quick validation |\n\n## Repository Structure\n\n### Python Packages (`python/packages/`)\n- `autogen-core`: Core agent runtime, model interfaces, and base components\n- `autogen-agentchat`: High-level multi-agent conversation APIs  \n- `autogen-ext`: Extensions for specific model providers and tools\n- `autogen-studio`: Web-based IDE for agent workflows\n- `agbench`: Benchmarking suite for agent performance\n- `magentic-one-cli`: Multi-agent team CLI application\n\n### .NET Projects (`dotnet/src/`)\n- `AutoGen`: Legacy 0.2-style .NET packages (being deprecated)\n- `Microsoft.AutoGen.*`: New event-driven .NET packages\n- `AutoGen.Core`: Core .NET agent functionality\n- Multiple provider packages: OpenAI, Anthropic, Ollama, etc.\n\n### Key Configuration Files\n- `python/pyproject.toml`: Python workspace and tool configuration\n- `dotnet/global.json`: .NET SDK version requirements  \n- `dotnet/AutoGen.sln`: .NET solution file\n- `python/uv.lock`: Locked Python dependencies\n\n## Development Best Practices\n\n### Before Committing Changes\n**ALWAYS run these validation steps:**\n\n**Python:**\n```bash\ncd python && source .venv/bin/activate\npoe format    # Fix formatting\npoe lint      # Check code quality  \npoe mypy      # Type checking (6 minutes)\npoe docs-build # Verify docs build (1+ minutes)\n```\n\n**.NET:**\n```bash  \ncd dotnet && export PATH=\"$HOME/.dotnet:$PATH\"\ndotnet format --verify-no-changes  # Check formatting\ndotnet build --configuration Release --no-restore  # Build (53 seconds)\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Test (25 seconds)\n```\n\n### Key Directories Reference\n```\nautogen/\n├── python/                    # Python implementation\n│   ├── packages/             # Individual Python packages\n│   ├── docs/                 # Sphinx documentation\n│   ├── samples/              # Example code\n│   └── pyproject.toml        # Workspace configuration\n├── dotnet/                   # .NET implementation  \n│   ├── src/                  # Source projects\n│   ├── test/                 # Test projects\n│   ├── samples/              # Sample applications\n│   └── AutoGen.sln          # Solution file\n├── .github/workflows/        # CI/CD pipelines\n└── docs/                     # Additional documentation\n```\n\nThis framework supports creating both simple single-agent applications and complex multi-agent workflows with support for various LLM providers, tools, and deployment patterns.","isInternal":false,"tokens":2170,"sizeBytes":8858},{"name":"README.md","path":"python/packages/agbench/benchmarks/GAIA/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/README.md","title":"GAIA Documentation","category":"plugin-manifest","format":"markdown","content":"# GAIA Documentation\n\nPath: `python/packages/agbench/benchmarks/GAIA/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/README.md)","tokens":300,"sizeBytes":217},{"name":"README.md","path":"python/packages/agbench/benchmarks/HumanEval/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/README.md","title":"HumanEval Documentation","category":"plugin-manifest","format":"markdown","content":"# HumanEval Documentation\n\nPath: `python/packages/agbench/benchmarks/HumanEval/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/README.md)","tokens":300,"sizeBytes":232},{"name":"README.md","path":"python/packages/agbench/benchmarks/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/README.md","title":"benchmarks Documentation","category":"plugin-manifest","format":"markdown","content":"# benchmarks Documentation\n\nPath: `python/packages/agbench/benchmarks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/README.md)","tokens":300,"sizeBytes":213},{"name":"README.md","path":"python/packages/agbench/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/README.md","title":"agbench Documentation","category":"plugin-manifest","format":"markdown","content":"# AutoGenBench\n\nAutoGenBench (agbench) is a tool for repeatedly running a set of pre-defined AutoGen tasks in a setting with tightly-controlled initial conditions. With each run, AutoGenBench will start from a blank slate. The agents being evaluated will need to work out what code needs to be written, and what libraries or dependencies to install, to solve tasks. The results of each run are logged, and can be ingested by analysis or metrics scripts (such as `agbench tabulate`). By default, all runs are conducted in freshly-initialized docker containers, providing the recommended level of consistency and safety.\n\nAutoGenBench works with all AutoGen 0.1.*, and 0.2.* versions.\n\n## Technical Specifications\n\nIf you are already an AutoGenBench pro, and want the full technical specifications, please review the [contributor&#39;s guide](CONTRIBUTING.md).\n\n## Docker Requirement\n\nAutoGenBench also requires Docker (Desktop or Engine). **It will not run in GitHub codespaces**, unless you opt for native execution (which is strongly discouraged). To install Docker Desktop see [https://www.docker.com/products/docker-desktop/](https://www.docker.com/products/docker-desktop/).\n\nIf you are working in WSL, you can follow the instructions below to set up your environment:\n\n1. Install Docker Desktop. After installation, restart is needed, then open Docker Desktop, in Settings, Ressources, WSL Integration, Enable integration with additional distros – Ubuntu\n2. Clone autogen and export `AUTOGEN_REPO_BASE`. This environment variable enables the Docker containers to use the correct version agents.\n    ```bash\n    git clone git@github.com:microsoft/autogen.git\n    export AUTOGEN_REPO_BASE=<path_to_autogen>\n    ```\n\n## Installation and Setup\n\n[Deprecated currently] **To get the most out of AutoGenBench, the `agbench` package should be installed**. At present, the easiest way to do this is to install it via `pip`.\n\n\nIf you would prefer working from source code (e.g., for development, or to utilize an alternate branch), simply clone the [AutoGen](https://github.com/microsoft/autogen) repository, then install `agbench` via:\n\n```\npip install -e autogen/python/packages/agbench\n```\n\nAfter installation, you must configure your API keys. As with other AutoGen applications, AutoGenBench will look for the OpenAI keys in the OAI_CONFIG_LIST file in the current working directory, or the OAI_CONFIG_LIST environment variable. This behavior can be overridden using a command-line parameter described later.\n\nIf you will be running multiple benchmarks, it is often most convenient to leverage the environment variable option. You can load your keys into the environment variable by executing:\n\n```\nexport OAI_CONFIG_LIST=$(cat ./OAI_CONFIG_LIST)\n```\n\nIf an OAI_CONFIG_LIST is *not* provided (by means of file or environment variable), AutoGenBench will use the OPENAI_API_KEY environment variable instead.\n\nFor some benchmark scenarios, additional keys may be required (e.g., keys for the Bing Search API). These can be added to an `ENV.json` file in the current working folder. An example `ENV.json` file is provided below:\n\n```\n{\n    \"BING_API_KEY\": \"xxxyyyzzz\"\n}\n```\n\n## A Typical Session\n\nOnce AutoGenBench and necessary keys are installed, a typical session will look as follows:\n\n\n\nNavigate to HumanEval\n\n```bash\ncd autogen/python/packages/agbench/benchmarks/HumanEval\n```\n**Note:** The following instructions are specific to the HumanEval benchmark. For other benchmarks, please refer to the README in the respective benchmark folder, e.g.,: [AssistantBench](benchmarks/AssistantBench/README.md).\n\n\nCreate a file called ENV.json with the following (required) contents (If you're using MagenticOne), if using Azure:\n\n```json\n{\n    \"CHAT_COMPLETION_KWARGS_JSON\": \"{}\",\n    \"CHAT_COMPLETION_PROVIDER\": \"azure\"\n}\n```\n\nYou can also use the openai client by replacing the last two entries in the ENV file by:\n\n- `CHAT_COMPLETION_PROVIDER='openai'`\n- `CHAT_COMPLETION_KWARGS_JSON` with the following JSON structure:\n\n```json\n{\n  \"api_key\": \"REPLACE_WITH_YOUR_API\",\n  \"model\": \"REPLACE_WITH_YOUR_MODEL\"\n}\n```\n\nNow initialize the tasks.\n\n```bash\npython Scripts/init_tasks.py\n```\n\nNote: This will attempt to download HumanEval\n\n\nOnce the script completes, you should now see a folder in your current directory called `Tasks` that contains one JSONL file per template in `Templates`.\n\nNow to run a specific subset of HumanEval use:\n\n```bash\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nYou should see the command line print the raw logs that shows the agents in action To see a summary of the results (e.g., task completion rates), in a new terminal run the following:\n\n```bash\nagbench tabulate Results/human_eval_MagenticOne\n```\n\nWhere:\n\n- `agbench run Tasks/human_eval_MagenticOne.jsonl` runs the tasks defined in `Tasks/human_eval_MagenticOne.jsonl`\n- `agbench tablue results/human_eval_MagenticOne` tabulates the results of the run\n\nEach of these commands has extensive in-line help via:\n\n- `agbench --help`\n- `agbench run --help`\n- `agbench tabulate --help`\n- `agbench remove_missing --help`\n\n**NOTE:** If you are running `agbench` from within the repository, you need to navigate to the appropriate scenario folder (e.g., `scenarios/HumanEval`) and run the `Scripts/init_tasks.py` file.\n\nMore details of each command are provided in the sections that follow.\n\n\n## Running AutoGenBench\n\nTo run a benchmark (which executes the tasks, but does not compute metrics), simply execute:\n\n```\ncd [BENCHMARK]\nagbench run Tasks/*.jsonl\n```\n\nFor example,\n\n```\ncd HumanEval\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe default is to run each task once. To run each scenario 10 times, use:\n\n```\nagbench run --repeat 10 Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe `agbench` command-line tool allows a number of command-line arguments to control various parameters of execution. Type ``agbench -h`` to explore these options:\n\n```\n'agbench run' will run the specified autogen scenarios for a given number of repetitions and record all logs and trace information. When running in a Docker environment (default), each run will begin from a common, tightly controlled, environment. The resultant logs can then be further processed by other scripts to produce metrics.\n\npositional arguments:\n  scenario      The JSONL scenario file to run. If a directory is specified,\n                then all JSONL scenarios in the directory are run. (default:\n                ./scenarios)\n\noptions:\n  -h, --help            show this help message and exit\n  -c CONFIG, --config CONFIG\n                        The environment variable name or path to the OAI_CONFIG_LIST (default: OAI_CONFIG_LIST).\n  -r REPEAT, --repeat REPEAT\n                        The number of repetitions to run for each scenario (default: 1).\n  -s SUBSAMPLE, --subsample SUBSAMPLE\n                        Run on a subsample of the tasks in the JSONL file(s). If a decimal value is specified, then run on\n                        the given proportion of tasks in each file. For example \"0.7\" would run on 70% of tasks, and \"1.0\"\n                        would run on 100% of tasks. If an integer value is specified, then randomly select *that* number of\n                        tasks from each specified JSONL file. For example \"7\" would run tasks, while \"1\" would run only 1\n                        task from each specified JSONL file. (default: 1.0; which is 100%)\n  -m MODEL, --model MODEL\n                        Filters the config_list to include only models matching the provided model name (default: None, which\n                        is all models).\n  --requirements REQUIREMENTS\n                        The requirements file to pip install before running the scenario.\n  -d DOCKER_IMAGE, --docker-image DOCKER_IMAGE\n                        The Docker image to use when running scenarios. Can not be used together with --native. (default:\n                        'agbench:default', which will be created if not present)\n  --native              Run the scenarios natively rather than in docker. NOTE: This is not advisable, and should be done\n                        with great caution.\n```\n\n## Results\n\nBy default, the AutoGenBench stores results in a folder hierarchy with the following template:\n\n``./results/[scenario]/[task_id]/[instance_id]``\n\nFor example, consider the following folders:\n\n``./results/default_two_agents/two_agent_stocks/0``\n``./results/default_two_agents/two_agent_stocks/1``\n\n...\n\n``./results/default_two_agents/two_agent_stocks/9``\n\nThis folder holds the results for the ``two_agent_stocks`` task of the ``default_two_agents`` tasks file. The ``0`` folder contains the results of the first instance / run. The ``1`` folder contains the results of the second run, and so on. You can think of the _task_id_ as mapping to a prompt, or a unique set of parameters, while the _instance_id_ defines a specific attempt or run.\n\nWithin each folder, you will find the following files:\n\n- *timestamp.txt*: records the date and time of the run, along with the version of the autogen-agentchat library installed\n- *console_log.txt*: all console output produced by Docker when running AutoGen. Read this like you would a regular console.\n- *[agent]_messages.json*: for each Agent, a log of their messages dictionaries\n- *./coding*: A directory containing all code written by AutoGen, and all artifacts produced by that code.\n\n## Contributing or Defining New Tasks or Benchmarks\n\nIf you would like to develop -- or even contribute -- your own tasks or benchmarks, please review the [contributor&#39;s guide](CONTRIBUTING.md) for complete technical details.\n","isInternal":false,"tokens":2189,"sizeBytes":9633},{"name":"README.md","path":"python/packages/autogen-agentchat/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-agentchat/README.md","title":"autogen-agentchat Documentation","category":"plugin-manifest","format":"markdown","content":"# AutoGen AgentChat\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/index.html)\n\nAgentChat is a high-level API for building multi-agent applications.\nIt is built on top of the [`autogen-core`](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html) package.\nFor beginner users, AgentChat is the recommended starting point.\nFor advanced users, [`autogen-core`](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html)'s event-driven\nprogramming model provides more flexibility and control over the underlying components.\n\nAgentChat provides intuitive defaults, such as **Agents** with preset\nbehaviors and **Teams** with predefined [multi-agent design patterns](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/design-patterns/intro.html).\n","isInternal":false,"tokens":184,"sizeBytes":857},{"name":"README.md","path":"python/packages/autogen-core/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-core/README.md","title":"autogen-core Documentation","category":"plugin-manifest","format":"markdown","content":"# AutoGen Core\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html)\n\nAutoGen core offers an easy way to quickly build event-driven, distributed, scalable, resilient AI agent systems. Agents are developed by using the [Actor model](https://en.wikipedia.org/wiki/Actor_model). You can build and run your agent system locally and easily move to a distributed system in the cloud when you are ready.\n","isInternal":false,"tokens":94,"sizeBytes":444},{"name":"README.md","path":"python/packages/autogen-ext/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/README.md","title":"autogen-ext Documentation","category":"plugin-manifest","format":"markdown","content":"# AutoGen Extensions\n\n- [Documentation](https://microsoft.github.io/autogen/stable/user-guide/extensions-user-guide/index.html)\n\nAutoGen is designed to be extensible. The `autogen-ext` package contains many different component implementations maintained by the AutoGen project. However, we strongly encourage others to build their own components and publish them as part of the ecosytem.\n","isInternal":false,"tokens":79,"sizeBytes":388},{"name":"README.md","path":"python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md","title":"task_centric_memory Documentation","category":"plugin-manifest","format":"markdown","content":"# task_centric_memory Documentation\n\nPath: `python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/experimental/task_centric_memory/README.md)","tokens":300,"sizeBytes":306},{"name":"README.md","path":"python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md","title":"_host Documentation","category":"plugin-manifest","format":"markdown","content":"# _host Documentation\n\nPath: `python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/README.md)","tokens":300,"sizeBytes":258},{"name":"README.md","path":"python/packages/autogen-magentic-one/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-magentic-one/README.md","title":"autogen-magentic-one Documentation","category":"plugin-manifest","format":"markdown","content":"# Magentic-One\n\n> Magentic-One is now available as part of the `autogen-agentchat` library.\n> Please see the [user guide](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html) for information.\n\n> Looking for the original implementation of Magentic-One? It is available [here](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one).\n\n[Magentic-One](https://aka.ms/magentic-one-blog) is a generalist multi-agent system for solving open-ended web and file-based tasks across a variety of domains. It represents a significant step forward for multi-agent systems, achieving competitive performance on a number of agentic benchmarks (see the [technical report](https://arxiv.org/abs/2411.04468) for full details).\n\nWhen originally released in [November 2024](https://aka.ms/magentic-one-blog) Magentic-One was [implemented directly on the `autogen-core` library](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one). We have now ported Magentic-One to use `autogen-agentchat`, providing a more modular and easier to use interface. To this end, the older implementation is deprecated, but can be accessed at [https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one](https://github.com/microsoft/autogen/tree/v0.4.4/python/packages/autogen-magentic-one).\n\nMoving forward, the Magentic-One orchestrator [MagenticOneGroupChat](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.teams.html#autogen_agentchat.teams.MagenticOneGroupChat) is now simply an AgentChat team, supporting all standard AgentChat agents and features. Likewise, Magentic-One's [MultimodalWebSurfer](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.web_surfer.html#autogen_ext.agents.web_surfer.MultimodalWebSurfer), [FileSurfer](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.file_surfer.html#autogen_ext.agents.file_surfer.FileSurfer), and [MagenticOneCoderAgent](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.teams.magentic_one.html) agents are now broadly available as AgentChat agents, to be used in any AgentChat workflows.\n\nLastly, there is a helper class, [MagenticOne](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.teams.magentic_one.html#autogen_ext.teams.magentic_one.MagenticOne), which bundles all of this together as it was in the paper with minimal configuration\n\n## Citation\n\n```\n@misc{fourney2024magenticonegeneralistmultiagentsolving,\n      title={Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks},\n      author={Adam Fourney and Gagan Bansal and Hussein Mozannar and Cheng Tan and Eduardo Salinas and Erkang and Zhu and Friederike Niedtner and Grace Proebsting and Griffin Bassman and Jack Gerrits and Jacob Alber and Peter Chang and Ricky Loynd and Robert West and Victor Dibia and Ahmed Awadallah and Ece Kamar and Rafah Hosn and Saleema Amershi},\n      year={2024},\n      eprint={2411.04468},\n      archivePrefix={arXiv},\n      primaryClass={cs.AI},\n      url={https://arxiv.org/abs/2411.04468},\n}\n```\n","isInternal":false,"tokens":802,"sizeBytes":3185},{"name":"README.md","path":"python/packages/autogen-studio/frontend/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/README.md","title":"frontend Documentation","category":"plugin-manifest","format":"markdown","content":"# frontend Documentation\n\nPath: `python/packages/autogen-studio/frontend/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/README.md)","tokens":300,"sizeBytes":221},{"name":"README.md","path":"python/packages/autogen-studio/frontend/src/components/shared/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/shared/README.md","title":"shared Documentation","category":"plugin-manifest","format":"markdown","content":"# shared Documentation\n\nPath: `python/packages/autogen-studio/frontend/src/components/shared/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/shared/README.md)","tokens":300,"sizeBytes":263},{"name":"README.md","path":"python/packages/autogen-studio/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/README.md","title":"autogen-studio Documentation","category":"plugin-manifest","format":"markdown","content":"# AutoGen Studio\n\n[![PyPI version](https://badge.fury.io/py/autogenstudio.svg)](https://badge.fury.io/py/autogenstudio)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/autogenstudio)\n\n![ARA](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)\n\nAutoGen Studio is an AutoGen-powered AI app (user interface) to help you rapidly prototype AI agents, enhance them with skills, compose them into workflows and interact with them to accomplish tasks. It is built on top of the [AutoGen](https://microsoft.github.io/autogen) framework, which is a toolkit for building AI agents.\n\nCode for AutoGen Studio is on GitHub at [microsoft/autogen](https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio)\n\n> [!CAUTION]\n> AutoGen Studio is meant to help you rapidly prototype multi-agent workflows and demonstrate an example of end user interfaces built with AutoGen. It is **not meant to be a production-ready app**. Developers are encouraged to use the [AutoGen framework](https://microsoft.github.io/autogen) to build their own applications, implementing authentication, security and other features required for deployed applications.\n\n> [!WARNING]\n> AutoGen Studio is under active development. Expect breaking changes in upcoming releases.\n\n## A Note on Security\n\nAutoGen Studio is a research prototype and is **not meant to be used** in a production environment. Some baseline practices are encouraged e.g., using Docker code execution environment for your agents.\n\nHowever, other considerations such as rigorous tests related to jailbreaking, ensuring LLMs only have access to the right keys of data given the end user's permissions, and other security features are not implemented in AutoGen Studio.\n\nIf you are building a production application, please use the [AutoGen framework](https://microsoft.github.io/autogen) and implement the necessary security features.\n\n## Updates\n\n- **2024-11-14:** AutoGen Studio is being rewritten to use the updated AutoGen 0.4.0 api AgentChat api.\n- **2024-04-17:** April 17: AutoGen Studio database layer is now rewritten to use [SQLModel](https://sqlmodel.tiangolo.com/) (Pydantic + SQLAlchemy). This provides entity linking (skills, models, agents and workflows are linked via association tables) and supports multiple [database backend dialects](https://docs.sqlalchemy.org/en/20/dialects/) supported in SQLAlchemy (SQLite, PostgreSQL, MySQL, Oracle, Microsoft SQL Server). The backend database can be specified a `--database-uri` argument when running the application. For example, `autogenstudio ui --database-uri sqlite:///database.sqlite` for SQLite and `autogenstudio ui --database-uri postgresql+psycopg://user:password@localhost/dbname` for PostgreSQL.\n- **2024-03-12:** Default directory for AutoGen Studio is now /home/\\<USER\\>/.autogenstudio. You can also specify this directory using the `--appdir` argument when running the application. For example, `autogenstudio ui --appdir /path/to/folder`. This will store the database and other files in the specified directory e.g. `/path/to/folder/database.sqlite`. `.env` files in that directory will be used to set environment variables for the app.\n\n## Project Structure:\n\n- `autogenstudio/` contains code for the backend classes and web api (FastAPI)\n- `frontend/` contains code for the webui, built with Gatsby and TailwindCSS\n\n## Installation\n\nThere are two ways to install AutoGen Studio - from PyPi or from the source. We **recommend installing from PyPi** unless you plan to modify the source code.\n\n### Install from PyPi (Recommended)\n\nWe recommend using a virtual environment (e.g., venv) to avoid conflicts with existing Python packages. With Python 3.10 or newer active in your virtual environment, use pip to install AutoGen Studio:\n\n```bash\npip install -U autogenstudio\n```\n\n### Install from source\n\n_Note: This approach requires some familiarity with building interfaces in React._\n\n### Important: Git LFS Requirement\n\nAutoGen Studio uses Git Large File Storage (LFS) for managing image and other large files. If you clone the repository without git-lfs, you'll encounter build errors related to image formats.\n\n**Before cloning the repository:**\n\n1. Install git-lfs:\n\n   ```bash\n   # On Debian/Ubuntu\n   apt-get install git-lfs\n\n   # On macOS with Homebrew\n   brew install git-lfs\n\n   # On Windows with Chocolatey\n   choco install git-lfs\n   ```\n\n2. Set up git-lfs:\n   ```bash\n   git lfs install\n   ```\n\n**If you've already cloned the repository:**\n\n```bash\ngit lfs install\ngit lfs fetch --all\ngit lfs checkout  # downloads all missing image files to the working directory\n```\n\nThis setup is handled automatically if you use the dev container method of installation.\n\nYou have two options for installing from source: manually or using a dev container.\n\n#### A) Install from source manually\n\n1. Ensure you have Python 3.10+ and Node.js (version above 14.15.0) installed.\n2. Clone the AutoGen Studio repository and install its Python dependencies using `pip install -e .`\n3. Navigate to the `python/packages/autogen-studio/frontend` directory, install the dependencies, and build the UI:\n\n   ```bash\n   npm install -g gatsby-cli\n   npm install --global yarn\n   cd frontend\n   yarn install\n   yarn build\n   # Windows users may need alternative commands to build the frontend:\n   gatsby clean && rmdir /s /q ..\\\\autogenstudio\\\\web\\\\ui 2>nul & (set \\\"PREFIX_PATH_VALUE=\\\" || ver>nul) && gatsby build --prefix-paths && xcopy /E /I /Y public ..\\\\autogenstudio\\\\web\\\\ui\n   ```\n\n#### B) Install from source using a dev container\n\n1. Follow the [Dev Containers tutorial](https://code.visualstudio.com/docs/devcontainers/tutorial) to install VS Code, Docker and relevant extensions.\n2. Clone the AutoGen Studio repository.\n3. Open `python/packages/autogen-studio/`in VS Code. Click the blue button in bottom the corner or press F1 and select _\"Dev Containers: Reopen in Container\"_.\n4. Build the UI:\n\n   ```bash\n   cd frontend\n   yarn build\n   ```\n\n### Running the Application\n\nOnce installed, run the web UI by entering the following in your terminal:\n\n```bash\nautogenstudio ui --port 8081\n```\n\nThis command will start the application on the specified port. Open your web browser and go to <http://localhost:8081/> to use AutoGen Studio.\n\nAutoGen Studio also takes several parameters to customize the application:\n\n- `--host <host>` argument to specify the host address. By default, it is set to `localhost`.\n- `--appdir <appdir>` argument to specify the directory where the app files (e.g., database and generated user files) are stored. By default, it is set to the `.autogenstudio` directory in the user's home directory.\n- `--port <port>` argument to specify the port number. By default, it is set to `8080`.\n- `--reload` argument to enable auto-reloading of the server when changes are made to the code. By default, it is set to `False`.\n- `--database-uri` argument to specify the database URI. Example values include `sqlite:///database.sqlite` for SQLite and `postgresql+psycopg://user:password@localhost/dbname` for PostgreSQL. If this is not specified, the database URL defaults to a `database.sqlite` file in the `--appdir` directory.\n- `--upgrade-database` argument to upgrade the database schema to the latest version. By default, it is set to `False`.\n\nNow that you have AutoGen Studio installed and running, you are ready to explore its capabilities, including defining and modifying agent workflows, interacting with agents and sessions, and expanding agent skills.\n\n## AutoGen Studio Lite\n\nAutoGen Studio Lite provides a lightweight way to quickly prototype and experiment with AI agent teams. It's designed for rapid experimentation without the full database setup.\n\n### CLI Usage\n\nLaunch Studio Lite from the command line:\n\n```bash\n# Quick start with default team\nautogenstudio lite\n\n# Use custom team file\nautogenstudio lite --team ./my_team.json --port 8080\n\n# Custom session name\nautogenstudio lite --session-name \"My Experiment\" --auto-open\n```\n\n### Programmatic Usage\n\nUse Studio Lite directly in your Python code:\n\n```python\nfrom autogenstudio.lite import LiteStudio\n\n# Quick start with default team\nstudio = LiteStudio()\n# Use with AutoGen team objects\nfrom autogen_agentchat.teams import RoundRobinGroupChat\nteam = RoundRobinGroupChat([agent1, agent2], termination_condition=...)\n\n# Context manager usage\nwith LiteStudio(team=team) as studio:\n    # Studio runs in background\n    # Do other work here\n    pass\n```\n\n#### Local frontend development server\n\nSee `./frontend/README.md`\n\n## Contribution Guide\n\nWe welcome contributions to AutoGen Studio. We recommend the following general steps to contribute to the project:\n\n- Review the overall AutoGen project [contribution guide](https://github.com/microsoft/autogen?tab=readme-ov-file#contributing)\n- Please review the AutoGen Studio [roadmap](https://github.com/microsoft/autogen/issues/4006) to get a sense of the current priorities for the project. Help is appreciated especially with Studio issues tagged with `help-wanted`\n- Please initiate a discussion on the roadmap issue or a new issue to discuss your proposed contribution.\n- Submit a pull request with your contribution!\n- If you are modifying AutoGen Studio, it has its own devcontainer. See instructions in `.devcontainer/README.md` to use it\n- Please use the tag `proj-studio` for any issues, questions, and PRs related to Studio\n\n## FAQ\n\nPlease refer to the AutoGen Studio [FAQs](https://microsoft.github.io/autogen/docs/autogen-studio/faqs) page for more information.\n\n## Acknowledgements\n\nAutoGen Studio is Based on the [AutoGen](https://microsoft.github.io/autogen) project. It was adapted from a research prototype built in October 2023 (original credits: Gagan Bansal, Adam Fourney, Victor Dibia, Piali Choudhury, Saleema Amershi, Ahmed Awadallah, Chi Wang).\n","isInternal":false,"tokens":2284,"sizeBytes":9895},{"name":"README.md","path":"python/packages/autogen-test-utils/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-test-utils/README.md","title":"autogen-test-utils Documentation","category":"plugin-manifest","format":"markdown","content":"# test-utils\n","isInternal":false,"tokens":4,"sizeBytes":13},{"name":"README.md","path":"python/packages/component-schema-gen/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/component-schema-gen/README.md","title":"component-schema-gen Documentation","category":"plugin-manifest","format":"markdown","content":"# component-schema-gen\n\nThis is a tool to generate schema for built in components.\n\nSimply run `gen-component-schema` and it will print the schema to be used.\n","isInternal":false,"tokens":34,"sizeBytes":159},{"name":"README.md","path":"python/packages/magentic-one-cli/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/magentic-one-cli/README.md","title":"magentic-one-cli Documentation","category":"plugin-manifest","format":"markdown","content":"# magentic-one-cli\n","isInternal":false,"tokens":6,"sizeBytes":19},{"name":"README.md","path":"python/packages/pyautogen/README.md","rawUrl":"https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/pyautogen/README.md","title":"pyautogen Documentation","category":"plugin-manifest","format":"markdown","content":"# pyautogen\n\n> **NOTE:** This is a proxy package for the latest version of [`autogen-agentchat`](https://pypi.org/project/autogen-agentchat/). If you are looking for the 0.2.x version, please pin to `pyautogen~=0.2.0`.\n> To migrate from 0.2.x to the latest version, please refer to the [migration guide](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/migration-guide.html).\n> Read our [previous clarification regarding to forks](https://github.com/microsoft/autogen/discussions/4217).\n> We have regained admin access to this package.\n\nAutoGen is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans.\n\n- [Project homepage](https://github.com/microsoft/autogen)\n- [Documentation](https://microsoft.github.io/autogen/)\n- [Discord](https://aka.ms/autogen-discord)\n- [Contact](mailto:autogen@microsoft.com)\n","isInternal":false,"tokens":222,"sizeBytes":881}],"systemPromptSnippet":"<yakaai_skills repo=\"microsoft/autogen\">\n<!-- File: python/packages/agbench/benchmarks/GAIA/config.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/config.yaml)\n\n<!-- File: python/packages/agbench/benchmarks/GAIA/ENV.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# GAIA Skill\n\nPath: `python/packages/agbench/benchmarks/GAIA/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/ENV.yaml)\n\n<!-- File: python/packages/agbench/benchmarks/HumanEval/config.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/config.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/config.yaml)\n\n<!-- File: python/packages/agbench/benchmarks/HumanEval/ENV.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# HumanEval Skill\n\nPath: `python/packages/agbench/benchmarks/HumanEval/ENV.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/ENV.yaml)\n\n<!-- File: python/packages/agbench/CONTRIBUTING.md (Tokens: ~1872 | Category: anthropic-skill) -->\n# Contributing to AutoGenBench\n\nAs part of the broader AutoGen project, AutoGenBench welcomes community contributions. Contributions are subject to AutoGen's [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute), as well as a few additional AutoGenBench-specific requirements outlined here. You may also wish to develop your own private benchmark scenarios and the guidance in this document will help with such efforts as well. Below you will find the general requirements, followed by a detailed technical description.\n\n## General Contribution Requirements\nWe ask that all contributions to AutoGenBench adhere to the following:\n\n- Follow AutoGen's broader [contribution guidelines](https://microsoft.github.io/autogen/docs/Contribute)\n- All AutoGenBench benchmarks should live in a subfolder of `/benchmarks` alongside `HumanEval`, `GAIA`, etc.\n- Benchmark scenarios should include a detailed README.md, in the root of their folder, describing the benchmark and providing citations where warranted.\n- Benchmark data (tasks, ground truth, etc.) should be downloaded from their original sources rather than hosted in the AutoGen repository (unless the benchmark is original, and the repository *is* the original source)\n    - You can use the `Scripts/init_tasks.py` file to automate this download.\n- Basic scoring should be compatible with the `agbench tabulate` command (e.g., by outputting logs compatible with the default tabulation mechanism, or by providing a `Scripts/custom_tabulate.py` file)\n\nThese requirements are further detailed below, but if you simply copy the `HumanEval` folder, you will already be off to a great start.\n\n## Implementing and Running Benchmark Tasks\nAt the core of any benchmark is a set of tasks. To implement tasks that are runnable by AutoGenBench, you must adhere to AutoGenBench's templating and scenario expansion algorithms, as outlined below.\n\n### Task Definitions\n\nAll tasks are stored in JSONL files (in subdirectories under `./Tasks`). Each line of a tasks file is a JSON object with the following schema:\n\n```\n{\n   \"id\": string,\n   \"template\": dirname,\n   \"substitutions\" {\n       \"filename1\": {\n       \t   \"find_string1_1\": replace_string1_1,\n           \"find_string1_2\": replace_string1_2,\n           ...\n           \"find_string1_M\": replace_string1_N\n       }\n       \"filename2\": {\n       \t   \"find_string2_1\": replace_string2_1,\n           \"find_string2_2\": replace_string2_2,\n           ...\n           \"find_string2_N\": replace_string2_N\n       }\n   }\n}\n```\n\nFor example:\n\n```\n{\n    \"id\": \"two_agent_stocks_gpt4\",\n    \"template\": \"default_two_agents\",\n    \"substitutions\": {\n\t\"scenario.py\": {\n            \"__MODEL__\": \"gpt-4\",\n\t},\n\t\"prompt.txt\": {\n            \"__PROMPT__\": \"Plot and save to disk a chart of NVDA and TESLA stock price YTD.\"\n        }\n    }\n}\n```\n\nIn this example, the string `__MODEL__` will be replaced in the file `scenarios.py`, while the string `__PROMPT__` will be replaced in the `prompt.txt` file.\n\nThe `template` field can also take on a list value, but this usage is considered advanced and is not described here. See the `agbench/run_cmd.py` code, or the `GAIA` benchmark tasks files for additional information about this option.\n\n\n## Task Instance Expansion Algorithm\n\nOnce the tasks have been defined, as per above, they must be \"instantiated\" before they can be run. This instantiation happens automatically when the user issues the `agbench run` command and involves creating a local folder to share with Docker. Each instance and repetition gets its own folder along the path: `./results/[scenario]/[task_id]/[instance_id]`. For the sake of brevity we will refer to this folder as the `DEST_FOLDER`.\n\nThe algorithm for populating the `DEST_FOLDER` is as follows:\n\n1. Pre-populate DEST_FOLDER with all the basic starter files for running a scenario (found in `agbench/template`).\n2. Recursively copy the template folder specified in the JSONL line to DEST_FOLDER (if the JSON `template` attribute points to a folder) If the JSONs `template` attribute instead points to a file, copy the file, but rename it to `scenario.py`\n3. Apply any string replacements, as outlined in the prior section.\n4. Write a run.sh file to DEST_FOLDER that will be executed by Docker when it is loaded. The `run.sh` is described below.\n\n## Scenario Execution Algorithm\n\nOnce the task has been instantiated it is run (via run.sh). This script will execute the following steps:\n\n1. If a file named `global_init.sh` is present, run it.\n2. If a file named `scenario_init.sh` is present, run it.\n3. Install the requirements.txt file (if running in Docker)\n4. Run the task via `python scenario.py`\n5. If the scenario.py exited cleanly (exit code 0), then print \"SCENARIO.PY COMPLETE !#!#\"\n6. Clean up (delete cache, etc.)\n7. If a file named `scenario_finalize.sh` is present, run it.\n8. If a file named `global_finalize.sh` is present, run it.\n9. echo \"RUN COMPLETE !#!#\", signaling that all steps completed.\n\nNotably, this means that scenarios can add custom init and teardown logic by including `scenario_init.sh` and `scenario_finalize.sh` files.\n\nAt the time of this writing, the run.sh file is as follows:\n\n```sh\nexport AUTOGEN_TESTBED_SETTING=\"Docker\"\numask 000\n\n# Run the global init script if it exists\nif [ -f global_init.sh ] ; then\n    . ./global_init.sh\nfi\n\n# Run the scenario init script if it exists\nif [ -f scenario_init.sh ] ; then\n    . ./scenario_init.sh\nfi\n\n# Run the scenario\npip install -r requirements.txt\npython scenario.py\nEXIT_CODE=$?\nif [ $EXIT_CODE -ne 0 ]; then\n    echo SCENARIO.PY EXITED WITH CODE: $EXIT_CODE !#!#\nelse\n    echo SCENARIO.PY COMPLETE !#!#\nfi\n\n# Clean up\nif [ -d .cache ] ; then\n    rm -Rf .cache\nfi\n\n# Run the scenario finalize script if it exists\nif [ -f scenario_finalize.sh ] ; then\n    . ./scenario_finalize.sh\nfi\n\n# Run the global finalize script if it exists\nif [ -f global_finalize.sh ] ; then\n    . ./global_finalize.sh\nfi\n\necho RUN.SH COMPLETE !#!#\n```\n\nBe warned that this listing is provided here for illustration purposes, and may vary over time. The source of truth are the `run.sh` files found in the ``./results/[taskset]/[task_id]/[instance_id]`` folders.\n\n\n## Integrating with the `tabulate` \nThe above details are sufficient for defining and running tasks, but if you wish to support the `agbench tabulate`  commands, a few additional steps are required.\n\n### Tabulations\n\nIf you wish to leverage the default tabulation logic, it is as simple as arranging your `scenario.py` file to output the string \"ALL TESTS PASSED !#!#\" to the console in the event that a task was solved correctly.\n\nIf you wish to implement your own tabulation logic, simply create the file `Scripts/custom_tabulate.py` and include a `main(args)` method. Here, the `args` parameter will be provided by AutoGenBench, and is a drop-in replacement for `sys.argv`. In particular, `args[0]` will be the invocation command (similar to the executable or script name in `sys.argv`), and the remaining values (`args[1:]`) are the command line parameters.\n\nShould you provide a custom tabulation script, please implement `--help` and `-h` options for documenting your interface.\n\nThe `scenarios/GAIA/Scripts/custom_tabulate.py` is a great example of custom tabulation. It also shows how you can reuse some components of the default tabulator to speed up development.\n\n\n\n## Scripts/init_tasks.py\nFinally, you should provide an `Scripts/init_tasks.py` file, in your benchmark folder, and include a `main()` method therein. \n\nThis `init_tasks.py` script is a great place to download benchmarks from their original sources and convert them to the JSONL format required by AutoGenBench:\n- See `HumanEval/Scripts/init_tasks.py` for an example of how to expand a benchmark from an original GitHub repository.\n- See `GAIA/Scripts/init_tasks.py` for an example of how to expand a benchmark from `Hugging Face Hub`.\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/demonstration.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/self_teaching.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Configs Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/configs/teachability.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/add_topic.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/cell_towers_2_demo.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Insights Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/insights/liar_advice.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/10_liars.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/100_vampires.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/autogen_package.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_1.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml (Tokens: ~300 | Category: anthropic-skill) -->\n# Tasks Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/data_files/tasks/cell_towers_2.yaml)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Demonstration Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/demonstration/session.json)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Self_teaching Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/self_teaching/session.json)\n\n<!-- File: python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Teachability Skill\n\nPath: `python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-ext/tests/task_centric_memory/sessions/teachability/session.json)\n\n<!-- File: python/packages/autogen-studio/.devcontainer/devcontainer.json (Tokens: ~300 | Category: anthropic-skill) -->\n# .devcontainer Skill\n\nPath: `python/packages/autogen-studio/.devcontainer/devcontainer.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/.devcontainer/devcontainer.json)\n\n<!-- File: python/packages/autogen-studio/frontend/package.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/package.json)\n\n<!-- File: python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Gallery Skill\n\nPath: `python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/src/components/views/gallery/default_gallery.json)\n\n<!-- File: python/packages/autogen-studio/frontend/tsconfig.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Frontend Skill\n\nPath: `python/packages/autogen-studio/frontend/tsconfig.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/frontend/tsconfig.json)\n\n<!-- File: python/packages/autogen-studio/notebooks/team.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/team.json)\n\n<!-- File: python/packages/autogen-studio/notebooks/travel_team.json (Tokens: ~300 | Category: anthropic-skill) -->\n# Notebooks Skill\n\nPath: `python/packages/autogen-studio/notebooks/travel_team.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/autogen-studio/notebooks/travel_team.json)\n\n<!-- File: .github/copilot-instructions.md (Tokens: ~2170 | Category: copilot-instructions) -->\n# AutoGen Multi-Agent AI Framework\n\nAutoGen is a multi-language framework for creating AI agents that can act autonomously or work alongside humans. The project has separate Python and .NET implementations with their own development workflows.\n\nAlways reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.\n\n## Working Effectively\n\n### Prerequisites and Environment Setup\n\n**CRITICAL**: Install both .NET 8.0 and 9.0 for full compatibility:\n- Install uv package manager: `python3 -m pip install uv` \n- Install .NET 9.0 SDK: `wget https://dot.net/v1/dotnet-install.sh && chmod +x dotnet-install.sh && ./dotnet-install.sh --channel 9.0`\n- Install .NET 8.0 runtime: `./dotnet-install.sh --channel 8.0 --runtime dotnet && ./dotnet-install.sh --channel 8.0 --runtime aspnetcore`\n- Update PATH: `export PATH=\"$HOME/.dotnet:$PATH\"`\n\n### Python Development Workflow\n\n**Bootstrap and build Python environment:**\n```bash\ncd /home/runner/work/autogen/autogen/python\nuv sync --all-extras  # NEVER CANCEL: Takes 2 minutes. Set timeout to 300+ seconds.\nsource .venv/bin/activate\n```\n\n**Validate Python development:**\n```bash\n# Quick validation (under 1 second each)\npoe format  # Code formatting\npoe lint    # Linting with ruff\n\n# Type checking - NEVER CANCEL these commands\npoe mypy     # Takes 6 minutes. Set timeout to 600+ seconds.\npoe pyright  # Takes 41 seconds. Set timeout to 120+ seconds.\n\n# Individual package testing (core package example)\npoe --directory ./packages/autogen-core test  # Takes 10 seconds. Set timeout to 60+ seconds.\n\n# Documentation - NEVER CANCEL\npoe docs-build  # Takes 1 minute 16 seconds. Set timeout to 300+ seconds.\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: Python environment setup takes 2 minutes minimum\n- **NEVER CANCEL**: mypy type checking takes 6 minutes \n- **NEVER CANCEL**: Documentation build takes 1+ minutes\n- Format/lint tasks complete in under 1 second\n- Individual package tests typically complete in 10-60 seconds\n\n### .NET Development Workflow\n\n**Bootstrap and build .NET environment:**\n```bash\ncd /home/runner/work/autogen/autogen/dotnet\nexport PATH=\"$HOME/.dotnet:$PATH\"\ndotnet restore  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\ndotnet build --configuration Release  # NEVER CANCEL: Takes 53 seconds. Set timeout to 300+ seconds.\n```\n\n**Validate .NET development:**\n```bash\n# Unit tests - NEVER CANCEL\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Takes 25 seconds. Set timeout to 120+ seconds.\n\n# Format check (if build fails) \ndotnet format --verify-no-changes\n\n# Run samples\ncd samples/Hello\ndotnet run\n```\n\n**CRITICAL TIMING EXPECTATIONS:**\n- **NEVER CANCEL**: .NET restore takes 53 seconds minimum\n- **NEVER CANCEL**: .NET build takes 53 seconds minimum  \n- **NEVER CANCEL**: .NET unit tests take 25 seconds minimum\n- All build and test commands require appropriate timeouts\n\n### Complete Validation Workflow\n\n**Run full check suite (Python):**\n```bash\ncd /home/runner/work/autogen/autogen/python\nsource .venv/bin/activate\npoe check  # NEVER CANCEL: Runs all checks. Takes 7+ minutes total. Set timeout to 900+ seconds.\n```\n\n## Validation Scenarios\n\n### Manual Validation Requirements\nAlways manually validate changes by running complete user scenarios after making modifications:\n\n**Python validation scenarios:**\n1. **Import test**: Verify core imports work:\n   ```python\n   from autogen_agentchat.agents import AssistantAgent\n   from autogen_core import AgentRuntime\n   from autogen_ext.models.openai import OpenAIChatCompletionClient\n   ```\n\n2. **AutoGen Studio test**: Verify web interface can start:\n   ```bash\n   autogenstudio ui --help  # Should show help without errors\n   ```\n\n3. **Documentation test**: Build and verify docs generate without errors:\n   ```bash\n   poe docs-build && ls docs/build/index.html\n   ```\n\n**.NET validation scenarios:**\n1. **Sample execution**: Run Hello sample to verify runtime works:\n   ```bash\n   cd dotnet/samples/Hello && dotnet run --help\n   ```\n\n2. **Build validation**: Ensure all projects compile:\n   ```bash\n   dotnet build --configuration Release --no-restore\n   ```\n\n3. **Test execution**: Run unit tests to verify functionality:\n   ```bash\n   dotnet test --filter \"Category=UnitV2\" --configuration Release --no-build\n   ```\n\n## Common Issues and Workarounds\n\n### Network-Related Issues\n- **Python tests may fail** with network errors (tiktoken downloads, Playwright browser downloads) in sandboxed environments - this is expected\n- **Documentation intersphinx warnings** due to inability to reach external documentation sites - this is expected\n- **Individual package tests work better** than full test suite in network-restricted environments\n\n### .NET Runtime Issues  \n- **Requires both .NET 8.0 and 9.0**: Build uses 9.0 SDK but tests need 8.0 runtime\n- **Global.json specifies 9.0.100**: Must install exact .NET 9.0 version or later\n- **Path configuration critical**: Ensure `$HOME/.dotnet` is in PATH before system .NET\n\n### Python Package Issues\n- **Use uv exclusively**: Do not use pip/conda for dependency management\n- **Virtual environment required**: Always activate `.venv` before running commands\n- **Package workspace structure**: Project uses uv workspace with multiple packages\n\n## Timing Reference\n\n### Python Commands\n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `uv sync --all-extras` | 2 minutes | 300+ seconds | NEVER CANCEL |\n| `poe mypy` | 6 minutes | 600+ seconds | NEVER CANCEL |\n| `poe pyright` | 41 seconds | 120+ seconds | NEVER CANCEL |\n| `poe docs-build` | 1 min 16 sec | 300+ seconds | NEVER CANCEL |\n| `poe format` | <1 second | 30 seconds | Quick |\n| `poe lint` | <1 second | 30 seconds | Quick |\n| Individual package test | 10 seconds | 60+ seconds | May have network failures |\n\n### .NET Commands  \n| Command | Expected Time | Timeout | Notes |\n|---------|---------------|---------|-------|\n| `dotnet restore` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet build --configuration Release` | 53 seconds | 300+ seconds | NEVER CANCEL |\n| `dotnet test --filter \"Category=UnitV2\"` | 25 seconds | 120+ seconds | NEVER CANCEL |\n| `dotnet format --verify-no-changes` | 5-10 seconds | 60 seconds | Quick validation |\n\n## Repository Structure\n\n### Python Packages (`python/packages/`)\n- `autogen-core`: Core agent runtime, model interfaces, and base components\n- `autogen-agentchat`: High-level multi-agent conversation APIs  \n- `autogen-ext`: Extensions for specific model providers and tools\n- `autogen-studio`: Web-based IDE for agent workflows\n- `agbench`: Benchmarking suite for agent performance\n- `magentic-one-cli`: Multi-agent team CLI application\n\n### .NET Projects (`dotnet/src/`)\n- `AutoGen`: Legacy 0.2-style .NET packages (being deprecated)\n- `Microsoft.AutoGen.*`: New event-driven .NET packages\n- `AutoGen.Core`: Core .NET agent functionality\n- Multiple provider packages: OpenAI, Anthropic, Ollama, etc.\n\n### Key Configuration Files\n- `python/pyproject.toml`: Python workspace and tool configuration\n- `dotnet/global.json`: .NET SDK version requirements  \n- `dotnet/AutoGen.sln`: .NET solution file\n- `python/uv.lock`: Locked Python dependencies\n\n## Development Best Practices\n\n### Before Committing Changes\n**ALWAYS run these validation steps:**\n\n**Python:**\n```bash\ncd python && source .venv/bin/activate\npoe format    # Fix formatting\npoe lint      # Check code quality  \npoe mypy      # Type checking (6 minutes)\npoe docs-build # Verify docs build (1+ minutes)\n```\n\n**.NET:**\n```bash  \ncd dotnet && export PATH=\"$HOME/.dotnet:$PATH\"\ndotnet format --verify-no-changes  # Check formatting\ndotnet build --configuration Release --no-restore  # Build (53 seconds)\ndotnet test --configuration Release --filter \"Category=UnitV2\" --no-build  # Test (25 seconds)\n```\n\n### Key Directories Reference\n```\nautogen/\n├── python/                    # Python implementation\n│   ├── packages/             # Individual Python packages\n│   ├── docs/                 # Sphinx documentation\n│   ├── samples/              # Example code\n│   └── pyproject.toml        # Workspace configuration\n├── dotnet/                   # .NET implementation  \n│   ├── src/                  # Source projects\n│   ├── test/                 # Test projects\n│   ├── samples/              # Sample applications\n│   └── AutoGen.sln          # Solution file\n├── .github/workflows/        # CI/CD pipelines\n└── docs/                     # Additional documentation\n```\n\nThis framework supports creating both simple single-agent applications and complex multi-agent workflows with support for various LLM providers, tools, and deployment patterns.\n\n<!-- File: python/packages/agbench/benchmarks/GAIA/README.md (Tokens: ~300 | Category: plugin-manifest) -->\n# GAIA Documentation\n\nPath: `python/packages/agbench/benchmarks/GAIA/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/GAIA/README.md)\n\n<!-- File: python/packages/agbench/benchmarks/HumanEval/README.md (Tokens: ~300 | Category: plugin-manifest) -->\n# HumanEval Documentation\n\nPath: `python/packages/agbench/benchmarks/HumanEval/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/HumanEval/README.md)\n\n<!-- File: python/packages/agbench/benchmarks/README.md (Tokens: ~300 | Category: plugin-manifest) -->\n# benchmarks Documentation\n\nPath: `python/packages/agbench/benchmarks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/microsoft/autogen/HEAD/python/packages/agbench/benchmarks/README.md)\n\n<!-- File: python/packages/agbench/README.md (Tokens: ~2189 | Category: plugin-manifest) -->\n# AutoGenBench\n\nAutoGenBench (agbench) is a tool for repeatedly running a set of pre-defined AutoGen tasks in a setting with tightly-controlled initial conditions. With each run, AutoGenBench will start from a blank slate. The agents being evaluated will need to work out what code needs to be written, and what libraries or dependencies to install, to solve tasks. The results of each run are logged, and can be ingested by analysis or metrics scripts (such as `agbench tabulate`). By default, all runs are conducted in freshly-initialized docker containers, providing the recommended level of consistency and safety.\n\nAutoGenBench works with all AutoGen 0.1.*, and 0.2.* versions.\n\n## Technical Specifications\n\nIf you are already an AutoGenBench pro, and want the full technical specifications, please review the [contributor&#39;s guide](CONTRIBUTING.md).\n\n## Docker Requirement\n\nAutoGenBench also requires Docker (Desktop or Engine). **It will not run in GitHub codespaces**, unless you opt for native execution (which is strongly discouraged). To install Docker Desktop see [https://www.docker.com/products/docker-desktop/](https://www.docker.com/products/docker-desktop/).\n\nIf you are working in WSL, you can follow the instructions below to set up your environment:\n\n1. Install Docker Desktop. After installation, restart is needed, then open Docker Desktop, in Settings, Ressources, WSL Integration, Enable integration with additional distros – Ubuntu\n2. Clone autogen and export `AUTOGEN_REPO_BASE`. This environment variable enables the Docker containers to use the correct version agents.\n    ```bash\n    git clone git@github.com:microsoft/autogen.git\n    export AUTOGEN_REPO_BASE=<path_to_autogen>\n    ```\n\n## Installation and Setup\n\n[Deprecated currently] **To get the most out of AutoGenBench, the `agbench` package should be installed**. At present, the easiest way to do this is to install it via `pip`.\n\n\nIf you would prefer working from source code (e.g., for development, or to utilize an alternate branch), simply clone the [AutoGen](https://github.com/microsoft/autogen) repository, then install `agbench` via:\n\n```\npip install -e autogen/python/packages/agbench\n```\n\nAfter installation, you must configure your API keys. As with other AutoGen applications, AutoGenBench will look for the OpenAI keys in the OAI_CONFIG_LIST file in the current working directory, or the OAI_CONFIG_LIST environment variable. This behavior can be overridden using a command-line parameter described later.\n\nIf you will be running multiple benchmarks, it is often most convenient to leverage the environment variable option. You can load your keys into the environment variable by executing:\n\n```\nexport OAI_CONFIG_LIST=$(cat ./OAI_CONFIG_LIST)\n```\n\nIf an OAI_CONFIG_LIST is *not* provided (by means of file or environment variable), AutoGenBench will use the OPENAI_API_KEY environment variable instead.\n\nFor some benchmark scenarios, additional keys may be required (e.g., keys for the Bing Search API). These can be added to an `ENV.json` file in the current working folder. An example `ENV.json` file is provided below:\n\n```\n{\n    \"BING_API_KEY\": \"xxxyyyzzz\"\n}\n```\n\n## A Typical Session\n\nOnce AutoGenBench and necessary keys are installed, a typical session will look as follows:\n\n\n\nNavigate to HumanEval\n\n```bash\ncd autogen/python/packages/agbench/benchmarks/HumanEval\n```\n**Note:** The following instructions are specific to the HumanEval benchmark. For other benchmarks, please refer to the README in the respective benchmark folder, e.g.,: [AssistantBench](benchmarks/AssistantBench/README.md).\n\n\nCreate a file called ENV.json with the following (required) contents (If you're using MagenticOne), if using Azure:\n\n```json\n{\n    \"CHAT_COMPLETION_KWARGS_JSON\": \"{}\",\n    \"CHAT_COMPLETION_PROVIDER\": \"azure\"\n}\n```\n\nYou can also use the openai client by replacing the last two entries in the ENV file by:\n\n- `CHAT_COMPLETION_PROVIDER='openai'`\n- `CHAT_COMPLETION_KWARGS_JSON` with the following JSON structure:\n\n```json\n{\n  \"api_key\": \"REPLACE_WITH_YOUR_API\",\n  \"model\": \"REPLACE_WITH_YOUR_MODEL\"\n}\n```\n\nNow initialize the tasks.\n\n```bash\npython Scripts/init_tasks.py\n```\n\nNote: This will attempt to download HumanEval\n\n\nOnce the script completes, you should now see a folder in your current directory called `Tasks` that contains one JSONL file per template in `Templates`.\n\nNow to run a specific subset of HumanEval use:\n\n```bash\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nYou should see the command line print the raw logs that shows the agents in action To see a summary of the results (e.g., task completion rates), in a new terminal run the following:\n\n```bash\nagbench tabulate Results/human_eval_MagenticOne\n```\n\nWhere:\n\n- `agbench run Tasks/human_eval_MagenticOne.jsonl` runs the tasks defined in `Tasks/human_eval_MagenticOne.jsonl`\n- `agbench tablue results/human_eval_MagenticOne` tabulates the results of the run\n\nEach of these commands has extensive in-line help via:\n\n- `agbench --help`\n- `agbench run --help`\n- `agbench tabulate --help`\n- `agbench remove_missing --help`\n\n**NOTE:** If you are running `agbench` from within the repository, you need to navigate to the appropriate scenario folder (e.g., `scenarios/HumanEval`) and run the `Scripts/init_tasks.py` file.\n\nMore details of each command are provided in the sections that follow.\n\n\n## Running AutoGenBench\n\nTo run a benchmark (which executes the tasks, but does not compute metrics), simply execute:\n\n```\ncd [BENCHMARK]\nagbench run Tasks/*.jsonl\n```\n\nFor example,\n\n```\ncd HumanEval\nagbench run Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe default is to run each task once. To run each scenario 10 times, use:\n\n```\nagbench run --repeat 10 Tasks/human_eval_MagenticOne.jsonl\n```\n\nThe `agbench` command-line tool allows a number of command-line arguments to control various parameters of execution. Type ``agbench -h`` to explore these options:\n\n```\n'agbench run' will run the specified autogen scenarios for a given number of repetitions and record all logs and trace information. When running in a Docker environment (default), each run will begin from a common, tightly controlled, environment. The resultant logs can then be further processed by other scripts to produce metrics.\n\npositional arguments:\n  scenario      The JSONL scenario file to run. If a directory is specified,\n                then all JSONL scenarios in the directory are run. (default:\n                ./scenarios)\n\noptions:\n  -h, --help            show this help message and exit\n  -c CONFIG, --config CONFIG\n                        The environment variable name or path to the OAI_CONFIG_LIST (default: OAI_CONFIG_LIST).\n  -r REPEAT, --repeat REPEAT\n                        The number of repetitions to run for each scenario (default: 1).\n  -s SUBSAMPLE, --subsample SUBSAMPLE\n                        Run on a subsample of the tasks in the JSONL file(s). If a decimal value is specified, then run on\n                        the given proportion of tasks in each file. For example \"0.7\" would run on 70% of tasks, and \"1.0\"\n                        would run on 100% of tasks. If an integer value is specified, then randomly select *that* number of\n                        tasks from each specified JSONL file. For example \"7\" would run tasks, while \"1\" would run only 1\n                        task from each specified JSONL file. (default: 1.0; which is 100%)\n  -m MODEL, --model MODEL\n                        Filters the config_list to include only models matching the provided model name (default: None, which\n                        is all models).\n  --requirements REQUIREMENTS\n                        The requirements file to pip install before running the scenario.\n  -d DOCKER_IMAGE, --docker-image DOCKER_IMAGE\n                        The Docker image to use when running scenarios. Can not be used together with --native. (default:\n                        'agbench:default', which will be created if not present)\n  --native              Run the scenarios natively rather than in docker. NOTE: This is not advisable, and should be done\n                        with great caution.\n```\n\n## Results\n\nBy default, the AutoGenBench stores results in a folder hierarchy with the following template:\n\n``./results/[scenario]/[task_id]/[instance_id]``\n\nFor example, consider the following folders:\n\n``./results/default_two_agents/two_agent_stocks/0``\n``./results/default_two_agents/two_agent_stocks/1``\n\n...\n\n``./results/default_two_agents/two_agent_stocks/9``\n\nThis folder holds the results for the ``two_agent_stocks`` task of the ``default_two_agents`` tasks file. The ``0`` folder contains the results of the first instance / run. The ``1`` folder contains the results of the second run, and so on. You can think of the _task_id_ as mapping to a prompt, or a unique set of parameters, while the _instance_id_ defines a specific attempt or run.\n\nWithin each folder, you will find the following files:\n\n- *timestamp.txt*: records the date and time of the run, along with the version of the autogen-agentchat library installed\n- *console_log.txt*: all console output produced by Docker when running AutoGen. Read this like you would a regular console.\n- *[agent]_messages.json*: for each Agent, a log of their messages dictionaries\n- *./coding*: A directory containing all code written by AutoGen, and all artifacts produced by that code.\n\n## Contributing or Defining New Tasks or Benchmarks\n\nIf you would like to develop -- or even contribute -- your own tasks or benchmarks, please review the [contributor&#39;s guide](CONTRIBUTING.md) for complete technical details.\n\n<!-- ... +13 more modular skills available via /api/skills/microsoft/autogen?path={path} -->\n</yakaai_skills>"}