agent-framework Documentation

1. Project Overview & Quickstart (microsoft/agent-framework) Copy Chapter

File: README.md Copy Chapter

Welcome to Microsoft Agent Framework!

Microsoft Agent Framework (MAF) is an open, multi-language framework for building production-grade AI agents and multi-agent workflows in .NET and Python.

Microsoft Agent Framework is built for teams taking agents from prototype to production. It provides a consistent foundation for building, orchestrating, and operating agent systems across Python, .NET and Go, while keeping architecture choices open as requirements evolve, and supports a broad ecosystem including Microsoft Foundry, Azure OpenAI, OpenAI, and the GitHub Copilot SDK, with samples and hosting patterns for both local development and cloud deployment.

[!NOTE]
For the Go SDK, including its documentation, samples, contribution guidance, and issue tracker, visit microsoft/agent-framework-go.

Is this the right framework for you? Copy Chapter

MAF is a strong fit if you:

• are building agents and workflows you expect to run in production,
• need orchestration beyond a single prompt or stateless chat loop,
• want graph-based patterns such as sequential, concurrent, handoff, and group collaboration,
• care about durability, restartability, observability, governance, or human-in-the-loop control,
• need provider flexibility so your architecture can evolve without major rewrites.

Key Features Copy Chapter

Explore new MAF capabilities and real implementation patterns on the official blog.

• Python and C#/.NET Support: Full framework support for both Python and C#/.NET implementations with consistent APIs
  • Python packages | .NET source
• Go Support: For the Go SDK, including its documentation, samples, contribution guidance, and issue tracker, visit microsoft/agent-framework-go.
• Multiple Agent Provider Support: Support for various LLM providers with more being added continuously
  • Python examples | .NET examples
• Middleware: Flexible middleware system for request/response processing, exception handling, and custom pipelines
  • Python middleware | .NET middleware
• Orchestration Patterns & Workflows: Build multi-agent systems with graph-based workflows supporting sequential, concurrent, handoff, and group collaboration patterns; includes checkpointing, streaming, human-in-the-loop, and time-travel
  • Python workflows | .NET workflows
• Foundry Hosted Agents (new): Deploy and host your agents to Foundry-hosted infrastructure with just 2 additional lines of code
  • Python samples | .NET samples
• Observability: Built-in OpenTelemetry integration for distributed tracing, monitoring, and debugging
  • Python observability | .NET telemetry
• Declarative Agents: Define agents using YAML for faster setup and versioning
  • Declarative agent samples
• Agent Skills: Build domain-specific knowledge bases from multiple sources—files, inline code, class libraries—for agents to discover and use
  • Skills design
• AF Labs: Experimental packages for cutting-edge features including benchmarking, reinforcement learning, and research initiatives
  • Labs directory
• DevUI: Interactive developer UI for agent development, testing, and debugging workflows
  • See the DevUI in action

Table of Contents Copy Chapter

• Getting Started
  • Installation
  • Learning Resources
  • Quickstart
    • Basic Agent - Python
    • Basic Agent - .NET
• More Examples & Samples
• Community & Feedback
• Troubleshooting
• Contributor Resources

Getting Started Copy Chapter

Installation Copy Section

Python

pip install agent-framework
# This will install all sub-packages, see `python/packages` for individual packages.
# It may take a minute on first install on Windows.

.NET

dotnet add package Microsoft.Agents.AI
# For Foundry integration (used in the .NET quickstart below):
dotnet add package Microsoft.Agents.AI.Foundry
dotnet add package Azure.AI.Projects
dotnet add package Azure.Identity

Learning Resources Copy Section

• Overview - High level overview of the framework
• Quick Start - Get started with a simple agent
• Tutorials - Step by step tutorials
• User Guide - In-depth user guide for building agents and workflows
• Migration from Semantic Kernel - Guide to migrate from Semantic Kernel
• Migration from AutoGen - Guide to migrate from AutoGen

Quickstart Copy Section

Basic Agent - Python

Create a simple Azure Responses Agent that writes a haiku about the Microsoft Agent Framework

# pip install agent-framework
# Use `az login` to authenticate with Azure CLI
import os
import asyncio
from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from azure.identity import AzureCliCredential


async def main():
    # Initialize a chat agent with Microsoft Foundry
    # the endpoint, deployment name, and api version can be set via environment variables
    # or they can be passed in directly to the FoundryChatClient constructor
    agent = Agent(
      client=FoundryChatClient(
          credential=AzureCliCredential(),
          # project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
          # model=os.environ["FOUNDRY_MODEL_DEPLOYMENT_NAME"],
      ),
      name="HaikuAgent",
      instructions="You are an upbeat assistant that writes beautifully.",
    )

    print(await agent.run("Write a haiku about Microsoft Agent Framework."))

if __name__ == "__main__":
    asyncio.run(main())

Basic Agent - .NET

Create a simple Agent, using Microsoft Foundry that writes a haiku about the Microsoft Agent Framework

// This sample shows how to create and run a basic agent with AIProjectClient.AsAIAgent(...).

using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;

string endpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";

AIAgent agent =
    new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential())
    .AsAIAgent(model: deploymentName, instructions: "You are an upbeat assistant that writes beautifully.", name: "HaikuAgent");

// Once you have the agent, you can invoke it like any other AIAgent.
Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework."));

More Examples & Samples Copy Chapter

Python Copy Section

• Getting Started: progressive tutorial from hello-world to workflows
• Agent Concepts: deep-dive samples by topic (tools, middleware, providers, etc.)
• Workflows: workflow creation and integration with agents
• Hosting: A2A, self-hosted protocol helpers, and Foundry hosted agents. Durable Task and Azure Functions samples are in the Durable Agent Framework extension.
• End-to-End: full applications, evaluation, and demos

.NET Copy Section

• Getting Started: progressive tutorial from hello agent to workflows
• Agent Concepts: basic agent creation and tool usage
• Agent Providers: samples showing different agent providers
• Workflows: advanced multi-agent patterns and workflow orchestration
• Hosting: A2A and Foundry hosted agents. Durable agent and workflow samples are in the Durable Agent Framework extension.
• End-to-End: full applications and demos

Community & Feedback Copy Chapter

• Found a bug? File a GitHub issue to help us improve.
• Enjoying MAF? to show your support and help others discover the project.
• Have questions? Join our Discord or visit weekly office hours.

Troubleshooting Copy Chapter

Authentication Copy Section

Tip: DefaultAzureCredential is convenient for development but in production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid latency issues, unintended credential probing, and potential security risks from fallback mechanisms.

Environment Variables Copy Section

For environment variable configuration specific to each sample, refer to the README in the sample directory (Python samples | .NET samples).

Contributor Resources Copy Chapter

• Contributing Guide
• Code of Conduct
• Python Development Guide
• Design Documents
• Architectural Decision Records

Important Notes Copy Chapter

[!IMPORTANT]
If you use Microsoft Agent Framework to build applications that operate with any third-party servers, agents, code, or non-Azure Direct models (“Third-Party Systems”), you do so at your own risk. Third-Party Systems are Non-Microsoft Products under the Microsoft Product Terms and are governed by their own third-party license terms. You are responsible for any usage and associated costs.

We recommend reviewing all data being shared with and received from Third-Party Systems and being cognizant of third-party practices for handling, sharing, retention and location of data. It is your responsibility to manage whether your data will flow outside of your organization’s Azure compliance and geographic boundaries and any related implications, and that appropriate permissions, boundaries and approvals are provisioned.

You are responsible for carefully reviewing and testing applications you build using Microsoft Agent Framework in the context of your specific use cases, and making all appropriate decisions and customizations. This includes implementing your own responsible AI mitigations such as metaprompt, content filters, or other safety systems, and ensuring your applications meet appropriate quality, reliability, security, and trustworthiness standards. See also: Transparency FAQ

File: docs/design/python-package-setup.md Copy Chapter

Python Package design for Agent Framework

Design goals Copy Chapter

• Developer experience is key
  • the components needed for a basic agent with tools and a runtime should be importable from agent_framework without having to import from subpackages. This will be referred to as tier 0 components.
  • for more advanced components, tier 1 components, such as context providers, guardrails, vector data, text search, exceptions, evaluation, utils, telemetry and workflows, they should be importable from agent_framework.<component>, so for instance from agent_framework.vector_data import vectorstoremodel.
  • for parts of the package that are either additional functionality or integrations with other services (connectors) (tier 2), we use the term tier 2, however they should also be importable from agent_framework.<component>, so for instance from agent_framework.openai import OpenAIClient.
    • this means that the package structure is flat, and the components are grouped by functionality, not by type, so for instance from agent_framework.openai import OpenAIChatClient will import the OpenAI chat client, but also the OpenAI tools, and any other OpenAI related functionality.
    • There should not be a need for deeper imports from those packages, unless a good case is made for that, so the internals of the extensions packages should always be a folder with the name of the package, a __init__.py and one or more _files.py file, where the _files.py file contains the implementation details, and the __init__.py file exposes the public interface.
  • if a single file becomes too cumbersome (files are allowed to be 1k+ lines) it should be split into a folder with an __init__.py that exposes the public interface and a _files.py that contains the implementation details, with a __all__ in the init to expose the right things, if there are very large dependencies being loaded it can optionally using lazy loading to avoid loading the entire package when importing a single component.
  • as much as possible, related things are in a single file which makes understanding the code easier.
  • simple and straightforward logging and telemetry setup, so developers can easily add logging and telemetry to their code without having to worry about the details.
• Independence of connectors
  • To allow connectors to be treated as independent packages, we will use namespace packages for connectors, in principle this only includes the packages that we will develop in our repo, since that is easy to manage and maintain.
  • further advantages are that each package can have a independent lifecycle, versioning, and dependencies.
  • and this gives us insights into the usage, through pip install statistics, especially for connectors to services outside of Microsoft.
  • the goal is to group related connectors based on vendors, not on types, so for instance doing: import agent_framework.google will import connectors for all Google services, such as GoogleChatClient but also BigQueryCollection, etc.
  • All dependencies for a subpackage should be required dependencies in that package, and that package becomes a optional dependency in the main package as an extra with the same name, so in the main pyproject.toml we will have:
    toml Copy

    [project.optional-dependencies]
    google = [
        "agent-framework-google == 1.0.0"
    ]

  • this means developers can use pip install agent-framework[google] --pre to get AF with all Google connectors and dependencies, as well as manually installing the subpackage with pip install agent-framework-google --pre.

Sample getting started code Copy Section

from typing import Annotated
from agent_framework import Agent, ai_function
from agent_framework.openai import OpenAIChatClient

@ai_function(description="Get the current weather in a given location")
async def get_weather(location: Annotated[str, "The location as a city name"]) -> str:
    """Get the current weather in a given location."""
    # Implementation of the tool to get weather
    return f"The current weather in {location} is sunny."

agent = Agent(
    name="MyAgent",
    model_client=OpenAIChatClient(),
    tools=get_weather,
    description="An agent that can get the current weather.",
)
response = await agent.run("What is the weather in Amsterdam?")
print(response)

Global Package structure Copy Chapter

Overall the following structure is proposed:

• agent-framework
  • core components, will be exposed directly from agent_framework:
    • (single) agents (includes threads)
    • tools (includes MCP and OpenAPI)
    • types
    • context_providers
    • logging
    • workflows (includes multi-agent orchestration)
    • middleware
    • telemetry (user_agent)
  • advanced components, will be exposed from agent_framework.<component>:
    • vector_data (tbd, vector stores and other MEVD-like pieces)
    • text_search (tbd)
    • exceptions
    • evaluations (tbd)
    • utils (optional)
    • observability
  • vendor folders with connectors and integrations, will be exposed from agent_framework.<vendor>:
    • Code can be both in folder or in subpackage with lazy import.
    • See subpackage scope below for more detail
• tests
• samples
• extensions
  • azure
  • ...

All the init's in the subpackages will use lazy loading so avoid importing the entire package when importing a single component.
Internal imports will be done using relative imports, so that the package can be used as a namespace package.

File structure Copy Section

The resulting file structure will be as follows (not all things currently implemented, just an example):

packages/
    main/
        agent_framework/
            azure/
                __init__.py
                _chat_client.py
                ...
            microsoft/
                __init__.py
                _copilot_studio.py
                ...
            openai/
                __init__.py
                _chat_client.py
                _shared.py
                exceptions.py
            __init__.py
            __init__.pyi
            _agents.py
            _tools.py
            _models.py
            _logging.py
            _middleware.py
            _telemetry.py
            observability.py
            exceptions.py
            utils.py
            py.typed
        _workflow/
            __init__.py
            _workflow.py
            ...etc...
        tests/
            unit/
                test_types.py
            integration/
                test_chat_clients.py
        pyproject.toml
        ...
    azure-ai-agents/
        agent_framework-azure-ai-agents/
            __init__.py
            _chat_client.py
            ...
        tests/
            test_azure_ai_agents.py
        samples/ (optional)
            ...
        pyproject.toml
        ...
    redis/
        ...
    mem0/
        agent_framework-mem0/
            __init__.py
            _provider.py
            ...
        tests/
            test_mem0_provider.py
        samples/ (optional)
            ...
        pyproject.toml
        ...
    ...
samples/
    ...
pyproject.toml
LICENSE
uv.lock
.pre-commit-config.yaml

We might add a template subpackage as well, to make it easy to setup, this could be based on the first one that is added.

In the DEV_SETUP.md we will add instructions for how to deal with the path depth issues, especially on Windows, where the maximum path length can be a problem.

Subpackage scope Copy Section

Sub-packages are comprised of two parts, the code itself and the dependencies, the choice of when to use a subpackage and when to use a extra in the main package is based on the status of dependencies and/or possibilities of a external support mechanism. What this means is that:

• Integrations that need non-GA dependencies will be sub-packages and installed only when using a extra, so that we can avoid having non-GA dependencies in the main package.
• Integrations where the AF-code is still experimental, preview or release candidate will be sub-packages, so that we can avoid having non-GA code in the main package and we can version those packages properly.
• Integrations that are outside Microsoft and where we might not always be able to fast-follow breaking changes, will stay as sub-packages, to provide some isolation and to be able to version them properly.
• Integrations that are mature and that have released (GA) dependencies and features on the service side will be moved into the main package, the dependencies of those packages will stay installable under the same extra name, so that users do not have to change anything, and we then remove the subpackage itself.
• All subpackage imports in the code should be from a stable place, mostly vendor-based, so that when something moves from a subpackage to the main package, the import path does not change, so from agent_framework.microsoft import CopilotAgent will always work, even if it moves from the agent-framework-microsoft-copilot package to the main agent-framework package.
• The imports in those vendor namespaces (these won't be actual python namespaces, just the folders with a init.py file and any code) will do lazy loading and raise a meaningful error if the subpackage or dependencies are not installed, so that users know which extra to install with ease.
• On a case by case basis we can decide to create additional a extra, that combines multiple sub-packages and dependencies into one extra, so that users who work primarily with one platform can install everything they need with a single extra, for example (not implemented) you can install with the agent-framework[azure-purview] extra that only implement a PurviewMiddleware, or you can install with the agent-framework[azure] extra that includes all Azure related connectors, like purview, content-safety and others (all examples, not actual packages), regardless of where the code sits, these should always be importable from agent_framework.azure.
• Subpackage naming should also follow this, so in principle a package name is <vendor/folder>-<feature/brand>, so google-gemini, azure-purview, microsoft-copilotstudio, etc. For smaller vendors, where it's less likely to have a multitude of connectors, we can skip the feature/brand part, so mem0, redis, etc.
• For Microsoft services we will have two vendor folders, azure and microsoft, where azure contains all Azure services, while microsoft contains other Microsoft services, such as Copilot Studio Agents.

This setup was discussed at length and the decision is captured in ADR-0008.

Evolving the package structure

For each of the advanced components, we have two reason why we may split them into a folder, with an __init__.py and optionally a _files.py:

1. If the file becomes too large, we can split it into multiple _files, while still keeping the public interface in the __init__.py file, this is a non-breaking change
2. If we want to partially or fully move that code into a separate package.
   In this case we do need to lazy load anything that was moved from the main package to the subpackage, so that existing code still works, and if the subpackage is not installed we can raise a meaningful error.

Coding standards Copy Chapter

Coding standards will be maintained in the DEV_SETUP.md file.

Tooling Copy Section

uv and ruff are the main tools, for package management and code formatting/linting respectively.

Type checking

We currently can choose between mypy, pyright, ty and pyrefly for static type checking.
I propose we run mypy and pyright in GHA, similar to what AG already does. We might explore newer tools as a later date.

Task runner

AG already has experience with poe the poet, so let's start there, removing the MAKE file setup that SK uses.

Unit test coverage Copy Section

The goal is to have at least 80% unit test coverage for all code under both the main package and the subpackages.

Telemetry and logging Copy Section

Telemetry and logging are handled by the agent_framework.telemetry and agent_framework._logging packages.

Logging

Logging is considered as part of the basic setup, while telemetry is a advanced concept.
The telemetry package will use OpenTelemetry to provide a consistent way to collect and export telemetry data, similar to how we do this now in SK.

The logging will be simplified, there will be one logger in the base package:

• name: agent_framework - used for all logging in the abstractions and base components

Each of the other subpackages for connectors will have a similar single logger.

• name: agent_framework.openai
• name: agent_framework.azure

This means that when a logger is needed, it should be created like this:

from agent_framework import get_logger

logger = get_logger()
#or in a subpackage:
logger = get_logger('agent_framework.openai')

The implementation should be something like this:

# in file _logging.py
import logging

def get_logger(name: str = "agent_framework") -> logging.Logger:
    """
    Get a logger with the specified name, defaulting to 'agent_framework'.

    Args:
        name (str): The name of the logger. Defaults to 'agent_framework'.

    Returns:
        logging.Logger: The configured logger instance.
    """
    logger = logging.getLogger(name)
    # create the specifics for the logger, such as setting the level, handlers, etc.
    return logger

This will ensure that the logger is created with the correct name and configuration, and it will be consistent across the package.

Further there should be a easy way to configure the log levels, either through a environment variable or with a similar function as the get_logger.

This will not be allowed:

import logging

logger = logging.getLogger(__name__)

This is allowed but discouraged, if the get_logger function has been called at least once then this will return the same logger as the get_logger function, however that might not have happened and then the logging experience (in terms of formats and handlers, etc) is not consistent across the package:

import logging

logger = logging.getLogger("agent_framework")

Telemetry

Telemetry will be based on OpenTelemetry (OTel), and will be implemented in the agent_framework.telemetry package.

We will also add headers with user-agent strings where applicable, these will include agent-framework-python and the version.

We should consider auto-instrumentation and provide an implementation of it to the OTel community.

Build and release Copy Section

The build step will be done in GHA, adding the package to the release and then we call into Azure DevOps to use the ESRP pipeline to publish to pypi. This is how SK already works, we will just have to adapt it to the new package structure.

For now we will stick to semantic versioning, and all preview release will be tagged as such.

File: docs/decisions/0001-agent-run-response.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: westey-m date: 2025-07-10 deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub consulted: informed: Copy Chapter

Agent Run Responses Design

Context and Problem Statement Copy Chapter

Agents may produce lots of output during a run including

1. [Primary] General response messages to the caller (this may be in the form of text, including structured output, images, sound, etc.)
2. [Primary] Structured confirmation requests to the caller
3. [Secondary] Tool invocation activities executed (both local and remote). For information only.
4. Reasoning/Thinking output.
   1. [Primary] In some cases an LLM may return reasoning output intermixed with as part of the answer to the caller, since the caller's prompt asked for this detail in some way. This should be considered a specialization of 1.
   2. [Secondary] Reasonining models optionally produce reasoning output separate from the answer to the caller's question, and this should be considered secondary content.
5. [Secondary] Handoffs / transitions from agent to agent where an agent contains sub agents.
6. [Secondary] An indication that the agent is responding (i.e. typing) as if it's a real human.
7. Complete messages in addition to updates, when streaming
8. Id for long running process that is launched
9. and more

We need to ensure that with this diverse list of output, we are able to

• Support all with abstractions where needed
• Provide a simple getting started experience that doesn't overwhelm developers

Agent response data types Copy Section

When comparing various agent SDKs and protocols, agent output is often divided into two categories:

1. Result: A response from the agent that communicates the result of the agent's work to the caller in natural language (or images/sound/etc.). Let's call this Primary output.
   1. Includes cases where the agent finished because it requires more input from the user.
2. Progress: Updates while the agent is running, which are informational only, typically showing what the agent is doing, and does not allow any actions to be taken by the caller that modify the behavior of the agent before completing the run. Let's call this Secondary output.

A potential third category is:

3. Long Running: A response that does not contain a Primary response or Secondary updates, but rather a reference to a long running task.

Different use cases for Primary and Secondary output Copy Section

To solve complex problems, many agents must be used together. These agents typically have their own capabilities and responsibilities and communicate via input messages and final responses/handoff calls, while the internal workings of each agent is not of interest to the other agents participating in solving the problem.

When an agent is in conversation with one or more humans, the information that may be displayed to the user(s) can vary. E.g. When an agent is part of a conversation with multiple humans it may be asked to perform tasks by the humans, and they may not want a stream of distracting updates posted to the conversation, but rather just a final response. On the other hand, if an agent is being used by a single human to perform a task, the human may be waiting for the agent to complete the task. Therefore, they may be interested in getting updates of what the agent is doing.

Where agents are nested, consumers would also likely want to constrain the amount of data from an agent that bubbles up into higher level conversations to avoid exceeding the context window, therefore limiting it to the Primary response only.

Comparison with other SDKs / Protocols Copy Section

Approaches observed from the compared SDKs:

1. Response object with separate properties for Primary and Secondary
2. Response stream that contains Primary and Secondary entries and callers need to filter.
3. Response containing just Primary.

Decision Drivers Copy Chapter

• Solutions provides an easy to use experience for users who are getting started and just want the answer to a question.
• Solution must be extensible to future requirements, e.g. long running agent processes.
• Experience is in line or better than the best in class experience from other SDKs

Response Type Options Copy Chapter

• Option 1 Run: Messages List contains mix of Primary and Secondary content, RunStreaming: Stream of Primary + Secondary
  • Option 1.1 Secondary content do not use TextContent
  • Option 1.2 Presence of Secondary Content is determined by a runtime parameter
  • Option 1.3 Use ChatClient response types
  • Option 1.4 Return derived ChatClient response types
• Option 2 Run: Container with Primary and Secondary Properties, RunStreaming: Stream of Primary + Secondary
  • Option 2.1 Response types extend MEAI types
  • Option 2.2 New Response types
• Option 3 Run: Primary-only, RunStreaming: Stream of Primary + Secondary
• Option 4 Remove Run API and retain RunStreaming API only, which returns a Stream of Primary + Secondary.

Since the suggested options vary only for the non-streaming case, the following detailed explanations for each
focuses on the non-streaming case.

Option 1 Run: Messages List contains mix of Primary and Secondary content, RunStreaming: Stream of Primary + Secondary Copy Section

Run returns a Task<ChatResponse> and RunStreaming returns a IAsyncEnumerable<ChatResponseUpdate>.
For Run, the returned ChatResponse.Messages contains an ordered list of messages that contain both the Primary and Secondary content.

ChatResponse.Text automatically aggregates all text from any TextContent items in all ChatMessage items in the response.
If we can ensure that no updates ever contain TextContent, this will mean that ChatResponse.Text will always contain
the Primary response text. See option 1.1.
If we cannot ensure this, either the solution or usage becomes more complex, see 1.3 and 1.4.

Option 1.1 `TextContent`, `DataContent` and `UriContent` means Primary content

ChatResponse.Text aggregates all TextContent values, and no secondary updates use TextContent
so ChatResponse.Text will always contain the Primary content.

// Since the Text property contains the primary content, it's a simple getting started experience.
var response = await agent.RunAsync("Do Something");
Console.WriteLine(response.Text);

// Callers can still get access to all updates too.
foreach (var update in response.Messages)
{
    Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name);
}

// For streaming, it's possible to output the primary content by also using the Text property on each update.
await foreach (var update in agent.RunStreamingAsync("Do Something"))
{
    Console.Writeline(update.Text)
}

• PROS: Easy and familiar user experience, reuse response types from IChatClient. Similar experience for both streaming and non streaming.
• CONS: The agent response types cannot evolve separately from MEAI if needed.

Option 1.1a `TextContent`, `DataContent` and `UriContent` means Primary content, with custom Agent response types

Same as 1.1 but with custom Agent Framework response types.
The response types should preferably resemble ChatResponse types closely, to ensure user's have a fimilar experience when moving between the two.
Therefore something like AgentResponse.Text which also aggregates all TextContent values similar to 1.1 makes sense.

• PROS: Easy getting started experience, and response types can be customized for the Agent Framework where needed.
• CONS: More work to define custom response types.

Option 1.2 Presence of Secondary Content is determined by a runtime parameter

We can allow callers to choose whether to include secondary content in the list of response messages.
Open Question: Do we allow secondary content to use TextContent types?

// By default the response only has the primary content, so text
// contains the primary content, and it's a good starting experience.
var response = await agent.RunAsync("Do Something");
Console.WriteLine(response.Text);

// we can also optionally include updates via an option.
var response = await agent.RunAsync("Do Something", options: new() { IncludeUpdates = true });
// Callers can now access all updates.
foreach (var update in response.Messages)
{
    Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name);
}

• PROS: Easy getting started experience, reuse response types from IChatClient.
• CONS: Since the basic experience is the same as 1.1, and when you look at individual messages, you most likely want all anyway, it seems arbitrarily limiting compared to 1.1.

Option 2 Run: Container with Primary and Secondary Properties, RunStreaming: Stream of Primary + Secondary Copy Section

Run returns a new response type that has separate properties for the Primary Content and the Secondary Updates leading up to it.
The Primary content is available in the AgentResponse.Messages property while Secondary updates are in a new AgentResponse.Updates property.
AgentResponse.Text returns the Primary content text.

Since streaming would still need to return an IAsyncEnumerable of updates, the design would differ from non-streaming.
With non-streaming Primary and Secondary content is split into separate lists, while with streaming it's combined in one stream.

// Since text contains the primary content, it's a good getting started experience.
var response = await agent.RunAsync("Do Something");
Console.WriteLine(response.Text);

// Callers can still get access to all updates too.
foreach (var update in response.Updates)
{
    Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name);
}

• PROS: Primary content and Secondary Updates are categorised for non-streaming and therefore easy to distinguish and this design matches popular SDKs like AutoGen and OpenAI SDK.
• CONS: Requires custom response types and design would differ between streaming and non-streaming.

Option 3 Run: Primary-only, RunStreaming: Stream of Primary + Secondary Copy Section

Run returns a Task<ChatResponse> and RunStreaming returns a IAsyncEnumerable<ChatResponseUpdate>.
For Run, the returned ChatResponse.Messages contains only the Primary content messages.
ChatResponse.Text will contain the aggregate text of ChatResponse.Messages and therefore the primary content messages text.

// Since text contains the primary content response, it's a good getting started experience.
var response = await agent.RunAsync("Do Something");
Console.WriteLine(response.Text);

// Callers cannot get access to all updates, since only the primary content is in messages.
var primaryContentOnly = response.Messages.FirstOrDefault();

• PROS: Simple getting started experience, Reusing IChatClient response types.
• CONS: Intermediate updates are only availble in streaming mode.

Option 4: Remove Run API and retain RunStreaming API only, which returns a Stream of Primary + Secondary Copy Section

With this option, we remove the RunAsync method and only retain the RunStreamingAsync method, but
we add helpers to process the streaming responses and extract information from it.

// User can get the primary content through an extension method on the async enumerable stream.
var responses = agent.RunStreamingAsync("Do Something");
// E.g. an extension method that builds the primary content text.
Console.WriteLine(await responses.AggregateFinalResult());
// Or an extention method that builds complete messages from the updates.
Console.WriteLine(await responses.BuildMessage().Text);

// Callers can also iterate through all updates if needed
await foreach (var update in responses)
{
    Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name);
}

• PROS: Single API for streaming/non-streaming
• CONS: More complex to for inexperienced users.

Custom Response Type Design Options Copy Chapter

Option 1 Response types extend MEAI types Copy Section

class Agent
{
    public abstract Task<AgentResponse> RunAsync(
        IReadOnlyCollection<ChatMessage> messages,
        AgentThread? thread = null,
        AgentRunOptions? options = null,
        CancellationToken cancellationToken = default);

    public abstract IAsyncEnumerable<AgentResponseUpdate> RunStreamingAsync(
        IReadOnlyCollection<ChatMessage> messages,
        AgentThread? thread = null,
        AgentRunOptions? options = null,
        CancellationToken cancellationToken = default);
}

class AgentResponse : ChatResponse
{
}

public class AgentResponseUpdate : ChatResponseUpdate
{
}

• PROS: Fimilar response types for anyone already using MEAI.
• CONS: Agent response types cannot evolve separately.

Option 2 New Response types Copy Section

We could create new response types for Agents.
The new types could also exclude properties that make less sense for agents, like ConversationId, which is abstracted away by AgentThread, or ModelId, where an agent might use multiple models.

/* Detailed source-code truncated for AI context efficiency. */

• PROS: Agent response types can evolve separately. Types can still resemble MEAI response types to ensure a fimilar experience for developers.
• CONS: No automatic inheritence of new properties from MEAI. (this might also be a pro)

Long Running Processes Options Copy Chapter

Some agent protocols, like A2A, support long running agentic processes. When invoking the agent
in the non-streaming case, the agent may respond with an id of a process that was launched.

The caller is then expected to poll the service to get status updates using the id.
The caller may also subscribe to updates from the process using the id.

We therefore need to be able to support providing this type of response to agent callers.

• Option 1 Add a new AIContent type and ChatFinishReason for long running processes.
• Option 2 Add another property on a custom response type.

Option 1: Add another AIContent type and ChatFinishReason for long running processes Copy Section

public class AgentRunContent : AIContent
{
    public string AgentRunId { get; set; }
}

// Add a new long running chat finish reason.
public class ChatFinishReason
{
    public static ChatFinishReason LongRunning { get; } = new ChatFinishReason("long_running");
}

• PROS: Fits well into existing ChatResponse design.
• CONS: More complex for users to extract the required long running result (can be mitigated with extenion methods)

Option 2: Add another property on responses for AgentRun Copy Section

class AgentResponse
{
    ...
    public AgentRun RunReference { get; set; } // Reference to long running process
    ...
}


public class AgentResponseUpdate
{
    ...
    public AgentRun RunReference { get; set; } // Reference to long running process
    ...
}

// Add a new long running chat finish reason.
public class ChatFinishReason
{
    ...
    public static ChatFinishReason LongRunning { get; } = new ChatFinishReason("long_running");
    ...
}

// Can be added in future: Class representing long running processing by the agent
// that can be used to check for updates and status of the processing.
public class AgentRun
{
    public string AgentRunId { get; set; }
}

• PROS: Easy access to long running result values
• CONS: Requires custom response types.

Structured user input options (Work in progress) Copy Chapter

Some agent services may ask end users a question while also providing a list of options that the user can pick from or a template for the input required.
We need to decide whether to maintain an abstraction for these, so that similar types of structured input from different agents can be used by callers without
needing to break out of the abstraction.

Tool result options (Work in progress) Copy Chapter

We need to consider abstractions for AIContent derived types for tool call results for common tool types beyond Function calls, e.g. CodeInterpreter, WebSearch, etc.

StructuredOutputs Copy Chapter

Structured outputs is a valueable aspect of any Agent system, since it forces an Agent to produce output in a required format, and may include required fields. This allows turning unstructured data into structured data easily using a general purpose language model.

Not all agent types necessarily support this or necessarily support this in the same way.
Requesting a specific output schema at invocation time is widely supported by inference services though, and therefore inference based agents would support this well.
Custom agents on the other hand may not necessarily want to support this, and forcing all custom Agent implementations to have a final structured output step to produce this complicates implementations.
Custom agents may also have a built in output schema, that they always produce.

Options:

1. Support configuring the preferred structured output schema at agent construction time for those agents that support structured outputs.
2. Support configuring the preferred structured output schema at invocation time, and ignore/throw if not supported (similar to IChatClient)
3. Support both options with the invocation time schema overriding the construction time (or built in) schema if both are supported.

Note that where an agent doesn't support structured output, it may also be possible to use a decorator to produce structured output from the agent's unstructured response, thereby turning an agent that doesn't support this into one that does.

See Structured Outputs Support for a comparison on what other agent frameworks and protocols support.

To support a good user experience for structured outputs, I'm proposing that we follow the pattern used by MEAI.
We would add a generic version of AgentResponse<T>, that allows us to get the agent result already deserialized into our preferred type.
This would be coupled with generic overload extension methods for Run that automatically builds a schema from the supplied type and updates
the run options.

If we support requesting a schema at invocation time the following would be the preferred approach:

class Movie
{
    public string Title { get; set; }
    public string DirectorFullName { get; set; }
    public int ReleaseYear { get; set; }
}

AgentResponse<Movie[]> response = agent.RunAsync<Movie[]>("What are the top 3 children's movies of the 80s.");
Movie[] movies = response.Result

If we only support requesting a schema at agent creation time or where an agent has a built in schema, the following would be the preferred approach:

AgentResponse response = agent.RunAsync("What are the top 3 children's movies of the 80s.");
Movie[] movies = response.TryParseStructuredOutput<Movie[]>();

Decision Outcome Copy Chapter

Response Type Options Decision Copy Section

Option 1.1 with the caveate that we cannot control the output of all agents. However, as far as possible we should have appropriate AIContext derived types for
progress updates so that TextContent is not used for these.

Custom Response Type Design Options Decision Copy Section

Option 2 chosen so that we can vary Agent responses independently of Chat Client.

StructuredOutputs Decision Copy Section

We will not support structured output per run request, but individual agents are free to allow this on the concrete implementation or at construction time.
We will however add support for easily extracting a structured output type from the AgentResponse.

Addendum 1: AIContext Derived Types for different response types / Gap Analysis (Work in progress) Copy Chapter

We need to decide what AIContent types, each agent response type will be mapped to.

Addendum 2: Other SDK feature comparison Copy Chapter

Structured Outputs Support Copy Section

1. Configure Schema on Agent at Agent construction
2. Pass schema at Agent invocation

Response Reason Support Copy Section

File: docs/decisions/0002-agent-tools.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: {proposed} contact: {dmytrostruk} date: {2025-06-23} deciders: {stephentoub, markwallace-microsoft, RogerBarreto, westey-m} consulted: {} informed: {} Copy Chapter

Agent Tools

Context and Problem Statement Copy Chapter

AI agents increasingly rely on diverse tools like function calling, file search, and computer use, but integrating each tool often requires custom, inconsistent implementations. A unified abstraction for tool usage is essential to simplify development, ensure consistency, and enable scalable, reliable agent performance across varied tasks.

Decision Drivers Copy Chapter

• The abstraction must provide a consistent API for all tools to reduce complexity and improve developer experience.
• The design should allow seamless integration of new tools without significant changes to existing implementations.
• Robust mechanisms for managing tool-specific errors and timeouts are required for reliability.
• The abstraction should support a fallback approach to directly use unsupported or custom tools, bypassing standard abstractions when necessary.

Considered Options Copy Chapter

Option 1: Use ChatOptions.RawRepresentationFactory for Provider-Specific Tools Copy Section

Description

Utilize the existing ChatOptions.RawRepresentationFactory to inject provider-specific tools (e.g., for an AI provider like Foundry) without extending the AITool abstract class from Microsoft.Extensions.AI.

ChatOptions options = new()
{
    RawRepresentationFactory = _ => new ResponseCreationOptions()
    {
        Tools = { ... }, // backend-specific tools
    },
};

Pros

• No development work needed; leverages existing Microsoft.Extensions.AI functionality.
• Flexible for integrating tools from any AI provider without modifying the AITool.
• Minimal codebase changes, reducing the risk of introducing errors.

Cons

• Requires a separate mechanism to register tools, complicating the developer experience.
• Developers must know the specific AI provider (via IChatClient) to configure tools, reducing abstraction.
• Inconsistent with the AITool abstraction, leading to fragmented tool usage patterns.
• Poor tool discoverability, as they are not integrated into the AITool ecosystem.

Option 2: Add Provider-Specific AITool-Derived Types in Provider Packages Copy Section

Description

Create provider-specific tool types that inherit from the AITool abstract class within each AI provider’s package (e.g., a Foundry package could include Foundry-specific tools). The provider’s IChatClient implementation would natively recognize and process these AITool-derived types, eliminating the need for a separate registration mechanism.

Pros

• Integrates with the AITool abstract class, providing a consistent developer experience within the Microsoft.Extensions.AI.
• Eliminates the need for a special registration mechanism like RawRepresentationFactory.
• Enhances type safety and discoverability for provider-specific tools.
• Aligns with the standardized interface driver by leveraging AITool as the base class.

Cons

• Developers must know they are targeting a specific AI provider to select the appropriate AITool-derived types.
• Increases maintenance overhead for each provider’s package to support and update these tool types.
• Leads to fragmentation, as each provider requires its own set of AITool-derived types.
• Potential for duplication if multiple providers implement similar tools with different AITool derivatives.

Option 3: Create Generic AITool-Derived Abstractions in M.E.AI.Abstractions Copy Section

Description

Develop generic tool abstractions that inherit from the AITool abstract class in the M.E.AI.Abstractions package (e.g., HostedCodeInterpreterTool, HostedWebSearchTool). These abstractions map to common tool concepts across multiple AI providers, with provider-specific implementations handled internally.

Pros

• Provides a standardized AITool-based interface across AI providers, improving consistency and developer experience.
• Reduces the need for provider-specific knowledge by abstracting tool implementations.
• Highly extensible, supporting new AITool-derived types for common tool concepts (e.g., server-side MCP tools).

Cons

• Complex mapping logic needed to support diverse provider implementations.
• May not cover niche or provider-specific tools, necessitating a fallback mechanism.

Option 4: Hybrid Approach Combining Options 1, 2, and 3 Copy Section

Description

Implement a hybrid strategy where common tools use generic AITool-derived abstractions in M.E.AI.Abstractions (Option 3), provider-specific tools (e.g., for Foundry) are implemented as AITool-derived types in their respective provider packages (Option 2), and rare or unsupported tools fall back to ChatOptions.RawRepresentationFactory (Option 1).

Pros

• Balances developer experience and flexibility by using the best AITool-based approach for each tool type.
• Supports standardized AITool interfaces for common tools while allowing provider-specific and breakglass mechanisms.
• Extensible and scalable, accommodating both current and future tool requirements across AI providers.
• Addresses ancillary and intermediate content (e.g., MCP permissions) with generic types.

Cons

• Increases complexity by managing multiple AITool integration approaches within the same system.
• Requires clear documentation to guide developers on when to use each option.
• Potential for inconsistency if boundaries between approaches are not well-defined.
• Higher maintenance burden to support and test multiple tool integration paths.

More information Copy Chapter

AI Agent Tool Types Availability Copy Section

API Comparison Copy Section

Function Calling

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/function-calling?pivots=rest

Message Request:

{
  "tools": [
    { 
      "type": "function",
      "function": {
        "description": "{string}",
        "name": "{string}",
        "parameters": "{JSON Schema object}"
      }
    }
  ]
}

Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "function",
      "function": {
        "name": "{string}",
        "arguments": "{JSON object}",
      }
    }
  ]
}

OpenAI Assistant API
Source: https://platform.openai.com/docs/assistants/tools/function-calling

Message Request:

{
  "tools": [
    { 
      "type": "function",
      "function": {
        "description": "{string}",
        "name": "{string}",
        "parameters": "{JSON Schema object}"
      }
    }
  ]
}

Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "function",
      "function": {
        "name": "{string}",
        "arguments": "{JSON object}",
      }
    }
  ]
}

OpenAI ChatCompletion API
Source: https://platform.openai.com/docs/guides/function-calling?api-mode=chat

Message Request:

{
  "tools": [
    { 
      "type": "function",
      "function": {
        "description": "{string}",
        "name": "{string}",
        "parameters": "{JSON Schema object}"
      }
    }
  ]
}

Tool Call Response:

[
  {
    "id": "{string}",
    "type": "function",
    "function": {
      "name": "{string}",
      "arguments": "{JSON object}",
    }
  }
]

OpenAI Responses API
Source: https://platform.openai.com/docs/guides/function-calling?api-mode=responses

Message Request:

{
  "tools": [
    { 
      "type": "function",
      "description": "{string}",
      "name": "{string}",
      "parameters": "{JSON Schema object}"
    }
  ]
}

Tool Call Response:

[
  {
    "id": "{string}",
    "call_id": "{string}",
    "type": "function_call",
    "name": "{string}",
    "arguments": "{JSON object}"
  }
]

Amazon Bedrock Agents
Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax

CreateAgentActionGroup Request:

{
  "functionSchema": {
    "name": "{string}",
    "description": "{string}",
    "parameters": {
      "type": "{string | number | integer | boolean | array}",
      "description": "{string}",
      "required": "{boolean}"
    }
  }
}

Tool Call Response:

{
  "invocationInputs": [
    {
      "functionInvocationInput": {
        "actionGroup": "{string}",
        "function": "{string}",
        "parameters": [
          {
            "name": "{string}",
            "type": "{string | number | integer | boolean | array}",
            "value": {}
          }
        ]
      }
    }
  ]
}

Google
Source: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#rest

Message Request:

{
  "tools": [
    {
      "functionDeclarations": [
        {
          "name": "{string}",
          "description": "{string}",
          "parameters": "{JSON Schema object}"
        }
      ]
    }
  ]
}

Tool Call Response:

{
  "content": {
    "role": "model",
    "parts": [
      {
        "functionCall": {
          "name": "{string}",
          "args": {
            "{argument_name}": {}
          }
        }
      }
    ]
  }
}

Anthropic
Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview

Message Request:

{
  "tools": [
    {
      "name": "{string}",
      "description": "{string}",
      "input_schema": "{JSON Schema object}"
    }
  ]
}

Tool Call Response:

{
  "id": "{string}",
  "model": "{string}",
  "stop_reason": "tool_use",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "{string}"
    },
    {
      "type": "tool_use",
      "id": "{string}",
      "name": "{string}",
      "input": {
        "argument_name": {}
      }
    }
  ]
}

Commonalities

• Standardized Tool Definition: All providers use a JSON-based structure for defining tools, including a type field (commonly "function") and a function object with name, description, and parameters (often following JSON Schema).
• Tool Call Response Structure: Responses typically include a list of tool calls with an id, type, and details about the function called (e.g., name and arguments), enabling consistent handling of function invocations.
• JSON Schema for Parameters: Parameters for functions are defined using JSON Schema objects across most providers, facilitating a unified approach to parameter validation and processing.
• Extensibility: The structure allows for additional metadata or fields (e.g., call_id, actionGroup), suggesting potential for abstraction to support provider-specific extensions while maintaining core compatibility.

Code Interpreter

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter-samples?pivots=rest-api

.NET Support: ✅

Message Request:

{
  "tools": [
    { 
      "type": "code_interpreter"
    }
  ],
  "tool_resources": {
    "code_interpreter": {
      "file_ids": ["{string}"],
      "data_sources": [
        {
          "type": {
            "id_asset": "{string}",
            "uri_asset": "{string}"
          },
          "uri": "{string}"
        }
      ]
    }
  }
}

Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "code_interpreter",
      "code_interpreter": {
        "input": "{string}",
        "outputs": [
          {
            "type": "image",
            "file_id": "{string}"
          },
          {
            "type": "logs",
            "logs": "{string}"
          }
        ]
      }
    }
  ]
}

OpenAI Assistant API
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter-samples?pivots=rest-api

.NET Support: ✅

Message Request:

{
  "tools": [
    { 
      "type": "code_interpreter"
    }
  ],
  "tool_resources": {
    "code_interpreter": {
      "file_ids": ["{string}"]
    }
  }
}

Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "code",
      "code": {
        "input": "{string}",
        "outputs": [
          {
            "type": "logs",
            "logs": "{string}"
          }
        ]
      }
    }
  ]
}

OpenAI Responses API
Source: https://platform.openai.com/docs/guides/tools-code-interpreter

.NET Support: ❌ (currently in development: GitHub issue)

Message Request:

{
  "tools": [
    { 
      "type": "code_interpreter",
      "container": { "type": "auto" }
    }
  ]
}

Tool Call Response:

[
  {
    "id": "{string}",
    "code": "{string}",
    "type": "code_interpreter_call",
    "status": "{string}",
    "container_id": "{string}",
    "results": [
      {
        "type": "logs",
        "logs": "{string}"
      },
      {
        "type": "files",
        "files": [
          {
            "file_id": "{string}",
            "mime_type": "{string}"
          }
        ]
      }
    ]
  }
]

Amazon Bedrock Agents
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-enable-code-interpretation.html

.NET Support: ❌ (Amazon SDK has IChatClient implementation but lacks ChatOptions.RawRepresentationFactory)

CreateAgentActionGroup Request:

{
  "actionGroupName": "{string}",
  "parentActionGroupSignature": "AMAZON.CodeInterpreter",
  "actionGroupState": "ENABLED"
}

Tool Call Response:

{
  "trace": {
    "orchestrationTrace": {
      "invocationInput": {
        "invocationType": "ACTION_GROUP_CODE_INTERPRETER",
        "codeInterpreterInvocationInput": {
          "code": "{string}",
          "files": ["{string}"]
        }
      },
      "observation": {
        "codeInterpreterInvocationOutput": {
          "executionError": "{string}",
          "executionOutput": "{string}",
          "executionTimeout": "{boolean}",
          "files": ["{string}"],
          "metadata": {
            "clientRequestId": "{string}",
            "endTime": "{timestamp}",
            "operationTotalTimeMs": "{long}",
            "startTime": "{timestamp}",
            "totalTimeMs": "{long}",
            "usage": {
              "inputTokens": "{integer}",
              "outputTokens": "{integer}"
            }
          }
        }
      }
    }
  }
}

Google
Source: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/code-execution#googlegenaisdk_tools_code_exec_with_txt-drest

.NET Support: ❌ (official SDK lacks IChatClient implementation.)

Message Request:

{
  "contents": {
    "role": "{string}",
    "parts": { 
      "text": "{string}" 
    }
  },
  "tools": [
    {
      "codeExecution": {}
    }
  ]
}

Tool Call Response:

{
  "content": {
    "role": "model",
    "parts": [
      {
        "executableCode": {
          "language": "{string}",
          "code": "{string}"
        }
      },
      {
        "codeExecutionResult": {
          "outcome": "{string}",
          "output": "{string}"
        }
      }
    ]
  }
}

Anthropic
Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/code-execution-tool

.NET Support: ❌

<li>[Anthropic.SDK](https://github.com/tghamm/Anthropic.SDK) - uses `code_interpreter` instead of `code_execution` and lacks a possibility to specify file id.</li>
<li>[Anthropic by tryAGI](https://github.com/tryAGI/Anthropic) - has `code_execution` implementation, but it's in beta and can't be used as a tool.</li>

Message Request:

{
  "tools": [
    {
      "name": "code_execution",
      "type": "code_execution_20250522"
    }
  ]
}

Tool Call Response:

{
  "role": "assistant",
  "container": {
    "id": "{string}",
    "expires_at": "{timestamp}"
  },
  "content": [
    {
      "type": "server_tool_use",
      "id": "{string}",
      "name": "code_execution",
      "input": {
        "code": "{string}"
      }
    },
    {
      "type": "code_execution_tool_result",
      "tool_use_id": "{string}",
      "content": {
        "type": "code_execution_result",
        "stdout": "{string}",
        "stderr": "{string}",
        "return_code": "{integer}"
      }
    }
  ]
}

Commonalities

• Tool Type Specification: Providers consistently define a code_interpreter tool type within the tools array, indicating support for code execution capabilities.
• Input and Output Handling: Requests include mechanisms to specify code input (e.g., input or code fields), and responses return execution outputs, such as logs or files, in a structured format.
• File Resource Support: Most providers allow associating files with the code interpreter (e.g., via file_ids or files), enabling data input/output for code execution.
• Execution Metadata: Responses often include metadata about the execution process (e.g., status, logs, or executionError), which can be abstracted for standardized error handling and result processing.

Search and Retrieval

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/file-search-upload-files?pivots=rest

File Search Request:

{
  "tools": [
    { 
      "type": "file_search"
    }
  ],
  "tool_resources": { 
    "file_search": {
      "vector_store_ids": ["{string}"],
      "vector_stores": [
        {
          "name": "{string}",
          "configuration": {
            "data_sources": [
              {
                "type": {
                  "id_asset": "{string}",
                  "uri_asset": "{string}"
                },
                "uri": "{string}"
              }
            ]
          }
        }
      ]
    }
  }
}

File Search Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "file_search",
      "file_search": {
        "ranking_options": {
          "ranker": "{string}",
          "score_threshold": "{float}"
        },
        "results": [
          {
            "file_id": "{string}",
            "file_name": "{string}",
            "score": "{float}",
            "content": [
              {
                "text": "{string}",
                "type": "{string}"
              }
            ]
          }
        ]
      }
    }
  ]
}

Azure AI Search Request:

{
  "tools": [
    { 
      "type": "azure_ai_search"
    }
  ],
  "tool_resources": { 
    "azure_ai_search": {
      "indexes": [
        {
          "index_connection_id": "{string}",
          "index_name": "{string}",
          "query_type": "{string}"
        }
      ]
    }
  }
}

Azure AI Search Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "azure_ai_search",
      "azure_ai_search": {} // From documentation: Reserved for future use
    }
  ]
}

OpenAI Assistant API
Source: https://platform.openai.com/docs/assistants/tools/file-search

Message Request:

{
  "tools": [
    { 
      "type": "file_search"
    }
  ],
  "tool_resources": { 
    "file_search": {
      "vector_store_ids": ["string"]
    }
  }
}

Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "file_search",
      "file_search": {
        "ranking_options": {
          "ranker": "{string}",
          "score_threshold": "{float}"
        },
        "results": [
          {
            "file_id": "{string}",
            "file_name": "{string}",
            "score": "{float}",
            "content": [
              {
                "text": "{string}",
                "type": "{string}"
              }
            ]
          }
        ]
      }
    }
  ]
}

OpenAI Responses API
Source: https://platform.openai.com/docs/api-reference/responses/create

Message Request:

{
  "tools": [
    { 
      "type": "file_search"
    }
  ],
  "tool_resources": { 
    "file_search": {
      "vector_store_ids": ["string"]
    }
  }
}

Tool Call Response:

{
  "output": [
    {
      "id": "{string}",
      "queries": ["{string}"],
      "status": "{in_progress | searching | incomplete | failed | completed}",
      "type": "file_search_call",
      "results": [
        {
          "attributes": {},
          "file_id": "{string}",
          "filename": "{string}",
          "score": "{float}",
          "text": "{string}"
        }
      ]
    }
  ]
}

Amazon Bedrock Agents
Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeAgent.html

Message Request:

{
  "sessionState": {
    "knowledgeBaseConfigurations": [
      { 
          "knowledgeBaseId": "{string}",
          "retrievalConfiguration": { 
             "vectorSearchConfiguration": { 
                "filter": {},
                "implicitFilterConfiguration": { 
                   "metadataAttributes": [ 
                      { 
                         "description": "{string}",
                         "key": "{string}",
                         "type": "{string}"
                      }
                   ],
                   "modelArn": "{string}"
                },
                "numberOfResults": "{number}",
                "overrideSearchType": "{string}",
                "rerankingConfiguration": { 
                   "bedrockRerankingConfiguration": { 
                      "metadataConfiguration": { 
                         "selectionMode": "{string}",
                         "selectiveModeConfiguration": {}
                      },
                      "modelConfiguration": { 
                         "additionalModelRequestFields": { 
                            "string" : "{JSON string}"
                         },
                         "modelArn": "{string}"
                      },
                      "numberOfRerankedResults": "{number}"
                   },
                   "type": "{string}"
                }
             }
          }
       }
    ]
  }
}

Tool Call Response:

{
  "trace": {
    "orchestrationTrace": {
      "invocationInput": {
        "invocationType": "KNOWLEDGE_BASE",
        "knowledgeBaseLookupInput": {
          "knowledgeBaseId": "{string}",
          "text": "{string}"
        }
      },
      "observation": {
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseLookupOutput": {
          "retrievedReferences": [
            {
              "metadata": {},
              "content": {
                "byteContent": "{string}",
                "row": [
                  {
                    "columnName": "{string}",
                    "columnValue": "{string}",
                    "type": "{BLOB | BOOLEAN | DOUBLE | NULL | LONG | STRING}"
                  }
                ],
                "text": "{string}",
                "type": "{TEXT | IMAGE | ROW}"
              }
            }
          ],
          "metadata": {
            "clientRequestId": "{string}",
            "endTime": "{timestamp}",
            "operationTotalTimeMs": "{long}",
            "startTime": "{timestamp}",
            "totalTimeMs": "{long}",
            "usage": {
              "inputTokens": "{integer}",
              "outputTokens": "{integer}"
            }
          }
        }
      }
    }
  }
}

Google
Source: https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-vertex-ai-search

Message Request:

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "text": "{string}"
        }
      ]
    }
  ],
  "tools": [
    {
      "retrieval": {
        "vertexAiSearch": {
          "datastore": "{string}"
        }
      }
    }
  ]
}

Tool Call Response:

{
  "content": {
    "role": "model",
    "parts": [
      {
        "text": "{string}"
      }
    ]
  },
  "groundingMetadata": {
      "retrievalQueries": [
        "{string}"
      ],
      "groundingChunks": [
        {
          "retrievedContext": {
            "uri": "{string}",
            "title": "{string}"
          }
        }
      ],
      "groundingSupport": [
        {
          "segment": {
            "startIndex": "{number}",
            "endIndex": "{number}"
          },
          "segment_text": "{string}",
          "supportChunkIndices": ["{number}"],
          "confidenceScore": ["{number}"]
        }
      ]
    }
}

Commonalities

• Vector Store Integration: Providers like Azure and OpenAI use vector_store_ids or similar constructs to reference vector stores for file search, suggesting a common approach to retrieval-augmented generation.
• Search Configuration: Requests include configurations for search (e.g., vectorSearchConfiguration, ranking_options), allowing customization of retrieval parameters like result count or ranking.
• Result Structure: Responses contain a list of search results with fields like file_id, score, and content or text, enabling consistent processing of retrieved data.
• Metadata Inclusion: Search responses often include metadata (e.g., score, timestamp, usage), which can be abstracted for unified analytics and performance tracking.

Web Search

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/bing-code-samples?pivots=rest

Bing Search Message Request:

{
  "tools": [
    { 
      "type": "bing_grounding",
      "bing_grounding": {
        "search_configurations": [
          {
            "connection_id": "{string}",
            "count": "{number}", 
            "market": "{string}", 
            "set_lang": "{string}", 
            "freshness": "{string}",
          }
        ]
      }
    }
  ]
}

Bing Search Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "function",
      "bing_grounding": {} // From documentation: Reserved for future use
    }
  ]
}

OpenAI ChatCompletion API
Source: https://platform.openai.com/docs/guides/tools-web-search?api-mode=chat

Message Request:

{
  "web_search_options": {},
  "messages": [
    {
      "role": "user",
      "content": "{string}"
    }
  ]
}

Tool Call Response:

[
  {
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "{string}",
      "annotations": [
        {
          "type": "url_citation",
          "url_citation": {
            "end_index": "{number}",
            "start_index": "{number}",
            "title": "{string}",
            "url": "{string}"
          }
        }
      ]
    }
  }
]

OpenAI Responses API
Source: https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses

Message Request:

{
  "tools": [
    {
      "type": "web_search_preview"
    }
  ],
  "input": "{string}"
}

Tool Call Response:

{
  "output": [
    {
      "type": "web_search_call",
      "id": "{string}",
      "status": "{string}"
    },
    {
      "id": "{string}",
      "type": "message",
      "status": "{string}",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "{string}",
          "annotations": [
            {
              "type": "url_citation",
              "start_index": "{number}",
              "end_index": "{string}",
              "url": "{string}",
              "title": "{string}"
            }
          ]
        }
      ]
    }
  ]
}

Google
Source: https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-google-search

Message Request:

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "text": "{string}"
        }
      ]
    }
  ],
  "tools": [
    {
      "googleSearch": {}
    }
  ]
}

Tool Call Response:

{
  "content": {
    "role": "model",
    "parts": [
      {
        "text": "{string}"
      }
    ]
  },
  "groundingMetadata": {
    "webSearchQueries": [
      "{string}"
    ],
    "searchEntryPoint": {
      "renderedContent": "{string}"
    },
    "groundingChunks": [
      {
        "web": {
          "uri": "{string}",
          "title": "{string}",
          "domain": "{string}"
        }
      }
    ],
    "groundingSupports": [
      {
        "segment": {
          "startIndex": "{number}",
          "endIndex": "{number}",
          "text": "{string}"
        },
        "groundingChunkIndices": [
          "{number}"
        ],
        "confidenceScores": [
          "{number}"
        ]
      }
    ],
    "retrievalMetadata": {}
  }
}

Anthropic
Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/web-search-tool

Message Request:

{
  "tools": [
    {
      "name": "web_search",
      "type": "web_search_20250305",
      "max_uses": "{number}",
      "allowed_domains": ["{string}"],
      "blocked_domains": ["{string}"],
      "user_location": {
        "type": "approximate",
        "city": "{string}",
        "region": "{string}",
        "country": "{string}",
        "timezone": "{string}"
      }
    }
  ]
}

Tool Call Response:

{
  "role": "assistant",
  "content": [
    {
      "type": "server_tool_use",
      "id": "{string}",
      "name": "web_search",
      "input": {
        "query": "{string}"
      }
    },
    {
      "type": "web_search_tool_result",
      "tool_use_id": "{string}",
      "content": [
        {
          "type": "web_search_result",
          "url": "{string}",
          "title": "{string}",
          "encrypted_content": "{string}",
          "page_age": "{string}"
        }
      ]
    },
    {
      "text": "{string}",
      "type": "text",
      "citations": [
        {
          "type": "web_search_result_location",
          "url": "{string}",
          "title": "{string}",
          "encrypted_index": "{string}",
          "cited_text": "{string}"
        }
      ]
    }
  ]
}

Commonalities

• Tool-Based Activation: Providers define web search as a tool (e.g., web_search, bing_grounding, googleSearch), typically within a tools array, allowing standardized activation of search capabilities.
• Query Input: Requests support passing a search query (e.g., via input, content, or query), enabling a unified interface for initiating searches.
• Result Annotations: Responses include search results with metadata like url, title, and sometimes confidenceScores or citations, which can be abstracted for consistent result presentation.
• Grounding Metadata: Most providers include grounding metadata (e.g., groundingMetadata, annotations), facilitating traceability and validation of search results.

Remote MCP Servers

OpenAI Responses API
Source: https://platform.openai.com/docs/guides/tools-remote-mcp

Message Request:

{
  "tools": [
    { 
      "type": "mcp",
      "server_label": "{string}",
      "server_url": "{string}",
      "require_approval": "{string}"
    }
  ]
}

Tool Call Response:

{
  "output": [
    {
      "id": "{string}",
      "type": "mcp_list_tools",
      "server_label": "{string}",
      "tools": [
        {
          "name": "{string}",
          "input_schema": "{JSON Schema object}"
        }
      ]
    },
    {
      "id": "{string}",
      "type": "mcp_call",
      "approval_request_id": "{string}",
      "arguments": "{JSON string}",
      "error": "{string}",
      "name": "{string}",
      "output": "{string}",
      "server_label": "{string}"
    }
  ]
}

Google
Source: https://google.github.io/adk-docs/tools/mcp-tools/#using-mcp-tools-in-your-own-agent-out-of-adk-web

async def get_agent_async():
toolset = MCPToolset(
    tool_filter=['read_file', 'list_directory'] # Optional: filter specific tools
    connection_params=SseServerParams(url="http://remote-server:port/path", headers={...})
)

# Use in an agent
root_agent = LlmAgent(
    model='model', # Adjust model name if needed based on availability
    name='agent_name',
    instruction='agent_instructions',
    tools=[toolset], # Provide the MCP tools to the ADK agent
)
return root_agent, toolset

Anthropic
Source: https://docs.anthropic.com/en/docs/agents-and-tools/mcp-connector

Message Request:

{
  "messages": [
    {
      "role": "user", 
      "content": "{string}"
    }
  ],
  "mcp_servers": [
    {
      "type": "url",
      "url": "{string}",
      "name": "{string}",
      "tool_configuration": {
        "enabled": true,
        "allowed_tools": ["{string}"]
      },
      "authorization_token": "{string}"
    }
  ]
}

Tool Use Response:

{
  "type": "mcp_tool_use",
  "id": "{string}",
  "name": "{string}",
  "server_name": "{string}",
  "input": { "param1": "{object}", "param2": "{object}" }
}

Tool Result Response:

{
  "type": "mcp_tool_result",
  "tool_use_id": "{string}",
  "is_error": "{boolean}",
  "content": [
    {
      "type": "text",
      "text": "{string}"
    }
  ]
}

Commonalities

• Server Configuration: Providers specify remote servers via URL and metadata (e.g., server_url, url, name), enabling a standardized way to connect to external MCP services.
• Tool Integration: MCP tools are integrated into the tools or mcp_servers array, allowing agents to interact with remote tools in a consistent manner.
• Input/Output Structure: Requests and responses include structured input (e.g., input, arguments) and output (e.g., output, content), supporting abstraction for tool execution workflows.
• Authorization Support: Most providers include mechanisms for authentication (e.g., authorization_token, headers), which can be abstracted for secure communication with remote servers.

Computer Use

OpenAI Responses API
Source: https://platform.openai.com/docs/guides/tools-computer-use

Message Request:

{
  "tools": [
    {
      "type": "computer_use_preview",
      "display_width": "{number}",
      "display_height": "{number}",
      "environment": "{browser | mac | windows | ubuntu}"
    }
  ]
}

Tool Call Response:

{
  "output": [
    {
      "type": "reasoning",
      "id": "{string}",
      "summary": [
        {
          "type": "summary_text",
          "text": "{string}"
        }
      ]
    },
    {
      "type": "computer_call",
      "id": "{string}",
      "call_id": "{string}",
      "action": {
        "type": "{click | double_click | drag | keypress | move | screenshot | scroll | type | wait}",
        // Other properties are associated with specific action type.
      },
      "pending_safety_checks": [],
      "status": "{in_progress | completed | incomplete}"
    }
  ]
}

Amazon Bedrock Agents
Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/agent-computer-use-handle-tools.html

CreateAgentActionGroup Request:

{
  "actionGroupName": "{string}",
  "parentActionGroupSignature": "ANTHROPIC.Computer",
  "actionGroupState": "ENABLED"
}

Tool Call Response:

{
  "returnControl": {
    "invocationId": "{string}",
    "invocationInputs": [
      {
        "functionInvocationInput": {
          "actionGroup": "{string}",
          "actionInvocationType": "RESULT",
          "agentId": "{string}",
          "function": "{string}",
          "parameters": [
            {
              "name": "{string}",
              "type": "string",
              "value": "{string}"
            }
          ]
        }
      }
    ]
  }
}

Anthropic
Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/computer-use-tool

Message Request:

{
  "tools": [
    {
      "type": "computer_20250124",
      "name": "computer",
      "display_width_px": "{number}",
      "display_height_px": "{number}",
      "display_number": "{number}"
    },
  ]
}

Tool Call Response:

{
  "role": "assistant",
  "content": [
    {
      "type": "tool_use",
      "id": "{string}",
      "name": "{string}",
      "input": "{object}"
    }
  ]
}

Commonalities

• Tool Type Definition: Providers define a computer use tool (e.g., computer_use_preview, computer_20250124, ANTHROPIC.Computer) within the tools array, indicating support for computer interaction capabilities.
• Action Specification: Responses include actions (e.g., click, keypress, type) with associated parameters, enabling standardized interaction with computer environments.
• Environment Configuration: Requests allow specifying the environment (e.g., browser, windows, display_width), which can be abstracted for cross-platform compatibility.
• Status Tracking: Responses include status indicators (e.g., status, pending_safety_checks), facilitating consistent monitoring of computer use tasks.

OpenAPI Spec Tool

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/openapi-spec-samples?pivots=rest-api
Source: https://learn.microsoft.com/en-us/rest/api/aifoundry/aiagents/run-steps/get-run-step?view=rest-aifoundry-aiagents-v1&tabs=HTTP#runstepopenapitoolcall

Message Request:

{
  "tools": [
    { 
      "type": "openapi",
      "openapi": {
        "description": "{string}",
        "name": "{string}",
        "auth": {
          "type": "{string}"
        },
        "spec": "{OpenAPI specification object}"
      }
    }
  ]
}

Tool Call Response:

{
  "tool_calls": [
    {
      "id": "{string}",
      "type": "openapi",
      "openapi": {} // From documentation: Reserved for future use
    }
  ]
}

Amazon Bedrock Agents
Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax

CreateAgentActionGroup Request:

{
  "apiSchema": {
    "payload": "{JSON or YAML OpenAPI specification string}"
  }
}

Tool Call Response:

{
  "invocationInputs": [
    {
      "apiInvocationInput": {
        "actionGroup": "{string}",
        "apiPath": "{string}",
        "httpMethod": "{string}",
        "parameters": [
          {
            "name": "{string}",
            "type": "{string}",
            "value": "{string}"
          }
        ]
      }
    }
  ]
}

Commonalities

• OpenAPI Specification: Both providers support defining tools using OpenAPI specifications, either as a JSON/YAML payload or a structured spec object, enabling standardized API integration.
• Tool Type Identification: The tool is identified as openapi or via an apiSchema, providing a clear entry point for OpenAPI-based tool usage.
• Parameter Handling: Responses include parameters (e.g., parameters, apiPath, httpMethod) for API invocation, which can be abstracted for unified API call execution.

Stateful Functions

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/azure-functions-samples?pivots=rest

Message Request:

{
  "tools": [
    { 
      "type": "azure_function",
      "azure_function": {
        "function": {
          "name": "{string}",
          "description": "{string}",
          "parameters": "{JSON Schema object}"
        },
        "input_binding": {
          "type": "storage_queue",
          "storage_queue": {
            "queue_service_endpoint": "{string}",
            "queue_name": "{string}"
          }
        },
        "output_binding": {
          "type": "storage_queue",
          "storage_queue": {
            "queue_service_endpoint": "{string}",
            "queue_name": "{string}"
          }
        }
      }
    }
  ]
}

Tool Call Response: Not specified in the documentation.

Amazon Bedrock Agents
Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax

CreateAgentActionGroup Request:

{
  "apiSchema": {
    "payload": "{JSON or YAML OpenAPI specification string}"
  }
}

Tool Call Response:

{
  "invocationInputs": [
    {
      "apiInvocationInput": {
        "actionGroup": "{string}",
        "apiPath": "{string}",
        "httpMethod": "{string}",
        "parameters": [
          {
              "name": "{string}",
              "type": "{string}",
              "value": "{string}"
          }
        ]
      }
    }
  ]
}

Commonalities

• API-Driven Interaction: Both providers use API-based structures (e.g., apiSchema, azure_function) to define stateful functions, enabling integration with external services.
• Parameter Specification: Requests include parameter definitions (e.g., parameters, JSON Schema object), supporting standardized input handling.

Text Editor

Anthropic
Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/text-editor-tool

Message Request:

{
  "tools": [
    {
      "type": "text_editor_20250429",
      "name": "str_replace_based_edit_tool"
    }
  ]
}

Tool Call Response:

{
  "role": "assistant",
  "content": [
    {
      "type": "tool_use",
      "id": "{string}",
      "name": "str_replace_based_edit_tool",
      "input": {
        "command": "{string}",
        "path": "{string}"
      }
    }
  ]
}

Microsoft Fabric

Microsoft Foundry Agent Service
Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/fabric?pivots=rest

Message Request:

{
  "tools": [
    { 
      "type": "fabric_dataagent",
      "fabric_dataagent": {
        "connections": [
          {
            "connection_id": "{string}"
          }
        ]
      }
    }
  ]
}

Tool Call Response: Not specified in the documentation.

Image Generation

OpenAI Responses API
Source: https://platform.openai.com/docs/guides/tools-image-generation

Message Request:

{
  "tools": [
    {
      "type": "image_generation"
    }
  ]
}

Tool Call Response:

{
  "output": [
    {
      "type": "image_generation_call",
      "id": "{string}",
      "result": "{Base64 string}",
      "status": "{string}"
    }
  ]
}

Decision Outcome Copy Chapter

TBD.

File: docs/decisions/0003-agent-opentelemetry-instrumentation.md Copy Chapter

status: proposed contact: rogerbarreto date: 2025-07-14 deciders: stephentoub, markwallace-microsoft, rogerbarreto, westey-m informed: {} Copy Chapter

Agent OpenTelemetry Instrumentation

Context and Problem Statement Copy Chapter

Currently, the Agent Framework lacks comprehensive observability and telemetry capabilities, making it difficult for developers to monitor agent performance, track usage patterns, debug issues, and gain insights into agent behavior in production environments. While the underlying ChatClient implementations may have their own telemetry, there is no standardized way to capture agent-specific metrics and traces that provide visibility into agent operations, token usage, response times, and error patterns at the agent abstraction level.

Decision Drivers Copy Chapter

• Compliance: The implementation should adhere to established OpenTelemetry semantic conventions for agents, ensuring consistency and interoperability with existing telemetry systems.
• Observability Requirements: Developers need comprehensive telemetry to monitor agent performance, track usage patterns, and debug issues in production environments.
• Standardization: The solution must follow established OpenTelemetry semantic conventions and integrate seamlessly with existing .NET telemetry infrastructure.
• Microsoft.Extensions.AI Alignment: The implementation should follow the exact patterns and conventions established by Microsoft.Extensions.AI's OpenTelemetry instrumentation.
• Non-Intrusive Design: Telemetry should be optional and not impact the core agent functionality or performance when disabled.
• Agent-Level Insights: The telemetry should capture agent-specific operations without duplicating underlying ChatClient telemetry.
• Extensibility: The solution should support future enhancements and additional telemetry scenarios.

Considered Options Copy Chapter

Option 1: Direct Integration into Core Agent Classes Copy Section

Embed OpenTelemetry instrumentation directly into the base Agent class and ChatClientAgent implementations.

Pros

• Automatic telemetry for all agent implementations
• No additional wrapper classes needed
• Consistent telemetry across all agents

Cons

• Violates single responsibility principle
• Increases complexity of core agent classes
• Makes telemetry mandatory rather than optional
• Harder to test and maintain
• Couples telemetry concerns with business logic

Option 2: Aspect-Oriented Programming (AOP) Approach Copy Section

Use interceptors or AOP frameworks to inject telemetry behavior into agent methods.

Pros

• Clean separation of concerns
• Non-intrusive to existing code
• Can be applied selectively

Cons

• Adds complexity with AOP framework dependencies
• Runtime overhead for interception
• Harder to debug and understand
• Not consistent with Microsoft.Extensions.AI patterns

Option 3: OpenTelemetryAgent Wrapper Pattern Copy Section

Create a delegating OpenTelemetryAgent wrapper class that implements the Agent interface and wraps any existing agent with telemetry instrumentation, following the exact pattern of Microsoft.Extensions.AI's OpenTelemetryChatClient.

Pros

• Follows established Microsoft.Extensions.AI patterns exactly
• Clean separation of concerns
• Optional and non-intrusive
• Easy to test and maintain
• Consistent with .NET telemetry conventions
• Supports any agent implementation
• Provides agent-level telemetry without duplicating ChatClient telemetry

Cons

• Requires explicit wrapping of agents
• Additional object allocation for wrapper

Decision Outcome Copy Chapter

Chosen option: "OpenTelemetryAgent Wrapper Pattern", because it follows the established Microsoft.Extensions.AI patterns exactly, provides clean separation of concerns, maintains optional telemetry, and offers the best balance of functionality, maintainability, and consistency with existing .NET telemetry infrastructure.

Implementation Details Copy Section

The implementation includes:

1. OpenTelemetryAgent Wrapper Class: A delegating agent that wraps any Agent implementation with telemetry instrumentation
2. AgentOpenTelemetryConsts: Comprehensive constants for telemetry attribute names and metric definitions
3. Extension Methods: .WithOpenTelemetry() extension method for easy agent wrapping
4. Comprehensive Test Suite: Full test coverage following Microsoft.Extensions.AI testing patterns

Telemetry Data Captured Copy Section

Activities/Spans:

• agent.operation.name (agent.run, agent.run_streaming)
• agent.request.id, agent.request.name, agent.request.instructions
• agent.request.message_count, agent.request.thread_id
• agent.response.id, agent.response.message_count, agent.response.finish_reason
• agent.usage.input_tokens, agent.usage.output_tokens
• Error information and activity status codes

Metrics:

• Operation duration histogram with proper buckets
• Token usage histogram (input/output tokens)
• Request count counter
• All metrics tagged with operation type and agent name

Consequences Copy Section

• Good: Provides comprehensive agent-level observability following established patterns
• Good: Non-intrusive and optional implementation that doesn't affect core functionality
• Good: Consistent with Microsoft.Extensions.AI telemetry conventions
• Good: Easy to integrate with existing OpenTelemetry infrastructure
• Good: Supports debugging, monitoring, and performance analysis
• Neutral: Requires explicit wrapping of agents with .WithOpenTelemetry()
• Neutral: Additional object allocation for telemetry wrapper

Validation Copy Chapter

The implementation is validated through:

1. Comprehensive Unit Tests: 16 test methods covering all scenarios including success, error, streaming, and edge cases
2. Integration Testing: Step05 telemetry sample demonstrating real-world usage
3. Pattern Compliance: Exact adherence to Microsoft.Extensions.AI OpenTelemetry patterns
4. Semantic Convention Compliance: Follows OpenTelemetry semantic conventions for telemetry data

More Information Copy Chapter

Usage Example Copy Section

// Create TracerProvider
using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSource(AgentOpenTelemetryConsts.DefaultSourceName)
    .AddConsoleExporter()
    .Build();

// Create and wrap agent with telemetry
var baseAgent = new ChatClientAgent(chatClient, options);
using var telemetryAgent = baseAgent.WithOpenTelemetry();

// Use agent normally - telemetry is captured automatically
var response = await telemetryAgent.RunAsync(messages);

Relationship to Microsoft.Extensions.AI Copy Section

This implementation follows the exact patterns established by Microsoft.Extensions.AI's OpenTelemetry instrumentation, ensuring consistency across the AI ecosystem and leveraging proven patterns for telemetry integration.

File: docs/decisions/0004-foundry-sdk-extensions.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: proposed contact: markwallace-microsoft date: 2025-08-06 deciders: markwallace-microsoft, westey-m, quibitron, trrwilson consulted: informed: Copy Chapter

`Azure.AI.Agents.Persistent` package Extensions Methods for Agent Framework

Context and Problem Statement Copy Chapter

To align the Azure.AI.Agents.Persistent package and Agent Framework a set of extensions methods have been created which allow a developer to create or retrieve an AIAgent using the PersistentAgentsClient.
The purpose of this ADR is to decide where these extension methods should live.

Decision Drivers Copy Chapter

• Provide the optimum experience for developers.
• Avoid adding additional dependencies to the Azure.AI.Agents.Persistent package (and not in the future)

Considered Options Copy Chapter

• Add the extension methods to the Azure.AI.Agents.Persistent package and change it's dependencies
• Add the extension methods to the Azure.AI.Agents.Persistent package without changing it's dependencies
• Add the extension methods to a Microsoft.Extensions.AI.Azure package

Add the extension methods to the `Azure.AI.Agents.Persistent` package and change it's dependencies Copy Section

• Azure.AI.Agents.Persistent would depend on Microsoft.Extensions.AI instead of Microsoft.Extensions.AI.Abstractions

• Good because, extension methods are in the Azure.AI.Agents.Persistent package and can be easily kept up-to-date

• Good because, developers don't need to explicitly depend on a new package to get Agent Framework functionality

• Bad because, it introduces additional dependencies which would possibly grow overtime

- Add the extension methods to the `Azure.AI.Agents.Persistent` package without changing it's dependencies Copy Section

• Azure.AI.Agents.Persistent would depend on Microsoft.Extensions.AI.Abstractions (as it currently does)

• ChatClientAgent and FunctionInvokingChatClient would move to Microsoft.Extensions.AI.Abstractions

• Good because, extension methods are in the Azure.AI.Agents.Persistent package and can be easily kept up-to-date

• Good because, developers don't need to explicitly depend on a new package to get Agent Framework functionality

• Good because, it introduces minimal additional dependencies

• Bad because, it adds additional dependencies to Microsoft.Extensions.AI.Abstractions and these additional dependencies add up as transitive to Azure.AI.Agents.Persistent`

Add the extension methods to a `Microsoft.Extensions.AI.Azure` package Copy Section

• Introduce a new package called Microsoft.Extensions.AI.Azure where the extension methods would live

• Azure.AI.Agents.Persistent does not change

• Good because, it introduces no additional dependencies to Azure.AI.Agents.Persistent package

• Bad because, extension methods are not in the Azure.AI.Agents.Persistent package and cannot be easily kept up-to-date

• Bad because, developers need to explicitly depend on a new package to get Agent Framework functionality

Decision Outcome Copy Chapter

Chosen option: "Add the extension methods to a Microsoft.Extensions.AI.Azure package", because
it introduces no additional dependencies to Azure.AI.Agents.Persistent package.

File: docs/decisions/0005-python-naming-conventions.md Copy Chapter

status: accepted contact: eavanvalkenburg date: 2025-09-04 deciders: markwallace-microsoft, dmytrostruk, peterychang, ekzhu, sphenry consulted: taochenosu, alliscode, moonbox3, johanste Copy Chapter

Python naming conventions and renames (ADR)

Context and Problem Statement Copy Chapter

The project has a public .NET surface and a Python surface. During a cross-language alignment effort the community proposed renames to make the Python surface more idiomatic while preserving discoverability and mapping to the .NET names. This ADR captures the final naming decisions (or the proposed ones), the rationale, and the alternatives considered and rejected.

Decision drivers Copy Chapter

• Follow Python naming conventions (PEP 8) where appropriate (snake_case for functions and module-level variables, PascalCase for classes).
• Preserve conceptual parity with .NET names to make it easy for developers reading both surfaces to correlate types and behaviors.
• Avoid ambiguous or overloaded names in Python that could conflict with stdlib, common third-party packages, or existing package/module names.
• Prefer clarity and discoverability in the public API surface over strict symmetry with .NET when Python conventions conflict.
• Minimize churn and migration burden for existing Python users where backwards compatibility is feasible.

Principles applied Copy Chapter

• Map .NET PascalCase class names to PascalCase Python classes when they represent types.
• Map .NET method/field names that are camelCase to snake_case in Python where they will be used as functions or module-level attributes.
• When a .NET name is an acronym or initialism, use Python-friendly casing (e.g., Http -> HTTP in classes, but acronyms in function names should be lowercased per PEP 8 where sensible).
• Avoid names that shadow common stdlib modules (e.g., logging, asyncio) or widely used third-party modules.
• When multiple reasonable Python names exist, prefer the one that communicates intent most clearly to Python users, and record rejected alternatives in the table with justification.

Renaming table Copy Chapter

The table below represents the majority of the naming changes discussed in issue #506. Each row has:

• Original and/or .NET name — the canonical name used in dotnet or earlier Python variants.
• New name — the chosen Python name.
• Status — accepted if the new name differs from the original, rejected if unchanged.
• Reasoning — short rationale why the new name was chosen.
• Rejected alternatives — other candidate new names that were considered and rejected; include the rejected 'new name' values and the reason each was rejected.

Naming guidance Copy Chapter

In general Python tends to prefer shorter names, while .NET tends to prefer more descriptive names. The table above captures the specific renames agreed upon, but in general the following guidelines were applied:

• Use PEP 8 for generic naming conventions (snake_case for functions and module-level variables, PascalCase for classes).

When mapping .NET names to Python:

• Remove AI prefix when appropriate, as it is often redundant in the context of an AI SDK.
• Remove Chat prefix when the context is clear (e.g., Role and FinishReason).
• Use Protocol suffix for interfaces/protocols to clarify their purpose.
• Use Base prefix for base classes that are not abstract but serve as a common ancestor for internal implementations.
• When readability improves while it is still easy to understand what it does and how it maps to the .NET name, prefer the shorter name.

File: docs/decisions/0006-userapproval.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: westey-m date: 2025-09-12 deciders: sergeymenshykh, markwallace-microsoft, rogerbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub, peterychang consulted: informed: Copy Chapter

Agent User Approvals Content Types and FunctionCall approvals Design

Context and Problem Statement Copy Chapter

When agents are operating on behalf of a user, there may be cases where the agent requires user approval to continue an operation.
This is complicated by the fact that an agent may be remote and the user may not immediately be available to provide the approval.

Inference services are also increasingly supporting built-in tools or service side MCP invocation, which may require user approval before the tool can be invoked.

This document aims to provide options and capture the decision on how to model this user approval interaction with the agent caller.

See various features that would need to be supported via this type of mechanism, plus how various other frameworks support this:

• Also see dotnet issue 6492, which discusses the need for a similar pattern in the context of MCP approvals.
• Also see the openai human-in-the-loop guide.
• Also see the openai MCP guide.
• Also see MCP Approval Requests from OpenAI.
• Also see Microsoft Foundry MCP Approvals.
• Also see MCP Elicitation requests

Decision Drivers Copy Chapter

• Agents should encapsulate their internal logic and not leak it to the caller.
• We need to support approvals for local actions as well as remote actions.
• We need to support approvals for service-side tool use, such as remote MCP tool invocations
• We should consider how other user input requests will be modeled, so that we can have a consistent approach for user input requests and approvals.

Considered Options Copy Chapter

1. Return a FunctionCallContent to the agent caller, that it executes Copy Section

This introduces a manual function calling element to agents, where the caller of the agent is expected to invoke the function if the user approves it.

This approach is problematic for a number of reasons:

• This may not work for remote agents (e.g. via A2A), where the function that the agent wants to call does not reside on the caller's machine.
• The main value prop of an agent is to encapsulate the internal logic of the agent, but this leaks that logic to the caller, requiring the caller to know how to invoke the agent's function calls.
• Inference services are introducing their own approval content types for server side tool or function invocation, and will not be addressed by this approach.

2. Introduce an ApprovalCallback in AgentRunOptions and ChatOptions Copy Section

This approach allows a caller to provide a callback that the agent can invoke when it requires user approval.

This approach is easy to use when the user and agent are in the same application context, such as a desktop application, where the application can show the approval request to the user and get their response from the callback before continuing the agent run.

This approach does not work well for cases where the agent is hosted in a remote service, and where there is no user available to provide the approval in the same application context.
For cases like this, the agent needs to be suspended, and a network response must be sent to the client app. After the user provides their approval, the client app must call the service that hosts the agent again, with the user's decision, and the agent needs to be resumed. However, with a callback, the agent is deep in the call stack and cannot be suspended or resumed like this.

class AgentRunOptions
{
    public Func<ApprovalRequestContent, Task<ApprovalResponseContent>>? ApprovalCallback { get; set; }
}

agent.RunAsync("Please book me a flight for Friday to Paris.", thread, new AgentRunOptions
{
    ApprovalCallback = async (approvalRequest) =>
    {
        // Show the approval request to the user in the appropriate format.
        // The user can then approve or reject the request.
        // The optional FunctionCallContent can be used to show the user what function the agent wants to call with the parameter set:
        // approvalRequest.FunctionCall?.Arguments.

        // If the user approves:
        return true;
    }
});

3. Introduce new ApprovalRequestContent and ApprovalResponseContent types Copy Section

The agent would return an ApprovalRequestContent to the caller, which would then be responsible for getting approval from the user in whatever way is appropriate for the application.
The caller would then invoke the agent again with an ApprovalResponseContent to the agent containing the user decision.

When an agent returns an ApprovalRequestContent, the run is finished for the time being, and to continue, the agent must be invoked again with an ApprovalResponseContent on the same thread as the original request. This doesn't of course have to be the exact same thread object, but it should have the equivalent contents as the original thread, since the agent would have stored the ApprovalRequestContent in its thread state.

The ApprovalRequestContent could contain an optional FunctionCallContent if the approval is for a function call, along with any additional information that the agent wants to provide to the user to help them make a decision.

It is up to the agent to decide when and if a user approval is required, and therefore when to return an ApprovalRequestContent.

ApprovalRequestContent and ApprovalResponseContent will not necessarily always map to a supported content type for the underlying service or agent thread storage.
Specifically, when we are deciding in the IChatClient stack to ask for approval from the user, for a function call, this does not mean that the underlying ai service or
service side thread type (where applicable) supports the concept of a function call approval request. While we can store the approval requests and response in local
threads, service managed threads won't necessarily support this. For service managed threads, there will therefore be no long term record of the approval request in the chat history.
We should however log approvals so that there is a trace of this for debugging and auditing purposes.

Suggested Types:

/* Detailed source-code truncated for AI context efficiency. */

4. Introduce new Container UserInputRequestContent and UserInputResponseContent types Copy Section

This approach is similar to the ApprovalRequestContent and ApprovalResponseContent types, but is more generic and can be used for any type of user input request, not just approvals.

There is some ambiguity with this approach. When using an LLM based agent the LLM may return a text response about missing user input.
E.g the LLM may need to invoke a function but the user did not supply all necessary information to fill out all arguments.
Typically an LLM would just respond with a text message asking the user for the missing information.
In this case, the message is not distinguishable from any other result message, and therefore cannot be returned to the caller as a UserInputRequestContent, even though it is conceptually a type of unstructured user input request. Ultimately our types are modeled to make it easy for callers to decide on the right way to represent this to users. E.g. is it just a regular message to show to users, or do we need a special UX for it.

Suggested Types:

/* Detailed source-code truncated for AI context efficiency. */

5. Introduce new Base UserInputRequestContent and UserInputResponseContent types Copy Section

This approach is similar to option 4, but the UserInputRequestContent and UserInputResponseContent types are base classes rather than generic container types.

Suggested Types:

/* Detailed source-code truncated for AI context efficiency. */

Decision Outcome Copy Chapter

Chosen option 5.

Appendices Copy Chapter

ChatClientAgent Approval Process Flow Copy Section

1. User passes a User message to the agent with a request.
2. Agent calls IChatClient with any functions registered on the agent.
   (IChatClient has FunctionInvokingChatClient)
3. Model responds with FunctionCallContent indicating function calls required.
4. FunctionInvokingChatClient decorator identifies any function calls that require user approval and returns an FunctionApprovalRequestContent.
   (If there are multiple parallel function calls, all function calls will be returned as FunctionApprovalRequestContent even if only some require approval.)
5. Agent updates the thread with the FunctionApprovalRequestContent (or this may have already been done by a service threaded agent).
6. Agent returns the FunctionApprovalRequestContent to the caller which shows it to the user in the appropriate format.
7. User (via caller) invokes the agent again with FunctionApprovalResponseContent.
8. Agent adds the FunctionApprovalResponseContent to the thread.
9. Agent calls IChatClient with the provided FunctionApprovalResponseContent.
10. Agent invokes IChatClient with FunctionApprovalResponseContent and the FunctionInvokingChatClient decorator identifies the response as an approval for the function call.
    Any rejected approvals are converted to FunctionResultContent with a message indicating that the function invocation was denied.
    Any approved approvals are executed by the FunctionInvokingChatClient decorator.
11. FunctionInvokingChatClient decorator passes the FunctionCallContent and FunctionResultContent for the approved and rejected function calls to the model.
12. Model responds with the result.
13. FunctionInvokingChatClient returns the FunctionCallContent, FunctionResultContent, and the result message to the agent.
14. Agent responds to caller with the same messages and updates the thread with these as well.

CustomAgent Approval Process Flow Copy Section

1. User passes a User message to the agent with a request.
2. Agent adds this message to the thread.
3. Agent executes various steps.
4. Agent encounters a step for which it requires user input to continue.
5. Agent responds with an UserInputRequestContent and also adds it to its thread.
6. User (via caller) invokes the agent again with UserInputResponseContent.
7. Agent adds the UserInputResponseContent to the thread.
8. Agent responds to caller with result message and thread is updated with the result message.

Sequence Diagram: FunctionInvokingChatClient with built in Approval Generation Copy Section

This is a ChatClient Approval Stack option has been proven to work via a proof of concept implementation.

/* Detailed source-code truncated for AI context efficiency. */

Sequence Diagram: Post FunctionInvokingChatClient ApprovalGeneratingChatClient - Multiple function calls with partial approval Copy Section

This is a discarded ChatClient Approval Stack option, but is included here for reference.

/* Detailed source-code truncated for AI context efficiency. */

Sequence Diagram: Pre FunctionInvokingChatClient ApprovalGeneratingChatClient - Multiple function calls with partial approval Copy Section

This is a discarded ChatClient Approval Stack option, but is included here for reference.

It doesn't work for the scenario where we have multiple function calls for the same function in serial with different arguments.

Flow:

• AGCC turns AIFunctions into AIFunctionDefinitions (not invocable) and FICC ignores these.
• We get back a FunctionCall for one of these and it gets approved.
• We invoke the FICC again, this time with an AIFunction.
• We call the service with the FCC and FRC.
• We get back a new Function call for the same function again with different arguments.
• Since we were passed an AIFunction instead of an AIFunctionDefinition, we now incorrectly execute this FC without approval.

/* Detailed source-code truncated for AI context efficiency. */

File: docs/decisions/0007-agent-filtering-middleware.md Copy Chapter

status: proposed contact: rogerbarreto date: 2025-09-15 deciders: markwallace-microsoft, rogerbarreto, westey-m, dmytrostruk, sergeymenshykh informed: {} Copy Chapter

Agent Filtering Middleware Design

Context and Problem Statement Copy Chapter

The current Agent Framework lacks a standardized, extensible mechanism for intercepting and processing agent execution. Developers need the ability to add custom filters/middleware to intercept and modify agent behavior at various stages of the execution pipeline. While the framework has basic agent abstractions with RunAsync and RunStreamingAsync methods, and standards like approval workflows, there is no middleware that allows developers to intercept and modify agent behavior at different agent execution contexts.

The challenge is to design an architecture that supports:

• Multiple execution contexts (invocation, function calls, approval requests, error handling)
• Support for both streaming and non-streaming scenarios
• Dependency injection friendly setup

Decision Drivers Copy Chapter

• Agents should be able to intercept and modify agent behavior at various stages of the execution pipeline.
• The design should be simple and intuitive for developers to understand and use.
• The design should be extensible to support new execution contexts and scenarios.
• The design should support both manual and dependency injection configuration.
• The design should allow flexible custom behaviors provided by enough context information.
• The design should be exception friendly and allow clear error handling and recovery mechanisms.

Other AI Agent Framework Analysis Copy Chapter

This section provides an analysis of how other major AI agent frameworks handle filtering, middleware, hooks, or similar interception capabilities. The goal is to identify ubiquitous language, design patterns, and approaches that could inform our Agent Middleware design also providing valuable insights into achieving a more idiomatic designs.

Overview Comparison Table Copy Section

Considered Options Copy Chapter

Option 1: Semantic Kernel Approach Copy Section

Similar to the Semantic Kernel kernel filters this option involves exposing different interface and properties for each specialized filter.

/* Detailed source-code truncated for AI context efficiency. */

Pros

• Clean separation of concerns
• Follows established patterns in Semantic Kernel and easy migration path
• No resistance or complaints from the community when used in Semantic Kernel
• Composable and reusable filter components

Cons

• Adding more filters may require adding more properties to the agent class.
• Filters are not always used, and adding this responsibility to the AIAgent abstraction level, may be an overkill.

Option 2: Agent Filter Decorator Pattern Copy Section

Similar to the OpenTelemetryAgent and the DelegatingChatClient in Microsoft.Extensions.AI, this option involves creating decorator agents that wrap the inner agent and allow interception of method calls. The current POC implementation demonstrates two approaches:

2a. Direct Decorator Implementation (GuardrailCallbackAgent)

/* Detailed source-code truncated for AI context efficiency. */

2b. Context-Based Middleware (RunningCallbackHandlerAgent)

The POC also includes a context-based approach using RunningCallbackHandlerAgent that wraps the agent and provides a context object for middleware processing:

// Internal implementation that supports the .Use() pattern
internal sealed class RunningCallbackHandlerAgent : DelegatingAIAgent
{
    private readonly Func<AgentInvokeCallbackContext, Func<AgentInvokeCallbackContext, Task>, Task> _func;

    internal RunningCallbackHandlerAgent(AIAgent innerAgent, Func<AgentInvokeCallbackContext, Func<AgentInvokeCallbackContext, Task>, Task> func) : base(innerAgent)
    {
        this._func = func;
    }

    public override async Task<AgentResponse> RunAsync(IEnumerable<ChatMessage> messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default)
    {
        var context = new AgentInvokeCallbackContext(this, messages, thread, options, isStreaming: false, cancellationToken);

        async Task CoreLogicAsync(AgentInvokeCallbackContext ctx)
        {
            var response = await this.InnerAgent.RunAsync(ctx.Messages, ctx.Thread, ctx.Options, ctx.CancellationToken);
            ctx.SetRawResponse(response);
        }

        await this._func(context, CoreLogicAsync);

        return context.RunResponse!;
    }
}

2c. Function Invocation Filtering

The POC also demonstrates function invocation filtering using a similar decorator pattern:

// Function invocation middleware using .Use() pattern
var agent = persistentAgentsClient.CreateAIAgent(model)
    .AsBuilder()
    .Use((functionInvocationContext, next, ct) =>
    {
        Console.WriteLine($"IsStreaming: {functionInvocationContext!.IsStreaming}");
        return next(functionInvocationContext.Arguments, ct);
    })
    .Use((functionInvocationContext, next, ct) =>
    {
        Console.WriteLine($"City Name: {(functionInvocationContext!.Arguments.TryGetValue("location", out var location) ? location : "not provided")}");
        return next(functionInvocationContext.Arguments, ct);
    })
    .Build();

This demonstrates that the current POC supports both agent-level and function-level filtering through consistent patterns.

Pros

• Clean separation of concerns
• Follows established patterns in Microsoft.Extensions.AI (DelegatingChatClient, OpenTelemetryAgent)
• Non-intrusive to existing agent implementations
• Supports both manual and DI configuration through builder pattern
• Context-specific processing middleware with AgentInvokeCallbackContext
• Composable and reusable filter components
• Flexible implementation allowing both direct decorators and context-based middleware
• Seamless integration with builder pattern using .Use() method
• Support for both streaming and non-streaming scenarios
• Rich context object providing access to messages, thread, options, and response handling

Option 3: Dedicated Processor Component for Middleware Copy Section

This approach involves creating a dedicated CallbackMiddlewareProcessor that manages collections of ICallbackMiddleware instances. The current POC implementation demonstrates this pattern with the CallbackEnabledAgent and processor architecture.

Current POC Implementation

/* Detailed source-code truncated for AI context efficiency. */

Function Invocation Filtering

The POC also demonstrates function invocation filtering using the processor pattern:

// Processor-based function invocation middleware
var agent = persistentAgentsClient.CreateAIAgent(model)
    .AsBuilder()
    .UseCallbacks(config =>
    {
        config.AddCallback(new UsedApiFunctionInvocationCallback());
        config.AddCallback(new CityInformationFunctionInvocationCallback());
    }).Build();

internal sealed class UsedApiFunctionInvocationCallback : CallbackMiddleware<AgentFunctionInvocationCallbackContext>
{
    public override async Task OnProcessAsync(AgentFunctionInvocationCallbackContext context, Func<AgentFunctionInvocationCallbackContext, Task> next, CancellationToken cancellationToken)
    {
        Console.WriteLine($"IsStreaming: {context!.IsStreaming}");

        await next(context);
    }
}

internal sealed class CityInformationFunctionInvocationCallback : CallbackMiddleware<AgentFunctionInvocationCallbackContext>
{
    public override async Task OnProcessAsync(AgentFunctionInvocationCallbackContext context, Func<AgentFunctionInvocationCallbackContext, Task> next, CancellationToken cancellationToken)
    {
        Console.WriteLine($"City Name: {(context!.Arguments.TryGetValue("location", out var location) ? location : "not provided")}");
        await next(context);
    }
}

This demonstrates that the current POC supports both agent-level and function-level filtering through consistent patterns.

Processor Implementation

The CallbackMiddlewareProcessor manages the filter pipeline and chain execution:

public sealed class CallbackMiddlewareProcessor
{
    // For thread-safety when used as a Singleton
    private readonly ConcurrentBag _agentCallbacks = [];

    public CallbackMiddlewareProcessor(IEnumerable? callbacks = null)
    {
        if (callbacks is not null)
        {
            foreach (var callback in callbacks)
            {
                AddCallback(callback);
            }
        }
    }

    internal CallbackMiddlewareProcessor AddCallback(ICallbackMiddleware middleware)
    {
        switch (middleware)
        {
            case CallbackMiddleware<AgentInvokeCallbackContext>:
                this._agentCallbacks.Add(middleware);
                break;
            default:
                throw new ArgumentException($"The middleware type '{middleware.GetType().FullName}' is not supported.", nameof(middleware));
        }

        return this;
    }

    public async Task ProcessAsync<TContext>(TContext context, Func<TContext, Task> coreLogic, CancellationToken cancellationToken = default)
        where TContext : CallbackContext
    {
        var applicableCallbacks = this.GetApplicableCallbacks<TContext>().ToList();
        await this.InvokeChainAsync(context, applicableCallbacks, 0, coreLogic, cancellationToken);
    }

    private IEnumerable GetApplicableCallbacks<TContext>()
        where TContext : CallbackContext
    {
        return this._agentCallbacks.Where(callback => callback.CanProcess<TContext>());
    }
}

CallbackEnabledAgent Implementation

public sealed class CallbackEnabledAgent : DelegatingAIAgent
{
    private readonly CallbackMiddlewareProcessor _callbacksProcessor;

    public CallbackEnabledAgent(AIAgent agent, CallbackMiddlewareProcessor? callbackMiddlewareProcessor) : base(agent)
    {
        this._callbacksProcessor = callbackMiddlewareProcessor ?? new();
    }

    public override async Task<AgentResponse> RunAsync(
        IEnumerable<ChatMessage> messages,
        AgentThread? thread = null,
        AgentRunOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        AgentInvokeCallbackContext roamingContext = null!;

        async Task CoreLogic(AgentInvokeCallbackContext ctx)
        {
            roamingContext ??= ctx;
            var result = await this.InnerAgent.RunAsync(ctx.Messages, ctx.Thread, ctx.Options, ctx.CancellationToken);

            ctx.SetRawResponse(result);
        }

        await this._callbacksProcessor.ProcessAsync<AgentInvokeCallbackContext>(
            new AgentInvokeCallbackContext(
                agent: this,
                messages: messages,
                thread,
                options,
                isStreaming: false,
                cancellationToken),
            CoreLogic,
            cancellationToken);

        return roamingContext.RunResponse!;
    }
}

Pros

• Flexibility: Use shared processor for multiple agents or create per-agent instances
• Clean fluent configuration API with .UseCallbacks() builder method
• Type-safe middleware registration with CallbackMiddleware<TContext> base class
• Thread-safe processor implementation using ConcurrentBag
• Extensible context system with AgentInvokeCallbackContext providing rich execution context
• Seamless integration with existing agent builder pattern
• Support for both streaming and non-streaming scenarios in middleware
• Clear separation between middleware logic and agent core functionality
• Simplicity: Agents stay lean, middleware is externalized to processor
• Extensibility: Add new contexts/filters without changing agent implementation

Cons

• Additional complexity with processor class and context management
• Requires understanding of middleware lifecycle and context passing
• Type switching in processor for different middleware types
• Roaming context pattern needed to capture specialized contexts through middleware chain

APPENDIX 1: Proposed Middleware Contexts Copy Chapter

The following context classes would be needed to support the filtering architecture:

public abstract class AgentContext
{
    // For scenarios where the filter is processed by multiple agents sounds very desirable to provide access to the invoking agent
    public AIAgent Agent { get; } 

    public AgentRunOptions? Options { get; set; } // Options are allowed to be set by filters

    protected AgentContext(AIAgent agent, AgentRunOptions? options)
    {
        Agent = agent;
        Options = options;
    }
}

public class AgentRunContext : AgentContext
{
    public IList<ChatMessage> Messages { get; set; }
    public AgentResponse? Response { get; set; }
    public AgentThread? Thread { get; }

    public AgentRunContext(AIAgent agent, IList<ChatMessage> messages, AgentThread? thread, AgentRunOptions? options)
        : base(agent, options)
    {
        Messages = messages;
        Thread = thread;
    }
}

public class AgentFunctionInvocationContext : AgentToolContext
{
    // Similar to MEAI.FunctionInvocationContext
    public AIFunction Function { get; set; }
    public AIFunctionArguments Arguments { get; set; }
    public FunctionCallContent CallContent { get; set; }
    public IList<ChatMessage> Messages { get; set; }
    public ChatOptions? Options { get; set; }
    public int Iteration { get; set; }
    public int FunctionCallIndex { get; set; }
    public int FunctionCount { get; set; }
    public bool Terminate { get; set; }
    public bool IsStreaming { get; set; }
}

APPENDIX 2: Setting Up Middleware Options Copy Chapter

1. Semantic Kernel Setup Copy Section

Has the benefit of clear separation of concerns, but this approach requires developers
to manage and maintain separate collections for each filter type, increasing code complexity and maintenance overhead.

// Use Case
var agent = new MyAgent();
agent.RunFilters.Add(new MyAgentRunFilter());
agent.RunFilters.Add(new MyMultipleFilterImplementation());
agent.FunctionCallFilters.Add(new MyAgentFunctionCallFilter());
agent.FunctionCallFilters.Add(new MyMultipleFilterImplementation());
agent.AYZFilters.Add(new MyAgentAYZFilter());
agent.AYZFilters.Add(new MyMultipleFilterImplementation());



// Impl
interface IAgentRunFilter
{
    Task OnRunAsync(AgentRunContext context, Func<AgentRunContext, Task> next, CancellationToken cancellationToken = default);
}
interface IAgentFunctionCallFilter
{
    Task OnFunctionCallAsync(AgentFunctionCallContext context, Func<AgentFunctionCallContext, Task> next, CancellationToken cancellationToken = default);
}

Pros

• Clean separation of concerns
• Follows established patterns in Semantic Kernel and easy migration path
• No resistance or complaints from the community when used in Semantic Kernel

Cons

• Adding more filters may require adding more properties to the agent/processor class.
• Adding more filters requires bigger code changes downstream to callers.

2. Setup with Generic Method Copy Section

Instead of properties, exposing as a method may be more appropriate while still maintaining those filters in separate buckets internally.

// Use Case
var agent = new MyAgent();
agent.AddFilters<RunFilter>([new MyAgentRunFilter(), new MyMultipleFilterImplementation()]);
agent.AddFilters<FunctionCallFilter>([new MyAgentFunctionCallFilter(), new MyMultipleFilterImplementation()]);
agent.AddFilters<AYZFilter>([new MyAgentAYZFilter(), new MyMultipleFilterImplementation()]);

Pros

• Clean separation of concerns
• Cleaner API for adding filters compared to option 1
• No resistance or complaints from the community when used in Semantic Kernel

Cons

• Adding more filters may require adding more properties to the agent/processor class.
• Adding more filters requires bigger code changes downstream to callers.

3. Setup with Filter Hierarchy, Fully Generic Setup Copy Section

In a more generic approach, filters can be grouped in the same bucket and processed based on the context.
One generic interface for all filters, with context-specific implementations.
Allow simple grouping of filters in the same list and adding new filter types with low code-changes.

/* Detailed source-code truncated for AI context efficiency. */

Pros

• Simple grouping of filters in the same list, help with DI registration and filtering iteration
• Lower maintenance and learning curve when adding new filter types
• Can be combined with other patterns like the AgentFilterProcessor

Cons

• Less clear separation of concerns compared to dedicated filter types
• Requires extra runtime type checking and casting for context-specific processing

Decision Outcome Copy Chapter

• Option 2 (Decorator Pattern) is the preferred approach for the following reasons:
  • Adding a processor pattern seems an overkill as we can achieve same results without introducing new abstractions and complexity.
  • Direct decorator on agents and tools for agent and function invocation middleware.
  • Support for Context-based middleware also leveraging closer patterns to Semantic Kernel filters.
  • Agent Builder pattern integration with .Use() method for fluent configuration

Key POC Insights:

1. Both patterns actually work
2. The decorator pattern offers more direct control and simpler and more flexible implementation
3. The processor seems an overkill compared to decorator as it adds more extra abstractions and complexity
4. Function invocation filtering is supported in both patterns
5. Streaming scenarios are well-supported in both approaches
6. Function approval request filtering is supported in both patterns
7. Builder pattern added as part of the POC is a must-have and mades both approaches developer-friendly

Appendix: Other AI Agent Framework Analysis Details Copy Chapter

LangChain

LangChain uses callbacks for interception, which can be passed at runtime or during construction.

Naming (Python): Callbacks (BaseCallbackHandler)
Supports: Y (read/write)
Observation: Uses observer pattern with event methods for interception (e.g., on_chain_start); supports agent actions and errors; handlers can read inputs/outputs and modify metadata or raise exceptions to influence flow.

Python Example: For more details, see the official documentation: Callbacks - Python LangChain.

from langchain_core.callbacks import BaseCallbackHandler

class MyHandler(BaseCallbackHandler):
    def on_chain_start(self, serialized, inputs, **kwargs):
        inputs['number'] += 1  # Modify inputs (write capability)
        print("Chain started!")

handler = MyHandler()

# Pass callback at runtime
chain.invoke({"number": 25}, {"callbacks": [handler]})

# Or at constructor time
chain = SomeChain(callbacks=[handler])
chain.invoke({"number": 25})

Naming (JS): Callbacks (BaseCallbackHandler)
Supports: Y (read/write)
Observation: Similar observer pattern to Python, with event methods adapted for JS async handling; supports chain/agent interception; handlers can read inputs/outputs and modify metadata or raise exceptions to influence flow.

JS Example: For more details, see the official documentation: Callbacks - LangChain.js. (Adapted for async handling in JS.)

import { BaseCallbackHandler } from "@langchain/core/callbacks/base";

class MyHandler extends BaseCallbackHandler {
  name = "my_handler";

  async handleChainStart(chain, inputs) {
    inputs.number += 1;  # Modify inputs (write capability)
    console.log("Chain started!");
  }
}

const handler = new MyHandler();

// Pass callback at runtime
await chain.invoke({ number: 25 }, { callbacks: [handler] });

// Or at constructor time
const chainWithHandler = new SomeChain({ callbacks: [handler] });
await chainWithHandler.invoke({ number: 25 });

LangGraph

LangGraph inherits callbacks from LangChain and often uses them with handlers for observability (e.g., via Langfuse).

Naming (Python): Hooks/Callbacks (inherited from LangChain)
Supports: Y (read/write)
Observation: Event-driven with runtime handlers; integrates callbacks for observability in graphs; inherits LangChain's ability to read/modify metadata or interrupt execution.

For more details, see the official documentation (inherited from LangChain): Callbacks - Python LangChain. Here's an example of streaming with a callback handler (Python):

from langfuse.langchain import CallbackHandler
from langchain_core.messages import HumanMessage

class MyLangfuseHandler(CallbackHandler):
    def on_chain_start(self, serialized, inputs, **kwargs):
        inputs['messages'][0].content += " modified"  # Modify input messages (write capability)
        super().on_chain_start(serialized, inputs, **kwargs)

langfuse_handler = MyLangfuseHandler()

# Stream with callback in config
for s in graph.stream(
    {"messages": [HumanMessage(content="What is Langfuse?")]},
    config={"callbacks": [langfuse_handler]}
):
    print(s)

AutoGen

AutoGen supports middleware-like behavior in both languages.

Naming (Python): Reply Functions (register_reply)
Supports: Y (read/write)
Observation: Reply functions intercept and process messages; middleware-like for agent replies; can directly modify messages or replies before continuing.

Python Example: For more details, see the official documentation: agentchat.conversable_agent | AutoGen 0.2. Uses register_reply to add reply functions that intercept and process messages.

def print_messages(recipient, messages, sender, config): 
    if "callback" in config and config["callback"] is not None:
        callback = config["callback"]
        callback(sender, recipient, messages[-1])
    messages[-1]["content"] += " modified"  # Modify last message content (write capability)
    print(f"Messages sent to: {recipient.name} | num messages: {len(messages)}")
    return False, None  # required to ensure the agent communication flow continues

user_proxy.register_reply(
    [autogen.Agent, None],
    reply_func=print_messages, 
    config={"callback": None},
)

assistant.register_reply(
    [autogen.Agent, None],
    reply_func=print_messages, 
    config={"callback": None},
)

Naming (C#): Middleware (MiddlewareAgent)
Supports: Y (read/write)
Observation: Decorator/wrapper with middleware delegates for message modification; delegates can read and alter message content or options.

C# Example: For more details, see the official documentation: Use middleware in an agent - AutoGen for .NET. Registers middleware to modify messages.

// Register middleware to modify messages
var middlewareAgent = new MiddlewareAgent(innerAgent: agent);
middlewareAgent.Use(async (messages, options, agent, ct) =>
{
    if (messages.Last() is TextMessage lastMessage && lastMessage.Content.Contains("Hello World"))
    {
        lastMessage.Content = $"[middleware] {lastMessage.Content}";  # Modify message content (write capability)
        return lastMessage;
    }
    return await agent.GenerateReplyAsync(messages, options, ct);
});

Semantic Kernel

Semantic Kernel uses filters added to the kernel for interception during function invocation, prompt rendering, etc. Implementations differ by language: C# use interfaces, while Python uses functions and decorators.

Naming (C#): Filters (IFunctionInvocationFilter, etc.)
Supports: Y (read/write)
Observation: Interface-based middleware for function/prompt interception; filters can read and modify context, arguments, or results.

C# Example: For more details, see the official documentation: Semantic Kernel Filters | Microsoft Learn. Adding a function invocation filter using interfaces.

using Microsoft.SemanticKernel;

IKernelBuilder builder = Kernel.CreateBuilder();
builder.Services.AddSingleton();

Kernel kernel = builder.Build();

// Alternatively, add directly
kernel.FunctionInvocationFilters.Add(new LoggingFilter(logger));

// Define the filter
public sealed class LoggingFilter(ILogger logger) : IFunctionInvocationFilter
{
    public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
    {
        context.Arguments["new_arg"] = "modified_value";  # Modify arguments by adding a new key (write capability)
        logger.LogInformation("Invoking {FunctionName}", context.Function.Name);
        await next(context);
        logger.LogInformation("Invoked {FunctionName}", context.Function.Name);
    }
}

Naming (Python): Filters (add_filter, @kernel.filter decorator)
Supports: Y (read/write)
Observation: Function and decorator-based for interception; no explicit interfaces like C#, focuses on async functions for filters; can read and modify context/arguments/results.

Python Example: For more details, see the official documentation: Semantic Kernel Filters | Microsoft Learn. Adding function invocation filters (one as a standalone function and one via decorator).

/* Detailed source-code truncated for AI context efficiency. */

CrewAI

CrewAI uses event listeners for callbacks.

Naming (Python): Events/Callbacks (BaseEventListener)
Supports: Y (read)
Observation: Event-driven orchestration with listeners for workflows; listeners can observe events (e.g., read source/event data) but are primarily for logging/reactions without direct modification of workflow state.

For more details, see the official documentation: Event Listeners - CrewAI Documentation. Here's an example of setting up a custom listener (Python):

from crewai.utilities.events import (
    CrewKickoffStartedEvent,
    BaseEventListener,
    crewai_event_bus
)

class MyCustomListener(BaseEventListener):
    def setup_listeners(self, crewai_event_bus):
        @crewai_event_bus.on(CrewKickoffStartedEvent)
        def on_crew_started(source, event):
            print(f"Crew '{event.crew_name}' started!")

my_listener = MyCustomListener()  # Automatically registers on init

# Use in a crew
crew = Crew(agents=[...], tasks=[...])

LlamaIndex

LlamaIndex uses callback managers with handlers.

Naming (Python): Callbacks (CallbackManager, BaseCallbackHandler)
Supports: Y (read)
Observation: Observer pattern with event methods for queries and tools; handlers can observe events/payloads (e.g., read prompts/responses) but are designed for debugging/tracing without modifying execution context.

For more details, see the official documentation: Callbacks - LlamaIndex. Here's an example setup (Python):

from llama_index.core.callbacks import CallbackManager, LlamaDebugHandler

debug_handler = LlamaDebugHandler()  # Concrete handler subclassing BaseCallbackHandler
callback_manager = CallbackManager([debug_handler])

# Assign to components, e.g., an index or query engine
index = VectorStoreIndex.from_documents(documents, callback_manager=callback_manager)
query_engine = index.as_query_engine()
response = query_engine.query("What is this about?")

Haystack

Haystack does not support explicit middleware or filters like the others. Instead, it uses a modular pipeline architecture for interception via components (e.g., ConditionalRouter for routing based on conditions like tool calls) and observability through logging/tracing integrations (e.g., Langfuse).

Naming (Python): N/A (Pipeline Components/Routers)
Supports: N (Pipeline-based interception)
Observation: Relies on modular pipelines for implicit interception but lacks explicit middleware/filters; custom components can read/write data flow via routing/transformations, but this is compositional rather than hook-based interception.

For more details, see the official documentation: Pipelines - Haystack Documentation. Here's an example of pipeline-based interception with a custom collector component (Python):

/* Detailed source-code truncated for AI context efficiency. */

OpenAI Swarm

OpenAI Swarm does not provide native support for middleware, filters, callbacks, or hooks. While interception can be achieved through custom implementations (e.g., function wrappers, client subclassing, or manual tool execution with execute_tools=False), this requires the caller to implement their own logic, which is not considered built-in framework support.

Naming (Python): N/A
Supports: N
Observation: No explicit middleware/filters; interception requires custom wrappers or manual handling (e.g., function decorators, client subclassing), lacking native framework support for built-in components to accept such modifications.

For more details, see the official GitHub repository: OpenAI Swarm GitHub. No native code examples available for interception; custom approaches are possible but not framework-native.

Atomic Agents

Atomic Agents does not support explicit middleware, callbacks, hooks, or filters. Its modularity allows composable components, but no dedicated interception mechanisms are documented.

Naming (Python): N/A (Composable Components)
Supports: N
Observation: No explicit middleware/filters; modularity allows composable units but no dedicated interception hooks or callbacks for custom reading/modification mid-execution.

No specific code examples available for interception.

Smolagents (Hugging Face)

Smolagents does not support explicit middleware, callbacks, hooks, or filters; it focuses on simple agent building.

Naming (Python): N/A
Supports: N
Observation: No explicit support; focuses on simple agent building without interception mechanisms or hooks for reading/modifying execution.

For more details, see the official documentation: Smolagents Docs. No specific code examples available for interception.

Phidata (Agno)

Phidata (Agno) does not support explicit middleware, callbacks, hooks, or filters; agents rely on tools and memory.

Naming (Python): N/A
Supports: N
Observation: No explicit middleware/filters; agents use tools/memory but no interception hooks for custom reading/modification of calls.

For more details, see the official documentation: Phidata Docs. No specific code examples available for interception.

PromptFlow (Microsoft)

PromptFlow supports tracing for LLM interactions, which acts like callbacks for debugging and iteration.

Naming (Python): Tracing
Supports: N (Tracing only)
Observation: Supports tracing for LLM interactions, acting as callbacks for debugging/iteration; tracing is read-only for observability/telemetry without options to modify context or intercept calls beyond logging.

For more details, see the official documentation: Tracing in PromptFlow. No direct code examples in the browsed content, but tracing is integrated into flow debugging (Python).

n8n

n8n's AI Agent node inherits callbacks from LangChain for observability in workflows.

Naming (JS/TS): Callbacks (inherited from LangChain)
Supports: Y (read/write)
Observation: AI Agent node uses LangChain under the hood, inheriting callbacks for observability; supports reading/modifying metadata or interrupting flow as in LangChain.

For more details, see the official documentation: AI Agent Node Docs. (Inherits from LangChain; refer to LangChain docs for callback examples.) No specific n8n-unique code in the content, but uses LangChain's observer pattern. Here's an adapted LangChain JS example for consistency:

import { BaseCallbackHandler } from "@langchain/core/callbacks/base";

class MyHandler extends BaseCallbackHandler {
  name = "my_handler";

  async handleChainStart(chain, inputs) {
    inputs.number += 1;  # Modify inputs (write capability)
    console.log("Chain started!");
  }
}

const handler = new MyHandler();

// Pass callback at runtime
await chain.invoke({ number: 25 }, { callbacks: [handler] });

// Or at constructor time
const chainWithHandler = new SomeChain({ callbacks: [handler] });
await chainWithHandler.invoke({ number: 25 });

2. Official Technical Reference & Guides (microsoft/microsoft.github.io) Copy Chapter

[microsoft.github.io](https://microsoft.github.io)

Many open source projects released by Microsoft products, services and teams also have
deployments powered by GitHub Pages on this domain.

To discover Microsoft-released open source projects, learn about Microsoft's open source
approach and other information, check us out online at opensource.microsoft.com.

If we can help answer questions about the open source program at Microsoft, you can
contact opensource at microsoft com - although we are not able to provide support
for projects through this mechanism.

Microsoft GitHub resources Copy Chapter

• Default repo templates including SECURITY.MD and code of conduct files: microsoft/repo-templates
• Code of Conduct (FAQ)

Microsoft open source resources Copy Chapter

• Open Source Blog
• The Open Source Show on Channel 9
• https://opensource.microsoft.com/

Resources for Microsoft employees Copy Chapter

• docs.opensource.microsoft.com
• Link your GitHub Account and join GitHub organizations automatically

How to identify an official Microsoft organization on GitHub Copy Chapter

• Official Microsoft GitHub orgs have a professional avatar and include a web address at Microsoft.com or an associated official domain
• Official Microsoft orgs are Verified on GitHub - look for the domain verification indicator on GitHub