semantic-kernel Documentation

1. Project Overview & Quickstart (microsoft/semantic-kernel) Copy Chapter

File: README.md Copy Chapter

Semantic Kernel

[!IMPORTANT]
Semantic Kernel is now Microsoft Agent Framework! Microsoft Agent Framework (MAF) is the enterprise‑ready successor to Semantic Kernel. Microsoft Agent Framework is now available at version 1.0 as a production-ready release: stable APIs, and a commitment to long-term support. Whether you're building a single assistant or orchestrating a fleet of specialized agents, Microsoft Agent Framework 1.0 gives you enterprise-grade multi-agent orchestration, multi-provider model support, and cross-runtime interoperability via A2A and MCP.

Learn more about Semantic Kernel and Agent Framework here: Semantic Kernel and Microsoft Agent Framework on the Agent Framework blog, and try out the Semantic Kernel migration guide.

Build intelligent AI agents and multi-agent systems with this enterprise-ready orchestration framework

What is Semantic Kernel? Copy Chapter

Semantic Kernel is a model-agnostic SDK that empowers developers to build, orchestrate, and deploy AI agents and multi-agent systems. Whether you're building a simple chatbot or a complex multi-agent workflow, Semantic Kernel provides the tools you need with enterprise-grade reliability and flexibility.

System Requirements Copy Chapter

• Python: 3.10+
• .NET: .NET 10.0+
• Java: JDK 17+
• OS Support: Windows, macOS, Linux

Key Features Copy Chapter

• Model Flexibility: Connect to any LLM with built-in support for OpenAI, Azure OpenAI, Hugging Face, NVidia and more
• Agent Framework: Build modular AI agents with access to tools/plugins, memory, and planning capabilities
• Multi-Agent Systems: Orchestrate complex workflows with collaborating specialist agents
• Plugin Ecosystem: Extend with native code functions, prompt templates, OpenAPI specs, or Model Context Protocol (MCP)
• Vector DB Support: Seamless integration with Azure AI Search, Elasticsearch, Chroma, and more
• Multimodal Support: Process text, vision, and audio inputs
• Local Deployment: Run with Ollama, LMStudio, or ONNX
• Process Framework: Model complex business processes with a structured workflow approach
• Enterprise Ready: Built for observability, security, and stable APIs

Installation Copy Chapter

First, set the environment variable for your AI Services:

Azure OpenAI:

export AZURE_OPENAI_API_KEY=AAA....

or OpenAI directly:

export OPENAI_API_KEY=sk-...

Python Copy Section

pip install semantic-kernel

.NET Copy Section

dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Agents.Core

Java Copy Section

See semantic-kernel-java build for instructions.

Quickstart Copy Chapter

Basic Agent - Python Copy Section

Create a simple assistant that responds to user prompts:

import asyncio
from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion

async def main():
    # Initialize a chat agent with basic instructions
    agent = ChatCompletionAgent(
        service=AzureChatCompletion(),
        name="SK-Assistant",
        instructions="You are a helpful assistant.",
    )

    # Get a response to a user message
    response = await agent.get_response(messages="Write a haiku about Semantic Kernel.")
    print(response.content)

asyncio.run(main()) 

# Output:
# Language's essence,
# Semantic threads intertwine,
# Meaning's core revealed.

Basic Agent - .NET Copy Section

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;

var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
                Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT"),
                Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"),
                Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY")
                );
var kernel = builder.Build();

ChatCompletionAgent agent =
    new()
    {
        Name = "SK-Agent",
        Instructions = "You are a helpful assistant.",
        Kernel = kernel,
    };

await foreach (AgentResponseItem<ChatMessageContent> response 
    in agent.InvokeAsync("Write a haiku about Semantic Kernel."))
{
    Console.WriteLine(response.Message);
}

// Output:
// Language's essence,
// Semantic threads intertwine,
// Meaning's core revealed.

Agent with Plugins - Python Copy Section

Enhance your agent with custom tools (plugins) and structured output:

import asyncio
from typing import Annotated
from pydantic import BaseModel
from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, OpenAIChatPromptExecutionSettings
from semantic_kernel.functions import kernel_function, KernelArguments

class MenuPlugin:
    @kernel_function(description="Provides a list of specials from the menu.")
    def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
        return """
        Special Soup: Clam Chowder
        Special Salad: Cobb Salad
        Special Drink: Chai Tea
        """

    @kernel_function(description="Provides the price of the requested menu item.")
    def get_item_price(
        self, menu_item: Annotated[str, "The name of the menu item."]
    ) -> Annotated[str, "Returns the price of the menu item."]:
        return "$9.99"

class MenuItem(BaseModel):
    price: float
    name: str

async def main():
    # Configure structured output format
    settings = OpenAIChatPromptExecutionSettings()
    settings.response_format = MenuItem

    # Create agent with plugin and settings
    agent = ChatCompletionAgent(
        service=AzureChatCompletion(),
        name="SK-Assistant",
        instructions="You are a helpful assistant.",
        plugins=[MenuPlugin()],
        arguments=KernelArguments(settings)
    )

    response = await agent.get_response(messages="What is the price of the soup special?")
    print(response.content)

    # Output:
    # The price of the Clam Chowder, which is the soup special, is $9.99.

asyncio.run(main())

Agent with Plugin - .NET Copy Section

using System.ComponentModel;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.ChatCompletion;

var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
                Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT"),
                Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"),
                Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY")
                );
var kernel = builder.Build();

kernel.Plugins.Add(KernelPluginFactory.CreateFromType<MenuPlugin>());

ChatCompletionAgent agent =
    new()
    {
        Name = "SK-Assistant",
        Instructions = "You are a helpful assistant.",
        Kernel = kernel,
        Arguments = new KernelArguments(new PromptExecutionSettings() { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() })

    };

await foreach (AgentResponseItem<ChatMessageContent> response 
    in agent.InvokeAsync("What is the price of the soup special?"))
{
    Console.WriteLine(response.Message);
}

sealed class MenuPlugin
{
    [KernelFunction, Description("Provides a list of specials from the menu.")]
    public string GetSpecials() =>
        """
        Special Soup: Clam Chowder
        Special Salad: Cobb Salad
        Special Drink: Chai Tea
        """;

    [KernelFunction, Description("Provides the price of the requested menu item.")]
    public string GetItemPrice(
        [Description("The name of the menu item.")]
        string menuItem) =>
        "$9.99";
}

Multi-Agent System - Python Copy Section

Build a system of specialized agents that can collaborate:

/* Detailed source-code truncated for AI context efficiency. */

Where to Go Next Copy Chapter

1. 📖 Try our Getting Started Guide or learn about Building Agents
2. 🔌 Explore over 100 Detailed Samples
3. 💡 Learn about core Semantic Kernel Concepts

API References Copy Section

• C# API reference
• Python API reference

Troubleshooting Copy Chapter

Common Issues Copy Section

• Authentication Errors: Check that your API key environment variables are correctly set
• Model Availability: Verify your Azure OpenAI deployment or OpenAI model access

Getting Help Copy Section

• Check our GitHub issues for known problems
• Search the Discord community for solutions
• Include your SDK version and full error messages when asking for help

Join the community Copy Chapter

We welcome your contributions and suggestions to the SK community! One of the easiest ways to participate is to engage in discussions in the GitHub repository. Bug reports and fixes are welcome!

For new features, components, or extensions, please open an issue and discuss with us before sending a PR. This is to avoid rejection as we might be taking the core in a different direction, but also to consider the impact on the larger ecosystem.

To learn more and get started:

• Read the documentation

• Learn how to contribute to the project

• Ask questions in the GitHub discussions

• Ask questions in the Discord community

• Attend regular office hours and SK community events

• Follow the team on our blog

Contributor Wall of Fame Copy Chapter

Code of Conduct Copy Chapter

This project has adopted the
Microsoft Open Source Code of Conduct.
For more information, see the
Code of Conduct FAQ
or contact [email protected]
with any additional questions or comments.

License Copy Chapter

Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT license.

File: docs/decisions/0001-madr-architecture-decisions.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted date: 2023-05-29 deciders: dluc, shawncal, hathind, alliscode consulted: informed: Copy Chapter

Use Markdown Any Decision Records to track Semantic Kernel Architecture Decisions

Context and Problem Statement Copy Chapter

We have multiple different language versions of the Semantic Kernel under active development i.e., C#, Python, Java and Typescript.
We need a way to keep the implementations aligned with regard to key architectural decisions e.g., we are reviewing a change to the format used to store
semantic function configuration (config.json) and when this change is agreed it must be reflected in all of the Semantic Kernel implementations.

MADR is a lean template to capture any decisions in a structured way. The template originated from capturing architectural decisions and developed to a template allowing to capture any decisions taken.
For more information see

Decision Drivers Copy Chapter

• Architecture changes and the associated decision making process should be transparent to the community.
• Decision records are stored in the repository and are easily discoverable for teams involved in the various language ports.

Considered Options Copy Chapter

• Use MADR format and store decision documents in the repository.

Decision Outcome Copy Chapter

Chosen option:

Pros and Cons of the Options Copy Chapter

Use MADR format and store decision documents in the repository Copy Section

How would we use ADR's to track technical decisions?

1. Copy docs/decisions/adr-template.md to docs/decisions/NNNN-title-with-dashes.md, where NNNN indicates the next number in sequence.
   1. Check for existing PR's to make sure you use the correct sequence number.
   2. There is also a short form template docs/decisions/adr-short-template.md
2. Edit NNNN-title-with-dashes.md.
   1. Status must initially be proposed
   2. List of deciders must include the aliases of the people who will sign off on the decision.
   3. The relevant EM and dluc must be listed as deciders or informed of all decisions.
   4. You should list the aliases of all partners who were consulted as part of the decision.
3. For each option list the good, neutral and bad aspects of each considered alternative.
   1. Detailed investigations can be included in the More Information section inline or as links to external documents.
4. Share your PR with the deciders and other interested parties.
   1. Deciders must be listed as required reviewers.
   2. The status must be updated to accepted once a decision is agreed and the date must also be updated.
   3. Approval of the decision is captured using PR approval.
5. Decisions can be changed later and superseded by a new ADR. In this case it is useful to record any negative outcomes in the original ADR.

• Good, because lightweight format which is easy to edit
• Good, because this uses the standard Git review process for commenting and approval
• Good, because decisions and review process are transparent to the community

File: docs/decisions/0002-java-folder-structure.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted date: 2013-06-19 deciders: shawncal,johnoliver consulted: informed: Copy Chapter

Java Folder Structure

Context and Problem Statement Copy Chapter

A port of the Semantic Kernel to Java is under development in the experimental-java branch. The folder structure being used has diverged from the .Net implementation.
The purpose of this ADR is to document the folder structure that will be used by the Java port to make it clear to developers how to navigate between the .Net and Java implementations.

Decision Drivers Copy Chapter

• Goal is to learn for SDKs that already have excellent multiple language support e.g., Azure SDK
• The Java SK should follow the general design guidelines and conventions of Java. It should feel natural to a Java developer.
• Different language versions should be consistent with the .Net implementation. In cases of conflict, consistency with Java conventions is the highest priority.
• The SK for Java and .Net should feel like a single product developed by a single team.
• There should be feature parity between Java and .Net. Feature status must be tracked in the FEATURE_MATRIX

Considered Options Copy Chapter

Below is a comparison of .Net and Java Folder structures

dotnet/src
           Connectors
           Extensions
           IntegrationTests
           InternalUtilities
           SemanticKernel.Abstractions
           SemanticKernel.MetaPackage
           SemanticKernel.UnitTests
           SemanticKernel
           Skills

Some observations:

• The src folder is at the very start of the folder structure, which reduces flexibility
• The use of the Skills term is due to change

java
     api-test
     samples
     semantickernel-api
     semantickernel-bom
     semantickernel-connectors-parent
     semantickernel-core-skills
     semantickernel-core
     semantickernel-extensions-parent

Some observations:

• Using lowercase folder name with the - delimiter is idiomatic Java
• The src folders are located as close as possible to the source files e.g., semantickernel-api/src/main/java, this is idiomatic Java
• Unit tests are contained together with the implementation
• The samples are located within the java folder and each sample runs standalone

Decision Outcome Copy Chapter

Follow these guidelines:

• The folder names will match those used (or planned for .Net) but in the idiomatic Java folder naming convention
• Use bom instead of MetaPackage as the latter is .Net centric
• Use api instead of Abstractions as the latter is .Net centric
• Move semantickernel-core-skills to a new plugins folder and rename to plugins-core
• Use the term plugins instead of skills and avoid introducing technical debt

File: docs/decisions/0003-support-multiple-native-function-args.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: markwallace-microsoft date: 2023-06-16 deciders: shawncal,dluc consulted: informed: Copy Chapter

Add support for multiple native function arguments of many types

Context and Problem Statement Copy Chapter

Move native functions closer to a normal C# experience.

Decision Drivers Copy Chapter

• Native skills can now have any number of parameters. The parameters are populated from context variables of the same name. If no context variable exists for that name, it'll be populated with a default value if one was supplied via either an attribute or a default parameter value, or if there is none, the function will fail to be invoked. The first parameter may also be populated from "input" if it fails to get input by its name or default value.
• Descriptions are now specified with the .NET DescriptionAttribute, and DefaultValue with the DefaultValueAttribute. The C# compiler is aware of the DefaultValueAttribute and ensures the type of the value provided matches that of the type of the parameter. Default values can now also be specified using optional parameter values.
• SKFunction is now purely a marker attribute, other than for sensitivity. It's sole purpose is to subset which public members are imported as native functions when a skill is imported. It was already the case that the attribute wasn't needed when importing a function directly from a delegate; that requirement has also been lifted when importing from a MethodInfo.
• SKFunctionContextParameterAttribute has been obsoleted and will be removed subsequently. DescriptionAttribute, DefaultValueAttribute, and SKName attribute are used instead. In rare situations where the method needs access to a variable that's not defined in its signature, it can use the SKParameter attribute on the method, which does have Description and DefaultValue optional properties.
• SKFunctionInputAttribute has been obsoleted and will be removed subsequently. DescriptionAttribute, DefaultValueAttribute, and SKName attribute are used instead (the latter with "Input" as the name). However, the need to use SKName should be exceedingly rare.
• InvokeAsync will now catch exceptions and store the exception into the context. This means native skills should handle all failures by throwing exceptions rather than by directly interacting with the context.
• Updated name selection heuristic to strip off an "Async" suffix for async methods. There are now very few reasons to use [SKName] on a method.
• Added support for ValueTasks as return types, just for completeness so that developers don't need to think about it. It just works.
• Added ability to accept an ILogger or CancellationToken into a method; they're populated from the SKContext. With that, there are very few reasons left to pass an SKContext into a native function.
• Added support for non-string arguments. All C# primitive types and many core .NET types are supported, with their corresponding TypeConverters used to parse the string context variable into the appropriate type. Custom types attributed with TypeConverterAttribute may also be used, and the associated TypeConverter will be used as is appropriate. It's the same mechanism used by UI frameworks like WinForms as well as ASP.NET MVC.
• Similarly, added support for non-string return types.

Decision Outcome Copy Chapter

PR 1195

More Information Copy Chapter

Example

Before:

[SKFunction("Adds value to a value")]
[SKFunctionName("Add")]
[SKFunctionInput(Description = "The value to add")]
[SKFunctionContextParameter(Name = "Amount", Description = "Amount to add")]
public Task<string> AddAsync(string initialValueText, SKContext context)
{
    if (!int.TryParse(initialValueText, NumberStyles.Any, CultureInfo.InvariantCulture, out var initialValue))
    {
        return Task.FromException<string>(new ArgumentOutOfRangeException(
            nameof(initialValueText), initialValueText, "Initial value provided is not in numeric format"));
    }

    string contextAmount = context["Amount"];
    if (!int.TryParse(contextAmount, NumberStyles.Any, CultureInfo.InvariantCulture, out var amount))
    {
        return Task.FromException<string>(new ArgumentOutOfRangeException(
            nameof(context), contextAmount, "Context amount provided is not in numeric format"));
    }

    var result = initialValue + amount;
    return Task.FromResult(result.ToString(CultureInfo.InvariantCulture));
}

After:

[SKFunction, Description("Adds an amount to a value")]
public int Add(
    [Description("The value to add")] int value,
    [Description("Amount to add")] int amount) =>
    value + amount;

Example

Before:

[SKFunction("Wait a given amount of seconds")]
[SKFunctionName("Seconds")]
[SKFunctionInput(DefaultValue = "0", Description = "The number of seconds to wait")]
public async Task SecondsAsync(string secondsText)
{
    if (!decimal.TryParse(secondsText, NumberStyles.Any, CultureInfo.InvariantCulture, out var seconds))
    {
        throw new ArgumentException("Seconds provided is not in numeric format", nameof(secondsText));
    }

    var milliseconds = seconds * 1000;
    milliseconds = (milliseconds > 0) ? milliseconds : 0;

    await this._waitProvider.DelayAsync((int)milliseconds).ConfigureAwait(false);
}

After:

[SKFunction, Description("Wait a given amount of seconds")]
public async Task SecondsAsync([Description("The number of seconds to wait")] decimal seconds)
{
    var milliseconds = seconds * 1000;
    milliseconds = (milliseconds > 0) ? milliseconds : 0;

    await this._waitProvider.DelayAsync((int)milliseconds).ConfigureAwait(false);
}

Example

Before:

/* Detailed source-code truncated for AI context efficiency. */

After:

[SKFunction, Description("Add an event to my calendar.")]
public async Task AddEventAsync(
    [Description("Event subject"), SKName("input")] string subject,
    [Description("Event start date/time as DateTimeOffset")] DateTimeOffset start,
    [Description("Event end date/time as DateTimeOffset")] DateTimeOffset end,
    [Description("Event location (optional)")] string? location = null,
    [Description("Event content/body (optional)")] string? content = null,
    [Description("Event attendees, separated by ',' or ';'.")] string? attendees = null)
{
    if (string.IsNullOrWhiteSpace(subject))
    {
        throw new ArgumentException($"{nameof(subject)} variable was null or whitespace", nameof(subject));
    }

    CalendarEvent calendarEvent = new()
    {
        Subject = subject,
        Start = start,
        End = end,
        Location = location,
        Content = content,
        Attendees = attendees is not null ? attendees.Split(new[] { ',', ';' }, StringSplitOptions.RemoveEmptyEntries) : Enumerable.Empty<string>(),
    };

    this._logger.LogInformation("Adding calendar event '{0}'", calendarEvent.Subject);
    await this._connector.AddEventAsync(calendarEvent).ConfigureAwait(false);
}

File: docs/decisions/0004-error-handling.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: SergeyMenshykh date: 2023-06-23 deciders: shawncal consulted: stephentoub informed: Copy Chapter

Error handling improvements

Disclaimer Copy Chapter

This ADR describes problems and their solutions for improving the error handling aspect of SK. It does not address logging, resiliency, or observability aspects.

Context and Problem Statement Copy Chapter

Currently, there are several aspects of error handling in SK that can be enhanced to simplify SK code and SK client code, while also ensuring consistency and maintainability:

• Exception propagation. SK has a few public methods, like Kernel.RunAsync and SKFunction.InvokeAsync, that handle exceptions in a non-standard way. Instead of throwing exceptions, they catch and store them within the SKContext. This deviates from the standard error handling approach in .NET, which expects a method to either execute successfully if its contract is fulfilled or throw an exception if the contract is violated. Consequently, when working with the .NET version of the SK SDK, it becomes challenging to determine whether a method executed successfully or failed without analyzing specific properties of the SKContext instance. This can lead to a frustrating experience for developers using the .NET SK SDK.

• Improper exception usage. Some SK components use custom SK exceptions instead of standard .NET exceptions to indicate invalid arguments, configuration issues, and so on. This deviates from the standard approach for error handling in .NET and may frustrate SK client code developers.

• Exception hierarchy. Half of the custom SK exceptions are derived from SKException, while the other half are directly derived from Exception. This inconsistency in the exception hierarchy does not contribute to a cohesive exception model.

• Unnecessary and verbose exceptions A few SK components, such as the Kernel or Planner, have exceptions at their level, namely PlanningException or KernelException, that are not truly necessary and can be easily replaced by SKException and a few of its derivatives. SK clients might become dependent on them, making it challenging to remove them later if SK needs to discontinue their usage. Additionally, SK has an exception type for each SK memory connector - PineconeMemoryException, QdrantMemoryException that does not add any additional information and only differs by name while having the same member signatures. This makes it impossible for SK client code to handle them in a consolidated manner. Instead of having a single catch block, SK client code needs to include a catch block for each component implementation. Moreover, SK client code needs to be updated every time a new component implementation is added or removed.

• Missing original exception details. Certain SK exceptions do not preserve the original failure or exception details and do not expose them through their properties. This omission prevents SK client code from understanding the problem and handling it properly.

Decision Drivers Copy Chapter

• Exceptions should be propagated to the SK client code instead of being stored in the SKContext. This adjustment will bring SK error handling in line with the .NET approach.
• The SK exception hierarchy should be designed following the principle of "less is more." It is easier to add new exceptions later, but removing them can be challenging.
• .NET standard exception types should be preferred over SK custom ones because they are easily recognizable, do not require any maintenance, can cover common error scenarios, and provide meaningful and standardized error messages.
• Exceptions should not be wrapped in SK exceptions when passing them up to a caller, unless it helps in constructing actionable logic for either SK or SK client code.

Considered Options Copy Chapter

• Simplify existing SK exception hierarchy by removing all custom exceptions types except the SKException one and any other type that is actionable. Use SKException type instead of the removed ones unless more details need to be conveyed in which case create a derived specific exception.
• Modify SK code to throw .NET standard exceptions, such as ArgumentOutOfRangeException or ArgumentNullException, when class argument values are not provided or are invalid, instead of throwing custom SK exceptions. Analyze SK exception usage to identify and fix other potential areas where standard .NET exceptions can be used instead.
• Remove any code that wraps unhandled exceptions into AIException or any other SK exception solely for the purpose of wrapping. In most cases, this code does not provide useful information to action on it, apart from a generic and uninformative "Something went wrong" message.
• Identify all cases where the original exception is not preserved as an inner exception of the rethrown SK exception, and address them.
• Create a new exception HttpOperationException, which includes a StatusCode property, and implement the necessary logic to map the exception from HttpStatusCode, HttpRequestException, or Azure.RequestFailedException. Update existing SK code that interacts with the HTTP stack to throw HttpOperationException in case of a failed HTTP request and assign the original exception as its inner exception.
• Modify all SK components that currently store exceptions to SK context to rethrow them instead.
• Simplify the SK critical exception handling functionality by modifying the IsCriticalException extension method to exclude handling of StackOverflowException and OutOfMemoryException exceptions. This is because the former exception is not thrown, so the calling code won't be executed, while the latter exception doesn't necessarily prevent the execution of recovery code.

File: docs/decisions/0005-kernel-hooks-phase1.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: rogerbarreto date: 2023-05-29 deciders: rogerbarreto, shawncal, stephentoub consulted: informed: Copy Chapter

Kernel/Function Handlers - Phase 1

Context and Problem Statement Copy Chapter

A Kernel function caller needs to be able to handle/intercept any function execution in the Kernel before and after it was attempted. Allowing it to modify the prompt, abort the execution, or modify the output and many other scenarios as follows:

• Pre-Execution / Function Invoking

  • Get: SKContext
  • Set: Modify input parameters sending to the function
  • Set: Abort/Cancel pipeline execution
  • Set: Skip function execution

• Post-Execution / Function Invoked

  • Get: LLM Model Result (Tokens Usage, Stop Sequence, ...)
  • Get: SKContext
  • Get: Output parameters
  • Set: Modify output parameters content (before returning the output)
  • Set: Cancel pipeline execution
  • Set: Repeat function execution

Out of Scope (Will be in phase 2) Copy Chapter

• Pre-Execution / Function Invoking

  • Get: Rendered Prompt
  • Get: Current settings used
  • Set: Modify the Rendered Prompt

• Post-Execution / Function Invoked

  • Get: Rendered Prompt
  • Get: Current settings used

Decision Drivers Copy Chapter

• Architecture changes and the associated decision making process should be transparent to the community.
• Decision records are stored in the repository and are easily discoverable for teams involved in the various language ports.
• Simple, Extensible and easy to understand.

Considered Options Copy Chapter

1. Callback Registration + Recursive
2. Single Callback
3. Event Based Registration
4. Middleware
5. ISKFunction Event Support Interfaces

Pros and Cons of the Options Copy Chapter

1. Callback Registration Recursive Delegate (Kernel, Plan, Function) Copy Section

• Specified on plan and function level as a configuration be able to specify what are the callback Handlers that will be triggered.

Pros:

• Common pattern for observing and also changing data exposed as parameter into the delegate signature for (Get/Set) scenarios
• Registering a callback gives back the registration object that can be used to cancel the execution of the function in the future.
• Recursive approach, allows to register multiple callbacks for the same event, and also allows to register callbacks on top of pre existing callbacks.

Cons:

• Registrations may use more memory and might not be garbage collected in the recursive approach, only when the function or the plan is disposed.

2. Single Callback Delegate (Kernel, Plan, Function) Copy Section

• Specified on kernel level as a configuration be able to specify what are the callback Handlers that will be triggered.
  • Specified on function creation: As part of the function constructor be able to specify what are the callback Handlers that will be triggered.
  • Specified on function invocation: As part of the function invoke be able to specify what are the callback Handlers as a parameter that will be triggered.

Pros:

• Common pattern for observing and also changing data exposed as parameter into the delegate signature for (Get/Set) scenarios

Cons:

• Limited to only one method observing a specific event (Pre Post and InExecution). - Function When used as parameter, three new parameters would be needed as part of the function. (Specified on function invocation) - Extra Cons on

3. Event Base Registration (Kernel only) Copy Section

Expose events on both IKernel and ISKFunction that the call can can be observing to interact.

Pros:

• Multiple Listeners can registered for the same event
• Listeners can be registered and unregistered at will
• Common pattern (EventArgs) for observing and also changing data exposed as parameter into the event signature for (Get/Set) scenarios

Cons:

• Event handlers are void, making the EventArgs by reference the only way to modify the data.
• Not clear how supportive is this approach for asynchronous pattern/multi threading
• Won't support ISKFunction.InvokeAsync

4. Middleware (Kernel Only) Copy Section

Specified on Kernel level, and would only be used using IKernel.RunAsync operation, this pattern would be similar to asp.net core middlewares, running the pipelines with a context and a requestdelegate next for controlling (Pre/Post conditions)

Pros:

• Common pattern for handling Pre/Post Setting/Filtering data

Cons:

• Functions can run on their own instance, middlewares suggest more complexity and the existence of an external container/manager (Kernel) to intercept/observe function calls.

5. ISKFunction Event Support Interfaces Copy Section

```

/* Detailed source-code truncated for AI context efficiency. */

### Pros and Cons

Pros:

- `Kernel` is not aware of `SemanticFunction` implementation details or any other `ISKFunction` implementation
- Extensible to show dedicated EventArgs per custom `ISKFunctions` implementation, including prompts for semantic functions
- Extensible to support future events on the Kernel thru the `ISKFunctionEventSupport<NewEvent>` interface
- Functions can have their own EventArgs specialization.
- Interface is optional, so custom `ISKFunctions` can choose to implement it or not

Cons:

- Any custom functions now will have to responsibility implement the `ISKFunctionEventSupport` interface if they want to support events.
- `Kernel` will have to check if the function implements the interface or not, and if not, it will have to throw an exception or ignore the event.
- Functions implementations that once were limited to InvokeAsync now need to be scattered across multiple places and handle the state of the execution related to content that needs to be get at the beginning or at the end of the invocation.

## Main Questions

- Q: Post Execution Handlers should execute right after the LLM result or before the end of the function execution itself?
  A: Currently post execution Handlers are executed after function execution.

- Q: Should Pre/Post Handlers be many (pub/sub) allowing registration/deregistration?
  A: By using the standard .NET event implementation, this already supports multiple registrations as well as deregistrations managed by the caller.

- Q: Setting Handlers on top of pre existing Handlers should be allowed or throw an error?
  A: By using the standard .NET event implementation, the standard behavior will not throw an error and will execute all the registered handlers.

- Q: Setting Handlers on Plans should automatically cascade this Handlers for all the inner steps + overriding existing ones in the process?
  A: Handlers will be triggered before and after each step is executed the same way the Kernel RunAsync pipeline works.

- Q: When a pre function execution handler intents to cancel the execution, should further handlers in the chain be called or not?
  A: Currently the standard .net behavior is to call all the registered handlers. This way function execution will solely depends on the final state of the Cancellation Request after all handlers were called.

## Decision Outcome

Chosen option: **3. Event Base Registration (Kernel only)**

This approach is the simplest and take the benefits of the standard .NET event implementation.

Further changes will be implemented to fully support all the scenarios in phase 2.

---

## File: docs/decisions/0006-open-api-dynamic-payload-and-namespaces.md

---
status: superseded by [ADR-0062](0062-open-api-payload.md)
contact: SergeyMenshykh
date: 2023-08-15
deciders: shawncal
consulted:
informed:
---

# Dynamic payload building for PUT and POST RestAPI operations and parameter namespacing

## Context and Problem Statement

Currently, the SK OpenAPI does not allow the dynamic creation of payload/body for PUT and POST RestAPI operations, even though all the required metadata is available. One of the reasons the functionality was not fully developed originally, and eventually removed is that JSON payload/body content of PUT and POST RestAPI operations might contain properties with identical names at various levels. It was not clear how to unambiguously resolve their values from the flat list of context variables. Another reason the functionality has not been added yet is that the 'payload' context variable, along with RestAPI operation data contract schema(OpenAPI, JSON schema, Typings?) should have been sufficient for LLM to provide fully fleshed-out JSON payload/body content without the need to build it dynamically.



## Decision Drivers

- Create a mechanism that enables the dynamic construction of the payload/body for PUT and POST RestAPI operations.
- Develop a mechanism(namespacing) that allows differentiation of payload properties with identical names at various levels for PUT and POST RestAPI operations.
- Aim to minimize breaking changes and maintain backward compatibility of the code as much as possible.

## Considered Options

- Enable the dynamic creation of payload and/or namespacing by default.
- Enable the dynamic creation of payload and/or namespacing based on configuration.

## Decision Outcome

Chosen option: "Enable the dynamic creation of payload and/or namespacing based on configuration". This option keeps things compatible, so the change won't affect any SK consumer code. Additionally, it lets SK consumer code easily control both mechanisms, turning them on or off based on the scenario.

## Additional details

### Enabling dynamic creation of payload

In order to enable the dynamic creation of payloads/bodies for PUT and POST RestAPI operations, please set the `EnableDynamicPayload` property of the `OpenApiSkillExecutionParameters` execution parameters to `true` when importing the AI plugin:

```csharp
var plugin = await kernel.ImportPluginFunctionsAsync("<skill name>", new Uri("<chatGPT-plugin>"), new OpenApiSkillExecutionParameters(httpClient) { EnableDynamicPayload = true });

To dynamically construct a payload for a RestAPI operation that requires payload like this:

{
  "value": "secret-value",
  "attributes": {
    "enabled": true
  }
}

Please register the following arguments in context variables collection:

var contextVariables = new ContextVariables();
contextVariables.Set("value", "secret-value");
contextVariables.Set("enabled", true);

Enabling namespacing Copy Section

To enable namespacing, set the EnablePayloadNamespacing property of the OpenApiSkillExecutionParameters execution parameters to true when importing the AI plugin:

var plugin = await kernel.ImportPluginFunctionsAsync("<skill name>", new Uri("<chatGPT-plugin>"), new OpenApiSkillExecutionParameters(httpClient) { EnablePayloadNamespacing = true });

Remember that the namespacing mechanism depends on prefixing parameter names with their parent parameter name, separated by dots. So, use the 'namespaced' parameter names when adding arguments for them to the context variables. Let's consider this JSON:

{
  "upn": "<sender upn>",
  "receiver": {
    "upn": "<receiver upn>"
  },
  "cc": {
    "upn": "<cc upn>"
  }
}

It contains upn properties at different levels. The the argument registration for the parameters(property values) will look like:

var contextVariables = new ContextVariables();
contextVariables.Set("upn", "<sender-upn-value>");
contextVariables.Set("receiver.upn", "<receiver-upn-value>");
contextVariables.Set("cc.upn", "<cc-upn-value>");

File: docs/decisions/0007-prompt-extract-template-engine.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: markwallace-microsoft date: 2023-08-25 deciders: shawncal consulted: informed: Copy Chapter

Extract the Prompt Template Engine from Semantic Kernel core

Context and Problem Statement Copy Chapter

The Semantic Kernel includes a default prompt template engine which is used to render Semantic Kernel prompts i.e., skprompt.txt files. The prompt template is rendered before being send to the AI to allow the prompt to be generated dynamically e.g., include input parameters or the result of a native or semantic function execution.
To reduce the complexity and API surface of the Semantic Kernel the prompt template engine is going to be extracted and added to it's own package.

The long term goal is to enable the following scenarios:

1. Implement a custom template engine e.g., using Handlebars templates. This is supported now but we want to simplify the API to be implemented.
2. Support using zero or many template engines.

Decision Drivers Copy Chapter

• Reduce API surface and complexity of the Semantic Kernel core.
• Simplify the IPromptTemplateEngine interface to make it easier to implement a custom template engine.
• Make the change without breaking existing clients.

Decision Outcome Copy Chapter

• Create a new package called Microsoft.SemanticKernel.TemplateEngine.
• Maintain the existing namespace for all prompt template engine code.
• Simplify the IPromptTemplateEngine interface to just require implementation of RenderAsync.
• Dynamically load the existing PromptTemplateEngine if the Microsoft.SemanticKernel.TemplateEngine assembly is available.

File: docs/decisions/0008-support-generic-llm-request-settings.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: markwallace-microsoft date: 2023-9-15 deciders: shawncal consulted: stephentoub, lemillermicrosoft, dmytrostruk informed: Copy Chapter

Refactor to support generic LLM request settings

Context and Problem Statement Copy Chapter

The Semantic Kernel abstractions package includes a number of classes (CompleteRequestSettings, ChatRequestSettings, PromptTemplateConfig.CompletionConfig) which are used to support:

1. Passing LLM request settings when invoking an AI service
2. Deserialization of LLM requesting settings when loading the config.json associated with a Semantic Function

The problem with these classes is they include OpenAI specific properties only. A developer can only pass OpenAI specific requesting settings which means:

1. Settings may be passed that have no effect e.g., passing MaxTokens to Huggingface
2. Settings that do not overlap with the OpenAI properties cannot be sent e.g., Oobabooga supports additional parameters e.g., do_sample, typical_p, ...

Link to issue raised by the implementer of the Oobabooga AI service: https://github.com/microsoft/semantic-kernel/issues/2735

Decision Drivers Copy Chapter

• Semantic Kernel abstractions must be AI Service agnostic i.e., remove OpenAI specific properties.
• Solution must continue to support loading Semantic Function configuration (which includes AI request settings) from config.json.
• Provide good experience for developers e.g., must be able to program with type safety, intellisense, etc.
• Provide a good experience for implementors of AI services i.e., should be clear how to define the appropriate AI Request Settings abstraction for the service they are supporting.
• Semantic Kernel implementation and sample code should avoid specifying OpenAI specific request settings in code that is intended to be used with multiple AI services.
• Semantic Kernel implementation and sample code must be clear if an implementation is intended to be OpenAI specific.

Considered Options Copy Chapter

• Use dynamic to pass request settings
• Use object to pass request settings
• Define a base class for AI request settings which all implementations must extend

Note: Using generics was discounted during an earlier investigation which Dmytro conducted.

Decision Outcome Copy Chapter

Proposed: Define a base class for AI request settings which all implementations must extend.

Pros and Cons of the Options Copy Chapter

Use `dynamic` to pass request settings Copy Section

The IChatCompletion interface would look like this:

public interface IChatCompletion : IAIService
{
    ChatHistory CreateNewChat(string? instructions = null);

    Task> GetChatCompletionsAsync(
        ChatHistory chat,
        dynamic? requestSettings = null,
        CancellationToken cancellationToken = default);

    IAsyncEnumerable GetStreamingChatCompletionsAsync(
        ChatHistory chat,
        dynamic? requestSettings = null,
        CancellationToken cancellationToken = default);
}

Developers would have the following options to specify the requesting settings for a semantic function:

// Option 1: Use an anonymous type
await kernel.InvokeSemanticFunctionAsync("Hello AI, what can you do for me?", requestSettings: new { MaxTokens = 256, Temperature = 0.7 });

// Option 2: Use an OpenAI specific class
await kernel.InvokeSemanticFunctionAsync(prompt, requestSettings: new OpenAIRequestSettings() { MaxTokens = 256, Temperature = 0.7 });

// Option 3: Load prompt template configuration from a JSON payload
string configPayload = @"{
    ""schema"": 1,
    ""description"": ""Say hello to an AI"",
    ""type"": ""completion"",
    ""completion"": {
        ""max_tokens"": 60,
        ""temperature"": 0.5,
        ""top_p"": 0.0,
        ""presence_penalty"": 0.0,
        ""frequency_penalty"": 0.0
    }
}";
var templateConfig = JsonSerializer.Deserialize(configPayload);
var func = kernel.CreateSemanticFunction(prompt, config: templateConfig!, "HelloAI");
await kernel.RunAsync(func);

PR: https://github.com/microsoft/semantic-kernel/pull/2807

• Good, SK abstractions contain no references to OpenAI specific request settings
• Neutral, because anonymous types can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., temperature or combine properties for different AI services e.g., max_tokens (OpenAI) and max_new_tokens (Oobabooga).
• Bad, because it's not clear to developers what they should pass when creating a semantic function
• Bad, because it's not clear to implementors of a chat/text completion service what they should accept or how to add service specific properties.
• Bad, there is no compiler type checking for code paths where the dynamic argument has not been resolved which will impact code quality. Type issues manifest as RuntimeBinderException's and may be difficult to troubleshoot. Special care needs to be taken with return types e.g., may be necessary to specify an explicit type rather than just var again to avoid errors such as Microsoft.CSharp.RuntimeBinder.RuntimeBinderException : Cannot apply indexing with [] to an expression of type 'object'

Use `object` to pass request settings Copy Section

The IChatCompletion interface would look like this:

public interface IChatCompletion : IAIService
{
    ChatHistory CreateNewChat(string? instructions = null);

    Task> GetChatCompletionsAsync(
        ChatHistory chat,
        object? requestSettings = null,
        CancellationToken cancellationToken = default);

    IAsyncEnumerable GetStreamingChatCompletionsAsync(
        ChatHistory chat,
        object? requestSettings = null,
        CancellationToken cancellationToken = default);
}

The calling pattern is the same as for the dynamic case i.e. use either an anonymous type, an AI service specific class e.g., OpenAIRequestSettings or load from JSON.

PR: https://github.com/microsoft/semantic-kernel/pull/2819

• Good, SK abstractions contain no references to OpenAI specific request settings
• Neutral, because anonymous types can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., temperature or combine properties for different AI services e.g., max_tokens (OpenAI) and max_new_tokens (Oobabooga).
• Bad, because it's not clear to developers what they should pass when creating a semantic function
• Bad, because it's not clear to implementors of a chat/text completion service what they should accept or how to add service specific properties.
• Bad, code is needed to perform type checks and explicit casts. The situation is slightly better than for the dynamic case.

Define a base class for AI request settings which all implementations must extend Copy Section

The IChatCompletion interface would look like this:

public interface IChatCompletion : IAIService
{
    ChatHistory CreateNewChat(string? instructions = null);

    Task> GetChatCompletionsAsync(
        ChatHistory chat,
        AIRequestSettings? requestSettings = null,
        CancellationToken cancellationToken = default);

    IAsyncEnumerable GetStreamingChatCompletionsAsync(
        ChatHistory chat,
        AIRequestSettings? requestSettings = null,
        CancellationToken cancellationToken = default);
}

AIRequestSettings is defined as follows:

public class AIRequestSettings
{
    /// 
    /// Service identifier.
    /// 
    [JsonPropertyName("service_id")]
    [JsonPropertyOrder(1)]
    public string? ServiceId { get; set; } = null;

    /// 
    /// Extra properties
    /// 
    [JsonExtensionData]
    public Dictionary<string, object>? ExtensionData { get; set; }
}

Developers would have the following options to specify the requesting settings for a semantic function:

// Option 1: Invoke the semantic function and pass an OpenAI specific instance
var result = await kernel.InvokeSemanticFunctionAsync(prompt, requestSettings: new OpenAIRequestSettings() { MaxTokens = 256, Temperature = 0.7 });
Console.WriteLine(result.Result);

// Option 2: Load prompt template configuration from a JSON payload
string configPayload = @"{
    ""schema"": 1,
    ""description"": ""Say hello to an AI"",
    ""type"": ""completion"",
    ""completion"": {
        ""max_tokens"": 60,
        ""temperature"": 0.5,
        ""top_p"": 0.0,
        ""presence_penalty"": 0.0,
        ""frequency_penalty"": 0.0
        }
}";
var templateConfig = JsonSerializer.Deserialize(configPayload);
var func = kernel.CreateSemanticFunction(prompt, config: templateConfig!, "HelloAI");

await kernel.RunAsync(func);

It would also be possible to use the following pattern:

this._summarizeConversationFunction = kernel.CreateSemanticFunction(
    SemanticFunctionConstants.SummarizeConversationDefinition,
    skillName: nameof(ConversationSummarySkill),
    requestSettings: new AIRequestSettings()
    {
        ExtensionData = new Dictionary<string, object>()
        {
            { "Temperature", 0.1 },
            { "TopP", 0.5 },
            { "MaxTokens", MaxTokens }
        }
    });

The caveat with this pattern is, assuming a more specific implementation of AIRequestSettings uses JSON serialization/deserialization to hydrate an instance from the base AIRequestSettings, this will only work if all properties are supported by the default JsonConverter e.g.,

• If we have MyAIRequestSettings which includes a Uri property. The implementation of MyAIRequestSettings would make sure to load a URI converter so that it can serialize/deserialize the settings correctly.
• If the settings for MyAIRequestSettings are sent to an AI service which relies on the default JsonConverter then a NotSupportedException exception will be thrown.

PR: https://github.com/microsoft/semantic-kernel/pull/2829

• Good, SK abstractions contain no references to OpenAI specific request settings
• Good, because it is clear to developers what they should pass when creating a semantic function and it is easy to discover what service specific request setting implementations exist.
• Good, because it is clear to implementors of a chat/text completion service what they should accept and how to extend the base abstraction to add service specific properties.
• Neutral, because ExtensionData can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., temperature or combine properties for different AI services e.g., max_tokens (OpenAI) and max_new_tokens (Oobabooga).

File: docs/decisions/0009-support-multiple-named-args-in-template-function-calls.md Copy Chapter

These are optional elements. Feel free to remove any of them.

status: accepted contact: dmytrostruk date: 2013-06-16 deciders: shawncal, hario90 consulted: dmytrostruk, matthewbolanos informed: lemillermicrosoft Copy Chapter

Add support for multiple named arguments in template function calls

Context and Problem Statement Copy Chapter

Native functions now support multiple parameters, populated from context values with the same name. Semantic functions currently only support calling native functions with no more than 1 argument. The purpose of these changes is to add support for calling native functions within semantic functions with multiple named arguments.

Decision Drivers Copy Chapter

• Parity with Guidance
• Readability
• Similarity to languages familiar to SK developers
• YAML compatibility

Considered Options Copy Chapter

Syntax idea 1: Using commas Copy Section

{{Skill.MyFunction street: "123 Main St", zip: "98123", city:"Seattle", age: 25}}

Pros:

• Commas could make longer function calls easier to read, especially if spaces before and after the arg separator (a colon in this case) are allowed.

Cons:

• Guidance doesn't use commas
• Spaces are already used as delimiters elsewhere so the added complexity of supporting commas isn't necessary

Syntax idea 2: JavaScript/C#-Style delimiter (colon) Copy Section

{{MyFunction street:"123 Main St" zip:"98123" city:"Seattle" age: "25"}}

Pros:

• Resembles JavaScript Object syntax and C# named argument syntax

Cons:

• Doesn't align with Guidance syntax which uses equal signs as arg part delimiters
• Too similar to YAML key/value pairs if we support YAML prompts in the future. It's likely possible to support colons as delimiters but would be better to have a separator that is distinct from normal YAML syntax.

Syntax idea 3: Python/Guidance-Style delimiter Copy Section

{{MyFunction street="123 Main St" zip="98123" city="Seattle"}}

Pros:

• Resembles Python's keyword argument syntax
• Resembles Guidance's named argument syntax
• Not too similar to YAML key/value pairs if we support YAML prompts in the future.

Cons:

• Doesn't align with C# syntax

Syntax idea 4: Allow whitespace between arg name/value delimiter Copy Section

{{MyFunction street="123 Main St" zip="98123" city="Seattle"}}

Pros:

• Follows the convention followed by many programming languages of whitespace flexibility where spaces, tabs, and newlines within code don't impact a program's functionality

Cons:

• Promotes code that is harder to read unless commas can be used (see Using Commas)
• More complexity to support
• Doesn't align with Guidance which doesn't support spaces before and after the = sign.

Decision Outcome Copy Chapter

Chosen options: "Syntax idea 3: Python/Guidance-Style keyword arguments", because it aligns well with Guidance's syntax and is the most compatible with YAML and "Syntax idea 4: Allow whitespace between arg name/value delimiter" for more flexible developer experience.

Additional decisions:

• Continue supporting up to 1 positional argument for backward compatibility. Currently, the argument passed to a function is assumed to be the $input context variable.

Example

{{MyFunction "inputVal" street="123 Main St" zip="98123" city="Seattle"}}

• Allow arg values to be defined as strings or variables ONLY, e.g.

{{MyFunction street=$street zip="98123" city="Seattle"}}

If function expects a value other than a string for an argument, the SDK will use the corresponding TypeConverter to parse the string provided when evaluating the expression.

2. Official Technical Reference & Guides (microsoft/microsoft.github.io) Copy Chapter

[microsoft.github.io](https://microsoft.github.io)

Many open source projects released by Microsoft products, services and teams also have
deployments powered by GitHub Pages on this domain.

To discover Microsoft-released open source projects, learn about Microsoft's open source
approach and other information, check us out online at opensource.microsoft.com.

If we can help answer questions about the open source program at Microsoft, you can
contact opensource at microsoft com - although we are not able to provide support
for projects through this mechanism.

Microsoft GitHub resources Copy Chapter

• Default repo templates including SECURITY.MD and code of conduct files: microsoft/repo-templates
• Code of Conduct (FAQ)

Microsoft open source resources Copy Chapter

• Open Source Blog
• The Open Source Show on Channel 9
• https://opensource.microsoft.com/

Resources for Microsoft employees Copy Chapter

• docs.opensource.microsoft.com
• Link your GitHub Account and join GitHub organizations automatically

How to identify an official Microsoft organization on GitHub Copy Chapter

• Official Microsoft GitHub orgs have a professional avatar and include a web address at Microsoft.com or an associated official domain
• Official Microsoft orgs are Verified on GitHub - look for the domain verification indicator on GitHub