### Index

# Workflow Core

Workflow Core is a light weight workflow engine targeting .NET Standard.  Think: long running processes with multiple tasks that need to track state.  It supports pluggable persistence and concurrency providers to allow for multi-node clusters.

## Installing

Install the NuGet package "WorkflowCore"

Using nuget
```
PM> Install-Package WorkflowCore
```

Using .net cli
```
dotnet add package WorkflowCore
```

## Fluent API

Define workflows with the fluent API.

```c#
public class MyWorkflow : IWorkflow
{
    public void Build(IWorkflowBuilder<MyData> builder)
    {    
        builder
           .StartWith<Task1>()
           .Then<Task2>()
           .Then<Task3>;
    }
}
```

---

### Getting Started

# Basic Concepts

## Steps

A workflow consists of a series of connected steps.  Each step can have inputs and produce outputs that can be passed back to the workflow within which it exists.

Steps are defined by creating a class that inherits from the `StepBody` or `StepBodyAsync` abstract classes and implementing the Run/RunAsync method.  They can also be created inline while defining the workflow structure.

### First we define some steps

```C#
public class HelloWorld : StepBody
{
    public override ExecutionResult Run(IStepExecutionContext context)
    {
        Console.WriteLine("Hello world");
        return ExecutionResult.Next();
    }
}
```
*The `StepBody` and `StepBodyAsync` class implementations are constructed by the workflow host which first tries to use IServiceProvider for dependency injection, if it can't construct it with this method, it will search for a parameterless constructor*

### Then we define the workflow structure by composing a chain of steps.  This is done by implementing the IWorkflow interface

```C#
public class HelloWorldWorkflow : IWorkflow
{
    public string Id => "HelloWorld";
    public int Version => 1;

    public void Build(IWorkflowBuilder<object> builder)
    {
        builder
            .StartWith<HelloWorld>()
            .Then<GoodbyeWorld>();
    }  
}
```
The *IWorkflow* interface also has a readonly Id property and readonly Version property.  These are used by the workflow host to identify a workflow definition.

This workflow implemented in JSON would look like this
```json
{
  "Id": "HelloWorld",
  "Version": 1,
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "Bye"
    },        
    {
      "Id": "Bye",
      "StepType": "MyApp.GoodbyeWorld, MyApp"
    }
  ]
}
```


### You can also define your steps inline

```C#
public class HelloWorldWorkflow : IWorkflow
{
    public string Id => "HelloWorld";
    public int Version => 1;

    public void Build(IWorkflowBuilder<object> builder)
    {
        builder
            .StartWith(context =>
            {
                Console.WriteLine("Hello world");
                return ExecutionResult.Next();
            })
            .Then(context =>
            {
                Console.WriteLine("Goodbye world");
                return ExecutionResult.Next();
            });
    }
}
```

Each running workflow is persisted to the chosen persistence provider between each step, where it can be picked up at a later point in time to continue execution.  The outcome result of your step can instruct the workflow host to defer further execution of the workflow until a future point in time or in response to an external event.

## Host

The workflow host is the service responsible for executing workflows.  It does this by polling the persistence provider for workflow instances that are ready to run, executes them and then passes them back to the persistence provider to by stored for the next time they are run.  It is also responsible for publishing events to any workflows that may be waiting on one.

### Setup

Use the *AddWorkflow* extension method for *IServiceCollection* to configure the workflow host upon startup of your application.
By default, it is configured with *MemoryPersistenceProvider* and *SingleNodeConcurrencyProvider* for testing purposes.  You can also configure a DB persistence provider at this point.

```C#
services.AddWorkflow();
```

### Usage

When your application starts, grab the workflow host from the built-in dependency injection framework *IServiceProvider*.  Make sure you call *RegisterWorkflow*, so that the workflow host knows about all your workflows, and then call *Start()* to fire up the thread pool that executes workflows.  Use the *StartWorkflow* method to initiate a new instance of a particular workflow.


```C#
var host = serviceProvider.GetService<IWorkflowHost>();            
host.RegisterWorkflow<HelloWorldWorkflow>();
host.Start();

host.StartWorkflow("HelloWorld", 1, null);

Console.ReadLine();
host.Stop();
```

## Passing data between steps

Each step is intended to be a black-box, therefore they support inputs and outputs.  These inputs and outputs can be mapped to a data class that defines the custom data relevant to each workflow instance.

The following sample shows how to define inputs and outputs on a step, it then shows how define a workflow with a typed class for internal data and how to map the inputs and outputs to properties on the custom data class.

```C#
//Our workflow step with inputs and outputs
public class AddNumbers : StepBody
{
    public int Input1 { get; set; }

    public int Input2 { get; set; }

    public int Output { get; set; }

    public override ExecutionResult Run(IStepExecutionContext context)
    {
        Output = (Input1 + Input2);
        return ExecutionResult.Next();
    }
}

//Our class to define the internal data of our workflow
public class MyDataClass
{
    public int Value1 { get; set; }
    public int Value2 { get; set; }
    public int Answer { get; set; }
}

//Our workflow definition with strongly typed internal data and mapped inputs & outputs
public class PassingDataWorkflow : IWorkflow<MyDataClass>
{  
    public void Build(IWorkflowBuilder<MyDataClass> builder)
    {
        builder            
            .StartWith<AddNumbers>()
                .Input(step => step.Input1, data => data.Value1)
                .Input(step => step.Input2, data => data.Value2)
                .Output(data => data.Answer, step => step.Output)
            .Then<CustomMessage>()
                .Input(step => step.Message, data => "The answer is " + data.Answer.ToString());
    }
    ...
}

```

or in jSON format
```json
{
  "Id": "AddWorkflow",
  "Version": 1,
  "DataType": "MyApp.MyDataClass, MyApp",
  "Steps": [
	{
      "Id": "Add",
      "StepType": "MyApp.AddNumbers, MyApp",
      "NextStepId": "ShowResult",
      "Inputs": { 
          "Input1": "data.Value1",
          "Input2": "data.Value2" 
       },
      "Outputs": { 
          "Answer": "step.Output" 
      }
    },    
    {
      "Id": "ShowResult",
      "StepType": "MyApp.CustomMessage, MyApp",
      "Inputs": { 
          "Message": "\"The answer is \" + data.Answer" 
       }
    }
  ]
}
```

or in YAML format
```yaml
Id: AddWorkflow
Version: 1
DataType: MyApp.MyDataClass, MyApp
Steps:
- Id: Add
  StepType: MyApp.AddNumbers, MyApp
  NextStepId: ShowResult
  Inputs:
    Input1: data.Value1
    Input2: data.Value2
  Outputs:
    Answer: step.Output
- Id: ShowResult
  StepType: MyApp.CustomMessage, MyApp
  Inputs:
    Message: '"The answer is " + data.Answer'
```


## Injecting dependencies into steps

If you register your step classes with the IoC container, the workflow host will use the IoC container to construct them and therefore inject any required dependencies.  This example illustrates the use of dependency injection for workflow steps.

Consider the following service

```C#
public interface IMyService
{
    void DoTheThings();
}
...
public class MyService : IMyService
{
    public void DoTheThings()
    {
        Console.WriteLine("Doing stuff...");
    }
}
```

Which is consumed by a workflow step as follows

```C#
public class DoSomething : StepBody
{
    private IMyService _myService;

    public DoSomething(IMyService myService)
    {
        _myService = myService;
    }

    public override ExecutionResult Run(IStepExecutionContext context)
    {
        _myService.DoTheThings();
        return ExecutionResult.Next();
    }
}
```

Simply add both the service and the workflow step as transients to the service collection when setting up your IoC container.
(Avoid registering steps as singletons, since multiple concurrent workflows may need to use them at once.)

```C#
IServiceCollection services = new ServiceCollection();
services.AddLogging();
services.AddWorkflow();
            
services.AddTransient<DoSomething>();
services.AddTransient<IMyService, MyService>();
```

---

### Wip/Multiple Outcomes

### Multiple outcomes / forking

A workflow can take a different path depending on the outcomes of preceeding steps.  The following example shows a process where first a random number of 0 or 1 is generated and is the outcome of the first step.  Then, depending on the outcome value, the workflow will either fork to (TaskA + TaskB) or (TaskC + TaskD)

```C#
public class MultipleOutcomeWorkflow : IWorkflow
{
    public void Build(IWorkflowBuilder<object> builder)
    {
        builder
            .StartWith<RandomOutput>(x => x.Name("Random Step"))
                .When(data => 0).Do(then => then
                    .StartWith<TaskA>()
                    .Then<TaskB>())
                .When(data => 1).Do(then => then
                    .StartWith<TaskC>()
                    .Then<TaskD>())
            .Then<SayGoodbye>();
    }
}
```

---

### Wip/Steps Deep

The first time a particular step within the workflow is called, the PersistenceData property on the context object is *null*.  The ExecutionResult produced by the Run method can either cause the workflow to proceed to the next step by providing an outcome value, instruct the workflow to sleep for a defined period or simply not move the workflow forward.  If no outcome value is produced, then the step becomes re-entrant by setting PersistenceData, so the workflow host will call this step again in the future buy will populate the PersistenceData with it's previous value.

For example, this step will initially run with *null* PersistenceData and put the workflow to sleep for 12 hours, while setting the PersistenceData to *new Object()*.  12 hours later, the step will be called again but context.PersistenceData will now contain the object constructed in the previous iteration, and will now produce an outcome value of *null*, causing the workflow to move forward.

```C#
public class SleepStep : StepBody
{
    public override ExecutionResult Run(IStepExecutionContext context)
    {
        if (context.PersistenceData == null)
            return ExecutionResult.Sleep(Timespan.FromHours(12), new Object());
        else
            return ExecutionResult.Next();
    }
}
```

---

### Activities

# Activities

An activity is defined as an item on an external queue of work, that a workflow can wait for.

In this example the workflow will wait for `activity-1`, before proceeding.  It also passes the value of `data.Value1` to the activity, it then maps the result of the activity to `data.Value2`.

Then we create a worker to process the queue of activity items.  It uses the `GetPendingActivity` method to get an activity and the data that a workflow is waiting for.


```C#
public class ActivityWorkflow : IWorkflow<MyData>
{
    public void Build(IWorkflowBuilder<MyData> builder)
    {
        builder                
            .StartWith<HelloWorld>()
            .Activity("activity-1", (data) => data.Value1)
                .Output(data => data.Value2, step => step.Result)
            .Then<PrintMessage>()
                .Input(step => step.Message, data => data.Value2);
    }
             
}
...

var activity = host.GetPendingActivity("activity-1", "worker1", TimeSpan.FromMinutes(1)).Result;

if (activity != null)
{
    Console.WriteLine(activity.Parameters);
    host.SubmitActivitySuccess(activity.Token, "Some response data");
}

```

The JSON representation of this step would look like this

```json
{
    "Id": "activity-step",
    "StepType": "WorkflowCore.Primitives.Activity, WorkflowCore",
    "Inputs": 
    {
        "ActivityName": "\"activity-1\"",
        "Parameters": "data.Value1" 
    },
    "Outputs": { "Value2": "step.Result" }
}
```

## JSON / YAML API

The `Activity` step can be configured using inputs as follows

| Field                  | Description                 |
| ---------------------- | --------------------------- |
| CancelCondition        | Optional expression to specify a cancel condition  |
| Inputs.ActivityName    | Expression to specify the activity name            |
| Inputs.Parameters      | Expression to specify the parameters to pass the activity worker                |
| Inputs.EffectiveDate   | Optional expression to specify the effective date  |


```json
{
    "Id": "MyActivityStep",
    "StepType": "WorkflowCore.Primitives.Activity, WorkflowCore",
    "NextStepId": "...",
    "CancelCondition": "...",
    "Inputs": {
        "ActivityName": "\"my-activity\"",
        "Parameters": "data.SomeValue"
    }
}
```
```yaml
Id: MyActivityStep
StepType: WorkflowCore.Primitives.Activity, WorkflowCore
NextStepId: "..."
CancelCondition: "..."
Inputs:
  ActivityName: '"my-activity"'
  EventKey: '"Key1"'
  Parameters: data.SomeValue

```

---

### Azure Ai Foundry

# Azure AI Foundry Extension

The Azure AI Foundry extension enables building AI-powered, agentic workflows with WorkflowCore. It provides workflow steps for LLM invocation, automatic tool execution, embeddings, vector search, and human-in-the-loop review patterns.

## Installation

```bash
dotnet add package WorkflowCore.AI.AzureFoundry
```

## Overview

This extension adds six new workflow step types:

| Step | Description |
|------|-------------|
| `ChatCompletion` | Invoke LLMs with conversation history |
| `AgentLoop` | Agentic workflows with automatic tool calling |
| `ExecuteTool` | Manual tool execution |
| `GenerateEmbedding` | Create vector embeddings |
| `VectorSearch` | Semantic search with Azure AI Search |
| `HumanReview` | Pause for human approval |

## Configuration

### Basic Setup

```csharp
services.AddWorkflow();

services.AddAzureFoundry(options =>
{
    options.Endpoint = "https://myresource.services.ai.azure.com";
    options.ApiKey = "your-api-key";
    options.DefaultModel = "gpt-4o";
});
```

### Configuration Options

| Option | Type | Description |
|--------|------|-------------|
| `Endpoint` | string | Azure AI Foundry endpoint URL |
| `ApiKey` | string | API key for authentication |
| `Credential` | TokenCredential | Azure AD credential (alternative to ApiKey) |
| `DefaultModel` | string | Default LLM model name |
| `DefaultEmbeddingModel` | string | Default embedding model |
| `DefaultTemperature` | float | Default creativity level (0-1) |
| `DefaultMaxTokens` | int | Default response token limit |
| `SearchEndpoint` | string | Azure AI Search endpoint (optional) |
| `SearchApiKey` | string | Azure AI Search API key (optional) |

## Chat Completion

The simplest way to invoke an LLM in your workflow:

```csharp
public class SimpleChatWorkflow : IWorkflow<ChatData>
{
    public void Build(IWorkflowBuilder<ChatData> builder)
    {
        builder
            .StartWith(context => ExecutionResult.Next())
            .ChatCompletion(cfg => cfg
                .SystemPrompt("You are a helpful assistant")
                .UserMessage(data => data.Question)
                .OutputTo(data => data.Answer));
    }
}
```

### With Conversation History

Enable multi-turn conversations:

```csharp
.ChatCompletion(cfg => cfg
    .SystemPrompt("You are a helpful assistant")
    .UserMessage(data => data.Question)
    .WithHistory()  // Maintains conversation context
    .OutputTo(data => data.Answer));
```

## Agentic Workflows

The `AgentLoop` step enables autonomous AI agents that can use tools to accomplish tasks:

```csharp
public class SupportAgentWorkflow : IWorkflow<SupportData>
{
    public void Build(IWorkflowBuilder<SupportData> builder)
    {
        builder
            .StartWith(context => ExecutionResult.Next())
            .AgentLoop(cfg => cfg
                .SystemPrompt(@"You are a customer support agent.
                    Use the available tools to help customers.
                    Always search the knowledge base before answering.")
                .Message(data => data.CustomerQuery)
                .WithTool<SearchKnowledgeBase>()
                .WithTool<CreateTicket>()
                .WithTool<SendEmail>()
                .MaxIterations(10)
                .OutputTo(data => data.Response));
    }
}
```

### How Agent Loop Works

1. The LLM receives the user message and tool definitions
2. If the LLM decides to use a tool, it returns a tool call request
3. The step executes the tool and feeds the result back to the LLM
4. This continues until the LLM provides a final response (or max iterations)

```
User Message → LLM → Tool Call → Tool Execution → Result → LLM → ... → Final Response
```

## Creating Tools

Tools extend the LLM's capabilities by allowing it to take actions:

```csharp
public class SearchKnowledgeBase : IAgentTool
{
    private readonly IKnowledgeBaseService _kb;

    public SearchKnowledgeBase(IKnowledgeBaseService kb)
    {
        _kb = kb;
    }

    public string Name => "search_knowledge_base";
    
    public string Description => 
        "Search the knowledge base for articles matching the query";
    
    public string ParametersSchema => @"{
        ""type"": ""object"",
        ""properties"": {
            ""query"": { 
                ""type"": ""string"", 
                ""description"": ""Search query"" 
            },
            ""category"": { 
                ""type"": ""string"", 
                ""description"": ""Optional category filter"" 
            }
        },
        ""required"": [""query""]
    }";

    public async Task<ToolResult> ExecuteAsync(
        string toolCallId, 
        string arguments, 
        CancellationToken ct)
    {
        var args = JsonSerializer.Deserialize<SearchArgs>(arguments);
        var results = await _kb.SearchAsync(args.Query, args.Category, ct);
        
        if (results.Any())
        {
            return ToolResult.Succeeded(
                toolCallId, 
                Name, 
                JsonSerializer.Serialize(results));
        }
        
        return ToolResult.Succeeded(
            toolCallId, 
            Name, 
            "No articles found matching the query.");
    }
}
```

### Registering Tools

```csharp
// In your DI setup
services.AddSingleton<SearchKnowledgeBase>();
services.AddSingleton<CreateTicket>();

// After building service provider
var toolRegistry = serviceProvider.GetRequiredService<IToolRegistry>();
toolRegistry.Register(serviceProvider.GetRequiredService<SearchKnowledgeBase>());
toolRegistry.Register(serviceProvider.GetRequiredService<CreateTicket>());
```

## Human-in-the-Loop

For workflows requiring human oversight of AI outputs:

```csharp
public class ContentReviewWorkflow : IWorkflow<ContentData>
{
    public void Build(IWorkflowBuilder<ContentData> builder)
    {
        builder
            .StartWith(context => ExecutionResult.Next())
            
            // Generate content with AI
            .ChatCompletion(cfg => cfg
                .SystemPrompt("Generate marketing copy for the product")
                .UserMessage(data => data.ProductDescription)
                .OutputTo(data => data.DraftContent))
            
            // Human reviews before publishing
            .HumanReview(cfg => cfg
                .Content(data => data.DraftContent)
                .Reviewer(data => data.AssignedEditor)
                .Prompt("Review this AI-generated marketing copy")
                .OnApproved(data => data.ApprovedContent)
                .OnDecision(data => data.ReviewDecision))
            
            // Continue based on decision
            .If(data => data.ReviewDecision == ReviewDecision.Approved)
                .Do(then => then
                    .Then<PublishContent>()
                        .Input(step => step.Content, data => data.ApprovedContent));
    }
}
```

### Getting the Event Key

There are two ways to get the event key for completing a review:

**Option 1: Use the workflow ID (simplest)**

By default, if you don't provide a `CorrelationId`, the event key equals the workflow ID:

```csharp
// Start workflow
var workflowId = await host.StartWorkflow("ContentReview", data);

// Later, complete the review using workflowId as the event key
await host.PublishEvent("HumanReview", workflowId, reviewAction);
```

**Option 2: Use a custom correlation ID**

Provide your own correlation ID (e.g., a ticket ID, request ID) for easier integration:

```csharp
// In your workflow
.HumanReview(cfg => cfg
    .Content(data => data.DraftContent)
    .CorrelationId(data => data.TicketId)  // Use your own ID
    .OnApproved(data => data.ApprovedContent))

// Complete the review using your known ID
await host.PublishEvent("HumanReview", "TICKET-12345", reviewAction);
```

**Option 3: Capture the event key in workflow data**

Output the event key to your workflow data for later use:

```csharp
.HumanReview(cfg => cfg
    .Content(data => data.DraftContent)
    .OnEventKey(data => data.ReviewEventKey)  // Capture the key
    .OnApproved(data => data.ApprovedContent))
```

### Completing Reviews

From your UI or API, publish an event to complete the review:

```csharp
await workflowHost.PublishEvent(
    "HumanReview",
    eventKey,  // The workflow ID, custom correlation ID, or captured event key
    new ReviewAction
    {
        Decision = ReviewDecision.Approved,
        Reviewer = "editor@example.com",
        Comments = "Approved with minor edits",
        ModifiedContent = "Updated content..."  // Optional, for modifications
    });
```

## RAG (Retrieval-Augmented Generation)

Combine vector search with LLM generation for knowledge-grounded responses:

```csharp
public class RAGWorkflow : IWorkflow<RAGData>
{
    public void Build(IWorkflowBuilder<RAGData> builder)
    {
        builder
            .StartWith(context => ExecutionResult.Next())
            
            // Search for relevant documents
            .VectorSearch(cfg => cfg
                .Input(s => s.Query, data => data.UserQuestion)
                .Input(s => s.IndexName, data => "company-docs")
                .Input(s => s.TopK, data => 5)
                .Output(s => s.Results, data => data.RelevantDocs))
            
            // Generate answer grounded in documents
            .ChatCompletion(cfg => cfg
                .SystemPrompt(data => $@"Answer based on these documents:
                    {string.Join("\n", data.RelevantDocs.Select(d => d.Content))}
                    If the answer isn't in the documents, say so.")
                .UserMessage(data => data.UserQuestion)
                .OutputTo(data => data.Answer));
    }
}
```

## Embeddings

Generate embeddings for semantic search or similarity:

```csharp
.GenerateEmbedding(cfg => cfg
    .Input(s => s.Text, data => data.Document)
    .Output(s => s.Embedding, data => data.DocumentVector));
```

## Authentication

### API Key (Simplest)

```csharp
options.ApiKey = Environment.GetEnvironmentVariable("AZURE_AI_API_KEY");
```

### Managed Identity (Production)

```csharp
options.Credential = new ManagedIdentityCredential();
```

### Service Principal

```csharp
options.Credential = new ClientSecretCredential(
    tenantId: "your-tenant-id",
    clientId: "your-client-id",
    clientSecret: "your-client-secret"
);
```

## Best Practices

### 1. Set Iteration Limits

Always set `MaxIterations` on `AgentLoop` to prevent runaway costs:

```csharp
.AgentLoop(cfg => cfg
    .MaxIterations(10)  // Stop after 10 LLM calls
    ...);
```

### 2. Write Clear Tool Descriptions

The LLM uses descriptions to decide when to use tools:

```csharp
// ❌ Bad
public string Description => "Gets weather";

// ✅ Good  
public string Description => 
    "Get the current weather conditions for a specific city. " +
    "Returns temperature, humidity, and conditions.";
```

### 3. Use System Prompts Effectively

Guide the agent's behavior with clear instructions:

```csharp
.AgentLoop(cfg => cfg
    .SystemPrompt(@"You are a customer support agent.
        
        Guidelines:
        1. Always be polite and professional
        2. Search the knowledge base before answering
        3. If you can't help, create a support ticket
        4. Never share sensitive customer data")
    ...);
```

### 4. Track Token Usage

Monitor costs by tracking token consumption:

```csharp
.ChatCompletion(cfg => cfg
    ...
    .OutputTokensTo(data => data.TokensUsed));

// In your application
logger.LogInformation("Request used {Tokens} tokens", data.TokensUsed);
```

### 5. Handle Tool Errors Gracefully

Return meaningful error messages from tools:

```csharp
public async Task<ToolResult> ExecuteAsync(...)
{
    try
    {
        var result = await DoWork();
        return ToolResult.Succeeded(id, Name, result);
    }
    catch (NotFoundException)
    {
        return ToolResult.Succeeded(id, Name, 
            "No results found. Try a different search query.");
    }
    catch (Exception ex)
    {
        logger.LogError(ex, "Tool execution failed");
        return ToolResult.Failed(id, Name, 
            "An error occurred. Please try again.");
    }
}
```

## Samples

See the [sample project](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample.AzureFoundry) for complete working examples.

## Troubleshooting

### 404 Resource Not Found

Ensure your endpoint ends correctly:
- Azure AI Foundry: `https://resource.services.ai.azure.com`
- The extension automatically appends `/models` to the endpoint

### Authentication Errors

1. Verify your API key or credentials
2. Check that your Azure AD app has the required permissions
3. For managed identity, ensure the identity has access to the AI resource

### Tool Not Being Called

1. Check the tool description is clear about when to use it
2. Verify the tool is registered in the `IToolRegistry`
3. Check the tool's `ParametersSchema` is valid JSON Schema

---

### Control Structures

# Control Structures

## Decision Branches

You can define multiple independent branches within your workflow and select one based on an expression value.

#### Fluent API

For the fluent API, we define our branches with the `CreateBranch()` method on the workflow builder.  We can then select a branch using the `Branch` method.

The select expressions will be matched to the branch listed via the `Branch` method, and the matching next step(s) will be scheduled to execute next.  Matching multiple next steps will result in parallel branches running.

This workflow will select `branch1` if the value of `data.Value1` is `one`, and `branch2` if it is `two`.
```c#
var branch1 = builder.CreateBranch()
    .StartWith<PrintMessage>()
        .Input(step => step.Message, data => "hi from 1")
    .Then<PrintMessage>()
        .Input(step => step.Message, data => "bye from 1");

var branch2 = builder.CreateBranch()
    .StartWith<PrintMessage>()
        .Input(step => step.Message, data => "hi from 2")
    .Then<PrintMessage>()
        .Input(step => step.Message, data => "bye from 2");


builder
    .StartWith<HelloWorld>()
    .Decide(data => data.Value1)
        .Branch((data, outcome) => data.Value1 == "one", branch1)
        .Branch((data, outcome) => data.Value1 == "two", branch2);
```

#### JSON / YAML API

Hook up your branches via the `SelectNextStep` property, instead of a `NextStepId`.  The expressions will be matched to the step Ids listed in `SelectNextStep`, and the matching next step(s) will be scheduled to execute next.

```json
{
  "Id": "DecisionWorkflow",
  "Version": 1,
  "DataType": "MyApp.MyData, MyApp",
  "Steps": [
    {
      "Id": "decide",
      "StepType": "...",
      "SelectNextStep":
      {
        "Branch1": "<<result expression to match for branch 1>>",
        "Branch2": "<<result expression to match for branch 2>>"
      }
    },
    {
      "Id": "Branch1",
      "StepType": "MyApp.PrintMessage, MyApp",
      "Inputs": 
	  { 
		  "Message": "\"Hello from 1\"" 
	  }
    },
    {
      "Id": "Branch2",
      "StepType": "MyApp.PrintMessage, MyApp",
      "Inputs": 
	  { 
	    "Message": "\"Hello from 2\"" 
	  }
    }
  ]
}
```

```yaml
Id: DecisionWorkflow
Version: 1
DataType: MyApp.MyData, MyApp
Steps:
- Id: decide
  StepType: WorkflowCore.Primitives.Decide, WorkflowCore
  Inputs:
    Expression: <<input expression to evaluate>>
  OutcomeSteps:
    Branch1: '<<result expression to match for branch 1>>'
    Branch2: '<<result expression to match for branch 2>>'
- Id: Branch1
  StepType: MyApp.PrintMessage, MyApp
  Inputs:
    Message: '"Hello from 1"'
- Id: Branch2
  StepType: MyApp.PrintMessage, MyApp
  Inputs:
    Message: '"Hello from 2"'
```


## Parallel ForEach

Use the .ForEach method to start a parallel for loop

#### Fluent API

```C#
public class ForEachWorkflow : IWorkflow
{
    public string Id => "Foreach";
    public int Version => 1;

    public void Build(IWorkflowBuilder<object> builder)
    {
        builder
            .StartWith<SayHello>()
            .ForEach(data => new List<int>() { 1, 2, 3, 4 })
                .Do(x => x
                    .StartWith<DisplayContext>()
                        .Input(step => step.Message, (data, context) => context.Item)
                    .Then<DoSomething>())
            .Then<SayGoodbye>();
    }        
}
```

#### JSON / YAML API

```json
{
  "Id": "MyForEachStep",
  "StepType": "WorkflowCore.Primitives.ForEach, WorkflowCore",
  "NextStepId": "...",
  "Inputs": { "Collection": "<<expression to evaluate>>" },
  "Do": [[
      {
        "Id": "do1",
        "StepType": "MyApp.DoSomething1, MyApp",
        "NextStepId": "do2"
      },
      {
        "Id": "do2",
        "StepType": "MyApp.DoSomething2, MyApp"
      }
  ]]
}
```
```yaml
Id: MyForEachStep
StepType: WorkflowCore.Primitives.ForEach, WorkflowCore
NextStepId: "..."
Inputs:
  Collection: "<<expression to evaluate>>"
Do:
- - Id: do1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: do2
  - Id: do2
    StepType: MyApp.DoSomething2, MyApp
```


## While Loops

Use the .While method to start a while construct

#### Fluent API

```C#
public class WhileWorkflow : IWorkflow<MyData>
{
    public string Id => "While";
    public int Version => 1;

    public void Build(IWorkflowBuilder<MyData> builder)
    {
        builder
            .StartWith<SayHello>()
            .While(data => data.Counter < 3)
                .Do(x => x
                    .StartWith<DoSomething>()
                    .Then<IncrementStep>()
                        .Input(step => step.Value1, data => data.Counter)
                        .Output(data => data.Counter, step => step.Value2))
            .Then<SayGoodbye>();
    }        
}
```

#### JSON / YAML API

```json
{
  "Id": "MyWhileStep",
  "StepType": "WorkflowCore.Primitives.While, WorkflowCore",
  "NextStepId": "...",
  "Inputs": { "Condition": "<<expression to evaluate>>" },
  "Do": [[
      {
        "Id": "do1",
        "StepType": "MyApp.DoSomething1, MyApp",
        "NextStepId": "do2"
      },
      {
        "Id": "do2",
        "StepType": "MyApp.DoSomething2, MyApp"
      }
  ]]
}
```
```yaml
Id: MyWhileStep
StepType: WorkflowCore.Primitives.While, WorkflowCore
NextStepId: "..."
Inputs:
  Condition: "<<expression to evaluate>>"
Do:
- - Id: do1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: do2
  - Id: do2
    StepType: MyApp.DoSomething2, MyApp

```


## If Conditions

Use the .If method to start an if condition

#### Fluent API

```C#
public class IfWorkflow : IWorkflow<MyData>
{ 
    public void Build(IWorkflowBuilder<MyData> builder)
    {
        builder
            .StartWith<SayHello>()
            .If(data => data.Counter < 3).Do(then => then
                .StartWith<PrintMessage>()
                    .Input(step => step.Message, data => "Value is less than 3")
            )
            .If(data => data.Counter < 5).Do(then => then
                .StartWith<PrintMessage>()
                    .Input(step => step.Message, data => "Value is less than 5")
            )
            .Then<SayGoodbye>();
    }        
}
```

#### JSON / YAML API

```json
{
    "Id": "MyIfStep",
    "StepType": "WorkflowCore.Primitives.If, WorkflowCore",
    "NextStepId": "...",
    "Inputs": { "Condition": "<<expression to evaluate>>" },
    "Do": [[
        {
          "Id": "do1",
          "StepType": "MyApp.DoSomething1, MyApp",
          "NextStepId": "do2"
        },
        {
          "Id": "do2",
          "StepType": "MyApp.DoSomething2, MyApp"
        }
    ]]
}
```
```yaml
Id: MyIfStep
StepType: WorkflowCore.Primitives.If, WorkflowCore
NextStepId: "..."
Inputs:
  Condition: "<<expression to evaluate>>"
Do:
- - Id: do1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: do2
  - Id: do2
    StepType: MyApp.DoSomething2, MyApp

```


## Parallel Paths

Use the .Parallel() method to branch parallel tasks

#### Fluent API

```C#
public class ParallelWorkflow : IWorkflow<MyData>
{
    public string Id => "parallel-sample";
    public int Version => 1;

    public void Build(IWorkflowBuilder<MyData> builder)
    {
        builder
            .StartWith<SayHello>()
            .Parallel()
                .Do(then => 
                    then.StartWith<Task1dot1>()
                        .Then<Task1dot2>()
                .Do(then =>
                    then.StartWith<Task2dot1>()
                        .Then<Task2dot2>()
                .Do(then =>
                    then.StartWith<Task3dot1>()
                        .Then<Task3dot2>()
            .Join()
            .Then<SayGoodbye>();
    }        
}
```

#### JSON / YAML API

```json
{
      "Id": "MyParallelStep",
      "StepType": "WorkflowCore.Primitives.Sequence, WorkflowCore",
      "NextStepId": "...",
      "Do": [
		[ 
		  {
		    "Id": "Branch1.Step1",
		    "StepType": "MyApp.DoSomething1, MyApp",
		    "NextStepId": "Branch1.Step2"
		  },
		  {
		    "Id": "Branch1.Step2",
		    "StepType": "MyApp.DoSomething2, MyApp"
		  }
		],			
		[ 
		  {
		    "Id": "Branch2.Step1",
		    "StepType": "MyApp.DoSomething1, MyApp",
		    "NextStepId": "Branch2.Step2"
		  },
		  {
		    "Id": "Branch2.Step2",
		    "StepType": "MyApp.DoSomething2, MyApp"
		  }
		]
	  ]
}
```
```yaml
Id: MyParallelStep
StepType: WorkflowCore.Primitives.Sequence, WorkflowCore
NextStepId: "..."
Do:
- - Id: Branch1.Step1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: Branch1.Step2
  - Id: Branch1.Step2
    StepType: MyApp.DoSomething2, MyApp
- - Id: Branch2.Step1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: Branch2.Step2
  - Id: Branch2.Step2
    StepType: MyApp.DoSomething2, MyApp
```


## Schedule

Use `.Schedule` to register a future set of steps to run asynchronously in the background within your workflow.

#### Fluent API

```c#
builder
    .StartWith(context => Console.WriteLine("Hello"))
    .Schedule(data => TimeSpan.FromSeconds(5)).Do(schedule => schedule
        .StartWith(context => Console.WriteLine("Doing scheduled tasks"))
    )
    .Then(context => Console.WriteLine("Doing normal tasks"));
```

#### JSON / YAML API

```json
{
  "Id": "MyScheduleStep",
  "StepType": "WorkflowCore.Primitives.Schedule, WorkflowCore",
  "Inputs": { "Interval": "<<expression to evaluate>>" },
  "Do": [[
      {
        "Id": "do1",
        "StepType": "MyApp.DoSomething1, MyApp",
        "NextStepId": "do2"
      },
      {
        "Id": "do2",
        "StepType": "MyApp.DoSomething2, MyApp"
      }
  ]]
}
```
```yaml
Id: MyScheduleStep
StepType: WorkflowCore.Primitives.Schedule, WorkflowCore
Inputs:
  Interval: "<<expression to evaluate>>"
Do:
- - Id: do1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: do2
  - Id: do2
    StepType: MyApp.DoSomething2, MyApp
```


### Delay

The `Delay` step will pause the current branch of your workflow for a specified period.

#### JSON / YAML API

```json
{
  "Id": "MyDelayStep",
  "StepType": "WorkflowCore.Primitives.Delay, WorkflowCore",
  "NextStepId": "...",
  "Inputs": { "Period": "<<expression to evaluate>>" }
}
```
```yaml
Id: MyDelayStep
StepType: WorkflowCore.Primitives.Delay, WorkflowCore
NextStepId: "..."
Inputs:
  Period: "<<expression to evaluate>>"
```


## Recur

Use `.Recur` to setup a set of recurring background steps within your workflow, until a certain condition is met

#### Fluent API

```c#
builder
    .StartWith(context => Console.WriteLine("Hello"))
    .Recur(data => TimeSpan.FromSeconds(5), data => data.Counter > 5).Do(recur => recur
        .StartWith(context => Console.WriteLine("Doing recurring task"))
    )
    .Then(context => Console.WriteLine("Carry on"));
```

#### JSON / YAML API

```json
{
  "Id": "MyScheduleStep",
  "StepType": "WorkflowCore.Primitives.Recur, WorkflowCore",
  "Inputs": { 
    "Interval": "<<expression to evaluate>>",
    "StopCondition": "<<expression to evaluate>>" 
  },
  "Do": [[
      {
        "Id": "do1",
        "StepType": "MyApp.DoSomething1, MyApp",
        "NextStepId": "do2"
      },
      {
        "Id": "do2",
        "StepType": "MyApp.DoSomething2, MyApp"
      }
  ]]
}
```
```yaml
Id: MyScheduleStep
StepType: WorkflowCore.Primitives.Recur, WorkflowCore
Inputs:
  Interval: "<<expression to evaluate>>"
  StopCondition: "<<expression to evaluate>>"
Do:
- - Id: do1
    StepType: MyApp.DoSomething1, MyApp
    NextStepId: do2
  - Id: do2
    StepType: MyApp.DoSomething2, MyApp
```

---

### Elastic Search

# Elasticsearch plugin for Workflow Core

A search index plugin for Workflow Core backed by Elasticsearch, enabling you to index your workflows and search against the data and state of them.

## Installing

Install the NuGet package "WorkflowCore.Providers.Elasticsearch"

Using Nuget package console
```
PM> Install-Package WorkflowCore.Providers.Elasticsearch
```

Using .NET CLI
```
dotnet add package WorkflowCore.Providers.Elasticsearch
```


## Configuration

Use the `.UseElasticsearch` extension method on `IServiceCollection` when building your service provider

```
using Nest;
...
services.AddWorkflow(cfg =>
{
	...
	cfg.UseElasticsearch(new ConnectionSettings(new Uri("http://localhost:9200")), "index_name");
});
```

## Usage

Inject the `ISearchIndex` service into your code and use the `Search` method.

```
Search(string terms, int skip, int take, params SearchFilter[] filters)
```

#### terms

A whitespace separated string of search terms, an empty string will match everything.
This will do a full text search on the following default fields
 * Reference
 * Description
 * Status
 * Workflow Definition

 In addition you can search data within your own custom data object if it implements `ISearchable`

 ```
 using WorkflowCore.Interfaces;
 ...
 public class MyData : ISearchable
{
    public string StrValue1 { get; set; }
    public string StrValue2 { get; set; }

    public IEnumerable<string> GetSearchTokens()
    {
        return new List<string>()
        {
            StrValue1,
            StrValue2
        };    
    }
}
 ```

 ##### Examples

 Search all fields for "puppies"
 ```
 searchIndex.Search("puppies", 0, 10);
 ```

#### skip & take

Use `skip` and `take` to page your search results.  Where `skip` is the result number to start from and `take` is the page size.

#### filters

You can also supply a list of filters to apply to the search, these can be applied to both the standard fields as well as any field within your custom data objects.
There is no need to implement `ISearchable` on your data object in order to use filters against it.

The following filter types are available
 * ScalarFilter
 * DateRangeFilter
 * NumericRangeFilter
 * StatusFilter

 These exist in the `WorkflowCore.Models.Search` namespace.

 ##### Examples

 Filtering by reference
 ```
 using WorkflowCore.Models.Search;
 ...

 searchIndex.Search("", 0, 10, ScalarFilter.Equals(x => x.Reference, "My Reference"));
 ```

 Filtering by workflows started after a date
 ```
 searchIndex.Search("", 0, 10, DateRangeFilter.After(x => x.CreateTime, startDate));
 ```

 Filtering by workflows completed within a period
 ```
 searchIndex.Search("", 0, 10, DateRangeFilter.Between(x => x.CompleteTime, startDate, endDate));
 ```

 Filtering by workflows in a state
 ```
 searchIndex.Search("", 0, 10, StatusFilter.Equals(WorkflowStatus.Complete));
 ```

 Filtering against your own custom data class
 ```

 class MyData
 {
	public string Value1 { get; set; }
	public int Value2 { get; set; }
 }

 searchIndex.Search("", 0, 10, ScalarFilter.Equals<MyData>(x => x.Value1, "blue moon"));
 searchIndex.Search("", 0, 10, NumericRangeFilter.LessThan<MyData>(x => x.Value2, 5))
 ```

---

### Enhanced Test Reporting

# Enhanced Test Reporting for GitHub Actions

This document explains the enhanced test reporting capabilities that have been added to the GitHub Actions workflow.

## Overview

The GitHub Actions workflow has been enhanced to provide detailed, individual test results for all test suites in the Workflow Core project. This addresses the requirement to see detailed, individual test results from tests run by GitHub workflows.

## Key Enhancements

### 1. Detailed Test Output
- **Enhanced Verbosity**: Changed from `--verbosity normal` to `--verbosity detailed`
- **Detailed Console Logging**: Added `--logger "console;verbosity=detailed"` for comprehensive console output
- **Individual Test Results**: Each test case now shows its execution status, duration, and any error details

### 2. TRX Test Result Files
- **TRX Format**: Added `--logger "trx;LogFileName={TestSuite}.trx"` to generate XML test result files
- **Structured Data**: TRX files contain structured test data including:
  - Test names and fully qualified names
  - Test outcomes (Passed, Failed, Skipped)
  - Execution times and durations
  - Error messages and stack traces for failed tests
  - Test categories and traits

### 3. GitHub Actions Test Reporting
- **Test Reporter Integration**: Added `dorny/test-reporter@v1` action to display test results in the GitHub UI
- **PR Integration**: Test results are automatically displayed in pull request checks
- **Visual Test Summary**: Failed tests are highlighted with detailed error information
- **Test Status Annotations**: Test results appear as GitHub Actions annotations

### 4. Test Result Artifacts
- **Downloadable Results**: Test result files are uploaded as artifacts for each job
- **Persistent Storage**: Test results are available for download even after workflow completion
- **Individual Job Results**: Each test suite (Unit, Integration, MongoDB, etc.) has separate artifacts

## What You'll See

### In GitHub Actions Logs
Before (old format):
```
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Passed!  - Failed:     0, Passed:    25, Skipped:     0, Total:    25
```

After (enhanced format):
```
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

  Passed WorkflowCore.UnitTests.Services.ExecutionResultProcessorFixture.should_advance_workflow [< 1 ms]
  Passed WorkflowCore.UnitTests.Services.ExecutionResultProcessorFixture.should_branch_children [2 ms]
  Failed WorkflowCore.UnitTests.Services.SomeTest.example_failing_test [15 ms]
    Error Message:
     Assert.Equal() Failure
     Expected: True
     Actual:   False
    Stack Trace:
      at WorkflowCore.UnitTests.Services.SomeTest.example_failing_test() in /path/to/test.cs:line 42

Test Run Summary:
  Total tests: 25
    Passed: 24
    Failed: 1
    Skipped: 0
```

### In GitHub Pull Requests
- ✅ **Test Status Checks**: Clear pass/fail status for each test suite
- 📊 **Test Summary**: Number of passed, failed, and skipped tests
- 🔍 **Detailed Failure Information**: Click-through to see specific test failures
- 📁 **Downloadable Artifacts**: Access to complete test result files

### Available Artifacts
Each test job now produces downloadable artifacts:
- `unit-test-results`: Unit test TRX files and logs
- `integration-test-results`: Integration test TRX files and logs
- `mongodb-test-results`: MongoDB-specific test results
- `mysql-test-results`: MySQL-specific test results
- `postgresql-test-results`: PostgreSQL-specific test results
- `redis-test-results`: Redis-specific test results
- `sqlserver-test-results`: SQL Server-specific test results
- `elasticsearch-test-results`: Elasticsearch-specific test results
- `oracle-test-results`: Oracle-specific test results

## Benefits

1. **Individual Test Visibility**: See exactly which tests pass or fail
2. **Debugging Support**: Detailed error messages and stack traces
3. **Performance Monitoring**: Test execution times for performance analysis
4. **Historical Data**: Downloadable test results for trend analysis
5. **CI/CD Integration**: Better integration with GitHub's native test reporting features
6. **Developer Experience**: Faster identification of test issues in pull requests

## File Structure

After test execution, the following files are generated:
```
test-results/
├── UnitTests.trx
├── IntegrationTests.trx
├── MongoDBTests.trx
├── MySQLTests.trx
├── PostgreSQLTests.trx
├── RedisTests.trx
├── SQLServerTests.trx
├── ElasticsearchTests.trx
└── OracleTests.trx
```

Each TRX file contains detailed XML data about the test execution results that can be consumed by various reporting tools and integrated development environments.

---

### Error Handling

# Error handling

Each step can be configured with it's own error handling behavior, it can be retried at a later time, suspend the workflow or terminate the workflow.

### Fluent API

```C#
public void Build(IWorkflowBuilder<object> builder)
{
    builder                
        .StartWith<HelloWorld>()
            .OnError(WorkflowErrorHandling.Retry, TimeSpan.FromMinutes(10))
        .Then<GoodbyeWorld>();
}
```

### JSON / YAML API

ErrorBehavior

```json
{
    "Id": "...",
    "StepType": "...",
    "ErrorBehavior": "Retry / Suspend / Terminate / Compensate",
    "RetryInterval": "00:10:00"
}
```
```yaml
Id: "..."
StepType: "..."
ErrorBehavior: Retry / Suspend / Terminate / Compensate
RetryInterval: '00:10:00'
```

## Global Error handling

The WorkflowHost service also has a `.OnStepError` event which can be used to intercept exceptions from workflow steps on a more global level.

---

### Extensions

## Extensions

* [User (human) workflows](https://github.com/danielgerlag/workflow-core/tree/master/src/extensions/WorkflowCore.Users)
* [Azure AI Foundry](azure-ai-foundry.md) - AI-powered agentic workflows with LLM invocation, tool execution, and human-in-the-loop patterns

---

### External Events

# Events

A workflow can also wait for an external event before proceeding.  In the following example, the workflow will wait for an event called *"MyEvent"* with a key of *0*.  Once an external source has fired this event, the workflow will wake up and continue processing, passing the data generated by the event onto the next step.

```C#
public class EventSampleWorkflow : IWorkflow<MyDataClass>
{
    public void Build(IWorkflowBuilder<MyDataClass> builder)
    {
        builder
            .StartWith(context => ExecutionResult.Next())
            .WaitFor("MyEvent", data => "0")
                .Output(data => data.Value, step => step.EventData)
            .Then<CustomMessage>()
                .Input(step => step.Message, data => "The data from the event is " + data.Value);
    }
}
...
//External events are published via the host
//All workflows that have subscribed to MyEvent 0, will be passed "hello"
host.PublishEvent("MyEvent", "0", "hello");
```

## Effective Date

You can also specify an effective date when waiting for events, which allows you to respond to events that may have already occurred in the past, or only ones that occur after the effective date.


## JSON / YAML API

The `.WaitFor` can be implemented using inputs as follows

| Field                  | Description                 |
| ---------------------- | --------------------------- |
| CancelCondition        | Optional expression to specify a cancel condition  |
| Inputs.EventName       | Expression to specify the event name               |
| Inputs.EventKey        | Expression to specify the event key                |
| Inputs.EffectiveDate   | Optional expression to specify the effective date  |


```json
{
    "Id": "MyWaitStep",
    "StepType": "WorkflowCore.Primitives.WaitFor, WorkflowCore",
    "NextStepId": "...",
    "CancelCondition": "...",
    "Inputs": {
        "EventName": "\"Event1\"",
        "EventKey": "\"Key1\"",
        "EffectiveDate": "DateTime.Now"
    }
}
```
```yaml
Id: MyWaitStep
StepType: WorkflowCore.Primitives.WaitFor, WorkflowCore
NextStepId: "..."
CancelCondition: "..."
Inputs:
  EventName: '"Event1"'
  EventKey: '"Key1"'
  EffectiveDate: DateTime.Now

```

---

### Json Yaml

# Loading workflow definitions from JSON or YAML

Install the `WorkflowCore.DSL` package from nuget and call `AddWorkflowDSL` on your service collection.
Then grab the `DefinitionLoader` from the IoC container and call the `.LoadDefinition` method

```c#
using WorkflowCore.Interface;
...
var loader = serviceProvider.GetService<IDefinitionLoader>();
loader.LoadDefinition("<<json or yaml string here>>", Deserializers.Json);
```

## Common DSL

Both the JSON and YAML formats follow a common DSL, where step types within the workflow are referenced by the fully qualified class names.
Built-in step types typically live in the `WorklfowCore.Primitives` namespace.

| Field                   | Description                 |
| ----------------------- | --------------------------- |
| Id                      | Workflow Definition ID        |
| Version                 | Workflow Definition Version   |
| DataType                | Fully qualified assembly class name of the custom data object   |
| Steps[].Id              | Step ID (required unique key for each step)                     |
| Steps[].StepType        | Fully qualified assembly class name of the step                 |
| Steps[].NextStepId      | Step ID of the next step after this one completes               |
| Steps[].Inputs          | Optional Key/value pair of step inputs                          |
| Steps[].Outputs         | Optional Key/value pair of step outputs                         |
| Steps[].CancelCondition | Optional cancel condition                                       |

```json
{
  "Id": "HelloWorld",
  "Version": 1,
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "Bye"
    },        
    {
      "Id": "Bye",
      "StepType": "MyApp.GoodbyeWorld, MyApp"
    }
  ]
}
```
```yaml
Id: HelloWorld
Version: 1
Steps:
- Id: Hello
  StepType: MyApp.HelloWorld, MyApp
  NextStepId: Bye
- Id: Bye
  StepType: MyApp.GoodbyeWorld, MyApp
```

### Inputs and Outputs

Inputs and outputs can be bound to a step as a key/value pair object, 
* The `Inputs` collection, the key would match a property on the `Step` class and the value would be an expression with both the `data` and `context` parameters at your disposal.
* The `Outputs` collection, the key would match a property on the `Data` class and the value would be an expression with both the `step` as a parameter at your disposal.

Full details of the capabilities of  expression language can be found [here](https://github.com/StefH/System.Linq.Dynamic.Core/wiki/Dynamic-Expressions#expression-language)

```json
{
  "Id": "AddWorkflow",
  "Version": 1,
  "DataType": "MyApp.MyDataClass, MyApp",
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "Add"
    },
	{
      "Id": "Add",
      "StepType": "MyApp.AddNumbers, MyApp",
      "NextStepId": "Bye",
      "Inputs": { 
          "Value1": "data.Value1",
          "Value2": "data.Value2" 
       },
      "Outputs": { 
          "Answer": "step.Result" 
      }
    },    
    {
      "Id": "Bye",
      "StepType": "MyApp.GoodbyeWorld, MyApp"
    }
  ]
}
```
```yaml
Id: AddWorkflow
Version: 1
DataType: MyApp.MyDataClass, MyApp
Steps:
- Id: Hello
  StepType: MyApp.HelloWorld, MyApp
  NextStepId: Add
- Id: Add
  StepType: MyApp.AddNumbers, MyApp
  NextStepId: Bye
  Inputs:
    Value1: data.Value1
    Value2: data.Value2
  Outputs:
    Answer: step.Result
- Id: Bye
  StepType: MyApp.GoodbyeWorld, MyApp
```

```json
{
  "Id": "AddWorkflow",
  "Version": 1,
  "DataType": "MyApp.MyDataClass, MyApp",
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "Print"
    },
    {
      "Id": "Print",
      "StepType": "MyApp.PrintMessage, MyApp",
      "Inputs": { "Message": "\"Hi there!\"" }
    }
  ]
}
```
```yaml
Id: AddWorkflow
Version: 1
DataType: MyApp.MyDataClass, MyApp
Steps:
- Id: Hello
  StepType: MyApp.HelloWorld, MyApp
  NextStepId: Print
- Id: Print
  StepType: MyApp.PrintMessage, MyApp
  Inputs:
    Message: '"Hi there!"'

```

You can also pass object graphs to step inputs as opposed to just scalar values
```json
"inputs": 
{    
  "Body": {
      "Value1": 1,
      "Value2": 2
  },
  "Headers": {
      "Content-Type": "application/json"
  }
},
```

If you want to evaluate an expression for a given property of your object, simply prepend and `@` and pass an expression string
```json
"inputs": 
{    
  "Body": {
      "@Value1": "data.MyValue * 2",
      "Value2": 5
  },
  "Headers": {
      "Content-Type": "application/json"
  }
},
```

#### Enums

If your step has an enum property, you can just pass the string representation of the enum value and it will be automatically converted.

#### Environment variables available in input expressions

You can access environment variables from within input expressions.
usage:
```
environment["VARIABLE_NAME"]
```

---

### Multi Node Clusters

# Multi-node clusters

By default, the WorkflowHost service will run as a single node using the built-in queue and locking providers for a single node configuration.  Should you wish to run a multi-node cluster, you will need to configure an external queueing mechanism and a distributed lock manager to co-ordinate the cluster.  These are the providers that are currently available.

## Queue Providers

* SingleNodeQueueProvider *(Default built-in provider)*
* [Azure Storage Queues](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Azure)
* [Redis](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Redis)
* [RabbitMQ](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.QueueProviders.RabbitMQ)
* [AWS Simple Queue Service](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.AWS)


## Distributed lock managers

* SingleNodeLockProvider *(Default built-in provider)*
* [Azure Storage Leases](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Azure)
* [Redis](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Redis)
* [AWS DynamoDB](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.AWS)

---

### Performance

# Performance Test

Workflow-core version 3.7.0 was put under test to evaluate its performance. The setup used was single node with the default MemoryPersistenceProvider persistence provider.

## Methodology

- Test Environment - Test were run on following two environments one after the other to see how workflow-core performance with a lower vs higher hardware configuration.
  - Lower configuration
    - Cores: 8 vCPU ([Standard_D8s_v3](https://learn.microsoft.com/azure/virtual-machines/dv3-dsv3-series))
    - RAM: 32 GB
    - OS: Linux Ubuntu 20.04
    - dotNet 6
  - Higher configuration
    - Cores: 32 vCPU ([Standard_D32as_v4](https://learn.microsoft.com/azure/virtual-machines/dav4-dasv4-series))
    - RAM: 128 GB
    - OS: Linux Ubuntu 20.04
    - dotNet 6
- Test Workflow: Workflow consist of 3 basic steps. These 3 simple steps were chosen to test the performance of the workflow engine with minimal yet sufficient complexity and to avoid any external dependencies.
  - Step1 : Generate a [random number](https://learn.microsoft.com/dotnet/api/system.random?view=net-6.0) between 1 to 10 and print it on standard output.
  - Step2 :  [Conditional step](https://github.com/danielgerlag/workflow-core/blob/master/docs/control-structures.md)
    - Step 2.1: If value generate in step1 is > 5 then print it on standard output.
    - Step 2.2: If value generate in step1 is <= 5 then print it on standard output.
  - Step3: Prints a good bye message on standard output.
- Test tools:
  - [NBomber](https://nbomber.com/docs/getting-started/overview/) was used as performance testing framework with C# console app as base.

- Test scenarios:
  - Each type of test run executed for 20 minutes.
  - NBomber Load Simulation of type [KeepConstant](https://nbomber.com/docs/using-nbomber/basic-api/load-simulation#keep-constant) copies was used. This type of simulation keep a constant amount of Scenario copies(instances) for a specific period.
  - Concurrent copies [1,2,3,4,5,6,7,8,10,12,14,16,32,64,128,256,512,1024] were tested.
    - For example if we take Concurrent copies=4 and Duration=20 minutes this means that NBomber will ensure that we have 4 instance of Test Workflow running in parallel for 20 minutes.

## Results

- Workflow per seconds - Below tables shows how many workflows we are able to execute per second on two different environment with increasing number of concurrent copies.

| **Concurrent Copies** | **8 vCPU** | **32 vCPU** |
| :-------------------: | :--------: | :---------: |
|         **1**         |   300.6    |    504.7    |
|         **2**         |   310.3    |    513.1    |
|         **3**         |   309.6    |    519.3    |
|         **4**         |   314.7    |    521.3    |
|         **5**         |   312.4    |    519.0    |
|         **6**         |   314.7    |    517.7    |
|         **7**         |   318.9    |    516.7    |
|         **8**         |   318.4    |    517.5    |
|        **10**         |   322.6    |    517.1    |
|        **12**         |   319.7    |    517.6    |
|        **14**         |   322.4    |    518.1    |
|        **16**         |   327.0    |    515.5    |
|        **32**         |   327.7    |    515.8    |
|        **64**         |   330.7    |    523.7    |
|        **128**        |   332.8    |    526.9    |
|        **256**        |   332.8    |    529.1    |
|        **512**        |   332.8    |    529.1    |
|       **1024**        |   341.3    |    529.1    |



- Latency - Shows Mean, P99 and P50 latency in milliseconds on two different environment with increasing number of concurrent copies.

| **Concurrent Copies** | **Mean 8 vCPU** | **Mean 32 vCPU** | **P.99 8 vCPU** | **P.99 32 vCPU** | **P.50 8 vCPU** | **P.50 32 vCPU** |
| :-------------------: | :-------------: | :--------------: | :-------------: | :--------------: | :-------------: | :--------------: |
|         **1**         |      3.32       |       1.98       |      12.67      |       2.49       |      3.13       |       1.85       |
|         **2**         |      6.43       |       3.89       |      19.96      |       5.67       |      6.17       |       3.65       |
|         **3**         |      9.67       |       5.77       |      24.96      |       8.2        |      9.14       |       5.46       |
|         **4**         |      12.7       |       7.76       |      27.44      |      13.57       |      12.02      |       7.22       |
|         **5**         |      15.99      |       9.63       |      34.59      |      41.89       |      15.14      |       9.08       |
|         **6**         |      19.05      |      11.58       |      38.69      |      45.92       |      18.02      |      10.93       |
|         **7**         |      21.94      |      13.54       |      42.18      |       48.9       |      20.72      |      12.66       |
|         **8**         |      25.11      |      15.45       |      44.35      |      51.04       |      23.92      |      14.54       |
|        **10**         |      30.98      |      19.33       |      52.29      |      56.64       |      29.31      |      18.21       |
|        **12**         |      37.52      |      23.18       |      59.2       |      63.33       |      35.42      |      21.82       |
|        **14**         |      43.44      |      27.01       |      67.33      |      67.58       |      41.28      |      25.55       |
|        **16**         |      48.93      |      31.03       |      72.06      |      72.77       |      46.11      |      28.93       |
|        **32**         |      97.65      |      62.03       |     130.05      |      104.96      |      94.91      |      58.02       |
|        **64**         |     193.53      |      122.24      |     235.14      |      168.45      |     191.49      |      115.26      |
|        **128**        |     384.63      |      243.74      |     449.79      |      294.65      |     379.65      |      236.67      |
|        **256**        |     769.13      |      486.82      |     834.07      |      561.66      |     766.46      |      498.22      |
|        **512**        |     1538.29     |      968.02      |     1725.44     |     1052.67      |     1542.14     |      962.05      |
|       **1024**        |     2999.36     |     1935.32      |     3219.46     |     2072.57      |     3086.34     |     1935.36      |



## References

- [NBomber](https://nbomber.com/docs/getting-started/overview/)

---

### Persistence

# Persistence

Since workflows are typically long running processes, they will need to be persisted to storage between steps.
There are several persistence providers available as separate Nuget packages.

* MemoryPersistenceProvider *(Default provider, for demo and testing purposes)*
* [MongoDB](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Persistence.MongoDB)
* [SQL Server](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Persistence.SqlServer)
* [PostgreSQL](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Persistence.PostgreSQL)
* [Sqlite](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Persistence.Sqlite)
* [Amazon DynamoDB](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.AWS)
* [Cosmos DB](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Azure)
* [Azure Table Storage](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Azure)
* [Redis](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Providers.Redis)
* [Oracle](https://github.com/danielgerlag/workflow-core/tree/master/src/providers/WorkflowCore.Persistence.Oracle)

## Implementing a custom persistence provider

To implement a custom persistence provider, create a class that implements `IPersistenceProvider` interface:

```csharp
public interface IPersistenceProvider : IWorkflowRepository, ISubscriptionRepository, IEventRepository, IScheduledCommandRepository
{        
    Task PersistErrors(IEnumerable<ExecutionError> errors, CancellationToken cancellationToken = default);
    void EnsureStoreExists();
}
```

The `IPersistenceProvider` interface combines four repository interfaces:

### IWorkflowRepository
Handles workflow instance storage and retrieval:
- `CreateNewWorkflow` - Create and store a new workflow instance
- `PersistWorkflow` - Update an existing workflow instance
- `GetWorkflowInstance` - Retrieve a specific workflow instance
- `GetRunnableInstances` - Get workflow instances ready for execution

### IEventRepository  
Manages workflow events:
- `CreateEvent` - Store a new event
- `GetEvent` - Retrieve a specific event
- `GetRunnableEvents` - Get events ready for processing
- `MarkEventProcessed/Unprocessed` - Update event status

### ISubscriptionRepository
Handles event subscriptions:
- `CreateEventSubscription` - Create new event subscription
- `GetSubscriptions` - Query subscriptions for events
- `TerminateSubscription` - Remove a subscription
- `SetSubscriptionToken/ClearSubscriptionToken` - Manage subscription locking

### IScheduledCommandRepository
For future command scheduling (optional):
- `ScheduleCommand` - Schedule a command for future execution
- `ProcessCommands` - Execute scheduled commands
- `SupportsScheduledCommands` - Indicates if provider supports this feature

Once implemented, register your provider:

```csharp
services.AddWorkflow(options => 
{
    options.UsePersistence(sp => new MyCustomPersistenceProvider());
});
```

---

### Sagas

# Saga transaction with compensation

A Saga allows you to encapsulate a sequence of steps within a saga transaction and specify compensation steps for each.

In the sample, `Task2` will throw an exception, then `UndoTask2` and `UndoTask1` will be triggered.

```c#
builder
    .StartWith(context => Console.WriteLine("Begin"))
    .Saga(saga => saga
        .StartWith<Task1>()
            .CompensateWith<UndoTask1>()
        .Then<Task2>()
            .CompensateWith<UndoTask2>()
        .Then<Task3>()
            .CompensateWith<UndoTask3>()
    )
        .CompensateWith<CleanUp>()
    .Then(context => Console.WriteLine("End"));
```

## Retry policy for failed saga transaction

This particular example will retry the saga every 5 seconds, but you could also simply fail completely, and process a master compensation task for the whole saga.

```c#
builder
    .StartWith(context => Console.WriteLine("Begin"))
    .Saga(saga => saga
        .StartWith<Task1>()
            .CompensateWith<UndoTask1>()
        .Then<Task2>()
            .CompensateWith<UndoTask2>()
        .Then<Task3>()
            .CompensateWith<UndoTask3>()
    )
    .OnError(Models.WorkflowErrorHandling.Retry, TimeSpan.FromSeconds(5))
    .Then(context => Console.WriteLine("End"));
```

## Compensate entire saga transaction

You could also only specify a master compensation step, as follows

```c#
builder
    .StartWith(context => Console.WriteLine("Begin"))
        .Saga(saga => saga
            .StartWith<Task1>()
            .Then<Task2>()
            .Then<Task3>()
    )
        .CompensateWith<UndoEverything>()
    .Then(context => Console.WriteLine("End"));
```

## Passing parameters to compensation steps

Parameters can be passed to a compensation step as follows

```c#
builder
    .StartWith<SayHello>()
        .CompensateWith<PrintMessage>(compensate => 
        {
            compensate.Input(step => step.Message, data => "undoing...");
        })
```

## Expressing a saga in JSON or YAML

A saga transaction can be expressed in JSON or YAML, by using the `WorkflowCore.Primitives.Sequence` step and setting the `Saga` parameter to `true`.

The compensation steps can be defined by specifying the `CompensateWith` parameter.

```json
{
  "Id": "Saga-Sample",
  "Version": 1,
  "DataType": "MyApp.MyDataClass, MyApp",
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "MySaga"
    },    
    {
      "Id": "MySaga",
      "StepType": "WorkflowCore.Primitives.Sequence, WorkflowCore",
      "NextStepId": "Bye",
      "Saga": true,
      "Do": [
        [
          {
            "Id": "do1",
            "StepType": "MyApp.Task1, MyApp",
            "NextStepId": "do2",
            "CompensateWith": [
              {
                "Id": "undo1",
                "StepType": "MyApp.UndoTask1, MyApp"
              }
            ]
          },
          {
            "Id": "do2",
            "StepType": "MyApp.Task2, MyApp",
            "CompensateWith": [
              {
                "Id": "undo2-1",
                "NextStepId": "undo2-2",
                "StepType": "MyApp.UndoTask2, MyApp"
              },
              {
                "Id": "undo2-2",
                "StepType": "MyApp.DoSomethingElse, MyApp"
              }
            ]
          }
        ]
      ]
    },    
    {
      "Id": "Bye",
      "StepType": "MyApp.GoodbyeWorld, MyApp"
    }
  ]
}
```

```yaml
Id: Saga-Sample
Version: 1
DataType: MyApp.MyDataClass, MyApp
Steps:
- Id: Hello
  StepType: MyApp.HelloWorld, MyApp
  NextStepId: MySaga
- Id: MySaga
  StepType: WorkflowCore.Primitives.Sequence, WorkflowCore
  NextStepId: Bye
  Saga: true
  Do:
  - - Id: do1
      StepType: MyApp.Task1, MyApp
      NextStepId: do2
      CompensateWith:
      - Id: undo1
        StepType: MyApp.UndoTask1, MyApp
    - Id: do2
      StepType: MyApp.Task2, MyApp
      CompensateWith:
      - Id: undo2-1
        NextStepId: undo2-2
        StepType: MyApp.UndoTask2, MyApp
      - Id: undo2-2
        StepType: MyApp.DoSomethingElse, MyApp
- Id: Bye
  StepType: MyApp.GoodbyeWorld, MyApp

```

---

### Samples

# Samples

[Hello World](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample01)

[Passing Data](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample03)

[Events](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample04)

[Activity Workers](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample18)

[Dependency Injection](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample15)

[Parallel ForEach](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample09)

[While loop](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample10)

[If](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample11)

[Parallel Tasks](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample13)

[Saga Transactions](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample17)

[Scheduled Background Tasks](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample16)

[Recurring Background Tasks](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample14)

[Multiple outcomes](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample12)

[Deferred execution & re-entrant steps](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample05)

[Looping](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample02)

[Exposing a REST API](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WebApiSample)

[Human(User) Workflow](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample08)

[Workflow Middleware](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample19)

## AI & Agentic Workflow Samples

[Azure AI Foundry - Chat, Agents & Tools](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample.AzureFoundry) - Interactive sample demonstrating:
  - Simple LLM chat completion
  - Agentic workflows with automatic tool execution (weather, calculator)
  - Human-in-the-loop approval workflows

---

### Test Helpers

# Test helpers for Workflow Core

Provides support writing tests for workflows built on WorkflowCore

## Installing

Install the NuGet package "WorkflowCore.Testing"

```
PM> Install-Package WorkflowCore.Testing
```

## Usage

### With xUnit

* Create a class that inherits from WorkflowTest
* Call the Setup() method in the constructor
* Implement your tests using the helper methods
	* StartWorkflow()
	* WaitForWorkflowToComplete()
	* WaitForEventSubscription()
	* GetStatus()
	* GetData()
	* UnhandledStepErrors

```C#
public class xUnitTest : WorkflowTest<MyWorkflow, MyDataClass>
{
    public xUnitTest()
    {
        Setup();
    }

    [Fact]
    public void MyWorkflow()
    {
        var workflowId = StartWorkflow(new MyDataClass() { Value1 = 2, Value2 = 3 });
        WaitForWorkflowToComplete(workflowId, TimeSpan.FromSeconds(30));

        GetStatus(workflowId).Should().Be(WorkflowStatus.Complete);
        UnhandledStepErrors.Count.Should().Be(0);
        GetData(workflowId).Value3.Should().Be(5);
    }
}
```


### With NUnit

* Create a class that inherits from WorkflowTest and decorate it with the *TestFixture* attribute
* Override the Setup method and decorate it with the *SetUp* attribute
* Implement your tests using the helper methods
	* StartWorkflow()
	* WaitForWorkflowToComplete()
	* WaitForEventSubscription()
	* GetStatus()
	* GetData()
	* UnhandledStepErrors

```C#
[TestFixture]
public class NUnitTest : WorkflowTest<MyWorkflow, MyDataClass>
{
    [SetUp]
    protected override void Setup()
    {
        base.Setup();
    }

    [Test]
    public void NUnit_workflow_test_sample()
    {
        var workflowId = StartWorkflow(new MyDataClass() { Value1 = 2, Value2 = 3 });
        WaitForWorkflowToComplete(workflowId, TimeSpan.FromSeconds(30));

        GetStatus(workflowId).Should().Be(WorkflowStatus.Complete);
        UnhandledStepErrors.Count.Should().Be(0);
        GetData(workflowId).Value3.Should().Be(5);
    }

}
```

---

### Using With Aspnet Core

# Using with ASP.NET Core
## How to configure within an ASP.NET Core application

In your startup class, use the `AddWorkflow` extension method to configure workflow core services, and then register your workflows and start the host when you configure the app.
```c#
public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }

    public IConfiguration Configuration { get; }

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddMvc();
        services.AddWorkflow(cfg =>
        {
            cfg.UseMongoDB(@"mongodb://mongo:27017", "workflow");
            cfg.UseElasticsearch(new ConnectionSettings(new Uri("http://elastic:9200")), "workflows");
        });
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseMvc();

        var host = app.ApplicationServices.GetService<IWorkflowHost>();
        host.RegisterWorkflow<TestWorkflow, MyDataClass>();
        host.Start();
    }
}
```

## Usage

Now simply inject the services you require into your controllers
* IWorkflowController
* IWorkflowHost
* ISearchIndex
* IPersistenceProvider

---

### Workflow Middleware

# Workflow Middleware

Workflows can be extended with Middleware that run before/after workflows start/complete as well as around workflow steps to provide flexibility in implementing cross-cutting concerns such as [log correlation](https://www.frakkingsweet.com/net-core-log-correlation-easy-access-to-headers/), [retries](https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/implement-http-call-retries-exponential-backoff-polly), and other use-cases.

This is done by implementing and registering `IWorkflowMiddleware` for workflows or `IWorkflowStepMiddleware` for steps.

## Step Middleware

Step middleware lets you run additional code around the execution of a given step and alter its behavior. Implementing a step middleware should look familiar to anyone familiar with [ASP.NET Core's middleware pipeline](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/middleware/?view=aspnetcore-3.1) or [`HttpClient`'s `DelegatingHandler` middleware](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/http-requests?view=aspnetcore-3.1#outgoing-request-middleware).

### Usage

First, create your own middleware class that implements `IWorkflowStepMiddleware`. Here's an example of a middleware that adds workflow ID and step ID to the log correlation context of every workflow step in your app.

**Important:** You must make sure to call `next()` as part of your middleware. If you do not do this, your step will never run.

```cs
public class LogCorrelationStepMiddleware : IWorkflowStepMiddleware
{
    private readonly ILogger<LogCorrelationStepMiddleware> _log;

    public LogCorrelationStepMiddleware(
        ILogger<LogCorrelationStepMiddleware> log)
    {
        _log = log;
    }

    public async Task<ExecutionResult> HandleAsync(
        IStepExecutionContext context,
        IStepBody body,
        WorkflowStepDelegate next)
    {
        var workflowId = context.Workflow.Id;
        var stepId = context.Step.Id;

        // Uses log scope to add a few attributes to the scope
        using (_log.BeginScope("{@WorkflowId}", workflowId))
        using (_log.BeginScope("{@StepId}", stepId))
        {
            // Calling next ensures step gets executed
            return await next();
        }
    }
}
```

Here's another example of a middleware that uses the [Polly](https://github.com/App-vNext/Polly) dotnet resiliency library to implement retries on workflow steps based off a custom retry policy.

```
/* Detailed source-code truncated for AI context efficiency. */
```

## Pre/Post Workflow Middleware

Workflow middleware run either before a workflow starts or after a workflow completes and can be used to hook into the workflow lifecycle or alter the workflow itself before it is started.

### Pre Workflow Middleware

These middleware get run before the workflow is started and can potentially alter properties on the `WorkflowInstance`.

The following example illustrates setting the `Description` property on the `WorkflowInstance` using a middleware that interprets the data on the passed workflow. This is useful in cases where you want the description of the workflow to be derived from the data passed to the workflow.

Note that you use `WorkflowMiddlewarePhase.PreWorkflow` to specify that it runs before the workflow starts.

**Important:** You should call `next` as part of the workflow middleware to ensure that the next workflow in the chain runs.

```cs
// AddDescriptionWorkflowMiddleware.cs
public class AddDescriptionWorkflowMiddleware : IWorkflowMiddleware
{
    public WorkflowMiddlewarePhase Phase =>
        WorkflowMiddlewarePhase.PreWorkflow;

    public Task HandleAsync(
        WorkflowInstance workflow,
        WorkflowDelegate next
    )
    {
        if (workflow.Data is IDescriptiveWorkflowParams descriptiveParams)
        {
            workflow.Description = descriptiveParams.Description;
        }

        return next();
    }
}

// IDescriptiveWorkflowParams.cs
public interface IDescriptiveWorkflowParams
{
    string Description { get; }
}

// MyWorkflowParams.cs
public MyWorkflowParams : IDescriptiveWorkflowParams
{
    public string Description => $"Run task '{TaskName}'";

    public string TaskName { get; set; }
}
```

### Exception Handling in Pre Workflow Middleware

Pre workflow middleware exception handling gets treated differently from post workflow middleware. Since the middleware runs before the workflow starts, any exceptions thrown within a pre workflow middleware will bubble up to the `StartWorkflow` method and it is up to the caller of `StartWorkflow` to handle the exception and act accordingly.

```cs
public async Task MyMethodThatStartsAWorkflow()
{
    try
    {
        await host.StartWorkflow("HelloWorld", 1, null);
    }
    catch(Exception ex)
    {
        // Handle the exception appropriately
    }
}
```

### Post Workflow Middleware

These middleware get run after the workflow has completed and can be used to perform additional actions for all workflows in your app.

The following example illustrates how you can use a post workflow middleware to print a summary of the workflow to console.

Note that you use `WorkflowMiddlewarePhase.PostWorkflow` to specify that it runs after the workflow completes.

**Important:** You should call `next` as part of the workflow middleware to ensure that the next workflow in the chain runs.

```cs
public class PrintWorkflowSummaryMiddleware : IWorkflowMiddleware
{
    private readonly ILogger<PrintWorkflowSummaryMiddleware> _log;

    public PrintWorkflowSummaryMiddleware(
        ILogger<PrintWorkflowSummaryMiddleware> log
    )
    {
        _log = log;
    }

    public WorkflowMiddlewarePhase Phase =>
        WorkflowMiddlewarePhase.PostWorkflow;

    public Task HandleAsync(
        WorkflowInstance workflow,
        WorkflowDelegate next
    )
    {
        if (!workflow.CompleteTime.HasValue)
        {
            return next();
        }

        var duration = workflow.CompleteTime.Value - workflow.CreateTime;
        _log.LogInformation($@"Workflow {workflow.Description} completed in {duration:g}");

        foreach (var step in workflow.ExecutionPointers)
        {
            var stepName = step.StepName;
            var stepDuration = (step.EndTime - step.StartTime) ?? TimeSpan.Zero;
            _log.LogInformation($"  - Step {stepName} completed in {stepDuration:g}");
        }

        return next();
    }
}
```

### Exception Handling in Post Workflow Middleware

Post workflow middleware exception handling gets treated differently from pre workflow middleware. At the time that the workflow completes, your workflow has ran already so an uncaught exception would be difficult to act on.

By default, if a workflow middleware throws an exception, it will be logged and the workflow will complete as normal. This behavior can be changed, however.

To override the default post workflow error handling for all workflows in your app, just register a new `IWorkflowMiddlewareErrorHandler` in the dependency injection framework with your custom behavior as follows.

```cs
// CustomMiddlewareErrorHandler.cs
public class CustomHandler : IWorkflowMiddlewareErrorHandler
{
    public Task HandleAsync(Exception ex)
    {
        // Handle your error asynchronously
    }
}

// Startup.cs
public void ConfigureServices(IServiceCollection services)
{
    // Other workflow configuration
    services.AddWorkflow();

    // Should go after .AddWorkflow()
    services.AddTransient<IWorkflowMiddlewareErrorHandler, CustomHandler>();
}
```

## Registering Middleware

In order for middleware to take effect, they must be registered with the built-in dependency injection framework using the convenience helpers.

**Note:** Middleware will be run in the order that they are registered with middleware that are registered earlier running earlier in the chain and finishing later in the chain. For pre/post workflow middleware, all pre middleware will be run before a workflow starts and all post middleware will be run after a workflow completes.

```cs
public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        ...

        // Add workflow middleware
        services.AddWorkflowMiddleware<AddDescriptionWorkflowMiddleware>();
        services.AddWorkflowMiddleware<PrintWorkflowSummaryMiddleware>();

        // Add step middleware
        services.AddWorkflowStepMiddleware<LogCorrelationStepMiddleware>();
        services.AddWorkflowStepMiddleware<PollyRetryMiddleware>();

        ...
    }
}
```

## More Information

See the [Workflow Middleware](https://github.com/danielgerlag/workflow-core/tree/master/src/samples/WorkflowCore.Sample19) sample for full examples of workflow middleware in action.

---

### README

# Workflow Core

[](https://ci.appveyor.com/project/danielgerlag/workflow-core)
[<img src="https://api.gitsponsors.com/api/badge/img?id=73864802" height="20">](https://api.gitsponsors.com/api/badge/link?p=xj6mObb7nZAJGyuABfd8nD5XWf3SE4oUfw0vmCgSiJeIfNlzJAej0FWX8oFdYm6D7bvZpCf6qANVBNPWid4dRQ==)

Workflow Core is a light weight embeddable workflow engine targeting .NET Standard.  Think: long running processes with multiple tasks that need to track state.  It supports pluggable persistence and concurrency providers to allow for multi-node clusters.

### Announcements

#### New related project: Conductor
Conductor is a stand-alone workflow server as opposed to a library that uses Workflow Core internally. It exposes an API that allows you to store workflow definitions, track running workflows, manage events and define custom steps and scripts for usage in your workflows.

https://github.com/danielgerlag/conductor

## Documentation

See [Tutorial here.](https://workflow-core.readthedocs.io)

## Fluent API

Define your workflows with the fluent API.

```c#
public class MyWorkflow : IWorkflow
{
    public void Build(IWorkflowBuilder<MyData> builder)
    {    
        builder
           .StartWith<Task1>()
           .Then<Task2>()
           .Then<Task3>();
    }
}
```

## JSON / YAML Workflow Definitions

Define your workflows in JSON or YAML, need to install WorkFlowCore.DSL

```json
{
  "Id": "HelloWorld",
  "Version": 1,
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "Bye"
    },        
    {
      "Id": "Bye",
      "StepType": "MyApp.GoodbyeWorld, MyApp"
    }
  ]
}
```

```yaml
Id: HelloWorld
Version: 1
Steps:
- Id: Hello
  StepType: MyApp.HelloWorld, MyApp
  NextStepId: Bye
- Id: Bye
  StepType: MyApp.GoodbyeWorld, MyApp
```

## Sample use cases

* New user workflow
```c#
public class MyData
{
	public string Email { get; set; }
	public string Password { get; set; }
	public string UserId { get; set; }
}

public class MyWorkflow : IWorkflow
{
    public void Build(IWorkflowBuilder<MyData> builder)
    {    
        builder
            .StartWith<CreateUser>()
                .Input(step => step.Email, data => data.Email)
                .Input(step => step.Password, data => data.Password)
                .Output(data => data.UserId, step => step.UserId)
           .Then<SendConfirmationEmail>()
               .WaitFor("confirmation", data => data.UserId)
           .Then<UpdateUser>()
               .Input(step => step.UserId, data => data.UserId);
    }
}
```

* Saga Transactions

```c#
public class MyWorkflow : IWorkflow
{
    public void Build(IWorkflowBuilder<MyData> builder)
    {    
        builder
            .StartWith<CreateCustomer>()
            .Then<PushToSalesforce>()
                .OnError(WorkflowErrorHandling.Retry, TimeSpan.FromMinutes(10))
            .Then<PushToERP>()
                .OnError(WorkflowErrorHandling.Retry, TimeSpan.FromMinutes(10));
    }
}
```

```c#
builder
    .StartWith<LogStart>()
    .Saga(saga => saga
        .StartWith<Task1>()
            .CompensateWith<UndoTask1>()
        .Then<Task2>()
            .CompensateWith<UndoTask2>()
        .Then<Task3>()
            .CompensateWith<UndoTask3>()
    )
    .OnError(Models.WorkflowErrorHandling.Retry, TimeSpan.FromMinutes(10))
    .Then<LogEnd>();
```

## Persistence

Since workflows are typically long running processes, they will need to be persisted to storage between steps.
There are several persistence providers available as separate Nuget packages.

* MemoryPersistenceProvider *(Default provider, for demo and testing purposes)*
* [MongoDB](src/providers/WorkflowCore.Persistence.MongoDB)
* [Cosmos DB](src/providers/WorkflowCore.Providers.Azure)
* [Amazon DynamoDB](src/providers/WorkflowCore.Providers.AWS)
* [SQL Server](src/providers/WorkflowCore.Persistence.SqlServer)
* [PostgreSQL](src/providers/WorkflowCore.Persistence.PostgreSQL)
* [Sqlite](src/providers/WorkflowCore.Persistence.Sqlite)
* [MySQL](src/providers/WorkflowCore.Persistence.MySQL)
* [Redis](src/providers/WorkflowCore.Providers.Redis)
* [Oracle](src/providers/WorkflowCore.Persistence.Oracle)

## Search

A search index provider can be plugged in to Workflow Core, enabling you to index your workflows and search against the data and state of them.
These are also available as separate Nuget packages.
* [Elasticsearch](src/providers/WorkflowCore.Providers.Elasticsearch)

## Extensions

* [Azure AI Foundry](src/extensions/WorkflowCore.AI.AzureFoundry)
* [User (human) workflows](src/extensions/WorkflowCore.Users)


## Samples

* [Hello World](src/samples/WorkflowCore.Sample01)

* [Multiple outcomes](src/samples/WorkflowCore.Sample12)

* [Passing Data](src/samples/WorkflowCore.Sample03)

* [Parallel ForEach](src/samples/WorkflowCore.Sample09)

* [Sync ForEach](src/samples/WorkflowCore.Sample09s)

* [While Loop](src/samples/WorkflowCore.Sample10)

* [If Statement](src/samples/WorkflowCore.Sample11)

* [Events](src/samples/WorkflowCore.Sample04)

* [Activity Workers](src/samples/WorkflowCore.Sample18)

* [Parallel Tasks](src/samples/WorkflowCore.Sample13)

* [Saga Transactions (with compensation)](src/samples/WorkflowCore.Sample17)

* [Scheduled Background Tasks](src/samples/WorkflowCore.Sample16)

* [Recurring Background Tasks](src/samples/WorkflowCore.Sample14)

* [Dependency Injection](src/samples/WorkflowCore.Sample15)

* [Deferred execution & re-entrant steps](src/samples/WorkflowCore.Sample05)

* [Human(User) Workflow](src/samples/WorkflowCore.Sample08)

* [Looping](src/samples/WorkflowCore.Sample02)

* [Exposing a REST API](src/samples/WebApiSample)

* [Testing](src/samples/WorkflowCore.TestSample01)


## Contributors

* **Daniel Gerlag** - *Initial work*
* **Jackie Ja**
* **Aaron Scribner**
* **Roberto Paterlini**

## Related Projects

* [Conductor](https://github.com/danielgerlag/conductor) (Stand-alone workflow server built on Workflow Core)

## Ports

* [JWorkflow (Java)](https://github.com/danielgerlag/jworkflow)
* [workflow-es (Node.js)](https://github.com/danielgerlag/workflow-es)
* [liteflow (Python)](https://github.com/danielgerlag/liteflow)

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

---