BenchmarkDotNet Documentation

Powerful .NET library for benchmarking

11,468 stars C# C# 1 file · ~7,759 tokens #benchmark#benchmarking#c-sharp#csharp#dotnet#hacktoberfest#performance
RAW Doc

1. Project Overview & Quickstart (dotnet/BenchmarkDotNet)

File: README.md

BenchmarkDotNet helps you to transform methods into benchmarks, track their performance, and share reproducible measurement experiments.
It's no harder than writing unit tests!
Under the hood, it performs a lot of magic that guarantees reliable and precise results thanks to the perfolizer and pragmastat statistical engine.
BenchmarkDotNet protects you from popular benchmarking mistakes and warns you if something is wrong with your benchmark design or obtained measurements.
The results are presented in a user-friendly form that highlights all the important facts about your experiment.
BenchmarkDotNet is already adopted by 27400+ GitHub projects including
.NET Runtime,
.NET Compiler,
.NET Performance,
and many others.

It's easy to start writing benchmarks, check out the following example
(copy-pastable version is here):

cs
[SimpleJob(RuntimeMoniker.Net472, baseline: true)]
[SimpleJob(RuntimeMoniker.NetCoreApp30)]
[SimpleJob(RuntimeMoniker.NativeAot70)]
[SimpleJob(RuntimeMoniker.Mono)]
[RPlotExporter]
public class Md5VsSha256
{
    private SHA256 sha256 = SHA256.Create();
    private MD5 md5 = MD5.Create();
    private byte[] data;

    [Params(1000, 10000)]
    public int N;

    [GlobalSetup]
    public void Setup()
    {
        data = new byte[N];
        new Random(42).NextBytes(data);
    }

    [Benchmark]
    public byte[] Sha256() => sha256.ComputeHash(data);

    [Benchmark]
    public byte[] Md5() => md5.ComputeHash(data);
}

BenchmarkDotNet automatically
runs the benchmarks on all the runtimes,
aggregates the measurements,
and prints a summary table with the most important information:

text
/* Detailed source-code truncated for AI context efficiency. */

The measured data can be exported to different formats (md, html, csv, xml, json, etc.) including plots:

Supported runtimes: .NET 5+, .NET Framework 4.6.1+, .NET Core 2.0+, Mono, NativeAOT
Supported languages: C#, F#, Visual Basic
Supported OS: Windows, Linux, macOS
Supported architectures: x86, x64, ARM, ARM64, Wasm and LoongArch64

Features

BenchmarkDotNet has tons of features that are essential in comprehensive performance investigations.
Four aspects define the design of these features:
simplicity, automation, reliability, and friendliness.

Simplicity

You shouldn't have to be an experienced performance engineer if you want to write benchmarks.
You can design very complicated performance experiments in the declarative style using simple APIs.

For example, if you want to parameterize your benchmark,
mark a field or a property with [Params(1, 2, 3)]: BenchmarkDotNet will enumerate all of the specified values
and run benchmarks for each case.
If you want to compare benchmarks with each other,
mark one of the benchmarks as the baseline
via [Benchmark(Baseline = true)]: BenchmarkDotNet will compare it with all of the other benchmarks.
If you want to compare performance in different environments, use jobs.
For example, you can run all the benchmarks on .NET 8.0 and Mono via
[SimpleJob(RuntimeMoniker.Net80)] and [SimpleJob(RuntimeMoniker.Mono)].

If you don't like attributes, you can call most of the APIs via the fluent style and write code like this:

cs
ManualConfig.CreateEmpty() // A configuration for our benchmarks
    .AddJob(Job.Default // Adding first job
        .WithRuntime(ClrRuntime.Net472) // .NET Framework 4.7.2
        .WithPlatform(Platform.X64) // Run as x64 application
        .WithJit(Jit.LegacyJit) // Use LegacyJIT instead of the default RyuJIT
        .WithGcServer(true) // Use Server GC
    ).AddJob(Job.Default // Adding second job
        .AsBaseline() // It will be marked as baseline
        .WithEnvironmentVariable("Key", "Value") // Setting an environment variable
        .WithWarmupCount(0) // Disable warm-up stage
    );

If you prefer command-line experience, you can configure your benchmarks via
the console arguments
in any console application (other types of applications are not supported).

Automation

Reliable benchmarks always include a lot of boilerplate code.

Let's think about what you should do in a typical case.
First, you should perform a pilot experiment and determine the best number of method invocations.
Next, you should execute several warm-up iterations and ensure that your benchmark achieved a steady state.
After that, you should execute the main iterations and calculate some basic statistics.
If you calculate some values in your benchmark, you should use it somehow to prevent dead code elimination.
If you use loops, you should care about the effect of the loop unrolling on your results
(which may depend on the processor architecture).
Once you get results, you should check for some special properties of the obtained performance distribution
like multimodality or extremely high outliers.
You should also evaluate the overhead of your infrastructure and deduct it from your results.
If you want to test several environments, you should perform the measurements in each of them and manually aggregate the results.

If you write this code from scratch, it's easy to make a mistake and spoil your measurements.
Note that it's a shortened version of the full checklist that you should follow during benchmarking:
there are a lot of additional hidden pitfalls that should be handled appropriately.
Fortunately, you shouldn't worry about it because
BenchmarkDotNet will perform this boring and time-consuming stuff for you.

Moreover, the library can help you with some advanced tasks that you may want to perform during the investigation.
For example,
BenchmarkDotNet can measure the managed and
native memory traffic
and print disassembly listings for your benchmarks.

Reliability

A lot of hand-written benchmarks produce wrong numbers that lead to incorrect business decisions.
BenchmarkDotNet protects you from most of the benchmarking pitfalls and allows achieving high measurement precision.

You shouldn't worry about the perfect number of method invocation, the number of warm-up and actual iterations:
BenchmarkDotNet tries to choose the best benchmarking parameters and
achieve a good trade-off between the measurement prevision and the total duration of all benchmark runs.
So, you shouldn't use any magic numbers (like "We should perform 100 iterations here"),
the library will do it for you based on the values of statistical metrics.

BenchmarkDotNet also prevents benchmarking of non-optimized assemblies that were built using DEBUG mode because
the corresponding results will be unreliable.
The library will print a warning if you have an attached debugger,
if you use a hypervisor (HyperV, VMware, VirtualBox),
or if you have any other problems with the current environment.

During 6+ years of development, we faced dozens of different problems that may spoil your measurements.
Inside BenchmarkDotNet, there are a lot of heuristics, checks, hacks, and tricks that help you to
increase the reliability of the results.

Friendliness

Analysis of performance data is a time-consuming activity that requires attentiveness, knowledge, and experience.
BenchmarkDotNet performs the main part of this analysis for you and presents results in a user-friendly form.

After the experiments, you get a summary table that contains a lot of useful data about the executed benchmarks.
By default, it includes only the most important columns,
but they can be easily customized.
The column set is adaptive and depends on the benchmark definition and measured values.
For example, if you mark one of the benchmarks as a baseline,
you will get additional columns that will help you to compare all the benchmarks with the baseline.
By default, it always shows the Mean column,
but if we detected a vast difference between the Mean and the Median values,
both columns will be presented.

BenchmarkDotNet tries to find some unusual properties of your performance distributions and prints nice messages about it.
For example, it will warn you in case of multimodal distribution or high outliers.
In this case, you can scroll the results up and check out ASCII-style histograms for each distribution
or generate beautiful png plots using [RPlotExporter].

BenchmarkDotNet doesn't overload you with data; it shows only the essential information depending on your results:
it allows you to keep the summary small for primitive cases and extend it only for complicated cases.
Of course, you can request any additional statistics and visualizations manually.
If you don't customize the summary view,
the default presentation will be as much user-friendly as possible. :)

Learn more about benchmarking

BenchmarkDotNet is not a silver bullet that magically makes all of your benchmarks correct and analyzes the measurements for you.
Even if you use this library, you still should know how to design benchmark experiments and how to make correct conclusions based on the raw data.
If you want to know more about benchmarking methodology and good practices,
it's recommended to read a book by Andrey Akinshin (the BenchmarkDotNet project lead): "Pro .NET Benchmarking".
Use this in-depth guide to correctly design benchmarks, measure key performance metrics of .NET applications, and analyze results.
This book presents dozens of case studies to help you understand complicated benchmarking topics.
You will avoid common pitfalls, control the accuracy of your measurements, and improve the performance of your software.

Contributions are welcome!

BenchmarkDotNet is already a stable full-featured library that allows performing performance investigation on a professional level.
And it continues to evolve!
We add new features all the time, but we have too many new cool ideas.
Any help will be appreciated.
You can develop new features, fix bugs, improve the documentation, or do some other cool stuff.

If you want to contribute, check out the
Contributing guide and
up-for-grabs issues.
If you have new ideas or want to complain about bugs, feel free to create a new issue.
Let's build the best tool for benchmarking together!

Code of Conduct

This project has adopted the code of conduct defined by the Contributor Covenant
to clarify expected behavior in our community.
For more information, see the .NET Foundation Code of Conduct.

File: docs/articles/contributing/building.md

Building

There are two recommended options to build BenchmarkDotNet from source:

Visual Studio

Once all the necessary tools are in place, building is trivial. Simply open solution file BenchmarkDotNet.slnx that lives at the base of the repository and run Build action.

Command-line

Cake (C# Make) is a cross platform build automation system with a C# DSL to do things like compiling code, copy files/folders, running unit tests, compress files and build NuGet packages.

The build currently depends on the following prerequisites:

In order to run various build tasks from terminal, use build.cmd file in the repository root.
build.cmd is a cross-platform script that can be used the same way on Windows, Linux, and macOS.
When executed without arguments, it prints help information with list of all available build tasks.

File: docs/articles/contributing/debugging.md

Debugging

There should be two debug profiles available in VS drop down

File: docs/articles/contributing/documentation.md

Documentation

BenchmarkDotNet uses DocFX as a documentation generation tool.

Hints

  • If you want to provide a link to API, you can use
    cross references by
    UID.
    For example,
    [SimpleJobAttribute](xref:BenchmarkDotNet.Attributes.SimpleJobAttribute) and
    @BenchmarkDotNet.Attributes.SimpleJobAttribute
    will be transformed to
    SimpleJobAttribute.

Notes

DocFX uses the following syntax inside block quote for different types of notes:

markdown
> [!NOTE]
> note content
> [!TIP]
> tip content
> [!WARNING]
> warning content
> [!IMPORTANT]
> important content
> [!Caution]
> caution content

It will be transformed to:

[!NOTE]
note content

[!TIP]
tip content

[!WARNING]
warning content

[!IMPORTANT]
important content

[!Caution]
caution content

Building documentation locally

You can build documentation locally with the help of the docs-build build task:

text
build.cmd docs-build

See also

File: docs/articles/contributing/miscellaneous.md

#Miscellaneous topics

F#

We have full F# support, all you have to do is to run dotnet restore to download the compilers etc.

Chat room

How can I help?

Here is a list of up-for-grabs issues

File: docs/articles/contributing/running-tests.md

Running Tests

To run all tests just run the following command in the repo root:

cmd
dotnet test -c Release BenchmarkDotNet.slnx

Most of the tests projects target net472 and net8.0. If the change that you want to test is not specific to any particular runtime, you can run the tests for one of them.

cmd
dotnet test -c Release -f net8.0 BenchmarkDotNet.slnx

You should be able to run all of tests from your IDE as well.

Verify Tests

For some unit tests (e.g. for exporter tests) BenchmarkDotNet uses Verify.

  • The expected value for each test is stored in a *.verified.txt file located near the test source file in the repository. Verify generates verified file's names automatically according test name and its parameters. This files must be added under the source control.
  • It also creates a *.received file for each failed test. You can use diff tools for convenient file comparison. By default you can find test run results on the test runner console as usual. You can comment out the line result.DisableDiff() in VerifySettingsFactory.Create method and then Verify will open KDiff for each failed test. This way you can easily understand what's the difference between verified and received values and choose the correct one.

Running tests on GitHub Actions with GitHub CLI

This section describes how to run tests on GitHub Actions by using CLI.

Prerequisites

The following tools are required to run tests on CI with gh workflow run command.

Run gh auth login command to authenticate with your GitHub account after installation.

Common parameters of `gh workflow run` command

Following parameters need to specified to invoke workflow with gh workflow run command.

name description
<workflow-name> Specify workflow name (e.g. run-tests-selected.yaml)
--repo Specify target repository name with {owner}/{repo} format (e.g. dotnet/BenchmarkDotNet)
--ref Specify target branch name

[!NOTE]
gh run workflow command requires Write permission on target repository.
Contributors who don't have write permission to dotnet/BenchmarkDotNet need to run workflow on a forked repository.

Parameters of `run-tests-selected.yaml` workflow

run-tests-selected.yaml accept following input parameters. (Passed with -f or --field)

name default description
runs_on ubuntu-latest GitHub Actions runner image name (windows-latest windows-11-arm ubuntu-latest macos-latest macos-26-intel)
project tests/BenchmarkDotNet.IntegrationTests Specify path of project directory
framework net8.0 Target Framework (e.g. net8.0, net472)
filter BenchmarkDotNet Test filter text (It's used for dotnet test --filter) Use default value when running all tests
iteration_count 1 Count of test loop (It's expected to be used for flaky tests)
skip_setup_additional_tools false Skip setup additional tools for Wasm/NativeAOT tests

Command examples

Run `CompletedWorkItemCountIsAccurate` test

pwsh
gh workflow run run-tests-selected.yaml --repo dotnet/BenchmarkDotNet --ref master -f runs_on=windows-latest -f filter=CompletedWorkItemCountIsAccurate

File: docs/articles/configs/analysers.md

uid: docs.analysers name: Analysers

Analysers

An analyser can analyse summary of your benchmarks and produce some useful warnings.
For example, EnvironmentAnalyser warns you,
if you build your application in the DEBUG mode or run it with an attached debugger.

File: docs/articles/configs/columns.md

uid: docs.columns name: Columns

Columns

A column is a column in the summary table.

Default columns

In this section, default columns (which be added to the Summary table by default) are presented.
Some of columns are optional, i.e. they can be omitted (it depends on the measurements from the summary).

Target

There are 3 default columns which describes the target benchmark:
Namespace, Type, Method. Namespace and Type will be omitted
when all the benchmarks have the same namespace or type name.
Method column always be a part of the summary table.

Job

There are many different job characteristics,
but the summary includes only characteristics which has at least one non-default value.

Statistics

There are also a lot of different statistics which can be considered.
It will be really hard to analyse the summary table, if all of the available statistics will be shown.
Fortunately, BenchmarkDotNet has some heuristics for statistics columns and shows only important columns.
For example, if all of the standard deviations are zero (we run our benchmarks against Dry job),
this column will be omitted.
The standard error will be shown only for cases when we are failed to achieve required accuracy level.

Only Mean will be always shown.
If the distribution looks strange,
BenchmarkDotNet could also print additional columns like Median or P95 (95th percentile).

If you need specific statistics, you always could add them manually.

Params

If you have params, the corresponded columns will be automatically added.

Diagnosers

If you turned on diagnosers which providers additional columns, they will be also included in the summary page.

Custom columns

Of course, you can define own custom columns and use it everywhere. Here is the definition of TagColumn:

[!code-csharpIntroTagColumn.cs]

[!includeIntroTagColumn]

File: docs/articles/configs/configoptions.md

#cspell:ignore configoptions uid: docs.configoptions name: Configoptions

Config Options

The config options let you customize some behavior of BenchmarkDotNet - mainly regarding the output.
Available config options are:

  • ConfigOptions.Default - No configuration option is set - this is the default.
  • ConfigOptions.KeepBenchmarkFiles - All auto-generated files should be kept after running the benchmarks (by default they are removed).
  • ConfigOptions.JoinSummary - All benchmarks results should be joined into a single summary (by default we have a summary per type).
  • ConfigOptions.StopOnFirstError - Benchmarking should be stopped after the first error (by default it's not).
  • ConfigOptions.DisableOptimizationsValidator - Mandatory optimizations validator should be entirely turned off.
  • ConfigOptions.DontOverwriteResults - The exported result files should not be overwritten (by default they are overwritten).
  • ConfigOptions.DisableLogFile - Disables the log file written on disk.

All of these options could be combined and are available as CLI (Comand Line Interface) option (except DisableOptimizationsValidator), see Console Arguments for further information how to use the CLI.

Any of these options could be used either in object style config or fluent style config:

Object style config

cs
public class Config : ManualConfig
{
    public Config()
    {
        // Using the WithOptions() factory method:
        this.WithOptions(ConfigOptions.JoinSummary)
            .WithOptions(ConfigOptions.DisableLogFile);
        
        // Or (The ConfigOptions Enum is defined as a BitField)
        this.WithOptions(ConfigOptions.JoinSummary | ConfigOptions.DisableLogFile);

    }
}

Fluent style config

cs
public static void Run()
    {
        BenchmarkRunner
            .Run(
                ManualConfig
                    .Create(DefaultConfig.Instance)
                    .WithOptions(ConfigOptions.JoinSummary)
                    .WithOptions(ConfigOptions.DisableLogFile)
                    // or
                    .WithOptions(ConfigOptions.JoinSummary | ConfigOptions.DisableLogFile));
    }

2. Official Technical Reference & Guides (dotnet/docs)

.NET Docs








This repository contains the conceptual documentation for .NET. The .NET documentation site is built from multiple repositories:

API reference documentation is published from the following repositories:

These API reference doc repositories are public. Only some accept issues and pull requests. Others are pass-through repositories because API reference is generated directly from the /// in the product source. Some folders in dotnet-api-docs use the product repository as the source of truth.

The C# language specification documentation comes from the following two repositories:

Our team's tasks are tracked in our project boards. You'll see monthly sprint projects, along with long-running projects for major documentation updates. The projects contain documentation issues across the repositories that build .NET docs. Issues are tracked in the relevant repositories. We have a large community using these resources. We make our best effort to respond to issues in a timely fashion. To create a new issue, click the "Open a documentation issue" button at the bottom of any of our published docs, or choose one of the available templates. The control at the bottom of each article automatically routes you to the correct repo and fills in some relevant information based on the article.

:purple_heart: Contribute

We welcome contributions to help us improve and complete the .NET docs. This is a very large repo, covering a large area. If this is your first visit, see the Contributor guide for information on working with us. Look for issues labeled help wanted :label: for ideas to get started.

Before submitting a PR with 3rd party dependencies, see our policy on 3rd party dependencies.

We work to merge or close PRs in a timely fashion. We regularly review and approve PRs. We encourage contributors to respond to comments in a similar timely fashion. Many times, reviews identify small changes such as spelling or grammar issues, or word choice. Maintainers can accept those suggestions and merge the PR. In other situations, maintainers ask for more significant changes before a PR is ready to merge. We expect contributors to make those changes. Maintainers will help by answering questions or pointing to other resources, if needed.

We consider PRs to be abandoned when they've had no activity (either commits or discussion) in the past 30 days. Team members will close PRs that have been abandoned. Contributors can reopen and continue to work.

:bookmark_tabs: Code of conduct

This project has adopted the code of conduct defined by the Contributor Covenant
to clarify expected behavior in our community. For more information, see the .NET Foundation: Code of Conduct.

:octocat: GitHub Action workflows

  • : Adds a comment to PRs that were not automated, but rather manually created that target the live branch.
  • : Automatically approves and auto-merges PRs originating from the dependabbot[bot].
  • : Creates a PR to generate the "What's new" article on the first of every month.
  • : The current status for the entire repositories Markdown linter status.
  • : Runs various Markdown verifications, beyond the linter, such as ensuring links and redirects are valid.
  • : Builds the site for the PR in context, and verifies the build reporting either, success, warnings, or error.
  • : Custom .NET build validation, locates code impacted by a PR, and builds.
  • : Runs monthly, creating issues on projects that target .NET versions that are out of support.
  • : Automatically updates the dependabot configuration weekly, but only if required.
  • : Automatically synchronizes issues with Quest (Azure DevOps).
  • : Manual bulk import of issues into Quest (Azure DevOps).

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#benchmark Browse benchmark catalog → #benchmarking Browse benchmarking catalog → #c-sharp Browse c-sharp catalog → #csharp Browse csharp catalog → #dotnet Browse dotnet catalog → #hacktoberfest Browse hacktoberfest catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →