Catch2 Documentation

File: README.md Copy Chapter

What is Catch2? Copy Chapter

Catch2 is mainly a unit testing framework for C++, but it also
provides basic micro-benchmarking features, and simple BDD macros.

Catch2's main advantage is that using it is both simple and natural.
Test names do not have to be valid identifiers, assertions look like
normal C++ boolean expressions, and sections provide a nice and local way
to share set-up and tear-down code in tests.

Example unit test

#include <catch2/catch_test_macros.hpp>

#include <cstdint>

uint32_t factorial( uint32_t number ) {
    return number <= 1 ? number : factorial(number-1) * number;
}

TEST_CASE( "Factorials are computed", "[factorial]" ) {
    REQUIRE( factorial( 1) == 1 );
    REQUIRE( factorial( 2) == 2 );
    REQUIRE( factorial( 3) == 6 );
    REQUIRE( factorial(10) == 3'628'800 );
}

Example microbenchmark

#include <catch2/catch_test_macros.hpp>
#include <catch2/benchmark/catch_benchmark.hpp>

#include <cstdint>

uint64_t fibonacci(uint64_t number) {
    return number < 2 ? number : fibonacci(number - 1) + fibonacci(number - 2);
}

TEST_CASE("Benchmark Fibonacci", "[!benchmark]") {
    REQUIRE(fibonacci(5) == 5);

    REQUIRE(fibonacci(20) == 6'765);
    BENCHMARK("fibonacci 20") {
        return fibonacci(20);
    };

    REQUIRE(fibonacci(25) == 75'025);
    BENCHMARK("fibonacci 25") {
        return fibonacci(25);
    };
}

Note that benchmarks are not run by default, so you need to run it explicitly
with the [!benchmark] tag.

Catch2 v3 has been released! Copy Chapter

You are on the devel branch, where the v3 version is being developed.
v3 brings a bunch of significant changes, the big one being that Catch2
is no longer a single-header library. Catch2 now behaves as a normal
library, with multiple headers and separately compiled implementation.

The documentation is slowly being updated to take these changes into
account, but this work is currently still ongoing.

For migrating from the v2 releases to v3, you should look at our
documentation. It provides a simple
guidelines on getting started, and collects most common migration
problems.

For the previous major version of Catch2 look into the v2.x branch
here on GitHub.

How to use it Copy Chapter

This documentation comprises these three parts:

• Why do we need yet another C++ Test Framework?
• Tutorial - getting started
• Reference section - all the details

More Copy Chapter

• Issues and bugs can be raised on the Issue tracker on GitHub
• For discussion or questions please use our Discord
• See who else is using Catch2 in Open Source Software
  or commercially.

File: docs/assertions.md Copy Chapter

Assertion Macros

Contents
Natural Expressions
Floating point comparisons
Exceptions
Matcher expressions
Thread Safety
Expressions with commas

Most test frameworks have a large collection of assertion macros to capture all possible conditional forms (_EQUALS, _NOTEQUALS, _GREATER_THAN etc).

Catch is different. Because it decomposes natural C-style conditional expressions most of these forms are reduced to one or two that you will use all the time. That said there is a rich set of auxiliary macros as well. We'll describe all of these here.

Most of these macros come in two forms:

Natural Expressions Copy Chapter

The REQUIRE family of macros tests an expression and aborts the test case if it fails.
The CHECK family are equivalent but execution continues in the same test case even if the assertion fails. This is useful if you have a series of essentially orthogonal assertions and it is useful to see all the results rather than stopping at the first failure.

• REQUIRE( expression ) and
• CHECK( expression )

Evaluates the expression and records the result. If an exception is thrown, it is caught, reported, and counted as a failure. These are the macros you will use most of the time.

Examples:

CHECK( str == "string value" );
CHECK( thisReturnsTrue() );
REQUIRE( i == 42 );

Expressions prefixed with ! cannot be decomposed. If you have a type
that is convertible to bool and you want to assert that it evaluates to
false, use the two forms below:

• REQUIRE_FALSE( expression ) and
• CHECK_FALSE( expression )

Note that there is no reason to use these forms for plain bool variables,
because there is no added value in decomposing them.

Example:

Status ret = someFunction();
REQUIRE_FALSE(ret); // ret must evaluate to false, and Catch2 will print
                    // out the value of ret if possibly

Other limitations Copy Section

Note that expressions containing either of the binary logical operators,
&& or ||, cannot be decomposed and will not compile. The reason behind
this is that it is impossible to overload && and || in a way that
keeps their short-circuiting semantics, and expression decomposition
relies on overloaded operators to work.

Simple example of an issue with overloading binary logical operators
is a common pointer idiom, p && p->foo == 2. Using the built-in &&
operator, p is only dereferenced if it is not null. With overloaded
&&, p is always dereferenced, thus causing a segfault if
p == nullptr.

If you want to test expression that contains && or ||, you have two
options.

1. Enclose it in parentheses. Parentheses force evaluation of the expression
   before the expression decomposition can touch it, and thus it cannot
   be used.

2. Rewrite the expression. REQUIRE(a == 1 && b == 2) can always be split
   into REQUIRE(a == 1); REQUIRE(b == 2);. Alternatively, if this is a
   common pattern in your tests, think about using Matchers.
   instead. There is no simple rewrite rule for ||, but I generally
   believe that if you have || in your test expression, you should rethink
   your tests.

Floating point comparisons Copy Chapter

Comparing floating point numbers is complex, and so it has its own
documentation page.

Exceptions Copy Chapter

• REQUIRE_NOTHROW( expression ) and
• CHECK_NOTHROW( expression )

Expects that no exception is thrown during evaluation of the expression.

• REQUIRE_THROWS( expression ) and
• CHECK_THROWS( expression )

Expects that an exception (of any type) is be thrown during evaluation of the expression.

• REQUIRE_THROWS_AS( expression, exception type ) and
• CHECK_THROWS_AS( expression, exception type )

Expects that an exception of the specified type is thrown during evaluation of the expression. Note that the exception type is extended with const& and you should not include it yourself.

• REQUIRE_THROWS_WITH( expression, string or string matcher ) and
• CHECK_THROWS_WITH( expression, string or string matcher )

Expects that an exception is thrown that, when converted to a string, matches the string or string matcher provided (see next section for Matchers).

e.g.

REQUIRE_THROWS_WITH( openThePodBayDoors(), ContainsSubstring( "afraid" ) && ContainsSubstring( "can't do that" ) );
REQUIRE_THROWS_WITH( dismantleHal(), "My mind is going" );

• REQUIRE_THROWS_MATCHES( expression, exception type, matcher for given exception type ) and
• CHECK_THROWS_MATCHES( expression, exception type, matcher for given exception type )

Expects that exception of exception type is thrown and it matches provided matcher (see the documentation for Matchers).

Please note that the THROW family of assertions expects to be passed a single expression, not a statement or series of statements. If you want to check a more complicated sequence of operations, you can use a C++11 lambda function.

REQUIRE_NOTHROW([&](){
    int i = 1;
    int j = 2;
    auto k = i + j;
    if (k == 3) {
        throw 1;
    }
}());

Matcher expressions Copy Chapter

To support Matchers a slightly different form is used. Matchers have their own documentation.

• REQUIRE_THAT( lhs, matcher expression ) and
• CHECK_THAT( lhs, matcher expression )

Matchers can be composed using &&, || and ! operators.

Thread Safety Copy Chapter

Currently assertions in Catch are not thread safe.
For more details, along with workarounds, see the section on the limitations page.

Expressions with commas Copy Chapter

Because the preprocessor parses code using different rules than the
compiler, multiple-argument assertions (e.g. REQUIRE_THROWS_AS) have
problems with commas inside the provided expressions. As an example
REQUIRE_THROWS_AS(std::pair(1, 2), std::invalid_argument);
will fail to compile, because the preprocessor sees 3 arguments provided,
but the macro accepts only 2. There are two possible workarounds.

1. Use typedef:

using int_pair = std::pair;
REQUIRE_THROWS_AS(int_pair(1, 2), std::invalid_argument);

This solution is always applicable, but makes the meaning of the code
less clear.

2. Parenthesize the expression:

TEST_CASE_METHOD((Fixture), "foo", "[bar]") {
    SUCCEED();
}

This solution is not always applicable, because it might require extra
changes on the Catch's side to work.

Home

File: docs/benchmarks.md Copy Chapter

Authoring benchmarks

Introduced in Catch2 2.9.0.

Writing benchmarks is not easy. Catch simplifies certain aspects but you'll
always need to take care about various aspects. Understanding a few things about
the way Catch runs your code will be very helpful when writing your benchmarks.

First off, let's go over some terminology that will be used throughout this
guide.

• User code: user code is the code that the user provides to be measured.
• Run: one run is one execution of the user code. Sometimes also referred
  to as an iteration.
• Sample: one sample is one data point obtained by measuring the time it takes
  to perform a certain number of runs. One sample can consist of more than one
  run if the clock available does not have enough resolution to accurately
  measure a single run. All samples for a given benchmark execution are obtained
  with the same number of runs.

Execution procedure Copy Chapter

Now I can explain how a benchmark is executed in Catch. There are three main
steps, though the first does not need to be repeated for every benchmark.

1. Environmental probe: before any benchmarks can be executed, the clock's
   resolution is estimated. A few other environmental artifacts are also estimated
   at this point, like the cost of calling the clock function, but they almost
   never have any impact in the results.

2. Estimation: the user code is executed a few times to obtain an estimate of
   the amount of runs that should be in each sample. This also has the potential
   effect of bringing relevant code and data into the caches before the actual
   measurement starts.

3. Measurement: all the samples are collected sequentially by performing the
   number of runs estimated in the previous step for each sample.

This already gives us one important rule for writing benchmarks for Catch: the
benchmarks must be repeatable. The user code will be executed several times, and
the number of times it will be executed during the estimation step cannot be
known beforehand since it depends on the time it takes to execute the code.
User code that cannot be executed repeatedly will lead to bogus results or
crashes.

Benchmark specification Copy Chapter

Benchmarks can be specified anywhere inside a Catch test case.
There is a simple and a slightly more advanced version of the BENCHMARK macro.

Let's have a look how a naive Fibonacci implementation could be benchmarked:

std::uint64_t Fibonacci(std::uint64_t number) {
    return number < 2 ? 1 : Fibonacci(number - 1) + Fibonacci(number - 2);
}

Now the most straight forward way to benchmark this function, is just adding a BENCHMARK macro to our test case:

TEST_CASE("Fibonacci") {
    CHECK(Fibonacci(0) == 1);
    // some more asserts..
    CHECK(Fibonacci(5) == 8);
    // some more asserts..

    // now let's benchmark:
    BENCHMARK("Fibonacci 20") {
        return Fibonacci(20);
    };

    BENCHMARK("Fibonacci 25") {
        return Fibonacci(25);
    };

    BENCHMARK("Fibonacci 30") {
        return Fibonacci(30);
    };

    BENCHMARK("Fibonacci 35") {
        return Fibonacci(35);
    };
}

There's a few things to note:

• As BENCHMARK expands to a lambda expression it is necessary to add a semicolon after
  the closing brace (as opposed to the first experimental version).
• The return is a handy way to avoid the compiler optimizing away the benchmark code.

Running this already runs the benchmarks and outputs something similar to:

-------------------------------------------------------------------------------
Fibonacci
-------------------------------------------------------------------------------
C:\path\to\Catch2\Benchmark.tests.cpp(10)
...............................................................................
benchmark name                                  samples       iterations    est run time
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
Fibonacci 20                                            100       416439   83.2878 ms
                                                       2 ns         2 ns         2 ns
                                                       0 ns         0 ns         0 ns

Fibonacci 25                                            100       400776   80.1552 ms
                                                       3 ns         3 ns         3 ns
                                                       0 ns         0 ns         0 ns

Fibonacci 30                                            100       396873   79.3746 ms
                                                      17 ns        17 ns        17 ns
                                                       0 ns         0 ns         0 ns

Fibonacci 35                                            100       145169   87.1014 ms
                                                     468 ns       464 ns       473 ns
                                                      21 ns        15 ns        34 ns

Advanced benchmarking Copy Section

The simplest use case shown above, takes no arguments and just runs the user code that needs to be measured.
However, if using the BENCHMARK_ADVANCED macro and adding a Catch::Benchmark::Chronometer argument after
the macro, some advanced features are available. The contents of the simple benchmarks are invoked once per run,
while the blocks of the advanced benchmarks are invoked exactly twice:
once during the estimation phase, and another time during the execution phase.

BENCHMARK("simple"){ return long_computation(); };

BENCHMARK_ADVANCED("advanced")(Catch::Benchmark::Chronometer meter) {
    set_up();
    meter.measure([] { return long_computation(); });
};

These advanced benchmarks no longer consist entirely of user code to be measured.
In these cases, the code to be measured is provided via the
Catch::Benchmark::Chronometer::measure member function. This allows you to set up any
kind of state that might be required for the benchmark but is not to be included
in the measurements, like making a vector of random integers to feed to a
sorting algorithm.

A single call to Catch::Benchmark::Chronometer::measure performs the actual measurements
by invoking the callable object passed in as many times as necessary. Anything
that needs to be done outside the measurement can be done outside the call to
measure.

The callable object passed in to measure can optionally accept an int
parameter.

meter.measure([](int i) { return long_computation(i); });

If it accepts an int parameter, the sequence number of each run will be passed
in, starting with 0. This is useful if you want to measure some mutating code,
for example. The number of runs can be known beforehand by calling
Catch::Benchmark::Chronometer::runs; with this one can set up a different instance to be
mutated by each run.

std::vector<std::string> v(meter.runs());
std::fill(v.begin(), v.end(), test_string());
meter.measure([&v](int i) { in_place_escape(v[i]); });

Note that it is not possible to simply use the same instance for different runs
and resetting it between each run since that would pollute the measurements with
the resetting code.

It is also possible to just provide an argument name to the simple BENCHMARK macro to get
the same semantics as providing a callable to meter.measure with int argument:

BENCHMARK("indexed", i){ return long_computation(i); };

Constructors and destructors Copy Section

All of these tools give you a lot mileage, but there are two things that still
need special handling: constructors and destructors. The problem is that if you
use automatic objects they get destroyed by the end of the scope, so you end up
measuring the time for construction and destruction together. And if you use
dynamic allocation instead, you end up including the time to allocate memory in
the measurements.

To solve this conundrum, Catch provides class templates that let you manually
construct and destroy objects without dynamic allocation and in a way that lets
you measure construction and destruction separately.

BENCHMARK_ADVANCED("construct")(Catch::Benchmark::Chronometer meter) {
    std::vector<Catch::Benchmark::storage_for<std::string>> storage(meter.runs());
    meter.measure([&](int i) { storage[i].construct("thing"); });
};

BENCHMARK_ADVANCED("destroy")(Catch::Benchmark::Chronometer meter) {
    std::vector<Catch::Benchmark::destructable_object<std::string>> storage(meter.runs());
    for(auto&& o : storage)
        o.construct("thing");
    meter.measure([&](int i) { storage[i].destruct(); });
};

Catch::Benchmark::storage_for<T> objects are just pieces of raw storage suitable for T
objects. You can use the Catch::Benchmark::storage_for::construct member function to call a constructor and
create an object in that storage. So if you want to measure the time it takes
for a certain constructor to run, you can just measure the time it takes to run
this function.

When the lifetime of a Catch::Benchmark::storage_for<T> object ends, if an actual object was
constructed there it will be automatically destroyed, so nothing leaks.

If you want to measure a destructor, though, we need to use
Catch::Benchmark::destructable_object<T>. These objects are similar to
Catch::Benchmark::storage_for<T> in that construction of the T object is manual, but
it does not destroy anything automatically. Instead, you are required to call
the Catch::Benchmark::destructable_object::destruct member function, which is what you
can use to measure the destruction time.

The optimizer Copy Section

Sometimes the optimizer will optimize away the very code that you want to
measure. There are several ways to use results that will prevent the optimiser
from removing them. You can use the volatile keyword, or you can output the
value to standard output or to a file, both of which force the program to
actually generate the value somehow.

Catch adds a third option. The values returned by any function provided as user
code are guaranteed to be evaluated and not optimised out. This means that if
your user code consists of computing a certain value, you don't need to bother
with using volatile or forcing output. Just return it from the function.
That helps with keeping the code in a natural fashion.

Here's an example:

// may measure nothing at all by skipping the long calculation since its
// result is not used
BENCHMARK("no return"){ long_calculation(); };

// the result of long_calculation() is guaranteed to be computed somehow
BENCHMARK("with return"){ return long_calculation(); };

However, there's no other form of control over the optimizer whatsoever. It is
up to you to write a benchmark that actually measures what you want and doesn't
just measure the time to do a whole bunch of nothing.

To sum up, there are two simple rules: whatever you would do in handwritten code
to control optimization still works in Catch; and Catch makes return values
from user code into observable effects that can't be optimized away.

Adapted from nonius' documentation.

File: docs/ci-and-misc.md Copy Chapter

Tooling integration (CI, test runners and so on)

Contents
Continuous Integration systems
Bazel test runner integration
Low-level tools
CMake

This page talks about Catch2's integration with other related tooling,
like Continuous Integration and 3rd party test runners.

Continuous Integration systems Copy Chapter

Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (currently we also offer TeamCity, TAP, Automake and SonarQube reporters).

Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them.

XML Reporter Copy Section

-r xml

The XML Reporter writes in an XML format that is specific to Catch.

The advantage of this format is that it corresponds well to the way Catch works (especially the more unusual features, such as nested sections) and is a fully streaming format - that is it writes output as it goes, without having to store up all its results before it can start writing.

The disadvantage is that, being specific to Catch, no existing build servers understand the format natively. It can be used as input to an XSLT transformation that could convert it to, say, HTML - although this loses the streaming advantage, of course.

JUnit Reporter Copy Section

-r junit

The JUnit Reporter writes in an XML format that mimics the JUnit ANT schema.

The advantage of this format is that the JUnit Ant schema is widely understood by most build servers and so can usually be consumed with no additional work.

The disadvantage is that this schema was designed to correspond to how JUnit works - and there is a significant mismatch with how Catch works. Additionally the format is not streamable (because opening elements hold counts of failed and passing tests as attributes) - so the whole test run must complete before it can be written.

TeamCity Reporter Copy Section

-r teamcity

The TeamCity Reporter writes TeamCity service messages to stdout. In order to be able to use this reporter an additional header must also be included.

Being specific to TeamCity this is the best reporter to use with it - but it is completely unsuitable for any other purpose. It is a streaming format (it writes as it goes) - although test results don't appear in the TeamCity interface until the completion of a suite (usually the whole test run).

Automake Reporter Copy Section

-r automake

The Automake Reporter writes out the meta tags expected by automake via make check.

TAP (Test Anything Protocol) Reporter Copy Section

-r tap

Because of the incremental nature of Catch's test suites and ability to run specific tests, our implementation of TAP reporter writes out the number of tests in a suite last.

SonarQube Reporter Copy Section

-r sonarqube
SonarQube Generic Test Data XML format for tests metrics.

Bazel test runner integration Copy Chapter

Catch2 understands some of the environment variables Bazel uses to control
test execution. Specifically it understands

• JUnit output path via XML_OUTPUT_FILE
• Test filtering via TESTBRIDGE_TEST_ONLY
• Test sharding via TEST_SHARD_INDEX, TEST_TOTAL_SHARDS, and TEST_SHARD_STATUS_FILE
• Creating a file to signal premature test exit via TEST_PREMATURE_EXIT_FILE
• Setting the RNG seed via TEST_RANDOM_SEED

Support for XML_OUTPUT_FILE was introduced in Catch2 3.0.1

Support for TESTBRIDGE_TEST_ONLY and sharding was introduced in Catch2 3.2.0

Support for TEST_PREMATURE_EXIT_FILE and TEST_RANDOM_SEED was introduced in Catch2 3.11.0

This integration is enabled via either a compile time configuration
option, or via BAZEL_TEST environment
variable set to "1".

Support for BAZEL_TEST was introduced in Catch2 3.1.0

Note that if both the Bazel environment var and command line option for
something are used, the environment variable wins.

Low-level tools Copy Chapter

CodeCoverage module (GCOV, LCOV...) Copy Section

If you are using GCOV tool to get testing coverage of your code, and are not sure how to integrate it with CMake and Catch, there should be an external example over at https://github.com/claremacrae/catch_cmake_coverage

pkg-config Copy Section

Catch2 provides a rudimentary pkg-config integration, by registering itself
under the name catch2. This means that after Catch2 is installed, you
can use pkg-config to get its include path: pkg-config --cflags catch2.

gdb and lldb scripts Copy Section

Catch2's extras folder also contains two simple debugger scripts,
gdbinit for gdb and lldbinit for lldb. If loaded into their
respective debugger, these will tell it to step over Catch2's internals
when stepping through code.

CMake Copy Chapter

As it has been getting kinda long, the documentation of Catch2's
integration with CMake has been moved to its own page.

Home

File: docs/cmake-integration.md Copy Chapter

CMake integration

Contents
CMake targets
Automatic test registration
CMake project options
CATCH_CONFIG_* customization options in CMake
Installing Catch2 from git repository
Installing Catch2 from vcpkg
Installing Catch2 from Bazel

Because we use CMake to build Catch2, we also provide a couple of
integration points for our users.

1. Catch2 exports a (namespaced) CMake target
2. Catch2's repository contains CMake scripts for automatic registration
   of TEST_CASEs in CTest

CMake targets Copy Chapter

Catch2's CMake build exports two targets, Catch2::Catch2, and
Catch2::Catch2WithMain. If you do not need custom main function,
you should be using the latter (and only the latter). Linking against
it will add the proper include paths and link your target together with
2 static libraries that implement Catch2 and its main respectively.
If you need custom main, you should link only against Catch2::Catch2.

This means that if Catch2 has been installed on the system, it should
be enough to do

find_package(Catch2 3 REQUIRED)
# These tests can use the Catch2-provided main
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)

# These tests need their own main
add_executable(custom-main-tests test.cpp test-main.cpp)
target_link_libraries(custom-main-tests PRIVATE Catch2::Catch2)

These targets are also provided when Catch2 is used as a subdirectory.
Assuming Catch2 has been cloned to lib/Catch2, you only need to replace
the find_package call with add_subdirectory(lib/Catch2) and the snippet
above still works.

Another possibility is to use FetchContent:

Include(FetchContent)

FetchContent_Declare(
  Catch2
  GIT_REPOSITORY https://github.com/catchorg/Catch2.git
  GIT_TAG        v3.8.1 # or a later release
)

FetchContent_MakeAvailable(Catch2)

add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)

Automatic test registration Copy Chapter

Catch2's repository also contains three CMake scripts that help users
with automatically registering their TEST_CASEs with CTest. They
can be found in the extras folder, and are

1. Catch.cmake (and its dependency CatchAddTests.cmake)
2. ParseAndAddCatchTests.cmake (deprecated)
3. CatchShardTests.cmake (and its dependency CatchShardTestsImpl.cmake)

If Catch2 has been installed in system, both of these can be used after
doing find_package(Catch2 REQUIRED). Otherwise you need to add them
to your CMake module path.

`Catch.cmake` and `CatchAddTests.cmake` Copy Section

Catch.cmake provides function catch_discover_tests to get tests from
a target. This function works by running the resulting executable with
--list-test flag, and then parsing the output to find all existing tests.

Usage

cmake_minimum_required(VERSION 3.16)

project(baz LANGUAGES CXX VERSION 0.0.1)

find_package(Catch2 REQUIRED)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2)

include(CTest)
include(Catch)
catch_discover_tests(tests)

When using FetchContent, include(Catch) will fail unless
CMAKE_MODULE_PATH is explicitly updated to include the extras
directory.

# ... FetchContent ...
#
list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/extras)
include(CTest)
include(Catch)
catch_discover_tests(tests)

Customization

catch_discover_tests can be given several extra arguments:

catch_discover_tests(target
                     [TEST_SPEC arg1...]
                     [EXTRA_ARGS arg1...]
                     [WORKING_DIRECTORY dir]
                     [TEST_PREFIX prefix]
                     [TEST_SUFFIX suffix]
                     [PROPERTIES name1 value1...]
                     [TEST_LIST var]
                     [REPORTER reporter]
                     [OUTPUT_DIR dir]
                     [OUTPUT_PREFIX prefix]
                     [OUTPUT_SUFFIX suffix]
                     [DISCOVERY_MODE ]
                     [SKIP_IS_FAILURE]
                     [ADD_TAGS_AS_LABELS]
)

• TEST_SPEC arg1...

Specifies test cases, wildcarded test cases, tags and tag expressions to
pass to the Catch executable alongside the --list-test-names-only flag.

• EXTRA_ARGS arg1...

Any extra arguments to pass on the command line to each test case.

• WORKING_DIRECTORY dir

Specifies the directory in which to run the discovered test cases. If this
option is not provided, the current binary directory is used.

• TEST_PREFIX prefix

Specifies a prefix to be added to the name of each discovered test case.
This can be useful when the same test executable is being used in multiple
calls to catch_discover_tests(), with different TEST_SPEC or EXTRA_ARGS.

• TEST_SUFFIX suffix

Same as TEST_PREFIX, except it specific the suffix for the test names.
Both TEST_PREFIX and TEST_SUFFIX can be specified at the same time.

• PROPERTIES name1 value1...

Specifies additional properties to be set on all tests discovered by this
invocation of catch_discover_tests.

• TEST_LIST var

Make the list of tests available in the variable var, rather than the
default <target>_TESTS. This can be useful when the same test
executable is being used in multiple calls to catch_discover_tests().
Note that this variable is only available in CTest.

• REPORTER reporter

Use the specified reporter when running the test case. The reporter will
be passed to the test runner as --reporter reporter.

• OUTPUT_DIR dir

If specified, the parameter is passed along as
--out dir/<test_name> to test executable. The actual file name is the
same as the test name. This should be used instead of
EXTRA_ARGS --out foo to avoid race conditions writing the result output
when using parallel test execution.

• OUTPUT_PREFIX prefix

May be used in conjunction with OUTPUT_DIR.
If specified, prefix is added to each output file name, like so
--out dir/prefix<test_name>.

• OUTPUT_SUFFIX suffix

May be used in conjunction with OUTPUT_DIR.
If specified, suffix is added to each output file name, like so
--out dir/<test_name>suffix. This can be used to add a file extension to
the output file name e.g. ".xml".

• DISCOVERY_MODE mode

If specified allows control over when test discovery is performed.
For a value of POST_BUILD (default) test discovery is performed at build time.
For a value of PRE_TEST test discovery is delayed until just prior to test
execution (useful e.g. in cross-compilation environments).
DISCOVERY_MODE defaults to the value of the
CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE variable if it is not passed when
calling catch_discover_tests. This provides a mechanism for globally
selecting a preferred test discovery behavior.

Note that on Apple Silicon with the Xcode generator you must use PRE_TEST,
e.g. catch_discover_tests(tests DISCOVERY_MODE PRE_TEST). With the default
POST_BUILD mode the build fails with Result: Subprocess killed, because
macOS on Apple Silicon refuses to run unsigned binaries and Xcode code-signs
the test executable only after the post-build script that POST_BUILD
mode uses to run it for test discovery. PRE_TEST avoids this by delaying
discovery until test time, when the executable is already signed. The same
limitation affects CMake's gtest_discover_tests; see
Catch2 #2411 and
CMake #21845.

• SKIP_IS_FAILURE

Skipped tests will be marked as failed instead.

• ADD_TAGS_AS_LABELS

Add the tags from tests as labels to CTest.

`ParseAndAddCatchTests.cmake` Copy Section

⚠ This script is deprecated
in Catch2 2.13.4 and superseded by the above approach using catch_discover_tests.
See #2092 for details.

ParseAndAddCatchTests works by parsing all implementation files
associated with the provided target, and registering them via CTest's
add_test. This approach has some limitations, such as the fact that
commented-out tests will be registered anyway. More serious, only a
subset of the assertion macros currently available in Catch can be
detected by this script and tests with any macros that cannot be
parsed are silently ignored.

Usage

cmake_minimum_required(VERSION 3.16)

project(baz LANGUAGES CXX VERSION 0.0.1)

find_package(Catch2 REQUIRED)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2)

include(CTest)
include(ParseAndAddCatchTests)
ParseAndAddCatchTests(tests)

Customization

ParseAndAddCatchTests provides some customization points:

• PARSE_CATCH_TESTS_VERBOSE -- When ON, the script prints debug
  messages. Defaults to OFF.
• PARSE_CATCH_TESTS_NO_HIDDEN_TESTS -- When ON, hidden tests (tests
  tagged with either of [.] or [.foo]) will not be registered.
  Defaults to OFF.
• PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME -- When ON, adds fixture
  class name to the test name in CTest. Defaults to ON.
• PARSE_CATCH_TESTS_ADD_TARGET_IN_TEST_NAME -- When ON, adds target
  name to the test name in CTest. Defaults to ON.
• PARSE_CATCH_TESTS_ADD_TO_CONFIGURE_DEPENDS -- When ON, adds test
  file to CMAKE_CONFIGURE_DEPENDS. This means that the CMake configuration
  step will be re-ran when the test files change, letting new tests be
  automatically discovered. Defaults to OFF.

Optionally, one can specify a launching command to run tests by setting the
variable OptionalCatchTestLauncher before calling ParseAndAddCatchTests. For
instance to run some tests using MPI and other sequentially, one can write

set(OptionalCatchTestLauncher ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} ${NUMPROC})
ParseAndAddCatchTests(mpi_foo)
unset(OptionalCatchTestLauncher)
ParseAndAddCatchTests(bar)

`CatchShardTests.cmake` Copy Section

CatchShardTests.cmake was introduced in Catch2 3.1.0.

CatchShardTests.cmake provides a function
catch_add_sharded_tests(TEST_BINARY) that splits tests from TEST_BINARY
into multiple shards. The tests in each shard and their order is randomized,
and the seed changes every invocation of CTest.

Currently there are 3 customization points for this script:

• SHARD_COUNT - number of shards to split target's tests into
• REPORTER - reporter spec to use for tests
• TEST_SPEC - test spec used for filtering tests

Example usage:

include(CatchShardTests)

catch_add_sharded_tests(foo-tests
  SHARD_COUNT 4
  REPORTER "xml::out=-"
  TEST_SPEC "A"
)

catch_add_sharded_tests(tests
  SHARD_COUNT 8
  REPORTER "xml::out=-"
  TEST_SPEC "B"
)

This registers total of 12 CTest tests (4 + 8 shards) to run shards
from foo-tests test binary, filtered by a test spec.

Note that this script is currently a proof-of-concept for reseeding
shards per CTest run, and thus does not support (nor does it currently
aim to support) all customization points from
catch_discover_tests.

CMake project options Copy Chapter

Catch2's CMake project also provides some options for other projects
that consume it. These are:

• BUILD_TESTING -- When ON and the project is not used as a subproject,
  Catch2's test binary will be built. Defaults to ON.
• CATCH_INSTALL_DOCS -- When ON, Catch2's documentation will be
  included in the installation. Defaults to ON.
• CATCH_INSTALL_EXTRAS -- When ON, Catch2's extras folder (the CMake
  scripts mentioned above, debugger helpers) will be included in the
  installation. Defaults to ON.
• CATCH_DEVELOPMENT_BUILD -- When ON, configures the build for development
  of Catch2. This means enabling test projects, warnings and so on.
  Defaults to OFF.

Enabling CATCH_DEVELOPMENT_BUILD also enables further configuration
customization options:

• CATCH_BUILD_TESTING -- When ON, Catch2's SelfTest project will be
  built. Defaults to ON. Note that Catch2 also obeys BUILD_TESTING CMake
  variable, so both of them need to be ON for the SelfTest to be built,
  and either of them can be set to OFF to disable building SelfTest.
• CATCH_BUILD_EXAMPLES -- When ON, Catch2's usage examples will be
  built. Defaults to OFF.
• CATCH_BUILD_EXTRA_TESTS -- When ON, Catch2's extra tests will be
  built. Defaults to OFF.
• CATCH_BUILD_FUZZERS -- When ON, Catch2 fuzzing entry points will
  be built. Defaults to OFF.
• CATCH_ENABLE_WERROR -- When ON, adds -Werror or equivalent flag
  to the compilation. Defaults to ON.
• CATCH_BUILD_SURROGATES -- When ON, each header in Catch2 will be
  compiled separately to ensure that they are self-sufficient.
  Defaults to OFF.

`CATCH_CONFIG_*` customization options in CMake Copy Chapter

CMake support for CATCH_CONFIG_* options was introduced in Catch2 3.0.1

Due to the new separate compilation model, all the options from the
Compile-time configuration docs can also be set
through Catch2's CMake. To set them, define the option you want as ON,
e.g. -DCATCH_CONFIG_NOSTDOUT=ON.

Note that setting the option to OFF doesn't disable it. To force disable
an option, you need to set the _NO_ form of it to ON, e.g.
-DCATCH_CONFIG_NO_COLOUR_WIN32=ON.

To summarize the configuration option behaviour with an example:

Installing Catch2 from git repository Copy Chapter

If you cannot install Catch2 from a package manager (e.g. Ubuntu 16.04
provides catch only in version 1.2.0) you might want to install it from
the repository instead. Assuming you have enough rights, you can just
install it to the default location, like so:

$ git clone https://github.com/catchorg/Catch2.git
$ cd Catch2
$ cmake -B build -S . -DBUILD_TESTING=OFF
$ sudo cmake --build build/ --target install

If you do not have superuser rights, you will also need to specify
CMAKE_INSTALL_PREFIX
when configuring the build, and then modify your calls to
find_package
accordingly.

Installing Catch2 from vcpkg Copy Chapter

Alternatively, you can build and install Catch2 using vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install catch2

The catch2 port in vcpkg is kept up to date by microsoft team members and community contributors.
If the version is out of date, please create an issue or pull request on the vcpkg repository.

Installing Catch2 from Bazel Copy Chapter

Catch2 is now a supported module in the Bazel Central Registry. You only need to add one line to your MODULE.bazel file;
please see https://registry.bazel.build/modules/catch2 for the latest supported version.

You can then add catch2_main to each of your C++ test build rules as follows:

cc_test(
    name = "example_test",
    srcs = ["example_test.cpp"],
    deps = [
        ":example",
        "@catch2//:catch2_main",
    ],
)

Home

File: docs/command-line.md Copy Chapter

Command line

Contents
Specifying which tests to run
Choosing a reporter to use
Breaking into the debugger
Showing results for successful tests
Aborting after a certain number of failures
Listing available tests, tags or reporters
Sending output to a file
Naming a test run
Eliding assertions expected to throw
Make whitespace visible
Warnings
Reporting timings
Load test names to run from a file
Specify the order test cases are run
Specify a seed for the Random Number Generator
Identify framework and version according to the libIdentify standard
Wait for key before continuing
Skip all benchmarks
Specify the number of benchmark samples to collect
Specify the number of resamples for bootstrapping
Specify the confidence-interval for bootstrapping
Disable statistical analysis of collected benchmark samples
Specify the amount of time in milliseconds spent on warming up each test
Usage
Specify the section/generator element to run
Filenames as tags
Override output colouring
Test Sharding
Allow running the binary without tests
Output verbosity
Create file to guard against silent early termination

Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available.
Click one of the following links to take you straight to that option - or scroll on to browse the available options.

<test-spec> ...
-h, -?, --help
-s, --success
-b, --break
-e, --nothrow
-i, --invisibles
-o, --out
-r, --reporter
-n, --name
-a, --abort
-x, --abortx
-w, --warn
-d, --durations
-f, --input-file
-c, --section
-g, --generator-index
-p, --path-filter
-#, --filenames-as-tags

--list-tests
--list-tags
--list-reporters
--list-listeners
--order
--rng-seed
--libidentify
--wait-for-keypress
--skip-benchmarks
--benchmark-samples
--benchmark-resamples
--benchmark-confidence-interval
--benchmark-no-analysis
--benchmark-warmup-time
--colour-mode
--shard-count
--shard-index
--allow-running-no-tests
--verbosity

Specifying which tests to run Copy Chapter

<test-spec> ...

By providing a test spec, you filter which tests will be run. If you call
Catch2 without any test spec, then it will run all non-hidden test
cases. A test case is hidden if it has the [!benchmark] tag, any tag
with a dot at the start, e.g. [.] or [.foo].

There are three basic test specs that can then be combined into more
complex specs:

• Full test name, e.g. "Test 1".

  This allows only test cases whose name is "Test 1".

• Wildcarded test name, e.g. "*Test", or "Test*", or "*Test*".

  This allows any test case whose name ends with, starts with, or contains
  in the middle the string "Test". Note that the wildcard can only be at
  the start or end.

• Tag name, e.g. [some-tag].

  This allows any test case tagged with "[some-tag]". Remember that some
  tags are special, e.g. those that start with "." or with "!".

You can also combine the basic test specs to create more complex test
specs. You can:

• Concatenate specs to apply all of them, e.g. [some-tag][other-tag].

  This allows test cases that are tagged with both "[some-tag]" and
  "[other-tag]". A test case with just "[some-tag]" will not pass the filter,
  nor will test case with just "[other-tag]".

• Comma-join specs to apply any of them, e.g. [some-tag],[other-tag].

  This allows test cases that are tagged with either "[some-tag]" or
  "[other-tag]". A test case with both will obviously also pass the filter.

  Note that commas take precedence over simple concatenation. This means
  that [a][b],[c] accepts tests that are tagged with either both "[a]" and
  "[b]", or tests that are tagged with just "[c]".

• Negate the spec by prepending it with ~, e.g. ~[some-tag].

  This rejects any test case that is tagged with "[some-tag]". Note that
  rejection takes precedence over other filters.

  Note that negations always binds to the following basic test spec.
  This means that ~[foo][bar] negates only the "[foo]" tag and not the
  "[bar]" tag.

Note that when Catch2 is deciding whether to include a test, first it
checks whether the test matches any negative filters. If it does,
the test is rejected. After that, the behaviour depends on whether there
are positive filters as well. If there are no positive filters, all
remaining non-hidden tests are included. If there are positive filters,
only tests that match the positive filters are included.

You can also match test names with special characters by escaping them
with a backslash ("\"), e.g. a test named "Do A, then B" is matched
by "Do A\, then B" test spec. Backslash also escapes itself.

Examples Copy Section

Given these TEST_CASEs,

TEST_CASE("Test 1") {}

TEST_CASE("Test 2", "[.foo]") {}

TEST_CASE("Test 3", "[.bar]") {}

TEST_CASE("Test 4", "[.][foo][bar]") {}

this is the result of these filters

./tests                      # Selects only the first test, others are hidden
./tests "Test 1"             # Selects only the first test, other do not match
./tests ~"Test 1"            # Selects no tests. Test 1 is rejected, other tests are hidden
./tests "Test *"             # Selects all tests.
./tests [bar]                # Selects tests 3 and 4. Other tests are not tagged [bar]
./tests ~[foo]               # Selects test 1, because it is the only non-hidden test without [foo] tag
./tests [foo][bar]           # Selects test 4.
./tests [foo],[bar]          # Selects tests 2, 3, 4.
./tests ~[foo][bar]          # Selects test 3. 2 and 4 are rejected due to having [foo] tag
./tests ~"Test 2"[foo]       # Selects test 4, because test 2 is explicitly rejected
./tests [foo][bar],"Test 1"  # Selects tests 1 and 4.
./tests "Test 1*"            # Selects test 1, wildcard can match zero characters

Note: Using plain asterisk on a command line can cause issues with shell
expansion. Make sure that the asterisk is passed to Catch2 and is not
interpreted by the shell.

Choosing a reporter to use Copy Chapter

-r, --reporter <reporter[::key=value]*>

Reporters are how the output from Catch2 (results of assertions, tests,
benchmarks and so on) is formatted and written out. The default reporter
is called the "Console" reporter and is intended to provide relatively
verbose and human-friendly output.

Reporters are also individually configurable. To pass configuration options
to the reporter, you append ::key=value to the reporter specification
as many times as you want, e.g. --reporter xml::out=someFile.xml or
--reporter custom::colour-mode=ansi::Xoption=2.

The keys must either be prefixed by "X", in which case they are not parsed
by Catch2 and are only passed down to the reporter, or one of options
hardcoded into Catch2. Currently there are 3 supported options:

• "out"
• "colour-mode"
• "verbosity"

Support for per-reporter verbosity option was added in Catch2 vX.Y.Z

Note that the reporter might still check the X-prefixed options for
validity, and throw an error if they are wrong.

Support for passing arguments to reporters through the -r, --reporter flag was introduced in Catch2 3.0.1

There are multiple built-in reporters, you can see what they do by using the
--list-reporters
flag. If you need a reporter providing custom format outside of the already
provided ones, look at the "write your own reporter" part of the reporter
documentation.

This option may be passed multiple times to use multiple (different)
reporters at the same time. See the reporter documentation
for details on what the resulting behaviour is. Also note that at most one
reporter can be provided without the output-file part of reporter spec.
This reporter will use the "default" output destination, based on
the -o, --out option.

Support for using multiple different reporters at the same time was introduced in Catch2 3.0.1

Note: There is currently no way to escape :: in the reporter spec,
and thus the reporter names, or configuration keys and values, cannot
contain ::. As :: in paths is relatively obscure (unlike ':'), we do
not consider this an issue.

Breaking into the debugger Copy Chapter

-b, --break

Under most debuggers Catch2 is capable of automatically breaking on a test
failure. This allows the user to see the current state of the test during
failure.

Showing results for successful tests Copy Chapter

-s, --success

Usually you only want to see reporting for failed tests. Sometimes it's useful to see all the output (especially when you don't trust that that test you just added worked first time!).
To see successful, as well as failing, test results just pass this option. Note that each reporter may treat this option differently. The Junit reporter, for example, logs all results regardless.

Aborting after a certain number of failures Copy Chapter

-a, --abort
-x, --abortx [<failure threshold>]

If a REQUIRE assertion fails the test case aborts, but subsequent test cases are still run.
If a CHECK assertion fails even the current test case is not aborted.

Sometimes this results in a flood of failure messages and you'd rather just see the first few. Specifying -a or --abort on its own will abort the whole test run on the first failed assertion of any kind. Use -x or --abortx followed by a number to abort after that number of assertion failures.

Listing available tests, tags or reporters Copy Chapter

--list-tests
--list-tags
--list-reporters
--list-listeners

The --list* options became customizable through reporters in Catch2 3.0.1

The --list-listeners option was added in Catch2 3.0.1

--list-tests lists all registered tests matching specified test spec.
Usually this listing also includes tags, and potentially also other
information, like source location, based on verbosity and reporter's design.

--list-tags lists all tags from registered tests matching specified test
spec. Usually this also includes number of tests cases they match and
similar information.

--list-reporters lists all available reporters and their descriptions.

--list-listeners lists all registered listeners and their descriptions.

The --verbosity argument modifies the level of detail provided by the default --list* options
as follows:

Sending output to a file Copy Chapter

-o, --out <filename>

Use this option to send all output to a file, instead of stdout. You can
use - as the filename to explicitly send the output to stdout (this is
useful e.g. when using multiple reporters).

Support for - as the filename was introduced in Catch2 3.0.1

Filenames starting with "%" (percent symbol) are reserved by Catch2 for
meta purposes, e.g. using %debug as the filename opens stream that
writes to platform specific debugging/logging mechanism.

Catch2 currently recognizes 3 meta streams:

• %debug - writes to platform specific debugging/logging output
• %stdout - writes to stdout
• %stderr - writes to stderr

Support for %stdout and %stderr was introduced in Catch2 3.0.1

Naming a test run Copy Chapter

-n, --name <name for test run>

If a name is supplied it will be used by the reporter to provide an overall name for the test run. This can be useful if you are sending to a file, for example, and need to distinguish different test runs - either from different Catch executables or runs of the same executable with different options. If not supplied the name is defaulted to the name of the executable.

Eliding assertions expected to throw Copy Chapter

-e, --nothrow

Skips all assertions that test that an exception is thrown, e.g. REQUIRE_THROWS.

These can be a nuisance in certain debugging environments that may break when exceptions are thrown (while this is usually optional for handled exceptions, it can be useful to have enabled if you are trying to track down something unexpected).

Sometimes exceptions are expected outside of one of the assertions that tests for them (perhaps thrown and caught within the code-under-test). The whole test case can be skipped when using -e by marking it with the [!throws] tag.

When running with this option any throw checking assertions are skipped so as not to contribute additional noise. Be careful if this affects the behaviour of subsequent tests.

Make whitespace visible Copy Chapter

-i, --invisibles

If a string comparison fails due to differences in whitespace - especially leading or trailing whitespace - it can be hard to see what's going on.
This option transforms tabs and newline characters into \t and \n respectively when printing.

Warnings Copy Chapter

-w, --warn <warning name>

You can think of Catch2's warnings as the equivalent of -Werror (/WX)
flag for C++ compilers. It turns some suspicious occurrences, like a section
without assertions, into errors. Because these might be intended, warnings
are not enabled by default, but user can opt in.

You can enable multiple warnings at the same time.

There are currently two warnings implemented:

NoAssertions        // Fail test case / leaf section if no assertions
                        // (e.g. `REQUIRE`) is encountered.
    UnmatchedTestSpec   // Fail test run if any of the CLI test specs did
                        // not match any tests.
    InfiniteGenerators  // Fail if GENERATE would run infinitely

UnmatchedTestSpec was introduced in Catch2 3.0.1.

InfiniteGenerators was introduced in Catch2 3.13.0

Reporting timings Copy Chapter

-d, --durations <yes/no>

When set to yes Catch will report the duration of each test case, in seconds with millisecond precision. Note that it does this regardless of whether a test case passes or fails. Note, also, the certain reporters (e.g. Junit) always report test case durations regardless of this option being set or not.

-D, --min-duration <value>

--min-duration was introduced in Catch2 2.13.0

When set, Catch will report the duration of each test case that took more
than <value> seconds, in seconds with millisecond precision. This option is overridden by both
-d yes and -d no, so that either all durations are reported, or none
are.

Load test names to run from a file Copy Chapter

-f, --input-file <filename>

Provide the name of a file that contains a list of test case names,
one per line. Blank lines are skipped.

A useful way to generate an initial instance of this file is to combine
the --list-tests flag with
the --verbosity quiet option. You can also
use test specs to filter this list down to what you want first.

Specify the order test cases are run Copy Chapter

--order <decl|lex|rand>

Test cases are ordered one of three ways:

decl Copy Section

Declaration order.

Tests in the same translation unit are sorted using their declaration orders,
different TUs are sorted in an implementation (linking) dependent order.

lex Copy Section

Lexicographic order.

Tests are sorted by their name, their tags are ignored.

rand Copy Section

Randomized order. The default order.

Randomized order has been made default in Catch2 3.9.0

The order is dependent on Catch2's random seed (see
--rng-seed), and is subset invariant. What this means
is that as long as the random seed is fixed, running only some tests
(e.g. via tag) does not change their relative order.

The subset stability was introduced in Catch2 v2.12.0

Since the random order was made subset stable, we promise that given
the same random seed, the order of test cases will be the same across
different platforms, as long as the tests were compiled against identical
version of Catch2. We reserve the right to change the relative order
of tests cases between Catch2 versions, but it is unlikely to happen often.

Specify a seed for the Random Number Generator Copy Chapter

--rng-seed <'time'|'random-device'|number>

Sets the seed for random number generators used by Catch2. These are used
e.g. to shuffle tests when user asks for tests to be in random order.

Using time as the argument asks Catch2 generate the seed through call
to std::time(nullptr). This provides very weak randomness and multiple
runs of the binary can generate the same seed if they are started close
to each other.

Using random-device asks for std::random_device to be used instead.
If your implementation provides working std::random_device, it should
be preferred to using time. Catch2 uses std::random_device by default.

Identify framework and version according to the libIdentify standard Copy Chapter

--libidentify

See The LibIdentify repo for more information and examples.

Wait for key before continuing Copy Chapter

--wait-for-keypress <never|start|exit|both>

Will cause the executable to print a message and wait until the return/ enter key is pressed before continuing -
either before running any tests, after running all tests - or both, depending on the argument.

Skip all benchmarks Copy Chapter

--skip-benchmarks

Introduced in Catch2 3.0.1.

This flag tells Catch2 to skip running all benchmarks. Benchmarks in this
case mean code blocks in BENCHMARK and BENCHMARK_ADVANCED macros, not
test cases with the [!benchmark] tag.

Specify the number of benchmark samples to collect Copy Chapter

--benchmark-samples <# of samples>

Introduced in Catch2 2.9.0.

When running benchmarks a number of "samples" is collected. This is the base data for later statistical analysis.
Per sample a clock resolution dependent number of iterations of the user code is run, which is independent of the number of samples. Defaults to 100.

Specify the number of resamples for bootstrapping Copy Chapter

--benchmark-resamples <# of resamples>

Introduced in Catch2 2.9.0.

After the measurements are performed, statistical bootstrapping is performed
on the samples. The number of resamples for that bootstrapping is configurable
but defaults to 100000. Due to the bootstrapping it is possible to give
estimates for the mean and standard deviation. The estimates come with a lower
bound and an upper bound, and the confidence interval (which is configurable but
defaults to 95%).

Specify the confidence-interval for bootstrapping Copy Chapter

--benchmark-confidence-interval <confidence-interval>

Introduced in Catch2 2.9.0.

The confidence-interval is used for statistical bootstrapping on the samples to
calculate the upper and lower bounds of mean and standard deviation.
Must be between 0 and 1 and defaults to 0.95.

Disable statistical analysis of collected benchmark samples Copy Chapter

--benchmark-no-analysis

Introduced in Catch2 2.9.0.

When this flag is specified no bootstrapping or any other statistical analysis is performed.
Instead the user code is only measured and the plain mean from the samples is reported.

Specify the amount of time in milliseconds spent on warming up each test Copy Chapter

--benchmark-warmup-time

Introduced in Catch2 2.11.2.

Configure the amount of time spent warming up each test.

Usage Copy Chapter

-h, -?, --help

Prints the command line arguments to stdout

Specify the section/generator element to run Copy Chapter

-c, --section <section name>
-g, --generator-index <index in generator>
-p, --path-filter <path filter spec>

The generator and generic path filtering was added in Catch2 3.13.0

These arguments allow you to run specific section(s) in a test case, or
only get specific element from a generator. All the variants form a shared
stack of filters.

See the full documentation of path filtering for more details

Filenames as tags Copy Chapter

-#, --filenames-as-tags

This option adds an extra tag to all test cases. The tag is # followed
by the unqualified filename the test case is defined in, with the last
extension stripped out.

For example, tests within the file tests\SelfTest\UsageTests\BDD.tests.cpp
will be given the [#BDD.tests] tag.

Override output colouring Copy Chapter

--colour-mode <ansi|win32|none|default>

The --colour-mode option replaced the old --colour option in Catch2 3.0.1

Catch2 support two different ways of colouring terminal output, and by
default it attempts to make a good guess on which implementation to use
(and whether to even use it, e.g. Catch2 tries to avoid writing colour
codes when writing the results into a file).

--colour-mode allows the user to explicitly select what happens.

• --colour-mode ansi tells Catch2 to always use ANSI colour codes, even
  when writing to a file
• --colour-mode win32 tells Catch2 to use colour implementation based
  on Win32 terminal API
• --colour-mode none tells Catch2 to disable colours completely
• --colour-mode default lets Catch2 decide

--colour-mode default is the default setting.

Test Sharding Copy Chapter

--shard-count <#number of shards>, --shard-index <#shard index to run>

Introduced in Catch2 3.0.1.

When --shard-count <#number of shards> is used, the tests to execute
will be split evenly in to the given number of sets, identified by indices
starting at 0. The tests in the set given by
--shard-index <#shard index to run> will be executed. The default shard
count is 1, and the default index to run is 0.

Shard index must be less than number of shards. As the name suggests,
it is treated as an index of the shard to run.

Sharding is useful when you want to split test execution across multiple
processes, as is done with the Bazel test sharding.

Allow running the binary without tests Copy Chapter

--allow-running-no-tests

Introduced in Catch2 3.0.1.

By default, Catch2 test binaries return non-0 exit code if no tests were run,
e.g. if the binary was compiled with no tests, the provided test spec matched no
tests, or all tests were skipped at runtime. This flag
overrides that, so a test run with no tests still returns 0.

Output verbosity Copy Chapter

-v, --verbosity <quiet|normal|high>

Changing verbosity might change how many details Catch2's reporters output.
However, you should consider changing the verbosity level as a suggestion.
Not all reporters support all verbosity levels, e.g. because the reporter's
format cannot meaningfully change. In that case, the verbosity level is
ignored.

Verbosity defaults to normal.

Create file to guard against silent early termination Copy Chapter

--premature-exit-guard-file <path>

Introduced in Catch2 3.11.0

Tells Catch2 to create an empty file at specified path before the tests
start, and delete it after the tests finish. If the file is present after
the process stops, it can be assumed that the testing binary exited
prematurely, e.g. due to the OOM killer.

All directories in the path must already exist. If this option is used
and Catch2 cannot create the file (e.g. the location is not writable),
the test run will fail.

Home

File: docs/commercial-users.md Copy Chapter

Commercial users of Catch2

Catch2 is also widely used in proprietary code bases. This page contains
some of them that are willing to share this information.

If you want to add your organisation, please check that there is no issue
with you sharing this fact.

• Bloomberg
• Bloomlife
• Inscopix Inc.
• Locksley.CZ
• Makimo
• NASA
• Nexus Software Systems
• UX3D
• King

Home

File: docs/comparing-floating-point-numbers.md Copy Chapter

Comparing floating point numbers with Catch2

If you are not deeply familiar with them, floating point numbers can be
unintuitive. This also applies to comparing floating point numbers for
(in)equality.

This page assumes that you have some understanding of both FP, and the
meaning of different kinds of comparisons, and only goes over what
functionality Catch2 provides to help you with comparing floating point
numbers. If you do not have this understanding, we recommend that you first
study up on floating point numbers and their comparisons, e.g. by reading
this blog post.

Floating point matchers Copy Chapter

#include <catch2/matchers/catch_matchers_floating_point.hpp>

Matchers are the preferred way of comparing floating
point numbers in Catch2. We provide 3 of them:

• WithinAbs(double target, double margin),
• WithinRel(FloatingPoint target, FloatingPoint eps), and
• WithinULP(FloatingPoint target, uint64_t maxUlpDiff).

WithinRel matcher was introduced in Catch2 2.10.0

As with all matchers, you can combine multiple floating point matchers
in a single assertion. For example, to check that some computation matches
a known good value within 0.1% or is close enough (no different to 5
decimal places) to zero, we would write this assertion:

REQUIRE_THAT( computation(input),
        Catch::Matchers::WithinRel(expected, 0.001)
     || Catch::Matchers::WithinAbs(0, 0.000001) );

WithinAbs Copy Section

WithinAbs creates a matcher that accepts floating point numbers whose
difference with target is less-or-equal to the margin. Since float
can be converted to double without losing precision, only double
overload exists.

REQUIRE_THAT(1.0, WithinAbs(1.2, 0.2));
REQUIRE_THAT(0.f, !WithinAbs(1.0, 0.5));
// Notice that infinity == infinity for WithinAbs
REQUIRE_THAT(INFINITY, WithinAbs(INFINITY, 0));

WithinRel Copy Section

WithinRel creates a matcher that accepts floating point numbers that
are approximately equal to the target with a tolerance of eps.
Specifically, it matches if
|arg - target| <= eps * max(|arg|, |target|) holds. If you do not
specify eps, std::numeric_limits<FloatingPoint>::epsilon * 100
is used as the default.

// Notice that WithinRel comparison is symmetric, unlike Approx's.
REQUIRE_THAT(1.0, WithinRel(1.1, 0.1));
REQUIRE_THAT(1.1, WithinRel(1.0, 0.1));
// Notice that inifnity == infinity for WithinRel
REQUIRE_THAT(INFINITY, WithinRel(INFINITY));

WithinULP Copy Section

WithinULP creates a matcher that accepts floating point numbers that
are no more than maxUlpDiff
ULPs
away from the target value. The short version of what this means
is that there is no more than maxUlpDiff - 1 representable floating
point numbers between the argument for matching and the target value.

When using the ULP matcher in Catch2, it is important to keep in mind
that Catch2 interprets ULP distance slightly differently than
e.g. std::nextafter does.

Catch2's ULP calculation obeys these relations:

• ulpDistance(-x, x) == 2 * ulpDistance(x, 0)
• ulpDistance(-0, 0) == 0 (due to the above)
• ulpDistance(DBL_MAX, INFINITY) == 1
• ulpDistancE(NaN, x) == infinity

Important: The WithinULP matcher requires the platform to use the
IEEE-754 representation for
floating point numbers.

REQUIRE_THAT( -0.f, WithinULP( 0.f, 0 ) );

`Approx` Copy Chapter

#include <catch2/catch_approx.hpp>

We strongly recommend against using Approx when writing new code.
You should be using floating point matchers instead.

Catch2 provides one more way to handle floating point comparisons. It is
Approx, a special type with overloaded comparison operators, that can
be used in standard assertions, e.g.

REQUIRE(0.99999 == Catch::Approx(1));

Approx supports four comparison operators, ==, !=, <=, >=, and can
also be used with strong typedefs over doubles. It can be used for both
relative and margin comparisons by using its three customization points.
Note that the semantics of this is always that of an or, so if either
the relative or absolute margin comparison passes, then the whole comparison
passes.

The downside to Approx is that it has a couple of issues that we cannot
fix without breaking backwards compatibility. Because Catch2 also provides
complete set of matchers that implement different floating point comparison
methods, Approx is left as-is, is considered deprecated, and should
not be used in new code.

The issues are

• All internal computation is done in doubles, leading to slightly
  different results if the inputs were floats.
• Approx's relative margin comparison is not symmetric. This means
  that Approx( 10 ).epsilon(0.1) != 11.1 but Approx( 11.1 ).epsilon(0.1) == 10.
• By default, Approx only uses relative margin comparison. This means
  that Approx(0) == X only passes for X == 0.

Approx details Copy Section

If you still want/need to know more about Approx, read on.

Catch2 provides a UDL for Approx; _a. It resides in the Catch::literals
namespace, and can be used like this:

using namespace Catch::literals;
REQUIRE( performComputation() == 2.1_a );

Approx has three customization points for the comparison:

• epsilon - epsilon sets the coefficient by which a result
  can differ from Approx's value before it is rejected.
  Defaults to std::numeric_limits<float>::epsilon()*100.

Approx target = Approx(100).epsilon(0.01);
100.0 == target; // Obviously true
200.0 == target; // Obviously still false
100.5 == target; // True, because we set target to allow up to 1% difference

• margin - margin sets the absolute value by which
  a result can differ from Approx's value before it is rejected.
  Defaults to 0.0.

Approx target = Approx(100).margin(5);
100.0 == target; // Obviously true
200.0 == target; // Obviously still false
104.0 == target; // True, because we set target to allow absolute difference of at most 5

• scale - scale is used to change the magnitude of Approx for the relative check.
  By default, set to 0.0.

Scale could be useful if the computation leading to the result worked
on a different scale than is used by the results. Approx's scale is added
to Approx's value when computing the allowed relative margin from the
Approx's value.

Home

File: docs/configuration.md Copy Chapter

Compile-time configuration

Contents
Prefixing Catch macros
Terminal colour
Console width
stdout
Fallback stringifier
Default reporter
Bazel support
C++11 toggles
C++17 toggles
Other toggles
Enabling stringification
Disabling exceptions
Disabling deprecation warnings
Overriding Catch's debug break (-b)
Static analysis support
Thread safety in assertions (and messages)

Catch2 is designed to "just work" as much as possible, and most of the
configuration options below are changed automatically during compilation,
according to the detected environment. However, this detection can also
be overridden by users, using macros documented below, and/or CMake options
with the same name.

Prefixing Catch macros Copy Chapter

CATCH_CONFIG_PREFIX_ALL       // Prefix all macros with CATCH_
CATCH_CONFIG_PREFIX_MESSAGES  // Prefix only message macros ((UNSCOPED_)INFO, WARN, (UNSCOPED_)CAPTURE)

To keep test code clean and uncluttered Catch uses short macro names (e.g. TEST_CASE and REQUIRE). Occasionally these may conflict with identifiers from platform headers or the system under test. In this case the above identifier can be defined. This will cause all the Catch user macros to be prefixed with CATCH_ (e.g. CATCH_TEST_CASE and CATCH_REQUIRE).

Terminal colour Copy Chapter

CATCH_CONFIG_COLOUR_WIN32     // Force enables compiling colouring impl based on Win32 console API
CATCH_CONFIG_NO_COLOUR_WIN32  // Force disables ...

Yes, Catch2 uses the british spelling of colour.

Catch2 attempts to autodetect whether the Win32 console colouring API,
SetConsoleTextAttribute, is available, and if it is available it compiles
in a console colouring implementation that uses it.

This option can be used to override Catch2's autodetection and force the
compilation either ON or OFF.

Console width Copy Chapter

CATCH_CONFIG_CONSOLE_WIDTH = x // where x is a number

Catch formats output intended for the console to fit within a fixed number of characters. This is especially important as indentation is used extensively and uncontrolled line wraps break this.
By default a console width of 80 is assumed but this can be controlled by defining the above identifier to be a different value.

stdout Copy Chapter

CATCH_CONFIG_NOSTDOUT

To support platforms that do not provide std::cout, std::cerr and
std::clog, Catch does not use them directly, but rather calls
Catch::cout, Catch::cerr and Catch::clog. You can replace their
implementation by defining CATCH_CONFIG_NOSTDOUT and implementing
them yourself, their signatures are:

std::ostream& cout();
std::ostream& cerr();
std::ostream& clog();

You can see an example of replacing these functions here.

Fallback stringifier Copy Chapter

By default, when Catch's stringification machinery has to stringify
a type that does not specialize StringMaker, does not overload operator<<,
is not an enumeration and is not a range, it uses "{?}". This can be
overridden by defining CATCH_CONFIG_FALLBACK_STRINGIFIER to name of a
function that should perform the stringification instead.

All types that do not provide StringMaker specialization or operator<<
overload will be sent to this function (this includes enums and ranges).
The provided function must return std::string and must accept any type,
e.g. via overloading.

Note that if the provided function does not handle a type and this type
requires to be stringified, the compilation will fail.

Default reporter Copy Chapter

Catch's default reporter can be changed by defining macro
CATCH_CONFIG_DEFAULT_REPORTER to string literal naming the desired
default reporter.

This means that defining CATCH_CONFIG_DEFAULT_REPORTER to "console"
is equivalent with the out-of-the-box experience.

Bazel support Copy Chapter

Compiling Catch2 with CATCH_CONFIG_BAZEL_SUPPORT force-enables Catch2's
support for Bazel's environment variables (normally Catch2 looks for
BAZEL_TEST=1 env var first).

This can be useful if you are using older versions of Bazel, that do not
yet have BAZEL_TEST env var support.

CATCH_CONFIG_BAZEL_SUPPORT was introduced in Catch2 3.0.1.

CATCH_CONFIG_BAZEL_SUPPORT was deprecated in Catch2 3.1.0.

C++11 toggles Copy Chapter

CATCH_CONFIG_CPP11_TO_STRING // Use `std::to_string`

Because we support platforms whose standard library does not contain
std::to_string, it is possible to force Catch to use a workaround
based on std::stringstream. On platforms other than Android,
the default is to use std::to_string. On Android, the default is to
use the stringstream workaround. As always, it is possible to override
Catch's selection, by defining either CATCH_CONFIG_CPP11_TO_STRING or
CATCH_CONFIG_NO_CPP11_TO_STRING.

C++17 toggles Copy Chapter

CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS  // Override std::uncaught_exceptions (instead of std::uncaught_exception) support detection
CATCH_CONFIG_CPP17_STRING_VIEW          // Override std::string_view support detection (Catch provides a StringMaker specialization by default)
CATCH_CONFIG_CPP17_VARIANT              // Override std::variant support detection (checked by CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER)
CATCH_CONFIG_CPP17_OPTIONAL             // Override std::optional support detection (checked by CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER)
CATCH_CONFIG_CPP17_BYTE                 // Override std::byte support detection (Catch provides a StringMaker specialization by default)

CATCH_CONFIG_CPP17_STRING_VIEW was introduced in Catch2 2.4.1.

Catch contains basic compiler/standard detection and attempts to use
some C++17 features whenever appropriate. This automatic detection
can be manually overridden in both directions, that is, a feature
can be enabled by defining the macro in the table above, and disabled
by using _NO_ in the macro, e.g. CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS.

Other toggles Copy Chapter

CATCH_CONFIG_COUNTER                    // Use __COUNTER__ to generate unique names for test cases
CATCH_CONFIG_WINDOWS_SEH                // Enable SEH handling on Windows
CATCH_CONFIG_FAST_COMPILE               // Sacrifices some (rather minor) features for compilation speed
CATCH_CONFIG_POSIX_SIGNALS              // Enable handling POSIX signals
CATCH_CONFIG_WINDOWS_CRTDBG             // Enable leak checking using Windows's CRT Debug Heap
CATCH_CONFIG_DISABLE_STRINGIFICATION    // Disable stringifying the original expression
CATCH_CONFIG_DISABLE                    // Disables assertions and test case registration
CATCH_CONFIG_WCHAR                      // Enables use of wchart_t
CATCH_CONFIG_EXPERIMENTAL_REDIRECT      // Enables the new (experimental) way of capturing stdout/stderr
CATCH_CONFIG_USE_ASYNC                  // Force parallel statistical processing of samples during benchmarking
CATCH_CONFIG_ANDROID_LOGWRITE           // Use android's logging system for debug output
CATCH_CONFIG_GLOBAL_NEXTAFTER           // Use nextafter{,f,l} instead of std::nextafter
CATCH_CONFIG_GETENV                     // System has a working `getenv`
CATCH_CONFIG_USE_BUILTIN_CONSTANT_P     // Use __builtin_constant_p to trigger warnings

CATCH_CONFIG_ANDROID_LOGWRITE and CATCH_CONFIG_GLOBAL_NEXTAFTER were introduced in Catch2 2.10.0

CATCH_CONFIG_GETENV was introduced in Catch2 3.2.0

CATCH_CONFIG_USE_BUILTIN_CONSTANT_P was introduced in Catch2 3.8.0

Currently Catch enables CATCH_CONFIG_WINDOWS_SEH only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support.

CATCH_CONFIG_POSIX_SIGNALS is on by default, except when Catch is compiled under Cygwin, where it is disabled by default (but can be force-enabled by defining CATCH_CONFIG_POSIX_SIGNALS).

CATCH_CONFIG_GETENV is on by default, except when Catch2 is compiled for
platforms that lacks working std::getenv (currently Windows UWP and
Playstation).

CATCH_CONFIG_WINDOWS_CRTDBG is off by default. If enabled, Windows's
CRT is used to check for memory leaks, and displays them after the tests
finish running. This option only works when linking against the default
main, and must be defined for the whole library build.

CATCH_CONFIG_WCHAR is on by default, but can be disabled. Currently
it is only used in support for DJGPP cross-compiler.

With the exception of CATCH_CONFIG_EXPERIMENTAL_REDIRECT,
these toggles can be disabled by using _NO_ form of the toggle,
e.g. CATCH_CONFIG_NO_WINDOWS_SEH.

CATCH_CONFIG_USE_BUILTIN_CONSTANT_P is ON by default for Clang and GCC
(but as far as possible, not for other compilers masquerading for these
two). However, it can cause bugs where the enclosed code is evaluated, even
though it should not be, e.g. in #2925.

`CATCH_CONFIG_FAST_COMPILE` Copy Section

This compile-time flag speeds up compilation of assertion macros by ~20%,
by disabling the generation of assertion-local try-catch blocks for
non-exception family of assertion macros ({REQUIRE,CHECK}{``,_FALSE, _THAT}).
This disables translation of exceptions thrown under these assertions, but
should not lead to false negatives.

CATCH_CONFIG_FAST_COMPILE has to be either defined, or not defined,
in all translation units that are linked into single test binary.

`CATCH_CONFIG_DISABLE_STRINGIFICATION` Copy Section

This toggle enables a workaround for VS 2017 bug. For details see known limitations.

`CATCH_CONFIG_DISABLE` Copy Section

This toggle removes most of Catch from given file. This means that TEST_CASEs are not registered and assertions are turned into no-ops. Useful for keeping tests within implementation files (ie for functions with internal linkage), instead of in external files.

This feature is considered experimental and might change at any point.

Inspired by Doctest's DOCTEST_CONFIG_DISABLE

Enabling stringification Copy Chapter

By default, Catch does not stringify some types from the standard library. This is done to avoid dragging in various standard library headers by default. However, Catch does contain these and can be configured to provide them, using these macros:

CATCH_CONFIG_ENABLE_PAIR_STRINGMAKER     // Provide StringMaker specialization for std::pair
CATCH_CONFIG_ENABLE_TUPLE_STRINGMAKER    // Provide StringMaker specialization for std::tuple
CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER  // Provide StringMaker specialization for std::variant, std::monostate (on C++17)
CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER // Provide StringMaker specialization for std::optional (on C++17)
CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS     // Defines all of the above

CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER was introduced in Catch2 2.4.1.

CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER was introduced in Catch2 2.6.0.

Disabling exceptions Copy Chapter

Introduced in Catch2 2.4.0.

By default, Catch2 uses exceptions to signal errors and to abort tests
when an assertion from the REQUIRE family of assertions fails. We also
provide an experimental support for disabling exceptions. Catch2 should
automatically detect when it is compiled with exceptions disabled, but
it can be forced to compile without exceptions by defining

CATCH_CONFIG_DISABLE_EXCEPTIONS

Note that when using Catch2 without exceptions, there are 2 major
limitations:

1. If there is an error that would normally be signalled by an exception,
   the exception's message will instead be written to Catch::cerr and
   std::terminate will be called.
2. If an assertion from the REQUIRE family of macros fails,
   std::terminate will be called after the active reporter returns.

There is also a customization point for the exact behaviour of what
happens instead of exception being thrown. To use it, define

CATCH_CONFIG_DISABLE_EXCEPTIONS_CUSTOM_HANDLER

and provide a definition for this function:

namespace Catch {
    [[noreturn]]
    void throw_exception(std::exception const&);
}

Disabling deprecation warnings Copy Chapter

Introduced in Catch2 3.9.0

Catch2 has started using the C++ macro [[deprecated]] to mark things
that are deprecated and should not be used any more. If you need to
temporarily disable these warnings, use

CATCH_CONFIG_NO_DEPRECATION_ANNOTATIONS

Catch2 currently does not support more fine-grained deprecation warning
control, nor do we plan to.

Overriding Catch's debug break (`-b`) Copy Chapter

Introduced in Catch2 2.11.2.

You can override Catch2's break-into-debugger code by defining the
CATCH_BREAK_INTO_DEBUGGER() macro. This can be used if e.g. Catch2 does
not know your platform, or your platform is misdetected.

The macro will be used as is, that is, CATCH_BREAK_INTO_DEBUGGER();
must compile and must break into debugger.

Static analysis support Copy Chapter

Introduced in Catch2 3.4.0.

Some parts of Catch2, e.g. SECTIONs, can be hard for static analysis
tools to reason about. Catch2 can change its internals to help static
analysis tools reason about the tests.

Catch2 automatically detects some static analysis tools (initial
implementation checks for clang-tidy and Coverity), but you can override
its detection (in either direction) via

CATCH_CONFIG_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT     // force enables static analysis help
CATCH_CONFIG_NO_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT  // force disables static analysis help

As the name suggests, this is currently experimental, and thus we provide
no backwards compatibility guarantees.

DO NOT ENABLE THIS FOR BUILDS YOU INTEND TO RUN. The changed internals
are not meant to be runnable, only "scannable".

Thread safety in assertions (and messages) Copy Chapter

Introduced in Catch2 3.9.0

Made non-experimental in Catch2 3.12.0

Catch2 can optionally support thread-safe assertions, that means, multiple
user-spawned threads can use the assertion macros at the same time. Due
to the performance cost this imposes even on single-threaded usage, Catch2
defaults to non-thread-safe assertions.

CATCH_CONFIG_THREAD_SAFE_ASSERTIONS     // enables thread safe assertions
CATCH_CONFIG_NO_THREAD_SAFE_ASSERTIONS  // force-disables thread safe assertions

See the documentation on thread safety in Catch2
for details on which macros are safe and other notes.

Home

File: docs/contributing.md Copy Chapter

Contributing to Catch2

Contents
Using Git(Hub)
Testing your changes
Writing documentation
Writing code
CoC
Using LLMs when contributing

So you want to contribute something to Catch2? That's great! Whether it's
a bug fix, a new feature, support for additional compilers - or just
a fix to the documentation - all contributions are very welcome and very
much appreciated. Of course so are bug reports, other comments, and
questions, but generally it is a better idea to ask questions in our
Discord, than in the issue tracker.

This page covers some guidelines and helpful tips for contributing
to the codebase itself.

Using Git(Hub) Copy Chapter

Ongoing development happens in the devel branch for Catch2 v3, and in
v2.x for maintenance updates to the v2 versions.

Commits should be small and atomic. A commit is atomic when, after it is
applied, the codebase, tests and all, still works as expected. Small
commits are also preferred, as they make later operations with git history,
whether it is bisecting, reverting, or something else, easier.

When submitting a pull request please do not include changes to the
amalgamated distribution files. This means do not include them in your
git commits!

When addressing review comments in a MR, please do not rebase/squash the
commits immediately. Doing so makes it harder to review the new changes,
slowing down the process of merging a MR. Instead, when addressing review
comments, you should append new commits to the branch and only squash
them into other commits when the MR is ready to be merged. We recommend
creating new commits with git commit --fixup (or --squash) and then
later squashing them with git rebase --autosquash to make things easier.

Testing your changes Copy Chapter

Note: Running Catch2's tests requires Python3

Catch2 has multiple layers of tests that are then run as part of our CI.
The most obvious one are the unit tests compiled into the SelfTest
binary. These are then used in "Approval tests", which run (almost) all
tests from SelfTest through a specific reporter and then compare the
generated output with a known good output ("Baseline"). By default, new
tests should be placed here.

To configure a Catch2 build with just the basic tests, use the basic-tests
preset, like so:

# Assuming you are in Catch2's root folder

cmake -B basic-test-build -S . -DCMAKE_BUILD_TYPE=Debug --preset basic-tests

However, not all tests can be written as plain unit tests. For example,
checking that Catch2 orders tests randomly when asked to, and that this
random ordering is subset-invariant, is better done as an integration
test using an external check script. Catch2 integration tests are written
using CTest, either as a direct command invocation + pass/fail regex,
or by delegating the check to a Python script.

Catch2 is slowly gaining more and more types of tests, currently Catch2
project also has buildable examples, "ExtraTests", and CMake config tests.
Examples present a small and self-contained snippets of code that
use Catch2's facilities for specific purpose. Currently they are assumed
passing if they compile.

ExtraTests then are expensive tests, that we do not want to run all the
time. This can be either because they take a long time to run, or because
they take a long time to compile, e.g. because they test compile time
configuration and require separate compilation.

Finally, CMake config tests test that you set Catch2's compile-time
configuration options through CMake, using CMake options of the same name.

These test categories can be enabled one by one, by passing
-DCATCH_BUILD_EXAMPLES=ON, -DCATCH_BUILD_EXTRA_TESTS=ON, and
-DCATCH_ENABLE_CONFIGURE_TESTS=ON when configuring the build.

Catch2 also provides a preset that promises to enable all test types,
all-tests.

The snippet below will build & run all tests, in Debug compilation mode.

# 1. Regenerate the amalgamated distribution (some tests are built against it)
./tools/scripts/generateAmalgamatedFiles.py

# 2. Configure the full test build
cmake -B debug-build -S . -DCMAKE_BUILD_TYPE=Debug --preset all-tests

# 3. Run the actual build
cmake --build debug-build

# 4. Run the tests using CTest
ctest -j 4 --output-on-failure -C Debug --test-dir debug-build

^{snippet source | anchor}

For convenience, the above commands are in the script tools/scripts/buildAndTest.sh, and can be run like this:

cd Catch2
./tools/scripts/buildAndTest.sh

A Windows version of the script is available at tools\scripts\buildAndTest.cmd.

If you added new tests, you will likely see ApprovalTests failure.
After you check that the output difference is expected, you should
run tools/scripts/approve.py to confirm them, and include these changes
in your commit.

Writing documentation Copy Chapter

If you have added new feature to Catch2, it needs documentation, so that
other people can use it as well. This section collects some technical
information that you will need for updating Catch2's documentation, and
possibly some generic advise as well.

Technicalities Copy Section

First, the technicalities:

• If you have introduced a new document, there is a simple template you
  should use. It provides you with the top anchor mentioned to link to
  (more below), and also with a backlink to the top of the documentation:

<a id="top"></a>
# Cool feature

> [Introduced](https://github.com/catchorg/Catch2/pull/123456) in Catch2 X.Y.Z

Text that explains how to use the cool feature.


---

[Home](Readme.md#top)

• Crosslinks to different pages should target the top anchor, like this
  [link to contributing](contributing.md#top).

• We introduced version tags to the documentation, which show users in
  which version a specific feature was introduced. This means that newly
  written documentation should be tagged with a placeholder, that will
  be replaced with the actual version upon release. There are 2 styles
  of placeholders used through the documentation, you should pick one that
  fits your text better (if in doubt, take a look at the existing version
  tags for other features).

  • > [Introduced](link-to-issue-or-PR) in Catch2 X.Y.Z - this
    placeholder is usually used after a section heading
  • > X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch2 X.Y.Z
  • this placeholder is used when you need to tag a subpart of something,
    e.g. a list

• For pages with more than 4 subheadings, we provide a table of contents
  (ToC) at the top of the page. Because GitHub markdown does not support
  automatic generation of ToC, it has to be handled semi-manually. Thus,
  if you've added a new subheading to some page, you should add it to the
  ToC. This can be done either manually, or by running the
  updateDocumentToC.py script in the scripts/ folder.

Contents Copy Section

Now, for some content tips:

• Usage examples are good. However, having large code snippets inline
  can make the documentation less readable, and so the inline snippets
  should be kept reasonably short. To provide more complex compilable
  examples, consider adding new .cpp file to examples/.

• Don't be afraid to introduce new pages. The current documentation
  tends towards long pages, but a lot of that is caused by legacy, and
  we know that some of the pages are overly big and unfocused.

• When adding information to an existing page, please try to keep your
  formatting, style and changes consistent with the rest of the page.

• Any documentation has multiple different audiences, that desire
  different information from the text. The 3 basic user-types to try and
  cover are:

  • A beginner to Catch2, who requires closer guidance for the usage of Catch2.
  • Advanced user of Catch2, who want to customize their usage.
  • Experts, looking for full reference of Catch2's capabilities.

Writing code Copy Chapter

If want to contribute code, this section contains some simple rules
and tips on things like code formatting, code constructions to avoid,
and so on.

C++ standard version Copy Section

Catch2 currently targets C++14 as the minimum supported C++ version.
Features from higher language versions should be used only sparingly,
when the benefits from using them outweigh the maintenance overhead.

Example of good use of polyfilling features is our use of conjunction,
where if available we use std::conjunction and otherwise provide our
own implementation. The reason it is good is that the surface area for
maintenance is quite small, and std::conjunction can directly use
compiler built-ins, thus providing significant compilation benefits.

Example of bad use of polyfilling features would be to keep around two
sets of metaprogramming in the stringification implementation, once
using C++14 compliant TMP and once using C++17's if constexpr. While
the C++17 would provide significant compilation speedups, the maintenance
cost would be too high.

Formatting Copy Section

To make code formatting simpler for the contributors, Catch2 provides
its own config for clang-format. However, because it is currently
impossible to replicate existing Catch2's formatting in clang-format,
using it to reformat a whole file would cause massive diffs. To keep
the size of your diffs reasonable, you should only use clang-format
on the newly changed code.

Code constructs to watch out for Copy Section

This section is a (sadly incomplete) listing of various constructs that
are problematic and are not always caught by our CI infrastructure.

Naked exceptions and exceptions-related function

If you are throwing an exception, it should be done via CATCH_ERROR
or CATCH_RUNTIME_ERROR in internal/catch_enforce.hpp. These macros will handle
the differences between compilation with or without exceptions for you.
However, some platforms (IAR) also have problems with exceptions-related
functions, such as std::current_exceptions. We do not have IAR in our
CI, but luckily there should not be too many reasons to use these.
However, if you do, they should be kept behind a
CATCH_CONFIG_DISABLE_EXCEPTIONS macro.

Avoid `std::move` and `std::forward`

std::move and std::forward provide nice semantic name for a specific
static_cast. However, being function templates they have surprisingly
high cost during compilation, and can also have a negative performance
impact for low-optimization builds.

You should be using CATCH_MOVE and CATCH_FORWARD macros from
internal/catch_move_and_forward.hpp instead. They expand into the proper
static_cast, and avoid the overhead of std::move and std::forward.

Unqualified usage of functions from C's stdlib

If you are using a function from C's stdlib, please include the header
as <cfoo> and call the function qualified. The common knowledge that
there is no difference is wrong, QNX and VxWorks won't compile if you
include the header as <cfoo> and call the function unqualified.

User-Defined Literals (UDL) for Catch2' types

Due to messy standardese and ... not great ... implementation of
-Wreserved-identifier in Clang, avoid declaring UDLs as

Approx operator "" _a(long double);

and instead declare them as

Approx operator ""_a(long double);

Notice that the second version does not have a space between the "" and
the literal suffix.

New source file template Copy Section

If you are adding new source file, there is a template you should use.
Specifically, every source file should start with the licence header:

//              Copyright Catch2 Authors
    // Distributed under the Boost Software License, Version 1.0.
    //   (See accompanying file LICENSE.txt or copy at
    //        https://www.boost.org/LICENSE_1_0.txt)

    // SPDX-License-Identifier: BSL-1.0

The include guards for header files should follow the pattern {FILENAME}_INCLUDED.
This means that for file catch_matchers_foo.hpp, the include guard should
be CATCH_MATCHERS_FOO_HPP_INCLUDED, for catch_generators_bar.hpp, the include
guard should be CATCH_GENERATORS_BAR_HPP_INCLUDED, and so on.

Adding new `CATCH_CONFIG` option Copy Section

When adding new CATCH_CONFIG option, there are multiple places to edit:

• CMake/CatchConfigOptions.cmake - this is used to generate the
  configuration options in CMake, so that CMake frontends know about them.
• docs/configuration.md - this is where the options are documented
• src/catch2/catch_user_config.hpp.in - this is template for generating
  catch_user_config.hpp which contains the materialized configuration
• BUILD.bazel - Bazel does not have configuration support like CMake,
  and all expansions need to be done manually
• other files as needed, e.g. catch2/internal/catch_config_foo.hpp
  for the logic that guards the configuration

CoC Copy Chapter

This project has a CoC. Please adhere to it
while contributing to Catch2.

Using LLMs when contributing Copy Chapter

I do not care whether you used LLM for your contribution. What I care
about is the quality of the contribution, not whether you made it through
prompting LLM, letting GAs run wild, dictated it into the computer, wrote
it using your nose or used butterflies to flip the correct bits.

The flipside of this is that I am also not going to iterate your LLM for
you. If a PR looks LLM generated and does not pass the muster to be merged
as-is, I am going to close it.

This documentation will always be in-progress as new information comes
up, but we are trying to keep it as up to date as possible.

Home