## File: README.md [](https://www.swift.org/sswg/incubation-process.html#graduated-level) # SwiftNIO SwiftNIO is a cross-platform asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients. It's like [Netty](https://netty.io), but written for Swift. ### Repository organization The SwiftNIO project is split across multiple repositories: Repository | NIO 2 --- | --- [https://github.com/apple/swift-nio][repo-nio] SwiftNIO core | `from: "2.0.0"` [https://github.com/apple/swift-nio-ssl][repo-nio-ssl] TLS (SSL) support | `from: "2.0.0"` [https://github.com/apple/swift-nio-http2][repo-nio-http2] HTTP/2 support | `from: "1.0.0"` [https://github.com/apple/swift-nio-extras][repo-nio-extras] useful additions around SwiftNIO | `from: "1.0.0"` [https://github.com/apple/swift-nio-transport-services][repo-nio-transport-services] first-class support for macOS, iOS, tvOS, and watchOS | `from: "1.0.0"` [https://github.com/apple/swift-nio-ssh][repo-nio-ssh] SSH support | `.upToNextMinor(from: "0.2.0")` [https://github.com/apple/swift-nio-quic][repo-nio-quic] QUIC support (in active development) | `branch: "main"` [https://github.com/apple/swift-nio-http3][repo-nio-http3] HTTP/3 support (in active development) | `branch: "main"` Within this repository we have a number of products that provide different functionality. This package contains the following products: - `NIO`. This is an umbrella module exporting `NIOCore`, `NIOEmbedded` and `NIOPosix`. - `NIOCore`. This provides the core abstractions and types for using SwiftNIO (see ["Conceptual Overview"](#conceptual-overview) for more details). Most NIO extension projects that provide things like new [`EventLoop`s][el] and [`Channel`s][c] or new protocol implementations should only need to depend on `NIOCore`. - `NIOPosix`. This provides the primary [`EventLoopGroup`], [`EventLoop`][el], and [`Channel`s][c] for use on POSIX-based systems. This is our high performance core I/O layer. In general, this should only be imported by projects that plan to do some actual I/O, such as high-level protocol implementations or applications. - `NIOEmbedded`. This provides [`EmbeddedChannel`][ec] and [`EmbeddedEventLoop`][eel], implementations of the `NIOCore` abstractions that provide fine-grained control over their execution. These are most often used for testing, but can also be used to drive protocol implementations in a way that is decoupled from networking altogether. - `NIOConcurrencyHelpers`. This provides a few low-level concurrency primitives that are used by NIO implementations, such as locks and atomics. - `NIOFoundationCompat`. This extends a number of NIO types for better interoperation with Foundation data types. If you are working with Foundation data types such as `Data`, you should import this. - `NIOTLS`. This provides a few common abstraction types for working with multiple TLS implementations. Note that this doesn't provide TLS itself: please investigate [swift-nio-ssl][repo-nio-ssl] and [swift-nio-transport-services][repo-nio-transport-services] for concrete implementations. - `NIOHTTP1`. This provides a low-level HTTP/1.1 protocol implementation. - `NIOWebSocket`. This provides a low-level WebSocket protocol implementation. - `NIOTestUtils`. This provides a number of helpers for testing projects that use SwiftNIO. - `_NIOFileSystem`. This provides `async` APIs for interacting with the file system. ### Protocol Implementations Below you can find a list of a few protocol implementations that are done with SwiftNIO. This is a non-exhaustive list of protocols that are either part of the SwiftNIO project or are accepted into the [SSWG](https://swift.org/server)'s incubation process. All of the libraries listed below do all of their I/O in a non-blocking fashion using SwiftNIO. #### Low-level protocol implementations Low-level protocol implementations are often a collection of [`ChannelHandler`][ch]s that implement a protocol but still require the user to have a good understanding of SwiftNIO. Often, low-level protocol implementations will then be wrapped in high-level libraries with a nicer, more user-friendly API. Protocol | Client(Sends requests) | Server(Responds to requests) | Repository | Module | Comment --- | --- | --- | --- | --- | --- HTTP/1 | ✅| ✅ | [apple/swift-nio](https://github.com/apple/swift-nio) | [`NIOHTTP1`][nioh1] | official NIO project HTTP/2 | ✅| ✅ | [apple/swift-nio-http2](https://github.com/apple/swift-nio-http2) | [`NIOHTTP2`][nioh2] | official NIO project WebSocket | ✅| ✅ | [apple/swift-nio](https://github.com/apple/swift-nio) | [`NIOWebSocket`][niows] | official NIO project TLS | ✅ | ✅ | [apple/swift-nio-ssl](https://github.com/apple/swift-nio-ssl) | [`NIOSSL`][niossl] | official NIO project SSH | ✅ | ✅ | [apple/swift-nio-ssh][repo-nio-ssh] | [`NIOSSH`][niossh] | official NIO project #### High-level implementations High-level implementations are usually libraries that come with an API that doesn't expose SwiftNIO's [`ChannelPipeline`][cp] and can therefore be used with very little (or no) SwiftNIO-specific knowledge. The implementations listed below do still do all of their I/O in SwiftNIO and integrate really well with the SwiftNIO ecosystem. Protocol | Client(Sends requests) | Server(Responds to requests) | Repository | Module | Comment --- | --- | --- | --- | --- | --- HTTP | ✅| ❌ | [swift-server/async-http-client](https://github.com/swift-server/async-http-client) | `AsyncHTTPClient` | SSWG community project gRPC | ✅| ✅ | [grpc/grpc-swift](https://github.com/grpc/grpc-swift) | `GRPC` | also offers a low-level API; SSWG community project APNS | ✅ | ❌ | [swift-server-community/APNSwift](https://github.com/swift-server-community/APNSwift) | `APNSwift` | SSWG community project PostgreSQL | ✅ | ❌ | [vapor/postgres-nio](https://github.com/vapor/postgres-nio) | `PostgresNIO` | SSWG community project Redis | ✅ | ❌ | [swift-server/RediStack](https://github.com/swift-server/RediStack) | `RediStack` | SSWG community project ### Supported Versions ### SwiftNIO 2 This is the current version of SwiftNIO and will be supported for the foreseeable future. ### Swift Versions We commit to support the most recently released Swift version and the last two minor releases before that unless this is impossible to do in one codebase. In addition checks are run against the latest beta release (if any) as well as the nightly Swift builds and the intent is that these should pass. The minimum Swift version supported by SwiftNIO releases are detailed below: SwiftNIO | Minimum Swift Version --------------------|---------------------- `2.0.0 ..< 2.30.0` | 5.0 `2.30.0 ..< 2.40.0` | 5.2 `2.40.0 ..< 2.43.0` | 5.4 `2.43.0 ..< 2.51.0` | 5.5.2 `2.51.0 ..< 2.60.0` | 5.6 `2.60.0 ..< 2.65.0` | 5.7 `2.65.0 ..< 2.76.0` | 5.8 `2.76.0 ..< 2.83.0` | 5.9 `2.83.0 ..< 2.87.0` | 5.10 `2.87.0 ..< 2.98.0` | 6.0 `2.98.0 ...` | 6.1 ### SwiftNIO 1 SwiftNIO 1 is considered end of life - it is strongly recommended that you move to a newer version. The Core NIO team does not actively work on this version. No new features will be added to this version but PRs which fix bugs or security vulnerabilities will be accepted until the end of May 2022. If you have a SwiftNIO 1 application or library that you would like to migrate to SwiftNIO 2, please check out the [migration guide](docs/migration-guide-NIO1-to-NIO2.md) we prepared for you. The latest released SwiftNIO 1 version supports Swift 4.0, 4.1, 4.2, and 5.0. ### Supported Platforms SwiftNIO aims to support all of the platforms where Swift is supported. Currently, it is developed and tested on macOS and Linux, and is known to support the following operating system versions: * Ubuntu 18.04+ * macOS 10.9+, iOS 7+; (macOS 10.14+, iOS 12+, tvOS 12+ or watchOS 6+ with [swift-nio-transport-services][repo-nio-transport-services]) SwiftNIO has experimental support on OpenBSD for all SwiftNIO libraries _except_ for `_NIOFileSystem`, which is not yet supported. You can use all other SwiftNIO libraries on OpenBSD by adding them as dependencies in `Package.swift`. ### Compatibility SwiftNIO follows [SemVer 2.0.0](https://semver.org/#semantic-versioning-200) with a separate document declaring [SwiftNIO's Public API](docs/public-api.md). What this means for you is that you should depend on SwiftNIO with a version range that covers everything from the minimum SwiftNIO version you require up to the next major version. In SwiftPM that can be easily done specifying for example `from: "2.0.0"` meaning that you support SwiftNIO in every version starting from 2.0.0 up to (excluding) 3.0.0. SemVer and SwiftNIO's Public API guarantees should result in a working program without having to worry about testing every single version for compatibility. ## Conceptual Overview SwiftNIO is fundamentally a low-level tool for building high-performance networking applications in Swift. It particularly targets those use-cases where using a "thread-per-connection" model of concurrency is inefficient or untenable. This is a common limitation when building servers that use a large number of relatively low-utilization connections, such as HTTP servers. To achieve its goals, SwiftNIO extensively uses "non-blocking I/O": hence the name! Non-blocking I/O differs from the more common blocking I/O model because the application does not wait for data to be sent to or received from the network: instead, SwiftNIO asks for the kernel to notify it when I/O operations can be performed without waiting. SwiftNIO does not aim to provide high-level solutions like, for example, web frameworks do. Instead, SwiftNIO is focused on providing the low-level building blocks for these higher-level applications. When it comes to building a web application, most users will not want to use SwiftNIO directly: instead, they'll want to use one of the many great web frameworks available in the Swift ecosystem. Those web frameworks, however, may choose to use SwiftNIO under the covers to provide their networking support. The following sections will describe the low-level tools that SwiftNIO provides, and provide a quick overview of how to work with them. If you feel comfortable with these concepts, then you can skip right ahead to the other sections of this README. ### Basic Architecture The basic building blocks of SwiftNIO are the following 8 types of objects: - [`EventLoopGroup`][elg], a protocol, provided by `NIOCore`. - [`EventLoop`][el], a protocol, provided by `NIOCore`. - [`Channel`][c], a protocol, provided by `NIOCore`. - [`ChannelHandler`][ch], a protocol, provided by `NIOCore`. - `Bootstrap`, several related structures, provided by `NIOCore`. - [`ByteBuffer`][bb], a struct, provided by `NIOCore`. - [`EventLoopFuture`][elf], a generic class, provided by `NIOCore`. - [`EventLoopPromise`][elp], a generic struct, provided by `NIOCore`. All SwiftNIO applications are ultimately constructed of these various components. #### EventLoops and EventLoopGroups The basic I/O primitive of SwiftNIO is the event loop. The event loop is an object that waits for events (usually I/O related events, such as "data received") to happen and then fires some kind of callback when they do. In almost all SwiftNIO applications there will be relatively few event loops: usually only one or two per CPU core the application wants to use. Generally speaking, event loops run for the entire lifetime of your application, spinning in an endless loop dispatching events. Event loops are gathered together into event loop *groups*. These groups provide a mechanism to distribute work around the event loops. For example, when listening for inbound connections the listening socket will be registered on one event loop. However, we don't want all connections that are accepted on that listening socket to be registered with the same event loop, as that would potentially overload one event loop while leaving the others empty. For that reason, the event loop group provides the ability to spread load across multiple event loops. In SwiftNIO today there is one [`EventLoopGroup`][elg] implementation, and two [`EventLoop`][el] implementations. For production applications there is the [`MultiThreadedEventLoopGroup`][mtelg], an [`EventLoopGroup`][elg] that creates a number of threads (using the POSIX [`pthreads`][pthreads] library) and places one `SelectableEventLoop` on each one. The `SelectableEventLoop` is an event loop that uses a selector (either [`kqueue`][kqueue] or [`epoll`][epoll] depending on the target system) to manage I/O events from file descriptors and to dispatch work. These [`EventLoop`s][el] and [`EventLoopGroup`s][elg] are provided by the `NIOPosix` module. Additionally, there is the [`EmbeddedEventLoop`][eel], which is a dummy event loop that is used primarily for testing purposes, provided by the `NIOEmbedded` module. [`EventLoop`][el]s have a number of important properties. Most vitally, they are the way all work gets done in SwiftNIO applications. In order to ensure thread-safety, any work that wants to be done on almost any of the other objects in SwiftNIO must be dispatched via an [`EventLoop`][el]. [`EventLoop`][el] objects own almost all the other objects in a SwiftNIO application, and understanding their execution model is critical for building high-performance SwiftNIO applications. #### Channels, Channel Handlers, Channel Pipelines, and Channel Contexts While [`EventLoop`][el]s are critical to the way SwiftNIO works, most users will not interact with them substantially beyond asking them to create [`EventLoopPromise`][elp]s and to schedule work. The parts of a SwiftNIO application most users will spend the most time interacting with are [`Channel`][c]s and [`ChannelHandler`][ch]s. Almost every file descriptor that a user interacts with in a SwiftNIO program is associated with a single [`Channel`][c]. The [`Channel`][c] owns this file descriptor, and is responsible for managing its lifetime. It is also responsible for processing inbound and outbound events on that file descriptor: whenever the event loop has an event that corresponds to a file descriptor, it will notify the [`Channel`][c] that owns that file descriptor. [`Channel`][c]s by themselves, however, are not useful. After all, it is a rare application that doesn't want to do anything with the data it sends or receives on a socket! So the other important part of the [`Channel`][c] is the [`ChannelPipeline`][cp]. A [`ChannelPipeline`][cp] is a sequence of objects, called [`ChannelHandler`][ch]s, that process events on a [`Channel`][c]. The [`ChannelHandler`][ch]s process these events one after another, in order, mutating and transforming events as they go. This can be thought of as a data processing pipeline; hence the name [`ChannelPipeline`][cp]. All [`ChannelHandler`][ch]s are either Inbound or Outbound handlers, or both. Inbound handlers process "inbound" events: events like reading data from a socket, reading socket close, or other kinds of events initiated by remote peers. Outbound handlers process "outbound" events, such as writes, connection attempts, and local socket closes. Each handler processes the events in order. For example, read events are passed from the front of the pipeline to the back, one handler at a time, while write events are passed from the back of the pipeline to the front. Each handler may, at any time, generate either inbound or outbound events that will be sent to the next handler in whichever direction is appropriate. This allows handlers to split up reads, coalesce writes, delay connection attempts, and generally perform arbitrary transformations of events. In general, [`ChannelHandler`][ch]s are designed to be highly re-usable components. This means they tend to be designed to be as small as possible, performing one specific data transformation. This allows handlers to be composed together in novel and flexible ways, which helps with code reuse and encapsulation. [`ChannelHandler`][ch]s are able to keep track of where they are in a [`ChannelPipeline`][cp] by using a [`ChannelHandlerContext`][chc]. These objects contain references to the previous and next channel handler in the pipeline, ensuring that it is always possible for a [`ChannelHandler`][ch] to emit events while it remains in a pipeline. SwiftNIO ships with many [`ChannelHandler`][ch]s built in that provide useful functionality, such as HTTP parsing. In addition, high-performance applications will want to provide as much of their logic as possible in [`ChannelHandler`][ch]s, as it helps avoid problems with context switching. Additionally, SwiftNIO ships with a few [`Channel`][c] implementations. In particular, it ships with `ServerSocketChannel`, a [`Channel`][c] for sockets that accept inbound connections; `SocketChannel`, a [`Channel`][c] for TCP connections; and `DatagramChannel`, a [`Channel`][c] for UDP sockets. All of these are provided by the `NIOPosix` module. It also provides [`EmbeddedChannel`][ec], a [`Channel`][c] primarily used for testing, provided by the `NIOEmbedded` module. ##### A Note on Blocking One of the important notes about [`ChannelPipeline`][cp]s is that they are thread-safe. This is very important for writing SwiftNIO applications, as it allows you to write much simpler [`ChannelHandler`][ch]s in the knowledge that they will not require synchronization. However, this is achieved by dispatching all code on the [`ChannelPipeline`][cp] on the same thread as the [`EventLoop`][el]. This means that, as a general rule, [`ChannelHandler`][ch]s **must not** call blocking code without dispatching it to a background thread. If a [`ChannelHandler`][ch] blocks for any reason, all [`Channel`][c]s attached to the parent [`EventLoop`][el] will be unable to progress until the blocking call completes. This is a common concern while writing SwiftNIO applications. If it is useful to write code in a blocking style, it is highly recommended that you dispatch work to a different thread when you're done with it in your pipeline. #### Bootstrap While it is possible to configure and register [`Channel`][c]s with [`EventLoop`][el]s directly, it is generally more useful to have a higher-level abstraction to handle this work. For this reason, SwiftNIO ships a number of `Bootstrap` objects whose purpose is to streamline the creation of channels. Some `Bootstrap` objects also provide other functionality, such as support for Happy Eyeballs for making TCP connection attempts. Currently SwiftNIO ships with three `Bootstrap` objects in the `NIOPosix` module: [`ServerBootstrap`][sbootstrap], for bootstrapping listening channels; [`ClientBootstrap`][cbootstrap], for bootstrapping client TCP channels; and [`DatagramBootstrap`][dbootstrap] for bootstrapping UDP channels. #### ByteBuffer The majority of the work in a SwiftNIO application involves shuffling buffers of bytes around. At the very least, data is sent and received to and from the network in the form of buffers of bytes. For this reason it's very important to have a high-performance data structure that is optimized for the kind of work SwiftNIO applications perform. For this reason, SwiftNIO provides [`ByteBuffer`][bb], a fast copy-on-write byte buffer that forms a key building block of most SwiftNIO applications. This type is provided by the `NIOCore` module. [`ByteBuffer`][bb] provides a number of useful features, and in addition provides a number of hooks to use it in an "unsafe" mode. This turns off bounds checking for improved performance, at the cost of potentially opening your application up to memory correctness problems. In general, it is highly recommended that you use the [`ByteBuffer`][bb] in its safe mode at all times. For more details on the API of [`ByteBuffer`][bb], please see our API documentation, linked below. #### Promises and Futures One major difference between writing concurrent code and writing synchronous code is that not all actions will complete immediately. For example, when you write data on a channel, it is possible that the event loop will not be able to immediately flush that write out to the network. For this reason, SwiftNIO provides [`EventLoopPromise`][elp] and [`EventLoopFuture`][elf] to manage operations that complete *asynchronously*. These types are provided by the `NIOCore` module. An [`EventLoopFuture`][elf] is essentially a container for the return value of a function that will be populated *at some time in the future*. Each [`EventLoopFuture`][elf] has a corresponding [`EventLoopPromise`][elp], which is the object that the result will be put into. When the promise is succeeded, the future will be fulfilled. If you had to poll the future to detect when it completed that would be quite inefficient, so [`EventLoopFuture`][elf] is designed to have managed callbacks. Essentially, you can chain callbacks off the future that will be executed when a result is available. The [`EventLoopFuture`][elf] will even carefully arrange the scheduling to ensure that these callbacks always execute on the event loop that initially created the promise, which helps ensure that you don't need too much synchronization around [`EventLoopFuture`][elf] callbacks. Another important topic for consideration is the difference between how the promise passed to `close` works as opposed to `closeFuture` on a [`Channel`][c]. For example, the promise passed into `close` will succeed after the [`Channel`][c] is closed down but before the [`ChannelPipeline`][cp] is completely cleared out. This will allow you to take action on the [`ChannelPipeline`][cp] before it is completely cleared out, if needed. If it is desired to wait for the [`Channel`][c] to close down and the [`ChannelPipeline`][cp] to be cleared out without any further action, then the better option would be to wait for the `closeFuture` to succeed. There are several functions for applying callbacks to [`EventLoopFuture`][elf], depending on how and when you want them to execute. Details of these functions is left to the API documentation. ### Design Philosophy SwiftNIO is designed to be a powerful tool for building networked applications and frameworks, but it is not intended to be the perfect solution for all levels of abstraction. SwiftNIO is tightly focused on providing the basic I/O primitives and protocol implementations at low levels of abstraction, leaving more expressive but slower abstractions to the wider community to build. The intention is that SwiftNIO will be a building block for server-side applications, not necessarily the framework those applications will use directly. Applications that need extremely high performance from their networking stack may choose to use SwiftNIO directly in order to reduce the overhead of their abstractions. These applications should be able to maintain extremely high performance with relatively little maintenance cost. SwiftNIO also focuses on providing useful abstractions for this use-case, such that extremely high performance network servers can be built directly. The core SwiftNIO repository will contain a few extremely important protocol implementations, such as HTTP, directly in tree. However, we believe that most protocol implementations should be decoupled from the release cycle of the underlying networking stack, as the release cadence is likely to be very different (either much faster or much slower). For this reason, we actively encourage the community to develop and maintain their protocol implementations out-of-tree. Indeed, some first-party SwiftNIO protocol implementations, including our TLS and HTTP/2 bindings, are developed out-of-tree! ## Documentation - [API documentation](https://swiftpackageindex.com/apple/swift-nio/documentation/nio) ## Example Usage There are currently several example projects that demonstrate how to use SwiftNIO. - **chat client** https://github.com/apple/swift-nio/tree/main/Sources/NIOChatClient - **chat server** https://github.com/apple/swift-nio/tree/main/Sources/NIOChatServer - **echo client** https://github.com/apple/swift-nio/tree/main/Sources/NIOEchoClient - **echo server** https://github.com/apple/swift-nio/tree/main/Sources/NIOEchoServer - **UDP echo client** https://github.com/apple/swift-nio/tree/main/Sources/NIOUDPEchoClient - **UDP echo server** https://github.com/apple/swift-nio/tree/main/Sources/NIOUDPEchoServer - **HTTP client** https://github.com/apple/swift-nio/tree/main/Sources/NIOHTTP1Client - **HTTP server** https://github.com/apple/swift-nio/tree/main/Sources/NIOHTTP1Server - **WebSocket client** https://github.com/apple/swift-nio/tree/main/Sources/NIOWebSocketClient - **WebSocket server** https://github.com/apple/swift-nio/tree/main/Sources/NIOWebSocketServer To build & run them, run following command, replace TARGET_NAME with the folder name under `./Sources` ```bash swift run TARGET_NAME ``` For example, to run NIOHTTP1Server, run following command: ```bash swift run NIOHTTP1Server ``` ## Getting Started SwiftNIO primarily uses [SwiftPM](https://swift.org/package-manager/) as its build tool, so we recommend using that as well. If you want to depend on SwiftNIO in your own project, it's as simple as adding a `dependencies` clause to your `Package.swift`: ```swift dependencies: [ .package(url: "https://github.com/apple/swift-nio.git", from: "2.0.0") ] ``` and then adding the appropriate SwiftNIO module(s) to your target dependencies. The syntax for adding target dependencies differs slightly between Swift versions. For example, if you want to depend on the `NIOCore`, `NIOPosix` and `NIOHTTP1` modules, specify the following dependencies: #### Swift 5.4 and newer (`swift-tools-version:5.4`) dependencies: [.product(name: "NIOCore", package: "swift-nio"), .product(name: "NIOPosix", package: "swift-nio"), .product(name: "NIOHTTP1", package: "swift-nio")] ### Using Xcode Package support If your project is set up as an Xcode project and you're using Xcode 11+, you can add SwiftNIO as a dependency to your Xcode project by clicking File -> Swift Packages -> Add Package Dependency. In the upcoming dialog, please enter `https://github.com/apple/swift-nio.git` and click Next twice. Finally, select the targets you are planning to use (for example `NIOCore`, `NIOHTTP1`, and `NIOFoundationCompat`) and click finish. Now will be able to `import NIOCore` (as well as all the other targets you have selected) in your project. To work on SwiftNIO itself, or to investigate some of the demonstration applications, you can clone the repository directly and use SwiftPM to help build it. For example, you can run the following commands to compile and run the example echo server: ```bash swift build swift test swift run NIOEchoServer ``` To verify that it is working, you can use another shell to attempt to connect to it: ```bash echo "Hello SwiftNIO" | nc localhost 9999 ``` If all goes well, you'll see the message echoed back to you. To work on SwiftNIO in Xcode, you can just open the `Package.swift` file in Xcode and use Xcode's support for SwiftPM Packages. ## Developing SwiftNIO *Note*: This section is only relevant if you would like to develop SwiftNIO yourself. You can ignore the information here if you just want to use SwiftNIO as a SwiftPM package. For the most part, SwiftNIO development is as straightforward as any other SwiftPM project. With that said, we do have a few processes that are worth understanding before you contribute. For details, please see `CONTRIBUTING.md` in this repository. ### Prerequisites SwiftNIO's `main` branch is the development branch for the next releases of SwiftNIO 2, it's Swift 5-only. To be able to compile and run SwiftNIO and the integration tests, you need to have a few prerequisites installed on your system. #### macOS - Xcode 11.4 or newer, Xcode 12 recommended. ### Linux - Swift 5.7 or newer from [swift.org/download](https://swift.org/download/#releases). We always recommend to use the latest released version. - netcat (for integration tests only) - lsof (for integration tests only) - shasum (for integration tests only) #### Ubuntu 18.04 ``` # install swift tarball from https://swift.org/downloads apt-get install -y git curl libatomic1 libxml2 netcat-openbsd lsof perl ``` ### Fedora 28+ ``` dnf install swift-lang /usr/bin/nc /usr/bin/lsof /usr/bin/shasum ``` [ch]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/channelhandler [c]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/channel [chc]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/channelhandlercontext [ec]: https://swiftpackageindex.com/apple/swift-nio/documentation/nioembedded/embeddedchannel [el]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/eventloop [eel]: https://swiftpackageindex.com/apple/swift-nio/documentation/nioembedded/embeddedeventloop [elg]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/eventloopgroup [bb]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/bytebuffer [elf]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/eventloopfuture [elp]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/eventlooppromise [cp]: https://swiftpackageindex.com/apple/swift-nio/documentation/niocore/channelpipeline [sbootstrap]: https://swiftpackageindex.com/apple/swift-nio/documentation/nioposix/serverbootstrap [cbootstrap]: https://swiftpackageindex.com/apple/swift-nio/documentation/nioposix/clientbootstrap [dbootstrap]: https://swiftpackageindex.com/apple/swift-nio/documentation/nioposix/datagrambootstrap [mtelg]: https://swiftpackageindex.com/apple/swift-nio/documentation/nioposix/multithreadedeventloopgroup [nioh1]: https://swiftpackageindex.com/apple/swift-nio/documentation/niohttp1 [nioh2]: https://swiftpackageindex.com/apple/swift-nio-http2/documentation/niohttp2 [niows]: https://swiftpackageindex.com/apple/swift-nio/documentation/niowebsocket [niossl]: https://swiftpackageindex.com/apple/swift-nio-ssl/documentation/niossl [niossh]: https://swiftpackageindex.com/apple/swift-nio-ssh/documentation/niossh [pthreads]: https://en.wikipedia.org/wiki/POSIX_Threads [kqueue]: https://en.wikipedia.org/wiki/Kqueue [epoll]: https://en.wikipedia.org/wiki/Epoll [repo-nio]: https://github.com/apple/swift-nio [repo-nio-extras]: https://github.com/apple/swift-nio-extras [repo-nio-http2]: https://github.com/apple/swift-nio-http2 [repo-nio-ssl]: https://github.com/apple/swift-nio-ssl [repo-nio-transport-services]: https://github.com/apple/swift-nio-transport-services [repo-nio-ssh]: https://github.com/apple/swift-nio-ssh [repo-nio-quic]: https://github.com/apple/swift-nio-quic [repo-nio-http3]: https://github.com/apple/swift-nio-http3 --- ## File: docs/advanced-performance-analysis.md # Advanced performance analysis for CPU-bound programs Most performance problems can be handily debugged with [Instruments, `perf`, or FlameGraphs](https://github.com/swift-server/guides/blob/main/docs/performance.md). At times however, there are changes in performance that are very hard to understand and sometimes very counter intuitive. ## Motivating example (feel free to skip) A motivating example is [this pull request](https://github.com/apple/swift-nio/pull/1733) which is supposed to improve the performance of `ByteToMessageDecoder`s in very specific cases. The expectation would be that most performance tests remain the same and everything that uses a `ByteToMessageDecoder` either remains the same or becomes a little faster. Of course, all the low-level microbenchmarks that don't even use `ByteToMessageDecoder`s should be totally unaffected by this change. For the most part that was the case, except for two benchmarks in SwiftNIO's performance test suite for which the [results were](https://github.com/apple/swift-nio/pull/1733#issuecomment-766917189) very unexpected: | benchmark name | run time with PR | run time before PR | which is faster | difference | | --- | --- | --- | --- | --- | | circular_buffer_into_byte_buffer_1kb | 0.05089386 | 0.041163118 | previous | 23% | | circular_buffer_into_byte_buffer_1mb | 0.094321349 | 0.082286637 | previous | 14% | This means that for whatever reason, the totally unrelated benchmarks `circular_buffer_into_byte_buffer_*` got _slower_ by 23% and 14% respectively (measured in wall clock seconds) despite using none of the changed code. ## Wall clock time To start out, we need to verify if the benchmark results are stable regarding wall clock time. If there is too much variation, then it is likely that too much I/O or allocations etc are involved. If a benchmark is unstable because of I/O or allocations, there are better tools available than what is discussed here, for example `strace`, Instrument's "Allocations" instrument, or Xcode's Memory Graph Debugger. This document solely discusses _CPU-bound_ and mostly single-threaded performance analysis, ie. you should see one CPU core pegged at 100% for the whole duration of the benchmark. ## Executed CPU instructions If you have two versions of a program, one faster and one slower and you want to find out what is going on, then often it makes sense to look at the number of CPU instructions executed during the benchmark. On Linux -- assuming you have access to the PMU (Performance Monitoring Unit) -- this is rather straightforward perf stat -- ./benchmark arguments The output will look similar to ``` Performance counter stats for './my-benchmark args': 449.96 msec task-clock # 1.001 CPUs utilized 36 context-switches # 0.080 K/sec 9 cpu-migrations # 0.020 K/sec 2989 page-faults # 0.007 M/sec 1736698259 cycles # 3.860 GHz 53504877 stalled-cycles-frontend # 3.08% frontend cycles idle 5927885177 instructions # 3.41 insn per cycle # 0.01 stalled cycles per insn 1422566509 branches # 3161.555 M/sec 375326 branch-misses # 0.03% of all branches 0.449450377 seconds time elapsed 0.442788000 seconds user 0.008050000 seconds sys ``` First, we can confirm that it is nicely CPU bound (`1.001 CPUs utilized`). We can also see that we executed about 5.9B instructions during this benchmark run. The number of instructions executed itself is not very meaningful but it can be very useful when comparing between versions of the same code. For example, let's assume we changed some code and want to confirm in a benchmark that indeed we have made it faster, then it can be very useful to see that we have also reduced the number of instructions. With `perf` you could run `perf stat -e instructions -- ./benchmark-before-change` and `perf stat -e instructions -- ./benchmark-after-change` and confirm that: 1. The runtime (_time elapsed_) has gone down and 2. the number of instructions executed has gone down too. Note that the time it takes to run an instruction varies wildly so it is in theory possible to reduce the number of instructions but make performance actually worse. An example would be if you traded a lot of simple arithmetic for memory accesses. Said that, in most cases where you reduced the number of instructions significantly, you will also see an improvement in the actual runtime. As a control, the number of instructions per cycle can also be useful as it would go down if you traded faster instructions for slower instructions. Another place where instruction counts come in very handy is when you see this maybe impossible-sounding situation: the run time varies significantly but the number of executed instructions is _the same_. The motivating example is exactly this situation: The code executed is the _same_, confirmed by manually checking the generated assembly for the code run as well as checking that the number of instructions is almost equal. Because manual assembly analysis can easily go wrong, it's advisable to compare the number of executed instructions even if you convinced yourself that the executed code is the same. Of course, there are other options like single-stepping in a debugger but that gets tedious quickly. If the executed code and the inputs are the same, then the number of executed instructions should also be the same (ie. < 0.01% variance). That begs the question: What can we do if we have the runtime differs despite running the exact same instructions? Assuming the code is CPU-bound, the answer is probably that the CPU cannot execute the instructions at the same velocity because it internally has to wait for something (such as memory, more instructions being decoded, another resource, ...). Many of these situations are referred to as CPU stalls. ## CPU stalls When ending up in situations where a difference in run time is not clearly reflected in the instructions executed, it may be worth looking into CPU stalls. Modern CPUs may be unable to reach their full speed for a variety of reasons. For example it could be that the instruction decoder can't decode the instructions as fast as they could execute (be retired). Or instructions may have to wait to be executed because the data required is slow to load (cache misses etc). The easiest way to look into such micro-architectural issues are through the CPU's performance counters and with `perf` they're quite easy to access. A good starting point for tracing stalls may be perf stat -e instructions,stalled-cycles-frontend,uops_executed.stall_cycles,resource_stalls.any,cycle_activity.stalls_mem_any -- ./benchmark Which instructs `perf` to collect the following performance counters: - `instructions`: The number of CPU instructions executed, - `stalled-cycles-frontend`: cycles where nothing happened because instructions cannot be decoded fast enough, - `uops_executed.stall_cycles`: the number of cycles where no (micro-)operations were dispatched to be executed on a given core, - `resource_stalls.any`: Stalls that occurred because the backend had to wait for resources (eg. memory) - `cycle_activity.stalls_mem_any`: execution stalls while memory subsystem has an outstanding load. which will yield output similar to ``` Performance counter stats for './fast circular_buffer_into_byte_buffer_1kb': 5927396603 instructions 64091849 stalled-cycles-frontend 38429768 uops_executed.stall_cycles 13278071 resource_stalls.any 22148955 cycle_activity.stalls_mem_any 0.452895518 seconds time elapsed ``` And just like for the instruction counts, the absolute counts of the stalled cycles are not that important, but they can be very important as a comparison. Going back to the motivating example, where we could see: fast version (`perf stat -e counters... -- ./fast`): ``` Performance counter stats for './fast circular_buffer_into_byte_buffer_1kb': 5927396603 instructions 34307940 uops_executed.stall_cycles 13109779 resource_stalls.any 18148622 cycle_activity.stalls_mem_any 0.409785892 seconds time elapsed ``` slow version (`perf stat -e counters... -- ./slow`): ``` Performance counter stats for './slow circular_buffer_into_byte_buffer_1kb': 5927358294 instructions 308782310 uops_executed.stall_cycles 14159547 resource_stalls.any 288072686 cycle_activity.stalls_mem_any 0.510635133 seconds time elapsed ``` What we can immediately see is that `resource_stalls.any` is pretty much the same in both cases, but `uops_executed.stall_cycles` and `cycle_activity.stalls_mem_any` are about an order of magnitude higher for the slow case. Remember, this is with the number of instructions being almost the same. So in this case, we would have identified that the slowdown comes from us getting a lot of "stall cycles" in the CPU. Those are cycles where the CPU was free to run some (micro-)instructions but could not. Unfortunately, from this picture alone, it is unclear _why_ exactly the CPU couldn't actually run some more (micro-)operations. We’ve only learned a little so far, we need to dig deeper. One strategy is to run perf list | less and in there, search for "stall" (using `/stall`) and "penalty". This will yield some results of available performance counters that allow you to identify which particular stall it may be. It's worth to know that you can specify a bunch of them in one command line (comma separated). Figuring out which ones to choose is really an exercise in brute force with a little educated guessing, there aren't _that many_ such counters so it won't take you too long. In the motivating example, the culprit turned out to be `dsb2mite_switches.penalty_cycles` ``` Performance counter stats for './fast circular_buffer_into_byte_buffer_1kb': 9943811 dsb2mite_switches.penalty_cycles 0.408032273 seconds time elapsed ``` vs ``` Performance counter stats for './slow circular_buffer_into_byte_buffer_1kb': 229285609 dsb2mite_switches.penalty_cycles 0.511033898 seconds time elapsed ``` The names of these stalls/penalties can be quite opaque and understanding what exactly is going on can be rather hard without a pretty good understanding of how a particular CPU micro-architecture works. Nevertheless, being able to pinpoint why a slowdown happens can be very important as it may give information on what to change in the code. In case you're wondering what `dsb2mite_switches.penalty_cycles` is, a web search revealed a nicely written [blogpost](https://easyperf.net/blog/2018/01/18/Code_alignment_issues) which discusses something very similar. Bottom line is that modern Intel CPUs have a MITE (Micro-operation Translation Engine) which seems to read the instruction stream in 16 byte chunks. So if the alignment of your code is changed (for example by adding/removing an unrelated function to/from your binary) you may incur a penalty if your instructions aren't aligned by 16 bytes (remember, Intel instructions are variable length). This does sound scary (and for some code it is) but the good news is that this should _very rarely_ matter outside of micro-benchmarks. If your whole run time is doing exactly what that micro-benchmark is doing, you should probably care. But most real-world code does much more and more diverse work than some arbitrary micro-benchmarks. So in this particular example we can safely ignore the "regression" which was just unlucky placement of the hot function in the binary. Nevertheless it was important to understand why the regression happened so we could safely decide it does not matter in this instance. ## The bad news: PMU access Unfortunately, in some environments like Docker for Mac and other solutions relying on virtualisation it can be hard to get access to the PMU (the CPU's performance measuring unit). Check out this [guide](https://github.com/swift-server/guides/blob/main/docs/linux-perf.md#getting-perf-to-work) for tips on getting `perf` with PMU support to work in your environment. --- ## File: docs/debugging-allocations.md # How to debug allocation regressions Usually, your journey will start by detecting that an allocation test regressed. So let's first start by looking at how to run the allocation counter tests. ## Running allocation counter tests The allocation counter tests are usually run by NIO's integration test framework which lives in `IntegrationTests/`. In there, you first want to identify the test suite and test case that runs all the allocation counter tests. For the `swift-nio` repository that is IntegrationTests/tests_04_performance/test_01_allocation_counts.sh Next to the test case (`test_01_allocation_counts.sh`) that runs the allocation counter tests should be a resource directory which contains the individual allocation counter tests. For `swift-nio` that is IntegrationTests/tests_04_performance/test_01_resources From here on, this document expects that you are within this directory, so you might want to start with cd IntegrationTests/tests_04_performance/test_01_resources/ To run all the allocation tests, simply invoke the runner script in there ./run-nio-alloc-counter-tests.sh The invocation of this script will take a while because it has to build all of SwiftNIO in release mode first, then it compiles the integration tests and runs them all, multiple times... ### Understanding the output of the allocation counter tests The output of this script will look something like ``` - /Users/johannes/devel/swift-nio/IntegrationTests/tests_04_performance/test_01_resources/test_future_lots_of_callbacks.swift test_future_lots_of_callbacks.remaining_allocations: 0 test_future_lots_of_callbacks.total_allocations: 75001 test_future_lots_of_callbacks.total_allocated_bytes: 4138056 DEBUG: [["remaining_allocations": 0, "total_allocations": 75001, "total_allocated_bytes": 4138056], ["total_allocations": 75001, "remaining_allocations": 0, "total_allocated_bytes": 4138056], ["total_allocated_bytes": 4138056, "total_allocations": 75001, "remaining_allocations": 0], ["total_allocated_bytes": 4138056, "total_allocations": 75001, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocations": 75001, "total_allocated_bytes": 4138056], ["total_allocations": 75001, "remaining_allocations": 0, "total_allocated_bytes": 4138056], ["total_allocated_bytes": 4138056, "total_allocations": 75001, "remaining_allocations": 0], ["total_allocations": 75001, "total_allocated_bytes": 4138056, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocations": 75001, "total_allocated_bytes": 4138056], ["total_allocations": 75001, "total_allocated_bytes": 4138056, "remaining_allocations": 0]] ``` with this kind of block repeated for each allocation counter test. Let's go through and understand each line. The first line is the name of the specific allocation test. The most relevant part is the file name (`test_future_lots_of_callbacks.swift`). For this test, we seem to be testing how many allocations futures with many callbacks are doing. ``` - /Users/johannes/devel/swift-nio/IntegrationTests/tests_04_performance/test_01_resources/test_future_lots_of_callbacks.swift ``` Next, we see ``` test_future_lots_of_callbacks.remaining_allocations: 0 test_future_lots_of_callbacks.total_allocations: 75001 test_future_lots_of_callbacks.total_allocated_bytes: 4138056 ``` which are the aggregate values. The first line says `remaining_allocations: 0`, which means that this allocation test didn't leak, which is good! Then, we see test_future_lots_of_callbacks.total_allocations: 75001 which means that by the end of the test, we saw 75001 allocations in total. As a rule, we usually run the workload of every allocation test 1000 times. That means you want to divide 75001 by the 1000 runs to get the number of allocations per run. In other words: each iteration of this allocation test allocated 75 times. The extra one allocation is just some noise that we have to ignore: these are usually inserted by operations in the Swift runtime that need to initialize some state. In many (especially multi-threaded test cases) there is some noise, which is why we run the workload repeatedly, which makes it easy to tell the signal from the noise. Finally, we see test_future_lots_of_callbacks.total_allocated_bytes: 4138056 Which is the total number of bytes allocated by this test. Last of all, we see ``` DEBUG: [["remaining_allocations": 0, "total_allocations": 75001, "total_allocated_bytes": 4138056], ["total_allocations": 75001, "remaining_allocations": 0, "total_allocated_bytes": 4138056], ["total_allocated_bytes": 4138056, "total_allocations": 75001, "remaining_allocations": 0], ["total_allocated_bytes": 4138056, "total_allocations": 75001, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocations": 75001, "total_allocated_bytes": 4138056], ["total_allocations": 75001, "remaining_allocations": 0, "total_allocated_bytes": 4138056], ["total_allocated_bytes": 4138056, "total_allocations": 75001, "remaining_allocations": 0], ["total_allocations": 75001, "total_allocated_bytes": 4138056, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocations": 75001, "total_allocated_bytes": 4138056], ["total_allocations": 75001, "total_allocated_bytes": 4138056, "remaining_allocations": 0]] ``` which is literally just the exact numbers of each of the 10 runs we considered. Note, the lines are prefixed with `DEBUG:` to make analysis of the output easier, it does _not_ mean we're running in debug mode. If you see a lot of fluctuation between the runs, then the allocation test doesn't allocate deterministically which would be something you want to fix. In the example above, we're totally stable to the last byte of allocated memory, so we have nothing to worry about with this test. ## Important notes The exact allocation counts are _not_ directly comparable between different operating systems and even different versions of the same operating system. Every new Swift version will also introduce differences (hopefully decreasing over time). This means that if you get hit by a CI failure, you will have to first reproduce the regression locally by running the allocation counter tests once on `main` and then again on your failing branch. ## Debugging a regression in a particular test Because it takes quite a while to run the whole allocation counter test suite, there are some shortcuts you definitely want to know about. First, let's have a look at how we can run just single test. ### Running a single test Running a single test is very straightforward, pass the test's `.swift` file to the runner script. For example: ./run-nio-alloc-counter-tests.sh test_future_lots_of_callbacks.swift You will notice that it now runs faster but also that it still compiles SwiftNIO for every single time you run the program. So that's still rather slow. If you look closely at the output you will notice that it prints messages such as ``` Fetching /private/tmp/.nio_alloc_counter_tests_5jMMhk/swift-nio Fetching /private/tmp/.nio_alloc_counter_tests_5jMMhk/HookedFunctions Fetching /private/tmp/.nio_alloc_counter_tests_5jMMhk/AtomicCounter ``` If you change your working directory to common directory above (`/private/tmp/.nio_alloc_counter_tests_5jMMhk`), then you will find a regular Swift package there and you can just re-run the allocation counter tests using ``` cd /tmp/.nio_alloc_counter_tests_5jMMhk swift run -c release ``` which should run very quickly. But _do not_ forget to pass `-c release`, otherwise the results will be meaningless. ### Debugging with other tools Often, you will want to use external tools such as Instruments or `dtrace` to analyse why a performance test regressed. Unfortunately, that is a little harder than you might expect because to implement the allocation tests, the allocation counter test framework actually [hijacks](https://github.com/apple/swift-nio/blob/main/IntegrationTests/tests_04_performance/test_01_resources/README.md) functions like `malloc` and `free`. The counting of allocations happens in the hijacked or 'hooked' functions. To use an external memory analysis tool, it's best to not do the hooking and fortunately it's very straightforward to switch the hooking off. To disable the hooking, run the runner script with `-- -n` followed by the file name of the test you would like to debug. For example ./run-nio-alloc-counter-tests.sh -- -n test_future_lots_of_callbacks.swift When you run this, you will notice that every allocation counter test now outputs ``` Fetching /private/tmp/.nio_alloc_counter_tests_0UDNXS/HookedFunctions Fetching /private/tmp/.nio_alloc_counter_tests_0UDNXS/swift-nio Fetching /private/tmp/.nio_alloc_counter_tests_0UDNXS/AtomicCounter [...] test_future_lots_of_callbacks.total_allocations: 0 test_future_lots_of_callbacks.remaining_allocations: 0 test_future_lots_of_callbacks.total_allocated_bytes: 0 DEBUG: [["total_allocations": 0, "remaining_allocations": 0, "total_allocated_bytes": 0], ["remaining_allocations": 0, "total_allocations": 0, "total_allocated_bytes": 0], ["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["total_allocated_bytes": 0, "total_allocations": 0, "remaining_allocations": 0], ["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["total_allocated_bytes": 0, "remaining_allocations": 0, "total_allocations": 0], ["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["total_allocated_bytes": 0, "remaining_allocations": 0, "total_allocations": 0], ["remaining_allocations": 0, "total_allocated_bytes": 0, "total_allocations": 0], ["total_allocated_bytes": 0, "total_allocations": 0, "remaining_allocations": 0]] ``` See all the zeroes? Those are all zero because the hooking is disabled. In this mode the output itself is not very useful but you can now switch to using other tools like Intruments or `dtrace` to debug your allocations. As explained before, you can change your working directory in the temporary directory (for example `cd /private/tmp/.nio_alloc_counter_tests_0UDNXS`) and invoke `swift run -c release` from there to run the allocation counter benchmark. Note that if you disabled hooking (by passing `-- -n`) it will still print only zeroes. ### Debugging with Instruments 1. Start Instruments 2. Choose the "Allocations" instrument 3. Tap the executable selector in the toolbar 4. Click "Choose Target..." 5. Tick "Show Hidden Files" 6. Press Cmd+Shift+G 7. Enter the temporary directory path of a non-hooked version of the allocation counter tests (for example `/private/tmp/.nio_alloc_counter_tests_0UDNXS`) 8. Select `.build/release/` 9. Find the binary that is named like your allocation counter test (eg. `test_lots_of_future_callbacks`) 10. Click "Choose" 11. Press the record button Now you should have a full Instruments allocations trace of the test run. To make sense of what's going on, switch the allocation lifespan from "Created & Persisted" to "Created & Destroyed". After that you should be able to see the type of the allocation and how many times it got allocated. Clicking on the little arrow next to the type name will reveal each individual allocation of this type and the responsible stack trace. ### Debugging with `malloc-aggregation.d` or `malloc-aggreation.bt` SwiftNIO ships a `dtrace` script called `dev/malloc-aggregation.d` which can give you an aggregation of all allocations by stack trace, and an equivalent `bpftrace` script called `dev/malloc-aggregation.bt`. This means that you will see the stack trace of each allocation that happened and how many times this particular stack trace allocated. Here's an example of how to use the `dtrace` script: ``` sudo ~/path/to/swift-nio/dev/malloc-aggregation.d -c .build/release/test_future_lots_of_callbacks ``` The `bpftrace` script can be invoked very similarly: ``` sudo ~/path/to/swift-nio/dev/malloc-aggregation.bt -c .build/release/test_future_lots_of_callbacks ``` However, for `bpftrace` it tends to work better if pid-based tracing is used instead. This is because `bpftrace` has a [known limitation where symbolication fails if the process being traced has existed before `bpftrace` does](https://github.com/bpftrace/bpftrace/issues/2118#issuecomment-1008694821). This can still be resolved using tools like `llvm-symbolizer`, but it's trickier. The output will look something like ``` [...] libsystem_malloc.dylib`malloc libswiftCore.dylib`swift_slowAlloc+0x19 libswiftCore.dylib`swift_allocObject+0x27 test_1_reqs_1000_conn`SelectableEventLoop.run()+0x231 test_1_reqs_1000_conn`closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:selectorFactory:initializer:)+0x106 test_1_reqs_1000_conn`partial apply for closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:selectorFactory:initializer:)+0x2d test_1_reqs_1000_conn`thunk for @escaping @callee_guaranteed (@guaranteed NIOThread) -> ()+0xf test_1_reqs_1000_conn`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed NIOThread) -> ()+0x11 test_1_reqs_1000_conn`closure #1 in static ThreadOpsPosix.run(handle:args:detachThread:)+0x1e4 libsystem_pthread.dylib`_pthread_start+0x94 libsystem_pthread.dylib`thread_start+0xf 231000 ``` repeated many times. Each block means that the specific stack trace is responsible for some number of allocations: in the example above, `231000` allocations. Remember that the allocation counter tests run each test 11 times (1 warm up run and 10 measured runs), so this means that in each test run, the above stack trace allocated 21000 times. Also remember that we usually run the workload we want to test 1000 times. That means that in the real world, the above stack trace accounts for 21 out of the total number of allocations (231000 allocations / 11 runs / 1000 executions per run = 21). When you are looking at allocations in the setup of the test the numbers may be split into one allocation set of 10000 and another of 1000 - for measured code vs warm up run. The output from `malloc-aggreation.d` can also be diffed using the `stackdiff-dtrace.py` script. This can be helpful to track down where additional allocations were made. Right now this strategy doesn't work for `bpftrace`, but a simple modification to the `stackdiff-dtrace` script should enable it. The `stdout` from `malloc-aggregation.d` for the two runs to compare should be written to files, and then passsed to `stackdiff-dtrace.py`: ```bash ~/path/to/swift-nio/dev/stackdiff-dtrace.py stack_aggregation.old stack_aggregation.new ``` Where `stack_aggregation.old` and `stack_aggregation.new` are the two outputs from `malloc-aggregation.d` to compare. The script will output the aggregated stacks which appear: - only in `stack_aggregation.old`, - only in `stack_aggregation.new`, and - any differences in numbers for stacks that appear in both. Similar stack traces will be aggregated into a headline number for matching but where different the full stack traces which made up the collection will be printed indented. Allocations of less than 1000 in number are ignored - as the tests all run multiples of 1000 runs any values below this are almost certainly from setup in libraries we are using rather than as a consequence of our measured run. Let's look at some output. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` These stacks are only *slighly* different; differing only in the name similar to `$s3NIO7Channel_pxq_q0_r1_lyypypAA15EventLoopFutureCyytGIsegnnr_Iegnr_AaB_pypypAEIegnno_Ieggo_TR`. They're otherwise the same so we can ignore them and look at more output. ``` ### only in AFTER [... (previously dismissed stacktrace)] 2000 libsystem_malloc.dylib`malloc libswiftCore.dylib`swift_slowAlloc libswiftCore.dylib`swift_allocObject test_1_reqs_1000_conn`doRequests(group:number:) test_1_reqs_1000_conn`closure #1 in run(identifier:) test_1_reqs_1000_conn`partial apply for closure #1 in measureOne #1 (throwAway:_:) in measureAll(_:) test_1_reqs_1000_conn`partial apply for thunk for @callee_guaranteed () -> (@error @owned Error) test_1_reqs_1000_conn`thunk for @callee_guaranteed () -> (@error @owned Error)partial apply 1000 libsystem_malloc.dylib`malloc libswiftCore.dylib`swift_slowAlloc+0x19 libswiftCore.dylib`swift_allocObject+0x27 test_1_reqs_1000_conn`doRequests(group:number:)+0x66 [...] 1000 libsystem_malloc.dylib`malloc libswiftCore.dylib`swift_slowAlloc+0x19 libswiftCore.dylib`swift_allocObject+0x27 test_1_reqs_1000_conn`doRequests(group:number:)+0x1c4 [...] ### only in BEFORE [... (previously dismissed stacktrace)] ### different numbers before: 10000, after: 20000 AFTER 10000 libsystem_malloc.dylib`malloc libswiftCore.dylib`swift_slowAlloc+0x19 libswiftCore.dylib`swift_allocObject+0x27 test_1_reqs_1000_conn`doRequests(group:number:)+0x66 [...] ``` Now we see there's another stacktrace in the `AFTER` section which has no corresponding stacktrace in `BEFORE`. From the stack we can see it's originating from `doRequests(group:number:)`. In this instance we were working on options applied in this function so it appears we have added allocations. We have also increased the number of allocations which are reported in the different numbers section where stack traces have been paired but with different numbers of allocations. ### Debugging with 'heaptrack' In some cases we don't have access to Linux kernel headers, which prevents us from using `bpftrace`, but there is the [heaptrack](https://github.com/KDE/heaptrack) tool which supports diff analysis of two versions. Tested on Ubuntu 20.04 (and likely working on other distributions) Install it with: ``` sudo apt-get install heaptrack ``` Then using the instructions previously, you need to build the test without hooks such that heaptrack can hook instead, e.g.: ``` cd IntegrationTests/tests_04_performance/test_01_resources/ ./run-nio-alloc-counter-tests.sh -- -n test_1000_autoReadGetAndSet.swift ``` and then cd to the temp directory from the output (find the tmp directory as above), e.g.: ``` cd /tmp/.nio_alloc_counter_tests_5jMMhk ``` Finally, run the binary with heaptrack two times — first we do it for `main` to get a baseline: ``` ubuntu@ip-172-31-25-161 /t/.nio_alloc_counter_tests_GRusAy> SWIFTNIO_URING_DISABLED=1 heaptrack .build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet heaptrack output will be written to "/tmp/.nio_alloc_counter_tests_GRusAy/heaptrack.test_1000_autoReadGetAndSet.84341.gz" starting application, this might take some time... SWIFTNIO_URING_DISABLED set, disabling liburing. test_1000_autoReadGetAndSet.total_allocations: 0 test_1000_autoReadGetAndSet.total_allocated_bytes: 0 test_1000_autoReadGetAndSet.remaining_allocations: 0 DEBUG: [["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocated_bytes": 0, "total_allocations": 0], ["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocated_bytes": 0, "total_allocations": 0], ["remaining_allocations": 0, "total_allocated_bytes": 0, "total_allocations": 0], ["remaining_allocations": 0, "total_allocations": 0, "total_allocated_bytes": 0], ["total_allocated_bytes": 0, "total_allocations": 0, "remaining_allocations": 0], ["total_allocated_bytes": 0, "total_allocations": 0, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocations": 0, "total_allocated_bytes": 0], ["total_allocations": 0, "remaining_allocations": 0, "total_allocated_bytes": 0]] free(): invalid pointer Aborted (core dumped) heaptrack stats: allocations: 319347 leaked allocations: 107 temporary allocations: 68 Heaptrack finished! Now run the following to investigate the data: heaptrack --analyze "/tmp/.nio_alloc_counter_tests_GRusAy/heaptrack.test_1000_autoReadGetAndSet.84341.gz" ``` Then run it a second time for the feature branch: ``` ubuntu@ip-172-31-25-161 /t/.nio_alloc_counter_tests_ARusXy> heaptrack .build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet heaptrack output will be written to "/tmp/.nio_alloc_counter_tests_GRusAy/heaptrack.test_1000_autoReadGetAndSet.84372.gz" starting application, this might take some time... test_1000_autoReadGetAndSet.remaining_allocations: 0 test_1000_autoReadGetAndSet.total_allocations: 0 test_1000_autoReadGetAndSet.total_allocated_bytes: 0 DEBUG: [["remaining_allocations": 0, "total_allocations": 0, "total_allocated_bytes": 0], ["total_allocated_bytes": 0, "total_allocations": 0, "remaining_allocations": 0], ["total_allocated_bytes": 0, "remaining_allocations": 0, "total_allocations": 0], ["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["total_allocations": 0, "remaining_allocations": 0, "total_allocated_bytes": 0], ["remaining_allocations": 0, "total_allocations": 0, "total_allocated_bytes": 0], ["total_allocations": 0, "total_allocated_bytes": 0, "remaining_allocations": 0], ["remaining_allocations": 0, "total_allocated_bytes": 0, "total_allocations": 0], ["remaining_allocations": 0, "total_allocations": 0, "total_allocated_bytes": 0], ["remaining_allocations": 0, "total_allocated_bytes": 0, "total_allocations": 0]] free(): invalid pointer Aborted (core dumped) heaptrack stats: allocations: 673989 leaked allocations: 117 temporary allocations: 341011 Heaptrack finished! Now run the following to investigate the data: heaptrack --analyze "/tmp/.nio_alloc_counter_tests_GRusAy/heaptrack.test_1000_autoReadGetAndSet.84372.gz" ubuntu@ip-172-31-25-161 /t/.nio_alloc_counter_tests_GRusAy> ``` Here we could see that we had 673989 allocations in the feature branch version and 319347 in `main`, so clearly a regression. Finally, we can analyze the output as a diff from these runs using `heaptrack_print`: ``` heaptrack_print -T -d heaptrack.test_1000_autoReadGetAndSet.84341.gz heaptrack.test_1000_autoReadGetAndSet.84372.gz | swift demangle ``` `-T` gives us the temporary allocations (as it in this case was not a leak, but a transient alloaction - if you have leaks remove `-T`). The output can be quite long, but in this case as we look for transient allocations, scroll down to: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` And here we could fairly quickly see that the transient extra allocations was due to extra debug printing and querying of environment variables: ``` NIO.URing.getEnvironmentVar(Swift.String) -> Swift.String? at /home/ubuntu/swiftnio/swift-nio/Sources/NIO/LinuxURing.swift:291 in /tmp/.nio_alloc_counter_tests_GRusAy/.build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet NIO.URing._debugPrint(@autoclosure () -> Swift.String) -> () ``` And this code will be removed before final integration of the feature branch, so the diff will go away. --- ## File: docs/io_uring.md # SwiftNIO io_uring support status [Linux] ## Background Linux has fairly recently received a new asynchrounus I/O interface — `io_uring` — that allows high-performance AIO with a minimum amount of system calls. Some background information at [LWN](https://lwn.net/Articles/810414/) and [kernel.dk](https://kernel.dk/io_uring.pdf) ## Current status and supported platforms SwiftNIO has basic support for using `io_uring` as a notification mechanism. `io_uring` is available on Linux kernels starting from 5.1 and requires support for the POLL operations. SwiftNIO optionally supports multishot polls and poll updates through a configuration option - these are new interfaces that are expected to available from the future release 5.13 (as of this writing). The `io_uring` implementation has primarily been tested on version 5.12rc. ## Performance expectations Currently the `io_uring` implementation does not yet use the underlying interface to its full potential - it is only used for polling and not yet for asynchronous operation of send/recv/read/write. This means performance characteristics currently will be similar to the default `epoll()` (or slightly worse), but you should test for your specific use case as YMMV. This is expected to change in the future as the `io_uring` interface is used more extensively. ## How to use and requirements To enable `io_uring`, SwiftNIO needs to be built and run on a machine that has the [liburing](https://github.com/axboe/liburing) library installed (typically in `/usr/local`). If built with liburing, it must be available and work on the production machine. To build SwiftNIO so it uses io_uring instead of epoll, you must specify additional flags when building: `swift build -Xcc -DSWIFTNIO_USE_IO_URING=1 -Xlinker -luring -Xswiftc -DSWIFTNIO_USE_IO_URING` To build it so also use the new poll update / multishot polls (will need kernel 5.13+) you need to do: `swift build -Xcc -DSWIFTNIO_USE_IO_URING=1 -Xlinker -luring -Xswiftc -DSWIFTNIO_USE_IO_URING -Xswiftc -DSWIFTNIO_IO_URING_MULTISHOT` ## Debug output There are currently three debug outputs that can be compiled in (e.g. to verify that `liburing` is used as expected and for development troubleshooting) `-Xswiftc -DSWIFTNIO_IO_URING_DEBUG_SELECTOR` `-Xswiftc -DSWIFTNIO_IO_URING_DEBUG_URING` `-Xswiftc -DSWIFTNIO_IO_URING_DEBUG_DUMP_CQE` ## Roadmap The initial support for `io_uring` is primarily intended to give people interested in it that ability to try it out and provide feedback - in the future the expectation is to add full support for `io_uring` which is expected to give significant performance improvements on Linux systems supporting it. The idea is also to add SQPOLL together with fine tune settings such that it can be pinned on a specific CPU. --- ## File: docs/migration-guide-NIO1-to-NIO2.md # NIO 1 to NIO 2 migration guide Please take a look at the [migration guide included with SwiftNIO 2.29.0](https://github.com/apple/swift-nio/blob/2.29.0/docs/migration-guide-NIO1-to-NIO2.md), the last version with Swift 5.0 support. [SwiftNIO 2.29.0](https://github.com/apple/swift-nio/releases/tag/2.29.0) includes the `_NIO1APIShims` module which can help to ease the migration. --- ## File: docs/optimization-tips.md Swift NIO Optimization Tips =========================== The following document is a list of tips and advice for writing high performance code using Swift NIO. General Optimization Tips ------------------------- Although not specific to NIO, the [Swift GitHub repository][swift-repo] has a list of tips and tricks for [writing high performance Swift code][swift-optimization-tips] and is a great place to start. Wrapping types in `NIOAny` -------------------------- Anything passing through a NIO pipeline that is not special-cased by `NIOAny` (i.e. is not one of `ByteBuffer`, `FileRegion`, `IOData` or `AddressedEnvelope`) needs to have careful attention paid to its size. If the type isn't one of the aforementioned special cases then it must be no greater than 24 bytes in size to avoid a heap-allocation each time it is added to a `NIOAny`. The size of a type can be checked using [`MemoryLayout.size`][docs-memory-layout]. If the type being wrapped is a value type then it can be narrowed by placing it in a copy-on-write box. Alternatively, if the type is an `enum` with associated data, then it is possible for the associated data to be boxed by the compiler by making it an `indirect case`. [swift-repo]: https://github.com/apple/swift [swift-optimization-tips]: https://github.com/apple/swift/blob/main/docs/OptimizationTips.rst [docs-memory-layout]: https://developer.apple.com/documentation/swift/memorylayout --- ## File: docs/public-api.md # SwiftNIO 2 Public API The SwiftNIO project (which includes all of `github.com/apple/swift-nio*`) aims to follow [SemVer 2.0.0](https://semver.org/spec/v2.0.0.html) which requires projects to declare a public API. This document along with the [API documentation](https://swiftpackageindex.com/apple/swift-nio/main/documentation) declare SwiftNIO's API. ## What are acceptable uses of NIO's Public API ### 1. Using NIO types, methods, and modules It is acceptable and expected to use any exported type and call any exported method from the `NIO*` modules unless the type or function name start with an underscore (`_`). For `init`s this includes the first parameter's name. ##### What's the point of this restriction? If we prefix something with an underscore or put it into one of NIO's internal modules, we can't commit to an API for it and don't expect users to ever use this. Also, there might be different underscored functions/types depending on the platform. ##### Examples - ✅ `channel.close(promise: nil)` - ❌ `channel._channelCore.flush0()`, underscored property - ❌ `import CNIOAtomics`, module name doesn't start with NIO - ❌ `ByteBuffer(_enableSuperSpecialAllocationMode: true)`, as the initialiser's first argument is underscored --- ### 2. Conforming NIO types to protocols It is acceptable to conform NIO types to protocols _you control_, i.e. a protocol in _your codebase_. It is not acceptable to conform any NIO types to protocols in the standard library or outside of your control. ##### What's the point of this restriction? NIO in a later version may conform its type to said protocol, or the package that provides the protocol may conform the NIO type to _their_ protocol. ##### Examples - ✅ `extension EventLoopFuture: MyOwnProtocol { ... }`, assuming `MyOwnProtocol` lives in your codebase - ❌ `extension EventLoopFuture: DebugStringConvertible { ... }`, `DebugStringConvertible` is a standard library protocol --- ### 3. Conforming your types to NIO protocols It is acceptable to conform _your own types_ to NIO protocols but it is not acceptable to conform types you do not control to NIO protocols. ##### What's the point of this restriction? NIO in a later version might add this conformance or the owner of the type may add the conformance. ##### Examples - ✅ `extension MyHandler: ChannelHandler { ... }` - ❌ `extension Array: EventLoopGroup where Element: EventLoop { ... }`, as `Array` lives in the standard library --- ### 4. Extending NIO types It is acceptable to extend NIO types with `public` methods/properties that either use one of your types as a non-default argument or are prefixed in a way that prevents clashes. Adding `private`/`internal` methods/properties is always acceptable. #### What's the point of these restrictions? NIO might later add a member function/property with the same signature as yours. ##### Examples - ✅ `extension ByteBuffer { public mutating func readMyType(at: Int) -> MyType {...} }`, acceptable because it returns a `MyType` - ✅ `extension ByteBuffer { public mutating func myProjectReadInteger(at: Int) -> Int {...} } `, acceptable because prefixed with `myProject` - ✅ `extension ByteBuffer { internal mutating func readFloat(at: Int) -> Float {...} }`, acceptable even though `Float` is not in your control because the function is `internal` - ❌ `extension ByteBuffer { public mutating func readBool(at: Int) -> Bool {...} }`, because `Bool` is a standard library type not in your control ## Promises the NIO team make Before releasing a new major version, i.e. SwiftNIO 3.0.0 we promise the following: ### 1. No global namespace additions that aren't prefixed In a minor or patch version, all global types/modules/functions that we will add will have `NIO*`/`nio*`/`CNIO*`/`cnio*`/`c_nio*` prefixes, this includes C symbols that are visible to others. Be aware that we don't control the exact versions of the system libraries installed on your system so we can't make any guarantees about the symbols exported by the system libraries we depend on. ##### What does this mean concretely? - ✅ We might add a new global type called `NIOJetPack`. - ✅ We might add a new module called `NIOHelpers`. - ✅ We might add a new global functions called `nioRun` (very unlikely) or `c_nio_SSL_write` (more likely). - ❌ We will not add a global new type called `Tomato`. - ❌ We will not add a new module called `Helpers`. - ❌ We will not add a new global function called `SSL_write`. - ❌ We will not add a new system library dependency without a new major version. #### Why is this useful to you? Let's assume your application is composed of three modules: 1. `NIO`, which in version 2.0.0 does not declare a `public struct Tomato` 2. `Caprese`, which declares a `public struct Tomato` 3. `MyApp` Let's also assume that `MyApp` has both `import Caprese` and `import NIO` and it uses `Tomato` (from `Caprese`). For example `let t = Tomato()`. If now in version `NIO` in version 2.1.0 introduces a `public struct Tomato { ... }` as well, then now compilation of `MyApp`'s compilation will fail will an ambiguity error. With the guarantee that in minor versions NIO will only introduce `NIO`-prefixed types, this can not happen because NIO would call its type `NIOTomato` rather than `Tomato`. C functions are all in a global namespace so even worse issues might hit you if NIO were to just introduce exported C functions with very general names in minor releases. ## What's the point of codifying all this? We believe that it is important for the ecosystem that all parties can adopt each others' new versions quickly and easily. Therefore we always strongly recommend depending on all the projects in the NIO family which have a major version greater than or equal to 1 up to the next _major_ version. By following the guidelines set out above we should all not run into any trouble even when run with newer versions. Example: .package(url: "https://github.com/apple/swift-nio.git", from: "2.0.0"), Needless to say if you require at least version `2.3.4` you would specify `from: "2.3.4"`. The important part is that you use `from: "2.3.4"` or `.upToNextMajor(from: "2.3.4")` and not `.exact` or `.upToNextMinor`. ## What happens if you ignore these guidelines? We are trying to be nice people and we ❤️ our users so we will never break anybody just because they didn't perfectly stick to these guidelines. But just ignoring those guidelines might mean rewriting some of your code, debugging random build, or runtime failures that we didn't hit in the pre-release testing. We do have a source compatibility suite to which you can [ask to be added](https://forums.swift.org/t/register-as-swiftnio-user-to-get-ahead-of-time-security-notifications-be-added-to-the-source-compatibility-suite/17792) and we try not to break you (within reason). But it is impossible for us to test all of our users' projects and we don't want to lose the ability to move fast without breaking things. Certain failures like clashing protocol conformances can have delicate failure modes. --- ## File: docs/public-async-nio-apis.md # Async NIO bridges This is a summary of all the new APIs we introduced to make NIO and the various network protocols work with new async interfaces. The intention of this document is to quickly outline the incompatibilities in the original API and then show a holistic view over all the new APIs. ## What APIs are needed? The first problem that we had to tackle was bridging NIO's `Channel` to Swift Concurrency. To do so we introduced two new foundational types - the `NIOAsyncSequenceProducer` and the `NIOAsyncWriter`. Those allow us to bridge the read and the write side of the `Channel` while propagating the back-pressure across the bridge. On top of those two types, we built out the `NIOAsyncChannel` which allows users to bridge a `Channel` into Swift Concurrency. To do this it inserts two channel handlers which bridge the read and write side using the `NIOAsyncSequenceProducer` and the `NIOAsyncWriter`. Next up we had to look at the bootstraps. Here the import part is that the `Channel`s **must** be wrapped at the correct timing otherwise there is the potential that reads might be dropped. This is not problematic for most of the bootstraps since they call their various `channelInitializer`s and `childChannelInitializer`s at the right time. However, there was one tricky bootstrap - `ServerBootstrap`. The `ServerBootstrap` multiplexes the incoming connections and we have to make sure that the wrapping of the child channels happens at the correct time. Additionally, the new bootstrap APIs **must** be able to relay the type information of the configured channels to the `bind`/`connect` methods. The next thing we had to tackle was networking protocols that dynamically re-configure the `ChannelPipeline`. The two examples that we provide implementations for are HTTP/1 protocol upgrades and Application Protocol Negotiation (ALPN) via TLS. Similar to the bootstraps we have to ensure that the type information is retained so that users can correctly identify which reconfiguration path has been taken. Lastly, we had to look at how to handle protocols that multiplex, like HTTP/2. Multiplexing protocols need to expose a typed async interface to consume new inbound connections/streams and to open new outbound connections/streams where applicable. ## Proposed APIs This section contains all the new APIs that we are adding and gives us a holistic overview to review them. ### `NIOAsyncChannel` ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Bootstraps ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### HTTPUpgrade ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Websocket ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### ALPN ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### HTTP/2.0 ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ## File: docs/workarounds.md # Workarounds The following list of PRs and commits were applied in order to work around bugs or cases where the Swift compiler was unable to sufficiently optimize the code: - https://github.com/apple/swift-nio/pull/1374 - https://github.com/apple/swift-nio-ssl/pull/176 - https://github.com/apple/swift-nio/pull/1325 - https://github.com/apple/swift-nio/pull/1299 - https://github.com/apple/swift-nio/pull/1252 - https://github.com/apple/swift-nio/pull/494 - https://github.com/apple/swift-nio/pull/420 - https://github.com/apple/swift-nio/commit/abc963cfe1e1d4856c41421c9d53ea778102e9e8 - https://github.com/apple/swift-nio/pull/1814 - https://github.com/apple/swift-nio/pull/1956 - https://github.com/apple/swift-nio/pull/1961 - https://github.com/apple/swift-nio/pull/2046 - https://github.com/apple/swift-nio/pull/3303 - https://github.com/apple/swift-nio/pull/3452 - https://github.com/apple/swift-nio/pull/3454 - https://github.com/apple/swift-nio/pull/3455 --- ## File: IntegrationTests/tests_04_performance/test_01_resources/README.md # Allocation Counting Test This briefly describes how the allocation counting test works. ## How does it work? This is possibly the simplest implementation that counts memory allocations (`malloc` and friends) and frees (mostly `free`). It just maintains two atomic variables which count the number of mallocs and the number of frees respectively. We run a simple HTTP1 example -- 1000 requests and responses generated by a simple SwiftNIO based client and server -- and then evaluate the number of mallocs and frees. The difference `mallocs - frees` should be pretty much 0 and the number of `mallocs` should remain stable (or decrease) across commits. We can't establish a perfect baseline as the exact number of allocations depends on your operating system, libc and Swift version. ### How are the functions hooked? Usually in UNIX it's enough to just define a function, for example ```C void free(void *ptr) { ... } ``` in the main binary and all modules will use this `free` function instead of the real one from the `libc`. For Linux, this is exactly what we're doing, the `bootstrap` binary defines such a `free` function in its `main.c`. On Darwin (macOS/iOS/...) however that is not the case and you need to use [dyld's interpose feature](https://books.google.co.uk/books?id=K8vUkpOXhN4C&lpg=PA73&ots=OMjhRWWwUu&dq=dyld%20interpose&pg=PA73#v=onepage&q=dyld%20interpose&f=false). The odd thing is that dyld's interposing _only_ works if it's in a `.dylib` and not from a binary's main executable. Therefore we need to build a slightly strange SwiftPM package: - `bootstrap`: The main executable's main module (written in C) so we can hook the `free` function on Linux. - `BootstrapSwift`: A SwiftPM module (written in Swift) called in from `bootstrap` which implements the actual SwiftNIO benchmark (and therefore depends on the `NIO` module). - `HookedFunctions`: A separate SwiftPM package that builds a shared library (`.so` on Linux, `.dylib` on Darwin) which contains the `replacement_malloc`, `replacement_free`, etc functions which just increment an atomic integers representing the number of operations. On Darwin, we use `DYLD_INTERPOSE` in this module, interposing libc functions with our `replacement_` functions. This needs to be a separate SwiftPM package as otherwise its code would just live inside of the `bootstrap` executable and the dyld interposing feature wouldn't work. - `AtomicCounter`: SwiftPM package (written in C) that implements the atomic counters. It needs to be a separate package as both `BoostrapSwift` (to read the allocation counter) as well as `HookedFree` (to increment the allocation counter) depend on it. ## What benchmark is run? We run a single TCP connection over which 1000 HTTP requests are made by a client written in NIO, responded to by a server also written in NIO. We re-run the benchmark 10 times and return the lowest number of allocations that has been made. ## Why do I have to set a baseline? By default this test should always succeed as it doesn't actually compare the number of allocations to a certain number. The reason is that this number varies ever so slightly between operating systems and Swift versions. At the time of writing on macOS we got roughly 326k allocations and on Linux 322k allocations for 1000 HTTP requests & responses. To set a baseline simply run ```bash export MAX_ALLOCS_ALLOWED_1000_reqs_1_conn=327000 ``` or similar to set the maximum number of allocations allowed. If the benchmark exceeds these allocations the test will fail.