iggy Documentation

Technical Documentation: apache/iggy

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`) Copy Section

Contents: What is Apache Iggy? ·
Features ·
Architecture ·
Clustering ·
Version ·
Supported languages SDK ·
CLI ·
Web UI ·
Connectors ·
Kafka gateway ·
Model Context Protocol ·
Docker ·
Kubernetes ·
Configuration ·
Run the server ·
Try the CLI ·
Examples ·
SDK ·
Benchmarks ·
Contributing

What is Apache Iggy? Copy Chapter

Iggy is a persistent message streaming platform written in Rust, supporting QUIC, WebSocket, TCP (custom binary specification) and HTTP (regular REST API) transport protocols, capable of processing millions of messages per second at ultra-low latency.

Iggy provides exceptionally high throughput and performance while utilizing minimal computing resources.

This is not yet another extension running on top of existing infrastructure, such as Kafka or SQL database.

Iggy is a persistent message streaming log built from the ground up using low-level I/O with thread-per-core shared nothing architecture, io_uring and compio for maximum speed and efficiency.

The name is an abbreviation for the Italian Greyhound - small yet extremely fast dogs, the best in their class. See the lovely Fabio & Cookie ❤️

Features Copy Chapter

• Highly performant, persistent append-only log for message streaming
• Very high throughput for both writes and reads
• Low latency and predictable resource usage thanks to the Rust compiled language (no GC) and io_uring.
• User authentication and authorization with granular permissions and Personal Access Tokens (PAT)
• Support for multiple streams, topics and partitions
• Support for multiple transport protocols (QUIC, WebSocket, TCP, HTTP)
• Fully operational RESTful API which can be optionally enabled
• Available client SDK in multiple languages
• Thread per core shared nothing design together with io_uring guarantee the best possible performance on modern Linux systems.
• Works directly with binary data, avoiding enforced schema and serialization/deserialization overhead
• Custom zero-copy (de)serialization, which greatly improves the performance and reduces memory usage.
• Configurable server features (e.g. caching and transport protocols), plus per-topic segment size, durability and flush thresholds
• Server-side storage of consumer offsets
• Multiple ways of polling the messages:
  • By offset (using the indexes)
  • By timestamp (using the time indexes)
  • First/Last N messages
  • Next N messages for the specific consumer
• Optional poll auto-commit; processing guarantees depend on application processing and offset-commit ordering
• Consumer groups providing the message ordering and horizontal scaling across the connected clients
• Message expiry with auto deletion based on the configurable retention policy
• Additional features such as server side message deduplication
• Multi-tenant support via abstraction of streams which group topics
• TLS support for all transport protocols (TCP, WebSocket, QUIC, HTTPS)
• Connectors - sinks, sources and data transformations based on the custom Rust plugins
• Model Context Protocol - provide context to LLM with MCP server
• Optional server-side as well as client-side data encryption using AES-256-GCM
• Optional metadata support in the form of message headers
• Server-side message compression is not supported yet. Topic compression values
  are reserved for future disk/network compression support; use message headers
  for manual compression today (see examples/rust/src/message-headers/message-compression).
• Optional data backups and archiving to disk or S3 compatible cloud storage (e.g. AWS S3)
• Prometheus metrics for the server and connectors runtime, plus OpenTelemetry logs & traces in the connectors runtime. Server OTLP export is unavailable pending runtime integration.
• Built-in CLI to manage the streaming server installable via cargo install iggy-cli
• Built-in benchmarking app to test the performance
• Single binary deployment without an external broker or database; dynamically linked builds still require operating-system libraries
• Running as a single node or as a cluster, with data replication based on Viewstamped Replication (VSR)

Architecture Copy Chapter

This is the high-level architecture of the Iggy message streaming server, where extremely high performance and ultra low and stable tail latencies are the primary goals. The server is designed to handle high throughput and very low latency (sub-millisecond tail latencies), making it suitable for real-time applications. For more details, please refer to the documentation.

Clustering Copy Chapter

Clustering is built into the standard iggy-server binary, with no separate build or feature flag. The same server runs as a single node or as a cluster, and cluster.enabled in the configuration chooses which. Clustering is disabled by default.

Replicas stay consistent using Viewstamped Replication (VSR), which commits on a quorum and elects a new primary when the current one fails. See the clustering documentation for configuration and deployment.

Version Copy Chapter

The official releases follow semver (for example server-0.9.0), and the Docker images also carry the latest tag (apache/iggy:latest).

The server, SDKs and CLI are versioned independently. For example, server 0.9.0 ships with Rust SDK 0.11.0 and CLI 0.14.0. The server compatibility table lists the SDK version for each server release.

We also publish edge releases (versions with an -edge.N suffix, and apache/iggy:edge) for the SDKs and the Docker images. They track the latest changes, but are not guaranteed to be stable and are not recommended for production use.

Supported languages SDK Copy Chapter

• Rust
• C#
• Java
• Python
• Node.js (TypeScript)
• Go

C++ and PHP are work in progress.

CLI Copy Chapter

The interactive CLI is implemented under core/cli, to provide the best developer experience. This is a great addition to the Web UI, especially for all the developers who prefer using the console tools.

Iggy CLI can be installed with cargo install iggy-cli and then simply accessed by typing iggy in your terminal.

Web UI Copy Chapter

There's a dedicated Web UI for the server, which allows managing the streams, topics, partitions, browsing the messages and so on. This is an ongoing effort to build a comprehensive dashboard for administrative purposes of the Iggy server. Check the Web UI in the /web directory. The docker image for Web UI is available, and can be fetched via docker pull apache/iggy-web-ui.

Connectors Copy Chapter

The highly performant and modular runtime for statically typed, yet dynamically loaded connectors. Ingest the data from the external sources and push it further to the Iggy streams, or fetch the data from the Iggy streams and push it further to the external sources. Create your own Rust plugins by simply implementing either the Source or Sink trait and build custom pipelines for the data processing.

## Configure a sink or source connector, depending on your needs in its own config file.
type = "sink"
key = "quickwit"
enabled = true
version = 0
name = "Quickwit sink"
path = "target/release/libiggy_connector_quickwit_sink"
plugin_config_format = "yaml"

[[streams]]
stream = "qw"
topics = ["records"]
schema = "json"
batch_length = 1000
poll_interval = "5ms"
consumer_group = "qw_sink_connector"

[transforms.add_fields]
enabled = true

[[transforms.add_fields.fields]]
key = "service_name"
value.static = "qw_connector"

[[transforms.add_fields.fields]]
key = "timestamp"
value.computed = "timestamp_millis"

[transforms.delete_fields]
enabled = true
fields = ["email", "created_at"]

Kafka gateway Copy Chapter

The Kafka gateway is a Kafka wire protocol listener in front of Iggy, intended to let existing Kafka clients use Iggy. It is in development and not yet part of a release. See its README for the current status and how to run it.

Model Context Protocol Copy Chapter

The Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to LLMs. The Iggy MCP Server is an implementation of the MCP protocol for the message streaming infrastructure. It can be used to provide context to LLMs in real-time, allowing for more accurate and relevant responses.

Docker Copy Chapter

The official images are on Docker Hub. Images tagged latest are built from stable releases, and edge images are built from the latest master branch. Images for the other tools, such as the connectors runtime and the MCP server, are also on Docker Hub.

The published image listens on 127.0.0.1 inside the container, so the host can't reach it by default. This command makes it reachable and sets the root credentials:

docker run -d --name iggy \
  --cap-add=SYS_NICE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
  -e IGGY_ROOT_USERNAME=iggy -e IGGY_ROOT_PASSWORD=iggy \
  -e IGGY_TCP_ADDRESS=0.0.0.0:8090 -e IGGY_HTTP_ADDRESS=0.0.0.0:3000 \
  -e IGGY_NODE_ADVERTISED_ADDRESS=localhost \
  -p 8090:8090 -p 3000:3000 \
  -v iggy:/app/local_data \
  apache/iggy

The CLI is on the image's PATH:

docker exec -it iggy iggy -u iggy -p iggy stream list

The SYS_NICE capability, the seccomp setting and the unlimited locked memory are required. The Docker & Helm page explains why, and has a docker-compose.yml example and the Helm chart.

On operating systems other than Linux, Docker runs in a VM, so the container might run slower.

To build the image from source instead, run docker compose up from the root of the repository. That image listens on all interfaces and keeps its data in /local_data. The container is named iggy-server, and the CLI is at /iggy. The compose file requires IGGY_ROOT_PASSWORD from your shell (IGGY_ROOT_PASSWORD=... docker compose up) and defaults IGGY_ROOT_USERNAME to iggy.

Kubernetes Copy Chapter

A Helm chart for the server and the Web UI is in helm/charts/iggy:

helm install iggy ./helm/charts/iggy --set server.image.tag=0.9.0

See the chart README and the Docker & Helm page for its settings.

Configuration Copy Chapter

The default configuration can be found in config.toml file in core/server directory.

The configuration file is loaded from the current working directory, but you can specify the path to the configuration file by setting IGGY_CONFIG_PATH environment variable, for example export IGGY_CONFIG_PATH=core/server/config.toml (or other command depending on OS).

When config file is not found, the default values from embedded config.toml file are used.

Topic creation accepts two independent policies: durability for message acknowledgments and consumer_offset_durability for explicit offset stores and deletes. Both default to replicated. This means VSR quorum commit without waiting for stable storage. persisted also requires recoverable stable-storage copies on the replication quorum. Both policies normally store data on disk. Poll auto-commit remains asynchronous and is not covered by the poll response's completion.

The data directory is configured with path or IGGY_PATH. The layout beneath it is streams/<stream>/topics/<topic>/partitions/, with fixed directory names.

The HTTP Iggy-Durability header reports replicated or persisted for awaited writes, and none for early dispatch acceptance.

Segment flush thresholds control scheduling, independently of acknowledgment durability.

Rust HTTP callers can use HttpClient::send_messages_with_durability to read the advertised guarantee alongside confirmations.

The CLI exposes --durability persisted and --consumer-offset-durability persisted on topic create. Select either independently. The policy names describe completion guarantees and do not prescribe an I/O syscall.

For the detailed documentation of the configuration file, please refer to the configuration section.

Run the server Copy Chapter

Build the project (the longer compilation time is due to LTO enabled in release profile:

cargo build

Run the tests:

cargo test

Set root user credentials before the first start:

Iggy requires credentials to authenticate requests to the server.
Set the root user before starting the server.

(macOS/Linux)

export IGGY_ROOT_USERNAME=iggy
export IGGY_ROOT_PASSWORD=iggy

(Windows(Powershell))

$env:IGGY_ROOT_USERNAME = "iggy"
$env:IGGY_ROOT_PASSWORD = "iggy"

When root does not exist, the leader needs both variables or --with-default-root-credentials below to create it. The server never generates a root password and never writes either credential to its output or log files.

Start the server:

cargo run --bin iggy-server

All the data used by the server will be persisted under the local_data directory by default, unless specified differently in the configuration (see path in config.toml).

One can use default root credentials with optional --with-default-root-credentials.
This flag is equivalent to setting IGGY_ROOT_USERNAME=iggy and IGGY_ROOT_PASSWORD=iggy, plus
it should only be used for development and testing.

cargo run --bin iggy-server -- --with-default-root-credentials

The leader creates root once and records the username and password hash in the metadata journal. Followers apply that same operation. Restarts recover the stored account without the variables or the flag, even before the first metadata checkpoint. Supplying different credentials does not replace the stored account. A half-set pair or an out-of-range value still aborts startup. To change the root password, use the API or the CLI.

For configuration options and detailed help:

cargo run --bin iggy-server -- --help

You can also use environment variables to override any configuration setting:

• Override TCP address
  IGGY_TCP_ADDRESS=127.0.0.1:8090 cargo run --bin iggy-server

• Set custom data path
  IGGY_PATH=/data/iggy cargo run --bin iggy-server

• Enable HTTP transport
  IGGY_HTTP_ENABLED=true cargo run --bin iggy-server

• Set custom root user credentials
  IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy cargo run --bin iggy-server

To quickly generate the sample data:

cargo run --bin data-seeder-tool

Try the CLI Copy Chapter

The commands below use the iggy CLI from the core/cli crate, run against the server started above.

Create a stream with name dev (numerical ID will be assigned by server automatically) using default credentials and tcp transport (available transports: quic, websocket, tcp, http, default tcp):

cargo run --bin iggy -- --transport tcp --username --password stream create dev

List available streams:

cargo run --bin iggy -- --username --password stream list

Get dev stream details:

cargo run --bin iggy -- -u -p stream get dev

Create a topic named sample (numerical ID will be assigned by server automatically) for stream dev, with 2 partitions (IDs 0 and 1), no topic compression (none), and disabled message expiry (skipped optional parameter). Other compression values are reserved for future server-side support:

cargo run --bin iggy -- -u -p topic create dev sample 2 none

List available topics for stream dev:

cargo run --bin iggy -- -u -p topic list dev

Get topic details for topic sample in stream dev:

cargo run --bin iggy -- -u -p topic get dev sample

Send the first message 'hello world' to the stream dev to topic sample and partition 0:

cargo run --bin iggy -- -u -p message send --partition-id 0 dev sample "hello world"

Send a second message 'lorem ipsum' to the same stream, topic and partition:

cargo run --bin iggy -- -u -p message send --partition-id 0 dev sample "lorem ipsum"

Poll messages by a regular consumer with ID 1 from the stream dev for topic sample and partition with ID 0, starting with offset 0, messages count 2, with auto commit (storing consumer offset on server):

cargo run --bin iggy -- -u -p message poll --consumer 1 --offset 0 --message-count 2 --auto-commit dev sample 0

Finally, restart the server to see it is able to load the persisted data.

The HTTP API endpoints can be found in server.http file, which can be used with REST Client extension for VS Code.

To see the detailed logs from the CLI/server, run it with the RUST_LOG=trace environment variable.

Examples Copy Chapter

You can find comprehensive sample applications under the examples/rust directory. These examples showcase various usage patterns of the Iggy client SDK, from basic operations to advanced multi-tenant scenarios.

For detailed information about available examples and how to run them, please see the Examples README.

There are also examples for C#, Go, Java, Node.js, PHP and Python.

SDK Copy Chapter

Iggy comes with the Rust SDK, which is available on crates.io.

The SDK provides both, low-level client for the specific transport, which includes the message sending and polling along with all the administrative actions such as managing the streams, topics, users etc., as well as the high-level client, which abstracts the low-level details and provides the easy-to-use API for both, message producers and consumers.

You can find the more examples, including the multi-tenant one under the examples directory.

// Create the Iggy client
let client = IggyClient::from_connection_string("iggy://user:secret@localhost:8090")?;

// Create a producer for the given stream and one of its topics
let mut producer = client
    .producer("dev01", "events")?
    .direct( // Use either direct (instant) or background message sending
        DirectConfig::builder()
            .batch_length(1000)
            .linger_time(IggyDuration::from_str("1ms")?)
            .build(),
    )
    .partitioning(Partitioning::balanced())
    .build();

producer.init().await?;

// Send some messages to the topic
let messages = vec![IggyMessage::from_str("Hello Apache Iggy")?];
producer.send(messages).await?;

// Create a consumer for the given stream and one of its topics
let mut consumer = client
    .consumer_group("my_app", "dev01", "events")?
    .auto_commit(AutoCommit::IntervalOrWhen(
        NonZeroIggyDuration::from_str("1s")?,
        AutoCommitWhen::ConsumingAllMessages,
    ))
    .create_consumer_group_if_not_exists()
    .auto_join_consumer_group()
    .polling_strategy(PollingStrategy::next())
    .poll_interval(IggyDuration::from_str("1ms")?)
    .batch_length(1000)
    .build();

consumer.init().await?;

// Start consuming the messages
while let Some(message) = consumer.next().await {
    // Handle the message
}

Benchmarks Copy Chapter

Benchmarks should be the first-class citizens. We believe that performance is crucial for any system, and we strive to provide the best possible performance for our users. Please check, why we believe that the transparent
benchmarking is so important.

We've also built the benchmarking platform where anyone can upload the benchmarks and compare the results with others. Source code for the platform is available in the core/bench/dashboard directory.

For the benchmarking purposes, we've developed the dedicated iggy-bench tool, which is a part of the iggy project. It is a command-line tool that allows you to run the variety of fully customizable benchmarks.

To benchmark the project, first build the project in release mode:

cargo build --release

Start iggy-server separately, then run the benchmarking app with the desired options:

1. Sending (writing) benchmark

   bash Copy

   cargo run --bin iggy-bench -r -- pinned-producer tcp

2. Polling (reading) benchmark

   bash Copy

   cargo run --bin iggy-bench -r -- pinned-consumer tcp

3. Parallel sending and polling benchmark

   bash Copy

   cargo run --bin iggy-bench -r -- pinned-producer-and-consumer tcp

4. Balanced sending to multiple partitions benchmark

   bash Copy

   cargo run --bin iggy-bench -r -- balanced-producer tcp

5. Consumer group polling benchmark:

   bash Copy

   cargo run --bin iggy-bench -r -- balanced-consumer-group tcp

6. Parallel balanced sending and polling from consumer group benchmark:

   bash Copy

   cargo run --bin iggy-bench -r -- balanced-producer-and-consumer-group tcp

7. End to end producing and consuming benchmark (single task produces and consumes messages in sequence):

   bash Copy

   cargo run --bin iggy-bench -r -- end-to-end-producing-consumer tcp

8. End to end producing and consuming through a consumer group:

   bash Copy

   cargo run --bin iggy-bench -r -- end-to-end-producing-consumer-group tcp

The benchmark connects to a running server and creates the streams, topics, and partitions needed by the selected workload. Use iggy-bench --help and iggy-bench examples for all benchmark variants, transports, and topic-option examples. Both message and consumer-offset durability independently default to replicated.

For example, to run the benchmark for the already started server, provide the additional argument --server-address 127.0.0.1:8090.

Iggy is already capable of processing millions of messages per second at the microseconds range for p99+ latency Depending on the hardware, transport protocol (quic, websocket, tcp or http) and payload size (messages-per-batch * message-size) you might expect over 5000 MB/s (e.g. 5M of 1 KB msg/sec) throughput for writes and reads.

Please refer to the mentioned benchmarking platform where you can browse the results achieved on the different hardware configurations, using the different Iggy server versions.

Host preparation Copy Section

Check io_uring access, process limits, memory headroom, CPU/NUMA placement, and sustained disk/network capacity before comparing runs. Measure host-tuning changes with the same workload and durability policies.

Use the benchmark host checklist for practical setup and repeatable measurements. The Linux tuning guide explains swappiness, huge pages, writeback, CPU placement, and networking, with commands and upstream references.

Contributing Copy Chapter

Please see Contributing

Overview (`bdd/README.md`) Copy Section

Cross-SDK BDD Testing

This directory contains cross-SDK Behavior-Driven Development (BDD) tests for Apache Iggy, designed to ensure consistency across different language SDKs.

Structure Copy Chapter

/* Detailed source-code truncated for AI context efficiency. */

Usage Copy Chapter

Quick Start Copy Section

# Usage: ../scripts/run-bdd-tests.sh [--coverage] <sdk> [feature]
#   sdk:     rust | python | php | go | go-race | node | csharp | java | all | clean  (default: all)
#   feature: basic_messaging | leader_redirection | all  (default: all)

# Run all features for all SDKs
../scripts/run-bdd-tests.sh all

# Run specific SDK tests (all features)
../scripts/run-bdd-tests.sh rust
../scripts/run-bdd-tests.sh python
../scripts/run-bdd-tests.sh go
../scripts/run-bdd-tests.sh node
../scripts/run-bdd-tests.sh csharp
../scripts/run-bdd-tests.sh java
../scripts/run-bdd-tests.sh php

# Run only basic_messaging feature for Rust SDK
../scripts/run-bdd-tests.sh rust basic_messaging

# Run only leader_redirection
../scripts/run-bdd-tests.sh all leader_redirection

# Every suite runs against iggy-server. Build the binaries first:
#   cargo build --bin iggy-server --bin iggy

# Clean up Docker resources
../scripts/run-bdd-tests.sh clean

Requirements Copy Section

• Docker and Docker Compose
• The tests build the latest Iggy server from source in debug mode for faster compilation

How it Works Copy Section

1. Server Containers: docker-compose.server.yml runs a single Iggy server; docker-compose.cluster.yml adds a leader + follower pair for cluster scenarios
2. SDK Containers: Each SDK has its own container with the appropriate runtime and dependencies
3. Shared Features: All SDKs test against the same .feature files for consistency
4. Health Checks: Containers wait for the server to be healthy before running tests

Adding New SDKs Copy Section

To add a new SDK (e.g., Node.js):

1. Create node/ directory
2. Add node/Dockerfile with appropriate runtime and dependencies
3. Create node/tests/ directory with BDD implementation
4. Add node-bdd service to docker-compose.yml
5. Update ../scripts/run-bdd-tests.sh script
6. Update components.yml file to include the new SDK files

CI/CD Integration Copy Section

GitHub Actions workflow: ci-test-bdd.yml

Development Copy Chapter

For Rust SDK Copy Section

The Rust implementation is located in bdd/rust/ and linked via Docker volumes.

For Python SDK Copy Section

The Python implementation is in bdd/python/tests/ and needs to be updated as the Python SDK API evolves.

For Go SDK Copy Section

The Go implementation is located in bdd/go/tests/.

For Node SDK Copy Section

The node.js BDD test are run by cucumber-js, bdd test code is located at foreign/node/src/bdd

For csharp SDK Copy Section

The csharp implementation is located at foreign/csharp/Iggy_SDK.Tests.BDD

For Java SDK Copy Section

The Java implementation is located in java/src/test/

For PHP SDK Copy Section

The PHP implementation is located in bdd/php/tests/ and runs through PHPUnit.

Adding New Scenarios Copy Section

Add new .feature files to the bdd/scenarios/ directory and implement the corresponding step definitions in each SDK's test directory.

Overview (`examples/README.md`) Copy Section

Apache Iggy Examples

Here you can find the examples of how to use Apache Iggy with the different programming languages. Please note that the features available in each language may vary.

Overview (`gateways/README.md`) Copy Section

Apache Iggy Gateways

Protocol gateways that let existing clients talk to Iggy without changing the core server wire surface.

Each gateway is a separate workspace crate under gateways/<name>/.

Overview (`rfcs/README.md`) Copy Section

Iggy RFCs

The "RFC" (request for comments) process is intended to provide a consistent
and controlled path for changes to Iggy (such as new features) so that all
stakeholders can be confident about the direction of the project.

Many changes, including bug fixes and documentation improvements can be
implemented and reviewed via the normal GitHub pull request workflow.

Some changes though are "substantial", and we ask that these be put through a
bit of a design process and produce a consensus among the Iggy community and
the maintainers.

Table of Contents Copy Chapter

• Opening
• Table of Contents
• When you need to follow this process
• Before creating an RFC
• What the process is
• RFC numbering
• The RFC life-cycle
• Help this is all too informal
• License
• Contributions

When you need to follow this process Copy Chapter

You need to follow this process if you intend to make "substantial" changes to
the Iggy server, the wire protocol, the SDKs, the connectors runtime, or the RFC
process itself. What constitutes a "substantial" change is evolving based on
community norms and varies depending on what part of the project you are
proposing to change, but may include the following.

• Any change to the wire protocol or the on-disk format that is not a bugfix.
• A new storage, replication, or consumption model, or a change to how an
  existing one behaves.
• Removing features, including those behind a feature flag.
• Large additions to the SDK surface that every SDK must then implement.

Some changes do not require an RFC:

• Rephrasing, reorganizing, refactoring, or otherwise "changing shape does
  not change meaning".
• Additions that strictly improve objective, numerical quality criteria
  (warning removal, speedup, better platform coverage, more parallelism, trap
  more errors, etc.)
• Additions only likely to be noticed by other developers-of-iggy,
  invisible to users-of-iggy.

If you submit a pull request to implement a new feature without going through
the RFC process, it may be closed with a polite request to submit an RFC first.

Before creating an RFC Copy Chapter

A hastily-proposed RFC can hurt its chances of acceptance. Low quality
proposals, proposals for previously-rejected features, or those that don't fit
into the near-term roadmap, may be quickly rejected, which can be demotivating
for the unprepared contributor. Laying some groundwork ahead of the RFC can
make the process smoother.

Although there is no single way to prepare for submitting an RFC, it is
generally a good idea to pursue feedback from other project developers
beforehand, to ascertain that the RFC may be desirable; having a consistent
impact on the project requires concerted effort toward consensus-building.

The most common preparations for writing and submitting an RFC include talking
the idea over on our Discord server, discussing the topic in GitHub
Discussions, and occasionally posting "pre-RFCs" there. A pre-RFC thread is
separate from the RFC's own Discussion, which is opened later, once the text
exists and has had its initial review. You may file issues on this repo for
discussion, but these are not actively looked at by the maintainers.

As a rule of thumb, receiving encouraging feedback from long-standing project
developers, and particularly from the maintainers, is a good indication that
the RFC is worth pursuing.

What the process is Copy Chapter

In short, to get a major feature added to Iggy, one must first get the RFC
merged into the rfcs/ directory of this repository as a markdown file. At
that point the RFC is "active" and may be implemented with the goal of eventual
inclusion into Iggy.

• Fork the Iggy repository.
• Copy rfcs/rfc-template.md to rfcs/rfc-0-my-feature.md (where
  "my-feature" is descriptive). Don't assign an RFC number yet; 0 marks an
  unassigned number, and one is assigned when the RFC is accepted and the file
  is renamed accordingly. See RFC numbering.
• Fill in the RFC. Put care into the details: RFCs that do not present
  convincing motivation, demonstrate lack of understanding of the design's
  impact, or are disingenuous about the drawbacks or alternatives tend to
  be poorly-received.
• Submit a pull request titled rfc(<scope>): <subject>, choosing the scope
  as described in the commit message section of CONTRIBUTING.md, which also
  covers a scope that matches nothing yet. The pull request carries the text.
  A maintainer gives it an initial review within a week: whether the change
  needs an RFC at all, whether the text is complete enough to debate, and
  editorial comments on the writing. If the change does not need an RFC, the
  pull request is closed and the change goes through the normal pull request
  workflow. If the text is not complete enough to debate, the maintainer
  labels the pull request S-waiting-on-author until it is.
• Now that your RFC has an open pull request, update the "RFC PR" link at the
  top of the file to point at it.
• A maintainer marks the RFC ready for discussion by labeling the pull request
  S-waiting-on-review. The label stays for the lifetime of the RFC, because
  the debate happens outside the pull request and the stale bot otherwise
  closes a pull request after 14 days without activity on it.
• Once the RFC is ready for discussion, open a GitHub
  Discussion in the RFCs category titled RFC: <feature>, linking the pull
  request and quoting the summary, and update the "Discussion" link at the top
  of the file to point at it. The Discussion is where the design is debated;
  the pull request keeps editorial review.
• Build consensus and integrate feedback. RFCs that have broad support are
  much more likely to make progress than those that don't receive any
  comments. Feel free to reach out to the maintainers in particular to get
  help identifying stakeholders and obstacles.
• The maintainers will discuss the RFC, as much as possible in the Discussion
  itself. Offline discussion will be summarized in the Discussion.
• RFCs rarely go through this process unchanged, especially as alternatives
  and drawbacks are shown. You can make edits, big and small, to the RFC to
  clarify or change the design, but make changes as new commits to the pull
  request, and reply in the Discussion with the commit hash and permalinks to
  the changed lines, explaining your changes. Specifically, do not squash or
  rebase commits after they are visible on the pull request.
• At some point, a maintainer will propose a "motion for final comment period"
  (FCP) in the Discussion, along with a disposition for the RFC (merge or
  close).
  • This step is taken when enough of the tradeoffs have been discussed that
    the maintainers are in a position to make a decision. That does not require
    consensus amongst all participants in the Discussion (which is usually
    impossible). However, the argument supporting the disposition on the RFC
    needs to have already been clearly articulated, and there should not be a
    strong consensus against that position outside of the maintainers.
    Maintainers use their best judgment in taking this step, and the FCP itself
    ensures there is ample time and notification for stakeholders to push
    back if it is made prematurely.
  • For RFCs with lengthy discussion, the motion to FCP is usually preceded by
    a summary comment trying to lay out the current state of the discussion
    and major tradeoffs/points of disagreement.
  • Before actually entering FCP, the maintainers must sign off; this is often
    the point at which many of them first review the RFC in full depth. The
    maintainer who proposed the motion records the sign-offs in the
    Discussion and announces there when the FCP starts and when it ends.
• The FCP lasts ten calendar days, so that it is open for at least 5 business
  days. The Discussion is mirrored to the dev mailing list, but replies on
  the list do not reach the Discussion, so final objections must be raised in
  the Discussion itself.
• In most cases, the FCP period is quiet, and the RFC is either merged or
  closed. However, sometimes substantial new arguments or ideas are raised,
  the FCP is canceled, and the RFC goes back into development mode. If the
  reasoning behind the decision is not clear from the Discussion, the
  maintainers add a comment there describing it.

RFC numbering Copy Chapter

RFC numbers are small sequential integers without zero padding, assigned when
an RFC is accepted, so the first RFC is rfc-1. When an FCP ends with a merge
disposition, the maintainer who announced its end opens the tracking issue for
the implementation and tells the author the next free number. The author then
renames rfcs/rfc-0-my-feature.md to that number, for example
rfcs/rfc-7-my-feature.md, and fills in the "Iggy Issue" link, before the
final approvals, since a push after approval dismisses it. The pull request is
then squash-merged, and the maintainer posts a summary comment in the
Discussion linking the merged RFC and the tracking issue.

Pull request numbers are not used as RFC numbers because this repository's
counter is shared with every other pull request, which would make RFC numbers
large, sparse, and meaningless as a sequence. While an RFC is under review it
is referred to by its feature name.

The RFC life-cycle Copy Chapter

Once an RFC becomes "active" then authors may implement it and submit the
feature as a pull request to the Iggy repository. Being "active" is not a
rubber stamp, and in particular still does not mean the feature will ultimately
be merged; it does mean that in principle all the major stakeholders have
agreed to the feature and are amenable to merging it.

Every accepted RFC has an associated issue tracking its implementation in the
Iggy repository, so it can be assigned a priority via the triage process that
the maintainers use for all issues. If you are interested in working on the
implementation for an "active" RFC, but cannot determine if someone else is
already working on it, feel free to ask on that issue.

Furthermore, the fact that a given RFC has been accepted and is "active"
implies nothing about what priority is assigned to its implementation, nor does
it imply anything about whether an Iggy developer has been assigned the task of
implementing the feature. While it is not necessary that the author of the
RFC also write the implementation, it is by far the most effective way to see
an RFC through to completion: authors should not expect that other project
developers will take on responsibility for implementing their accepted feature.

Modifications to "active" RFCs can be done in follow-up pull requests. We
strive to write each RFC in a manner that it will reflect the final design of
the feature; but the nature of the process means that we cannot expect every
merged RFC to actually reflect what the end result will be at the time of the
next major release.

In general, once accepted, RFCs should not be substantially changed. Only very
minor changes should be submitted as amendments. More substantial changes
should be new RFCs, with a note added to the original RFC. Exactly what counts
as a "very minor change" is up to the maintainers to decide.

Help this is all too informal Copy Chapter

The process is intended to be as lightweight as reasonable for the present
circumstances. As usual, we are trying to let the process be driven by
consensus and community norms, not impose more structure than necessary.

License Copy Chapter

The contents of this directory are licensed under the
Apache License, Version 2.0,
like the rest of the repository.

This README and rfc-template.md are adapted from the
rust-lang/rfcs repository, which is
licensed under the MIT and Apache 2.0 licenses.

Contributions Copy Section

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
licensed as above, without any additional terms or conditions.

Overview (`web/README.md`) Copy Section

Apache Iggy Web UI

This project hosts the web user interface for Apache Iggy. The web UI is built using SvelteKit.

Overview Copy Chapter

The Iggy Web UI provides a user-friendly panel for managing various aspects of the Iggy platform, including streams, topics, partitions, and more.

The docker image is available, and can be fetched via docker pull apache/iggy-web-ui:edge.

Tooling Copy Chapter

• Node.js: use a version supported by the current frontend toolchain, ^22.17.0 || >=24.
• Package manager: npm
• pnpm and yarn are not part of the supported workflow for this package. CI, Docker builds, and the committed lockfile use npm.

Getting Started Copy Section

1. Run Iggy server first

   sh Copy

   docker pull apache/iggy:edge

   sh Copy

   docker run --security-opt seccomp=unconfined -p 3000:3000 -p 8090:8090 \
     -e IGGY_ROOT_USERNAME=iggy -e IGGY_ROOT_PASSWORD=iggy \
     -e IGGY_HTTP_ADDRESS=0.0.0.0:3000 -e IGGY_TCP_ADDRESS=0.0.0.0:8090 \
     -e IGGY_NODE_ADVERTISED_ADDRESS=localhost apache/iggy:edge

2. Clone the repository:

   sh Copy

   git clone https://github.com/apache/iggy.git

3. Build the project:

   sh Copy

   cd iggy/web
   npm ci

4. Run the project:

   sh Copy

   npm run dev

   To expose port in local network run:

   sh Copy

   npm run dev -- --host --port 3333

   Set PUBLIC_IGGY_API_URL in web/.env to the HTTP address reachable by your browser, for example:

   sh Copy

   PUBLIC_IGGY_API_URL=http://127.0.0.1:3000

   This applies to both container and source-built servers. Use the host-published HTTP port for a container.

Roadmap Copy Chapter

• Authorization
• Allow manual interval setting for auto-refresh
• Introduce dark mode
• Enable CRUD operations on Streams
• Enable CRUD operations on Topics
• Messages management
• General UI improvements
• Permission management (in progress)
• GitHub actions CI/CD
• Publish to Dockerhub as separate Image

Contributing Copy Chapter

Feel free to contribute to the project. Your feedback and contributions are highly appreciated!

Overview (`.github/review-bot/README.md`) Copy Section

Skill review bot

A repository committer comments /skill <name> on a pull request. A headless agent runs that skill against the pull request head, and the findings arrive as one pull request review with inline comments.

Commands Copy Chapter

• /skill <name> runs the skill on the default branch at .claude/skills/<name>/SKILL.md, or .agents/skills/<name>/SKILL.md.
• /skill answers with the list of available skill names.

Any committer can write a skill and use it this way. The gate is a repository write permission, so the PR author alone cannot start a run.

The run reads the skill file, so a skill marked disable-model-invocation: true still works here. That flag keeps a skill out of an agent's own reach, which is the point of a human typing the command.

The two stages Copy Chapter

1. pr-skill-review-run.yml runs on issue_comment. First it parses the command and gates the author. It then makes sure that the skill exists on the default branch. After that it installs the Claude Code CLI, checks out the pull request head, and runs the agent. The agent writes findings.json.
2. pr-skill-review-post.yml runs on workflow_run, where the token can write. It reads the trigger comment back from the API, so the run artifact cannot aim the review elsewhere. Then it anchors each finding on a changed line and posts one review.

The review body opens with the summary and a count per severity. A finding with no anchor goes into the body as text. When the head moved during the run, the whole set goes to the body. The poster answers every conclusion, a cancelled run included. It also answers a pull request that closed while the run worked, and a post that the API refused.

The agent also writes flip_author_label, which says whether the findings need the author before merge. If it is true and the review shows a critical or warning finding, the poster replaces S-waiting-on-review with S-waiting-on-author. That is the same move as /author. The label moves in that direction only. A review that asks for nothing leaves the label as it is, and /ready moves the pull request back to the review queue. A draft gets no state label.

What the agent can do Copy Chapter

The agent reads. It cannot run a command, build the workspace or start a test. The run installs no Rust toolchain, so no build script, proc macro, cargo configuration or rustc wrapper from the pull request can execute.

• Tools: Read, Grep, Glob, Write and Agent. The CLI runs with --restricted, which removes every command tool and confines the file tools to two directories: the checkout and the agent directory under the runner temp. Subagents get the same set.
• Writes: one allow rule, the agent directory. --permission-prompts none denies every other write, a write into the checkout included.
• Inputs: the diff, its stat and file list, the CI status of the commit, and comment-style.md, all in the agent directory.
• Model: the Claude Code CLI at CLAUDE_CODE_VERSION, pointed at the DeepSeek Anthropic-compatible endpoint with the model in DEEPSEEK_MODEL. Both pins live in one job-level env block, and the posted review closes with a footer that names them.

A skill written for a terminal session still works. The prompt tells the agent which file replaces which command: the diff files replace git diff and gh pr diff, and the CI status file replaces a build or a test run. A finding that only a test can settle is dropped.

Watching a run Copy Chapter

The agent step streams each tool call into the job log, one line per call, and each denied call as a deny: line. An in-progress run is therefore readable in the Actions UI. The raw event stream and the CLI error output sit in the review-out artifact next to findings.json.

The job log closes with the cost of the run. The line prices the token counts at DeepSeek rates and at two Claude tiers, next to the estimate the CLI itself reports.

Nothing appears on the pull request while a run works. An issue_comment run of a fork receives a read-only token, so the first workflow cannot post. The answer arrives with the review at the end.

Files Copy Chapter

• prompt.md - the task and the output contract for the agent.
• comment-style.md - how each posted comment must read, wording rules included.

Both files come from the default branch, never from the pull request under review.

Secret Copy Chapter

DEEPSEEK_AUTH_TOKEN holds the DeepSeek key. Add it as a repository secret on GitHub, or with gh secret set DEEPSEEK_AUTH_TOKEN.

Cost and duration Copy Chapter

A run installs one npm package and reads code. Expect a few minutes of wall clock for a small skill, and up to 45 minutes for one that spawns subagents, on one hosted runner. The agent tokens come from the DeepSeek key.

Security notes Copy Chapter

• No pull request code runs. The agent has no shell, and nothing in the run builds, tests or installs from the checkout.
• The DeepSeek key sits in the environment of the agent step. Without a shell the agent cannot read that environment, and /proc is outside its working directories. Keep the key scoped to this bot and renewable all the same.
• No GitHub token reaches the agent step. The checkout sets persist-credentials: false and the step receives no token, so the agent has no way to post or to push.
• The control file that names the pull request to answer sits outside the agent directory, so the agent cannot rewrite it. The poster reads the trigger comment, its author and its pull request from the API regardless.
• The poster acts only on a run of the issue_comment event, which always runs the default branch copy of the run workflow. A fork pull request can add a workflow with the same name, and the poster ignores its runs.
• .claude and .agents are deleted from the pull request checkout, and the default branch copy is restored. A pull request therefore cannot redefine the skill that reviews it. The same holds for AGENTS.md and CLAUDE.md at any depth, which the CLI loads as its own orders.
• Project configuration files, hooks and MCP servers never load. --restricted ignores the project configuration files, and --strict-mcp-config with no MCP configuration loads no server.
• The pull request text is data. The prompt says so, and a comment in the diff that addresses the reviewer can become a finding, never a command. That is a rule for the model, not a fence. A model that follows such a comment can still only read the tree and write its own directory.
• The pull request can still steer the words that the model writes. The poster first deletes the invisible characters from every model string: Unicode tags, bidi controls, zero-width characters and variation selectors. Then it adds a zero-width space that stops HTML, mentions, links and math.
• The review shows counts per severity, not a verdict from the model. The prompt keeps the verdict of a skill out of the summary. The poster also deletes a Verdict: APPROVE or Verdict: REQUEST CHANGES that opens a sentence. The verdict reaches the pull request only through flip_author_label, as the label move above. A pull request that steers the model can therefore at most keep its own label, or send itself to the author queue.
• No cache is read or written. The run has no build, so it has no cache to poison.

Overview (`core/ai/README.md`) Copy Section

Apache Iggy AI

This module contains the AI-related components of the Apache Iggy message streaming infrastructure.

Please check the MCP Server for the details about Iggy MCP (Model Context Protocol) server.

Overview (`core/bench/README.md`) Copy Section

Apache Iggy Bench CLI

The interactive Bench CLI allows you to perform various benchmarking on the Apache Iggy server.

Iggy Bench CLI can be installed with cargo install iggy-bench and then simply accessed by typing iggy-bench in your terminal.

The WebSocket transport command is websocket, with ws as a shorthand.

Producer and consumer counts default to six. Pinned workloads also default to six streams. Override the counts for the workload and available CPUs.

Examples and topic options Copy Chapter

Start Iggy before running benchmarks. The tool connects to an existing server. Run iggy-bench examples for all eight benchmark kinds, their aliases, all four transports, and topic-option combinations. Global options precede the benchmark kind, kind-specific options follow it, and the server address follows the transport.

Both --durability and --consumer-offset-durability independently default to replicated. The following runs use the same workload with different topic policies:

# Both policies replicated, the default.
iggy-bench balanced-producer-and-consumer-group tcp

# Persisted message acknowledgments with replicated offsets.
iggy-bench --durability persisted balanced-producer-and-consumer-group tcp

# Replicated message acknowledgments with persisted offsets.
iggy-bench --consumer-offset-durability persisted balanced-producer-and-consumer-group tcp

# Both policies persisted.
iggy-bench --durability persisted --consumer-offset-durability persisted balanced-producer-and-consumer-group tcp

These options apply when the benchmark creates topics. They do not change existing topics with --reuse-streams. Both policies normally write data to disk. persisted adds a stable-storage completion requirement. Consumer workloads using poll auto-commit still receive asynchronous poll responses, so this flag does not turn their poll latency into a measurement of acknowledged offset-store latency.

--messages-required-to-save controls a topic's segment-flush cadence independently of durability. --max-topic-size and --message-expiry are kind-specific topic options. Use the selected kind's --help to inspect supported flags. OS writeback settings do not replace these policies.

Host preparation Copy Chapter

Use a dedicated server and a separate load generator where possible. If they share a host, assign disjoint CPU sets and leave CPU capacity for kernel and network work. Match the deployment's CPU topology, storage, replication-group size, and durability policies.

Read the Linux tuning guide before reserving huge pages. vm.nr_hugepages=2048 reserves 4 GiB only with 2 MiB pages. MIMALLOC_RESERVE_HUGE_OS_PAGES=2048 requests 2048 one-GiB pages, or 2 TiB. They are different controls. The guide explains allocator configuration, THP, service limits, permissions, and reboot persistence.

Repeatable measurements Copy Section

1. Record the server and benchmark revisions, build profile, kernel, VM type, CPU allocation, allocator, memory limits, filesystem, and provisioned disk/network capacity.
2. Keep workload shape, both durability policies, replication-group size, and topic options identical between compared runs. Use an explicit warmup and repeat runs.
3. Observe vmstat 1, iostat -xz 1, mpstat -P ALL 1, memory/I/O pressure, and network errors. Confirm the client has spare CPU and network capacity.
4. Run long enough to include segment flushes, checkpoints, and sustained device limits. Distinguish warm-cache tests, cold-cache tests, and storage throughput. Do not drop caches during a production workload.
5. Change one host setting at a time, retain its previous value, and compare throughput, p50/p99/p99.9 latency, memory, and errors. Record successful changes in provisioning and recheck after reboot.

The output subcommand records results and accepts context for the run:

iggy-bench --warmup-time 10s --total-data 10GiB balanced-producer tcp output --identifier baseline --remark replicated-defaults

Choose --total-data for the duration and storage behavior being tested. This example is not a universal sufficient data volume.

Overview (`core/binary_protocol/README.md`) Copy Section

iggy_binary_protocol

Wire protocol types and codec for the Apache Iggy binary protocol, shared between server and SDK. This crate is an internal building block. Most users should depend on the iggy SDK crate instead.

The language-neutral wire spec for the login protocol-version handshake (packed version layout, ClientVersionInfo prefix, gate semantics, rejection frame offsets) lives in the version module docs.

Links Copy Chapter

• Project website
• GitHub repository
• Iggy SDK on crates.io

License Copy Chapter

Licensed under the Apache License, Version 2.0. See LICENSE.

Overview (`core/cli/README.md`) Copy Section

Apache Iggy CLI

Interactive CLI for Apache Iggy. Manage streams, topics, partitions, users, consumer groups, and produce/consume messages from your terminal.

Installation Copy Chapter

cargo install iggy-cli

Run iggy --help for the full command tree.

Links Copy Chapter

• Project website
• GitHub repository
• Iggy SDK on crates.io

License Copy Chapter

Licensed under the Apache License, Version 2.0. See LICENSE.

Overview (`core/common/README.md`) Copy Section

iggy_common

Common types and traits shared across the Apache Iggy ecosystem (server, SDK, CLI, tooling). This crate is an internal building block. Most users should depend on the iggy SDK crate instead.

Links Copy Chapter

• Project website
• GitHub repository
• Iggy SDK on crates.io

License Copy Chapter

Licensed under the Apache License, Version 2.0. See LICENSE.

Overview (`core/connectors/README.md`) Copy Section

Apache Iggy Connectors

The highly performant and modular runtime for statically typed, yet dynamically loaded connectors. Ingest the data from the external sources and push it further to the Iggy streams, or fetch the data from the Iggy streams and push it further to the external sources. Create your own Rust plugins by simply implementing either the Source or Sink trait and build custom pipelines for the data processing.

The docker image is available, and can be fetched via docker pull apache/iggy-connect:edge.

Features Copy Chapter

• High Performance: Utilizes Rust's performance characteristics to ensure fast data ingestion and egress.
• Low memory footprint: Designed with memory efficiency in mind, minimizing the memory footprint of the connectors.
• Modular Design: Designed with modularity in mind, allowing for easy extension and customization.
• Dynamic Loading: Supports dynamic loading of plugins, enabling seamless integration with various data sources and sinks.
• Statically Typed: Ensures type safety and compile-time checks, reducing runtime errors.
• Easy Customization: Provides a simple interface for implementing custom connectors, making it easy to create new plugins.
• Data transformation: Supports data transformation with the help of existing functions.
• Powerful configuration: Define your sinks, sources, and transformations in the configuration file or fetch them from a remote HTTP API.
• Flexible configuration providers: Support for local file-based and HTTP-based configuration providers for centralized configuration management.
• Observability: Prometheus metrics with per-stage latency histograms, plus an opt-in per-batch tracing benchmark target.
• Structured logging: Selectable text or JSON log format via [logging] configuration.

Quick Start Copy Chapter

Run these commands from the root of the same Iggy source checkout used for the server and plugins. This guide targets server 0.9.0, including its edge builds.

1. Build the server, CLI, runtime and quick-start plugins:

   bash Copy

   cargo build --release -p server -p iggy-cli -p iggy-connectors \
     -p iggy_connector_random_source -p iggy_connector_stdout_sink \
     -p iggy_connector_quickwit_sink

   For a debug build, omit --release and replace target/release with target/debug in both the commands and plugin paths. Make sure that the plugins specified in core/connectors/runtime/example_config/connectors/ directory under path are available. The configuration must be provided in toml format.
   The example directory also enables connectors for ClickHouse, Delta Lake, Apache Doris, Apache Iceberg, and InfluxDB. Without their backing services (or their compiled plugins) these are reported with the Error status, but they don't block the remaining connectors. Set enabled = false in their files to skip them entirely.

2. Run docker compose -f examples/rust/src/sink-data-producer/docker-compose.yml up -d, which will start the Quickwit server to be used by an example sink connector. At this point, you can access the Quickwit UI at http://localhost:7280 - check this dashboard again later on, after the events index will be created.

3. In the terminal that will run the connectors, set the runtime configuration path:

   bash Copy

   export IGGY_CONNECTORS_CONFIG_PATH=core/connectors/runtime/example_config/config.toml

4. Start the Iggy server in a separate terminal with credentials matching the sample connector configuration:

   bash Copy

   IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy cargo run --bin iggy-server --release

   With the server running, create the example streams and topics using the CLI from this checkout. An existing server must have these credentials, or you must adjust the commands and connector configuration to match it.

   bash Copy

   target/release/iggy --username iggy --password iggy stream create example_stream
   target/release/iggy --username iggy --password iggy topic create example_stream example_topic 1 none 1d --durability persisted
   target/release/iggy --username iggy --password iggy stream create qw
   target/release/iggy --username iggy --password iggy topic create qw records 1 none 1d

5. Execute cargo run --example sink-data-producer --release, which sends 100 batches of messages to previously created qw stream and records topic (this will be used by the Quickwit sink connector).

6. Start the connector runtime cargo run --bin iggy-connectors --release in the terminal configured in step 3. The Quickwit sink indexes the produced records in the events index. At the same time, you should see the new messages being added to the example_stream stream and example_topic topic by the Random source connector - you can start the Iggy Web UI to browse the data. The messages will have applied the basic fields transformations.

Runtime Copy Chapter

All the connectors are implemented as Rust libraries and can be used as a part of the connector runtime. The runtime is responsible for managing the lifecycle of the connectors and providing the necessary infrastructure for the connectors to run. For more information, please refer to the runtime documentation.

Plugin path resolution Copy Chapter

The path field in connector configs points to the shared library (.so, .dylib, .dll). The runtime resolves it as follows:

1. Extension — if the path has no recognized extension, the OS-native one is appended automatically (.so on Linux, .dylib on macOS, .dll on Windows).

2. Absolute paths — used as-is.

3. Relative paths — searched in order, returning the first match:

   • the literal relative path (from working directory)
   • directory of the runtime binary (filename only)
   • current working directory (filename only)
   • /usr/lib, /usr/lib64, /lib, /lib64, /usr/local/lib, /usr/local/lib64

Examples:

# Relative — resolved against search dirs; extension appended on Linux
path = "target/release/libiggy_connector_stdout_sink"

# Absolute — used directly
path = "/opt/iggy/plugins/libiggy_connector_stdout_sink.so"

If the library is not found, the runtime logs all searched paths to help diagnose the issue.

Sink Copy Chapter

Sinks are responsible for consuming the messages from the configured stream(s) and topic(s) and sending them further to the specified destination. For example, the Quickwit sink connector is responsible for sending the messages to the Quickwit indexer.

Please refer to the Sink documentation for the details about the configuration and the sample implementation.

When implementing Sink, make sure to use the sink_connector! macro to expose the FFI interface and allow the connector runtime to register the sink with the runtime. The macro also exports the connector's version (from Cargo.toml) which is reported in the runtime's /stats endpoint.
Each sink should have its own, custom configuration, which is passed along with the unique plugin ID via expected new() method.

Available Sinks Copy Section

• Doris Sink - loads JSON messages into Apache Doris tables via the Stream Load HTTP API
• Elasticsearch Sink - sends messages to Elasticsearch indices
• Iceberg Sink - writes data to Apache Iceberg tables via REST catalog
• Meilisearch Sink - indexes messages in Meilisearch
• PostgreSQL Sink - stores messages in PostgreSQL database tables
• Quickwit Sink - indexes messages in Quickwit search engine
• RabbitMQ Sink - publishes messages to RabbitMQ exchanges via AMQP 0.9.1
• Reshift Sink - stores messages in Redshift warehouse tables via S3 as staging
• S3 Sink - writes messages to Amazon S3 and S3-compatible stores (MinIO, R2, B2, DO Spaces)
• Stdout Sink - prints messages to standard output (useful for debugging/development)
• SurrealDB Sink - writes messages into SurrealDB with deterministic record IDs for idempotent replay

Source Copy Chapter

Sources produce messages to an Iggy stream and topic. Configure one [[streams]] entry per source instance: the runtime retains only the last configured producer. For example, the Random source generates messages for that stream and topic.

Please refer to the Source documentation for the details about the configuration and the sample implementation.

Available Sources Copy Section

• Elasticsearch Source - polls documents from Elasticsearch indices
• PostgreSQL Source - reads rows from PostgreSQL tables with multiple consumption strategies (delete after read, mark as processed, timestamp tracking)
• Random Source - generates random test messages (useful for testing/development)

Building the connectors Copy Chapter

New connector can be built simply by implementing either Sink or Source trait. Please check the sink or source documentation, as well as the existing examples under core/connectors/sinks and core/connectors/sources.

Transformations Copy Chapter

Field transformations (depending on the supported payload formats) can be applied to the messages either before they are sent to the specified topic (e.g. when produced by the source connectors), or before consumed by the sink connectors. To add a transformation, implement the Transform trait in the SDK, add its TransformType variant and extend transforms::from_config. Each transform may have its own, custom configuration.

To find out more about the transforms, stream decoders or encoders, please refer to the SDK documentation.

Overview (`core/sdk/README.md`) Copy Section

Apache Iggy Rust SDK

Official Rust client SDK for Apache Iggy, the persistent message streaming platform written in Rust. The SDK ships a low-level transport client (QUIC, TCP, HTTP, WebSocket) for direct command access and a high-level producer/consumer API with batching, consumer groups, and auto-commit.

Features Copy Chapter

• Transports: TCP (custom binary), QUIC, HTTP, WebSocket. One unified IggyClient API across all four.
• TLS: TCP and WebSocket expose TLS connection-string options; QUIC always uses TLS; HTTP uses an HTTPS URL configured through the builder.
• Connection strings: iggy:// (TCP default), iggy+tcp://, iggy+quic://, iggy+http://, iggy+ws://. Binary transports apply credentials on connect(); HTTP requires an explicit login. Option keys and reconnection support differ by transport.
• Authentication: username/password and Personal Access Tokens (PAT).
• Async, non-blocking client built on Tokio with custom zero-copy (de)serialization.
• High-level builders on IggyClient: producer(stream, topic), consumer(name, stream, topic, partition), and consumer_group(name, stream, topic).
• Producer modes: direct (awaited send) and background (buffered with parallel shard workers using OrderedSharding or BalancedSharding). Configurable batch length / size and linger time.
• Partitioning: balanced, messages_key, or explicit partition_id. Custom Partitioner is pluggable.
• Consumer: standalone or consumer-group over binary transports; HTTP supports standalone consumers only. Consumed as an async Stream. Polling strategies: next, offset, timestamp, first, last.
• Auto-commit offset policies: Interval, When, After, IntervalOrWhen, IntervalOrAfter, or disabled.
• Stream builder (IggyStream, IggyStreamProducer, IggyStreamConsumer) for declarative producer + consumer setup on shared or separate stream/topic.
• Reliability: managed replica connections, heartbeat, ordered mutation lanes, explicit recovery of uncertain mutations, and offset auto-commit.
• Message features: optional headers (HeaderKey / HeaderValue), client-side AES-256-GCM encryption (via Aes256GcmEncryptor), topic compression metadata (None and Gzip; no runtime compression yet), server-honored message expiry, and server-side deduplication.
• Admin: stream/topic/partition CRUD, consumer-group management, server-side consumer offsets, system stats.

Installation Copy Chapter

Run from your application crate. Use a release compatible with your server; for
unreleased changes, build the SDK and server from the same source checkout.

This checkout requires matching servers and SDKs with fresh storage. Older data
directories are refused without modification; --fresh deletes existing data
in the configured directory. See the server recovery guide.
Mixed versions and rolling upgrades are unsupported. Binary clients share one
logical session across replica links using BindSession (15) and discover
partition primaries with GetPartitionRoute (105). Commands 14, 103, 104 and 123
are retired. HTTP clients use server-side forwarding.

cargo add iggy

All four transports are included; this crate declares no optional Cargo features.

The durable session API adds required session_identity and session_bind_secret
methods to VsrSessionControl. External transport implementations must provide
them, retain their logical identity and bind proof across transport reconnects,
and clear them on explicit logout. VsrSessionSealed is an implementation marker,
not a restriction on external implementations. The bind proof is a bearer
credential for internal session control and must never be logged or exposed.
refresh_session_password and refresh_session_username are removed. A
transport that signs in again applies a committed credential change where its
calls complete.

An established session has its own lifetime. Password or PAT changes and PAT
expiry do not end it. Logout, session lease expiry, or user deactivation end the
session and prevent its bind proof from restoring it.

Mutation recovery Copy Chapter

TCP, QUIC and WebSocket serialize mutations within each partition history and
within the metadata lane. Different lanes can progress concurrently. An
IggyError::OperationUncertain(operation_id) retains the exact request for
recovery. A later mutation on the same lane returns
IggyError::LaneBlocked(operation_id) until that operation is resolved or abandoned.

Call client.resolve_operation(operation_id, deadline).await, where deadline
is a std::time::Instant. A proved success returns Ok(()); a proved committed
business rejection returns its original error. Both release the lane. Another
OperationUncertain keeps the request available for a later recovery attempt.
The client keeps the proved outcomes of its last max_pending_operations
replays, also across a session reset. So a later resolve returns the same
outcome without a request, also after a producer or a consumer of the client
resolved the operation first.
client.abandon_operation(operation_id).await releases the lane while leaving
the outcome unknown. A session reset drops retained requests. An
OperationNotRetained error does not establish the outcome. A reset or an
abandon came before a proved outcome, or the proved outcome aged out.
Recovery is unavailable over HTTP.

A committed password change or rename of the signed-in user updates the
credentials that the client signs in with again. This holds for a typed call, a
raw request and a change that resolve_operation proves. A committed rejection,
such as a wrong current password, leaves them unchanged.

The client defaults to 256 active or retained mutations and 64 MiB of frozen
request frames. Configure positive limits through PendingOperationsConfig
on the binary transport builder, or through connection-string options:

iggy://iggy:iggy@localhost:8090?max_pending_operations=256&max_pending_operation_bytes=67108864

Admission waits within the call deadline before allocating a request ID or
sending. A frame larger than the byte limit fails locally. Completion,
abandonment and session reset release operation capacity; bytes remain charged
while a writer still holds the frame. Replaying a retained request reuses its
reservation.

Auto-commit and Next polls share their partition's mutation barrier while the
call is active. They retain their weak acknowledgement: cancellation or failover
can leave progress unknown. Error 57 or a lost reply after send ends the call
immediately without replay; errors 58/89 still retry known non-admission.
These polls are not retained for recovery.

Quick start Copy Chapter

Start a source server from the repository root in a separate terminal:

cargo run --bin iggy-server -- --fresh --with-default-root-credentials

Use disposable replica data with --fresh. Environment credentials override the
flag, and recovered credentials are not replaced. The sample expects iggy/iggy
and requires iggy, Tokio and futures-util in the application.

/* Detailed source-code truncated for AI context efficiency. */

For lower-level control over individual commands (login, stream/topic management, raw send, polling by offset or timestamp), use the transport-specific clients directly. See the examples and the Rust SDK docs.

For IggyConsumerConfig, partitions_count controls topic creation only. An ordinary consumer uses partition 0 unless the builder's partition_id or the config's with_partition_id selects another partition. Code that previously used partitions_count to select an existing partition must set partition_id explicitly. Consumer-group assignment ignores partition_id.

Versioning Copy Chapter

Stable releases follow semver (x.y.z). Edge releases (x.y.z-edge.N) are cut from master between stable versions and may include unreleased fixes; pin to a stable version for production.

Resources Copy Chapter

• Rust SDK docs
• High-level SDK guide
• Stream builder guide
• Project documentation
• Runnable examples: getting-started, basic, new-sdk, stream-builder, multi-tenant, message-envelope, message-headers, tcp-tls, sink-data-producer.
• Benchmarking platform
• GitHub repository
• Discord community

Related crates Copy Chapter

• iggy_common: shared types and traits.
• iggy_binary_protocol: wire protocol codec.
• iggy-cli: command-line tool, cargo install iggy-cli.

License Copy Chapter

Licensed under the Apache License, Version 2.0. See LICENSE.

Overview (`core/server/README.md`) Copy Section

Apache Iggy Server

The core server component of Apache Iggy: a persistent, append-only log for message streaming. It runs thread-per-core and shared-nothing on io_uring (through compio), and commits every write through Viewstamped Replication (VSR), so the same binary serves a standalone node and a multi-node cluster.

Clients connect over TCP (custom binary protocol), QUIC, WebSocket, or the HTTP REST API.

Running Copy Chapter

IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=... cargo run --bin iggy-server --release

The leader needs IGGY_ROOT_USERNAME and IGGY_ROOT_PASSWORD to create root. For local development, --with-default-root-credentials sets both to iggy. The leader records root in the metadata journal and replicates the same username and password hash to followers. Restarts recover root without these inputs. The server never generates a root password or writes either credential value to its output or log files.

Missing or empty storage is initialized automatically. Existing incompatible
storage is refused without modification. --fresh deletes existing data in
the configured system path and must be an explicit reset, never a restart
default. A new cluster starts automatically when its metadata WAL and
superblock are absent. A node with retained metadata rejoins through recovery.
Loss of a replica's entire metadata directory is outside the automatic recovery
guarantee, including loss on every replica. Empty metadata cannot distinguish a
new cluster from a previously used one. Restore a verified consistent backup
through a validated recovery procedure before restarting erased metadata disks.

The Docker image apache/iggy:latest ships the server together with the CLI; the edge tag tracks the latest development build.

The image binds every listener to 0.0.0.0 so the container is reachable from outside it. A wildcard bind says which interfaces accept connections, not where a client reaches the server, so the address to publish in cluster metadata has to be supplied and the server refuses to start without it. On a single host with published ports that address is localhost:

docker run -p 3000:3000 -p 8090:8090 \
  -e IGGY_NODE_ADVERTISED_ADDRESS=localhost \
  -e IGGY_ROOT_USERNAME=iggy -e IGGY_ROOT_PASSWORD=... apache/iggy:latest

Use the hostname or load balancer address clients actually dial when they are not on the same host. The Helm chart derives it from the Service DNS name.

To run one node of a cluster, pass its replica ID from the cluster.nodes roster:

cargo run --bin iggy-server --release -- --replica-id 0

Command-line arguments include --replica-id, --fresh (-f), --with-default-root-credentials, and --list-config-env-vars. Other settings come from configuration.

Upgrading Copy Chapter

Deploy matching server and SDK releases together. Rolling upgrades and mixed
server versions are unsupported. Partition control frames from older servers
carry incarnation zero and are rejected, so a mixed-version cluster cannot
complete partition view changes. The IGGY-BOUND-GRANTS-4 storage gate
requires fresh storage and rejects older data directories without modifying them.
Cached replies now preserve a committed-result marker, which older receipts lack.
No in-place upgrade or implicit migration is supported. A separately validated
offline migration or restore procedure would be required to reuse existing data.

The server no longer generates a root password when IGGY_ROOT_USERNAME and IGGY_ROOT_PASSWORD are unset. Supply both variables on every replica before the first cluster start so any promoted node can initialize root. Use --with-default-root-credentials only in development. A cluster node without credentials waits for root to be replicated. Once root is persisted, restarts need neither.

Releases up to 0.9 printed a generated password to stdout, and 0.9 also wrote it to the log file. Change any root password that was generated, and delete log files and container logs that still hold it. Snapshots redact that legacy log line.

All data directories from earlier storage formats must be wiped before upgrading. The storage-format check rejects them before metadata replay. There is no in-place upgrade or automatic migration.

After storage-format admission, a checkpoint above op 0 preserves initialized metadata. An empty checkpoint at op 0 does not prove that root exists. Without an initialized checkpoint, startup searches the current-format journal for root creation at op 1 because repair can write operations out of order. A journal with complete entries but no validated root creation is rejected before replay.

The error identifies the journal path and any unexpected op 1 fields. Restore a compatible data directory from this storage format, or use --fresh to delete local data. In multi-node clusters, use --fresh on at most one replica at a time while the others hold quorum.

The diagnostic snapshot (POST /snapshot, the binary GET_SNAPSHOT_FILE command, iggy snapshot) now requires the manage_servers permission. read_servers alone is rejected, because the archive carries raw server logs and host diagnostics.

All existing self-issued JWT access and refresh tokens are rejected after this
upgrade. HTTP clients and the web UI must log in again. In
[[http.jwt.trusted_issuers]], replace user_id with username, naming the
account the issuer may act as. The corresponding environment variable changes
from IGGY_HTTP_JWT_TRUSTED_ISSUERS_<n>_USER_ID to
IGGY_HTTP_JWT_TRUSTED_ISSUERS_<n>_USERNAME.

Cluster metadata includes replica_id. Java's ClusterNode record adds a leading
replicaId component, changing constructor calls and record patterns. C# callers
must initialize the required ReplicaId property. The CLI's cluster metadata list
appends a replica ID column. Swift's ClusterNode initializer now requires
replicaID. C++'s FFI ClusterNode, Rust's iggy_common::ClusterNode and
iggy_binary_protocol::responses::system::ClusterNodeResponse add the public
replica_id field; update aggregate and struct initializers.
Go, Java, Node.js and C# clients use command 105's new
partition routes and refresh on NotPrimary (89); use their matching releases.

Configuration Copy Chapter

Settings are read from config.toml, resolved relative to the working directory. Set IGGY_CONFIG_PATH to load a different file.

Any single value can be overridden with an IGGY_-prefixed environment variable that mirrors the TOML path:

IGGY_TCP_ADDRESS=127.0.0.1:8090 IGGY_HTTP_ENABLED=false cargo run --bin iggy-server

Cluster membership, quorum and replica addressing live under [cluster].

Protocol 0.11.1 uses client-owned registration proofs and BindSession (15) to
share a logical session across connections. Commands 14, 103, 104 and 123 are
retired. Partition routing uses 105, polling uses 100 and offset reads use 120. Disconnecting a
connection does not log out its logical session. Server-observed activity renews
the session lease, and expiry or explicit logout retires it before capacity is
reused.

External group offsets belong to groups managed outside Iggy, such as a Kafka
gateway. They require no Iggy group membership and can exceed the partition's
message-offset range.

The bind proof is an independent session credential. Password changes and PAT
revocation or expiry do not end an established session. Binding checks that the
owner still exists and is active; current permissions still govern each request.
Explicit logout, lease expiry and user deactivation end session access.

clients_table_max and dedup_clients_max nominate immutable limits when the
first operation commits in each metadata or partition group. Recovery and state
transfer preserve those committed limits. Configuration changes affect groups
that have not committed a limit yet; existing groups log a mismatch and retain
their committed capacity. At capacity, new sessions or writers are refused until
ordered retirement releases slots. Live retry protection is never evicted.

HTTP writes using one session and partition serialize through the previous
write's reply or bounded deadline. With ack=none, a request returns 202 after
its dispatch, while the next request to that partition waits for the previous
write to settle. Waiters acquire the in-flight permit after the partition gate,
so they do not consume the session's budget for other partitions.

Durability defaults remain Replicated, and ordinary SDK sends and explicit
offset writes support the configured policy. Crash-safe send retries require
Persisted; crash-safe explicit offset retries require Persisted and
Quorum. Weaker policies can lose data and receipts on a crash. A retained
receipt still replays its original result, but a lost receipt cannot prevent
another execution. NoAck and internal auto-commit polls retain their weaker
completion contracts.

Auto-commit polls are at-most-once: the owner can advance an offset before the
gateway's final authorization check. If the session ends or permissions are
revoked during that return hop, the response is denied and the batch can be lost.

Topics have an immutable partition_resize_policy creation option. The default,
mutable, permits partition addition and deletion without a bounded cutover or
per-key ordering across partition layouts. fixed rejects both operations with
PartitionResizeDisabled (2023). Existing messages stay in their original
partitions. Neither policy prevents retention or deletion of the topic.
Set the option through TopicCreateOptions::partition_resize_policy, HTTP's
options map, or the CLI's --set partition_resize_policy=fixed.

Upgrade recovery Copy Chapter

This release changes protocol and storage formats. Servers and compatible
clients deploy together; mixed versions and rolling upgrades are unsupported.
Peers verify protocol, release and storage-format identity before admission.
Executable packaging does not affect that identity, so stripping or rebuilding
the same compatible release does not by itself prevent a replica from joining.

This checkout uses development protocol 0.11.1. Finalize its version and minimum
before the next stable release. Released server-0.9.0 uses protocol 0.11.0 and
accepts every 0.11.x patch. An incompatible release therefore needs a new minor
protocol version, such as 0.12.0, for that released server to reject new clients
before decoding their login.

The protocol number is independent of server and SDK package versions. Its
current and minimum values are maintained manually for stable releases. The
client login gate checks the full inclusive range, including patch versions.
Both bounds stay equal unless compatibility with older released protocols is
explicitly supported. Edge builds require matching clients and servers, even
when their protocol numbers match.

An unsupported data directory is refused before WAL scanning or file changes.
Replacing the binary does not migrate old data. Restore or migrate retained data
only through a separately verified procedure; copying a format marker is not
a migration.

Partition recovery refuses a superblock whose nonzero log_view
is below the partition's committed created_view. It also refuses a WAL
certificate with a nonzero log_view below that floor. The log names the
superblock directory or the prepares-{revision} WAL directory and the
recorded views. The affected partition is tombstoned locally and is not served;
its durable files are preserved.

For a superblock with log_view zero, recovery raises view to at least
created_view without changing log_view. The raised view must be persisted
before any view-scoped send, just as when no superblock record exists.

Such state can have been written by older servers, including server-0.9.0,
after restarting before the partition's first superblock write. The creation
view was not restored on that path. Raising a nonzero log_view during an
upgrade would certify history that the replica may not hold, so the server
does not repair those records automatically.

Partition directories hold a created.revision file with the revision that
created the partition. Recovery deletes a directory whose file names an older
revision. The partition then starts empty, as if it had no directory, and in a
cluster the replica copies its data from the other replicas.

A directory that an older server made has no such file. A delete that did not
finish before a restart leaves such a directory on disk, and a later create
with the same stream, topic and partition ids finds it. Recovery then loads the
leftover directory as the new partition. Only a leftover superblock with a
nonzero log_view below created_view trips the superblock refusal. The WAL
refusal does not apply, because each partition has its own prepares-{revision}
WAL directory. Recovery logs adopting a nonempty partition directory without created.revision before adopting such a directory, since it cannot distinguish
a live legacy incarnation from a deleted one.

If recovery logs falls below committed creation view or the adoption warning
above, look at the history of the affected topic or partition first. If it was
deleted and created again with the same ids, delete it again, and then create
it again. The delete removes the leftover directory on every replica. It also
removes every message in that topic or partition, so do this only if you can
lose them.

In every other case:

1. Pause writes to the affected partition and stop the affected cluster before
   changing any recovery files. Keep the refusal logs.
2. Preserve a consistent copy of every replica's metadata, partition
   superblocks, WAL directories and segment files together with its configuration.
3. Restore only from a verified consistent backup, or use a separately validated
   recovery procedure based on retained replica history. Confirm that it preserves
   acknowledged writes and agrees with the committed partition creation metadata.

There is no automatic in-place migration for below-floor log certificates. Do not edit
view numbers or delete superblocks or WAL directories to bypass the refusal:
an empty history could then replace committed data during a view change.

Before a partition can serve, initialization publishes its incarnation in
partition-initialization/<namespace>/created.revision under the system data
directory. This atomic record remains outside the partition directory, including
after that directory is lost. Preserve it with metadata and partition backups.
Storage format IGGY-BOUND-GRANTS-4 requires this initialization contract;
older data directories are refused before mutation.

Retirement also writes retirement.fence in that external namespace directory
before counting a permanently failed partition as retired. The fence names the
failed incarnation and keeps it offline on subsequent boots, even if its
partition directory is replaced. Healthy partitions and new logins can continue.
If the fence cannot be persisted, the session remains retained and retirement
retries. Temporary teardown tombstones and state transfer do not qualify as
permanent failures.

Partition retirement barriers batch up to 128 ended sessions per commit. Each
session still requires retirement reports covering every allocated partition
from a common replica quorum before metadata releases its registry slot. Segment
deletion does not invalidate those reports; changes to partition incarnations
do. Metadata snapshot format 10 persists that separate revision.

A committed partition that has never initialized on this replica can finish
initialization after a crash, including when all replicas stopped before their
first WAL frontiers were published. An initialized partition with no durable
prepare-WAL frontier remains fenced as missing history. A singleton refuses to
serve it; a replicated partition requires history from a healthy peer. If every
replica lost initialized history, restarting does not clear the fence.

Preserve the files and restore a verified consistent backup or use a validated
recovery procedure. If the partition is independently known to be empty, or its
data may be discarded, delete and recreate it through the metadata API. This
creates a new partition incarnation and removes the old data. Missing files
alone are not evidence that discarding that data is safe. Do not create a WAL
frontier or remove initialization records or recovery fences manually.

Systemd integration Copy Chapter

Build with the systemd feature to enable readiness and watchdog notifications:

cargo build --bin iggy-server --release --features systemd

The server sends READY=1 only after every enabled transport is bound and accepting, so a unit ordered after it can dial as soon as it is notified. When the unit sets WatchdogSec=, the server pings WATCHDOG=1 at half that interval. On shutdown it sends STOPPING=1, which stops a long drain from counting against the watchdog.

Overview (`core/simulator/README.md`) Copy Section

Apache Iggy Simulator

A deterministic harness that runs an Iggy cluster inside a single thread. The replicas are the real shard::IggyShard, running the real message pump, VSR consensus, metadata state machine and partitions. Only the environment under them is swapped: clock, journal storage, superblock, bus and network are in-memory doubles, and every task runs on a cooperative executor whose poll order is drawn from the run's seed.

A run is a pure function of that seed. Failures print it, and --seed <value> replays the same task interleaving, the same dropped packets, the same crashes, down to the prepare timestamps recorded in the log. Three production consensus bugs have been found this way, each of them reproducible from one seed and a handful of flags.

The crate is publish = false: a tool to run against the workspace, not a dependency.

Quick start Copy Chapter

# 10k ticks of partition-plane traffic, 3 replicas, perfect network
cargo run -p simulator --bin workload-fuzz

# metadata plane against a hostile network with crashes and restarts
cargo run -p simulator --bin workload-fuzz -- \
  --seed 48 --ticks 4000 --plane metadata --faults heavy \
  --crash-prob 0.02 --restart-prob 0.08 --crash-primary

# the properties below, pinned as regression tests
cargo test -p simulator

# scripted demo: send, create and delete a stream, crash a follower, poll
cargo run -p simulator --bin simulator-ui

Nothing here needs Docker, a server binary, or a network stack. core/integration is where real processes are spawned.

Real code, simulated environment Copy Chapter

The simulator feature on shard, metadata, partitions and server_common exposes four seams that only this crate uses: init_partition (bypasses the reconciler), seed_single_partition (bypasses metadata consensus), hold_borrow_across_await (feeds the borrow detector a known-bad case), and a fixed-salt password hash so replicated user metadata replays. None has a production caller, and a -p iggy-server build excludes all four.

Determinism Copy Chapter

SimSeeds::derive(seed) splits the run's seed into one PRNG stream per consumer:

Children are drawn from a parent PRNG in field order, so fields are appended to SimSeeds, never inserted: an insertion moves every child after it and re-locks every seeded baseline.

Time never flows on its own. DetExecutor::advance_time is the only thing that moves it, sleeps anchor their deadline at creation (matching compio::time::sleep, which the pump's re-arm relies on), and timers fire in (deadline, seq) order so equal deadlines resolve FIFO. run_pumps gives run_until_stalled a 100k poll budget; exhausting it means a task is spin-waking, and the panic names the seed and the schedule hash.

One tick Copy Chapter

Simulator::step() returns the client replies delivered during it, in four phases:

1. Fire the consensus tick timer (CONSENSUS_TICK_INTERVAL: view change, retransmits) and run every pump to quiescence.
2. Deliver ready packets into the shard routers. The receiving shard is a seeded draw and so usually not the owner, which makes the frame take the real router hop, as production does when the coordinator homes a peer connection on an arbitrary shard. Under --shell a client packet enters through deliver_client_request instead of raw dispatch.
3. Run the pumps again over the delivered frames and their loopback follow-ups, then drain each replica's outbox into the network.
4. Advance network time.

Everything a step produces is on the wire before network time moves, so a reply cannot chain into another delivery inside the same tick.

The workload driver Copy Chapter

workload::run_with_faults is the loop the fuzzer and most tests share. Per tick it advances the workload clock, steps the fault injector, then resends timed-out requests and handshakes before sampling anything new, since a timed-out request still holds its client's slot. It then samples at most one request per idle client, steps the simulator, classifies the replies, recovers evicted clients, and asserts the per-tick invariants.

Each workload action has a module under workload/ops/, 23 of them, one per Action variant, exposing the same seven items: Input, Outcome, OUTCOMES, sample, build_message, classify_reply, predicted_effect. Dispatch runs through the op_dispatch! macro, so an action without a module is a compile error rather than a silently unsampled op. Variant order in Action is part of the determinism contract: append, never insert or reorder.

A resend reuses the original encoded message verbatim. Rebuilding it would draw a fresh request id, and the metadata client table dedups on that id: a renumbered retry commits a second time instead of returning the cached reply. Resends move to the next replica, so a client whose primary died finds the new one. Each client holds one request in flight (CLIENT_REQUEST_QUEUE_MAX).

What gets checked Copy Chapter

Agreement is asserted over the committed prefix, not over equal heads. A replica that missed the last commit broadcast, or rejoined a moment ago, is allowed to trail; what it may not do is hold different history at an op it did commit.

The oracle reports what it actually compared (ops_compared, replicas_compared, namespaces_checked) because a check that ran against an empty chain passes silently.

Using it from a test Copy Chapter

let seed = 0xC0FF_EE00;
let client_id: u128 = 1;
let network = PacketSimulatorOptions {
    node_count: 3,
    client_count: 1,
    seed,
    ..PacketSimulatorOptions::default()
};
let mut sim = Simulator::new(3, std::iter::once(client_id), network);
let client = SimClient::new(client_id);
let ns = IggyNamespace::new(1, 1, 0);
sim.init_partition(ns);
sim.register_client_with_primary(&client);

let mut options = WorkloadOptions::new(seed, 3, vec![ns]);
options.weights = ActionWeights::new(&[
    (Action::CreateStream, 50),
    (Action::DeleteStream, 25),
    (Action::SendMessages, 25),
]);
let mut wl = Workload::new(options);

let replies = workload::run(&mut sim, &mut wl, &[client], 2_000, u64::MAX);
assert!(replies > 0);
assert!(oracle::drive_to_quiesce(&mut sim, &mut wl, 5_000));
oracle::assert_converged(&sim, &mut wl);

MemoryPool::init_pool has to run first with pooling disabled, since PooledBuffer::from panics on an uninitialised pool.
For the dispatch path use Simulator::with_shards_shell, where shell_login and seed_stream_topic_partition replace register_client_with_primary. To assert what was injected rather than trusting the probabilities to have fired, pass a caller-owned FaultInjector to run_with_faults; replica_crash and replica_restart drive faults directly.

workload-fuzz Copy Chapter

workload-fuzz: seed=1 ticks=500 clients=1 replicas=3 plane=Partition faults=None shell=false crash_prob=0 quiesce=true
network: loss=0 replay=0 delay=1..3 partition=None/Symmetric p_partition=0 p_unpartition=0 stability=0/0 clog=0 clog_ticks=0 link_capacity=64
ran 500 ticks; 50 replies; crashes=0 restarts=0 still down: 0
coverage: replies_seen=50 replies_unknown=0 committed_rejections=0 samples_none=0 resends=0 denials=2 transients=0 evictions=0
  SendMessages: 28 commits, 0 denied (last status 0), 0 transient (last code 0)
  StoreConsumerOffset: 16 commits, 0 denied (last status 0), 0 transient (last code 0)
  DeleteConsumerOffset: 4 commits, 2 denied (last status 3021), 0 transient (last code 0)
quiesced and converged (leader-relative; entity oracle: held; evictions=0; ops_compared=1 replicas_compared=2 namespaces_checked=1)
coverage: replies_seen=51 replies_unknown=0 committed_rejections=0 samples_none=0 resends=0 denials=2 transients=0 evictions=0
commands delivered: Request=52 Prepare=80 PrepareOk=78 Reply=52 Commit=56 RequestPrepares=3 RepairPrepare=4 RepairDone=3
commands never delivered: Ping Pong PingClient PongClient StartViewChange DoViewChange StartView Eviction ...
workload-fuzz: OK (seed=1)

The two banner lines are the run's identity: every network parameter is printed, not the interesting subset, because under --faults swarm those values are the run. Coverage prints twice, before the quiesce assert and after the drain, so a failed drain still reports what the run managed to do. The command coverage lines answer "did this path get exercised?" by counting at delivery, so a command listed as delivered was seen on the wire rather than inferred from the source.

An invalid network configuration exits 2 before the simulator starts. Every other failure is a panic, and the hook prints reproduce with --seed N ahead of it.

Flags Copy Section

--plane picks the op mix: partition (writes and consumer offsets), metadata (replicated metadata mutations), mixed (stream creates over a write-heavy base), uniform (every action equally likely, the widest per-tick coverage).

--faults picks a whole network profile and the individual network flags override single fields of it.
Severity costs progress, every lost frame paying a resend timeout: on one namespace with one client, a 3-replica cluster drains roughly 440 replies in 5000 ticks on a perfect network, 240 under light, 40 under heavy.
Budget ticks accordingly instead of reading a low reply count as a stall.
--faults swarm derives every network parameter from the seed, which is what a campaign wants: none, light and heavy are three fixed points, so a thousand seeds against heavy is the same network a thousand times.

The gates exist because a run that commits nothing compares empty against empty and reports success. --min-commits and --min-ops-compared default to 1 for that reason; --require-entity-oracle additionally fails a run whose entity oracle was disarmed by an eviction and never re-armed.

--heal-before-quiesce restarts every crashed replica and stops the network drawing faults before the drain. It is off by default: a drain that fails with a replica still down is how a wedge shows up, and healing first resolves the wedge instead of reporting it.

Checkpoints Copy Section

A checkpoint is forced by the metadata WAL running low on slots, so --journal-slots N is what makes one happen. Without it the journal is unbounded and WAL drain, snapshot_op movement, RangeEvicted and metadata state transfer are all unreachable however hard the cluster is driven. N must exceed SnapshotCoordinator::CHECKPOINT_MARGIN, or every commit checkpoints and the run measures that rather than the workload.

Snapshots are written to a real directory, per process, whose path is printed. It refuses an existing --data-dir unless --reuse-data-dir says otherwise, because a restart now recovers from snapshot.bin and the directory would be input to the new run.

Reproducing and diagnosing Copy Chapter

RUST_BACKTRACE=1 RUST_LOG=shard=debug,consensus=debug \
  cargo run -p simulator --bin workload-fuzz -- --seed 13 --plane metadata \
  --crash-prob 0.01 --restart-prob 0.05 --ticks 30000

Omitting --seed draws one and prints it on the banner, so an exploratory run stays replayable. The panic hook replaces the default one, so RUST_BACKTRACE is honoured explicitly and only when set; a campaign would otherwise be buried in backtraces. RUST_LOG selects the server's own tracing, which is the only record of a request dropped after logging, the shape that wedges a client's in-flight slot.

A failed drain prints every outstanding request with its action, target and attempt count, then one line per replica carrying its commit range, repair barrier, transfer state, primary claim and armed repair session, which usually names the failure without a debugger:

replica 2: live | metadata ... commit=1407..1414 barrier=1359 transferring=false primary=false repair=..1407@0

Tests Copy Chapter

The suite lives at the end of src/lib.rs, with unit tests next to the modules they cover. Names follow given_X_when_Y_should_Z where there is a meaningful given; older behavioural names remain where there is not. Every test that touches replies initialises MemoryPool first, as above.

What the suite pins, by example: workload_replay_is_deterministic and multi_shard_replay_is_deterministic (the seed contract), committed_metadata_agrees_across_replicas, given_advanced_view_when_metadata_replica_restarts_should_recover_view_from_superblock, given_superblock_write_fails_when_primary_crashes_should_withhold_votes_and_not_elect, checkpointing_cluster_serves_a_chunked_state_transfer, shell_detects_partition_borrow_held_across_await.

Constructors: Simulator::new (one shard per replica), with_shards (metadata on shard 0, partitions hash-assigned across all shards), with_shards_shell (the same plus the real dispatch handlers), with_checkpoints. Multi-shard and shell-on-multi-shard are library-only; workload-fuzz always builds one shard per replica.

Not modelled Copy Chapter

• Client heartbeat verification and periodic consumer-group session expiry. Disconnect detaches a local binding and leaves the durable session live. The consumer-group tests invoke ordered cleanup explicitly; they do not cover the production timer or automatic liveness reporting.
• Storage faults, beyond two knobs on the superblock. SimSuperblock::set_fail_writes and set_yield_writes inject a persistent write fault and an fsync-wide suspension point; MemStorage under the journal never fails and never tears a write. Partition superblocks are storeless, which leaves partition view recovery untested.
• Segment files. Partition messages live in memory; they survive a restart only because the harness carries RetainedPartitionState across the rebuild.
• Physical partition durability. The consensus model uses persisted topic policies and retains the partition journal, committed frontier and retry receipts together across restarts. Storage is ideal: it cannot lose a write or reorder persistence barriers. WAL/checkpoint fault tests and real-server crash tests must prove those physical properties separately.
• Packet corruption. Packets are delayed, dropped, duplicated, partitioned and clogged, never mangled.
• Multi-shard metadata. Shard 0 owns the only metadata consensus group.
• I/O of any kind except the checkpoint files: no io_uring, no sockets, no wall clock.

Layout Copy Chapter

src/
├── lib.rs              Simulator: cluster construction, step(), crash/restart, partition seeding, tests
├── executor/           DetExecutor (seeded cooperative scheduler) + virtual clock
├── packet.rs           PacketSimulator: delay, loss, replay, partitions, clogs, link capacity
├── network.rs          Network: passthrough over PacketSimulator, submit/step/heal
├── bus.rs              SimOutbox: per-replica staging that consensus sends into
├── replica.rs          Replica type alias and new_shard() wiring, mirroring server bootstrap
├── deps.rs             SimClock, MemStorage, SimJournal, SimSuperblock
├── client.rs           SimClient: builds real wire requests
├── seeds.rs            SimSeeds: one PRNG stream per consumer
├── ready_queue.rs      min-heap with reservoir-sampled random ready removal
├── workload/           driver, op modules, shadow, auditor, invariants, state checker, oracle
└── bin/
    ├── workload-fuzz.rs    the fuzzer
    └── simulator-ui.rs     scripted demo run

CI Copy Chapter

A change under core/simulator/** marks the rust-simulator component in .github/config/components.yml, which scopes the Rust test tasks to it. The component is split out of rust-cluster so simulator-only changes do not trigger the foreign SDK suites. No fuzzing campaign runs in CI today; workload-fuzz is driven by hand, and a seed that fails should arrive as a test or as a finding write-up.

Commits touching this crate take the simulator scope: fix(simulator): ....

Overview (`examples/csharp/README.md`) Copy Section

Iggy Examples

This directory contains comprehensive sample applications that showcase various usage patterns of the Iggy client SDK. To learn more about building applications with Iggy, please refer to the getting started guide.

Running Examples Copy Chapter

Run from the Iggy repository root with .NET 10, using the server and SDK from the same checkout.
Initialize a new server data directory with the credentials used by the examples:

IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy cargo run --bin iggy-server

For an existing server, use its configured credentials. Run each producer before its consumer.

For server configuration options and help:

cargo run --bin iggy-server -- --help

You can also customize the server using environment variables:

# Enable HTTP transport and set its address
IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy IGGY_HTTP_ENABLED=true IGGY_HTTP_ADDRESS=127.0.0.1:3000 cargo run --bin iggy-server

Basic Examples Copy Chapter

Getting Started Copy Section

Perfect introduction for newcomers to Iggy:

dotnet run --project  examples/csharp/src/GettingStarted/Iggy_SDK.Examples.GettingStarted.Producer
dotnet run --project  examples/csharp/src/GettingStarted/Iggy_SDK.Examples.GettingStarted.Consumer

These examples use IIggyClient with TCP transport and demonstrate stream/topic creation with basic message handling.

Basic Usage Copy Section

Core functionality with detailed configuration options:

dotnet run --project  examples/csharp/src/Basic/Iggy_SDK.Examples.Basic.Producer
dotnet run --project  examples/csharp/src/Basic/Iggy_SDK.Examples.Basic.Consumer

Demonstrates fundamental client connection, authentication, batch message sending, and polling over TCP or HTTP.

Message Pattern Examples Copy Chapter

Message Headers Copy Section

Shows metadata management using custom headers:

dotnet run --project  examples/csharp/src/MessageHeaders/Iggy_SDK.Examples.MessageHeaders.Producer
dotnet run --project  examples/csharp/src/MessageHeaders/Iggy_SDK.Examples.MessageHeaders.Consumer

Demonstrates using HeaderKey/HeaderValue for message metadata instead of payload-based typing, with header-based message type dispatch in the consumer.

Message Envelopes Copy Section

JSON envelope pattern for polymorphic message handling:

dotnet run --project  examples/csharp/src/MessageEnvelope/Iggy_SDK.Examples.MessageEnvelope.Producer
dotnet run --project  examples/csharp/src/MessageEnvelope/Iggy_SDK.Examples.MessageEnvelope.Consumer

Uses MessagesGenerator to create OrderCreated, OrderConfirmed, and OrderRejected messages wrapped in JSON envelopes for type identification.

Security Examples Copy Chapter

TCP/TLS Copy Section

Demonstrates secure TLS-encrypted TCP connections:

dotnet run --project  examples/csharp/src/TcpTls/Iggy_SDK.Examples.TcpTls.Producer
dotnet run --project  examples/csharp/src/TcpTls/Iggy_SDK.Examples.TcpTls.Consumer

Uses IggyClientConfigurator with TlsSettings (Enabled, Hostname, CertificatePath) to establish TLS-encrypted TCP connections with CA certificate verification. Run from the repository root so the CA path resolves. Start the server with the example certificate:

IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy \
IGGY_TCP_TLS_ENABLED=true \
IGGY_TCP_TLS_CERT_FILE=core/certs/iggy_cert.pem \
IGGY_TCP_TLS_KEY_FILE=core/certs/iggy_key.pem \
cargo run --bin iggy-server

Example Structure Copy Chapter

All examples can be executed directly from the repository. Follow these steps:

1. Start the matching server with the credentials shown above
2. Run the producer and consumer using the concrete commands above
3. Check source code

These examples use IggyClient with TCP transport and demonstrate automatic stream/topic creation with basic message handling.

The examples are automatically tested via scripts/run-examples-from-readme.sh --language csharp to ensure they remain functional and up-to-date with the latest API changes.

Overview (`examples/go/README.md`) Copy Section

Iggy Examples

This directory contains a getting-started producer and consumer using the Go SDK over TCP, with optional TLS. To learn more about building applications with Iggy, please refer to the getting started guide.

Running Examples Copy Chapter

Use Go 1.25 or newer. The local go.mod replaces the SDK dependency with ../../foreign/go, so build the server from the same checkout. Start it from the repository root, then run the Go commands from examples/go.

For disposable development data and no root credential environment overrides:

cargo run --bin iggy-server -- --fresh --with-default-root-credentials

--fresh wipes this replica's local data directory. IGGY_ROOT_USERNAME and IGGY_ROOT_PASSWORD take precedence over the flag. Bootstrap settings do not replace recovered credentials, including credentials recovered from cluster peers. The examples use iggy/iggy and numeric stream/topic IDs 0; run the producer first, before creating any other streams or topics.

For server configuration options and help:

cargo run --bin iggy-server -- --help

You can also customize the server using environment variables:

## Example: Enable HTTP transport and set custom address
IGGY_HTTP_ENABLED=true IGGY_HTTP_ADDRESS=127.0.0.1:3000 cargo run --bin iggy-server -- --with-default-root-credentials

Each producer sends 5 batches of 10 messages to partition 0. Each consumer reads that partition independently and exits after 5 batches; multiple consumers read the same messages and do not join a consumer group.

Basic Examples Copy Chapter

Getting Started Copy Section

Perfect introduction for newcomers to Iggy:

go run ./getting-started/producer/main.go
go run ./getting-started/consumer/main.go

TLS Examples Copy Chapter

From the repository root, start a disposable TLS server with the development certificate pair and the same credential prerequisites:

IGGY_TCP_TLS_ENABLED=true \
IGGY_TCP_TLS_CERT_FILE=core/certs/iggy_cert.pem \
IGGY_TCP_TLS_KEY_FILE=core/certs/iggy_key.pem \
cargo run --bin iggy-server -- --fresh --with-default-root-credentials

These certificates are for development only. From examples/go, run:

go run ./getting-started/producer/main.go --tcp-server-address localhost:8090 --tls --tls-ca-file ../../core/certs/iggy_ca_cert.pem
go run ./getting-started/consumer/main.go --tcp-server-address localhost:8090 --tls --tls-ca-file ../../core/certs/iggy_ca_cert.pem

Example Structure Copy Chapter

All examples can be executed directly from the repository. Follow these steps:

1. Start the Iggy server: the Go SDK speaks the VSR wire protocol, so the
   examples need a VSR server.
   Use the source-server command and prerequisites above.
2. Run the producer, then the consumer: use the commands above from examples/go.
3. Check source code: Examples include detailed comments explaining concepts and usage patterns

These examples use IggyClient with TCP transport and demonstrate automatic stream/topic creation with basic message handling.

The examples are automatically tested via scripts/run-examples-from-readme.sh --language go to ensure they remain functional and up-to-date with the latest API changes.

Overview (`examples/java/README.md`) Copy Section

Iggy Java Examples

This directory contains comprehensive sample applications that showcase various usage patterns of the Iggy java client SDK, from basic operations to advanced multi-tenant scenarios.

Java 17 or newer is required. The included Gradle wrapper downloads the pinned Gradle version. The project uses includeBuild to compile the SDK from foreign/java in the same checkout.

Running Examples Copy Chapter

These examples target server 0.9.0 and speak the VSR (Viewstamped Replication) wire protocol. Build the server and SDK from the same checkout for unreleased changes. Run the Gradle tasks below from examples/java, with each producer before its consumer.

Iggy requires valid credentials to authenticate client requests. The examples assume that the server is using the default root credentials, set through environment variables before starting the server:

Linux, from the repository root:

export IGGY_ROOT_USERNAME=iggy
export IGGY_ROOT_PASSWORD=iggy
cargo run --bin iggy-server -- --fresh --with-default-root-credentials

PowerShell environment variables:

$env:IGGY_ROOT_USERNAME = "iggy"
$env:IGGY_ROOT_PASSWORD = "iggy"

Note
This setup is intended only for development and testing, not production use.

By default, all server data is stored in the local_data directory (this can be changed via IGGY_PATH).

--fresh wipes this replica's local data directory. Environment credentials take precedence over --with-default-root-credentials. Bootstrap settings do not replace recovered credentials, and a fresh cluster replica can recover them from peers. On an existing server, use the credentials that were configured for it.

You can also customize the server using environment variables:

## Example: set a custom TCP address
IGGY_TCP_ADDRESS=127.0.0.1:8090 cargo run --bin iggy-server -- --with-default-root-credentials

Basic Examples Copy Chapter

Getting Started Copy Section

A good introduction for newcomers to Iggy:

./gradlew runGettingStartedProducer
./gradlew runGettingStartedConsumer

Message Headers Copy Section

Shows metadata management using custom headers:

./gradlew runMessageHeadersProducer
./gradlew runMessageHeadersConsumer

Uses a message_type header to choose the application handler for each order event.

Message Envelopes Copy Section

JSON envelope pattern for polymorphic message handling:

./gradlew runMessageEnvelopeProducer
./gradlew runMessageEnvelopeConsumer

Uses MessagesGenerator to create OrderCreated, OrderConfirmed, and OrderRejected messages wrapped in JSON envelopes for type identification.

Advanced Examples Copy Chapter

Multi-Tenant Architecture Copy Section

Complex example demonstrating enterprise-level isolation:

./gradlew runMultiTenantProducer
./gradlew runMultiTenantConsumer

Features multiple tenant setup, user creation with stream-specific permissions, concurrent producers/consumers across tenants, and security isolation.

High-Volume Data Generation Copy Section

Testing and benchmarking support:

./gradlew runSinkDataProducer

Produces 100 batches of 1000 to 1099 messages with generated user records.

Stream Builder Examples Copy Chapter

Stream Builder Copy Section

Producing and consuming messages in one class:

./gradlew runStreamBasic

Uses the blocking client to create a stream and topic, send three messages, and poll them. It deletes its test_stream stream after the run.

Async Client Examples Copy Chapter

Async Producer Copy Section

Non-blocking batch production with concurrent request submission:

./gradlew runAsyncProducer

Shows:

• CompletableFuture chaining patterns
• Submitting multiple sends without blocking

Async Consumer Copy Section

Non-blocking async consumption with advanced patterns:

./gradlew runAsyncConsumer

Shows:

• Backpressure management (don't poll faster than you can process)
• Error recovery with exponential backoff
• Thread pool separation (Netty I/O threads vs. processing threads)
• Offset-based polling with CompletableFuture

CRITICAL ASYNC PATTERN - Thread Pool Management:

The async client uses Netty's event loop threads for I/O operations. NEVER block these threads with:

• .join() or .get() inside thenApply/thenAccept
• Thread.sleep()
• Blocking database calls
• Long-running computations

If your message processing involves blocking operations, offload to a separate thread pool using thenApplyAsync(fn, executor).

Security Examples Copy Chapter

TCP/TLS Copy Section

Demonstrates secure TLS-encrypted TCP connections:

./gradlew runTcpTlsProducer
./gradlew runTcpTlsConsumer

These examples require a TLS-enabled Iggy server. From the repository root, start a disposable server with the development certificates:

IGGY_TCP_TLS_ENABLED=true \
IGGY_TCP_TLS_CERT_FILE=core/certs/iggy_cert.pem \
IGGY_TCP_TLS_KEY_FILE=core/certs/iggy_key.pem \
cargo run --bin iggy-server -- --fresh --with-default-root-credentials

Uses IggyTcpClientBuilder.enableTls() and tlsCertificate("../../core/certs/iggy_ca_cert.pem") with CA verification. Run the clients from examples/java so that path resolves. The same data and credential prerequisites apply; these certificates are for development only.

Blocking vs. Async - When to Use Each Copy Chapter

The Iggy Java SDK provides two client types: blocking (synchronous) and async (non-blocking). Choose based on your use case:

Use Blocking Client When Copy Section

• Writing scripts, CLI tools, or simple applications
• Sequential code is easier to reason about
• Integration tests

Use Async Client When Copy Section

• Need high throughput
• Application is already async/reactive (Spring WebFlux, Vert.x)
• Want to compose non-blocking requests with CompletableFuture
• Building services that handle many concurrent streams

Key Async Patterns Copy Chapter

CompletableFuture Chaining Copy Section

client.connect()
    .thenCompose(v -> client.login())
    .thenCompose(identity -> client.streams().createStream("my-stream"))
    .thenAccept(stream -> System.out.println("Created: " + stream.name()))
    .exceptionally(ex -> {
        System.err.println("Error: " + ex.getMessage());
        return null;
    });

Submitting Multiple Sends Copy Section

List<CompletableFuture<SendMessagesResponse>> sends = new ArrayList<>();
for (int i = 0; i < 10; i++) {
    sends.add(client.messages().sendMessages(...));
}
CompletableFuture.allOf(sends.toArray(new CompletableFuture[0])).join();

The client accepts these calls without blocking, but its single VSR-pinned TCP
connection processes them in order. Batch more messages into each send to improve
throughput.

Thread Pool Offloading Copy Section

// WRONG - blocks Netty event loop
client.messages().pollMessages(...)
    .thenAccept(polled -> {
        saveToDatabase(polled);  // blocking I/O!
    });

// CORRECT - offloads to processing pool
var processingPool = Executors.newFixedThreadPool(8);
client.messages().pollMessages(...)
    .thenAcceptAsync(polled -> {
        saveToDatabase(polled);  // runs on processingPool
    }, processingPool);

Overview (`examples/node/README.md`) Copy Section

Iggy Examples

This directory contains comprehensive sample applications that showcase various usage patterns of the Iggy client SDK for Node.js, from basic operations to advanced scenarios. To learn more about building applications with Iggy, please refer to the getting started guide.

Running Examples Copy Chapter

These examples target server 0.9.0 and use the SDK from the same checkout. Start the server from the repository root in a separate terminal:

cargo run --bin iggy-server -- --fresh --with-default-root-credentials

--fresh deletes existing local server data. Explicit credential environment variables override the default-root flag, and bootstrap settings do not replace recovered credentials. See the repository development prerequisites.

For server configuration options and help, from the repository root:

cargo run --bin iggy-server -- --help

You can also customize the server using environment variables:

# Enable HTTP transport and set the TCP address
IGGY_HTTP_ENABLED=true IGGY_TCP_ADDRESS=127.0.0.1:8090 \
  cargo run --bin iggy-server -- --fresh --with-default-root-credentials

From the repository root, build the local SDK before installing the examples:

npm --prefix foreign/node ci
npm --prefix foreign/node run build
npm --prefix examples/node ci
cd examples/node

Run the following scripts from examples/node. Set DEBUG=iggy:examples* to see progress. The separate producer and consumer scripts do not consistently share resources: envelope/header consumers create empty topics, and the tenant producer removes its streams before exiting. Getting-started/basic producers spread batches over five partitions while their consumers poll partition 0. The stream-builder example below is a self-contained send-and-consume walkthrough.

Basic Examples Copy Chapter

Getting Started Copy Section

Producer and consumer scripts:

npm run test:getting-started:producer
npm run test:getting-started:consumer

Basic Usage Copy Section

Core functionality with detailed configuration options:

npm run test:basic:producer
npm run test:basic:consumer

Demonstrates fundamental client connection, authentication, batch message sending, and polling with support for TCP transport.

Message Envelope Copy Section

Working with message envelopes:

npm run test:message-envelope:producer
npm run test:message-envelope:consumer

Demonstrates a JSON envelope containing message_type and a serialized payload.

Message Headers Copy Section

Dispatching by a message type stored in the payload:

npm run test:message-headers:producer
npm run test:message-headers:consumer

This example stores messageType and data in a JSON payload wrapper.

Multi-Tenant Copy Section

Multi-tenant application patterns:

npm run test:multi-tenant:producer
npm run test:multi-tenant:consumer

Each script uses one client with the supplied credentials to access separate tenant streams. Consumers poll with Consumer.Single.

Stream Builder Copy Section

Creating a stream and topic, then sending and consuming messages:

npm run test:stream-builder

Uses the standard client API to send and consume three messages, then deletes the resources it created.

Sink Data Producer Copy Section

Generating messages for sink connectors:

npm run test:sink-data-producer

Produces 100 batches of 100-499 JSON records. On completion it deletes the selected topic and stream, including resources that already existed.

TLS Examples Copy Chapter

TCP/TLS Copy Section

Producer and consumer examples using TLS-encrypted TCP connections with custom CA certificates:

npm run test:tcp-tls:producer
npm run test:tcp-tls:consumer

These examples require a TLS-enabled Iggy server. From the repository root:

IGGY_TCP_TLS_ENABLED=true \
IGGY_TCP_TLS_CERT_FILE=core/certs/iggy_cert.pem \
IGGY_TCP_TLS_KEY_FILE=core/certs/iggy_key.pem \
cargo run --bin iggy-server -- --fresh --with-default-root-credentials

The clients use transport: 'TLS', connect to localhost, and load ../../core/certs/iggy_ca_cert.pem relative to examples/node. These are repository development certificates.

Local checks Copy Chapter

npm run test:unit
npm run lint

Overview (`examples/php/README.md`) Copy Section

Iggy PHP Examples

This directory contains sample applications that show how to use the Apache
Iggy PHP SDK extension.

Running Examples Copy Chapter

Start the server from the repository root:

cargo run --bin iggy-server -- --fresh --with-default-root-credentials

From examples/php, build the PHP extension and point PHP at it:

(cd ../../foreign/php && cargo build)
export PHP_IGGY_EXTENSION="$(pwd)/../../foreign/php/target/debug/libiggy_php.so"

On macOS, use ../../foreign/php/target/debug/libiggy_php.dylib instead.

Getting Started Copy Section

php -d extension="${PHP_IGGY_EXTENSION:-../../foreign/php/target/debug/libiggy_php.so}" getting-started/producer.php
php -d extension="${PHP_IGGY_EXTENSION:-../../foreign/php/target/debug/libiggy_php.so}" getting-started/consumer.php

Basic Usage Copy Section

php -d extension="${PHP_IGGY_EXTENSION:-../../foreign/php/target/debug/libiggy_php.so}" basic/producer.php
php -d extension="${PHP_IGGY_EXTENSION:-../../foreign/php/target/debug/libiggy_php.so}" basic/consumer.php

The examples use IGGY_CONNECTION_STRING when it is set; use a binary transport
with credentials for automatic login. Otherwise they connect over TCP using
IGGY_HOST and IGGY_PORT, then log in with IGGY_USERNAME and IGGY_PASSWORD
without URL encoding. Defaults are 127.0.0.1:8090 and iggy/iggy.

Overview (`examples/python/README.md`) Copy Section

Iggy Examples

This directory contains Python SDK examples for connection configuration, sending and polling messages, user headers, and TLS. To learn more about building applications with Iggy, please refer to the getting started guide.

Running Examples Copy Chapter

These examples target server 0.9.0. For unreleased changes, build the SDK and
server from the same source checkout. Start the server in a separate terminal,
from the repository root:

# Server 0.9.0
docker run --rm \
  --cap-add=SYS_NICE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
  -p 8090:8090 \
  -e IGGY_TCP_ADDRESS=0.0.0.0:8090 \
  -e IGGY_NODE_ADVERTISED_ADDRESS=localhost \
  -e IGGY_ROOT_USERNAME=iggy -e IGGY_ROOT_PASSWORD=iggy \
  apache/iggy:0.9.0

# Or build from source
cargo run --bin iggy-server -- --with-default-root-credentials --fresh

The container variables expose the TCP listener and bootstrap iggy/iggy for
new data. Stored credentials are not replaced, and environment credentials take
precedence over the source command's default-credentials flag. Use --fresh
only with disposable local replica data.

For server configuration options and help:

cargo run --bin iggy-server -- --help

You can also customize the server using environment variables:

# Enable HTTP transport and set its address
IGGY_HTTP_ENABLED=true IGGY_HTTP_ADDRESS=127.0.0.1:3000 cargo run --bin iggy-server -- --with-default-root-credentials

With Python 3.11 or newer and Rust/Cargo available, install dependencies from
examples/python. uv selects the local SDK path in pyproject.toml; pip needs
that path explicitly:

# Using uv
uv sync

# Using pip with the dependencies declared in pyproject.toml
python -m venv .venv
source .venv/bin/activate
pip install ../../foreign/python .

Basic Examples Copy Chapter

High-Level Producer and Consumer Copy Section

The Python high-level producer API is a port of the Rust high-level producer
API. For detailed producer behavior and configuration, see the
Rust high-level SDK documentation.

The high-level producer binds the destination once, initializes missing
resources, applies producer-level batching and retry settings, and shuts down
deterministically through an async context manager. The high-level consumer
joins a consumer group, polls all assigned partitions, invokes an async handler,
and commits each message after it has been handled.

Run either producer first. Both create the stream and topic and publish 12
messages for the consumer, which exits after receiving all of them. producer.py
uses direct mode and waits for server confirmations; background_producer.py
uses bounded background workers and flushes them on context-manager exit:

# Using uv
uv run high-level/producer.py
uv run high-level/consumer.py

# Or use the background producer before starting the same consumer
uv run high-level/background_producer.py
uv run high-level/consumer.py

# Without using uv
python high-level/producer.py
python high-level/consumer.py

# Or use the background producer before starting the same consumer
python high-level/background_producer.py
python high-level/consumer.py

The existing examples below use the low-level IggyClient.send_messages() API
and remain useful when each call needs to specify its own destination.

Getting Started Copy Section

Perfect introduction for newcomers to Iggy:

# Using uv
uv run getting-started/producer.py
uv run getting-started/consumer.py

# Without using uv
python getting-started/producer.py
python getting-started/consumer.py

Basic Usage Copy Section

Core functionality with detailed configuration options:

# Using uv
uv run basic/producer.py
uv run basic/consumer.py

# Without using uv
python basic/producer.py
python basic/consumer.py

Demonstrates client connection, authentication, batch message sending, and polling
over TCP, QUIC, or WebSocket. HTTP requires an explicit login call; its
connection-string credentials are not applied automatically.

Message Headers Copy Section

Shows how to attach and read Python SDK user headers with str, bytes, bool, int, and float values. Two variants share their logic through message-headers/common.py:

• plain-headers/ uses the convenient dict[str, str | bytes | bool | int | float] form; the SDK infers a wire type for each value.
• typed-headers/ uses explicit HeaderKey/HeaderValue for full control over the wire type.

Both producers store typed headers on the wire. The plain consumer converts them to Python scalars, while the typed consumer preserves and inspects the explicit header kinds.

# Using uv
uv run message-headers/plain-headers/producer.py
uv run message-headers/plain-headers/consumer.py
uv run message-headers/typed-headers/producer.py
uv run message-headers/typed-headers/consumer.py

# Without using uv
python message-headers/plain-headers/producer.py
python message-headers/plain-headers/consumer.py
python message-headers/typed-headers/producer.py
python message-headers/typed-headers/consumer.py

TLS Examples Copy Chapter

To test with a TLS-enabled server, start the server with TLS configured (see main README), then run:

uv run getting-started/producer.py --tcp-server-address localhost:8090 --tls --tls-ca-file ../../core/certs/iggy_ca_cert.pem
uv run getting-started/consumer.py --tcp-server-address localhost:8090 --tls --tls-ca-file ../../core/certs/iggy_ca_cert.pem

Overview (`examples/rust/README.md`) Copy Section

Iggy Examples

This directory contains comprehensive sample applications that showcase various usage patterns of the Iggy client SDK, from basic operations to advanced multi-tenant scenarios. To learn more about building applications with Iggy, please refer to the getting started guide.

Running Examples Copy Chapter

Run all commands from the repository root, with the server in a separate terminal.
For unreleased changes, use the SDK and server from the same checkout.

Iggy requires valid credentials to authenticate client requests. The examples assume that the server is using the default root credentials, which can be enabled in one of two ways:

1. Start the server with default credentials:

   bash Copy

   cargo run --bin iggy-server -- --with-default-root-credentials

2. Set the appropriate environment variables before starting the server with cargo run --bin iggy-server:

   macOS/Linux:

   bash Copy

   export IGGY_ROOT_USERNAME=iggy
   export IGGY_ROOT_PASSWORD=iggy

   Windows(Powershell):

   bash Copy

   $env:IGGY_ROOT_USERNAME = "iggy"
   $env:IGGY_ROOT_PASSWORD = "iggy"

Note
This setup is intended only for development and testing, not production use.

By default, all server data is stored in the local_data directory (this can be changed via system.path in config.toml).

Root credentials bootstrap a new state. Environment credentials override the default-credential flag; bootstrap settings do not replace recovered credentials.
Once the server has created and populated the data directory, the existing stored credentials will always be used, and supplying the --with-default-root-credentials flag or setting the environment variables will no longer override them.

If the server has already been started once and your example returns Error: InvalidCredentials, then this means the stored credentials differ from the defaults.

You can reset the credentials in one of two ways:

1. Delete the existing data directory, then start the server again with the default-credential flag or environment variables.

2. Use the --fresh flag to force a reset:

   bash Copy

   cargo run --bin iggy-server -- --with-default-root-credentials --fresh

   This deletes this replica's local data and re-initializes it. Use disposable development data. In a cluster, a fresh replica can recover credentials from peers; a new cluster requires explicit root credentials.

For server configuration options and help:

cargo run --bin iggy-server -- --help

You can also customize the server using environment variables:

## Example: Enable HTTP transport and set custom address
IGGY_HTTP_ENABLED=true IGGY_HTTP_ADDRESS=127.0.0.1:3000 cargo run --bin iggy-server -- --with-default-root-credentials

You can run multiple producers and consumers simultaneously to observe how messages are distributed across clients. Most examples support configurable options via the Args struct, including transport protocol, stream/topic/partition settings, consumer ID, message size, and more.

Basic Examples Copy Chapter

Getting Started Copy Section

Perfect introduction for newcomers to Iggy:

cargo run --example getting-started-producer
cargo run --example getting-started-consumer

These examples use IggyClientBuilder with TCP transport and demonstrate stream/topic creation with basic message handling. Run the producer before the consumer on a fresh server: the consumer expects stream and topic IDs 0.

Basic Usage Copy Section

Core functionality with detailed configuration options:

cargo run --example basic-producer
cargo run --example basic-consumer

Demonstrates fundamental client connection, authentication, batch message sending, and polling with support for TCP/QUIC/HTTP/WebSocket protocols.

To run the pair over HTTP:

cargo run --example basic-producer -- \
  --transport http
cargo run --example basic-consumer -- \
  --transport http

Message Pattern Examples Copy Chapter

Message Headers Copy Section

Shows metadata management using custom headers:

cargo run --example message-headers-type-producer
cargo run --example message-headers-type-consumer

Demonstrates using HeaderKey/HeaderValue for message metadata instead of payload-based typing, with header-based message routing.

Shows manual message compression using headers. This is the current workaround while built-in compression is unsupported:

cargo run --example message-headers-compression-producer
cargo run --example message-headers-compression-consumer

Demonstrates typed header keys and values with various data types (strings, integers, floats, booleans, raw bytes):

cargo run --example typed-headers-producer
cargo run --example typed-headers-consumer

Message Envelopes Copy Section

JSON envelope pattern for polymorphic message handling:

cargo run --example message-envelope-producer
cargo run --example message-envelope-consumer

Uses MessagesGenerator to create OrderCreated, OrderConfirmed, and OrderRejected messages wrapped in JSON envelopes for type identification.

Advanced Examples Copy Chapter

Multi-Tenant Architecture Copy Section

Complex example demonstrating enterprise-level isolation:

cargo run --example multi-tenant-producer
cargo run --example multi-tenant-consumer

Features multiple tenant setup, user creation with stream-specific permissions, concurrent producers/consumers across tenants, and security isolation. Configurable via environment variables (TENANTS_COUNT, PRODUCERS_COUNT, etc.).

New SDK API Copy Section

Modern, ergonomic SDK usage patterns:

cargo run --example new-sdk-producer
cargo run --example new-sdk-consumer

Showcases the newer SDK APIs with simplified setup, automatic topic creation, consumer groups, and AutoCommit configuration.

High-Volume Data Generation Copy Section

Testing and benchmarking support:

cargo run --example sink-data-producer

Produces 100 batches of 100 to 499 random user records, with a direct-send request limit of 1000 messages. Connection and stream settings are configurable via environment variables.

Stream Builder Examples Copy Chapter

Basic Stream Management Copy Section

IggyStream abstraction for producer/consumer pairs:

cargo run --example stream-basic
cargo run --example stream-producer
cargo run --example stream-consumer

Advanced Configuration Copy Section

Comprehensive configuration examples with detailed documentation:

cargo run --example stream-producer-config
cargo run --example stream-consumer-config

These examples document all available configuration options including partitioning strategies, retry policies, batching, AutoCommit strategies, polling strategies, and retry mechanisms.

Security Examples Copy Chapter

TCP/TLS Copy Section

Demonstrates secure TLS-encrypted TCP connections using custom CA certificates:

cargo run --example tcp-tls-producer
cargo run --example tcp-tls-consumer

These examples require a TLS-enabled Iggy server. Start the server with:

IGGY_TCP_TLS_ENABLED=true \
IGGY_TCP_TLS_CERT_FILE=core/certs/iggy_cert.pem \
IGGY_TCP_TLS_KEY_FILE=core/certs/iggy_key.pem \
cargo run --bin iggy-server -- --fresh --with-default-root-credentials

Use the test certificates only for local development. The --fresh data-reset and credential qualifications above apply.

Uses IggyClientBuilder with TLS options (with_tls_enabled, with_tls_domain, with_tls_ca_file) to establish TLS-encrypted TCP connections with CA certificate verification.

Example Structure Copy Chapter

All examples can be executed directly from the repository. Follow these steps:

1. Start the Iggy server using the credential setup above
2. Run desired example: cargo run --example EXAMPLE_NAME
3. Check source code: Examples include detailed comments explaining concepts and usage patterns

Most examples use shared utilities from examples/rust/src/shared/ including:

• Message type definitions (orders, events)
• Message generation utilities
• Common argument parsing
• Client setup helpers

The examples are automatically tested via scripts/run-examples-from-readme.sh --language rust to ensure they remain functional and up-to-date with the latest API changes.

Overview (`foreign/cpp/README.md`) Copy Section

Iggy C++ Client

C++ client for Apache Iggy message streaming.

Bazel uses the Cargo and Rust toolchain provisioned by rules_rust. The Rust bridge builds locally into a separate target directory in the Bazel output tree. Use the Bazel version pinned in .bazelversion with a compatible Java runtime (Java 21 or newer for Bazel 9.2.0).

Build commands

# Build library
bazel build //:iggy-cpp

# Unit tests
bazel test //:unit

# Low-level integration tests (require a running server)
bazel test //:e2e

Binary mutations retain an uncertain operation until it is resolved or abandoned.
IggyException::OperationId() carries an exact std::uint64_t for
operation_uncertain, lane_blocked, and operation_not_retained.
Keep using the same client to recover:

try {
    client.CreateStream("events");
} catch (const iggy::IggyException &error) {
    if (error.Kind() == "operation_uncertain" && error.OperationId()) {
        client.ResolveOperation(*error.OperationId(), std::chrono::seconds(30));
    } else {
        throw;
    }
}

ResolveOperation retries the retained identity and proves a successful commit
without returning the original response payload. A proved committed business
rejection throws its original error. Both outcomes release the retained operation
and its lane. An operation_uncertain result remains retained.
AbandonOperation releases local state without undoing or determining the server
result. operation_not_retained does not prove noncommit, so submitting the old
payload again can duplicate work. A lane_blocked ID belongs to an earlier call;
resolve it before submitting the new request again. Polls remain transient and
are never retained or replayed.

Disconnect() and session reset discard retained operations without settling
their outcomes. Reconnecting cannot recover those discarded operations.
The client keeps the proved outcomes of its last max_operations replays, also
across a session reset. A later ResolveOperation of a proved operation returns
the same outcome without a request.
TCP, QUIC, and WebSocket support ResolveOperation and AbandonOperation;
HTTP reports feature_unavailable for both.

The blocking wrapper populates IggyException::Code() and Kind() for Rust SDK
errors from these methods:

• Login, Logout, ResolveOperation, and AbandonOperation.
• CreateUser, UpdateUser, and DeleteUser.
• CreateStream, UpdateStream, and DeleteStream.
• CreateTopic, UpdateTopic, and DeleteTopic.
• CreatePartitions, DeletePartitions, and DeleteSegments.
• CreateConsumerGroup, DeleteConsumerGroup, JoinConsumerGroup, and LeaveConsumerGroup.
• StoreConsumerOffset, StoreConsumerPosition, and DeleteConsumerOffset.

Other methods, including reads and Connect, Disconnect, and Shutdown, report
errors through the exception message only. C++ and bridge-local argument errors
also leave Code() and OperationId() empty and Kind() empty, including for the
methods listed above.

Direct generated-bridge callers pass a local iggy::ffi::ErrorInfo as the last
argument to login/logout, mutations, polls, raw commands, and recovery calls.
It is reset at entry and populated only when the Rust SDK returns an IggyError.
Bridge-local argument errors leave it empty. Inspect kind, code, and, when
has_operation_id is true, operation_id. Use separate values for concurrent
calls; error state is never stored globally or on the client.

Concurrent calls on one client are safe. Moving, assigning, or destroying the
client while any call is in progress is undefined behavior.

StoreConsumerOffset and DeleteConsumerOffset require an explicit partition on
TCP, QUIC, and WebSocket; an absent partition returns invalid_identifier.
An omitted partition, including the default on DeleteConsumerOffset, is for
HTTP only.

The binary session defaults to 256 active or retained mutations and 64 MiB of
frozen request bytes. Configure both limits with
Builder().WithPendingOperations(max_operations, max_bytes). Both must be positive.
FromConnectionString accepts the same limits as max_pending_operations and
max_pending_operation_bytes query parameters for TCP, QUIC, and WebSocket.
HTTP continues to use its configured transport behavior and has no retained
operation recovery.

Overview (`foreign/csharp/README.md`) Copy Section

C# SDK for [Iggy](https://github.com/apache/iggy) [](https://www.nuget.org/packages/Apache.Iggy)

Overview Copy Chapter

The Apache Iggy C# SDK provides a comprehensive client library for interacting with Iggy message streaming servers. It
offers a modern, async-first API with support for multiple transport protocols and comprehensive message streaming
capabilities.

Getting Started Copy Chapter

Installation Copy Section

Install the NuGet package:

dotnet add package Apache.Iggy --version 0.9.0

The package installation and API examples target server/SDK 0.9.0. For source builds, use the server and SDK
from the same checkout. The SDK targets .NET 8 and .NET 10; repository examples require .NET 10.
0.9.0 includes the independent message and consumer-offset durability options.

This checkout uses binary protocol 0.11.1. Servers and applicable SDKs must deploy together; mixed versions and
rolling upgrades are unsupported. This checkout requires fresh storage and refuses older data directories
without modification. --fresh deletes existing data in the configured directory.
See the server recovery guide.
Register creates a shared logical session; other TCP connections authenticate with BindSession (15), using the
parent's identity and proof. A disconnect removes its binding without ending the session. Logout or committed expiry
ends it; a new session must not replay an unresolved mutation from the old one.
GetPartitionRoute (105) discovers partition primaries. Commands 14, 103, 104 and 123 are retired.
Ordinary SDK sends and explicit offset mutations support the configured durability, including the Replicated default.
Crash-safe send retries require Persisted message durability; crash-safe explicit offset retries require Persisted
offset durability and Quorum acknowledgement. Weaker policies can lose data and receipts on a crash. HTTP NoAck's
202 confirms dispatch, not partition admission or commit, and supplies no recoverable caller receipt.

Supported Protocols Copy Section

The SDK supports two transport protocols:

• TCP - Binary protocol for optimal performance and lower latency (recommended)
• HTTP - RESTful JSON API for stateless operations

Over TCP the SDK speaks the VSR consensus framing, which is the only wire protocol the server accepts.

See Viewstamped Replication (VSR) for what that means for the client API.

Creating a Client Copy Section

The SDK is built around the IIggyClient interface. The API fragments below reuse this client and
these imports. Create the named resources before read, update, delete, send, or poll operations:

using System.Buffers;
using System.Text;
using System.Text.Json;
using Apache.Iggy;
using Apache.Iggy.Configuration;
using Apache.Iggy.Consumers;
using Apache.Iggy.Contracts;
using Apache.Iggy.Contracts.Auth;
using Apache.Iggy.Enums;
using Apache.Iggy.Factory;
using Apache.Iggy.Headers;
using Apache.Iggy.IggyClient;
using Apache.Iggy.Kinds;
using Apache.Iggy.Messages;
using Apache.Iggy.Publishers;
using Microsoft.Extensions.Logging;
using Partitioning = Apache.Iggy.Kinds.Partitioning;


using var client = IggyClientFactory.CreateClient(new IggyClientConfigurator
{
    BaseAddress = "127.0.0.1:8090",
    Protocol = Protocol.Tcp
});

await client.ConnectAsync();
await client.LoginUserAsync("iggy", "iggy");

Optionally, you can provide an ILoggerFactory for diagnostics and debugging (defaults to
NullLoggerFactory.Instance). Add Microsoft.Extensions.Logging.Console to use AddConsole:

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder
        .AddFilter("Apache.Iggy", LogLevel.Information)
        .AddConsole();
});

using var client = IggyClientFactory.CreateClient(new IggyClientConfigurator
{
    BaseAddress = "127.0.0.1:8090",
    Protocol = Protocol.Tcp,
    LoggerFactory = loggerFactory
});

await client.ConnectAsync();

Configuration Copy Section

The IggyClientConfigurator provides comprehensive configuration options:

using var client = IggyClientFactory.CreateClient(new IggyClientConfigurator
{
    BaseAddress = "127.0.0.1:8090",
    Protocol = Protocol.Tcp,

    // Socket buffer sizes in bytes (optional, null = OS default)
    ReceiveBufferSize = null,
    SendBufferSize = null,

    // Idle ping keeping the session alive under server-side heartbeat verification (TCP only).
    // Default 5 seconds, must be between 1 millisecond and about 49 days. Init-only.
    HeartbeatInterval = TimeSpan.FromSeconds(5),

    // Automatic reconnection with exponential backoff (enabled by default, infinite retries)
    ReconnectionSettings = new ReconnectionSettings
    {
        Enabled = true,
        MaxRetries = 0,              // 0 = infinite retries
        InitialDelay = TimeSpan.FromSeconds(5),
        MaxDelay = TimeSpan.FromSeconds(30),
        WaitAfterReconnect = TimeSpan.FromSeconds(1),
        UseExponentialBackoff = true,
        BackoffMultiplier = 2.0
    },

    // Auto-login after connection. Reconnect first binds the retained session; configured or
    // remembered LoginUserAsync credentials allow a fresh login after that session ends
    AutoLoginSettings = AutoLoginSettings.For("iggy", "iggy"),
    // or AutoLoginSettings.ForPersonalAccessToken("your_token")

    // Optional: logging
    LoggerFactory = loggerFactory
});

await client.ConnectAsync();

TCP applies the connection, heartbeat, TLS, and auto-login settings. HTTP ConnectAsync does no
network work and requires explicit login. For TLS configuration, see the TcpTls example.

Connection String Copy Section

A TCP client can also be created from the connection string format shared with the other SDKs. The
credentials become the auto-login settings, so the client signs in on ConnectAsync:

using var client = IggyClientFactory.CreateClient("iggy://iggy:[email protected]:8090");
await client.ConnectAsync();

// A personal access token instead of a username and password, plus options
using var tokenClient = IggyClientFactory.CreateClient(
    "iggy+tcp://[email protected]:8090?heartbeat_interval=10s&reconnection_retries=5");

// Parse into a configurator to adjust the remaining settings
var config = IggyClientConfigurator.FromConnectionString("iggy://iggy:[email protected]:8090");
config.LoggerFactory = loggerFactory;

reestablish_after and nodelay are accepted so that strings shared with other SDKs keep parsing, but
they change nothing. The client has no reestablish_after window after a lost connection, and it always opens
sockets with NoDelay, so nodelay=false does not turn Nagle back on even though the Rust and Node.js SDKs leave
Nagle on by default.

reconnection_retries=0 differs from the Rust SDK. After a lost connection, Rust still makes one reconnect
attempt, but this client fails the request at once. It also differs from ReconnectionSettings.MaxRetries,
where 0 means unlimited, so the parser maps reconnection_retries=0 to Enabled = false instead.

Reconnection uses no exponential backoff, and after each reconnect the client waits
ReconnectionSettings.WaitAfterReconnect (1 second by default). Credentials are taken literally: they are
not percent-decoded and must not contain @ or :. Bracketed IPv6 hosts such as
iggy://iggy:iggy@[::1]:8090 are accepted. Durations look like 500ms, 5s or 1m30s. Only iggy:// and
iggy+tcp:// are supported. A malformed string throws FormatException, whose message never
contains the connection string.

Viewstamped Replication (VSR) Copy Chapter

Over TCP every request is wrapped in a 256-byte consensus header, the client registers a consensus session at
login, and replicated writes complete after the required quorum acknowledges them.
Durability.Persisted additionally waits for persistence on that quorum; explicit offset stores have
an independent ConsumerOffsetDurability policy. A NoAck success is a weak acknowledgement and does not prove commit.
Uncertain replicated operations can be resolved through the recovery methods below.

using var client = IggyClientFactory.CreateClient(new IggyClientConfigurator
{
    BaseAddress = "127.0.0.1:8090",
    Protocol = Protocol.Tcp,

    // Upper bound on a reply frame the server announces, 64 MiB by default.
    MaxResponseFrameSize = 64 * 1024 * 1024,
    MaxPendingOperations = 256,
    MaxPendingOperationBytes = 64L * 1024 * 1024,

    AutoLoginSettings = AutoLoginSettings.For("iggy", "iggy")
});

await client.ConnectAsync();

What changes under VSR Copy Section

• Login registers a shared session in this checkout. LoginUserAsync / LoginWithPersonalAccessTokenAsync
  authenticate Register. Auxiliary and reconnected transports bind the retained identity and proof without another Register.
  Disconnecting removes only that binding; Logout, committed lease expiry or user deactivation ends the session.
  A StaleClient eviction also removes only the binding of that link. The reconnect binds the same session
  again, so its consumer-group memberships stay.
  A committed StaleClient result means that the server ended the session, for example because the user was
  deleted or deactivated. The client drops that session, and with reconnection enabled it signs in with configured
  or remembered credentials. A StaleClient refusal before commit means only that the server detached the link
  that carried it from the session. A live session also gets this refusal after a replica replaced its client
  table. The client closes that link and binds the session again on a new link. If the session ended, that bind
  fails with Unauthenticated and the client signs in again. Otherwise the session, its retained operations and
  its consumer-group memberships stay.
  Password changes and PAT changes or expiry do not end an established session.
  A terminal BindSession refusal (Unauthenticated or StaleClient) permits a fresh login. A transient refusal
  retains the identity while that replica's background reconnect retries the bind. An unresolved mutation
  cannot be replayed under a fresh session. The older 0.9.0 connection-lifetime rule does not apply to this checkout.
• Replica links are independent. The client proactively opens one shared session link per roster replica.
  Calls use the advertised metadata or partition primary. Replies are matched by request id, so concurrent
  reads and different partition groups can progress independently; cancelling one call preserves other calls.
  With a one-node roster, the client uses BaseAddress instead of the address that the node advertises, so a
  port mapping or a port-forward works.
• The client picks partitions. Balanced and message-key partitioning are resolved
  client-side (xxHash32 matches Rust for identical encoded key bytes and partition counts), and consumer-group polls round-robin
  over the partitions the coordinator assigned to this client.
• Consumer groups are assignment-based. JoinConsumerGroupAsync makes this client a member; the assignment
  is synced on demand and refreshed on every PingAsync. Partition counts are cached for 30 seconds, so a topic
  another client widens is picked up without waiting for a ping.
• Credentials are bounds-checked locally. A username outside 3-50 bytes, a password outside 3-100 bytes or a
  personal access token outside 1-255 bytes is rejected before the register body is framed.
• PingAsync costs more than a ping. Besides the ping it re-syncs the assignment of every consumer group
  this client has joined, so it makes one extra round trip per joined group. The TCP client pings on its own
  every HeartbeatInterval (5 seconds by default) while connected, so an idle session survives the server's
  heartbeat verification and assignments stay fresh; a lost session is repaired by the regular reconnect and
  auto login.

Retries and failed requests Copy Section

Within one call deadline, the SDK retries an unresolved replicated mutation using its original session,
request id, context, and frozen frame. Metadata mutations share one lane; partition mutations share a lane
by incarnation and partition id, including requests using different names for the same partition.

• IggyInvalidStatusCodeException carries the server status. FromServer distinguishes a server verdict
  from a local error, and Committed identifies a proved committed result.
• VsrRequestOutcomeUnknownException.OperationId identifies a retained replicated mutation after its call
  ends without proof. Its lane stays blocked. VsrLaneBlockedException.OperationId identifies the operation
  blocking a subsequent mutation in the same lane; independent groups and pure reads can continue.
• ResolveOperationAsync(operationId, timeout, token) replays that frozen operation. A proved success or
  business rejection releases the lane; a rejection throws its original status. Further uncertainty retains it.
  The client keeps the proved outcomes of its last MaxPendingOperations replays, also across a session
  reset. A later resolve of the same operation returns the same outcome without a request.
• AbandonOperationAsync(operationId) releases local ownership without determining the outcome. It does
  not undo a committed operation. The operation stops counting against MaxPendingOperations at once.
  A ResolveOperationAsync that is still running is not canceled and keeps the lane until it ends, so later
  mutations of the lane still wait behind its replay.
  VsrOperationNotRetainedException means that no replay proved the outcome
  before a session reset, such as logout or session replacement, or an abandon. A proved outcome that aged
  out also causes it. It never proves non-commit and never authorizes an automatic fresh resend.

Auto-commit and Next polls hold a transient partition lane but are not retained or replayed after uncertainty.
Their unknown-outcome error has no operation id. IggyConsumer reports it through its error event and polls again.
Under AutoCommitMode.Auto, the server can store the offset of a batch whose reply was lost. A Next poll then
starts after that batch, so the consumer skips it. Behind a retained mutation these polls throw
VsrLaneBlockedException. IggyConsumer propagates it without resolving the blocker and ends its polling loop.
The application decides how to resume. Background IggyPublisher sends report unknown outcomes through the
message-batch-failed event; direct sends throw them to the caller.

IggyConsumer offset stores and deletes resolve any blocking operation before their own request. A background
IggyPublisher send resolves only a blocking batch of its own. Both continue after a proved commit or committed
rejection. The client keeps that outcome for the caller that received the operation id, such as the receiver of
the message-batch-failed event. That caller reads it with ResolveOperationAsync.

MaxPendingOperations and MaxPendingOperationBytes bound active plus retained mutation frames across the
client, defaulting to 256 operations and 64 MiB. Admission waits within the call deadline before allocating an
id or frozen frame. A replay shares its reservation; outstanding writers keep bytes charged until released.
Both limits must be positive. The byte limit includes the 256-byte header and the mutation body.
Smaller positive limits are valid but reject mutations that cannot fit; reads and login remain available.
Reads and control requests have a separate per-link limit of 256 pending calls and a 32-frame write queue.
Their bodies, including unknown raw non-replicated commands, are outside the mutation byte budget.

Limitations Copy Section

• TCP uses VSR framing; HTTP uses the separate REST API. Join/leave, GetMeAsync, segment deletion,
  and raw binary requests are TCP-only and throw FeatureUnavailableException on HTTP.
• StoreOffsetAsync / DeleteOffsetAsync need an explicit partition id under VSR: the broker does not
  resolve a null partition for a consumer-offset request, so passing one throws client-side.
• Polling a missing topic throws IggyInvalidStatusCodeException. An existing topic with no available
  messages returns an empty poll.

Behaviour changes for existing clients Copy Section

• IIggyPublisher.FlushUnsavedBufferAsync was removed. Set TopicOptions.Durability to
  Durability.Persisted when creating a topic to wait for message persistence.
• MaxResponseFrameSize bounds the reply frames the VSR reader accepts. A reply larger than the 64 MiB
  default is refused and the connection is dropped, so raise it if a single response legitimately exceeds that
  • a large GetSnapshotAsync is the usual case.
• Clients built with IggyConsumerBuilder / IggyPublisherBuilder now auto-login with the credentials passed
  to WithConnection. Before, a builder-created client came back from a
  reconnect unauthenticated; now the credentials are held for the lifetime of the connection and replayed.
• The TCP client pings each ready replica every HeartbeatInterval (5 seconds). Each replica reconnects
  independently, with at most four concurrent dials, a two-second dial and TLS deadline, and jittered
  backoff capped at ten seconds. MaxRetries bounds one burst of consecutive retries per replica, and zero
  allows unlimited retries. While the client is signed in, each heartbeat restarts a replica link that used
  up its burst, with a fresh budget, so the client recovers from a longer outage without an application call.
  Logout and Dispose stop the restarts. Invalid local CA configuration and rejected credentials surface as
  errors. A caller's cancellation stops that call while the link owner continues reconnecting. Request replay
  has its own deadline.
  Configured auto-login credentials take precedence over credentials remembered from successful
  username/password or PAT login. Without either, reconnecting cannot restore the authenticated session.
  A committed password change or rename of the signed-in user updates these credentials. This includes a
  change that ResolveOperationAsync proves. A committed rejection leaves them unchanged.
• AutoLoginSettings properties are now init-only, as is IggyClientConfigurator.HeartbeatInterval. Build
  them with an object initializer or the AutoLoginSettings.For / AutoLoginSettings.ForPersonalAccessToken
  factories instead of assigning after construction.
• IggyConsumerBuilder / IggyPublisherBuilder accept a personal access token through the
  WithConnection(protocol, address, personalAccessToken, ...) overload, as an alternative to a username and
  password.
• The SDK now ships a dependency on System.IO.Hashing, used for the client-side message-key partitioner.
• TCP sockets use NoDelay to avoid delaying trailing request segments behind acknowledgements.

Authentication Copy Chapter

User Login Copy Section

Use the credentials configured for the server. The setup below initializes a new server with iggy / iggy:

var response = await client.LoginUserAsync("iggy", "iggy");

Creating Users Copy Section

Create new users with customizable permissions:

var permissions = new Permissions
{
    Global = new GlobalPermissions
    {
        ManageServers = true,
        ManageUsers = true,
        ManageStreams = true,
        ManageTopics = true,
        PollMessages = true,
        ReadServers = true,
        ReadStreams = true,
        ReadTopics = true,
        ReadUsers = true,
        SendMessages = true
    }
};

await client.CreateUserAsync("test_user", "secure_password", UserStatus.Active, permissions);

// Login with the new user
var loginResponse = await client.LoginUserAsync("test_user", "secure_password");

Personal Access Tokens Copy Section

Create and use Personal Access Tokens (PAT) for programmatic access:

// Create a PAT
var patResponse = await client.CreatePersonalAccessTokenAsync("api-token", TimeSpan.FromHours(1));

// Login with PAT
await client.LoginWithPersonalAccessTokenAsync(patResponse!.Token);

Streams and Topics Copy Chapter

Creating Streams Copy Section

await client.CreateStreamAsync("my-stream");

You can reference streams by either numeric ID or name:

var streamById = Identifier.Numeric(0);
var streamByName = Identifier.String("my-stream");

Creating Topics Copy Section

Every stream contains topics for organizing messages:

var streamId = Identifier.String("my-stream");

await client.CreateTopicAsync(
    streamId,
    name: "my-topic",
    partitionsCount: 3,
    compressionAlgorithm: CompressionAlgorithm.None,
    messageExpiry: TimeSpan.MaxValue,  // never expire
    maxTopicSize: 1024 * 1024 * 1024   // 1 GiB
);

Note: Stream and topic names use hyphens instead of spaces. Iggy automatically replaces spaces with hyphens.

Publishing Messages Copy Chapter

Sending Messages Copy Section

Send messages using the publisher interface:

var streamId = Identifier.String("my-stream");
var topicId = Identifier.String("my-topic");

var messages = new List<Message>
{
    new(Guid.NewGuid(), "Hello, Iggy!"u8.ToArray()),
    new(1, "Another message"u8.ToArray())
};

await client.SendMessagesAsync(
    streamId,
    topicId,
    Partitioning.None(),  // balanced partitioning
    messages
);

Partitioning Strategies Copy Section

Control which partition receives each message:

// Balanced partitioning (default)
Partitioning.None();

// Send to specific partition
Partitioning.PartitionId(1);

// Key-based partitioning (string)
Partitioning.EntityIdString("user-123");

// Key-based partitioning (integer)
Partitioning.EntityIdInt(12345);

// Key-based partitioning (GUID)
Partitioning.EntityIdGuid(Guid.NewGuid());

User-Defined Headers Copy Section

Add custom headers to messages with typed values:

var headers = new Dictionary<HeaderKey, HeaderValue>
{
    { HeaderKey.FromString("correlation_id"), HeaderValue.FromString("req-123") },
    { HeaderKey.FromString("priority"), HeaderValue.FromInt32(1) },
    { HeaderKey.FromString("timeout"), HeaderValue.FromInt64(5000) },
    { HeaderKey.FromString("confidence"), HeaderValue.FromFloat(0.95f) },
    { HeaderKey.FromString("is_urgent"), HeaderValue.FromBool(true) },
    { HeaderKey.FromString("request_id"), HeaderValue.FromGuid(Guid.NewGuid()) }
};

var messages = new List<Message>
{
    new(Guid.NewGuid(), "Message with headers"u8.ToArray(), headers)
};

await client.SendMessagesAsync(
    streamId,
    topicId,
    Partitioning.PartitionId(1),
    messages
);

Message durability Copy Section

There is no on-demand flush command. Set TopicOptions.Durability = Durability.Persisted when creating
a topic to wait for message persistence on the required quorum. ConsumerOffsetDurability controls
explicit offset-store completion independently. Flush thresholds alone do not provide that guarantee.

Consumer Groups Copy Chapter

Creating Consumer Groups Copy Section

Coordinate message consumption across multiple consumers:

var groupResponse = await client.CreateConsumerGroupAsync(
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    "my-consumer-group"
);

Joining and Leaving Groups Copy Section

Note: Join/Leave operations are only supported on TCP protocol and will throw FeatureUnavailableException on HTTP.

// Join a consumer group
await client.JoinConsumerGroupAsync(
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    Identifier.String("my-consumer-group")
);

// Leave a consumer group
await client.LeaveConsumerGroupAsync(
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    Identifier.String("my-consumer-group")
);

Consuming Messages Copy Chapter

Fetching Messages Copy Section

Fetch a batch of messages:

var polledMessages = await client.PollMessagesAsync(new MessageFetchRequest
{
    StreamId = streamId,
    TopicId = topicId,
    Consumer = Consumer.New(1), // or Consumer.Group("my-consumer-group") for consumer group
    Count = 10,
    PartitionId = 0, // optional, null for consumer group
    PollingStrategy = PollingStrategy.Next(),
    AutoCommit = true
});

foreach (var message in polledMessages.Messages)
{
    Console.WriteLine($"Message: {Encoding.UTF8.GetString(message.Payload)}");
}

With AutoCommit = true, the server advances/submits the offset before application processing.
The poll reply does not wait for durable completion of that store. Use explicit offset stores after
processing when that distinction matters.

Polling Strategies Copy Section

Control where message consumption starts:

// Start from a specific offset
PollingStrategy.Offset(1000);

// Start from a specific timestamp (microseconds since epoch)
PollingStrategy.Timestamp(1699564800000000);

// Start from the first message
PollingStrategy.First();

// Start from the last message
PollingStrategy.Last();

// Start from the next unread message
PollingStrategy.Next();

Offset Management Copy Chapter

Storing Offsets Copy Section

Store the current consumer position:

await client.StoreOffsetAsync(
    Consumer.New(1),
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    offset: 42,
    partitionId: 0
);

Retrieving Offsets Copy Section

Get the current stored offset:

var offsetInfo = await client.GetOffsetAsync(
    Consumer.New(1),
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    partitionId: 0
);

Console.WriteLine($"Stored offset: {offsetInfo!.StoredOffset}");

Deleting Offsets Copy Section

Clear stored offsets:

await client.DeleteOffsetAsync(
    Consumer.New(1),
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    partitionId: 0
);

System Operations Copy Chapter

Cluster Information Copy Section

Get cluster metadata and node information:

var metadata = await client.GetClusterMetadataAsync();

Server Statistics Copy Section

Retrieve server performance metrics:

var stats = await client.GetStatsAsync();

Health Checks Copy Section

Verify server connectivity:

await client.PingAsync();

Client Information Copy Section

Get information about connected clients:

var clients = await client.GetClientsAsync();
var currentClient = await client.GetMeAsync();

Snapshots Copy Section

Capture a system snapshot as a compressed ZIP archive:

var snapshotBytes = await client.GetSnapshotAsync(
    SnapshotCompression.Zstd,
    new List<SystemSnapshotType>
    {
        SystemSnapshotType.ServerLogs,
        SystemSnapshotType.ServerConfig,
        SystemSnapshotType.ResourceUsage
    }
);

// Or capture everything
var fullSnapshot = await client.GetSnapshotAsync(
    SnapshotCompression.Deflated,
    new List<SystemSnapshotType> { SystemSnapshotType.All }
);

Available compression methods: Stored, Deflated, Bzip2, Zstd, Lzma, Xz.

Available snapshot types: FilesystemOverview, ProcessList, ResourceUsage, Test, ServerLogs, ServerConfig,
All.

Segment Management Copy Section

Delete up to N of the oldest sealed segments from a partition; the active segment is retained:

await client.DeleteSegmentsAsync(
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    partitionId: 1,
    segmentsCount: 2
);

Event Subscription Copy Chapter

Subscribe to connection events:

Func<ConnectionStateChangedEventArgs, Task> handler = async args =>
{
    Console.WriteLine($"Current connection state: {args.CurrentState}");
    await Task.CompletedTask;
};

client.SubscribeConnectionEvents(handler);

// Later
client.UnsubscribeConnectionEvents(handler);

Advanced: IggyPublisher Copy Chapter

High-level publisher with background sending and retries. Retry settings apply to background sends;
maxAttempts includes the first send. Await the queue drain before disposal:

using System.Buffers;
using System.Text;
using System.Text.Json;
using Apache.Iggy;
using Apache.Iggy.Consumers;
using Apache.Iggy.Kinds;
using Apache.Iggy.Messages;
using Apache.Iggy.Publishers;

await using var publisher = IggyPublisherBuilder.Create(
    client,
    Identifier.String("my-stream"),
    Identifier.String("my-topic")
)
.WithPartitioning(Partitioning.None())
.WithBackgroundSending(enabled: true, batchSize: 100)
.WithRetry(maxAttempts: 3)
.Build();

await publisher.InitAsync();

var messages = new List<Message>
{
    new(Guid.NewGuid(), "Message 1"u8.ToArray()),
    new(0, "Message 2"u8.ToArray())
};

await publisher.SendMessagesAsync(messages);

// Drain the background queue, then dispose
await publisher.WaitUntilAllSendsAsync();

For automatic object serialization, use the typed variant:

await using var publisher = IggyPublisherBuilder<Order>.Create(
    client,
    Identifier.String("orders-stream"),
    Identifier.String("orders-topic"),
    new OrderSerializer()
).Build();

await publisher.InitAsync();
await publisher.SendAsync(new List<Order> { new(Guid.NewGuid(), 99.90m) });

record Order(Guid OrderId, decimal Amount);

class OrderSerializer : ISerializer<Order>
{
    public void Serialize(Order data, IBufferWriter<byte> writer) =>
        writer.Write(JsonSerializer.SerializeToUtf8Bytes(data));
}

Advanced: IggyConsumer Copy Chapter

High-level consumer with automatic offset management and consumer groups:

using System.Text;
using Apache.Iggy;
using Apache.Iggy.Consumers;
using Apache.Iggy.Kinds;

await using var consumer = IggyConsumerBuilder.Create(
    client,
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    Consumer.New(1)
)
.WithPollingStrategy(PollingStrategy.Next())
.WithBatchSize(10)
.WithAutoCommitMode(AutoCommitMode.Auto)
.Build();

await consumer.InitAsync();

await foreach (var message in consumer.ReceiveAsync())
{
    var payload = Encoding.UTF8.GetString(message.Message.Payload);
    Console.WriteLine($"Offset {message.CurrentOffset}: {payload}");
}

For consumer groups (load-balanced across multiple consumers):

await using var consumer = IggyConsumerBuilder.Create(
    client,
    Identifier.String("my-stream"),
    Identifier.String("my-topic"),
    Consumer.Group("my-group")
)
.WithConsumerGroup("my-group", createIfNotExists: true, joinGroup: true)
.WithPollingStrategy(PollingStrategy.Next())
.WithAutoCommitMode(AutoCommitMode.AfterReceive)
.Build();

await consumer.InitAsync();

await foreach (var message in consumer.ReceiveAsync())
{
    var payload = Encoding.UTF8.GetString(message.Message.Payload);
    Console.WriteLine($"Partition {message.PartitionId}: {payload}");
}

For automatic deserialization:

var builder = IggyConsumerBuilder<OrderEvent>.Create(
    client,
    Identifier.String("orders-stream"),
    Identifier.String("orders-topic"),
    Consumer.Group("order-processors"),
    new OrderDeserializer()
);
builder.WithPollingStrategy(PollingStrategy.Next());
builder.WithAutoCommitMode(AutoCommitMode.AfterReceive);

await using var consumer = builder.Build();
await consumer.InitAsync();

await foreach (var message in consumer.ReceiveDeserializedAsync())
{
    if (message.Status == MessageStatus.Success)
    {
        Console.WriteLine($"Order: {message.Data?.OrderId}");
    }
}

record OrderEvent(Guid OrderId, decimal Amount);

class OrderDeserializer : IDeserializer<OrderEvent>
{
    public OrderEvent Deserialize(ReadOnlyMemory<byte> data) =>
        JsonSerializer.Deserialize<OrderEvent>(data.Span)!;
}

API Reference Copy Chapter

The SDK provides the following main interfaces:

• IIggyClient - Main client interface (aggregates all features)
• IIggyPublisher - Per-call message publishing interface
• IIggyConsumer - Per-call message consumption interface
• IIggyStream - Stream management
• IIggyTopic - Topic management
• IIggyOffset - Offset management
• IIggyConsumerGroup - Consumer group operations
• IIggyPartition - Partition operations
• IIggySegment - Segment management
• IIggyUsers - User and authentication management
• IIggySystem - System and cluster operations
• IIggyPersonalAccessToken - Personal access token management

Additionally, builder-based APIs are available:

• IggyPublisherBuilder / IggyPublisherBuilder - Fluent publisher configuration
• IggyConsumerBuilder / IggyConsumerBuilder - Fluent consumer configuration

Running Examples Copy Chapter

Examples are located in examples/csharp/src/ in root iggy directory. Available examples:

• GettingStarted - Basic producer/consumer setup
• Basic - Simple message publishing and consuming
• MessageHeaders - Using custom message headers
• MessageEnvelope - Envelope pattern for message serialization
• NewSdk - High-level IggyPublisher/IggyConsumer API

Start the matching source server from the repository root (use configured credentials for an existing data directory):

IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy cargo run --bin iggy-server

Run an example from the examples/csharp/ directory. Run TcpTls from the repository root so its
certificate paths resolve:

dotnet run -c Release --project src/GettingStarted/Iggy_SDK.Examples.GettingStarted.Producer
dotnet run -c Release --project src/GettingStarted/Iggy_SDK.Examples.GettingStarted.Consumer

Integration Tests Copy Chapter

Integration tests are located in Iggy_SDK.Tests.Integration/. Tests can run against:

• A dockerized Iggy server with TestContainers
• A local Iggy server (set IGGY_SERVER_HOST environment variable)

Requirements Copy Section

• .NET 10 SDK
• Docker (for TestContainers tests)

Running Integration Tests Locally Copy Section

1. Dockerization

The suite runs against iggy-server. TCP only: the SDK frames TCP with the
VSR wire protocol, the cluster serves reads from the primary, and the HTTP surface has no equivalent path to
route them through.

cargo build --bin iggy-server --bin iggy

docker build --no-cache -f core/server/Dockerfile --platform linux/amd64 --target runtime-prebuilt --build-arg PREBUILT_IGGY_SERVER=target/debug/iggy-server --build-arg PREBUILT_IGGY_CLI=target/debug/iggy -t iggy-server:test .

2. Build the Test Project

dotnet build foreign/csharp/Iggy_SDK.Tests.Integration

3. Run test

cd foreign/csharp
export IGGY_SERVER_DOCKER_IMAGE=iggy-server:test
dotnet test -f net10.0 --project Iggy_SDK.Tests.Integration --no-build --verbosity diagnostic

IGGY_SERVER_DOCKER_IMAGE defaults to iggy-server:test, so the export above is only needed to point
at a different image. Rider and Visual Studio need nothing configured.

Useful Resources Copy Chapter

• Iggy Documentation
• NuGet Package

ROADMAP - TODO Copy Chapter

• Error handling with status codes and descriptions
• Add support for ASP.NET Core Dependency Injection

Overview (`foreign/go/README.md`) Copy Section

Go SDK for Iggy

Official Go client SDK for Apache Iggy message streaming.

The client speaks the VSR wire protocol over TCP, with or without TLS.
Concurrent requests share one managed connection per replica and one logical
session. VSR is the only protocol it supports.

Installation Copy Chapter

The current source requires Go 1.25 or newer. From your application module,
install a release compatible with your server:

go get github.com/apache/iggy/foreign/go

Unversioned go get selects the latest stable release. VSR support starts
with v0.9.0. For unreleased changes,
build both SDK and server from the same checkout; examples/go/go.mod
replaces this module with the local SDK source.

Running a server Copy Chapter

From the repository root, build and start a VSR server using a new,
disposable data directory. The root environment variables bootstrap a new
instance; they do not replace credentials recovered from disk or peers:

cargo build --bin iggy-server

IGGY_PATH=/tmp/iggy-go \
IGGY_TCP_ADDRESS=127.0.0.1:8090 \
IGGY_HTTP_ENABLED=false IGGY_QUIC_ENABLED=false IGGY_WEBSOCKET_ENABLED=false \
IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy \
target/debug/iggy-server

QUIC, WebSocket and HTTP are enabled by default on ports 8080, 8092 and 3000.
Disable the ones you do not need so they cannot race with another process.

Delivery semantics Copy Chapter

SendMessages returns any placements the server reports. Replicated mutations
replay the same frozen request, client identity, and request ID after an uncertain
reply. Once uncertain, only a matching final committed-result proof can settle
the operation. A reconnect binds the existing session, also after a stale client
eviction; a terminal session refusal ends its retained operations and permits a
new registration. While the client registers that new session by itself, a call
that it did not send fails with 58 instead of waiting for the session. The caller
can retry that call. If the registration fails, these calls fail with 40 and the
application must sign in.

A reply with status 30 (StaleClient) can deny one attempt, because the server
no longer attaches the link to the session. Usually the session ended, for
example because an administrator deleted or deactivated the user. But after a
state transfer, which copies the state of another replica, a replica also denies
a live session with 30. So the client closes only that link, as after a stale
client eviction, and the next link binds the session again. If the bind
succeeds, the session keeps its retained operations and its consumer group
membership. If the server refuses the bind, the client registers a new session,
as after any terminal session refusal.

A committed 30 means that the replicated metadata ended the session, so the
client drops the session at once. A committed 30 still settles an uncertain
operation, like any other committed rejection.

The client registers a new session with the WithAutoLogin credentials, or
else with those of the last explicit sign-in. When a password change or a rename
of the user behind those credentials commits, the client updates them. This holds
for typed calls, SendBinaryRequest and ResolveOperation. A rejected change
leaves the credentials unchanged.

If a mutation remains uncertain when its call ends, errors.As exposes
errors.UncertainOutcome with an exact uint64 OperationID. The SDK retains
that request and blocks later mutations on its lane with errors.LaneBlocked
carrying the same ID. Other partitions and pure reads can continue. Metadata
mutations share one lane; partition mutations use the partition incarnation
and partition ID.

Call ResolveOperation(ctx, operationID) to resume that exact request. A proven
business rejection returns its original error and releases the lane. The client
keeps the last maxOperations outcomes that its replays proved, also across a
session reset. So a later ResolveOperation of the same ID returns that outcome
without a request, also after another caller resolved the operation first.

Adding these two methods to contracts.Client requires external implementations,
wrappers, and mocks to implement them. The exported
IggyTcpClient.HandleLeaderRedirection method has been removed; replica routing
and reconnection are managed by the client.

If the application accepts the uncertainty, call
AbandonOperation(ctx, operationID) to release local recovery ownership.
Abandoning does not cancel server execution or make a new application send safe
from duplication. A ResolveOperation that is still running keeps the lane
until it ends, so later mutations still queue behind its replay. The client also
keeps the outcome that this replay proves. errors.OperationNotRetained does not
establish the outcome. Either the client never proved it, because a session
reset or an abandon came first, or the proven outcome aged out. An unknown ID
gets the same error.

tcp.WithPendingOperations(maxOperations, maxBytes) configures the shared bound
for active and retained mutation frames. Both values must be positive. Defaults
are 256 operations and 64 MiB, including each 256-byte header. Admission waits
within the call deadline before allocating a request ID; an oversized frame
fails locally. Writer references keep their bytes charged after abandonment or
session reset until the writer releases them. Calls without an earlier context
deadline have a 30-second operation budget.

Managed connections reconnect independently, with at most four simultaneous
dials and bounded exponential jitter. Each replica connection dials the address
that its node advertises in the cluster roster. With a one-node roster, it dials
the tcp.WithServerAddress endpoint instead, so a single node behind a port
mapping or a port-forward stays reachable. tcp.WithHeartbeatInterval controls
activity on every ready replica connection; the default is five seconds and a
nonpositive interval disables it. Logout, disconnect, and close stop the tasks.

After a session loss, the client registers a new session by itself. A
registration that loses its link or reaches no endpoint goes again after the
reconnect backoff. A refusal by the server, a disconnect or a close stops it.
A sign-in by the application also stops it. If that sign-in fails, the client
signs in again only when the application does. The sign-in does not wait for a
pending automatic registration, and the late reply cannot replace the new
session.

An established session has its own lifetime. Password or PAT changes and PAT
expiry do not end it. Logout, session lease expiry, or user deactivation end the
session and prevent its bind proof from restoring it.

Crash durability follows the topic's durability policy: replicated
confirms replication, while persisted also waits for the required replicas
to persist the message data. An empty confirmation list is a valid success
but does not by itself prove that new messages were appended.

In a cluster, sends, polls, and consumer offset operations use the authoritative
partition route (GetPartitionRoute, command 105). Metadata and membership
operations use the metadata primary. Every replica link binds the same session,
so partition routing preserves consumer-group membership. Route aliases share
view and refusal fences; route and topic caches have bounded capacity.

Use SDKs and servers from the same protocol generation and fresh storage for
this wire-contract cutover. The server refuses the previous storage format;
there is no implicit migration or fallback to retired routing commands.

Next and auto-commit polls can advance progress before replying. A 57 reply,
a lost reply or an eviction after sending ends that call immediately with 57,
releases its transient lane barrier, and creates no retained operation. A later
application poll is a new operation and may skip messages whose progress already
advanced. Pure reads retry within their deadline. Known non-admission replies 58
and 89 can retry without changing the frozen request. Manual offset writes are
replicated mutations and use the exact-ID recovery API above.

Testing Copy Chapter

Unit tests need nothing running:

go test ./...

The end-to-end suite runs against a server at the address in
IGGY_TCP_ADDRESS and skips when that variable is unset:

IGGY_TCP_ADDRESS=127.0.0.1:8090 go test ./tests

Add IGGY_TCP_TLS_ENABLED=true to run the TLS cases against a server started
with IGGY_TCP_TLS_ENABLED=true and the certificate pair in core/certs.

From the repository root, the integration harness builds a three-node fixture,
seeds messages, moves metadata leadership independently of the partition
primary, and runs the Go regression. It requires Go on PATH:

cargo build --bin iggy-server --bin iggy
cargo test -p integration given_split_primaries_when_go_group_ -- --ignored --test-threads=1

To use an existing fixture, set its coordinator address and topic. The test
expects eight messages per partition by default; override that with the
positive integer IGGY_POLL_ROUTING_MESSAGES_PER_PARTITION when needed:

IGGY_TCP_ADDRESS=127.0.0.1:20016 \
IGGY_POLL_ROUTING_STREAM=sdk-primary-routing \
IGGY_POLL_ROUTING_TOPIC=go \
go test ./tests -run TestE2E_SplitPrimaryPollsPreserveCoordinatorMembership

Contributing Copy Chapter

Before creating a pull request, please run golangci-lint and fix any reported lint issues:

golangci-lint run

Overview (`foreign/java/README.md`) Copy Section

Java SDK for Apache Iggy

Official Java client SDK for Apache Iggy message streaming.

This is part of the Apache Iggy monorepo. For the main project, see the root repository.

Installation Copy Chapter

These examples target server 0.9.0. SDK 0.9.0 is on Maven Central and works with server 0.9.0. The older 0.8.0 artifact speaks the previous TCP protocol and does not work with server 0.9.0. Java 17 or newer is required.

Add the dependency to your project:

Gradle:

implementation 'org.apache.iggy:iggy:0.9.0'

Maven:

<dependency>
    <groupId>org.apache.iggy</groupId>
    <artifactId>iggy</artifactId>
    <version>0.9.0</version>
</dependency>

See Maven Central for all published versions.

Snapshot Versions Copy Section

Development builds of the next release carry the version 0.9.1-SNAPSHOT. Get them from the ASF snapshot repository:

Gradle:

repositories {
    mavenCentral()
    maven {
        url = uri("https://repository.apache.org/content/repositories/snapshots/")
    }
}

dependencies {
    implementation 'org.apache.iggy:iggy:0.9.1-SNAPSHOT'
}

Maven:

<repositories>
    <repository>
        <id>apache-snapshots</id>
        <url>https://repository.apache.org/content/repositories/snapshots/</url>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

<dependencies>
    <dependency>
        <groupId>org.apache.iggy</groupId>
        <artifactId>iggy</artifactId>
        <version>0.9.1-SNAPSHOT</version>
    </dependency>
</dependencies>

Unreleased 0.11.1 protocol Copy Section

The source checkout uses protocol 0.11.1. Build it with a matching server.
CommandCode.System.ATTACH_CONSUMER_SESSION (code 14) was removed. Use
CommandCode.System.BIND_SESSION (code 15), which requires the registered
session's private bind secret. Auxiliary poll connections reuse that session
instead of logging in independently.

Transport loss retains the logical identity and reconnect uses BindSession.
A definitive stale or unauthenticated bind reply permits a fresh Register, and
transient replies retain the identity. A stale-client eviction during the bind
is a lost connection, so the next connection binds the same session again. An
unauthenticated eviction during the bind ends the session. The bind proof is an
independent session credential: password/PAT changes or expiry do not end an
established session. Logout, lease expiry and user deactivation end session
access.

A reply with status 30 (STALE_CLIENT) means that the server no longer attaches
the connection to the session. The client closes that connection, and the next
one binds the session again. A live session keeps its retained operations and
its group membership. An ended session gets a refused bind, and the client signs
in again with the remembered sign-in or stays signed out.

RetryPolicy limits one burst of redials after a connection loss, not the life
of the client. A heartbeat that fails starts a new burst with a fresh budget. If
the client lost its session and every sign-in of a burst lost its link, the
burst ends with the client signed out. A heartbeat that the server answers then
signs in again with a fresh budget. A logout, a rejected sign-in and close()
end the automatic sign-in. With RetryPolicy.noRetry(), the client stays signed
out until the caller signs in.

An explicit sign-in ends the automatic sign-in as soon as it starts. It does not
wait for an automatic Register whose reply is late, and that reply cannot
replace the new session. If the explicit sign-in succeeds, it becomes the
automatic sign-in. If it fails, the client does not sign in again as the
previous user.

A committed password change or rename of the signed-in user updates the
credentials of the automatic sign-in. It also updates the builder credentials
that login() uses when they name that user. This holds for typed calls,
sendBinaryRequest and resolveOperation. A rejected change leaves the
credentials unchanged.

Quick Start Copy Chapter

Iggy.tcpClientBuilder() handles the routing and the session for you. Over TCP and TCP/TLS, the client keeps the consumer group membership on the coordinator. It maintains one shared-session connection per advertised replica and multiplexes partition requests on those connections. With a one-node roster, the client dials the configured endpoint instead of the advertised one. This keeps a single node behind a port map or a port-forward reachable.

Start the server with the example prerequisites and matching credentials. The following snippets show alternative clients. Close a blocking client with close() or an async client with close().join() when finished.

TCP Client (Blocking) Copy Section

import org.apache.iggy.Iggy;

// Create and connect with auto-login
var client = Iggy.tcpClientBuilder()
    .blocking()
    .host("localhost")
    .port(8090)
    .credentials("iggy", "iggy")
    .buildAndLogin();

// Or build, connect, and login separately
var client = Iggy.tcpClientBuilder()
    .blocking()
    .host("localhost")
    .port(8090)
    .build();
client.connect();
client.users().login("iggy", "iggy");

Connection String Copy Section

fromConnectionString configures a TCP client from an iggy:// connection string:

// Blocking client
var client = Iggy.tcpClientBuilder()
    .fromConnectionString("iggy://iggy:iggy@localhost:8090")
    .buildAndLogin();

// Async client, with TLS and a custom CA certificate
var asyncClient = Iggy.tcpClientBuilder()
    .async()
    .fromConnectionString("iggy+tcp://admin:[email protected]:8090?tls=true&tls_ca_file=/certs/ca.pem")
    .buildAndLogin()
    .join();

The connection string accepts iggy:// and iggy+tcp:// with : credentials and the
tls and tls_ca_file options, each at most once. tls_ca_file requires tls=true, and a string without tls
keeps the TLS setting already on the builder. Personal access tokens and other options are not supported yet
and are rejected with an IggyInvalidArgumentException.

TCP Client (Async) Copy Section

import org.apache.iggy.Iggy;

// Create async client
var asyncClient = Iggy.tcpClientBuilder()
    .async()
    .host("localhost")
    .port(8090)
    .credentials("iggy", "iggy")
    .buildAndLogin()
    .join();

// Or with manual connect and login
var asyncClient = Iggy.tcpClientBuilder()
    .async()
    .host("localhost")
    .build();
asyncClient.connect().join();
asyncClient.users().login("iggy", "iggy").join();

HTTP Client Copy Section

import org.apache.iggy.Iggy;

// Using URL
var httpClient = Iggy.httpClientBuilder()
    .blocking()
    .url("http://localhost:3000")
    .credentials("iggy", "iggy")
    .buildAndLogin();

// Using host/port
var httpClient = Iggy.httpClientBuilder()
    .blocking()
    .host("localhost")
    .port(3000)
    .credentials("iggy", "iggy")
    .buildAndLogin();

TLS Support Copy Section

Both TCP and HTTP clients support TLS:

// TCP with TLS
var secureClient = Iggy.tcpClientBuilder()
    .blocking()
    .host("iggy-server.example.com")
    .port(8090)
    .enableTls()
    .tlsCertificate("/path/to/ca.pem")  // Optional custom CA
    .credentials("admin", "secret")
    .buildAndLogin();

// HTTPS
var secureHttpClient = Iggy.httpClientBuilder()
    .blocking()
    .host("iggy-server.example.com")
    .port(443)
    .enableTls()
    .credentials("admin", "secret")
    .buildAndLogin();

Builder Options Copy Section

The client builders support additional configuration:

var client = Iggy.tcpClientBuilder()
    .blocking()
    .host("localhost")
    .port(8090)
    .connectionTimeout(Duration.ofSeconds(10))
    .requestTimeout(Duration.ofSeconds(30))
    .retryPolicy(RetryPolicy.exponentialBackoff())
    .credentials("iggy", "iggy")
    .buildAndLogin();

Event Loop Threads Copy Section

Each TCP client shares one event loop group across its replica connections. By default
the group has one thread. An application that opens many clients can instead register them all on
one caller-owned group. The clients never shut that group down. Close the clients first,
then shut the group down:

var group = new MultiThreadIoEventLoopGroup(2, NioIoHandler.newFactory());

var producer = Iggy.tcpClientBuilder()
    .blocking()
    .eventLoopGroup(group)
    .credentials("iggy", "iggy")
    .buildAndLogin();
var consumer = Iggy.tcpClientBuilder()
    .blocking()
    .eventLoopGroup(group)
    .credentials("iggy", "iggy")
    .buildAndLogin();

// ... later
producer.close();
consumer.close();
group.shutdownGracefully();

Do not block in a completion callback. Callbacks run on the group's loops, so a blocked
callback stalls every client that shares the group.

Version Information Copy Section

// Get SDK version
String version = Iggy.version();  // e.g., "0.9.0"

// Get detailed version info
IggyVersion info = Iggy.versionInfo();
info.getVersion();     // Version string
info.getBuildTime();   // Build timestamp
info.getGitCommit();   // Git commit hash
info.getUserAgent();   // User-Agent string for HTTP

Exception Handling Copy Chapter

The SDK's custom exception types inherit from IggyException. When you join a failed future, the cause can come back inside a CompletionException. The HTTP client's close() method declares IOException. Handle these two boundaries as well as the specific SDK errors.

Retained TCP operations Copy Section

Replicated mutations serialize within their metadata or partition lane. If a sent mutation's
outcome cannot be established, IggyOperationException carries
IggyErrorCode.OPERATION_UNCERTAIN and its exact operation ID. A later mutation in the same
lane returns LANE_BLOCKED with that ID. Unrelated partitions and pure reads continue.

Only a reply that refuses a sent mutation, or a committed result for it, settles it. A lost
connection, an eviction or a malformed reply leaves it uncertain. After a lost connection, a
stale-client eviction or a malformed reply, the call replays the same request on a new
connection within its deadline. If the server refuses to bind the session again, the client
registers a new session and drops the operation.

An eviction and a reply can carry the same status, for example 30 (STALE_CLIENT) or 40
(UNAUTHENTICATED). Only the eviction is a lost connection, so the call does not replay a
request that a reply refused. A refused first attempt ends with that error. After an earlier
sent attempt without a proved reply, the call ends with OPERATION_UNCERTAIN.

Both blocking and async TCP clients expose resolveOperation(operationId, timeout) and
abandonOperation(operationId). Resolve reuses the original request bytes and returns the
response body, or the proved business error. A resolve that finds a replay of the same operation
running waits for it within its own timeout. If that wait runs out, the resolve fails with
OPERATION_UNCERTAIN. Java long preserves all 64 operation-ID bits, including negative values.
When you display an ID, use Long.toUnsignedString(operationId).

The client keeps the proved outcomes of its last replays, up to the operation count of
.pendingOperations. A later resolve returns the same outcome without a request, also after a
session reset. Abandon releases local retention and frees the pending operation slot at once. It
does not cancel or undo server work. A running replay keeps the lane until the replay ends, so a
later mutation in that lane waits for it. That replay still keeps the outcome that it proves.
OPERATION_NOT_RETAINED provides no evidence that the original mutation was uncommitted. It
means that a session reset or an abandon came before a proved outcome, or that the proved outcome
aged out. Do not resend it automatically.

The TCP builders accept .pendingOperations(count, bytes), defaulting to 256 mutations
and 64 MiB across active and retained frames. Admission waits within the call's existing
deadline; an oversized frame fails locally. Cancellation removes only the caller's waiter.
A mutation already sent may remain retained. The byte limit counts admitted frozen frames and
writer references; API input snapshots and queued serialization buffers are outside this limit.

Auto-commit and Next polls use a temporary partition ordering barrier. Their success
acknowledges local admission, not replicated offset durability. Code 57, a lost or malformed
reply, or an eviction ends the call with code 57. Progress is unknown, and the call
releases the barrier. These polls are never retained or replayed after uncertainty. Explicit non-admission
(58 or 89) may retry within the same deadline.

Examples Copy Chapter

See the Java Examples directory for runnable applications demonstrating the SDK:

• GettingStartedProducer: synchronous message production with batch sending
• GettingStartedConsumer: synchronous consumption with polling loops
• AsyncProducer: non-blocking batch production with concurrent request submission
• AsyncConsumer: async consumption with backpressure and error recovery

The examples README describes blocking and async clients, CompletableFuture patterns, and thread pool management.

For Apache Flink integration, see the Flink Connector Library.

Building from Source Copy Chapter

This project uses the Gradle Wrapper. Due to Apache Software Foundation policy, the gradle-wrapper.jar binary is not checked into the repository. Instead, the gradlew script automatically downloads it on first run.

# Build the project
./gradlew build

# Run tests
./gradlew test

The wrapper script will:

1. Download gradle-wrapper.jar from the official Gradle repository if missing
2. Verify the SHA256 checksum for security
3. Execute the requested Gradle command

No manual Gradle installation is required.

Note: Only the Unix shell wrapper (gradlew) is provided. On Windows, use WSL or Git Bash, or install Gradle manually.

Contributing Copy Chapter

Before opening a pull request:

1. Format code: ./gradlew spotlessApply
2. Validate build: ./gradlew check
3. Use AssertJ for assertions: Write test assertions with AssertJ (assertThat(...)) instead of JUnit assertions.

These steps keep the code style compliant and make sure that all tests and checkstyle validations pass.

Overview (`foreign/node/README.md`) Copy Section

Apache Iggy Node.js Client

Apache Iggy Node.js client written in typescript, it currently only supports tcp & tls transports.

Disclaimer: although all iggy commands & basic client/stream are implemented this is still a WIP, provided as is, and has still a long way to go to be considered "battle tested".

note: This lib started as iggy-bin ( github / npm) before migrating under iggy-rs org. package [email protected] is equivalent to @iggy.rs/[email protected] and migrating again under apache iggy monorepo ( github and is now published on npmjs as apache-iggy

note: previous works on node.js http client has been moved to iggy-node-http-client (moved on 04 July 2024)

install Copy Chapter

npm i --save apache-iggy@edge

Use matching SDK and server builds from the coordinated VSR release. Older builds are incompatible. This release requires fresh server storage (IGGY-BOUND-GRANTS-4). Existing incompatible data is refused unchanged. For a local server with the example credentials, follow the example setup.

basic usage Copy Chapter

Response frame limit Copy Section

Compatibility note: response frames larger than maxResponseFrameSize (default 64 MiB) are rejected and close the connection. Raise the limit in the client configuration when polling very large batches.

VSR framing Copy Section

The SDK speaks the VSR wire protocol exclusively and requires an Iggy VSR
server:

import { SimpleClient, getRawClient } from "apache-iggy";

const config = {
  transport: "TCP" as const,
  options: { host: "127.0.0.1", port: 8090 },
  credentials: { username: "iggy", password: "iggy" },
};
const client = new SimpleClient(getRawClient(config));
const stats = await client.system.getStats();

Codes absent from the SDK command table use Operation::NonReplicated and
carry the command code in the request header's reserved field. The server
remains authoritative for classifying or rejecting extension commands.

Sends always carry an explicit partition id. The client resolves
Partitioning.Balanced round-robin and Partitioning.MessageKey as XXH32
(seed 0) of the key modulo the topic's partition count, as the Rust SDK does.
The count is cached per topic. The client reads it again after a refused send,
and after the client deletes a stream or a topic or creates or deletes
partitions. A raw SendMessages command without an explicit partition id
fails with error code 5 before the client sends it.

Partition commands carry a context containing the partition incarnation, owner
generation and metadata operation. Pass the context of an earlier poll reply to
keep an offset write or continuation tied to that history. Otherwise the client
captures the route context. Non-admission retries and uncertain-operation replay
preserve the first captured context, request identity and payload. Route refreshes
change the destination. Lifecycle error 88 retries with a fresh ID and the same
context. A send without caller-supplied context can refresh history once after
error 87; explicit contexts return that error. A new call can capture newer
metadata progress.

Route discovery uses command 105. Sends, polls and offset reads/writes use the
partition primary; other commands use the metadata primary. One logical client
owns a shared session, a monotonic request counter and one multiplexed link per
replica. Healthy links survive primary changes. Each link dials the address
that its node advertises in the cluster roster. With a one-node roster, the
link dials the configured endpoint instead. A single node behind a port
mapping or a port-forward thus stays reachable. The legacy poolSize option
accepts only one logical client; it does not limit replica links or concurrency.
The public Client._pool property has been removed. Applications should use the
client methods directly; the client owns its replica connections.

VSR authentication translates password and personal-access-token login into the
Register handshake. Replacement links bind the existing session. Confirmed
session expiry requires a new Register; an uncertain mutation is never replayed
under that new session. An eviction that ends the session does not prove the
outcome of a mutation that the client already wrote. That call ends with
OperationUncertainError. A stale client eviction drops only its link, because
the server keeps the session. The client binds the same session on a new link,
as after any lost link, so retained operations and group membership stay.
Explicit relogin logs out the previous session first.

A reply with status 30 (StaleClient) says that the server no longer attaches
the link to the session. This happens, for example, after the user was deleted
or deactivated. A 30 refusal before commit fails a first attempt with that
error. After an earlier written attempt, it does not prove the outcome, so that
call ends with OperationUncertainError. The client closes that link and binds
the session on a new link, as after any lost link. A replica that replaced its
client table refuses a live session the same way, so the bind decides. A live
session stays, with its retained operations and group membership. An ended
session refuses the bind, and the client signs in again.

A committed 30 comes from the replicated session table, so that session ended.
It is a proved outcome, like any other committed rejection, and the client
drops the session at once and signs in again.

Mutation recovery and limits Copy Section

Mutations share one FIFO lane per partition history, with a separate metadata
lane. An uncertain mutation remains retained and blocks later mutations in its
lane. OperationUncertainError.operationId and LaneBlockedError.operationId
are exact bigint values. Use resolveOperation(operationId, { deadline }) to
replay the retained frame, where deadline is an absolute Date.now() timestamp
in milliseconds. The default operation budget is 30 seconds, including admission,
routing, link waits and retries.

Resolution requires correlated committed-result proof. A proved business
rejection returns the original error and releases the lane. An unproved denial
keeps an uncertain operation retained. The client keeps the proved outcomes of
its last maxPendingOperations replays, also across a session reset. If a
resolver already proved the outcome, a later resolveOperation of the same ID
returns it without a request. A resolveOperation call that waits for another
replay of the same ID and reaches its deadline fails with
OperationUncertainError.

abandonOperation(operationId) releases the client lane without proving that
the mutation failed. A replay that is already running keeps the lane until it
ends, and the client keeps the outcome that it proves. Logout, session reset and
destruction discard retained operations with their outcomes still unknown.
OperationNotRetainedError does not establish the outcome: a reset or an
abandon came before a proved outcome, or the proved outcome aged out.

Breaking change: Client.destroy() aborts in-flight calls immediately and
discards retained operations. It does not wait for outstanding calls to finish.
Await those calls before destroying the client when their results are needed.

Auto-commit and Next polls hold their partition lane only during the active call.
Error 57 or a lost reply ends the call immediately with unknown progress and no
retained operation ID. These polls never replay after uncertainty. Known
non-admission errors 58/89 remain retryable; pure reads may retry lost replies.
Before uncertainty, lifecycle error 88 retries with a fresh ID within the same
deadline. After uncertainty, only committed-result proof can release the lane.

Configure maxPendingOperations (default 256) and maxPendingOperationBytes
(default 64 MiB) on the client. Admission waits before allocating an ID or freezing
a mutation frame. Both limits must be positive. The byte budget includes the
256-byte header and body; smaller positive budgets reject mutations that cannot
fit while reads and login remain available. Oversized frames fail locally.
Completion, abandonment and reset release the operation slot. Writer aliases keep bytes charged until they release
the frame. These limits cover admitted active/retained frames, not caller buffers,
queued API input snapshots, or read/control bodies. sendBinaryRequest preserves
its immediate payload snapshot before awaiting a custom provider.

An established session has its own lifetime. Password or PAT changes and PAT
expiry do not end it. Logout, session lease expiry, or user deactivation end the
session and prevent its bind proof from restoring it.

The client pings every heartbeatInterval milliseconds, 5000 by default, which
keeps an idle session alive when the server's [heartbeat] eviction is enabled.
heartbeatInterval also accepts a duration expression such as "10s" or
"1h 30m", like the Rust SDK.
With the default heartbeat settings, a client that sends nothing for 36 s
(1.2 x the 30 s interval) stops renewing its session, and the server ends the
session about 36 s later. The verifier checks once per interval and also evicts
the connection of a consumer-group member that is silent past 36 s. If the
client interval exceeds 36 s, the verifier evicts an idle group member. If it
exceeds about 72 s, or is 0 to disable client heartbeats, any idle session
expires. Any other unusable value is rejected instead of silently disabling the
heartbeat.

A replica link redials on its own after it drops, with jittered exponential
backoff. reconnect.maxRetries limits one burst of redials, not the life of
the client. While the client stays signed in, each heartbeat restarts a link
that used its budget, with a fresh budget. A request does not redial a link. It
retries within its deadline until the link returns. destroy() and logout stop
the restarts.

When every link used its budget, a request can fail before its deadline with
NotConnectedError. When no endpoint accepts a dial within its retry budget, a
sign-in also fails with NotConnectedError, and its cause holds the last dial
error. In both cases a later call can succeed, because the heartbeat restarts
the links and each sign-in dials again. If reconnection or heartbeats are
disabled, nothing restarts a link that used its budget, so the request fails
with a plain Error instead.

When a poll fails with status 30, 40, 57, 58 or 89, singleConsumerStream and
groupConsumerStream poll again after interval. They also poll again after a
poll deadline, after a poll with unknown progress and after
NotConnectedError. The client binds the session again, signs in again,
reconnects or routes again after these on its own. Any other error ends the
stream. If the reply of an auto-commit poll is lost, the stream can skip that
batch.

singleConsumerStream and groupConsumerStream sign in before they return the
stream, and groupConsumerStream also joins its group. If the sign-in or the
join fails, the call destroys its client and rejects with that error. If no
endpoint accepts a dial within the retry budget, the error is
NotConnectedError. If the 30 second sign-in deadline ends first, the error is
VsrResponseTimeoutError. With the default reconnect values, the deadline
usually ends first. When a stream closes, it destroys its client.

sendBinaryRequest(code, payload) sends an arbitrary command code. Known replicated commands use their registered operation, while unknown codes reach the server as non-replicated requests and are rejected by servers that do not register them.

import { ResponseError } from "apache-iggy";

try {
  await client.sendBinaryRequest(60_000, Buffer.from("opaque request"));
} catch (error) {
  if (error instanceof ResponseError) {
    console.error(error.commandCode, error.errorCode);
  }
}

The client includes its npm package version and the binary wire protocol
version in VSR registration. An incompatible server rejects registration with
a protocol-version error instead of accepting a mismatched wire contract.

import { Client } from "apache-iggy";

const credentials = { username: "iggy", password: "iggy" };

const client = new Client({
  transport: "TCP",
  options: { port: 8090, host: "127.0.0.1" },
  credentials,
});

const stats = await client.system.getStats();

Connection strings Copy Section

Every client constructor (except SimpleClient see note) also accepts a
connection string instead of a config object:

import { Client } from "apache-iggy";

const client = new Client("iggy://iggy:[email protected]:8090");
const stats = await client.system.getStats();

Supported schemes are iggy:// (TCP, default) and iggy+tcp://. Credentials
are username:password or a single personal access token. Options mirror the
other SDKs: tls, tls_domain, tls_ca_file, reconnection_retries,
reconnection_interval, heartbeat_interval, nodelay, max_pending_operations
and max_pending_operation_bytes. reestablish_after
is accepted for format compatibility but has no Node equivalent.

note: SimpleClient does not accept a connection string: it wraps an existing
RawClient instance rather than building one from configuration. Pass the
connection string to Client, SingleClient or getRawClient and hand the
resulting raw client to SimpleClient if needed.

option limits Copy Section

Durations accept the same expressions as the Rust SDK, for example 500ms,
10s, 1h 30m, 5d, 2w, 1y; matching is case-insensitive and
0, unlimited, disabled and none map to zero. Unit-less numbers such as
5 are rejected.

use sources Copy Chapter

Cluster polling uses command 15 (BindSession), route discovery 105 and poll
100. Commands 14, 103, 104 and 123 are retired. Pause consumers for the coordinated
server/SDK upgrade and restart them to rejoin their groups. There is no fallback
to retired polling commands.

Install Copy Section

npm ci

build Copy Section

npm run build

test Copy Section

note: use env var IGGY_TCP_ADDRESS="host:port" to set the server
address for e2e tests. bdd tests need more variables, see below.

unit tests

npm run test:unit

e2e tests

e2e test expect an iggy-server at tcp://127.0.0.1:8090

npm run test:e2e

bdd tests

the bdd suite has no defaults and fails when IGGY_TCP_ADDRESS,
IGGY_ROOT_USERNAME or IGGY_ROOT_PASSWORD is missing. from the repository
root run

./scripts/run-bdd-tests.sh node

the script starts the server and sets every variable, so none of them have to be
exported by hand. to iterate against a server you started yourself, see
src/bdd/README.md.

run all test

npm run test runs unit, bdd and e2e tests suite against an iggy-server at
tcp://127.0.0.1:8090, started with the same root credentials

IGGY_TCP_ADDRESS=127.0.0.1:8090 IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy \
  npm run test

lint Copy Section

npm run lint

Overview (`foreign/php/README.md`) Copy Section

iggy-php

PHP extension bindings for Apache Iggy, built in Rust with
ext-php-rs.

This repository is experimental. The Rust Iggy SDK is async and Tokio-based, but
this extension exposes Iggy\Client as a blocking synchronous PHP API. Each call
drives the lazy global Tokio runtime and blocks the calling PHP thread until the
future resolves; it does not provide fiber-aware or non-blocking I/O.

Requirements Copy Chapter

• Rust and Cargo
• PHP 8.3 or newer with php-config (non-thread-safe builds only, no ZTS)
• cargo-php
• Composer, for installing PHPUnit
• Docker, for running the integration test server

On macOS with Homebrew PHP:

export PATH="/opt/homebrew/opt/php/bin:$PATH"
export PHP=/opt/homebrew/opt/php/bin/php
export PHP_CONFIG=/opt/homebrew/opt/php/bin/php-config

Build Copy Chapter

cargo build --release

Generate IDE stubs after changing the exported PHP API:

cargo build
cargo php stubs target/debug/libiggy_php.so -o /tmp/iggy-php.stubs.php

Stub generation requires a debug build. On macOS, use libiggy_php.dylib.
Preserve the existing Apache license header when updating iggy-php.stubs.php.
The CI lint job regenerates this file and fails if the checked-in stubs drift
from the Rust signatures.

Install Copy Chapter

cargo php install --release --yes

If the extension is already enabled, reinstall it with:

cargo php remove --yes
cargo php install --release --yes

Verify PHP can load it:

php -r 'var_dump(extension_loaded("iggy-php"));'

Run Iggy Copy Chapter

Use Iggy 0.9.0. When testing unreleased SDK changes, build the server from the
same source checkout.

docker run --rm --name iggy-php-test \
  --cap-add=SYS_NICE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
  -p 8090:8090 \
  -p 3000:3000 \
  -e IGGY_TCP_ADDRESS=0.0.0.0:8090 -e IGGY_HTTP_ADDRESS=0.0.0.0:3000 \
  -e IGGY_NODE_ADVERTISED_ADDRESS=localhost \
  -e IGGY_ROOT_USERNAME=iggy -e IGGY_ROOT_PASSWORD=iggy \
  apache/iggy:0.9.0

You can also run a local server from the repository root:

cargo run --bin iggy-server -- --fresh --with-default-root-credentials

The root variables bootstrap a new data directory; they do not replace stored
credentials. Environment credentials override --with-default-root-credentials.
Use --fresh only with disposable development data: it deletes local replica state.

The tests assume:

• host: 127.0.0.1
• port: 8090
• username: iggy
• password: iggy

Override them with IGGY_TCP_ADDRESS, IGGY_ROOT_USERNAME, and IGGY_ROOT_PASSWORD.

Usage Copy Chapter

<?php

$client = new \Iggy\Client('127.0.0.1:8090');
$client->connect();
$client->loginUser('iggy', 'iggy');

$stream = 'php-stream';
$topic = 'php-topic';
$partitionId = 0;

$client->createStream($stream);
$client->createTopic($stream, $topic, 1, null, null, null, null);

$client->sendMessages($stream, $topic, $partitionId, [
    new \Iggy\SendMessage('hello from PHP 1'),
    new \Iggy\SendMessage('hello from PHP 2'),
]);

$messages = $client->pollMessages(
    $stream,
    $topic,
    $partitionId,
    \Iggy\PollingStrategy::first(),
    10,
    true,
);

foreach ($messages as $message) {
    echo $message->payload(), PHP_EOL;
}

Consumer group callbacks require a finite message limit. The partition id
argument is ignored for a consumer group, since the member reads the partitions
the server assigns to it. After the example above, each loop below consumes one
of its two messages:

<?php

$consumer = $client->consumerGroup(
    'php-consumer',
    $stream,
    $topic,
    null,
    \Iggy\PollingStrategy::next(),
    10,
    \Iggy\AutoCommit::disabled(),
    true,
    true,
    1_000_000,
    null,
    null,
    null,
    false,
);

$consumer->consumeMessages(
    function (\Iggy\ReceiveMessage $message) use ($consumer): void {
        echo $message->payload(), PHP_EOL;
        $consumer->storePosition($message->position());
    },
    1,
);

foreach ($consumer->iterMessages() as $message) {
    echo $message->payload(), PHP_EOL;
    $consumer->storePosition($message->position());

    break;
}

Tests Copy Chapter

Run the Dockerized integration suite:

docker compose -f docker-compose.test.yml up --build --abort-on-container-exit --exit-code-from php-tests

Run the PHP test suite:

composer install
composer test

Run Rust verification with a matching PHP embedding library (libphp) available
to the linker and dynamic loader:

cargo test --features ext-php-rs/embed

TLS tests are opt-in because they require a TLS-enabled Iggy server and certificate
setup. Set IGGY_TLS_CONNECTION_STRING to enable TLS connection tests. Set
IGGY_TLS_PLAINTEXT_ADDRESS to run the negative plaintext-to-TLS test.

TLS connection strings use the Rust SDK connection-string format, for example:

iggy+tcp://iggy:[email protected]:8090?tls=true&tls_domain=localhost&tls_ca_file=/path/to/ca.pem

API Notes Copy Chapter

• new Iggy\Client($address, new Iggy\PendingOperationsConfig(256, 64 * 1024 * 1024))
  configures client-wide active/retained mutation limits. Both limits must be positive;
  omission uses 256 operations and 64 MiB of frozen headers and bodies. Admission waits
  within the operation deadline before assigning an ID. Queued writer references keep
  bytes charged after abandonment or reset until their final owner releases the frame.
  TCP, QUIC and WebSocket connection strings accept max_pending_operations and
  max_pending_operation_bytes, for example
  iggy+quic://iggy:iggy@localhost:8080?max_pending_operations=256&max_pending_operation_bytes=67108864.
• Iggy\Exception\PendingOperationException preserves the read-only operation_id
  decimal string and kind (uncertain, lane_blocked, or not_retained).
  getCode() returns the SDK error code. Keep IDs as strings: PHP integers cannot
  represent the full unsigned 64-bit range. lane_blocked identifies an earlier
  operation, and the blocked call was not sent. not_retained does not prove
  that the earlier operation failed to commit.
• Iggy\Client::resolveOperation(string $operation_id, ?int $timeout_micros = null)
  replays that retained operation, with a positive timeout (30 seconds by default).
  A proved committed business rejection raises its original exception and frees the
  lane. An uncertain result keeps the operation retained for another resolution attempt.
  abandonOperation(string $operation_id) releases it without proving whether it
  committed. Both methods reject malformed or overflowing IDs. HTTP does not support
  retained-operation recovery. A session reset, disconnect or shutdown discards
  retained operations. The client keeps the proved outcomes of its last
  max_pending_operations replays, also across a session reset. A later
  resolveOperation() of a proved operation returns the same outcome without a request.
• Methods are exposed to PHP as camelCase, for example createStream() and
  pollMessages().
• Classes live in the Iggy namespace, for example Iggy\Client and
  Iggy\SendMessage.
• Partition IDs use the Iggy partition index. For a topic with one partition, use 0.
• Passing null as the partition to storeOffset() or deleteOffset() uses the
  current consumer partition, and is rejected until at least one message has been
  polled. Pass an explicit partition id before the first poll.
• consumeMessages() requires an explicit finite limit. It does not run forever
  by default.
• iterMessages() returns a PHP Iterator and can be used with foreach.
  Break out of the loop when the caller's processing limit or shutdown signal is reached.
• AutoCommit::when() may queue an offset commit before the PHP callback runs.
  If callback success must control commits, use AutoCommit::disabled() and call
  storePosition($message->position()) after the callback work succeeds.
• Every received message keeps the partition context of the poll that delivered it:
  the partition incarnation, which changes when the partition is deleted and recreated,
  and the consumer group owner generation. Iggy\Consumer::storePosition() commits a
  message's position() under that context, so a commit after a recreation or an owner
  change is refused instead of landing on the newer partition. storeOffset() commits
  under the context of the latest message consumed from that partition.
• Iggy\PollingStrategy::withContext() continues a poll at an offset under the context
  that produced it, for example
  PollingStrategy::offset($message->offset() + 1)->withContext($message->context()).
  A stale context fails the poll. To continue in another process, save the context's
  incarnation, owner_generation and metadata_op with the offset and rebuild it with
  new Iggy\PartitionContext(...).
• A poll or offset write without a caller context uses the context of its route. It can
  fail once with HistoryUnavailable (87), or ConsumerGroupPartitionNotOwned (5009)
  for a consumer group, after another client deleted and recreated the partition. The
  failed route is dropped, and the next call routes again. Both errors surface as
  Iggy\Exception\IggyException.
• Iggy\PollingStrategy::timestamp() and Iggy\PollingStrategy::timestampMicros()
  expect microseconds since the Unix epoch. Use
  Iggy\PollingStrategy::timestampSeconds() for PHP time() values.
• PHP strings are passed as named identifiers, including strings that contain
  only digits. PHP integers are passed as numeric identifiers.
• Iggy\SendMessage::payload and Iggy\ReceiveMessage::payload() copy the payload
  bytes into a PHP string on each read. Cache large payloads in PHP if they will
  be read repeatedly.
• Message IDs and checksums are returned as decimal strings. Offset and timestamp
  getters return PHP integers and are limited to PHP_INT_MAX.
• Iggy\Client::sendBinaryRequest(int $code, string $payload): string sends a
  command code and payload and returns the raw response body.
• Iggy\Client is synchronous and blocks the current PHP thread.
• The extension owns a lazy global Tokio runtime. Do not call pcntl_fork() after
  the first Iggy SDK call; the child process inherits file descriptors but not
  Tokio worker threads. Runtime initialization failure is unrecoverable and aborts
  extension use.

Overview (`foreign/python/README.md`) Copy Section

apache-iggy

Apache Iggy is the persistent message streaming platform written in Rust, supporting QUIC, TCP and HTTP transport protocols, capable of processing millions of messages per second.

Installation Copy Chapter

Basic Installation Copy Section

# Using uv in an existing project
uv add apache-iggy

# Using pip
python3 -m venv .venv
source .venv/bin/activate
pip install apache-iggy

Prerequisites Copy Section

• Python 3.11+

Published wheels include the Rust extension; installing a wheel does not require
Rust. Building from source and running the development checks below also requires:

• Rust toolchain: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
• uv: curl -LsSf https://astral.sh/uv/install.sh | sh
• All checks tooling from CONTRIBUTING.md.
• Docker

Use an SDK release compatible with your server. For unreleased changes, build
the SDK and server from the same source checkout.

Local Development Copy Section

IMPORTANT: All commands are supposed to be ran from foreign/python unless it's specified to run in repository's root folder.

1. Build a project for development

   With uv:

   bash Copy

   # Create a venv
   uv venv
   
   # Sync the environment without updating it
   uv sync --frozen --all-extras --no-install-project
   
   # Build the project -- this builds the rust extension into the venv (debug profile) - re-run after any rust change
   uv run --no-sync maturin develop

   With pip:

   bash Copy

   # Create a venv
   python3 -m venv .venv
   
   # Activate the venv
   source .venv/bin/activate
   
   # Install the dependencies
   pip install -e ".[all]"
   
   # Build the project -- this builds the rust extension into the venv (debug profile) - re-run after any rust change
   maturin develop

2. Run the server to be able to run the tests (this blocks the terminal - run steps 3-5 in a separate one). --fresh deletes local_data/ on every run - drop it if you have existing data you want to keep.

   bash Copy

   # run from the repository's root directory
   cargo run --bin iggy-server -- --with-default-root-credentials --fresh

3. Run the tests

   uv:

   bash Copy

   uv run --no-sync pytest tests/ -v

   pip:

   bash Copy

   pytest tests/ -v # make sure iggy-server is running and the venv is activated

4. To update the stubs, after changing the pyo3 API surface, use

   bash Copy

   # run from foreign/python
   cargo run --bin stub_gen

5. Before committing, test the pre-commit and pre-push hooks. prek only inspects staged content, so stage your work first:

   bash Copy

   git add -A
   prek run # runs pre-commit hooks
   prek run --hook-stage pre-push
   # if a hook modifies files, re-run `git add -A` and `prek run`.

   These are some of the essential commands prek is running, so it's recommended to run them manually before
   running prek / committing / pushing. This list is not exhaustive and other hook failures are possible.

   bash Copy

   uv run --no-sync ruff format .

   bash Copy

   uv run --no-sync ruff check --fix .

   bash Copy

   cargo fmt --manifest-path Cargo.toml

   bash Copy

   cargo clippy --manifest-path Cargo.toml --all-targets --all-features -- -D warnings

   bash Copy

   # run from the repository's root directory
   ./scripts/ci/markdownlint.sh --fix foreign/python/README.md # read the diff after applying this, sometimes it gives unwanted results, e.g. messing up enumerations

Client Configuration Copy Chapter

IggyClient takes a server address, a TcpConfig, a QuicConfig, an
HttpConfig, or a WebSocketConfig:

import asyncio
from datetime import timedelta

from apache_iggy import AutoLogin, IggyClient, TcpConfig, TcpReconnectionConfig


async def main():
    client = IggyClient(
        TcpConfig(
            server_address="127.0.0.1:8090",
            auto_login=AutoLogin.username_password("iggy", "iggy"),
            reconnection=TcpReconnectionConfig(
                enabled=True,
                max_retries=10,
                interval=timedelta(seconds=2),
                reestablish_after=timedelta(seconds=30),
            ),
            heartbeat_interval=timedelta(seconds=5),
            # tls_enabled=True,
            # tls_domain="localhost",
            # tls_ca_file="../../core/certs/iggy_ca_cert.pem",
            # tls_validate_certificate=True,
            # nodelay=True,
        )
    )
    await client.connect()


asyncio.run(main())

IggyClient(...) also accepts a QuicConfig for the QUIC transport, an
HttpConfig for the HTTP transport, and a WebSocketConfig for the WebSocket
transport. examples/python/getting-started/producer.py shows each swap in
context.

Established binary sessions survive transport loss. Password/PAT changes and
PAT expiry do not end these sessions; logout, session lease expiry and account
deactivation do. Reconnection binds the retained session before attempting a
fresh login.

HttpConfig differs from TCP in two ways. There is no reconnection policy and no
AutoLogin: connect() does not dial over HTTP, but it does start the
heartbeat that heartbeat_interval configures, so call it and then
login_user(...). And HTTP is single-consumer only: the consumer_group(...)
path always fails with Feature is unavailable, at the join by default and at
the returned consumer's first poll if you disable auto_join_consumer_group,
so disabling it is not a workaround. A direct
poll_messages(consumer=Consumer.Group(...)) fails the same way unless you
pass an explicit partition_id, and with one it degrades silently instead: the
consumer kind is not carried on the HTTP wire, so the group is served as an
ordinary consumer named after it, with no membership or partition assignment
behind it. Use Consumer.Single(...) with poll_messages(...). Delivery is
also at-least-once: the default retries=3 replays the full request body, so a
send whose response was lost is applied twice, and only retries=0 opts out.

import asyncio

from apache_iggy import HttpConfig, IggyClient


async def main():
    client = IggyClient(HttpConfig(api_url="http://127.0.0.1:3000"))
    await client.connect()
    await client.login_user("iggy", "iggy")


asyncio.run(main())

High-Level Producer Copy Chapter

The Python high-level producer API is a port of the Rust high-level producer
API. For detailed producer semantics and configuration guidance, see the
Rust high-level SDK documentation.

Use IggyClient.producer() when an application repeatedly publishes to one
stream and topic. Producer creation is asynchronous because it initializes the
destination before returning. By default, it creates a missing stream and topic,
uses balanced partitioning and sends directly in batches of up to 1,000 messages.
The binary driver retries each chunk within one 30-second deadline, preserving
its request identity. The producer adds no retry under a new identity.

The default mode is direct. Pass BackgroundProducerConfig to queue sends on
background workers instead.

import asyncio
from datetime import timedelta

from apache_iggy import DirectProducerConfig, IggyClient, Partitioning, SendMessage


async def main():
    client = IggyClient.from_connection_string("iggy+tcp://iggy:[email protected]:8090")
    await client.connect()

    producer = await client.producer(
        "orders",
        "created",
        partitioning=Partitioning.balanced(),
        mode=DirectProducerConfig(
            batch_length=500,
            linger_time=timedelta(milliseconds=5),
        ),
        create_stream_if_not_exists=True,
        create_topic_if_not_exists=True,
        topic_partitions_count=3,
        topic_message_expiry=None,
        topic_max_size=None,
    )

    async with producer:
        await producer.send_one(SendMessage("order-1"))
        response = await producer.send([SendMessage("order-2"), SendMessage("order-3")])
        print(f"Received {len(response.confirmations)} partition confirmations")


asyncio.run(main())

The producer is bound to the stream and topic passed to producer(). Use
send_with_partitioning(messages, partitioning) to override its partitioning
strategy for one call, or send_to(stream, topic, messages, partitioning) to
send to another existing destination. send_to() does not create or initialize
that destination.

Binary transports retain the original request identity after an uncertain
exchange. Persisted sends also provide crash-safe retry receipts; weaker topic
policies can lose data and receipts on a crash. HTTP NoAck confirms dispatch
only. An application resend creates a new request and can duplicate messages.

Mutation retries belong to the shared client driver. The producer no longer accepts
send_retries or send_retry_interval; passing either raises TypeError. HTTP
keeps its configured HTTP retry policy and does not support operation recovery.

PendingOperationError exposes the exact integer operation_id, the numeric
code, and kind ("uncertain", "lane_blocked", or "not_retained"). On the
same live client, await client.resolve_operation(operation_id, timeout=timedelta(seconds=30)) replays the original operation. A committed
business rejection raises its original error and releases the lane; uncertainty
keeps the operation retained. The default timeout is 30 seconds; it must be a
positive timedelta. await client.abandon_operation(operation_id) releases the
lane without establishing whether the operation committed. A session reset
forgets retained operations, so recovery must use the original session.
The client keeps the proved outcomes of its last max_operations replays, also
across a session reset. A later resolve_operation of a proved operation
returns the same outcome without a request.

TcpConfig, QuicConfig, and WebSocketConfig accept pending_operations= PendingOperationsConfig(max_operations=256, max_bytes=64 * 1024 * 1024). These
client-wide defaults bound active and retained mutations across every partition.
Admission waits before allocating a request ID. Queued writer frames keep their
bytes charged until their last reference is released, even after abandonment.
TCP, QUIC and WebSocket connection strings accept max_pending_operations and
max_pending_operation_bytes as positive integers, for example
iggy+tcp://iggy:iggy@localhost:8090?max_pending_operations=256&max_pending_operation_bytes=67108864.

A failed direct send raises ProducerSendError, which is a RuntimeError
subclass. Its cause property contains the underlying error text, and its
__cause__ preserves a structured PendingOperationError when applicable. The
optional operation_id identifies the retained send to resolve or abandon. Its
committed property contains confirmations from completed chunks, and failed
contains the remaining messages without usable confirmations. For an uncertain
cause, the first chunk of failed may already have committed. Resolve its
operation_id without submitting that chunk as a new send. For lane_blocked,
none of failed was sent: the ID names an earlier operation to resolve before
retrying. Abandonment or a session reset discards retained results, and
not_retained does not prove non-commit. Resubmitting an uncertain chunk can
duplicate messages. Before any safe resubmission, restore payloads changed by
encryption.

Background Mode Copy Section

A successful background send means the dispatcher accepted the messages. The
worker writes them later, so all four send methods return a
SendMessagesResponse with an empty confirmations list. It does not mean the
server has committed the messages.

from datetime import timedelta

from apache_iggy import (
    BackgroundProducerConfig,
    BackpressureMode,
    ProducerSharding,
    SendMessage,
)

producer = await client.producer(
    "orders",
    "created",
    mode=BackgroundProducerConfig(
        num_shards=4,
        linger_time=timedelta(milliseconds=10),
        batch_size=1024 * 1024,
        batch_length=100,
        max_buffer_size=32 * 1024 * 1024,
        failure_mode=BackpressureMode.block_with_timeout(timedelta(seconds=1)),
        max_in_flight=4,
        sharding=ProducerSharding.ORDERED,
    ),
)

async with producer:
    accepted = await producer.send_one(SendMessage("order-1"))
    assert accepted.confirmations == []

Each shard flushes when any configured condition is met: batch_length queued
send calls, batch_size reported bytes, or linger_time since the first send
entered an empty buffer. batch_length counts calls, not individual messages.
Zero disables either batching threshold, while a zero linger flushes as soon as
the worker receives a send.

ProducerSharding.ORDERED hashes the stream/topic destination so its sends use
one sequential worker and retain dispatch order. ProducerSharding.BALANCED
assigns consecutive sends round-robin across the shards for throughput; ordering
for one destination is not guaranteed.

max_buffer_size bounds bytes queued or in flight across the whole producer.
When it is full, BackpressureMode.block() waits indefinitely,
block_with_timeout(duration) waits up to that duration, and
fail_immediately() raises RuntimeError without accepting the batch. A single
batch larger than the whole budget always fails. A zero byte budget is
unlimited. max_in_flight separately bounds concurrent write requests across
all shards; zero uses the runtime maximum.

Binary driver retries and transport reconnection happen inside background workers.
The Python API does not currently expose a background error callback, so a write
that fails at its operation deadline is logged by the Rust SDK and its messages
are dropped from the worker queue. An uncertain chunk remains retained by the
client for recovery; it may already have committed.

See the complete runnable
background_producer.py
example for all background configuration fields and deterministic shutdown.

Cleanup is asynchronous and must be explicit. Prefer async with, as above, so
shutdown runs on both successful and exceptional exits. Otherwise, call
await producer.shutdown() in a finally block. Shutdown waits for active
sends, is safe to call more than once, and rejects later sends with
RuntimeError. In background mode it also drains every queue and flushes all
accepted messages before returning. Object destruction does not perform
asynchronous cleanup; dropping a background producer without shutdown() can
lose buffered messages.

Examples Copy Chapter

Refer to the examples/python/ directory for usage examples.

Contributing Copy Chapter

See CONTRIBUTING.md for contribution guidelines.

License Copy Chapter

Licensed under the Apache License 2.0. See LICENSE for details.

Overview (`foreign/swift/README.md`) Copy Section

Swift SDK for Iggy

Official Swift client SDK for Apache Iggy message streaming.

The SDK is built up in stages. The client is a TCP connection with optional TLS that
speaks the consensus protocol, signs in, keeps the session alive with a heartbeat,
reconnects, and follows the cluster leader. It manages streams, topics, partitions,
segments, users, personal access tokens, and consumer groups, sends and polls messages,
and stores consumer offsets. The producer and consumer, examples, and BDD scenarios
follow in later changes.

Swift supports the consensus wire protocol and checks commitment proof during
automatic retries. It does not yet expose retained-operation resolve/abandon APIs,
mutation lanes, or pending-operation limits. Those recovery features in the Rust
and other native SDKs are outside the current Swift implementation.

Requirements Copy Chapter

• Swift 6.0 or later (Swift 6 language mode, strict concurrency)
• macOS 13, iOS 16, tvOS 16, watchOS 9, visionOS 1, or Linux with a Swift 6 toolchain
• An Iggy server that speaks the consensus protocol (iggy-server from this repository)

The SDK depends on SwiftNIO,
swift-nio-ssl, and
swift-log.

Installation Copy Chapter

Swift Package Manager fetches a package from the root of a git repository, so the SDK
cannot be depended on straight from this monorepo's subdirectory. Add it through the
package mirror, which carries every release as a v<version> tag:

// Package.swift
dependencies: [
    .package(url: "https://github.com/apache/iggy-swift.git", from: "0.1.0")
],
targets: [
    .target(name: "MyApp", dependencies: [.product(name: "Iggy", package: "iggy-swift")])
]

From a checkout of this repository, or a vendored copy, depend on it by path:

dependencies: [
    .package(name: "apache-iggy", path: "../iggy/foreign/swift")
],
targets: [
    .target(name: "MyApp", dependencies: [.product(name: "Iggy", package: "apache-iggy")])
]

Usage Copy Chapter

Create a client, connect, and manage resources. With autoLogin set, connect() signs
in and every reconnect restores the session:

import Iggy

let client = IggyClient(address: "localhost:8090", autoLogin: .usernamePassword(username: "iggy", password: "iggy"))
try await client.connect()
let stream = try await client.createStream(name: "orders")
let topic = try await client.createTopic(streamID: "orders", name: "created", options: .init(partitionsCount: 3))
await client.shutdown()

One client owns one connection and is safe to share between tasks: requests are
serialized on the connection. disconnect() closes the connection until the next
connect(), shutdown() is final.

Configuration Copy Section

ClientConfiguration carries the address, TLS, the credentials to sign in with, the
reconnection policy, the heartbeat interval, and the connect and request timeouts:

var configuration = ClientConfiguration(address: "iggy.example:8090")
configuration.autoLogin = .personalAccessToken(token)
configuration.tls = TLSOptions(caFile: "/etc/iggy/ca.pem")
configuration.reconnection = ReconnectionOptions(maxRetries: 5, interval: .seconds(2))
let client = IggyClient(configuration: configuration)

The server certificate is verified against the system roots, or against caFile when
set, and against the host of the address unless TLSOptions.domain names another one.

Reconnection Copy Section

A request that hits a dropped connection reconnects, signs in again, and is replayed,
all within its requestTimeout, and only when it provably never reached the log:
reads, pings, and requests the server never admitted. A replicated write whose outcome
is unknown fails with transientNotCommitted instead of being applied twice. A client
left disconnected by a timeout or an outage reconnects on its next call or heartbeat
when it has credentials to sign in with. In a cluster the client reads the roster after
signing in, moves to the leader, and moves on when a node answers that it is not the
primary.

Messages Copy Section

sendMessages writes a batch to one partition. The server routes explicit partitions
only, so balanced (the default) and messagesKey are resolved on the client:
balanced round-robins over the topic's partitions, and a key is hashed with XXH32 the
way every other SDK hashes it. A message without an id gets a random one, written back
into the batch, so sending the same batch again after transientNotCommitted carries
the same ids. pollMessages reads from a partition, starting where the strategy says:

var batch: [IggyMessage] = ["placed"]
try await client.sendMessages(streamID: "orders", topicID: "created", partitioning: .messagesKey("customer-42"), messages: &batch)
let polled = try await client.pollMessages(
    streamID: "orders", topicID: "created", partitionID: 0, consumer: .consumer(1), strategy: .next, count: 100, autoCommit: true)
for message in polled.messages {
    print(message.offset, message.payloadString)
}

Every partition request carries the partition context the server requires: the
partition's incarnation, which changes when a deleted partition id is reused. The
client reads it from the partition's route and caches it. A poll reply carries the
context it was served under; hand it back with .offset(_:context:) to continue the
same partition, or in a ConsumerPosition to store a checkpoint. The server refuses
either with historyUnavailable once the partition was recreated, and a consumer-group
checkpoint with consumerGroupPartitionNotOwned once the partition moved to another
member.

Consumer groups Copy Section

A member polls without a partition id and the client takes the next partition of its
assignment, round-robin, syncing the assignment from the server when it changes. The
heartbeat syncs every joined group, so a member picks up new partitions on its own:

try await client.joinConsumerGroup(streamID: "orders", topicID: "created", groupID: "billing")
let batch = try await client.pollMessages(
    streamID: "orders", topicID: "created", partitionID: nil, consumer: .group("billing"), strategy: .next, count: 100, autoCommit: true)

A member holding no partitions gets an empty poll whose partition id is
PolledMessages.noAssignedPartition, and a client that is not a member gets
consumerGroupMemberNotFound. Membership belongs to the session. A reconnect resumes
the session and keeps it, but when the server no longer holds the session the client
signs in under a new one and the caller must call joinConsumerGroup again: the next
group poll fails with consumerGroupMemberNotFound until it does. In a cluster, polls
and offset writes go to the partition's primary over a second connection bound to the
same session.

Errors Copy Section

Every operation throws IggyError, a value carrying the typed code
(IggyErrorCode, one case per server error code), the raw wire code, and optional
context. A cancelled task gets CancellationError:

func handle(_ error: IggyError) {
    switch error.code {
    case .streamNameNotFound:
        print("create the stream first")
    default:
        print("iggy failed: \(error)")
    }
}

Testing Copy Chapter

cd foreign/swift
swift test

The client tests run against an in-process server that speaks the consensus framing
and is scripted per test, so they need no running iggy-server.

The end-to-end suite runs against a server at the address in IGGY_TCP_ADDRESS and
skips when that variable is unset. Start one from the repository root:

cargo build --bin iggy-server
IGGY_PATH=/tmp/iggy-swift IGGY_ROOT_USERNAME=iggy IGGY_ROOT_PASSWORD=iggy target/debug/iggy-server

Then, from foreign/swift:

IGGY_TCP_ADDRESS=127.0.0.1:8090 swift test --filter IggyE2ETests

Add IGGY_TCP_TLS_ENABLED=true to run it against a server started with
IGGY_TCP_TLS_ENABLED=true and the certificate pair in core/certs.

The hash, error-code, model, request, reply, and consensus header tests compare against
Tests/IggyTests/Fixtures/golden.json, a fixture dumped from the Rust protocol crates. Regenerate it after a protocol change, from
foreign/swift, and commit the result; CI fails when the checked-in copy is stale:

cargo run --manifest-path Tools/golden-vectors/Cargo.toml -- Tests/IggyTests/Fixtures/golden.json

Contributing Copy Chapter

Format the sources with the toolchain's formatter before opening a pull request; CI runs it
in lint mode and builds with warnings as errors:

swift format format --in-place --recursive Sources Tests Package.swift
swift format lint --strict --recursive Sources Tests Package.swift
swift build -Xswiftc -warnings-as-errors

Overview (`gateways/kafka/README.md`) Copy Section

Kafka gateway (`iggy-gateway-kafka`)

Foundation layer for apache/iggy#3421: a TCP listener on the Kafka wire port that decodes requests and validates scoped API keys and versions. With a bridge, Produce, Fetch, ListOffsets, Metadata, CreateTopics, DeleteTopics, CreatePartitions, DescribeConfigs, AlterConfigs and IncrementalAlterConfigs use Iggy. InitProducerId and consumer group coordination work with or without one.

Stub warning: with IGGY_KAFKA_BRIDGE_ENABLED=true, Produce, Fetch, ListOffsets, Metadata,
CreateTopics, DeleteTopics, CreatePartitions, DescribeConfigs, AlterConfigs and
IncrementalAlterConfigs use Iggy. With the bridge off (the default), those ten are stubs:
Produce, Fetch and ListOffsets answer retriable NOT_LEADER_OR_FOLLOWER (6), CreateTopics,
DeleteTopics, CreatePartitions, DescribeConfigs, AlterConfigs and IncrementalAlterConfigs answer
NOT_CONTROLLER (41), and Metadata reports every requested topic unknown. CreateTopics,
DeleteTopics, CreatePartitions and both alter APIs run as the bridge's own Iggy user: with the
bridge on and IGGY_KAFKA_SASL_ENABLED off (the default), any client that can reach this port
can create topics (up to 1000 partitions each), grow them by up to 1000 partitions per topic per
request (100 topics per request), delete topics and their data, or shorten a topic's
retention.ms low enough that the cleaner drops sealed segments, deleting data. With
IGGY_KAFKA_AUTO_CREATE_TOPICS_ENABLED=true and no listener that names principals, any client
that reaches this port can also create topics just by naming them in a Metadata request. When
the connection has a principal (SASL, or a client certificate mapped to an Iggy user),
DeleteTopics, CreatePartitions, AlterConfigs, IncrementalAlterConfigs and Metadata
auto-creation additionally require it to hold a global manage_topics or manage_streams;
without it every resource of a DeleteTopics, CreatePartitions, AlterConfigs or
IncrementalAlterConfigs request, and every missing topic a Metadata request would create,
fails TOPIC_AUTHORIZATION_FAILED (29) and nothing is written, created or deleted.
CreateTopics has no such gate yet - SASL on does not protect it. Every client reads and writes
as the bridge's Iggy user. Iggy's metadata.partitions_max defaults to 0 (unlimited), so set it
before exposing the gateway. See docs/SCOPE.md.

InitProducerId does real work too, with or without the bridge: it allocates a producer id, so a stock idempotent producer starts instead of failing at startup.

Consumer group coordination is not a stub either: FindCoordinator, JoinGroup, Heartbeat,
LeaveGroup, SyncGroup, DescribeGroups and ListGroups are real, with real membership,
rebalances, graceful leave, session expiry and admin views of that membership
(docs/CONSUMER_GROUPS.md). kafka-consumer-groups.sh --list,
--describe --members and --describe --state use those views. The default --describe
calls OffsetFetch (API key 9), which is not implemented, so that mode does not work. With the bridge off,
Metadata reports every topic unknown, so a consumer joins a group and is assigned 0 partitions.
With the bridge on, OffsetCommit and OffsetFetch keep each group's offsets in Iggy, so a group
resumes from its last commit, also after a gateway restart
(docs/OFFSET_STORAGE.md). Every Iggy server must run a build with the
external group consumer kind (#4392) before the first
commit. In a partly upgraded cluster, one commit can make a partition stop committing. In a
cluster, a node that has not applied a group's first commit answers "no offset", so a consumer on
auto.offset.reset=latest can skip records until
#4409.

Run Copy Chapter

cargo run -p iggy-gateway-kafka

Default bind: a PLAINTEXT listener on 127.0.0.1:9093. Environment variables:

Quick start (Docker Compose) Copy Chapter

The fastest way to see a Kafka client talk to Iggy: two containers, no local Rust toolchain to
build them. (Reading the record back below uses docker compose exec, not a local build either.)

Kafka wire (TCP 9093)              Iggy wire (TCP 8090)
  kcat / librdkafka  ─────────────────────▶  gateway  ─────────────────────▶  iggy-server
  (your Kafka client)                    iggy-gateway-kafka                  (real broker)

cd gateways/kafka
docker compose up --build

iggy-server pulls the prebuilt apache/iggy:edge image (refreshed on a master push only when server or iggy-cli is affected, or web or core/server/Dockerfile changes),
so only iggy-gateway-kafka compiles from source. Its Dockerfile mounts a cargo registry/target
cache, so a first run costs one real compile and every run after reuses it, even across a source
change elsewhere in the workspace. Testing a local iggy-server change: comment out image: and
uncomment build: in docker-compose.yml first. That build mounts cargo registry, git, and
target caches, so the first run is still a full release build of several minutes and later runs
reuse those mounts.

This starts iggy-server (the real Iggy broker) and iggy-gateway-kafka (bridge enabled,
pointed at that server) on a shared Docker network, with the gateway waiting for the server's
health check before it starts. From another terminal:

# Broker discovery works with just kcat - it has no create-topic mode of its own, though.
kcat -b 127.0.0.1:9093 -L

A full create-topic-then-produce flow needs a client that can send CreateTopics, unless
IGGY_KAFKA_AUTO_CREATE_TOPICS_ENABLED=true (see Auto topic creation). With Python's kafka-python (illustrative - not exercised by an
automated test, so adjust if the library's API has moved since this was written):

from kafka.admin import KafkaAdminClient, NewTopic
from kafka import KafkaProducer

admin = KafkaAdminClient(bootstrap_servers="127.0.0.1:9093")
admin.create_topics([NewTopic(name="orders", num_partitions=1, replication_factor=1)])

producer = KafkaProducer(bootstrap_servers="127.0.0.1:9093")
producer.send("orders", b"hello from kafka-python").get(timeout=5)

Read the record back with a real Kafka consumer - Fetch is wired to Iggy with the bridge on (see
the stub warning above), so kcat reads it the same way it would against a real broker:

kcat -b 127.0.0.1:9093 -C -t orders -o beginning -e

-e exits once kcat catches up rather than waiting for more. No consumer group needed for a
one-shot read; see docs/CONSUMER_GROUPS.md for group-based consuming.
Polling Iggy directly still works too, if you want to see the record from the other side:

docker compose exec iggy-server /usr/local/bin/iggy --username iggy \
  --password iggy-gateway-quickstart message poll kafka orders 0 --offset 0 --message-count 1

Ports: gateway 9093 (Kafka wire, the one a client dials), Iggy 8090 (TCP, iggy CLI/SDK),
3000/8080/8092 (Iggy HTTP/QUIC/WebSocket, unused by the gateway itself but published for
direct Iggy inspection).

Config: the compose file sets fixed dev-only root credentials (IGGY_ROOT_PASSWORD) so the
gateway's IGGY_KAFKA_IGGY_PASSWORD is known ahead of time - never reuse them outside a local
stack. See gateways/kafka/docker-compose.yml and the environment variable tables below for
every other knob.

Limitations (see the stub warning above for the full list): single gateway, single Iggy node,
SASL off by default. This is a development quick start, not a production deployment shape. If your
local apache/iggy:edge predates #4392, run
docker compose pull, because that server refuses every offset commit.

This quick start's topics use Iggy's default Durability::Replicated - a Produce ack is a
quorum commit, not an fsync. A docker kill or host power loss can lose acked records despite
the named Docker volume; the volume protects a clean docker compose down/up, not a crash.
Neither this quick start nor phase1_e2e_tests.rs exercises crash durability - the automated
test and the manual read-back above both only prove a produced record is readable from the
still-live process that wrote it. docs/MANUAL_TESTING.md's Category I covers a restart (a
graceful docker compose restart, which flushes on the way down) as a smoke test for "does the
volume actually persist across a container restart" - it is not a crash test and does not
contradict this limitation.

Success criteria (tracked for #3539): a new
contributor should reach a successful produce (and an Iggy-side read-back confirming it landed)
within 15 minutes of git clone, and every Phase 1 API key (CreateTopics, Metadata, Produce,
ListOffsets) has an automated test exercising the real compiled binaries end to end -
tests/phase1_e2e_tests.rs is that test, spawning real iggy-server and iggy-gateway-kafka
processes (not KafkaGateway::run in-process, as every other suite in this crate does) and
driving CreateTopics → Metadata → Produce → ListOffsets over real TCP, then reading the produced
records back through the Iggy SDK.

Test Copy Chapter

cargo test -p iggy-gateway-kafka

See docs/TEST_SUITE.md for the full suite catalog (cargo test -p iggy-gateway-kafka -- --list for the exact current test names - the count has drifted out of sync with the actual suites before, so it isn't pinned here).

Some api_handler_tests, server_e2e_tests, and version_firewall_tests cases require wire fixtures under tools/kafka-tool/kafka_messages/ (gitignored locally; CI generates them via scripts/ci-wire-fixtures.sh):

./gateways/kafka/scripts/ci-wire-fixtures.sh generate
cargo test -p iggy-gateway-kafka
./gateways/kafka/scripts/ci-wire-fixtures.sh cleanup   # optional

Or generate only the keys the tests need:

for key in 0 1 2 10 11 12 13 14 19 22; do
  cargo run -p kafka-message-gen -- generate \
    --output gateways/kafka/tools/kafka-tool/kafka_messages \
    --api-key "$key"
done

Manual testing Copy Chapter

Before check-in, run the procedure in docs/MANUAL_TESTING.md (smoke, version firewall, kcat, adversarial cases).

Scoped APIs Copy Chapter

See docs/SCOPE.md for #3421 deliverables, supported API key/version table, and post-foundation TODO backlog.

Design decisions Copy Chapter

• docs/BRIDGE_MAPPING.md — how a Kafka record becomes an Iggy message, and back
• docs/IDEMPOTENCE.md — InitProducerId, and why delivery is at-least-once
• docs/OFFSET_STORAGE.md — where Kafka consumer group offsets live
• docs/CONSUMER_GROUPS.md — group membership, rebalances, and why one gateway per bootstrap endpoint
• docs/AUTHENTICATION.md — how a Kafka client authenticates, and why PLAIN only
• docs/ACL_MAPPING.md — how Iggy permissions are described as Kafka ACLs
• docs/CONFIGS.md — DescribeConfigs, AlterConfigs and IncrementalAlterConfigs for topic retention.ms

Delivery guarantees Copy Section

Delivery through this gateway is at-least-once, and stays at-least-once across a gateway
restart. Transactions are not supported, and will not be. An idempotent Kafka producer is given
a producer id so that it starts, but its retries are not deduplicated: Produce uses the producer
id only to pick the producer's Iggy session, and ignores the epoch and sequence, so a retry after a
network timeout writes the record twice, and both copies reach the stream with their own offsets.

Iggy deduplicates writes on its own partition plane, and that does not close this gap, because it
guards the hop from the gateway to Iggy rather than the hop from the producer to the gateway.
docs/IDEMPOTENCE.md has the detail and what closing it needs.

Transactions are not supported, and will not be. AddPartitionsToTxn (24), AddOffsetsToTxn
(25), EndTxn (26) and TxnOffsetCommit (28) are never advertised, so a conforming client never
sends one; InitProducerId with a transactional_id answers UNSUPPORTED_VERSION (35); and a
Produce request carrying a transactional_id gets UNSUPPORTED_VERSION (35) on every partition
rather than having its records stored as if they were ordinary ones. None of those closes the
connection. docs/SCOPE.md's Transactions section has the ordering and the reasoning.

Consumer groups ([#3544](https://github.com/apache/iggy/issues/3544)) Copy Chapter

A stock consumer with a group.id joins, takes the partitions its own assignor hands it, commits,
and resumes from its commit. With the bridge on, commits live in Iggy, so a group also resumes after
a gateway restart. tests/kafka_client_e2e_tests.rs checks both with the Java consumer, and a
member that leaves hands its partition to the other at its commit.

Limits:

• One gateway per bootstrap endpoint. Group membership is gateway memory
  (docs/CONSUMER_GROUPS.md).
• Classic protocol only. A consumer with group.protocol=consumer (KIP-848) fails.
• group.instance.id is accepted but not honoured. A static member that restarts causes two
  rebalances.
• No DescribeGroups or ListGroups yet (#3548), so
  kafka-consumer-groups.sh cannot list or describe a group.
• No DeleteGroups or OffsetDelete, and offsets never expire. An operator deletes a group's offsets
  in Iggy (docs/OFFSET_STORAGE.md).
• Delivery is at-least-once. A consumer that stops before it commits reads those records again.

Listeners and TLS ([#4510](https://github.com/apache/iggy/issues/4510)) Copy Chapter

The gateway binds one listener per Kafka security protocol, named the way Kafka's own listeners
and advertised.listeners are. A client is pointed back, in Metadata and FindCoordinator, at the
listener it connected through and no other, so a client on the TLS port never learns the plaintext
one.

IGGY_KAFKA_LISTENERS=PLAINTEXT://127.0.0.1:9093,SSL://0.0.0.0:9094,SASL_SSL://0.0.0.0:9095 \
  IGGY_KAFKA_ADVERTISED_LISTENERS=SSL://kafka.example.com:9094,SASL_SSL://kafka.example.com:9095 \
  IGGY_KAFKA_TLS_CERT_FILE=/etc/iggy/kafka.pem IGGY_KAFKA_TLS_KEY_FILE=/etc/iggy/kafka.key \
  cargo run -p iggy-gateway-kafka

Each security protocol can be bound once, since the protocol is also the listener's name.

TLS is rustls. One certificate serves every SSL and SASL_SSL listener, and the files are
re-read every IGGY_KAFKA_TLS_RELOAD_INTERVAL_SECS: a rotated certificate takes effect for new
connections without a restart, a key that does not match leaves the previous material serving and
logs an error. Rotate by writing the new files elsewhere and renaming them into place: a chain
written in place can be read half written, and if what was read still parses and matches the key,
the truncated chain serves until the next interval. There is no self-signed mode; mint a
certificate (docs/MANUAL_TESTING.md has an openssl recipe) and give the Kafka client the CA.

The certificate's subject alternative names must cover every host in
IGGY_KAFKA_ADVERTISED_LISTENERS: a client verifies the name of each broker address Metadata
hands it, so a host missing from the certificate fails the client's next connection, not the
first. Client certificates are checked against the CA only; there is no CRL or OCSP revocation
checking, so revoke access by deleting or deactivating the Iggy user a certificate maps to. That
applies to new connections only: an open connection keeps the principal its handshake resolved,
and the permissions that came with it, until it closes.

kcat -b kafka.example.com:9094 -X security.protocol=SSL -X ssl.ca.location=ca.pem -L
kcat -b kafka.example.com:9095 -X security.protocol=SASL_SSL -X ssl.ca.location=ca.pem \
     -X sasl.mechanisms=PLAIN -X sasl.username=alice -X sasl.password=s3cret -L

Client certificates (mTLS) Copy Section

IGGY_KAFKA_TLS_CLIENT_AUTH=required (or requested) makes every SSL listener verify client
certificates against IGGY_KAFKA_TLS_CLIENT_CA_FILE, and the certificate names the principal
there. Its subject, rendered the way Java does (CN=alice,OU=eng,O=Example), goes through
IGGY_KAFKA_TLS_PRINCIPAL_MAPPING_RULES, and the result must be the name of an active Iggy user.
The gateway looks the user up through the bridge's own session, so the bridge is required and its
Iggy user needs read_users. A certificate whose subject no rule maps, or that names no such user,
is closed after the handshake, with the subject or name in the gateway's log at warn.

A SASL_SSL listener never asks for a client certificate, which is Kafka's default for that
protocol: SASL names the principal there.

With requested, a client that presents no certificate is Kafka's ANONYMOUS principal, which
holds no permissions. Once any listener names principals (a SASL listener, or an SSL listener
with client authentication), every connection that proved no identity is ANONYMOUS, on a
PLAINTEXT listener beside it too. The requests the gateway checks against a principal,
DeleteTopics, AlterConfigs and the auto-creation a Metadata request asks for, need
manage_topics and are refused for it; everything else is served as the bridge's user. A gateway whose listeners name nobody serves every request as the
bridge's user.

The rules are Kafka's syntax, tried in order: RULE:pattern/replacement/[LU] applies when the
pattern matches the whole subject, DEFAULT keeps the subject as is. Iggy accepts any username of
3 to 50 characters, but a raw subject is usually longer than that, and is rarely the name the
operator created, so a deployment that maps identities sets at least one rule. Patterns are a
Java-compatible subset: \d \s \w match ASCII only and . stops at Java's line terminators, as on
a broker, while (?i), the u, U, m, x and R flags, \p{..}, POSIX classes, \b, \v,
possessive quantifiers and the -- and ~~ class operations are refused at startup
(docs/AUTHENTICATION.md says why). A replacement that
names a group the pattern does not have is refused at startup. That is stricter than Kafka, which
writes such a $N literally: RULE:^CN=(.*)$/$2/ maps to $2 on a broker. The reverse case,
/$1\$2/, makes Kafka throw while the gateway maps alice to alice$2, which is harmless because
that Iggy user must still exist: