rustfs Documentation

File: README.md Copy Chapter

RustFS is a high-performance, distributed object storage system built in Rust—one of the most loved programming languages worldwide. RustFS combines the simplicity of MinIO with the memory safety and raw performance of Rust. It offers broad S3 API compatibility for supported features, is completely open-source, and is optimized for data lakes, AI, and big data workloads.

Unlike other storage systems, RustFS is released under the permissible Apache 2.0 license, avoiding the restrictions of AGPL. With Rust as its foundation, RustFS delivers superior speed and secure distributed features for next-generation object storage.

Feature & Status Copy Chapter

• High Performance: Built with Rust to ensure maximum speed and resource efficiency.
• Distributed Architecture: Scalable and fault-tolerant design suitable for large-scale deployments.
• S3 Compatibility: Seamless integration with common S3-compatible applications and tools; current coverage is tracked in the S3 compatibility matrix.
• OpenStack Swift API: Native support for Swift protocol with Keystone authentication.
• OpenStack Keystone Integration: Native support for OpenStack Keystone authentication with X-Auth-Token headers.
• Data Lake Support: Optimized for high-throughput big data and AI workloads.
• Open Source: Licensed under Apache 2.0, encouraging unrestricted community contributions and commercial usage.
• User-Friendly: Designed with simplicity in mind for easy deployment and management.

RustFS vs MinIO Performance Copy Chapter

Stress Test Environment:

https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a

RustFS vs Other Object Storage Copy Section

Staying ahead Copy Chapter

Star RustFS on GitHub and be instantly notified of new releases.

Quickstart Copy Chapter

To get started with RustFS, follow these steps:

1. One-click Installation (Option 1) Copy Section

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

2. Docker Quick Start (Option 2) Copy Section

The RustFS container runs as a non-root user rustfs (UID/GID 10001:10001). If you bind-mount host directories with Docker or Compose, every mounted path must be writable by that user, otherwise startup may fail with permission denied errors. This applies to data directories, log directories, and TLS certificate directories when RUSTFS_TLS_PATH is enabled.

# Create data and logs directories
mkdir -p data logs

# Change the owner of these directories
chown -R 10001:10001 data logs

# Using latest version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

# Using specific version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-rc.3

If you use podman instead of docker, you can install the RustFS with the below command

# Create data and logs directories
mkdir -p data logs

# Run the container (podman will automatically set the folders ownership)
podman run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data:Z,U -v $(pwd)/logs:/logs:Z,U rustfs/rustfs:latest

If you enable TLS with a bind-mounted certificate directory, prepare that mount the same way:

mkdir -p certs
chown -R 10001:10001 certs

You can also use Docker Compose. Using the docker-compose-simple.yml file in the root directory:

docker compose -f docker-compose-simple.yml up -d

Before running Compose with host bind mounts:

• Ensure every mounted host path is writable by 10001:10001.
• If you enable TLS, ensure the certificate mount for /opt/tls is also readable by 10001:10001.
• If matching host ownership is not practical, run the rustfs service with user: "<host-uid>:<host-gid>" instead.
• docker-compose-simple.yml includes a volume-permission-helper service for named volumes. docker-compose-simple.yml relies on you to prepare bind-mounted host paths in advance.

Similarly, you can run the command with podman

podman compose -f docker-compose-simple.yml up -d

Webhook notification quick start (Docker):

docker run -d --name rustfs -p 9000:9000 \
  -e RUSTFS_NOTIFY_ENABLE=true \
  -e RUSTFS_NOTIFY_WEBHOOK_ENABLE_PRIMARY=on \
  -e RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY=http://<host-ip>:3020/webhook \
  -e RUSTFS_NOTIFY_WEBHOOK_QUEUE_DIR_PRIMARY=/tmp/rustfs-events \
  -e RUSTFS_OUTBOUND_ALLOW_ORIGINS=http://<host-ip>:3020 \
  rustfs/rustfs:latest

Notes:

• RUSTFS_NOTIFY_ENABLE=true enables the global notify module switch.
• For ARN arn:rustfs:sqs::primary:webhook, use instance-scoped env vars with _PRIMARY.
• If queue dir is omitted, default is /opt/rustfs/events; ensure it is writable by the container runtime user.
• RUSTFS_NOTIFY_WEBHOOK_SKIP_TLS_VERIFY_PRIMARY defaults to false; enabling it skips webhook TLS certificate verification, allows MITM attacks, and emits a startup warning. Prefer RUSTFS_NOTIFY_WEBHOOK_CLIENT_CA_PRIMARY for private CAs.
• Since 1.0.0-beta.11, webhook endpoints on private or container networks
  (Docker Compose service names, host.docker.internal, RFC 1918 addresses) are
  blocked unless their exact scheme://host:port origin is listed in
  RUSTFS_OUTBOUND_ALLOW_ORIGINS (the origin only, without the path). See
  Outbound Connection Policy.

NOTE: We recommend reviewing the docker-compose.yml file before running. It defines several services including Grafana, Prometheus, and Jaeger, which are helpful for RustFS observability. If you wish to start Redis or Nginx containers, you can specify the corresponding profiles.

3\. Build from Source (Option 3) - Advanced Users Copy Section

For developers who want to build RustFS Docker images from source with multi-architecture support:

# Build multi-architecture images locally
./docker-buildx.sh --build-arg RELEASE=latest

# Build and push to registry
./docker-buildx.sh --push

# Build specific version
./docker-buildx.sh --release v1.0.0 --push

# Build for custom registry
./docker-buildx.sh --registry your-registry.com --namespace yourname --push

The docker-buildx.sh script supports:

• Multi-architecture builds: linux/amd64, linux/arm64
• Automatic version detection: Uses git tags or commit hashes
• Registry flexibility: Supports Docker Hub, GitHub Container Registry, etc.
• Build optimization: Includes caching and parallel builds

You can also use Make targets for convenience:

make docker-buildx                    # Build locally
make docker-buildx-push               # Build and push
make docker-buildx-version VERSION=v1.0.0  # Build specific version
make help-docker                      # Show all Docker-related commands

Heads-up (macOS cross-compilation): macOS keeps the default ulimit -n at 256, so cargo zigbuild or ./build-rustfs.sh --platform ... may fail with ProcessFdQuotaExceeded when targeting Linux. The build script attempts to raise the limit automatically, but if you still see the warning, run ulimit -n 4096 (or higher) in your shell before building.

4\. Build with Helm Chart (Option 4) - Cloud Native Copy Section

Follow the instructions in the Helm Chart README to install RustFS on a Kubernetes cluster.

For scanner pacing, cycle budgets, bitrot cadence, lifecycle transition status,
and single-node single-disk idle CPU tuning, see
Scanner Runtime Controls. For
repeatable scanner-pressure validation, see
Scanner Benchmark Runbook. For
drive timeout knobs on slow storage — including the walk stall budget that
governs ListObjects on large prefixes — see
Drive Timeout Tuning.

5\. Nix Flake (Option 5) Copy Section

If you have Nix with flakes enabled:

# Run directly without installing
nix run github:rustfs/rustfs

# Build the binary
nix build github:rustfs/rustfs
./result/bin/rustfs --help

# Or from a local checkout
nix build
nix run

6\. X-CMD (Option 6) Copy Section

If you are an x-cmd user:

# Run directly without installing
x rustfs

# Download the binary and install it to the global environment
x env use rustfs
rustfs --help

Accessing RustFS Copy Section

1. Access the Console: Open your web browser and navigate to http://localhost:9001 to access the RustFS console.
   • Default credentials: rustfsadmin / rustfsadmin
2. Create a Bucket: Use the console to create a new bucket for your objects.
3. Upload Objects: You can upload files directly through the console or use S3-compatible APIs/clients to interact with your RustFS instance.

NOTE: To access the RustFS instance via https, please refer to the TLS Configuration Docs.

OIDC Roles Claim (Microsoft Entra ID) Copy Section

RustFS supports mapping an OIDC claim containing role values into the existing
authorization pipeline. The roles_claim setting is optional: when unset or
empty, only the groups claim contributes to authorization (same as older
RustFS releases). For Microsoft Entra ID app roles, set roles_claim=roles so
both console admin checks and bucket IAM policies can evaluate those roles.

Example environment configuration (opt-in roles claim):

RUSTFS_IDENTITY_OPENID_ENABLE=on
RUSTFS_IDENTITY_OPENID_CONFIG_URL="https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration"
RUSTFS_IDENTITY_OPENID_CLIENT_ID="<client-id>"
RUSTFS_IDENTITY_OPENID_CLIENT_SECRET="<client-secret>"
RUSTFS_IDENTITY_OPENID_SCOPES="openid,profile,email"
RUSTFS_IDENTITY_OPENID_GROUPS_CLAIM="groups"
RUSTFS_IDENTITY_OPENID_ROLES_CLAIM="roles"

Policy condition example (evaluate app roles directly with jwt:roles; when
roles_claim is configured, RustFS also merges those values into jwt:groups
for backward compatibility with older policies):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["admin:*"],
      "Resource": ["arn:aws:s3:::*"],
      "Condition": {
        "ForAnyValue:StringEquals": {
          "jwt:roles": ["RustFS.ConsoleAdmin"]
        }
      }
    }
  ]
}

Documentation Copy Chapter

For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.

Getting Help Copy Chapter

If you have any questions or need assistance:

• Check the FAQ for common issues and solutions.
• Join our GitHub Discussions to ask questions and share your experiences.
• Open an issue on our GitHub Issues page for bug reports or feature requests.

Links Copy Chapter

• Documentation - The manual you should read
• Changelog - What we broke and fixed
• GitHub Discussions - Where the community lives
• Discord - Chat with the RustFS community

Contact Copy Chapter

• Bugs: GitHub Issues
• Business: [email protected]
• Jobs: [email protected]
• General Discussion: GitHub Discussions
• Contributing: CONTRIBUTING.md

Contributors Copy Chapter

RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.

Star History Copy Chapter

License Copy Chapter

Apache 2.0

RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.

File: crates/zip/README.md Copy Chapter

RustFS Zip - Archive Format Detection And Stream Decoding

📖 Overview Copy Chapter

RustFS Zip provides the archive primitives used by the RustFS archive extract flow:

• identify a compression format from an archive extension
• wrap an async reader in the matching stream decoder
• carry the shared default archive guardrails

Current Features Copy Chapter

• CompressionFormat::from_extension() for extension-based format detection, including tar-family suffixes such as tgz, tbz2, txz, and tzst
• CompressionFormat::get_decoder() for async stream decoding of gzip, bzip2, zlib, xz, and zstd, plus a pass-through reader for plain tar
• ArchiveLimits with the default entry count, entry size, total unpacked size, and path length guardrails

Current Boundaries Copy Chapter

• ZIP has no stream decoder: get_decoder() rejects CompressionFormat::Zip, because ZIP needs central-directory semantics that a forward-only stream cannot provide
• This crate detects formats and hands back decoders; archive iteration, entry writing, and extraction to disk belong to the caller
• ArchiveLimits carries the values only; enforcement and the resulting protocol error belong to the caller
• Archive extraction safety policy remains the responsibility of the RustFS caller for object-store flows

📚 Documentation Copy Chapter

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License Copy Chapter

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

File: crates/utils/README.md Copy Chapter

RustFS Utils - Utility Functions

📖 Overview Copy Chapter

RustFS Utils provides essential utility functions and common tools for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features Copy Chapter

• Cross-platform system operations and monitoring
• File system utilities with atomic operations
• Multi-algorithm compression and encoding support
• Cryptographic utilities and secure key generation
• Network utilities and protocol helpers
• Certificate handling and validation tools

📚 Documentation Copy Chapter

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License Copy Chapter

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

File: crates/trusted-proxies/README.md Copy Chapter

RustFS Trusted Proxies

The rustfs-trusted-proxies module provides secure and efficient management of trusted proxy servers within the RustFS
ecosystem. It is designed to handle multi-layer proxy architectures, ensuring accurate client IP identification while
maintaining a zero-trust security model.

Modes Copy Chapter

• Simple default: only trusts forwarding headers when the direct peer IP is
  internal.
• Legacy full mode: keeps the original proxy-chain validation, available
  via legacy_* helpers.

Features Copy Chapter

• Multi-Layer Proxy Validation: Supports Strict, Lenient, and HopByHop validation modes to accurately identify
  the real client IP address.
• Zero-Trust Security: Verifies every hop in the proxy chain against a configurable list of trusted networks.
• Cloud Integration: Automatic discovery of trusted IP ranges for major cloud providers including AWS, Azure, and
  GCP.
• High Performance: Utilizes the moka cache for fast lookup of validation results and axum for a
  high-performance web interface.
• Observability: Built-in support for Prometheus metrics and structured JSON logging via tracing.
• RFC 7239 Support: Full support for the modern Forwarded header alongside legacy X-Forwarded-For headers.

Configuration Copy Chapter

The module is configured primarily through environment variables:

Usage Copy Chapter

Initialization Copy Section

Initialize the global trusted proxy system at the start of your application (e.g., in main.rs):

// Initialize trusted proxies system
rustfs_trusted_proxies::init();

As a Middleware Copy Section

Integrate the trusted proxy validation into your Axum application or HTTP service stack:

use rustfs_trusted_proxies;

let app = Router::new()
    .route("/", get(handler))
    // Add the trusted proxy layer if enabled
    .option_layer(if rustfs_trusted_proxies::is_enabled() {
        Some(rustfs_trusted_proxies::layer().clone())
    } else {
        None
    });

Simple default mode Copy Section

The default mode only trusts forwarding headers from internal IPs.

RUSTFS_TRUSTED_PROXY_IMPLEMENTATION=simple

Legacy mode Copy Section

The original implementation is still available:

rustfs_trusted_proxies::legacy_init();
let layer = rustfs_trusted_proxies::LegacyTrustedProxyLayer::enabled(config, None);

Or switch the global default path:

RUSTFS_TRUSTED_PROXY_IMPLEMENTATION=legacy

Accessing Client Info Copy Section

Retrieve the verified client information in your handlers or other middleware:

use rustfs_trusted_proxies::ClientInfo;

async fn handler(req: Request) -> impl IntoResponse {
    if let Some(client_info) = req.extensions().get::<ClientInfo>() {
        println!("Real Client IP: {}", client_info.real_ip);
        println!("Is Trusted: {}", client_info.is_from_trusted_proxy);
    }
}

Development Copy Chapter

Pre-Commit Checklist Copy Section

Before committing, ensure all checks pass:

make pre-commit

Testing Copy Section

Run the test suite:

cargo test --workspace --exclude e2e_test

License Copy Chapter

Licensed under the Apache License, Version 2.0.

File: crates/signer/README.md Copy Chapter

RustFS Signer - Request Signing & Authentication

📖 Overview Copy Chapter

RustFS Signer provides AWS-compatible request signing and authentication capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features Copy Chapter

• AWS Signature Version 4 (SigV4) implementation
• Pre-signed URL generation and validation
• Multiple authentication methods (access key, STS token, IAM role)
• Streaming upload signature support
• Hardware-accelerated cryptographic operations
• Multi-region signature support

📚 Documentation Copy Chapter

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License Copy Chapter

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

File: crates/scanner/README.md Copy Chapter

RustFS Scanner

RustFS Scanner is the background maintenance scan loop. It handles usage
accounting, lifecycle expiry and transition admission, bucket replication repair
admission, scanner-driven heal/bitrot checks, and namespace alerts.

For operator-facing runtime controls, status fields, and tuning workflows, see
Scanner Runtime Controls.
For repeatable scanner-pressure validation, see
Scanner Benchmark Runbook.

Chinese documentation is available in README.zh-CN.md.

Development Copy Chapter

Build Copy Section

cargo build --package rustfs-scanner

Test Copy Section

cargo test --package rustfs-scanner

License Copy Chapter

Apache License 2.0

File: crates/s3select-query/README.md Copy Chapter

RustFS S3Select Query - SQL Query Engine

📖 Overview Copy Chapter

RustFS S3Select Query provides Apache DataFusion-powered SQL query engine capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features Copy Chapter

• Apache DataFusion integration for high-performance queries
• Vectorized processing with SIMD acceleration
• Parallel query execution across multiple threads
• Cost-based query optimization
• Support for complex SQL operations (joins, subqueries, window functions)
• Multiple data format support (Parquet, CSV, JSON, Arrow)

📚 Documentation Copy Chapter

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License Copy Chapter

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

File: crates/s3select-api/README.md Copy Chapter

RustFS S3Select API - SQL Query Interface

📖 Overview Copy Chapter

RustFS S3Select API provides AWS S3 Select compatible SQL query capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features Copy Chapter

• Standard SQL query support (SELECT, WHERE, GROUP BY, ORDER BY)
• Multiple data format support (CSV, JSON, Parquet, Arrow)
• Streaming processing for large files
• AWS S3 Select API compatibility
• Parallel query execution
• Predicate pushdown optimization

📚 Documentation Copy Chapter

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License Copy Chapter

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

File: crates/rio/README.md Copy Chapter

RustFS Rio - High-Performance I/O

📖 Overview Copy Chapter

RustFS Rio provides high-performance asynchronous I/O operations for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features Copy Chapter

• Zero-copy streaming I/O operations
• Hardware-accelerated encryption/decryption
• Multi-algorithm compression support
• Efficient buffer management and pooling
• Vectored I/O for improved throughput
• Real-time data integrity verification

📚 Documentation Copy Chapter

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License Copy Chapter

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.