## File: README.md

[](https://github.com/mozilla/sccache/actions?query=workflow%3Aci)
[](https://crates.io/crates/sccache)
[](https://chat.mozilla.org/#/room/#sccache:mozilla.org)

[](https://deps.rs/repo/github/mozilla/sccache)

[](https://codecov.io/gh/mozilla/sccache)


sccache - Shared Compilation Cache
==================================

sccache is a [ccache](https://ccache.dev/)-like compiler caching tool. It is used as a compiler wrapper and avoids compilation when possible, storing cached results either on [local disk](docs/Local.md) or in one of [several cloud storage backends](#storage-options). Multi-level caching with automatic backfill is supported for hierarchical cache architectures (see [Multi-Level Cache](docs/MultiLevel.md)).

sccache includes support for caching the compilation of Assembler, C/C++ code, [Rust](docs/Rust.md), as well as NVIDIA's CUDA using [nvcc](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html), and [clang](https://llvm.org/docs/CompileCudaWithLLVM.html), [AMD's ROCm HIP](https://rocm.docs.amd.com/projects/HIP/en/latest/index.html).

sccache also provides [icecream](https://github.com/icecc/icecream)-style distributed compilation (automatic packaging of local toolchains) for all supported compilers (including Rust). The distributed compilation system includes several security features that icecream lacks such as authentication, transport layer encryption, and sandboxed compiler execution on build servers. See [the distributed quickstart](docs/DistributedQuickstart.md) guide for more information.

sccache is also available as a [GitHub Actions](https://github.com/marketplace/actions/sccache-action) to facilitate the deployment using GitHub Actions cache.

---

Table of Contents (ToC)
=======================

* [Installation](#installation)
* [Usage](#usage)
* [Build Requirements](#build-requirements)
* [Build](#build)
* [Separating caches between invocations](#separating-caches-between-invocations)
* [Overwriting the cache](#overwriting-the-cache)
* [Debugging](#debugging)
* [Interaction with GNU `make` jobserver](#interaction-with-gnu-make-jobserver)
* [Known Caveats](#known-caveats)
* [Storage Options](#storage-options)
  * [Multi-Level (Hierarchical Caching)](docs/MultiLevel.md)
  * [Local](docs/Local.md)
  * [S3](docs/S3.md)
  * [R2](docs/S3.md#R2)
  * [Redis](docs/Redis.md)
  * [Memcached](docs/Memcached.md)
  * [Google Cloud Storage](docs/Gcs.md)
  * [Azure](docs/Azure.md)
  * [GitHub Actions](docs/GHA.md)
  * [WebDAV (Ccache/Bazel/Gradle compatible)](docs/Webdav.md)
  * [Alibaba OSS](docs/OSS.md)
  * [Tencent Cloud Object Storage](docs/COS.md)

---

## Installation

There are prebuilt x86-64 binaries available for Windows, Linux (a portable binary compiled against musl), and macOS [on the releases page](https://github.com/mozilla/sccache/releases/latest). Several package managers also include sccache packages, you can install the latest release from source using cargo, or build directly from a source checkout.

### macOS

On macOS sccache can be installed via [Homebrew](https://brew.sh/):

```bash
brew install sccache
```

or via [MacPorts](https://www.macports.org/):

```bash
sudo port install sccache
```

### Windows

On Windows, sccache can be installed via [scoop](https://scoop.sh/):

```
scoop install sccache
```
or `winget`:

```
winget install Mozilla.sccache
```

### Via cargo

If you have a Rust toolchain installed you can install sccache using cargo. **Note that this will compile sccache from source which is fairly resource-intensive. For CI purposes you should use prebuilt binary packages.**

```bash
cargo install sccache --locked
```

Or, using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall) to install a prebuilt binary:

```bash
cargo binstall sccache
```

### With Nix

Sccache is available in nixpkgs, so if you don't need the latest version you can use that:

```nix
buildInputs = [ pkgs.sccache ];
```

We also provide a flake with an overlay for getting the latest version:

```nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    sccache = {
      url = "github:mozilla/sccache";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, sccache, ... }:
    let
      system = "x86_64-linux";
      pkgs = import nixpkgs {
        inherit system;
        overlays = [ sccache.overlays.default ];
      };
    in {
      devShells.${system}.default = pkgs.mkShell {
        buildInputs = [ pkgs.sccache ];
      };
    };
}
```

Or use it directly from the flake without the overlay:

```bash
nix run github:mozilla/sccache -- --help
nix shell github:mozilla/sccache
```

---

Usage
-----

Running sccache is like running ccache: prefix your compilation commands with it, like so:

```bash
sccache gcc -o foo.o -c foo.c
```

If you want to use sccache for caching Rust builds you can define `build.rustc-wrapper` in the
[cargo configuration file](https://doc.rust-lang.org/cargo/reference/config.html).  For example, you can set it globally
in `$HOME/.cargo/config.toml` by adding:

```toml
[build]
rustc-wrapper = "/path/to/sccache"
```

Note that you need to use cargo 1.40 or newer for this to work.

Alternatively you can use the environment variable `RUSTC_WRAPPER`:

```bash
export RUSTC_WRAPPER=/path/to/sccache
cargo build
```

sccache supports gcc, clang, MSVC, rustc, [NVCC](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html), [NVC++](https://docs.nvidia.com/hpc-sdk//compilers/hpc-compilers-user-guide/index.html), [hipcc](https://rocm.docs.amd.com/projects/HIP/en/latest/index.html), and [Wind River's diab compiler](https://www.windriver.com/products/development-tools/#diab_compiler). Both gcc and msvc support Response Files, read more about their implementation [here](docs/ResponseFiles.md).

If you don't [specify otherwise](#storage-options), sccache will use a local disk cache.

sccache works using a client-server model, where the server runs locally on the same machine as the client. The client-server model allows the server to be more efficient by keeping some state in memory. The sccache command will spawn a server process if one is not already running, or you can run `sccache --start-server` to start the background server process without performing any compilation.

By default sccache server will listen on `127.0.0.1:4226`, you can specify environment variable `SCCACHE_SERVER_PORT` to use a different port or `SCCACHE_SERVER_UDS` to listen on unix domain socket. Abstract unix socket is also supported as long as the path is escaped following the [format](https://doc.rust-lang.org/std/ascii/fn.escape_default.html). For example:

```
% env SCCACHE_SERVER_UDS=$HOME/sccache.sock sccache --start-server # unix socket
% env SCCACHE_SERVER_UDS=\\x00sccache.sock sccache --start-server # abstract unix socket
```

You can run `sccache --stop-server` to terminate the server. It will also terminate after (by default) 10 minutes of inactivity.

Running `sccache --show-stats` will print a summary of cache statistics.

Some notes about using `sccache` with [Jenkins](https://jenkins.io) are [here](docs/Jenkins.md).

To use sccache with cmake, provide the following command line arguments to cmake 3.4 or newer:

```
-DCMAKE_C_COMPILER_LAUNCHER=sccache
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache
```

The process for using sccache with MSVC and cmake, depends on which version of cmake you're using. **For versions of cmake 3.24 and earlier**, to generate PDB files for debugging with MSVC, you can use the [`/Z7` option](https://docs.microsoft.com/en-us/cpp/build/reference/z7-zi-zi-debug-information-format?view=msvc-160). Alternatively, the `/Zi` option together with `/Fd` can work if `/Fd` names a different PDB file name for each object file created. Note that CMake sets `/Zi` by default, so if you use CMake, you can use `/Z7` by adding code like this in your CMakeLists.txt:

```cmake
if(CMAKE_BUILD_TYPE STREQUAL "Debug")
  string(REPLACE "/Zi" "/Z7" CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG}")
  string(REPLACE "/Zi" "/Z7" CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG}")
elseif(CMAKE_BUILD_TYPE STREQUAL "Release")
  string(REPLACE "/Zi" "/Z7" CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE}")
  string(REPLACE "/Zi" "/Z7" CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE}")
elseif(CMAKE_BUILD_TYPE STREQUAL "RelWithDebInfo")
  string(REPLACE "/Zi" "/Z7" CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO}")
  string(REPLACE "/Zi" "/Z7" CMAKE_C_FLAGS_RELWITHDEBINFO "${CMAKE_C_FLAGS_RELWITHDEBINFO}")
endif()
```

By default, sccache will fail your build if it fails to successfully communicate with its associated server. To have sccache instead gracefully failover to the local compiler without stopping, set the environment variable `SCCACHE_IGNORE_SERVER_IO_ERROR=1`.

**For versions of cmake 3.25 and later**, to compile with MSVC, you have to use the new `CMAKE_MSVC_DEBUG_INFORMATION_FORMAT` option, meant to configure the `-Z7` flag.  Additionally, you must set the cmake policy number 0141 to the NEW setting:
```cmake
set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>")
cmake_policy(SET CMP0141 NEW)
```

Example configuration where we automatically look for `sccache` in the `PATH`:
```cmake
find_program(SCCACHE sccache REQUIRED)

set(CMAKE_C_COMPILER_LAUNCHER ${SCCACHE})
set(CMAKE_CXX_COMPILER_LAUNCHER ${SCCACHE})
set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>")
cmake_policy(SET CMP0141 NEW)
```

Alternatively, if configuring cmake with MSVC on the command line, assuming that sccache is on the default search path:

```
cmake -DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache -DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=Embedded -DCMAKE_POLICY_CMP0141=NEW [...]
```

And you can build code as usual without any additional flags in the command line, which is useful for IDEs.


---

Build Requirements
------------------

sccache is a [Rust](https://www.rust-lang.org/) program. Building it requires `cargo` (and thus`rustc`). sccache currently requires **Rust 1.91.0**. We recommend you install Rust via [Rustup](https://rustup.rs/).

Build
-----

If you are building sccache for non-development purposes make sure you use `cargo build --release` to get optimized binaries:

```bash
cargo build --release [--no-default-features --features=s3|redis|gcs|memcached|azure|gha|webdav|oss|cos]
```

The list of features can be found in the `Cargo.toml` file, `[features]` section.

By default, `sccache` builds with support for all storage backends, but individual backends may be disabled by resetting the list of features and enabling all the other backends. Refer the [Cargo Documentation](http://doc.crates.io/manifest.html#the-features-section) for details on how to select features with Cargo.

### Building portable binaries

When building with the `dist-server` feature, `sccache` will depend on OpenSSL, which can be an annoyance if you want to distribute portable binaries. It is possible to statically link against OpenSSL using the `openssl/vendored` feature.

#### Linux

Build with `cargo` and use `ldd` to check that the resulting binary does not depend on OpenSSL anymore.

#### macOS

Build with `cargo` and use `otool -L` to check that the resulting binary does not depend on OpenSSL anymore.

#### Windows

On Windows, the binary might also depend on a few MSVC CRT DLLs that are not available on older Windows versions.

It is possible to statically link against the CRT using a `.cargo/config.toml` file with the following contents.

```toml
[target.x86_64-pc-windows-msvc]
rustflags = ["-Ctarget-feature=+crt-static"]
```

Build with `cargo` and use `dumpbin /dependents` to check that the resulting binary does not depend on MSVC CRT DLLs anymore.

When statically linking with OpenSSL, you will need Perl available in your `$PATH`.

---

Separating caches between invocations
-------------------------------------

In situations where several different compilation invocations
should not reuse the cached results from each other,
one can set `SCCACHE_C_CUSTOM_CACHE_BUSTER` to a unique value
that'll be mixed into the hash.
`MACOSX_DEPLOYMENT_TARGET` and `IPHONEOS_DEPLOYMENT_TARGET` variables
already exhibit such reuse-suppression behaviour.
There are currently no such variables for compiling Rust.

---

Overwriting the cache
---------------------

In situations where the cache contains broken build artifacts, it can be necessary to overwrite the contents in the cache. That can be achieved by setting the `SCCACHE_RECACHE` environment variable.

---

Debugging
---------

You can set the `SCCACHE_ERROR_LOG` environment variable to a path and set `SCCACHE_LOG` to get the server process to redirect its logging there (including the output of unhandled panics, since the server sets `RUST_BACKTRACE=1` internally).

    SCCACHE_ERROR_LOG=/tmp/sccache_log.txt SCCACHE_LOG=debug sccache

You can also set these environment variables for your build system, for example

    SCCACHE_ERROR_LOG=/tmp/sccache_log.txt SCCACHE_LOG=debug cmake --build /path/to/cmake/build/directory

Alternatively, if you are compiling locally, you can run the server manually in foreground mode by running `SCCACHE_START_SERVER=1 SCCACHE_NO_DAEMON=1 sccache`, and send logging to stderr by setting the [`SCCACHE_LOG` environment variable](https://docs.rs/env_logger/0.7.1/env_logger/#enabling-logging) for example. This method is not suitable for CI services because you need to compile in another shell at the same time.

    SCCACHE_LOG=debug SCCACHE_START_SERVER=1 SCCACHE_NO_DAEMON=1 sccache

---

Interaction with GNU `make` jobserver
-------------------------------------

sccache provides support for a [GNU make jobserver](https://www.gnu.org/software/make/manual/html_node/Job-Slots.html). When the server is started from a process that provides a jobserver, sccache will use that jobserver and provide it to any processes it spawns. (If you are running sccache from a GNU make recipe, you will need to prefix the command with `+` to get this behavior.) If the sccache server is started without a jobserver present it will create its own with the number of slots equal to the number of available CPU cores.

This is most useful when using sccache for Rust compilation, as rustc supports using a jobserver for parallel codegen, so this ensures that rustc will not overwhelm the system with codegen tasks. Cargo implements its own jobserver ([see the information on `NUM_JOBS` in the cargo documentation](https://doc.rust-lang.org/stable/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-build-scripts)) for rustc to use, so using sccache for Rust compilation in cargo via `RUSTC_WRAPPER` should do the right thing automatically.

---

Normalizing Paths with `SCCACHE_BASEDIRS`
-----------------------------------------

By default, sccache requires absolute paths to match for cache hits. To enable cache sharing across different build directories, you can set `SCCACHE_BASEDIRS` to strip a base directory from paths before hashing:

```bash
export SCCACHE_BASEDIRS=/home/user/project
```

You can also specify multiple base directories by separating them by `;` on Windows hosts and by `:` on any other operating system. When multiple directories are provided, the longest matching prefix is used:

```bash
export SCCACHE_BASEDIRS="/home/user/project:/home/user/workspace"
```

Path matching is **case-insensitive** on Windows and **case-sensitive** on other operating systems.

This is similar to ccache's `CCACHE_BASEDIR` and helps when:
* Building the same project from different directories
* Sharing cache between CI jobs with different checkout paths
* Multiple developers working with different username paths
* Working with multiple project checkouts simultaneously

**Note:** Only absolute paths are supported. Relative paths will prevent server from starting.

You can also configure this in the sccache config file:

```toml
# Single directory
basedirs = ["/home/user/project"]

# Or multiple directories
basedirs = ["/home/user/project", "/home/user/workspace"]
```

---

Known Caveats
-------------

### General

* By default, absolute paths to files must match to get a cache hit. To work around this, use `SCCACHE_BASEDIRS` (see above) to normalize paths before hashing.

### Rust

* Crates that invoke the system linker cannot be cached. This includes `bin`, `dylib`, `cdylib`, and `proc-macro` crates. You may be able to improve compilation time of large `bin` crates by converting them to a `lib` crate with a thin `bin` wrapper.
* Incrementally compiled crates cannot be cached. By default, in the debug profile Cargo will use incremental compilation for workspace members and path dependencies. [You can disable incremental compilation.](https://doc.rust-lang.org/cargo/reference/profiles.html#incremental)

[More details on Rust caveats](/docs/Rust.md)

### C++20 Modules

sccache has partial support for C++20 named modules when using **Clang**. The following flags are supported:

* `-fmodule-file=` and `-fmodule-file=<name>=` - importing precompiled module interfaces
* `-fmodule-output=` - generating module interface output alongside object files
* `--precompile` - compiling module interface units
* `-fmodules-reduced-bmi` - generating reduced BMI files

The following module-related flags are **not supported** and will bypass the cache:

* `-fmodules` and `-fcxx-modules` - Clang header modules (not C++20 named modules)
* `-fprebuilt-implicit-modules` and `-fprebuilt-module-path` - implicit module discovery

**GCC** and **MSVC** C++20 modules are not yet supported. Compilations using `-fmodules-ts` (GCC) or `/interface`, `/ifcOutput`, etc. (MSVC) will bypass the cache.

### User Agent

* Requests sent to your storage option of choice will have a user agent header indicating the current sccache version, e.g. `sccache/0.8.2`.

Storage Options
---------------

* [Local](docs/Local.md)
* [S3](docs/S3.md)
* [R2](docs/S3.md#R2)
* [Redis](docs/Redis.md)
* [Memcached](docs/Memcached.md)
* [Google Cloud Storage](docs/Gcs.md)
* [Azure](docs/Azure.md)
* [GitHub Actions](docs/GHA.md)
* [WebDAV (Ccache/Bazel/Gradle compatible)](docs/Webdav.md)
* [Alibaba OSS](docs/OSS.md)
* [Tencent Cloud Object Storage](docs/COS.md)

---

## File: docs/Architecture.md

# Sccache high level architecture

This schema shows at a high level how sccache decides whether to compile or to
reuse a cached result.


```mermaid
  flowchart LR
      id1[[Environment variables]] --> hash
      id2[[Compiler binary]] --> hash
      id3[[Compiler arguments]] --> hash
      id5[[Files]] --> |  | hash
      Compile --> Upload
      Storage[(Storage)] --> | yes | Download
      hash([hash]) --> | exists? | Storage
      Storage --> | no | Compile
      Upload --> Storage

      classDef input fill:#4fc3f7,stroke:#01579b,color:#000
      classDef store fill:#ffb74d,stroke:#e65100,color:#000
      classDef action fill:#81c784,stroke:#1b5e20,color:#000
      class id1,id2,id3,id5 input
      class Storage store
      class hash,Compile,Upload,Download action
```


For more details about how hash generation works, see [the caching documentation](Caching.md).


## Direct mode (preprocessor cache)

For C/C++, sccache can also cache the *preprocessor* result, so that a cache
lookup can skip preprocessing entirely. This is inspired by
[ccache's direct mode](https://ccache.dev/manual/3.7.9.html#_the_direct_mode)
and is described in the [local storage doc](Local.md).

Before computing the object-cache key, sccache looks up a **preprocessor cache
entry** keyed on the source file (path + contents) and the preprocessor
arguments. That entry records every file included by the source. If the entry
exists and all of those included files are unchanged, sccache reuses it and
never runs the preprocessor; otherwise it preprocesses normally and stores a
fresh entry. This step is shown as the blue region in the diagrams below and is
enabled by default (it is skipped for non-C/C++, when disabled, or when certain
flags such as `-Wp,*` / `-Xpreprocessor` are present).


## Execution modes

The caching logic above is the same regardless of *where* it runs. What differs
is which process actually runs the compiler and talks to the storage backend.

sccache is split in two:

- a short-lived **CLI process**, spawned once per compiler invocation (e.g. by
  `make -jN`), and
- a long-lived **daemon** (the background server), which holds the storage
  backend and the accumulated statistics.

The two communicate over a local IPC connection. There are two ways to divide
the work between them.

### Server-side mode (default)

The CLI forwards the whole compilation to the daemon. The daemon runs the
direct-mode preprocessor cache lookup, computes the hash, looks up the object
cache, runs the compiler on a miss, and stores the result. It writes the output
files (e.g. the object file) to disk itself; only the output streams (stdout /
stderr) and the exit code travel back to the CLI.

```mermaid
  sequenceDiagram
      participant B as Build (make -jN)
      participant C as sccache CLI
      participant D as sccache daemon
      participant S as Storage backend
      B->>C: sccache cc -c foo.c
      C->>D: Compile request (IPC)
      rect rgba(33, 150, 243, 0.18)
          note over D,S: Direct mode — preprocessor cache (C/C++)
          D->>S: lookup preprocessor entry (source + args)
          alt entry found & included files unchanged
              S-->>D: manifest → reuse, skip preprocessing
          else miss / stale / disabled
              D->>D: run preprocessor
              D->>S: store preprocessor entry
          end
      end
      rect rgba(76, 175, 80, 0.18)
          note over D,S: Object cache
          D->>S: lookup object (hash)
          alt cache hit
              S-->>D: cached object
          else cache miss
              D->>D: run compiler
              D->>S: store object
          end
      end
      D-->>C: CompileFinished (stdout / stderr / exit code)
      C-->>B: exit code + outputs
```

### Client-side mode (`SCCACHE_CLIENT_SIDE`)

The compile pipeline runs **in the CLI process itself**; the daemon is used only
as a shared gateway to the storage backend (and as a place to aggregate stats).

On startup the CLI performs a one-shot `StorageHandshake` to fetch the cache
metadata (cache mode, max size, basedirs, preprocessor-cache config). It then
runs the same pipeline locally — including direct mode — and forwards each
individual cache operation to the daemon over IPC. On the client side this is
implemented by `IpcStorage`, which implements the same `Storage` trait as every
other backend, so the rest of the compile pipeline is unchanged.

```mermaid
  sequenceDiagram
      participant B as Build (make -jN)
      participant C as sccache CLI (IpcStorage)
      participant D as sccache daemon
      participant S as Storage backend
      B->>C: sccache cc -c foo.c
      C->>D: StorageHandshake (IPC)
      D-->>C: cache metadata (mode, max_size, basedirs, …)
      rect rgba(33, 150, 243, 0.18)
          note over C,S: Direct mode — preprocessor cache (C/C++)
          C->>D: StorageGetPreprocessorEntry (key)
          D->>S: lookup
          alt entry found & included files unchanged
              D-->>C: manifest → reuse, skip preprocessing
          else miss / stale / disabled
              D-->>C: miss
              C->>C: run preprocessor
              C->>D: StoragePutPreprocessorEntry (key, bytes)
              D->>S: store
          end
      end
      rect rgba(76, 175, 80, 0.18)
          note over C,S: Object cache
          C->>C: compute hash
          C->>D: StorageGetPath / StorageGetRaw (key)
          D->>S: lookup
          alt cache hit
              S-->>D: path / bytes
              D-->>C: path / bytes
          else cache miss
              D-->>C: miss
              C->>C: run compiler
              C->>D: StoragePutRaw (key, bytes)
              D->>S: store
          end
      end
      C->>D: RecordStats (delta)
      C-->>B: exit code + outputs
```

`StorageGetPath` lets the CLI read a cached entry straight off disk when the
backend exposes a local path; for backends that don't (S3, Redis, …) the CLI
falls back to fetching the raw bytes with `StorageGetRaw`. Direct-mode entries
are exchanged with `StorageGetPreprocessorEntry` / `StoragePutPreprocessorEntry`.
Because each CLI process accumulates its own statistics, it flushes them to the
daemon with `RecordStats` before exiting.

Client-side mode is enabled with the `SCCACHE_CLIENT_SIDE` environment variable
(or the `client_side_mode` config key). It is expected to become the only
supported configuration in the future, with server-side mode eventually
removed. It is currently mutually exclusive with:

- **error logging** (`SCCACHE_ERROR_LOG`): the client always logs to stderr, and
  multiple concurrent CLI processes would race on the log file, and
- **distributed compilation** (a configured scheduler URL).

If either is in use, the setting is ignored and sccache stays in server-side
mode.

---

## File: docs/Azure.md

# Azure

sccache can store its cache in an Azure Blob Storage container using either a
shared-key connection string or Microsoft Entra ID (passwordless) authentication.
In both cases set `SCCACHE_AZURE_BLOB_CONTAINER` to the name of an _existing_
container — sccache will not create it for you.

## Shared key (connection string)

Set the `SCCACHE_AZURE_CONNECTION_STRING` environment variable to your storage
account connection string.

## Microsoft Entra ID (passwordless)

Use this when the storage account has shared-key access disabled. Instead of a
connection string, tell sccache which account to talk to with one of the
following (if both are set, `SCCACHE_AZURE_ENDPOINT` takes precedence):

* `SCCACHE_AZURE_STORAGE_ACCOUNT` — the storage account name. sccache builds the
  endpoint `https://{account}.blob.core.windows.net`.
* `SCCACHE_AZURE_ENDPOINT` — a full blob endpoint, for sovereign clouds
  (`https://{account}.blob.core.usgovcloudapi.net`,
  `https://{account}.blob.core.chinacloudapi.cn`) or custom DNS. Takes
  precedence over `SCCACHE_AZURE_STORAGE_ACCOUNT`. It must use `https`; plain
  `http` is accepted only for a loopback host.

`SCCACHE_AZURE_CONNECTION_STRING` and `SCCACHE_AZURE_STORAGE_ACCOUNT` /
`SCCACHE_AZURE_ENDPOINT` are mutually exclusive.

> **Azurite:** the local emulator authenticates with its well-known shared key,
> so point sccache at it with `SCCACHE_AZURE_CONNECTION_STRING` (the shared-key
> section above), not the passwordless path — the latter would require Azurite's
> non-default OAuth mode plus an ambient Entra credential.

Credentials are then resolved from the ambient environment, in this order:

1. **Service principal** — `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`.
2. **Workload identity** (AKS / Kubernetes / federated OIDC) — `AZURE_TENANT_ID`,
   `AZURE_CLIENT_ID`, `AZURE_FEDERATED_TOKEN_FILE`.
3. **Managed identity** via the instance-metadata endpoint (`169.254.169.254`) —
   Azure **VM / VM Scale Sets** and self-hosted CI agents running on them. No
   variables required; set `AZURE_CLIENT_ID` to select a specific user-assigned
   identity.

> **Managed-identity limitations.** App Service, Functions, Container Apps, and
> Container Instances expose managed identity only over the `IDENTITY_ENDPOINT`
> protocol, which this backend does not use — prefer **workload identity** or a
> **service principal** on those hosts. In any environment with an HTTP proxy
> configured, add `169.254.169.254` to `NO_PROXY`, or the metadata request is
> routed to the proxy and fails. Credentials resolve lazily, so a misconfigured
> passwordless setup builds successfully and fails only on the first cache
> access, not at server startup.

For **sovereign clouds**, `AZURE_AUTHORITY_HOST` is **required** for the service
principal and workload identity flows (it defaults to
`https://login.microsoftonline.com`): use `https://login.microsoftonline.us`
(US Gov) or `https://login.partner.microsoftonline.cn` (China). Managed identity
via IMDS does not need it. The identity needs a data-plane role on the container:
**Storage Blob Data Contributor** for `READ_WRITE` (the default), or **Storage
Blob Data Reader** when `SCCACHE_AZURE_RW_MODE=READ_ONLY`. sccache does not read
or log any `AZURE_*` value — they are consumed directly by the underlying storage
SDK when signing requests with an OAuth bearer token.

> **Note:** the underlying storage SDK also honours the ambient
> `AZBLOB_ACCOUNT_KEY` / `AZBLOB_SAS_TOKEN` variables, which take precedence over
> Entra ID credentials. Unset them if you intend to authenticate passwordlessly.

## Common options

You can define a prefix that will be prepended to the keys of all cache objects
created and read within the container, effectively creating a scope. To do that
use the `SCCACHE_AZURE_KEY_PREFIX` environment variable. This can be useful when
sharing a container with another application.

The `SCCACHE_AZURE_RW_MODE` environment variable can be set to `READ_ONLY` to
make sccache use this backend in read-only mode. The default is `READ_WRITE`.

**Important:** The environment variables are only taken into account when the
server starts, i.e. only on the first run.

---

## File: docs/Caching.md

# How caching works

To know if the storage contains the artifact we need, we are
computing some hashes to make sure the input is the same.

Because the configuration and environment matter, the hash
computation takes a few parameters into account.

## How hash keys are computed.

### Rust

We generate a blake3 digest for each file compiled.
In parallel, we also take into account in the hash:
* Path to the rustc executable
* Host triple for this rustc
* Path to the rustc sysroot
* digests of all the shared libraries in rustc's $sysroot/lib
* A shared, caching reader for rlib dependencies (for dist-client)
* Parsed arguments from the rustc invocation
  
See https://github.com/mozilla/sccache/blob/8567bbe2ba493153e76177c1f9a6f98cc7ba419f/src/compiler/rust.rs#L122 for the full list

### C/C++ compiler

For C/C++, the hash is generated with a blake3 digest of the preprocessed
file (-E with gcc/clang). For compilations that specify multiple `-arch` flags,
these flags are rewritten to their corresponding preprocessor defines to allow
pre-processing the file (e.g `-arch x86_64` is rewritten to `-D__X86_64__=1`),
this can be enabled by setting the environment variable
`SCCACHE_CACHE_MULTIARCH` but is disabled by default as it may not work in all
cases.

We also take into account in the hash:
* Hash of the compiler binary
* Programming language
* Flag required to compile for the given language
* File in which to generate dependencies.
* Commandline arguments for dependency generation
* Commandline arguments for the preprocessor
* Commandline arguments specifying the architecture to compile for
* Extra files that need to have their contents hashed
* Whether the compilation is generating profiling or coverage data
* Color mode
* Environment variables

See https://github.com/mozilla/sccache/blob/8567bbe2ba493153e76177c1f9a6f98cc7ba419f/src/compiler/c.rs#L84

### C/C++ preprocessor

In "preprocessor cache mode", [explained in the local doc](Local.md), an
extra key is computed to cache the preprocessor output itself. It is very close
to the C/C++ compiler one, but with additional elements:

* The path of the input file
* The hash of the input file

Note that some compiler options can disable preprocessor cache mode. As of this
writing, only `-Xpreprocessor` and `-Wp,*` do.

---

## File: docs/Configuration.md

# Available Configuration Options

## file

```
/* Detailed source-code truncated for AI context efficiency. */
```

sccache looks for its configuration file at the path indicated by env variable `SCCACHE_CONF`.

If no such env variable is set, sccache looks at default locations as below:
- Linux: `~/.config/sccache/config`
- macOS: `~/Library/Application Support/Mozilla.sccache/config`
- Windows: `%APPDATA%\Mozilla\sccache\config\config`

The latest `cache.XXX` entries may be found here: https://github.com/mozilla/sccache/blob/ffe3070f77ef3301c8ff718316e4ab017ec83042/src/config.rs#L300.

## env

Whatever is set by a file based configuration, it is overruled by the env
configuration variables

Note that some env variables may need sccache server restart to take effect.

### misc

* `SCCACHE_ALLOW_CORE_DUMPS` to enable core dumps by the server
* `SCCACHE_CONF` configuration file path
* `SCCACHE_BASEDIRS` base directory (or directories) to strip from paths for cache key computation. This is similar to ccache's `CCACHE_BASEDIR` and enables cache hits across different absolute paths when compiling the same source code. Multiple directories can be separated by `;` on Windows hosts and by `:` on any other operating system. When multiple directories are specified, the longest matching prefix is used. Path matching is **case-insensitive** on Windows and **case-sensitive** on other operating systems. Environment variable takes precedence over file configuration. Only absolute paths are supported; relative paths will cause an error and prevent the server from start.
* `SCCACHE_CACHED_CONF`
* `SCCACHE_IDLE_TIMEOUT` how long the local daemon process waits for more client requests before exiting, in seconds. Set to `0` to run sccache permanently
* `SCCACHE_STARTUP_NOTIFY` specify a path to a socket which will be used for server completion notification
* `SCCACHE_MAX_FRAME_LENGTH` how much data can be transferred between client and server
* `SCCACHE_NO_DAEMON` set to `1` to disable putting the server to the background
* `SCCACHE_CLIENT_SIDE` set to `1` to run the compile in the client process and use the daemon only as a gateway to the cache storage (see [the architecture doc](Architecture.md#client-side-mode-sccache_client_side)). This is the recommended mode and is expected to become the only supported configuration in the future. Ignored when `SCCACHE_ERROR_LOG` or distributed compilation is in use.
* `SCCACHE_CACHE_MULTIARCH` to disable caching of multi architecture builds.
* `SCCACHE_CACHE_ZSTD_LEVEL` to set zstd compression level of cache. the range is `1-22` and default is `3`.
  - For example, in `10`, it have about 0.9x size with about 1.6x time than default `3` (tested with compiling sccache code)
  - This option will only applied to newly compressed cache and don't affect existing cache.
  - If you want to be apply to all cache, you should reset cache and make new cache.
* `SCCACHE_LOG_MILLIS` when set (to any value), enables millisecond precision timestamps in log output instead of the default second precision.
* `SCCACHE_ERROR_LOG` path to a file where sccache will log errors
* `SCCACHE_LOG` log level, accepting standard env_logger values, see [env_logger documentation](https://docs.rs/env_logger/latest/env_logger/#enabling-logging) for details

### cache configs

#### multi-level cache

Multi-level caching enables hierarchical cache storage with automatic backfill. See the [Multi-Level Cache documentation](MultiLevel.md) for detailed information.

* `SCCACHE_MULTILEVEL_CHAIN` comma-separated list of cache backend names to use in hierarchy (e.g., `disk,redis,s3`)
  - Order matters: left-to-right is fast-to-slow (L0, L1, L2, ...)
  - Valid names: `disk`, `redis`, `memcached`, `s3`, `gcs`, `azure`, `gha`, `webdav`, `oss`, `cos`
  - Each level must be separately configured with its own environment variables
  - If not set, sccache uses single-level mode (legacy behavior)
* `SCCACHE_MULTILEVEL_WRITE_ERROR_POLICY` controls error handling on cache writes (default: `l0`)
  - `ignore` - never fail on write errors, log warnings only (most permissive)
  - `l0` - fail only if L0 (first level) write fails (default, balances reliability and performance)
  - `all` - fail if any read-write level fails (most strict)
  - Read-only levels are always skipped and never cause failures

**Basic example**:
```bash
export SCCACHE_MULTILEVEL_CHAIN="disk,redis,s3"
export SCCACHE_DIR="/tmp/cache"              # for disk level
export SCCACHE_REDIS_ENDPOINT="redis://..."  # for redis level
export SCCACHE_BUCKET="my-bucket"            # for s3 level
```

**Write policy examples**:
```bash
# Default: Fail only if disk write fails
export SCCACHE_MULTILEVEL_WRITE_ERROR_POLICY="l0"

# Best effort: Never fail on cache writes
export SCCACHE_MULTILEVEL_WRITE_ERROR_POLICY="ignore"

# Strict: Fail if any level write fails
export SCCACHE_MULTILEVEL_WRITE_ERROR_POLICY="all"
```

#### disk (local)

* `SCCACHE_DIR` local on disk artifact cache directory
* `SCCACHE_CACHE_SIZE` maximum size of the local on disk cache i.e. `2G` - default is 10G
* `SCCACHE_DIRECT` enable/disable preprocessor caching (see [the local doc](Local.md))
* `SCCACHE_LOCAL_RW_MODE` the mode that the cache will operate in (`READ_ONLY` or `READ_WRITE`)

#### s3 compatible

* `SCCACHE_BUCKET` s3 bucket to be used
* `SCCACHE_ENDPOINT` s3 endpoint
* `SCCACHE_REGION` s3 region, required if using AWS S3
* `SCCACHE_S3_USE_SSL` s3 endpoint requires TLS, set this to `true`
* `SCCACHE_S3_KEY_PREFIX` s3 key prefix (optional)
* `SCCACHE_S3_RW_MODE` allows to use s3 backend in read-only mode if set to `READ_ONLY`

The endpoint used then becomes `${SCCACHE_BUCKET}.s3-{SCCACHE_REGION}.amazonaws.com`.
If you are not using the default endpoint and `SCCACHE_REGION` is undefined, it
will default to `us-east-1`.

#### cloudflare r2

* `SCCACHE_BUCKET` is the name of your R2 bucket.
* `SCCACHE_ENDPOINT` must follow the format of `https://<ACCOUNT_ID>.r2.cloudflarestorage.com`. Note that the `https://` must be included. Your account ID can be found [here](https://developers.cloudflare.com/fundamentals/get-started/basic-tasks/find-account-and-zone-ids/).
* `SCCACHE_REGION` should be set to `auto`.
* `SCCACHE_S3_KEY_PREFIX` s3 key prefix (optional).

#### redis

* `SCCACHE_REDIS` full redis url, including auth and access token/passwd (deprecated).
* `SCCACHE_REDIS_ENDPOINT` redis url without auth and access token/passwd - single node configuration.
* `SCCACHE_REDIS_CLUSTER_ENDPOINTS` redis cluster urls, separated by comma - shared cluster configuration.
* `SCCACHE_REDIS_USERNAME` redis username (optional).
* `SCCACHE_REDIS_PASSWORD` redis password (optional).
* `SCCACHE_REDIS_DB` redis database (optional, default is 0).
* `SCCACHE_REDIS_EXPIRATION` / `SCCACHE_REDIS_TTL` ttl for redis cache, don't set for default behavior.
* `SCCACHE_REDIS_KEY_PREFIX` key prefix (optional).
* `SCCACHE_REDIS_RW_MODE` allows to use redis backend in read-only mode if set to `READ_ONLY`

The full url appears then as `redis://user:passwd@1.2.3.4:6379/?db=1`.

#### memcached

* `SCCACHE_MEMCACHED` is a deprecated alias for `SCCACHE_MEMCACHED_ENDPOINT`.
* `SCCACHE_MEMCACHED_ENDPOINT` memcached url.
* `SCCACHE_MEMCACHED_USERNAME` memcached username (optional).
* `SCCACHE_MEMCACHED_PASSWORD` memcached password (optional).
* `SCCACHE_MEMCACHED_EXPIRATION` ttl for memcached cache, don't set for default behavior.
* `SCCACHE_MEMCACHED_KEY_PREFIX` key prefix (optional).
* `SCCACHE_MEMCACHED_RW_MODE` allows to use memcached backend in read-only mode if set to `READ_ONLY`

#### gcs

* `SCCACHE_GCS_BUCKET`
* `SCCACHE_GCS_CREDENTIALS_URL`
* `SCCACHE_GCS_KEY_PATH`
* `SCCACHE_GCS_RW_MODE`

#### azure

* `SCCACHE_AZURE_BLOB_CONTAINER`
* `SCCACHE_AZURE_CONNECTION_STRING` shared-key connection string. Mutually exclusive with the Entra ID variables below.
* `SCCACHE_AZURE_STORAGE_ACCOUNT` storage account name for Microsoft Entra ID (passwordless) auth; the endpoint `https://{account}.blob.core.windows.net` is synthesized.
* `SCCACHE_AZURE_ENDPOINT` full blob endpoint override for Entra ID auth (sovereign clouds, custom DNS); must be `https` (plain `http` only for a loopback host); takes precedence over `SCCACHE_AZURE_STORAGE_ACCOUNT`.
* `SCCACHE_AZURE_KEY_PREFIX`
* `SCCACHE_AZURE_RW_MODE`

When using Entra ID, credentials are read from the ambient `AZURE_*` environment (`AZURE_TENANT_ID` / `AZURE_CLIENT_ID` / `AZURE_CLIENT_SECRET` for a service principal, `AZURE_FEDERATED_TOKEN_FILE` for workload identity, or IMDS for a managed identity). See [Azure.md](Azure.md).

#### gha

* `SCCACHE_GHA_CACHE_URL` / `ACTIONS_RESULTS_URL` GitHub Actions cache API URL
* `SCCACHE_GHA_RUNTIME_TOKEN` / `ACTIONS_RUNTIME_TOKEN` GitHub Actions access token
* `SCCACHE_GHA_CACHE_TO` cache key to write
* `SCCACHE_GHA_CACHE_FROM` comma separated list of cache keys to read from
* `SCCACHE_GHA_RW_MODE` allows to use GHA cache backend in read-only mode if set to `READ_ONLY`

#### webdav

* `SCCACHE_WEBDAV_ENDPOINT` a webdav service endpoint to store cache, such as `http://127.0.0.1:8080/my/webdav.php`.
* `SCCACHE_WEBDAV_KEY_PREFIX` specify the key prefix (subfolder) of cache (optional).
* `SCCACHE_WEBDAV_USERNAME` a username to authenticate with webdav service (optional).
* `SCCACHE_WEBDAV_PASSWORD` a password to authenticate with webdav service (optional).
* `SCCACHE_WEBDAV_TOKEN` a token to authenticate with webdav service (optional) - may be used instead of login & password.
* `SCCACHE_WEBDAV_RW_MODE` allows to use webdav backend in read-only mode if set to `READ_ONLY`

#### OSS

* `SCCACHE_OSS_BUCKET`
* `SCCACHE_OSS_ENDPOINT`
* `SCCACHE_OSS_KEY_PREFIX`
* `ALIBABA_CLOUD_ACCESS_KEY_ID`
* `ALIBABA_CLOUD_ACCESS_KEY_SECRET`
* `SCCACHE_OSS_NO_CREDENTIALS`
* `SCCACHE_OSS_RW_MODE`

#### Tencent Cloud Object Storage (COS)

* `SCCACHE_COS_BUCKET`
* `SCCACHE_COS_ENDPOINT`
* `SCCACHE_COS_KEY_PREFIX`
* `TENCENTCLOUD_SECRET_ID`
* `TENCENTCLOUD_SECRET_KEY`
* `SCCACHE_COS_RW_MODE`

---

## File: docs/Distributed.md

# Distributed sccache

Background:

 - You should read about JSON Web Tokens - https://jwt.io/.
   - HS256 in short: you can sign a piece of (typically unencrypted)
     data with a key. Verification involves signing the data again
     with the same key and comparing the result. As a result, if you
     want two parties to verify each others messages, the key must be
     shared beforehand.
 - Secure token's referenced below should be generated with a CSPRNG
   (your OS random number generator should suffice).
   For example, on Linux this is accessible with: `openssl rand -hex 64`.
 - When relying on random number generators (for generating keys or
   tokens), be aware that a lack of entropy is possible in cloud or
   virtualized environments in some scenarios.

## Overview

Distributed sccache consists of three parts:

 - the client, an sccache binary that wishes to perform a compilation on
   remote machines
 - the scheduler (`sccache-dist` binary), responsible for deciding where
   a compilation job should run
 - the server (`sccache-dist` binary), responsible for actually executing
   a build

All servers are required to be a 64-bit Linux or a FreeBSD install. Clients
may request compilation from Linux, Windows or macOS. Linux compilations will
attempt to automatically package the compiler in use, while Windows and macOS
users will need to specify a toolchain for cross-compilation ahead of time.

## Communication

The HTTP implementation of sccache has the following API, where all HTTP body content is encoded using [`bincode`](http://docs.rs/bincode):

 - scheduler
   - `POST /api/v1/scheduler/alloc_job`
      - Called by a client to submit a compilation request.
      - Returns information on where the job is allocated it should run.
   - `GET /api/v1/scheduler/server_certificate`
      - Called by a client to retrieve the (dynamically created) HTTPS
        certificate for a server, for use in communication with that server.
      - Returns a digest and PEM for the temporary server HTTPS certificate.
   - `POST /api/v1/scheduler/heartbeat_server`
      - Called (repeatedly) by servers to register as available for jobs.
   - `POST /api/v1/scheduler/job_state`
      - Called by servers to inform the scheduler of the state of the job.
   - `GET /api/v1/scheduler/status`
      - Returns information about the scheduler.
 - `server`
   - `POST /api/v1/distserver/assign_job`
      - Called by the scheduler to inform of a new job being assigned to this server.
      - Returns whether the toolchain is already on the server or needs submitting.
   - `POST /api/v1/distserver/submit_toolchain`
      - Called by the client to submit a toolchain.
   - `POST /api/v1/distserver/run_job`
      - Called by the client to run a job.
      - Returns the compilation stdout along with files created.

There are three axes of security in this setup:

1. Can the scheduler trust the servers?
2. Is the client permitted to submit and run jobs?
3. Can third parties see and/or modify traffic?

### Server Trust

If a server is malicious, they can return malicious compilation output to a user.
To protect against this, servers must be authenticated to the scheduler. You have three
means for doing this, and the scheduler and all servers must use the same mechanism.

Once a server has registered itself using the selected authentication, the scheduler
will trust the registered server address and use it for builds.

#### JWT HS256 (preferred)

This method uses secret key to create a per-IP-and-port token for each server.
Acquiring a token will only allow participation as a server if the attacker can
additionally impersonate the IP and port the token was generated for.

You *must* keep the secret key safe.

*To use it*:

Create a scheduler key with `sccache-dist auth generate-jwt-hs256-key` (which will
use your OS random number generator) and put it in your scheduler config file as
follows:

```
server_auth = { type = "jwt_hs256", secret_key = "YOUR_KEY_HERE" }
```

Now generate a token for the server, giving the IP and port the scheduler and clients can
connect to the server on (address `192.168.1.10:10501` here):

```
sccache-dist auth generate-jwt-hs256-server-token \
    --secret-key YOUR_KEY_HERE \
    --server 192.168.1.10:10501
```

*or:*

```
sccache-dist auth generate-jwt-hs256-server-token \
    --config /path/to/scheduler-config.toml \
    --server 192.168.1.10:10501
```

This will output a token (you can examine it with https://jwt.io if you're
curious) that you should add to your server config file as follows:

```
scheduler_auth = { type = "jwt_token", token = "YOUR_TOKEN_HERE" }
```

Done!

#### Token

This method simply shares a token between the scheduler and all servers. A token
leak from anywhere allows any attacker to participate as a server.

*To use it*:

Choose a 'secure token' you can share between your scheduler and all servers. The token must be a valid HTTP header value.

Put the following in your scheduler config file:

```
server_auth = { type = "token", token = "YOUR_TOKEN_HERE" }
```

Put the following in your server config file:

```
scheduler_auth = { type = "token", token = "YOUR_TOKEN_HERE" }
```

Done!

#### Insecure (bad idea)

*This route is not recommended*

This method uses a hardcoded token that effectively disables authentication and
provides no security at all.

*To use it*:

Put the following in your scheduler config file:

```
server_auth = { type = "DANGEROUSLY_INSECURE" }
```

Put the following in your server config file:

```
scheduler_auth = { type = "DANGEROUSLY_INSECURE" }
```

Done!

### Client Trust

If a client is malicious, they can cause a DoS of distributed sccache servers or
explore ways to escape the build sandbox. To protect against this, clients must
be authenticated.

Each client will use an authentication token for the initial job allocation request
to the scheduler. A successful allocation will return a job token that is used
to authorise requests to the appropriate server for that specific job.

This job token is a JWT HS256 token of the job id, signed with a server key.
The key for each server is randomly generated on server startup and given to
the scheduler during registration. This means that the server can verify users
without either a) adding client authentication to every server or b) needing
secret transfer between scheduler and server on every job allocation.

#### OAuth2

This is a group of similar methods for achieving the same thing - the client
retrieves a token from an OAuth2 service, and then submits it to the scheduler
which has a few different options for performing validation on that token.

*To use it*:

Put one of the following settings in your scheduler config file to determine how
the scheduler will validate tokens from the client:

```
# Use the known settings for Mozilla OAuth2 token validation
client_auth = { type = "mozilla" }

# Will forward the valid JWT token onto another URL in the `Bearer` header, with a
# success response indicating the token is valid. Optional `cache_secs` how long
# to cache successful authentication for.
client_auth = { type = "proxy_token", url = "...", cache_secs = 60 }
```

Additionally, each client should set up an OAuth2 configuration in the with one of
the following settings (as appropriate for your OAuth service):

```
# Use the known settings for Mozilla OAuth2 authentication
auth = { type = "mozilla" }

# Use the Authorization Code with PKCE flow. This requires a client id,
# an initial authorize URL (which may have parameters like 'audience' depending
# on your service) and the URL for retrieving a token after the browser flow.
auth = { type = "oauth2_code_grant_pkce", client_id = "...", auth_url = "...", token_url = "..." }

# Use the Implicit flow (typically not recommended due to security issues). This requires
# a client id and an authorize URL (which may have parameters like 'audience' depending
# on your service).
auth = { type = "oauth2_implicit", client_id = "...", auth_url = "..." }
```

The client should then run `sccache --dist-auth` and follow the instructions to retrieve
a token. This will be automatically cached locally for the token expiry period (manual
revalidation will be necessary after expiry).

#### Token

This method simply shares a token between the scheduler and all clients. A token
leak from anywhere allows any attacker to participate as a client.

*To use it*:

Choose a 'secure token' you can share between your scheduler and all clients. The token must be a valid HTTP header value.

Put the following in your scheduler config file:

```
client_auth = { type = "token", token = "YOUR_TOKEN_HERE" }
```

Put the following in your client config file:

```
auth = { type = "token", token = "YOUR_TOKEN_HERE" }
```

Done!

#### Insecure (bad idea)

*This route is not recommended*

This method uses a hardcoded token that effectively disables authentication and
provides no security at all.

*To use it*:

Put the following in your scheduler config file:

```
client_auth = { type = "DANGEROUSLY_INSECURE" }
```

Remove any `auth =` setting under the `[dist]` heading in your client config file
(it will default to this insecure mode).

Done!

### Eavesdropping and Tampering Protection

If third parties can see traffic to the servers, source code can be leaked. If third
parties can modify traffic to and from the servers or the scheduler, they can cause
the client to receive malicious compiled objects.

Securing communication with the scheduler is the responsibility of the sccache cluster
administrator - it is recommended to put a webserver with a HTTPS certificate in front
of the scheduler and instruct clients to configure their `scheduler_url` with the
appropriate `https://` address. The scheduler will verify the server's IP in this
configuration by inspecting the `X-Real-IP` header's value, if present. The webserver
used in this case should be configured to set this header to the appropriate value.

Securing communication with the server is performed automatically - HTTPS certificates
are generated dynamically on server startup and communicated to the scheduler during
the heartbeat. If a client does not have the appropriate certificate for communicating
securely with a server (after receiving a job allocation from the scheduler), the
certificate will be requested from the scheduler.

## Configuration

Use the `--config` argument to pass the path to its configuration file to `sccache-dist`.


### scheduler.toml

```toml
# The socket address the scheduler will listen on. It's strongly recommended
# to listen on localhost and put a HTTPS server in front of it.
public_addr = "127.0.0.1:10600"

[client_auth]
type = "token"
token = "my client token"

[server_auth]
type = "jwt_hs256"
secret_key = "my secret key"
```


#### [client_auth]

The `[client_auth]` section can be one of (sorted by authentication method):
```toml
# OAuth2
[client_auth]
type = "mozilla"

client_auth = { type = "proxy_token", url = "...", cache_secs = 60 }

# JWT
[client_auth]
type = "jwt_validate"
audience = "audience"
issuer = "issuer"
jwks_url = "..."

# Token
[client_auth]
type = "token"
token = "preshared token"

# None
[client_auth]
type = "DANGEROUSLY_INSECURE"
```


#### [server_auth]

The `[server_auth]` section can be can be one of:
```toml
[server_auth]
type = "jwt_hs256"
secret_key = "my secret key"

[server_auth]
type = "token"
token = "preshared token"

[server_auth]
type = "DANGEROUSLY_INSECURE"
```

### server.toml


```toml
# This is where client toolchains will be stored.
cache_dir = "/tmp/toolchains"
# The maximum size of the toolchain cache, in bytes.
# If unspecified the default is 10GB.
#toolchain_cache_size = 10737418240
# A public IP address and port that clients will use to connect to this builder.
public_addr = "192.168.1.1:10501"
# The socket address the builder will listen on. Falls back to public_addr.
#bind_address = "0.0.0.0:10501"
# The URL used to connect to the scheduler (should use https, given an ideal
# setup of a HTTPS server in front of the scheduler)
scheduler_url = "https://192.168.1.1"

[builder]
type = "overlay"
# The directory under which a sandboxed filesystem will be created for builds.
build_dir = "/tmp/build"
# The path to the bubblewrap version 0.3.0+ `bwrap` binary.
bwrap_path = "/usr/bin/bwrap"

[scheduler_auth]
type = "jwt_token"
# This will be generated by the `generate-jwt-hs256-server-token` command or
# provided by an administrator of the sccache cluster.
token = "my server's token"
```


#### [builder]

The `[builder]` section can be can be one of:
```toml
[builder]
type = "docker"

[builder]
type = "overlay"
# The directory under which a sandboxed filesystem will be created for builds.
build_dir = "/tmp/build"
# The path to the bubblewrap version 0.3.0+ `bwrap` binary.
bwrap_path = "/usr/bin/bwrap"

[builder]
type = "pot"
# Pot filesystem root
#pot_fs_root = "/opt/pot"
# Reference pot cloned when creating containers
#clone_from = "sccache-template"
# Command to invoke when calling pot
#pot_cmd = "pot"
# Arguments passed to `pot clone` command
#pot_clone_args = ["-i", "lo0|127.0.0.2"]

```


#### [scheduler_auth]

The `[scheduler_auth]` section can be can be one of:
```toml
[scheduler_auth]
type = "jwt_token"
token = "my server's token"

[scheduler_auth]
type = "token"
token = "preshared token"

[scheduler_auth]
type = "DANGEROUSLY_INSECURE"
```


# Building the Distributed Server Binaries

Until these binaries [are included in releases](https://github.com/mozilla/sccache/issues/393) I've put together a Docker container that can be used to easily build a release binary:
```
docker run -ti --rm -v $PWD:/sccache luser/sccache-musl-build:0.1 /bin/bash -c "cd /sccache; cargo build --release --target x86_64-unknown-linux-musl --features=dist-server && strip target/x86_64-unknown-linux-musl/release/sccache-dist && cd target/x86_64-unknown-linux-musl/release/ && tar czf sccache-dist.tar.gz sccache-dist"
```

---

## File: docs/DistributedFreeBSD.md

Distributed sccache on FreeBSD
==============================

Please read the [the distributed quickstart](DistributedQuickstart.md)
guide first.

Build and install from source
-----------------------------

```
cargo install --features="dist-client,dist-server" --path=.
```

Configure a FreeBSD build server
--------------------------------

On FreeBSD, the build server requires [pot](https://github.com/bsdpot/pot)
to sandbox execution:

```sh
pkg install pot
```

It's up to the user to create the reference pot that serves as a template
to clone from when instantiating image and build containers, e.g.:

```sh
pot create -p sccache-template -N alias -i "lo0|127.0.0.2" -t single -b 15.0
pot set-cmd -p sccache-template -c /usr/bin/true
pot set-attr -p sccache-template -A no-rc-script -V YES
pot snapshot -p sccache-template
```

Then, a server.conf like the one below is created, making use of the `pot`
builder type (commented out options show defaults):

```toml
# This is where client toolchains will be stored.
cache_dir = "/tmp/toolchains"
# The maximum size of the toolchain cache, in bytes.
# If unspecified the default is 10GB.
# toolchain_cache_size = 10737418240
# A public IP address and port that clients will use to connect to this builder.
public_addr = "192.168.1.1:10501"
# The URL used to connect to the scheduler (should use https, given an ideal
# setup of a HTTPS server in front of the scheduler)
scheduler_url = "https://192.168.1.1"

[builder]
type = "pot"
# Pot filesystem root
#pot_fs_root = "/opt/pot"
# Reference pot cloned when creating containers
#clone_from = "sccache-template"
# Command to invoke when calling pot
#pot_cmd = "pot"
# Arguments passed to `pot clone` command
#pot_clone_args = ["-i", "lo0|127.0.0.2"]

[scheduler_auth]
type = "jwt_token"
# This will be generated by calling
# `sccache-dist auth generate-jwt-hs256-server-token` or
# provided by an administrator of the sccache cluster.
token = "my server's token"
```

FreeBSD as a build client
-------------------------

On a FreeBSD client, make sure to add the right toolchains to
`~/.config/sccache/config`:

```toml
[dist]
# The URL used to connect to the scheduler (should use https, given an ideal
# setup of a HTTPS server in front of the scheduler)
scheduler_url = "http://127.0.0.1:10600"
# Used for mapping local toolchains to remote cross-compile toolchains. Empty in
# this example where the client and build server are both Linux.
#toolchains = []
# Size of the local toolchain cache, in bytes (5GB here, 10GB if unspecified).
toolchain_cache_size = 5368709120
cache_dir = "/home/user/.cache/sccache-dist-client"

[dist.auth]
type = "token"
# This should match the `client_auth` section of the scheduler config
# and was generated by calling `sccache-dist auth generate-jwt-hs256-key`
token = "my client token"

[[dist.toolchains]]
type = "path_override"
compiler_executable = "/usr/bin/cc"
archive = "/path/to/empty.tar.gz"
archive_compiler_executable = "/usr/bin/cc"

[[dist.toolchains]]
type = "path_override"
compiler_executable = "/usr/local/bin/rustc"
archive = "/path/to/rust-toolchain.tgz"
archive_compiler_executable = "/usr/local/bin/rustc"
```

Creating toolchain archives
---------------------------

The toolchain files from the examples above can be created like this:

```sh
pkg install gtar
gtar cvf - --files-from /dev/null | gzip >empty.tar.gz
pkg info -lq rust | gtar -cf - -T - | gzip >rust-toolchain.tgz
```

This just creates an empty file for the system compiler (as it is
included in the pot image anyway) and the toolchain for rustc is
created from the rust package installed on the system.

See [the distributed quickstart](DistributedQuickstart.md) guide for
instructions how to create other C toolchains using icecc-create-env.

Note: We use `gtar` (GNU tar) here, as the [flate2](
https://github.com/rust-lang/flate2-rs) crate has issues processing
sparse files created with `bsdtar`.

Cargo invocation example
------------------------

```sh
RUSTC_WRAPPER=~/.cargo/bin/sccache \
cargo build
```

---

## File: docs/DistributedQuickstart.md

sccache distributed compilation quickstart
==========================================

This is a quick start guide to getting distributed compilation working with sccache. This guide primarily covers Linux clients.
macOS and Windows clients are supported but have seen significantly less testing.

Get sccache binaries
--------------------

Either [install pre-built sccache binaries](https://github.com/mozilla/sccache#installation), or build sccache locally with the `dist-client` and `dist-server` features enabled:
```
cargo build --release --features="dist-client dist-server"
```

The `target/release/sccache` binary will be used on the client, and the `target/release/sccache-dist` binary will be used on the scheduler and build server.

If you're only planning to use the client, it is enabled by default, so just `cargo install sccache` should do the trick.

Configure a scheduler
---------------------

If you're adding a server to a cluster that has already been set up, skip ahead to [configuring a build server](#configure-a-build-server).

The scheduler is a daemon that manages compile request from clients and parcels them out to build servers. You only need one of these per sccache setup. Currently only Linux is supported for running the scheduler.

Create a scheduler.conf file to configure client/server authentication. A minimal example looks like:
```toml
# The socket address the scheduler will listen on. It's strongly recommended
# to listen on localhost and put a HTTPS server in front of it.
public_addr = "127.0.0.1:10600"

[client_auth]
type = "token"
token = "my client token" # Must be a valid HTTP header value.

[server_auth]
type = "jwt_hs256"
secret_key = "my secret key"
```

Mozilla build servers will typically require clients to be authenticated with the
[Mozilla identity system](https://github.com/mozilla-iam/mozilla-iam).

To configure for scheduler for this, the `client_auth` section should be as follows
so any client tokens are validated with the Mozilla service:

```
[client_auth]
type = "mozilla"
required_groups = ["group_name"]
```

Where `group_name` is a Mozilla LDAP group. Users will be required to belong to this group to successfully authenticate with the scheduler.

Start the scheduler by running:
```
sccache-dist scheduler --config scheduler.conf
```

Like the local server, the scheduler process will daemonize itself unless `SCCACHE_NO_DAEMON=1` is set. If the scheduler fails to start you may need to set `SCCACHE_LOG=trace` when starting it to get useful diagnostics.

Configure a build server
------------------------

A build server communicates with the scheduler and executes compiles requested by clients. Only Linux is supported for running a build server, but executing cross-compile requests from macOS/Windows clients is supported. You can also run a build server on FreeBSD, please see [distributed sccache on FreeBSD](DistributedFreeBSD.md).

The build server requires [bubblewrap](https://github.com/projectatomic/bubblewrap) to sandbox execution, at least version 0.3.0. Verify your version of bubblewrap *before* attempting to run the server. On Ubuntu 18.10+ you can `apt install bubblewrap` to install it. If you build from source you will need to first install your distro's equivalent of the `libcap-dev` package.

Create a server.conf file to configure authentication, storage locations, network addresses and the path to bubblewrap. A minimal example looks like:
```toml
# This is where client toolchains will be stored.
cache_dir = "/tmp/toolchains"
# The maximum size of the toolchain cache, in bytes.
# If unspecified the default is 10GB.
# toolchain_cache_size = 10737418240
# A public IP address and port that clients will use to connect to this builder.
public_addr = "192.168.1.1:10501"
# The URL used to connect to the scheduler (should use https, given an ideal
# setup of a HTTPS server in front of the scheduler)
scheduler_url = "https://192.168.1.1"

[builder]
type = "overlay"
# The directory under which a sandboxed filesystem will be created for builds.
build_dir = "/tmp/build"
# The path to the bubblewrap version 0.3.0+ `bwrap` binary.
bwrap_path = "/usr/bin/bwrap"

[scheduler_auth]
type = "jwt_token"
# This will be generated by the `generate-jwt-hs256-server-token` command or
# provided by an administrator of the sccache cluster.
token = "my server's token"
```

Due to bubblewrap requirements currently the build server *must* be run as root. Start the build server by running:
```
sudo sccache-dist server --config server.conf
```

As with the scheduler, if the build server fails to start you may need to set `SCCACHE_LOG=trace` to get useful diagnostics.

Configure a client
------------------

A client uses `sccache` to wrap compile commands, communicates with the scheduler to find available build servers, and communicates with build servers to execute the compiles and receive the results.

Clients that are not targeting linux64 require the `icecc-create-env` script or should be provided with an archive. `icecc-create-env` is part of `icecream` for packaging toolchains. You can install icecream to get this script (`apt install icecc` on Ubuntu), or download it from the git repository and place it in your `PATH`: `curl https://raw.githubusercontent.com/icecc/icecream/master/client/icecc-create-env.in > icecc-create-env && chmod +x icecc-create-env`. See [using custom toolchains](#using-custom-toolchains).

Create a client config file in `~/.config/sccache/config` (on Linux), `~/Library/Application Support/Mozilla.sccache/config` (on macOS), or `%APPDATA%\Mozilla\sccache\config\config` (on Windows). A minimal example looks like:
```toml
[dist]
# The URL used to connect to the scheduler (should use https, given an ideal
# setup of a HTTPS server in front of the scheduler)
scheduler_url = "https://192.168.1.1"
# Used for mapping local toolchains to remote cross-compile toolchains. Empty in
# this example where the client and build server are both Linux.
toolchains = []
# Size of the local toolchain cache, in bytes (5GB here, 10GB if unspecified).
toolchain_cache_size = 5368709120

[dist.auth]
type = "token"
# This should match the `client_auth` section of the scheduler config.
token = "my client token" # Must be a valid HTTP header value.
```

Clients using Mozilla build servers should configure their `dist.auth` section as follows:

```
[dist.auth]
type = "mozilla"
```

And retrieve a token from the Mozilla identity service by running `sccache --dist-auth`
and following the instructions. Completing this process will retrieve and cache a token
valid for 7 days.

Make sure to run `sccache --stop-server` and `sccache --start-server` if sccache was
running before changing the configuration.

You can check the status with `sccache --dist-status`, it should say something like:

```
$ sccache --dist-status
{"SchedulerStatus":["https://sccache1.corpdmz.ber3.mozilla.com/",{"num_servers":3,"num_cpus":56,"in_progress":24}]}
```

Using custom toolchains
-----------------------

Since Windows and macOS cannot automatically package toolchains, it is important to be
able to manually specify toolchains for distribution. This functionality is also available
on Linux.

Using custom toolchains involves adding a `dist.toolchains` section to your client config
file (you can add it multiple times to specify multiple toolchains).

On Linux and macOS:

```
[[dist.toolchains]]
type = "path_override"
compiler_executable = "/home/me/.mozbuild/clang/bin/clang"
archive = "/home/me/.mozbuild/toolchains/33d92fcd79ffef6e-clang-dist-toolchain.tar.gz"
archive_compiler_executable = "/builds/worker/toolchains/clang/bin/clang"
```

On Windows:

```
[[dist.toolchains]]
type = "path_override"
compiler_executable = "C:/clang/bin\\clang-cl.exe"
archive = "C:/toolchains/33d92fcd79ffef6e-clang-dist-toolchain.tar.gz"
archive_compiler_executable = "/builds/worker/toolchains/clang/bin/clang"
```

Where:
 - `compiler_executable` identifies the path that sccache will match against to activate
   this configuration (you need to be careful on Windows - paths can have slashes in both
   directions, and you may need to escape backslashes, as in the example)
 - `archive` is the gzip-compressed tar archive containing the compiler toolchain to distribute
   when `compiler_executable` is matched
 - `archive_compiler_executable` is the path within the archive the distributed
   compilation should invoke

A toolchain archive should be a Gzip compressed TAR archive, containing a filesystem
sufficient to run the compiler without relying on any external files. If you have archives
compatible with icecream (created with `icecc-create-env`, like
[these ones](https://github.com/jyavenard/mozilla-icecream) for macOS), they should also work
with sccache. To create a Windows toolchain, it is recommended that you download the [Clang
binaries for Ubuntu 16.04](http://releases.llvm.org/download.html) and extract them,
package up the toolchain using the extracted `bin/clang` file (requires
[PR #321](https://github.com/mozilla/sccache/pull/321)) and then insert `bin/clang-cl` at
the appropriate path as a symlink to the `bin/clang` binary.

Considerations when distributing from macOS
-------------------------------------------

When distributing from a macOS client, additional flags and configuration
may be required:

- An explicit target should be passed to the compiler, for instance by adding
  `--target=x86_64-apple-darwin16.0.0` to your build system's `CFLAGS`.
- An explicit toolchain archive will need to be configured, as described above.
  In case rust is being cached, the same version of `rustc` will need to be used
  for local compiles as is found in the distributed archive.
- The client config will be read from `~/Library/Application Support/Mozilla.sccache/config`,
  not `~/.config/sccache/config`.
- Some cross compilers may not understand some intrinsics used in more recent macOS
  SDKs. The 10.11 SDK is known to work.

Making a build server start at boot time
----------------------------------------

It is very easy with a systemd service to spawn the server on boot.

You can create a service file like `/etc/systemd/system/sccache-server.service`
with the following contents:

```ini
[Unit]
Description=sccache-dist server
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/path/to/sccache-dist server --config /path/to/server.conf

[Install]
WantedBy=multi-user.target
```

**Note** that if the `sccache-dist` binary is in a user's home directory, and
you're in a distro with SELinux enabled (like Fedora), you may need to use an
`ExecStart` line like:

```ini
ExecStart=/bin/bash -c "/home//path/to/sccache-dist server --config /home//path/to/server.conf"
```

This is because SELinux by default prevents services from running binaries in
home directories, for some reason. Using a shell works around that. An
alternative would be to move the `sccache-dist` binary to somewhere like
`/usr/local/bin`, but then you need to remember to update it manually.

After creating that file, you can ensure it's working and enable it by default
like:

```
# systemctl daemon-reload
# systemctl start sccache-server
# systemctl status # And check it's fine.
# systemctl enable sccache-server # This enables the service on boot
```

---

## File: docs/Gcs.md

# Google Cloud Storage

To use [Google Cloud Storage](https://cloud.google.com/storage/), you need to set
the `SCCACHE_GCS_BUCKET` environment variable to the name of the GCS bucket.

By default, SCCACHE on GCS will be read-only. To change this, set `SCCACHE_GCS_RW_MODE`
to either `READ_ONLY` or `READ_WRITE`.

You can also define a prefix that will be prepended to the keys of all cache objects
created and read within the GCS bucket, effectively creating a scope. To do that
use the `SCCACHE_GCS_KEY_PREFIX` environment variable. This can be useful when
sharing a bucket with another application.

## Credentials

Sccache is able to load credentials from various sources. Including:

- User Input: If `SCCACHE_GCS_KEY_PATH` has been set, we will load from this file
  first.
  - Service accounts JSONs
  - External accounts JSONs
- [Task Cluster](https://taskcluster.net/): If `SCCACHE_GCS_CREDENTIALS_URL` has
  been set, we will load token from this url first.
- Static: `GOOGLE_APPLICATION_CREDENTIALS`
- Well-known locations:
  - Windows: `%APPDATA%\gcloud\application_default_credentials.json`
  - macOS/Linux:
    - `$XDG_CONFIG_HOME/gcloud/application_default_credentials.json`
    - `$HOME/.config/gcloud/application_default_credentials.json`
- VM Metadata: Fetch token will the specified service account.

### Service accounts

To create such account, in GCP, go in `APIs and Services` => `Cloud Storage` =>
`Create credentials` => `Service account`. Then, once created, click on the account
then `Keys` => `Add key` => `Create new key`. Select the JSON format and here it
is. This JSON file is what `SCCACHE_GCS_KEY_PATH` expects.

The service account needs `Storage Object Admin` permissions on the bucket
(otherwise, sccache will fail with a simple `Permission denied`).

### External accounts

Such accounts require creating a [Workload Identity Pool and Workload Identity Provider].
This approach allows the environment (Azure, Aws, or other OIDC providers like Github)
to create a temporary service account grant without having to share a service account
JSON, which can be pretty powerful. An example on how to create such accounts is
[Google's guide on how to use it with Github].

After generating the external account JSON file, you may pass its path to `SCCACHE_GCS_KEY_PATH`.

Service accounts used by the pool must have `Storage Object Admin` permissions on
bucket as well.

## Verifying it works

To verify that it works, run:

```
export SCCACHE_GCS_BUCKET=
export SCCACHE_GCS_KEY_PATH=secret-gcp-storage.json
./sccache --show-stats
# you should see
[...]
Cache location                  GCS, bucket: Bucket(name=), key_prefix: (none)
```

## Deprecation

`SCCACHE_GCS_OAUTH_URL` have been deprecated and not supported, please use `SCCACHE_GCS_SERVICE_ACCOUNT` instead.

[Workload Identity Pool and Workload Identity Provider]: https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers
[Google's guide on how to use it with Github]: https://cloud.google.com/blog/products/identity-security/enabling-keyless-authentication-from-github-actions

---

## File: docs/GHA.md

# GitHub Actions

To use the [GitHub Actions cache](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows), you need to set `SCCACHE_GHA_ENABLED` to `on` to enable it.

By changing `SCCACHE_GHA_VERSION`, we can purge all the cache.

This cache type will need tokens like `ACTIONS_RESULTS_URL` and `ACTIONS_RUNTIME_TOKEN` to work. You can set these environmental variables using the following step in a GitHub Actions workflow.

```yaml
- name: Configure sccache
  uses: actions/github-script@v7
  with:
    script: |
      core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || '');
      core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
```

The `SCCACHE_GHA_RW_MODE` environment variable can be set to `READ_ONLY` to make sccache use this backend in read-only mode. The default is `READ_WRITE`.

## Behavior

In case sccache reaches the rate limit of the service, the build will continue, but the storage might not be performed.