# Technical Documentation: metacraft-labs/codetracer
> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/metacraft-labs/codetracer)
### Overview (`README.md`)
[](https://github.com/metacraft-labs/codetracer/actions/workflows/codetracer.yml)
[](https://discord.gg/aH5WTMnKHT)
# Introduction
CodeTracer is a user-friendly time-traveling debugger designed to support a wide range of programming languages.
It records the execution of a program into a sharable self-contained trace file. You can load the produced trace files in a GUI environment that allows you to move forward and backward
through the execution and to examine the history of all memory locations. They say a picture is worth a thousand words — well, a video is even better! Watch the demo below to see CodeTracer in action:
[](https://www.youtube.com/watch?v=xZsJ55JVqmU)
## The Benefits of Time-Travel
Compared to traditional debuggers, CodeTracer gives you two major superpowers:
* **Once you capture a bug in a recording, consider it squashed!**
Bugs that are hard to reproduce can be painful to fix — you’ve surely been there. Once such a bug is captured with CodeTracer, you'll rarely need more than 30 minutes to track it down! This is largely a consequence of the next superpower:
* **Most bugs are easily revealed when you know the origin of any value in the program.**
All programs produce output. Some examples are bytes generated as a response to a web request, pixels being drawn on your screen or perhaps a simple log line written to the terminal.
When CodeTracer creates a recording, it captures a user-extensible set of output events relevant to the program. The GUI displays these events in a searchable chronological event log.
Consider a misbehaving program that prints unexpected output to a log file midway through its execution. Clicking on the specific output event in CodeTracer will take you to the precise moment and code line where it was generated.
The unexpected value must be originating from some variable that's being passed to the logging function. With CodeTracer, you can now ask the question "Where did this value come from?". CodeTracer will find another moment in the execution, potentially multiple seconds earlier, in a completely different part of the program where this particular memory location was last written to.
This could be memory corruption or a genuine logical error. Either way, CodeTracer will report the origin. Let's say that you end up in the correct function that is responsible for computing the problematic value, but another input there leads to the issue. You can continue the search by repeating the question "Where did this input come from"? It usually takes just a few of these jumps to earlier moments in time to arrive at the root cause for the bug.
Every time you jump to a new moment in the execution, you can fearlessly explore your surroundings by stepping forward or backwards, having access to a back button that can always get you to any previous point of interest. At every point of the journey, our novel UI shows you details about the past and future program states at a glance and you know your position in the precisely recorded call trace of the program.
These features combined, make for a truly powerful debugging experience.
## Current state of the project and 2025 roadmap
The initial release of CodeTracer supports the Noir programming language. It has been developed in collaboration with the Blocksense team and currently requires the use of the [Blocksense Noir Compiler](https://github.com/blocksense-network/noir), which is included in the CodeTracer distribution.
CodeTracer’s open-source development is made possible by the generous support of Aztec Network, Arbitrum Foundation, and Aptos Foundation. During 2025, CodeTracer will evolve into a comprehensive Web3 development toolkit by gaining support for many additional smart contract and zero-knowledge programming languages. Most of its functionality will be packaged into reusable libraries that will power the creation of block explorers with advanced transaction tracing capabilities and omniscient debugging plugins for Visual Studio Code and other IDEs.
CodeTracer uses an [open format](https://github.com/metacraft-labs/runtime_tracing) for its trace files and we've started several community-driven projects which aim to add support for other programming languages:
* [Ruby](https://github.com/metacraft-labs/codetracer-ruby-recorder)
* [Python](https://github.com/metacraft-labs/codetracer-python-recorder)
Metacraft Labs is also developing an alternative back-end, capable of working with [RR](https://rr-project.org/) recordings, which will make CodeTracer suitable for debugging large-scale programs in a variety of system programming languages such as C/C++, Rust, Nim, D, Zig, Go, Fortran and FreePascal.
To shape our priorities and to help us understand the demographics of our user base better, please fill out [CodeTracer Developer Preferences Survey](https://form.typeform.com/to/M2Z28VFj?utm_source=Github).
To accelerate our development, please consider donating to our Open Collective campaign. Anyone who contributes more than €50 before June 2025 will get early access to our beta releases for system programming languages, forever.
[](https://opencollective.com/codetracer)
## The features of CodeTracer in more depth
Once you have time travel, you can re-imagine how a debugger works from the ground up! Here are some of the features of CodeTracer that set it apart from traditional debuggers:
### Omniscience
When CodeTracer breaks at a certain line of code, it knows not only the past but also the future. You can immediately see the values of all variables in the current function right next to the source code. This includes variables in loops where you can easily scroll through the iterations. Taken code branches are highlighted, while the non-executed code is immediately grayed out.
### Tracepoints
Many developers tend to transition from using interactive debugging early on in their careers, to mostly relying on carefully placed print statements as they start to face harder problems in distributed and real-time systems.
CodeTracer gives you the best of both worlds. A tracepoint allows you to see the effects of adding additional code to your program, without recompiling and rerecording it.
The added code can feature if statements, additional function calls and various ways to print or plot the captured data. The output is typically produced in seconds.
### Call Trace
Instead of a stack trace limited to a single moment in time, CodeTracer shows you the entire tree of function calls in the recorded execution. You can navigate and filter it in various ways and the clever collapsing and expanding algorithms still allow you to obtain a classic stack trace whenever you need it.
### State and History Explorer
Every variable in CodeTracer has a history. You can see all the values that the variable held during the entire execution of the program and for each of them you can jump to the originating expression that computed it. CodeTracer follows the path of trivial copies in assignments to save you extra time when tracking down more convoluted bugs.
### Event Log
The event log gives you a chronological view over anything of interest in your program, interleaved with the tracepoint outputs. Clicking on any event takes you to the precise moment in time when it was produced, which gives you excellent starting points for your investigations. Tracking down the origins of anomalous events and program states through the history of the involved variables is the essence of debugging with CodeTracer.
### Terminal Output
The terminal output panel renders the recorded `stdout` and `stderr` events like a standard terminal. Clicking anywhere in the output takes you to the exact moment when the specific character was generated. This allows for a fallback to a more traditional print-style debugging when the expressivity of tracepoints is not sufficient. In the future, CodeTracer will allow you to slide through the states of the terminal through time which will help for debugging highly interactive TUI apps.
### Mouse Stepping
Since CodeTracer is aware of all past and future control flow, it offers some convenient shortcuts for quickly navigating the program execution by interacting with its source code:
* Jump to a line: Middle-click on any undimmed line in the current function to quickly step to it. If you use a mouse without a middle button, you can achieve the same with `Ctrl+Click` or by pressing `Ctrl+F8`(as a special kind of `continue`) after navigating to the respective line with your keyboard.
* Jump into a call: Double middle-click on the function name in a call expression to jump into it. If you use a mouse without a middle button, you can achieve the same with `Ctrl+Alt+Click` or by pressing `Ctrl+F11`(as a special kind of `step-in`) after navigating to the respective call expression with your keyboard.
Explore the right-click context menu for additional operations.
### Scratchpad
The scratchpad provides a play area where you can pin values from different locations and moments in time. You can explore their differences both manually and algorithmically to gain quick insights into the behavior of your program.
# Installing CodeTracer
## Recommended: using the CodeTracer installer
>
> [!WARNING]
> On NixOS, this script currently installs CodeTracer as a raw AppImage, since our Nix package is not yet merged in nixpkgs.
> Nix users which want to use the package should install it manually from source.
On any system, you can use the CodeTracer installer script by running the following command:
```
user $ curl -fsSL https://get.codetracer.com/sh | sh
```
On Debian, Ubuntu, Fedora and RHEL the installer registers the Metacraft Labs package
repository (`deb.metacraft-labs.com` / `rpm.metacraft-labs.com`) and installs the `codetracer`
package, so later updates arrive with `apt upgrade` / `dnf upgrade`. On Arch it uses your AUR
helper, on Gentoo the metacraft-overlay, and on macOS the signed DMG from downloads.codetracer.com.
You may be asked for sudo access depending on your installation method.
This script is handy, because it installs CodeTracer in the best possible manner depending on your OS.
> [!CAUTION]
> CodeTracer requires ruby to be installed through [homebrew](https://brew.sh) for ruby support. It's recommended that you install homebrew if possible
> so that you can automatically get ruby support through the installer.
## Installing CodeTracer binaries manually
Click on the icons below that corresponds to your distribution or packaging model:
PGP Key signatures:
[](https://downloads.codetracer.com/CodeTracer-latest-arm64.dmg.asc)
[](https://downloads.codetracer.com/CodeTracer-latest-amd64.AppImage.asc)
[](https://downloads.codetracer.com/CodeTracer.pub.asc)
> [!CAUTION]
> When launched for the first time, macOS users will see the error message "CodeTracer is damaged and can't be opened".
> To resolve this problem, please execute the command `user $ xattr -c `.
>
> We expect this inconvenience will be remedied soon through our enrollment in the Apple Developer program which will ensure CodeTracer is properly signed and whitelisted by Apple.
> See [this discussion](https://discussions.apple.com/thread/253714860?sortBy=rank) for more details.
> [!CAUTION]
> Recording ruby on macOS requires you to install ruby through [homebrew](https://brew.sh), otherwise trying to record ruby programs will fail
> due to the built-in ruby binary on macOS being more than 7 years old.
>
> Once homebrew is installed, simply install ruby with `user $ brew install ruby`.
## Building from source
### Linux
The Metacraft Labs team is using [Nix](https://nixos.org/) to define a reproducible development environment for working on CodeTracer. Linux is our primary development platform, while some of our team members use macOS. Building on Windows will be supported in the near future.
To enter the Nix development environment, perform the following steps:
1) [Install Nix](https://zero-to-nix.com/start/install/).
2) Clone this repository.
3) At the repository root, execute `git submodule update --init --recursive`.
4) At the repository root, execute `nix develop` (or `direnv allow` for users of `direnv`).
5) In the resulting shell, you can build all targets by running `just build-once` or `just build` if you intend to make continuous changes to the source code.
#### NixOS
On NixOS, you can install our Nix package by running `just build-nix`.
### macOS
On macOS, we do not yet have support for Nix, instead we compile by using the non-nix-build scripts:
1. Clone this repository
1. Open a terminal and run `bash non-nix-build/build.sh`
1. Wait until it finishes installing and packaging
1. The end result will be a `CodeTracer.dmg` file that can be found under the `non-nix-build` folder.
1. For quicker testing without manually unpacking CodeTracer every time you can experiment with the contents of the `non-nix-build/CodeTracer.app` folder, which is equivalent to the final app bundle
### Windows (non-Nix bootstrap)
For Windows local development without Nix, use the scripts under `non-nix-build/windows/`:
1. Inspect pinned versions and source refs (Rust, Node.js, uv, Nim, ct-remote, Cap'n Proto, Tup): `non-nix-build/windows/toolchain-versions.env`
1. In Git Bash, source the environment (auto-installs missing tools): `source env.sh`
1. In PowerShell, dot-source the environment (auto-installs missing tools): `. .\env.ps1`
This workflow bootstraps pinned tools into a shared user cache (`%LOCALAPPDATA%/codetracer/windows-diy` by default) and keeps setup deterministic for Windows contributors.
When `env.sh` is sourced, it also ensures `libs/tree-sitter-nim/src/parser.c` is generated when missing or stale.
The parser script prefers local `libs/tree-sitter-nim/node_modules/.bin/tree-sitter` and falls back to an isolated cached `tree-sitter-cli` install when needed, so generation does not require building native addons for the full grammar package.
`env.sh` also restores caller shell options (it does not leave `set -e` enabled) and auto-installs missing Node deps in `node-packages` by default so `stylus.cmd`/`webpack.cmd` are available for `tup`/`just` (`WINDOWS_DIY_SETUP_NODE_DEPS=0` disables this behavior).
For `ct-remote`, the default is `CT_REMOTE_WINDOWS_SOURCE_MODE=auto`: on Windows x64 it uses local `../codetracer-ci` source publish output when available and otherwise falls back to pinned download, while on Windows arm64 it routes to local source and does not fall back to pinned x64 download. `CT_REMOTE_WINDOWS_SOURCE_RID` defaults by architecture (`win-x64` on x64, `win-arm64` on arm64) and remains overridable. Local-source publish now prefers `CT_REMOTE_WINDOWS_PUBLISH_SCRIPT` (default `/non-nix-build/windows/publish-desktop-client.ps1`) so host-global `dotnet` is not required when that helper is present. Use `CT_REMOTE_WINDOWS_SOURCE_MODE=local|download` and `CT_REMOTE_WINDOWS_SOURCE_REPO=` as needed (`download` is x64-only).
For Nim, the default is `NIM_WINDOWS_SOURCE_MODE=auto`: try building from pinned Nim/csources source refs with deterministic cache reuse, and fall back to pinned `nim-_x64.zip` when source bootstrap fails. Override with `NIM_WINDOWS_SOURCE_MODE=source|prebuilt`.
For Cap'n Proto, the default is `CAPNP_WINDOWS_SOURCE_MODE=auto`: use pinned prebuilt ZIP on Windows x64 and source mode on non-x64, with deterministic source-cache reuse. Override with `CAPNP_WINDOWS_SOURCE_MODE=source|prebuilt`.
For Tup, the default is `TUP_WINDOWS_SOURCE_MODE=auto`: build from pinned source (`https://github.com/zah/tup.git` ref `variants-for-windows`) with deterministic source-cache reuse to preserve Windows variant support. `auto` only falls back to prebuilt when you explicitly provide both `TUP_WINDOWS_PREBUILT_URL` and `TUP_WINDOWS_PREBUILT_SHA256`; otherwise it remains source-only by design. Override with `TUP_WINDOWS_SOURCE_MODE=source|prebuilt`.
## The CodeTracer CLI
When you launch the CodeTracer GUI, it will offer you the option to also install the CodeTracer CLI. It provides convenient ways to create and load trace files from the command-line or to integrate CodeTracer w
ith CI processes.
Run `ct --help` to see the full list of supported subcommands, but the most commonly used ones are the following:
`` can be a source file or a project folder (depending on the language):
1. `ct run ` - Creates a recording and load it in CodeTracer with a single command.
1. `ct record ` - Creates a trace file that can be loaded later or shared. On Windows, recordings use the TTD backend (`ct record --backend ttd`) and require an elevated shell. For Python scripts this reuses the interpreter you would get by running `python` in the same shell (honoring the `CODETRACER_PYTHON_INTERPRETER`, `PYTHON_EXECUTABLE`, `PYTHONEXECUTABLE`, and `PYTHON` environment variables before falling back to `PATH`) and that interpreter must have the `codetracer_python_recorder` package installed (run `ct doctor python` to verify).
1. `ct replay` - Launches the CodeTracer GUI with a previously recorded trace file. Common usages are:
* `ct replay` - Opens a simple console-based dialog to choose what recording you want to replay.
* `ct replay ` - Opens the last trace of an application.
* `ct replay --id=` - Opens a trace by its trace id.
* `ct replay --trace-folder=` - Opens a trace by its trace folder.
1. `ct` - Launches the startup screen of the CodeTracer GUI.
1. `ct help`/`ct --help` - Gives you a help message.
1. `ct version`/`ct --version` - Returns the current version of CodeTracer.
## Keyboard Shortcuts
You can learn all CodeTracer keyboard shortcuts by examining the main menu and the tooltips over all buttons in the interface.
The default configuration should be familiar to users experienced with Microsoft Visual Studio™.
Since CodeTracer provides a reverse counterpart to most traditional debugging operations, we typically use the `Shift` modifier to indicate this (e.g. `F10` is "Next Step" while `Shift+F10` is "Previous Step").
The user config file located at `~/.config/codetracer/.config.yml` allows you to specify custom shortcuts for all operations.
# Contributing
Check out our [Contributors Guide](./CONTRIBUTING.md) for more details.
## License
CodeTracer is distributed under the GNU Affero General Public License (AGPLv3).
The wasm-sysroot is copied from the tree-sitter repo: () : specifically . The C headers there are a modified version of glibc headers, which are originally under LGPL license.
# Miscellaneous
## Small Language Constructs
CodeTracer includes a miniature Lisp-like language called **small** used in some of the example programs. The constructs demonstrate how a language can integrate with the [`runtime_tracing`](https://github.com/metacraft-labs/runtime_tracing) format.
### Core constructs
| Construct | Example | Description |
|-----------|---------|-------------|
| `(defun name (params...) expr...)` | `(defun add-one (x) (add x 1))` | Define a function |
| `(set var value)` | `(set x 1)` | Assign a value |
| `(set-deref ref value)` | `(set-deref r 5)` | Write through a reference |
| `(vector v1 v2 ...)` | `(vector 1 2)` | Create a vector |
| `(# vec index)` | `(# v 0)` | Indexing |
| `(push vec value)` | `(push v 3)` | Append to a vector |
| `(loop i from to body)` | `(loop i 0 3 (print i))` | Simple loop |
| `(ref var)` | `(ref x)` | Create a reference |
| `(deref ref)` | `(deref r)` | Read from a reference |
| `(add a b)` | `(add 1 2)` | Integer addition |
| `(print expr ...)` | `(print x)` | Print to stdout |
| `(write-file path data)` | `(write-file "out.txt" "hi")` | Write a file |
### runtime\_tracing events
| Event | Purpose |
|-------|---------|
| `Write` | Terminal output produced by `print` |
| `WriteFile` | File creation via `write-file` |
| `WriteOther` | Other write operations |
| `Read` | Reads from standard input |
| `ReadFile` | Reading from files |
| `ReadOther` | Miscellaneous read operations |
| `ReadDir` | Directory listing |
| `OpenDir` | Opening a directory |
| `CloseDir` | Closing a directory |
| `Socket` | Network socket activity |
| `Open` | Opening files or sockets |
| `Error` | Error reporting |
| `TraceLogEvent` | Events generated by the tracer |
## Contributing
Check out our [Contributors Guide](./CONTRIBUTING.md) for more details.
## License
CodeTracer is distributed under the GNU Affero General Public License (AGPLv3).
The wasm-sysroot is copied from the tree-sitter repo: () : specifically . The C headers there are a modified version of glibc headers, which are originally under LGPL license.
---
### Overview (`browser-replay/README.md`)
# `browser-replay` — Browser-Based MCR Replay Client
This directory contains the browser-side CodeTracer MCR replay
client. The client loads a CodeTracer trace into an in-memory
WebAssembly virtual filesystem and serves the standard CodeTracer
DAP/replay protocol from a Web Worker — letting users step through
a recorded trace inside their browser without launching a native
`ct host` process.
For the full design and milestone history, see
[`codetracer-specs/Planned-Work/Browser-Replay.status.org`](../../codetracer-specs/Planned-Work/Browser-Replay.status.org)
and
[`codetracer-specs/Planned-Work/Unified-Browser-Replay-Architecture.md`](../../codetracer-specs/Planned-Work/Unified-Browser-Replay-Architecture.md).
## Two trace-loading paths
The client supports two trace-loading paths:
### 1. Gateway-authenticated load (M40, production)
The client fetches the recording manifest and trace bytes through
codetracer-ci's authenticated browser-replay gateway endpoints:
- `GET /api/v1/observability/gateway/manifests/{traceId}` — manifest fetch.
- `GET /api/v1/observability/gateway/ranges/{traceId}/{**objectKey}` — byte fetch with HTTP Range.
- `POST /api/v1/observability/gateway/credential-validate` — precondition probe (optional).
This is the path used in production. Users get to a CodeTracer
debug session via a codetracer-ci debug-session URL; codetracer-ci
issues a short-lived bearer token tied to the user's
`replay.read` role on the trace's tenant; the browser-replay client
loads the trace through the gateway with that token.
The endpoint reference lives at
[`codetracer-ci/rewrite-docs/04-apis-events/http-api.md`](../../codetracer-ci/rewrite-docs/04-apis-events/http-api.md)
section 4.11. The data-path overview is in
[`codetracer-specs/Observability-Platform/docs/direct-storage-data-path.md`](../../codetracer-specs/Observability-Platform/docs/direct-storage-data-path.md).
### 2. Static-asset load (legacy, tests)
The client can also fetch a trace directly from a static HTTP
server. This path predates M40 and is used by the local nginx
test harness (`test-server.sh`, `nginx.conf`) for development
iteration on the WASM module itself.
## URL contract
The client reads its configuration from the URL query string. The
`gateway-authenticated load` path is selected when both
`gatewayBaseUrl` and `traceId` are set; the legacy static path is
selected otherwise.
```
https:///index.html
?gatewayBaseUrl=https://codetracer.example.com
&traceId=019e0744-31cf-7ad9-b350-f8d2e3eda5a2
&authToken=
```
Required query parameters for the gateway path:
- `gatewayBaseUrl` — the codetracer-ci base URL. The client
appends `/api/v1/observability/gateway/manifests/{traceId}`
and `/api/v1/observability/gateway/ranges/{traceId}/{**}` to it.
- `traceId` — the trace UUID to load.
- `authToken` — bearer token. Sent as
`Authorization: Bearer ` on every fetch. Production
deployments mint short-lived tokens server-side and inject them
into the URL when redirecting users from a debug-session link.
Optional:
- `replicaIndex` (integer) — when the manifest exposes multiple
replicas per shard, force the client to use a specific one.
Defaults to `0`.
## Cross-origin and auth model
The gateway endpoints require user-bearer authentication and the
`replay.read` operation on the trace's tenant. The client sends
`Authorization: Bearer ` on every fetch. CORS preflights
are NOT required when the client is served from the same origin as
the gateway — the recommended deployment is a single nginx vhost
that serves both the static client assets and reverse-proxies to
codetracer-ci.
For end-to-end local testing without nginx, see
[`codetracer-ci/tests/CrossRepo.Test.Integration/BrowserReplayClientTests.cs`](../../codetracer-ci/tests/CrossRepo.Test.Integration/BrowserReplayClientTests.cs):
the cross-repo test mounts the `app/` bundle on the same port as
the gateway via `CrossRepoTestServer.MountStaticFiles`, so the
browser sees a single origin and Playwright can drive the whole
flow without CORS configuration.
## Files
| File | Purpose |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `app/index.html` | Thin scaffold; documents the URL contract; loads `gateway-client.js`. |
| `app/gateway-client.js` | Main-thread driver. Reads URL params; boots the worker; sends DAP `initialize` / `launch` / `configurationDone`. |
| `app/worker.js` | Web Worker that loads the WASM module. Two message types: `load-trace` (legacy static fetch) and `load-trace-from-gateway` (M40). |
| `app/pkg/db_backend.js` | Generated wasm-bindgen JS bindings (rebuilt from `src/db-backend/wasm-testing/`). |
| `app/pkg/db_backend_bg.wasm` | Generated WASM module. |
| `nginx.conf`, `start-server.sh`, `stop-server.sh`, `setup-certs.sh` | Local nginx test harness for the legacy static-fetch path. Not used by the gateway-authenticated path. |
| `test-server.sh` | One-shot wrapper around the nginx harness. |
| `traces/` | Hardcoded test traces for the legacy static path. |
## Testing
The end-to-end test is
[`codetracer-ci/tests/CrossRepo.Test.Integration/BrowserReplayClientTests.cs`](../../codetracer-ci/tests/CrossRepo.Test.Integration/BrowserReplayClientTests.cs)
(`browser_replay_client_fetches_manifest_and_ranges_through_gateway`).
It:
1. Spins up a real `DistributedHttpStorageNode` Python subprocess.
2. Spins up a real `CrossRepoTestServer` with the real
`BrowserGatewayEndpoints` route group and the real
`IS3Provider` / `IObservabilityAuditSink`.
3. Stages real payload bytes onto the storage node and seeds a
`ShardedMcrSegmentManifest` whose replica points at the node's
HTTP base URL.
4. Mounts the `browser-replay/app/` bundle on the same origin as
the gateway.
5. Launches headless Chromium via `Microsoft.Playwright`.
6. Navigates to
`index.html?gatewayBaseUrl=...&traceId=...&authToken=...`.
7. Hard-asserts the manifest fetch returned `200`, at least one
range fetch returned `206`, the storage node's `payloadReads`
counter strictly increased (proves the gateway HTTP-proxied to
the storage node), and the audit sink recorded
`gateway.manifest.read` and `gateway.range.read` events with
`status="allowed"`.
Run via:
```bash
cd codetracer-ci
nix develop ./nix -c dotnet test \
tests/CrossRepo.Test.Integration/CrossRepo.Test.Integration.csproj \
-c Release \
--filter "FullyQualifiedName~BrowserReplayClientTests" \
-p:NuGetAudit=false
```
## Building the WASM bundle
The `app/pkg/` directory is a build output and is `.gitignore`d.
Re-stage it from `src/db-backend/wasm-testing/pkg/` after building
the WASM module:
```bash
cd src/db-backend/wasm-testing
just build # or: cargo build --target wasm32-unknown-unknown
cd -
cp -r src/db-backend/wasm-testing/pkg/* browser-replay/app/pkg/
```
`build-dist.sh` automates this for distribution-ready bundles.
## Known limitations
- Materialized traces (Python / Ruby / JavaScript): the M40 gateway
endpoints serve materialized artifacts the same way as MCR shards
(each artifact has an object key in the manifest), but the
WASM-side replay engine for materialized traces is still in
development. The client returns an `unsupported-backend` error
message when handed a manifest with `kind=materialized_trace`
for a runtime that hasn't shipped its replay engine yet.
- Token refresh: the URL-embedded `authToken` is short-lived and
does NOT auto-refresh. Production deployments should mint
long-enough tokens to cover an expected replay session, or wire
in a refresh flow via the parent page.
---
### Overview (`get/README.md`)
# `get/` — source for https://get.codetracer.com
This directory is the source tree for CodeTracer's install/onboarding host,
`get.codetracer.com`, per
[product-install-domains.md](https://github.com/metacraft-labs/metacraft-dev-guidelines/blob/main/policies/product-install-domains.md).
It exposes three surfaces:
| URL | Served file | Purpose |
| --- | --- | --- |
| `https://get.codetracer.com/` | `index.html` | Onboarding / install landing page |
| `https://get.codetracer.com/sh` | `install-on-distributions.sh` (re-served) | `curl -fsSL https://get.codetracer.com/sh \| sh` |
| `https://get.codetracer.com/pwsh` | `install.ps1` or `pwsh.stub.ps1` | `irm https://get.codetracer.com/pwsh \| iex` |
## How it is built and served
`build-get.sh` assembles `dist/`:
- `index.html`, `_headers` — copied verbatim.
- `sh` — the repo's **canonical** `install-on-distributions.sh` with a provenance
header injected after the shebang (single-sourced, never a divergent copy).
- `pwsh` — the repo's `install.ps1` if present, otherwise `pwsh.stub.ps1`
(a "coming soon" stub that exits non-zero and points at the desktop app /
releases / WSL). CodeTracer has no PowerShell installer yet, so the stub ships.
`_headers` forces `Content-Type: text/plain; charset=utf-8` + a short cache TTL on
`/sh` and `/pwsh`, so `curl … | less` shows the source and a fixed URL always
yields the current bootstrapper.
The artifact **store** remains GitHub Releases / `downloads.codetracer.com`;
`get.codetracer.com` is only the advertised UX.
## Deploy
`.github/workflows/deploy-get.yml` runs `build-get.sh` and
`wrangler pages deploy get/dist --project-name=get-codetracer --branch=master`
on push to `stable` (paths `get/**`, `install-on-distributions.sh`, `install.ps1`).
The `get-codetracer` Pages project's production branch is `master`, so the
`--branch master` deploy publishes production (served on the custom domain).
Requires repo secrets `CLOUDFLARE_API_TOKEN` (Pages:Edit) and
`CLOUDFLARE_ACCOUNT_ID` (`803741d99690718276ea30950f690c46`).
Build locally: `bash get/build-get.sh && ls get/dist`.
---
### Overview (`refs/README.md`)
# References
Workspace-managed reference checkouts used for design study live here. They are
declared in the workspace manifests (`repos/codemirror-*.toml`) and cloned by
`repro`, so they are ignored by this repository the same way
`reprobuild/references/` is: the checkouts belong to the workspace, not to this
repo's history.
Product code must not import or link these checkouts directly.
Currently declared:
- `codemirror-state`, `codemirror-view`, `codemirror-commands`,
`codemirror-collab` — CodeMirror 6 (MIT). The design reference for the
headless editor ViewModel: change sets and transactions, the decoration and
widget model, the named-command vocabulary, and collaborative editing by
rebase against a central authority.
---
### Overview (`storybook/README.md`)
# CodeTracer Storybook
Storybook harness for isolated CodeTracer IsoNim panels, shell views,
building-block components, and default layouts.
## Commands
```bash
just build-storybook-components
just storybook-check-styles
just storybook
just storybook-build
```
The Nim bundle exports Storybook mount functions from
`src/frontend/storybook_components.nim` into `storybook/dist/components.js`.
Stories load that bundle and mount real ViewModels through the production
IsoNim views.
Storybook loads CodeTracer styles by parsing the production
`src/frontend/index.html`; do not copy app stylesheet lists or component visual
CSS into stories. Keep Storybook CSS limited to harness sizing/layout.
Fixtures may mock ViewModel/store state, but stories must not reimplement
component DOM in JavaScript. The DOM under test should come from the real
`src/frontend/viewmodel/views/*.nim` render or mount functions.
## Visual Review
The repeatable screenshot loop lives in `tools/visual-review/`:
```bash
nix shell nixpkgs#nodejs --command node tools/visual-review/capture-storybook.mjs
nix shell nixpkgs#nodejs --command node tools/visual-review/capture-storybook.mjs --view terminal-output --size laptop --no-build
```
The capture tool builds Storybook, serves `storybook/storybook-static`, captures
named stories at the configured viewport sizes, and writes diagnostics beside
the screenshots. It also writes HTML and computed-style dumps that can be
compared against reference dumps from older known-good builds:
```bash
nix shell nixpkgs#nodejs --command node tools/visual-review/compare-style-dumps.mjs \
--reference ../codetracer-main/ui-tests/reference-dumps/isonim-karax-reference-20260504T155511Z/20260504T160434Z_noir-terminal-visible.computed-styles.json \
--current tools/visual-review/dumps/terminal-output-laptop.computed-styles.json \
--out tools/visual-review/reports/terminal-vs-karax-reference.md
```
The generated `reports/`, `screenshots/`, and `dumps/` directories are ignored;
keep the project brief, screen briefs, and capture/comparison scripts in git.
---
### Overview (`ci/runner/README.md`)
# Self-hosted macOS runner defects — the register
These runners are **persistent**. Their home directory, work directory, git
config, nix store and background daemons survive between jobs, so a defect one
job leaves behind is inherited by whatever runs next — usually a different job,
often a different repository. That is what makes this class expensive: **the job
that fails is almost never the job at fault**, and the message it fails with
tends to name neither.
This file exists because the list below lived only in one person's head. Seven
defects were diagnosed over one campaign; the count was reported as five twice
before being reconciled upward. A register that is not in the tree gets
recounted from memory, and memory rounds down.
## Rules this list is maintained by
- **A fix needs a gate that was proved able to fail.** Not re-read — *run*
against the broken input, and seen to name what it found. Two checks in this
campaign's history could only ever pass: a `*.lock` sweep whose first version
matched anywhere (it was deleting `Cargo.lock`, `flake.lock`, `repro.lock` and
`yarn.lock` from sibling checkouts — a sweep meant to unblock checkout was
quietly unpinning the build), and a credential recount that `494d85395`
records as "itself a no-op".
- **Never weaken a check to make it green, and never edit an expected count
down.** Several corrections here went *up*.
- **Prefer planting the failure to arguing about the code.** Where a gate below
has a contract suite, the negative case is the real pre-fix file read out of
`origin/dev` rather than a mock of it.
- **A partial fix is recorded as partial.** Two entries below are mitigations
whose root cause is still unidentified or unaddressed; saying so is the point
of the entry.
---
## 1. The `repro` daemon never `chdir`s — CLOSED here, cause still upstream
**Symptom.** Any `repro build` on the runner dies with:
```
daemon-hosted build failed: No such file or directory
```
naming no path and no owner.
**Cause.** The `repro` user daemon is persistent and its endpoint is
deliberately stable across `nix develop` sessions, so one daemon serves every
job on a runner indefinitely. On macOS it is started via launchd, and when that
fails it falls back to a plain fork+setsid: `launchWithFork` in reprobuild's
`libs/repro_daemon_core/src/repro_daemon_core/runtime.nim` calls `setsid()` and
then `execve` **with no `chdir` between them**. `chdir` and `setCurrentDir`
appear nowhere in that library, and no launchd plist written here sets
`WorkingDirectory`. The daemon therefore inherits — and keeps — the working
directory of whichever `repro build` first spawned it. Once that directory is
gone (a re-cloned workspace, a deleted temp root), every later build dies in the
first statement of the daemon-side build executor, `getCurrentDir()`.
**State.** `a556399c0` diagnosed this and named two victims,
`reprobuild-macos-smoke` and `test-ui-tests (macos)`, but guarded only the
first. `ci/reprobuild/macos-daemon-build.sh` — the script that *deliberately*
runs `repro build --daemon=auto`, and so the likeliest thing on a runner to
leave a daemon behind — had no `repro daemon stop` anywhere, including its trap.
It now stops the daemon before building and on every exit path.
A second defect surfaced while fixing the first: that script's only
`trap ... EXIT` was armed **inside** `start_runquotad`, which returns early when
`RUNQUOTA_SOCKET` is already set. On that path the script had no exit handler at
all, so it leaked `runquotad` and its socket directory too. `trap` replaces
rather than composes; the handler is now armed once, at top level.
**Gate.** `ci/verdict/reprobuild-daemon-guard.sh`, suite
`ci/test/reprobuild-daemon-guard-test.sh` (9 assertions, in the bash lint lane).
It requires a stop before the first build, exactly one EXIT trap armed at top
level, and a stop inside the handler that trap names. Comments are stripped
before matching — these scripts discuss `repro daemon stop` at length, and a
checker satisfied by its own explanation is the defect it was written against.
It refuses to pass on an empty directory.
**STILL OPEN, upstream.** The durable fix is one line in another repo: perform
`chdir("/")` in the fork path, and let the daemon tolerate a vanished cwd.
Until then codetracer only stops being both cause and victim of its own jobs;
any other product's `repro build` on these runners can still poison them.
---
## 2. Stale git `extraheader` credential → HTTP 401 — MITIGATION, NOT A CURE
**Symptom.** `actions/checkout` fails to authenticate; git reports
`could not read Username`, which is a 401 in disguise.
**Cause.** A stale `http..extraHeader` in `$HOME/.gitconfig` carrying a dead
token. Git resolves `http..*` by **longest matching prefix**, so a narrower
stale entry outranks the fresh, correctly-scoped one that checkout installs.
**State.** An inline pre-checkout sweep,
`Evict stale git credentials from this runner's persistent home`, duplicated
across **8 jobs** in `.github/workflows/codetracer.yml` — lines 1802, 2034, 2236,
3569, 4094, 4651 (canonical copy, carries the full rationale), 5212 and 6085. It
must run *before* `actions/checkout`, so it cannot be a script this repository
provides. It matches only `http.*.extraheader`, `http.*.extraHeader` and
`url.*x-access-token*`, operates on `$HOME/.gitconfig` **by path** (not
`--global`, because XDG can redirect that), and recounts in pure bash — `grep`
and `sed` are not on a raw `run:` step's PATH on these nix-darwin runners.
Introduced by `083aa0c23`.
**STILL OPEN.** The step's own comment
(`.github/workflows/codetracer.yml:4698-4706`) says so:
> STILL UNKNOWN: WHO WRITES IT. `configure-git-auth.sh` in
> metacraft-labs/metacraft-github-actions installs exactly this key but
> deliberately passes it through `$GITHUB_ENV`, which is job-scoped, so on its
> documented path nothing should reach this file. Something on these runners
> writes it anyway (mcl-004's copy was stamped 08:15 on the day of the outage).
> Until that writer is found this step is a mitigation, not a cure — it runs
> before every checkout precisely because the poison can come back.
Finding the writer is the open work. Note also that the sweep asserts a
**non-zero** count on a poisoned host rather than merely a zero afterwards —
that distinction is what `494d85395` was about.
---
## 3. Stale `*.lock` sentinels blocking checkout — CLOSED
**Cause.** Git metadata locks (`index.lock` and friends) left by a killed job
make the next checkout fail.
**State.** `Sweep stale git lock files from the persistent work directory`,
`.github/workflows/codetracer.yml:5981`, in `visual-replay-regression-gate` —
the only persistent `[self-hosted, gpu]` job, hence the only copy.
**Read this before touching the pattern.** The first version matched
`-name '*.lock'` anywhere under the work directory and, measured on run
33734457928, deleted 42 files including `codetracer-trace-format/Cargo.lock`,
`flake.lock`, `nim-agents/repro.lock` and
`isonim/src/isonim/layout/yoga/yarn.lock`. A sweep meant to unblock checkout was
quietly unpinning the build. The current pattern is anchored:
```sh
find "$root" -type d -name .git -exec find {} -type f -name '*.lock' -mmin +60 ';'
```
The outer `find` restricts to directories literally named `.git`, so no
lockfile in a worktree is reachable; `-mmin +60` protects locks held by a
concurrent job. Both constraints are load-bearing — **do not flatten this into a
single `find`.**
Fixed by `083aa0c23`, anchored by `444cfa989`.
---
## 4. Read-only `.app` leftovers make runners uncleanable — CLOSED
**Cause.** A Nix-store-mode `.app` bundle is copied out read-only; the next
job's checkout cannot delete it.
**State.** `ci/runner/sweep-readonly-leftovers.sh`, plus a `chmod -R u+w`
remediation at `.github/workflows/codetracer.yml:6068-6083`. Fixed at the source
(the bundle is made writable when produced) *and* swept where already poisoned.
Landed in `444cfa989`.
**Gate.** `ci/test/readonly-leftovers-sweep-test.sh`, in the bash lint lane. It
asserts a **non-zero** count on a poisoned fixture, not merely a zero
afterwards.
---
## 5 & 6. `/usr/bin` missing from a raw `run:` step's PATH → exit 127 — CLOSED per site, STRUCTURALLY OPEN
These were logged as two defects and are one root cause, which is why they kept
recurring under new names.
**Cause.** `nix develop` prepends store paths and keeps the ambient PATH as the
tail. On these nix-darwin runners a raw `run:` step's ambient PATH has no
`/usr/bin`, so any Apple system tool is simply absent — exit 127, "command not
found", for a binary that is obviously installed.
**Victims, in the order they were found:** `codesign`, then `sw_vers` (via
`create-dmg`), then `hdiutil`. Each was fixed where it bit:
| Site | Fix | Commit |
|---|---|---|
| `codetracer.yml:5516` (`test-ui-tests`, ct-mcr) | `PATH="$PATH:/usr/bin"` | `3ac64e9f2` |
| `codetracer.yml:3727` (`dmg-build`) | `PATH="$PATH:/usr/bin:/usr/sbin"` | `5f6ab33ce` |
| `codetracer.yml:4252` (`dmg-lib-check`) | `PATH="$PATH:/usr/bin"` | `bf7bfd54d` |
| `codetracer.yml:5590` (`test-ui-tests`, build-once) | `PATH="$PATH:/usr/bin"` | `bed00876f` |
**Append, never prepend** — the devShell toolchain must keep winning; the point
is only to make the system tools reachable in the tail.
The last entry is the widest: `repro` passes PATH through to all 42 actions
(`0 hermetic, 22 inherited`), so the `dmg` action inherits whatever deficiency
the invoking step had.
**STILL OPEN, structurally.** Four independent point fixes for one root cause,
and the comment at `codetracer.yml:5573-5574` already records the second as "the
second confirmed instance of that one root cause". There is no runner-level or
workflow-level `env: PATH` fix, so a fifth site will fail the same way. A
workflow-level default, or a lane guard asserting `/usr/bin` is on PATH in every
raw `run:` step that invokes a system tool, would close the class rather than
the instance.
---
## 7. Recorder siblings cloned but never built — CLOSED
**Symptom, ruby.** A `require` failure from inside
`CodeTracer::Native.load_extension!`, naming a `.so`/`.bundle` the reader has
never seen.
**Symptom, js.** `codetracer-js-recorder is not built (…/packages/cli/dist/index.js)`
from `scripts/materialize-recording.sh`, repeated for every recording the
Playwright suite materialises — an hour into the run.
**Cause.** `grep -E 'rake compile|extconf|build-extension'` over `.github/`
returned **zero** hits. The ruby extension was built only by
`scripts/build-siblings.sh`, reached solely by the **Linux** `ct-test-providers`
job, while `test-non-gui` and `test-ui-tests` cloned the recorder on macOS and
ran against a tree with no compiled artefact. Separately, the fix for
"`test-ui-tests` never cloned `codetracer-js-recorder`" (`8616ff406`) added the
clone but no build — `npm run build` existed in this repository only inside that
step's own comment.
**Why it was silent rather than loud — the dominant defect of this campaign.**
`scripts/detect-siblings.sh` probed
`gems/codetracer-ruby-recorder/bin/codetracer-ruby-recorder`, a nine-line shim
**checked into git at mode 100755**. Present in every fresh clone, so the test
could not fail, detection printed `(native)` unconditionally, and the "not
built" warning beside it was unreachable code. The python recorder had the same
shape: it keyed off the source directory and a venv interpreter, neither of
which says the maturin/pyo3 extension was ever installed.
> **A detector that probes the wrong artefact does not merely fail to help — it
> actively converts a build error into a runtime mystery.** When adding a
> recorder, probe the file its *loader* opens, and check the probe against the
> loader's own resolution order.
**State.** Both probes now track the compiled artefact and resolve the platform
DLEXT (`.bundle` on macOS — `.so` is the Linux spelling and was wrong on every
Mac). `test-non-gui` and `test-ui-tests` build the ruby extension; `test-ui-tests`
builds the JS recorder. Both new steps assert the **artefact**, not the exit
code. `scripts/build-siblings.sh`'s hard-coded `.so` was made platform-aware,
since otherwise a green macOS build reports its own artefact missing.
**Python needs no workflow build step** — `nix/shells/main.nix`'s hook installs
the recorder into `.python-recorder-venv` on every `nix develop`, and writes
`.broken` (which `scripts/test-python-version-alignment.sh` fails on) when it
cannot. Asserting one would be a false failure. Checked, not assumed.
**Gates.** `ci/test/detect-siblings-recorder-artifacts-test.sh` (13 assertions)
and `ci/verdict/recorder-clone-implies-build.py` with
`ci/test/recorder-clone-implies-build-test.sh` (12 assertions), both in the bash
lint lane. Every "must warn" is paired with a "must not warn", because a probe
that only ever says one thing is the defect whichever thing it says.
The second gate follows `just` targets into the `Justfile`: `viewmodel-tests`
builds the JS recorder via `just test-vm-recorder-gated` →
`build-siblings.sh --only`, and the checker's first version called that a
violation. The rule was **not** relaxed to permit it — the checker was taught to
see it. A false positive trains the reader to skip the check, which costs
exactly what a missing check costs.
**Known remaining gap, not yet reached.** The nixos leg's `siblings:` block does
not carry the JS recorder either; that leg currently fails earlier, in the
storybook build, so the gap has never been exercised.
---
## Scoreboard
| # | Defect | State |
|---|---|---|
| 1 | repro daemon never `chdir`s | codetracer guarded; **cause open upstream** |
| 2 | stale `extraheader` → 401 | **open** — mitigation, writer unidentified |
| 3 | stale `*.lock` sentinels | closed |
| 4 | read-only `.app` leftovers | closed |
| 5 | `codesign` 127 | closed per site |
| 6 | `create-dmg`/`sw_vers`/`hdiutil` 127 | closed per site; **class open** |
| 7 | recorder cloned, never built | closed |
Four closed, three carrying open work — 1 upstream, 2 entirely, and 5/6 as a
class. Update this table in the same commit as the fix, not afterwards.
---
### Overview (`docs/book-isonim/README.md`)
# CodeTracer Book — working on docs.codetracer.com
This is the **CodeTracer documentation site** (published at
). It is a consumer of the
[isonim-docs](../../../isonim-docs) static-site framework: the Markdown in
`content/` is rendered by the framework's SSG/SSR, themed by the shared
[CodeTracer docs design system](../../../codetracer-design-system/docs/codetracer-docs.tokens.json).
## Prerequisites
Every task runs inside **this book's own dev shell** (provides `nim`, `just`,
`node`). The book consumes the [isonim-docs](../../../isonim-docs) static-site
framework, so it reuses that framework's self-contained dev shell — which
declares **isonim as its dependency** and reuses isonim's toolchain (Nim 2.2.4,
node, `just`). Nobody has to `nix develop ../../../isonim`.
With **direnv** the shell activates automatically on `cd` (see `./.envrc`):
```bash
cd codetracer/docs/book-isonim
just dev-docs
```
Without direnv, enter the framework's dev shell once…
```bash
cd codetracer/docs/book-isonim
nix develop path:../../../isonim-docs # the framework shell (isonim is its dep)
just dev-docs
```
…or prefix a single recipe with it:
```bash
nix develop path:../../../isonim-docs -c just dev-docs
```
All commands below assume you are in `codetracer/docs/book-isonim/` and in the
dev shell.
## Live preview (hot reload)
```bash
just dev-docs # http://127.0.0.1:8000 (loopback only)
just dev-docs-lan # same, reachable on your private LAN at http://:8000
just open-docs # open the running server in a browser
```
- `dev-docs` serves `content/` + the themed assets and **live-reloads every open
tab** on any content edit.
- The **first launch compiles** the dev server and pre-compiles the client JS
bundle (theme toggle, live search, sidebar collapse) — expect ~1 minute before
it starts serving; after that, page loads and edits are instant.
- Default host is loopback (`127.0.0.1`). Pass `0.0.0.0` (or set `AH_DEV_HOST`)
only on a trusted private network — it exposes the unauthenticated docs to the
LAN/tailnet.
## Build, one-shot preview, tests
```bash
just build # static build into build/ (what CI deploys via ci/deploy/docs.sh)
just serve-docs # one-shot SSR preview (no live reload)
just test # nav-order + dev-server tests
```
### If the build says `cannot open file: build_site`
Nim has no `--path` to the sibling framework. The book's `config.nims` — which
used to supply them, resolved from `currentSourcePath()` — is matched by the
repo-wide `*.nims` line in `codetracer/.gitignore`, so it is not in the
checkout. `ci/deploy/docs.sh` works around this by generating an absolute
`nim.cfg` before the build and deleting it after; do the same locally, once:
```bash
S="$(cd ../../.. && pwd)" # the workspace root holding the sibling repos
cat > nim.cfg </` folder.
### Moving or renaming a page
Its URL is published — on the released book, on `/nightly`, and in whatever
links readers have already shared — so a move needs a redirect or the old URL
starts 404ing. Add the pair to **`movedRoutes` in `src/redirects.nim`**:
```nim
(oldRoute: "/usage_guide/deep_review", newRoute: "/deep_review"),
```
The build then writes a meta-refresh stub at the old route's own `index.html`
(the only path GitHub Pages serves for the trailing-slash URL a reader has) and
adds the pair to `_redirects`, both carrying the channel prefix. The list is
**append-only**: deleting an entry re-breaks the URL it was added for. Adding a
new section is otherwise just a directory plus a `sectionOrder` entry in
`src/docs_config.nim`.
## Changing the look (design system)
The book is themed by the **shared** design system at
`codetracer-design-system/docs/codetracer-docs.tokens.json` — the same file the
other two docs sites use. To tweak it visually, launch the design-system editor
straight from here:
```bash
just design # http://127.0.0.1:8080 (the shared theme editor)
just design 8080 0.0.0.0 # reachable on the LAN
```
Keep `just dev-docs` running in another shell: a token you change and **Save** in
the editor persists to `codetracer-docs.tokens.json` and **hot-reloads the book
live, no rebuild**. You can also edit that JSON file by hand. Book-specific
layout/overrides live in `assets/style.css`. (The editor itself lives in
[`isonim-docs/site/design`](../../../isonim-docs/site/design/README.md).)
## Deployment
CI (`.github/workflows/codetracer.yml` → `ci/deploy/docs.sh`) builds this book
and publishes it to GitHub Pages, decoupled from the app build/test matrix.
There are **three channels**, chosen by the event and the branch:
| Trigger | URL | Built with |
|---|---|---|
| push to `stable` | (plus the archived mdBook at `/old`) | `CT_DOCS_BASE_PATH` unset |
| push to `dev` | | `CT_DOCS_BASE_PATH=/nightly` |
| pull request `N` | `https://docs.codetracer.com/pr/N/` | `CT_DOCS_BASE_PATH=/pr/N` |
A push to `main` publishes nothing. All channels commit to the same `gh-pages`
branch and each run replaces **only its own subtree**, so the other channels
survive untouched; the deploy is a fast-forward push, never a force-push.
### Pull-request previews
A pull request that changes anything under `docs/` gets its book published at
`/pr//`, and a single comment on the pull request links to it — edited in
place on every later push, never re-posted, and naming the commit it was built
from. When the pull request closes, `.github/workflows/docs-preview-cleanup.yml`
removes `/pr//` again and corrects that comment. A pull request that touches
no docs publishes nothing and says nothing.
Previews are **same-repository pull requests only**. A pull request from a fork
runs with a read-only token and no secrets, by design, because it builds code
written by whoever opened it; publishing from such a run would mean executing
untrusted build code with a token that can write to `gh-pages`. A fork pull
request therefore gets no preview (and no comment — with a read-only token the
job could not post one either).
### robots.txt
Search engines are asked to skip everything that is not the released book: the
**root** `robots.txt` (the only one crawlers read) carries `Disallow: /nightly/`
and `Disallow: /pr/`. Every released deploy regenerates that file from the build
and re-adds both lines; a preview deploy adds the `/pr/` line if it is missing,
so a preview is excluded from the moment it is first published rather than from
the next release. To publish a channel to search engines instead, drop its
`ensure_root_robots_disallow` call in `ci/deploy/docs.sh`.
### Base path
`CT_DOCS_BASE_PATH` is the URL prefix the build is hosted under: `src/build.nim`
passes it to `bookDocsConfig()`, and the framework then rewrites every
root-relative link, asset, stylesheet `url(...)`, search-index route and legacy
redirect stub to carry it. Build a prefixed variant locally with:
```bash
CT_DOCS_BASE_PATH=/nightly just build # or /pr/123
```
### Rehearsing a deploy
To rehearse one without pushing anything, from the repo root:
```bash
DOCS_DEPLOY_DRY_RUN=1 DOCS_DEPLOY_BRANCH=dev ./ci/deploy/docs.sh # or =stable
DOCS_DEPLOY_DRY_RUN=1 DOCS_DEPLOY_CHANNEL=preview DOCS_DEPLOY_PR=123 ./ci/deploy/docs.sh
```
It prints the staged tree, including how many files it preserved from the
channels it does not own. If the book build fails the deploy **fails**: it never
substitutes older content under the published URLs, and because nothing is
pushed the previously published site keeps serving.
The whole channel model — every channel, both preservation directions, the
robots.txt rules, the refusals, and real pushes/races/rejections against a local
bare repository — is covered by `ci/test/docs-deploy-channels-test.sh`; the
preview comment is covered by `ci/test/docs-preview-comment-test.sh`. Both run
standalone with no network access.
---
### Overview (`examples/noir_test_banking/readme.md`)
Explanation
Account Validation (account.nr)
validate_balance: Ensures that the sender's balance is sufficient to cover the transaction amount. If not, the program will abort.
verify_identity: Verifies the provided account hash against the stored account hash for both the sender and the recipient, ensuring the transaction is being initiated by the correct parties.
Transaction Execution (transaction.nr)
make_transaction:
Verifies the sender's and recipient's identities.
Validates that the sender has enough funds to complete the transaction.
Simulates the update of the recipient’s balance by adding the transaction amount.
Returns the updated balances for both the sender and recipient.
Main Logic (main.nr)
The main function coordinates the transaction.
It calls make_transaction to ensure the transaction is valid and then checks that the sender’s and recipient’s balances have been updated correctly after the transaction.
Proving the Transaction
In the Prover.toml, the values used for the transaction are as follows:
sender_balance = 1000: The sender has 1000 units.
transaction_amount = 200: The transaction amount is 200 units.
recipient_balance = 300: The recipient starts with 300 units.
sender_account_hash = "0xabc123": The hash of the sender’s account.
recipient_account_hash = "0xdef456": The hash of the recipient’s account.
With this setup, the transaction can be executed and verified using zero-knowledge proofs without ever revealing the actual balances or sensitive data of the accounts. The program simply proves that:
The sender has enough balance.
The identities of both parties are verified.
The transaction has been successfully executed, i.e., the sender’s balance is correctly updated, and the recipient’s balance is increased.
---
### Overview (`examples/ruby-project/README.md`)
# ruby
this is a simple interpreter written in ruby
it can execute ast defined as a ruby object:
```ruby
run(n(:module, [n(:binary_add, [n(:int, [0]), n(:int, [7])])]))
# rint(7)
```
it is a simple project useful to research ruby support for codetracer
and inferring types
it's also defined for python and c++
---
### Overview (`examples/ruby_exit_program/README.md`)
# Ruby Exit Methods
This project demonstrates various methods of exiting a Ruby program. Each script in the `bin/` directory illustrates a different method, providing insight into their usage and potential real-world applications.
## Scripts
- `exit_success`: Exits with a success code.
- `exit_with_error`: Exits with an error code.
- `abort`: Exits with an abort message.
- `unhandled_exception`: Raises an unhandled exception.
- `end_of_script`: Reaches the end of the script.
- `send_signal`: Sends a TERM signal.
- `thread_exit`: Exits the main thread.
- `exec_replace`: Replaces the Ruby process with another command.
- `stack_overflow`: Causes a stack overflow.
- `fork_bomb`: Initiates a fork bomb (use with caution).
## Usage
Each script can be run directly from the command line:
```bash
ruby bin/with_extensions/ or ruby bin/without_extension/
## Script Directories
- **`bin/with_extension/`**:
- Contains Ruby scripts with the `.rb` extension. These are primarily used during development and debugging when integration with IDEs and other development tools is required.
- **`bin/without_extension/`**:
- Contains executable scripts without the `.rb` extension, optimized for direct execution from the command line. These scripts provide a cleaner and more tool-like interface for users and are intended for production use or manual execution.
---
### Overview (`examples/rubytracer/README.md`)
# rubytracer
very small experiment ruby tracing
```bash
bin/rubytracer
```
Currently implemented just using the tracepoint ruby api. Some investigation
in directly doing this with gdb + Ruby VM shows it shouldn't be too hard, but
that server as a quick pre-experiment.
## infer types from trace
```bash
bin/rubytracer ../ruby-project/spec/ruby_spec.rb
ruby ../ruby-project/spec/x.rb
```
```ruby
Ruby::DSL:
instance:
class:
-> | | >]> ->
def n(kind, children)
-> <?>
def run(ast)
->
def rint(value)
Ruby::Node:
instance:
-> | | >]> ->
def initialize(kind, children)
class:
Ruby::Env:
instance:
->
def initialize(values)
->
def [](label)
class:
Ruby::Runner:
instance:
-> << | R::None>?>
def run(ast)
-> -> <?>
def run_binary_add(a, b)
->
def run_int(value)
->
def run_var(name)
<?>
def run_module()
class:
Ruby::RObj:
instance:
-> ]> ->
def initialize(type, values)
def value()
class:
```
The currently used type "system" is very simple: it has
* `atomic` and `class` types that
correspond to builtin types like String and to classes (currently no real difference)
* `none` type for nil
* `union`: `A | B` for supporting several possible
* `generic`: currently only matching empty arrays
* `concrete`: concretizing generic bases with real types
* `optional`: `A?` when it can be A or nil
We also unify concrete types with unifiable params.
It's probably not very good for a language like Ruby where something more similar to
duck typing like some form of structural typing will be better. However it's sufficient for demonstration in that simple prototype
We rewrite the spec so we can call the code in the `it` and we trace it and analyze it in
`trace.rb`.
---
### Overview (`non-nix-build/windows/README.md`)
# Windows DIY Bootstrap Maintenance
This directory contains the non-Nix Windows bootstrap workflow
and version pins.
Default install root:
- `%LOCALAPPDATA%/codetracer/windows-diy`
(PowerShell/bootstrap default)
- POSIX equivalent in Git Bash via
`source env.sh`
- PowerShell equivalent via
`. .\env.ps1`
- Override with `WINDOWS_DIY_INSTALL_ROOT` or bootstrap
`-InstallRoot`
`env.sh` sourcing behavior:
- When sourced from Git Bash, `env.sh` restores your previous
shell options on return and does not leave `set -e` enabled
in the caller shell.
- Missing Node tool deps are auto-provisioned by default
(`WINDOWS_DIY_SETUP_NODE_DEPS=1`) via `yarn install` in
`node-packages`, so `stylus.cmd`/`webpack.cmd` are available
for Tup/Just workflows.
- Set `WINDOWS_DIY_SETUP_NODE_DEPS=0` to disable this
auto-install and manage Node deps manually.
`env.ps1` dot-sourcing behavior (PowerShell):
- Dot-source with `. .\env.ps1` so
environment variables remain in your current shell.
- Mirrors the same bootstrap + PATH wiring goals as `env.sh`
for Windows-native shells.
- Supports the same key toggles (`WINDOWS_DIY_SYNC`,
`WINDOWS_DIY_SETUP_NODE_DEPS`,
`WINDOWS_DIY_ENSURE_TREE_SITTER_NIM_PARSER`).
- Applies shared runtime setup from
`non-nix-build/windows/setup-codetracer-runtime-env.ps1`:
- creates `ct_paths.json` when missing (required by
`ct.exe` wrapper)
- sets `CODETRACER_PREFIX`, `CODETRACER_REPO_ROOT_PATH`
- sets `CODETRACER_E2E_CT_PATH` to
`src/build-debug/bin/ct.exe` when available
- prepends `src/build-debug/bin` to PATH for direct `ct`
invocation
- Resolves `.NET` from `WINDOWS_DIY_DOTNET_ROOT` (if set),
then
`$WINDOWS_DIY_INSTALL_ROOT/dotnet/`,
then `%ProgramFiles%/dotnet`, exports `DOTNET_ROOT`, and
validates that the pinned `DOTNET_SDK_VERSION` is installed.
- Resolves Microsoft TTD + WinDbg packages through
`Get-AppxPackage` and exports:
- `WINDOWS_DIY_TTD_EXE`, `WINDOWS_DIY_TTD_DIR`,
`WINDOWS_DIY_TTD_VERSION`,
`WINDOWS_DIY_TTD_REPLAY_DLL`,
`WINDOWS_DIY_TTD_REPLAY_CPU_DLL`
- `WINDOWS_DIY_WINDBG_DIR`,
`WINDOWS_DIY_WINDBG_VERSION`
- debugger engine paths for machine APIs:
`WINDOWS_DIY_CDB_EXE`, `WINDOWS_DIY_DBGENG_DLL`,
`WINDOWS_DIY_DBGMODEL_DLL`,
`WINDOWS_DIY_DBGHELP_DLL`
- note: env scripts prefer
`C:\Windows\System32\dbgeng.dll`/`dbgmodel.dll`/`dbghelp.dll`
when available to avoid AppX ACL `LoadLibrary` failures
from `WindowsApps`.
- minimum version pins `TTD_MIN_VERSION`,
`WINDBG_MIN_VERSION`
- By default, validates TTD/WinDbg availability and pinned
minimum versions (`WINDOWS_DIY_ENSURE_TTD=1`). Set
`WINDOWS_DIY_ENSURE_TTD=0` to bypass this validation.
- Prepends Nim prebuilt `bin` (when available) before
source-built Nim `bin` so Windows runtime dependencies like
`libcrypto-1_1-x64.dll` are discoverable by `ct.exe` and
UI tests.
`env.sh` (Git Bash) now applies the same runtime setup via
`non-nix-build/windows/setup-codetracer-runtime-env.sh`.
## Git hooks (commit and push checks) without Nix
The checks a commit must pass are declared once, in `nix/pre-commit.nix`. The
Nix dev shell installs them as a `pre-commit` shim whose interpreter is a
`/nix/store` path, which Windows cannot execute. `env.ps1` installs the other
leg instead (`ci/dev/install-portable-git-hooks.sh`, also `just
install-portable-git-hooks`): a shim that runs the SAME hooks, read from
`nix/pre-commit.nix` by `ci/dev/portable-pre-commit.py`, through the
`pre-commit` framework with tools from `PATH`. Set
`WINDOWS_DIY_INSTALL_GIT_HOOKS=0` to skip it.
A tool that is missing does not skip its check: the commit that needs it fails,
naming the tool and the command that installs it. `just
portable-pre-commit-doctor` lists every hook and what this machine lacks. The
tools, at the versions Nix pins where a Windows build exists:
| Hooks | Install |
| --- | --- |
| the framework; whitespace, end-of-file, YAML, large files | `python -m pip install --user pre-commit==4.3.0 pre-commit-hooks==6.0.0` |
| `cspell`, `markdownlint-fix` | `npm install cspell@9.2.1 markdownlint-cli2@0.18.1` |
| `shellcheck`, `shfmt` | `scoop install shellcheck shfmt@3.12.0` |
| `taplo` | `cargo install taplo-cli --locked --version 0.10.0` |
| `rustfmt`, `cargo-check`, `clippy`, merge markers, submodule URLs | the Rust toolchain and Git Bash `env.ps1` provisions |
| `nixfmt-rfc-style` | no Windows build: commit `.nix` changes from a Nix host |
A checkout shared with a Nix shell (a WSL distribution on the same disk) has one
hooks directory, and each side reinstalls its own shim on entry. Re-run the
installer after using the other side.
The installer also removes the relative `core.hooksPath=.git/hooks` that
git-hooks.nix writes. Under that value a linked worktree (`git worktree add`)
runs no hooks at all, and nothing reports it. With the value unset, every
worktree uses the shared hooks directory.
## tree-sitter-nim parser generation
After `source env.sh` (Git Bash), the
workflow runs
`non-nix-build/ensure_tree_sitter_nim_parser.sh`, which
regenerates `libs/tree-sitter-nim/src/parser.c` when it is
missing or older than `libs/tree-sitter-nim/grammar.js`.
The script prefers the local CLI at
`libs/tree-sitter-nim/node_modules/.bin/tree-sitter` and uses
lockfile `npm ci` when `package-lock.json` is present. If
local CLI resolution fails, it falls back to an isolated
cached install of `tree-sitter-cli@` under
`.tools/tree-sitter-cli-cache/` so parser generation can
proceed without building this grammar package's native addon
dependencies.
Set `TREE_SITTER_NIM_FALLBACK_CLI_PACKAGE_SPEC` to override
the fallback npm package spec if needed, and set
`WINDOWS_DIY_ENSURE_TREE_SITTER_NIM_PARSER=0` to skip this
parser check.
## ct-remote source selection (local repo vs pinned download)
`bootstrap-windows-diy.ps1` supports two ct-remote bootstrap
inputs:
1. Preferred local source build from sibling repo
`../codetracer-ci` when available.
2. Existing pinned download fallback from
`downloads.codetracer.com`.
Behavior is controlled by environment variables:
- `CT_REMOTE_WINDOWS_SOURCE_MODE` (default: `auto`)
- `auto`: on Windows x64, use local source when available
and otherwise fall back to pinned download; on Windows
arm64, require local source and do not fall back to
pinned x64 download.
- `local`: require the local source path; fail if
unavailable.
- `download`: force pinned-download flow (Windows x64
only).
- `CT_REMOTE_WINDOWS_SOURCE_REPO` (default:
`../../../codetracer-ci` relative to
`non-nix-build/windows/bootstrap-windows-diy.ps1`)
- Override this path when the C# source repo is in a
different location.
- `CT_REMOTE_WINDOWS_PUBLISH_SCRIPT` -- default:
```text
/non-nix-build/windows/publish-desktop-client.ps1
```
- Override this path to use a different publish helper
script. Relative paths are resolved from
`CT_REMOTE_WINDOWS_SOURCE_REPO`.
- When this script is available, bootstrap uses it for
local-source publish and does not require host-global
`dotnet` on PATH.
- When this script is missing, bootstrap falls back to
direct `dotnet publish` behavior.
- `CT_REMOTE_WINDOWS_SOURCE_RID` (default: by architecture)
- `win-x64` on Windows x64.
- `win-arm64` on Windows arm64.
- Override is preserved; supported values are `win-x64`
and `win-arm64`.
Practical local-source publish path discovered from
`../codetracer-ci`:
- Project:
`apps/DesktopClient/DesktopClient.App/DesktopClient.App.csproj`
- Preferred publish command used by bootstrap
(defaults to `PublishAot=false` in that script):
```text
pwsh -File non-nix-build/windows/publish-desktop-client.ps1 -Rid -Configuration Release
```
- Fallback publish command (only when helper script is
missing):
`dotnet publish -c Release -r `
- Output consumed by bootstrap:
`runtime/publish/DesktopClient.App/DesktopClient.App(.exe)`
- Upstream CI workflow
`../codetracer-ci/.github/workflows/publish-desktop-client.yml`
currently publishes `win-x64` and has `win-arm64` commented
as not supported yet, so `win-arm64` local publish should
be treated as environment-dependent. Use
`CT_REMOTE_WINDOWS_SOURCE_RID=win-x64` as an explicit
fallback RID if needed.
## Nim source selection and cache layout
`bootstrap-windows-diy.ps1` supports source and prebuilt Nim
install paths:
- `NIM_WINDOWS_SOURCE_MODE` (default: `auto`)
- `auto`: try source build first; on Windows x64, source
bootstrap failure falls back to pinned prebuilt ZIP; on
non-x64, source failure is terminal and does not attempt
prebuilt fallback.
- `source`: require source build; fail if source bootstrap
fails.
- `prebuilt`: force pinned prebuilt ZIP flow (Windows x64
only).
- `NIM_WINDOWS_SOURCE_REPO` (default from pin:
`NIM_SOURCE_REPO`)
- `NIM_WINDOWS_SOURCE_REF` (default from pin:
`NIM_SOURCE_REF`)
- `NIM_WINDOWS_SOURCE_REVISION` (optional exact commit hash
override; skips ref resolution)
- `NIM_WINDOWS_CSOURCES_REPO` (default from pin:
`NIM_CSOURCES_REPO`)
- `NIM_WINDOWS_CSOURCES_REF` (default from pin:
`NIM_CSOURCES_REF`)
- `NIM_WINDOWS_CSOURCES_REVISION` (optional exact commit
hash override; skips ref resolution)
- `NIM_WINDOWS_SOURCE_CC` (default: `auto`)
- `auto`: prefer `gcc` when available, otherwise use MSVC
(`vcc`/`cl.exe`).
- `gcc`: require `gcc` for source bootstrap.
- `vcc`: require MSVC toolchain for source bootstrap.
- `NIM_WINDOWS_SOURCE_BOOTSTRAP_GCC_FROM_TUP_MSYS2`
(default: `1`)
- When `gcc` is not on PATH, bootstrap may provision/reuse
pinned Tup MSYS2 toolchain to supply `gcc`.
Deterministic cache paths under install root
(`$WINDOWS_DIY_INSTALL_ROOT`, default
`%LOCALAPPDATA%/codetracer/windows-diy`):
- Source build cache root:
`nim//cache/source//`
- Cached source install dir:
`nim//cache/source//nim-`
- Prebuilt install dir:
`nim//prebuilt/nim-`
- Selected install pointer:
`nim//nim.install.relative-path`
- Selected install metadata:
`nim//nim.install.meta`
- Source cache metadata:
`nim//cache/source//nim.source.meta`
`cache-key` is SHA256 over normalized build inputs:
- Nim version + architecture
- Nim repo/ref/resolved revision
- csources repo/ref/resolved revision
- compiler/toolchain hint snapshot (`CC`, `cl`, `gcc`
detection/version probes)
Reuse policy:
- If `nim.source.meta` matches current normalized inputs and
`nim --version` matches `NIM_VERSION`, bootstrap reuses the
cached source install and skips rebuild.
- Otherwise bootstrap rebuilds source into a new
deterministic cache key directory.
- In `auto` mode on Windows x64, source failure falls back
to prebuilt ZIP flow.
- In `auto` mode on non-x64 architectures, source failure is
returned directly and prebuilt fallback is skipped because
only `nim-_x64.zip` is pinned.
## Cap'n Proto source selection and cache layout
`bootstrap-windows-diy.ps1` supports source and prebuilt
Cap'n Proto install paths:
- `CAPNP_WINDOWS_SOURCE_MODE` (default: `auto`)
- `auto`: on Windows x64, use pinned prebuilt ZIP by
default; on non-x64, build from source.
- `source`: require source build.
- `prebuilt`: force pinned prebuilt ZIP flow (Windows x64
only).
- `CAPNP_WINDOWS_SOURCE_REPO` (default from pin:
`CAPNP_SOURCE_REPO`)
- `CAPNP_WINDOWS_SOURCE_REF` (default from pin:
`CAPNP_SOURCE_REF`)
- `CAPNP_WINDOWS_SOURCE_REVISION` (optional exact commit
hash override; skips ref resolution)
- `CAPNP_WINDOWS_CMAKE_GENERATOR` (optional explicit
generator override; defaults to `Ninja` when available,
else `Visual Studio 17 2022`)
Deterministic cache paths under install root
(`$WINDOWS_DIY_INSTALL_ROOT`, default
`%LOCALAPPDATA%/codetracer/windows-diy`):
- Source build cache root:
`capnp//cache/source//`
- Cached source install dir:
`capnp//cache/source//install`
- Prebuilt install dir:
`capnp//prebuilt/capnproto-tools-win32-`
- Selected install pointer:
`capnp//capnp.install.relative-path`
- Selected install metadata:
`capnp//capnp.install.meta`
- Source cache metadata:
`capnp//cache/source//capnp.source.meta`
`cache-key` is SHA256 over normalized build inputs:
- Cap'n Proto version + architecture
- Cap'n Proto repo/ref/resolved revision
- CMake generator + CMake version hint
- compiler/toolchain hint snapshot (`CC`, `cl`, `gcc`
detection/version probes)
Reuse policy:
- If `capnp.source.meta` matches current normalized inputs
and `capnp --version` matches `CAPNP_VERSION`, bootstrap
reuses the cached source install and skips rebuild.
- Otherwise bootstrap rebuilds source into a new
deterministic cache key directory.
- In `auto` mode on Windows x64, bootstrap uses pinned
prebuilt by default.
- In `auto` mode on non-x64 architectures, bootstrap selects
source mode directly.
## Tup source selection and cache layout
`bootstrap-windows-diy.ps1` supports pinned prebuilt Tup
bootstrap by default, with source mode still available:
- `TUP_WINDOWS_SOURCE_MODE` (default: `prebuilt`)
- `auto`: prefer pinned prebuilt ZIP first, then fallback
to source build if prebuilt bootstrap fails.
- `source`: require source build.
- `prebuilt`: require prebuilt install (uses pinned
`TUP_PREBUILT_*` defaults unless
`TUP_WINDOWS_PREBUILT_*` overrides are set).
- `TUP_WINDOWS_SOURCE_REPO` (default from pin:
`TUP_SOURCE_REPO`)
- `TUP_WINDOWS_SOURCE_REF` (default from pin:
`TUP_SOURCE_REF`)
- `TUP_WINDOWS_SOURCE_REVISION` (optional exact commit hash
override; skips ref resolution)
- `TUP_WINDOWS_SOURCE_BUILD_COMMAND` (default from pin:
`TUP_SOURCE_BUILD_COMMAND`, currently
`TUP_MINGW=1 TUP_MINGW32=0 ./bootstrap.sh`)
- When this remains at the pinned default, bootstrap now
generates and runs `codetracer-bootstrap-windows.sh`
inside the Tup source checkout (MinGW-only compile path
that avoids upstream `bootstrap.sh` Unix defaults and
does not require `pkg-config fuse`).
- Any non-default value is treated as an explicit override
and executed as-is in the MSYS2 shell.
- `TUP_WINDOWS_MSYS2_BASE_VERSION` (default from pin:
`TUP_MSYS2_BASE_VERSION`)
- `TUP_WINDOWS_MSYS2_PACKAGES` (default from pin:
`TUP_MSYS2_PACKAGES`)
- `TUP_WINDOWS_SELF_HOST_UPDATE` (default: unset/off)
- Off by default: bootstrap treats manually compiled
`build/tup.exe` as the final artifact and skips
`tup init`/`tup upd`.
- Set to `1`/`true`/`yes`/`on` only for debugging
self-host updates inside the staged source checkout.
- Pinned prebuilt defaults (toolchain pins):
- `TUP_PREBUILT_URL` (currently
`https://gittup.org/tup/win32/tup-latest.zip`)
- `TUP_PREBUILT_SHA256` (currently
`efa3a4ecb4e63a9b3f9ced9c18d15ef59fabaf1942c9fffa9a930fb5ebf853a0`)
- `TUP_PREBUILT_VERSION` (currently `latest`)
- Optional prebuilt override controls:
- `TUP_WINDOWS_PREBUILT_URL`
- `TUP_WINDOWS_PREBUILT_SHA256`
- `TUP_WINDOWS_PREBUILT_VERSION`
Deterministic cache paths under install root
(`$WINDOWS_DIY_INSTALL_ROOT`, default
`%LOCALAPPDATA%/codetracer/windows-diy`):
- Source build cache root:
`tup/cache/source//`
- Cached source install dir:
`tup/cache/source//install`
- Optional explicit prebuilt install dir:
`tup/prebuilt//install`
- Selected install pointer:
`tup/tup.install.relative-path`
- Selected install metadata: `tup/tup.install.meta`
- Source cache metadata:
`tup/cache/source//tup.source.meta`
`cache-key` is SHA256 over normalized build inputs:
- Tup source repo/ref/resolved revision
- requested source build command
- effective source build command identity (for default path
this is the codetracer-owned generated script version)
Host prerequisites for source mode:
- `git` must be present on PATH.
- Bootstrap now provisions an MSYS2 base
(`msys2-base-x86_64-.tar.xz`) under
`$WINDOWS_DIY_INSTALL_ROOT/tup/msys2//msys64`
and installs the pinned package set
(`mingw-w64-x86_64-gcc mingw-w64-x86_64-pkgconf make`)
before running the Tup source build command.
- Tup source bootstrap exports `TUP_MINGW=1` and
`TUP_MINGW32=0`, and prepends MinGW bin directories
(`mingw64/bin`, `mingw32/bin`) in the MSYS2 shell PATH so
Tup builds target MinGW compilers by default.
- In default mode, bootstrap emits
`codetracer-bootstrap-windows.sh` and compiles Tup directly
with the MinGW toolchain (including `src/compat/win32` and
`src/tup/server/windepfile.c`), then installs the compiled
`build/tup.exe` as the bootstrap artifact without running
self-host `tup init`/`tup upd`.
Reuse policy:
- If `tup.source.meta` matches current normalized inputs and
`tup --version` succeeds, bootstrap reuses the cached
source install and skips rebuild.
- `prebuilt` and `auto` modes use pinned upstream ZIP
URL/SHA by default for reproducibility. Use `source` mode
when you need to force source bootstrap.
## Version Bump Process (`toolchain-versions.env`)
1. Pick target versions and verify they exist upstream:
- `RUSTUP_VERSION`:
- `RUST_TOOLCHAIN_VERSION`:
- `NODE_VERSION`:
- `UV_VERSION`:
- `DOTNET_SDK_VERSION`:
(pin
exact SDK patch used by Windows non-Nix workflow)
- `WINDBG_MIN_VERSION`: minimum accepted AppX package
version for `Microsoft.WinDbg`
- `TTD_MIN_VERSION`: minimum accepted AppX package
version for `Microsoft.TimeTravelDebugging`
- `NIM_VERSION`:
(download
assets are under `https://nim-lang.org/download/`)
- `NIM_WIN_X64_SHA256`: SHA256 for
`nim-_x64.zip` from
`https://nim-lang.org/download/nim-_x64.zip.sha256`
- `NIM_SOURCE_REPO`: Nim source repository URL used for
source mode cache keying/build
- `NIM_SOURCE_REF`: Nim source ref to resolve
(tag/branch/refname)
- `NIM_CSOURCES_REPO`: Nim csources repository URL used
for source mode cache keying/build
- `NIM_CSOURCES_REF`: Nim csources ref to resolve
(tag/branch/refname)
- `CT_REMOTE_VERSION`: artifact revision at:
```text
https://downloads.codetracer.com/DesktopClient.App/DesktopClient.App-win-x64-.tar.gz
```
- `CT_REMOTE_WIN_X64_SHA256`: SHA256 for that exact
Windows archive.
- `CAPNP_VERSION`:
(Windows asset
`https://capnproto.org/capnproto-c++-win32-.zip`)
- `CAPNP_WIN_X64_SHA256`: SHA256 for that exact
`capnproto-c++-win32-.zip` asset. Cap'n Proto
currently does not publish a `.sha256` sidecar for this
ZIP, so we pin and verify the repo-managed hash.
- `CAPNP_SOURCE_REPO`: Cap'n Proto source repository URL
used for source mode cache keying/build.
- `CAPNP_SOURCE_REF`: Cap'n Proto source ref to resolve
(tag/branch/refname).
- `TUP_SOURCE_REPO`: Tup source repository URL used for
source mode cache keying/build (default pin points to
the variants-capable Windows fork).
- `TUP_SOURCE_REF`: Tup source ref to resolve
(tag/branch/refname), defaulting to
`variants-for-windows`.
- `TUP_SOURCE_BUILD_COMMAND`: Bash command executed inside
Tup source checkout to build Tup for Windows bootstrap
(default pin sets
`TUP_MINGW=1 TUP_MINGW32=0 ./bootstrap.sh`).
- `TUP_PREBUILT_VERSION`: version label for the pinned
official prebuilt Tup archive.
- `TUP_PREBUILT_URL`: prebuilt Tup archive URL (default
pin points to
`https://gittup.org/tup/win32/tup-latest.zip`).
- `TUP_PREBUILT_SHA256`: SHA256 for that exact prebuilt
Tup archive.
- `TUP_MSYS2_BASE_VERSION`: pinned MSYS2 base release
date (YYYYMMDD) used for Tup source prerequisites.
- `TUP_MSYS2_BASE_X64_SHA256`: SHA256 for
`msys2-base-x86_64-.tar.xz`.
- `TUP_MSYS2_PACKAGES`: space-separated MSYS2 package
list installed for Tup source builds (default pin uses
MinGW-targeted
`mingw-w64-x86_64-gcc mingw-w64-x86_64-pkgconf make`).
2. Update
`non-nix-build/windows/toolchain-versions.env`.
3. Keep
`non-nix-build/windows/bootstrap-windows-diy.ps1` in
sync with the same pinned versions.
4. Validate checksums from upstream checksum documents
before merging:
- Node.js archive must match `SHASUMS256.txt` from
`https://nodejs.org/dist/v/`.
- rustup installer must match
`rustup-init.exe.sha256` from
`https://static.rust-lang.org/rustup/archive///`.
- uv archive must match `.sha256` from the
corresponding GitHub release tag.
- Nim prebuilt archive must match
`nim-_x64.zip.sha256` from
`https://nim-lang.org/download/`.
- ct-remote archive must match the pinned
`CT_REMOTE_WIN_X64_SHA256` hash in
`toolchain-versions.env`.
- Cap'n Proto archive must match the pinned
`CAPNP_WIN_X64_SHA256` hash in
`toolchain-versions.env`.
### Nim hash bump helper
Use this command from PowerShell to compute and verify the
Nim hash you should pin:
```powershell
$version = "2.2.6"
$asset = "nim-$version`_x64.zip"
$url = "https://nim-lang.org/download/$asset"
$sidecarUrl = "$url.sha256"
$tmp = Join-Path $env:TEMP $asset
Invoke-WebRequest -Uri $url -OutFile $tmp -UseBasicParsing
$calculated = (Get-FileHash -Path $tmp -Algorithm SHA256).Hash.ToLowerInvariant()
$upstream = ((Invoke-WebRequest -Uri $sidecarUrl -UseBasicParsing).Content -split '\s+')[0].ToLowerInvariant()
"calculated=$calculated"
"upstream=$upstream"
if ($calculated -ne $upstream) { throw "Nim hash mismatch" }
```
## Windows note: ct-native-replay and UI tests
- `ct-native-replay` (from sibling repo
`../codetracer-native-backend`) can now be compiled on Windows
in a compile-only mode.
- Runtime RR replay/record behavior is still Linux-only, so
Windows flows that require active RR replay are expected to
return explicit unsupported errors.
- This means `ct record` for scenarios that require the RR
backend (for example some Noir flows) may still fail on
Windows even when the binary is present.
- Current observed Windows non-Nix issue: Noir recording can
hang in `nargo trace`, which leaves `trace-*` folders with
incomplete data (for example only `symbols.json`) and
causes Playwright UI tests to timeout waiting for core
components.
- Interim test-unblock recommendation: run Web-mode
program-agnostic/layout UI tests with
`CODETRACER_TRACE_PATH` set to a known non-Noir trace
folder (for example `src/tui/trace`) while investigating
the Noir hang.
- Windows-native time-travel backend is now implemented
behind `DebuggerBackend` via `ct-native-replay` using
Microsoft TTD + dbgeng.
- Enable native dbgeng control with
`CT_TTD_CONTROL_MODE=dbgeng` (optional: enforce
`CT_TTD_REQUIRE_NATIVE_DBGENG=1` to disable cdb
fallback).
- The Windows env scripts already export
`WINDOWS_DIY_TTD_*`, `WINDOWS_DIY_CDB_EXE`, and
`WINDOWS_DIY_DBGENG_DLL`; ensure those are available in
the shell that launches `ct-native-replay`.
- API surface reference lives in
`codetracer-native-backend/docs/dbgeng-api-surface.md`
alongside the implementation.
### ct-remote hash bump helper
Use this command from PowerShell to compute the hash you
should pin:
```powershell
$revision = "102d2c8"
$asset = "DesktopClient.App-win-x64-$revision.tar.gz"
$url = "https://downloads.codetracer.com/DesktopClient.App/$asset"
$tmp = Join-Path $env:TEMP $asset
Invoke-WebRequest -Uri $url -OutFile $tmp -UseBasicParsing
(Get-FileHash -Path $tmp -Algorithm SHA256).Hash.ToLowerInvariant()
```
### Cap'n Proto hash bump helper
Use this command from PowerShell to compute the hash you
should pin:
```powershell
$version = "1.3.0"
$asset = "capnproto-c++-win32-$version.zip"
$url = "https://capnproto.org/$asset"
$tmp = Join-Path $env:TEMP $asset
Invoke-WebRequest -Uri $url -OutFile $tmp -UseBasicParsing
(Get-FileHash -Path $tmp -Algorithm SHA256).Hash.ToLowerInvariant()
```
`bootstrap-windows-diy.ps1` keeps pinned ct-remote download
constrained to Windows x64
(`DesktopClient.App-win-x64-.tar.gz` +
`CT_REMOTE_WIN_X64_SHA256`). On Windows arm64,
`CT_REMOTE_WINDOWS_SOURCE_MODE=auto` now routes to
local-source mode and will not attempt x64 pinned-download
fallback. Nim source mode can be attempted on both `x64` and
`arm64`, while Nim prebuilt mode remains pinned to
`nim-_x64.zip` and therefore x64-only. Cap'n Proto
now supports
`CAPNP_WINDOWS_SOURCE_MODE=auto|source|prebuilt`: prebuilt
remains pinned to x64 ZIP, and source mode provides the
non-x64 path with deterministic source-cache reuse. Tup now
supports `TUP_WINDOWS_SOURCE_MODE=prebuilt|auto|source` with
default pinned prebuilt install from
`https://gittup.org/tup/win32/tup-latest.zip` and
`TUP_PREBUILT_SHA256`, while keeping source bootstrap support
pinned to `https://github.com/zah/tup.git` branch
`variants-for-windows`, MinGW-targeted bootstrap defaults
(`TUP_SOURCE_BUILD_COMMAND=TUP_MINGW=1 TUP_MINGW32=0 ./bootstrap.sh`)
and MSYS2 prerequisite pins (`TUP_MSYS2_*`) to provision
`mingw-w64-x86_64-gcc`, `mingw-w64-x86_64-pkgconf`, and
`make` reproducibly before source builds. Source defaults come
from `NIM_SOURCE_*`, `NIM_CSOURCES_*`, `CAPNP_SOURCE_*`, and
`TUP_SOURCE_*` pins.
### ct-remote arm64 blocker runbook
If Windows arm64 `ct-remote` bootstrap is still blocked, run
these exact URL probes first:
```powershell
$revision = "102d2c8"
$base = "https://downloads.codetracer.com/DesktopClient.App"
$urls = @(
"$base/DesktopClient.App-win-x64-$revision.tar.gz",
"$base/DesktopClient.App-win-arm64-$revision.tar.gz",
"$base/DesktopClient.App-win-aarch64-$revision.tar.gz",
"$base/DesktopClient.App-win-arm64ec-$revision.tar.gz"
)
foreach ($url in $urls) {
try {
$r = Invoke-WebRequest -UseBasicParsing -Method Head -Uri $url
"$url -> $([int]$r.StatusCode) $($r.StatusDescription)"
} catch {
$code = if ($_.Exception.Response -and $_.Exception.Response.StatusCode) { [int]$_.Exception.Response.StatusCode } else { "ERR" }
"$url -> $code"
}
}
```
Expected outcome as of 2026-02-09:
- `DesktopClient.App-win-x64-102d2c8.tar.gz` returns `200`.
- `DesktopClient.App-win-arm64-102d2c8.tar.gz` returns
`404`.
- `DesktopClient.App-win-aarch64-102d2c8.tar.gz` returns
`404`.
- `DesktopClient.App-win-arm64ec-102d2c8.tar.gz` returns
`404`.
Required inputs to unblock arm64 support:
1. Official arm64 artifact URL at
`downloads.codetracer.com` for the pinned revision (or an
approved new revision to pin), including exact filename
convention (`win-arm64` vs `win-aarch64`).
2. A trustworthy SHA256 value for that exact artifact. There
is currently no sidecar checksum document for ct-remote
assets, so this must come from release engineering
process/publish channel.
3. Confirmation whether x64 and arm64 must share
`CT_REMOTE_VERSION` or whether per-arch revisions are
allowed.
### Cap'n Proto non-x64 mode-routing probe
To verify that `CAPNP_WINDOWS_SOURCE_MODE=auto` selects
source mode for non-x64 architectures without requiring an
arm64 host, run:
```powershell
$env:WINDOWS_DIY_ARCH_OVERRIDE = "arm64"
$env:CAPNP_WINDOWS_SOURCE_MODE = "auto"
$env:CAPNP_WINDOWS_SOURCE_REVISION = "7dbb95989721016f8b590245ec7528c6ff03d1fe"
$env:WINDOWS_DIY_SKIP_NIM = "1"
$env:WINDOWS_DIY_SKIP_CT_REMOTE = "1"
pwsh -File non-nix-build/windows/bootstrap-windows-diy.ps1
```
Expected behavior:
- Bootstrap logs Cap'n Proto source-mode activity
(`Building Cap'n Proto ... from source` or
`Cap'n Proto ... source cache hit`).
- `capnp//capnp.install.meta` records
`requested_mode=auto` and `effective_mode=source`.
- Probe intentionally skips Nim and `ct-remote` via
`WINDOWS_DIY_SKIP_NIM=1` and
`WINDOWS_DIY_SKIP_CT_REMOTE=1` so Cap'n Proto route
selection can be validated in isolation.
## Required Validation Commands
Run these from repo root:
```powershell
pwsh -File non-nix-build/windows/validate-toolchain-versions.ps1
pwsh -Command "$errors = $null; [void][System.Management.Automation.Language.Parser]::ParseFile('non-nix-build/windows/bootstrap-windows-diy.ps1', [ref]$null, [ref]$errors); if ($errors.Count -gt 0) { $errors | ForEach-Object { Write-Error $_.Message }; exit 1 }"
pwsh -Command "$errors = $null; [void][System.Management.Automation.Language.Parser]::ParseFile('non-nix-build/windows/validate-toolchain-versions.ps1', [ref]$null, [ref]$errors); if ($errors.Count -gt 0) { $errors | ForEach-Object { Write-Error $_.Message }; exit 1 }"
bash -n env.sh
```
## Changelog and Status Update Expectations
- Add an entry to `CHANGELOG.md` describing the Windows
toolchain version bump.
- Update `windows-porting-initiative-status.md` when process
or automation changes.
---
### Overview (`src/tracer/README.md`)
tracer
-----------------
ruby tracer MOVED to a separate repo
original readme:
a very early version of a ruby codetracer tracer, that records data about
a running ruby program which is later used to simulate a limited replay of it
in codetracer.
### usage
```bash
ruby trace.rb
# -> trace.json
# and some debugging output currently
# TODO more options
# or
python3 trace.py
# -> trace.json
```
### run example
```bash
# enter codetracer dir, with enabled direnv
# so you have ruby from our nix shell in PATH
cd src/ruby-tracer
ruby trace.rb calc.rb
cat trace.json
# ..
```
### in codetracer
it should be callable by codetracer with
```bash
ct record
ct replay
```
not yet integrated fully for python
### TODO/problems
* optimizations
* cleanup code
* support more kind of ruby values: only several simples types supported currently
* others
---
### Overview (`test-programs/py_checklist/README.md`)
# Python Checklist Test Programs
This suite mirrors the `ruby_checklist` sample by collecting small,
targeted probes that exercise diverse areas of the Python language and
standard library. The code is intentionally verbose and heavily
commented so that engineers working across multiple languages can
quickly understand what a probe demonstrates and why it matters.
## Goals
- Capture representative call stacks for critical Python features so the
debugger and tracer integrations can be validated.
- Provide ready-made snippets that illustrate specific language
behaviors, pitfalls, and idioms for reference during development.
- Keep each probe independent and side-effect light so they can run in
isolation without hidden dependencies.
## Organization
- Minimum Python version: **3.11**. This allows us to cover structural
pattern matching, `except*` for exception groups, and other
post-3.10 features without fallbacks.
- Each feature cluster lives in its own module (for example,
`basics.py`, `functions_exceptions.py`, …). Modules should export
numbered `demo_*` functions that focus on a single behavior.
- A top-level `run_all()` function inside each module executes its demos
sequentially, printing numbered lines so missed events are easy to
spot in traces.
- Inline comments and docstrings must explain **what** the snippet does,
**why** someone would use it, and any common gotchas.
- Helpers that spin up threads or processes must protect their entry
points with `if __name__ == "__main__":` to stay portable across
platforms (especially Windows).
## Feature Coverage
| Area | Modules / Demos | Notes |
| --- | --- | --- |
| Literals, operators, control flow | `basics.run_all()` | Covers literals, slicing, unpacking, match/case, loop `else`. |
| Functions, decorators, exceptions | `functions_exceptions.run_all()` | Demonstrates call signatures, mutable defaults, closures, decorator stacking, caching, chained exceptions, `ExceptionGroup`. |
| Context managers & iterators | `contexts_iterators.run_all()` | Custom `__enter__/__exit__`, `contextlib`, iterator protocol, generators with `send`/`throw`/`close`. |
| Async & concurrency | `async_concurrency.run_all()` | Async context/iter, `create_task`, contextvars, threading/TLS, `ThreadPoolExecutor`, guarded multiprocessing. |
| Data model & classes | `data_model.run_all()` | `__repr__`, formatting, rich ops, properties with `__slots__`, attribute hooks, descriptors, subclass hooks, metaclass lifecycle. |
| Collections & dataclasses | `collections_dataclasses.run_all()` | Dataclasses with slots/order, enums and flags, namedtuple/deque/defaultdict/Counter, mapping proxy, pattern matching with `__match_args__`. |
| System utilities | `system_utils.run_all()` | `subprocess.run` with env overrides, tz-aware datetime math, `Decimal`/`Fraction`, deterministic random, regex capture groups. |
| Introspection & dynamic code | `introspection.run_all()` | `inspect.signature`, dynamic attributes, globals/locals snapshots, `eval`/`exec`/`compile`, AST parse/dump (with security notes in comments). |
| Import system | `imports_demo.run_all()` | Runtime module creation via `types.ModuleType`, adjusting `sys.path`, `importlib.import_module`, `__all__`, module caching notes. |
| Logging, warnings, GC, typing | `advanced_runtime.run_all()` | Scoped logging config, forced warnings, GC cycle collection with weakrefs/finalizers, runtime protocol checks, generics. |
| Miscellaneous constructs | `miscellaneous.run_all()` | `iter(callable, sentinel)`, `del`, custom mapping protocol, context manager without suppression. |
> **Planned additions:** Remaining entries from the original checklist
> (I/O & serialization, subprocess variants, JSON/pickle, etc.) can be
> layered in by following the same pattern—add `.py`, expose
> `run_all()`, then register it in `__main__.py`.
## Running The Suite
Once all modules are implemented, you will be able to run the entire
checklist from the project root:
```bash
python -m test-programs.python_checklist
```
During early development, it is fine to run individual modules directly
with `python test-programs/python_checklist/.py` to iterate on a
specific feature cluster.
## Contribution Guidelines
- Follow the numbering scheme diligently so console output gaps surface
immediately.
- Prefer deterministic behavior (avoid randomness unless the purpose is
to exercise the random module).
- Clean up temporary files, subprocesses, and other resources within the
probe itself. Each demo should leave the environment in the same state
it found it.
- If a snippet showcases risky behavior (e.g., `eval`), highlight the
security implications in comments.
This structure keeps the checklist maintainable and allows future
contributors to plug in additional probes without surprising teammates.
---
### Overview (`test-programs/rs_checklist/README.md`)
# Rust Checklist Programs
This suite mirrors the existing Python and Ruby checklists with a
Rust-specific set of probes that exercise language features and
standard-library behaviors useful for debugging CodeTracer. Programs are
small, verbose, and grouped by topic so you can run them together or
drill into a single cluster.
## Running
```bash
# from repo root
nix develop -c cargo run -p rs_checklist
# to run a single module binary (examples)
nix develop -c cargo run -p rs_checklist --bin lexical_bindings
nix develop -c cargo run -p rs_checklist --bin traits_generics
```
The crate targets Rust 1.85+/edition 2024 to capture newer drop-order
and capture semantics. Each module prints a heading before running its
scenarios so missing events are easy to spot.
## Layout
- `src/main.rs` wires the modules and prints section headers.
- `src/macros_support.rs` holds traits used by the proc-macro derive.
- `macros/` is a tiny proc-macro crate providing `#[derive(AutoHello)]`
to demonstrate attribute macros.
Modules (roughly aligned to the provided coverage map):
- `lexical_bindings`: bindings, shadowing vs `mut`, `const`/`static`/`static mut`,
`thread_local!`, raw identifiers for reserved keywords, name resolution.
- `iterators_collections`: literals/ranges/struct forms, operator overloading,
coercions/casts, closures for Fn/FnMut/FnOnce, custom iterators,
Vec/VecDeque/LinkedList/HashMap/BTreeMap/BinaryHeap, formatting, Unicode-safe
slicing.
- `ownership_borrowing`: Copy vs move, Clone/Drop, partial moves, RAII drop
ordering (2024 edition semantics), lifetimes/HRTBs, reborrowing, raw refs.
- `patterns_control`: destructuring (tuples/structs/slices), `ref`/`ref mut`,
or-patterns, guards, range patterns, let-else, `if let` chains, labelled loops,
diverging arms using `!`.
- `traits_generics`: supertraits, default methods, trait objects vs `dyn`,
associated types/consts, GATs, `impl Trait` in args/returns, async fn in traits,
const generics, PhantomData.
- `errors_runtime`: Result/Option with `?` and `ok_or`, custom error impls,
file I/O (read/write/missing), `Instant` timing, `panic!` hooks plus
`catch_unwind`.
- `concurrency_async`: threads with Arc/RwLock, barriers, mpsc + sync
channels, atomics with orderings, condvars/Once, async/await with joins and a
manual `Future`.
- `smart_pointers`: Box raw conversions, Rc cycles + Weak, Cell/RefCell borrow
panic, Arc, Cow, pinning `PhantomPinned` self-references,
NonZero/NonNull pointer niches.
- `unsafe_macros_const`: unsafe blocks, `MaybeUninit`, unions, `transmute`,
repr(C) + extern "C" functions, macro_rules + proc-macro derive,
cfg attributes, const fn/const blocks, visibility modifiers, Any/type_id
introspection, size/align checks.
Each module exposes a `run()` invoked by `main` with numbered `println!`
output to keep trace review predictable.
---
### Overview (`test-programs/stylus_fund_tracker/README.md`)
# Stylus Hello World
Project starter template for writing Arbitrum Stylus programs in Rust using the [stylus-sdk](https://github.com/OffchainLabs/stylus-sdk-rs). It includes a Rust implementation of a basic counter Ethereum smart contract:
```js
// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;
contract Counter {
uint256 public number;
function setNumber(uint256 newNumber) public {
number = newNumber;
}
function increment() public {
number++;
}
}
```
To set up more minimal example that still uses the Stylus SDK, use `cargo stylus new --minimal ` under [OffchainLabs/cargo-stylus](https://github.com/OffchainLabs/cargo-stylus).
## Quick Start
Install [Rust](https://www.rust-lang.org/tools/install), and then install the Stylus CLI tool with Cargo
```bash
cargo install --force cargo-stylus cargo-stylus-check
```
Add the `wasm32-unknown-unknown` build target to your Rust compiler:
```
rustup target add wasm32-unknown-unknown
```
You should now have it available as a Cargo subcommand:
```bash
cargo stylus --help
```
Then, clone the template:
```
git clone https://github.com/OffchainLabs/stylus-hello-world && cd stylus-hello-world
```
### Testnet Information
All testnet information, including faucets and RPC endpoints can be found [here](https://docs.arbitrum.io/stylus/reference/testnet-information).
### ABI Export
You can export the Solidity ABI for your program by using the `cargo stylus` tool as follows:
```bash
cargo stylus export-abi
```
which outputs:
```js
/**
* This file was automatically generated by Stylus and represents a Rust program.
* For more information, please see [The Stylus SDK](https://github.com/OffchainLabs/stylus-sdk-rs).
*/
// SPDX-License-Identifier: MIT-OR-APACHE-2.0
pragma solidity ^0.8.23;
interface ICounter {
function number() external view returns (uint256);
function setNumber(uint256 new_number) external;
function mulNumber(uint256 new_number) external;
function addNumber(uint256 new_number) external;
function increment() external;
}
```
Exporting ABIs uses a feature that is enabled by default in your Cargo.toml:
```toml
[features]
export-abi = ["stylus-sdk/export-abi"]
```
## Deploying
You can use the `cargo stylus` command to also deploy your program to the Stylus testnet. We can use the tool to first check
our program compiles to valid WASM for Stylus and will succeed a deployment onchain without transacting. By default, this will use the Stylus testnet public RPC endpoint. See here for [Stylus testnet information](https://docs.arbitrum.io/stylus/reference/testnet-information)
```bash
cargo stylus check
```
If successful, you should see:
```bash
Finished release [optimized] target(s) in 1.88s
Reading WASM file at stylus-hello-world/target/wasm32-unknown-unknown/release/stylus-hello-world.wasm
Compressed WASM size: 8.9 KB
Program succeeded Stylus onchain activation checks with Stylus version: 1
```
Next, we can estimate the gas costs to deploy and activate our program before we send our transaction. Check out the [cargo-stylus](https://github.com/OffchainLabs/cargo-stylus) README to see the different wallet options for this step:
```bash
cargo stylus deploy \
--private-key-path= \
--estimate-gas
```
You will then see the estimated gas cost for deploying before transacting:
```bash
Deploying program to address e43a32b54e48c7ec0d3d9ed2d628783c23d65020
Estimated gas for deployment: 1874876
```
The above only estimates gas for the deployment tx by default. To estimate gas for activation, first deploy your program using `--mode=deploy-only`, and then run `cargo stylus deploy` with the `--estimate-gas` flag, `--mode=activate-only`, and specify `--activate-program-address`.
Here's how to deploy:
```bash
cargo stylus deploy \
--private-key-path=
```
The CLI will send 2 transactions to deploy and activate your program onchain.
```bash
Compressed WASM size: 8.9 KB
Deploying program to address 0x457b1ba688e9854bdbed2f473f7510c476a3da09
Estimated gas: 1973450
Submitting tx...
Confirmed tx 0x42db…7311, gas used 1973450
Activating program at address 0x457b1ba688e9854bdbed2f473f7510c476a3da09
Estimated gas: 14044638
Submitting tx...
Confirmed tx 0x0bdb…3307, gas used 14044638
```
Once both steps are successful, you can interact with your program as you would with any Ethereum smart contract.
## Calling Your Program
This template includes an example of how to call and transact with your program in Rust using [ethers-rs](https://github.com/gakonst/ethers-rs) under the `examples/counter.rs`. However, your programs are also Ethereum ABI equivalent if using the Stylus SDK. **They can be called and transacted with using any other Ethereum tooling.**
By using the program address from your deployment step above, and your wallet, you can attempt to call the counter program and increase its value in storage:
```rs
abigen!(
Counter,
r#"[
function number() external view returns (uint256)
function setNumber(uint256 number) external
function increment() external
]"#
);
let counter = Counter::new(address, client);
let num = counter.number().call().await;
println!("Counter number value = {:?}", num);
let _ = counter.increment().send().await?.await?;
println!("Successfully incremented counter via a tx");
let num = counter.number().call().await;
println!("New counter number value = {:?}", num);
```
Before running, set the following env vars or place them in a `.env` file (see: [.env.example](./.env.example)) in this project:
```
RPC_URL=https://sepolia-rollup.arbitrum.io/rpc
STYLUS_CONTRACT_ADDRESS=
PRIV_KEY_PATH=
```
Next, run:
```
cargo run --example counter --target=
```
Where you can find `YOUR_ARCHITECTURE` by running `rustc -vV | grep host`. For M1 Apple computers, for example, this is `aarch64-apple-darwin` and for most Linux x86 it is `x86_64-unknown-linux-gnu`
## Build Options
By default, the cargo stylus tool will build your project for WASM using sensible optimizations, but you can control how this gets compiled by seeing the full README for [cargo stylus](https://github.com/OffchainLabs/cargo-stylus). If you wish to optimize the size of your compiled WASM, see the different options available [here](https://github.com/OffchainLabs/cargo-stylus/blob/main/OPTIMIZING_BINARIES.md).
## Peeking Under the Hood
The [stylus-sdk](https://github.com/OffchainLabs/stylus-sdk-rs) contains many features for writing Stylus programs in Rust. It also provides helpful macros to make the experience for Solidity developers easier. These macros expand your code into pure Rust code that can then be compiled to WASM. If you want to see what the `stylus-hello-world` boilerplate expands into, you can use `cargo expand` to see the pure Rust code that will be deployed onchain.
First, run `cargo install cargo-expand` if you don't have the subcommand already, then:
```
cargo expand --all-features --release --target=
```
Where you can find `YOUR_ARCHITECTURE` by running `rustc -vV | grep host`. For M1 Apple computers, for example, this is `aarch64-apple-darwin`.
## License
This project is fully open source, including an Apache-2.0 or MIT license at your choosing under your own copyright.
---
### Overview (`examples/ruby/project-euler/README.md`)
project-euler-ruby
==================
Answers to , in ruby.
## Expectations
Ruby 1.9.3
---
### Overview (`src/public/third_party/README.md`)
This directory stores third-party libraries used by the frontend. Files that are in folders are guaranteed to be third-party.
Files in the root of this directory are either single-file third-party libraries, or are our custom modifications and/or glue code.
So if you do something like a `grep -r` while in the process of renaming something, or when tracking where a certain style or javascript
behaviour is coming from, make sure to also check out the files in the root directory in this folder!
---
### Overview (`src/db-backend/test-programs/reprobuild_hcr_in_codetracer/README.md`)
# Reprobuild HCR In CodeTracer Fixture
This fixture is intentionally shaped for the full Reprobuild HCR in
CodeTracer gate, not the older same-process direct-patch prototype.
The accepted test path is:
- Reprobuild builds the initial native target with HCR-required compiler
flags and build graph metadata.
- CodeTracer launches the target under MCR recording from process start.
- The edit driver replaces `src/patchable.c` with generation 1 through a
normal filesystem write.
- The production Reprobuild HCR coordinator rebuilds the changed translation
unit and delivers a direct patch to the in-process agent.
- CodeTracer DAP validates old and new source generations in live recording
and replay.
The fixture files deliberately avoid any in-process shortcut to the patching
layer. The Rust gate scans this directory for those shortcuts before running.
---
### Overview (`src/public/third_party/monaco-themes/README.md`)
codetracer monaco-themes fork:
-----------------------------------
* added our custom codetracer themes in themes/customThemes : Victor Velitchkov's work
* 6.10.2025: (alexander): removed yarn.lock, package.json, index.html, webpack.config.js and `scripts/`, because we don't use them and to prevent receiving dependabot alerts for them
## monaco-themes
See extended demo at [https://editor.bitwiser.in](https://editor.bitwiser.in)
A list of theme definitions to be used with [monaco-editor](https://microsoft.github.io/monaco-editor/) in browser. See [minimal demo]('https://bitwiser.in/monaco-themes/')
### Usage
```sh
npm install monaco-themes
```
#### API
##### Using `
```
##### Using webpack/node
```js
const parseTmTheme = require('monaco-themes').parseTmTheme;
```
#### Directly using themes
##### With webpack
```js
const monaco = /* require monaco */
import('monaco-themes/themes/Monokai.json')
.then(data => {
monaco.editor.defineTheme('monokai', data);
})
```
##### Independently
Download this [repository](https://github.com/brijeshb42/monaco-themes/archive/master.zip) and extract and save `themes` directory in your project.
```js
/* load monaco */
fetch('/themes/Monokai.json')
.then(data => data.json())
.then(data => {
monaco.editor.defineTheme('monokai', data);
monaco.editor.setTheme('monokai');
})
```
---
### Overview (`src/public/third_party/wnumb-1.2.0/README.md`)
wNumb
=====
wNumb - JavaScript Number & Money formatting
# Documentation
Documentation and examples are available on [refreshless.com/wnumb](https://refreshless.com/wnumb/).
# Changelog
### 1.2.0 (*2019-10-29*)
- Changed: License is now MIT
- Added: Prettier code formatter
- Added: Minified version
### 1.1.0 (*2017-02-04*)
- Changed: Renamed `postfix` option to the proper `suffix`. `postfix` is remapped internally for backward compatibility;
# License
Licensed MIT, so free for personal and commercial use.
---
### Overview (`docs/book-isonim/static/img/deep_review/README.md`)
# Generated screenshots
The `*.png` files in this directory are **generated assets**, not hand-made
ones. The pages of the `deep_review` section serve them from
`/assets/img/deep_review/`.
Regenerate them with:
```bash
just capture-deep-review-assets
```
That builds a real two-commit Noir repository, records it twice with the
shipping `ct record`, collects a real dataset with the shipping
`ct review collect`, opens it with the shipping `ct review`, and photographs the
running window — so the images always show the shipping UI rather than a drawing
of it. It needs a built `ct` (`just build-once`) plus `nargo`, `Xvfb`, `xdotool`
and ImageMagick; if any is missing it fails with a named remedy and a non-zero
status rather than leaving stale images in place.
| File | Where it is used | What it shows |
| ------------------- | ---------------- | ------------- |
| `review-window.png` | `deep_review/index.md` (Introduction) | The whole review: the VCS panel, the diff tab, and the Agent Activity panel. |
| `diff-tab.png` | `deep_review/reading.md` | A close-up of the diff tab: the invocation stepper, the loop stepper, the value chips, the expand control. |
| `vcs-panel.png` | `deep_review/reading.md` | A close-up of the VCS panel — the review's navigation surface: the `Review: ` header, the trace-context selector, the totals, the `Unified Diff` toggle and the changed-file row with its coverage badge. Added by DS-1, when the section split gave that panel a walkthrough of its own and the whole-window shot proved too small to read it in. |
All three come from **one** capture run, so the commit id in the header is the
same picture to picture. Regenerating one image without the others is how they
start disagreeing; the recipe writes all three.
## Why they are checked in
The book is published from CI on every push, and the capture needs an X display
and the Noir toolchain. Making the doc build depend on that would make the docs
unpublishable whenever the capture environment is unavailable — so the images
are committed, and the recipe above is how you refresh them.
`tests/test_nav_order_matches_summary.nim` resolves every `/assets/...` link in
the content against this tree, so a page may only reference an image that
actually exists here; deleting one fails the suite rather than 404ing in a
browser. `tests/test_deep_review_section.nim` closes the other half of the
loop: every image the section references must also be one the capture script
above actually writes, so an image cannot survive here as an orphan that
nothing regenerates.
## Why Noir
The recording is a **materialized** trace, the trace kind the db-backend
collector was added for. DeepReview is not an rr-only feature, and the pictures
in the book should not imply that it is. A native/rr capture would additionally
need an rr-capable machine (`perf_event_paranoid <= 1`).
---
### Overview (`docs/book-isonim/static/img/visual_recordings/README.md`)
# Generated screenshots
The `*.png` files in this directory are **generated assets**, not hand-made
ones. `usage_guide/visual_recordings.md` serves them from
`/assets/img/visual_recordings/`.
Regenerate them with:
```bash
just capture-book-assets
```
That records a real GL trace and drives the real visual-replay player, so the
images always show the shipping UI rather than a drawing of it. It needs a
built `ct_gfx_player` (`codetracer-visual-replay`) and a built `ct_cli`
(`codetracer-native-recorder`); if either is missing it fails with a named
remedy and a non-zero status rather than leaving stale images in place.
## Why they are checked in
The book is published from CI on every push to `main`, and the capture needs a
GPU-capable player plus two recorder siblings. Making the doc build depend on
that would make the docs unpublishable whenever the capture environment is
unavailable — so the images are committed, and this recipe is how you refresh
them.
Two things keep that from rotting into "nobody regenerates these":
- `tests/test_nav_order_matches_summary.nim` resolves every `/assets/...`
link against this tree, so a page may only reference an image that actually
exists here. Deleting one fails the suite rather than 404ing in a browser.
- The recipe above is the single documented way to produce them, and it is
wired into the reprobuild graph alongside the book build.
This file replaced a `PLACEHOLDERS.txt` which said the real captures went to
`docs/book/src/generated/visual_recordings/` — the *old* mdBook's output
directory, which is not checked in and which this book never reads. The
placeholder images were shipped as if they were the real thing.
---
### Book Isonim Content Index (`docs/book-isonim/content/index.md`)
---
title: Welcome to CodeTracer Docs
description: CodeTracer is a user-friendly time-traveling debugger designed to support a wide range of programming languages. Install it, record your first trace, and explore the usage guide.
order: 1
---
# Welcome to CodeTracer Docs
:::hero title="Welcome to CodeTracer Docs"
:::button href="/getting_started/installation" variant="primary"
Get Started
:::button href="https://github.com/metacraft-labs/codetracer/issues" variant="secondary"
Support
:::
## Overview
CodeTracer is a user-friendly time-traveling debugger designed to support a wide range of programming languages. It records the execution of a program into a shareable, self-contained trace file, which you can load to move forward and backward through the execution to examine the state of memory and variables at any point.
Use the sidebar to get started with installation, learn how to record and replay traces for your language, and explore the usage guide.
:::video xZsJ55JVqmU title="CodeTracer - Noir Release Demo"
## Start here
:::cards
:::card title="Getting Started" icon="/assets/img/icon__start.svg" href="/getting_started"
Install CodeTracer and record your first trace — Noir, Stylus, WASM, Ruby and Python.
:::card title="Reference" icon="/assets/img/icon__style.svg" href="/reference/build_systems"
Build systems, contributing, troubleshooting, environment variables and building the docs.
:::card title="Usage Guide" icon="/assets/img/icon__components.svg" href="/usage_guide"
The command-line interface, the GUI replay interface, tracepoints and the CodeTracer shell.
:::
## Popular articles
:::cards variant="compact"
:::card title="Introduction" href="/getting_started/introduction"
Getting Started
:::card title="Installation" href="/getting_started/installation"
Getting Started
:::card title="Tracepoints" href="/usage_guide/tracepoints"
Usage Guide
:::card title="Graphical interface" href="/usage_guide/gui"
Usage Guide
:::card title="Command-line interface" href="/usage_guide/cli"
Usage Guide
:::card title="Build systems" href="/reference/build_systems"
Reference
:::
---
### Book Isonim Content Deep Review Index (`docs/book-isonim/content/deep_review/index.md`)
---
title: Introduction
order: 0
---
# DeepReview
A diff is a proposal written in the language of text edits. It tells you,
exactly and completely, what the author changed. It cannot tell you what the
change **did**.
So a reviewer infers. You read `let contribution = (i as Field) * x;`, you build
a model of the loop in your head, you decide the arithmetic comes out right, and
you approve. Only the first of those steps is evidence. The rest is a simulation
you ran in your head against a program you have not seen run — and it is where
review defects come from.
**DeepReview is code review with the recordings attached.** You give CodeTracer
a diff together with one or more recordings of that code actually running, and
it renders the diff with what each changed line did: which lines the run
executed, how many times, and which values they held on each pass through a
loop.
## The questions it lets you answer
Each of these is a question about a change that a diff cannot answer at all, and
that a reviewer otherwise answers by inference, by asking the author, or by
checking the branch out and running it themselves.
**Did this line run at all?** Every line inside a recorded function is shaded
executed or not executed, on the diff itself. A hunk that no recording ever
entered looks different from one that ran on every pass, so "this branch is
never taken in practice" stops being a suspicion and becomes something you can
see.
**What value did it hold — on which pass through the loop?** Executed lines
carry inline value chips, and a loop the recording entered gets an iteration
stepper above it. Stepping it walks the same line through the values it actually
held: `contribution` through `0`, `5`, `10`, `15`, and `sum` through their
running total. You are reading measurements, not re-deriving them.
**Was this function called once, or forty times?** A recorded function carries an
invocation stepper (`‹ main: call 1 / 2 ›`), and the file rows carry coverage
counts. A helper the change touched lightly and a helper it put on a hot path
are different reviews, and the dataset already knows which one you are looking
at.
**Which parts of the change did the runs never exercise?** The same shading, read
the other way. It is the honest version of "is this tested?" — not a promise
that the untouched lines are wrong, but a list of the ones no recording has
anything to say about.
**What did the agent that produced this change actually do?** When a review is
collected inside an agent session, the Agent Activity panel shows that session's
conversation, loaded by reference against the agent backend. The change, the
evidence for it, and the reasoning that produced it are in one window.
## What a review actually is
Two commands and one artefact:
```sh
ct review collect … # build a review dataset from recordings + a diff
ct review # open it
```
A **review dataset** is a self-contained `review.json`: the diff, the source of
each changed file, per-line coverage, the recorded flow (steps and values), and
the call tree. It is produced once and can be opened later, on another machine,
without the original recordings — which is what makes a review something you can
hand to somebody rather than a session you have to be present for.
DeepReview adds **no panel of its own**. A review routes its data into three
panels CodeTracer already has — the VCS panel, the Editor, and the Agent
Activity panel — and leaves your layout otherwise alone. Everything you already
know about the editor still applies inside a review.
## What it does not tell you
Worth stating in the same breath, because a tool that shows execution next to a
diff invites a stronger reading than it has earned:
- **It does not tell you the change is correct.** It tells you what happened in
the runs you recorded. Judging whether that is what *should* have happened is
still the review.
- **It is not a test report.** A review dataset carries no test results at all —
no names, no statuses, no durations — so no surface reports them, and none
shows a zero in their place. `ct test`
is a separate command, and issues **no test certificates** — see
[Not yet available](/deep_review/not_yet_available) for exactly how far it
goes. Nothing in a review claims otherwise.
- **Coverage is observation, never reachability.** A recording can say a line
was not observed. It can never say a line cannot be reached, and a review does
not pretend the two are the same.
- **It reviews the changed code, not everything it calls.** Flow is collected for
the functions the diff touched; a callee gets a call count and coverage, but no
overlay of its own.
The full list of the things that are deliberately not there yet lives on
[Not yet available](/deep_review/not_yet_available). It is worth a read before
your first review: everything on it is real behaviour you will meet, written
down so you do not read it as a bug or, worse, as a measurement.
## Where to go next
- [Collecting a review](/deep_review/collecting) — the two commands, a worked
example end to end, and what happens with native (rr) recordings.
- [Reading a review](/deep_review/reading) — the three panels, the flow overlay
and the steppers.
- [The agent workflow](/deep_review/agent_workflow) — how an agent produces a
review, and how to make it do so reliably.
- [Not yet available](/deep_review/not_yet_available) — the deferrals, named.
---
### Book Isonim Content Getting Started Index (`docs/book-isonim/content/getting_started/index.md`)
---
title: Getting Started
order: 0
---
# Overview
Welcome to the CodeTracer's getting started section. This guide will help you understand how to record and replay programs in each of the supported programming languages.
## Core Concept: Record and Replay
Unlike traditional debuggers that attach to a running process, CodeTracer works by first **recording** your application's execution into a trace file. This trace captures everything that happens during the run.
Once a recording is made, you can **replay** it as many times as you need in the CodeTracer GUI. This allows you to inspect the application's state at any point in time, move forwards and backwards through the execution, and use powerful features like tracepoints without having to run your application again.
Think of it like recording a video of your program's execution that you can then analyze in detail.
## How to Use This Guide
This guide is structured to help you get started quickly and then dive deeper into the features that interest you.
### General-Purpose Languages
These languages are recorded directly via the `ct` CLI (`ct record`, `ct run`):
- [**Python**](./python.md): Steps for installing the recorder, creating a trace, and replaying it with your interpreter.
- [**Ruby**](./ruby.md): Examples and steps how to trace a Ruby program.
- [**JavaScript / TypeScript**](./javascript.md): Instrument and record JS/TS programs.
- [**WASM**](./wasm.md): Examples and steps how to trace a WASM program.
### Zero-Knowledge Languages
- [**Noir**](./noir.md): Examples and steps how to trace a Noir program.
- [**Circom**](./circom.md): Trace witness generation for Circom zero-knowledge circuits.
- [**Miden**](./miden.md): Trace programs on the Miden STARK-based VM.
- [**Leo (Aleo)**](./leo.md): Trace Leo smart contracts on the Aleo network.
### Smart Contract Languages
These languages use dedicated recorder binaries. Record with the language-specific recorder, then view with `ct replay --trace-folder `:
- [**Solidity (EVM)**](./solidity.md): Trace Solidity contracts via debug_traceTransaction.
- [**Stylus**](./stylus.md): Examples and steps how to trace a Stylus contract.
- [**Cairo (StarkNet)**](./cairo.md): Trace Cairo programs and StarkNet contracts.
- [**Aiken (Cardano)**](./aiken.md): Step through UPLC CEK machine execution.
- [**Cadence (Flow)**](./cadence.md): Trace Cadence contracts via interpreter instrumentation.
- [**Move (Sui / Aptos)**](./move.md): Trace Move VM execution on Sui and Aptos.
- [**Solana**](./solana.md): Trace Solana SBF programs with DWARF source mapping.
- [**Sway (FuelVM)**](./sway.md): Trace Sway contracts on FuelVM.
- [**PolkaVM (ink!)**](./polkavm.md): Trace PolkaVM programs and ink! contracts.
- [**Tolk (TON)**](./tolk.md): Trace Tolk/FunC contracts on the TON blockchain.
---
### Book Isonim Content Usage Guide Index (`docs/book-isonim/content/usage_guide/index.md`)
---
title: Usage Guide
order: 0
---
# Overview
Welcome to the CodeTracer Usage Guide. This guide will help you understand how to use CodeTracer effectively to debug your applications.
## Core Concept: Record and Replay
Unlike traditional debuggers that attach to a running process, CodeTracer works by first **recording** your application's execution into a trace file. This trace captures everything that happens during the run.
Once a recording is made, you can **replay** it as many times as you need in the CodeTracer GUI. This allows you to inspect the application's state at any point in time, move forwards and backwards through the execution, and use powerful features like tracepoints without having to run your application again.
Think of it like recording a video of your program's execution that you can then analyze in detail.
## How to Use This Guide
This guide is structured to help you get started quickly and then dive deeper into the features that interest you.
* **To learn about the different ways to interact with CodeTracer**, see the following sections:
* [**Graphical User Interface (GUI)**](./gui.md): Learn how to use the visual interface to replay traces, inspect state, and set tracepoints.
* [**Command-Line Interface (CLI)**](./cli.md): Discover how to record traces and manage trace files from your terminal.
* **To learn about specific features**, see these sections:
* [**Tracepoints**](./tracepoints.md): A deep dive into using tracepoints for advanced debugging scenarios.
* **To review a change with its recordings attached**, see the
[**DeepReview**](/deep_review) section: what each changed line did — which
lines ran, with which values — shown on the diff itself, plus the agent
handoff.
* **For advanced use cases**, explore these topics:
* [**CodeTracer Shell**](./codetracer_shell.md): For integrating CodeTracer with complex or custom build systems.
---
### Book Src Getting Started Overview (`docs/book/src/getting_started/overview.md`)
# Overview
Welcome to the CodeTracer's getting started section. This guide will help you understand how to record and replay programs in each of the supported programming languages.
## Core Concept: Record and Replay
Unlike traditional debuggers that attach to a running process, CodeTracer works by first **recording** your application's execution into a trace file. This trace captures everything that happens during the run.
Once a recording is made, you can **replay** it as many times as you need in the CodeTracer GUI. This allows you to inspect the application's state at any point in time, move forwards and backwards through the execution, and use powerful features like tracepoints without having to run your application again.
Think of it like recording a video of your program's execution that you can then analyze in detail.
## How to Use This Guide
This guide is structured to help you get started quickly and then dive deeper into the features that interest you.
### General-Purpose Languages
These languages are recorded directly via the `ct` CLI (`ct record`, `ct run`):
- [**Python**](./python.md): Steps for installing the recorder, creating a trace, and replaying it with your interpreter.
- [**Ruby**](./ruby.md): Examples and steps how to trace a Ruby program.
- [**JavaScript / TypeScript**](./javascript.md): Instrument and record JS/TS programs.
- [**WASM**](./wasm.md): Examples and steps how to trace a WASM program.
### Zero-Knowledge Languages
- [**Noir**](./noir.md): Examples and steps how to trace a Noir program.
- [**Circom**](./circom.md): Trace witness generation for Circom zero-knowledge circuits.
- [**Miden**](./miden.md): Trace programs on the Miden STARK-based VM.
- [**Leo (Aleo)**](./leo.md): Trace Leo smart contracts on the Aleo network.
### Smart Contract Languages
These languages use dedicated recorder binaries. Record with the language-specific recorder, then view with `ct replay --trace-folder `:
- [**Solidity (EVM)**](./solidity.md): Trace Solidity contracts via debug_traceTransaction.
- [**Stylus**](./stylus.md): Examples and steps how to trace a Stylus contract.
- [**Cairo (StarkNet)**](./cairo.md): Trace Cairo programs and StarkNet contracts.
- [**Aiken (Cardano)**](./aiken.md): Step through UPLC CEK machine execution.
- [**Cadence (Flow)**](./cadence.md): Trace Cadence contracts via interpreter instrumentation.
- [**Move (Sui / Aptos)**](./move.md): Trace Move VM execution on Sui and Aptos.
- [**Solana**](./solana.md): Trace Solana SBF programs with DWARF source mapping.
- [**Sway (FuelVM)**](./sway.md): Trace Sway contracts on FuelVM.
- [**PolkaVM (ink!)**](./polkavm.md): Trace PolkaVM programs and ink! contracts.
- [**Tolk (TON)**](./tolk.md): Trace Tolk/FunC contracts on the TON blockchain.
---
### Book Src Usage Guide Overview (`docs/book/src/usage_guide/overview.md`)
# Overview
Welcome to the CodeTracer Usage Guide. This guide will help you understand how to use CodeTracer effectively to debug your applications.
## Core Concept: Record and Replay
Unlike traditional debuggers that attach to a running process, CodeTracer works by first **recording** your application's execution into a trace file. This trace captures everything that happens during the run.
Once a recording is made, you can **replay** it as many times as you need in the CodeTracer GUI. This allows you to inspect the application's state at any point in time, move forwards and backwards through the execution, and use powerful features like tracepoints without having to run your application again.
Think of it like recording a video of your program's execution that you can then analyze in detail.
## How to Use This Guide
This guide is structured to help you get started quickly and then dive deeper into the features that interest you.
* **To learn about the different ways to interact with CodeTracer**, see the following sections:
* [**Graphical User Interface (GUI)**](./gui.md): Learn how to use the visual interface to replay traces, inspect state, and set tracepoints.
* [**Command-Line Interface (CLI)**](./cli.md): Discover how to record traces and manage trace files from your terminal.
* **To learn about specific features**, see these sections:
* [**Tracepoints**](./tracepoints.md): A deep dive into using tracepoints for advanced debugging scenarios.
* **For advanced use cases**, explore these topics:
* [**CodeTracer Shell**](./codetracer_shell.md): For integrating CodeTracer with complex or custom build systems.
---
### A (`docs/a.md`)
### CI
* We pass `-Ddisable32bit=ON` to rr on CI because I might need to set up additional g++ things (g++-9 ?)
* We build builds with more cached binaries/sources (e.g. rr, nim) for master push-es and "full rebuild" for merge requests, pushing directly to stable is forbidden from GitLab settings
* I should stop changing `/home/codetracer/codetracer/` state on server out of builds
### Local
I sometimes do
`set -gx PATH /home/user/codetracer/src/build-debug $PATH`
to get many codetracer-related binaries in my path: `codetracer`, `tester` and maybe others
# Fix preloading
# Fix history
# Fix the tests to work on nim and c++
# Monitor memory performance: and maybe gc values
# Pseudo: show how sum is working, stepping through complexity with preloading multiline/parallel
# Nim: nilcheck, go through the whole nilcheck with breakpoint/jump function with preloading multiline/parallel
# Nim: nilcheck, fix a bug in ref checking
# Tracepoints:
Tracepoint: `active` and `not active`, `valid` and `invalid`
Trace: `running` and `stopped`.
We can just run one trace at a time.
It always runs all the `active` and `valid` tracepoints.
Basically it works like inserted logging.
We can add a tracepoint to path and line for now.
If we want conditional tracepoints, we use `if`.
We can't add two tracepoints to the same location.
If the path and location exist, it's valid.
# Actions:
`debug`: `loading`, `ready`, `running` and `finished`.
# `interrupt` ?
We can only run moves when it's `ready`. Otherwise, we ignore the moves: we don't buffer them
or something.
Or we buffer them: and we try to show the remaining items: however can we "ignore" those? maybe no, because we can go back easily.
With `loading` we don't buffer yet.
If we get stuck, we should `pause` which should get us into `ready` state.
Errors: we just show the place where we are. we get to `ready`.
`finished` : just have a "run" which should get us to a breakpoint or main if no breakpoint?
Should we hit main if we have a breakpoint?
# event-jump:
We jump to an event:
If we hit an error, we should try to present an ok error message and just leave where we are
If we dont : we get to the new position or we pause
Can we cancel ? Pausing should be basically cancelling and should try to get us back to the previous location
but maybe thats cancel? And pausing should just interrupt? But then resuming is hard
so in this case probably keep a pause/cancel for now.
# calltrace-jump:
We jump to an event:
error: error message, leave where we are
else: new position or pause
# events prestarting:
We just get to those
---
### Async Notes (`docs/async_notes.md`)
async/iterator stepping:
* Current state: stepping literally, special keys, start of a multi-lang support focused on concurrency/async(?)
* Think of C/C++/Rust as well
* Think of both `stdlib`(and our patched version of it) `asyncdispatch` and `chronos`
* What is co-step-in:
It's continuing to next line in the current context(iterator/async function) or entering a call called by us:
even if it happens later and invokes additional calls internally(? if awaited, otherwise not)
* What is co-next:
It's continue to next line in current context(iterator/async function) or if context finished, on callsite if possible(? maybe we jump/finish in other place because of a signal or other)
* What is co-step-in for iterators:
Taken from stdlib of nim: `sequtils.nim`, filter
credit to authors and maintainers of the nim stdlib
```nim
for i in 0 ..< s.len:
if pred(s[i]):
yield s[i]
```
Normal step-in if no yield
Step-in for yield as well? Stepping in caller which should send to actual user
* What is co-next for iterators:
normal next if no yield
continue to next entrance in internal function (probably name_iter) if yield or step-out if finished(but this should happen on last entrance in internal function? Maybe we might not hit that if something happens while yielding)
* What about async:
Expand a bit of code
* Eventually TODO: fix `colonenv_ = .. newObj(..);` location directive from `internalNew` in `system.nim` to local function
* C++: coroutines: c++ co-next seems to work, but still requires sometimes two steps and is probably buggy, co-step-in doesn't
* C `libdill`: TODO
* Rust async_std/tokio: TODO
* Nim `asyncdispatch`/`chronos`: TODO
* Detect context:
* Co-step-in:
* Normal: do a normal step-in and then check context again: if normal/generator: ok, else if wrapper: run to generator context
* Wrapper: run to generator context(find equivalent generator function, make a breakpoint there, disable others, continue to it, delete the breakpoint, enable others): TODO what if we are finished, and we don't hit the generator? Maybe check some fields? After run, check context: if generator: ok, else: error(not expected)
* Async generator/generator: (same as normal) do a normal step-in/co-next and then check context again: if normal/same generator: ok, else if wrapper: if not finished: run to same generator context, else: next to normal/other generator
* Co-next:
* Normal: do a normal next and then check context again: if normal/generator: ok, else if wrapper: run to generator context(maybe we finish a function and return to a wrapper)
* Wrapper: run to generator context, after run check context: if generator: ok, else: error(not expected)
* Async generator/generator: do a normal next, check context again: if not in the same generator: we probably returned(yielded) so if not finished: run to same generator context, else: next to normal/other generator
else: ok
OK, for C++ it's different: we might keep a separate logic for now
libdill:
* normal-or-worker context, prologue, wrapper, potential-switch context
step-in or next: beginning, otherwise same
normal-or-worker: step-in/next and check context again: then for normal-or-worker: ok, otherwise do the same as others
prologue: finish, step-in again
wrapper: continue to potential-switch context and then next until normal-or-worker
potential-switch: next until normal-or-worker
rust:
async-std:
we hit a rr(?) problem when reverse-next in async function: maybe because of smart next handling/custom gdb rust commands?
otherwise step and use `{{closure}}` version as equivalent to generator context
tokio:
similar rr(?) problem when reverse-next in async function
otherwise step and use `{{closure}}` similarly to async-std
---
### Build (`docs/build.md`)
notes about build and dependencies
### monaco-editor notes
* TODO: list
* Temporary : copy all to `editor.main.js`
IMPORTANT: change that to normal build or a simpler auto operation ?
### golden-layout notes
* For now build in a copied repo of it
* Copy its `dist/umd/golden-layout.js` file to `src/public/third_party/goldenlayout.js`
* Comment out the `// else if(typeof define === 'function' && define.amd) .. // define(["jquery"], factory);`
TODO: license
---
### Call Interleave (`docs/call_interleave.md`)
call interleave
==================
An idea for a possible feature:
We can interleave event and other info with callstack
and calltrace:
```markdown
---
|___ a1()
|
|
PGP Key signatures:
[](https://downloads.codetracer.com/CodeTracer-latest-arm64.dmg.asc)
[](https://downloads.codetracer.com/CodeTracer-latest-amd64.AppImage.asc)
[](https://downloads.codetracer.com/CodeTracer.pub.asc)
> [!CAUTION]
> When launched for the first time, macOS users will see the error message "CodeTracer is damaged and can't be opened".
> To resolve this problem, please execute the command `user $ xattr -c