File: crates/obscura/README.md
obscura
Embeddable Rust API for the Obscura
headless browser. Drive a real V8 + DOM browser (Browser, Page, Element,CookieStore) directly from Rust, with no separate process or CDP round-trips.
Install
This crate is not published to crates.io, so depend on it via git. Building it
compiles Obscura from source, including its embedded V8 (deno_core), so the
first build is large and slow.
[dependencies]
obscura = { git = "https://github.com/h4ckf0r0day/obscura", features = ["api"] }
tokio = { version = "1", features = ["rt", "macros"] }
anyhow = "1"Usage
use obscura::Browser;
use std::time::Duration;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let browser = Browser::builder()
.stealth(true)
.storage_dir("/tmp/cookies")
.build()?;
let mut page = browser.new_page().await?;
page.goto("https://example.com").await?;
let el = page.wait_for_selector("a", Duration::from_secs(5)).await?;
println!("{} -> {:?}", el.text(), el.attribute("href"));
Ok(())
}See examples/basic.rs for a runnable version (cargo run --example basic).
File: docs/Adding-a-CDP-method-or-Web-API.md
Two recipes for the two most common extensions: a new CDP method, and a new JS Web API.
Adding a CDP method
Worked example: MyDomain.doThing that takes { name } and returns { ok }.
1. Add the handler
Create or edit a file under crates/obscura-cdp/src/domains/:
// crates/obscura-cdp/src/domains/my_domain.rs
use serde_json::{json, Value};
use crate::dispatch::CdpContext;
pub async fn do_thing(
params: &Value,
_ctx: &mut CdpContext,
_session_id: &Option<String>,
) -> Result<Value, String> {
let name = params.get("name")
.and_then(|v| v.as_str())
.ok_or("missing name")?;
// do the work
Ok(json!({ "ok": true, "name": name }))
}2. Register in the dispatcher
In crates/obscura-cdp/src/dispatch.rs, add a match arm:
"MyDomain.doThing" => domains::my_domain::do_thing(&req.params, ctx, &req.session_id).await,3. Test it
crates/obscura-cdp/tests/cdp_my_domain.rs:
use obscura_cdp::dispatch::{dispatch, CdpContext};
use obscura_cdp::types::CdpRequest;
use serde_json::json;
#[tokio::test(flavor = "current_thread")]
async fn my_domain_do_thing_returns_ok() {
let mut ctx = CdpContext::new();
let resp = dispatch(&CdpRequest {
id: 1,
method: "MyDomain.doThing".into(),
params: json!({ "name": "test" }),
session_id: None,
}, &mut ctx).await;
assert!(resp.error.is_none());
assert_eq!(resp.result.unwrap()["ok"], true);
}Run:
cargo nextest run --release --features render -p obscura-cdp my_domainAdding a Web API
Worked example: crypto.subtle.digest, real implementation backed by a Rust hash op.
1. Add the Rust op
In crates/obscura-js/src/ops.rs:
#[op2]
#[buffer]
fn op_subtle_digest(#[string] algorithm: &str, #[buffer] data: &[u8]) -> Vec<u8> {
use sha1::Digest as _;
match algorithm.to_ascii_uppercase().as_str() {
"SHA-1" => sha1::Sha1::digest(data).to_vec(),
"SHA-256" => sha2::Sha256::digest(data).to_vec(),
"SHA-384" => sha2::Sha384::digest(data).to_vec(),
"SHA-512" => sha2::Sha512::digest(data).to_vec(),
_ => sha2::Sha256::digest(data).to_vec(),
}
}2. Register the op
In the same file, build_extension():
ops: std::borrow::Cow::Owned(vec![
op_dom(),
op_console_msg(),
// ...
op_subtle_digest(),
]),3. Add the JS shim
In crates/obscura-js/js/bootstrap.js:
globalThis.crypto = globalThis.crypto || {};
globalThis.crypto.subtle = globalThis.crypto.subtle || {};
globalThis.crypto.subtle.digest = function digest(algorithm, data) {
const algName = typeof algorithm === 'string' ? algorithm : algorithm.name;
const bytes = data instanceof ArrayBuffer
? new Uint8Array(data)
: new Uint8Array(data.buffer, data.byteOffset, data.byteLength);
const out = Deno.core.ops.op_subtle_digest(algName, bytes);
return Promise.resolve(out.buffer);
};4. Add a dependency if needed
crates/obscura-js/Cargo.toml:
sha1 = "0.10"
sha2 = "0.10"5. Smoke test
cargo build --release --features render
./target/release/obscura fetch https://example.com --eval "
crypto.subtle.digest('SHA-256', new TextEncoder().encode('hi'))
.then(buf => Array.from(new Uint8Array(buf)).map(b => b.toString(16).padStart(2, '0')).join(''))
"Tips
- Keep the JS shim thin. All side effects go through ops.
- Use
Promise.resolveto keep async-shaped APIs callable from sync ops. - Match the spec: Web API names and shapes are checked by Puppeteer / Playwright wrappers.
- DOM mutations go through
op_dom, not new ops. - For events that need to fire across handlers, use the existing
_makeListenerBoxhelper inbootstrap.js.
Worked examples in the tree
- CDP method with intercept:
crates/obscura-cdp/src/domains/page.rsdo_navigate. - Web API with op + JS shim:
crypto.subtle.digest(above). - Web API in pure JS (no op):
DOMParserinbootstrap.js. - Web API with async event firing:
WebSocket,IntersectionObserverinbootstrap.js.
File: docs/Architecture-overview.md
Obscura is a workspace of nine crates.
obscura-cli CLI entry point. fetch, serve, scrape, mcp.
obscura-cdp Chrome DevTools Protocol server. WebSocket, dispatch, domain handlers.
obscura-browser Page type, navigation, lifecycle events.
obscura-js V8 runtime via deno_core. bootstrap.js + Rust ops.
obscura-dom DOM tree implementation.
obscura-net HTTP client, stealth client, cookie jar, robots cache, tracker blocklist.
obscura-mcp Model Context Protocol server.
obscura-render CSS cascade, retained layout, text shaping, and CPU paint.
obscura Embeddable Rust library API (Browser, Page, Element, CookieStore).Request flow
A Page.navigate from a CDP client:
CDP client (Puppeteer)
โ WebSocket frame
โผ
obscura-cdp/server.rs accept, route by sessionId
โ
โผ
obscura-cdp/dispatch.rs method router, acquires v8_lock
โ
โผ
obscura-cdp/domains/page.rs Page.navigate handler
โ
โผ
obscura-browser/page.rs navigate_with_wait
โ
โโโโบ obscura-net/client.rs HTTP fetch
โ
โโโโบ obscura-dom/tree.rs parse HTML into the tree
โ
โโโโบ obscura-js/runtime.rs run inline scripts
โ
โโโโบ bootstrap.js + ops.rs DOM bindingsThe dispatcher emits CDP events (Network.requestWillBeSent, Page.frameNavigated, Page.lifecycleEvent) back to the client through the same WebSocket.
Rendering flow
obscura-render consumes the shared DOM and computed style state. Taffy
provides the flex/grid foundation; Obscura adds browser formatting behavior,
text shaping, intrinsic replaced-element sizing, retained geometry, scrolling,
and CPU-backed paint. obscura-js exposes renderer-owned geometry to DOM APIs,obscura-browser prepares resources and owns capture, and obscura-cdp maps
screenshots, screencast frames, and raster PDF output onto CDP.
Layout is retained between captures and invalidated by relevant DOM, style,
viewport, scroll, animation, font, and resource changes. The same geometry
therefore drives browser APIs and paint instead of maintaining separate
measurement and screenshot models.
Page-owned layout and paint never fetch resources synchronously. Missing
resources load through the page transport, with URL blocking checked both
for navigation/screenshot warmup and for later renderer misses. The DOM-onlyPage::screenshot fallback (for example after Page::suspend_js) also performs
no network requests: external images and fonts absent from its cache remain
unavailable. Resume the page and prepare its resources before capture when
those assets are required. Standalone obscura-render callers retain their
existing synchronous-loader behavior.
The relevant paths are Page::screenshot_with_animation_sample andPage::render_resource_candidates in crates/obscura-browser/src/page.rs,ObscuraJsRuntime::take_render_resource_requests incrates/obscura-js/src/runtime.rs, and RenderResourceCache incrates/obscura-render/src/paint.rs.
Single V8 isolate
All pages in a process share one V8 isolate. The isolate is single-threaded by design.
obscura_js::v8_lock::global() is a tokio::sync::Mutex that serializes V8 work. A handler that wants to run JS must acquire the lock first:
let _guard = obscura_js::v8_lock::global().lock().await;
page.evaluate(expr).awaitThe dispatcher routes long-running operations (navigation, eval) through process_with_interception in server.rs, which spawns the work onto the tokio LocalSet and releases the dispatcher to keep handling other CDP messages.
This is why Target.createTarget from many concurrent clients works: each newPage returns immediately while the actual navigation runs in a spawned task.
Child frames
iframe srcdoc uses a native child realm and document, not a copy in the
parent DOM. Its URL is about:srcdoc; relative URLs inherit the creator's
document base, and unsandboxed Web Storage uses the creator's origin.srcdoc takes precedence over src, including an empty value. Parser input,
connected innerHTML, property/attribute changes and removal use the same
loading path. Replaced or detached documents are retired before stale loads
can run. The iframe's load event follows classic author scripts.
Navigation runs as a posted task, after microtasks. Parsing uses srcdoc
standards mode without requiring a doctype and honors script-disablednoscript content.
Sandboxed srcdoc enforces opaque document access and script disabling,
including inherited restrictions on nested frames. This does not implement
every HTML sandbox policy. Frame module scripts remain unsupported.
Child-frame CDP evaluation routing and iframe content compositing remain
separate compatibility gaps; loading a document does not establish support
for Playwright frame locators or iframe screenshot fidelity.
Robustness
One page cannot hang or crash the process. obscura-js/runtime.rs provides a V8 termination watchdog (arm_watchdog, run_event_loop_bounded) that terminates the isolate from a separate thread when synchronous work overruns a budget, because tokio::time::timeout cannot preempt synchronous V8. It bounds the post-load settle, the navigation event-loop pumps, and --eval. The complete script phase is bounded by OBSCURA_SCRIPT_DEADLINE_MS; enhancement modules have a shorter per-module graph-loading/evaluation budget controlled by OBSCURA_MODULE_BUDGET_MS, while modules mounting an empty SPA shell receive the full script deadline. obscura-js/cdp_watchdog.rs is a single shared watchdog the dispatcher arms around every CDP command, so a runaway page cannot hold the V8 lock and wedge other sessions (tunable via OBSCURA_CDP_COMMAND_TIMEOUT_MS). op_dom is wrapped in catch_unwind so a DOM-op panic degrades to a null result instead of aborting the process through V8's FFI frame, and obscura-dom/tree.rs rejects cyclic reparenting that would make tree walks loop forever. Scripted fetch()/XHR and module network requests are timeout-bounded (OBSCURA_FETCH_TIMEOUT_MS), and the one-shot fetch CLI has a process-level hard deadline as a final backstop.
JS bridge
obscura-js/js/bootstrap.js provides the browser globals: document, window, navigator, location, observers, fetch, indexedDB, etc.
obscura-js/src/ops.rs registers Rust ops that the bootstrap calls into:
Deno.core.ops.op_dom('insert_before', parentNid, refNid, newNid);Adding a Web API usually means:
- JS shim in
bootstrap.jsthat exposes the API surface. - Rust op in
ops.rsthat performs the side effect (DOM mutation, fetch, crypto). - Register the op in
build_extension().
Worked example: Adding a CDP method or Web API.
Classic Web Workers
The JavaScript shim executes each classic Worker source once and retains its
message handlers and lexical state. Bare onmessage assignments target the
worker scope, and messages posted before the source loads are queued until
initialization finishes. Terminating a worker discards pending messages.
Workers remain emulated within the page runtime, not separate V8 isolates or
OS threads. This is not a complete WorkerGlobalScope implementation.
CDP session model
Each CDP client connection gets attached to one or more targets.
Session IDs are "{targetId}-session". The dispatcher routes by sessionId in the incoming frame to the right Page.
Targets are created by Target.createTarget. Closing the WebSocket detaches all sessions but leaves the pages running.
Lifecycle
Lifecycle events are emitted by obscura-browser/lifecycle.rs as the page transitions:
init โ commit โ domcontentloaded โ load โ networkidle2 โ networkidle0waitUntil on Page.navigate blocks until the requested level is reached. The Puppeteer / Playwright goto resolves on the matching Page.lifecycleEvent client-side.
Storage
--storage-dir persists cookies to cookies.json and localStorage to localStorage/<origin>-<digest>.json: read on process start, written after each navigation that changed something and on graceful shutdown. sessionStorage is origin-keyed in memory only, per tab.
Stealth
--stealth swaps the default reqwest client for obscura-net/wreq_client.rs, which presents a real browser's TLS ClientHello, ALPN, and cipher order (a consistent Chrome fingerprint, not a randomized one) so the TLS layer matches the User-Agent and JS surfaces. It also applies the bundled tracker blocklist before any request leaves the process. Scripted fetch()/XHR go through the same stealth client, so subresource requests carry the same fingerprint as the navigation. --stealth is a global CLI flag that applies to fetch, serve, scrape, and mcp.
Workspace conventions
- One crate per layer. Cross-crate calls go through the layer above, not sideways.
- All async is
tokiowith aLocalSetbecause V8 is!Send. - All DOM ops go through
op_domto keep the JS/Rust boundary narrow.
File: docs/Build-from-source.md
Requirements
- Rust 1.75+ (rustup.rs)
- C compiler (gcc or clang)
- ~5 GB free disk space (V8 compiles from source on first build)
Build time depends on the machine and cache state. Cargo reuses unchanged
dependencies, but a release source edit can still require substantial compilation
and linking. For the scoped incremental runtime workflow, see
Fast edit/test loop.
Build
git clone https://github.com/h4ckf0r0day/obscura.git
cd obscura
cargo build --release -p obscura-cli --bins --features renderBinary is at ./target/release/obscura.
This produces the release binary with geometry, screenshots, screencasting,
and PDF export.
Rendering and stealth
cargo build --release -p obscura-cli --bins --features render,stealthThis is the complete rendering build with the stealth wreq/BoringSSL transport,
TLS fingerprint randomization, browser-identity protections, and tracker
blocklist. See Configure stealth and proxies.
Without rendering
cargo build --release -p obscura-cli --bins --no-default-features
cargo build --release -p obscura-cli --bins --no-default-features --features stealthThe second command keeps stealth while excluding layout, screenshots,
screencasting, and PDF export.
The stealth feature builds BoringSSL and generates Rust bindings. In addition
to the default requirements, install CMake, Clang, and the libclang/LLVM
development libraries. On Ubuntu/Debian:
sudo apt-get install build-essential cmake clang libclang-dev llvm-devOn macOS, install the Xcode Command Line Tools and CMake. On Windows, install
the Visual Studio C++ Build Tools, CMake, and LLVM/Clang. Ensure the directory
containing libclang is available through LIBCLANG_PATH if bindgen cannot
locate it automatically.
On macOS 26 with the standalone Command Line Tools, Apple Clang may not find
libc++ while compiling BoringSSL. Use the active SDK for that build:
SDK_PATH="$(xcrun --show-sdk-path)"
SDKROOT="$SDK_PATH" CXXFLAGS="-isystem $SDK_PATH/usr/include/c++/v1" \
cargo build --release -p obscura-cli --bins --features render,stealthOpenSSL on older systems
If the build fails on the vendored OpenSSL with an AVX-512 assembler error (common on older VPS hosts):
OPENSSL_NO_VENDOR=1 cargo build --release -p obscura-cli --bins --features renderUses the system OpenSSL instead.
Run from the build
./target/release/obscura --version
./target/release/obscura fetch https://example.com --eval "document.title"Install system-wide:
cargo install --path crates/obscura-cli --features renderTests
cargo nextest run --release --features render --no-fail-fastIntegration suite:
python3 tests/test_all.pyUse cargo nextest, not cargo test: runtime tests require process isolation
because the engine owns a single V8 isolate per process.
File: docs/CLI-reference.md
`obscura`
Most top-level flags apply to every subcommand. --storage-dir is the
exception: it is declared on serve and fetch, and the top-level copy is read
only by mcp, which exports it to OBSCURA_STORAGE_DIR. Pass it after the
subcommand to use it with serve or fetch.
-v, --verbose Enable info logging
-p, --port <PORT> CDP port (default 9222)
--proxy <URL> HTTP or SOCKS5 proxy
--stealth Consistent browser fingerprint + tracker blocking
--obey-robots Respect robots.txt
--user-agent <UA> Override the User-Agent
--storage-dir <DIR> Persistent cookies and localStorage
--allow-private-network Permit loopback / RFC1918 / link-local
--v8-flags <FLAGS> Raw V8 flags, applied at startup
-h, --help Help
-V, --version Version`obscura fetch `
Load a URL and print its content or an evaluated expression.
--dump <FORMAT> html | text | links | markdown | original | assets | cookies
(default html)
--selector <CSS> Narrow output to a CSS selector
--wait <SECONDS> Fixed post-load delay; omitted uses adaptive settle (5s cap)
--timeout <SECONDS> Navigation timeout (default 30)
--wait-until <LEVEL> domcontentloaded | load | networkidle2 | networkidle0
(default load)
--user-agent <UA> Override the User-Agent
--proxy <URL> HTTP or SOCKS5 proxy
--stealth Consistent browser fingerprint + tracker blocking (global)
-e, --eval <JS> Evaluate JS, print the result as JSON
-o, --output <FILE> Write to a file instead of stdout
-s, --screenshot <FILE> Capture the settled page as PNG (single URL)
-q, --quiet Suppress info logging
-v, --verbose Enable verbose logging--screenshot requires a render-enabled build. It uses a 1280ร720 viewport by
default and may be combined with --eval; the expression runs before capture,
which is useful for scrolling or preparing page state. It is not available in--file batch mode.
When --wait is omitted, Obscura drives timers and async work until the page
becomes quiescent, with a five-second ceiling. Supplying --wait N instead
requests a fixed N-second delay. --timeout separately bounds navigation.
--dump values:
| Value | Output |
|---|---|
html |
Rendered HTML (default) |
text |
Plain text |
markdown |
Markdown conversion |
links |
Every <a href>, one URL per line |
assets |
Every external resource, one JSON object per line (DOM assets plus fetch()/XHR requests) |
original |
Raw HTTP response body (binary-safe, bypasses the engine) |
cookies |
All cookies in the jar as a JSON array, including HttpOnly cookies invisible to document.cookie |
`obscura serve`
Run the CDP server. Puppeteer and Playwright connect over WebSocket.
-p, --port <PORT> CDP port (default 9222)
--host <HOST> Bind host (default 127.0.0.1)
--proxy <URL> HTTP or SOCKS5 proxy
--user-agent <UA> Override the User-Agent
--stealth Consistent browser fingerprint + tracker blocking (global)
--workers <N> Worker processes (default 1)
--worker-max-connections <N> Recycle a worker after N CDP connections (--workers > 1)
--worker-max-rss <MB> Recycle a worker above this resident size (Linux)
--worker-max-age <SECS> Recycle a worker after this long
--worker-drain-timeout <SECS> Let a replaced worker finish live connections (default 120)
--font-dir <DIR> Recursively load fonts once per worker (repeatable; render build)
--allow-file-access Permit CDP clients to navigate to file:// URLs
--storage-dir <DIR> Persistent cookies and localStorage
--allow-private-network Permit loopback / RFC1918 / link-local
-q, --quiet Suppress info logging
-v, --verbose Enable info loggingDefault endpoint is ws://127.0.0.1:9222.
`obscura scrape [URLS]...`
Run a JS expression across many URLs in parallel.
-e, --eval <JS> JS to run on each page
--concurrency <N> Parallel pages (default 10)
--format <FORMAT> Output format (default json)
--timeout <SECONDS> Per-URL timeout (default 60)
--proxy <URL> HTTP or SOCKS5 proxy
--stealth Consistent browser fingerprint + tracker blocking (global)
--allow-private-network Permit loopback / RFC1918 / link-local
-q, --quiet Suppress info logging
-v, --verbose Enable verbose logging--stealth, --proxy, and --allow-private-network are global flags: they work before or after any subcommand, so each worker in a scrape run inherits stealth too.
Read URLs from stdin with -:
cat urls.txt | obscura scrape - --eval "document.title" --concurrency 20Requires obscura-worker next to obscura in PATH.
`obscura mcp`
Run obscura as an MCP server.
--http HTTP transport instead of stdio
--font-dir <DIR> Load additional fonts before rendering (repeatable; render build)
--host <HOST> HTTP bind host (default 127.0.0.1)
--port <PORT> HTTP port (default 3000)
--proxy <URL> HTTP or SOCKS5 proxy
--user-agent <UA> Override the User-Agent
--stealth Consistent browser fingerprint + tracker blocking (global)
--allow-private-network Permit loopback / RFC1918 / link-local
-v, --verbose Enable info logging--host only applies with --http. The default 127.0.0.1 keeps the server loopback-only. A non-loopback bind requires OBSCURA_MCP_TOKEN (at least 32 bytes); use OBSCURA_MCP_ALLOWED_ORIGINS as well when a browser-based client needs access.
Default transport is stdio. See Use the MCP server.
Render-enabled builds add browser_screenshot and browser_pdf to the MCP
tool list. Streaming screencasts are available through CDP rather than MCP.
Worker navigation responses
obscura-worker accepts newline-delimited JSON commands on stdin and emits one
JSON response per command on stdout. For example:
{"cmd":"navigate","url":"https://example.com/"}A completed navigation includes the final document's HTTP status alongside the
existing title and URL:
{"ok":true,"result":{"title":"Example Domain","url":"https://example.com/","status":200}}status is the main document's final HTTP response code after redirects, not a
script, image, or child frame's response. HTTP errors such as 404 and 503 still
return ok: true: the server responded and the error document can be inspected
with dump_html, dump_text, or evaluate. Navigation without an HTTP response,
such as about:blank, a data: URL, or a tracker blocked before the request,
returns status: null. A transport or
navigation failure retains the existing {"ok":false,"error":"..."} envelope
without a result object.
The status field is additive. Navigation timing and the other worker commands
are unchanged.
Worker navigation readiness
obscura-worker accepts newline-delimited JSON commands on stdin and writes
one response per command to stdout. Select a readiness mode per navigation:
{"cmd":"navigate","url":"https://example.com","waitUntil":"domcontentloaded"}waitUntil accepts load, domcontentloaded, networkidle0, or networkidle2.
Omitting it preserves the existing load default. The option applies only to
that request. Invalid values return an ok: false response without navigating;
the worker remains available for subsequent commands.
These modes use the existing Page::navigate_with_wait behavior. Parser scripts
run in every mode. Network-idle modes additionally drive the event loop until
active requests remain at or below zero or two, respectively, for 500 ms, with
the engine's existing five-second idle deadline. domcontentloaded uses the
engine's DOM readiness boundary; it does not change script execution or resource
warmup semantics. Successful responses retain their existing title, url, and status
fields.
File: docs/Configure-stealth-and-proxies.md
Stealth mode
obscura fetch https://example.com --stealth
obscura serve --stealth
obscura scrape url1 url2 --stealth
obscura mcp --stealth--stealth is a global flag, so it works before or after the subcommand and applies to fetch, serve, scrape, and mcp. In a scrape run each worker inherits it.
What --stealth changes:
- Uses the wreq HTTP client with browser-matching TLS fingerprints (ClientHello, ALPN, cipher order).
- Loads a tracker blocklist that drops requests to known analytics and fingerprinting endpoints.
- Bundles webpki roots instead of relying on the system store.
Requires a build that includes the stealth feature. Use a -stealth archive
with rendering or a -no-render-stealth archive without it. To build the
rendering variant yourself:
cargo build --release -p obscura-cli --bins --features render,stealthOmit rendering with cargo build --release -p obscura-cli --bins --no-default-features --features stealth.
What stealth handles
- Basic bot detection that checks TLS fingerprint or User-Agent.
- Sites that rely on third-party analytics being reachable.
What stealth does not handle
- Cloudflare interactive challenges.
- Datadome and Akamai bot manager active challenges.
- CAPTCHAs.
- IP-based rate limiting (use proxies).
- Passive TCP/IP stack fingerprinting. Classifiers that inspect the raw SYN
packet (tcp_options order, window size, window scaling, ip_id) can distinguish
Linux from macOS even when the TLS and HTTP layers are identical, because those
fields are set by the OS kernel, not by Obscura. The same binary on Linux
(tcp_options: "M1460,S,T,N,W10", window 64240) and macOS
(tcp_options: "M1460,N,W6,N,N,T,S,E,E", window 65535) produces a different
passive fingerprint. Obscura operates in userspace and cannot override these
kernel defaults. If TCP/IP passive detection is a concern, run Obscura inside
a container or VM whose kernel matches the target fingerprint, or route
connections through a proxy whose TCP stack presents the desired profile.
Proxies
HTTP proxy:
obscura fetch https://example.com --proxy http://proxy.example.com:8080
obscura serve --proxy http://proxy.example.com:8080With auth:
obscura fetch https://example.com --proxy http://user:[email protected]:8080SOCKS5:
obscura fetch https://example.com --proxy socks5://proxy.example.com:1080Custom User-Agent
obscura fetch https://example.com --user-agent "Mozilla/5.0 (...) ..."
obscura serve --user-agent "Mozilla/5.0 (...) ..."Default UA matches a recent Chrome on the build platform.
Browser profile, timezone, and geolocation
The engine presents one of a built-in pool of realistic browser profiles (a mix of Windows and macOS, recent Chrome versions). Each profile keeps navigator.platform, navigator.userAgentData (platform and platform version), and the UA string internally consistent, so the surfaces a site fingerprints agree with each other. There is no GPU renderer among them: canvas.getContext('webgl') returns null, so a page cannot read a renderer string at all.
A single stable profile is used by default. One IP cycling through different identities is itself a signal, so rotation is opt-in:
OBSCURA_PROFILE=2 obscura serve # pin a specific profile by index
OBSCURA_ROTATE_PROFILE=1 obscura serve # random profile per browser contextTimezone is driven by the process zone so Date (getTimezoneOffset, toString) and Intl.DateTimeFormat report the same region. Default is Europe/Berlin; set it to match the exit IP:
OBSCURA_TIMEZONE=America/New_York obscura servenavigator.geolocation reports configurable coordinates. Set them as lat,lon and keep them consistent with the timezone and proxy region:
OBSCURA_GEOLOCATION="40.7128,-74.0060" obscura serveKeep these aligned. A rotated or mismatched profile carries no matching TLS or timezone fingerprint, so when you pin a proxy region or TLS fingerprint, leave rotation off and set the timezone and geolocation to the same region. See Environment variables for the full list.
Combine
obscura serve \
--stealth \
--proxy http://user:[email protected]:8080 \
--user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 ..."File: docs/Connect-Puppeteer-or-Playwright.md
Obscura speaks the Chrome DevTools Protocol over WebSocket. Puppeteer and
Playwright can connect to its CDP endpoint for the supported workflows below.
Start the server
obscura serve --port 9222obscura listening on ws://127.0.0.1:9222Puppeteer
npm install puppeteer-coreconst puppeteer = require('puppeteer-core');
const browser = await puppeteer.connect({
browserWSEndpoint: 'ws://127.0.0.1:9222',
});
const page = await browser.newPage();
await page.goto('https://example.com');
console.log(await page.title()); // "Example Domain"
await browser.disconnect();Use puppeteer-core, not puppeteer. The puppeteer package bundles a Chrome download.
Playwright
npm install playwrightconst { chromium } = require('playwright');
const browser = await chromium.connectOverCDP('ws://127.0.0.1:9222');
const context = browser.contexts()[0] || await browser.newContext();
const page = await context.newPage();
await page.goto('https://example.com');
console.log(await page.title());
await browser.close();Use connectOverCDP, not connect. Playwright's connect speaks Playwright's own protocol, which obscura does not implement.
`waitUntil`
Default is domcontentloaded. For full subresource load:
await page.goto('https://example.com', { waitUntil: 'load' });| Value | Returns when |
|---|---|
domcontentloaded |
HTML parsed, scripts ran (default) |
load |
All subresources finished |
networkidle2 |
โค2 network connections active for 500ms |
networkidle0 |
0 network connections active for 500ms |
Supported
page.goto,page.reload,page.goBack,page.goForwardpage.evaluate,page.evaluateHandlepage.click,page.type,page.fill,page.focuspage.waitForSelector,page.waitForFunction,page.waitForNavigationpage.cookies,page.setCookie,context.cookiespage.setRequestInterception, block / modifypage.exposeFunctionpage.content,page.title,page.urlpage.screenshotfor viewport, clipped, and full-page capturepage.pdffor raster-backed print output- raw CDP
Page.startScreencastwith frame acknowledgements (page.createCDPSession()
in Puppeteer;context.newCDPSession(page)in Playwright)
DOM-agent frameworks such as browser-use also connect: obscura implements DOMSnapshot.captureSnapshot and Target.targetInfoChanged for perception, and DOM.focus so a focused field receives Input.dispatchKeyEvent keystrokes.
Capture example
await page.setViewport({ width: 1440, height: 1000 });
await page.screenshot({ path: 'viewport.png' });
await page.screenshot({ path: 'full-page.png', fullPage: true });
await page.pdf({ path: 'page.pdf', format: 'A4', printBackground: true });Rendering is included in official binaries and requires --features render
for source builds. The client-specific guides cover scrolling, raw CDP
screencasting, and current output limits.
Raw CDP node queries
DOM.getAttributes, DOM.requestNode, DOM.requestChildNodes, andDOM.getNodeForLocation support the top-level light DOM. Request the document
with DOM.getDocument before publishing frontend node IDs. Ancestors and
requested children arrive through session-scoped DOM.setChildNodes events;
navigation clears the node cache and emits DOM.documentUpdated.
Live mutation events are not implemented yet. Request the document again to
publish newly inserted children of an already-published parent. requestChildNodes acceptspierce: true on plain light-DOM subtrees, but rejects requests that would
require frame or shadow-tree traversal. Detached-node publication andignorePointerEventsNone: true hit testing are not supported. The remaining work is tracked in
#1291.
Element visibility queries
Render builds implement element.checkVisibility() using the rendered flat tree.
Disconnected elements, display: none subtrees, display: contents elements,
closed-details content, and unassigned light-DOM children have no visible box.
Zero-sized boxes are not automatically rejected. Queries use the element's owning
document, including iframe documents.
The opacityProperty / checkOpacity and visibilityProperty /checkVisibilityCSS options enable the corresponding CSS checks. Without
rendering, only inline styles are checked. CSS content-visibility: hidden andcontent-visibility: auto skipping are not implemented, including thecontentVisibilityAuto option.
Listbox option/optgroup boxes are also not rendered, so their visibility queries
currently return false even when Chromium returns true.
Native table rows and sections can be revealed with explicit display: table-row,table-row-group, table-header-group, and table-footer-group declarations.
CSS-created tables with explicit rows/groups share column tracks and expose row
and group geometry. Full anonymous-table fixup for mixed content is not implemented;
unsupported CSS table structures retain ordinary box layout rather than dropping
their non-table children.
Current limits
- Pages share one V8 isolate. CPU-bound JavaScript on one page can delay others.
- PDF output is raster-backed; text is not selectable and tagged PDF,
headers/footers, outlines, and full CSS paged media are not implemented. - Service workers, native media playback, some Web APIs, and long-tail CSS or
compositor effects are still incomplete relative to Chromium.
File: docs/Environment-variables.md
Runtime
`OBSCURA_ALLOW_PRIVATE_NETWORK`
Allow fetches to loopback (127.0.0.0/8), RFC1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), and link-local (169.254.0.0/16, including the 169.254.169.254 cloud-metadata endpoint) addresses. The deny-set also covers the unspecified address (0.0.0.0 / ::), IPv6 unique-local (fc00::/7), and any IPv4-mapped form of the above. Off by default to block SSRF.
The guard validates at DNS-resolution time as well as on literal hosts, so a public hostname that resolves to a forbidden address is rejected at connect time (DNS-rebinding safe), not just hosts written as raw IPs.
Truthy values: 1, true, yes, on.
OBSCURA_ALLOW_PRIVATE_NETWORK=1 obscura fetch http://localhost:8080Per-process equivalent: --allow-private-network on any subcommand.
`OBSCURA_NAV_TIMEOUT_MS`
Hard ceiling on a single navigation. Default 30000 (30 seconds). Applies to Page.navigate and the CLI fetch command.
OBSCURA_NAV_TIMEOUT_MS=60000 obscura serve`OBSCURA_NAV_CHAIN_LIMIT`
How many documents a navigation chain may load, the first navigation included. Default 10, which allows the requested document and nine navigations the page itself triggers via location assignments or form submissions. Raise the value for an endpoint that chains longer for good reasons, such as an SSO handover across several providers. The low default is what stops a page that resets location on every load.
A zero is raised to 1. This loads the requested document. If the page wants to chain further afterwards, the call reports an error, as at any other limit. A value the engine does not read as a number is replaced by the default. This also applies to a negative value and to a value with a trailing space.
The time budget is not tied to this limit. A longer chain usually also needs a higher OBSCURA_NAV_TIMEOUT_MS, because its default of 30 seconds applies to the whole chain and not to the individual document.
OBSCURA_NAV_CHAIN_LIMIT=20 obscura serve`OBSCURA_SCRIPT_DEADLINE_MS`
Soft deadline for the complete page script-execution phase, including classic scripts and ES modules. Default 30000 (30 seconds). Raise it for a heavy SPA whose initial module is responsible for mounting an otherwise empty document. The engine also uses this value as a hard V8 watchdog budget, with a one-second grace period, so a synchronous script cannot run forever.
OBSCURA_SCRIPT_DEADLINE_MS=60000 obscura serve`OBSCURA_MODULE_BUDGET_MS`
Per-module graph-loading and evaluation budget for modules that enhance an already-rendered page. Default 3000 (3 seconds). Raise it when a module such as the Vite HMR client legitimately needs longer to evaluate:
OBSCURA_MODULE_BUDGET_MS=10000 obscura serveThis shorter budget applies when the document body already contains more than 50 descendant nodes, where modules are normally progressive enhancement and should not delay navigation indefinitely. For an unmounted SPA shell, Obscura instead gives each module the full OBSCURA_SCRIPT_DEADLINE_MS budget so the app has time to mount. Module network requests remain independently bounded by OBSCURA_FETCH_TIMEOUT_MS.
`OBSCURA_CDP_COMMAND_TIMEOUT_MS`
Per-command deadline for the CDP server. A hung page (a runaway Runtime.evaluate, a synchronous DOM op) is terminated after this budget so one bad session cannot hold the shared V8 lock and stall the others. Default 60000 (60 seconds); 0 disables it. Navigation self-bounds via OBSCURA_NAV_TIMEOUT_MS well under this.
OBSCURA_CDP_COMMAND_TIMEOUT_MS=30000 obscura serve`OBSCURA_CDP_TOKEN`
Bearer token for the CDP discovery and WebSocket endpoints. It is optional on
loopback. A non-loopback bind is refused unless this is set to at least 32
bytes. Pass it as Authorization: Bearer <token> in the CDP client's headers.
OBSCURA_CDP_TOKEN="$(openssl rand -hex 32)" obscura serve --host 0.0.0.0`OBSCURA_FETCH_TIMEOUT_MS`
Request timeout for scripted fetch(), XMLHttpRequest, and ES-module loads. Without it a request to a server that accepts the connection but never responds (including a CORS preflight) hangs forever and the XHR is stuck with no completion event. Default 30000 (30 seconds).
OBSCURA_FETCH_TIMEOUT_MS=15000 obscura serve`OBSCURA_PROXY`
Default proxy URL used by obscura-worker for the parallel scrape command when no --proxy flag is set.
OBSCURA_PROXY=http://proxy.example.com:8080 obscura scrape - < urls.txtStealth and identity
These tune the browser identity the engine presents so it stays internally consistent. See Configure stealth and proxies for the full picture.
`OBSCURA_BLOCK_TRACKERS`
Controls the tracker blocklist used by the stealth HTTP transport. It is on by
default so --stealth retains its current privacy-first behavior. Set it to0, false, no, or off to keep the stealth TLS/browser fingerprint while
allowing tracker requests.
The setting is read when a stealth client is created. Values are case-insensitive
and surrounding whitespace is ignored; unset, empty, and unrecognized values keep
blocking enabled. Non-stealth transport settings and SSRF protection are unchanged.
OBSCURA_BLOCK_TRACKERS=0 obscura --stealth fetch https://example.com`OBSCURA_TIMEZONE`
Pins the process timezone before V8/ICU reads it, so Date (getTimezoneOffset, toString) and Intl.DateTimeFormat report one consistent zone. Default Europe/Berlin. Set it to match the exit IP's region.
OBSCURA_TIMEZONE=America/New_York obscura serve`OBSCURA_GEOLOCATION`
Override the coordinates the navigator.geolocation shim reports, as lat,lon. Without it the shim reports a fixed default. Keep it consistent with OBSCURA_TIMEZONE and the proxy region.
OBSCURA_GEOLOCATION="40.7128,-74.0060" obscura serve`OBSCURA_PROFILE`
Pin a specific browser profile from the built-in pool by index (0-based). Each profile keeps navigator.platform, userAgentData, the UA string, and the GPU renderer internally consistent. Without it a single stable profile is used.
OBSCURA_PROFILE=2 obscura serve`OBSCURA_ROTATE_PROFILE`
Opt into picking a random profile per browser context instead of the stable default. Leave it off when you pin a TLS fingerprint, proxy region, or timezone, since a rotated profile would no longer match those.
OBSCURA_ROTATE_PROFILE=1 obscura serveMCP
`OBSCURA_MCP_ALLOWED_ORIGINS`
Comma-separated Origin allowlist for the HTTP MCP transport (obscura mcp --http). Browser requests are refused by default; when set, only listed origins are accepted. Native, non-browser MCP clients (which send no Origin) are always allowed.
OBSCURA_MCP_ALLOWED_ORIGINS="https://app.example.com" obscura mcp --http --host 0.0.0.0`OBSCURA_MCP_TOKEN`
Bearer token for MCP HTTP requests. It is optional on loopback. A non-loopback
bind is refused unless this is set to at least 32 bytes.
OBSCURA_MCP_TOKEN="$(openssl rand -hex 32)" obscura mcp --http --host 0.0.0.0Logging
`RUST_LOG`
Standard tracing filter. Common settings:
RUST_LOG=obscura=info obscura serve
RUST_LOG=obscura=debug obscura serve
RUST_LOG=obscura_cdp=trace,obscura_browser=debug obscura serve--verbose on the CLI is equivalent to RUST_LOG=obscura=info.
Build
`OPENSSL_NO_VENDOR`
Forces cargo build to use the system OpenSSL instead of compiling the vendored copy. Set to 1 on hosts where the vendored OpenSSL fails (older VPS with AVX-512 issues).
OPENSSL_NO_VENDOR=1 cargo build --release --features renderV8
V8 flags are passed via --v8-flags, not environment variables:
obscura --v8-flags "--max-old-space-size=2048 --expose-gc" serveDefaults are --max-old-space-size=4096 --max-semi-space-size=4 --optimize-for-size on 64-bit systems. In the current V8, --optimize-for-size overrides the requested semi-space cap to 1 MiB, not 4 MiB. The old-generation ceiling remains 4 GB.
User flags are appended after the defaults, but V8 applies flag implications afterwards. To change the young-generation cap, also disable size optimization, for example obscura --v8-flags "--no-optimize-for-size --max-semi-space-size=8" serve. This can reduce GC CPU at the cost of more memory; measure the complete workload before adopting it. Use --v8-flags "--print-flag-values" to inspect the effective settings after V8 initialization.
HTTP proxy environment
Obscura does not honor HTTP_PROXY / HTTPS_PROXY / NO_PROXY. Use --proxy or OBSCURA_PROXY.
File: docs/Extract-data.md
--dump formats the page output without writing JavaScript.
obscura fetch https://example.com --dump html
obscura fetch https://example.com --dump text
obscura fetch https://example.com --dump markdown
obscura fetch https://example.com --dump links
obscura fetch https://example.com --dump assets
obscura fetch https://example.com --dump original
obscura fetch https://example.com --dump cookies`html`
Rendered HTML after JavaScript runs. Default.
obscura fetch https://news.ycombinator.com --dump html > hn.html`text`
Plain text. No markup.
obscura fetch https://en.wikipedia.org/wiki/Rust_(programming_language) --dump text`markdown`
Markdown conversion: headings, lists, links, code blocks, images.
obscura fetch https://docs.example.com/page --dump markdown > page.md`links`
Every <a href> on the page, one per line.
obscura fetch https://example.com --dump links`assets`
Every external resource (stylesheets, scripts, images, fonts, iframes), plus the URLs the page requested through fetch()/XHR, one JSON object per line.
obscura fetch https://example.com --dump assets`original`
The raw HTML the server sent, before JavaScript ran.
obscura fetch https://my-spa.example --dump original > before.html
obscura fetch https://my-spa.example --dump html > after.html
diff before.html after.html`cookies`
Every cookie in the jar as a JSON array, including HttpOnly cookies that document.cookie cannot see. Useful for capturing session tokens set by anti-bot challenges.
obscura fetch https://example.com --dump cookiesWith `--wait-until`
--dump runs after the wait condition:
obscura fetch https://my-spa.example --wait-until load --dump markdownPipe and redirect
obscura fetch https://example.com --dump markdown > example.md
obscura fetch https://example.com --dump text --quiet | wc -wReader libraries
After loading a reader library such as Mozilla Readability,
run it on a document clone to leave the page unchanged:
const article = new Readability(document.cloneNode(true)).parse();Obscura does not bundle Readability. Supply the library through your automation
client before evaluating this expression. Document clones preserve metadata and
live form-control state. Attribute clones retain their value and namespace.DOMParser HTML documents preserve the document structure and root attributes,
inherit the parser's document URL, and do not execute scripts. XML parsing still
uses the existing limited XML implementation.
File: docs/Installation.md
Linux x86_64
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
tar xzf obscura-x86_64-linux.tar.gz
./obscura --versionLinux ARM64
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-linux.tar.gz
tar xzf obscura-aarch64-linux.tar.gz
./obscura --versionLinux builds target Ubuntu 22.04 and require glibc 2.35+.
macOS Apple Silicon
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-macos.tar.gz
tar xzf obscura-aarch64-macos.tar.gz
./obscura --versionmacOS Intel
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-macos.tar.gz
tar xzf obscura-x86_64-macos.tar.gz
./obscura --versionWindows
Download the .zip from Releases, extract, run obscura.exe --version.
Arch Linux (AUR)
yay -S obscura-browserDocker
docker run -d --name obscura -p 127.0.0.1:9222:9222 h4ckf0r0day/obscuraImage: h4ckf0r0day/obscura. Built on distroless/cc:nonroot, with no shell or package manager in the runtime image, running as uid 65532. Note the -p 127.0.0.1:... above: it publishes the port to host loopback only. A mounted --storage-dir must be writable by uid 65532 โ see Run in production at scale.
Official archives and the Docker image include the rendering engine. Source
builders must pass --features render; see Build from source.
From source
See Build from source.
What's in the archive
obscura: CLI and CDP server.obscura-worker: helper for the parallelscrapecommand. Keep both in the same directory.
Archive suffixes identify the feature set: no suffix includes rendering,-stealth includes rendering and stealth, -no-render includes neither, and-no-render-stealth includes stealth without rendering.
Smoke test
./obscura fetch https://example.com --eval "document.title"
./obscura fetch https://example.com --screenshot smoke.pngExpected output: "Example Domain", followed by a nonempty PNG at smoke.png.
Troubleshooting
cannot execute binary file: wrong arch. Check uname -m.
GLIBC_2.35 not found: distro is older than Ubuntu 22.04. Use Docker or build from source.
macOS Gatekeeper warning: xattr -d com.apple.quarantine ./obscura.
File: docs/Intercept-and-modify-requests.md
CDP Fetch.enable lets you inspect, block, or modify every request the page makes.
Block by resource type
Puppeteer:
await page.setRequestInterception(true);
page.on('request', req => {
if (['image', 'media', 'font'].includes(req.resourceType())) {
req.abort();
} else {
req.continue();
}
});Playwright:
await page.route('**/*', route => {
if (['image', 'media', 'font'].includes(route.request().resourceType())) {
route.abort();
} else {
route.continue();
}
});Block by URL pattern
// Puppeteer
page.on('request', req => {
const url = req.url();
if (url.includes('google-analytics.com') || url.includes('doubleclick.net')) {
req.abort();
} else {
req.continue();
}
});// Playwright
await page.route(/google-analytics\.com|doubleclick\.net/, route => route.abort());Modify headers
// Puppeteer
page.on('request', req => {
req.continue({
headers: { ...req.headers(), 'X-Custom': 'value' },
});
});// Playwright
await page.route('**/*', route => {
route.continue({
headers: { ...route.request().headers(), 'X-Custom': 'value' },
});
});Return a fake response
// Puppeteer
page.on('request', req => {
if (req.url().endsWith('/api/feature-flags')) {
req.respond({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ newDashboard: true }),
});
} else {
req.continue();
}
});// Playwright
await page.route('**/api/feature-flags', route => {
route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ newDashboard: true }),
});
});Strip analytics in production scrapes
const BLOCK = [
'google-analytics.com',
'googletagmanager.com',
'doubleclick.net',
'facebook.net',
'segment.io',
'mixpanel.com',
'hotjar.com',
];
page.on('request', req => {
if (BLOCK.some(host => req.url().includes(host))) {
req.abort();
} else {
req.continue();
}
});Built-in: --stealth ships with a tracker blocklist that handles most of these without per-script setup. See Configure stealth and proxies.
From the Rust library
The patterns above drive interception over CDP from Puppeteer or Playwright. If you embed the engine with the obscura crate, the same capability is a native API on Page: on_request / on_response callbacks, an enable_interception() channel that can block, mock, or rewrite requests, and add_preload_script to run code before the page's own scripts. See Use as a Rust library.
File: docs/Markdown-extraction.md
--dump markdown converts the rendered page to markdown.
obscura fetch https://example.com --dump markdownWhat gets converted
- Headings (
<h1>through<h6>) - Paragraphs, line breaks
- Bold, italic, code spans
- Links (with
href) - Images (with
srcandalt) - Ordered and unordered lists
- Block quotes
- Code blocks (``,
<code>) - Tables
What gets stripped
<script>,<style>,<noscript>- Inline styles
- ARIA attributes
- Tracking pixels and beacons
Save to file
obscura fetch https://docs.example.com/page --dump markdown -o page.mdFor RAG / LLM context
obscura fetch https://docs.example.com/page --dump markdown --quiet--quiet strips info logging so the output is just markdown.
Wait for SPA content
For pages that render content client-side:
obscura fetch https://my-spa.example --wait-until load --dump markdownNarrow to a region
--selector restricts the conversion to a CSS selector:
obscura fetch https://example.com --selector "main" --dump markdown
obscura fetch https://example.com --selector "article.post" --dump markdownUseful for skipping nav, sidebars, and footers.