## File: README.md A Model Context Protocol (MCP) server and CLI that provides tools for agent use when working on iOS and macOS projects. [](https://github.com/getsentry/MobileBuildMCP/actions/workflows/ci.yml) [](https://badge.fury.io/js/mobilebuildmcp) [](https://opensource.org/licenses/MIT) [](https://nodejs.org/) [](https://developer.apple.com/xcode/) [](https://www.apple.com/macos/) [](https://modelcontextprotocol.io/) [](https://deepwiki.com/getsentry/MobileBuildMCP) [](https://www.agentaudit.dev/skills/xcodebuildmcp) [](https://pkg.pr.new/~/getsentry/MobileBuildMCP) ## Installation MobileBuildMCP ships as a single package with two modes: a **CLI** for direct terminal use and an **MCP server** for AI coding agents. Either install method gives you both. ### Option A — Homebrew ```bash brew tap getsentry/xcodebuildmcp brew install mobilebuildmcp ``` ### Option B — npm (Node.js 18+) ```bash npm install mobilebuildmcp@latest ``` Verify either install: ```bash mobilebuildmcp --help ``` ### Connect your MCP client Drop-in config snippets for Cursor, Claude Code, Codex, can be found in the official docs page [MCP Clients](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/clients.mdx). Most clients can also run the MCP server on demand via `npx -y mobilebuildmcp@latest mcp` without a global install. ## Requirements - macOS 14.5 or later - Xcode 16.x or later - Node.js 18.x or later (not required for Homebrew installation) ## Skills MobileBuildMCP now includes two optional agent skills: - **MCP Skill**: Primes the agent with instructions on how to use the MCP server's tools (optional when using the MCP server). - **CLI Skill**: Primes the agent with instructions on how to navigate the CLI (recommended when using the CLI). To install with a global binary: ```bash mobilebuildmcp init ``` Or install directly via npx without a global install: ```bash npx -y mobilebuildmcp@latest init ``` For further information on installing skills, see [Agent Skills](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/skills.mdx). ## Notes - MobileBuildMCP requests xcodebuild to skip macro validation to avoid errors when building projects that use Swift Macros. - Device tools require code signing to be configured in Xcode. See [Device Code Signing](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/device-signing.mdx). ## Privacy MobileBuildMCP uses Sentry for internal runtime error telemetry only. For details and opt-out instructions, see [Privacy & Telemetry](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/privacy.mdx). ## CLI MobileBuildMCP provides a unified command-line interface. The `mcp` subcommand starts the MCP server, while all other commands provide direct terminal access to tools: ```bash # Install globally npm install mobilebuildmcp@latest # Start the MCP server (for MCP clients) mobilebuildmcp mcp # List available tools mobilebuildmcp tools # Build for simulator mobilebuildmcp simulator build --scheme MyApp --project-path ./MyApp.xcodeproj # Prepare portable test products without running tests mobilebuildmcp simulator build --scheme MyApp --project-path ./MyApp.xcodeproj --simulator-name "iPhone 17" --build-for-testing --test-products-path ./MyApp.xctestproducts # Run previously prepared test products mobilebuildmcp simulator test --test-products-path ./MyApp.xctestproducts --simulator-name "iPhone 17" ``` Check for updates and upgrade in place: ```bash mobilebuildmcp upgrade --check mobilebuildmcp upgrade --yes ``` The CLI uses a per-workspace daemon for stateful operations (log capture, debugging, etc.) that auto-starts when needed. See the [CLI guide](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/cli.mdx) for full documentation. ## Documentation - Installation: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/installation.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/installation.mdx) - Setup: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/setup.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/setup.mdx) - MCP clients: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/clients.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/clients.mdx) - CLI usage: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/cli.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/cli.mdx) - Configuration and options: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/configuration.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/configuration.mdx) - Tools reference: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/tools.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/tools.mdx) - Troubleshooting: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/troubleshooting.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/troubleshooting.mdx) - Privacy: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/privacy.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/privacy.mdx) - Skills: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/skills.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/skills.mdx) - Contributing: [https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/contributing.mdx](https://github.com/getsentry/xcodebuildmcp.com/blob/main/app/docs/_content/contributing.mdx) ## Licence This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. For third-party licensing notices see the [THIRD_PARTY_LICENSES](THIRD_PARTY_LICENSES) file for details. For npm package attributions see the [THIRD_PARTY_PACKAGE_LICENSES](THIRD_PARTY_PACKAGE_LICENSES.md) file for details. --- ## File: benchmarks/claude-ui/README.md # Claude UI benchmark harness Local/manual harness for running Claude Code against configurable tool surfaces and auditing UI automation behavior. The default suite configuration targets the development MobileBuildMCP MCP server. The harness: - reads a suite YAML file from `benchmarks/claude-ui/suites/` - reads the referenced prompt Markdown file from disk and feeds it to `claude -p` - creates, boots, waits for, and opens a fresh temporary simulator before Claude launches for each suite run by default - writes an isolated per-run MCP workspace config with the suite defaults and temporary `simulatorId` when MCP is enabled - generates a Claude MCP config pointing at `node build/cli.js mcp` with `MOBILEBUILDMCP_CWD` set to that isolated workspace when MCP is enabled - optionally preflights configured first-run prompts before Claude launches, outside the measured run - deletes the temporary simulator at the end of the suite, best effort, using only the ID created by the harness - writes artifacts under `out.nosync/claude-benchmarks///` - runs the bundled `parse_claude_conversation.py` parser against Claude's stream JSONL - audits tool counts, configured tracked tool calls, UI automation calls, wall clock, stumbles, and observed tool sequence differences - prints a structured per-suite report and (for `--all`) an aggregate summary - optionally prints machine-readable JSON with `--json` - can render an existing `result.json` or artifact directory with `--from-result` without rerunning Claude This is intentionally not part of the normal test suite because it launches Claude and drives local simulators/apps. ## Commands Build first, then run a suite: ```bash npm run build npx tsx benchmarks/claude-ui/run.ts --suite weather ``` Shortcut: ```bash npm run bench:claude-ui -- --suite weather ``` CLI-backed suites should use benchmark-local skills rather than global Claude/agent skills. Keep private/local CLI suites and assets under `benchmarks/claude-ui/local/`; that directory is ignored by git. For local CLI skills, point the suite at the local directory with `claude.pluginDirs`. Do not rely on skills installed in `~/.claude`, `~/.agents`, or the repository root; the benchmark run can isolate Claude's working directory so only the suite's configured tool skill is visible. Run every committed suite YAML plus any private local suites under `benchmarks/claude-ui/local/suites/`: ```bash npm run bench:claude-ui -- --all ``` Print machine-readable output from a new run: ```bash npm run bench:claude-ui -- --suite reminders --json ``` Request an exact Claude model for controlled comparisons: ```bash npm run bench:claude-ui -- --suite weather --model claude-sonnet-4-7 npm run bench:claude-ui:mobilebuildmcp -- --model claude-sonnet-4-7 ``` The `--model` CLI option overrides `claude.model` from the suite YAML for that run. Render an existing result without rerunning Claude: ```bash npm run bench:claude-ui -- --from-result out.nosync/claude-benchmarks/reminders/20260522T130926Z npm run bench:claude-ui -- --from-result out.nosync/claude-benchmarks/reminders/20260522T130926Z/result.json --json ``` New runs use the bundled parser at `benchmarks/claude-ui/parse_claude_conversation.py`. Pass `--parser /path/to/parse_claude_conversation.py` or set `CLAUDE_UI_BENCHMARK_PARSER` only when testing a different parser. `--from-result` does not need a parser because it only re-renders existing artifacts. ## Suite YAML shape ```yaml name: weather prompt: ../prompts/weather.md workingDirectory: example_projects/Weather sessionDefaults: projectPath: Weather.xcodeproj scheme: Weather simulatorName: iPhone 17 Pro Max temporarySimulator: true firstRunPromptDismissals: labels: - Continue - Not Now timeoutSeconds: 12 baseline: totalToolCalls: 19 trackedToolCalls: 18 mcpToolCalls: 18 uiAutomationCalls: 16 wallClockSeconds: 125 tools: snapshot_ui: 1 tap: 9 baselineToolSequence: - session_show_defaults - build_run_sim - snapshot_ui failurePatterns: - STALE_ELEMENT_REF - SNAPSHOT_MISSING - WAIT_TIMEOUT ignoredFailurePatterns: - element_disabled ``` Baselines are recorded comparison data, not completion gates. Metric rows show the current observed value, the recorded baseline value, and the delta. Use baselines to compare against the best or representative successful run recorded so far, not as made-up correctness thresholds. When recording new baselines, only write baseline values from clean successful runs. A clean run means Claude completed the task, Claude exited with status `0`, the parser exited with status `0`, there were no parse errors, and there were no unexpected terminal/process/task failures. Old-baseline metric drift does not disqualify a run. Retry a suite up to three total attempts when trying to establish a baseline. If no attempt produces a clean successful run, remove that suite's `baseline:` block rather than leaving stale recorded data behind. Report `no clean baseline after 3 attempts / no baseline recorded` in your notes instead. Tool sequence differences are reported as observed comparison data because real Claude runs can choose equally valid UI paths. Sequence differences do not affect task/process completion status. `sessionDefaults` are written to a harness-owned config at `/mcp-workspace/.mobilebuildmcp/config.yaml`. The generated Claude MCP config sets `MOBILEBUILDMCP_CWD` to `/mcp-workspace`, so the dev MCP server reads only the benchmark config instead of any repo or example-project `.mobilebuildmcp/config.yaml`. Unknown keys fail fast. Relative path defaults such as `projectPath`, `workspacePath`, and `derivedDataPath` are resolved against the suite `workingDirectory` before being written because the MCP server cwd is the isolated workspace. ## Configuring Claude and tracked tools Suites can override the Claude invocation without changing harness code. Omit this block for the default MobileBuildMCP MCP behavior. ```yaml claude: model: claude-sonnet-4-7 useMcpServer: false tools: - Bash allowedTools: - Bash(vendorcli *) - Bash(xcodebuild *) appendSystemPrompt: | Use the simulator from `$CLAUDE_UI_BENCHMARK_SIMULATOR_ID`. You may use the configured local CLI and xcodebuild directly. pluginDirs: - benchmarks/claude-ui/local/skills/vendor-cli isolatedWorkingDirectory: true extraArgs: - --setting-sources - project,local toolAnalysis: matchers: - kind: bashCommand commandPrefix: vendorcli ui screen shortName: vendorcli.screen uiAutomation: true - kind: bashCommand commandPrefix: vendorcli ui tap shortName: vendorcli.tap uiAutomation: true - kind: bashCommand commandPrefix: xcodebuild shortName: xcodebuild ``` `claude.model` is the canonical suite-level model request. Do not put `--model` or `--model=` in `claude.extraArgs`; the config parser rejects those forms so suite config and CLI overrides cannot disagree. Pass `--model ` to override the suite model for controlled comparison runs. `claude.useMcpServer: false` writes an empty per-run MCP config and passes it with `--strict-mcp-config`, so project/user MCP servers cannot leak into CLI-only benchmark runs. The harness still prepares the simulator lifecycle and exports `CLAUDE_UI_BENCHMARK_SIMULATOR_ID`, `CLAUDE_UI_BENCHMARK_RUN_DIR`, and `CLAUDE_UI_BENCHMARK_WORKING_DIRECTORY` to Claude. `appendSystemPrompt` also supports `{simulatorId}`, `{runDirectory}`, and `{workingDirectory}` placeholders. `claude.pluginDirs` is passed to Claude as one `--plugin-dir` argument per configured path, resolved from the repository root. Use this for suite-specific local/private CLI skills. `claude.isolatedWorkingDirectory: true` runs Claude from the per-run artifact directory instead of the suite working directory, which prevents repository/project skills from being discovered implicitly. When using an isolated working directory, include absolute `{workingDirectory}` paths in prompts for build commands or project files. `toolAnalysis.matchers` defines what the analyzer treats as benchmark-relevant. `namePrefix` matchers track MCP-style tool names and can strip the prefix or final `__` segment. `bashCommand` matchers track Claude `Bash` tool calls whose `command` starts with the configured prefix. `uiAutomation: true` marks a tracked command as UI automation; xcodebuild commands can be tracked without counting as UI automation. `ignoredFailurePatterns` removes configured, known non-terminal tool-result errors from observed stumbles and from `failurePatterns` matches. Keep these patterns narrow and suite-specific. This is useful for CLI tools that return nonzero for exploratory probes while the agent can recover and still complete the user task. Use `preflightCommands` when a CLI tool needs host setup outside Claude's measured run, such as starting its companion app or validating local health: ```yaml preflightCommands: - open -a LocalBenchTool - sleep 3 - vendorcli status - vendorcli ui inspect --udid "$CLAUDE_UI_BENCHMARK_SIMULATOR_ID" - vendorcli ui home --udid "$CLAUDE_UI_BENCHMARK_SIMULATOR_ID" ``` ## Temporary simulator lifecycle By default, each suite creates a fresh simulator before Claude launches. The harness uses `sessionDefaults.simulatorName` as the `simctl create` device type name, captures the returned simulator ID, boots that simulator, waits for `simctl bootstatus -b`, opens Simulator.app to that device, applies a short UI-readiness delay, and writes the simulator ID as `sessionDefaults.simulatorId` in the isolated MCP workspace config. This makes Claude and the dev MCP server target a visible, booted, isolated simulator instead of reusing a previous run's state or spending benchmark calls on simulator boot/open setup. Simulator setup is deliberately outside the benchmark measurement boundary. The measured `wallClockSeconds` starts when the harness spawns Claude and stops when Claude exits. Tool-call counts are parsed only from Claude's JSONL transcript. The result JSON still records temporary simulator `setupDurationSeconds` under `run.temporarySimulator` so setup cost is visible without being compared against Claude task-efficiency baselines. Config contract: - Omit `temporarySimulator` for the default behavior: create and later delete a temporary simulator. - Set `temporarySimulator: false` with `sessionDefaults.simulatorName` to resolve, boot, open, and export an existing simulator by name without deleting it. - Set `sessionDefaults.simulatorId` to use an existing simulator. In this case the harness does not create or delete a simulator. - Do not set both `temporarySimulator: true` and `sessionDefaults.simulatorId`; the harness fails fast because deleting a user-provided simulator would be unsafe. Temporary simulator setup is required when enabled. If creation, boot, bootstatus, or Simulator.app opening fails, the suite fails loudly before Claude starts. Deletion is best effort in a `finally` block: failures are logged but do not mask the benchmark result or original error. `firstRunPromptDismissals` is an optional suite-level preflight for fresh simulator noise such as Apple first-run sheets. When configured, the harness launches `sessionDefaults.bundleId` before Claude starts, retries through transient UI-inspection failures, looks for any listed button labels, taps matching labels with AXe, then terminates the app. If the prompt state cannot be inspected or dismissed before `timeoutSeconds`, the suite fails before Claude starts. These preflight interactions are logged in `simulator-lifecycle.log`, but they are outside Claude's wall-clock measurement and do not appear in tool-call counts. Keep the labels generic and non-destructive, for example `Continue`, `Not Now`, or `OK`; do not configure sign-in, sync enablement, Settings, destructive, or data-deletion actions. Lifecycle details are written to `simulator-lifecycle.log`, including the `create`, `boot`, `bootstatus`, `open`, readiness delay, optional first-run prompt preflight, and deletion steps. `claude-command.log` also records the simulator ID used for the run. The terminal report shows the temporary simulator ID plus setup duration as `setup ... before Claude` when a temporary simulator is used. ## Terminal report Each suite renders as a structured report with a task-completion banner, aligned metric and tool tables, a stumbles section, and observed sequence differences. Baseline metric and sequence differences are observational. When run with `--all`, an aggregate summary follows the per-suite reports. ### Single suite ```text ──────────────────────────────────────────────────────────────────────── COMPLETED weather 1m 38.6s suite benchmarks/claude-ui/suites/weather.yml artifacts out.nosync/claude-benchmarks/weather/20260522T214044Z claude model requested=claude-sonnet-4-7 observed=claude-sonnet-4-7 version=1.2.3 exit claude=0 parser=0 Metrics METRIC ACTUAL BASELINE DELTA totalToolCalls 13 19 −6 mcpToolCalls 12 18 −6 uiAutomationCalls 10 16 −6 wallClockSeconds 98.62 125.00 −26.38 Tool calls (baseline-observed) TOOL ACTUAL BASELINE DELTA session_show_defaults 1 1 0 build_run_sim 1 1 0 snapshot_ui 1 1 0 tap 6 9 −3 batch 1 1 0 OBSERVED stumbles: 0 ``` ### Sequence differences When the tool sequence differs from the recorded sequence, the report includes unified-diff style hunks with baseline/actual index columns: ```text OBSERVED tool sequence: 4 missing from baseline, 0 additional @@ baseline[8..15] actual[8..11] @@ 8 8 tap 9 9 tap 10 − tap 11 10 swipe 12 11 tap 13 − swipe 14 − tap 15 − tap ``` `−` lines are baseline calls Claude skipped; `+` lines are calls Claude made beyond the baseline sequence. Dim lines are surrounding context. ### Stumbles and inspect hints When `stumbles` is non-zero the report lists the first few tool errors and pattern matches, and surfaces an `Inspect` block with the relevant artifact paths: ```text INCOMPLETE stumbles: 1 • tool errors: 1 boot_sim @ line 9: Boot failed: device not found Inspect result.json out.nosync/claude-benchmarks/reminders/20260522T213905Z/result.json transcript out.nosync/claude-benchmarks/reminders/20260522T213905Z/claude.jsonl stderr out.nosync/claude-benchmarks/reminders/20260522T213905Z/claude.stderr run dir out.nosync/claude-benchmarks/reminders/20260522T213905Z ``` ### Aggregate summary After `--all` (or multi-result `--from-result`) the harness appends: ```text ════════════════════════════════════════════════════════════════════════ Claude UI Benchmarks · Summary ════════════════════════════════════════════════════════════════════════ Suites: 3 total · 2 completed · 1 incomplete Duration: total 4m 49.8s · slowest reminders (1m 39.8s) Artifacts: out.nosync/claude-benchmarks/ ✓ COMPLETED weather 1m 38.6s sequence delta: 4m/0a ! INCOMPLETE reminders 1m 39.8s 1 stumble · sequence delta: 7m/4a ✓ COMPLETED contacts 1m 31.4s sequence delta: 2m/2a ════════════════════════════════════════════════════════════════════════ ``` `Nm/Ka` denotes "N missing / K additional" calls vs. `baselineToolSequence`. The renderer auto-detects TTY and adds ANSI color when stdout is a terminal and `NO_COLOR` is unset. Plain-text output (e.g. when piping to a file or under `NO_COLOR=1`) carries the same information without color codes. `--json` output is unchanged by this renderer: the JSON payload remains a single `BenchmarkResult` for `--suite` / single-result `--from-result`, and an array for `--all` / multi-result `--from-result`. ## Artifacts Each run writes: - `prompt.md` — exact suite prompt fed to Claude - `mcp-config.json` — generated Claude MCP config - `mcp-workspace/.mobilebuildmcp/config.yaml` — isolated MCP server config with effective suite defaults - `claude.jsonl` — Claude stream JSON output - `claude.stderr` — Claude stderr - `claude-command.log` — command, cwd, simulator ID, requested/observed model, `claude --version`, exit status, wall clock - `simulator-lifecycle.log` — temporary simulator create, boot, bootstatus, open, readiness, deletion commands, and simulator ID - `parsed/` — files written by `parse_claude_conversation.py` - `parse.log` / `parse.log.stderr` — parser output - `result.json` — full benchmark result, including requested model, observed model when Claude reports it, and `claude --version` output under `run.claude` --- ## File: example_projects/Weather/README.md # Atmos Weather Atmos Weather is a native SwiftUI weather app prototype for iOS. ## Launch Build and run the app with MobileBuildMCP: ```bash ../../build/cli.js simulator build-and-run ``` ## JSON fixtures Fixture JSON files live in: ```text WeatherTests/Fixtures/ ``` Current fixtures: - `WeatherTests/Fixtures/default-locations.json` - `WeatherTests/Fixtures/search-locations.json` - `WeatherTests/Fixtures/weather-report-loc-current-san-francisco.json` ## API schemas OpenAI-compatible API schema files live in: ```text Schemas/ ``` Current schemas: - `Schemas/default-locations.schema.json` - `Schemas/search-locations.schema.json` - `Schemas/weather-report.schema.json` These schemas describe the JSON response shape expected by the DTO layer. ## Expected API endpoints The production client is `URLSessionWeatherAPIClient`. It currently expects a JSON API rooted at: ```text https://api.atmosweather.example/v1 ``` All endpoints are `GET` requests. | Purpose | Method | Path | Request shape | Schema | | --- | --- | --- | --- | --- | | Default saved locations | `GET` | `/locations/default` | No path params, query params, or body. | `Schemas/default-locations.schema.json` | | Search locations | `GET` | `/locations/search` | Query string: `query=` | `Schemas/search-locations.schema.json` | | Weather report for a location | `GET` | `/weather/{locationID}` | Path param: `locationID=` | `Schemas/weather-report.schema.json` | ### Request examples Default locations: ```http GET /v1/locations/default ``` Search locations: ```http GET /v1/locations/search?query=San%20Francisco ``` Weather report: ```http GET /v1/weather/loc-current-san-francisco ``` ### Response expectations - Responses must be JSON. - Successful responses should use a `2xx` HTTP status code. - Non-`2xx` responses are treated as API failures. ## Tests Run the app test suite through MobileBuildMCP: ```bash ../../build/cli.js simulator test ``` The app uses bundled deterministic weather data so UI tests do not depend on the production API endpoint.