Latitude can import the traces you already have in another observability platform, a
one-time backfill so you do not start from an empty project when you migrate. For the
whole migration path, live tracing included, see Migrate to Latitude. Imports run from Project settings →
Imports and finish in the background — you can close the tab.

Pick your platform to get started:

How an import runs Copy Chapter

Every platform is imported from its cloud offering — you pick a region, never a URL.
Self-hosted deployments are not supported yet.

What gets imported Copy Chapter

Imported spans carry import.* metadata (platform, source project, and the original trace
and span ids), so you can always tell imported history from live telemetry. If your
organization has a PII redaction policy, it applies to imported traces exactly as it
does to live ones.

An imported trace goes through the same pipeline as one that arrives live: it is grouped
into sessions, embedded for semantic search, indexed, and evaluated by flaggers. Because of
that it is billed the same way: one credit per trace. Re-importing the same trace does
not charge twice.

Limits Copy Chapter

Traces are imported newest first, so an import that stops early — it reached your trace
limit or your plan's remaining usage — keeps the most recent history rather than an
arbitrary slice. A stopped import is marked capped, everything imported stays, and
Continue resumes it from where it stopped.

Retrying Copy Chapter

Imports are idempotent: every imported span gets a deterministic id derived from its
source ids, so importing the same range twice updates the same spans instead of duplicating
them. If an import fails or is cancelled, Retry resumes from where it stopped — you
will be asked for credentials again, because Latitude never keeps them after an import
ends.

Agent Dispatch Claude Code (`docs/agent-dispatch/claude-code.mdx`) Copy Section

title: Claude Code agent dispatch Copy Chapter

Claude Code agent dispatch lets Latitude fire a Claude Code routine when a signal needs engineering follow-up. Latitude sends the routine a prompt with signal context, sample traces, and a link back to Latitude.

Prerequisites Copy Chapter

• Agent dispatch enabled for your Latitude workspace
• Access to Claude Code routines
• A Claude Code routine that is allowed to work in the repository you want it to modify
• The routine trigger token from Claude Code

Prepare a routine Copy Chapter

1. Open Claude Code routines.
2. Create or select the routine Latitude should trigger.
3. Use a routine description that tells Claude Code to investigate Latitude signals, implement fixes, run relevant checks, and report what changed.
4. Copy the routine trigger token from the routine API section.
5. Copy the routine page URL from your browser. Latitude extracts the routine trigger ID from this URL.

A good routine description is:

Inspect the Latitude signal, identify the regression or newly discovered issue, implement the fix, run the relevant checks, and report what changed.

Connect Claude Code Copy Chapter

1. In Latitude, open your project and go to Settings → Integrations.
2. Open Claude Code and click Connect.
3. Paste the routine token.
4. Paste the routine URL, for example https://claude.ai/code/routines/trig_....
5. Click Connect.

Latitude stores the routine token encrypted and uses it only to fire the configured routine.

Configure dispatches Copy Chapter

After Claude Code is connected, open the Claude Code integration detail page in Latitude.

• Triggers choose which Latitude events can fire the routine.
• Routine trigger ID is extracted from the routine URL during setup and can be edited if you rotate routines.
• Max dispatches per day limits how many routine runs Latitude can create.
• Cooldown prevents repeated dispatches for the same source during a short period.

The default prompt asks Claude Code to inspect the signal, use Latitude context and MCP tools when available, identify the likely root cause, implement the smallest correct fix, add a regression test if appropriate, and open a pull request or report the result according to the routine's environment.

What Latitude sends to Claude Code Copy Chapter

Latitude calls the Claude Code routines API with:

• The configured routine trigger ID
• The encrypted routine token
• The rendered dispatch prompt as the routine input text

Dispatch history records whether Claude Code accepted the request and links to the Claude Code session when Claude returns a URL.

Troubleshooting Copy Chapter

• Paste the routine page URL, not just the ID: the connection form validates URLs and extracts trig_... from the path.
• Authentication error: rotate the routine token in Claude Code and reconnect the integration.
• Dispatch request rejected: confirm the routine still exists and the trigger ID matches the routine you want Latitude to fire.
• Routine runs but cannot fix code: review the routine's repository access and instructions in Claude Code.

Agent Dispatch Cursor (`docs/agent-dispatch/cursor.mdx`) Copy Section

title: Cursor agent dispatch Copy Chapter

Cursor agent dispatch lets Latitude start a Cursor agent when a signal needs engineering follow-up. Latitude sends a prompt with the signal context, sample traces, and a link back to Latitude. Cursor runs in the repository you configure and can open a pull request with the fix.

Prerequisites Copy Chapter

• Agent dispatch enabled for your Latitude workspace
• A Cursor workspace with API access
• A Cursor API key from the workspace that should run the agent
• Repository access in Cursor for the repo where fixes should be proposed

Connect Cursor Copy Chapter

1. In Latitude, open your project and go to Settings → Integrations.
2. Open Cursor and click Connect.
3. Open Cursor API keys, sign in to the right workspace, and create a new API key for Latitude.
4. Paste the API key in Latitude.
5. Choose the repository Cursor should work in, or paste the repository URL if it is not listed.
6. Optionally set a branch or ref. Latitude sends this as Cursor's starting ref.
7. Click Connect.

Latitude stores the API key encrypted and uses it only to list repositories and create Cursor agents for dispatches.

Configure dispatches Copy Chapter

After Cursor is connected, open the Cursor integration detail page in Latitude.

• Triggers choose which Latitude events can start a Cursor agent.
• Repository is the Git repository Cursor receives in the dispatch request.
• Branch is optional. Leave it empty to let Cursor use the repository default.
• Max dispatches per day limits how many runs Latitude can create for this integration.
• Cooldown prevents repeated dispatches for the same source during a short period.

The default prompt asks Cursor to inspect the signal, use Latitude context and MCP tools when available, identify the likely root cause, implement the smallest correct fix, add a regression test if appropriate, and open a pull request.

What Latitude sends to Cursor Copy Chapter

Latitude creates a Cursor agent through Cursor's API with:

• The rendered dispatch prompt
• The repository URL
• The optional starting ref
• autoCreatePr enabled by default

Dispatch history records whether Cursor accepted the request and links to the Cursor run when Cursor returns a URL.

Troubleshooting Copy Chapter

• Repository list does not load: verify the API key belongs to the expected Cursor workspace and still has access.
• Authentication error: revoke the old key in Cursor, create a new key, and reconnect the integration.
• Dispatch request rejected: check that the repository URL and branch/ref are valid for the Cursor workspace.
• No pull request appears: open the run from the dispatch history and review the agent result in Cursor.

Agent Dispatch Linear (`docs/agent-dispatch/linear.mdx`) Copy Section

title: Linear agent dispatch Copy Chapter

Linear agent dispatch creates a Linear issue when a Latitude signal needs follow-up. The issue contains the rendered Latitude prompt, signal context, sample trace evidence, and a link back to Latitude.

Prerequisites Copy Chapter

• Agent dispatch enabled for your Latitude workspace
• A Linear workspace where Latitude should create issues
• A Linear personal API key with access to the target workspace
• A Linear team where Latitude can create issues

Connect Linear Copy Chapter

1. In Latitude, open your project and go to Settings → Integrations.
2. Open Linear and click Connect.
3. Open Linear API settings for the workspace where Latitude should create issues.
4. Create a personal API key for Latitude and copy it before leaving Linear.
5. Paste the API key in Latitude.
6. Choose the Linear team where new issues should be created.
7. Click Connect.

Latitude stores the API key encrypted and uses it only to list teams, list members, and create issues.

Configure dispatches Copy Chapter

After Linear is connected, open the Linear integration detail page in Latitude.

• Triggers choose which Latitude events can create Linear issues. Linear currently supports the New signal trigger.
• Linear team is the team where Latitude creates issues.
• Assignee is optional. Leave it empty to create unassigned issues.
• Max dispatches per day limits how many issues Latitude can create.
• Cooldown prevents repeated issue creation for the same source during a short period.

What Latitude sends to Linear Copy Chapter

Latitude creates issues through Linear's GraphQL API.

The issue title uses the signal name and trigger, for example:

[Latitude] Payment timeout spike — signal.discovered

The issue description contains the rendered dispatch prompt and the Latitude deep link. The prompt includes the project, signal, trend metrics, sample trace IDs, sample conversation excerpts when available, and instructions for investigating the issue.

Dispatch history records whether Linear accepted the request and links to the created issue.

Troubleshooting Copy Chapter

• Teams do not load: verify the API key belongs to the expected Linear workspace and can access teams.
• Authentication error: revoke the old key in Linear, create a new key, and reconnect the integration.
• Dispatch request rejected: confirm the selected team still exists and that the optional assignee is a member Linear can assign issues to.
• Duplicate issues: reduce the max dispatches per day or increase the cooldown in the integration settings.

Agent Dispatch Webhooks (`docs/agent-dispatch/webhooks.mdx`) Copy Section

title: Agent dispatch webhooks Copy Chapter

Agent dispatch webhooks let Latitude send signal follow-up work to any HTTPS endpoint you control.

Endpoint requirements Copy Chapter

Your endpoint must:

• Accept POST requests over public HTTPS.
• Return a 2xx response when the dispatch is accepted.
• Return 401 or 403 for authentication failures.
• Return 429 with an optional Retry-After header when you want Latitude to retry later.

Latitude rejects webhook URLs that are not HTTPS or resolve to private/internal IP addresses.

Request body Copy Chapter

Latitude sends JSON with the trigger, the dispatch context, and the prompt text assembled for the agent.

{
  "trigger": "signal.discovered",
  "context": {
    "trigger": "signal.discovered",
    "organizationName": "Acme Inc.",
    "projectName": "Checkout API",
    "projectSlug": "checkout-api",
    "deepLinkUrl": "https://console.latitude.so/projects/checkout-api/signals/sig_123",
    "signal": {
      "id": "sig_123",
      "name": "Payment timeout spike"
    }
  },
  "prompt": "Investigate the Latitude signal and propose the next follow-up action..."
}

The exact context shape depends on the trigger source, but trigger, project identity, a Latitude deep link, and the rendered prompt are always included.

Headers Copy Chapter

Latitude includes two headers on every delivery:

Verify the signature Copy Chapter

When you connect the webhook integration, Latitude shows a webhook secret once. Store it securely and use it to verify X-Latitude-Signature.

import { createHmac, timingSafeEqual } from "node:crypto"

function verifyLatitudeSignature(rawBody: string, signatureHeader: string, secret: string) {
  const expected = `sha256=${createHmac("sha256", secret).update(rawBody).digest("hex")}`
  return timingSafeEqual(Buffer.from(signatureHeader), Buffer.from(expected))
}

Verify the signature against the raw request body before parsing JSON.

Return external run metadata Copy Chapter

A successful handler can return an optional JSON body that identifies the agent run it started:

{
  "externalAgentId": "agent-scope",
  "externalRunId": "run_123",
  "deepLinkUrl": "https://agents.example.com/runs/run_123"
}

All fields are optional. IDs must be non-empty strings, and deepLinkUrl must be an absolute HTTP or HTTPS URL.
Latitude records each valid field in dispatch history and uses deepLinkUrl as the link to the external run.

An empty body, a non-JSON body, or invalid fields do not reject an otherwise successful dispatch. When the response does
not include a valid deepLinkUrl, Latitude links to the configured webhook URL instead.

Retry behavior Copy Chapter

Latitude retries transport failures, 429, and 5xx responses. A 4xx response other than 429 is treated as a configuration or authentication failure and is not retried indefinitely.

Annotations Flaggers (`docs/annotations/flaggers.mdx`) Copy Section

title: Flaggers Copy Chapter

Flaggers

Flaggers are Latitude's built-in automatic annotators. Every project is provisioned with a fixed set of flaggers — one per known failure category — and each completed trace is checked against the enabled flaggers. When a flagger matches, Latitude writes an annotation score directly on the trace, no human review required.

Flaggers replace what used to be called system annotation queues. The categories and detection logic are the same; the surface is different. Instead of routing matches into a queue for review, flaggers write a published annotation immediately and that annotation flows through the rest of the system: scores, issue discovery, analytics, and alignment.

Available Flaggers Copy Chapter

Each project starts with these flaggers provisioned. Some use deterministic rules; the rest use lightweight LLM classifiers.

How Flaggers Run Copy Chapter

When a trace completes (after the trace-end debounce window), Latitude runs the enabled flaggers against it. Each flagger evaluates the trace in three phases:

1. Required context check: The flagger confirms the trace has the data it needs (for example, the Output Schema Validation flagger needs a generation span with a declared schema).
2. Deterministic detection: The flagger applies its rules. The result is matched, no-match, or ambiguous.
3. LLM detection (for LLM-capable flaggers only): If the deterministic step returned ambiguous, an LLM judges the trace. Sampled traces from the no-match path are also re-evaluated with the LLM, so the flagger can catch cases the deterministic rules missed.

If any phase produces a match, Latitude writes a published annotation score on the trace with the flagger's feedback. That annotation behaves like any other:

• It feeds issue discovery
• It contributes to evaluation alignment
• It shows up on the trace, in scores analytics, and in any saved search that surfaces matching traces

Flagger annotations are marked with a system source. They don't sit in a draft state waiting for human review; they're immediately part of your project's signal.

Suppression Copy Section

Some flaggers suppress others to avoid double-counting:

• Refusal is suppressed by Jailbreaking and NSFW: a correct refusal of an unsafe request isn't over-refusal.
• Laziness is suppressed by Trashing: an agent stuck cycling tools isn't separately "lazy".

Suppression only applies when the higher-priority flagger actually matched. If Jailbreaking returns no-match, Refusal can still match.

Configuring Flaggers Copy Chapter

Open Project Settings to see the flaggers list. For each flagger you can adjust:

• Enabled: Turn the flagger on or off for this project. A disabled flagger never runs and never writes annotations.
• Sampling: An integer between 0 and 100 that controls what fraction of no-match traces are escalated to the LLM step. Only meaningful for LLM-capable flaggers. Deterministic flaggers ignore sampling because their rules either match or they don't.

Sampling is the main lever for cost and noise control. A flagger that's too noisy can be turned down; a flagger that's missing too many real cases can be turned up.

Flaggers vs Search vs Inline Annotations Copy Chapter

All three feed the same scores system, the same issue discovery pipeline, and the same alignment metrics. The difference is who decides which trace gets attention: a flagger, a person searching, or a person opening a trace directly.

Flagger Annotations and Issues Copy Chapter

Because flagger annotations are published with a system source, they flow into issue discovery automatically. A trace that gets a frustration annotation from the flagger will be clustered into the relevant issue alongside human-annotated and evaluation-detected occurrences, with no extra wiring.

This is the simplest way to see flaggers in action: enable them on a project, let them run for a week, and watch what comes out the other side of issue discovery.

Next Steps Copy Chapter

• Annotations Overview: How annotations connect to scores, issues, and evaluations
• Inline Annotations: Leave human feedback on traces
• Search: Build cohorts that flaggers don't cover
• Issues: How flagger annotations become trackable failure patterns

Annotations Inline Annotations (`docs/annotations/inline-annotations.md`) Copy Section

title: Inline Annotations Copy Chapter

Inline Annotations

Inline annotations are the primary way to leave human feedback on a trace. Any trace you can open — from the Traces page, from a search result, from an issue's logs, from a session view — has an annotation panel you can use.

How Inline Annotations Work Copy Chapter

When viewing any trace in Latitude:

1. Open the trace detail view by clicking on a trace.
2. Look for the annotation panel on the right side of the screen.
3. Create an annotation:
   • Conversation-level: Click the annotation button to assess the overall interaction.
   • Message-level: Click on a specific message to annotate just that part of the conversation.
   • Text-range: Select a substring within a message to anchor the annotation to that exact span.
4. Provide your verdict (thumbs up / thumbs down) and feedback.
5. Optionally link the annotation to an issue.

Annotations save as drafts immediately so a page refresh won't lose your work. They finalize automatically after 5 minutes of inactivity. Once finalized, they feed into analytics, issue discovery, and evaluation alignment alongside annotations created by flaggers and the API.

A Typical Review Workflow Copy Chapter

The most efficient way to work through a batch of traces is to start from a search or a saved search:

1. Run a search (or open a saved one) that matches the cohort you want to review — for example, "failed payments last week" or "checkout flows over 5 steps".
2. Open the first matching trace.
3. Read the conversation, annotate, and move to the next trace.
4. Repeat until the cohort is covered. The saved search's Annotated / Total columns let your team see progress.

For shared review work, assign the saved search to a teammate. Assignment is a lightweight ownership signal; everyone can still see and open the search.

When to Use Inline Annotations Copy Chapter

Inline annotations cover the full range of review work:

• Systematic review of a cohort: Use a saved search to scope, then annotate each match.
• Ad-hoc spot checks: You're browsing traces and notice something worth flagging.
• Issue investigation: You're drilling into a specific issue and want to annotate the linked traces.
• Demo and training: Showing a new team member how annotation works.
• Supplementary feedback: Adding context to a trace that already has scores or flagger annotations.

If you want detection without human review for a fixed set of known failure categories, look at flaggers instead.

Inline Annotations and Issues Copy Chapter

When creating an inline annotation, you can:

• Leave issue assignment automatic: Let Latitude's discovery pipeline decide which issue the annotation belongs to (or create a new one) when the score is finalized.
• Link to an existing issue: Associate your annotation with a known failure pattern, bypassing automatic discovery for that score.

Once finalized, failed annotations feed into the issue discovery pipeline automatically. You don't need to create issues manually.

Persisted Highlights Copy Chapter

Annotations with a message or text-range anchor leave a highlight in the conversation view. Clicking a highlight focuses the matching annotation card in the panel, so you can jump between the conversation and the feedback that pinned it.

Next Steps Copy Chapter

• Annotations Overview: How the annotation system works
• Flaggers: Automatic annotators for common failure categories
• Search: Find the traces you want to annotate
• Issues: How annotations connect to issue tracking

Datasets Add Traces (`docs/datasets/add-traces.mdx`) Copy Section

title: Add traces to a dataset Copy Chapter

Where this fits: Part of Refine. It turns the traces behind a failure into reusable rows in a dataset, ready for regression testing.

The most realistic test cases are the interactions that actually happened. You can add traces to a dataset directly from where you find them: the trace list, Search results, or a signal.

Add selected traces Copy Chapter

Add a whole cohort Copy Chapter

You are not limited to hand-picked traces. When you add to a dataset from a filtered view, a saved search, or a signal, the matching traces are added as a group, so a saved search for a behavior or a signal's full set of failures becomes a dataset in one action.

Next step Copy Chapter

• Add expected output: record what the agent should have done so a test can check against it.

Datasets Custom Columns (`docs/datasets/custom-columns.mdx`) Copy Section

title: Custom columns Copy Chapter

Where this fits: Part of Refine. Custom columns let a dataset carry fields beyond the four built-ins, so a row can hold exactly the data your tests and workflows need.

Every dataset starts with four built-in columns — Input, Output, Expected output, and Metadata. Custom columns let you add more fields of your own (for example a label, a category, or a reference id) alongside them. A newly added column is empty on every existing row; it only gets a value when you fill that cell.

Manage columns Copy Chapter

Open a dataset and click Columns in the top bar. From the popover you can:

Editing custom values Copy Chapter

Open any row to edit its cells. Active custom columns show their own editable fields next to the built-in ones, and the values you enter are saved with the row just like the built-ins. You can also set custom values programmatically — keyed by column identifier — over the API, the SDKs, or MCP; see Edit row contents.

How columns affect the rest of the dataset Copy Chapter

• Adding, renaming, removing, or reordering a column does not create a new dataset version. Only writing actual cell data does. So shaping your columns is free to iterate on.
• CSV export honors your active columns. Exports include your custom columns (under their display name) in column order and leave out removed columns. Built-in columns keep their standard input / output / expected_output / metadata headers.
• Removed columns are excluded from the table, row reads, the API, and exports until you re-add them.

Manage columns from your coding agent Copy Chapter

Through the MCP server, an agent like Claude or Cursor can manage a dataset's columns for you — list columns (including removed ones), add, rename, reorder, remove, and restore them — the same operations available in the UI.

Next step Copy Chapter

• Add expected output: record the known-good answer for a row.
• Regression testing: replay the dataset against your agent.

Datasets Edit Rows (`docs/datasets/edit-rows.mdx`) Copy Section

title: Edit row contents Copy Chapter

Where this fits: Part of Refine. Editing rows lets you curate a dataset after it is built — most commonly to add expected output to rows imported from traces.

Every dataset row is made of cells: the four built-in fields — Input, Output, Expected output, and Metadata — plus any custom columns you have added. You can edit those cells by hand in the UI, or programmatically over the API, the SDKs, or an MCP agent.

Edit a row in the UI Copy Chapter

Open a dataset and open any row to edit its cells inline. Built-in fields and active custom columns each show their own editable field; the values you enter are saved with the row. See Add expected output for the most common case.

Edit a row programmatically Copy Chapter

Updates are partial: you send only the cells you want to change, and every cell you omit keeps its current value. This makes it safe to, for example, set expectedOutput on a row without resending its input or output.

How cells are addressed:

• Built-in cells use their field name: input, output, expectedOutput, metadata.
• Custom columns are set through a custom map keyed by the column's stable identifier (not its display name). Custom values are merged onto the row's existing ones, so columns you omit are left untouched. Unknown or removed columns are rejected.

A successful edit creates a new dataset version and returns its id.

HTTP API Copy Section

PATCH /v1/projects/{projectSlug}/datasets/{datasetSlug}/rows/{rowId}

curl -X PATCH \
  https://api.latitude.so/v1/projects/my-project/datasets/my-dataset/rows/ROW_ID \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "expectedOutput": "The correct answer",
    "custom": { "COLUMN_IDENTIFIER": "high" }
  }'

The response carries the new dataset version:

{ "versionId": "os4gxr0mhcocrydibwupq4jk", "version": 10 }

SDKs Copy Section

const client = new LatitudeClient({ apiKey: process.env.LATITUDE_API_KEY! });

const { version } = await client.datasets.updateRow(
  "my-project",
  "my-dataset",
  "ROW_ID",
  {
    expectedOutput: "The correct answer",
    custom: { COLUMN_IDENTIFIER: "high" },
  },
);
```

from latitude_sdk import LatitudeClient

client = LatitudeClient(api_key=os.environ["LATITUDE_API_KEY"])

result = client.datasets.update_row(
    "my-project",
    "my-dataset",
    "ROW_ID",
    expected_output="The correct answer",
    custom={"COLUMN_IDENTIFIER": "high"},
)
```

From your coding agent (MCP) Copy Section

Through the MCP server, an agent like Claude or Cursor can edit a row for you — for example, "in my dataset From Traces Drawer, set the expected output of the failing row to …". The agent lists the dataset's rows and columns to resolve the ids, then applies the same partial update described above. For the live tool list and full schemas, see the API reference.

Next step Copy Chapter

• Add expected output: the most common reason to edit a row.
• Custom columns: add, rename, reorder, and remove the columns a row can hold.
• Regression testing: replay the curated dataset against your agent.

Datasets Expected Output (`docs/datasets/expected-output.mdx`) Copy Section

title: Add expected output Copy Chapter

Where this fits: Part of Refine. Expected output turns a dataset of real traces into a test set with a known-good answer for regression testing.

Expected output is the correct or desired answer for a dataset row. It lets a test compare your agent's actual output against a known-good result, rather than only checking the output in isolation. Expected output is optional, since many evaluations check a response on its own merits, but it is what makes a row a precise regression case.

Add expected output to a row Copy Chapter

Where the expected answer comes from Copy Chapter

When you build a dataset from a failing signal, the agent's actual output was wrong, which is why the signal exists. The expected output is the response the agent should have produced. Common sources:

• the correct answer a human reviewer would give
• the behavior described in the signal or in an annotation
• a corrected version of the original output

Add it programmatically Copy Chapter

You can also set expected output (or any other cell) over the API, the SDKs, or an MCP agent — handy for filling in answers in bulk or straight from your coding agent after building a dataset. See Edit row contents.

Next step Copy Chapter

• Regression testing: replay the dataset against your agent and check the results.
• Simulate agents locally: drive multi-turn conversations from the same rows.

Deployment Cluster (`docs/deployment/cluster.mdx`) Copy Section

title: Cluster Copy Chapter

Cluster runs Latitude on Kubernetes through a cloud-agnostic Helm chart: a Deployment per application service, a one-shot migrations job, ingress, and bundled infrastructure (Postgres, ClickHouse, Redis, Temporal, SeaweedFS) — every piece replaceable by a managed equivalent. It deploys the published Docker images.

Quick start Copy Chapter

```bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/latitude-dev/latitude-llm.git
cd latitude-llm
git sparse-checkout set charts/latitude
cd charts/latitude
```

```yaml my-values.yaml
config:
  webUrl: https://latitude.example.com
  apiUrl: https://api.latitude.example.com
  ingestUrl: https://ingest.latitude.example.com
  extraEnv:
    # Email transport — required for magic-link sign-in (choose one provider)
    - name: LAT_SMTP_HOST
      value: smtp.example.com
    - name: LAT_SMTP_PORT
      value: "587"
    - name: LAT_SMTP_USER
      value: apikey
    - name: LAT_SMTP_FROM
      value: [email protected]

secrets:
  masterEncryptionKey: # openssl rand -hex 32
  betterAuthSecret: # openssl rand -hex 32
  # Optional secret env vars (email passwords, AI provider keys, OAuth
  # client secrets) — delivered through the chart-managed Secret.
  extra:
    LAT_SMTP_PASS: your-smtp-password

postgres:
  auth:
    password: # openssl rand -hex 16
    runtimePassword: # openssl rand -hex 16

clickhouse:
  auth:
    password: # openssl rand -hex 16

ingress:
  className: nginx
  # tls: [...]
```

<Warning>
  The secrets are permanent for the installation — keep this file (or move the keys into a pre-created Secret and set `secrets.existingSecret`). Every future `helm upgrade` must receive the same values.
</Warning>

```bash
helm install latitude . \
  --namespace latitude --create-namespace \
  --values my-values.yaml --timeout 15m
```

```bash
kubectl get pods -n latitude
kubectl get jobs -n latitude
```

What gets deployed Copy Chapter

Secrets Copy Chapter

Optional non-secret settings (email transport, LAT_AI_* model selection, integrations) go in config.extraEnv as standard EnvVar entries; optional secret values (AI provider keys, email passwords, OAuth client secrets) go in secrets.extra, which merges them into the chart-managed Secret — never as extraEnv literals, which would sit readable in the pod spec.

To manage secrets yourself (e.g. with an external secrets operator), create a Secret carrying the chart's expected keys and set secrets.existingSecret — the chart then renders no Secret of its own. The key list is in the chart README.

See the secrets reference for what each application secret does.

AI features Copy Chapter

The stack boots and core observability (ingest + trace viewing) works without API keys. AI-dependent features stay disabled until you add credentials for the providers you configure — keys in secrets.extra, LAT_AI_* provider/model selection in config.extraEnv:

Every provider and model is selectable per feature through environment variables — see the AI configuration reference.

Scaling Copy Chapter

The five application services are stateless and scale horizontally:

helm upgrade latitude . --values my-values.yaml --set api.replicas=3
# or imperatively:
kubectl scale deploy/latitude-api -n latitude --replicas=3

Every service ships with resource requests (HPA-ready); add limits and HorizontalPodAutoscalers to taste. The bundled stateful services stay single-instance by design — for HA move them to managed equivalents (Bring your own infrastructure).

Retention Copy Chapter

Telemetry is kept for config.telemetryRetentionDays (default 3650, ten years), stamped on each row as it is ingested. With LAT_BILLING_ENABLED off, nothing is enforced: ingestion is never rejected for billing and no plan shortens retention, though per-organization usage is still counted. Changing the value applies to newly ingested data — see widening retention on existing data.

Backups Copy Chapter

Persistent state lives in the PVCs of postgres, clickhouse, redis-bullmq, and seaweedfs. Use your storage provider's volume snapshots, plus:

• Postgres — kubectl exec -n latitude latitude-postgres-0 -- pg_dump -U latitude latitude (the source of truth for projects, users, and metadata).
• ClickHouse — volume snapshots or ClickHouse BACKUP (span/telemetry data).
• Object store — volume snapshots (or rely on your managed S3's durability).

Upgrading Copy Chapter

The chart version tracks the Latitude release, and the images default to the chart's appVersion — so the chart always deploys the release it shipped with. To upgrade, pull the new chart and run:

helm upgrade latitude . --values my-values.yaml

To deviate from the chart's pinned release, set image.tag (X.Y.Z with no leading v, or latest). Migrations run as a pre-upgrade hook — the schema is upgraded before the new pods roll. Latitude migrations are backward-compatible, so the previous release keeps working during the rollout.

Health and observability Copy Chapter

Every service exposes an HTTP health endpoint, wired as readiness and liveness probes (/health; the web app uses /api/health; workers/workflows listen on internal ports 9090/9091). OpenTelemetry export is available through the LAT_OBSERVABILITY_* variables via config.extraEnv.

TLS and reverse proxy Copy Chapter

The chart's Ingress routes the three public hosts (taken from config.{web,api,ingest}Url) to the web, api, and ingest Services over plain HTTP inside the cluster — terminate TLS at your ingress controller. Reference your certificate Secrets in ingress.tls, or have cert-manager issue them via ingress.annotations:

ingress:
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
  tls:
    - secretName: latitude-tls
      hosts:
        - latitude.example.com
        - api.latitude.example.com
        - ingest.latitude.example.com

Keep the config.*Url values on https:// — they feed the LAT_*_URL and CORS/trusted-origin settings.

Custom domain Copy Chapter

Latitude exposes three public surfaces — the web UI, the API (which also hosts the MCP server), and ingest (OTLP traces). You set them through config.webUrl, config.apiUrl, and config.ingestUrl; the Ingress derives its three hosts from those same values, and the chart wires the rest of the URL contract for you — LAT_TRUSTED_ORIGINS and LAT_CORS_ALLOWED_ORIGINS are derived from config.webUrl.

So a standard three-host setup needs nothing beyond getting those three URLs right and pointing DNS at the Ingress:

config:
  webUrl: https://app.your-domain.com
  apiUrl: https://api.your-domain.com
  ingestUrl: https://ingest.your-domain.com

Each must be the real, externally-resolvable https:// URL (scheme included): the chart feeds them into the in-cluster LAT_*_URL, CORS, and trusted-origin settings and routes the matching Ingress host to each Service. The LAT_*_PORT values stay cluster-internal — the Ingress maps your hostnames to them, so you leave them unchanged. Add the certificates as shown under TLS and reverse proxy.

If you serve the web UI from more than one origin (e.g. a vanity host alongside the canonical one), override the allowlists explicitly through config.extraEnv:

config:
  extraEnv:
    - name: LAT_TRUSTED_ORIGINS
      value: https://app.your-domain.com,https://latitude.your-domain.com
    - name: LAT_CORS_ALLOWED_ORIGINS
      value: https://app.your-domain.com,https://latitude.your-domain.com

Bring your own infrastructure Copy Chapter

Every bundled dependency is an independent toggle: set <dep>.enabled: false and fill its external: block, and the chart wires the right LAT_* configuration at your existing or managed instance instead of deploying the bundle.

postgres:
  enabled: false
  external:
    databaseUrl: postgres://latitude_app:...@my-rds:5432/latitude
    adminDatabaseUrl: postgres://latitude:...@my-rds:5432/latitude

temporal:
  enabled: false
  external:
    address: my-namespace.a1b2c.tmprl.cloud:7233
    namespace: my-namespace.a1b2c
    apiKey: ...

Troubleshooting Copy Chapter

Next steps Copy Chapter

Deployment Configuration (`docs/deployment/configuration.mdx`) Copy Section

title: Configuration Copy Chapter

Every application variable is prefixed with LAT_*. A few infrastructure variables (POSTGRES_*, CLICKHOUSE_*) are read directly by the bundled database containers. The committed .env.example is the canonical starting point — it documents both local development and production: copy it to .env.production and apply the values marked # For production on each line.

Each variable is either required (the deployment won't boot or function without it) or optional — it has a safe default, or it unlocks a feature that stays disabled until you set it (SSO, AI, email).

Secrets Copy Chapter

Latitude needs two secrets, generated uniquely per deployment — never reuse the example values:

Generate each with:

openssl rand -hex 32

Moreover, remember to change the default infrastructure passwords (POSTGRES_PASSWORD_*, CLICKHOUSE_PASSWORD...) too!

Email Copy Chapter

A working email transport is required for anyone to log in. Local development captures mail in Mailpit; production is not bundled with an email service — configure exactly one provider.

AI Copy Chapter

The stack boots and core observability (ingest + trace viewing) works without API keys — AI-dependent features stay disabled, or fail, until you add the relevant provider key:

Every internal AI capability is configurable through LAT_AI_* environment variables. With nothing set, the defaults uses Amazon Bedrock for generation and Voyage AI for embeddings and reranking.

Providers Copy Section

You have to set the credentials for whichever providers you end up using:

Generation Copy Section

Each feature resolves every setting through a three-tier fallback:

1. Per-feature variable LAT_AI_<FEATURE>_<SETTING>
2. Global variable LAT_AI_GENERATION_<SETTING>
3. Built-in default minimax.minimax-m2.5

Available <SETTING>:

• PROVIDER
• MODEL
• REASONING (none/default/minimal/low/medium/high/xhigh)
• TEMPERATURE
• MAX_TOKENS

Embeddings Copy Section

Embeddings are configured globally through LAT_AI_EMBEDDING_PROVIDER and LAT_AI_EMBEDDING_MODEL. It defaults to voyage / voyage-4-large, and powers semantic trace/signal search, search highlights, and signal clustering. Without a working embeddings provider, search falls back to lexical.

Once an embedding model is used on a live deployment, changing it is a difficult task. Different models produce incompatible vector spaces and existing data is never re-embedded, so switching breaks semantic search, signal matching, and clustering.

The model must also emit 2048-dimensional vectors — the dimension is fixed by the database schema and is not configurable.

Reranking Copy Section

Reranking is configured globally through LAT_AI_RERANKING_PROVIDER and LAT_AI_RERANKING_MODEL. It defaults to voyage / rerank-2.5, and refines signal-discovery candidate matching. Reranking is optional — when unavailable, discovery falls back to embedding-order matching.

SSO Copy Chapter

By default users sign in with email magic links. Add an OAuth provider to enable social sign-in:

Integrations Copy Chapter

Optional, and cleanly no-op when unset:

Infrastructure Copy Chapter

These wire Latitude to its datastores, depending on your chosen deployment option you can either point them to the bundled infrastructure services or bring your own infrastructure.

Application Copy Section

Postgres Copy Section

Primary store for projects, users, and metadata. Requires the vector (pgvector) extension.

ClickHouse Copy Section

OLAP store for spans and telemetry, in a dedicated database.

Redis Copy Section

Two Redis roles — a cache and the BullMQ job queue (same server or separate). Latitude namespaces all keys under latitude:, so it can share an instance with other apps.

Temporal Copy Section

Workflow engine for durable background processes.

Object storage Copy Section

Stores ingested payloads, datasets, and exports — shared by all services. Pick one of two drivers with LAT_STORAGE_DRIVER: fs writes to the local filesystem (a path or volume shared by every service), or s3 points at any S3-compatible service — AWS S3, SeaweedFS, MinIO, or any other managed or self-hosted store.

Deployment Single Host (`docs/deployment/single-host.mdx`) Copy Section

title: Single-host Copy Chapter

Single-host runs the entire Latitude product on one machine: all five application services, a one-shot migrations job, and bundled infrastructure (Postgres, ClickHouse, Redis, Temporal, SeaweedFS). It uses the published Docker images and a single .env.production, all defined in docker-stack.yml.

Quick start Copy Chapter

```bash
base=https://raw.githubusercontent.com/latitude-dev/latitude-llm/development
curl -fsSL --create-dirs "$base/docker-stack.yml"              -o docker-stack.yml
curl -fsSL --create-dirs "$base/.env.example"                  -o .env.example
curl -fsSL --create-dirs "$base/docker/init-db.sh"             -o docker/init-db.sh
curl -fsSL --create-dirs "$base/docker/seaweedfs/init.sh"      -o docker/seaweedfs/init.sh
curl -fsSL --create-dirs "$base/docker/clickhouse/storage.xml" -o docker/clickhouse/storage.xml
```

Give permissions to the initialization scripts for postgres and seaweedfs:

```bash
chmod +x docker/init-db.sh docker/seaweedfs/init.sh
```

```bash
cp .env.example .env.production
```

```bash
echo "LAT_MASTER_ENCRYPTION_KEY=$(openssl rand -hex 32)"
echo "LAT_BETTER_AUTH_SECRET=$(openssl rand -hex 32)"
```

Moreover, remember to change the default infrastructure passwords (`POSTGRES_PASSWORD_*`, `CLICKHOUSE_PASSWORD`...) too!

The `migrations` container runs first and the application services wait for it to finish.

<Tip>
  To pin a specific release instead of `latest`, set `LAT_IMAGE_TAG=X.Y.Z` in `.env.production` (image tags have no leading `v`).
</Tip>

What gets deployed Copy Chapter

Secrets Copy Chapter

All configuration, secrets included, lives in .env.production. Keep .env.production out of version control and back it up with your other operational secrets.

See the secrets reference for what each application secret does.

AI features Copy Chapter

The stack boots and core observability (ingest + trace viewing) works without API keys. AI-dependent features stay disabled until you add credentials for the providers you configure:

Every provider and model is selectable per feature through environment variables — see the AI configuration reference.

Scaling Copy Chapter

docker-stack.yml also runs under Docker Swarm (docker swarm init, then docker stack deploy -c docker-stack.yml latitude), which is the better fit for scaling. The stateless application services — web, api, ingest, workers, workflows — scale horizontally: raise their deploy.replicas in the file (or docker service scale latitude_api=3), and Swarm load-balances across replicas, including across extra machines joined with docker swarm join.

The bundled stateful services (Postgres, ClickHouse, Redis, Temporal, SeaweedFS) stay single-instance — for HA, point them at managed/external instances (Bring your own infrastructure) or move to the Cluster deployment. On plain Compose (one host), the services with no published ports — workers and workflows — can still scale directly: docker compose -f docker-stack.yml up -d --scale workers=3.

Retention Copy Chapter

A self-hosted deployment keeps telemetry for LAT_TELEMETRY_RETENTION_DAYS (default 3650, ten years). With LAT_BILLING_ENABLED off, no ingest request is ever rejected for billing and no plan shortens retention. Usage is still counted per organization, so the credit figures on the billing page stay populated; nothing enforces them.

Retention is stamped on each row as it is ingested, so lowering or raising the value applies to new data only. To widen the window on data you have already ingested, update the stamp in ClickHouse:

ALTER TABLE spans UPDATE retention_days = 3650 WHERE retention_days < 3650;
ALTER TABLE traces UPDATE retention_days = 3650 WHERE retention_days < 3650;
ALTER TABLE sessions UPDATE retention_days = 3650 WHERE retention_days < 3650;
ALTER TABLE trace_search_documents UPDATE retention_days = 3650 WHERE retention_days < 3650;
ALTER TABLE trace_message_occurrences UPDATE retention_days = 3650 WHERE retention_days < 3650;

ClickHouse deletes a row retention_days + 30 days after its start time, so the extra month is your window to notice and correct a retention setting before anything is removed.

Backups Copy Chapter

Persisted state lives in the postgres_data, clickhouse_data, seaweedfs_data, and redis_bullmq_data volumes. Back them up regularly:

• Postgres — docker compose -f docker-stack.yml exec postgres pg_dump -U "$POSTGRES_USER" "$POSTGRES_DB" (the source of truth for projects, users, and metadata).
• ClickHouse — back up clickhouse_data or use ClickHouse BACKUP (span/telemetry data).
• Object store — back up seaweedfs_data (or rely on your managed S3's durability).

Upgrading Copy Chapter

Pin a specific X.Y.Z tag for Latitude's images in production so upgrades are deliberate. For easiness, you can bump LAT_IMAGE_TAG to a new release (or use :latest) in .env.production. Then pull the new images and restart the services.

docker compose --env-file .env.production -f docker-stack.yml pull
docker compose --env-file .env.production -f docker-stack.yml up -d

The migrations container runs automatically on every up, applying any new Postgres and ClickHouse migrations before the application services start.

Health and observability Copy Chapter

Every service exposes an HTTP health endpoint, wired as the container healthchecks in docker-stack.yml (/health; the web app uses /api/health; workers/workflows listen on internal ports 9090/9091). Check them with docker compose -f docker-stack.yml ps — every service should report healthy. OpenTelemetry export is available through the LAT_OBSERVABILITY_* variables in .env.production.

TLS and reverse proxy Copy Chapter

docker-stack.yml publishes web (3000), api (3001), and ingest (3002) on the host over plain HTTP. For anything internet-facing, put a TLS-terminating reverse proxy (Caddy, nginx, Traefik) in front, route your domain(s) to those ports, and set the LAT_*_URL / *_ORIGINS values to the public https:// URLs.

Custom domain Copy Chapter

Latitude exposes three public surfaces, each typically on its own hostname behind your reverse proxy: the web UI, the API (which also hosts the MCP server), and ingest (OTLP traces). .env.example ships these pointing at localhost; for a real domain you must update the public URLs and the origin allowlists, so that browsers, the API, and MCP clients all agree on where each service lives.

In .env.production:

# Public URLs — what users, SDKs, and MCP clients connect to
# (https, terminated by your reverse proxy)
LAT_WEB_URL=https://app.your-domain.com
LAT_API_URL=https://api.your-domain.com
LAT_INGEST_URL=https://ingest.your-domain.com

# Origin allowlists — must contain your web origin, or browser calls to the API
# are rejected and sign-in fails. Comma-separate to allow several origins.
LAT_TRUSTED_ORIGINS=https://app.your-domain.com
LAT_CORS_ALLOWED_ORIGINS=https://app.your-domain.com

Then restart so the services pick up the new values:

docker compose --env-file .env.production -f docker-stack.yml up -d

The hostnames are yours to pick — three subdomains (above), three separate domains, or one domain with path routing — as long as each LAT_*_URL exactly matches (scheme included) the URL your reverse proxy serves, and the proxy forwards each host to the matching service port. Leave the LAT_*_PORT values as their defaults (3000 / 3001 / 3002): they're the ports each service listens on internally, not part of the public hostname.

Bring your own infrastructure Copy Chapter

Each infra service in docker-stack.yml is a clearly-marked, independently removable block. To use an existing or managed datastore, comment out its block and point the matching LAT_* value at your instance:

Next steps Copy Chapter

Development Contributing (`docs/development/contributing.mdx`) Copy Section

title: Contributing Copy Chapter

Latitude is open source under the MIT License, and contributions are welcome — code, docs, bug reports, and feature ideas alike.

Ways to contribute Copy Chapter

• Pick up a good first issue — curated, newcomer-friendly tasks.
• Open and vote on issues.
• Improve the docs.

For typos, small docs fixes, and clearly-scoped bugs, just open a PR. For new features or anything significant, open an issue first so we can discuss the approach — undiscussed changes may be rejected.

If you like the project but don't have time to contribute code: star the repo, share Latitude with people who'd find it useful or mention it at meetups or in your project's README.

Reporting issues Copy Chapter

Search existing issues first. A good bug report has expected vs. actual behavior, exact repro steps, and your environment. Found a security vulnerability? Don't open a public issue — check out the Security Policy.

Opening a pull request Copy Chapter

We're a small team, we read everything but may take a few days, longer for big changes. Stale or out-of-scope PRs may be closed, but you're welcome to reopen.

Community Copy Chapter

We all hang out in our Slack community — a good place to ask questions and share what you're building.

Development Setup (`docs/development/setup.mdx`) Copy Section

title: Setup Copy Chapter

Infrastructure (Postgres, ClickHouse, Redis, Mailpit, Temporal) runs in Docker, while the five Latitude services run directly on your host for fast hot reload.

Core observability (trace ingest and viewing) works out of the box. AI-dependent features light up as you add the documented provider keys, see AI features.

Prerequisites Copy Chapter

• Node.js 25 and Python 3.13. If you use mise, mise install provisions both.
• pnpm 10. Enable it with corepack enable.
• Docker. Docker Desktop or any Docker Engine, used for the infrastructure containers.
• goose — the ClickHouse migration tool. Install with brew install goose (or see goose releases on other platforms).
• uv — the Python package manager for the evaluation optimizer runtime. Install with brew install uv (or see the uv docs).

Local setup Copy Chapter

This also configures the git pre-commit hooks via the `prepare` script.

```bash
cd packages/platform/op-gepa/python
uv venv && uv sync --all-extras --all-groups
```

Latitude includes a Python sidecar for the evaluation optimizer runtime.

```bash
cp .env.example .env.development
cp .env.example .env.test
```

Then, set `NODE_ENV=test` in `.env.test` (`.env.development` already defaults to `development`). And point `LAT_STORAGE_FS_ROOT` (in both environments) at an **absolute** path for the local object store (the default is `/tmp/latitude`, created automatically).

Some platform packages must be compiled before the database migrations run.

`docker-compose.yml` is for local **infrastructure-only**: Postgres (with `pgvector` extension), ClickHouse, two Redis instances (cache + BullMQ), Mailpit (local SMTP), and self-hosted Temporal instance and its UI.

Runs the Postgres migrations (Drizzle Kit) and the ClickHouse migrations (goose).

Optionally create a sample organization, with a default project, a handful of users, an API key, and realistic sample telemetry across Postgres and ClickHouse — enough to sign in and exercise traces, search, signals, and evaluations immediately.

<CodeGroup>
```bash tmuxinator
# Infra + all services + Postgres studio, each in its own pane.
# Requires tmux and tmuxinator. Runs `docker compose up` for you,
# so you can skip the standalone infra step above.
pnpm tmux
```

```bash turbo
# Runs every workspace `dev` task together.
pnpm dev
```

```bash manual
# Spin up each service separately.
pnpm --filter @app/web dev
pnpm --filter @app/api dev
pnpm --filter @app/ingest dev
pnpm --filter @app/workers dev
pnpm --filter @app/workflows dev
```
</CodeGroup>

Local services Copy Chapter

Once everything is running, these are the local endpoints you can access:

Contributing workflow Copy Chapter

The pre-commit hook runs pnpm check, pnpm typecheck, and pnpm knip. It's configured automatically on pnpm install; to (re)configure it in an existing clone run pnpm prepare.

Before opening a pull requests, run the same checks CI runs:

You can scope any task to a single package with --filter, and pass flags after --:

pnpm --filter @app/api test                          # one package
pnpm --filter @app/api test -- src/some-file.test.ts # one file
pnpm --filter @app/api test -- -t "health endpoint"  # by test name

Check out the full Contributing guidelines.

AI features Copy Chapter

The stack boots and core observability (ingest + trace viewing) works without API keys, and the full test suite is guaranteed to pass with no keys set. However, there are AI-dependent features that you must add the relevant keys to your environment file to enable them:

Every provider and model is selectable per feature through environment variables — see the AI configuration reference.

What's next Copy Chapter

Evaluations Alignment (`docs/evaluations/alignment.md`) Copy Section

title: Evaluation Alignment Copy Chapter

Evaluation Alignment

Alignment measures how closely your automated evaluations match human judgment. It answers a critical question: Can you trust your evaluations?

Why Alignment Matters Copy Chapter

Automated evaluations are only useful if they agree with what a human reviewer would say. Without alignment tracking:

• You don't know if an evaluation is too strict or too lenient
• You can't detect when an evaluation starts disagreeing with reality
• You have no signal for when to recalibrate your evaluation scripts

Alignment gives you that signal. When alignment drops, it's time to review your evaluation scripts and update them based on recent human annotations.

How Alignment Works Copy Chapter

Alignment is computed when both an evaluation and a human annotation have scored the same trace. Latitude compares their pass/fail verdicts and computes metrics from a stored confusion matrix.

The only persisted alignment primitive is the confusion matrix. All derived metrics (MCC, accuracy, F1) are computed from those stored counts on read.

Matthews Correlation Coefficient (MCC) Copy Section

MCC is the primary alignment metric. It's a balanced measure that works well even when pass/fail rates are imbalanced. MCC ranges from -1 to +1:

Confusion Matrix Copy Section

The confusion matrix breaks down agreement into four categories:

• True Positive: Both evaluation and human say "pass"
• True Negative: Both say "fail"
• False Positive: Evaluation says "pass" but human says "fail" (evaluation is too lenient)
• False Negative: Evaluation says "fail" but human says "pass" (evaluation is too strict)

This tells you the direction of disagreement, not just the magnitude. Accuracy and F1 are also derivable from the same counts.

Viewing Alignment Copy Chapter

Each evaluation's detail page shows alignment metrics when annotation data exists. You'll see:

• Current MCC and trend over time
• Confusion matrix for the selected time period
• Last aligned timestamp
• A manual realignment button

Alignment and Evaluation Generation Copy Chapter

When you generate an evaluation from an issue, alignment is core to the generation process:

1. Latitude collects annotation-derived ground truth: at least one finalized, failed annotation linked to the issue (positive examples), plus available negative examples
2. The optimizer generates candidate scripts and evaluates them against this ground truth
3. The best script is selected based on ordered objectives: maximize MCC, then minimize cost, then minimize duration
4. The confusion matrix is stored on the evaluation

There is no minimum negative-example count. A monitor can be created from a single positive occurrence, and its alignment may be weak at first. As users add more annotations, automatic realignment improves the monitor over time.

Automatic Realignment Copy Chapter

Once an evaluation exists, Latitude keeps it calibrated:

• Incremental refresh: When the script hash hasn't changed, new examples are evaluated and added to the existing confusion matrix
• Full re-optimization: When alignment (MCC) degrades beyond a tolerance threshold, the optimizer runs a full pass
• Debounced scheduling: Metric recomputation at most once per hour; full re-optimization at most once every eight hours
• Manual realignment: Available from the evaluation dashboard, rate-limited

Improving Alignment Copy Chapter

When alignment is low:

1. Review the confusion matrix: Is the evaluation too strict or too lenient?
2. Examine false positives and false negatives: Look at specific traces where the evaluation and human disagree. What did the evaluation miss?
3. Add more annotations: More human-reviewed traces give the optimizer better signal for realignment
4. Trigger manual realignment: After adding annotations, use the realignment button to refresh

This iterative calibration process is how you keep automated evaluations trustworthy over time.

Next Steps Copy Chapter

• Annotations: How human review produces the ground truth for alignment
• Flaggers: Automatic annotators that contribute to alignment data
• Search: Build cohorts of traces to annotate
• Issues: How failed evaluations become trackable issues

Evaluations Custom Scripts (`docs/evaluations/custom-scripts.mdx`) Copy Section

title: Custom scripts Copy Chapter

A custom script is a detector you write by hand in JavaScript. It reads one session and returns a score saying how strongly a behavior is present. Reach for it when a set of conditions is too rigid and an LLM judge is too loose, or when you need logic that combines several checks.

Conditions and LLM judges compile down to the same kind of script, so a custom script can express anything they can, plus whatever else you write.

The shape of a script Copy Chapter

Your code runs as the body of an async function, so you can use await at the top level. It must return a score built with Score, Passed, or Failed.

// Match sessions where the assistant said it couldn't help.
const messages = session.conversation
const lastReply = messages.length ? messages[messages.length - 1].content : ''
return lastReply.includes("I can't help") ? Passed() : Failed()

Passed() and Failed() are shorthand for a full match (1) and no match (0). Use Score(value, feedback) when you want a strength in between, or a note explaining the verdict.

Returning a verdict Copy Chapter

A script returns a strength between 0 and 1, not a pass or fail. Latitude compares that strength against a threshold (0.5 by default): at or above it, the behavior counts as present and the session joins the signal.

feedback is optional free text. Latitude stores it on the score and uses it when it groups and displays matches, so a short reason pays off later.

The session object Copy Chapter

Every script receives one global, session: a read-only snapshot of the conversation being checked. It is frozen, so you can read it but not change it.

Numbers are in raw units: durations are nanoseconds, costs are microcents (one US cent is 1,000,000 microcents), and token counts are integers. Most scripts read conversation, tags, metadata, and the tool data rather than the cost and token fields.

Messages Copy Section

session.conversation is the deduplicated transcript for the whole session. Each entry is { role, content }.

const userTurns = session.conversation.filter((m) => m.role === 'user').length
return Score(userTurns > 5 ? 1 : 0, `${userTurns} user turns`)

Interpolating the array into a string renders it as [role] content lines, one per message, which is handy for prompts:

const transcript = `${session.conversation}`
// [user] where is my order?
// [assistant] let me check that for you...

Traces and tools Copy Section

session.traces breaks the session down by trace. Each trace carries its own rollups plus the models, providers, finish reasons, and tool calls it used.

Each tool call has this shape:

// Match sessions where any tool call failed.
const failed = session.traces.some((t) => t.tools.some((tool) => tool.error))
return failed ? Passed() : Failed()

Built-in functions Copy Chapter

Alongside the score helpers, a few functions are always available, and two more appear only when your script calls them.

semanticSimilarity(query) Copy Section

Returns the highest cosine similarity between query and any message in the session, from 0 to 1 (0 when the session has no messages). It is async.

const score = await semanticSimilarity('the user is angry or frustrated')
return score >= 0.55 ? Passed(score) : Failed(score)

Latitude reuses the message embeddings computed when the session was ingested and embeds only your query, so this stays cheap. As a starting point, about 0.4 is broad, 0.55 is balanced, and 0.7 is strict. A session that hasn't been embedded yet is scored once its embeddings are ready.

llm(prompt, options) Copy Section

Sends a prompt to an LLM and returns a structured object matching schema. Latitude manages the model and the system prompt. schema is required and must be built with z (below). It is async.

const result = await llm(
  `Did the assistant promise a refund in this conversation?\n${session.conversation}`,
  { schema: z.object({ promised: z.boolean(), quote: z.string() }) },
)
return result.promised ? Passed(1, result.quote) : Failed()

z Copy Section

A schema builder for shaping llm() output and parse() input. It mirrors a subset of Zod: z.string(), z.number(), z.boolean(), z.literal(), z.enum(), z.array(), z.object(), and z.union(), with .optional(), .nullable(), .describe(), .min(), .max(), and .int().

parse(value, schema) Copy Section

Validates a value against a z schema and returns it, or throws if it doesn't match. Useful for checking JSON your agent produced.

const messages = session.conversation
const output = messages.length ? messages[messages.length - 1].content : ''
try {
  parse(JSON.parse(output), z.object({ answer: z.string(), confidence: z.number() }))
  return Failed(0, 'output matched the expected schema')
} catch {
  return Passed(1, 'output was missing or malformed')
}

What isn't available Copy Section

The sandbox is deliberately small. There is no network access, no fetch, no timers, no Date.now or Math.random, and no Node or browser APIs. The only import allowed is zod. A script's verdict depends only on the session and any llm() call, which keeps results reproducible.

Where scripts run and their limits Copy Chapter

Detectors run in the background, never inside your app's request path. When a session finishes, Latitude runs the matching detectors on it. A preview runs the same way, over recent sessions, on demand.

Latitude picks a resource budget from what your script uses:

Every script gets 64 MiB of memory. These limits are enforced automatically and are generous for typical detectors.

Errors are handled at two points:

• Invalid JavaScript is caught when you save, so a script that won't compile can't be stored.
• A script that throws at runtime, or exceeds its time or memory budget, scores that session as errored. Errored sessions show up in the preview. A detector that errors often is flagged as unhealthy.

LLM and embedding calls cost money, so use the sampling control in the Scope step to check a slice of traffic rather than every session.

Editing and detaching Copy Chapter

If you build a detector with conditions or an LLM judge, the Custom script tab shows the exact script Latitude compiled from your settings, read-only. Choose "Edit as custom script" to take it over by hand. This clears the conditions or criteria form and switches the detector to your script.

It is a one-way move. Once the script is the source of truth, the settings form is gone. Switching between the conditions and judge tabs before you detach keeps both drafts, so detach only when you're ready to hand-write.

Related pages Copy Chapter

• Detection methods: conditions and LLM judges, the other two ways to define a detector
• Create a signal: the full creation flow
• Scores: what a detector produces when it matches
• Sessions: what a session is

Evaluations Detection Methods (`docs/evaluations/detection-methods.mdx`) Copy Section

title: Detection methods Copy Chapter

When you create a signal, you define how Latitude decides whether a session belongs to it. That check is the signal's evaluation, and you can build it three ways.

All three produce the same kind of detector, and a signal has one active detector at a time. You can switch methods while you build or edit a signal.

Set of conditions Copy Chapter

A set of conditions checks concrete facts about a session. Each condition is a small deterministic test. Conditions are free and run instantly, so this is the method to start with.

Add up to 10 conditions and choose how they combine:

• All: every condition must hold (AND).
• Any: at least one condition must hold (OR).

The match selector appears once you have two or more conditions. A single condition just has to hold.

Condition types Copy Section

A few details are worth knowing:

• Where (for text match) picks which messages to look at: the last assistant message, any assistant message, any user message, any tool message, or the whole conversation.
• Comparison operators read as "greater than", "at least", "less than", and "at most".
• Metric works in display units: duration in milliseconds, cost in dollars. The available metrics are duration, cost, total or input or output tokens, error count, trace count, and span count. Aggregation decides whether a metric is summed across the session or compared per trace.
• Tool failed and Error look at the error status of a tool call or trace, not at the content of a tool's output. A tool that returns the word "error" in an otherwise normal result does not count as failed.
• Semantic similarity compares messages to a query using the embeddings Latitude already computed for the session. It is the one condition that isn't instant, because it works on embeddings.

Semantic similarity sensitivity Copy Section

The sensitivity presets map to a similarity threshold:

Broad matches loosely and catches more. Strict matches only close paraphrases. Start with Balanced and adjust from the preview, or set an exact threshold and operator under Advanced.

LLM as judge Copy Chapter

An LLM judge reads each matching session and decides whether the behavior is present, with a short reason. You describe the behavior in plain language in the "A session matches when..." field:

the user got frustrated, repeating themselves, complaining, or giving up before getting a useful answer

Use a judge for behavior that is hard to pin to a fixed rule: tone, frustration, whether the answer actually resolved the request, or whether the model made something up. Latitude writes the prompt and manages the model, so you only supply the criteria.

Each check sends a session to an LLM, which costs money and takes longer than a condition. On high traffic, lower the sampling rate in the Scope step so a judge checks a representative slice instead of every session.

Custom script Copy Chapter

A custom script is JavaScript you write that reads a session and returns a score. It can express anything conditions and judges can, plus logic they can't, such as combining several checks or inspecting tool arguments.

The Custom scripts page documents the full API: the session object, the built-in functions, and the limits scripts run under.

Choosing a method Copy Chapter

• If the behavior is a concrete fact you can name, use a set of conditions. It is free, fast, and easy to reason about.
• If the behavior is fuzzy or semantic, use an LLM judge.
• If you need logic the builder can't express, write a custom script.

You aren't locked in. Editing a signal lets you switch methods, and the Custom script tab always shows the script your conditions or criteria compiled to, so you can start simple and take over by hand later.

Related pages Copy Chapter

• Create a signal: where you pick a method and scope the detector
• Custom scripts: the full scripting reference
• Triggers: scope and sampling for a detector
• Evaluations overview: how detectors fit the wider evaluation model

Evaluations Triggers (`docs/evaluations/triggers.md`) Copy Section

title: Evaluation Triggers Copy Chapter

Evaluation Triggers

Every evaluation has a trigger configuration that determines which traces it evaluates and how. Triggers give you precise control over what an evaluation monitors without modifying the evaluation script itself.

How Triggers Work Copy Chapter

When a trace completes (after a debounce window with no new spans), Latitude checks it against every active evaluation's trigger configuration. Trigger checks are evaluated in a specific order:

1. Filter: Does the trace match the evaluation's filter criteria?
2. Sampling: Does it pass the sample rate check?
3. Turn / Debounce: Which turn does the evaluation target, and should execution be debounced?

If a trace passes all checks, the evaluation runs. If it fails at any stage, it's skipped.

Triggers use the same shared filter system as the trace view and saved searches. The filtering capabilities you see in the trace dashboard or in the Search page are the same ones available for evaluation triggers.

Trigger Fields Copy Chapter

Filter Copy Section

Select which traces the evaluation monitors using any combination of the shared filters:

• Status: Only evaluate traces with errors, or only successful traces
• Models: Only evaluate traces that used specific models
• Providers: Only evaluate traces from specific providers
• Tags: Only evaluate traces with specific tags
• Cost: Only evaluate traces above or below a cost threshold
• Duration: Only evaluate traces above or below a duration threshold
• Custom metadata: Filter on any metadata.* fields your application sends

An empty filter means "match all traces."

Sampling Copy Section

The percentage of matching traces that the evaluation actually runs against, from 0 to 100. This controls cost and processing time while still giving you statistical coverage.

• Setting sampling to 0 effectively pauses the evaluation.
• New evaluations generated from issues default to 10% sampling.

Turn Copy Section

Controls which trace or turn the evaluation runs on:

• every: Run on every completed trace (the default)
• first: Run only on the first trace/turn in a session
• last: Run only on the last trace/turn in a session

This is useful when your evaluation only makes sense at the start or end of a conversation.

Debounce Copy Section

A debounce time in seconds. When set, the evaluation waits for the debounce period after the trace completes before executing. This is useful for batching or rate-limiting evaluation execution.

Trigger Examples Copy Chapter

Monitor all production traces for jailbreak attempts:

• Filter: metadata environment = "production"
• Sampling: 100%
• Turn: every
• Debounce: 0

Spot-check expensive traces for quality:

• Filter: cost > $0.50
• Sampling: 25%
• Turn: every
• Debounce: 0

Evaluate only the last turn of each session:

• Filter: (empty: match all)
• Sampling: 10%
• Turn: last
• Debounce: 0

Triggers, Search, and Annotations Copy Chapter

Triggers, search, and annotations form a feedback loop:

1. An evaluation monitors traces with a broad trigger
2. Failed scores feed into issue discovery
3. A saved search scopes a reviewer to the failing cohort
4. Reviewers leave inline annotations on those traces
5. Human annotations measure alignment with the evaluation

Triggers determine the scope of automated monitoring; saved searches and annotations determine the scope of human oversight. Flaggers handle the same loop fully automatically for a fixed list of well-known categories.

Next Steps Copy Chapter

• Alignment: How human annotations calibrate evaluations
• Evaluations Overview: How evaluation scripts work
• Annotations: The human side of the feedback loop
• Search: Build cohorts of traces to review

Getting Started Cli (`docs/getting-started/cli.mdx`) Copy Section

title: CLI Copy Chapter

Overview Copy Chapter

The latitude CLI is a single, self-contained binary that exposes your Latitude organization on the command line. Like the MCP server, its commands are generated directly from the Latitude API, so the command surface automatically stays in sync with the platform. For the live list of commands and their input/output schemas, check the API reference.

It's built for two audiences:

• Humans, a fast, scriptable way to inspect and manage projects, traces, datasets, members, keys, and more without leaving the terminal.

• Agents, a zero-dependency way for an agent (Claude Code, Cursor, Codex, …) to act on Latitude without wiring up an MCP connection, using --format json for machine-readable output and --schema for machine-readable help.

  Prefer a network connection? The MCP server exposes the same capabilities over OAuth.

Installation Copy Chapter

The CLI ships as a pre-built binary on our GitHub Releases — grab the latest cli-vX.Y.Z release. Download the archive for your platform, extract it, and put the latitude binary somewhere on your PATH.

tar -xzf latitude.tar.gz
chmod +x latitude
sudo mv latitude /usr/local/bin/latitude   # or any directory on your PATH

latitude --version
```


  On MacOS the first run may be blocked by Gatekeeper. Allow it with `xattr -d com.apple.quarantine /usr/local/bin/latitude`, or via **System Settings → Privacy & Security**.

```powershell
latitude --version
```

Then verify it works and, optionally, set up shell completion and the man page.

Authentication Copy Chapter

The CLI authenticates with an organization-scoped API key. Create one in the Latitude UI under Settings → Keys → API Keys. There are two ways to give the key to the CLI:

```bash
latitude auth login --with-token
```

Paste your API key when prompted and press Enter. It's saved to the keyring under `latitude:ApiKeyAuth` and reused on every subsequent command. Manage it with:

```bash
latitude auth status    # show which credential sources are set, for each scheme
latitude auth logout    # remove the stored key from the keyring
```

Need more than one key, like one for production and one for your sandbox? Use [profiles](#profiles).

```bash
export LATITUDE_API_KEY="lat_..."
latitude projects list
```

`LATITUDE_API_KEY` takes precedence over the keyring, so it's easy to override the stored credential per-session. `latitude auth status` shows every source the CLI can see and which one wins.

The CLI also reads a .env file from the current directory, or from the nearest parent directory that has one. The LATITUDE_API_KEY you keep there for tracing authenticates the CLI too. Variables exported in your shell win over .env. Settings that change where requests go or how TLS is checked (LATITUDE_BASE_URL, the proxy and CA bundle variables, LATITUDE_INSECURE) are ignored when they come from .env, so export those from your shell instead.

Profiles Copy Chapter

A profile is a named setup for the CLI with its own API key, stored in the keyring. With profiles you can keep several keys side by side and choose one per command, per shell, or as your default, instead of re-exporting LATITUDE_API_KEY every time you switch.

Production and sandbox Copy Section

The most common use is switching between your project's live data and its sandbox. Both use the same project slugs. The API key alone decides which one a command reads: a live key from Settings → Keys → API Keys, or a lat_sandbox_ key from Sandbox configuration inside the sandbox. Give each key its own profile:

# Production: create the profile, make it the default, then paste your live key
latitude profiles create production --use
latitude auth login --with-token -p production

# Sandbox: create the profile, then paste your lat_sandbox_ key
latitude profiles create sandbox
latitude auth login --with-token -p sandbox

Commands now run against production by default. Add -p sandbox (or --profile sandbox) to run one against the sandbox. The project slug is the same in both, so set it once:

export LATITUDE_PROJECT_SLUG=my-app

latitude traces list                                 # production traces of my-app
latitude traces list -p sandbox                      # sandbox traces of my-app
latitude traces get --trace-id <traceId> -p sandbox  # one sandbox trace

To change the default instead of passing -p each time:

latitude profiles use sandbox       # sandbox is now the default
latitude profiles use production    # and back
export LATITUDE_PROFILE=sandbox     # or pick a profile for the current shell only

Managing profiles Copy Section

latitude profiles list                        # every profile, with the default marked
latitude profiles current                     # the profile this command would use, and why
latitude profiles show sandbox                # one profile's settings
latitude auth login --with-token -p sandbox   # replace the sandbox key
latitude profiles remove sandbox              # delete the profile and its stored key
latitude profiles use -                       # clear the default profile

Profiles live in a profiles.toml file in the CLI's config directory: ~/Library/Application Support/latitude/ on MacOS, ~/.config/latitude/ on Linux, and %APPDATA%\latitude\ on Windows. The file never holds a key. Each key stays in the keyring under latitude:ApiKeyAuth#.

A profile can also carry a default project, which helps when your profiles point at different projects:

latitude profiles create support-bot --project-slug support-bot

A profile's project wins over LATITUDE_PROJECT_SLUG and --global-project-slug. Passing --project-slug on a command still overrides it.

Which key a command uses Copy Section

The CLI uses the first of these that's set:

1. The profile passed with -p / --profile.
2. LATITUDE_API_KEY, from your shell or a .env file.
3. The profile named by LATITUDE_PROFILE, or else the default set with latitude profiles use.
4. The key stored by latitude auth login with no profile selected.

A selected profile only ever sends its own key. If it has none stored, the command fails to authenticate instead of falling back to another key.

Usage Copy Chapter

The general shape is latitude <resource> <command> [flags], where each resource mirrors an area of the Latitude API:

latitude --help                 # list every resource
latitude projects --help        # list the commands for a resource
latitude projects list          # run one

latitude projects create --help # discover a command's flags/arguments

Resources include projects, traces, datasets, and more — run latitude --help for the full list. Common examples:

latitude account get                       # the current org + your role
latitude usage get                         # credits used this period, by area and project
latitude projects usage --project-slug X   # the same, for one project
latitude projects list                     # every project in the org
latitude projects create --name "My App"   # create a project
latitude traces list --help                # inspect traces (project-scoped — see --help for the flags)
latitude api-keys list

Most resources are project-scoped and take --project-slug. Set the project once for a session with LATITUDE_PROJECT_SLUG (or the global --global-project-slug flag) and you can drop it from individual commands — --project-slug still wins wherever you pass it:

export LATITUDE_PROJECT_SLUG=my-app
latitude traces list                       # uses my-app
latitude traces list --project-slug other  # uses other
latitude projects list                     # unaffected — not project-scoped

A profile can carry its own default project as well.

A few global flags worth knowing (run latitude --help for the full list):

• --profile <name> / -p — run the command under a named profile.
• --query <JMESPath> — project/filter the response before it's formatted, e.g. --query "items[].slug". When an API operation has its own query parameter (like the free-text search on traces list), that parameter is exposed as --query-param instead.
• --dry-run — validate the request locally without sending it to the API.
• --quiet / -q — suppress success output (errors still print to stderr).
• --human — force table output even when stdout is piped.
• --schema — print a machine-readable JSON description of a command's inputs and outputs, the agent-facing counterpart to --help.
• --debug — dump the raw HTTP request and response to stderr.

API commands also take a few request options:

• --params <JSON> — pass parameters as a JSON object. These override the individual flags.
• --json <JSON|-> — on commands that send a request body, pass the whole body as JSON. Use - to read it from stdin.
• --retries <N> / --no-retry — change how many times a failed request is retried, or turn retries off.

Output formats Copy Chapter

Every command accepts --format to control how results are rendered. This makes the CLI equally good for humans reading a terminal and for scripts or agents parsing output.

latitude projects list --format json
latitude projects list --format csv > projects.csv
latitude traces list --format jsonl | jq .   # stream trace-by-trace

# Combine with --query to reshape before formatting
latitude projects list --query "items[].{slug: slug, name: name}" --format yaml

You can also set a default format for a session with the LATITUDE_OUTPUT environment variable (e.g. export LATITUDE_OUTPUT=json); the --format flag always overrides it. --human is shorthand for --format table, for when you pipe output but still want the table.

Environment variables Copy Chapter

The standard HTTPS_PROXY / HTTP_PROXY / NO_PROXY / SSL_CERT_FILE variables are honored as well. Any of these can also go in a .env file, except LATITUDE_BASE_URL and the proxy, CA bundle, and TLS variables, which the CLI only reads from your shell (see Authentication).

Shell completion Copy Chapter

Generate a completion script for your shell — bash, zsh, fish, powershell, or elvish — and load it to get tab-completion for every resource, command, and flag:

Man page Copy Chapter

The CLI can emit its own manual page in roff format, so man latitude works like any native tool:

latitude man > latitude.1
man ./latitude.1
# or install it system-wide:
sudo mv latitude.1 /usr/local/share/man/man1/ && man latitude

For Agents Copy Chapter

The CLI pairs naturally with agents: --format json/jsonl for structured output, --schema for machine-readable help, and latitude generate-skills to teach an agent the command surface — everything it needs to drive Latitude locally with no MCP connection. Have agents pass -p on every command when you use profiles, so each call names its environment and can't be redirected by a LATITUDE_API_KEY in the environment. Prefer the MCP server when you'd rather connect the agent over the network with OAuth.

Getting Started Concepts (`docs/getting-started/concepts.mdx`) Copy Section

title: Core Concepts Copy Chapter

Core Concepts

Latitude organizes your work into a hierarchy of organizations and projects, with all reliability features scoped to the right level.

Organizations Copy Chapter

An organization is the top-level ownership boundary. It represents your team or company.

Organizations own:

• Members: Team members who can access the organization's projects
• API keys: Used for programmatic access from your applications
• Settings: Organization-wide defaults, including monitoring behavior

All data in Latitude is organization-scoped. Members of one organization cannot see another organization's data.

Projects Copy Chapter

A project is the main product boundary for reliability workflows. Each project typically corresponds to one agent, application, or feature you want to monitor.

Everything you work with day-to-day is project-scoped:

• Traces and sessions: Your agent's captured interactions
• Search and saved searches: Find traces by meaning and bookmark useful cohorts
• Flaggers: Automatic annotators that surface common failure categories
• Evaluations: Automated monitoring scripts
• Issues: Discovered failure patterns
• Simulations: Pre-ship test runs
• Scores: All verdicts from evaluations, annotations, and custom sources

When you create a project, Latitude provisions a full set of flaggers — automatic annotators for common failure categories like jailbreaking, frustration, and tool call errors. They start running on every completed trace right away, so you get useful signal before configuring anything else.

Project Settings Copy Section

Projects can override organization-level defaults:

• Keep monitoring: Controls whether issue-linked evaluations stay active after an issue is resolved. When set at the project level, it overrides the organization default. When a user resolves an issue, this setting determines the default state of the "keep monitoring" toggle in the confirmation dialog.

How Settings Cascade Copy Chapter

Settings resolve from the most specific scope to the broadest:

1. Project settings: Checked first
2. Organization settings: Used as fallback when the project doesn't override

For example, if a project has keepMonitoring set to true, that applies regardless of the organization setting. If the project doesn't set it, the organization default is used.

Access Model Copy Chapter

• Web users authenticate through browser sessions with an active organization context
• API clients authenticate with API keys scoped to an organization
• Users without an active organization are routed to onboarding

Next Steps Copy Chapter

• Developer Quick Start: Connect your first agent
• No-Code Quick Start: Explore the Latitude UI
• Observability: How telemetry flows into projects

Getting Started How To Use Latitude (`docs/getting-started/how-to-use-latitude.mdx`) Copy Section

title: How to use Latitude Copy Chapter

Latitude works best as a continuous loop for production agents: observe real traffic, understand what is going wrong, and refine your agent until it is fixed and stays fixed. The product is organized around that loop: Observe, Understand, Refine.

The core workflow Copy Chapter

If you have not connected your app yet, follow [Start tracing](../telemetry/start-tracing).

What to focus on first Copy Chapter

If you are setting up Latitude for the first time:

1. Connect tracing for one production agent.
2. Add userId and sessionId so traces group by user and conversation.
3. Search for one failure mode your team already cares about, or browse Behaviors to see what stands out.
4. Annotate representative traces with specific feedback.
5. Watch the Signals page for grouped patterns, and let the built-in monitors notify you.
6. Generate evaluations for the signals you want to score continuously.

Work with agents and self-host Copy Chapter

• MCP: manage your workspace straight from Claude, Cursor, and other agents through the MCP server.
• Self-hosting: run Latitude in your own infrastructure, from a single host to a full cluster.

Why this works Copy Chapter

Latitude does not require you to define every possible failure upfront. You discover failures from real traffic, validate them with human review, and turn important patterns into automated monitoring. Over time, the system becomes a living map of what goes wrong in your agent and whether your fixes are working.

Getting Started Introduction (`docs/getting-started/introduction.mdx`) Copy Section

title: Introduction Copy Chapter

Introduction

Latitude is an open-source agent observability platform. It helps teams turn live agent traffic and human judgment into measurable, improvable quality. So you can ship AI features with confidence.

What Latitude Does Copy Chapter

Latitude gives you a closed-loop system for monitoring, evaluating, and improving AI agents in production:

• Observe: Capture every interaction your agents have with users through automatic telemetry. See spans, traces, and sessions in real time.
• Score: Attach quantitative verdicts to every interaction. Scores come from automated evaluations, human annotations, or your own custom logic.
• Discover: When scores indicate failures, Latitude automatically groups similar problems into issues: named, trackable failure patterns.
• Evaluate: Generate monitoring scripts from discovered issues. These evaluations run continuously on live traffic, catching regressions before users notice.
• Align: Keep your automated evaluations honest by measuring how well they agree with human reviewers. Latitude tracks alignment metrics so you know when machine judgment drifts from human judgment.
• Simulate: Before shipping changes, run your agent against test scenarios locally or in CI. Reuse the same evaluation scripts that monitor production.
• Improve: Use everything you've learned to make your agents better, then repeat.

How It All Connects Copy Chapter

Every feature in Latitude feeds the next. Observability captures interactions. Evaluations score them. Failed scores surface issues. Issues generate new evaluations. Human annotations keep evaluations calibrated. Simulations prevent regressions before they reach production.

The result is a continuous improvement cycle where your agents get better with every interaction.

Key Concepts Copy Chapter

Next Steps Copy Chapter

• New to Latitude? Read Core Concepts to understand organizations, projects, and how everything fits together.
• Developers: Follow the Developer Quick Start to connect your first agent and see traces in Latitude.
• Team leads and PMs: Follow the No-Code Quick Start for a walkthrough of the Latitude web UI.

Getting Started Mcp (`docs/getting-started/mcp.mdx`) Copy Section

title: MCP Server Copy Chapter

Overview Copy Chapter

The Latitude MCP is a remote, OAuth-authenticated, streamable HTTP Model Context Protocol server, living at https://api.latitude.so/v1/mcp.

It is the portal that AI agents — Claude, Cursor, Codex, Gemini, Zed, OpenCode, and others — can use to read and manage your Latitude workspace.

The MCP exposes everything you can do in the Latitude UI as tools — managing projects, members, keys, traces, annotations, scores, searches, issues, datasets, and more.

Tools are dynamically generated from the Latitude API, so the catalog automatically stays in sync with the platform. For the live list of tools, their descriptions, and their input/output schemas, you can check the API reference

To see or revoke connected agents at any time, go to your organization's Settings → Keys and look at the OAuth Keys section. Revoking immediately cuts off the agent's access.

Installation Copy Chapter

The MCP is compatible with all the most common agents, harnesses or IDEs you use. We have built extensions for many of them, otherwise check the Manual section tab with the exact install steps. If your agent isn't listed below, tell your agent to add the Latitude MCP as a Remote Streamable HTTP MCP server with URL https://api.latitude.so/v1/mcp and OAuth enabled — those are the only two things the agent needs to know!

Cursor Desktop Copy Section

Cursor CLI Copy Section

Claude Code Desktop Copy Section

Claude Code CLI Copy Section

Codex Desktop Copy Section

Codex CLI Copy Section

Gemini CLI Copy Section

Google Antigravity Copy Section

Zed Copy Section

OpenCode Desktop Copy Section

OpenCode CLI Copy Section

Or edit `~/.config/opencode/opencode.json` directly:
    ```json
    {
      "mcp": {
        "latitude": {
          "type": "remote",
          "url": "https://api.latitude.so/v1/mcp"
        }
      }
    }
    ```
  </Step>
  <Step title="Verify installation">
    Verify the Latitude MCP was added running:
    ```bash
    opencode mcp list
    ```
  </Step>
  <Step title="Authenticate">
    Run (outside of OpenCode):
    ```bash
    opencode mcp auth latitude
    ```
  </Step>
  <Step title="Authorize">
    Sign in (if needed) and pick the organization you want this OpenCode Agent to access in the consent screen.
  </Step>    
</Steps>

Getting Started Migrate From V1 (`docs/getting-started/migrate-from-v1.mdx`) Copy Section

title: Migrate from Latitude V1 Copy Chapter

Latitude V1 was a prompt engineering platform. You wrote prompts in PromptL, published them, and your app called them through the Latitude gateway. Latitude V2 is an observability and signal-discovery platform for agents you run yourself. Your code calls the model provider directly, and Latitude watches through OpenTelemetry.

This guide is for teams with a V1 workspace who need to move before V1 is sunset. It maps each V1 concept to its V2 equivalent, then walks through a real migration: LinguaAI, a FastAPI language tutor whose two prompts ran through sdk.prompts.run() on V1. The walkthrough drives the control plane from a coding agent over the Latitude MCP, because that is the quickest way to do it. Every MCP call has a REST, CLI, and SDK twin, listed at the end.

Treat V2 onboarding as a fresh start rather than a data migration. Your prompts move into your codebase, your golden datasets move over as CSV, and traces, signals and evaluations are rebuilt from live traffic within days.

What changed Copy Chapter

Latitude is no longer on the critical path of your application. If Latitude is down, your agent still answers. You lose observability for that window and nothing else.

Concept map Copy Chapter

Use this table as the checklist for your own migration.

Before you start Copy Chapter

1. Export your prompts while your V1 workspace is still active. latitude.prompts.getAll() (SDK 5) or GET https://gateway.latitude.so/api/v3/projects/{projectId}/versions/live/documents returns every PromptL document with its content and config. Commit them to your repository; they are the source material for moving the prompts into code.

2. Export every dataset. Download the CSV from the Datasets page, or list them with GET /api/v3/datasets and pull rows with GET /api/v3/dataset-rows?datasetId=...&page=1&pageSize=100. Note which column is the label.

3. List your evaluations per prompt: name, type (judge, rule, human), criteria, and whether it ran live. Each becomes a signal or an annotation habit.

4. Write down your customIdentifier scheme. It usually maps to a session id or a user id.

5. Pick a project boundary. V2 projects are per agent. LinguaAI's grammar checker and quiz writer share one project because they share one product surface and one user base.

6. Connect the MCP to your coding agent. For Claude Code:

   bash Copy

   claude mcp add --transport http latitude https://api.latitude.so/v1/mcp --scope user

   Then type /mcp, pick latitude, authenticate, and choose the organization. Other agents are covered on the MCP page. The MCP is OAuth-only; an API key on that URL gets a 401 by design. Revoke the agent any time under Settings, Keys, OAuth Keys.

7. Point your coding agent at the V2 docs. Older V1 pages survive in search caches, and agents sometimes pull them in. Tell the agent to use https://docs.latitude.so/llms.txt as its index.

The example: LinguaAI Copy Chapter

LinguaAI is a FastAPI service with two endpoints. POST /grammar/check corrects a learner's sentence and POST /quiz/generate writes a three-question vocabulary quiz. On V1 both endpoints called sdk.prompts.run() against PromptL documents named grammar-check and vocab-quiz. The grammar prompt had a live LLM-as-judge evaluation ("Misses a grammar error") and a golden dataset (grammar-golden.csv) used for batch experiments before publishing.

The before and after code is in the examples/migrations/linguaai-v1-to-v2 folder of the Latitude repository.

Create the V2 project Copy Section

Ask your agent:

Create a Latitude project called LinguaAI and give me its slug.

The agent calls createProject with {"name": "LinguaAI"} and gets back the slug linguaai. Slugs are permanent; renaming never changes them. Put the slug in .env as LATITUDE_PROJECT_SLUG. Create an organization API key under Settings, Keys and store it as LATITUDE_API_KEY.

Move the prompts into code Copy Section

The V1 document grammar-check.promptl:

---
provider: anthropic
model: claude-3-5-sonnet-latest
temperature: 0
---
<system>
You are LinguaAI's grammar coach. The learner is studying {{ language }}.
Check the learner's text for grammar errors.
Reply ONLY with JSON of this shape: ...
</system>

<user>
Language: {{ language }}
Text: {{ text }}
</user>

becomes a Python template with a version string (app/prompts.py):

GRAMMAR_CHECK_V2 = """You are LinguaAI's grammar coach. The learner is studying {language}.
Check the learner's text for grammar errors.
Reply ONLY with JSON of this shape: ..."""

GRAMMAR_CHECK_VERSIONS = {
    "2.0.0": ("grammar-check@2", GRAMMAR_CHECK_V2),
    "2.1.0": ("grammar-check@3", GRAMMAR_CHECK_V3),
}

def grammar_check_prompt(language: str) -> tuple[str, str]:
    version, template = GRAMMAR_CHECK_VERSIONS[APP_RELEASE]
    return version, template.format(language=language)

The frontmatter became arguments to the provider call, so check each one against the provider SDK you use. The Anthropic Python SDK 1.x, for example, no longer accepts temperature on messages.create(). {{ language }} became {language}; the <system> block is the system argument and the `` block is the first message. The published-version concept became a version string that travels to Latitude as metadata.prompt_version, which you can filter and compare on.

Replace the gateway call with a direct call plus telemetry Copy Section

V1 (backend/routers/grammar.py):

from latitude_sdk import Latitude, LatitudeOptions, RunPromptOptions

sdk = Latitude(api_key, LatitudeOptions(project_id=int(project_id), version_uuid=version_uuid))

result = await sdk.prompts.run("grammar-check", RunPromptOptions(parameters={"text": req.text, "language": req.language}))
data = _extract_json(result.response.text)

V2 (app/telemetry.py and app/services.py):

# telemetry.py: import this before any Anthropic client exists
import anthropic
from latitude_telemetry import Latitude

latitude = Latitude(
    api_key=os.environ["LATITUDE_API_KEY"],
    project=os.environ["LATITUDE_PROJECT_SLUG"],
    instrumentations={"anthropic": anthropic},
)

# services.py: one capture() per use case
from anthropic import Anthropic
from latitude_telemetry import capture

client = Anthropic()

def check_grammar(text, language, *, user_id, session_id):
    version, system = prompts.grammar_check_prompt(language)
    raw = capture(
        "grammar-check",
        lambda: client.messages.create(
            model=prompts.MODEL, max_tokens=600, system=system,
            messages=[{"role": "user", "content": f"Language: {language}\nText: {text}"}],
        ).content[0].text,
        {
            "user_id": user_id,
            "session_id": session_id,
            "tags": ["grammar", f"release-{prompts.APP_RELEASE}", "production"],
            "metadata": {"release": prompts.APP_RELEASE, "prompt_version": version, "language": language},
        },
    )
    return _extract_json(raw)

What to get right:

• Construct Latitude(...) once, at import time, before the provider client. Pass the same module your code imports (anthropic, not a wrapper around it).
• Wrap the model call in capture(). It creates no spans of its own. It attaches user, session, tags and metadata to the spans the instrumentation creates inside the callback. The route handler reads X-User-Id and X-Session-Id headers and passes them down; the smoke script passes them directly.
• Put the release in a tag as well as in metadata. Tags are the cohort key for session filters and Experiments; metadata holds exact values for search.
• Flush before a short-lived process exits (latitude.flush(), then latitude.shutdown()). Servers export in the background, but scripts, tests and jobs can exit before the last batch ships.

If you would rather have your coding agent do the whole migration, the latitude-migrate skill runs every step in this guide, from exporting your V1 prompts to verifying traces:

Install the latitude-migrate skill from github.com/latitude-dev/skills and use it to move this app from Latitude V1 to V2.

To instrument only, the latitude-telemetry skill does the audit, the plan, and the verification:

Install the latitude-telemetry skill from github.com/latitude-dev/skills and use it to add Latitude tracing to this app.

Run traffic and verify it with the MCP Copy Section

Run the smoke script once per release, so there are two cohorts to compare later:

APP_RELEASE=2.0.0 python -m scripts.smoke
APP_RELEASE=2.1.0 python -m scripts.smoke

Then ask your agent:

List the latest grammar traces in linguaai and open one. Show me the conversation, the metadata, and who the user was.

listTraces with {"filters": {"tags": [{"op": "eq", "value": "grammar"}]}} returns rows with traceId, sessionId, userId, tags, models, tokensInput, tokensOutput, costTotalMicrocents and rootSpanName. getTrace adds metadata and the conversation in OpenTelemetry GenAI format: system instructions, the user message, the assistant reply. listSessions shows each study session with its trace count and cost, and listUsers shows the three learners.

On the first trace, check that metadata.release and metadata.prompt_version carry what you set, and that the conversation renders as system, user, assistant rather than as one blob. If the model and token counts are zero, the instrumentation is not wrapping the client you actually call.

Recreate the evaluation as a signal Copy Section

The V1 live evaluation "Misses a grammar error" becomes a signal with a judge evaluation. Ask your agent:

Create a signal in linguaai called "Missed grammar error" for traces tagged grammar. Use an LLM judge: the behaviour is present when the learner's text has an error that the corrections list does not cover, or when is_correct is true although the text has an error.

createSignal takes the name, a description, a pre-gate filters set, and evaluation.settings of kind judge with the criteria in plain language. Latitude returns the slug (LIN-O84M in our run) and creates the evaluation behind it. For the V1 programmatic rule "output must be valid JSON", use kind rule with a json_output condition instead.

Plan for these differences from V1:

• A signal detector collects forward from creation. It does not scan history, so create signals before you generate the traffic you want judged.
• Judge evaluations default to 10 percent sampling. On a low-traffic demo, open the signal in the UI and set the scope to 100 percent, or it may never fire.
• The criteria text is compiled into the detector. Avoid backticks in it. In our run, a criteria string with backticks was rejected with a syntax error.

You do not have to define every signal yourself. Annotations, flaggers (frustration, refusal, empty output, tool errors) and custom scores feed discovery, which groups repeated failures into named signals on its own.

Bring your golden dataset Copy Section

Two sources feed the V2 dataset: real traces, and the CSV you exported from V1.

Create a dataset called grammar-regressions in linguaai. Import every grammar trace from release 2.0.0 with no errors as rows.

createDataset returns the slug grammar-regressions. importDatasetRowsFromTraces with {"traces": {"by": "filters", "filters": {"tags": [...], "metadata.release": [{"op": "eq", "value": "2.0.0"}], "errorCount": [{"op": "eq", "value": 0}]}}} returned nine row ids. Each row's input is the user message list, output is the assistant reply, and metadata.traceId points back at the trace.

For the V1 CSV, map parameter columns into input and the label into expectedOutput (scripts/import_v1_dataset.py):

rows = [
    {"input": {"text": r["text"], "language": r["language"]},
     "expectedOutput": r["expected_output"],
     "metadata": {"source": "latitude-v1-golden-dataset"}}
    for r in csv.DictReader(fh)
]
client.datasets.insert_rows(project, "grammar-regressions", rows=rows)

That is insertDatasetRows; six rows landed. Then fill expected outputs on the trace-imported rows you want to check precisely:

Set the expected output of the row whose input contains "If I would have known" to "If I had known, I would have come earlier."

updateDatasetRow changes only the cells you send and bumps the dataset version.

Regression test by replaying the dataset Copy Section

V1 ran batch evaluations inside Latitude. V2 hands the dataset to your own runner. tests/test_regression.py pulls every row with the SDK, replays the grammar check, applies the returned corrections to the original text, and compares the result with the expected output. Every replay is tagged simulation and agent-<release> and carries the dataset slug and row id as metadata, so it never counts as production and an Experiment can compare runs.

LATITUDE_DATASET_SLUG=grammar-regressions APP_RELEASE=2.1.0 pytest -q

The first full run failed one row. The V1 label read "Yo estoy muy cansado hoy." and the coach returned "Estoy muy cansado hoy.", which is also correct Spanish. Exact-match labels are brittle, which is why the judge evaluation exists for semantic checks. We kept the label, added a custom column alsoAccepts with addDatasetColumn, filled it on that row with updateDatasetRow, and taught the test to accept either answer. Nine of nine pass.

In CI, run the same command on every change to prompts, tools or models. To keep score in Latitude, post each verdict as a custom score with createScore and a sourceId such as ci-regression; failing scores cluster into signals like any other failure. Because the replays carry the same release tag as production traffic, exclude the simulation tag from any cohort or monitor that should only see real users (see the experiment below).

Compare prompt versions with an Experiment Copy Section

V1 compared two prompt variants over a dataset. V2 compares two slices of real traffic across every metric Latitude tracks. Ask your agent:

Create an experiment in linguaai comparing sessions tagged release-2.0.0 (baseline) with sessions tagged release-2.1.0, excluding sessions tagged simulation, over the last 7 days.

createExperiment takes variants with a filterSet, a query and a timeRange. Each variant's tag filter here is [{"op": "eq", "value": "release-2.1.0"}, {"op": "neq", "value": "simulation"}]. The comparison covers sessions, users, cost, tokens, latency percentiles, tool usage and signal occurrences per cohort, with deltas against the baseline. Metrics fill in after sessions close, about five minutes after their last trace. Use updateExperiment (HTTP PUT) to change variants later.

V1's prompts.chat(conversationUuid, ...) implied Latitude stored conversation history server-side; V2 has no such storage, so a multi-turn use case now keeps its own message array, keyed by the session id.
Give each release its own sessions. A session that mixes traces from two releases cannot be assigned to one cohort; the smoke script suffixes session ids with the release for that reason. In our first attempt the regression replays also landed in the 2.1.0 cohort, because they carry the release tag; the simulation exclusion fixed it.

Annotate, monitor, dispatch Copy Section

The V1 human-in-the-loop evaluation becomes ordinary annotation. From the trace view, or through the MCP:

Annotate trace 2a27c6f2... in linguaai as passed with the feedback "Correctly recognised a well-formed sentence and returned is_correct=true with no spurious corrections."

createAnnotation takes value (0 to 1), passed, feedback and the trace. Failed annotations are the primary input to signal discovery. Once a signal matters, monitor it. A monitor opens an incident and notifies in-app, by email or in Slack, and agent dispatch can send the signal with its example traces to Cursor, Claude Code or Linear.

Decommission V1 Copy Section

• Remove latitude-sdk<6 (Python) or @latitude-data/sdk<6 (TypeScript) and every prompts.run, prompts.chat, logs.create and evaluations.annotate call.
• Delete LATITUDE_PROJECT_ID and LATITUDE_VERSION_UUID from your environment. V2 uses LATITUDE_PROJECT_SLUG.
• Keep the exported PromptL files in your repository history and delete them once every prompt has a code owner.
• Rotate the V1 API key.

Gotchas from this migration Copy Chapter

• Provider parameters are yours now. The PromptL frontmatter hid provider differences; Anthropic's Python SDK 1.x rejected temperature on messages.create(). Check each setting you carry over.
• Sessions belong to one release. Reusing session ids across releases produced sessions that no cohort filter could own. Suffix the id or start new sessions on deploy.
• Split cohorts on tags. Tag filters worked on sessions and in Experiments in our run, while a metadata.release filter on sessions did not match even though the sessions showed the field. Metadata is for search and for filtering traces.
• Keep replays out of production cohorts. Regression runs carry the release tag too, so exclude simulation in experiment variants and monitors.
• Signals collect forward and sample at 10 percent by default. Create them first, then generate traffic, and raise sampling in the UI for low-volume projects.
• No backticks in judge criteria. The criteria is compiled, and a backtick broke it with a syntax error.
• A V1 prompt with type: agent and a Latitude-hosted tool (latitude/extract and similar) needs a real implementation and a hand-written tool-call loop in V2; there is no hosted-tool equivalent. The latitude-migrate skill's promptl-to-code.md has a full example.
• Flush before exit. Short scripts lose their last batch without flush() and shutdown().
• The MCP is OAuth-only. An API key on the MCP URL returns 401. Use the API key for the SDK, CLI and REST, and OAuth for the MCP.
• Old docs linger in agent memory. Point coding agents at docs.latitude.so/llms.txt.

Frequently asked Copy Chapter

Can I import my V1 logs? No. V2 imports exist for Langfuse, LangSmith and Braintrust. Start V2 from live traffic; a week of production gives discovery enough to work with.

Where do non-technical teammates edit prompts now? They do not. Prompts live in code. Product and domain experts work through annotations, signals, datasets and expected outputs, which is where their judgment changes the agent's behaviour.

What replaces prompt suggestions? Agent dispatch. A signal carries the evidence to your coding agent, which proposes the fix as a pull request, and the GitHub integration resolves the signal when the PR merges.

What about self-hosting? V2 self-hosts from a single Docker host to a Kubernetes cluster; see Deployment.

Reference: the MCP calls used in this guide Copy Chapter

The same operations are available as latitude <resource> <command> in the CLI and as client.<resource>.<method>() in the SDKs.

Getting Started Quick Start Dev (`docs/getting-started/quick-start-dev.md`) Copy Section

title: Developer Quick Start Copy Chapter

Developer Quick Start

This guide walks you through connecting an existing AI agent to Latitude. By the end, you'll have live traces flowing into your project and understand how scores and annotations work.

Prerequisites Copy Chapter

• A Latitude account (sign up at latitude.so)
• An existing AI-powered application using a supported provider or framework

Step 1: Create a Project Copy Chapter

After signing in, create a new project from the dashboard. Projects are the main boundary for all reliability features: issues, evaluations, flaggers, saved searches, and simulations are all scoped to a project.

Give your project a descriptive name that matches the agent or feature you're monitoring.

Step 2: Connect Telemetry Copy Chapter

Latitude captures your agent's interactions through OpenTelemetry-compatible telemetry. See the Telemetry section for detailed setup instructions for your specific provider or framework.

Once telemetry is connected, every LLM call, tool invocation, and agent step your application makes will appear as spans in Latitude. Related spans are grouped into traces (single interactions) and sessions (multi-turn conversations).

Step 3: View Your Traces Copy Chapter

Navigate to your project in the Latitude dashboard. You should see traces appearing in real time as your agent handles requests.

Each trace shows:

• The full conversation between user and agent
• Individual spans (LLM calls, tool calls, etc.)
• Timing, token usage, and cost
• Any scores attached to the trace

Step 4: Explore Scores Copy Chapter

Scores are the fundamental measurement unit. Every score is a normalized value between 0 and 1 with a pass/fail verdict and human-readable feedback.

Scores come from three sources:

1. Evaluations: automated scripts that run on your traces
2. Annotations: human review verdicts from your team
3. Custom: scores you submit from your own code via the API

Your project starts with default flaggers that automatically annotate traces for common problems like jailbreaking, refusals, frustration, and tool call errors. Flagger annotations are written directly on matching traces and feed straight into issue discovery, so you'll see issues forming from your live traffic without configuring anything.

Step 5: Explore with Search and Annotate Copy Chapter

Open the Search page in your project. This is where you build cohorts of traces to investigate or review.

1. Type a query like "failed payments" or "frustrated user". Search blends keywords and meaning, so close paraphrases work.
2. Add filters from the toolbar to narrow further (status, model, cost, custom metadata, etc.).
3. Click into any matching trace to read the conversation.
4. Use the annotation panel on the right to leave human feedback. Pick conversation-level, message-level, or text-range scope, give a thumbs-up or thumbs-down verdict, and add a short explanation.

When a search becomes a regular part of your workflow, click Save search to bookmark it. Saved searches surface on the Search landing page with assignee, last-found timestamp, and annotated-vs-total review progress. See Saved Searches for the full lifecycle.

Your annotations feed into issue discovery and evaluation alignment alongside the annotations that flaggers create automatically.

What's Next Copy Chapter

• Observability: Understand spans, traces, and sessions in depth
• Search: Find traces by meaning and bookmark useful cohorts
• Scores: Learn how the scoring system works
• Annotations: Build human review workflows
• Flaggers: Automatic annotators for common failure categories
• Evaluations: Set up automated monitoring
• Issues: Understand how failure patterns are discovered
• Simulations: Test your agent before shipping

Getting Started Quick Start Pm (`docs/getting-started/quick-start-pm.md`) Copy Section

title: No-Code Quick Start Copy Chapter

No-Code Quick Start

This guide walks you through the Latitude web interface. You'll learn how to navigate the product, review agent interactions, annotate conversations, and understand issues, all without writing code.

Prerequisites Copy Chapter

• A Latitude account (sign up at latitude.so)
• A project with telemetry already connected (ask your development team to set this up using the Developer Quick Start)

Understanding the Dashboard Copy Chapter

When you open a project, you'll see the main navigation with these sections:

• Search: Find traces by meaning, and bookmark useful searches for later
• Traces: Every interaction your agent has had, shown as a timeline
• Issues: Failure patterns discovered from your agent's interactions
• Datasets: Saved trace collections for offline analysis and simulations
• Settings: Project configuration, including flaggers

Exploring Traces Copy Chapter

The Traces page shows every interaction your agent has processed. Each row represents one complete interaction, from user request to agent response.

Click on a trace to see:

• The conversation: The full message exchange between user and agent
• Spans: Individual steps the agent took (LLM calls, tool uses, etc.)
• Metadata: Timing, token usage, cost, and any custom metadata
• Scores: Any evaluation, annotation, or custom scores attached to this trace

Use the filter sidebar to narrow traces by status, cost, duration, model, provider, tags, or custom metadata.

Finding Traces with Search Copy Chapter

The Search page is where you go to investigate a specific kind of trace. Type a plain-English query — "failed payments", "frustrated user", "long latency on signup" — and Latitude returns the most relevant traces, ranked by a hybrid of keyword and semantic match. Use "quotes" for exact phrases.

Filters work alongside the query, so you can scope to a time range, a specific model, traces with errors, or any custom metadata your application sends.

When a search is one you'll come back to, click Save search in the toolbar. Saved searches appear on the Search landing page in a table with the columns:

• Saved search: The name, plus a preview of the query and filters
• Last found: When the most recent matching trace appeared
• Assigned To: A team member responsible for reviewing matches
• Annotated: How many matching traces have been reviewed
• Total: How many traces currently match

This is the workflow that used to live in the old Annotation Queues page. Instead of a managed queue with start/finish semantics, you scope your own cohort with a saved search and the Annotated/Total ratio tells you how far through it your team is. See Saved Searches for details.

Automatic Detection with Flaggers Copy Chapter

Some failure categories are common enough that Latitude detects them for you. Every project starts with a set of built-in flaggers running on every completed trace:

• Jailbreaking: Attempts to bypass safety constraints
• NSFW: Sexual or otherwise inappropriate content
• Refusal: The agent refuses requests it should handle
• Frustration: Clear user dissatisfaction
• Forgetting: The agent loses conversation context
• Laziness: The agent avoids doing the requested work
• Trashing: The agent cycles between tools without making progress
• Tool Call Errors: Failed tool invocations
• Output Schema Validation: Structured output didn't conform to the declared schema
• Empty Response: The assistant returned an empty or degenerate response

When a flagger matches, it writes an annotation directly on the trace — no queue, no human approval step. That annotation feeds into issue discovery, scores analytics, and evaluation alignment the same way a human annotation would. You can adjust which flaggers are enabled and how aggressively they sample under Project Settings. See Flaggers for the full list and detection logic.

Reviewing Traces Copy Chapter

To leave human feedback on a trace, open it from any list (Search, Traces, an issue's logs) and use the annotation panel on the right:

• Click anywhere in the conversation to create a message-level annotation, or use the button for a conversation-level one.
• Mark it as positive (thumbs up) or negative (thumbs down).
• Write feedback describing what you observed.
• Optionally link it to an existing issue, or leave issue assignment automatic.

A typical review session combines saved searches and inline annotations: open the saved search you're responsible for, click the first trace, annotate, move on. The saved search's Annotated count goes up as you work.

Understanding Issues Copy Chapter

The Issues page shows failure patterns your agent is experiencing. Issues are discovered automatically when failed scores share similar feedback.

Each issue has:

• A name and description summarizing the failure pattern
• A lifecycle state: New, Escalating, Resolved, Regressed, or Ignored
• Linked evaluations that monitor for this issue on live traffic
• Occurrence trends showing how often the issue appears

You can:

• Generate an evaluation from an issue to start automated monitoring
• Resolve an issue when you believe it's fixed (with an option to keep monitoring for regressions)
• Ignore an issue that isn't worth tracking

Understanding Evaluations Copy Chapter

The Evaluations page shows automated scripts that score your agent's interactions in real time.

Evaluations are typically generated from issues. When you click "Generate Evaluation" on an issue, Latitude creates a monitoring script that watches for that failure pattern on live traffic.

Each evaluation shows:

• Score trends over time
• Alignment with human annotations (how well the automation agrees with human reviewers)
• Trigger configuration (which traces it monitors and how often)

Understanding Scores Copy Chapter

Scores are the fundamental unit of measurement in Latitude. Every score has:

• A value between 0 and 1
• A pass/fail verdict
• Feedback text describing the verdict
• A source: evaluation, annotation (human review), or custom

Scores appear throughout the product: on traces, in evaluation dashboards, in issue details, and in simulation reports.

What's Next Copy Chapter

• Search: Build cohorts of traces with hybrid search
• Saved Searches: Bookmark useful searches and assign ownership
• Flaggers: Built-in automatic annotators for common failures
• Scores: Deep dive into how scores work
• Annotations: Human review workflows
• Issues: Learn about issue lifecycle and management
• Evaluations: Understand automated monitoring

Getting Started Self Healing Agents (`docs/getting-started/self-healing-agents.mdx`) Copy Section

title: Self-Healing Agents Copy Chapter

A self-healing agent is one whose production failures reach a coding agent automatically, with the evidence attached, so remediation starts from the detection instead of from a person relaying context. Latitude closes that loop end to end: traffic is observed, recurring patterns become tracked signals, escalations dispatch your coding agent, and the fix lands as a pull request that a person reviews, backed by a regression test in your CI.

How the loop closes Copy Chapter

Where humans stay Copy Chapter

Two points of the loop are human by design. Annotations are the ground truth that evaluations are aligned against, and a person reviews every pull request before it merges. Everything between those points, finding examples, reconstructing context, and carrying trace ids between tools, is what the loop automates.

Set it up Copy Chapter

1. Start tracing one production agent.
2. Annotate a handful of real failures, or let flaggers do it.
3. Generate an evaluation for the signal that matters most.
4. Add a monitor for any metric you cannot afford to miss.
5. Configure agent dispatch under Settings → Integrations.

Your coding agent can drive the whole workspace through the MCP server:

claude mcp add --transport http latitude https://api.latitude.so/v1/mcp

For other coding agents, see the MCP server docs.

Getting Started Skills (`docs/getting-started/skills.mdx`) Copy Section

title: Skills Copy Chapter

Overview Copy Chapter

The fastest way to add Latitude to an app is to let your coding agent do it. Latitude publishes a set of Agent Skills that teach an agent (Claude Code, Cursor, Codex, Windsurf, OpenCode, …) how to instrument your code, drive the CLI, and confirm that real traces arrive — following Latitude best practices.

There are two entry points, depending on whether you already have a Latitude account.

The skills Copy Chapter

All skills live in the public latitude-dev/skills repo. Most agents can install them with the skills CLI:

npx skills add https://github.com/latitude-dev/skills --skill latitude-setup,latitude-cli,latitude-telemetry,latitude-artifacts

latitude-setup builds on latitude-cli and latitude-telemetry and ends by building a first Artifact from its own template. latitude-artifacts is for the reports you ask for after that, so the command above installs all four.

No account yet Copy Chapter

If you don't have a Latitude account, paste this prompt into your coding agent:

Install the `latitude-setup` skill from `github.com/latitude-dev/skills`, and use it to add Latitude tracing to this app following best practices.

The latitude-setup skill orchestrates the whole zero-account flow:

1. Installs the latitude CLI and the Latitude skills.
2. Bootstraps a temporary account — no signup — with an API key and a project.
3. Instruments your app (via the latitude-telemetry skill) against that project.
4. Runs your real code and inspects the resulting traces, iterating until they look right.
5. Cleans up the trace noise and hands you a claim link.
6. Builds your first Artifact, an HTML page showing what the telemetry captured from the verified session, with a button to claim the workspace, and hands it back with the claim link.

Open the claim link in your browser to take ownership of the temporary account and keep it. Unclaimed temporary accounts expire automatically.

The skill presents a plan and waits for your approval before it edits any code, and it never prints your API key in the chat.

Already have an account Copy Chapter

If you're already signed in, skip the temporary account and point the agent straight at your existing project:

Install the `latitude-telemetry` skill from `github.com/latitude-dev/skills`, and use it to add Latitude tracing to this app following best practices.

Give the agent your LATITUDE_API_KEY and LATITUDE_PROJECT_SLUG when it asks — or connect the MCP server so it can look them up — and it will instrument your app and verify traces the same way.

Once traces are flowing, head to the telemetry guides for provider-specific details, the CLI and MCP server pages to keep driving Latitude from your terminal or agent, or Artifacts to turn that data into reports and dashboards without opening the web app.

More Api Reference (`docs/more/api-reference.mdx`) Copy Section

title: API Copy Chapter

API Reference

The Latitude API reference is hosted separately.

Open the API Reference

More Artifacts (`docs/more/artifacts.mdx`) Copy Section

title: Artifacts Copy Chapter

Overview Copy Chapter

An Artifact is a self-contained HTML page that your coding agent builds from your Latitude data: a one-off report that answers a specific question, or a KPI dashboard you reopen every morning with fresh numbers.

Latitude doesn't ship a dashboard builder. Instead, everything you see in the Latitude web app is exposed as operations over the MCP, the CLI, the API and the SDKs: traces, sessions, spans, tools, users, memory, signals, incidents, experiments, saved searches, and a composable analytics query on top of all of it. You ask your agent a question, the agent makes the calls, renders the page and writes the findings. Latitude provides the data; the agent provides the page. You may never need to open the Latitude web app.

npx skills add https://github.com/latitude-dev/skills --skill latitude-artifacts

Three ways to build one Copy Chapter

The page is the same in all three. It embeds the data as a JSON blob, so a refresh never touches the layout, and it never calls the Latitude API from the browser. Refreshable artifacts need an API key in the environment.

Example prompts Copy Chapter

Paste any of these into an agent connected to the Latitude MCP (or with the CLI authenticated). Each one gives the agent a question, a window and a target; the latitude-artifacts skill fills in the rest, and asks a couple of short questions when something is unclear.

One-off reports Copy Section

Build an artifact of this project's LLM reliability and cost over the last 30 days: volume, error rate, p95 latency and spend, broken down by model and by tool, with what changed versus the previous 30 days.

How are users interacting with my agent's memory? Build an artifact for the last 14 days: which memory stores exist and how big they are, how much they change per day, who reads and writes them, what gets retrieved most, and what memory costs per session.

Which of my agent's tools fail the most, and why? Build an artifact with per-tool call volume, error rate and p95 latency, the clustered error messages for the three worst tools, and the ten slowest tool calls.

Yesterday's error rate spiked. Build a post-mortem artifact: when it started and ended, which models, tools and users were affected, the signals and incidents that fired, and the most likely root cause with links to example traces.

Compare gpt-4.1 and claude-sonnet-4 in this project over the last two weeks: latency percentiles, cost per trace, error rate, cache hit rate and tool-call counts, as an artifact with a short recommendation.

Build a weekly quality digest artifact: ongoing, new and escalating signals with their occurrence trends, incidents by severity, behaviour clusters that grew or shrank, and the pass rate of every signal broken down by model.

Refreshable dashboards Copy Section

Create a refreshable artifact I can open every morning with yesterday's numbers: traces, sessions, unique users, error rate, p95 latency, spend and tokens, each with a 14-day trend, plus the top 10 tools by calls and the ongoing signals. Save it to artifacts/daily.html with a refresh script I can run from this repo.

Set up a weekly ops artifact for this project that refreshes on a GitHub Actions schedule every Monday at 08:00 UTC and publishes to GitHub Pages. Sections: reliability, cost by model and provider, tool failures, new users, and incidents.

Build a per-customer usage artifact refreshed daily: for each identified user, traces, sessions, tokens, cost and the tools they hit most, sorted by spend, with a cost histogram and the users whose spend grew fastest week over week.

What the agent can query Copy Chapter

Every read below is one operation with the same name on the MCP, the CLI, the API and the SDKs: the MCP tool listTools is latitude tools list on the CLI, GET /v1/projects/{projectSlug}/tools on the API and client.tools.list(...) in the SDKs. The agent reads each operation's schema before calling it; the live catalog is the API reference.

All of these are read-only. The write operations (creating signals, monitors, datasets, annotations) are available to the same agent, so an artifact can end with an action: "create a monitor on this" or "save this search".

The aggregate query: `queryAnalytics` Copy Chapter

Most tiles and charts in an artifact are one queryAnalytics call: a metric over a filtered stream, optionally split by a breakdown and bucketed over time.

• Metric: the number each row reduces to (count, errorRate, percentile(duration), sum(cost), …).
• Breakdown: the dimension to split across, one row per value (model, tool, signalId, …).
• Time bucket: optional granularity (hour, day, week) that turns a number into a trend.