Technical Documentation: cypress-io/cypress
ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference
Overview (README.md)
The web has evolved. Finally, testing has too.
What is Cypress?
Installing
Install Cypress for Mac, Linux, or Windows, then get started.
npm install cypress --save-devor
yarn add cypress --devor
pnpm add cypress --save-devContributing
Please see our Contributing Guideline which explains repo organization, linting, testing, and other steps.
License
This project is licensed under the terms of the MIT license.
Badges
Configure a badge for your project's README to show your test status or test count in the Cypress Cloud.
Or let the world know your project is using Cypress with the badge below.
[](https://www.cypress.io/)Guides README (`guides/README.md`)
Guides
The guides directory contains information about specific things encountered when contributing to the Cypress project that fall outside the scope of CONTRIBUTING.md, which is more geared towards everyday development in this repo.
For information about using Cypress for testing web applications, visit docs.cypress.io.
For information related to a specific package in the monorepo, see that package's respective README.md.
For general contributor information, check out CONTRIBUTING.md.
Table of Contents
- App lifecycle
- App <--> Cloud Testing
- CI
- Building release artifacts
- Code signing
- Debug logs
- Determining the next version of Cypress to be released
- E2E Open Mode Testing
- Error handling
- ESLint Migration Guide: Monorepo Alignment
- GraphQL Subscriptions - Overview and Test Guide
- Patching packages
- Protocol development
- Purging Specific URLs from Cloudflare cache
- Release process
- Studio development
- Testing other projects
- Testing strategy and style guide (draft)
- V8 Snapshots
- Writing cross-platform JavaScript
- Writing the Cypress Changelog
- Writing and Migrating To TypeScript
Ai README (`docs/ai/README.md`)
AI Tooling for the Cypress Monorepo
This repo is AI-ready. The goal of these docs is to make effective AI workflows
discoverable alongside the code, with minimal process overhead.
Note: Some references below point to Cypress-internal resources (e.g. AI Hub).
These are optional background/context and are not required to contribute to this repository.
All repo-local AI guidance should remain usable without access to internal systems.
Canonical Source of Truth
Canonical workflows and patterns are curated in the AI Hub (internal reference):
- Patterns: https://aihub.cypress.io/patterns
- Playbooks: https://aihub.cypress.io/playbooks
- Tools (skills installer, etc.): https://aihub.cypress.io/tools
- Recordings / notes: https://aihub.cypress.io/forum
- Browse by tag: https://aihub.cypress.io/browse-by-tag
If you improve a workflow/pattern materially, prefer capturing it in the Hub
(PR-reviewed) and linking from here.
Tool Entrypoints
Claude Code CLI
Docs — Claude Code walks upward from your
current working directory to the git root at launch, loading every CLAUDE.md it finds.
CLAUDE.md files in this repo:
/* Detailed source-code truncated for AI context efficiency. */Each CLAUDE.md (except the root) is a thin wrapper — title heading + @AGENTS.md.
All substantive content lives in the sibling AGENTS.md. This keeps CLAUDE.md files
minimal while making them self-documenting via the import.
Codex CLI
Docs — Codex walks downward
from the git root to your current working directory, concatenating AGENTS.md files
root-first (32 KiB limit total).
AGENTS.md files in this repo mirror the CLAUDE.md list above, with the same paths
but AGENTS.md filename instead. The root AGENTS.md provides full project context;
workspace and package AGENTS.md files add scoped detail.
Cursor
Docs — Cursor reads .cursor/rules/*.mdc and
natively reads AGENTS.md files.
Existing Cursor rules in this repo:
.cursor/BUGBOT.md— PR review checklist (security, performance, Cypress-specific rules)packages/electron/.cursor/rules/electron-upgrade.mdc— Electron upgrade guide
| Type | Frontmatter | When active |
|---|---|---|
| Always Apply | alwaysApply: true |
Every session |
| Specific Files | globs: ["app/**"] |
When editing a matching file |
| Agent-requested | description: "..." only |
Agent decides based on description |
| Manual | none | Only when mentioned via @rule-name in chat |
Why AGENTS.md Is the Single Source of Truth
Cursor and Codex read AGENTS.md natively; Claude Code reads it via @import. One edit
propagates to all three tools. The only separately-maintained content is the workflow
orchestration section in root CLAUDE.md — that is Claude Code-specific and intentionally
kept separate.
Contributing
Hub contributions are preferred for canonical patterns and playbooks. Repo-local scaffolds
(AGENTS.md / CLAUDE.md) should stay minimal — factual descriptions of what exists,
not aspirational guides.
If you add a new package to the monorepo, add a CLAUDE.md and AGENTS.md alongside
it following the patterns in existing packages.
When repo structure, commands, or conventions change, update the relevant AGENTS.md /CLAUDE.md files in the same PR (similar to keeping READMEs accurate).
Guides App Cloud Testing (`guides/app-cloud-testing.md`)
App <--> Cloud Testing
Testing communication between the Cypress App and Cypress Cloud can be complex depending on how much data is going back and forth. For example, we recently completed the Debug page feature which involved writing some e2e tests to verify that the page was functioning correctly with the expected data from Cypress Cloud. In this document, we'll share some of our findings from the process of writing these tests for the Debug page.
Working with GraphQL requests
We use GraphQL to interface with Cypress Cloud, so there are some things to know when writing e2e tests between the App and Cloud.
GraphQL requests over fetch
By default, GraphQL queries use web sockets to fetch data. This doesn't let us intercept these messages and return a mock response using cy.intercept to get our app into the desired state for the test. We can set the GraphQL client to use fetch rather than WS by adding this hook at the top of our spec file.
You can see an example of this in our spec for the Debug page.
Cypress.on('window:before:load', (win) => {
win.__CYPRESS_GQL_NO_SOCKET__ = 'true'
})Now that the GraphQL requests to Cypress Cloud are happening over fetch, we can use cy.intercept to intercept the requests and do whatever we need to do (in most cases, return a JSON fixture as a response to the request).
Note that this intercepts requests between the App and Server. This means that any logic that we have in the data-context layer for this query will not be executed by the test because the query will be intercepted before it runs.
Creating JSON fixtures from GraphQL requests
Another benefit of making our GraphQL requests over fetch is that we can easily see the real responses from Cypress Cloud in the Network tab of our browser developer tools. This is especially useful when we have large responses coming back from Cypress Cloud that we want to mock in our tests.
We can start our fixture by copying the real response from the dev tools response into our fixture and then modifying individual fields to tweak our app state. Then we can intercept the GraphQL request and return our fixture.
You can see an example of this in our spec for the Debug page.
cy.intercept('query-Debug', {
fixture: 'debug-Passing/gql-Debug.json',
})Intercepting subscriptions and requests from the server
There are a few situations in which we need to use cy.remoteGraphQLIntercept to intercept a request:
- If the request originates from the server
- If the request is a subscription (these will always be over web sockets, so cy.intercept doesn't work)
- If there is server-side logic in the resolver that we want to cover with our test
cy.remoteGraphQLIntercept intercepts the request at the server level, so we are able to return fixture data here and have the resolver continue on as if this is the data that was returned from Cypress Cloud.
You can see an example of this in our spec for the Debug page.
import RelevantRunsDataSource_RunsByCommitShas from '../fixtures/gql-RelevantRunsDataSource_RunsByCommitShas.json'
beforeEach(() => {
cy.remoteGraphQLIntercept((obj, _testState, options) => {
if (obj.operationName === 'RelevantRunsDataSource_RunsByCommitShas') {
obj.result.data = options.RelevantRunsDataSource_RunsByCommitShas.data
}
return obj.result
}, { RelevantRunsDataSource_RunsByCommitShas })
})Guides App Lifecycle (`guides/app-lifecycle.md`)
App Lifecycle
This documents the lifecycle of the application, specifically related to managing the current project,
and the various states & inputs that can feed into state changes, and how they are managed
- Application starts via
cypress open | run --flags - The input is run through
cli/lib/cli.jsfor normalization - The normalized input is passed into the server, eventually getting to
server/lib/modes/index.ts - The
DataContextclass receives the testing mode (run|open), and themodeOptions(CLI Flags) - We call
ctx.initialize, which based on themodereturns a promise for series of steps needed - The
DataContextshould act as the global source of truth for all state in the application. It should be passed along where possible. In theserverpackage, we can import/usegetCtxso we don't need to pass it down the chain. - The CLI flags & environment variables are used set the initial state of the
coreData
1. TODO: rename toappState? - In
openmode, if the--globalflag is passed, we start in "global" mode, which allows us to select multiple projects - Once a project is selected, either via the CLI being run within a project, or via the
--projectflag, we launch into project mode
Project Lifecycle
- Once a project is selected, we source the config from
cypress.config.js, or wherever the config is specified via the--configFileCLI flag: - Read the
globalBrowsers - Execute the
configFilein a child process & reply back with the config, and the require.cache files in the child process - If there is an error sourcing the config file, we set an error on the
currentProjectin the root state - We source
cypress.env.jsonand validate (if it exists)
**Config Precedence:**
- Runtime, inline:
it('should do the thing', { retries: { run: 3 } } portfrom spawned server- Returned from
setupNodeEvents(as these get the options from the CLI) - Sourced from CLI
- Sourced from
cypress.env.json - Sourced from
cypress.config.{js|ts} - Default config options
**Merging**
Config options are deeply merged:
# CLI:
cypress run --env FOO=bar
# cypress.config.js
env: {
FOO: 'test'
},
e2e: {
setupNodeEvents (on, config) {
return require('@cypress/code-coverage')(on, config)
},
env: {
e2eRunner: true
}
}
# Would Result in
{
env: { FOO: 'bar', e2eRunner: true }
}Steps of Sourcing / Execution
- Application Start
- CLI args & environment are parsed into an "options" object, which is passed along to create the initial application config
- Browsers are sourced from the machine at startup
- CLI options
--config baseUrl=http://example.com,--envare gathered for merging later
- Project Initialization
- When we have a "projectRoot", we execute the
cypress.config.{js|ts}, and read thecypress.env.json- this will be persisted on the state object, so we can compare the diff as we detect/watch changes to these files- The child process will also send back a list of files that have been sourced so we can watch them for changes to re-execute the config. We may want to warn against importing things top-level, so as to minimize the work done in child-process blocking the config
- We also pull the "saved state" for the user from the FS App data
- We only do this in "open mode"
- At this point, we do a first-pass at creating a known config shape, merging the info together into a single object, picking out the "allowed" list of properties to pass to the
setupNodeEvents
- When we have a "projectRoot", we execute the
- setupNodeEvents
- Once we have selected a
testingType, we execute thesetupNodeEvents, passing an "allowed" list of options as the second argument to the function. At this point, we have merged in any CLI options, env vars,- If they return a new options object, we merge it with the one we passed in
- Once we have selected a
- config → FullConfig
- At this point we have the entire config, and we can set the
resolvedproperty which includes the origin of where the config property was resolved from
- At this point we have the entire config, and we can set the
Guides Building Release Artifacts (`guides/building-release-artifacts.md`)
Building Release Artifacts
The cypress npm package consists of two main parts:
- The
cypressnpm package.tgz(built fromcli)- Contains the command line tool
cypress, type definitions, and the Module API. - End users install this via npm to the project's
node_modulesdirectory or via Yarn or pnpm similarly.
- Contains the command line tool
- The "binary"
.zip(built from thepackagesdirectory)- Contains the Electron app,
ffmpeg, and built versions of the packages from thepackagessub-directories (frontend-shared,reporterandweb-configare not separately included).- Also contains all the production dependencies of the above.
- This is installed when the
cliis installed or whencypress installis run, to a system cache.
- Contains the Electron app,
This guide has instructions for building both.
Building the npm package
:warning: Note: The steps in this section are automated in CI, and you should not need to do them yourself when going through the release process.
Building a new npm package is two commands:
- Increment the version in the root
package.json yarn lerna run build-cli
The steps above:
- Build the
cypressnpm package - Transpile the code into ES5 to be compatible with the common Node versions
- Put the result into the
cli/buildfolder.
Building the binary
:warning: Note: The steps in this section are automated in CI, and you should not need to do them yourself when going through the release process.
The npm package requires a corresponding binary of the same version. In production, it will try to retrieve the binary from the Cypress CDN if it is not cached locally.
You can build the Cypress binary locally by running yarn binary-build, then package the binary by running yarn binary-package. You can use Linux to build the Cypress binary (just like it is in CI) by running yarn binary-build and yarn binary-package inside of yarn docker.
If you're on macOS and building locally, you'll need a code-signing certificate in your keychain, which you can get by following the instructions on Apple's website. Also, you'll also most likely want to skip notarization since it requires an Apple Developer Program account - set SKIP_NOTARIZATION=1 when building locally to do this. More info about code signing in CI.
yarn binary-zip can be used to zip the built binary together.
Tips
If you want to speed up the time it takes to package the binary, set V8_SNAPSHOT_DISABLE_MINIFY=1
If you are on an M1, you need to set RESET_ADHOC_SIGNATURE=1 in order to be able to actually run the binary after packaging it.
Guides Code Signing (`guides/code-signing.md`)
Code Signing
Code signing is done for the Windows and Mac distributions of Cypress when they are built in CI.
electron-builder handles code signing during the create-build-artifacts jobs. This guide assumes that the reader is already familiar with electron-builder's Code Signing documentation.
Rotating the Mac code signing key
- On a Mac, log in to Xcode using Cypress's Apple developer program identity.
- Follow Apple's Create, export, and delete signing certificates instructions:
- Follow "View signing certificates".
- Follow "Create a signing certificate", and choose the type of "Developer ID Application" when prompted.
- Follow "Export a signing certificate". Set a strong passphrase when prompted, which will later become
CSC_KEY_PASSWORD.
- Upload the exported, encrypted
.p12file to the Code Signing folder in Google Drive and obtain a public direct download link. - Within the
test-runner:sign-mac-binaryCircleCI context, setCSC_LINKto that direct download URL and setCSC_KEY_PASSWORDto the passphrase used to encrypt thep12file.
Rotating the Windows code signing key
- Generate a certificate signing request (CSR) file using
openssl. For example:shell# generate a new private key openssl genrsa -out win-code-signing.key 4096 # create a CSR using the private key openssl req -new -key win-code-signing.key -out win-code-signing.csr - Obtain a certificate by submitting the CSR to SSL.com using the Cypress SSL.com account.
- If renewing, follow the renewal instructions.
- If rotating, contact SSL.com's support to request certificate re-issuance.
- Obtain the full certificate chain from SSL.com's dashboard in ASCII-armored PEM format and save it as
win-code-signing.crt. (-----BEGIN PRIVATE KEY-----,-----BEGIN CERTIFICATE-----) - Using
openssl, convert the plaintext PEM public and private key to binary PKCS#12/PFX format and encrypt it with a strong passphrase, which will later becomeCSC_KEY_PASSWORD.shell➜ openssl pkcs12 -export -inkey win-code-signing.key -in win-code-signing.crt -out encrypted-win-code-signing.pfx Enter Export Password: <password> Verifying - Enter Export Password: <password> - Upload the
encrypted-win-code-signing.pfxfile to the Code Signing folder in Google Drive and obtain a public direct download link. - Within the
test-runner:sign-windows-binaryCircleCI context, setCSC_LINKto that direct download URL and setCSC_KEY_PASSWORDto the passphrase used to encrypt thepfxfile.
Guides Continuous Integration (`guides/continuous-integration.md`)
Continuous Integration
This guide provides an overview of the CI/CD pipelines used in the Cypress repository.
CircleCI
CircleCI is our primary CI system, handling testing, building, and release workflows.
Workflows
Pull Request Workflow
Runs on PRs targeting develop or release/* branches (excluding draft PRs).
| Stage | What It Does |
|---|---|
| Build | Installs dependencies, builds packages, generates V8 snapshots |
| Lint & Type Check | ESLint, TypeScript compilation, type linting |
| Unit Tests | Runs unit tests across packages |
| Integration Tests | Driver tests (Chrome, Firefox, Electron, WebKit), server tests, component tests |
| System Tests | End-to-end system tests across browsers |
| NPM Package Tests | Validates framework integrations (React, Vue, Angular, etc.) |
Note: External contributor PRs require manual approval before running tests that use secrets.
Binary jobs do not run on pull requests. Building and verifying the Electron
binary is the most expensive part of the pipeline, so it runs post-merge ondevelop instead. There is no opt-in and no path-based override: a PR that
changes binary or packaging code is not covered until it merges, and breakage is
caught by the develop build. See notify-binary-failure below.
Full Workflow
Runs on pushes to develop and release/* branches. Includes everything from the PR workflow plus:
Note: a webhook push to develop is path-filtered the same way PRs are — only the job groups touched by the merge's changed files run. Everything else runs unfiltered: release/* pushes, and, on develop, the scheduled nightly cron and any manually triggered run. Filtering never skips the binary/packaging chain, npm-release, or verify-release-readiness.
| Stage | What It Does |
|---|---|
| Linux x64 Build | Full build, packaging, and binary verification for Linux x64 |
| Binary Creation | Triggers the cypress-publish-binary pipeline to build the Linux x64 Electron binaries |
| Binary Verification | Tests the built Linux binary against kitchensink, recipes, and real-world apps |
| Binary Failure Alerts | Comments on the commit, tagging its author, when a binary job fails |
| Release Preparation | Validates release readiness, prepares npm packages |
Linux ARM64, macOS Intel, macOS Apple Silicon, and Windows do not build on every develop or release/* push — see "Multi-Platform Builds" below.
Multi-Platform Builds
Linux ARM64, macOS Intel, macOS Apple Silicon, and Windows build from a CircleCI Scheduled Pipeline instead of every push — one schedule for develop, a second retargeted at whichever release/* branch is currently active. See the "Scheduled platform CI" section of .circleci/AGENTS.md for the schedule ids and parameters. A manual Trigger Pipeline run can also produce all four on demand, for any branch, via run-platform-workflows=true or force-persist-artifacts=true.
Triggers
| Trigger | Workflow | Notes |
|---|---|---|
| PR opened/updated | Pull Request | Skipped for draft PRs |
Push to develop |
Full | Linux x64 is path-filtered by changed files; binary builds and release gating always run. Other platforms run from the develop schedule, not this push |
Push to release/* |
Full | Linux x64 runs the complete test suite and binary builds, unfiltered. Other platforms run from the release schedule |
| Scheduled pipeline | Full | Nightly cron; always unfiltered on develop, even though it runs on that branch |
| Manual (CircleCI UI) | Configurable | Can run the full Linux x64 workflow, or all four scheduled platforms, on any branch; run-all-jobs=true forces everything on Linux x64 |
Key Jobs
build- Core build job that all test jobs depend onready-to-release- Gate job that ensures all tests pass before releasenpm-release- Publishes packages to npm (develop/release branches only)create-and-trigger-packaging-artifacts- Initiates binary build processnotify-binary-failure- Comments on the commit when a binary job fails ondevelop, tagging its author. It requires every binary job atterminal, so it
runs once they have all settled however they settled, then reads their real
statuses from the CircleCI API and exits quietly unless one of them isfailed-
a cancelled or skipped job is not a breakage. Scope comes from the job's own
dependency list, so adding a binary job to itsrequires:block is all that is
needed to cover it.developis squash-merged, so the commit author is the author of the PR that
produced it, and the@tag notifies them whatever their GitHub notification
settings are. A commit whose author GitHub cannot resolve still gets a comment,
just without the tag.It needs
CIRCLE_TOKEN(from thepublish-binarycontext) andGH_TOKEN(fromtest-runner:npm-release, the same pairingverify-release-readinessuses).
GitHub Actions
GitHub Actions handle automation, security scanning, and repository maintenance.
Automation
| Workflow | Trigger | Purpose |
|---|---|---|
| Update Browser Versions | Daily at 8am UTC | Checks for new Chrome stable/beta versions and creates PRs to update CI config |
| Update V8 Snapshot Cache | Daily + release branch pushes | Regenerates V8 snapshots to maintain startup performance |
Security
| Workflow | Trigger | Purpose |
|---|---|---|
| Snyk SCA Scan | PRs to develop/release | Scans dependencies for known vulnerabilities |
| Snyk Static Analysis | PRs to develop/release | Performs code security analysis |
PR Validation
| Workflow | Trigger | Purpose |
|---|---|---|
| Semantic Pull Request | PR opened/edited | Validates PR title follows conventional commit format and has changelog entry |
Repository Maintenance
| Workflow | Trigger | Purpose |
|---|---|---|
| Stale Issues/PRs | Daily at 1:30am UTC | Marks inactive issues/PRs as stale after 180 days, closes after 14 more days |
| Triage Workflows | Issue/PR opened | Adds external contributor items to the triage project board |
Compliance
| Workflow | Trigger | Purpose |
|---|---|---|
| Upload Release Asset | Release published | Generates Software Bill of Materials (SBOM) and attaches to GitHub release |
Configuration Files
- CircleCI:
.circleci/config.yml(entry point),.circleci/src/(source configs) - GitHub Actions:
.github/workflows/
For information about modifying the CircleCI configuration, see .circleci/README.md.
Guides Cy Prompt Development (`guides/cy-prompt-development.md`)
`cy.prompt` Development
In production, the code used to facilitate the prompt command will be retrieved from the Cloud.
To run against locally developed cy.prompt:
- Clone the
cypress-servicesrepo- Run
yarn && yarn all - Run
yarn watchinapp/packages/cy-prompt
- Run
- Set:
CYPRESS_INTERNAL_ENV=<environment>(e.g.stagingorproductionif you want to hit those deployments ofcypress-servicesordevelopmentif you want to hit a locally running version ofcypress-services)CYPRESS_LOCAL_CY_PROMPT_PATHto the path to thecypress-services/app/packages/cy-prompt/dist/developmentdirectory
To run against a deployed version of cy.prompt:
- Set:
CYPRESS_INTERNAL_ENV=<environment>(e.g.stagingorproductionif you want to hit those deployments ofcypress-servicesordevelopmentif you want to hit a locally running version ofcypress-services)
Regardless of running against local or deployed cy.prompt:
- Clone the
cypressrepo- Run
yarn - Run
yarn cypress:open - Log In to the Cloud via the App
- Run
To run against a deployed version of cy.prompt:
- Set:
CYPRESS_INTERNAL_ENV=<environment>(e.g.stagingorproductionif you want to hit those deployments ofcypress-servicesordevelopmentif you want to hit a locally running version ofcypress-services)
Types
The prompt bundle provides the types for the app, driver, and server interfaces that are used within the Cypress code. To incorporate the types into the code base, run:
yarn gulp downloadPromptTypesor to reference a local cypress_services repo:
CYPRESS_LOCAL_CY_PROMPT_PATH=<path-to-cypress-services/app/packages/cy-prompt/dist/development-directory> yarn gulp downloadPromptTypesTesting
Unit/Component Testing
The code that supports cloud cy.prompt and lives in the cypress monorepo is unit, integration, and e2e tested in a similar fashion to the rest of the code in the repo. See the contributing guide for more specifics.
The code that supports cloud cy.prompt and lives in the cypress-services monorepo has unit tests that live alongside the code in that monorepo.
Guides Debug Logs (`guides/debug-logs.md`)
Debug Logs
Many Cypress packages use the debug module to log runtime info to the console.
Choosing a namespace
The naming scheme for debug namespaces should generally follow this pattern:
cypress:{packageName}:{relative path to file from src root, using : to separate directories, minus index if applicable}
# examples:
# packages/server/lib/util/file.js -> cypress:server:util:file
# packages/launcher/windows/index.ts -> cypress:launcher:windowscypress-verbose can be used instead of cypress if the logs are overly verbose and would make the output of DEBUG=cypress:* unreadable.
Exceptions to these rules:
- The
cliusescypress:cli:*. - NPM packages should use
{moduleName}as a prefix instead ofcypress, likecypress-webpack-preprocessorfornpm/webpack-preprocessor. - In some places, like per-request in the
proxypackage, it's more useful to attachdebugmessages to something besides the module (like individual HTTP requests). In that case, it's okay to create namespaces as you see fit. But at least begin withcypress:{packageName}orcypress-verbose:{packageName}
Using debug logs
Pass the DEBUG environment variable to select a set of logs to print to stderr. Example selectors:
# frequently useful to get a sense of what is happening in the app at a high level
DEBUG=cypress:*
# print all info and verbose logs, but don't print verbose logs from `some-noisy-package`
DEBUG=cypress:*,cypress-verbose:*,-cypress-verbose:some-noisy-package:*
# print out verbose per-request data for proxied HTTP requests
DEBUG=cypress-verbose:proxy:http
# in the browser, set `localStorage.DEBUG`:
localStorage.DEBUG = 'cypress:driver,cypress:driver:*'For more info, see the public documentation for printing debug logs and the debug module docs
Guides E2e Open Testing (`guides/e2e-open-testing.md`)
E2E Open Mode Testing
The "Open Mode" E2E tests in packages/app & packages/launchpad are the "system tests" for the open mode. This means that they hit the actual node server and act as a realistic "real world" scenario as much as possible. The main idea of these tests is to act as comprehensive "end-to-end" test for the actual server execution, and act as a "black box", testing for functional behavior as high-level as possible - only tapping into lower-level details when it's necessary.
Example Test:
beforeEach(() => {
// Scaffold a project, from system-tests/fixtures
cy.scaffoldProject('todos')
// "Open" the project, as though you did cypress open --project 'path/to/todos'
cy.openProject('todos')
// Open the project, passing '--e2e' in the argv for cypress open
cy.openProject('todos', ['--e2e'])
// cypress open --global
cy.openGlobalMode()
})
it('should onboard a todos project', () => {
cy.visitLaunchpad()
cy.get('[e2e-project]').click()
cy.withCtx(async (ctx) => {
await ctx.actions.file.writeFileInProject('cypress.config.ts', 'export default {}') // Adds a file
})
})Testing the App:
it('should open todos in the app', () => {
cy.startAppServer() // starts the express server used to run the "app"
cy.visitApp() // visits the app page, without launching the browser
cy.specsPageIsVisible()
cy.get('[href=#/runs]').click()
cy.get('[href=#/settings]').click()
})Remote GraphQL Schema Mocking
When we hit the remote GraphQL server, we mock against the same mocked schema we use client-side in the component tests. If we want to modify these responses we can use cy.remoteGraphQLIntercept to tap in and modify the mocked payloads to simulate different states:
cy.remoteGraphQLIntercept(async (obj) => {
// Currently, all remote requests go through here, we want to use this to modify the
// remote request before it's used and avoid touching the login query
if (obj.result.data?.cloudProjectBySlug.runs?.nodes) {
obj.result.data.cloudProjectBySlug.runs.nodes = []
}
return obj.result
})Guides Error Handling (`guides/error-handling.md`)
Error Handling in Cypress
Clear, consistent, errors are one of the important parts of the Cypress experience. When something goes wrong, there should be clear, actionable feedback for the user about exactly what went wrong, where it went wrong, and next steps on how to fix.
@packages/errors
All error related logic for the server should be added to @packages/errors.
Errors Development Workflow
Adding and editing errors is best done with the help of an ANSI Comparison tool. Cursor/VSCode offer the vscode-ansi comparison tool.
Errors can be compared directly in your code error. Comparing old vs new errors can be done via comparing the git diff or the snapshot diff.
Editing or Adding New Errors and Updating Snapshots
- Add (or update) errors in
packages/errors/src/errors.ts - Add test cases to
visualSnapshotErrors.spec.ts. - Run
yarn testin thepackages/errorsdirectory. - See the
.ansifile generated/edited for your added test case. The file should be named by the error key. (e.g.AUTOMATION_SERVER_DISCONNECTED). - click the "Open Preview to the Side" icon in the top right portion of your code editor.
- Verify the changes expected.
- To make edits, re-run the
yarn test -ucommand.-uwill update the test snapshots. - Run
yarn testafter updating the snapshot to validate the changes were applied. - Commit the files changed in
./test/__snapshots__. - If you added or removed an error key, regenerate and commit
packages/data-context/schemas/schema.graphqlin the same PR (yarn workspace @packages/data-context build).
Technical Overview
errors.ts: A key/value mapping of known errors to functions returning "ErrorTemplates", described below, also includes/re-exports several helper utilities:get/getError: builds & retrieves the error as aCypressError, should be the main way we retrieve errors throughout Cypress. Aliased aserrors.getfor existing use in the server packagethrow/throwErr: Get & throw the error, so we can spy/stub it in a test. Aliased aserrors.throwErrfor existing use in the server packagelogWarning: Logs the error as a warning to the console, aliased aserrors.logfor existing use in the server package
errTemplate.ts: Tagged template literal formatting the error as described belowstackUtils.ts: Utilities for working with a stack trace, extended by the driver package
errTemplate
The errTemplate is a tagged template literal. It allows us to maintain consistent behavior & formatting in our error messages, when we see a variable, we format it depending on the target environment.
The error message returns a message that defaults to being formatted for the terminal, and has a forBrowser method which returns the error message where the variables are wrapped in backticks '`' for Markdown display in the browser.
Return Value of errTemplate (ErrTemplateResult):
{
// Will always exist, this is the terminal-formatted error message
message: string,
// Will always exist, this is the browser-formatted error message
messageMarkdown: string,
details?: string, // Exists if there is `details()` call in the errTemplate
originalError?: ErrorLike // Exists if an error was passed into the `details()`
}Example:
CANNOT_TRASH_ASSETS: (arg1: string) => {
return errTemplate`\
Warning: We failed to trash the existing run results.
This error will not affect or change the exit code.
${details(arg1)}`
},In this case, arg1 will be highlighted in yellow when printed to the terminal.
FAKE_ERROR: (arg1: string, arg2: Error) => {
return errTemplate`\
The fake file is missing or invalid.
Your \`fakeFile\` is set to ${arg1}, but either the file is missing, it contains a syntax error, or threw an error when required. The \`fakeFile\` must be a \`.js\` or \`.ts\` file.
Or you might have renamed the extension of your \`fakeFile\`. If that's the case, restart the test runner.
Please fix this, or set \`fakeFile\` to \`false\` if a plugins file is not necessary for your project.
${details(arg2)}
`
},arg1 will be highlighted in blue for the terminal, and wrapped in backticks when called with forBrowser. Details will be printed in yellow as a stack trace when printed to the terminal, or shown as a stack-trace in the browser.
Error Wrapping
Any time we know about an edge case that is an error, we should define an error in errors.ts. This error should be retrieved by getError, which converts it to a CypressError.
The CypressError is an Error containing the message returned from the errTemplate. The stack is set to that of the originalError if it exists (i.e. the error object passed into details), otherwise it's the stack from where the getError / throwError is called.
The CypressError has an isCypressErr prop which we use as a duck-type guard against exiting the process when logging exceptions. It also maintains a reference to the originalError if it exists.
Child-Process Errors
All errors that occur in a child process spawned by Cypress should be sent over the ipc bridge using util.serializeError.
This ensures the name, message, stack, and any other relevant details are preserved and can be handled by the standard process of Cypress' error standardization / wrapping.
Guides Eslint Migration (`guides/eslint-migration.md`)
ESLint Migration Guide: Monorepo Alignment
Migration Checklist
Batch 1: Small npm utilities
- npm/grep ✅ COMPLETED
- npm/puppeteer ✅ COMPLETED
- npm/mount-utils ✅ COMPLETED
- npm/cypress-schematic ✅ COMPLETED
Batch 2: Framework adapters
- npm/react ✅ COMPLETED
- npm/vue ✅ COMPLETED
- npm/svelte ✅ COMPLETED
- npm/angular ✅ COMPLETED
Batch 3: Build-related
- npm/webpack-batteries-included-preprocessor
- npm/webpack-preprocessor
- npm/webpack-dev-server
- npm/vite-plugin-cypress-esm
- npm/vite-dev-server
Batch 4a: Core packages (part 1)
- packages/frontend-shared
- packages/icons
- packages/launcher
- packages/https-proxy
- packages/proxy
- packages/net-stubbing
- packages/driver
- packages/reporter
- packages/server
Batch 4b: Core packages (part 2)
- packages/runner
- packages/extension
- packages/network
- packages/socket
- packages/telemetry
- packages/launchpad
- packages/errors
- packages/data-context
- packages/app
Batch 4c: Core packages (part 3)
- cli
- packages/agent-info ✅ COMPLETED
- packages/cypress-sessions ✅ COMPLETED
- packages/electron ✅ COMPLETED
- packages/eslint-config ✅ COMPLETED — the shared config lints itself
- packages/stderr-filtering ✅ COMPLETED
- packages/config
- packages/root
- packages/resolve-dist
- packages/packherd-require
- packages/v8-snapshot-require
- packages/web-config
- packages/types
- packages/example
Batch 5: Test and script folders
- system-tests
- scripts
This guide describes how to migrate all packages in the Cypress monorepo to the new unified ESLint configuration (@packages/eslint-config).
Why Migrate?
- Consistency: Enforce the same linting rules and plugins across all packages.
- Simplicity: Remove the need for custom plugins like
@cypress/eslint-plugin-dev. - Maintainability: Make it easier to update and maintain lint rules.
- Modernization: Replace obsolete TSLint with the appropriate ESLint TypeScript plugin (
@typescript-eslint).
Migration Strategy
1. **Lint-Staged Configuration**
lint-staged hands each staged file to the nearest package.json that has a lint-staged key, and runs that config's tasks from its directory. Routing is therefore per package, not in the root:
- Migrated packages carry their own
lint-stagedkey (step 4 below), so their files runeslint --fixfrom the package directory, resolving the package's local ESLint 9 and itseslint.config.ts. - Everything else falls through to the root
package.json, whose single*.{js,jsx,ts,tsx,json,eslintrc,vue}pattern runs ESLint 8 with the root.eslintrc.js. A pattern without a/matches at any depth, so it covers every directory on its own; adding directory-specific patterns beside it would lint those files twice.
Migrating a package needs no change to the root config. scripts/unit/lint-staged-spec.ts fails if a workspace with an eslint.config.* has no lint-staged key. Add the package to the --scope list of the root lint:fix script too.
2. **Batch Packages for Migration**
- Batch by directory/type to keep PRs manageable and reduce risk.
- Batch size: 4–8 packages per PR, grouped by similarity.
2. **Migration Steps for Each Package**
For each package in the batch:
- Remove old ESLint config and plugin references:
- Delete
.eslintrc,.eslintrc.json, or.eslintrc.jsin the package. - Delete
.eslintignore— flat config does not read it. Anything in it that is still needed becomes anignoresentry ineslint.config.ts. - Remove any references to
@cypress/eslint-plugin-devinpackage.json(if present). - Remove TSLint configs: Delete
tslint.jsonand removetslintdependencies frompackage.json.
- Delete
- Add a new ESLint config file:
- Create
eslint.config.tsin the package root:tsimport baseConfig from '@packages/eslint-config' export default baseConfig - To keep linting the package's JSON files, spread
jsonConfigafterbaseConfig, as described in Lint JSON withjsonConfig.
- Create
- Ensure dependencies are up to date:
- Remove any package-local ESLint plugins now provided by the shared config.
- Remove TSLint-related dependencies (the new config includes
@typescript-eslint). - Add the shared ESLint config as a dev dependency:
- In each migrated package, add
@packages/eslint-configtodevDependencies(use a relative file path if not published to npm).
- In each migrated package, add
- Add ESLint as a dev dependency:
- Since
@packages/eslint-confighas ESLint as a peer dependency, addeslint: "^9.18.0"todevDependencies.
- Since
- Add lint-staged configuration:
- Add a
lint-stagedsection to the package'spackage.json:json{ "lint-staged": { "**/*.{js,jsx,ts,tsx}": "eslint --fix" } } - This ensures that when files are staged for commit, they are automatically linted and fixed using the package's local ESLint configuration.
- Add a
- Run lint and autofix:
- From the package root, run:text
npx eslint . --ext .js,.ts,.tsx,.jsx --fix - Manually fix any remaining lint errors.
- From the package root, run:
- Verify TypeScript configuration:
- Ensure the package has a valid
tsconfig.jsonthat works with the new ESLint config. - Run
npx tsc --noEmitto check for TypeScript compilation errors. - Verify that the new ESLint config can properly parse TypeScript files in the package.
- Ensure the package has a valid
- Run tests for the package to ensure nothing broke.
- Commit changes with a clear message, e.g.:text
chore(npm/grep): migrate to @packages/eslint-config and remove legacy eslint-plugin-dev
4. **Open a PR for Each Batch**
- Keep each migration PR focused (one batch per PR).
- List all affected packages in the PR description.
- Include a checklist for each package:
- Removed old ESLint config
- Added new config
- Added lint-staged configuration
- Ran lint and fixed errors
- Ran tests
5. **Document Issues or Gaps**
- If you hit any missing rules or plugin gaps, note them for follow-up.
- If a package needs a custom override, add it in a local
eslint.config.ts(prefer to upstream to the shared config if possible).
6. **Ignore Build Output Centrally**
Compiled output (cjs/, esm/, dist/) is ignored centrally, in both config homes, and neither list should be duplicated per package:
- root
.eslintrc.jsignorePatterns— for the packages still on eslintrc packages/eslint-config/src/baseConfig.tsignores— for the migrated ones
A package-level .eslintignore cannot do this job, which is worth knowing before reaching for one. ESLint 8 merges every ignore source into a single predicate based at the common ancestor of their base paths, and the root .eslintrc.js ignorePatterns pulls that ancestor up to the repo root. Entries from an .eslintignore are flagged loose, so they are re-based without being re-prefixed: a package-relative entry keeps its spelling and is then matched against the repo root. Anything containing a / is anchored there and silently matches nothing — packages/resolve-dist/.eslintignore listed cjs/index.js and ESLint linted the file anyway, failing linux-lint on develop. Only unanchored spellings survive the re-base:
| Pattern | Result |
|---|---|
**/tsconfig.json |
matches at any depth |
cjs/ |
matches at any depth (a trailing slash is not an anchor) |
cjs/index.js |
anchored at the repo root, matches nothing |
/src/thing.ts |
re-prefixed with the package path, matches |
ESLINT_USE_FLAT_CONFIG packages are not affected: flat config does not read .eslintignore at all, so a migrated package must express ignores as an ignores entry. Delete any .eslintignore left behind when migrating a package — it is dead weight once the package is on flat config.
Note also that cjs/esm are anchored to the workspace roots (packages/*, npm/*, tooling/*) rather than written as **/esm/**. System-tests fixtures ship handwritten sources under an esm/ directory, and an unanchored pattern would stop linting them without any signal.
7. **Lint JSON with `jsonConfig`**
Under eslintrc, the @cypress/dev/general preset ran every *.json file through eslint-plugin-json-format. That plugin works by patching ESLint 8's Linter, so it has no flat-config equivalent, and baseConfig matches no .json files: a migrated package lints no JSON at all unless it opts in. To opt in, spread jsonConfig after baseConfig:
import { baseConfig, jsonConfig } from '@packages/eslint-config'
export default [
...baseConfig,
...jsonConfig,
]jsonConfig is built on eslint-plugin-jsonc and keeps what the old preset enforced:
Old preset (eslint-plugin-json-format) |
jsonConfig |
|---|---|
| File must parse as JSON | Parse errors, plus jsonc/recommended-with-json (no comments, no trailing commas, no duplicate keys) |
Reformatted to JSON.stringify(value, null, 2) plus a final newline |
jsonc/indent, the jsonc/*-newline rules, @stylistic/eol-last, jsonc/array-bracket-spacing, which also collapses [ ], and @stylistic/no-multiple-empty-lines set to 0. All of them autofix. |
package.json keys in the "pro" order, then in sort-package-json's order |
package-json/order-properties in sort-package-json's standard order, with no "pro" list. Every top-level key is checked, and keys that order does not know go last in alphabetical order |
package.json scripts and dependency maps sorted |
package-json/sort-collections (from eslint-plugin-package-json), which uses sort-package-json too. It keeps pre/post hooks and name:variant scripts next to their base script, and sorts the dependency maps in ascending order. exports is left out of its default list, because the order of conditions there is significant |
Skipped tsconfig.json, .vscode/** and package-lock.json |
Same skips, widened to every tsconfig*.json variant, since tsc reads all of them as JSONC |
@eslint/json was considered and rejected. It is a separate ESLint language, so every JS rule in baseConfig that has no files restriction would need fencing off. It has no formatting rules, and its sort-keys only sorts alphabetically, so it cannot express the package.json order. eslint-plugin-jsonc parses JSON into an AST that the existing JS rules ignore and autofixes the layout, and eslint-plugin-package-json autofixes package.json ordering on the same AST. eslint-plugin-jsonc is pinned to v2 because v3 needs ESLint 9.38 or later. eslint-plugin-package-json is pinned to 0.x for a related reason: 1.x moves to jsonc-eslint-parser v3 and declares a peer dependency on @eslint/json.
Two differences from the old preset remain:
- Top-level key order: the old preset used the "pro" order, which put
scriptsand the dependency maps near the top.jsonConfigusessort-package-json's standard order, soscriptsand the dependencies follow the metadata (license,main,types,filesand so on). Nearly everypackage.jsonin the repo follows the "pro" order today, so the firsteslint --fixafter opting in reorders the top-level keys. This one is deliberate. - Empty objects:
{ }is not collapsed to{}.jsonc/object-curly-spacingskips empty objects, and no off-the-shelf rule covers them.
When a package opts in, add json to its lint-staged glob. Without jsonConfig, staging a .json file only produces a "File ignored because no matching configuration was supplied" warning. Running eslint --fix once on opt-in sorts any package.json that drifted while the package was not linting JSON. @packages/errors is the reference opt-in.
8. **Deprecate and Remove Old Plugin**
@cypress/eslint-plugin-devis alreadyprivateand no longer publishes.7.0.0stays on npm for anything pinned to it.- Its two custom rules (
arrow-body-multiline-braces,skip-comment) are mirrored in@packages/eslint-configunder the same@cypress/devnamespace, so a package gets the same behavior either side of the migration and existingeslint-disable @cypress/dev/...comments keep resolving. Until the plugin is gone, a change to either rule belongs in both. - The plugin still cannot be deleted: the root
.eslintrc.jsand the nested eslintrc files extend itsgeneral/tests/reactpresets, so it is what every unmigrated package lints against. Once all packages are migrated, remove it from the repo and CI, along with the root.eslintrc.js, the root.eslintignore, andeslint8 from the rootpackage.json.
9. **Lint-Staged After the Migration**
The root lint-staged config is already a single catch-all pattern, so it needs no change when the last package migrates. Once the root itself moves to flat config, that same pattern runs ESLint 9.
10. **Update Lerna/Monorepo Config**
- Ensure all packages reference the new config in their
package.json/eslint.config.ts. - Update documentation and developer onboarding guides.
Example: Migrating a Batch
- Select batch: e.g.,
npm/grep,npm/puppeteer,npm/mount-utils,npm/cypress-schematic - For each package:
- Remove
.eslintrc*files - Add
eslint.config.tsas above - Remove local plugin deps
- Run lint and fix errors
- Run tests
- Remove
- Commit and open PR
Troubleshooting Guide
Common Issues and Solutions
1. **Jiti Version Compatibility Error**
Error: Error: You are using an outdated version of the 'jiti' library. Please update to the latest version of 'jiti' to ensure compatibility and access to the latest features.
Solution:
- Add
jiti: "^2.4.2"to the package'sdevDependencies - ESLint 9.x requires jiti >= 2.2.0, but monorepo dependencies might provide older versions
2. **TypeScript Project Service Errors**
Error: was not found by the project service. Consider either including it in the tsconfig.json or including it in allowDefaultProject
Solutions:
- Create/update
tsconfig.jsonthat extends the base config:json{ "extends": "../../packages/ts/tsconfig.json", "compilerOptions": { "esModuleInterop": true, "allowJs": true, "checkJs": false }, "include": [ "src/**/*", "cypress/**/*", "*.js", "*.ts", "*.jsx", "*.tsx" ], "exclude": ["node_modules", "dist"] } - For Cypress packages, ensure
cypress/tsconfig.jsonincludes all test files:json{ "compilerOptions": { "types": ["cypress"] }, "include": [ "**/*.ts", "**/*.js" ] } - Add
allowDefaultProject: trueto ESLint config for problematic files:ts{ files: ['**/*.js', '**/*.ts', '**/*.jsx', '**/*.tsx'], languageOptions: { parserOptions: { allowDefaultProject: true, }, }, }
3. **Skip Comment Rule Violations**
Error: @cypress/dev/skip-comment rule violations for it.skip() tests
Solution:
- Replace
eslint-disable-next-line @cypress/dev/skip-commentwith proper explanatory comments:js// NOTE: This test is skipped for demonstration purposes it.skip('first test', () => {})
4. **Rule Mismatches in Pre-commit Hook**
Error: ESLint rule violations that don't match the package's ESLint configuration (e.g., Unexpected console statement when no-console is off in package config)
Root Cause: The package has no lint-staged key in its package.json, so its files fall through to the root config and are linted by ESLint 8 against .eslintrc.js.
Solution: Add the lint-staged key from step 4. See Lint-Staged Configuration.
5. **ESLint Script Changes**
Before ESLint 9.x:
"lint": "eslint . --ext .js,.ts"After ESLint 9.x:
"lint": "eslint"ESLint 9.x auto-detects file extensions, so --ext flag is no longer needed.
6. **Package Dependencies**
Required additions to package.json:
{
"devDependencies": {
"@packages/eslint-config": "0.0.0-development",
"eslint": "^9.18.0",
"jiti": "^2.4.2"
}
}Why jiti? ESLint 9 loads TypeScript flat config files (eslint.config.ts) at runtime via jiti, not via a static import in package source. Add it as an explicit devDependency in each migrated package so Yarn resolves jiti@^2.4.2 locally instead of a hoisted older version (see troubleshooting §1).
Why doesn't yarn health-check / knip flag jiti? Knip ties the eslint binary to eslint.config.ts as config infrastructure and does not treat jiti as an unused dependency in migrated npm packages. It is still required at lint runtime even though no source file imports it.
7. **Custom Package Rules**
If a package needs custom rules, extend the base config:
import baseConfig from '@packages/eslint-config'
export default [
...baseConfig,
{
files: ['**/*.js', '**/*.ts', '**/*.jsx', '**/*.tsx'],
rules: {
'no-console': 'off',
'no-restricted-syntax': 'off',
},
},
{
files: ['cypress/**/*.js', 'cypress/**/*.ts'],
languageOptions: {
globals: {
Cypress: 'readonly',
cy: 'readonly',
},
},
},
]Migration Checklist Template
For each package, ensure you've completed:
- Removed
.eslintrc*files - Created
eslint.config.tswith proper configuration - Added required dependencies (
eslint,@packages/eslint-config,jiti) - Added lint-staged configuration to
package.json - Spread
jsonConfigafterbaseConfigand includedjsonin the lint-staged glob (see Lint JSON withjsonConfig) - Added the package to the
--scopelist of the rootlint:fixscript - Created/updated
tsconfig.jsonthat extends base config - Updated ESLint scripts (removed
--extflag) - Ran
yarn lintsuccessfully - Ran tests to ensure nothing broke
Tips
- Use a tracking issue or project board to coordinate and document progress.
- If a package is especially noisy, consider splitting it into its own PR.
- Communicate with the team about the migration timeline and process.
- Test the pre-commit hook before pushing to ensure lint-staged works correctly.
Happy linting!
Guides Esm Migration (`guides/esm-migration.md`)
ESM Migration Guide: Monorepo Alignment
Migration Checklist
Phase 1: Convert Packages to TypeScript
Criteria
- No require statements are to be used in
.tsfiles - Unit tests in package are written in TypeScript
- Does not include scripts or system test migrations
Notes
When migrating some of these projects away from the ts-node entry see @packages/scaffold-config example, it is somewhat difficult to make separate browser/node entries as the v8-snapshot tsconfig.json is using an older style of module resolution where the exports key inside a package's package.json is not well supported. Because of this, we need to find ways to bundle code that is needed internally in the browser vs in node without them being a part of the same bundle. This is a temporary work around until we are able to get every package being able to build as an ES Module, which as that point we can re assess how the Cypress binary is being built as well as v8-snapshots, and will allow us to reconfigure this packages to export content in a more proper fashion. We are currently doing something similar in the following packages:
@packages/scaffold-config@packages/socket@packages/telemetry
Status
NPM Packages
- cli ✅ COMPLETED
- npm/angular ✅ COMPLETED
- npm/cypress-schematic ✅ COMPLETED
npm/eslint-plugin-dev— SKIP: private, no longer published, and deleted once every package is on@packages/eslint-config. See the ESLint migration guide.- npm/grep ✅ COMPLETED
- npm/mount-utils ✅ COMPLETED
- npm/puppeteer ✅ COMPLETED
- npm/react ✅ COMPLETED
- npm/svelte ✅ COMPLETED
- npm/vite-dev-server ✅ COMPLETED
- vite-plugin-cypress-esm ✅ COMPLETED
- npm/vue ✅ COMPLETED
- npm/webpack-batteries-included-preprocessor ✅ COMPLETED
- npm/webpack-dev-server ✅ COMPLETED
- npm/webpack-preprocessor ✅ COMPLETED
Binary Packages
- packages/app PARTIAL - low priority: frontend package
- packages/config ✅ COMPLETED
- packages/data-context PARTIAL - entry point is JS
- packages/driver ✅ COMPLETED - source complete, cypress tests need migration
- packages/electron ✅ COMPLETED
- packages/error ✅ COMPLETED
- packages/eslint-config ✅ COMPLETED
- packages/example
- packages/extension ✅ COMPLETED
- packages/frontend-shared PARTIAL - entry point is JS
- packages/electron ✅ COMPLETED
- packages/https-proxy - ✅ COMPLETED
- packages/icons ✅ COMPLETED
- packages/launcher ✅ COMPLETED
- packages/launchpad ✅ COMPLETED
- packages/net-stubbing ✅ COMPLETED
- packages/network ✅ COMPLETED
- packages/network-tools ✅ COMPLETED
- packages/packherd-require ✅ COMPLETED
- packages/proxy PARTIAL - entry point is JS
- packages/reporter ✅ COMPLETED
- packages/resolve-dist ✅ COMPLETED
- packages/root ✅ COMPLETED
- packages/runner ✅ COMPLETED
- packages/scaffold-config ✅ COMPLETED
- packages/server PARTIAL - many source/test files in JS. highest priority
- packages/socket ✅ COMPLETED
- packages/stderr-filtering ✅ COMPLETED
- packages/telemetry ✅ COMPLETED
- packages/ts PARTIAL - ultimate goal is removal and likely not worth the effort to convert
- packages/types ✅ COMPLETED
- packages/v8-snapshot-require
- packages/web-config
Phase 2: Convert Package tests from Mocha to Vitest
Status
NPM Packages
- cli ✅ COMPLETED
- npm/cypress-schematic ✅ COMPLETED
- npm/eslint-plugin-dev ✅ COMPLETED
- npm/grep ✅ COMPLETED
- npm/puppeteer ✅ COMPLETED
- npm/vite-dev-server ✅ COMPLETED
- npm/webpack-batteries-included-preprocessor ✅ COMPLETED
- npm/webpack-dev-server ✅ COMPLETED
- npm/webpack-preprocessor ✅ COMPLETED
Binary Packages
- packages/config ✅ COMPLETED
- packages/data-context COMPLETED (migrated from
mocha/sinon/chaitojest). See package README for more details as to whyjestovervitest - packages/driver ✅ COMPLETED
- packages/electron ✅ COMPLETED
- packages/error ✅ COMPLETED
- packages/extension ✅ COMPLETED
- packages/https-proxy ✅ COMPLETED
- packages/electron ✅ COMPLETED
- packages/icons ✅ COMPLETED
- packages/launcher ✅ COMPLETED
- packages/net-stubbing ✅ COMPLETED
- packages/network ✅ COMPLETED
- packages/network-tools ✅ COMPLETED
- packages/packherd-require ✅ COMPLETED
- packages/proxy ✅ COMPLETED
- packages/scaffold-config ✅ COMPLETED
- packages/server
- packages/socket ✅ COMPLETED
- packages/stderr-filtering ✅ COMPLETED
- packages/telemetry ✅ COMPLETED
- packages/ts - ultimate goal is removal and likely not worth the effort to convert
- packages/types ✅ COMPLETED
- packages/v8-snapshot-require ✅ COMPLETED
Phase 3: Bundle ESM/CJS versions of NPM packages
TBD: details will be clearer at the end of Phase 2
Lint ignores build output centrally, so a package that emits cjs/ and esm/ needs no ignore entries of its own: the root .eslintrc.js ignorePatterns covers packages still on eslintrc, baseConfig.ts ignores covers the ones on flat config, and any emitter under packages/*, npm/* or tooling/* is covered as soon as it is added. Two things to keep in mind if Phase 3 changes the output layout: the linux-lint job restores a built workspace, so generated output is on disk when ESLint runs and an uncovered emitter fails CI rather than only local yarn build && yarn lint; and output directories outside those workspace roots (or named something other than cjs/esm/dist) need their pattern added centrally. See the ESLint migration guide for why this cannot be done with a per-package .eslintignore.
Phase 4: Run Cypress server as an ESM package
TBD: details will be clearer at the end of Phase 2
Guides Graphql Subscriptions (`guides/graphql-subscriptions.md`)
GraphQL Subscriptions Overview & Test Guide:
In GraphQL there are currently 3 types of Operations: query, mutation, subscription
Query is similar to a rest-ful GET request:
query MyAppData {
cloudUser { # CloudUser type
id
email
fullName
}
currentProject { # Project type
id
name
...CurrentProjectCard
}
app { # App type
localSettings {
...LocalSettingsView
}
}
}Whereas a mutation is like a POST request, where you ask for what you want back (what changed b/c of the mutation):
mutation MyAppMutation($testingType: TestingTypeEnum) {
chooseTestingType(testingType: $testingType) { # Project type
id
currentTestingType
}
}A subscription is used when you want to receive information about data that has changed scoped to an individual event:
subscription MyAppSubscription {
projectUpdated { # Project type
id
isLoadingConfig
isLoadingSetupNodeEvents
}
}The distinction between subscriptions & queries is that subscriptions can only have a single top-level field. You can think of the subscription like an "event name" in socket.io, except that it's strongly typed & part of the overall GraphQL schema, meaning that you don't need to do any extra work for it to merge into the normalized cache & refresh affected parts of the view.
// will merge & update any views that depend on `isLoadingConfig` / `isLoadingSetupNodeEvents`
useSubscription({ query: MyAppSubscriptionDocument })We can also use subscriptions more granularly as an event emitter, basically as a strongly typed socket.io emitter:
subscription OnSpecChange {
onSpecUpdate {
specPath
reason
}
}useSubscription({ query: OnSpecChangeDocument }, (prev, next) => {
if (data.specPath === currentSpecPath && reason === 'DELETED') {
// navigate to another page
} else {
// Rerun spec
}
return next
})Client Details:
One thing to be aware of, is the subscription is only mounted/responded to when the containing component is mounted on the page. Therefore, one rule of thumb when using subscriptions is to ensure that they are declared in the subscription dependent view that is highest in the hierarchy so that the subscription can be active when it needs to be. This location is often alongside a query rather than a fragment.
Server Docs
Subscriptions are implemented on the server as an AsyncIterator. This is handled for us by the graphql-ws package.
To add a new Subscription field, add a new entry in the gql-Subscriptions:
t.field('browserStatusChange', {
type: CurrentProject,
description: 'Status of the currently opened browser',
subscribe: (source, args, ctx) => ctx.emitter.subscribeTo('browserStatusChange'),
resolve: (source, args, ctx) => ({}),
})And then add the corresponding method in DataEmitterActions
browserStatusChange () {
this._emit('browserStatusChange')
}Testing
If you want to TDD the subscription being added & working directly in isolation, one recommended approach is to add one spec file per-subscription in the /app/cypress/e2e/subscription directory. This file can cover both the app & launchpad handling of a given subscription.
Example: authChange-subscription.cy.ts
Guides Next Version (`guides/next-version.md`)
Next Version
When the Cypress CLI and binary are built in CI, the version number they share is determined by the script in /scripts/get-next-version.js.
In most cases, the script will correctly determine the next version. If it needs to be overridden, the environment variable NEXT_VERSION can be set to the desired version (NEXT_VERSION=1.2.3). This is commonly desired on a release branch or when trying to force a specific major version.
The get-next-version.js script follows these steps:
- If the environment variable
NEXT_VERSIONexists, printNEXT_VERSIONand exit. - Otherwise, analyze the commits to the current branch since the last release, using semantic commit messages. Print out the calculated version.
- Only commits that touch files in
packages/*orcli/*are considered.- This is done so that commits to the
npm/packages do not necessarily bump the CLI and binary versions.
- This is done so that commits to the
- This project uses the
angularcommit message style.- The prefix
fix:will trigger a patch version bump. - The prefix
feat:will trigger a minor version bump. - A commit with the footer containing
BREAKING CHANGE:should trigger a major version bump.
- The prefix
- Only commits that touch files in
You can debug the script locally by running it using node ./scripts/get-next-version.js. It will analyze the commits on your current branch and print out the calculated version number.
Guides Patch Package (`guides/patch-package.md`)
Patching packages
Sometimes we need to patch node_modules that are not in our control in order to fix bugs or add features. There are a few ways to do this:
- Fork the package to the
cypress-ioorg and install via Git hash - Re-publish a patched version under the
@cypressorg on NPM - Patch the package using the
patch-packageutility on install/build
In most cases, it is best to use patch-package. Using patch-package has a number of advantages over #1 and #2:
patch-packageavoids the need for maintaining yet another repo ornpm/packagepatch-packageavoids the need for keeping version numbers/Git hashes synced inpackage.json/yarn.lockin the monorepopatch-packagemakes it easy to review changes in the context of a single PR to thecypressrepo, as opposed to having to review changes in 2+ repospatch-packageside-steps a bug in Yarn that causes extremely confusing behavior when installing/caching Git dependencies
The only times where we cannot use patch-package is if we need to make a patch that is not included in the binary. The cli and npm/ packages have their transitive dependencies installed by the user's package manager, so we cannot use patch-package to patch them.
For example: @cypress/request is used in the CLI, so we maintain a separate NPM package.
Also, we cannot include Git dependencies (#1) in any NPM packages, because not all users can install Git dependencies: #6752
Upstreaming patches
If your patch is general purpose, you should submit a PR to the dependency's repo and create an issue in the cypress repo that tracks your upstream PR.
Once your upstream PR is merged, we can bump the version of the patched module in the monorepo and remove the patch, along with associated maintenance burden.
Testing patches
All patches require tests.
Along with regular unit/integration/etc. tests against unbuilt Cypress, there should be at least one test for the patch that uses the built version of Cypress. This prevents regressions from a patch not being applied as expected when we build Cypress.
You can add a test for your patch against the built binary in a couple of ways:
- Create a
binary-system-testthat tests that the patched behavior is correct in the built binary. - Add an expectation to
scripts/binary/util/testStaticAssets.jsthat asserts the patch is applied. - Add some other test that runs against the built binary in CI.
Guides Protocol Development (`guides/protocol-development.md`)
Protocol Development
In production, the capture code used to capture and communicate test data will be retrieved from the Cloud. However, in order to develop the capture code locally, developers will:
- Clone the
cypress-servicesrepo (this requires that you be a member of the Cypress organization)- Run
yarn - Run
yarn watchinapp/packages/capture-protocol
- Run
- Clone the
cypressrepo- Run
yarn - Execute
CYPRESS_LOCAL_PROTOCOL_PATH=path/to/cypress-services/app/packages/capture-protocol/dist/index.js CYPRESS_INTERNAL_ENV=staging yarn cypress:run --record --key <record_key> --projecton a project in record mode
- Run
Guides Purging Cloudflare Cache (`guides/purging-cloudflare-cache.md`)
Purging Specific URLs from Cloudflare cache
In some cases, pre-released cypress binaries will 404 when being downloaded from the Cypress CDN.
yarn add -D cypress@https://cdn.cypress.io/beta/npm/vX.Y.Z/OPERATING_ARCH/BRANCH_AND_HASH_NAME/cypress.tgz
npm ERR! URL:https://cdn.cypress.io/beta/binary/vX.Y.Z/OPERATING_ARCH/BRANCH_AND_HASH_NAME/cypress.zip
npm ERR! Error: Failed downloading the Cypress binary.
npm ERR! Response code: 404
npm ERR! Response message: Not Found
npm ERR!When this happens, the Cloudflare URL cache for the pre-released binary likely needs to be cleared. This can be done with the following command:
node ./scripts/binary.js purge-urls --f ./path_to_file_with_urls_to_purgeWhere ./path_to_file_with_urls_to_purge is a text file of URLs separated by new lines. For example, the below text file would clear the cache for a given pre-release:
https://cdn.cypress.io/beta/npm/vX.Y.Z/win32-x64/BRANCH_AND_HASH_NAME/cypress.tgz
https://cdn.cypress.io/beta/binary/vX.Y.Z/win32-x64/BRANCH_AND_HASH_NAME/cypress.zip
https://cdn.cypress.io/beta/npm/vX.Y.Z/linux-arm64/BRANCH_AND_HASH_NAME/cypress.tgz
https://cdn.cypress.io/beta/binary/vX.Y.Z/linux-arm64/BRANCH_AND_HASH_NAME/cypress.zip
https://cdn.cypress.io/beta/npm/vX.Y.Z/linux-x64/BRANCH_AND_HASH_NAME/cypress.tgz
https://cdn.cypress.io/beta/binary/vX.Y.Z/linux-x64/BRANCH_AND_HASH_NAME/cypress.zip
https://cdn.cypress.io/beta/npm/vX.Y.Z/darwin-x64/BRANCH_AND_HASH_NAME/cypress.tgz
https://cdn.cypress.io/beta/binary/vX.Y.Z/darwin-x64/BRANCH_AND_HASH_NAME/cypress.zip
https://cdn.cypress.io/beta/npm/vX.Y.Z/darwin-arm64/BRANCH_AND_HASH_NAME/cypress.tgz
https://cdn.cypress.io/beta/binary/vX.Y.Z/darwin-arm64/BRANCH_AND_HASH_NAME/cypress.zipNote: you likely need to clear out the corresponding .zip files alongside the .tgz (included in the example)
You will need the Cloudflare cache variables mentioned in the release process.
Guides Release Process (`guides/release-process.md`)
Release Process
These procedures concern the release process for the Cypress binary and cypress npm module.
The @cypress/-namespaced NPM packages that live inside the /npm directory are automatically published to npm (with semantic-release) upon being merged into develop. You can read more about this in CONTRIBUTING.md.
Anyone can build the binary and npm package locally, but you can only deploy the Cypress application and publish the npm module cypress if you are a member of the cypress npm organization.
Publishing
Prerequisites
Ensure you have the following permissions set up:
- An AWS account with permission to access and write to the AWS S3, i.e. the Cypress CDN.
- Permissions for your npm account to publish the
cypresspackage.
Set up an AWS SSO profile with the Team-CypressApp-Prod role. The release scripts assumes the name of your profile is
prod. Make sure to open the "App Developer" expando for some necessary config values. Your AWS config file should end up looking like the following:text[profile prod] sso_start_url = <start_url> sso_region = <region> sso_account_id = <account_id> sso_role_name = <role_name> region = <region> cli_pager = <pager>Set up the following environment variables:
For the
release-automationsstep, you will need setup the following envs:- GitHub token - Found in 1Password.
- The
cypress-botGitHub app credentials. Found in 1Password.
textGITHUB_TOKEN="..." GITHUB_APP_CYPRESS_INSTALLATION_ID= GITHUB_APP_ID= GITHUB_PRIVATE_KEY=For purging the Cloudflare cache (needed for the
prepare-release-artifactsscript in step 6 andbinary-releasein step 14), you'll needCF_ZONEIDandCF_TOKENset. These can be found in 1Password.textCF_ZONEID="..." CF_TOKEN="..."
Ensure that you have the following repositories checked out locally and ready to contribute to:
If you don't have access to 1Password, ask a team member who has done a deploy.
Before Publishing a New Version
In order to publish a new version of the cypress package to the npm registry, CI must build and test it across multiple platforms and test projects. CI is set up to do the following on every commit to develop:
- Build the npm package with the next target version baked in.
- Build the Linux, Mac & Windows binaries on CircleCI. Linux x64 builds on every commit; Linux ARM64, Mac, and Windows build from the nightly schedule or a manual trigger (see below).
- Upload the binaries and the new npm package to the AWS S3 Bucket
cdn.cypress.iounder the "beta" folder. - Launch test projects using the newly-uploaded package & binary instead of installing from the npm registry.
Multiple test projects are launched for each target operating system and the results are reported
back to GitHub using status checks so that you can see if a change has broken real-world usage
of Cypress. You can see the progress of the test projects by opening the status checks on GitHub:
Steps to Publish a New Version
In the following instructions, "X.Y.Z" is used to denote the next version of Cypress being published.
Note: It is advisable to notify the team that the develop branch is locked down prior to beginning the release process
Install and test the pre-release version to make sure everything is working. See Install Pre-Release Version docs for more details.
- Install the new version:
- Globally:
npm install <cypress.tgz path> - or in a project:
npm i -D cypress@file:<cypress.tgz path>
- Globally:
- Run a quick, manual smoke test:
cypress open- Go into a project, run a quick test, make sure things look right
- Optionally, install the new version into an established project and run the tests there
- cypress-realworld-app uses yarn and represents a typical consumer implementation.
- Optionally, do more thorough tests, for example test the new version of Cypress against the Cypress Cloud repo.
- Install the new version:
Ensure all changes to the links manifest to
on.cypress.iohave been merged todevelopand deployed.(optional) Create a Release PR -
Bump, submit, get approvals on, and merge a new PR. This PR should:- Bump the
packages/exampledependency if there is a newcypress-example-kitchensinkversion, andyarnto ensure the lockfile is up to date. - Follow the writing the Cypress Changelog release steps to update the
cli/CHANGELOG.md. - NOTE: If edits to the
cli/CHANGELOG.mdare not needed, nor is a new version ofcypress-example-kitchensink, this step can be skipped and the last build offdevelopcan be used below.
- Bump the
Once the
developbranch is passing in CI and you have confirmed thecypress-bothas commented on the commit with the pre-release versions fordarwin-x64,darwin-arm64,linux-x64,linux-arm64, andwin32-x64, publishing can proceed.darwin-x64,darwin-arm64,linux-arm64, andwindowsdon't build on everydevelopcommit — they run from a nightly CircleCI Scheduled Pipeline instead (see.circleci/AGENTS.md). Merges todevelopcut off at 8PM EST the day before a release. Wait for that night's scheduled build to land on the release sha, and confirm it's green, before starting release day. If the schedule missed the sha or failed, trigger it manually withforce-persist-artifacts=trueon that exact sha.Tips for getting a green build:
- If the
windowsworkflow is failing with timeout errors, you can retry from the last failed step. - Sometimes a test can get stuck in a failing state between attempts on the
windowsworkflow. In these cases, kicking off a full run of the workflow can help get it into a passing state. - If the
linux-x64workflow fails due to a flaky test but percy finalizes the build, you must restart the workflow from the failed steps. Restarting the entire workflow after a finalized Percy build can cause Percy to fail the next attempt with a "Build has already been finalized" error, requiring pushing a new commit to start fresh.
- If the
Log into AWS SSO with
aws sso login --profile <name_of_profile>. If you have setup your credentials under a different profile thanprod, be sure to set theAWS_PROFILEenvironment variable to that profile name for the remaining steps. For example, if you are usingproductioninstead ofprod, doexport AWS_PROFILE=production.Use the
prepare-release-artifactsscript (Mac/Linux only) to prepare the latest commit to a stable release. When you run this script, the following happens:- the binaries for
<commit sha>are moved frombetato thedesktopfolder for<new target version>in S3 - the Cloudflare cache for this version is purged
- the pre-prod
cypress.tgzNPM package is converted to a stable NPM package ready for release - if on MacOS, you will need to prefix the command with
COPYFILE_DISABLE=1as OSX may include hidden files in the.tgzfile of the binary
shellyarn prepare-release-artifacts --sha <commit sha> --version <new target version>You can pass
--dry-runto see the commands this would run under the hood.- the binaries for
Validate you are logged in to
npmwithnpm whoami. Otherwise log in interactively withnpm login, as the npm service account. The login prompts for a 2FA one-time password, so the service account must have 2FA enabled.
If you are not already a Cypress package maintainer, contact a team member who is to get you added.Publish the generated npm package under the
devtag, using the npm service account. Trusted publishing only works from CI, so thecypresspackage is published by hand. This step needs only the interactive 2FA login from the previous step. It does not use theNPM_TOKENin theorg-npm-credentialsCircleCI context, and rotating or revoking that token does not affect it.shellnpm publish /tmp/cypress-prod.tgz --tag devDouble-check that the new version has been published under the
devtag usingnpm view cypress dist-tags. Thelatesttag should still point to the previous version. Example output:text$ npm view cypress dist-tags { latest: '14.5.3', dev: '14.5.4' }Note: It may take several minutes for
npm viewto reflect the latest version info.Test
[email protected]to make sure everything is working.- Install the new version:
npm install [email protected] - Run a quick, manual smoke test:
cypress open- Go into a project, run a quick test, make sure things look right
- Install the new version into an established project and run the tests there
- cypress-realworld-app uses yarn and represents a typical consumer implementation.
- Optionally, do more thorough tests, for example test the new version of Cypress against the Cypress Cloud repo.
- Install the new version:
Review the release-specific documentation and changelog PR in cypress-documentation. If there is not already a release-specific PR open, create one.
- Copy the changelog content for this version from the release PR above into
/docs/app/references/changelog.mdx. Adjust anydocs.cypress.iolinks to use host-relative paths. - Add the release date in
_Released MMM DD, YYYY_format below the version header in thechangelog.mdxentry. - Merge any release-specific documentation changes into the main release PR.
- You can view the doc's branch deploy preview by clicking 'Details' on the PR's
netlify-cypress-docs/deploy-previewGitHub status check.
- Copy the changelog content for this version from the release PR above into
Create new Docker images in
cypress-docker-imageswith the new Cypress version and the latest stable browsers. Create one PR in which you update factory/.env:CYPRESS_VERSION— the new Cypress version.CHROME_VERSIONandCHROME_FOR_TESTING_VERSION— look each up separately. They come from different sources, so their latest stable versions often differ.EDGE_VERSIONFIREFOX_VERSION— Firefox ships a new major version every 2 weeks, so expect this to have changed since the last release.GECKODRIVER_VERSION— only if a newer release exists.
Find each version at the link in the comment above it in
factory/.env. If a browser is already on its latest stable version, say so in the PR description so reviewers know it was checked. Don't changeFACTORY_VERSIONor the factory CHANGELOG for these updates. Ensure the Docker images are reviewed and have passing tests before proceeding.Make the new npm version the "latest" version by updating the dist-tag
latestto point to the new version:shellnpm dist-tag add [email protected]Run
binary-releaseto update the download server's manifest. This will also ensure the binary for the version is downloadable for each system.shellyarn binary-release --version X.Y.ZMerge the documentation PR from step 11 and the new docker image PR created in step 12 to release the image.
If needed, deploy the updated
cypress-example-kitchensinktoexample.cypress.ioby following these instructions under "Deployment".- Build
@packages/examplewithyarn workspace @packages/example build - Inspect the contents of
./packages/example/buildbefore deploying, and ensure it looks correct - Run
yarn workspace @packages/example deploy. This adds changes fromcypress-example-kitchensinkto a commit in thegh-pagesbranch, which will deploy to production with its own CI. - Check the deployed site at
https://example.cypress.ioto ensure the new changes deployed correctly.
- Build
Once the release is complete, tag the commit that was released: the
<commit sha>passed toprepare-release-artifactsin step 6.shellgit fetch origin develop git log -1 --oneline origin/develop # confirm it matches the step 6 sha git tag -a vX.Y.Z -m vX.Y.Z <commit sha> git push origin vX.Y.ZCreate a new GitHub release. Choose the tag you created previously and add contents to match previous releases.
Add a comment to each GH issue that has been resolved with the new published version. Download the
releaseData.jsonartifact from theverify-release-readinessCircleCI job and run the following command inside of cypress-io/release-automations:shellnpm run do:comment -- --release-data <path_to_releaseData.json>Confirm there are no issues from the release with the label stage: pending release left.
Notify the team that
developis reopen. The release announcement to the releases channel is automatically handled by the GitHub bot. If additional details are needed, they can posted as a reply to the release announcement.If utilizing the
SKIP_RELEASE_CHANGELOG_VALIDATION_FOR_BRANCHESto override and skip changelog validation for this release, change its value as needed or delete it from CircleCI so that subsequent releases and PRs will go through changelog validation.Check all
cypress-test-*andcypress-example-*repositories, and if there is a branch namedx.y.zfor testing the features or fixes from the newly published versionx.y.z, update that branch to refer to the newly published NPM version inpackage.json. Then, get the changes approved and merged into that project's main branch. For projects without ax.y.zbranch, you can go to the Renovate dependency issue and check the box next toUpdate dependency cypress to X.Y.Z. It will automatically create a PR. Once it passes, you can merge it. Try updating at least the following projects:Be sure to rotate the
NPM_TOKENvariable inside theorg-npm-credentialscontext inside CircleCI. This token is separate from the interactive login used to publishcypress. If an npm token was created when logging in, please delete this token now.
Take a break, you deserve it! 👉😎👉
Guides Studio Development (`guides/studio-development.md`)
Studio Development
Studio is delivered from Cypress Cloud. In production, the Studio bundle is retrieved from the Cloud at runtime. For local development, the environment variable CYPRESS_LOCAL_STUDIO_PATH can be used to run against a local Studio build from the cypress-services repo.
To run against locally developed Studio:
- Clone the
cypress-servicesrepo (requires Cypress organization membership)- Run
yarn - Run
yarn watchinapp/packages/studio - If developing against locally running
cypress-services: runyarn devin thecypress-servicesdirectory first
- Run
- Set environment variables:
CYPRESS_INTERNAL_ENV=<environment>(e.g.staging,production, ordevelopmentfor a localcypress-servicesinstance)CYPRESS_LOCAL_STUDIO_PATH: path tocypress-services/app/packages/studio/dist/development(overrides Cloud bundle when set)
- In the
cypressrepo:- Run
yarnandyarn cypress:open - Log in to Cypress Cloud in the app
- Use a project that has Studio enabled (e.g. in Cypress (staging) or Cypress Internal Org with the
studio-aifeature enabled for that project) - Open a project with E2E tests and use "Add Commands to Test" from a test to enter Studio
- Run
Note: With CYPRESS_LOCAL_STUDIO_PATH or when running the app from a local clone, error reporting is bypassed and errors are logged to the browser or Node console. With CYPRESS_LOCAL_STUDIO_PATH, the local Studio code is watched so you can change it without restarting the app.
Types
The studio bundle provides the types for the app and server interfaces that are used within the Cypress code. To incorporate the types into the code base, run:
yarn gulp downloadStudioTypesor to reference a local cypress_services repo:
CYPRESS_LOCAL_STUDIO_PATH=<path-to-cypress-services/app/packages/studio/dist/development-directory> yarn gulp downloadStudioTypesTesting
Unit/Component Testing
The code that supports cloud Studio and lives in the cypress monorepo is unit and component tested in a similar fashion to the rest of the code in the repo. See the contributing guide for more specifics.
The code that supports cloud Studio and lives in the cypress-services monorepo has unit and component tests that live alongside the code in that monorepo.
Cypress in Cypress Testing
Helpers for testing Studio with Cypress-in-Cypress live in packages/app/cypress/e2e/studio/helper.ts. The launchStudio helper:
- Loads a project (by default the
studiofixture project). - Navigates to the target spec (by default
specName.cy.js). - Enters Studio (new test or existing test via the
createNewTestparameter). - Waits for the test run to finish in Studio mode.
Those tests use the Studio bundle from the Cloud. The studio project uses a canary projectId so it receives the latest Cloud Studio build. For local Studio changes, set process.env.CYPRESS_LOCAL_STUDIO_PATH to your local studio build; Studio is enabled in that environment in packages/frontend-shared/cypress/e2e/e2ePluginSetup.ts.
In order to properly engage with Studio AI, we choose to simulate the cloud interactions that enable it via something like:
cy.mockNodeCloudRequest({
url: '/studio/testgen/n69px6/enabled',
method: 'get',
body: { enabled: true },
})To ensure that we get the same results from our Studio AI calls every time, we simulate them via something like:
const aiOutput = 'cy.get(\'button\').should(\'have.text\', \'Increment\')'
cy.mockNodeCloudRequest({
url: '/studio/testgen/n69px6/generate',
method: 'post',
body: { recommendations: [{ content: aiOutput }] },
})The above two helpers actually mock out the Node requests so we still test the interface between the browser and node with these tests.
Also, since protocol does not work properly on the inner Cypress of Cypress in Cypress tests, we choose to create a dummy protocol which means we need to provide a simulated CDP full snapshot that will be sent to AI via something like:
cy.mockStudioFullSnapshot({
id: 1,
nodeType: 1,
nodeName: 'div',
localName: 'div',
nodeValue: 'div',
children: [],
shadowRoots: [],
})Guides Testing Other Projects (`guides/testing-other-projects.md`)
Testing other projects
In develop and any other branch configured in the CircleCI config, the Cypress binary and npm package are built and uploaded to cdn.cypress.io. Then, tests are run, using a variety of real-world example repositories.
Two main strategies are used to spawn these test projects:
test-binary-against-repojobs- Remote CI
`test-binary-against-repo` jobs
A number of CI jobs in .circleci/config.yml clone test projects and run tests as part of cypress-io/cypress's CI pipeline.
You can find a list of test projects that do this by searching for usage of the test-binary-against-repo step.
Similarly to "Remote CI" test projects, Local CI test projects will attempt to check out a branch that is named after the next version (X.Y.Z) if one exists in the test project git repo.
One advantage to local CI is that it does not require creating commits to another repo.
`binary-system-tests`
System tests in /system-tests/test-binary are run against the built Cypress App in CI. For more details, see the README.
Guides Testing Strategy And Styleguide (`guides/testing-strategy-and-styleguide.md`)
DRAFT - CYPRESS TESTING STRATEGY + STYLE GUIDE
Status of this document
This draft is a starting point to help us converge on some common practices in terms of how we write tests, and a reference point for how we can review test PRs with each other. There are many valid style choices to make in writing tests, and some of them are contradictory, so a consistent approach is useful.
I'm starting by documenting the practices that I see us currently doing in the new parts of the codebase, or things we seem to have agreed we'd like to do. Not exhaustive, and not prescriptive. The Testing Strategy section existed already in Google Docs.
Testing Strategy
The purpose of this document is to detail the various ways to test the cypress repo including the Cypress App.
Testing Goals
The goal of testing is to instill confidence in the features being built and the bugs being fixed as well as save development hours.
Test driven development (TDD) is a core tenet to how we write tests at Cypress. Our approach is to Shift-left testing whenever possible.
Testing Types
- Unit - Tests the functionality of a single section of code, like a single function.
- Integration - Tests the functionality of a feature, with some portions (like a backend) mocked.
- End-to-end (E2E) - Tests the functionality of a feature with no mocking. It tests all pieces of the feature working together.
- Component - Tests a single component or group of components.
- System - A type of E2E test that tests an entire project in the context of Cypress.
- Snapshot - Captures text output and compares them to previously captured text output (like testing output for a CLI).
- Screenshot - Captures screenshots and compares them to previously captured screenshots.
- Performance - Tests the speed, response time, stability, reliability, scalability and resource usage of the application.
Testing Environments
Tests can run in different environments:
- Different browsers
- Different OSs
- Node.js
Testing Tools
- Mocha
- Cypress
- Snapshot tests using snap-shot-it - Use
SNAPSHOT_UPDATE=1 <test command>to update them, but make sure to check the diff to ensure it's the right update. - Percy snapshot tests - These only run in CI and can be managed from the GitHub status check on a PR
Testing Workflow
Developing features and fixing bugs should be test driven.
An example of building out a front-end facing feature may go like this:
- Check user requirements in the ticket/issue you’re assigned to.
- Write a failing component test for a features required in the user story.
- Build the piece of the component needed so that the failing test passes.
- Repeat steps 2-3 until the component satisfies the user requirements.
- Write out all CSS required for the feature to be design complete.
- Add a screenshot test.
- Write an E2E test to ensure all pieces and integrations work together.
- Add performance tests if necessary to ensure the feature doesn’t introduce regressions on performance.
An example of building out a feature/change to the CLI may go like this:
- Check user requirements in the ticket/issue you’re assigned to.
- Write a failing unit/integration test for the feature required in the user story.1. Write the logic for the new feature/change.
- Capture/update a snapshot test for any changes to the printed console output.
Component and E2E Tests in App, Launchpad, and Frontend-Shared Packages
Component Tests
Component tests should assert all expected behaviors of a component from a user perspective, as well as the "contract" of the component from a developer perspective. It is useful to write component tests as the primary driver of component development. This helps with development speed (no need to keep getting the whole app into the state you want), and test completeness.
For user-facing behavior, we interact with (or remount) the component to get it into all desired states and confirm they are valid. Each state should have a Percy snapshot in addition to regular Cypress assertions.
For developer-facing behavior - props received, events emitted, any other side effects, anything not covered in the UI tests can be explicitly asserted on its own. Props are often fully covered by UI assertions, but events emitted by the component to be used in parent components are not. So we can stub event handlers and verify that they are called in response to particular interactions.
Many components in the unified App and Launchpad define the data they require from GraphQL with a Fragment inside the component's script section, and receive the data from that Fragment in a prop called gql, so mocking out scenarios usually involves tweaking the mocked GraphQL data before the component mounts using mountFragment().
E2E Tests
Certain side effects, like GraphQL mutations, do not fire from component tests, but can be monitored from an E2E test with cy.intercept. And some entire packages, like reporter are independent apps that can be mounted in an E2E test and tested for interactions with other parts of the system.
Testing Style Guide
String constants
Strings used in tests for locating elements or asserting content in other ways are imported from the i18n constants. All strings present in the App or Launchpad UI should be found in some form in en-US.json. If a plain string is found in the UI code (we have a few of them), it should be moved into the constants file and then imported to both the component and the test code.
Element Locators
When relevant, element locators can assert something useful about the nature or context of the element in the DOM. Often for accessibility reasons it is important what elements are rendered. If there is nothing in particular that matters about the DOM, or if needed to disambiguate, then data-cy attributes can be used to locate elements.
Testing Interactive Elements
If the test will interact with an element, or assert the state of an interactive element, always prefer to locate that element with its accessible name (and, usually, an assertion about the element itself). Without an accessible name, the element will not be described correctly by a screen reader, which means certain disabled users would not know what the control or form field is for. Since we want every user to be able to interact with our app, a test that interacts with something should fail if there is no label for it.
Examples:
cy.contains('button', 'Log In').click()cy.findByLabelText('open navigation', {
selector: 'button',
}).click()Label-like locators that are not labels
We should be cautious with a locator like cy.findByPlaceholderText('My placeholder') to target a form input, as an input that only has a placeholder, but no label, is not fully accessible. Even though it may be useful to assert the placeholder contents for its own sake, we should prefer to locate an input by label when interacting.
Non-interactive elements
When assertions are made against non-interactive elements, if the surrounding HTML is relevant from an accessibility perspective, assert the relevant parts of the HTML:
cy.contains('h1', 'Page Title!').should('be.visible')cy.contains('h2', 'Modal Title').should('be.visible')This is a judgement call, there is a spectrum between asserting useful things about the DOM and writing a brittle assertion like this that adds no value to the test:
cy.contains('div > div > .special-list > li > span', 'Home Link')`In general, prefer to limit assertions about the nature of the DOM to a small surface area near the target element. Even then, only assert the DOM if there is a reason that changing the DOM in that area would harm the use experience - for example by breaking accessibility.
Data-cy attributes
Sometimes the specific details of an element don't matter so we don't assert what kind of element it is. Or the contents are dynamic and can't be controlled in the test, so we can't use the content to locate the element. In these cases data-cy attributes can be used:
cy.get('[data-cy="success-toast"]').should('be.visible') // just make sure it existsdata-cy can also be combined with other selectors to help target a specific element we've identified in our code, while continuing to assert the appropriate DOM and content:
cy.contains('[data-cy="success-toast"] h2', 'Success!').should('be.visible') // toast with correct heading element and test existsBe cautious when using only data-cy to locate an element, because doing so specifically tells the test not to care about what kind of element it is, or what it contains. So we should be sure that's what is intended. This means we should rarely, if ever, rely on data-cy alone to locate an element we will interact with. It should always be combined with an assertion about the label/name of that element.
An exception would be when testing a something like card in the UI, if the whole card is supposed to be clickable. While there should be some focusable element inside the card for keyboard and screenreader users, the card itself does not need a label but should still be tested, alongside the properly labelled accessible control for the same function:
cy.get('[data-cy="ui-card"]').click() // test clicking at the level of the card itself
// ... assert expected state ...
cy.contains('[data-cy="ui-card"] button', 'Activate').click() // test the accessible trigger for that card's functionality
// ... assert expected state ...Visibility Checks
Asserting should('be.visible') is useful when testing elements that the test won't interact with, as in certain situations it is possible for elements to be found in the DOM with the content we expect, but to still be unexpectedly hidden from the user with CSS or covered by another element. When interacting, that visibility check is already built in before cy.click or cy.type for example.
Finding elements by role
Use the built-in cy.getByRole() rather than Testing Library's findByRole. For a role that HTML has an element for, such as button, link, or heading, it only matches that element by default: cy.getByRole('button', { name: 'Log In' }) finds a , not a . The ARIA role of button could be added to an element that does not have the expected keyboard behaviors implemented, and the test would still pass if that element matched. It is also a good accessibility practice to not use ARIA to recreate the existing functionality of HTML elements, but instead use the elements directly.
Widget roles that HTML has no element for, such as menuitem or tab, need no option. Pass native: false for elements that get their role from a component library, such as the dialog, radio, and option elements rendered by Headless UI: cy.getByRole('dialog', { name: 'Settings', native: false }). To narrow by state, chain a filter: .filter('[aria-expanded="false"]'), .filter('[aria-checked="true"]'), or .filter('h2') for a heading level.
Cypress-Testing-Library
Feel free to use its other queries, such as findByTestId or findByLabelText, often in tests if it makes the test easier to write or understand, except where using it provides less confidence than a plain Cypress selector.
Visual Appearance
Avoid specifying specific CSS color values in tests. Prefer Percy snapshots to validate that the approved appearance isn't changing unexpectedly.
Guides Typescript (`guides/typescript.md`)
Writing and Migrating To TypeScript
Style Guidance
In general, the ts.dev style guide shall be adhered to as close as possible, with the caveat that end-of-line semicolons are omitted unless necessary to prevent syntax errors.
Type Definitions
Prefer interface to type when possible. Interfaces are never inlined, which solves some potential issues with generated .d.ts files.
When to use type:
- The type being declared must be a Union type
- Declaration merging may happen when it should not
- Complicated types using utility types
Object Literals
When working with a structural implementation, always declare the type.
Correct:
interface Foo {
bar: string
}
const foo: Foo = {
bar: 'baz',
}Incorrect:
const foo = {
bar: 'baz',
}Function Definitions
Always declare types for parameters and return values. Declaring parameter types ensures that callers are providing the correct parameters. Declaring the return type ensures that the function body is returning what is intended.
Exception: inlined single-line arrow functions can usually have their type declarations inferred. This is especially the case if the ultimate return value is declared. Example:
const odds: string[] = [1,2,3,4,5]
.filter(n => n % 2)
.map(n => String(n))Prefer inline types. Parameter types and return types often do not need to be referenced outside of the function itself. Bag parameters (object literals with more than two properties) should be declared as a named interface. If possible, do not export this interface. If a dependent needs to reference the bag type because it is composing an object literal in preparation to call the function in question, that dependent should use the Parameters utility type instead. This way the interface name can change without necessitating a change to the dependent. Use this technique, example below, as a last resort. Prefer composing the parameter bag inline, rather than assigning it to an identifier.
Choosing definition styles is almost more of an art than a science. Conciseness and flexibility often go hand in hand, but do not compromise on enforcing correctness to achieve conciseness. Fancy type manipulation with mapped types derived from templated types can be satisfying to figure out, but they often indicate leaky abstractions, vague overloads, and so forth. Use with caution.
Incorrect:
// foo.ts
export const foo({ a, b, n}: { a: string, b: boolean, n: number}): void { }
// baz.ts
export interface BazParam {
foobar: string
}
export const baz(bag: BazParam) {}
// bar.ts
import { foo } from './foo'
import { BazParam, baz } from './baz'
const bag = {
a: 'a',
b: false,
n: 2
}
foo(bag)
const bazBag: BazParam = {
foobar: 'buzz'
}
baz(bazBag)Better:
// foo.ts
interface FooParams {
a: string
b: boolean
n: number
}
export const foo({ a, b, n }: FooParams) {
}
// bar.ts
import { foo } from './foo'
const FooParams = Parameters<typeof foo>[0]
const bag: FooParams = { a: 'a', b: false, n: 2 }
foo(bag)Best:
// foo.ts
interface FooParams {
a: string
b: boolean
n: number
}
export const foo({ a, b, n }: FooParams) {
}
// bar.ts
import { foo} from './foo'
foo({
a: 'a',
b: false,
n: 2
})Type Assertions
Prefer a runtime check that narrows the type over an as assertion, since an assertion silences the compiler without checking anything:
- Declare a predicate's return type as a type guard (
(value: unknown): value is ShadowRoot) so callers narrow without casting. - Declare a function that throws on invalid input as an assertion function (
asserts value is Options), so the code after the call sees the validated type. TypeScript only narrows through an assertion function declared withfunctionor given an explicit type, such as a method on a typed interface. - Use
'prop' in valueto narrow before reading a property that only some subtypes have.
When an assertion has to stay, such as the string[] that Object.keys returns, add a one-line comment that starts "Safe because" and says what guarantees the type.
DRY
If you find yourself defining more than two very similar interfaces or types, consider what can be DRY'd. Is it purely the types, or is there further refactoring that should be done? When DRY, prefer defining the type as close to the implementation that depends on that type as possible. Define parameter bags next to the function they're an argument for, and export them only when necessary. New interfaces should almost never need to be declared for intermediary variables in a function - if you find yourself doing this, there are probably structural refactors that should be considered before DRYing the type definitions.
Exception to DRY:
Type declarations composed for public API consumption & definition don't need to be as rigorously DRY'd. Readability and understandability is preferred over conciseness.
Migration Success Criteria
- No unnecessary
.jsin the repository - The repository passes linting based on the
@typescript-eslint/recommended-type-checkedrule set - The repository uses modern eslint: stylistic for formatting, various non-formatting-related plugins for code correctness, and the modern flat config pattern.
- Optional / Stretch goal: Define a style guide for the repository that may extend past basic linting and formatting constraints. Initial thoughts:
- There are no circular dependencies
- There are no deep imports from undeclared exports
Execution
For shared language purposes, both packages and files can be categorized as "leaf," "shared," or "root" nodes.
- A Leaf node has no monorepo dependencies
- A Shared node depends on one or more nodes, and is the dependent of one or more node.
- A Root node is not imported by any nodes other than
./cli
Stage 1: Typescript Shift
Working through each package from leaf to shared to root nodes, and files within each package from leaf to shared to root files:
.jsfiles are renamed to.ts- Type definitions are added only when adding them does not have a domino effect
- When adding a type definition would have a domino effect, use an
anytype. Because we can warn on the use ofanytypes, this serves as an implicit "todo" to come back and fix it. - Add definitely-typed dependencies as necessary
- Add
@ts-expect-errordirectives as necessary, each with a description of the error it suppresses (// @ts-expect-error - <reason>).@typescript-eslint/ban-ts-commentrequires the description and bans@ts-ignoreand@ts-nocheckin packages on the flat config (@packages/eslint-config); packages still on.eslintrcdo not enforce it yet. The directive fails type checking once the error is gone, so it stays an implicit "todo".
Stage 2: Reduce Warnings
Beginning again with packages from leaf to root, and within each from leaf file to root file, begin to address warnings. Addressing warnings should be prioritized based on code quality impact. Loosely:
no-require-importsis a prerequisite to ESM. We userequireintentionally in several places, so this requires additional consideration.no-explicit-anyensures that we're defining types and interfaces for function parameters and return values. This is the number one tool to help catch bugs and incorrect code.anytypes are a useful escape hatch when first converting a project, but quickly become a crutch. Do the easiest ones first - if you start in on one and realize it has a wide ranging domino effect, add a comment about which other files are involved and move on. That type may become more straightforward to define once other explicitanytypes are filled in.require-await,await-thenable, andno-floating-promiseshelp reduce logical errors and unnecessary microtasks with asynchronous code.
Stage 3: Enforce previously warned rules as errors
As warnings are removed from packages, those warnings should be converted to errors so they do not re-occur. These rules are specifically called out when overridden in the base eslint configuration.
Helpful Tips
- If you're working on converting a large file with many exports, try extracting each export into its own file. This can help reduce the scope of necessary changes, and may help resolve circular dependencies.
- Circular dependencies can cause havoc - use madge to detect if the file you're working on is included in a circular dependency cycle, and resolve that before converting the file.
- If you find yourself on a change cascade, pause and back up. Use
anytypes to help get to the finish line - we'll warn on them, and they can be fixed later.
Guides V8 Snapshots (`guides/v8-snapshots.md`)
V8 Snapshots
In order to improve start up time, Cypress uses electron mksnapshot for generating v8 snapshots for both development and production.
Snapshot Generation
At a high level, snapshot generation works by creating a single snapshot JS file out of all Cypress server code. In order to do this some code needs to be translated to be "snapshottable". For specifics on what can and can't be snapshot, see these requirements. The JS file that gets generated then runs through electron mksnapshot which creates the actual binary snapshot that can be used to load the entire JS file into memory almost instantaneously.
Locally, a v8 snapshot is generated in a post install step and set up to only include node modules in the snapshot. In this way, cypress code can be modified without having to regenerate a snapshot. If you do want or need to regenerate the snapshot for development you can run:
yarn build-v8-snapshot-devOn CI and for binary builds we run:
yarn build-v8-snapshot-prodwhich will include both node modules and cypress code in the snapshot.
Environment Variables
DISABLE_SNAPSHOT_REQUIRE- disables snapshot require and the snapshot build process when runningyarn installV8_SNAPSHOT_DISABLE_MINIFY- disables the minification process during a V8 snapshot production build. This speeds up the build time greatly. It is useful when building the Cypress binary locally
Cache
Because the V8 snapshot process involves analyzing every file to determine whether it can be properly snapshot or not, this process can be time consuming. In order to make this process faster, there is a cache file that is stored at tooling/v8-snapshot/cache//snapshot-meta.json. This file specifies the snapshotability of each file. The snapshot process uses this snapshot meta file as a starting guess as to all of the files snapshot status. It can then detect new problems per file and thus only have to process new or newly problematic files, thus speeding up the overall process.
This cache should be maintained and updated over time. Rather than having this maintenance be a manual process done by developers, a github action is used. It is scheduled to run nightly and issues a PR to develop with any changes to the cache files. Because the iterative snapshot process is good at transitioning files from a healthy status to unhealthy but not the other way, we generate the snapshot from scratch weekly to try and keep the cache as optimal as possible. Generating from scratch is a very lengthy process (on the order of hours) which is why this is only done weekly.
Troubleshooting
Missing Dependency
Background on Snapshot Dependencies
When we generate snapshots, we create a metadata file that contains information about all dependencies included in the snapshot. In theory, these dependencies could be removed from the binary to save space, since they're already included in the snapshot. This helps offset the size increase from the snapshot itself.
However, not all dependencies can be safely removed. There are two main categories of dependencies that must be preserved:
- Dependencies (and their sub-dependencies) that are used in child processes
- Dependencies (and their sub-dependencies) that are dynamically loaded and cannot be determined by
esbuild
To handle this, we maintain a list of these dependencies and use esbuild to retrieve all their child dependencies. This list is defined in binary-cleanup.js. We then remove these dependencies from the list of v8 snapshot metadata dependencies, leaving only the ones that are safe to remove.
Handling Missing Dependencies
If you encounter a missing dependency error that looks something like:
/* Detailed source-code truncated for AI context efficiency. */Follow these steps:
- Look through the stack trace of the missing dependency error
- Try to identify where the dependency is being dynamically required/imported
- Validate that the dynamically required/imported dependency is missing from the binary
- Add the dependency to the list in binary-cleanup.js
This will ensure the dependency is preserved in the binary and available when needed.
Local Development
If you're running into problems locally, either with generating the snapshot or at runtime, a good first step is to clean everything and start from scratch. This command will accomplish that (note that it will delete any new unstaged files, so if you want to keep them, either stash them or stage them):
git clean -fxd && yarnGeneration
If the build v8 snapshot command is taking a long time to run on Circle CI, the snapshot cache probably needs to be updated. Run the Update V8 Snapshot Cache github action against your branch to generate the snapshots for you on all platforms. You can choose to commit directly to your branch or alternatively issue a PR to your branch.
If the build v8 snapshot command fails, you can sometimes see which file is causing the problem via the stack trace. Running the build snapshot command with DEBUG=*snap* set will give you more information. Sometimes you can narrow the issue down to a specific file. If this is the case, you can try removing that file from the snapshot cache at tooling/v8-snapshot/cache//snapshot-meta.json. If this works, check in the changes and the file will get properly updated in the cache during the next automatic update.
Package `exports` Subpaths
Our custom snapshot engine cannot resolve named subpath exports in package.json (for example "exports": { "./types": "./cjs/types/index.js" }).
Do not add package subpath exports until the snapshot engine supports them.
If a Full Snapshot Rebuild Still Fails
Occasionally, a full rebuild will still fail. This is typically a sign that there is a problem with the code we are generating, with Electron's mksnapshot or even deeper with the pure v8 mksnapshot. The first thing to be done is to figure out where the problem actually is.
To determine if the problem is with the pure v8 mksnapshot, check out, build, and run the v8 mksnapshot code. Then you can run the built mksnapshot against the generated snapshot file at tooling/v8-snapshot/cache/darwin/snapshot.js. For example on an M1, you would run tools/dev/gm.py arm64.release and once that builds, you can run mksnapshot via: out/arm64.release/mksnapshot . Then, it's just a matter of figuring out where that commit is that introduces the problem. A good strategy is to check out the tag of the release corresponding to the electron version where everything is working and verify it works. Then check out the tag of the release corresponding to the electron version where everything is not working and verify that it is broken. Then use a binary search on the commits in between to find the place where things break.
To determine if the problem is with Electron's mksnapshot, check out, build and run the electron mksnapshot code. Using Electron's build tools is strongly recommended. An example of what the process looks like would be running e build mksnapshot to build the mksnapshot target and then executing src/out/Release/mksnapshot to try and generate the snapshot. A good strategy is to check out the tag of the release corresponding to the electron version where everything is working and verify it works. Then check out the tag of the release corresponding to the electron version where everything is not working and verify that it is broken. Then use a binary search on the commits in between to find the place where things break. If you have determined that the problem is not with the pure v8 mksnapshot, then the issue is likely with one of the patches that electron applied on top of v8.
To determine if the problem is with the code we are generating, run V8_SNAPSHOT_DISABLE_MINIFY=1 DEBUG=*snap* yarn build-v8-snapshot-prod. The failure will spell out a command like node /Users/user1/cypress/tooling/electron-mksnapshot/dist/mksnapshot-bin.js /Users/user1/cypress/tooling/v8-snapshot/cache/darwin/snapshot.js --output_dir /private/var/folders/9z/qzyh61x16tv5y874h_8297m40000gq/T/cy-v8-snapshot-bin. Running that by itself will then spell out another command like /private/var/folders/9z/qzyh61x16tv5y874h_8297m40000gq/T/mksnapshot-workdir/mksnapshot /Users/user1/cypress/tooling/v8-snapshot/cache/darwin/snapshot.js --turbo_instruction_scheduling --stress-turbo-late-spilling --target_os=mac --target_arch=arm64 --embedded_src gen/v8/embedded.S --embedded_variant Default --random-seed 314159265 --startup_blob snapshot_blob.bin --no-native-code-counters. This last command can be used for further iterative troubleshooting. A good approach for troubleshooting involves looking at the snapshot file (/Users/user1/cypress/tooling/v8-snapshot/cache/darwin/snapshot.js in the above example) and commenting out various files referenced in the // tooling/v8-snapshot/cache/darwin/snapshot-entry.js section. If you can narrow it down to a specific file causing the problem, you can further try and narrow it down to a line of code in that file. If it is a specific line, there are two options: tweak the line in the source code to see if there's a way to cause it not to have a problem or fix the code generation at https://github.com/cypress-io/esbuild.
Runtime
If you're experiencing issues during runtime, you can try and narrow down where the problem might be via a few different scenarios:
- If the problem occurs with the binary, but not in the monorepo, chances are something is being removed during the binary cleanup step that shouldn't be
- If the problem occurs with running
yarn build-v8-snapshot-prodbut notyarn build-v8-snapshot-dev, then that means there's a problem with a cypress file and not a node module dependency. Chances are that a file is not being flagged properly (e.g. healthy when it should be deferred or norewrite). - If the problem occurs with both
yarn build-v8-snapshot-prodandyarn build-v8-snapshot-devbut does not occur when using theDISABLE_SNAPSHOT_REQUIREenvironment variable, then that means there's a problem with a node module dependency. Chances are that a file is not being flagged properly (e.g. healthy when it should be deferred or norewrite). - If the problem still occurs when using the
DISABLE_SNAPSHOT_REQUIREenvironment variable, then that means the problem is not snapshot related.
Guides Writing Cross Platform Javascript (`guides/writing-cross-platform-javascript.md`)
Writing Cross-Platform JavaScript
Cypress works on Linux, macOS and Windows. This includes both installing from npm, as well as for local development. Code should be written in a platform agnostic style.
Handling File Paths
Throughout the code base, we access the file system in various ways, and need to be conscious of how we do so to ensure Cypress can be used and developed seamlessly on multiple platforms. One thing to keep in mind is file paths and file separators. macOS and Linux systems use /, and Windows uses \.
As a general rule, we want to use native paths where possible. There are a few reasons for this. Wherever we display a file path, we want to use the native file separator, since that is what the user will expect on their platform. In general, we can use the Node.js path module to handle this:
// on linux-like systems
path.join('cypress', 'e2e') //=> `cypress/e2e`
// on Windows
path.join('cypress', 'e2e') //=> `cypress\e2e`There are some exceptions to this, namely the globby module, which only supports / (see here) when writing glob patterns. In these cases, where an API is posix only, you can use path.posix.join, which will always use /, even on a Windows system:
// don't do
const files = await globby('my-project\cypress\e2e\**\*')
// do
const files = await globby('my-project/cypress/e2e/**/*')
// or you can convert it by splitting by path.sep
// and joining with `path.posix.join`
const glob = path.posix.join('my-project/cypress/e2e/**/*'.split(path.sep))The general rule of using path where possible applies to moving around the file system, too:
path.resolve('../', '/../', '../')
// '/home' on Linux
// '/Users' on OSX
// 'C:\\Users' on WindowsIn general, you want to avoid writing file system code using / and \, and use Node.js APIs where possible - those are cross platform and guaranteed to work.
Use Node.js Scripts
For many developers, it's tempting to write a quick bash script to automate tasks. Maybe you'd like to delete all .js files, so you add a script to package.json:
{
"scripts": {
"clean": "rm -rf **/*.js"
}
}This will stop developers on Windows from running yarn clean unless they are specifically using a POSIX shell (like Git Bash). Instead, opt for a Node.js script where possible, or use a cross-platform Node.js module. In this case, we could use the rimraf module:
{
"devDependencies": {
"rimraf": "5.0.10"
},
"scripts": {
"clean": "rimraf --glob '**/*.js'"
}
}Now your script is cross-platform.
Use the os Module
You can use the os module to handle platform differences. One such example is line endings; \n on linux systems, and \r\n on Windows. Instead. use os.EOL. To check the current platform, use os.arch():
import os from 'os'
os.EOL // \n on linux, \r\n on windows
os.platform()
// 'linux' on Linux
// 'win32' on Windows (32-bit / 64-bit)
// 'darwin' on OSXGuides Writing The Cypress Changelog (`guides/writing-the-cypress-changelog.md`)
Cypress App - Managing the Release Changelog
Cypress prefers hand tailored release notes over auto generated release notes, primarily, user experience is highly valued at Cypress.
When to Add an Entry
The changelog should include anything that was merged into the develop branch of the Cypress repo that is a user-affecting change. This includes:
breaking- A breaking change that will require an MVBdependency- A change to a dependency that impacts the userdeprecation- An API deprecation notice for usersfeat- A new featurefix- A bug fix or regression fixmisc- A miscellaneous user-facing change, like a UI update which is not a fix or enhancement to how Cypress worksperf- A code change that improves performance
These prefixes do not require an entry:
chore- Changes to the build process or auxiliary tools and librariesdocs- Documentation only changesrefactor- A code change that neither fixes a bug nor adds a featurerevert- Reverts a previous committest- Adding missing or correcting existing testsinternal- Work that is real and may well ship publicly later, but is not reachable by users yet — typically behind a feature flag, an internal-only build, or a gated rollout. Its entry is written in the release that actually exposes it, asfeatorfixat that point.
Choosing fix asserts the change ships to users, so do not then skip the entry because the impact looks internal. Use chore or refactor if the change truly has no user-facing effect, or internal if it is user-facing but not yet reachable.
This changelog describes the binary. Validation only considers changed files under cli/ or packages/ (excluding cli/CHANGELOG.md itself), so a pull request touching only npm/, system-tests/, scripts/, tooling/, or docs needs no entry — and should probably not carry a user-facing prefix. The npm/* packages are versioned and published independently by semantic-release and never appear here.
Writing Guidelines
The changelog is formatted like the following. If there is not a pending changelog for the next release, add these sections.
md## <RELEASE_VERSION> **<CHANGE_SECTION:** - <CHANGELOG_ENTRY>The file is parsed by position, so a few rules are enforced as hard errors by
parse-changelog.js:- Line 1 stays the HTML comment and line 2 must be the
## X.Y.Zheading, with a blank line after it and after every section header. - Only the change sections in the table below are valid section headers. Anything else throws, naming the offending line number.
- A section header may not appear twice in the same release — a repeat throws
Duplicate section header, naming the line. Add to the existing section instead. - Parsing stops at the next
## X.Y.Z, so only the topmost release is validated.
- Line 1 stays the HTML comment and line 2 must be the
Each changelog entry is written and merged with the associated user-facing code change in
cli/CHANGELOG.md.The changelog entry should be added to the associated change section. The supported change sections for the changelog (that should be listed in the order below) are:
change type (by order of impact) change section details -- Summary A description of the overall changes, usually only provided for breaking changes or large features, written in coordination with Cypress's marketing to match the language used around the release. It may also link to relevant blogs. Example — note: **Summary:**is not a valid section header incli/CHANGELOG.mdand will fail validation. Add the summary when the release notes are published todocs.cypress.io(see step 11 of the Release Process), not to this file.breakingBreaking Changes Link to the Migration Guide (if any) at the beginning of this section. For each one explain the change, how it affects users, and how users can mitigate the effects of the change (unless it's covered in the Migration Guide). Example deprecationDeprecations Explain each deprecation and that it will be removed in a future release. Example perfPerformance Example featFeatures Example fixBugfixes Example miscMisc A user-facing change to how Cypress looks or describes itself, such as a redesigned UI like the browser picker, a correction to the public TypeScript types, or new wording in an error message. Example dependencyDependency Updates A list of dependencies that were updated, downgraded, or removed as well as the version it was changed from. Example You may have several changes around a feature that make sense to group. Feel free to do so to make more sense to users consuming the changelog. Example
Do not refer to 'we' when writing a changelog item. We want to phrase the changelog in a way that emphasizes how the user is impacted. Additionally 'we' may not have addressed the issue, an outside contributor may have.
- Example: Instead of 'We fixed a situation where a cross-origin error could incorrectly throw in Chrome' write 'Cross-origin errors will no longer incorrectly throw in Chrome in certain situations'.
Be as direct as possible in explaining the changes, but with enough clarity that the user understands the full impact. Users should never have to click on the link to the issue/PR to understand the change that happened and absolutely never have to look at the code to understand the change. If you cannot yourself understand the change from the Changelog entry, add more context.
- Do not include implementation details (e.g. internal function/file names, refactors, the specific code path that changed, or how the fix works under the hood). Describe only the user-facing impact — what behavior the user will now observe. If a change has no user-facing impact, it does not belong in the changelog.
Order the changelog items in order of impact. The most impactful features/bugfixes should be ordered first.
If a changelog item is a regression, the description should start with
Fixed a regression in [9.1.0](#9-1-0)with a link to the release that introduced it.For each changelog item, there should be a link to the issue(s) it addresses, or to the PR it was addressed in when there is no corresponding issue. Validation matches an entry by these links, so every issue (or the PR) must appear in it. Link issues as
https://github.com/cypress-io/cypress/issues/<n>and pull requests ashttps://github.com/cypress-io/cypress/pull/<n>. The lead-in phrase depends on the change type, perchange-categories.js:fixandperf, with issues:Fixes [#1234](https://github.com/cypress-io/cypress/issues/1234).fixandperf, no issue:Fixed in [#1234](https://github.com/cypress-io/cypress/pull/1234).- All other change types, with issues:
Addresses [#1234](https://github.com/cypress-io/cypress/issues/1234). - All other change types, no issue:
Addressed in [#1234](https://github.com/cypress-io/cypress/pull/1234). - Multiple issues are sorted ascending and joined with a comma and
and:Fixes [#12](https://github.com/cypress-io/cypress/issues/12), [#13](https://github.com/cypress-io/cypress/issues/13) and [#14](https://github.com/cypress-io/cypress/issues/14).
When a changelog item references a Cypress command, plugin event, or configuration option, link the first mention to its documentation for consistency with existing entries. Either link form is acceptable: the
https://on.cypress.io/<slug>short link or an absolutehttps://docs.cypress.io/URL.- Example (
on.cypress.io):[`cy.visit()`](https://on.cypress.io/visit),[`before:spec`](https://on.cypress.io/before-spec-api). - Example (
docs.cypress.io):[`before:spec`](https://docs.cypress.io/api/plugins/before-spec-api),[`numTestsKeptInMemory`](https://docs.cypress.io/app/references/configuration#Global).
- Example (
If a change is only observable under a specific run context (e.g.
cypress runvs.cypress open, or behind an experimental flag), state which context is affected so users can tell whether the change applies to them.A release often ships between the commit that adds an entry and the review that merges it. Re-check placement every time you pull
develop, and again before marking a pull request ready: if a new## X.Y.Zheading has appeared above your entry, move the entry up into the new topmost release, keeping it under the same named section there. This is silent in both directions — the entry still reads correctly in the diff, and validation parses only the topmost release, so an entry stranded under an already-shipped version is simply never seen.
Dependency Updates and Security Advisories
Most dependency entries exist because a security scan reported an advisory, and users (and their own security teams) scan this section for a specific advisory ID. Follow the established shape: upgraded what, from and to which versions, to address which advisory, as reported in security scans.
- Upgraded `undici` from `6.26.0` to `6.27.0` to address a [CRLF Injection](https://security.snyk.io/vuln/SNYK-JS-UNDICI-17372658) vulnerability reported in security scans. Addressed in [#34121](https://github.com/cypress-io/cypress/pull/34121).- Open with
Upgradedand the package name in backticks, thenfrom `X` to `Y`— or justto `Y`when the previous version was a range. - Always link the advisory, using whichever source reported it:
https://www.cve.org/CVERecord?id=CVE-…,https://github.com/advisories/GHSA-…,https://security.snyk.io/vuln/SNYK-JS-…, orhttps://nvd.nist.gov/vuln/detail/CVE-…. - Use the vulnerability class as the link text when the advisory names one — Denial of Service, Remote Code Execution, Prototype Pollution, CRLF Injection, ReDoS. Fall back to the bare advisory ID when it has no descriptive name. A Snyk link may carry the CVE after it in parentheses.
- Close the clause with "reported in security scans", which is what makes these entries greppable. Use
vulnerabilitieswhen listing more than one advisory. - Say where the vulnerability actually lived when it is not the package itself — a bundled binary, or a transitive copy — and name any sibling dependencies bumped to keep versions aligned, and why.
- Do not claim Cypress was exploitable. These entries state that the advisory no longer appears in scans; they do not assess whether the vulnerable code path was reachable.
Formatting
Package Names and Versions
Enclose the following in backticks (`):
| Item | Pattern | Example |
|---|---|---|
| npm package name | lower case, hyphen / underscore optional | tar-fs |
| semver version | X.Y.Z (no v) |
3.1.0 |
| npm semver ranges | caret ^, tilde ~, comparator |
^3.0.0 ~0.6.1 >=15.0.4 |
No backticks for:
| Item | Pattern | Example |
|---|---|---|
| npm package major version | X |
15 |
Verifying Your Entry
Run the same check CI's verify-release-readiness job runs:
GH_TOKEN=<token> node ./scripts/semantic-commits/validate-binary-changelog.jsTwo prerequisites, both of which fail before the script reaches any changelog logic:
- A root
yarninstall. The script loadsbluebirdand@octokit/corefrom the rootnode_modules. Without them it exits atCannot find module 'bluebird'— a missing dependency, not a problem with your entry. GH_TOKEN. It queries the GitHub API for the last release and this branch's commits, and throwsThe GH_TOKEN env is not set.without one. Any token with public repo read access works;gh auth tokensupplies one.
Run it on your feature branch. The gate that limits full-changelog validation to develop and release/X.Y.Z applies only under CIRCLECI, so a local run validates the whole changelog whatever branch you are on.
It reports one of:
A changelog entry was not found in cli/CHANGELOG.md.The changelog does not include the **<Section>:** section.Found the changelog entry in the wrong section.- a missing-links error that prints the exact expected string to paste.
- a parse error naming the offending line number, for an unknown or duplicate section header or a missing blank line.
The script checks placement and links only. Wording, ordering, and whether the entry is understandable are left to reviewers, so a passing run does not mean the writing guidelines above have been met.
Release
At the time of the release, the releaser will:
- ensure the Changelog is coherent
- ensure the change sections are in the correct order
- ensure that the entries are ordered by impact
Each Cypress release results in an update to the changelog.mdx file in the cypress-documentation repository to be published on the doc site. See Example pull request adding a new changelog section to the repository.
Npm Vue Docs Styles (`npm/vue/docs/styles.md`)
styles
If your component imports its own style, the style should be applied during the Cypress test.
// MyComponent.vue
<template>
<div class="darkred">
<p class="green">I get my style from main.css</p>
</div>
</template>
<style lang="scss">
@import "./src/styles/main.css";
</style>You can also load styles in the following ways:
Import in the spec file
import '../styles/main.css'
const myComponent = {
template: '<button class="orange"><slot/></button>'
}
mount(myComponent)Import in the component support file
If you have stylesheets that should apply to all of your components, you can import those from your component support file.
// cypress/support/component.js
import '../styles/main.css'
...
// MyComponent.spec.js
const myComponent = {
template: '<button class="orange"><slot/></button>'
}
mount(myComponent)--- METRICS ---
- Files Extracted: 28
- Estimated Token Budget: ~37829 tokens
- Canonical Reference: https://codewiki.google/github.com/cypress-io/cypress