{"owner":"airbytehq","repo":"airbyte","hasSkills":true,"totalSkillsCount":42,"totalTokensCount":32019,"categories":["anthropic-skill","plugin-manifest","subagent-persona"],"hasMcp":false,"mcpConfig":null,"found":[".claude/skills/my-prs/SKILL.md","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/SKILL.md","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/SKILL.md","airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/SKILL.md","airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/SKILL.md","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/SKILL.md","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/SKILL.md","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/SKILL.md","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/SKILL.md","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/SKILL.md","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/SKILL.md","tools/bin/admin-service-account.yaml","tools/bin/run-harness-process.yaml","tools/internal/README.md","tools/schema_generator/README.md","tools/site/README.md","airbyte-integrations/connectors/source-commercetools/README.md","docs/release_notes/agents/readme.md","docs/ai-agents/get-started/developer-quickstart/skills/_category_.json","docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md","docs/ai-agents/get-started/developer-quickstart/skills/codex.md","docs/ai-agents/get-started/developer-quickstart/skills/lovable.md","docs/ai-agents/get-started/developer-quickstart/skills/readme.md","airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json","airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json","airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json"],"skills":{".claude/skills/my-prs/SKILL.md":"---\nname: my-prs\ndescription: List PRs opened today that are assigned to me or the move-destinations team, excluding external contributors\ndisable-model-invocation: true\nallowed-tools: Bash(gh:*), Bash(date:*), Read\nargument-hint: [days-back]\n---\n\n# My PRs - Daily PR Review Dashboard\n\nThis skill fetches and displays PRs that need your attention:\n\n1. **PRs assigned directly to you** (@frifriSF59)\n2. **PRs with review requested from the move-destinations team**\n\nAll results are filtered to:\n- Only show PRs opened today (or within `$ARGUMENTS` days if specified)\n- Exclude PRs from users who are not members of the airbytehq organization\n\n## Execution\n\nRun the fetch script to get the PR list:\n\n```bash\nbash .claude/skills/my-prs/scripts/fetch-prs.sh\n```\n\nIf an argument is provided (number of days), modify the date filter accordingly:\n- `$ARGUMENTS` = number of days to look back (default: 0 = today only)\n- Example: `/my-prs 3` looks at PRs from the last 3 days\n\nFor custom date ranges, use:\n```bash\ngh pr list --repo airbytehq/airbyte --assignee @me --search \"created:>=YYYY-MM-DD\" --json number,title,author,url\n```\n\n## After Fetching\n\nAfter displaying the PRs, offer to:\n1. Open a specific PR in the browser (`gh pr view --web`)\n2. Show details of a specific PR (`gh pr view `)\n3. Show the diff of a specific PR (`gh pr diff `)\n4. Check CI status (`gh pr checks `)\n\n## Quick Actions\n\nThe user may want to:\n- **Review a PR**: Use `/review ` if available\n- **Check PR comments**: `gh pr view --comments`\n- **See who approved**: `gh pr view --json reviews --jq '.reviews[] | select(.state==\"APPROVED\") | .author.login'`\n","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/SKILL.md":"---\nname: source-mssql-e2e-tests\ndescription: Stand up a local SQL Server 2022 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mssql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mssql test or repro that is not CDC-specific.\n---\n\n# source-mssql-e2e-tests\n\nLocal end-to-end test harness for `source-mssql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../db-harness-lib/);\nthis skill keeps the MSSQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a SQL Server\n2022 container named `source-mssql-db-backend`, lets you apply arbitrary\nSQL fixtures, and runs Airbyte protocol commands against any\n`airbyte/source-mssql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a non-CDC bug against `source-mssql` locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mssql:` for connector\n development.\n- As the building block for the\n [`source-mssql-e2e-cdc-tests`](../source-mssql-e2e-cdc-tests/SKILL.md)\n skill, which adds CDC enable plus worked-example fixtures on top.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install airbyte-internal-ops`\n puts `airbyte-ops` on `$PATH`; alternatively prefix every call with\n `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mssql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mcr…/mssql/server:2022-latest as source-mssql-db-backend\n│ ├── apply-sql.sh # docker cp + docker exec sqlcmd -i\n│ ├── reset-databases.sh # docker exec sqlcmd → drop every non-system database (used by run.sh --reset=fixture)\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # non-CDC config; host=mssql-db-backend placeholder\n └── sql/\n ├── .gitignore # `.tmp/` — subtree for uncommitted per-bug scratch fixtures\n └── 00-init-base.sql # CREATE DATABASE TestDb + dbo.sample table\n```\n\nThe engine shim and backend lifecycle scripts remain in this skill's\n`scripts/` directory; the shared orchestration, default teardown, and\nprotocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mssql-db-backend`. Hard-coded in\n `scripts/start-backend.sh` and `apply-sql.sh`. The shared\n `db-harness-lib/scripts/stop-backend.sh` removes this container by\n `BACKEND_NAME`; the engine shim supplies the default. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mssql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per\n protocol command (`spec/`, `check/`, `discover/`, `read/`) holding that\n command's `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n This is the same layout the ops repo's `connector-regression-test.yml`\n workflow uses under `/tmp/regression_test_artifacts`, so CI can upload\n it per command.\n- Both containers (the SQL Server backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765);\n once `--network` is supported the bridge-IP dance can collapse.\n- SQL Server image is pinned to `mcr.microsoft.com/mssql/server:2022-latest`,\n not `latest`, for stable major-version behavior across CU patches.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover`'s output, run `read`, tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mssql\n\n# Single version, full sweep.\npoe e2e-local --test-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, non-CDC (airbyte-ops's built-in comparator).\npoe e2e-local --test-version=dev --control-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, CDC (per-image reset between the two runs).\npoe e2e-local --test-version=dev --control-version=5.0.0 --reset=fixture \\\n --config-template=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json \\\n --sync-mode=incremental --cursor-field=_ab_cdc_cursor --streams=users \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/00-init-cdc.sql \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/.sql\n\n# One command only.\npoe e2e-local --command=read --test-version=5.0.0\n```\n\nThe CDC example needs an incremental catalog (`--sync-mode=incremental\n--cursor-field=_ab_cdc_cursor`) restricted with `--streams` to exclude\n`dbo.systranschemas`; the general rule is in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md#cdc-config-templates-need-an-incremental-catalog).\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed — separating an\ninfrastructure failure (`ERROR`, no verdict was produced) from a failed\ntest verdict (`FAIL`), as the ops workflow's final-status step does. A\n`read` whose `discover` failed earlier in the sweep is reported `SKIPPED`,\nnot `ERROR` — an invalid-config case is expected to fail `discover`, and\nthere is then no catalog for `read` to use. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to\none command instead exits with the connector's own exit code, so a repro\ncan still assert on it.\n\nPrefer a published tag over `--test-version=dev`: for code on a pushed PR\nbranch, publish a pre-release and pass its `-preview.` tag,\nwhich skips the Gradle build and gives reviewers a tag they can re-run\nagainst — see\n[Getting a target image](../../../AGENTS.md#getting-a-target-image). When\nyou do use `--test-version=dev`, build the target image first, or pass\n`--build` to have `run.sh` run `:dockerBuildx` for you. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures` (run against whatever state the backend already has;\nfor the second/later `run.sh` invocation of a multi-phase driver, when\nre-applying the initial fixture would wipe intermediate state),\n`--step-name`, `--catalog` (skip discover-derived generation),\n`--state=PATH` (pass a saved STATE file to the read as `--state-path`;\nfor use by multi-phase drivers), `--sync-mode=incremental`,\n`--cursor-field`, `--streams`, `--config-template`,\n`--reset=none|fixture|backend`, `--keep-backend`, and `--` to forward\nextra args to `airbyte-ops`. Per-command timeouts match the workflow's\n(30/30/60/180 minutes) and are overridable with\n`TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n### Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst each command's artifacts, `run.sh` accepts a small set of\nexpectation flags that it enforces itself before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `--expect-test=pass\\|fail` | Overall target verdict. `pass` = every executed command's status was `pass`; `fail` = one or more failed. |\n| `--expect-control=pass\\|fail` | Same for the control sweep (comparison-mode runs only; requires `--control-version`). |\n| `--min-records=N` | Target read's `stdout.txt` must contain ≥N `RECORD` messages. |\n| `--min-states=N` | Target read's `stdout.txt` must contain ≥N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target's `` step's `` (`stdout` \\| `stderr` \\| `any`) must match `` at least N times (default 1). `` (`spec` \\| `check` \\| `discover` \\| `read`) is optional and defaults to `read`. Repeatable. |\n| `--forbid-match=[:]:` | Same shape but must match zero times. Repeatable. |\n\nAll match assertions run against the **target-side** artifacts of the\nnamed command (or `read` if no `` prefix). Under\n`--control-version + --reset=none` those live at\n`$ARTIFACTS_DIR//target/` (created by airbyte-ops's comparison\nmode); under `--reset=fixture|backend` at\n`$ARTIFACTS_DIR/target//`; under single-version at\n`$ARTIFACTS_DIR//`. `run.sh` picks the right path automatically.\n\nThe match-spec grammar disambiguates the leading `:` from the\n`:` on the first colon-separated field (command names and\nchannel names don't overlap), and strips a trailing `:N` only when the\nlast field is a positive integer. So a regex containing `:` still\nparses: `--expect-match=stderr:io.debezium.DebeziumException` reads as\ncommand `read` (default), channel `stderr`, regex\n`io.debezium.DebeziumException`, count 1. `--expect-match=check:stderr:should be positive`\nreads as command `check`, channel `stderr`, regex `should be positive`,\ncount 1.\n\n`--min-records` and `--min-states` are read-step only — they count\nRECORD / STATE envelopes, which are read-specific — and have no\n`:` prefix.\n\nAny expectation failure exits non-zero regardless of the command-level\nverdicts and appends an `**Expectation failures:**` section to the\nsummary. Migrating an existing driver script's `grep -q '' || exit 1`\nblock is straightforward — drop the block and add\n`--expect-match=stderr:X` to the shared `run.sh` invocation.\n\n### Multi-phase drivers\n\n`run.sh` runs one sweep. A driver script composes multi-phase flows\n(read → mutate → read-with-state) by calling `run.sh` more than once\nwith `--step-name` for each invocation and `--state=` to feed the\nsecond read the state extracted from the first. `extract-state.py`\nlives in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\nbecause it walks Airbyte STATE messages, which is a protocol-level\noperation and not CDC-specific.\n\nThe CDC skill's [`cases/11451.sh`](../source-mssql-e2e-cdc-tests/cases/11451.sh)\nis a worked example: it invokes the engine shim's `run.sh` around an\nintermediate `apply-sql.sh` mutation and the shared\n`db-harness-lib/scripts/extract-state.py` step, and asserts on the\nconnector's stderr. New multi-phase drivers can follow the same shape\nbut call `run.sh --state=PATH` directly instead of smuggling\n`--state-path` through trailing args.\n\n### Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` once per\n protocol command with both `--test-image` and `--control-image`, so\n the two images run sequentially against a single backend and\n `airbyte-ops`'s built-in comparators emit the target-vs-control diff\n (record counts, primary keys, per-record, schema). Right for non-CDC\n full-refresh work: the diff is meaningful and the shared-backend\n assumption is safe because a full-refresh read does not mutate the\n upstream. This is the current behavior for any caller that doesn't\n set `--reset`.\n- **`--reset=fixture`** — `run.sh` runs the whole sweep against the\n control image first, drops every non-system database, re-applies the\n fixtures, then runs the sweep against the target image. Two\n single-version `airbyte-ops` calls, no built-in comparator; artifacts\n land under `$REPRO_OUT//{control,target}//`.\n Right for **CDC comparisons**, where reusing the backend leaves the\n target reading against a warm capture instance and an advanced log\n position — the built-in diff can look clean while being meaningless.\n The SQL Server log-LSN clock is server-wide and keeps ticking across\n the reset, so per-record LSN columns and STATE offsets still differ\n between runs; gate the verdict on record-level shape rather than on\n those values.\n- **`--reset=backend`** — same as `--reset=fixture` but also recreates\n the backend container between the two sweeps, resetting the LSN clock\n at ~15s of extra startup cost. Use only when the reproduction depends\n on matching LSN sequences across runs.\n\n`--reset` has no effect without `--control-version` and produces a\nwarning; the flag governs the reset between two image runs, and there\nis no second run in single-version mode.\n\n## Asserting on output\n\nInline assertions in driver scripts. Suggested helpers:\n\n- Exit code: check `$?` immediately after the call.\n- AirbyteMessage shape: `jq -e 'select(.type == \"RECORD\")' $REPRO_OUT//read/stdout.txt`.\n- Connector-side log shape: `grep -c '' $REPRO_OUT//read/stderr.txt`.\n- Connection status: `jq -e 'select(.type == \"CONNECTION_STATUS\" and .connectionStatus.status == \"SUCCEEDED\")' $REPRO_OUT//check/stdout.txt`.\n\nIn comparison mode with `--reset=none` each command's directory holds\n`target/` and `control/` subdirectories (created by `airbyte-ops`).\nWith `--reset=fixture|backend` the split lives one level up:\n`$REPRO_OUT//{control,target}//`.\n\nPass `-- --enable-debug-logs=True` to `run.sh` to set `LOG_LEVEL=DEBUG`\non the connector container. That surfaces the\nDebezium \"Adding table … to the list of capture schema tables\" lines\nthat some assertions rely on.\n\n## Common gotchas\n\n- _No SQL Server Agent_ → CDC capture / cleanup jobs never run, and\n Debezium silently produces no records. `start-backend.sh` sets\n `MSSQL_AGENT_ENABLED=true` for this reason.\n- _Race on first `sqlcmd`_ → SQL Server 2022 takes around 15 seconds to\n accept connections cold. `start-backend.sh` polls until `SELECT 1`\n succeeds before returning.\n- _Configured-stream fields the validator rejects as null_ → the\n bulk-CDK schema validator requires `is_file_based`, `cursor_field`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files` on every\n `ConfiguredAirbyteStream`; omitting them fails with\n `Null value is not allowed. (code: 1021)`. This is not 4.3.x-only —\n verified on `4.4.12` and `5.0.0` too. `discover` does not emit\n `is_file_based`, so the catalog `run.sh` derives fills it in, and the\n catalog fixtures shipped with this and dependent skills populate all of\n them.\n- _A failed `check` does not fail a single-version run_ → the CDK exits\n 0 even when it emits `CONNECTION_STATUS` with `status: FAILED`, so the\n `check` step passes. Assert on the status message yourself (see\n [Asserting on output](#asserting-on-output)) rather than on the exit\n code when the repro hinges on `check`. Comparison mode does surface\n it, because it reads the report's verdict rather than the exit code.\n\n## Tear-down\n\n`run.sh` removes the backend container in an `EXIT` trap, so a run leaves\nnothing behind unless you passed `--keep-backend`. After a\n`--keep-backend` run, or to also discard the collected artifacts:\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\" # idempotent; docker rm -f no-ops when absent\nrm -rf \"${REPRO_OUT:-/tmp/source-mssql-repro}\"\n```\n","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/SKILL.md":"---\nname: source-mssql-e2e-cdc-tests\ndescription: Reproduce CDC bugs against source-mssql by enabling CDC on the local SQL Server backend, applying per-bug SQL fixtures, and running the connector via airbyte-ops. Composes on top of source-mssql-e2e-tests. Worked examples for airbytehq/oncall#11451, #12094, and #12162 ship with the skill.\n---\n\n# source-mssql-e2e-cdc-tests\n\nLocal CDC bug-reproduction harness for `source-mssql`. Builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe MSSQL engine skill\n([`source-mssql-e2e-tests`](../source-mssql-e2e-tests/SKILL.md)):\ngeneric skill stands up the SQL Server backend and runs the connector;\nthis skill adds CDC enable, CDC-aware config / catalog templates, and\nper-bug **case scripts** under `cases/` that call `run.sh` with the\nappropriate fixtures and `--expect-*` assertions.\n\n## When to use this skill\n\n- Reproducing a CDC-mode bug against `source-mssql` locally.\n- Verifying a fix by re-running an existing case with\n `VERSION=dev ./cases/.sh` (after `:dockerBuildx`).\n- Verifying that a fix which _loosens_ CDC offset validation still\n rejects genuinely invalid state — `cases/11451.sh` is the\n invalid-state case for LSN availability; see\n [Invalid-state case](#invalid-state-case-for-lsn-availability-fixes).\n- Authoring a new repro for a customer-reported CDC bug. Drop a SQL\n fixture under `fixtures/sql/` and a `cases/.sh` script\n that invokes `run.sh` with the right fixtures and `--expect-*`\n assertions.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, a clone of `airbytehq/airbyte`.\n- The backend container `source-mssql-db-backend` started by\n `../source-mssql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials.\n\n## Layout\n\n```\nsource-mssql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── 11451.sh # airbytehq/oncall#11451 — LSN-range regression in 4.3.4+ (multi-phase; invalid-state case)\n│ ├── 12094.sh # airbytehq/oncall#12094 — schema-history bloat\n│ └── 12162.sh # airbytehq/oncall#12162 — whitespace in stream name\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json\n ├── catalogs/\n │ ├── users-cdc.json\n │ └── order-items-cdc.json\n └── sql/\n ├── 00-init-cdc.sql\n ├── repro-11451-lsn-cleanup.sql\n ├── repro-12094-schema-history.sql\n └── repro-12162-spaces-in-name.sql\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit walks Airbyte STATE messages, which is protocol-level and not\nCDC-specific.\n\n## Conventions\n\n- Cases keep the backend up across invocations (`--keep-backend`).\n Start the backend once at the top of a session, run any subset of\n cases, tear down when done — see the Usage section below.\n- Fixtures are re-applied at the start of each `run.sh` invocation, so\n they must be idempotent (`00-init-cdc.sql` uses `IF DB_ID(...) IS NULL`\n guards, per-bug fixtures use similar shape). Re-applying a fixture\n during a multi-phase case does not reset the CDC state that the\n previous phase established.\n- Configured catalogs populate the bulk-CDK-required fields\n (`is_file_based`, `generation_id`, `minimum_generation_id`,\n `sync_id`, `destination_object_name`, `include_files`) so the same\n fixtures drive repros across `source-mssql:4.3.x` and `4.4.x`.\n- Cases default `VERSION=4.4.2` (except `11451.sh` which splits into\n `BASELINE_VERSION`/`TARGET_VERSION`). Override with\n `VERSION=4.3.4 ./cases/12162.sh` to test against an earlier version,\n or `VERSION=dev` after a local\n `./gradlew :airbyte-integrations:connectors:source-mssql:dockerBuildx`\n to test a fix.\n- **Assertions** are declarative via `run.sh`'s expectation flags:\n `--expect-test`, `--expect-match=[:]:[:N]`,\n `--forbid-match`, `--min-records`, `--min-states`. The runner\n enforces them and exits non-zero on any failure. The `:`\n prefix defaults to `read` when omitted; set it explicitly\n (`check:stderr:…`, `discover:stderr:…`) for check-time or\n discover-time signatures.\n- Multi-phase cases (`11451.sh`, and any future read → mutate →\n read-with-state repros) capture Airbyte STATE messages between reads\n via the shared\n [`extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\n (`uv`-PEP-723 standalone; pass `` as its argument or pipe\n stdin) and feed the file back into the second read with\n `run.sh --state=PATH`.\n Use `--step-name=/` to give each phase its own artifact\n subtree.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mssql-repro\n\n# 1. Bring up the backend (once per session). Cases pass --keep-backend,\n# so they don't tear it down between runs.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# 2. Run any case. Each case's --expect-* flags gate its own exit code\n# (non-zero on any expectation failure), so bash's `set -e` will fail\n# the sequence on the first failing case if you chain them.\n\"$SKILL/cases/12162.sh\"\n\"$SKILL/cases/12094.sh\"\n\"$SKILL/cases/11451.sh\"\n\n# 3. (After fix) verify by retargeting `dev` or a fixed version.\nVERSION=dev \"$SKILL/cases/12162.sh\"\n\n# 4. Tear down.\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\nrm -rf \"$REPRO_OUT\"\n```\n\nEach case is a shell script that composes `run.sh` invocations:\n\n1. Applies its SQL fixtures (via `--fixture=…`; `run.sh` calls\n `apply-sql.sh` internally).\n2. Renders the config against the running backend (via\n `--config-template=…`).\n3. Runs the target protocol command with any `--catalog` / `--state`.\n4. Gates its exit code on `--expect-*` flags — the runner enforces\n them and exits non-zero on any failure, so the case script has no\n inline assertion logic to maintain.\n\nMulti-phase cases (`11451.sh`) call `run.sh` more than once, with\n`extract-state.py` and `apply-sql.sh` between invocations. Each phase\nuses `--step-name=/` to keep its artifacts separate under\n`$REPRO_OUT///`.\n\n## Worked examples\n\n### airbytehq/oncall#12162 — whitespace in stream name\n\n`cases/12162.sh`. Creates `dbo.[Order Items]` (note the space), enables\nCDC on it, and runs `read` against it. Case asserts\n`--expect-test=fail`, `--expect-match=stderr:io.debezium.DebeziumException`,\nand `--expect-match=stderr:message.key.columns` — i.e. the read exits\nnon-zero and Debezium's `stderr.txt` contains its\n`Connector configuration is not valid. The 'message.key.columns' value\nis invalid` rejection.\n\nRoot cause: `MsSqlServerDebeziumOperations.buildMessageKeyColumns()`\njoins `schema.table:pkcol` strings without filtering or escaping\nidentifiers that contain whitespace or `:`, and Debezium rejects them\nat engine startup. Fix: pre-filter such streams (Debezium falls back\nto the table's native PK from system tables).\n\n### airbytehq/oncall#12094 — schema-history bloat\n\n`cases/12094.sh`. Creates 30 noise tables in `dbo` (CDC not enabled on\nthem), runs the baseline `read` against `dbo.users`, and asserts\n`--expect-match='stdout:Adding table CdcTest\\..* to the list of capture schema tables:30'`\n(the trailing `:30` is the count threshold; the fixture creates 30\nnoise tables). Adjust with `MIN_LOADED=`. Expected outcome: at\nleast 30 \"Adding table\" lines even though the configured catalog has\na single stream.\n\nRoot cause: `withSchemaHistory()` in the bulk-CDK Debezium properties\nsets `schema.history.internal.store.only.captured.databases.ddl=true`\nbut not `…captured.tables.ddl=true`. Same harness reproduces equivalent\nbehavior on `source-mysql` and `source-postgres` because the property\nlives in the bulk-CDK `extract-cdc` toolkit.\n\n### airbytehq/oncall#11451 — LSN-range regression in 4.3.4+\n\n`cases/11451.sh`. Multi-phase:\n\n1. Baseline `read` on clean CdcTest (`BASELINE_VERSION`, default `4.4.2`).\n Case asserts `--expect-test=pass --min-states=1` so a missing STATE\n fails the case immediately rather than later at replay.\n2. `extract-state.py` on the baseline stdout, then\n `apply-sql.sh repro-11451-lsn-cleanup.sql` runs\n `sys.sp_cdc_cleanup_change_table`\n with a `low_water_mark` past the saved LSN — advancing\n `fn_cdc_get_min_lsn('dbo_users')` past the baseline offset.\n3. Replay `read` on `TARGET_VERSION` (default `4.3.4`) with the stale\n state. Case asserts `--expect-test=fail`,\n `--expect-match=stderr:Saved offset no longer present`, and\n `--expect-match='stderr:is no longer available in SQL Server transaction logs'`.\n\nRoot cause: pre-`4.3.4` the LSN-range query computed `min` via\n`sys.fn_cdc_get_min_lsn('')`, which always returns\n`0x00000000000000000000`. `4.3.4` swapped to\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance)) FROM cdc.change_tables`,\nwhich returns the actual per-instance min — a value that can advance\npast a saved offset on geo-replicas with aggressive cleanup. The\nsaved-offset-rejection guard then fires even though the data is\nstill present. Investigation lives at\n[`airbytehq/oncall#11451`](https://github.com/airbytehq/oncall/issues/11451).\n\n#### Invalid-state case for LSN-availability fixes\n\nThis case's Phase 3 is a _genuinely_ expired saved offset: Phase 2\nadvances `fn_cdc_get_min_lsn` past the captured LSN, so rejecting the\nreplay is the correct behavior, not the bug. That makes it the\ninvalid-state counterpart to any repro built around a saved LSN that is\nstill valid — for example an unauthorized capture instance returning\n`0x0` and poisoning\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance))`.\n\nSo when a change relaxes what the offset-availability check aggregates\nor tolerates, run this case against the image built from that change:\n\n```bash\nTARGET_VERSION=dev \"$SKILL/cases/11451.sh\"\n```\n\nIt must still fail with the same two `stderr` signatures the case\nasserts on. A pass here means the guard stopped firing on state it is\nsupposed to reject, which turns an actionable \"reset the connection\"\nerror into a silently incomplete sync — the symptom-gone evidence from\nthe valid-LSN repro cannot distinguish that from a correct fix.\n\n## Authoring a new repro\n\n1. Drop a SQL fixture in `fixtures/sql/repro--.sql`.\n Make it idempotent (`IF NOT EXISTS` guards for `CREATE`,\n `IF EXISTS` guards for `DROP`) — cases re-apply fixtures on every\n `run.sh` invocation.\n2. (Optional) Drop a CDC-aware catalog in `fixtures/catalogs/`.\n3. Drop a case script in `cases/.sh` modeled on the\n existing ones. `set -euo pipefail`; default `VERSION=4.4.2` with a\n `${VERSION:-…}` override; invoke `run.sh` with the right\n `--command`, `--fixture=`, `--config-template=`, `--catalog=`, and\n `--expect-*` flags. Pass `--keep-backend` so the shared-session\n Usage flow works. For check-time or discover-time signatures, use\n `--expect-match=check:stderr:…` / `--expect-match=discover:stderr:…`\n (the `:` prefix defaults to `read` when omitted).\n4. Verify locally:\n ```bash\n \"$GENERIC/scripts/start-backend.sh\"\n \"$SKILL/cases/.sh\"\n BACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\n ```\n5. Note the worked example in this `SKILL.md`'s \"Worked examples\"\n section with: customer-symptom one-liner, `--expect-*` assertions\n the case gates on, root-cause one-liner.\n\n### Multi-phase repros\n\nWhen the reproduction is single-phase (one `run.sh` invocation is\nenough to trigger the bug), follow the recipe above. Reach for a\nmulti-phase shape only when the bug needs an intermediate mutation\nthat depends on state from a first read — the canonical example is\nread → extract STATE → mutate the server based on that STATE → replay\nwith the stale STATE. Model on [`cases/11451.sh`](cases/11451.sh) —\nthe three primitives it composes are:\n\n- **`--step-name=/`** — per-phase artifact dirs under\n `$REPRO_OUT//`. `cases/11451.sh` uses `11451/baseline` and\n `11451/stale`; a new case would use its own bug number and phase\n names.\n- **`extract-state.py`** (implemented in `db-harness-lib`) — reads a phase's\n `read/stdout.txt` and emits a JSON array of AirbyteStateMessage\n objects, which is what `run.sh --state=PATH` expects.\n- **`--skip-fixtures`** on the second/later `run.sh` invocation —\n opts out of the initial-fixture reapply so the intermediate state\n the previous phase established survives. Otherwise `run.sh`'s\n default fixture-application would re-run `00-init-cdc.sql`, which\n drops and recreates `CdcTest` and wipes the mutation. Rejects\n `--fixture=` in the same invocation (either apply fixtures or skip\n them).\n\nThe between-phase mutation itself is a plain `apply-sql.sh` call\nagainst a fixture in `fixtures/sql/`. Nothing about the mutation is\nspecial-cased in `run.sh` — the case script drives the sequencing.\n","airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/SKILL.md":"---\nname: source-bigquery-e2e-tests\ndescription: Run the Airbyte protocol smoke sweep for airbyte/source-bigquery: against the shared BigQuery integration project inside a per-run PROVEFIX_ dataset (B3 remote-backend mode of db-harness-lib), with a dry-run orphan sweep.\n---\n\n# source-bigquery-e2e-tests\n\nThis harness runs the source-bigquery protocol regression sweep against an isolated dataset in a real BigQuery project. Orchestration comes from `airbyte-integrations/db-harness-lib`; the engine scripts own the dataset, fixtures, and credentials.\n\n## Prerequisites\n\n- `gcloud` with ambient access to Secret Manager in `GSM_PROJECT`.\n- `bq` and `docker`.\n- `airbyte-ops` (or `uvx airbyte-internal-ops`).\n- A BigQuery service account with dataset create/delete, query, and source-read permissions in `BIGQUERY_PROJECT`.\n\nCloud Storage Read API coverage requires `airbyte-ops cloud connector regression-test --extra-env`, added in [airbyte-ops-mcp#349](https://github.com/airbytehq/airbyte-ops-mcp/pull/349). The shim defaults `BIGQUERY_AIRBYTE_EDITION` to `CLOUD` and adds `AIRBYTE_EDITION=CLOUD` when supported. Set `BIGQUERY_AIRBYTE_EDITION=''` to disable the check and injection. Older `airbyte-ops` versions warn and continue using the query API.\n\n## Run\n\nFrom this connector directory:\n\n```bash\npoe e2e-local --test-version=1.0.1\n```\n\nThe Poe task forwards the shared harness flags; use `poe e2e-local --help` for the full list. Common options include `--command=check`, `--skip-read`, `--step-name=NAME`, `--keep-backend`, and `--reset=fixture`. Environment overrides include `REPRO_OUT`, `GSM_PROJECT`, `BIGQUERY_PROJECT`, `BIGQUERY_LOCATION`, `BIGQUERY_SECRET_NAME`, `BIGQUERY_CONFIG_FILE`, `AIRBYTE_OPS`, and `BIGQUERY_AIRBYTE_EDITION`.\n\n## Dataset lifecycle\n\nEach run creates `PROVEFIX_` in `BIGQUERY_PROJECT` and records ownership in `$REPRO_OUT/.provefix-owned-PROVEFIX_`. An existing dataset is reused only with that marker; otherwise the harness refuses to touch it and points to `sweep-orphans.sh`. The default teardown drops the owned dataset and verifies it is absent. `--reset=fixture` drops and recreates the run dataset between comparison sweeps. `sweep-orphans.sh --dry-run` lists old `PROVEFIX_*` datasets without deleting them; `--older-than-hours=N` defaults to 4.\n\n`gcloud`/`bq` authentication uses `$REPRO_OUT/.gcloud`, separate from the ambient credentials used to fetch the GSM secret. The fetched service-account key and isolated auth directory are removed at teardown.\n\n## Expected smoke result\n\nBoth runs pass CHECK, DISCOVER `smoke_table` and `smoke_view`, and READ three records from each stream. The installed CLI warns when it lacks `--extra-env`, so that run exercises only the query API. With `BIGQUERY_AIRBYTE_EDITION=CLOUD` and the #349 CLI, connector logs show `smoke_table` opening a Storage Read API session and `smoke_view` falling back to the query API because views are unsupported by the Storage Read API.\n\nThere is no bad-vs-fixed proof case yet: the 1.0.1 views fallback requires a service account without BigQuery Read Session User, and none exists in GSM.\n","airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/SKILL.md":"---\nname: source-mongodb-v2-e2e-tests\ndescription: Stand up a local MongoDB 7.0 single-node replica set, apply mongosh JavaScript fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mongodb-v2: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any non-CDC source-mongodb-v2 test or prove-fix comparison.\n---\n\n# source-mongodb-v2-e2e-tests\n\nLocal end-to-end test harness for `source-mongodb-v2`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MongoDB backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MongoDB 7.0 container named\n`source-mongodb-v2-db-backend` configured as a single-node replica set\n(`rs0`), lets you apply arbitrary `mongosh` JavaScript fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mongodb-v2:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: non-CDC (full-refresh / initial-snapshot) behavior. Change-stream\nresume-token replay and CDC comparison flows are not covered by this skill.\n\n## When to use this skill\n\n- Reproducing a `source-mongodb-v2` bug locally without Atlas credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-mongodb-v2:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM, Atlas, or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mongodb-v2-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mongo:7.0 --replSet rs0; rs.initiate()\n│ ├── apply-sql.sh # docker exec mongosh with a .js fixture on stdin\n│ ├── reset-databases.sh # drop every non-system database\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # SELF_MANAGED_REPLICA_SET config; host placeholder\n └── js/\n └── 00-init-base.js # test_db.sample with three documents\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but MongoDB\nfixtures are `mongosh` scripts, not SQL. Each fixture runs with `test_db`\n(`BACKEND_DB`) as the current `db`; use `db.getSiblingDB(...)` for others.\n\n## Conventions\n\n- Container name: `source-mongodb-v2-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The backend runs without `--auth`; the config's\n `database_config` omits `username`/`password`. Add them to a copied\n template if you need to reproduce an auth-specific issue.\n- Replica set: `rs0` (`BACKEND_REPLSET`). The connector requires a replica\n set even for non-CDC reads. `start-backend.sh` registers the single member\n under the container's bridge IP so the connector container, which performs\n replica-set discovery from the seed list, can reach the advertised host.\n Do not change the member host to `localhost`: the connector would then\n fail to connect.\n- Initial database: `test_db`. Override with `BACKEND_DB`; keep\n `database_config.databases` in the config template in sync.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mongodb-v2-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `database_config.connection_string` to\n `mongodb://:27017/?replicaSet=rs0` at runtime.\n- The backend image is pinned to `mongo:7.0`. Override with `BACKEND_IMAGE=…`\n for a version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mongodb-v2\n\n# Single version, full sweep.\npoe e2e-local --test-version=2.0.7\n\n# Target vs. control comparison.\npoe e2e-local --test-version=dev --control-version=2.0.6 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=2.0.7\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. Under CI the table is also\nappended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **Schema enforcement.** The base template sets\n `database_config.schema_enforced: true`, so `discover` samples documents\n and emits typed fields. Set it to `false` in a copied template to\n reproduce schemaless (`data` blob) behavior.\n- **`_id` types.** Fixtures use integer `_id`s for readability; use\n `ObjectId()` in a fixture when the bug depends on ObjectId serialization.\n- **Schema inference needs documents.** An empty collection created with\n `db.createCollection()` is still listed by `discover`, but with\n `schema_enforced: true` its schema has no sampled fields beyond `_id`.\n Insert at least one document in the fixture for every field you expect\n in the catalog.\n- **Reset drops databases, not the replica set.** `--reset=fixture` removes\n all non-system databases and reapplies fixtures; the oplog and resume\n tokens survive. This only matters once change-stream (CDC) coverage\n lands (HYD-147 phase 2); non-CDC sweeps do not read the oplog. Use\n `--reset=backend` when a run must start from a fresh oplog.\n- **Databases are created on first write.** MongoDB has no `CREATE\n DATABASE`; `test_db` exists only once a fixture inserts into it, so a\n fixture that only calls `db.createCollection(...)` or writes to a\n `getSiblingDB(...)` database leaves `test_db` absent and `discover` empty.\n `00-init-base.js` inserts into `test_db`; keep that in any fixture set you\n pass with `--fixture`.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections, Atlas clusters\n holding customer data, or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mongodb-v2-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/SKILL.md":"---\nname: source-dynamodb-e2e-tests\ndescription: Stand up a local DynamoDB Local backend, apply JSON fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-dynamodb: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for a cursor-incremental source-dynamodb test or prove-fix comparison.\n---\n\n# source-dynamodb-e2e-tests\n\nLocal end-to-end test harness for `source-dynamodb`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the DynamoDB Local backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a DynamoDB Local container named\n`source-dynamodb-db-backend`, applies JSON fixtures with the AWS CLI, and runs\nAirbyte protocol commands against any `airbyte/source-dynamodb:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: no CDC mode exists; incremental reads use a user-defined cursor\nattribute. Assume-role / IAM / STS behavior is not covered: DynamoDB Local\ndoes not provide STS, and the connector's `endpoint` only overrides the\nDynamoDB client, so a published connector image cannot send AssumeRole\nrequests to this emulator. State this limitation in the evidence plan rather\nthan claiming coverage.\n\n## When to use this skill\n\n- Reproducing a `source-dynamodb` bug locally without AWS credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-dynamodb:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need AWS, IAM, or Cloud credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-dynamodb-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── aws-cli.sh # AWS CLI container and backend bridge-IP helpers\n│ ├── start-backend.sh # docker run DynamoDB Local with -sharedDb\n│ ├── apply-sql.sh # delete/create tables and put fixture items\n│ ├── reset-databases.sh # delete every table\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # user credentials, region, endpoint placeholder\n └── json/\n ├── 00-init-base.json # sample with a numeric cursor and three items\n ├── attribute-types.json\n └── empty-table.json\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but DynamoDB\nfixtures are JSON documents, not SQL. Each fixture has a `tables` array. Each\ntable entry contains a `create` object accepted by `CreateTable\n--cli-input-json` and an `items` array in DynamoDB attribute-value JSON:\n\n```json\n{\n \"tables\": [\n {\n \"create\": {\n \"TableName\": \"sample\",\n \"KeySchema\": [{\"AttributeName\": \"id\", \"KeyType\": \"HASH\"}],\n \"AttributeDefinitions\": [{\"AttributeName\": \"id\", \"AttributeType\": \"S\"}],\n \"BillingMode\": \"PAY_PER_REQUEST\"\n },\n \"items\": [{\"id\": {\"S\": \"e1\"}, \"seq\": {\"N\": \"1\"}}]\n }\n ]\n}\n```\n\n## Conventions\n\n- Container name: `source-dynamodb-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The config uses dummy `User` credentials and\n `us-east-1`; DynamoDB Local does not authenticate them.\n- `-sharedDb` is required. Without it, DynamoDB Local namespaces tables by\n access key and region, so tables created by the fixture client can be\n invisible to the connector client.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-dynamodb-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `.endpoint` to `http://:8000` at runtime. The AWS CLI\n helper resolves the same bridge IP for its endpoint.\n- DynamoDB Local is pinned to `amazon/dynamodb-local:3.3.2`. Override with\n `BACKEND_IMAGE=…` for a version-specific reproduction. The AWS CLI helper\n image is pinned to `amazon/aws-cli:2.38.2`; override with\n `AWS_CLI_IMAGE=…`.\n- `attribute-types.json` exercises DynamoDB attribute-value families and a\n composite-key table. The connector's source-defined primary key is the\n partition key only; its sort key is not part of that primary key.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-dynamodb\n\n# Single version, full sweep.\npoe e2e-local --test-version=0.4.1\n\n# Target vs. control comparison.\npoe e2e-local --test-version=0.4.1 --control-version=0.3.11 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=0.4.1\n```\n\nThe sweep runs every command against the backend and reports each result\nrather than stopping at the first failure, then prints a summary table. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **DynamoDB key attributes.** `AttributeDefinitions` contains only the\n table's key attributes; non-key attributes such as `seq` and `label` appear\n in fixture items and are discovered by sampling them.\n- **Sampled attributes.** Discovery samples table items and unions the\n attributes it sees. Keep at least one item with each field you want in the\n catalog. A field missing from an item is a useful missing-attribute case,\n but it may not be emitted if it is absent from every sampled item.\n- **Configured-catalog nulls.** DynamoDB discover omits `namespace`,\n `source_defined_cursor`, and `is_resumable`, and 0.4.0 accepts them absent;\n it rejects `namespace: null`. Keep them omitted through an Airbyte Ops build\n that serializes configured catalogs with `exclude_none=True`; no shared\n catalog-generator defaults are needed.\n- **Empty tables.** The connector discovers key attributes from\n `DescribeTable` when there are no items to sample, so an empty table remains\n in the catalog with its key fields.\n- **Comparison catalogs.** Use `--reset=fixture` for comparisons where\n discovery output changes between image tags. In particular, for the\n integer-cursor regression below, the old tag discovers `seq` as\n `[\"null\",\"integer\"]` while the fixed tag discovers it as `number` with\n `airbyte_type: integer`. A paired `--reset=none` run can reuse the target's\n discovered catalog for the control read and mask the control bug, because\n the crash is driven by the cursor's catalog JSON-schema type. Fixture reset\n runs separate sweeps, so each image derives its own catalog.\n- **No role coverage.** Do not use DynamoDB Local to claim assume-role, IAM,\n or STS coverage. There is no STS emulator here, and `endpoint` only\n overrides the DynamoDB client endpoint.\n- **Do not use real AWS.** This harness is for local testing only and must\n never be used against customer AWS accounts, tables containing customer\n data, or Airbyte Cloud.\n\n## Worked example: integer cursor, airbyte#71171\n\nThe historical integer-cursor crash can be proven with the base fixture. The\ncontrol image fails with\n`IllegalArgumentException: No enum constant\nio.airbyte.protocol.models.JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER`\nand emits no records; the fixed image reads all three records and emits\n`STATE` with `cursor: \"3\"`.\n\nUse `--reset=fixture` so the 0.3.11 control and 0.4.0 target each run their\nown `discover` and `read`. The versions discover different cursor schemas,\nand the crash depends on that schema in the configured catalog.\nThis command also requires an Airbyte Ops build with the configured-catalog\n`exclude_none=True` fix: DynamoDB discovery omits `namespace`, and older Ops\nbuilds serialize that omission as `namespace: null`, which 0.4.0 rejects.\n\n```bash\nREPRO_OUT=\"$HOME/dynamodb-hyd310-evidence/patched-ops-1792531195\" \\\n poe e2e-local \\\n --step-name=integer-cursor-71171 \\\n --control-version=0.3.11 \\\n --test-version=0.4.0 \\\n --reset=fixture \\\n --sync-mode=incremental \\\n --cursor-field=seq \\\n --streams=sample \\\n --expect-control=fail \\\n --expect-test=pass \\\n --min-records=3 \\\n --min-states=1 \\\n --forbid-match=read:any:JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER\n```\n\nVerified on 2026-10-20 with the Airbyte Ops null-namespace fix:\n\n| Command | Control (`0.3.11`) | Target (`0.4.0`) |\n|---------|---------|---------|\n| `SPEC` | PASS | PASS |\n| `CHECK` | PASS | PASS |\n| `DISCOVER` | PASS | PASS |\n| `READ` | FAIL — integer-enum crash, 0 records, 0 states | PASS — 3 records, 1 state (`cursor: \"3\"`) |\n\nThe paired command exited 0: the control reproduced the historical failure,\nwhile the target passed and emitted the expected records and state.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-dynamodb-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/SKILL.md":"---\nname: source-postgres-e2e-cdc-tests\ndescription: Reproduce PostgreSQL logical-decoding CDC behavior against a local PostgreSQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-postgres-e2e-tests and db-harness-lib.\n---\n\n# source-postgres-e2e-cdc-tests\n\nLocal CDC test harness for `source-postgres`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic PostgreSQL skill\n([`source-postgres-e2e-tests`](../source-postgres-e2e-tests/SKILL.md)). The\ngeneric skill starts a PostgreSQL 16 backend and runs the connector; this skill\nsupplies the logical-decoding CDC config, CDC catalog, SQL fixtures, and case\nscripts.\n\nPostgreSQL CDC uses logical decoding from a publication and replication slot.\nThe generic PostgreSQL backend starts with the WAL and replication settings\nrequired by this skill, so this skill does not start a second backend or add\nper-run server configuration.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-postgres` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new PostgreSQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-postgres-db-backend`, started by\n `../source-postgres-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-postgres-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with PostgreSQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # publication, slot, table, and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\nThe fixture uses the generic backend database `test_db`, because PostgreSQL\ncannot drop the database to which `apply-sql.sh` is connected. It resets the\n`users` table, publication, and logical replication slot in that database.\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic PostgreSQL backend starts with `wal_level=logical`,\n `max_replication_slots=10`, and `max_wal_senders=10`. Do not add per-run\n server configuration or start another backend in this skill.\n- `00-init-cdc.sql` drops and recreates the `users` table, publication, and\n `airbyte_slot` in `test_db` on every application. Re-applying it resets the\n fixture, which is why `cases/incremental-replay.sh` passes `--skip-fixtures`\n on its second phase.\n- PostgreSQL 3.8.5 discovers `_ab_cdc_lsn` as the source-defined cursor and\n does not include the newer generated catalog fields in discover output.\n The configured catalog includes `is_file_based`, generation metadata, and\n destination metadata because the pinned image requires those fields when\n initializing `read`.\n- Cases default to `VERSION=3.8.5`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-postgres-repro\n\n# Start the logical-decoding-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-postgres-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the logical-decoding-ready backend, CDC config, catalog metadata, and\ninitial load work together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved LSN\nresumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No PostgreSQL CDC fixtures\nor worked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep CDC setup in the\n configured database and do not add per-run server configuration.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`. Use the source-defined cursor\n emitted by `discover` for the pinned connector version.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.8.5}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses `psql` on stdin, so fixtures may contain\nmultiple statements. Logical replication settings are established when the\ngeneric backend starts; no additional CDC setup is needed in a case beyond the\npublication and replication slot in the SQL fixture.\n","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/SKILL.md":"---\nname: source-mysql-e2e-cdc-tests\ndescription: Reproduce MySQL binlog CDC behavior against a local MySQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-mysql-e2e-tests and db-harness-lib.\n---\n\n# source-mysql-e2e-cdc-tests\n\nLocal CDC test harness for `source-mysql`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic MySQL skill\n([`source-mysql-e2e-tests`](../source-mysql-e2e-tests/SKILL.md)). The generic\nskill starts a MySQL 8.0 backend and runs the connector; this skill supplies\nthe binlog CDC config, CDC catalog, SQL fixtures, and case scripts.\n\nMySQL CDC is binlog-based and server-wide. The generic backend already starts\nMySQL with ROW binlogging, full row images, and GTIDs, so this skill does not\nenable CDC per table or start a second backend.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-mysql` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new MySQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-mysql-db-backend`, started by\n `../source-mysql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-mysql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with MySQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # dedicated cdc_test database and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic MySQL backend starts with `--log-bin`, `--binlog-format=ROW`,\n `--binlog-row-image=FULL`, `--gtid-mode=ON`, and\n `--enforce-gtid-consistency=ON`. Do not add per-table CDC enable steps or\n engine-specific backend startup to this skill.\n- `00-init-cdc.sql` drops and recreates the dedicated `cdc_test` database on\n every application. Re-applying it wipes existing rows and any accumulated\n binlog-relative table state, which is why `cases/incremental-replay.sh`\n passes `--skip-fixtures` on its second phase.\n- CDC catalogs include the bulk-CDK fields (`is_file_based`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files`) and use\n `[\"_ab_cdc_cursor\"]` as the cursor field.\n- Cases default to `VERSION=3.53.4`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mysql-repro\n\n# Start the binlog-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-mysql-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the binlog-ready backend, CDC config, catalog metadata, and initial load\nwork together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved binlog\nposition resumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No MySQL CDC fixtures or\nworked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep any CDC database\n setup dedicated to the fixture and do not add per-table CDC enable steps.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.53.4}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses the plain MySQL client on stdin, so fixtures\nmay contain multiple statements. The MySQL server's binlog and GTID settings\nare established when the generic backend starts; no additional CDC setup is\nneeded in a case.\n","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/SKILL.md":"---\nname: source-mysql-e2e-tests\ndescription: Stand up a local MySQL 8.0 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mysql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mysql test or prove-fix comparison.\n---\n\n# source-mysql-e2e-tests\n\nLocal end-to-end test harness for `source-mysql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MySQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MySQL 8.0 container named\n`source-mysql-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mysql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a `source-mysql` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mysql:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mysql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mysql:8.0 with ROW binlog + GTIDs\n│ ├── apply-sql.sh # docker exec mysql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mysql-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Root password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mysql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the MySQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `mysql:8.0`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n- The backend starts with ROW-format binary logging, a full row image,\n and GTIDs. This leaves the container ready for a CDC driver without\n restarting it.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mysql\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.53.3\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.53.2\n\n# One command only.\npoe e2e-local --command=read --test-version=3.53.3\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose `discover`\nfailed earlier is reported `SKIPPED`, not `ERROR`. Under CI the table is\nalso appended to `$GITHUB_STEP_SUMMARY`. A run limited to one command\ninstead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`. Use this for CDC\n comparisons so the target does not read a warm capture instance.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the MySQL\n container between sweeps. Use this only when matching binlog or GTID\n sequences matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC needs ROW binlogging.** `start-backend.sh` sets\n `--binlog-format=ROW`, `--binlog-row-image=FULL`, and a server ID.\n The connecting user must have replication privileges; the root user\n created by the fixture has them.\n- **MySQL config shapes are specific.** `ssl_mode` uses `preferred`,\n `required`, `verify_ca`, or `verify_identity`; the base config uses\n `preferred` for the local plaintext backend. The\n `replication_method.method` value is `STANDARD` or `CDC`.\n- **`tinyint(1)` mapping.** The connector maps `TINYINT(1)` to boolean by\n default. Set `treat_tinyint1_as_integer: true` in a config when\n reproducing a typing issue.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mysql-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/SKILL.md":"---\nname: source-postgres-e2e-tests\ndescription: Stand up a local PostgreSQL 16 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-postgres: for ad-hoc end-to-end testing.\n---\n\n# source-postgres-e2e-tests\n\nLocal end-to-end test harness for `source-postgres`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the PostgreSQL backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a PostgreSQL 16 container named\n`source-postgres-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-postgres:` image via\n`airbyte-ops cloud connector regression-test`.\n\nThis skill supports standard (non-CDC) local sweeps. CDC fixtures and\nreplication setup are not yet stood up in this skill.\n\n## When to use this skill\n\n- Reproducing a `source-postgres` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-postgres:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-postgres-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run postgres:16 with logical replication\n│ ├── apply-sql.sh # docker exec psql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-postgres-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Postgres password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-postgres-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the PostgreSQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `postgres:16`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-postgres\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.8.5\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.8.4\n\n# One command only.\npoe e2e-local --command=read --test-version=3.8.5\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose\n`discover` failed earlier is reported `SKIPPED`, not `ERROR`. Under CI the\ntable is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to one\ncommand instead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the\n PostgreSQL container between sweeps. Use this only when matching database\n initialization state matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC is not included in this skill.** The lifecycle starts PostgreSQL 16\n with logical replication enabled so the composed\n [`source-postgres-e2e-cdc-tests`](../source-postgres-e2e-cdc-tests/SKILL.md)\n skill can use the same backend. CDC fixtures and case scripts live in that\n skill.\n- **PostgreSQL config shapes are specific.** `ssl_mode.mode` uses\n `prefer` for the local plaintext backend. The\n `replication_method.method` value for a standard sync is `Standard`.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-postgres-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/SKILL.md":"---\nname: source-snowflake-e2e-tests\ndescription: Run the Airbyte protocol sweep for airbyte/source-snowflake: against the shared Snowflake integration account inside a per-run PROVEFIX_ schema (B3 remote-backend mode of db-harness-lib); prove-fix comparisons, cursor canary, orphan sweep.\n---\n\n# source-snowflake-e2e-tests\n\nRemote end-to-end harness for `source-snowflake`. The shared\n`db-harness-lib` orchestration runs protocol commands against a per-run schema\nin the shared Snowflake integration account.\n\n## Prerequisites\n\n- Docker, [`uv`](https://docs.astral.sh/uv/), and `jq`.\n- `gcloud` authenticated with read access to the GSM secret, or an existing\n `SNOWFLAKE_CONFIG_FILE`.\n- A clone of `airbytehq/airbyte`.\n\n## Layout\n\n```\nsource-snowflake-e2e-tests/\n├── SKILL.md\n├── ../../../poe_tasks.toml (connector-root entrypoint)\n├── scripts/\n│ ├── run.sh, sf.py\n│ ├── fetch-config.sh, render-config.sh\n│ ├── start-backend.sh, reset-databases.sh, stop-backend.sh\n│ ├── apply-sql.sh, sweep-orphans.sh\n├── fixtures/configs/base.template.json, utc-session.template.json\n├── fixtures/sql/00-cursor-canary-base.sql\n├── fixtures/sql/01-cursor-canary-boundary-row.sql\n└── cases/cursor-upper-bound.sh\n```\n\n## Isolation model\n\n- Every run uses `PROVEFIX_` and asserts that the schema is absent\n before both control and target runs.\n- A marker owned by the run permits reuse of the same schema between canary\n phases; unmarked pre-existing schemas are refused.\n- Control and target use the same schema with `--reset=fixture`; remote\n `--reset=backend` is rejected.\n- Teardown drops the schema in the run's exit trap.\n- `scripts/sweep-orphans.sh` lists or drops old `PROVEFIX_*` schemas, defaulting\n to four hours.\n\nThe default run id is\n`YYYYMMDDhhmmss__`, producing a\nschema-safe `PROVEFIX_` name.\n\nThe schema is owned by the scripts, while `sf.py` is the only code that talks\nto Snowflake. The default config comes from GSM secret\n`SECRET_SOURCE-SNOWFLAKE_KEY_PAIR__CREDS` in project\n`dataline-integration-testing`; fetched configs are marked with a sidecar and\nonly those configs are removed during teardown, so caller-supplied configs\nsurvive. Set `SNOWFLAKE_CONFIG_FILE` to reuse a caller-supplied config.\n\n## Cursor upper-bound canary\n\nThe canary proves the cursor precision regression from airbytehq/airbyte#82705:\n\n```bash\ncases/cursor-upper-bound.sh\n```\n\nIt compares fixed `1.1.1` with control `1.1.0` in two phases. Phase 1 runs the\ninitial sync on the control image with exact-microsecond base rows, producing\nthe checkpoint a user on the bad version would have. Phase 2 inserts a\nsub-microsecond boundary row and replays that state against both versions:\n1.1.0 loses the row because it floors the upper bound, while 1.1.1 emits it.\n\nThe harness reuses the `INTEGRATION_TEST_WAREHOUSE_DESTINATION` X-Small\nwarehouse shared with destination-snowflake integration tests. Never touch\nschemas outside `PROVEFIX_*`.\n\n## Gotchas\n\nSnowflake uppercases identifiers, so use\n`--cursor-field=UPDATED_AT --streams=CURSOR_CANARY`. Rendered configs contain\nthe private key; treat `REPRO_OUT` as 0600 material and never attach it.\nTIMESTAMP_NTZ cursor bounds are bound as TIMESTAMP_LTZ by the JDBC driver and\ncompared through the session TIMEZONE; with the account default\n(America/Los_Angeles), narrow incremental windows match nothing.\nThe harness sets `jdbc_url_params=TIMEZONE=UTC` so the canary isolates #82705;\nthis session-timezone behavior appears to be a separate connector bug worth\nits own issue.\nThe `--catalog` CDC guard (#85752) never fires here: source-snowflake has no\nCDC replication method.\n","tools/bin/admin-service-account.yaml":"apiVersion: rbac.authorization.k8s.io/v1\nkind: Role\nmetadata:\n name: airbyte-admin-role\nrules:\n - apiGroups: [\"*\"]\n resources: [\"jobs\", \"pods\", \"pods/log\", \"pods/exec\", \"pods/attach\"]\n verbs: [\"get\", \"list\", \"watch\", \"create\", \"update\", \"patch\", \"delete\"] # over-permission for now\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: RoleBinding\nmetadata:\n name: default-admin-binding\nroleRef:\n apiGroup: \"\"\n kind: Role\n name: airbyte-admin-role\nsubjects:\n - kind: ServiceAccount\n name: default\n","tools/bin/run-harness-process.yaml":"# Bin Skill\n\nPath: `tools/bin/run-harness-process.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/bin/run-harness-process.yaml)","tools/internal/README.md":"# internal Documentation\n\nPath: `tools/internal/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/internal/README.md)","tools/schema_generator/README.md":"# schema_generator Documentation\n\nPath: `tools/schema_generator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/schema_generator/README.md)","tools/site/README.md":"# site Documentation\n\nPath: `tools/site/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/site/README.md)","airbyte-integrations/connectors/source-commercetools/README.md":"# source-commercetools Documentation\n\nPath: `airbyte-integrations/connectors/source-commercetools/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-commercetools/README.md)","docs/release_notes/agents/readme.md":"# Subagent: readme\n\nPath: `docs/release_notes/agents/readme.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/release_notes/agents/readme.md)","docs/ai-agents/get-started/developer-quickstart/skills/_category_.json":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/_category_.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/_category_.json)","docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md)","docs/ai-agents/get-started/developer-quickstart/skills/codex.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/codex.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/codex.md)","docs/ai-agents/get-started/developer-quickstart/skills/lovable.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/lovable.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/lovable.md)","docs/ai-agents/get-started/developer-quickstart/skills/readme.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/readme.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/readme.md)","airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json":"# shared Plugin Manifest\n\nPath: `airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json)","airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json)","airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json)","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json)","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json)"},"files":{".claude/skills/my-prs/SKILL.md":"---\nname: my-prs\ndescription: List PRs opened today that are assigned to me or the move-destinations team, excluding external contributors\ndisable-model-invocation: true\nallowed-tools: Bash(gh:*), Bash(date:*), Read\nargument-hint: [days-back]\n---\n\n# My PRs - Daily PR Review Dashboard\n\nThis skill fetches and displays PRs that need your attention:\n\n1. **PRs assigned directly to you** (@frifriSF59)\n2. **PRs with review requested from the move-destinations team**\n\nAll results are filtered to:\n- Only show PRs opened today (or within `$ARGUMENTS` days if specified)\n- Exclude PRs from users who are not members of the airbytehq organization\n\n## Execution\n\nRun the fetch script to get the PR list:\n\n```bash\nbash .claude/skills/my-prs/scripts/fetch-prs.sh\n```\n\nIf an argument is provided (number of days), modify the date filter accordingly:\n- `$ARGUMENTS` = number of days to look back (default: 0 = today only)\n- Example: `/my-prs 3` looks at PRs from the last 3 days\n\nFor custom date ranges, use:\n```bash\ngh pr list --repo airbytehq/airbyte --assignee @me --search \"created:>=YYYY-MM-DD\" --json number,title,author,url\n```\n\n## After Fetching\n\nAfter displaying the PRs, offer to:\n1. Open a specific PR in the browser (`gh pr view --web`)\n2. Show details of a specific PR (`gh pr view `)\n3. Show the diff of a specific PR (`gh pr diff `)\n4. Check CI status (`gh pr checks `)\n\n## Quick Actions\n\nThe user may want to:\n- **Review a PR**: Use `/review ` if available\n- **Check PR comments**: `gh pr view --comments`\n- **See who approved**: `gh pr view --json reviews --jq '.reviews[] | select(.state==\"APPROVED\") | .author.login'`\n","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/SKILL.md":"---\nname: source-mssql-e2e-tests\ndescription: Stand up a local SQL Server 2022 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mssql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mssql test or repro that is not CDC-specific.\n---\n\n# source-mssql-e2e-tests\n\nLocal end-to-end test harness for `source-mssql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../db-harness-lib/);\nthis skill keeps the MSSQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a SQL Server\n2022 container named `source-mssql-db-backend`, lets you apply arbitrary\nSQL fixtures, and runs Airbyte protocol commands against any\n`airbyte/source-mssql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a non-CDC bug against `source-mssql` locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mssql:` for connector\n development.\n- As the building block for the\n [`source-mssql-e2e-cdc-tests`](../source-mssql-e2e-cdc-tests/SKILL.md)\n skill, which adds CDC enable plus worked-example fixtures on top.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install airbyte-internal-ops`\n puts `airbyte-ops` on `$PATH`; alternatively prefix every call with\n `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mssql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mcr…/mssql/server:2022-latest as source-mssql-db-backend\n│ ├── apply-sql.sh # docker cp + docker exec sqlcmd -i\n│ ├── reset-databases.sh # docker exec sqlcmd → drop every non-system database (used by run.sh --reset=fixture)\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # non-CDC config; host=mssql-db-backend placeholder\n └── sql/\n ├── .gitignore # `.tmp/` — subtree for uncommitted per-bug scratch fixtures\n └── 00-init-base.sql # CREATE DATABASE TestDb + dbo.sample table\n```\n\nThe engine shim and backend lifecycle scripts remain in this skill's\n`scripts/` directory; the shared orchestration, default teardown, and\nprotocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mssql-db-backend`. Hard-coded in\n `scripts/start-backend.sh` and `apply-sql.sh`. The shared\n `db-harness-lib/scripts/stop-backend.sh` removes this container by\n `BACKEND_NAME`; the engine shim supplies the default. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mssql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per\n protocol command (`spec/`, `check/`, `discover/`, `read/`) holding that\n command's `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n This is the same layout the ops repo's `connector-regression-test.yml`\n workflow uses under `/tmp/regression_test_artifacts`, so CI can upload\n it per command.\n- Both containers (the SQL Server backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765);\n once `--network` is supported the bridge-IP dance can collapse.\n- SQL Server image is pinned to `mcr.microsoft.com/mssql/server:2022-latest`,\n not `latest`, for stable major-version behavior across CU patches.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover`'s output, run `read`, tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mssql\n\n# Single version, full sweep.\npoe e2e-local --test-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, non-CDC (airbyte-ops's built-in comparator).\npoe e2e-local --test-version=dev --control-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, CDC (per-image reset between the two runs).\npoe e2e-local --test-version=dev --control-version=5.0.0 --reset=fixture \\\n --config-template=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json \\\n --sync-mode=incremental --cursor-field=_ab_cdc_cursor --streams=users \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/00-init-cdc.sql \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/.sql\n\n# One command only.\npoe e2e-local --command=read --test-version=5.0.0\n```\n\nThe CDC example needs an incremental catalog (`--sync-mode=incremental\n--cursor-field=_ab_cdc_cursor`) restricted with `--streams` to exclude\n`dbo.systranschemas`; the general rule is in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md#cdc-config-templates-need-an-incremental-catalog).\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed — separating an\ninfrastructure failure (`ERROR`, no verdict was produced) from a failed\ntest verdict (`FAIL`), as the ops workflow's final-status step does. A\n`read` whose `discover` failed earlier in the sweep is reported `SKIPPED`,\nnot `ERROR` — an invalid-config case is expected to fail `discover`, and\nthere is then no catalog for `read` to use. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to\none command instead exits with the connector's own exit code, so a repro\ncan still assert on it.\n\nPrefer a published tag over `--test-version=dev`: for code on a pushed PR\nbranch, publish a pre-release and pass its `-preview.` tag,\nwhich skips the Gradle build and gives reviewers a tag they can re-run\nagainst — see\n[Getting a target image](../../../AGENTS.md#getting-a-target-image). When\nyou do use `--test-version=dev`, build the target image first, or pass\n`--build` to have `run.sh` run `:dockerBuildx` for you. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures` (run against whatever state the backend already has;\nfor the second/later `run.sh` invocation of a multi-phase driver, when\nre-applying the initial fixture would wipe intermediate state),\n`--step-name`, `--catalog` (skip discover-derived generation),\n`--state=PATH` (pass a saved STATE file to the read as `--state-path`;\nfor use by multi-phase drivers), `--sync-mode=incremental`,\n`--cursor-field`, `--streams`, `--config-template`,\n`--reset=none|fixture|backend`, `--keep-backend`, and `--` to forward\nextra args to `airbyte-ops`. Per-command timeouts match the workflow's\n(30/30/60/180 minutes) and are overridable with\n`TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n### Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst each command's artifacts, `run.sh` accepts a small set of\nexpectation flags that it enforces itself before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `--expect-test=pass\\|fail` | Overall target verdict. `pass` = every executed command's status was `pass`; `fail` = one or more failed. |\n| `--expect-control=pass\\|fail` | Same for the control sweep (comparison-mode runs only; requires `--control-version`). |\n| `--min-records=N` | Target read's `stdout.txt` must contain ≥N `RECORD` messages. |\n| `--min-states=N` | Target read's `stdout.txt` must contain ≥N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target's `` step's `` (`stdout` \\| `stderr` \\| `any`) must match `` at least N times (default 1). `` (`spec` \\| `check` \\| `discover` \\| `read`) is optional and defaults to `read`. Repeatable. |\n| `--forbid-match=[:]:` | Same shape but must match zero times. Repeatable. |\n\nAll match assertions run against the **target-side** artifacts of the\nnamed command (or `read` if no `` prefix). Under\n`--control-version + --reset=none` those live at\n`$ARTIFACTS_DIR//target/` (created by airbyte-ops's comparison\nmode); under `--reset=fixture|backend` at\n`$ARTIFACTS_DIR/target//`; under single-version at\n`$ARTIFACTS_DIR//`. `run.sh` picks the right path automatically.\n\nThe match-spec grammar disambiguates the leading `:` from the\n`:` on the first colon-separated field (command names and\nchannel names don't overlap), and strips a trailing `:N` only when the\nlast field is a positive integer. So a regex containing `:` still\nparses: `--expect-match=stderr:io.debezium.DebeziumException` reads as\ncommand `read` (default), channel `stderr`, regex\n`io.debezium.DebeziumException`, count 1. `--expect-match=check:stderr:should be positive`\nreads as command `check`, channel `stderr`, regex `should be positive`,\ncount 1.\n\n`--min-records` and `--min-states` are read-step only — they count\nRECORD / STATE envelopes, which are read-specific — and have no\n`:` prefix.\n\nAny expectation failure exits non-zero regardless of the command-level\nverdicts and appends an `**Expectation failures:**` section to the\nsummary. Migrating an existing driver script's `grep -q '' || exit 1`\nblock is straightforward — drop the block and add\n`--expect-match=stderr:X` to the shared `run.sh` invocation.\n\n### Multi-phase drivers\n\n`run.sh` runs one sweep. A driver script composes multi-phase flows\n(read → mutate → read-with-state) by calling `run.sh` more than once\nwith `--step-name` for each invocation and `--state=` to feed the\nsecond read the state extracted from the first. `extract-state.py`\nlives in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\nbecause it walks Airbyte STATE messages, which is a protocol-level\noperation and not CDC-specific.\n\nThe CDC skill's [`cases/11451.sh`](../source-mssql-e2e-cdc-tests/cases/11451.sh)\nis a worked example: it invokes the engine shim's `run.sh` around an\nintermediate `apply-sql.sh` mutation and the shared\n`db-harness-lib/scripts/extract-state.py` step, and asserts on the\nconnector's stderr. New multi-phase drivers can follow the same shape\nbut call `run.sh --state=PATH` directly instead of smuggling\n`--state-path` through trailing args.\n\n### Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` once per\n protocol command with both `--test-image` and `--control-image`, so\n the two images run sequentially against a single backend and\n `airbyte-ops`'s built-in comparators emit the target-vs-control diff\n (record counts, primary keys, per-record, schema). Right for non-CDC\n full-refresh work: the diff is meaningful and the shared-backend\n assumption is safe because a full-refresh read does not mutate the\n upstream. This is the current behavior for any caller that doesn't\n set `--reset`.\n- **`--reset=fixture`** — `run.sh` runs the whole sweep against the\n control image first, drops every non-system database, re-applies the\n fixtures, then runs the sweep against the target image. Two\n single-version `airbyte-ops` calls, no built-in comparator; artifacts\n land under `$REPRO_OUT//{control,target}//`.\n Right for **CDC comparisons**, where reusing the backend leaves the\n target reading against a warm capture instance and an advanced log\n position — the built-in diff can look clean while being meaningless.\n The SQL Server log-LSN clock is server-wide and keeps ticking across\n the reset, so per-record LSN columns and STATE offsets still differ\n between runs; gate the verdict on record-level shape rather than on\n those values.\n- **`--reset=backend`** — same as `--reset=fixture` but also recreates\n the backend container between the two sweeps, resetting the LSN clock\n at ~15s of extra startup cost. Use only when the reproduction depends\n on matching LSN sequences across runs.\n\n`--reset` has no effect without `--control-version` and produces a\nwarning; the flag governs the reset between two image runs, and there\nis no second run in single-version mode.\n\n## Asserting on output\n\nInline assertions in driver scripts. Suggested helpers:\n\n- Exit code: check `$?` immediately after the call.\n- AirbyteMessage shape: `jq -e 'select(.type == \"RECORD\")' $REPRO_OUT//read/stdout.txt`.\n- Connector-side log shape: `grep -c '' $REPRO_OUT//read/stderr.txt`.\n- Connection status: `jq -e 'select(.type == \"CONNECTION_STATUS\" and .connectionStatus.status == \"SUCCEEDED\")' $REPRO_OUT//check/stdout.txt`.\n\nIn comparison mode with `--reset=none` each command's directory holds\n`target/` and `control/` subdirectories (created by `airbyte-ops`).\nWith `--reset=fixture|backend` the split lives one level up:\n`$REPRO_OUT//{control,target}//`.\n\nPass `-- --enable-debug-logs=True` to `run.sh` to set `LOG_LEVEL=DEBUG`\non the connector container. That surfaces the\nDebezium \"Adding table … to the list of capture schema tables\" lines\nthat some assertions rely on.\n\n## Common gotchas\n\n- _No SQL Server Agent_ → CDC capture / cleanup jobs never run, and\n Debezium silently produces no records. `start-backend.sh` sets\n `MSSQL_AGENT_ENABLED=true` for this reason.\n- _Race on first `sqlcmd`_ → SQL Server 2022 takes around 15 seconds to\n accept connections cold. `start-backend.sh` polls until `SELECT 1`\n succeeds before returning.\n- _Configured-stream fields the validator rejects as null_ → the\n bulk-CDK schema validator requires `is_file_based`, `cursor_field`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files` on every\n `ConfiguredAirbyteStream`; omitting them fails with\n `Null value is not allowed. (code: 1021)`. This is not 4.3.x-only —\n verified on `4.4.12` and `5.0.0` too. `discover` does not emit\n `is_file_based`, so the catalog `run.sh` derives fills it in, and the\n catalog fixtures shipped with this and dependent skills populate all of\n them.\n- _A failed `check` does not fail a single-version run_ → the CDK exits\n 0 even when it emits `CONNECTION_STATUS` with `status: FAILED`, so the\n `check` step passes. Assert on the status message yourself (see\n [Asserting on output](#asserting-on-output)) rather than on the exit\n code when the repro hinges on `check`. Comparison mode does surface\n it, because it reads the report's verdict rather than the exit code.\n\n## Tear-down\n\n`run.sh` removes the backend container in an `EXIT` trap, so a run leaves\nnothing behind unless you passed `--keep-backend`. After a\n`--keep-backend` run, or to also discard the collected artifacts:\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\" # idempotent; docker rm -f no-ops when absent\nrm -rf \"${REPRO_OUT:-/tmp/source-mssql-repro}\"\n```\n","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/SKILL.md":"---\nname: source-mssql-e2e-cdc-tests\ndescription: Reproduce CDC bugs against source-mssql by enabling CDC on the local SQL Server backend, applying per-bug SQL fixtures, and running the connector via airbyte-ops. Composes on top of source-mssql-e2e-tests. Worked examples for airbytehq/oncall#11451, #12094, and #12162 ship with the skill.\n---\n\n# source-mssql-e2e-cdc-tests\n\nLocal CDC bug-reproduction harness for `source-mssql`. Builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe MSSQL engine skill\n([`source-mssql-e2e-tests`](../source-mssql-e2e-tests/SKILL.md)):\ngeneric skill stands up the SQL Server backend and runs the connector;\nthis skill adds CDC enable, CDC-aware config / catalog templates, and\nper-bug **case scripts** under `cases/` that call `run.sh` with the\nappropriate fixtures and `--expect-*` assertions.\n\n## When to use this skill\n\n- Reproducing a CDC-mode bug against `source-mssql` locally.\n- Verifying a fix by re-running an existing case with\n `VERSION=dev ./cases/.sh` (after `:dockerBuildx`).\n- Verifying that a fix which _loosens_ CDC offset validation still\n rejects genuinely invalid state — `cases/11451.sh` is the\n invalid-state case for LSN availability; see\n [Invalid-state case](#invalid-state-case-for-lsn-availability-fixes).\n- Authoring a new repro for a customer-reported CDC bug. Drop a SQL\n fixture under `fixtures/sql/` and a `cases/.sh` script\n that invokes `run.sh` with the right fixtures and `--expect-*`\n assertions.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, a clone of `airbytehq/airbyte`.\n- The backend container `source-mssql-db-backend` started by\n `../source-mssql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials.\n\n## Layout\n\n```\nsource-mssql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── 11451.sh # airbytehq/oncall#11451 — LSN-range regression in 4.3.4+ (multi-phase; invalid-state case)\n│ ├── 12094.sh # airbytehq/oncall#12094 — schema-history bloat\n│ └── 12162.sh # airbytehq/oncall#12162 — whitespace in stream name\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json\n ├── catalogs/\n │ ├── users-cdc.json\n │ └── order-items-cdc.json\n └── sql/\n ├── 00-init-cdc.sql\n ├── repro-11451-lsn-cleanup.sql\n ├── repro-12094-schema-history.sql\n └── repro-12162-spaces-in-name.sql\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit walks Airbyte STATE messages, which is protocol-level and not\nCDC-specific.\n\n## Conventions\n\n- Cases keep the backend up across invocations (`--keep-backend`).\n Start the backend once at the top of a session, run any subset of\n cases, tear down when done — see the Usage section below.\n- Fixtures are re-applied at the start of each `run.sh` invocation, so\n they must be idempotent (`00-init-cdc.sql` uses `IF DB_ID(...) IS NULL`\n guards, per-bug fixtures use similar shape). Re-applying a fixture\n during a multi-phase case does not reset the CDC state that the\n previous phase established.\n- Configured catalogs populate the bulk-CDK-required fields\n (`is_file_based`, `generation_id`, `minimum_generation_id`,\n `sync_id`, `destination_object_name`, `include_files`) so the same\n fixtures drive repros across `source-mssql:4.3.x` and `4.4.x`.\n- Cases default `VERSION=4.4.2` (except `11451.sh` which splits into\n `BASELINE_VERSION`/`TARGET_VERSION`). Override with\n `VERSION=4.3.4 ./cases/12162.sh` to test against an earlier version,\n or `VERSION=dev` after a local\n `./gradlew :airbyte-integrations:connectors:source-mssql:dockerBuildx`\n to test a fix.\n- **Assertions** are declarative via `run.sh`'s expectation flags:\n `--expect-test`, `--expect-match=[:]:[:N]`,\n `--forbid-match`, `--min-records`, `--min-states`. The runner\n enforces them and exits non-zero on any failure. The `:`\n prefix defaults to `read` when omitted; set it explicitly\n (`check:stderr:…`, `discover:stderr:…`) for check-time or\n discover-time signatures.\n- Multi-phase cases (`11451.sh`, and any future read → mutate →\n read-with-state repros) capture Airbyte STATE messages between reads\n via the shared\n [`extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\n (`uv`-PEP-723 standalone; pass `` as its argument or pipe\n stdin) and feed the file back into the second read with\n `run.sh --state=PATH`.\n Use `--step-name=/` to give each phase its own artifact\n subtree.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mssql-repro\n\n# 1. Bring up the backend (once per session). Cases pass --keep-backend,\n# so they don't tear it down between runs.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# 2. Run any case. Each case's --expect-* flags gate its own exit code\n# (non-zero on any expectation failure), so bash's `set -e` will fail\n# the sequence on the first failing case if you chain them.\n\"$SKILL/cases/12162.sh\"\n\"$SKILL/cases/12094.sh\"\n\"$SKILL/cases/11451.sh\"\n\n# 3. (After fix) verify by retargeting `dev` or a fixed version.\nVERSION=dev \"$SKILL/cases/12162.sh\"\n\n# 4. Tear down.\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\nrm -rf \"$REPRO_OUT\"\n```\n\nEach case is a shell script that composes `run.sh` invocations:\n\n1. Applies its SQL fixtures (via `--fixture=…`; `run.sh` calls\n `apply-sql.sh` internally).\n2. Renders the config against the running backend (via\n `--config-template=…`).\n3. Runs the target protocol command with any `--catalog` / `--state`.\n4. Gates its exit code on `--expect-*` flags — the runner enforces\n them and exits non-zero on any failure, so the case script has no\n inline assertion logic to maintain.\n\nMulti-phase cases (`11451.sh`) call `run.sh` more than once, with\n`extract-state.py` and `apply-sql.sh` between invocations. Each phase\nuses `--step-name=/` to keep its artifacts separate under\n`$REPRO_OUT///`.\n\n## Worked examples\n\n### airbytehq/oncall#12162 — whitespace in stream name\n\n`cases/12162.sh`. Creates `dbo.[Order Items]` (note the space), enables\nCDC on it, and runs `read` against it. Case asserts\n`--expect-test=fail`, `--expect-match=stderr:io.debezium.DebeziumException`,\nand `--expect-match=stderr:message.key.columns` — i.e. the read exits\nnon-zero and Debezium's `stderr.txt` contains its\n`Connector configuration is not valid. The 'message.key.columns' value\nis invalid` rejection.\n\nRoot cause: `MsSqlServerDebeziumOperations.buildMessageKeyColumns()`\njoins `schema.table:pkcol` strings without filtering or escaping\nidentifiers that contain whitespace or `:`, and Debezium rejects them\nat engine startup. Fix: pre-filter such streams (Debezium falls back\nto the table's native PK from system tables).\n\n### airbytehq/oncall#12094 — schema-history bloat\n\n`cases/12094.sh`. Creates 30 noise tables in `dbo` (CDC not enabled on\nthem), runs the baseline `read` against `dbo.users`, and asserts\n`--expect-match='stdout:Adding table CdcTest\\..* to the list of capture schema tables:30'`\n(the trailing `:30` is the count threshold; the fixture creates 30\nnoise tables). Adjust with `MIN_LOADED=`. Expected outcome: at\nleast 30 \"Adding table\" lines even though the configured catalog has\na single stream.\n\nRoot cause: `withSchemaHistory()` in the bulk-CDK Debezium properties\nsets `schema.history.internal.store.only.captured.databases.ddl=true`\nbut not `…captured.tables.ddl=true`. Same harness reproduces equivalent\nbehavior on `source-mysql` and `source-postgres` because the property\nlives in the bulk-CDK `extract-cdc` toolkit.\n\n### airbytehq/oncall#11451 — LSN-range regression in 4.3.4+\n\n`cases/11451.sh`. Multi-phase:\n\n1. Baseline `read` on clean CdcTest (`BASELINE_VERSION`, default `4.4.2`).\n Case asserts `--expect-test=pass --min-states=1` so a missing STATE\n fails the case immediately rather than later at replay.\n2. `extract-state.py` on the baseline stdout, then\n `apply-sql.sh repro-11451-lsn-cleanup.sql` runs\n `sys.sp_cdc_cleanup_change_table`\n with a `low_water_mark` past the saved LSN — advancing\n `fn_cdc_get_min_lsn('dbo_users')` past the baseline offset.\n3. Replay `read` on `TARGET_VERSION` (default `4.3.4`) with the stale\n state. Case asserts `--expect-test=fail`,\n `--expect-match=stderr:Saved offset no longer present`, and\n `--expect-match='stderr:is no longer available in SQL Server transaction logs'`.\n\nRoot cause: pre-`4.3.4` the LSN-range query computed `min` via\n`sys.fn_cdc_get_min_lsn('')`, which always returns\n`0x00000000000000000000`. `4.3.4` swapped to\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance)) FROM cdc.change_tables`,\nwhich returns the actual per-instance min — a value that can advance\npast a saved offset on geo-replicas with aggressive cleanup. The\nsaved-offset-rejection guard then fires even though the data is\nstill present. Investigation lives at\n[`airbytehq/oncall#11451`](https://github.com/airbytehq/oncall/issues/11451).\n\n#### Invalid-state case for LSN-availability fixes\n\nThis case's Phase 3 is a _genuinely_ expired saved offset: Phase 2\nadvances `fn_cdc_get_min_lsn` past the captured LSN, so rejecting the\nreplay is the correct behavior, not the bug. That makes it the\ninvalid-state counterpart to any repro built around a saved LSN that is\nstill valid — for example an unauthorized capture instance returning\n`0x0` and poisoning\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance))`.\n\nSo when a change relaxes what the offset-availability check aggregates\nor tolerates, run this case against the image built from that change:\n\n```bash\nTARGET_VERSION=dev \"$SKILL/cases/11451.sh\"\n```\n\nIt must still fail with the same two `stderr` signatures the case\nasserts on. A pass here means the guard stopped firing on state it is\nsupposed to reject, which turns an actionable \"reset the connection\"\nerror into a silently incomplete sync — the symptom-gone evidence from\nthe valid-LSN repro cannot distinguish that from a correct fix.\n\n## Authoring a new repro\n\n1. Drop a SQL fixture in `fixtures/sql/repro--.sql`.\n Make it idempotent (`IF NOT EXISTS` guards for `CREATE`,\n `IF EXISTS` guards for `DROP`) — cases re-apply fixtures on every\n `run.sh` invocation.\n2. (Optional) Drop a CDC-aware catalog in `fixtures/catalogs/`.\n3. Drop a case script in `cases/.sh` modeled on the\n existing ones. `set -euo pipefail`; default `VERSION=4.4.2` with a\n `${VERSION:-…}` override; invoke `run.sh` with the right\n `--command`, `--fixture=`, `--config-template=`, `--catalog=`, and\n `--expect-*` flags. Pass `--keep-backend` so the shared-session\n Usage flow works. For check-time or discover-time signatures, use\n `--expect-match=check:stderr:…` / `--expect-match=discover:stderr:…`\n (the `:` prefix defaults to `read` when omitted).\n4. Verify locally:\n ```bash\n \"$GENERIC/scripts/start-backend.sh\"\n \"$SKILL/cases/.sh\"\n BACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\n ```\n5. Note the worked example in this `SKILL.md`'s \"Worked examples\"\n section with: customer-symptom one-liner, `--expect-*` assertions\n the case gates on, root-cause one-liner.\n\n### Multi-phase repros\n\nWhen the reproduction is single-phase (one `run.sh` invocation is\nenough to trigger the bug), follow the recipe above. Reach for a\nmulti-phase shape only when the bug needs an intermediate mutation\nthat depends on state from a first read — the canonical example is\nread → extract STATE → mutate the server based on that STATE → replay\nwith the stale STATE. Model on [`cases/11451.sh`](cases/11451.sh) —\nthe three primitives it composes are:\n\n- **`--step-name=/`** — per-phase artifact dirs under\n `$REPRO_OUT//`. `cases/11451.sh` uses `11451/baseline` and\n `11451/stale`; a new case would use its own bug number and phase\n names.\n- **`extract-state.py`** (implemented in `db-harness-lib`) — reads a phase's\n `read/stdout.txt` and emits a JSON array of AirbyteStateMessage\n objects, which is what `run.sh --state=PATH` expects.\n- **`--skip-fixtures`** on the second/later `run.sh` invocation —\n opts out of the initial-fixture reapply so the intermediate state\n the previous phase established survives. Otherwise `run.sh`'s\n default fixture-application would re-run `00-init-cdc.sql`, which\n drops and recreates `CdcTest` and wipes the mutation. Rejects\n `--fixture=` in the same invocation (either apply fixtures or skip\n them).\n\nThe between-phase mutation itself is a plain `apply-sql.sh` call\nagainst a fixture in `fixtures/sql/`. Nothing about the mutation is\nspecial-cased in `run.sh` — the case script drives the sequencing.\n","airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/SKILL.md":"---\nname: source-bigquery-e2e-tests\ndescription: Run the Airbyte protocol smoke sweep for airbyte/source-bigquery: against the shared BigQuery integration project inside a per-run PROVEFIX_ dataset (B3 remote-backend mode of db-harness-lib), with a dry-run orphan sweep.\n---\n\n# source-bigquery-e2e-tests\n\nThis harness runs the source-bigquery protocol regression sweep against an isolated dataset in a real BigQuery project. Orchestration comes from `airbyte-integrations/db-harness-lib`; the engine scripts own the dataset, fixtures, and credentials.\n\n## Prerequisites\n\n- `gcloud` with ambient access to Secret Manager in `GSM_PROJECT`.\n- `bq` and `docker`.\n- `airbyte-ops` (or `uvx airbyte-internal-ops`).\n- A BigQuery service account with dataset create/delete, query, and source-read permissions in `BIGQUERY_PROJECT`.\n\nCloud Storage Read API coverage requires `airbyte-ops cloud connector regression-test --extra-env`, added in [airbyte-ops-mcp#349](https://github.com/airbytehq/airbyte-ops-mcp/pull/349). The shim defaults `BIGQUERY_AIRBYTE_EDITION` to `CLOUD` and adds `AIRBYTE_EDITION=CLOUD` when supported. Set `BIGQUERY_AIRBYTE_EDITION=''` to disable the check and injection. Older `airbyte-ops` versions warn and continue using the query API.\n\n## Run\n\nFrom this connector directory:\n\n```bash\npoe e2e-local --test-version=1.0.1\n```\n\nThe Poe task forwards the shared harness flags; use `poe e2e-local --help` for the full list. Common options include `--command=check`, `--skip-read`, `--step-name=NAME`, `--keep-backend`, and `--reset=fixture`. Environment overrides include `REPRO_OUT`, `GSM_PROJECT`, `BIGQUERY_PROJECT`, `BIGQUERY_LOCATION`, `BIGQUERY_SECRET_NAME`, `BIGQUERY_CONFIG_FILE`, `AIRBYTE_OPS`, and `BIGQUERY_AIRBYTE_EDITION`.\n\n## Dataset lifecycle\n\nEach run creates `PROVEFIX_` in `BIGQUERY_PROJECT` and records ownership in `$REPRO_OUT/.provefix-owned-PROVEFIX_`. An existing dataset is reused only with that marker; otherwise the harness refuses to touch it and points to `sweep-orphans.sh`. The default teardown drops the owned dataset and verifies it is absent. `--reset=fixture` drops and recreates the run dataset between comparison sweeps. `sweep-orphans.sh --dry-run` lists old `PROVEFIX_*` datasets without deleting them; `--older-than-hours=N` defaults to 4.\n\n`gcloud`/`bq` authentication uses `$REPRO_OUT/.gcloud`, separate from the ambient credentials used to fetch the GSM secret. The fetched service-account key and isolated auth directory are removed at teardown.\n\n## Expected smoke result\n\nBoth runs pass CHECK, DISCOVER `smoke_table` and `smoke_view`, and READ three records from each stream. The installed CLI warns when it lacks `--extra-env`, so that run exercises only the query API. With `BIGQUERY_AIRBYTE_EDITION=CLOUD` and the #349 CLI, connector logs show `smoke_table` opening a Storage Read API session and `smoke_view` falling back to the query API because views are unsupported by the Storage Read API.\n\nThere is no bad-vs-fixed proof case yet: the 1.0.1 views fallback requires a service account without BigQuery Read Session User, and none exists in GSM.\n","airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/SKILL.md":"---\nname: source-mongodb-v2-e2e-tests\ndescription: Stand up a local MongoDB 7.0 single-node replica set, apply mongosh JavaScript fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mongodb-v2: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any non-CDC source-mongodb-v2 test or prove-fix comparison.\n---\n\n# source-mongodb-v2-e2e-tests\n\nLocal end-to-end test harness for `source-mongodb-v2`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MongoDB backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MongoDB 7.0 container named\n`source-mongodb-v2-db-backend` configured as a single-node replica set\n(`rs0`), lets you apply arbitrary `mongosh` JavaScript fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mongodb-v2:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: non-CDC (full-refresh / initial-snapshot) behavior. Change-stream\nresume-token replay and CDC comparison flows are not covered by this skill.\n\n## When to use this skill\n\n- Reproducing a `source-mongodb-v2` bug locally without Atlas credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-mongodb-v2:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM, Atlas, or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mongodb-v2-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mongo:7.0 --replSet rs0; rs.initiate()\n│ ├── apply-sql.sh # docker exec mongosh with a .js fixture on stdin\n│ ├── reset-databases.sh # drop every non-system database\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # SELF_MANAGED_REPLICA_SET config; host placeholder\n └── js/\n └── 00-init-base.js # test_db.sample with three documents\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but MongoDB\nfixtures are `mongosh` scripts, not SQL. Each fixture runs with `test_db`\n(`BACKEND_DB`) as the current `db`; use `db.getSiblingDB(...)` for others.\n\n## Conventions\n\n- Container name: `source-mongodb-v2-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The backend runs without `--auth`; the config's\n `database_config` omits `username`/`password`. Add them to a copied\n template if you need to reproduce an auth-specific issue.\n- Replica set: `rs0` (`BACKEND_REPLSET`). The connector requires a replica\n set even for non-CDC reads. `start-backend.sh` registers the single member\n under the container's bridge IP so the connector container, which performs\n replica-set discovery from the seed list, can reach the advertised host.\n Do not change the member host to `localhost`: the connector would then\n fail to connect.\n- Initial database: `test_db`. Override with `BACKEND_DB`; keep\n `database_config.databases` in the config template in sync.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mongodb-v2-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `database_config.connection_string` to\n `mongodb://:27017/?replicaSet=rs0` at runtime.\n- The backend image is pinned to `mongo:7.0`. Override with `BACKEND_IMAGE=…`\n for a version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mongodb-v2\n\n# Single version, full sweep.\npoe e2e-local --test-version=2.0.7\n\n# Target vs. control comparison.\npoe e2e-local --test-version=dev --control-version=2.0.6 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=2.0.7\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. Under CI the table is also\nappended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **Schema enforcement.** The base template sets\n `database_config.schema_enforced: true`, so `discover` samples documents\n and emits typed fields. Set it to `false` in a copied template to\n reproduce schemaless (`data` blob) behavior.\n- **`_id` types.** Fixtures use integer `_id`s for readability; use\n `ObjectId()` in a fixture when the bug depends on ObjectId serialization.\n- **Schema inference needs documents.** An empty collection created with\n `db.createCollection()` is still listed by `discover`, but with\n `schema_enforced: true` its schema has no sampled fields beyond `_id`.\n Insert at least one document in the fixture for every field you expect\n in the catalog.\n- **Reset drops databases, not the replica set.** `--reset=fixture` removes\n all non-system databases and reapplies fixtures; the oplog and resume\n tokens survive. This only matters once change-stream (CDC) coverage\n lands (HYD-147 phase 2); non-CDC sweeps do not read the oplog. Use\n `--reset=backend` when a run must start from a fresh oplog.\n- **Databases are created on first write.** MongoDB has no `CREATE\n DATABASE`; `test_db` exists only once a fixture inserts into it, so a\n fixture that only calls `db.createCollection(...)` or writes to a\n `getSiblingDB(...)` database leaves `test_db` absent and `discover` empty.\n `00-init-base.js` inserts into `test_db`; keep that in any fixture set you\n pass with `--fixture`.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections, Atlas clusters\n holding customer data, or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mongodb-v2-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/SKILL.md":"---\nname: source-dynamodb-e2e-tests\ndescription: Stand up a local DynamoDB Local backend, apply JSON fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-dynamodb: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for a cursor-incremental source-dynamodb test or prove-fix comparison.\n---\n\n# source-dynamodb-e2e-tests\n\nLocal end-to-end test harness for `source-dynamodb`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the DynamoDB Local backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a DynamoDB Local container named\n`source-dynamodb-db-backend`, applies JSON fixtures with the AWS CLI, and runs\nAirbyte protocol commands against any `airbyte/source-dynamodb:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: no CDC mode exists; incremental reads use a user-defined cursor\nattribute. Assume-role / IAM / STS behavior is not covered: DynamoDB Local\ndoes not provide STS, and the connector's `endpoint` only overrides the\nDynamoDB client, so a published connector image cannot send AssumeRole\nrequests to this emulator. State this limitation in the evidence plan rather\nthan claiming coverage.\n\n## When to use this skill\n\n- Reproducing a `source-dynamodb` bug locally without AWS credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-dynamodb:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need AWS, IAM, or Cloud credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-dynamodb-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── aws-cli.sh # AWS CLI container and backend bridge-IP helpers\n│ ├── start-backend.sh # docker run DynamoDB Local with -sharedDb\n│ ├── apply-sql.sh # delete/create tables and put fixture items\n│ ├── reset-databases.sh # delete every table\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # user credentials, region, endpoint placeholder\n └── json/\n ├── 00-init-base.json # sample with a numeric cursor and three items\n ├── attribute-types.json\n └── empty-table.json\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but DynamoDB\nfixtures are JSON documents, not SQL. Each fixture has a `tables` array. Each\ntable entry contains a `create` object accepted by `CreateTable\n--cli-input-json` and an `items` array in DynamoDB attribute-value JSON:\n\n```json\n{\n \"tables\": [\n {\n \"create\": {\n \"TableName\": \"sample\",\n \"KeySchema\": [{\"AttributeName\": \"id\", \"KeyType\": \"HASH\"}],\n \"AttributeDefinitions\": [{\"AttributeName\": \"id\", \"AttributeType\": \"S\"}],\n \"BillingMode\": \"PAY_PER_REQUEST\"\n },\n \"items\": [{\"id\": {\"S\": \"e1\"}, \"seq\": {\"N\": \"1\"}}]\n }\n ]\n}\n```\n\n## Conventions\n\n- Container name: `source-dynamodb-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The config uses dummy `User` credentials and\n `us-east-1`; DynamoDB Local does not authenticate them.\n- `-sharedDb` is required. Without it, DynamoDB Local namespaces tables by\n access key and region, so tables created by the fixture client can be\n invisible to the connector client.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-dynamodb-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `.endpoint` to `http://:8000` at runtime. The AWS CLI\n helper resolves the same bridge IP for its endpoint.\n- DynamoDB Local is pinned to `amazon/dynamodb-local:3.3.2`. Override with\n `BACKEND_IMAGE=…` for a version-specific reproduction. The AWS CLI helper\n image is pinned to `amazon/aws-cli:2.38.2`; override with\n `AWS_CLI_IMAGE=…`.\n- `attribute-types.json` exercises DynamoDB attribute-value families and a\n composite-key table. The connector's source-defined primary key is the\n partition key only; its sort key is not part of that primary key.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-dynamodb\n\n# Single version, full sweep.\npoe e2e-local --test-version=0.4.1\n\n# Target vs. control comparison.\npoe e2e-local --test-version=0.4.1 --control-version=0.3.11 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=0.4.1\n```\n\nThe sweep runs every command against the backend and reports each result\nrather than stopping at the first failure, then prints a summary table. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **DynamoDB key attributes.** `AttributeDefinitions` contains only the\n table's key attributes; non-key attributes such as `seq` and `label` appear\n in fixture items and are discovered by sampling them.\n- **Sampled attributes.** Discovery samples table items and unions the\n attributes it sees. Keep at least one item with each field you want in the\n catalog. A field missing from an item is a useful missing-attribute case,\n but it may not be emitted if it is absent from every sampled item.\n- **Configured-catalog nulls.** DynamoDB discover omits `namespace`,\n `source_defined_cursor`, and `is_resumable`, and 0.4.0 accepts them absent;\n it rejects `namespace: null`. Keep them omitted through an Airbyte Ops build\n that serializes configured catalogs with `exclude_none=True`; no shared\n catalog-generator defaults are needed.\n- **Empty tables.** The connector discovers key attributes from\n `DescribeTable` when there are no items to sample, so an empty table remains\n in the catalog with its key fields.\n- **Comparison catalogs.** Use `--reset=fixture` for comparisons where\n discovery output changes between image tags. In particular, for the\n integer-cursor regression below, the old tag discovers `seq` as\n `[\"null\",\"integer\"]` while the fixed tag discovers it as `number` with\n `airbyte_type: integer`. A paired `--reset=none` run can reuse the target's\n discovered catalog for the control read and mask the control bug, because\n the crash is driven by the cursor's catalog JSON-schema type. Fixture reset\n runs separate sweeps, so each image derives its own catalog.\n- **No role coverage.** Do not use DynamoDB Local to claim assume-role, IAM,\n or STS coverage. There is no STS emulator here, and `endpoint` only\n overrides the DynamoDB client endpoint.\n- **Do not use real AWS.** This harness is for local testing only and must\n never be used against customer AWS accounts, tables containing customer\n data, or Airbyte Cloud.\n\n## Worked example: integer cursor, airbyte#71171\n\nThe historical integer-cursor crash can be proven with the base fixture. The\ncontrol image fails with\n`IllegalArgumentException: No enum constant\nio.airbyte.protocol.models.JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER`\nand emits no records; the fixed image reads all three records and emits\n`STATE` with `cursor: \"3\"`.\n\nUse `--reset=fixture` so the 0.3.11 control and 0.4.0 target each run their\nown `discover` and `read`. The versions discover different cursor schemas,\nand the crash depends on that schema in the configured catalog.\nThis command also requires an Airbyte Ops build with the configured-catalog\n`exclude_none=True` fix: DynamoDB discovery omits `namespace`, and older Ops\nbuilds serialize that omission as `namespace: null`, which 0.4.0 rejects.\n\n```bash\nREPRO_OUT=\"$HOME/dynamodb-hyd310-evidence/patched-ops-1792531195\" \\\n poe e2e-local \\\n --step-name=integer-cursor-71171 \\\n --control-version=0.3.11 \\\n --test-version=0.4.0 \\\n --reset=fixture \\\n --sync-mode=incremental \\\n --cursor-field=seq \\\n --streams=sample \\\n --expect-control=fail \\\n --expect-test=pass \\\n --min-records=3 \\\n --min-states=1 \\\n --forbid-match=read:any:JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER\n```\n\nVerified on 2026-10-20 with the Airbyte Ops null-namespace fix:\n\n| Command | Control (`0.3.11`) | Target (`0.4.0`) |\n|---------|---------|---------|\n| `SPEC` | PASS | PASS |\n| `CHECK` | PASS | PASS |\n| `DISCOVER` | PASS | PASS |\n| `READ` | FAIL — integer-enum crash, 0 records, 0 states | PASS — 3 records, 1 state (`cursor: \"3\"`) |\n\nThe paired command exited 0: the control reproduced the historical failure,\nwhile the target passed and emitted the expected records and state.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-dynamodb-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/SKILL.md":"---\nname: source-postgres-e2e-cdc-tests\ndescription: Reproduce PostgreSQL logical-decoding CDC behavior against a local PostgreSQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-postgres-e2e-tests and db-harness-lib.\n---\n\n# source-postgres-e2e-cdc-tests\n\nLocal CDC test harness for `source-postgres`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic PostgreSQL skill\n([`source-postgres-e2e-tests`](../source-postgres-e2e-tests/SKILL.md)). The\ngeneric skill starts a PostgreSQL 16 backend and runs the connector; this skill\nsupplies the logical-decoding CDC config, CDC catalog, SQL fixtures, and case\nscripts.\n\nPostgreSQL CDC uses logical decoding from a publication and replication slot.\nThe generic PostgreSQL backend starts with the WAL and replication settings\nrequired by this skill, so this skill does not start a second backend or add\nper-run server configuration.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-postgres` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new PostgreSQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-postgres-db-backend`, started by\n `../source-postgres-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-postgres-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with PostgreSQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # publication, slot, table, and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\nThe fixture uses the generic backend database `test_db`, because PostgreSQL\ncannot drop the database to which `apply-sql.sh` is connected. It resets the\n`users` table, publication, and logical replication slot in that database.\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic PostgreSQL backend starts with `wal_level=logical`,\n `max_replication_slots=10`, and `max_wal_senders=10`. Do not add per-run\n server configuration or start another backend in this skill.\n- `00-init-cdc.sql` drops and recreates the `users` table, publication, and\n `airbyte_slot` in `test_db` on every application. Re-applying it resets the\n fixture, which is why `cases/incremental-replay.sh` passes `--skip-fixtures`\n on its second phase.\n- PostgreSQL 3.8.5 discovers `_ab_cdc_lsn` as the source-defined cursor and\n does not include the newer generated catalog fields in discover output.\n The configured catalog includes `is_file_based`, generation metadata, and\n destination metadata because the pinned image requires those fields when\n initializing `read`.\n- Cases default to `VERSION=3.8.5`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-postgres-repro\n\n# Start the logical-decoding-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-postgres-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the logical-decoding-ready backend, CDC config, catalog metadata, and\ninitial load work together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved LSN\nresumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No PostgreSQL CDC fixtures\nor worked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep CDC setup in the\n configured database and do not add per-run server configuration.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`. Use the source-defined cursor\n emitted by `discover` for the pinned connector version.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.8.5}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses `psql` on stdin, so fixtures may contain\nmultiple statements. Logical replication settings are established when the\ngeneric backend starts; no additional CDC setup is needed in a case beyond the\npublication and replication slot in the SQL fixture.\n","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/SKILL.md":"---\nname: source-mysql-e2e-cdc-tests\ndescription: Reproduce MySQL binlog CDC behavior against a local MySQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-mysql-e2e-tests and db-harness-lib.\n---\n\n# source-mysql-e2e-cdc-tests\n\nLocal CDC test harness for `source-mysql`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic MySQL skill\n([`source-mysql-e2e-tests`](../source-mysql-e2e-tests/SKILL.md)). The generic\nskill starts a MySQL 8.0 backend and runs the connector; this skill supplies\nthe binlog CDC config, CDC catalog, SQL fixtures, and case scripts.\n\nMySQL CDC is binlog-based and server-wide. The generic backend already starts\nMySQL with ROW binlogging, full row images, and GTIDs, so this skill does not\nenable CDC per table or start a second backend.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-mysql` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new MySQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-mysql-db-backend`, started by\n `../source-mysql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-mysql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with MySQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # dedicated cdc_test database and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic MySQL backend starts with `--log-bin`, `--binlog-format=ROW`,\n `--binlog-row-image=FULL`, `--gtid-mode=ON`, and\n `--enforce-gtid-consistency=ON`. Do not add per-table CDC enable steps or\n engine-specific backend startup to this skill.\n- `00-init-cdc.sql` drops and recreates the dedicated `cdc_test` database on\n every application. Re-applying it wipes existing rows and any accumulated\n binlog-relative table state, which is why `cases/incremental-replay.sh`\n passes `--skip-fixtures` on its second phase.\n- CDC catalogs include the bulk-CDK fields (`is_file_based`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files`) and use\n `[\"_ab_cdc_cursor\"]` as the cursor field.\n- Cases default to `VERSION=3.53.4`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mysql-repro\n\n# Start the binlog-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-mysql-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the binlog-ready backend, CDC config, catalog metadata, and initial load\nwork together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved binlog\nposition resumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No MySQL CDC fixtures or\nworked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep any CDC database\n setup dedicated to the fixture and do not add per-table CDC enable steps.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.53.4}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses the plain MySQL client on stdin, so fixtures\nmay contain multiple statements. The MySQL server's binlog and GTID settings\nare established when the generic backend starts; no additional CDC setup is\nneeded in a case.\n","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/SKILL.md":"---\nname: source-mysql-e2e-tests\ndescription: Stand up a local MySQL 8.0 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mysql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mysql test or prove-fix comparison.\n---\n\n# source-mysql-e2e-tests\n\nLocal end-to-end test harness for `source-mysql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MySQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MySQL 8.0 container named\n`source-mysql-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mysql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a `source-mysql` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mysql:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mysql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mysql:8.0 with ROW binlog + GTIDs\n│ ├── apply-sql.sh # docker exec mysql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mysql-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Root password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mysql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the MySQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `mysql:8.0`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n- The backend starts with ROW-format binary logging, a full row image,\n and GTIDs. This leaves the container ready for a CDC driver without\n restarting it.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mysql\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.53.3\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.53.2\n\n# One command only.\npoe e2e-local --command=read --test-version=3.53.3\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose `discover`\nfailed earlier is reported `SKIPPED`, not `ERROR`. Under CI the table is\nalso appended to `$GITHUB_STEP_SUMMARY`. A run limited to one command\ninstead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`. Use this for CDC\n comparisons so the target does not read a warm capture instance.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the MySQL\n container between sweeps. Use this only when matching binlog or GTID\n sequences matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC needs ROW binlogging.** `start-backend.sh` sets\n `--binlog-format=ROW`, `--binlog-row-image=FULL`, and a server ID.\n The connecting user must have replication privileges; the root user\n created by the fixture has them.\n- **MySQL config shapes are specific.** `ssl_mode` uses `preferred`,\n `required`, `verify_ca`, or `verify_identity`; the base config uses\n `preferred` for the local plaintext backend. The\n `replication_method.method` value is `STANDARD` or `CDC`.\n- **`tinyint(1)` mapping.** The connector maps `TINYINT(1)` to boolean by\n default. Set `treat_tinyint1_as_integer: true` in a config when\n reproducing a typing issue.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mysql-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/SKILL.md":"---\nname: source-postgres-e2e-tests\ndescription: Stand up a local PostgreSQL 16 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-postgres: for ad-hoc end-to-end testing.\n---\n\n# source-postgres-e2e-tests\n\nLocal end-to-end test harness for `source-postgres`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the PostgreSQL backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a PostgreSQL 16 container named\n`source-postgres-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-postgres:` image via\n`airbyte-ops cloud connector regression-test`.\n\nThis skill supports standard (non-CDC) local sweeps. CDC fixtures and\nreplication setup are not yet stood up in this skill.\n\n## When to use this skill\n\n- Reproducing a `source-postgres` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-postgres:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-postgres-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run postgres:16 with logical replication\n│ ├── apply-sql.sh # docker exec psql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-postgres-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Postgres password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-postgres-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the PostgreSQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `postgres:16`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-postgres\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.8.5\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.8.4\n\n# One command only.\npoe e2e-local --command=read --test-version=3.8.5\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose\n`discover` failed earlier is reported `SKIPPED`, not `ERROR`. Under CI the\ntable is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to one\ncommand instead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the\n PostgreSQL container between sweeps. Use this only when matching database\n initialization state matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC is not included in this skill.** The lifecycle starts PostgreSQL 16\n with logical replication enabled so the composed\n [`source-postgres-e2e-cdc-tests`](../source-postgres-e2e-cdc-tests/SKILL.md)\n skill can use the same backend. CDC fixtures and case scripts live in that\n skill.\n- **PostgreSQL config shapes are specific.** `ssl_mode.mode` uses\n `prefer` for the local plaintext backend. The\n `replication_method.method` value for a standard sync is `Standard`.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-postgres-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/SKILL.md":"---\nname: source-snowflake-e2e-tests\ndescription: Run the Airbyte protocol sweep for airbyte/source-snowflake: against the shared Snowflake integration account inside a per-run PROVEFIX_ schema (B3 remote-backend mode of db-harness-lib); prove-fix comparisons, cursor canary, orphan sweep.\n---\n\n# source-snowflake-e2e-tests\n\nRemote end-to-end harness for `source-snowflake`. The shared\n`db-harness-lib` orchestration runs protocol commands against a per-run schema\nin the shared Snowflake integration account.\n\n## Prerequisites\n\n- Docker, [`uv`](https://docs.astral.sh/uv/), and `jq`.\n- `gcloud` authenticated with read access to the GSM secret, or an existing\n `SNOWFLAKE_CONFIG_FILE`.\n- A clone of `airbytehq/airbyte`.\n\n## Layout\n\n```\nsource-snowflake-e2e-tests/\n├── SKILL.md\n├── ../../../poe_tasks.toml (connector-root entrypoint)\n├── scripts/\n│ ├── run.sh, sf.py\n│ ├── fetch-config.sh, render-config.sh\n│ ├── start-backend.sh, reset-databases.sh, stop-backend.sh\n│ ├── apply-sql.sh, sweep-orphans.sh\n├── fixtures/configs/base.template.json, utc-session.template.json\n├── fixtures/sql/00-cursor-canary-base.sql\n├── fixtures/sql/01-cursor-canary-boundary-row.sql\n└── cases/cursor-upper-bound.sh\n```\n\n## Isolation model\n\n- Every run uses `PROVEFIX_` and asserts that the schema is absent\n before both control and target runs.\n- A marker owned by the run permits reuse of the same schema between canary\n phases; unmarked pre-existing schemas are refused.\n- Control and target use the same schema with `--reset=fixture`; remote\n `--reset=backend` is rejected.\n- Teardown drops the schema in the run's exit trap.\n- `scripts/sweep-orphans.sh` lists or drops old `PROVEFIX_*` schemas, defaulting\n to four hours.\n\nThe default run id is\n`YYYYMMDDhhmmss__`, producing a\nschema-safe `PROVEFIX_` name.\n\nThe schema is owned by the scripts, while `sf.py` is the only code that talks\nto Snowflake. The default config comes from GSM secret\n`SECRET_SOURCE-SNOWFLAKE_KEY_PAIR__CREDS` in project\n`dataline-integration-testing`; fetched configs are marked with a sidecar and\nonly those configs are removed during teardown, so caller-supplied configs\nsurvive. Set `SNOWFLAKE_CONFIG_FILE` to reuse a caller-supplied config.\n\n## Cursor upper-bound canary\n\nThe canary proves the cursor precision regression from airbytehq/airbyte#82705:\n\n```bash\ncases/cursor-upper-bound.sh\n```\n\nIt compares fixed `1.1.1` with control `1.1.0` in two phases. Phase 1 runs the\ninitial sync on the control image with exact-microsecond base rows, producing\nthe checkpoint a user on the bad version would have. Phase 2 inserts a\nsub-microsecond boundary row and replays that state against both versions:\n1.1.0 loses the row because it floors the upper bound, while 1.1.1 emits it.\n\nThe harness reuses the `INTEGRATION_TEST_WAREHOUSE_DESTINATION` X-Small\nwarehouse shared with destination-snowflake integration tests. Never touch\nschemas outside `PROVEFIX_*`.\n\n## Gotchas\n\nSnowflake uppercases identifiers, so use\n`--cursor-field=UPDATED_AT --streams=CURSOR_CANARY`. Rendered configs contain\nthe private key; treat `REPRO_OUT` as 0600 material and never attach it.\nTIMESTAMP_NTZ cursor bounds are bound as TIMESTAMP_LTZ by the JDBC driver and\ncompared through the session TIMEZONE; with the account default\n(America/Los_Angeles), narrow incremental windows match nothing.\nThe harness sets `jdbc_url_params=TIMEZONE=UTC` so the canary isolates #82705;\nthis session-timezone behavior appears to be a separate connector bug worth\nits own issue.\nThe `--catalog` CDC guard (#85752) never fires here: source-snowflake has no\nCDC replication method.\n","tools/bin/admin-service-account.yaml":"apiVersion: rbac.authorization.k8s.io/v1\nkind: Role\nmetadata:\n name: airbyte-admin-role\nrules:\n - apiGroups: [\"*\"]\n resources: [\"jobs\", \"pods\", \"pods/log\", \"pods/exec\", \"pods/attach\"]\n verbs: [\"get\", \"list\", \"watch\", \"create\", \"update\", \"patch\", \"delete\"] # over-permission for now\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: RoleBinding\nmetadata:\n name: default-admin-binding\nroleRef:\n apiGroup: \"\"\n kind: Role\n name: airbyte-admin-role\nsubjects:\n - kind: ServiceAccount\n name: default\n","tools/bin/run-harness-process.yaml":"# Bin Skill\n\nPath: `tools/bin/run-harness-process.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/bin/run-harness-process.yaml)","tools/internal/README.md":"# internal Documentation\n\nPath: `tools/internal/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/internal/README.md)","tools/schema_generator/README.md":"# schema_generator Documentation\n\nPath: `tools/schema_generator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/schema_generator/README.md)","tools/site/README.md":"# site Documentation\n\nPath: `tools/site/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/site/README.md)","airbyte-integrations/connectors/source-commercetools/README.md":"# source-commercetools Documentation\n\nPath: `airbyte-integrations/connectors/source-commercetools/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-commercetools/README.md)","docs/release_notes/agents/readme.md":"# Subagent: readme\n\nPath: `docs/release_notes/agents/readme.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/release_notes/agents/readme.md)","docs/ai-agents/get-started/developer-quickstart/skills/_category_.json":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/_category_.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/_category_.json)","docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md)","docs/ai-agents/get-started/developer-quickstart/skills/codex.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/codex.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/codex.md)","docs/ai-agents/get-started/developer-quickstart/skills/lovable.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/lovable.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/lovable.md)","docs/ai-agents/get-started/developer-quickstart/skills/readme.md":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/readme.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/readme.md)","airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json":"# shared Plugin Manifest\n\nPath: `airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json)","airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json)","airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json)","airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json)","airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json)","airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json)","airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json)","airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json)"},"items":[{"name":"SKILL.md","path":".claude/skills/my-prs/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/.claude/skills/my-prs/SKILL.md","title":"my-prs","category":"anthropic-skill","format":"markdown","content":"---\nname: my-prs\ndescription: List PRs opened today that are assigned to me or the move-destinations team, excluding external contributors\ndisable-model-invocation: true\nallowed-tools: Bash(gh:*), Bash(date:*), Read\nargument-hint: [days-back]\n---\n\n# My PRs - Daily PR Review Dashboard\n\nThis skill fetches and displays PRs that need your attention:\n\n1. **PRs assigned directly to you** (@frifriSF59)\n2. **PRs with review requested from the move-destinations team**\n\nAll results are filtered to:\n- Only show PRs opened today (or within `$ARGUMENTS` days if specified)\n- Exclude PRs from users who are not members of the airbytehq organization\n\n## Execution\n\nRun the fetch script to get the PR list:\n\n```bash\nbash .claude/skills/my-prs/scripts/fetch-prs.sh\n```\n\nIf an argument is provided (number of days), modify the date filter accordingly:\n- `$ARGUMENTS` = number of days to look back (default: 0 = today only)\n- Example: `/my-prs 3` looks at PRs from the last 3 days\n\nFor custom date ranges, use:\n```bash\ngh pr list --repo airbytehq/airbyte --assignee @me --search \"created:>=YYYY-MM-DD\" --json number,title,author,url\n```\n\n## After Fetching\n\nAfter displaying the PRs, offer to:\n1. Open a specific PR in the browser (`gh pr view --web`)\n2. Show details of a specific PR (`gh pr view `)\n3. Show the diff of a specific PR (`gh pr diff `)\n4. Check CI status (`gh pr checks `)\n\n## Quick Actions\n\nThe user may want to:\n- **Review a PR**: Use `/review ` if available\n- **Check PR comments**: `gh pr view --comments`\n- **See who approved**: `gh pr view --json reviews --jq '.reviews[] | select(.state==\"APPROVED\") | .author.login'`\n","frontmatter":{"name":"my-prs","description":"List PRs opened today that are assigned to me or the move-destinations team, excluding external contributors","disable-model-invocation":true,"allowed-tools":"Bash(gh:*), Bash(date:*), Read","argument-hint":["days-back"],"allowedTools":["Bash(gh:*),","Bash(date:*),","Read"],"disableModelInvocation":true},"allowedTools":["Bash(gh:*),","Bash(date:*),","Read"],"isInternal":false,"tokens":455,"sizeBytes":1688},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":362},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/SKILL.md","title":"source-bigquery-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-bigquery-e2e-tests\ndescription: Run the Airbyte protocol smoke sweep for airbyte/source-bigquery: against the shared BigQuery integration project inside a per-run PROVEFIX_ dataset (B3 remote-backend mode of db-harness-lib), with a dry-run orphan sweep.\n---\n\n# source-bigquery-e2e-tests\n\nThis harness runs the source-bigquery protocol regression sweep against an isolated dataset in a real BigQuery project. Orchestration comes from `airbyte-integrations/db-harness-lib`; the engine scripts own the dataset, fixtures, and credentials.\n\n## Prerequisites\n\n- `gcloud` with ambient access to Secret Manager in `GSM_PROJECT`.\n- `bq` and `docker`.\n- `airbyte-ops` (or `uvx airbyte-internal-ops`).\n- A BigQuery service account with dataset create/delete, query, and source-read permissions in `BIGQUERY_PROJECT`.\n\nCloud Storage Read API coverage requires `airbyte-ops cloud connector regression-test --extra-env`, added in [airbyte-ops-mcp#349](https://github.com/airbytehq/airbyte-ops-mcp/pull/349). The shim defaults `BIGQUERY_AIRBYTE_EDITION` to `CLOUD` and adds `AIRBYTE_EDITION=CLOUD` when supported. Set `BIGQUERY_AIRBYTE_EDITION=''` to disable the check and injection. Older `airbyte-ops` versions warn and continue using the query API.\n\n## Run\n\nFrom this connector directory:\n\n```bash\npoe e2e-local --test-version=1.0.1\n```\n\nThe Poe task forwards the shared harness flags; use `poe e2e-local --help` for the full list. Common options include `--command=check`, `--skip-read`, `--step-name=NAME`, `--keep-backend`, and `--reset=fixture`. Environment overrides include `REPRO_OUT`, `GSM_PROJECT`, `BIGQUERY_PROJECT`, `BIGQUERY_LOCATION`, `BIGQUERY_SECRET_NAME`, `BIGQUERY_CONFIG_FILE`, `AIRBYTE_OPS`, and `BIGQUERY_AIRBYTE_EDITION`.\n\n## Dataset lifecycle\n\nEach run creates `PROVEFIX_` in `BIGQUERY_PROJECT` and records ownership in `$REPRO_OUT/.provefix-owned-PROVEFIX_`. An existing dataset is reused only with that marker; otherwise the harness refuses to touch it and points to `sweep-orphans.sh`. The default teardown drops the owned dataset and verifies it is absent. `--reset=fixture` drops and recreates the run dataset between comparison sweeps. `sweep-orphans.sh --dry-run` lists old `PROVEFIX_*` datasets without deleting them; `--older-than-hours=N` defaults to 4.\n\n`gcloud`/`bq` authentication uses `$REPRO_OUT/.gcloud`, separate from the ambient credentials used to fetch the GSM secret. The fetched service-account key and isolated auth directory are removed at teardown.\n\n## Expected smoke result\n\nBoth runs pass CHECK, DISCOVER `smoke_table` and `smoke_view`, and READ three records from each stream. The installed CLI warns when it lacks `--extra-env`, so that run exercises only the query API. With `BIGQUERY_AIRBYTE_EDITION=CLOUD` and the #349 CLI, connector logs show `smoke_table` opening a Storage Read API session and `smoke_view` falling back to the query API because views are unsupported by the Storage Read API.\n\nThere is no bad-vs-fixed proof case yet: the 1.0.1 views fallback requires a service account without BigQuery Read Session User, and none exists in GSM.\n","frontmatter":{"name":"source-bigquery-e2e-tests","description":"Run the Airbyte protocol smoke sweep for airbyte/source-bigquery: against the shared BigQuery integration project inside a per-run PROVEFIX_ dataset (B3 remote-backend mode of db-harness-lib), with a dry-run orphan sweep."},"isInternal":false,"tokens":775,"sizeBytes":3138},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":362},{"name":"00-init-base.json","path":"airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json","title":"Json Skill","category":"anthropic-skill","format":"json","content":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json)","tokens":300,"sizeBytes":351},{"name":"attribute-types.json","path":"airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json","title":"Json Skill","category":"anthropic-skill","format":"json","content":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json)","tokens":300,"sizeBytes":357},{"name":"empty-table.json","path":"airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json","title":"Json Skill","category":"anthropic-skill","format":"json","content":"# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json)","tokens":300,"sizeBytes":349},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/SKILL.md","title":"source-dynamodb-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-dynamodb-e2e-tests\ndescription: Stand up a local DynamoDB Local backend, apply JSON fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-dynamodb: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for a cursor-incremental source-dynamodb test or prove-fix comparison.\n---\n\n# source-dynamodb-e2e-tests\n\nLocal end-to-end test harness for `source-dynamodb`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the DynamoDB Local backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a DynamoDB Local container named\n`source-dynamodb-db-backend`, applies JSON fixtures with the AWS CLI, and runs\nAirbyte protocol commands against any `airbyte/source-dynamodb:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: no CDC mode exists; incremental reads use a user-defined cursor\nattribute. Assume-role / IAM / STS behavior is not covered: DynamoDB Local\ndoes not provide STS, and the connector's `endpoint` only overrides the\nDynamoDB client, so a published connector image cannot send AssumeRole\nrequests to this emulator. State this limitation in the evidence plan rather\nthan claiming coverage.\n\n## When to use this skill\n\n- Reproducing a `source-dynamodb` bug locally without AWS credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-dynamodb:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need AWS, IAM, or Cloud credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-dynamodb-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── aws-cli.sh # AWS CLI container and backend bridge-IP helpers\n│ ├── start-backend.sh # docker run DynamoDB Local with -sharedDb\n│ ├── apply-sql.sh # delete/create tables and put fixture items\n│ ├── reset-databases.sh # delete every table\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # user credentials, region, endpoint placeholder\n └── json/\n ├── 00-init-base.json # sample with a numeric cursor and three items\n ├── attribute-types.json\n └── empty-table.json\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but DynamoDB\nfixtures are JSON documents, not SQL. Each fixture has a `tables` array. Each\ntable entry contains a `create` object accepted by `CreateTable\n--cli-input-json` and an `items` array in DynamoDB attribute-value JSON:\n\n```json\n{\n \"tables\": [\n {\n \"create\": {\n \"TableName\": \"sample\",\n \"KeySchema\": [{\"AttributeName\": \"id\", \"KeyType\": \"HASH\"}],\n \"AttributeDefinitions\": [{\"AttributeName\": \"id\", \"AttributeType\": \"S\"}],\n \"BillingMode\": \"PAY_PER_REQUEST\"\n },\n \"items\": [{\"id\": {\"S\": \"e1\"}, \"seq\": {\"N\": \"1\"}}]\n }\n ]\n}\n```\n\n## Conventions\n\n- Container name: `source-dynamodb-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The config uses dummy `User` credentials and\n `us-east-1`; DynamoDB Local does not authenticate them.\n- `-sharedDb` is required. Without it, DynamoDB Local namespaces tables by\n access key and region, so tables created by the fixture client can be\n invisible to the connector client.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-dynamodb-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `.endpoint` to `http://:8000` at runtime. The AWS CLI\n helper resolves the same bridge IP for its endpoint.\n- DynamoDB Local is pinned to `amazon/dynamodb-local:3.3.2`. Override with\n `BACKEND_IMAGE=…` for a version-specific reproduction. The AWS CLI helper\n image is pinned to `amazon/aws-cli:2.38.2`; override with\n `AWS_CLI_IMAGE=…`.\n- `attribute-types.json` exercises DynamoDB attribute-value families and a\n composite-key table. The connector's source-defined primary key is the\n partition key only; its sort key is not part of that primary key.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-dynamodb\n\n# Single version, full sweep.\npoe e2e-local --test-version=0.4.1\n\n# Target vs. control comparison.\npoe e2e-local --test-version=0.4.1 --control-version=0.3.11 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=0.4.1\n```\n\nThe sweep runs every command against the backend and reports each result\nrather than stopping at the first failure, then prints a summary table. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **DynamoDB key attributes.** `AttributeDefinitions` contains only the\n table's key attributes; non-key attributes such as `seq` and `label` appear\n in fixture items and are discovered by sampling them.\n- **Sampled attributes.** Discovery samples table items and unions the\n attributes it sees. Keep at least one item with each field you want in the\n catalog. A field missing from an item is a useful missing-attribute case,\n but it may not be emitted if it is absent from every sampled item.\n- **Configured-catalog nulls.** DynamoDB discover omits `namespace`,\n `source_defined_cursor`, and `is_resumable`, and 0.4.0 accepts them absent;\n it rejects `namespace: null`. Keep them omitted through an Airbyte Ops build\n that serializes configured catalogs with `exclude_none=True`; no shared\n catalog-generator defaults are needed.\n- **Empty tables.** The connector discovers key attributes from\n `DescribeTable` when there are no items to sample, so an empty table remains\n in the catalog with its key fields.\n- **Comparison catalogs.** Use `--reset=fixture` for comparisons where\n discovery output changes between image tags. In particular, for the\n integer-cursor regression below, the old tag discovers `seq` as\n `[\"null\",\"integer\"]` while the fixed tag discovers it as `number` with\n `airbyte_type: integer`. A paired `--reset=none` run can reuse the target's\n discovered catalog for the control read and mask the control bug, because\n the crash is driven by the cursor's catalog JSON-schema type. Fixture reset\n runs separate sweeps, so each image derives its own catalog.\n- **No role coverage.** Do not use DynamoDB Local to claim assume-role, IAM,\n or STS coverage. There is no STS emulator here, and `endpoint` only\n overrides the DynamoDB client endpoint.\n- **Do not use real AWS.** This harness is for local testing only and must\n never be used against customer AWS accounts, tables containing customer\n data, or Airbyte Cloud.\n\n## Worked example: integer cursor, airbyte#71171\n\nThe historical integer-cursor crash can be proven with the base fixture. The\ncontrol image fails with\n`IllegalArgumentException: No enum constant\nio.airbyte.protocol.models.JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER`\nand emits no records; the fixed image reads all three records and emits\n`STATE` with `cursor: \"3\"`.\n\nUse `--reset=fixture` so the 0.3.11 control and 0.4.0 target each run their\nown `discover` and `read`. The versions discover different cursor schemas,\nand the crash depends on that schema in the configured catalog.\nThis command also requires an Airbyte Ops build with the configured-catalog\n`exclude_none=True` fix: DynamoDB discovery omits `namespace`, and older Ops\nbuilds serialize that omission as `namespace: null`, which 0.4.0 rejects.\n\n```bash\nREPRO_OUT=\"$HOME/dynamodb-hyd310-evidence/patched-ops-1792531195\" \\\n poe e2e-local \\\n --step-name=integer-cursor-71171 \\\n --control-version=0.3.11 \\\n --test-version=0.4.0 \\\n --reset=fixture \\\n --sync-mode=incremental \\\n --cursor-field=seq \\\n --streams=sample \\\n --expect-control=fail \\\n --expect-test=pass \\\n --min-records=3 \\\n --min-states=1 \\\n --forbid-match=read:any:JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER\n```\n\nVerified on 2026-10-20 with the Airbyte Ops null-namespace fix:\n\n| Command | Control (`0.3.11`) | Target (`0.4.0`) |\n|---------|---------|---------|\n| `SPEC` | PASS | PASS |\n| `CHECK` | PASS | PASS |\n| `DISCOVER` | PASS | PASS |\n| `READ` | FAIL — integer-enum crash, 0 records, 0 states | PASS — 3 records, 1 state (`cursor: \"3\"`) |\n\nThe paired command exited 0: the control reproduced the historical failure,\nwhile the target passed and emitted the expected records and state.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-dynamodb-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","frontmatter":{"name":"source-dynamodb-e2e-tests","description":"Stand up a local DynamoDB Local backend, apply JSON fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-dynamodb: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for a cursor-incremental source-dynamodb test or prove-fix comparison."},"isInternal":false,"tokens":2566,"sizeBytes":10303},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":370},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/SKILL.md","title":"source-mongodb-v2-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-mongodb-v2-e2e-tests\ndescription: Stand up a local MongoDB 7.0 single-node replica set, apply mongosh JavaScript fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mongodb-v2: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any non-CDC source-mongodb-v2 test or prove-fix comparison.\n---\n\n# source-mongodb-v2-e2e-tests\n\nLocal end-to-end test harness for `source-mongodb-v2`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MongoDB backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MongoDB 7.0 container named\n`source-mongodb-v2-db-backend` configured as a single-node replica set\n(`rs0`), lets you apply arbitrary `mongosh` JavaScript fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mongodb-v2:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: non-CDC (full-refresh / initial-snapshot) behavior. Change-stream\nresume-token replay and CDC comparison flows are not covered by this skill.\n\n## When to use this skill\n\n- Reproducing a `source-mongodb-v2` bug locally without Atlas credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-mongodb-v2:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM, Atlas, or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mongodb-v2-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mongo:7.0 --replSet rs0; rs.initiate()\n│ ├── apply-sql.sh # docker exec mongosh with a .js fixture on stdin\n│ ├── reset-databases.sh # drop every non-system database\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # SELF_MANAGED_REPLICA_SET config; host placeholder\n └── js/\n └── 00-init-base.js # test_db.sample with three documents\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but MongoDB\nfixtures are `mongosh` scripts, not SQL. Each fixture runs with `test_db`\n(`BACKEND_DB`) as the current `db`; use `db.getSiblingDB(...)` for others.\n\n## Conventions\n\n- Container name: `source-mongodb-v2-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The backend runs without `--auth`; the config's\n `database_config` omits `username`/`password`. Add them to a copied\n template if you need to reproduce an auth-specific issue.\n- Replica set: `rs0` (`BACKEND_REPLSET`). The connector requires a replica\n set even for non-CDC reads. `start-backend.sh` registers the single member\n under the container's bridge IP so the connector container, which performs\n replica-set discovery from the seed list, can reach the advertised host.\n Do not change the member host to `localhost`: the connector would then\n fail to connect.\n- Initial database: `test_db`. Override with `BACKEND_DB`; keep\n `database_config.databases` in the config template in sync.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mongodb-v2-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `database_config.connection_string` to\n `mongodb://:27017/?replicaSet=rs0` at runtime.\n- The backend image is pinned to `mongo:7.0`. Override with `BACKEND_IMAGE=…`\n for a version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mongodb-v2\n\n# Single version, full sweep.\npoe e2e-local --test-version=2.0.7\n\n# Target vs. control comparison.\npoe e2e-local --test-version=dev --control-version=2.0.6 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=2.0.7\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. Under CI the table is also\nappended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **Schema enforcement.** The base template sets\n `database_config.schema_enforced: true`, so `discover` samples documents\n and emits typed fields. Set it to `false` in a copied template to\n reproduce schemaless (`data` blob) behavior.\n- **`_id` types.** Fixtures use integer `_id`s for readability; use\n `ObjectId()` in a fixture when the bug depends on ObjectId serialization.\n- **Schema inference needs documents.** An empty collection created with\n `db.createCollection()` is still listed by `discover`, but with\n `schema_enforced: true` its schema has no sampled fields beyond `_id`.\n Insert at least one document in the fixture for every field you expect\n in the catalog.\n- **Reset drops databases, not the replica set.** `--reset=fixture` removes\n all non-system databases and reapplies fixtures; the oplog and resume\n tokens survive. This only matters once change-stream (CDC) coverage\n lands (HYD-147 phase 2); non-CDC sweeps do not read the oplog. Use\n `--reset=backend` when a run must start from a fresh oplog.\n- **Databases are created on first write.** MongoDB has no `CREATE\n DATABASE`; `test_db` exists only once a fixture inserts into it, so a\n fixture that only calls `db.createCollection(...)` or writes to a\n `getSiblingDB(...)` database leaves `test_db` absent and `discover` empty.\n `00-init-base.js` inserts into `test_db`; keep that in any fixture set you\n pass with `--fixture`.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections, Atlas clusters\n holding customer data, or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mongodb-v2-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","frontmatter":{"name":"source-mongodb-v2-e2e-tests","description":"Stand up a local MongoDB 7.0 single-node replica set, apply mongosh JavaScript fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mongodb-v2: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any non-CDC source-mongodb-v2 test or prove-fix comparison."},"isInternal":false,"tokens":1919,"sizeBytes":7611},{"name":"order-items-cdc.json","path":"airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json","title":"Catalogs Skill","category":"anthropic-skill","format":"json","content":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json)","tokens":300,"sizeBytes":365},{"name":"users-cdc.json","path":"airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","title":"Catalogs Skill","category":"anthropic-skill","format":"json","content":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","tokens":300,"sizeBytes":353},{"name":"cdc.template.json","path":"airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json)","tokens":300,"sizeBytes":356},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/SKILL.md","title":"source-mssql-e2e-cdc-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-mssql-e2e-cdc-tests\ndescription: Reproduce CDC bugs against source-mssql by enabling CDC on the local SQL Server backend, applying per-bug SQL fixtures, and running the connector via airbyte-ops. Composes on top of source-mssql-e2e-tests. Worked examples for airbytehq/oncall#11451, #12094, and #12162 ship with the skill.\n---\n\n# source-mssql-e2e-cdc-tests\n\nLocal CDC bug-reproduction harness for `source-mssql`. Builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe MSSQL engine skill\n([`source-mssql-e2e-tests`](../source-mssql-e2e-tests/SKILL.md)):\ngeneric skill stands up the SQL Server backend and runs the connector;\nthis skill adds CDC enable, CDC-aware config / catalog templates, and\nper-bug **case scripts** under `cases/` that call `run.sh` with the\nappropriate fixtures and `--expect-*` assertions.\n\n## When to use this skill\n\n- Reproducing a CDC-mode bug against `source-mssql` locally.\n- Verifying a fix by re-running an existing case with\n `VERSION=dev ./cases/.sh` (after `:dockerBuildx`).\n- Verifying that a fix which _loosens_ CDC offset validation still\n rejects genuinely invalid state — `cases/11451.sh` is the\n invalid-state case for LSN availability; see\n [Invalid-state case](#invalid-state-case-for-lsn-availability-fixes).\n- Authoring a new repro for a customer-reported CDC bug. Drop a SQL\n fixture under `fixtures/sql/` and a `cases/.sh` script\n that invokes `run.sh` with the right fixtures and `--expect-*`\n assertions.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, a clone of `airbytehq/airbyte`.\n- The backend container `source-mssql-db-backend` started by\n `../source-mssql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials.\n\n## Layout\n\n```\nsource-mssql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── 11451.sh # airbytehq/oncall#11451 — LSN-range regression in 4.3.4+ (multi-phase; invalid-state case)\n│ ├── 12094.sh # airbytehq/oncall#12094 — schema-history bloat\n│ └── 12162.sh # airbytehq/oncall#12162 — whitespace in stream name\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json\n ├── catalogs/\n │ ├── users-cdc.json\n │ └── order-items-cdc.json\n └── sql/\n ├── 00-init-cdc.sql\n ├── repro-11451-lsn-cleanup.sql\n ├── repro-12094-schema-history.sql\n └── repro-12162-spaces-in-name.sql\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit walks Airbyte STATE messages, which is protocol-level and not\nCDC-specific.\n\n## Conventions\n\n- Cases keep the backend up across invocations (`--keep-backend`).\n Start the backend once at the top of a session, run any subset of\n cases, tear down when done — see the Usage section below.\n- Fixtures are re-applied at the start of each `run.sh` invocation, so\n they must be idempotent (`00-init-cdc.sql` uses `IF DB_ID(...) IS NULL`\n guards, per-bug fixtures use similar shape). Re-applying a fixture\n during a multi-phase case does not reset the CDC state that the\n previous phase established.\n- Configured catalogs populate the bulk-CDK-required fields\n (`is_file_based`, `generation_id`, `minimum_generation_id`,\n `sync_id`, `destination_object_name`, `include_files`) so the same\n fixtures drive repros across `source-mssql:4.3.x` and `4.4.x`.\n- Cases default `VERSION=4.4.2` (except `11451.sh` which splits into\n `BASELINE_VERSION`/`TARGET_VERSION`). Override with\n `VERSION=4.3.4 ./cases/12162.sh` to test against an earlier version,\n or `VERSION=dev` after a local\n `./gradlew :airbyte-integrations:connectors:source-mssql:dockerBuildx`\n to test a fix.\n- **Assertions** are declarative via `run.sh`'s expectation flags:\n `--expect-test`, `--expect-match=[:]:[:N]`,\n `--forbid-match`, `--min-records`, `--min-states`. The runner\n enforces them and exits non-zero on any failure. The `:`\n prefix defaults to `read` when omitted; set it explicitly\n (`check:stderr:…`, `discover:stderr:…`) for check-time or\n discover-time signatures.\n- Multi-phase cases (`11451.sh`, and any future read → mutate →\n read-with-state repros) capture Airbyte STATE messages between reads\n via the shared\n [`extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\n (`uv`-PEP-723 standalone; pass `` as its argument or pipe\n stdin) and feed the file back into the second read with\n `run.sh --state=PATH`.\n Use `--step-name=/` to give each phase its own artifact\n subtree.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mssql-repro\n\n# 1. Bring up the backend (once per session). Cases pass --keep-backend,\n# so they don't tear it down between runs.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# 2. Run any case. Each case's --expect-* flags gate its own exit code\n# (non-zero on any expectation failure), so bash's `set -e` will fail\n# the sequence on the first failing case if you chain them.\n\"$SKILL/cases/12162.sh\"\n\"$SKILL/cases/12094.sh\"\n\"$SKILL/cases/11451.sh\"\n\n# 3. (After fix) verify by retargeting `dev` or a fixed version.\nVERSION=dev \"$SKILL/cases/12162.sh\"\n\n# 4. Tear down.\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\nrm -rf \"$REPRO_OUT\"\n```\n\nEach case is a shell script that composes `run.sh` invocations:\n\n1. Applies its SQL fixtures (via `--fixture=…`; `run.sh` calls\n `apply-sql.sh` internally).\n2. Renders the config against the running backend (via\n `--config-template=…`).\n3. Runs the target protocol command with any `--catalog` / `--state`.\n4. Gates its exit code on `--expect-*` flags — the runner enforces\n them and exits non-zero on any failure, so the case script has no\n inline assertion logic to maintain.\n\nMulti-phase cases (`11451.sh`) call `run.sh` more than once, with\n`extract-state.py` and `apply-sql.sh` between invocations. Each phase\nuses `--step-name=/` to keep its artifacts separate under\n`$REPRO_OUT///`.\n\n## Worked examples\n\n### airbytehq/oncall#12162 — whitespace in stream name\n\n`cases/12162.sh`. Creates `dbo.[Order Items]` (note the space), enables\nCDC on it, and runs `read` against it. Case asserts\n`--expect-test=fail`, `--expect-match=stderr:io.debezium.DebeziumException`,\nand `--expect-match=stderr:message.key.columns` — i.e. the read exits\nnon-zero and Debezium's `stderr.txt` contains its\n`Connector configuration is not valid. The 'message.key.columns' value\nis invalid` rejection.\n\nRoot cause: `MsSqlServerDebeziumOperations.buildMessageKeyColumns()`\njoins `schema.table:pkcol` strings without filtering or escaping\nidentifiers that contain whitespace or `:`, and Debezium rejects them\nat engine startup. Fix: pre-filter such streams (Debezium falls back\nto the table's native PK from system tables).\n\n### airbytehq/oncall#12094 — schema-history bloat\n\n`cases/12094.sh`. Creates 30 noise tables in `dbo` (CDC not enabled on\nthem), runs the baseline `read` against `dbo.users`, and asserts\n`--expect-match='stdout:Adding table CdcTest\\..* to the list of capture schema tables:30'`\n(the trailing `:30` is the count threshold; the fixture creates 30\nnoise tables). Adjust with `MIN_LOADED=`. Expected outcome: at\nleast 30 \"Adding table\" lines even though the configured catalog has\na single stream.\n\nRoot cause: `withSchemaHistory()` in the bulk-CDK Debezium properties\nsets `schema.history.internal.store.only.captured.databases.ddl=true`\nbut not `…captured.tables.ddl=true`. Same harness reproduces equivalent\nbehavior on `source-mysql` and `source-postgres` because the property\nlives in the bulk-CDK `extract-cdc` toolkit.\n\n### airbytehq/oncall#11451 — LSN-range regression in 4.3.4+\n\n`cases/11451.sh`. Multi-phase:\n\n1. Baseline `read` on clean CdcTest (`BASELINE_VERSION`, default `4.4.2`).\n Case asserts `--expect-test=pass --min-states=1` so a missing STATE\n fails the case immediately rather than later at replay.\n2. `extract-state.py` on the baseline stdout, then\n `apply-sql.sh repro-11451-lsn-cleanup.sql` runs\n `sys.sp_cdc_cleanup_change_table`\n with a `low_water_mark` past the saved LSN — advancing\n `fn_cdc_get_min_lsn('dbo_users')` past the baseline offset.\n3. Replay `read` on `TARGET_VERSION` (default `4.3.4`) with the stale\n state. Case asserts `--expect-test=fail`,\n `--expect-match=stderr:Saved offset no longer present`, and\n `--expect-match='stderr:is no longer available in SQL Server transaction logs'`.\n\nRoot cause: pre-`4.3.4` the LSN-range query computed `min` via\n`sys.fn_cdc_get_min_lsn('')`, which always returns\n`0x00000000000000000000`. `4.3.4` swapped to\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance)) FROM cdc.change_tables`,\nwhich returns the actual per-instance min — a value that can advance\npast a saved offset on geo-replicas with aggressive cleanup. The\nsaved-offset-rejection guard then fires even though the data is\nstill present. Investigation lives at\n[`airbytehq/oncall#11451`](https://github.com/airbytehq/oncall/issues/11451).\n\n#### Invalid-state case for LSN-availability fixes\n\nThis case's Phase 3 is a _genuinely_ expired saved offset: Phase 2\nadvances `fn_cdc_get_min_lsn` past the captured LSN, so rejecting the\nreplay is the correct behavior, not the bug. That makes it the\ninvalid-state counterpart to any repro built around a saved LSN that is\nstill valid — for example an unauthorized capture instance returning\n`0x0` and poisoning\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance))`.\n\nSo when a change relaxes what the offset-availability check aggregates\nor tolerates, run this case against the image built from that change:\n\n```bash\nTARGET_VERSION=dev \"$SKILL/cases/11451.sh\"\n```\n\nIt must still fail with the same two `stderr` signatures the case\nasserts on. A pass here means the guard stopped firing on state it is\nsupposed to reject, which turns an actionable \"reset the connection\"\nerror into a silently incomplete sync — the symptom-gone evidence from\nthe valid-LSN repro cannot distinguish that from a correct fix.\n\n## Authoring a new repro\n\n1. Drop a SQL fixture in `fixtures/sql/repro--.sql`.\n Make it idempotent (`IF NOT EXISTS` guards for `CREATE`,\n `IF EXISTS` guards for `DROP`) — cases re-apply fixtures on every\n `run.sh` invocation.\n2. (Optional) Drop a CDC-aware catalog in `fixtures/catalogs/`.\n3. Drop a case script in `cases/.sh` modeled on the\n existing ones. `set -euo pipefail`; default `VERSION=4.4.2` with a\n `${VERSION:-…}` override; invoke `run.sh` with the right\n `--command`, `--fixture=`, `--config-template=`, `--catalog=`, and\n `--expect-*` flags. Pass `--keep-backend` so the shared-session\n Usage flow works. For check-time or discover-time signatures, use\n `--expect-match=check:stderr:…` / `--expect-match=discover:stderr:…`\n (the `:` prefix defaults to `read` when omitted).\n4. Verify locally:\n ```bash\n \"$GENERIC/scripts/start-backend.sh\"\n \"$SKILL/cases/.sh\"\n BACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\n ```\n5. Note the worked example in this `SKILL.md`'s \"Worked examples\"\n section with: customer-symptom one-liner, `--expect-*` assertions\n the case gates on, root-cause one-liner.\n\n### Multi-phase repros\n\nWhen the reproduction is single-phase (one `run.sh` invocation is\nenough to trigger the bug), follow the recipe above. Reach for a\nmulti-phase shape only when the bug needs an intermediate mutation\nthat depends on state from a first read — the canonical example is\nread → extract STATE → mutate the server based on that STATE → replay\nwith the stale STATE. Model on [`cases/11451.sh`](cases/11451.sh) —\nthe three primitives it composes are:\n\n- **`--step-name=/`** — per-phase artifact dirs under\n `$REPRO_OUT//`. `cases/11451.sh` uses `11451/baseline` and\n `11451/stale`; a new case would use its own bug number and phase\n names.\n- **`extract-state.py`** (implemented in `db-harness-lib`) — reads a phase's\n `read/stdout.txt` and emits a JSON array of AirbyteStateMessage\n objects, which is what `run.sh --state=PATH` expects.\n- **`--skip-fixtures`** on the second/later `run.sh` invocation —\n opts out of the initial-fixture reapply so the intermediate state\n the previous phase established survives. Otherwise `run.sh`'s\n default fixture-application would re-run `00-init-cdc.sql`, which\n drops and recreates `CdcTest` and wipes the mutation. Rejects\n `--fixture=` in the same invocation (either apply fixtures or skip\n them).\n\nThe between-phase mutation itself is a plain `apply-sql.sh` call\nagainst a fixture in `fixtures/sql/`. Nothing about the mutation is\nspecial-cased in `run.sh` — the case script drives the sequencing.\n","frontmatter":{"name":"source-mssql-e2e-cdc-tests","description":"Reproduce CDC bugs against source-mssql by enabling CDC on the local SQL Server backend, applying per-bug SQL fixtures, and running the connector via airbyte-ops. Composes on top of source-mssql-e2e-tests. Worked examples for airbytehq/oncall#11451,"},"isInternal":false,"tokens":3535,"sizeBytes":13275},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":350},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/SKILL.md","title":"source-mssql-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-mssql-e2e-tests\ndescription: Stand up a local SQL Server 2022 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mssql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mssql test or repro that is not CDC-specific.\n---\n\n# source-mssql-e2e-tests\n\nLocal end-to-end test harness for `source-mssql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../db-harness-lib/);\nthis skill keeps the MSSQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a SQL Server\n2022 container named `source-mssql-db-backend`, lets you apply arbitrary\nSQL fixtures, and runs Airbyte protocol commands against any\n`airbyte/source-mssql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a non-CDC bug against `source-mssql` locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mssql:` for connector\n development.\n- As the building block for the\n [`source-mssql-e2e-cdc-tests`](../source-mssql-e2e-cdc-tests/SKILL.md)\n skill, which adds CDC enable plus worked-example fixtures on top.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install airbyte-internal-ops`\n puts `airbyte-ops` on `$PATH`; alternatively prefix every call with\n `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mssql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mcr…/mssql/server:2022-latest as source-mssql-db-backend\n│ ├── apply-sql.sh # docker cp + docker exec sqlcmd -i\n│ ├── reset-databases.sh # docker exec sqlcmd → drop every non-system database (used by run.sh --reset=fixture)\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # non-CDC config; host=mssql-db-backend placeholder\n └── sql/\n ├── .gitignore # `.tmp/` — subtree for uncommitted per-bug scratch fixtures\n └── 00-init-base.sql # CREATE DATABASE TestDb + dbo.sample table\n```\n\nThe engine shim and backend lifecycle scripts remain in this skill's\n`scripts/` directory; the shared orchestration, default teardown, and\nprotocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mssql-db-backend`. Hard-coded in\n `scripts/start-backend.sh` and `apply-sql.sh`. The shared\n `db-harness-lib/scripts/stop-backend.sh` removes this container by\n `BACKEND_NAME`; the engine shim supplies the default. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mssql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per\n protocol command (`spec/`, `check/`, `discover/`, `read/`) holding that\n command's `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n This is the same layout the ops repo's `connector-regression-test.yml`\n workflow uses under `/tmp/regression_test_artifacts`, so CI can upload\n it per command.\n- Both containers (the SQL Server backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765);\n once `--network` is supported the bridge-IP dance can collapse.\n- SQL Server image is pinned to `mcr.microsoft.com/mssql/server:2022-latest`,\n not `latest`, for stable major-version behavior across CU patches.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover`'s output, run `read`, tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mssql\n\n# Single version, full sweep.\npoe e2e-local --test-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, non-CDC (airbyte-ops's built-in comparator).\npoe e2e-local --test-version=dev --control-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, CDC (per-image reset between the two runs).\npoe e2e-local --test-version=dev --control-version=5.0.0 --reset=fixture \\\n --config-template=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json \\\n --sync-mode=incremental --cursor-field=_ab_cdc_cursor --streams=users \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/00-init-cdc.sql \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/.sql\n\n# One command only.\npoe e2e-local --command=read --test-version=5.0.0\n```\n\nThe CDC example needs an incremental catalog (`--sync-mode=incremental\n--cursor-field=_ab_cdc_cursor`) restricted with `--streams` to exclude\n`dbo.systranschemas`; the general rule is in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md#cdc-config-templates-need-an-incremental-catalog).\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed — separating an\ninfrastructure failure (`ERROR`, no verdict was produced) from a failed\ntest verdict (`FAIL`), as the ops workflow's final-status step does. A\n`read` whose `discover` failed earlier in the sweep is reported `SKIPPED`,\nnot `ERROR` — an invalid-config case is expected to fail `discover`, and\nthere is then no catalog for `read` to use. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to\none command instead exits with the connector's own exit code, so a repro\ncan still assert on it.\n\nPrefer a published tag over `--test-version=dev`: for code on a pushed PR\nbranch, publish a pre-release and pass its `-preview.` tag,\nwhich skips the Gradle build and gives reviewers a tag they can re-run\nagainst — see\n[Getting a target image](../../../AGENTS.md#getting-a-target-image). When\nyou do use `--test-version=dev`, build the target image first, or pass\n`--build` to have `run.sh` run `:dockerBuildx` for you. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures` (run against whatever state the backend already has;\nfor the second/later `run.sh` invocation of a multi-phase driver, when\nre-applying the initial fixture would wipe intermediate state),\n`--step-name`, `--catalog` (skip discover-derived generation),\n`--state=PATH` (pass a saved STATE file to the read as `--state-path`;\nfor use by multi-phase drivers), `--sync-mode=incremental`,\n`--cursor-field`, `--streams`, `--config-template`,\n`--reset=none|fixture|backend`, `--keep-backend`, and `--` to forward\nextra args to `airbyte-ops`. Per-command timeouts match the workflow's\n(30/30/60/180 minutes) and are overridable with\n`TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n### Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst each command's artifacts, `run.sh` accepts a small set of\nexpectation flags that it enforces itself before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `--expect-test=pass\\|fail` | Overall target verdict. `pass` = every executed command's status was `pass`; `fail` = one or more failed. |\n| `--expect-control=pass\\|fail` | Same for the control sweep (comparison-mode runs only; requires `--control-version`). |\n| `--min-records=N` | Target read's `stdout.txt` must contain ≥N `RECORD` messages. |\n| `--min-states=N` | Target read's `stdout.txt` must contain ≥N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target's `` step's `` (`stdout` \\| `stderr` \\| `any`) must match `` at least N times (default 1). `` (`spec` \\| `check` \\| `discover` \\| `read`) is optional and defaults to `read`. Repeatable. |\n| `--forbid-match=[:]:` | Same shape but must match zero times. Repeatable. |\n\nAll match assertions run against the **target-side** artifacts of the\nnamed command (or `read` if no `` prefix). Under\n`--control-version + --reset=none` those live at\n`$ARTIFACTS_DIR//target/` (created by airbyte-ops's comparison\nmode); under `--reset=fixture|backend` at\n`$ARTIFACTS_DIR/target//`; under single-version at\n`$ARTIFACTS_DIR//`. `run.sh` picks the right path automatically.\n\nThe match-spec grammar disambiguates the leading `:` from the\n`:` on the first colon-separated field (command names and\nchannel names don't overlap), and strips a trailing `:N` only when the\nlast field is a positive integer. So a regex containing `:` still\nparses: `--expect-match=stderr:io.debezium.DebeziumException` reads as\ncommand `read` (default), channel `stderr`, regex\n`io.debezium.DebeziumException`, count 1. `--expect-match=check:stderr:should be positive`\nreads as command `check`, channel `stderr`, regex `should be positive`,\ncount 1.\n\n`--min-records` and `--min-states` are read-step only — they count\nRECORD / STATE envelopes, which are read-specific — and have no\n`:` prefix.\n\nAny expectation failure exits non-zero regardless of the command-level\nverdicts and appends an `**Expectation failures:**` section to the\nsummary. Migrating an existing driver script's `grep -q '' || exit 1`\nblock is straightforward — drop the block and add\n`--expect-match=stderr:X` to the shared `run.sh` invocation.\n\n### Multi-phase drivers\n\n`run.sh` runs one sweep. A driver script composes multi-phase flows\n(read → mutate → read-with-state) by calling `run.sh` more than once\nwith `--step-name` for each invocation and `--state=` to feed the\nsecond read the state extracted from the first. `extract-state.py`\nlives in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\nbecause it walks Airbyte STATE messages, which is a protocol-level\noperation and not CDC-specific.\n\nThe CDC skill's [`cases/11451.sh`](../source-mssql-e2e-cdc-tests/cases/11451.sh)\nis a worked example: it invokes the engine shim's `run.sh` around an\nintermediate `apply-sql.sh` mutation and the shared\n`db-harness-lib/scripts/extract-state.py` step, and asserts on the\nconnector's stderr. New multi-phase drivers can follow the same shape\nbut call `run.sh --state=PATH` directly instead of smuggling\n`--state-path` through trailing args.\n\n### Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` once per\n protocol command with both `--test-image` and `--control-image`, so\n the two images run sequentially against a single backend and\n `airbyte-ops`'s built-in comparators emit the target-vs-control diff\n (record counts, primary keys, per-record, schema). Right for non-CDC\n full-refresh work: the diff is meaningful and the shared-backend\n assumption is safe because a full-refresh read does not mutate the\n upstream. This is the current behavior for any caller that doesn't\n set `--reset`.\n- **`--reset=fixture`** — `run.sh` runs the whole sweep against the\n control image first, drops every non-system database, re-applies the\n fixtures, then runs the sweep against the target image. Two\n single-version `airbyte-ops` calls, no built-in comparator; artifacts\n land under `$REPRO_OUT//{control,target}//`.\n Right for **CDC comparisons**, where reusing the backend leaves the\n target reading against a warm capture instance and an advanced log\n position — the built-in diff can look clean while being meaningless.\n The SQL Server log-LSN clock is server-wide and keeps ticking across\n the reset, so per-record LSN columns and STATE offsets still differ\n between runs; gate the verdict on record-level shape rather than on\n those values.\n- **`--reset=backend`** — same as `--reset=fixture` but also recreates\n the backend container between the two sweeps, resetting the LSN clock\n at ~15s of extra startup cost. Use only when the reproduction depends\n on matching LSN sequences across runs.\n\n`--reset` has no effect without `--control-version` and produces a\nwarning; the flag governs the reset between two image runs, and there\nis no second run in single-version mode.\n\n## Asserting on output\n\nInline assertions in driver scripts. Suggested helpers:\n\n- Exit code: check `$?` immediately after the call.\n- AirbyteMessage shape: `jq -e 'select(.type == \"RECORD\")' $REPRO_OUT//read/stdout.txt`.\n- Connector-side log shape: `grep -c '' $REPRO_OUT//read/stderr.txt`.\n- Connection status: `jq -e 'select(.type == \"CONNECTION_STATUS\" and .connectionStatus.status == \"SUCCEEDED\")' $REPRO_OUT//check/stdout.txt`.\n\nIn comparison mode with `--reset=none` each command's directory holds\n`target/` and `control/` subdirectories (created by `airbyte-ops`).\nWith `--reset=fixture|backend` the split lives one level up:\n`$REPRO_OUT//{control,target}//`.\n\nPass `-- --enable-debug-logs=True` to `run.sh` to set `LOG_LEVEL=DEBUG`\non the connector container. That surfaces the\nDebezium \"Adding table … to the list of capture schema tables\" lines\nthat some assertions rely on.\n\n## Common gotchas\n\n- _No SQL Server Agent_ → CDC capture / cleanup jobs never run, and\n Debezium silently produces no records. `start-backend.sh` sets\n `MSSQL_AGENT_ENABLED=true` for this reason.\n- _Race on first `sqlcmd`_ → SQL Server 2022 takes around 15 seconds to\n accept connections cold. `start-backend.sh` polls until `SELECT 1`\n succeeds before returning.\n- _Configured-stream fields the validator rejects as null_ → the\n bulk-CDK schema validator requires `is_file_based`, `cursor_field`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files` on every\n `ConfiguredAirbyteStream`; omitting them fails with\n `Null value is not allowed. (code: 1021)`. This is not 4.3.x-only —\n verified on `4.4.12` and `5.0.0` too. `discover` does not emit\n `is_file_based`, so the catalog `run.sh` derives fills it in, and the\n catalog fixtures shipped with this and dependent skills populate all of\n them.\n- _A failed `check` does not fail a single-version run_ → the CDK exits\n 0 even when it emits `CONNECTION_STATUS` with `status: FAILED`, so the\n `check` step passes. Assert on the status message yourself (see\n [Asserting on output](#asserting-on-output)) rather than on the exit\n code when the repro hinges on `check`. Comparison mode does surface\n it, because it reads the report's verdict rather than the exit code.\n\n## Tear-down\n\n`run.sh` removes the backend container in an `EXIT` trap, so a run leaves\nnothing behind unless you passed `--keep-backend`. After a\n`--keep-backend` run, or to also discard the collected artifacts:\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\" # idempotent; docker rm -f no-ops when absent\nrm -rf \"${REPRO_OUT:-/tmp/source-mssql-repro}\"\n```\n","frontmatter":{"name":"source-mssql-e2e-tests","description":"Stand up a local SQL Server 2022 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mssql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mssql test or repro that is not CDC-specific."},"isInternal":false,"tokens":4179,"sizeBytes":17583},{"name":"users-cdc.json","path":"airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","title":"Catalogs Skill","category":"anthropic-skill","format":"json","content":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","tokens":300,"sizeBytes":353},{"name":"cdc.template.json","path":"airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json)","tokens":300,"sizeBytes":356},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/SKILL.md","title":"source-mysql-e2e-cdc-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-mysql-e2e-cdc-tests\ndescription: Reproduce MySQL binlog CDC behavior against a local MySQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-mysql-e2e-tests and db-harness-lib.\n---\n\n# source-mysql-e2e-cdc-tests\n\nLocal CDC test harness for `source-mysql`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic MySQL skill\n([`source-mysql-e2e-tests`](../source-mysql-e2e-tests/SKILL.md)). The generic\nskill starts a MySQL 8.0 backend and runs the connector; this skill supplies\nthe binlog CDC config, CDC catalog, SQL fixtures, and case scripts.\n\nMySQL CDC is binlog-based and server-wide. The generic backend already starts\nMySQL with ROW binlogging, full row images, and GTIDs, so this skill does not\nenable CDC per table or start a second backend.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-mysql` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new MySQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-mysql-db-backend`, started by\n `../source-mysql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-mysql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with MySQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # dedicated cdc_test database and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic MySQL backend starts with `--log-bin`, `--binlog-format=ROW`,\n `--binlog-row-image=FULL`, `--gtid-mode=ON`, and\n `--enforce-gtid-consistency=ON`. Do not add per-table CDC enable steps or\n engine-specific backend startup to this skill.\n- `00-init-cdc.sql` drops and recreates the dedicated `cdc_test` database on\n every application. Re-applying it wipes existing rows and any accumulated\n binlog-relative table state, which is why `cases/incremental-replay.sh`\n passes `--skip-fixtures` on its second phase.\n- CDC catalogs include the bulk-CDK fields (`is_file_based`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files`) and use\n `[\"_ab_cdc_cursor\"]` as the cursor field.\n- Cases default to `VERSION=3.53.4`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mysql-repro\n\n# Start the binlog-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-mysql-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the binlog-ready backend, CDC config, catalog metadata, and initial load\nwork together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved binlog\nposition resumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No MySQL CDC fixtures or\nworked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep any CDC database\n setup dedicated to the fixture and do not add per-table CDC enable steps.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.53.4}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses the plain MySQL client on stdin, so fixtures\nmay contain multiple statements. The MySQL server's binlog and GTID settings\nare established when the generic backend starts; no additional CDC setup is\nneeded in a case.\n","frontmatter":{"name":"source-mysql-e2e-cdc-tests","description":"Reproduce MySQL binlog CDC behavior against a local MySQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-mysql-e2e-tests and db-harness-lib."},"isInternal":false,"tokens":1809,"sizeBytes":7340},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":350},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/SKILL.md","title":"source-mysql-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-mysql-e2e-tests\ndescription: Stand up a local MySQL 8.0 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mysql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mysql test or prove-fix comparison.\n---\n\n# source-mysql-e2e-tests\n\nLocal end-to-end test harness for `source-mysql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MySQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MySQL 8.0 container named\n`source-mysql-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mysql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a `source-mysql` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mysql:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mysql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mysql:8.0 with ROW binlog + GTIDs\n│ ├── apply-sql.sh # docker exec mysql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mysql-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Root password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mysql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the MySQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `mysql:8.0`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n- The backend starts with ROW-format binary logging, a full row image,\n and GTIDs. This leaves the container ready for a CDC driver without\n restarting it.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mysql\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.53.3\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.53.2\n\n# One command only.\npoe e2e-local --command=read --test-version=3.53.3\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose `discover`\nfailed earlier is reported `SKIPPED`, not `ERROR`. Under CI the table is\nalso appended to `$GITHUB_STEP_SUMMARY`. A run limited to one command\ninstead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`. Use this for CDC\n comparisons so the target does not read a warm capture instance.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the MySQL\n container between sweeps. Use this only when matching binlog or GTID\n sequences matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC needs ROW binlogging.** `start-backend.sh` sets\n `--binlog-format=ROW`, `--binlog-row-image=FULL`, and a server ID.\n The connecting user must have replication privileges; the root user\n created by the fixture has them.\n- **MySQL config shapes are specific.** `ssl_mode` uses `preferred`,\n `required`, `verify_ca`, or `verify_identity`; the base config uses\n `preferred` for the local plaintext backend. The\n `replication_method.method` value is `STANDARD` or `CDC`.\n- **`tinyint(1)` mapping.** The connector maps `TINYINT(1)` to boolean by\n default. Set `treat_tinyint1_as_integer: true` in a config when\n reproducing a typing issue.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mysql-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","frontmatter":{"name":"source-mysql-e2e-tests","description":"Stand up a local MySQL 8.0 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mysql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mysql test or prove-fix comparison."},"isInternal":false,"tokens":2458,"sizeBytes":10431},{"name":"users-cdc.json","path":"airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json","title":"Catalogs Skill","category":"anthropic-skill","format":"json","content":"# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)","tokens":300,"sizeBytes":365},{"name":"cdc.template.json","path":"airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json)","tokens":300,"sizeBytes":368},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/SKILL.md","title":"source-postgres-e2e-cdc-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-postgres-e2e-cdc-tests\ndescription: Reproduce PostgreSQL logical-decoding CDC behavior against a local PostgreSQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-postgres-e2e-tests and db-harness-lib.\n---\n\n# source-postgres-e2e-cdc-tests\n\nLocal CDC test harness for `source-postgres`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic PostgreSQL skill\n([`source-postgres-e2e-tests`](../source-postgres-e2e-tests/SKILL.md)). The\ngeneric skill starts a PostgreSQL 16 backend and runs the connector; this skill\nsupplies the logical-decoding CDC config, CDC catalog, SQL fixtures, and case\nscripts.\n\nPostgreSQL CDC uses logical decoding from a publication and replication slot.\nThe generic PostgreSQL backend starts with the WAL and replication settings\nrequired by this skill, so this skill does not start a second backend or add\nper-run server configuration.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-postgres` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new PostgreSQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-postgres-db-backend`, started by\n `../source-postgres-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-postgres-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with PostgreSQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # publication, slot, table, and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\nThe fixture uses the generic backend database `test_db`, because PostgreSQL\ncannot drop the database to which `apply-sql.sh` is connected. It resets the\n`users` table, publication, and logical replication slot in that database.\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic PostgreSQL backend starts with `wal_level=logical`,\n `max_replication_slots=10`, and `max_wal_senders=10`. Do not add per-run\n server configuration or start another backend in this skill.\n- `00-init-cdc.sql` drops and recreates the `users` table, publication, and\n `airbyte_slot` in `test_db` on every application. Re-applying it resets the\n fixture, which is why `cases/incremental-replay.sh` passes `--skip-fixtures`\n on its second phase.\n- PostgreSQL 3.8.5 discovers `_ab_cdc_lsn` as the source-defined cursor and\n does not include the newer generated catalog fields in discover output.\n The configured catalog includes `is_file_based`, generation metadata, and\n destination metadata because the pinned image requires those fields when\n initializing `read`.\n- Cases default to `VERSION=3.8.5`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-postgres-repro\n\n# Start the logical-decoding-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-postgres-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the logical-decoding-ready backend, CDC config, catalog metadata, and\ninitial load work together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved LSN\nresumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No PostgreSQL CDC fixtures\nor worked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep CDC setup in the\n configured database and do not add per-run server configuration.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`. Use the source-defined cursor\n emitted by `discover` for the pinned connector version.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.8.5}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses `psql` on stdin, so fixtures may contain\nmultiple statements. Logical replication settings are established when the\ngeneric backend starts; no additional CDC setup is needed in a case beyond the\npublication and replication slot in the SQL fixture.\n","frontmatter":{"name":"source-postgres-e2e-cdc-tests","description":"Reproduce PostgreSQL logical-decoding CDC behavior against a local PostgreSQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-postgres-e2e-tests and db-harness-lib."},"isInternal":false,"tokens":1874,"sizeBytes":7883},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":362},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/SKILL.md","title":"source-postgres-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-postgres-e2e-tests\ndescription: Stand up a local PostgreSQL 16 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-postgres: for ad-hoc end-to-end testing.\n---\n\n# source-postgres-e2e-tests\n\nLocal end-to-end test harness for `source-postgres`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the PostgreSQL backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a PostgreSQL 16 container named\n`source-postgres-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-postgres:` image via\n`airbyte-ops cloud connector regression-test`.\n\nThis skill supports standard (non-CDC) local sweeps. CDC fixtures and\nreplication setup are not yet stood up in this skill.\n\n## When to use this skill\n\n- Reproducing a `source-postgres` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-postgres:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-postgres-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run postgres:16 with logical replication\n│ ├── apply-sql.sh # docker exec psql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-postgres-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Postgres password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-postgres-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the PostgreSQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `postgres:16`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-postgres\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.8.5\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.8.4\n\n# One command only.\npoe e2e-local --command=read --test-version=3.8.5\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose\n`discover` failed earlier is reported `SKIPPED`, not `ERROR`. Under CI the\ntable is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to one\ncommand instead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the\n PostgreSQL container between sweeps. Use this only when matching database\n initialization state matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC is not included in this skill.** The lifecycle starts PostgreSQL 16\n with logical replication enabled so the composed\n [`source-postgres-e2e-cdc-tests`](../source-postgres-e2e-cdc-tests/SKILL.md)\n skill can use the same backend. CDC fixtures and case scripts live in that\n skill.\n- **PostgreSQL config shapes are specific.** `ssl_mode.mode` uses\n `prefer` for the local plaintext backend. The\n `replication_method.method` value for a standard sync is `Standard`.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-postgres-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n","frontmatter":{"name":"source-postgres-e2e-tests","description":"Stand up a local PostgreSQL 16 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-postgres: for ad-hoc end-to-end testing."},"isInternal":false,"tokens":2341,"sizeBytes":10099},{"name":"base.template.json","path":"airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json)","tokens":300,"sizeBytes":366},{"name":"utc-session.template.json","path":"airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json","title":"Configs Skill","category":"anthropic-skill","format":"json","content":"# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json)","tokens":300,"sizeBytes":380},{"name":"SKILL.md","path":"airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/SKILL.md","title":"source-snowflake-e2e-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: source-snowflake-e2e-tests\ndescription: Run the Airbyte protocol sweep for airbyte/source-snowflake: against the shared Snowflake integration account inside a per-run PROVEFIX_ schema (B3 remote-backend mode of db-harness-lib); prove-fix comparisons, cursor canary, orphan sweep.\n---\n\n# source-snowflake-e2e-tests\n\nRemote end-to-end harness for `source-snowflake`. The shared\n`db-harness-lib` orchestration runs protocol commands against a per-run schema\nin the shared Snowflake integration account.\n\n## Prerequisites\n\n- Docker, [`uv`](https://docs.astral.sh/uv/), and `jq`.\n- `gcloud` authenticated with read access to the GSM secret, or an existing\n `SNOWFLAKE_CONFIG_FILE`.\n- A clone of `airbytehq/airbyte`.\n\n## Layout\n\n```\nsource-snowflake-e2e-tests/\n├── SKILL.md\n├── ../../../poe_tasks.toml (connector-root entrypoint)\n├── scripts/\n│ ├── run.sh, sf.py\n│ ├── fetch-config.sh, render-config.sh\n│ ├── start-backend.sh, reset-databases.sh, stop-backend.sh\n│ ├── apply-sql.sh, sweep-orphans.sh\n├── fixtures/configs/base.template.json, utc-session.template.json\n├── fixtures/sql/00-cursor-canary-base.sql\n├── fixtures/sql/01-cursor-canary-boundary-row.sql\n└── cases/cursor-upper-bound.sh\n```\n\n## Isolation model\n\n- Every run uses `PROVEFIX_` and asserts that the schema is absent\n before both control and target runs.\n- A marker owned by the run permits reuse of the same schema between canary\n phases; unmarked pre-existing schemas are refused.\n- Control and target use the same schema with `--reset=fixture`; remote\n `--reset=backend` is rejected.\n- Teardown drops the schema in the run's exit trap.\n- `scripts/sweep-orphans.sh` lists or drops old `PROVEFIX_*` schemas, defaulting\n to four hours.\n\nThe default run id is\n`YYYYMMDDhhmmss__`, producing a\nschema-safe `PROVEFIX_` name.\n\nThe schema is owned by the scripts, while `sf.py` is the only code that talks\nto Snowflake. The default config comes from GSM secret\n`SECRET_SOURCE-SNOWFLAKE_KEY_PAIR__CREDS` in project\n`dataline-integration-testing`; fetched configs are marked with a sidecar and\nonly those configs are removed during teardown, so caller-supplied configs\nsurvive. Set `SNOWFLAKE_CONFIG_FILE` to reuse a caller-supplied config.\n\n## Cursor upper-bound canary\n\nThe canary proves the cursor precision regression from airbytehq/airbyte#82705:\n\n```bash\ncases/cursor-upper-bound.sh\n```\n\nIt compares fixed `1.1.1` with control `1.1.0` in two phases. Phase 1 runs the\ninitial sync on the control image with exact-microsecond base rows, producing\nthe checkpoint a user on the bad version would have. Phase 2 inserts a\nsub-microsecond boundary row and replays that state against both versions:\n1.1.0 loses the row because it floors the upper bound, while 1.1.1 emits it.\n\nThe harness reuses the `INTEGRATION_TEST_WAREHOUSE_DESTINATION` X-Small\nwarehouse shared with destination-snowflake integration tests. Never touch\nschemas outside `PROVEFIX_*`.\n\n## Gotchas\n\nSnowflake uppercases identifiers, so use\n`--cursor-field=UPDATED_AT --streams=CURSOR_CANARY`. Rendered configs contain\nthe private key; treat `REPRO_OUT` as 0600 material and never attach it.\nTIMESTAMP_NTZ cursor bounds are bound as TIMESTAMP_LTZ by the JDBC driver and\ncompared through the session TIMEZONE; with the account default\n(America/Los_Angeles), narrow incremental windows match nothing.\nThe harness sets `jdbc_url_params=TIMEZONE=UTC` so the canary isolates #82705;\nthis session-timezone behavior appears to be a separate connector bug worth\nits own issue.\nThe `--catalog` CDC guard (#85752) never fires here: source-snowflake has no\nCDC replication method.\n","frontmatter":{"name":"source-snowflake-e2e-tests","description":"Run the Airbyte protocol sweep for airbyte/source-snowflake: against the shared Snowflake integration account inside a per-run PROVEFIX_ schema (B3 remote-backend mode of db-harness-lib); prove-fix comparisons, cursor canary, orphan sweep."},"isInternal":false,"tokens":951,"sizeBytes":3745},{"name":"_category_.json","path":"docs/ai-agents/get-started/developer-quickstart/skills/_category_.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/_category_.json","title":"Skills Skill","category":"anthropic-skill","format":"json","content":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/_category_.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/_category_.json)","tokens":300,"sizeBytes":253},{"name":"claude-code.md","path":"docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md","title":"Skills Skill","category":"anthropic-skill","format":"markdown","content":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/claude-code.md)","tokens":300,"sizeBytes":251},{"name":"codex.md","path":"docs/ai-agents/get-started/developer-quickstart/skills/codex.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/codex.md","title":"Skills Skill","category":"anthropic-skill","format":"markdown","content":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/codex.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/codex.md)","tokens":300,"sizeBytes":239},{"name":"lovable.md","path":"docs/ai-agents/get-started/developer-quickstart/skills/lovable.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/lovable.md","title":"Skills Skill","category":"anthropic-skill","format":"markdown","content":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/lovable.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/lovable.md)","tokens":300,"sizeBytes":243},{"name":"readme.md","path":"docs/ai-agents/get-started/developer-quickstart/skills/readme.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/readme.md","title":"Skills Skill","category":"anthropic-skill","format":"markdown","content":"# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/readme.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/readme.md)","tokens":300,"sizeBytes":241},{"name":"admin-service-account.yaml","path":"tools/bin/admin-service-account.yaml","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/bin/admin-service-account.yaml","title":"Bin Skill","category":"anthropic-skill","format":"yaml","content":"apiVersion: rbac.authorization.k8s.io/v1\nkind: Role\nmetadata:\n name: airbyte-admin-role\nrules:\n - apiGroups: [\"*\"]\n resources: [\"jobs\", \"pods\", \"pods/log\", \"pods/exec\", \"pods/attach\"]\n verbs: [\"get\", \"list\", \"watch\", \"create\", \"update\", \"patch\", \"delete\"] # over-permission for now\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: RoleBinding\nmetadata:\n name: default-admin-binding\nroleRef:\n apiGroup: \"\"\n kind: Role\n name: airbyte-admin-role\nsubjects:\n - kind: ServiceAccount\n name: default\n","isInternal":false,"tokens":157,"sizeBytes":510},{"name":"run-harness-process.yaml","path":"tools/bin/run-harness-process.yaml","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/bin/run-harness-process.yaml","title":"Bin Skill","category":"anthropic-skill","format":"yaml","content":"# Bin Skill\n\nPath: `tools/bin/run-harness-process.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/bin/run-harness-process.yaml)","tokens":300,"sizeBytes":178},{"name":"README.md","path":"airbyte-integrations/connectors/source-commercetools/README.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-commercetools/README.md","title":"source-commercetools Documentation","category":"plugin-manifest","format":"markdown","content":"# source-commercetools Documentation\n\nPath: `airbyte-integrations/connectors/source-commercetools/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-commercetools/README.md)","tokens":300,"sizeBytes":259},{"name":"chatplugin.json","path":"airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json","title":"shared Plugin Manifest","category":"plugin-manifest","format":"json","content":"# shared Plugin Manifest\n\nPath: `airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-facebook-pages/source_facebook_pages/schemas/shared/chatplugin.json)","tokens":300,"sizeBytes":335},{"name":"README.md","path":"tools/internal/README.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/internal/README.md","title":"internal Documentation","category":"plugin-manifest","format":"markdown","content":"# internal Documentation\n\nPath: `tools/internal/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/internal/README.md)","tokens":300,"sizeBytes":171},{"name":"README.md","path":"tools/schema_generator/README.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/schema_generator/README.md","title":"schema_generator Documentation","category":"plugin-manifest","format":"markdown","content":"# schema_generator Documentation\n\nPath: `tools/schema_generator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/schema_generator/README.md)","tokens":300,"sizeBytes":195},{"name":"README.md","path":"tools/site/README.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/site/README.md","title":"site Documentation","category":"plugin-manifest","format":"markdown","content":"# site Documentation\n\nPath: `tools/site/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/tools/site/README.md)","tokens":300,"sizeBytes":159},{"name":"readme.md","path":"docs/release_notes/agents/readme.md","rawUrl":"https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/release_notes/agents/readme.md","title":"Subagent: readme","category":"subagent-persona","format":"markdown","content":"# Subagent: readme\n\nPath: `docs/release_notes/agents/readme.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/release_notes/agents/readme.md)","tokens":300,"sizeBytes":187}],"systemPromptSnippet":"\n\n---\nname: my-prs\ndescription: List PRs opened today that are assigned to me or the move-destinations team, excluding external contributors\ndisable-model-invocation: true\nallowed-tools: Bash(gh:*), Bash(date:*), Read\nargument-hint: [days-back]\n---\n\n# My PRs - Daily PR Review Dashboard\n\nThis skill fetches and displays PRs that need your attention:\n\n1. **PRs assigned directly to you** (@frifriSF59)\n2. **PRs with review requested from the move-destinations team**\n\nAll results are filtered to:\n- Only show PRs opened today (or within `$ARGUMENTS` days if specified)\n- Exclude PRs from users who are not members of the airbytehq organization\n\n## Execution\n\nRun the fetch script to get the PR list:\n\n```bash\nbash .claude/skills/my-prs/scripts/fetch-prs.sh\n```\n\nIf an argument is provided (number of days), modify the date filter accordingly:\n- `$ARGUMENTS` = number of days to look back (default: 0 = today only)\n- Example: `/my-prs 3` looks at PRs from the last 3 days\n\nFor custom date ranges, use:\n```bash\ngh pr list --repo airbytehq/airbyte --assignee @me --search \"created:>=YYYY-MM-DD\" --json number,title,author,url\n```\n\n## After Fetching\n\nAfter displaying the PRs, offer to:\n1. Open a specific PR in the browser (`gh pr view --web`)\n2. Show details of a specific PR (`gh pr view `)\n3. Show the diff of a specific PR (`gh pr diff `)\n4. Check CI status (`gh pr checks `)\n\n## Quick Actions\n\nThe user may want to:\n- **Review a PR**: Use `/review ` if available\n- **Check PR comments**: `gh pr view --comments`\n- **See who approved**: `gh pr view --json reviews --jq '.reviews[] | select(.state==\"APPROVED\") | .author.login'`\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-bigquery/.agents/skills/source-bigquery-e2e-tests/fixtures/configs/base.template.json)\n\n\n---\nname: source-bigquery-e2e-tests\ndescription: Run the Airbyte protocol smoke sweep for airbyte/source-bigquery: against the shared BigQuery integration project inside a per-run PROVEFIX_ dataset (B3 remote-backend mode of db-harness-lib), with a dry-run orphan sweep.\n---\n\n# source-bigquery-e2e-tests\n\nThis harness runs the source-bigquery protocol regression sweep against an isolated dataset in a real BigQuery project. Orchestration comes from `airbyte-integrations/db-harness-lib`; the engine scripts own the dataset, fixtures, and credentials.\n\n## Prerequisites\n\n- `gcloud` with ambient access to Secret Manager in `GSM_PROJECT`.\n- `bq` and `docker`.\n- `airbyte-ops` (or `uvx airbyte-internal-ops`).\n- A BigQuery service account with dataset create/delete, query, and source-read permissions in `BIGQUERY_PROJECT`.\n\nCloud Storage Read API coverage requires `airbyte-ops cloud connector regression-test --extra-env`, added in [airbyte-ops-mcp#349](https://github.com/airbytehq/airbyte-ops-mcp/pull/349). The shim defaults `BIGQUERY_AIRBYTE_EDITION` to `CLOUD` and adds `AIRBYTE_EDITION=CLOUD` when supported. Set `BIGQUERY_AIRBYTE_EDITION=''` to disable the check and injection. Older `airbyte-ops` versions warn and continue using the query API.\n\n## Run\n\nFrom this connector directory:\n\n```bash\npoe e2e-local --test-version=1.0.1\n```\n\nThe Poe task forwards the shared harness flags; use `poe e2e-local --help` for the full list. Common options include `--command=check`, `--skip-read`, `--step-name=NAME`, `--keep-backend`, and `--reset=fixture`. Environment overrides include `REPRO_OUT`, `GSM_PROJECT`, `BIGQUERY_PROJECT`, `BIGQUERY_LOCATION`, `BIGQUERY_SECRET_NAME`, `BIGQUERY_CONFIG_FILE`, `AIRBYTE_OPS`, and `BIGQUERY_AIRBYTE_EDITION`.\n\n## Dataset lifecycle\n\nEach run creates `PROVEFIX_` in `BIGQUERY_PROJECT` and records ownership in `$REPRO_OUT/.provefix-owned-PROVEFIX_`. An existing dataset is reused only with that marker; otherwise the harness refuses to touch it and points to `sweep-orphans.sh`. The default teardown drops the owned dataset and verifies it is absent. `--reset=fixture` drops and recreates the run dataset between comparison sweeps. `sweep-orphans.sh --dry-run` lists old `PROVEFIX_*` datasets without deleting them; `--older-than-hours=N` defaults to 4.\n\n`gcloud`/`bq` authentication uses `$REPRO_OUT/.gcloud`, separate from the ambient credentials used to fetch the GSM secret. The fetched service-account key and isolated auth directory are removed at teardown.\n\n## Expected smoke result\n\nBoth runs pass CHECK, DISCOVER `smoke_table` and `smoke_view`, and READ three records from each stream. The installed CLI warns when it lacks `--extra-env`, so that run exercises only the query API. With `BIGQUERY_AIRBYTE_EDITION=CLOUD` and the #349 CLI, connector logs show `smoke_table` opening a Storage Read API session and `smoke_view` falling back to the query API because views are unsupported by the Storage Read API.\n\nThere is no bad-vs-fixed proof case yet: the 1.0.1 views fallback requires a service account without BigQuery Read Session User, and none exists in GSM.\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/configs/base.template.json)\n\n\n# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/00-init-base.json)\n\n\n# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/attribute-types.json)\n\n\n# Json Skill\n\nPath: `airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-dynamodb/.agents/skills/source-dynamodb-e2e-tests/fixtures/json/empty-table.json)\n\n\n---\nname: source-dynamodb-e2e-tests\ndescription: Stand up a local DynamoDB Local backend, apply JSON fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-dynamodb: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for a cursor-incremental source-dynamodb test or prove-fix comparison.\n---\n\n# source-dynamodb-e2e-tests\n\nLocal end-to-end test harness for `source-dynamodb`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the DynamoDB Local backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a DynamoDB Local container named\n`source-dynamodb-db-backend`, applies JSON fixtures with the AWS CLI, and runs\nAirbyte protocol commands against any `airbyte/source-dynamodb:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: no CDC mode exists; incremental reads use a user-defined cursor\nattribute. Assume-role / IAM / STS behavior is not covered: DynamoDB Local\ndoes not provide STS, and the connector's `endpoint` only overrides the\nDynamoDB client, so a published connector image cannot send AssumeRole\nrequests to this emulator. State this limitation in the evidence plan rather\nthan claiming coverage.\n\n## When to use this skill\n\n- Reproducing a `source-dynamodb` bug locally without AWS credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-dynamodb:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need AWS, IAM, or Cloud credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-dynamodb-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── aws-cli.sh # AWS CLI container and backend bridge-IP helpers\n│ ├── start-backend.sh # docker run DynamoDB Local with -sharedDb\n│ ├── apply-sql.sh # delete/create tables and put fixture items\n│ ├── reset-databases.sh # delete every table\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # user credentials, region, endpoint placeholder\n └── json/\n ├── 00-init-base.json # sample with a numeric cursor and three items\n ├── attribute-types.json\n └── empty-table.json\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but DynamoDB\nfixtures are JSON documents, not SQL. Each fixture has a `tables` array. Each\ntable entry contains a `create` object accepted by `CreateTable\n--cli-input-json` and an `items` array in DynamoDB attribute-value JSON:\n\n```json\n{\n \"tables\": [\n {\n \"create\": {\n \"TableName\": \"sample\",\n \"KeySchema\": [{\"AttributeName\": \"id\", \"KeyType\": \"HASH\"}],\n \"AttributeDefinitions\": [{\"AttributeName\": \"id\", \"AttributeType\": \"S\"}],\n \"BillingMode\": \"PAY_PER_REQUEST\"\n },\n \"items\": [{\"id\": {\"S\": \"e1\"}, \"seq\": {\"N\": \"1\"}}]\n }\n ]\n}\n```\n\n## Conventions\n\n- Container name: `source-dynamodb-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The config uses dummy `User` credentials and\n `us-east-1`; DynamoDB Local does not authenticate them.\n- `-sharedDb` is required. Without it, DynamoDB Local namespaces tables by\n access key and region, so tables created by the fixture client can be\n invisible to the connector client.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-dynamodb-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `.endpoint` to `http://:8000` at runtime. The AWS CLI\n helper resolves the same bridge IP for its endpoint.\n- DynamoDB Local is pinned to `amazon/dynamodb-local:3.3.2`. Override with\n `BACKEND_IMAGE=…` for a version-specific reproduction. The AWS CLI helper\n image is pinned to `amazon/aws-cli:2.38.2`; override with\n `AWS_CLI_IMAGE=…`.\n- `attribute-types.json` exercises DynamoDB attribute-value families and a\n composite-key table. The connector's source-defined primary key is the\n partition key only; its sort key is not part of that primary key.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-dynamodb\n\n# Single version, full sweep.\npoe e2e-local --test-version=0.4.1\n\n# Target vs. control comparison.\npoe e2e-local --test-version=0.4.1 --control-version=0.3.11 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=0.4.1\n```\n\nThe sweep runs every command against the backend and reports each result\nrather than stopping at the first failure, then prints a summary table. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **DynamoDB key attributes.** `AttributeDefinitions` contains only the\n table's key attributes; non-key attributes such as `seq` and `label` appear\n in fixture items and are discovered by sampling them.\n- **Sampled attributes.** Discovery samples table items and unions the\n attributes it sees. Keep at least one item with each field you want in the\n catalog. A field missing from an item is a useful missing-attribute case,\n but it may not be emitted if it is absent from every sampled item.\n- **Configured-catalog nulls.** DynamoDB discover omits `namespace`,\n `source_defined_cursor`, and `is_resumable`, and 0.4.0 accepts them absent;\n it rejects `namespace: null`. Keep them omitted through an Airbyte Ops build\n that serializes configured catalogs with `exclude_none=True`; no shared\n catalog-generator defaults are needed.\n- **Empty tables.** The connector discovers key attributes from\n `DescribeTable` when there are no items to sample, so an empty table remains\n in the catalog with its key fields.\n- **Comparison catalogs.** Use `--reset=fixture` for comparisons where\n discovery output changes between image tags. In particular, for the\n integer-cursor regression below, the old tag discovers `seq` as\n `[\"null\",\"integer\"]` while the fixed tag discovers it as `number` with\n `airbyte_type: integer`. A paired `--reset=none` run can reuse the target's\n discovered catalog for the control read and mask the control bug, because\n the crash is driven by the cursor's catalog JSON-schema type. Fixture reset\n runs separate sweeps, so each image derives its own catalog.\n- **No role coverage.** Do not use DynamoDB Local to claim assume-role, IAM,\n or STS coverage. There is no STS emulator here, and `endpoint` only\n overrides the DynamoDB client endpoint.\n- **Do not use real AWS.** This harness is for local testing only and must\n never be used against customer AWS accounts, tables containing customer\n data, or Airbyte Cloud.\n\n## Worked example: integer cursor, airbyte#71171\n\nThe historical integer-cursor crash can be proven with the base fixture. The\ncontrol image fails with\n`IllegalArgumentException: No enum constant\nio.airbyte.protocol.models.JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER`\nand emits no records; the fixed image reads all three records and emits\n`STATE` with `cursor: \"3\"`.\n\nUse `--reset=fixture` so the 0.3.11 control and 0.4.0 target each run their\nown `discover` and `read`. The versions discover different cursor schemas,\nand the crash depends on that schema in the configured catalog.\nThis command also requires an Airbyte Ops build with the configured-catalog\n`exclude_none=True` fix: DynamoDB discovery omits `namespace`, and older Ops\nbuilds serialize that omission as `namespace: null`, which 0.4.0 rejects.\n\n```bash\nREPRO_OUT=\"$HOME/dynamodb-hyd310-evidence/patched-ops-1792531195\" \\\n poe e2e-local \\\n --step-name=integer-cursor-71171 \\\n --control-version=0.3.11 \\\n --test-version=0.4.0 \\\n --reset=fixture \\\n --sync-mode=incremental \\\n --cursor-field=seq \\\n --streams=sample \\\n --expect-control=fail \\\n --expect-test=pass \\\n --min-records=3 \\\n --min-states=1 \\\n --forbid-match=read:any:JsonSchemaPrimitiveUtil.JsonSchemaPrimitive.INTEGER\n```\n\nVerified on 2026-10-20 with the Airbyte Ops null-namespace fix:\n\n| Command | Control (`0.3.11`) | Target (`0.4.0`) |\n|---------|---------|---------|\n| `SPEC` | PASS | PASS |\n| `CHECK` | PASS | PASS |\n| `DISCOVER` | PASS | PASS |\n| `READ` | FAIL — integer-enum crash, 0 records, 0 states | PASS — 3 records, 1 state (`cursor: \"3\"`) |\n\nThe paired command exited 0: the control reproduced the historical failure,\nwhile the target passed and emitted the expected records and state.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-dynamodb-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mongodb-v2/.agents/skills/source-mongodb-v2-e2e-tests/fixtures/configs/base.template.json)\n\n\n---\nname: source-mongodb-v2-e2e-tests\ndescription: Stand up a local MongoDB 7.0 single-node replica set, apply mongosh JavaScript fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mongodb-v2: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any non-CDC source-mongodb-v2 test or prove-fix comparison.\n---\n\n# source-mongodb-v2-e2e-tests\n\nLocal end-to-end test harness for `source-mongodb-v2`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MongoDB backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MongoDB 7.0 container named\n`source-mongodb-v2-db-backend` configured as a single-node replica set\n(`rs0`), lets you apply arbitrary `mongosh` JavaScript fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mongodb-v2:` image\nvia `airbyte-ops cloud connector regression-test`.\n\nScope: non-CDC (full-refresh / initial-snapshot) behavior. Change-stream\nresume-token replay and CDC comparison flows are not covered by this skill.\n\n## When to use this skill\n\n- Reproducing a `source-mongodb-v2` bug locally without Atlas credentials.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of them,\n against any `airbyte/source-mongodb-v2:` for connector development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM, Atlas, or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mongodb-v2-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mongo:7.0 --replSet rs0; rs.initiate()\n│ ├── apply-sql.sh # docker exec mongosh with a .js fixture on stdin\n│ ├── reset-databases.sh # drop every non-system database\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # SELF_MANAGED_REPLICA_SET config; host placeholder\n └── js/\n └── 00-init-base.js # test_db.sample with three documents\n```\n\n`apply-sql.sh` keeps the db-harness-lib engine entrypoint name, but MongoDB\nfixtures are `mongosh` scripts, not SQL. Each fixture runs with `test_db`\n(`BACKEND_DB`) as the current `db`; use `db.getSiblingDB(...)` for others.\n\n## Conventions\n\n- Container name: `source-mongodb-v2-db-backend`. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- No authentication. The backend runs without `--auth`; the config's\n `database_config` omits `username`/`password`. Add them to a copied\n template if you need to reproduce an auth-specific issue.\n- Replica set: `rs0` (`BACKEND_REPLSET`). The connector requires a replica\n set even for non-CDC reads. `start-backend.sh` registers the single member\n under the container's bridge IP so the connector container, which performs\n replica-set discovery from the seed list, can reach the advertised host.\n Do not change the member host to `localhost`: the connector would then\n fail to connect.\n- Initial database: `test_db`. Override with `BACKEND_DB`; keep\n `database_config.databases` in the config template in sync.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mongodb-v2-repro}`, laid out as\n `$REPRO_OUT//{config.json,configured_catalog.json,/}`.\n- Both containers share Docker's default `bridge` network. The engine shim\n sets `CONFIG_HOST_JQ` so the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n rewrites `database_config.connection_string` to\n `mongodb://:27017/?replicaSet=rs0` at runtime.\n- The backend image is pinned to `mongo:7.0`. Override with `BACKEND_IMAGE=…`\n for a version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mongodb-v2\n\n# Single version, full sweep.\npoe e2e-local --test-version=2.0.7\n\n# Target vs. control comparison.\npoe e2e-local --test-version=dev --control-version=2.0.6 --reset=fixture\n\n# One command only.\npoe e2e-local --command=read --test-version=2.0.7\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. Under CI the table is also\nappended to `$GITHUB_STEP_SUMMARY`.\n\nPrefer a published target image for code on a pushed PR branch (publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry` and pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`). See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image).\nAll other `run.sh` options (`--command`, `--fixture`, `--skip-fixtures`,\n`--config-template`, `--catalog`, `--state=PATH`, `--reset`, `--expect-*`,\n`--min-records`, …) are documented in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md) and\n`poe e2e-local --help`.\n\n## Common gotchas\n\n- **Schema enforcement.** The base template sets\n `database_config.schema_enforced: true`, so `discover` samples documents\n and emits typed fields. Set it to `false` in a copied template to\n reproduce schemaless (`data` blob) behavior.\n- **`_id` types.** Fixtures use integer `_id`s for readability; use\n `ObjectId()` in a fixture when the bug depends on ObjectId serialization.\n- **Schema inference needs documents.** An empty collection created with\n `db.createCollection()` is still listed by `discover`, but with\n `schema_enforced: true` its schema has no sampled fields beyond `_id`.\n Insert at least one document in the fixture for every field you expect\n in the catalog.\n- **Reset drops databases, not the replica set.** `--reset=fixture` removes\n all non-system databases and reapplies fixtures; the oplog and resume\n tokens survive. This only matters once change-stream (CDC) coverage\n lands (HYD-147 phase 2); non-CDC sweeps do not read the oplog. Use\n `--reset=backend` when a run must start from a fresh oplog.\n- **Databases are created on first write.** MongoDB has no `CREATE\n DATABASE`; `test_db` exists only once a fixture inserts into it, so a\n fixture that only calls `db.createCollection(...)` or writes to a\n `getSiblingDB(...)` database leaves `test_db` absent and `discover` empty.\n `00-init-base.js` inserts into `test_db`; keep that in any fixture set you\n pass with `--fixture`.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections, Atlas clusters\n holding customer data, or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mongodb-v2-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n\n\n# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/order-items-cdc.json)\n\n\n# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json)\n\n\n---\nname: source-mssql-e2e-cdc-tests\ndescription: Reproduce CDC bugs against source-mssql by enabling CDC on the local SQL Server backend, applying per-bug SQL fixtures, and running the connector via airbyte-ops. Composes on top of source-mssql-e2e-tests. Worked examples for airbytehq/oncall#11451, #12094, and #12162 ship with the skill.\n---\n\n# source-mssql-e2e-cdc-tests\n\nLocal CDC bug-reproduction harness for `source-mssql`. Builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe MSSQL engine skill\n([`source-mssql-e2e-tests`](../source-mssql-e2e-tests/SKILL.md)):\ngeneric skill stands up the SQL Server backend and runs the connector;\nthis skill adds CDC enable, CDC-aware config / catalog templates, and\nper-bug **case scripts** under `cases/` that call `run.sh` with the\nappropriate fixtures and `--expect-*` assertions.\n\n## When to use this skill\n\n- Reproducing a CDC-mode bug against `source-mssql` locally.\n- Verifying a fix by re-running an existing case with\n `VERSION=dev ./cases/.sh` (after `:dockerBuildx`).\n- Verifying that a fix which _loosens_ CDC offset validation still\n rejects genuinely invalid state — `cases/11451.sh` is the\n invalid-state case for LSN availability; see\n [Invalid-state case](#invalid-state-case-for-lsn-availability-fixes).\n- Authoring a new repro for a customer-reported CDC bug. Drop a SQL\n fixture under `fixtures/sql/` and a `cases/.sh` script\n that invokes `run.sh` with the right fixtures and `--expect-*`\n assertions.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, a clone of `airbytehq/airbyte`.\n- The backend container `source-mssql-db-backend` started by\n `../source-mssql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials.\n\n## Layout\n\n```\nsource-mssql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── 11451.sh # airbytehq/oncall#11451 — LSN-range regression in 4.3.4+ (multi-phase; invalid-state case)\n│ ├── 12094.sh # airbytehq/oncall#12094 — schema-history bloat\n│ └── 12162.sh # airbytehq/oncall#12162 — whitespace in stream name\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json\n ├── catalogs/\n │ ├── users-cdc.json\n │ └── order-items-cdc.json\n └── sql/\n ├── 00-init-cdc.sql\n ├── repro-11451-lsn-cleanup.sql\n ├── repro-12094-schema-history.sql\n └── repro-12162-spaces-in-name.sql\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit walks Airbyte STATE messages, which is protocol-level and not\nCDC-specific.\n\n## Conventions\n\n- Cases keep the backend up across invocations (`--keep-backend`).\n Start the backend once at the top of a session, run any subset of\n cases, tear down when done — see the Usage section below.\n- Fixtures are re-applied at the start of each `run.sh` invocation, so\n they must be idempotent (`00-init-cdc.sql` uses `IF DB_ID(...) IS NULL`\n guards, per-bug fixtures use similar shape). Re-applying a fixture\n during a multi-phase case does not reset the CDC state that the\n previous phase established.\n- Configured catalogs populate the bulk-CDK-required fields\n (`is_file_based`, `generation_id`, `minimum_generation_id`,\n `sync_id`, `destination_object_name`, `include_files`) so the same\n fixtures drive repros across `source-mssql:4.3.x` and `4.4.x`.\n- Cases default `VERSION=4.4.2` (except `11451.sh` which splits into\n `BASELINE_VERSION`/`TARGET_VERSION`). Override with\n `VERSION=4.3.4 ./cases/12162.sh` to test against an earlier version,\n or `VERSION=dev` after a local\n `./gradlew :airbyte-integrations:connectors:source-mssql:dockerBuildx`\n to test a fix.\n- **Assertions** are declarative via `run.sh`'s expectation flags:\n `--expect-test`, `--expect-match=[:]:[:N]`,\n `--forbid-match`, `--min-records`, `--min-states`. The runner\n enforces them and exits non-zero on any failure. The `:`\n prefix defaults to `read` when omitted; set it explicitly\n (`check:stderr:…`, `discover:stderr:…`) for check-time or\n discover-time signatures.\n- Multi-phase cases (`11451.sh`, and any future read → mutate →\n read-with-state repros) capture Airbyte STATE messages between reads\n via the shared\n [`extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\n (`uv`-PEP-723 standalone; pass `` as its argument or pipe\n stdin) and feed the file back into the second read with\n `run.sh --state=PATH`.\n Use `--step-name=/` to give each phase its own artifact\n subtree.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mssql-repro\n\n# 1. Bring up the backend (once per session). Cases pass --keep-backend,\n# so they don't tear it down between runs.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# 2. Run any case. Each case's --expect-* flags gate its own exit code\n# (non-zero on any expectation failure), so bash's `set -e` will fail\n# the sequence on the first failing case if you chain them.\n\"$SKILL/cases/12162.sh\"\n\"$SKILL/cases/12094.sh\"\n\"$SKILL/cases/11451.sh\"\n\n# 3. (After fix) verify by retargeting `dev` or a fixed version.\nVERSION=dev \"$SKILL/cases/12162.sh\"\n\n# 4. Tear down.\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\nrm -rf \"$REPRO_OUT\"\n```\n\nEach case is a shell script that composes `run.sh` invocations:\n\n1. Applies its SQL fixtures (via `--fixture=…`; `run.sh` calls\n `apply-sql.sh` internally).\n2. Renders the config against the running backend (via\n `--config-template=…`).\n3. Runs the target protocol command with any `--catalog` / `--state`.\n4. Gates its exit code on `--expect-*` flags — the runner enforces\n them and exits non-zero on any failure, so the case script has no\n inline assertion logic to maintain.\n\nMulti-phase cases (`11451.sh`) call `run.sh` more than once, with\n`extract-state.py` and `apply-sql.sh` between invocations. Each phase\nuses `--step-name=/` to keep its artifacts separate under\n`$REPRO_OUT///`.\n\n## Worked examples\n\n### airbytehq/oncall#12162 — whitespace in stream name\n\n`cases/12162.sh`. Creates `dbo.[Order Items]` (note the space), enables\nCDC on it, and runs `read` against it. Case asserts\n`--expect-test=fail`, `--expect-match=stderr:io.debezium.DebeziumException`,\nand `--expect-match=stderr:message.key.columns` — i.e. the read exits\nnon-zero and Debezium's `stderr.txt` contains its\n`Connector configuration is not valid. The 'message.key.columns' value\nis invalid` rejection.\n\nRoot cause: `MsSqlServerDebeziumOperations.buildMessageKeyColumns()`\njoins `schema.table:pkcol` strings without filtering or escaping\nidentifiers that contain whitespace or `:`, and Debezium rejects them\nat engine startup. Fix: pre-filter such streams (Debezium falls back\nto the table's native PK from system tables).\n\n### airbytehq/oncall#12094 — schema-history bloat\n\n`cases/12094.sh`. Creates 30 noise tables in `dbo` (CDC not enabled on\nthem), runs the baseline `read` against `dbo.users`, and asserts\n`--expect-match='stdout:Adding table CdcTest\\..* to the list of capture schema tables:30'`\n(the trailing `:30` is the count threshold; the fixture creates 30\nnoise tables). Adjust with `MIN_LOADED=`. Expected outcome: at\nleast 30 \"Adding table\" lines even though the configured catalog has\na single stream.\n\nRoot cause: `withSchemaHistory()` in the bulk-CDK Debezium properties\nsets `schema.history.internal.store.only.captured.databases.ddl=true`\nbut not `…captured.tables.ddl=true`. Same harness reproduces equivalent\nbehavior on `source-mysql` and `source-postgres` because the property\nlives in the bulk-CDK `extract-cdc` toolkit.\n\n### airbytehq/oncall#11451 — LSN-range regression in 4.3.4+\n\n`cases/11451.sh`. Multi-phase:\n\n1. Baseline `read` on clean CdcTest (`BASELINE_VERSION`, default `4.4.2`).\n Case asserts `--expect-test=pass --min-states=1` so a missing STATE\n fails the case immediately rather than later at replay.\n2. `extract-state.py` on the baseline stdout, then\n `apply-sql.sh repro-11451-lsn-cleanup.sql` runs\n `sys.sp_cdc_cleanup_change_table`\n with a `low_water_mark` past the saved LSN — advancing\n `fn_cdc_get_min_lsn('dbo_users')` past the baseline offset.\n3. Replay `read` on `TARGET_VERSION` (default `4.3.4`) with the stale\n state. Case asserts `--expect-test=fail`,\n `--expect-match=stderr:Saved offset no longer present`, and\n `--expect-match='stderr:is no longer available in SQL Server transaction logs'`.\n\nRoot cause: pre-`4.3.4` the LSN-range query computed `min` via\n`sys.fn_cdc_get_min_lsn('')`, which always returns\n`0x00000000000000000000`. `4.3.4` swapped to\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance)) FROM cdc.change_tables`,\nwhich returns the actual per-instance min — a value that can advance\npast a saved offset on geo-replicas with aggressive cleanup. The\nsaved-offset-rejection guard then fires even though the data is\nstill present. Investigation lives at\n[`airbytehq/oncall#11451`](https://github.com/airbytehq/oncall/issues/11451).\n\n#### Invalid-state case for LSN-availability fixes\n\nThis case's Phase 3 is a _genuinely_ expired saved offset: Phase 2\nadvances `fn_cdc_get_min_lsn` past the captured LSN, so rejecting the\nreplay is the correct behavior, not the bug. That makes it the\ninvalid-state counterpart to any repro built around a saved LSN that is\nstill valid — for example an unauthorized capture instance returning\n`0x0` and poisoning\n`MIN(sys.fn_cdc_get_min_lsn(capture_instance))`.\n\nSo when a change relaxes what the offset-availability check aggregates\nor tolerates, run this case against the image built from that change:\n\n```bash\nTARGET_VERSION=dev \"$SKILL/cases/11451.sh\"\n```\n\nIt must still fail with the same two `stderr` signatures the case\nasserts on. A pass here means the guard stopped firing on state it is\nsupposed to reject, which turns an actionable \"reset the connection\"\nerror into a silently incomplete sync — the symptom-gone evidence from\nthe valid-LSN repro cannot distinguish that from a correct fix.\n\n## Authoring a new repro\n\n1. Drop a SQL fixture in `fixtures/sql/repro--.sql`.\n Make it idempotent (`IF NOT EXISTS` guards for `CREATE`,\n `IF EXISTS` guards for `DROP`) — cases re-apply fixtures on every\n `run.sh` invocation.\n2. (Optional) Drop a CDC-aware catalog in `fixtures/catalogs/`.\n3. Drop a case script in `cases/.sh` modeled on the\n existing ones. `set -euo pipefail`; default `VERSION=4.4.2` with a\n `${VERSION:-…}` override; invoke `run.sh` with the right\n `--command`, `--fixture=`, `--config-template=`, `--catalog=`, and\n `--expect-*` flags. Pass `--keep-backend` so the shared-session\n Usage flow works. For check-time or discover-time signatures, use\n `--expect-match=check:stderr:…` / `--expect-match=discover:stderr:…`\n (the `:` prefix defaults to `read` when omitted).\n4. Verify locally:\n ```bash\n \"$GENERIC/scripts/start-backend.sh\"\n \"$SKILL/cases/.sh\"\n BACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\"\n ```\n5. Note the worked example in this `SKILL.md`'s \"Worked examples\"\n section with: customer-symptom one-liner, `--expect-*` assertions\n the case gates on, root-cause one-liner.\n\n### Multi-phase repros\n\nWhen the reproduction is single-phase (one `run.sh` invocation is\nenough to trigger the bug), follow the recipe above. Reach for a\nmulti-phase shape only when the bug needs an intermediate mutation\nthat depends on state from a first read — the canonical example is\nread → extract STATE → mutate the server based on that STATE → replay\nwith the stale STATE. Model on [`cases/11451.sh`](cases/11451.sh) —\nthe three primitives it composes are:\n\n- **`--step-name=/`** — per-phase artifact dirs under\n `$REPRO_OUT//`. `cases/11451.sh` uses `11451/baseline` and\n `11451/stale`; a new case would use its own bug number and phase\n names.\n- **`extract-state.py`** (implemented in `db-harness-lib`) — reads a phase's\n `read/stdout.txt` and emits a JSON array of AirbyteStateMessage\n objects, which is what `run.sh --state=PATH` expects.\n- **`--skip-fixtures`** on the second/later `run.sh` invocation —\n opts out of the initial-fixture reapply so the intermediate state\n the previous phase established survives. Otherwise `run.sh`'s\n default fixture-application would re-run `00-init-cdc.sql`, which\n drops and recreates `CdcTest` and wipes the mutation. Rejects\n `--fixture=` in the same invocation (either apply fixtures or skip\n them).\n\nThe between-phase mutation itself is a plain `apply-sql.sh` call\nagainst a fixture in `fixtures/sql/`. Nothing about the mutation is\nspecial-cased in `run.sh` — the case script drives the sequencing.\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests/fixtures/configs/base.template.json)\n\n\n---\nname: source-mssql-e2e-tests\ndescription: Stand up a local SQL Server 2022 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mssql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mssql test or repro that is not CDC-specific.\n---\n\n# source-mssql-e2e-tests\n\nLocal end-to-end test harness for `source-mssql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../db-harness-lib/);\nthis skill keeps the MSSQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a SQL Server\n2022 container named `source-mssql-db-backend`, lets you apply arbitrary\nSQL fixtures, and runs Airbyte protocol commands against any\n`airbyte/source-mssql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a non-CDC bug against `source-mssql` locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mssql:` for connector\n development.\n- As the building block for the\n [`source-mssql-e2e-cdc-tests`](../source-mssql-e2e-cdc-tests/SKILL.md)\n skill, which adds CDC enable plus worked-example fixtures on top.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install airbyte-internal-ops`\n puts `airbyte-ops` on `$PATH`; alternatively prefix every call with\n `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mssql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mcr…/mssql/server:2022-latest as source-mssql-db-backend\n│ ├── apply-sql.sh # docker cp + docker exec sqlcmd -i\n│ ├── reset-databases.sh # docker exec sqlcmd → drop every non-system database (used by run.sh --reset=fixture)\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # non-CDC config; host=mssql-db-backend placeholder\n └── sql/\n ├── .gitignore # `.tmp/` — subtree for uncommitted per-bug scratch fixtures\n └── 00-init-base.sql # CREATE DATABASE TestDb + dbo.sample table\n```\n\nThe engine shim and backend lifecycle scripts remain in this skill's\n`scripts/` directory; the shared orchestration, default teardown, and\nprotocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mssql-db-backend`. Hard-coded in\n `scripts/start-backend.sh` and `apply-sql.sh`. The shared\n `db-harness-lib/scripts/stop-backend.sh` removes this container by\n `BACKEND_NAME`; the engine shim supplies the default. Override via\n `BACKEND_NAME=…` only for parallel test isolation; don't use customer\n connection names.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mssql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per\n protocol command (`spec/`, `check/`, `discover/`, `read/`) holding that\n command's `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n This is the same layout the ops repo's `connector-regression-test.yml`\n workflow uses under `/tmp/regression_test_artifacts`, so CI can upload\n it per command.\n- Both containers (the SQL Server backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765);\n once `--network` is supported the bridge-IP dance can collapse.\n- SQL Server image is pinned to `mcr.microsoft.com/mssql/server:2022-latest`,\n not `latest`, for stable major-version behavior across CU patches.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover`'s output, run `read`, tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mssql\n\n# Single version, full sweep.\npoe e2e-local --test-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, non-CDC (airbyte-ops's built-in comparator).\npoe e2e-local --test-version=dev --control-version=5.0.0 \\\n --fixture=.agents/skills/source-mssql-e2e-tests/fixtures/sql/00-init-base.sql\n\n# Target vs. control comparison, CDC (per-image reset between the two runs).\npoe e2e-local --test-version=dev --control-version=5.0.0 --reset=fixture \\\n --config-template=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/configs/cdc.template.json \\\n --sync-mode=incremental --cursor-field=_ab_cdc_cursor --streams=users \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/00-init-cdc.sql \\\n --fixture=.agents/skills/source-mssql-e2e-cdc-tests/fixtures/sql/.sql\n\n# One command only.\npoe e2e-local --command=read --test-version=5.0.0\n```\n\nThe CDC example needs an incremental catalog (`--sync-mode=incremental\n--cursor-field=_ab_cdc_cursor`) restricted with `--streams` to exclude\n`dbo.systranschemas`; the general rule is in the\n[db-harness-lib README](../../../../../db-harness-lib/README.md#cdc-config-templates-need-an-incremental-catalog).\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed — separating an\ninfrastructure failure (`ERROR`, no verdict was produced) from a failed\ntest verdict (`FAIL`), as the ops workflow's final-status step does. A\n`read` whose `discover` failed earlier in the sweep is reported `SKIPPED`,\nnot `ERROR` — an invalid-config case is expected to fail `discover`, and\nthere is then no catalog for `read` to use. Under\nCI the table is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to\none command instead exits with the connector's own exit code, so a repro\ncan still assert on it.\n\nPrefer a published tag over `--test-version=dev`: for code on a pushed PR\nbranch, publish a pre-release and pass its `-preview.` tag,\nwhich skips the Gradle build and gives reviewers a tag they can re-run\nagainst — see\n[Getting a target image](../../../AGENTS.md#getting-a-target-image). When\nyou do use `--test-version=dev`, build the target image first, or pass\n`--build` to have `run.sh` run `:dockerBuildx` for you. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures` (run against whatever state the backend already has;\nfor the second/later `run.sh` invocation of a multi-phase driver, when\nre-applying the initial fixture would wipe intermediate state),\n`--step-name`, `--catalog` (skip discover-derived generation),\n`--state=PATH` (pass a saved STATE file to the read as `--state-path`;\nfor use by multi-phase drivers), `--sync-mode=incremental`,\n`--cursor-field`, `--streams`, `--config-template`,\n`--reset=none|fixture|backend`, `--keep-backend`, and `--` to forward\nextra args to `airbyte-ops`. Per-command timeouts match the workflow's\n(30/30/60/180 minutes) and are overridable with\n`TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n### Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst each command's artifacts, `run.sh` accepts a small set of\nexpectation flags that it enforces itself before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `--expect-test=pass\\|fail` | Overall target verdict. `pass` = every executed command's status was `pass`; `fail` = one or more failed. |\n| `--expect-control=pass\\|fail` | Same for the control sweep (comparison-mode runs only; requires `--control-version`). |\n| `--min-records=N` | Target read's `stdout.txt` must contain ≥N `RECORD` messages. |\n| `--min-states=N` | Target read's `stdout.txt` must contain ≥N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target's `` step's `` (`stdout` \\| `stderr` \\| `any`) must match `` at least N times (default 1). `` (`spec` \\| `check` \\| `discover` \\| `read`) is optional and defaults to `read`. Repeatable. |\n| `--forbid-match=[:]:` | Same shape but must match zero times. Repeatable. |\n\nAll match assertions run against the **target-side** artifacts of the\nnamed command (or `read` if no `` prefix). Under\n`--control-version + --reset=none` those live at\n`$ARTIFACTS_DIR//target/` (created by airbyte-ops's comparison\nmode); under `--reset=fixture|backend` at\n`$ARTIFACTS_DIR/target//`; under single-version at\n`$ARTIFACTS_DIR//`. `run.sh` picks the right path automatically.\n\nThe match-spec grammar disambiguates the leading `:` from the\n`:` on the first colon-separated field (command names and\nchannel names don't overlap), and strips a trailing `:N` only when the\nlast field is a positive integer. So a regex containing `:` still\nparses: `--expect-match=stderr:io.debezium.DebeziumException` reads as\ncommand `read` (default), channel `stderr`, regex\n`io.debezium.DebeziumException`, count 1. `--expect-match=check:stderr:should be positive`\nreads as command `check`, channel `stderr`, regex `should be positive`,\ncount 1.\n\n`--min-records` and `--min-states` are read-step only — they count\nRECORD / STATE envelopes, which are read-specific — and have no\n`:` prefix.\n\nAny expectation failure exits non-zero regardless of the command-level\nverdicts and appends an `**Expectation failures:**` section to the\nsummary. Migrating an existing driver script's `grep -q '' || exit 1`\nblock is straightforward — drop the block and add\n`--expect-match=stderr:X` to the shared `run.sh` invocation.\n\n### Multi-phase drivers\n\n`run.sh` runs one sweep. A driver script composes multi-phase flows\n(read → mutate → read-with-state) by calling `run.sh` more than once\nwith `--step-name` for each invocation and `--state=` to feed the\nsecond read the state extracted from the first. `extract-state.py`\nlives in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py)\nbecause it walks Airbyte STATE messages, which is a protocol-level\noperation and not CDC-specific.\n\nThe CDC skill's [`cases/11451.sh`](../source-mssql-e2e-cdc-tests/cases/11451.sh)\nis a worked example: it invokes the engine shim's `run.sh` around an\nintermediate `apply-sql.sh` mutation and the shared\n`db-harness-lib/scripts/extract-state.py` step, and asserts on the\nconnector's stderr. New multi-phase drivers can follow the same shape\nbut call `run.sh --state=PATH` directly instead of smuggling\n`--state-path` through trailing args.\n\n### Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` once per\n protocol command with both `--test-image` and `--control-image`, so\n the two images run sequentially against a single backend and\n `airbyte-ops`'s built-in comparators emit the target-vs-control diff\n (record counts, primary keys, per-record, schema). Right for non-CDC\n full-refresh work: the diff is meaningful and the shared-backend\n assumption is safe because a full-refresh read does not mutate the\n upstream. This is the current behavior for any caller that doesn't\n set `--reset`.\n- **`--reset=fixture`** — `run.sh` runs the whole sweep against the\n control image first, drops every non-system database, re-applies the\n fixtures, then runs the sweep against the target image. Two\n single-version `airbyte-ops` calls, no built-in comparator; artifacts\n land under `$REPRO_OUT//{control,target}//`.\n Right for **CDC comparisons**, where reusing the backend leaves the\n target reading against a warm capture instance and an advanced log\n position — the built-in diff can look clean while being meaningless.\n The SQL Server log-LSN clock is server-wide and keeps ticking across\n the reset, so per-record LSN columns and STATE offsets still differ\n between runs; gate the verdict on record-level shape rather than on\n those values.\n- **`--reset=backend`** — same as `--reset=fixture` but also recreates\n the backend container between the two sweeps, resetting the LSN clock\n at ~15s of extra startup cost. Use only when the reproduction depends\n on matching LSN sequences across runs.\n\n`--reset` has no effect without `--control-version` and produces a\nwarning; the flag governs the reset between two image runs, and there\nis no second run in single-version mode.\n\n## Asserting on output\n\nInline assertions in driver scripts. Suggested helpers:\n\n- Exit code: check `$?` immediately after the call.\n- AirbyteMessage shape: `jq -e 'select(.type == \"RECORD\")' $REPRO_OUT//read/stdout.txt`.\n- Connector-side log shape: `grep -c '' $REPRO_OUT//read/stderr.txt`.\n- Connection status: `jq -e 'select(.type == \"CONNECTION_STATUS\" and .connectionStatus.status == \"SUCCEEDED\")' $REPRO_OUT//check/stdout.txt`.\n\nIn comparison mode with `--reset=none` each command's directory holds\n`target/` and `control/` subdirectories (created by `airbyte-ops`).\nWith `--reset=fixture|backend` the split lives one level up:\n`$REPRO_OUT//{control,target}//`.\n\nPass `-- --enable-debug-logs=True` to `run.sh` to set `LOG_LEVEL=DEBUG`\non the connector container. That surfaces the\nDebezium \"Adding table … to the list of capture schema tables\" lines\nthat some assertions rely on.\n\n## Common gotchas\n\n- _No SQL Server Agent_ → CDC capture / cleanup jobs never run, and\n Debezium silently produces no records. `start-backend.sh` sets\n `MSSQL_AGENT_ENABLED=true` for this reason.\n- _Race on first `sqlcmd`_ → SQL Server 2022 takes around 15 seconds to\n accept connections cold. `start-backend.sh` polls until `SELECT 1`\n succeeds before returning.\n- _Configured-stream fields the validator rejects as null_ → the\n bulk-CDK schema validator requires `is_file_based`, `cursor_field`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files` on every\n `ConfiguredAirbyteStream`; omitting them fails with\n `Null value is not allowed. (code: 1021)`. This is not 4.3.x-only —\n verified on `4.4.12` and `5.0.0` too. `discover` does not emit\n `is_file_based`, so the catalog `run.sh` derives fills it in, and the\n catalog fixtures shipped with this and dependent skills populate all of\n them.\n- _A failed `check` does not fail a single-version run_ → the CDK exits\n 0 even when it emits `CONNECTION_STATUS` with `status: FAILED`, so the\n `check` step passes. Assert on the status message yourself (see\n [Asserting on output](#asserting-on-output)) rather than on the exit\n code when the repro hinges on `check`. Comparison mode does surface\n it, because it reads the report's verdict rather than the exit code.\n\n## Tear-down\n\n`run.sh` removes the backend container in an `EXIT` trap, so a run leaves\nnothing behind unless you passed `--keep-backend`. After a\n`--keep-backend` run, or to also discard the collected artifacts:\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mssql/.agents/skills/source-mssql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nBACKEND_NAME=source-mssql-db-backend \"$LIB/scripts/stop-backend.sh\" # idempotent; docker rm -f no-ops when absent\nrm -rf \"${REPRO_OUT:-/tmp/source-mssql-repro}\"\n```\n\n\n# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests/fixtures/configs/cdc.template.json)\n\n\n---\nname: source-mysql-e2e-cdc-tests\ndescription: Reproduce MySQL binlog CDC behavior against a local MySQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-mysql-e2e-tests and db-harness-lib.\n---\n\n# source-mysql-e2e-cdc-tests\n\nLocal CDC test harness for `source-mysql`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic MySQL skill\n([`source-mysql-e2e-tests`](../source-mysql-e2e-tests/SKILL.md)). The generic\nskill starts a MySQL 8.0 backend and runs the connector; this skill supplies\nthe binlog CDC config, CDC catalog, SQL fixtures, and case scripts.\n\nMySQL CDC is binlog-based and server-wide. The generic backend already starts\nMySQL with ROW binlogging, full row images, and GTIDs, so this skill does not\nenable CDC per table or start a second backend.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-mysql` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new MySQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-mysql-db-backend`, started by\n `../source-mysql-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-mysql-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with MySQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # dedicated cdc_test database and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic MySQL backend starts with `--log-bin`, `--binlog-format=ROW`,\n `--binlog-row-image=FULL`, `--gtid-mode=ON`, and\n `--enforce-gtid-consistency=ON`. Do not add per-table CDC enable steps or\n engine-specific backend startup to this skill.\n- `00-init-cdc.sql` drops and recreates the dedicated `cdc_test` database on\n every application. Re-applying it wipes existing rows and any accumulated\n binlog-relative table state, which is why `cases/incremental-replay.sh`\n passes `--skip-fixtures` on its second phase.\n- CDC catalogs include the bulk-CDK fields (`is_file_based`,\n `generation_id`, `minimum_generation_id`, `sync_id`,\n `destination_object_name`, and `include_files`) and use\n `[\"_ab_cdc_cursor\"]` as the cursor field.\n- Cases default to `VERSION=3.53.4`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-mysql-repro\n\n# Start the binlog-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-mysql-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the binlog-ready backend, CDC config, catalog metadata, and initial load\nwork together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved binlog\nposition resumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No MySQL CDC fixtures or\nworked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep any CDC database\n setup dedicated to the fixture and do not add per-table CDC enable steps.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.53.4}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses the plain MySQL client on stdin, so fixtures\nmay contain multiple statements. The MySQL server's binlog and GTID settings\nare established when the generic backend starts; no additional CDC setup is\nneeded in a case.\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-mysql/.agents/skills/source-mysql-e2e-tests/fixtures/configs/base.template.json)\n\n\n---\nname: source-mysql-e2e-tests\ndescription: Stand up a local MySQL 8.0 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-mysql: for ad-hoc end-to-end testing. Use when you need a deterministic, throwaway local environment for any source-mysql test or prove-fix comparison.\n---\n\n# source-mysql-e2e-tests\n\nLocal end-to-end test harness for `source-mysql`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the MySQL backend lifecycle scripts, fixtures, and config\ntemplates. It stands up a MySQL 8.0 container named\n`source-mysql-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-mysql:` image via\n`airbyte-ops cloud connector regression-test`.\n\n## When to use this skill\n\n- Reproducing a `source-mysql` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-mysql:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-mysql-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run mysql:8.0 with ROW binlog + GTIDs\n│ ├── apply-sql.sh # docker exec mysql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-mysql-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Root password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-mysql-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the MySQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `mysql:8.0`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n- The backend starts with ROW-format binary logging, a full row image,\n and GTIDs. This leaves the container ready for a CDC driver without\n restarting it.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-mysql\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.53.3\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.53.2\n\n# One command only.\npoe e2e-local --command=read --test-version=3.53.3\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose `discover`\nfailed earlier is reported `SKIPPED`, not `ERROR`. Under CI the table is\nalso appended to `$GITHUB_STEP_SUMMARY`. A run limited to one command\ninstead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`. Use this for CDC\n comparisons so the target does not read a warm capture instance.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the MySQL\n container between sweeps. Use this only when matching binlog or GTID\n sequences matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC needs ROW binlogging.** `start-backend.sh` sets\n `--binlog-format=ROW`, `--binlog-row-image=FULL`, and a server ID.\n The connecting user must have replication privileges; the root user\n created by the fixture has them.\n- **MySQL config shapes are specific.** `ssl_mode` uses `preferred`,\n `required`, `verify_ca`, or `verify_identity`; the base config uses\n `preferred` for the local plaintext backend. The\n `replication_method.method` value is `STANDARD` or `CDC`.\n- **`tinyint(1)` mapping.** The connector maps `TINYINT(1)` to boolean by\n default. Set `treat_tinyint1_as_integer: true` in a config when\n reproducing a typing issue.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-mysql-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n\n\n# Catalogs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/catalogs/users-cdc.json)\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests/fixtures/configs/cdc.template.json)\n\n\n---\nname: source-postgres-e2e-cdc-tests\ndescription: Reproduce PostgreSQL logical-decoding CDC behavior against a local PostgreSQL backend with CDC-aware fixtures, catalogs, and smoke cases. Composes on source-postgres-e2e-tests and db-harness-lib.\n---\n\n# source-postgres-e2e-cdc-tests\n\nLocal CDC test harness for `source-postgres`. It builds on the\nengine-independent orchestration in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/) and\nthe generic PostgreSQL skill\n([`source-postgres-e2e-tests`](../source-postgres-e2e-tests/SKILL.md)). The\ngeneric skill starts a PostgreSQL 16 backend and runs the connector; this skill\nsupplies the logical-decoding CDC config, CDC catalog, SQL fixtures, and case\nscripts.\n\nPostgreSQL CDC uses logical decoding from a publication and replication slot.\nThe generic PostgreSQL backend starts with the WAL and replication settings\nrequired by this skill, so this skill does not start a second backend or add\nper-run server configuration.\n\n## When to use this skill\n\n- Reproducing a CDC-mode `source-postgres` bug locally.\n- Verifying a fix with a published connector image or `VERSION=dev` after\n building the connector locally.\n- Running the initial-load and incremental-replay smoke cases.\n- Authoring a new PostgreSQL CDC repro with an idempotent SQL fixture and a case\n script that invokes the generic skill's `scripts/run.sh`.\n\n## Prerequisites\n\nSame as the generic skill:\n\n- Docker, `uv`, `jq`, and a clone of `airbytehq/airbyte`.\n- The backend container `source-postgres-db-backend`, started by\n `../source-postgres-e2e-tests/scripts/start-backend.sh`.\n\nYou do not need GSM or Cloud admin credentials. Never run this harness against\na customer connection or Airbyte Cloud.\n\n## Layout\n\n```\nsource-postgres-e2e-cdc-tests/\n├── SKILL.md\n├── cases/\n│ ├── initial-load.sh # CDC initial-load smoke case\n│ └── incremental-replay.sh # read → mutate → read-with-state smoke case\n└── fixtures/\n ├── configs/\n │ └── cdc.template.json # CDC replication config\n ├── catalogs/\n │ └── users-cdc.json # users stream with PostgreSQL CDC metadata\n └── sql/\n ├── 00-init-cdc.sql # publication, slot, table, and rows\n └── mutate-insert-users.sql # row inserted between replay phases\n```\n\nThe fixture uses the generic backend database `test_db`, because PostgreSQL\ncannot drop the database to which `apply-sql.sh` is connected. It resets the\n`users` table, publication, and logical replication slot in that database.\n\n`extract-state.py` is implemented in\n[`airbyte-integrations/db-harness-lib/scripts/extract-state.py`](../../../../../db-harness-lib/scripts/extract-state.py);\nit extracts protocol-level STATE messages and is not engine-specific.\n\n## Conventions\n\n- Cases reuse the generic backend and pass `--keep-backend`. Start it once at\n the top of a session, run the cases, and tear it down when finished.\n- The generic PostgreSQL backend starts with `wal_level=logical`,\n `max_replication_slots=10`, and `max_wal_senders=10`. Do not add per-run\n server configuration or start another backend in this skill.\n- `00-init-cdc.sql` drops and recreates the `users` table, publication, and\n `airbyte_slot` in `test_db` on every application. Re-applying it resets the\n fixture, which is why `cases/incremental-replay.sh` passes `--skip-fixtures`\n on its second phase.\n- PostgreSQL 3.8.5 discovers `_ab_cdc_lsn` as the source-defined cursor and\n does not include the newer generated catalog fields in discover output.\n The configured catalog includes `is_file_based`, generation metadata, and\n destination metadata because the pinned image requires those fields when\n initializing `read`.\n- Cases default to `VERSION=3.8.5`, the current published image used by this\n harness. Override it with `VERSION=dev` after building a local image.\n- Assertions are declarative `run.sh` expectations. The runner enforces\n `--expect-test`, `--expect-match`, `--forbid-match`, `--min-records`, and\n `--min-states` before returning.\n- Multi-phase cases extract the latest STATE message with the shared\n `extract-state.py` and pass it to the next invocation with `--state=PATH`.\n\n## Usage\n\n```bash\nSKILL=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-cdc-tests\nGENERIC=airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests\nLIB=airbyte-integrations/db-harness-lib\nexport REPRO_OUT=/tmp/source-postgres-repro\n\n# Start the logical-decoding-ready backend once.\n\"$GENERIC/scripts/start-backend.sh\"\n\n# Run either smoke case. Both pass --keep-backend.\n\"$SKILL/cases/initial-load.sh\"\n\"$SKILL/cases/incremental-replay.sh\"\n\n# Use a locally built connector image after a code change.\nVERSION=dev \"$SKILL/cases/initial-load.sh\"\n\n# Tear down after the session.\nBACKEND_NAME=source-postgres-db-backend \\\n \"$LIB/scripts/stop-backend.sh\"\n```\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Use `--test-version=dev` only for code that is not on a pushed\nPR branch.\n\nEach case invokes the generic skill's engine shim, which delegates to\n`airbyte-integrations/db-harness-lib/scripts/run.sh`. The runner applies the\nfixture, renders the config, runs the requested connector command, stores\nartifacts under `$REPRO_OUT//`, and enforces the case assertions.\n\n## Worked examples\n\n### Initial CDC load\n\n`cases/initial-load.sh` applies `00-init-cdc.sql` and reads the configured\n`users` stream with `cdc.template.json` and `users-cdc.json`. It asserts a\npassing read with at least three records and one STATE message. This verifies\nthat the logical-decoding-ready backend, CDC config, catalog metadata, and\ninitial load work together.\n\n### Incremental replay\n\n`cases/incremental-replay.sh` runs a baseline read, extracts its STATE message,\ninserts `dave@example.com`, and replays from the saved state without\nre-applying fixtures. It asserts a passing replay containing\n`dave@example.com` and no `alice@example.com`, proving that the saved LSN\nresumes after the initial load.\n\nThese are smoke canaries, not per-bug reproductions. No PostgreSQL CDC fixtures\nor worked per-bug examples existed before this skill.\n\n## Authoring a new repro\n\n1. Add an idempotent SQL fixture under `fixtures/sql/`. Keep CDC setup in the\n configured database and do not add per-run server configuration.\n2. Add a CDC-aware catalog under `fixtures/catalogs/` when the case needs a\n stream shape not covered by `users-cdc.json`. Use the source-defined cursor\n emitted by `discover` for the pinned connector version.\n3. Add `cases/.sh` with `set -euo pipefail`, a `${VERSION:-3.8.5}`\n default, and a call to the generic skill's `scripts/run.sh`. Pass\n `--config-template`, `--catalog`, `--fixture`, `--keep-backend`, and the\n relevant `--expect-*` flags.\n4. For a multi-phase case, model the sequence on\n `cases/incremental-replay.sh`: baseline `read`, extract STATE from\n `$REPRO_OUT//read/stdout.txt`, mutate through the generic\n `apply-sql.sh`, then run with `--skip-fixtures --state=PATH`.\n5. Verify the case against a published image or `VERSION=dev`, inspect its\n artifacts, and keep the backend lifecycle in the generic skill.\n\nThe generic `apply-sql.sh` uses `psql` on stdin, so fixtures may contain\nmultiple statements. Logical replication settings are established when the\ngeneric backend starts; no additional CDC setup is needed in a case beyond the\npublication and replication slot in the SQL fixture.\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-postgres/.agents/skills/source-postgres-e2e-tests/fixtures/configs/base.template.json)\n\n\n---\nname: source-postgres-e2e-tests\ndescription: Stand up a local PostgreSQL 16 backend, apply SQL fixtures, and sweep the Airbyte protocol commands (spec → check → discover → read) against airbyte/source-postgres: for ad-hoc end-to-end testing.\n---\n\n# source-postgres-e2e-tests\n\nLocal end-to-end test harness for `source-postgres`. The engine-independent\norchestration lives in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/);\nthis skill keeps the PostgreSQL backend lifecycle scripts, fixtures, and\nconfig templates. It stands up a PostgreSQL 16 container named\n`source-postgres-db-backend`, lets you apply arbitrary SQL fixtures, and runs\nAirbyte protocol commands against any `airbyte/source-postgres:` image via\n`airbyte-ops cloud connector regression-test`.\n\nThis skill supports standard (non-CDC) local sweeps. CDC fixtures and\nreplication setup are not yet stood up in this skill.\n\n## When to use this skill\n\n- Reproducing a `source-postgres` bug locally.\n- Sweeping `spec` → `check` → `discover` → `read`, or running one of\n them, against any `airbyte/source-postgres:` for connector\n development.\n- Proving a fix by comparing a target image with a control image.\n\n## Prerequisites\n\n- Docker.\n- [`uv`](https://docs.astral.sh/uv/). `uv tool install\nairbyte-internal-ops` puts `airbyte-ops` on `$PATH`; alternatively prefix\n every call with `uvx airbyte-internal-ops`.\n- `jq`.\n- A clone of `airbytehq/airbyte`.\n\nYou do not need GSM or Cloud admin credentials. Local-only mode (no\n`--connection-id`) reads everything from local files.\n\n## Layout\n\n```\nsource-postgres-e2e-tests/\n├── SKILL.md\n├── scripts/\n│ ├── start-backend.sh # docker run postgres:16 with logical replication\n│ ├── apply-sql.sh # docker exec psql on stdin\n│ ├── reset-databases.sh # drop and recreate non-system databases\n│ └── run.sh # engine shim to db-harness-lib orchestration\n└── fixtures/\n ├── configs/\n │ └── base.template.json # STANDARD config; host placeholder\n └── sql/\n └── 00-init-base.sql # sample table with three rows\n```\n\nThe engine shim and lifecycle scripts remain in this skill's `scripts/`\ndirectory; the shared orchestration and protocol helpers live in\n[`airbyte-integrations/db-harness-lib/`](../../../../../db-harness-lib/).\nThe skill expects all script paths relative to the skill root.\n\n## Conventions\n\n- Container name: `source-postgres-db-backend`. Hard-coded in the lifecycle\n scripts. Override via `BACKEND_NAME=…` only for parallel test isolation;\n don't use customer connection names.\n- Postgres password: `test_password`. Override with `BACKEND_PASSWORD` when\n running an isolated local backend.\n- Initial database: `test_db`. Override with `BACKEND_DB` when using a\n different fixture database.\n- Working directory for rendered configs and run output:\n `${REPRO_OUT:-/tmp/source-postgres-repro}`. A `run.sh` invocation writes\n everything under `$REPRO_OUT//`: the rendered `config.json`,\n the derived `configured_catalog.json`, and one subdirectory per protocol\n command (`spec/`, `check/`, `discover/`, `read/`) holding that command's\n `stdout.txt`, `stderr.txt`, `report.md`, and `report.html`.\n- Both containers (the PostgreSQL backend and the connector launched by\n `airbyte-ops`) share Docker's default `bridge` network. The connector\n resolves the backend by its bridge IP, which the shared\n [`render-config.sh`](../../../../../db-harness-lib/scripts/render-config.sh)\n substitutes into the working config at runtime. Tracked upstream in\n [`airbytehq/airbyte-ops-mcp#765`](https://github.com/airbytehq/airbyte-ops-mcp/issues/765).\n- The backend image is pinned to `postgres:16`, not `latest`, for stable\n major-version behavior. Override with `BACKEND_IMAGE=…` for a\n version-specific reproduction.\n\n## Usage\n\n`scripts/run.sh` is the only supported entrypoint. It performs the whole\nsequence — start the backend, apply the fixtures, render the config, run\n`spec` → `check` → `discover`, derive the configured catalog from that\n`discover` output, run `read`, and tear down on exit:\n\n```bash\ncd airbyte-integrations/connectors/source-postgres\n\n# Single version, full sweep.\npoe e2e-local --test-version=3.8.5\n\n# Target vs. control comparison, non-CDC.\npoe e2e-local --test-version=dev --control-version=3.8.4\n\n# One command only.\npoe e2e-local --command=read --test-version=3.8.5\n```\n\nThe sweep runs every command against the one backend and reports each\nresult rather than stopping at the first failure, then prints a summary\ntable and exits non-zero if any command failed. A `read` whose\n`discover` failed earlier is reported `SKIPPED`, not `ERROR`. Under CI the\ntable is also appended to `$GITHUB_STEP_SUMMARY`. A run limited to one\ncommand instead exits with the connector's own exit code.\n\nPrefer a published target image for code on a pushed PR branch. Publish a\npre-release from the PR with the Airbyte Ops MCP tool\n`publish_connector_to_airbyte_registry`, then pass the resulting\n`-preview.<7-char-sha>` tag as `--test-version`. See\n[Getting a target image](../../../../../db-harness-lib/README.md#getting-a-target-image)\nfor details. Build the target image first when using `--test-version=dev`, or\npass `--build` to have `run.sh` run `:dockerBuildx` for code that is not on a\npushed PR branch. Other options:\n`--command=spec|check|discover|read` (default `all`), `--skip-read`,\n`--skip-fixtures`, `--step-name`, `--catalog`, `--state=PATH`,\n`--sync-mode=incremental`, `--cursor-field`, `--streams`,\n`--config-template`, `--fixture`, `--reset=none|fixture|backend`,\n`--keep-backend`, and `--` to forward extra args to `airbyte-ops`.\nPer-command timeouts match the workflow's (30/30/60/180 minutes) and are\noverridable with `TIMEOUT_MINUTES_{SPEC,CHECK,DISCOVER,READ}`.\n\n## Declarative expectations\n\nInstead of driver scripts hand-rolling `grep -q '' || exit 1`\nagainst command artifacts, `run.sh` accepts expectation flags that it\nenforces before returning:\n\n| Flag | Effect |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `--expect-test=pass\\|fail` | Overall target verdict. |\n| `--expect-control=pass\\|fail` | Control verdict in comparison mode; requires `--control-version` and `--reset=fixture` or `--reset=backend`. |\n| `--min-records=N` | Target read must contain at least N `RECORD` messages. |\n| `--min-states=N` | Target read must contain at least N `STATE` messages. |\n| `--expect-match=[:]:[:N]` | Target command output must match the regex at least N times. |\n| `--forbid-match=[:]:` | Target command output must not match the regex. |\n\nMatch assertions use the target-side artifacts for the named command and\ndefault to `read` when no command prefix is supplied. The accepted\nchannels are `stdout`, `stderr`, and `any`; command names are `spec`,\n`check`, `discover`, and `read`.\n\nAny expectation failure exits non-zero and is included in the generated\nsummary. For example, a driver can replace a `grep` assertion with\n`--expect-match=stderr:expected message`.\n\n## Comparison modes with `--control-version`\n\n- **`--reset=none` (default)** — `run.sh` invokes `airbyte-ops` with both\n `--test-image` and `--control-image`, so the images run sequentially\n against one backend and the built-in comparators emit a target-vs-control\n diff. This is appropriate for non-CDC full-refresh work.\n- **`--reset=fixture`** — `run.sh` runs the control sweep first, drops every\n non-system database, reapplies the fixtures, and then runs the target\n sweep. Artifacts land under\n `$REPRO_OUT//{control,target}//`.\n- **`--reset=backend`** — like `--reset=fixture`, but recreates the\n PostgreSQL container between sweeps. Use this only when matching database\n initialization state matters.\n\n`--reset` has no effect without `--control-version` and produces a warning.\nThe flag governs the reset between two image runs; there is no second run\nin single-version mode.\n\n## Common gotchas\n\n- **CDC is not included in this skill.** The lifecycle starts PostgreSQL 16\n with logical replication enabled so the composed\n [`source-postgres-e2e-cdc-tests`](../source-postgres-e2e-cdc-tests/SKILL.md)\n skill can use the same backend. CDC fixtures and case scripts live in that\n skill.\n- **PostgreSQL config shapes are specific.** `ssl_mode.mode` uses\n `prefer` for the local plaintext backend. The\n `replication_method.method` value for a standard sync is `Standard`.\n- **The connector resolves the backend by bridge IP.** The shared\n `render-config.sh` substitutes that IP into each rendered config because\n the local regression-test command does not currently accept `--network`.\n- **`check` failures may exit zero.** The connector can emit a failed\n `CONNECTION_STATUS` while exiting zero. Assert on the status message when\n a reproduction depends on check-time behavior.\n- **Do not use customer connections.** This harness is for local testing\n only and must never be used against customer connections or Airbyte Cloud.\n\n## Teardown\n\nThe shared library's default `stop-backend.sh` is idempotent:\n\n```bash\nBACKEND_NAME=source-postgres-db-backend \\\n airbyte-integrations/db-harness-lib/scripts/stop-backend.sh\nrm -rf \"$REPRO_OUT\"\nunset REPRO_OUT\n```\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/base.template.json)\n\n\n# Configs Skill\n\nPath: `airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/airbyte-integrations/connectors/source-snowflake/.agents/skills/source-snowflake-e2e-tests/fixtures/configs/utc-session.template.json)\n\n\n---\nname: source-snowflake-e2e-tests\ndescription: Run the Airbyte protocol sweep for airbyte/source-snowflake: against the shared Snowflake integration account inside a per-run PROVEFIX_ schema (B3 remote-backend mode of db-harness-lib); prove-fix comparisons, cursor canary, orphan sweep.\n---\n\n# source-snowflake-e2e-tests\n\nRemote end-to-end harness for `source-snowflake`. The shared\n`db-harness-lib` orchestration runs protocol commands against a per-run schema\nin the shared Snowflake integration account.\n\n## Prerequisites\n\n- Docker, [`uv`](https://docs.astral.sh/uv/), and `jq`.\n- `gcloud` authenticated with read access to the GSM secret, or an existing\n `SNOWFLAKE_CONFIG_FILE`.\n- A clone of `airbytehq/airbyte`.\n\n## Layout\n\n```\nsource-snowflake-e2e-tests/\n├── SKILL.md\n├── ../../../poe_tasks.toml (connector-root entrypoint)\n├── scripts/\n│ ├── run.sh, sf.py\n│ ├── fetch-config.sh, render-config.sh\n│ ├── start-backend.sh, reset-databases.sh, stop-backend.sh\n│ ├── apply-sql.sh, sweep-orphans.sh\n├── fixtures/configs/base.template.json, utc-session.template.json\n├── fixtures/sql/00-cursor-canary-base.sql\n├── fixtures/sql/01-cursor-canary-boundary-row.sql\n└── cases/cursor-upper-bound.sh\n```\n\n## Isolation model\n\n- Every run uses `PROVEFIX_` and asserts that the schema is absent\n before both control and target runs.\n- A marker owned by the run permits reuse of the same schema between canary\n phases; unmarked pre-existing schemas are refused.\n- Control and target use the same schema with `--reset=fixture`; remote\n `--reset=backend` is rejected.\n- Teardown drops the schema in the run's exit trap.\n- `scripts/sweep-orphans.sh` lists or drops old `PROVEFIX_*` schemas, defaulting\n to four hours.\n\nThe default run id is\n`YYYYMMDDhhmmss__`, producing a\nschema-safe `PROVEFIX_` name.\n\nThe schema is owned by the scripts, while `sf.py` is the only code that talks\nto Snowflake. The default config comes from GSM secret\n`SECRET_SOURCE-SNOWFLAKE_KEY_PAIR__CREDS` in project\n`dataline-integration-testing`; fetched configs are marked with a sidecar and\nonly those configs are removed during teardown, so caller-supplied configs\nsurvive. Set `SNOWFLAKE_CONFIG_FILE` to reuse a caller-supplied config.\n\n## Cursor upper-bound canary\n\nThe canary proves the cursor precision regression from airbytehq/airbyte#82705:\n\n```bash\ncases/cursor-upper-bound.sh\n```\n\nIt compares fixed `1.1.1` with control `1.1.0` in two phases. Phase 1 runs the\ninitial sync on the control image with exact-microsecond base rows, producing\nthe checkpoint a user on the bad version would have. Phase 2 inserts a\nsub-microsecond boundary row and replays that state against both versions:\n1.1.0 loses the row because it floors the upper bound, while 1.1.1 emits it.\n\nThe harness reuses the `INTEGRATION_TEST_WAREHOUSE_DESTINATION` X-Small\nwarehouse shared with destination-snowflake integration tests. Never touch\nschemas outside `PROVEFIX_*`.\n\n## Gotchas\n\nSnowflake uppercases identifiers, so use\n`--cursor-field=UPDATED_AT --streams=CURSOR_CANARY`. Rendered configs contain\nthe private key; treat `REPRO_OUT` as 0600 material and never attach it.\nTIMESTAMP_NTZ cursor bounds are bound as TIMESTAMP_LTZ by the JDBC driver and\ncompared through the session TIMEZONE; with the account default\n(America/Los_Angeles), narrow incremental windows match nothing.\nThe harness sets `jdbc_url_params=TIMEZONE=UTC` so the canary isolates #82705;\nthis session-timezone behavior appears to be a separate connector bug worth\nits own issue.\nThe `--catalog` CDC guard (#85752) never fires here: source-snowflake has no\nCDC replication method.\n\n\n# Skills Skill\n\nPath: `docs/ai-agents/get-started/developer-quickstart/skills/_category_.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/airbytehq/airbyte/HEAD/docs/ai-agents/get-started/developer-quickstart/skills/_category_.json)\n\n\n"}