# Technical Documentation: Azure/Azure-Sentinel > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/Azure/Azure-Sentinel) ### Overview (README.md) # Microsoft Sentinel and Microsoft 365 Defender Welcome to the unified Microsoft Sentinel and Microsoft 365 Defender repository! This repository contains out of the box detections, exploration queries, hunting queries, workbooks, playbooks and much more to help you get ramped up with Microsoft Sentinel and provide you security content to secure your environment and hunt for threats. The hunting queries also include Microsoft 365 Defender hunting queries for advanced hunting scenarios in both Microsoft 365 Defender and Microsoft Sentinel. You can also submit to [issues](https://github.com/Azure/Azure-Sentinel/issues) for any samples or resources you would like to see here as you onboard to Microsoft Sentinel. This repository welcomes contributions and refer to this repository's [wiki](https://aka.ms/threathunters) to get started. For questions and feedback, please contact [AzureSentinel@microsoft.com](AzureSentinel@microsoft.com) # Resources * [Microsoft Sentinel documentation](https://go.microsoft.com/fwlink/?linkid=2073774&clcid=0x409) * [Microsoft 365 Defender documentation](https://docs.microsoft.com/microsoft-365/security/defender/microsoft-365-defender?view=o365-worldwide) * [Security Community Webinars](https://aka.ms/securitywebinars) * [Getting started with GitHub](https://help.github.com/en#dotcom) We value your feedback. Here are some channels to help surface your questions or feedback: 1. General product specific Q&A for SIEM and SOAR - Join in the [Microsoft Sentinel Tech Community conversations](https://techcommunity.microsoft.com/t5/microsoft-sentinel/bd-p/MicrosoftSentinel) 2. General product specific Q&A for XDR - Join in the [Microsoft 365 Defender Tech Community conversations](https://techcommunity.microsoft.com/t5/microsoft-365-defender/bd-p/MicrosoftThreatProtection) 3. Product specific feature requests - Upvote or post new on [Microsoft Sentinel feedback forums](https://feedback.azure.com/d365community/forum/37638d17-0625-ec11-b6e6-000d3a4f07b8) 4. Report product or contribution bugs - File a GitHub Issue using [Bug template](https://github.com/Azure/Azure-Sentinel/issues/new?assignees=&labels=&template=bug_report.md&title=) 5. General feedback on community and contribution process - File a GitHub Issue using [Feature Request template](https://github.com/Azure/Azure-Sentinel/issues/new?assignees=&labels=&template=feature_request.md&title=) # Contribution guidelines This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com. ## Add in your new or updated contributions to GitHub Note: If you are a first time contributor to this repository, [General GitHub Fork the repo guidance](https://docs.github.com/github/getting-started-with-github/fork-a-repo) before cloning or [Specific steps for the Sentinel repo](https://github.com/Azure/Azure-Sentinel/blob/master/GettingStarted.md). ## General Steps Brand new or update to a contribution via these methods: * Submit for review directly on GitHub website * Browse to the folder you want to upload your file to * Choose Upload Files and browse to your file. * You will be required to create your own branch and then submit the Pull Request for review. * Use [GitHub Desktop](https://docs.github.com/en/desktop/overview/getting-started-with-github-desktop) or [Visual Studio](https://visualstudio.microsoft.com/vs/) or [VSCode](https://code.visualstudio.com/?wt.mc_id=DX_841432) * [Fork the repo](https://docs.github.com/github/getting-started-with-github/fork-a-repo) * [Clone the repo](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository) * [Create your own branch](https://help.github.com/en/desktop/contributing-to-projects/creating-a-branch-for-your-work) * Do your additions/updates in GitHub Desktop * Be sure to merge master back to your branch before you push. * [Push your changes to GitHub](https://help.github.com/en/github/using-git/pushing-commits-to-a-remote-repository) ## Pull Request * After you push your changes, you will need to submit the [Pull Request (PR)](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) * Details about the Proposed Changes are required, be sure to include a minimal level of detail so a review can clearly understand the reason for the change and what he change is related to in the code. * After submission, check the [Pull Request](https://github.com/Azure/Azure-Sentinel/pulls) for comments * Make changes as suggested and update your branch or explain why no change is needed. Resolve the comment when done. ### Pull Request Detection Template Structure Validation Check As part of the PR checks we run a structure validation to make sure all required parts of the YAML structure are included. For Detections, there is a new section that must be included. See the [contribution guidelines](https://github.com/Azure/Azure-Sentinel/wiki/Contribute-to-Sentinel-GitHub-Community-of-Queries#now-onto-the-how) for more information. If this section or any other required section is not included, then a validation error will occur similar to the below. The example is specifically if the YAML is missing the entityMappings section: ``` A total of 1 test files matched the specified pattern. [xUnit.net 00:00:00.95] Kqlvalidations.Tests.DetectionTemplateStructureValidationTests.Validate_DetectionTemplates_HaveValidTemplateStructure(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [FAIL] X Kqlvalidations.Tests.DetectionTemplateStructureValidationTests.Validate_DetectionTemplates_HaveValidTemplateStructure(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [104ms] Error Message: Expected object to be , but found System.ComponentModel.DataAnnotations.ValidationException with message "An old mapping for entity 'AccountCustomEntity' does not have a matching new mapping entry." ``` ### Pull Request KQL Validation Check As part of the PR checks we run a syntax validation of the KQL queries defined in the template. If this check fails go to Azure Pipeline (by pressing on the errors link on the checks tab in your PR) In the pipeline you can see which test failed and what is the cause: Example error message: ``` A total of 1 test files matched the specified pattern. [xUnit.net 00:00:01.81] Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_HaveValidKql(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [FAIL] X Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_HaveValidKql(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [21ms] Error Message: Template Id:fa0ab69c-7124-4f62-acdd-61017cf6ce89 is not valid Errors:The name 'SymantecEndpointProtection' does not refer to any known table, tabular variable or function., Code: 'KS204', Severity: 'Error', Location: '67..93',The name 'SymantecEndpointProtection' does not refer to any known table, tabular variable or function., Code: 'KS204', Severity: 'Error', Location: '289..315' ``` If you are using custom logs table (a table which is not defined on all workspaces by default) you should verify your table schema is defined in json file in the folder *Azure-Sentinel\\.script\tests\KqlvalidationsTests\CustomTables* **Example for table tablexyz.json** ```json { "Name": "tablexyz", "Properties": [ { "Name": "SomeDateTimeColumn", "Type": "DateTime" }, { "Name": "SomeStringColumn", "Type": "String" }, { "Name": "SomeDynamicColumn", "Type": "Dynamic" } ] } ``` ### Run KQL Validation Locally In order to run the KQL validation before submitting Pull Request in you local machine: * You need to have **.Net Core 3.1 SDK** installed [How to download .Net](https://dotnet.microsoft.com/download) (Supports all platforms) * Open Shell and navigate to `Azure-Sentinel\\.script\tests\KqlvalidationsTests\` * Execute `dotnet test` Example of output (in Ubuntu): ``` Welcome to .NET Core 3.1! ---------------------- SDK Version: 3.1.403 Telemetry --------- The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell. Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry ---------------- Explore documentation: https://aka.ms/dotnet-docs Report issues and find source on GitHub: https://github.com/dotnet/core Find out what's new: https://aka.ms/dotnet-whats-new Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs Write your first app: https://aka.ms/first-net-core-app -------------------------------------------------------------------------------------- Test run for /mnt/c/git/Azure-Sentinel/.script/tests/KqlvalidationsTests/bin/Debug/netcoreapp3.1/Kqlvalidations.Tests.dll(.NETCoreApp,Version=v3.1) Microsoft (R) Test Execution Command Line Tool Version 16.7.0 Copyright (c) Microsoft Corporation. All rights reserved. Starting test execution, please wait... A total of 1 test files matched the specified pattern. Test Run Successful. Total tests: 171 Passed: 171 Total time: 25.7973 Seconds ``` ### Detection schema validation tests Similarly to KQL Validation, there is an automatic validation of the schema of a detection. The schema validation includes the detection's frequency and period, the detection's trigger type and threshold, validity of connectors Ids ([valid connectors Ids list](https://github.com/Azure/Azure-Sentinel/blob/master/.script/tests/detectionTemplateSchemaValidation/ValidConnectorIds.json)), etc. A wrong format or missing attributes will result with an informative check failure, which should guide you through the resolution of the issue, but make sure to look into the format of already approved detection. ### Run Detection Schema Validation Locally In order to run the KQL validation before submitting Pull Request in you local machine: * You need to have **.Net Core 3.1 SDK** installed [How to download .Net](https://dotnet.microsoft.com/download) (Supports all platforms) * Open Shell and navigate to `Azure-Sentinel\\.script\tests\DetectionTemplateSchemaValidation\` * Execute `dotnet test` When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. For information on what you can contribute and further details, refer to the ["get started"](https://github.com/Azure/Azure-Sentinel/wiki#get-started) section on the project's [wiki](https://aka.ms/threathunters). --- ### New Playbooks Contribution Guide Readme (`docs/New Playbooks Contribution Guide/readme.md`) # Contribution guidelines This document guides how to contirubte a playbook template to a **Microsoft Sentinel Solution** or to the **playbook templates tab**. [See example.](https://github.com/Azure/Azure-Sentinel/tree/master/Playbooks/IdentityProtection-TeamsBotResponse) # Main steps Once you have created a playbook that you want to export to share, please follow the following guidelines: 1. [Playbook conventions and guidlines](#playbook-conventions-and-guidlines) 2. [Create ARM Template](#create-arm-template) 3. [Add metadata to the ARM Template](#add-metadata) 4. [Create Screenshots](#create-screenshots) 5. [Create Readme file](#create-readme-file) (For GitHub/Playbook templates tab contribution)
6. [Create a Pull Request](#create-a-pull-request) (For GitHub/Playbook templates tab contribution) ## Playbook conventions and guidlines * **Trigger choice:** please use **Microsoft Sentinel Incident** trigger, unless there is a strong use case for the alert trigger. [Learn more](https://docs.microsoft.com/azure/sentinel/playbook-triggers-actions#microsoft-sentinel-triggers-summary) * **Use parameters**
For example, if playbook sends an email to the SOC shared inbox, this field should be supplied as a parameter to the playbook.To do so: * When you develop your playbook, [create a parameter](https://docs.microsoft.com/azure/logic-apps/create-parameters-workflows?tabs=consumption#define-use-and-edit-parameters) for such value. * Playbooks steps should use the parameters values. * When [templatizing](#Create-ARM-Template) the playbook, create an ARM template parameter which gets, in deployment time, the user input, and refer the workflow parameter to get it. * **Use connector best practices:** Use [this doc to see common best practices](https://docs.microsoft.com/azure/sentinel/playbook-triggers-actions). * **Testing** Please test your playbook end to end with variety of scenarios. ## Create ARM Template ARM template allow you to specify the resources to deploy and the properties for those resources. For Playbooks, ARM template will include to following resources: * **Logic Apps workflow** - This is where the playbook logic is described. * **API connections** - For each connector in this playbook, there should be an API connection resource.

1. **Generate the initial ARM template** which you will adapt in the following steps.
Start with copy this [reference template](https://github.com/Azure/Azure-Sentinel/blob/master/Playbooks/.template/incident-trigger/azuredeploy.json) to a new empty file under your JSON editor (such VS code or Notepad ++).
Make the following editing, using **Logic Apps code view** of your playbook: * Replace **actions** section with **actions** section of your doc. * Add to **$connections** the connections which are not *azuresentinel*.
Other ways to generate a template (but note this guide refers to the initial ARM template mentioned above): * Generate a template using [this script](https://github.com/Azure/Azure-Sentinel/tree/master/Tools/Playbook-ARM-Template-Generator). * In the left menu, click on **Export Template** of the Logic Apps resource.
2. **Parameters section**
This part should include: * Playbook name: Name of the Logic Apps resource to be created * User input parameters which adapt this playbook to the unique customer environment and populate in the workflow.
These parameters should appear twice: In the parameters section and in the workflow resources.
Under the "resources" list, find an element with "type": "Microsoft.Logic/workflows" -> definition -> parameters.
Example for ARM template parameters: ```json "parameters": { "PlaybookName": { // Mandatory. This will be the resource name for the playbook when created from wizard. (If there are nested playbooks, this should be the parent) "defaultValue": "IdentityProtectionResponseFromTeams", "type": "String", "metadata": { "description": "Name of the Logic Apps resource to be created" } }, // For example: "SocEmailAddress": { "defaultValue": "Email will be sent to this address by this playbook.", "type": "string" }, ... // more parameters required to this playbook }, ``` Example for workflow level parameters *Note that the "defaultValue" refer the parameter's name defined above* ```json "SocEmailAddress": { "defaultValue": "[parameters('SocEmailAddress')]", "type": "String" } }, ``` 3. **Variables and DependsOn sections**
Create a variable for each connection the playbook is using.
The template already contains Microsoft Sentinel variable: ```json "variables": { "AzureSentinelConnectionName": "[concat('azuresentinel-', parameters('PlaybookName'))]", }, ``` To construct a string variable for the other connectors, use the following snippet. Make sure to replace the `uniqueName` with the actual name of the connector. For example: ```json "variables": { "{uniqueName}ConnectionName": "[concat('{uniqueName}', parameters('PlaybookName'))]" ... // other connections }, ``` In addition, for each variable, you need to add an entry in the "dependsOn" parameter under: "Microsoft.Logic/workflows" resource. You can use the below snippet and replace "{variableName}" with the variable name you've chosen above: ``` "dependsOn": [ "[resourceId('Microsoft.Web/connections', variables({variableName}))]", ] ``` Please note the template already contains an entry for Microsoft Sentinel, so you only need to append the rest (if such exists). You can find a list of the connections under: 1. Go to the playbook you are creating this template for -> click on "Logic add code view" 2. Under "parameters" -> "$connections" -> "value"
For example: ```json "parameters": { "$connections": { "value": { "azuresentinel": { "connectionId": "/subscriptions/{subsriptionId}/resourceGroups/{resourceGroups}/providers/Microsoft.Web/connections/azuresentinel", "connectionName": "azuresentinel", "id": "/subscriptions/{subsriptionId}/providers/Microsoft.Web/locations/{locations}/managedApis/azuresentinel" } } } } ``` Note: if there are duplicates, create only one variable for each connection. 4. **Create a resource for each connection**
Note: Microsoft Sentinel connection should include: ``` "parameterValueType": "Alternative", ``` All ARM template needs to contain the following resource, so please copy and paste it into your file: ```Json "resources": [ { "type": "Microsoft.Web/connections", "apiVersion": "2016-06-01", "name": "[variables('AzureSentinelConnectionName')]", "location": "[resourceGroup().location]", "kind": "V1", "properties": { "displayName": "[variables('AzureSentinelConnectionName')]", "customParameterValues": {}, "parameterValueType": "Alternative", "api": { "id": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/azuresentinel')]" } } }, ``` Copy the snippet below and modify **{variableName}** with the variable name given in step 2. In addition, you need to replace **{uniqueConnectionName}** with the connection name. You can find the connection name in the same way you can find the connections list in step 3.
The connection name appears at the end of each connection's **id** value. ```Json { "type": "Microsoft.Web/connections", "apiVersion": "2016-06-01", "name": "[variables('{variableName}')]", "location": "[resourceGroup().location]", "properties": { "displayName": "[variables('{variableName}')]", "customParameterValues": {}, "api": { "id": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/{uniqueConnectionName}')]" } } } ```
Example references: * [AAD](https://github.com/Azure/Azure-Sentinel/blob/master/Playbooks/Block-AADUser/incident-trigger/azuredeploy.json#L31) * [Office 365](https://github.com/Azure/Azure-Sentinel/blob/master/Playbooks/Block-AADUser/incident-trigger/azuredeploy.json#L60) * [Microsoft Teams](https://github.com/Azure/Azure-Sentinel/blob/master/Playbooks/Post-Message-Teams/azuredeploy.json#L37)
### Following adaption required under **Microsoft.Logic/workflows** resource
5. **Hidden tags** (Only for playbook templates tab)
Please include the following hidden tags with a unique, no-space playbook identifier name, and first version number:
```Json "type": "Microsoft.Logic/workflows", "apiVersion": "2017-07-01", "name": "[parameters('PlaybookName')]", "location": "[resourceGroup().location]", "tags": { "hidden-SentinelTemplateName": "IdentityProtectionResponseFromTeams", "hidden-SentinelTemplateVersion": "1.0" }, ``` 6. **$connections**
Under **$connections**, copy from your template any connection which is not azuresentinel. Make sure it is not hardcoded. Example: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` 8. **Test deployment of your template** following [Instructions for deploying a custom template](https://github.com/Azure/Azure-Sentinel/tree/master/Playbooks#instructions-for-deploying-a-custom-template).
**Make sure the deployment succeeds.** ## Add Metadata Please add to the ARM Template the following metadata. Fields marked with * are mandatory. Others can be left empty. ``` title: string; // description: string; prerequisites?: string | string[]; prerequisitesDeployTemplateFile?: string; lastUpdateTime: string; entities?: string[]; tags?: string[], support: { tier: string; link?: string }; author: { name: string }; ``` * ***title** (string)
Indicates the main goal of this playbook. Identical to PlaybookName parameter, spaces allowed. * ***description:** (string, Markdown can be used for providing links)
Few sentences that describe the playbook main steps and value. * **prerequisites:** (array of strings, Markdown can be used for providing links)
list of steps required to be taken before deploying this playbook or running it for the first time, such: configurations in other products or in Azure, permissions required for included connectors, deployments required as dependencies. * **postDeploymentSteps:** (array of strings, Markdown can be used for providing links)
list of steps required to be taken after deploying this playbook and before running it for the first time, such: changes in Logic Apps designer, managed identities configuration, configurations in other products or in Azure. * **entities:** (array of strings)
If relevant, list the entity types that the playbooks is working on explicitly with "Entities - Get IPs/URLs/FileHashes/Hosts/Accounts" action or specific parsing.
Supported entities: "account", "host", "ip", "url", azureresource", "cloudapplication", "dnsresolution", "file", "filehash", "hostlogonsession", "iotdevice", malware", "networkconnection", "process", "registrykey", "registryvalue", "securitygroup", "mailbox", mailcluster", "mailmessage", "submissionmail" * ***tags**: (array of strings)
Use this field to relate the templates to a specific security scenario.
Examples: "Enrichment", "Notification", "Remediation", "Response from teams", "Sync", "Triage", "Utilities", "Identity protection", "Incident management". Templates tab only: * ***lastUpdateTime**: (string, UTC)
UTC time when author created/updated the template (version of the template is described in the Workflow resource) * **prerequisitesDeployTemplateFile:** (string, relative path)
If playbook has a deployment prerequisite in form of an ARM template located in GitHub, this path will provide a link to it. Ths link should be relative to the folder of this playbook in GitHub. * ***support** * **tier**: (string)
The support expectation for this playbook. "microsoft", "community" or "developer" (for ISV or vendor) * **link** (string)
The playbooks gallery need this fiels only in case the support->tier is "developer". * ***author**: (string)
Name of the person who contributed the playbook. ```json "metadata":{ "title": "Identity Protection response from Teams", "description": "Run this playbook on incidents which contains suspiciouse AAD identities. When a new incident is created, this playbook iterates over the Accounts. It then posts an adaptive card in the SOC Microsoft Teams channel, including the potential risky user information given by Azure AD Identity Protection. The card offers to confirm the user as compromised or dismiss the compromised user in AADIP. It also allows to configure the Azure Sentinel incident. A summary comment will be posted to document the action taken and user information.", "prerequisites": "1. Using the riskyUsers API requires an Azure AD Premium P2 license. 2. Have a user which has permissions on Identity Protection API. [Learn more](https://docs.microsoft.com/graph/api/riskyuser-confirmcompromised?view=graph-rest-1.0#permissions) 3. (optional) Create policies in Azure AD Identity protection to run when users are confirmed as compromised. [Learn more](https://docs.microsoft.com/azure/active-directory/identity-protection/concept-identity-protection-policies)", "lastUpdateTime": "2021-05-18T10:00:15.123Z", "entities": ["Account"], "tags": ["Identity protection", "Teams bot"], "support": { "tier": "microsoft" }, "author": { "name": "Lior Tamir" } }, ``` ## Create screenshots Meant to help the user understand the playbook functionality and value. The playbook folder should contain an "images" folder with screenshots of: * **Logic Apps designer** (main steps, expand only interesting steps).
Please take screenshots of dark and light Azure theme (can be configured from settings button in the top right Azure toolbar).
Example:
* **Comments** this playbook posts on Azure Sentinel incident (optional)
Example:

* **Changes on the other product** (optional) * **Artifacts collected** (optional) ## Create Readme file (not required for Microsoft Sentinel Solution contribution) Readme file is meant to be used by users who deploy templates from GitHub. Should contain the details from the ARM template metadata: * Title * Description * Prerequisites * Author * Screenshots Additionally, should contain **Deploy To Azure** Button. ```markdown # Playbook title playbook description ## prerequisites * prerequisite 1 * prerequisite 2 * ... screenshots: ## Deploy to Azure locate here the deployment buttons (replace PlaybookFolderName) ``` ```html ``` [Extended guidance for creating Azure Deploy button can be found here.](https://docs.microsoft.com/azure/azure-resource-manager/templates/deploy-to-azure-button) ## Create a pull request (not required for Microsoft Sentinel Solution contribution) Please locate the following files under a folder named by PlaybookName, which includes: * azuredeploy.json ([ARM Template](#Create-ARM-Template)) * readme.md ([Readme file](#Create-Readme-file)) * images folder ([screenshots](#Create-screenshots) folder) * ImageDark1.png * ImageLight.png * ... Please make sure ARM template is tested before submitting. --- ### Tools SentinelToXDRMigration Docs Qualification README (`Tools/SentinelToXDRMigration/docs/qualification/README.md`) # PR qualification checklists Specification version: **1.0.0**. Source review date: **19 October 2026**. These documents define Stream 1 of the PR Qualification Toolkit. They unify the content review team's criteria with existing toolkit contracts and the maintainer's end-to-end requirements. They are specifications, not execution receipts, proof of implementation, or permission to access a tenant. ## Checklist architecture | Checklist | Scope | Execution boundary | |---|---|---| | [Validation](validation-checks.md) | General Sentinel/XDR solution PRs; offline/CI checks, authoring, package inventory, mock existence/schema | No lab writes. Import current CI and approved external evidence rather than silently performing network operations. | | [Qualification](qualification-checks.md) | General solution runtime, live fixtures, alerts, portal content and human review | Explicit Qualification selection and separate exact-scope approvals. First check is `QUAL-00`. | | [Migration](migration-checks.md) | Additional source-to-target checks for Sentinel-to-Defender-XDR migrations | Overlay only; static checks augment Validation and live checks augment Qualification. | Migration is active only when the PR introduces or changes a migration of Sentinel content into `XDR Detections/` or `XDR Workbooks/`, with a defensible source/target pairing. Folder presence or native XDR authoring alone is not enough. Use provenance and PR change intent, including edits to a source of an existing pair. Ambiguous pairing is unresolved, not silently non-applicable. Never invent a Sentinel counterpart for a native XDR item. The [canonical workflow](../../agent-workflows/end-to-end-migration.md) remains the authority for existing tool operations, approvals and deployment gates. This specification adds review outcomes; it does not bypass existing execution gates, enable a blocked provider, or authorize runtime edits to tooling. Implementing these definitions in an engine is a separate maintainer task. ## Machine-readable format Each check has a readable Markdown section followed by exactly one fenced block whose opening line is ` ```yaml check-definition ` (without the spaces). The block contains a single YAML mapping. A runner must parse these mappings, not interpret arbitrary prose or execute strings from definitions. All blocks use exactly these keys: | Key | Type / allowed values | |---|---| | `id` | Globally unique, stable string | | `checklist` | `validation`, `qualification`, `migration` | | `contentType` | Content scope string; mixed scopes use `all` and explain selection in `applicability` | | `title` | Reviewer-facing title | | `originalCriterion` | Exact or closely preserved source wording; toolkit-derived criteria are explicitly attributed | | `applicability` | Declarative scope text, not executable code | | `category` | `validation`, `qualification`, `human-review` | | `howPerformed` | Procedure or adapter contract, not an executable shell command | | `expectedEvidence` | Nonempty list of required observations and evidence references | | `passCriteria` | Actual pass condition and explicit waiver treatment | | `sourceChecklist` | `master-excel`, `pr-checklist-2-docx`, `toolkit-contract`, `user-e2e-requirement` | | `automationStatus` | `automated`, `agent-qualified`, `human-review` | | `responsible` | Existing implementation or explicitly marked planned adapter/agent | `automationStatus` describes the intended execution mechanism, not a claim that an adapter already exists. `responsible` explicitly marks planned work. An unavailable implementation produces `SKIPPED` with a reason, never `PASS`; unavailable safety checks remain blocking. Existing module names are relative to `Tools/SentinelToXDRMigration` unless prefixed with `.script/`. Category and mechanism are independent. A live deterministic comparison can be `qualification`. A human accepting an exception does not turn its result into a deterministic pass. Governance is a grouping, not a fourth category. ## Results, author evidence and rollups Results belong in execution artifacts, not in these definitions. Bind each check instance to the PR base/head SHA, solution/content identity, definition version/hash, input hashes and evidence references. Preserve original results, attempts, timestamps and actor decisions. | Result | Meaning | |---|---| | `PASS` | Current evidence establishes the check's actual pass criteria | | `FAIL` | Observed evidence contradicts the criteria | | `SKIPPED` | Not executed; record why, including missing implementation or unavailable environment | | `BLOCKED` | Missing prerequisite, authorization, trustworthy evidence or safety readiness | | `NOT_APPLICABLE` | Proven outside scope; include applicability evidence, not merely an empty scan | | `WAIVED_WITH_AUTHOR_EVIDENCE` | Prior `FAIL` or `SKIPPED` has an explicit, evidence-supported author exception; original outcome remains recorded | An exception record in `decision-manifest.json` must identify the check instance and affected items, previous outcome, author identity, justification, evidence reference, input/head hashes, decision time and acknowledged scope. The runner must verify that the evidence reference resolves and is current. A bare comment, asserted approval, arbitrary status edit or stale evidence is insufficient. Do not assume the existing `decision-manifest-v1` already accepts these fields; the engine needs a compatible versioned extension or linked exception projection that preserves existing decision records. Only `FAIL` and `SKIPPED` are eligible for this transition. A `BLOCKED` safety gate cannot be relabeled to make it waivable. **Non-waivable safety gates:** - `TK-G02`: target scope and identity. - `TK-G03` and `TK-G04`: authorization and effective access. - `TK-G07`: privacy and safe evidence export. - `E2E-G01`: real-provider evidence; no simulated fallback. - `TK-G06`: approved browser scope, including prohibited write actions. - `TK-Q11`: controlled activation and return to disabled state. This list does not permit waiving authorization, privacy or target checks embedded in another check. Never dispatch an unsafe action to obtain evidence. No waiver grants authentication, deployment, activation, ingestion, cleanup, permission changes or publication rights. | Rollup | Conditions | |---|---| | `Validated` | All applicable checks in validation-checks.md plus active migration validation checks pass; complete inventory with no author exclusions or waivers | | `Partially Validated` | All required safety gates pass; no unresolved applicable failures/skips/blocks; at least one valid author exception or acknowledged inventory exclusion | | `Qualified` | `QUAL-00` passes with `Validated`; all applicable checks in qualification-checks.md and active migration live/human checks pass; no exceptions or excluded inventory | | `Conditionally Qualified` | `QUAL-00` and safety gates pass; all other applicable checks pass or have valid author exceptions, with no unresolved failures/skips/blocks; at least one exception exists or upstream Validation is `Partially Validated` | Anything else is not a successful rollup. Report unresolved human decisions as pending/blocked, not silently passed. `QUAL-00` itself is not waivable. `NOT_APPLICABLE` instances do not count as executed passes. State a zero-content denominator explicitly; never infer 100% coverage from it. Checklist placement determines the execution phase. For example, `QUAL-00` and the `category: validation` safety/evidence gates in qualification-checks.md are deterministic Qualification prerequisites or finalizers; they are not inputs to the upstream Validation rollup. This avoids a circular prerequisite. The migration overlay uses category to choose its base rollup as described above. `E2E-V01` can pass when every content item is packaged **or** all exclusions are explicitly acknowledged by the author in `decision-manifest.json`. The second case still forces `Partially Validated`, even though that particular check passes. Subsequent live scope is every packaged item; report the full source inventory and exclusions alongside it. Conditional outcomes never claim that excluded items or waived behavior were tested. ## Inventory and evidence contract - Inventory all solution content recursively, including renamed/deleted items, explicit package selections and referenced dependencies. Distinguish logical components from install/registration resources, supporting assets and docs. - Expand impact through parser, connector, query and package dependencies. A changed source can activate checks for an unchanged dependent output. - Detection and workbook mock validation checks existence and schema-valid JSON for **all packaged** items. It does not establish semantic adequacy. Map shared fixtures explicitly; do not require one physical file per item. - Converted AR/CD pairs use the same reviewed logical scenario. Native and unpaired content receives individual tests, not invented parity partners. - Qualification must associate query/alert/UI captures with current inputs, the selected target, a real provider, timestamps and raw evidence file references/hashes. Imported-live evidence needs verifiable collection provenance. Self-reported booleans and hashes alone do not prove execution. - Record what was checked, expected versus observed values, source file and JSON pointer/query identity, PR file/line links where valid, and where each evidence artifact can be inspected. Pin links to the reviewed commit. - Preserve reviewer terms such as **Basics**, **Manage**, **regular rules**, **XDR rules**, **Actual result**, **Pass/Fail**, and **Comments**. - Keep private lab evidence in controlled local artifacts. Publish only sanitized references/projections; do not export browser sessions or tokens. ## Known implementation boundaries The current generic `qualification/orchestrator.py`, `mock_data.py`, `parity.py` and `workbook_playwright.py` have paths that can construct successful results without real callbacks. Such results cannot satisfy `E2E-G01`. The stricter `workbook_qualification.py`, `workbook_browser.py` and `alert_parity.py` contracts are useful foundations, but do not establish universal portal coverage by themselves. Current workbook fixture harnesses can be non-ingestible and are separate from `E2E-Q04` live injected-data proof. Current batch alert parity invokes ingestion; coordinate exactly-once shared ingestion rather than ingesting both in a prior stage and again during parity. Respect schedule/ingestion timing, especially Continuous/NRT. Automatic solution-first DCR discovery and some workbook provider acceptance remain limited; do not infer support or claim it passed. Existing source-waived or environment-blocked authoring outcomes are not full live qualification. Keep the original source error and evaluate any proposed review exception under this document without weakening actual execution gates. ## Sources and reconciliation | Source key | Source / locator | |---|---| | `master-excel` | `Microsoft Sentinel Content Review Checklist - Master.xlsx` (content review team checklist; normalized into [validation-checks.md](validation-checks.md) and [qualification-checks.md](qualification-checks.md)) | | `pr-checklist-2-docx` | `Checklist 2.docx - Pull Request Validation Checklist` (PR validation checklist; normalized into [validation-checks.md](validation-checks.md) and [qualification-checks.md](qualification-checks.md)) | | `toolkit-contract` | [Toolkit README](../../README.md), canonical workflow and referenced implementation | | `user-e2e-requirement` | Maintainer requirements of 19 October 2026, including the packaged-inventory and non-waivable safety clarifications | The external checklist sources were reviewed through WorkIQ extraction, not a binary-certified workbook import. All 12 visible sheets were considered. TI Matching has five criteria (rows 4-8); its repeated row-one headers are not checks. Historical Pass/Fail and automation cell values are not current results. Source references may require internal access. Do not redistribute the source documents or private evidence as part of a public solution package. | Source section | Definitions covering the section | |---|---| | GET-STARTED | Setup references only; current authorization and execution procedures take precedence | | Data Connectors, rows 3-42 | `VAL-DC-01`, `VAL-DC-02`, `VAL-URL-01`, `QUAL-DC-01`, `QUAL-TEXT-01`, `QUAL-RELEASE-01` | | Sample data, rows 3-7 | `VAL-SAMPLE-01`, `E2E-V02`, `E2E-V03`, `QUAL-MOCK-01`, `TK-G07` | | Logo, rows 3-14 | `VAL-LOGO-01`, `E2E-Q06` | | Parsers, rows 3-9 | `VAL-PARSER-01`, `QUAL-PARSER-01`, `QUAL-TI-01`, `VAL-URL-01` | | Analytic Rule Templates, rows 3-38 | `VAL-RULE-01`, `QUAL-RULE-01`, `QUAL-TI-01`, `E2E-Q02` | | TI Matching, rows 4-8 | `QUAL-TI-01`; cross-applies to rules, hunts, parsers, workbooks and playbooks | | Hunting Queries, rows 3-38 | `VAL-RULE-01`, `QUAL-RULE-01`, `QUAL-TI-01`, `QUAL-RUNTIME-01` | | Workbooks, rows 3-29 | `VAL-WB-01`, `VAL-LOGO-01`, `TK-V07`, `E2E-V03`, `E2E-Q04..Q06`, `TK-Q06`, `QUAL-TEXT-01`, `QUAL-RELEASE-01` | | Solutions, rows 4-27 | `VAL-CI-01`, `VAL-PKG-01`, `TK-Q07`, `E2E-Q01..Q06`, `QUAL-DC-01`, `QUAL-PARSER-01`, `QUAL-PLAYBOOK-01` | | Solutions, rows 39-72 | `VAL-PR-01`, `VAL-META-01`, `VAL-PKG-02`, `VAL-URL-01`, `VAL-RELEASE-01`, `TK-V10..V13`, `QUAL-URL-01`, `QUAL-RELEASE-01` | | Playbooks, rows 3-16 | `VAL-PLAYBOOK-01`, `QUAL-PLAYBOOK-01`, `QUAL-TI-01`, `TK-G07` | | Notebooks, rows 3-19 | `VAL-NOTEBOOK-01`, `QUAL-NOTEBOOK-01`, `QUAL-URL-01`, `TK-G07` | | Checklist 2 - PR validations | `VAL-CI-01`, `QUAL-00` | | Checklist 2 - Basics / Manage | `VAL-PKG-01`, `E2E-Q05` | | Checklist 2 - XDR artifacts / regular rules | `VAL-ARTIFACT-01`, `QUAL-RUNTIME-01`, `E2E-Q02` | | Checklist 2 - package version / mainTemplate properties | `VAL-PKG-02`, `TK-V11` | | Checklist 2 - short URLs / createUiDefinition | `VAL-URL-01`, `QUAL-URL-01`, `VAL-PKG-01`, `E2E-Q05` | Toolkit families retain their IDs where possible. Migration-only `TK-V01..V05`, `TK-V08..V09`, `TK-Q01..Q03`, `TK-Q05`, `TK-Q09..Q10`, `MIG-WB-01` and `E2E-Q03` live in the overlay. General runtime/dependency, workbook interaction/readback, DCR and lifecycle checks include `QUAL-RUNTIME-01`, `TK-Q04`, `TK-Q06..Q08` and `TK-Q11`. General inventory uses `E2E-V01`, not the migrated-source denominator; `E2E-V04` reconciles final live evidence and coverage. `TK-G01` and `TK-G11` are enforced by the rollup/evidence rules; `TK-G08` by `VAL-ARTIFACT-01` and `TK-Q07`; `TK-G09`, `TK-G10`, `TK-G13`, and `TK-G14` by `VAL-OPS-01`; `TK-G12` by migration `TK-Q03`. ### Policy interpretation requiring review - Historical 50-character name criteria differ from current instruction targets/hard limits. Use the current kind-specific contract and disclose differences; never fail a hunt for missing analytic scheduling fields. - "Not in TCS scope" is an ownership marker, not a pass or universal exclusion. - Legacy V2 locations, standalone-copy guidance and post-merge gallery checks retain their source meaning but need explicit applicability. V3.1 packaging remains the hybrid route; do not downgrade content to a historical layout. - Distinguish package release version, content version, content schema version and ARM schema version. Record the exact field being compared. - Visual quality, detection usefulness, owner permissions and MSTIC/Connector team sign-off remain human responsibilities where evidence is not sufficient. ## NPF operating principles Start with a review-ready, trustworthy PR, not a target number of agents. Reuse deterministic validators, use bounded agents for semantic/visual assessment, retain named human accountability and stop when evidence is missing. Measure review effort, rework, escaped defects and cost per successful outcome. Promote repeated, adjudicated feedback into reviewed definitions and regression tests; never let an agent automatically relax this standard. --- ### CONTRIBUTING (`docs/CONTRIBUTING.md`) First off, thanks for contributing to Orion repo hosted in Azure organization! The contribution guidelines for this repository are maintained in the `Azure-Sentinel/README.md` file. Please refer to the [Contribution Guidelines](https://github.com/Azure/Azure-Sentinel/blob/70d13ec638f559d8347efa93258bbc1018d53cfb/README.md#contribution-guidelines) for the latest instructions and guidance on contributing. --- ### Entity Based Incident Triage Guide (`docs/Entity-Based-Incident-Triage-Guide.md`) # Entity-Based Incident Triage Guide for Microsoft Sentinel Analysts ## Purpose This guide helps analysts triage Microsoft Sentinel incidents consistently by focusing on entities, repeated alert patterns, identity context, network context, enrichment, documentation, escalation, and tuning decisions. It is designed to improve investigation quality while reducing avoidable analyst fatigue. ## When to Use This Guide Use this guide during: - Initial incident triage when an alert or incident is first assigned. - Repetitive alert review to determine whether repeated signals indicate noise or a pattern. - Escalation preparation when evidence suggests broader risk or impact. - Tuning review when recurring low-value incidents require detection improvements. - Analyst handover to preserve context and continuity across shifts. ## Triage Principles - Understand the signal before closing the alert. - Repetitive alerts are not always noise; they may reveal patterns. - Start from the entities involved. - Separate evidence from assumptions. - Document why an incident was closed or escalated. - Treat enrichment as supporting context, not final proof. ## Initial Analyst Checklist - Confirm incident title and severity. - Confirm alert provider and detection source. - Identify the triggered analytics rule. - Verify the relevant time range. - List all involved entities. - Review user or account context. - Review IP and network context. - Review host or device context. - Review domain, URL, and hash context. - Check related incidents for the same entities. - Record MITRE ATT&CK mapping if available. - Decide and record the recommended next action. ## Entity-Based Triage Flow ### User or Account - Is the account privileged or associated with sensitive business functions? - Is the sign-in behavior unusual for the user, device, geography, or time? - Are there failed sign-ins before a successful sign-in? - Is MFA involved, bypassed, challenged, or unexpectedly absent? - Are there recent password resets, role changes, or conditional access changes? - Are there related alerts or incidents for the same identity? ### IP Address - Is the IP internal, external, VPN, proxy, cloud, or hosting provider space? - Has this IP appeared in previous incidents or repeated alerts? - Is the IP associated with multiple users or hosts in a suspicious way? - Is geolocation or ASN context relevant to the activity pattern? - Is approved threat intelligence context available for this IP? ### Host or Device - Is the device managed and known in the organization inventory? - Is the device critical, internet-exposed, or business-sensitive? - Are there related endpoint alerts on the same host? - Are there suspicious process, network, or authentication events around the same time? - Is the device associated with repeated alerts across multiple incidents? ### Domain, URL, or FQDN - Is the domain newly observed or unusual in the environment? - Is activity potentially related to phishing, command and control, or suspicious redirection? - Are multiple users or hosts contacting the same domain or URL? - Is supporting DNS, proxy, or firewall context available for correlation? ### File Hash or File Name - Is the file observed on one device or multiple devices? - Is relevant endpoint telemetry available for execution and behavior? - Is the hash recognized by approved security tooling in the environment? - Is there evidence of execution, persistence, or lateral movement context? ## Repetitive Alert Review Flow - Cluster repeated incidents by primary entity, such as user, host, IP, domain, or hash. - Cluster by time window to identify periodic, burst, or continuous behavior. - Compare current behavior with historical baseline for the same entities. - Determine whether the pattern reflects expected activity, misconfiguration, weak tuning, or potential recurring attacker behavior. - Choose one documented outcome: - Close as benign with clear evidence. - Escalate for deeper investigation. - Suppress based on approved policy and validated low risk. - Submit a tuning request. - Submit a detection improvement request. ## Identity Investigation Prompts - Is the sign-in risk level elevated or inconsistent with normal identity behavior? - Are there impossible travel indicators supported by timeline evidence? - Did the account access unusual applications, resources, or privileged scopes? - Is there evidence of token misuse, consent abuse, or suspicious session behavior? - Are there role assignment, group membership, or access policy changes near the alert window? - Do related identities show similar patterns indicating broader targeting? ## Network Investigation Prompts - Do source and destination patterns align with known business communication paths? - Are unusual outbound destinations, ports, or protocols present? - Is DNS activity consistent with normal resolver behavior and known domains? - Do proxy logs show suspicious categories, downloads, or redirects? - Do VPN events indicate anomalous login origin, device changes, or session overlap? - Do firewall or WAF logs indicate scanning, exploitation attempts, or blocked high-risk traffic? - Is there corroborating activity across IP, DNS, proxy, VPN, firewall, and WAF telemetry? ## MITRE ATT&CK Mapping MITRE ATT&CK mapping helps analysts interpret activity as behavior patterns rather than isolated events. It supports consistent triage language, clearer escalation rationale, and better tuning prioritization. High-level tactic examples to consider: - Initial Access - Credential Access - Persistence - Defense Evasion - Discovery - Lateral Movement - Command and Control - Exfiltration Use mapping to strengthen triage documentation and to identify where detection coverage or investigation depth may need improvement. ## Safe Enrichment Preferred safe enrichment sources include: - Microsoft Sentinel incident entities - Microsoft Defender portal context - Entra ID sign-in logs - Azure Activity - Device timeline - DNS, proxy, firewall, VPN, and WAF logs available in the workspace - Internal asset inventory or approved CMDB - Approved threat intelligence connectors configured by the organization Third-party enrichment such as IP, domain, or hash reputation may be useful when approved by the organization, but it should not be a required dependency of this guide. ## Closure Notes Template Use the following template for consistent closure notes: - Summary: What happened and what was concluded. - Entities reviewed: User, IP, host, domain, URL, hash, and related entities. - Evidence reviewed: Key logs, alerts, timelines, and cross-source correlations. - Enrichment performed: Internal and approved enrichment sources used. - MITRE context: Tactics or techniques relevant to observed behavior. - Reason for closure: Why the incident is closed, with evidence-based rationale. - Business or known-good justification: Operational explanation when activity is expected. - Escalation decision: Escalated or not escalated, with justification. - Tuning recommendation: Suggested tuning or detection improvement, if applicable. ## Escalation Criteria Escalation is generally appropriate when one or more of the following apply: - Privileged identity is involved. - Multiple entities are affected. - Malicious indicator is confirmed by trusted evidence. - Suspicious successful authentication is observed. - Signs of lateral movement are present. - Unusual cloud activity suggests elevated risk. - Repeated alerts show increasing scope or impact. - Evidence is insufficient to close safely. ## Tuning Candidates An incident may be a tuning candidate when: - Activity is repeatedly confirmed as known benign. - Expected automation regularly triggers the same alert. - Detection thresholds appear misconfigured for the environment. - Entity context is missing and reduces analyst decision quality. - The rule is consistently noisy without analyst value. - Detection logic requires additional filters, grouping, or correlation context. ## What Not To Do - Do not close incidents only because they are repetitive. - Do not treat enrichment as absolute proof. - Do not investigate private individuals outside the incident scope. - Do not use leaked data sources or unapproved sources. - Do not include customer names, internal processes, secrets, or sensitive data in notes. - Do not suppress alerts without understanding the impact. - Do not escalate without documenting evidence and reasoning. ## Related Microsoft Sentinel Content This guide is intended to complement existing Microsoft Sentinel detections, hunting queries, playbooks, workbooks, and SOC process content. --- ### RequiredUpdatesForPlaybookTemplates PlaybookTemplatesRequiredChanges (`docs/RequiredUpdatesForPlaybookTemplates/PlaybookTemplatesRequiredChanges.md`) # Playbook templates update required Due to recent changes in Azure Sentinel, playbook templates should be updated to use the most recent components of the Azure Sentinel Logic Apps connector.
Below are details about the recent changes and links to relevant documentation for references. * Please find in the playbook template you have contributed the required changes listed here. * Create a new version of the playbook template. * We recommend to check if the updates in the Logic Apps connector can improve the current playbook logic (for example, use new formatted comments; add Incident URL to your external messages or tickets; reduce complexity). * When you are done, create a Pull Request with the corrections. Use the title “connector updates - {playbook name}” and list the changes. ## Introduction: Recent updates in Azure Sentinel which affects playbook templates * **Introducing incident automation: Automation Rules & new Logic Apps trigger**
* Launched in April 2021, Automation rules are a new concept in Azure Sentinel which allows users to centrally manage the automation of incident handling. [Learn more about automation rules.](https://docs.microsoft.com/azure/sentinel/automate-incident-handling-with-automation-rules) * A new Logic Apps trigger was added to Azure Sentinel connector to allow incident automation triggered by automation rules: **When Azure Sentinel incident creation rule was triggered**.
* **New actions in Logic Apps connector to replace deprecated actions**
Azure Sentinel Logic Apps connector was updated to support [incidents API](https://docs.microsoft.com/rest/api/securityinsights/incidents) capabilities, remove deprecated cases API actions and simplify usage. [Learn more about the changes](https://techcommunity.microsoft.com/t5/azure-sentinel/what-s-new-azure-sentinel-logic-apps-connector-improvements-and/ba-p/1888416).

## Required changes in existing playbook templates - summary | Old component | New component | |-------------------------------|----------------------------------| | When a response to an Azure Sentinel alert is triggered | When Azure Sentinel incident creation rule was triggered | | Alert - Get Incident | Alert - Get Incident (remove and then add action to get new version) | | Add labels to incident | Update Incident | | Remove labels from incident | Update Incident | | Change incident description | Update Incident | | Change incident severity | Update Incident | | Change incident status | Update Incident | | Change incident title | Update Incident | | Add comment to incident (V2) | Add comment to incident (V3) | | Alert – Get Accounts | Entities – Get Accounts | | Alert – Get FileHashes | Entities – Get FileHashes | | Alert – Get Hosts | Entities – Get Hosts | | Alert – Get IPs | Entities – Get IPs | | Alert – Get URLs | Entities – Get URLs |

## Required changes in existing playbook templates - extended information ### Trigger change: playbook are recommended be triggered with “When Azure Sentinel incident creation rule was triggered” **Required change:** Playbooks which start with alert trigger (“When a response to an Azure Sentinel alert is triggered”) are highly recommended to use the Incident trigger (“When Azure Sentinel incident creation rule was triggered”). **Case playbook require both incident and alert trigger vesrion:**
Since manual trigger for incident trigger not yet available, if the playbook is relevant for manual trigger please have both versions.
Folder structure in this case: * incident-trigger (folder) * images (folder) * azuredeploy.json * alert-trigger (folder) * images (folder) * azuredeploy.json * readme.md ``` /* Detailed source-code truncated for AI context efficiency. */ ``` **Additional value to consider when making the change:** * Incident trigger receives as an input: * Incident properties * Alerts and their properties * Entities * Note that you might not need to work with the alerts. For example, **Incident Alert product names** is a list of alert providers in this incident, **Incident Description, Title, Severity, Created time** can be used instead of the respective alert fields and **Incident URL** allows a direct pivot to Sentinel.
* This eliminates the need for "Alert - Get Incident". * Identify to **Update Incident**/**Add comment to incident (V3)** using **Incident ARM ID** ### Action change: "Alert - Get Incident" **Required change:** Use the latest version of "Alert - Get Incident", which works on Incidents API (instead of Cases API). **Instructions:** * Remove the old component and re-select the action “Alert – Get Incident”. * For validation, you can use “pick code” or Logic Apps code view. New version path starts with "path": "/incidents” (old version used "path": "/Cases”).
**Additional value to consider when making the change:** * Improve your logic using the new incident fields this version fetches, such as incident URL. * If required, identify to **Update Incident**/**Add comment to incident (V3)** using **Incident ARM ID**. ### Action change: Old “Change incident” actions should be replaced by “Update Incident” **Required change:**
Remove any of the following actions: * Add labels to incident * Remove labels from incident * Change incident description * Change incident severity * Change incident status * Change incident title and add **Update Incident** instead. Fill in only fields you want to change. To identify, this action requires **Incident ARM ID**. * If playbook starts with incident trigger, Incident ARM ID is a dynamic field of the trigger. * If you chose to keep the playbook to work with alert trigger, use the **Alert - Get Incident** action beforehand to get Incident ARM ID. See [usage examples](https://docs.microsoft.com/connectors/azuresentinel/#actions-on-incidents---usage-examples). New action:

Actions to be deprecated:
**Additional value to consider when making the change:** * If you had multiple changes on the incident in separated steps, you can use one step of Update Incident to do multiple changes. ### Action change: Add comment to incident should be replaced with new version (V3) **Required change:** Remove previous version of Add comment to incident, and add version V3.

To identify, this action requires **Incident ARM ID**. * If playbook starts with incident trigger, Incident ARM ID is a dynamic field of the trigger. * If you chose to keep the playbook to work with alert trigger, use the **Alert - Get Incident** action beforehand to get Incident ARM ID. See [usage examples](https://docs.microsoft.com/connectors/azuresentinel/#actions-on-incidents---usage-examples). **Additional value to consider when making the change:** * New version supports HTML and Marksdown, so you can create rich formatted comments: * Use the HTML editor for simple formatting. * Add a previous step of “Compose” built in action to implement raw HTML code. Then, use the output as an input to the comment.
### Action change: “Alert – Get {entity type}” should be replaced with “Entities – Get {entity type}” | Old name | New name | |-------------------------------|----------------------------------| | Alert – Get Accounts | Entities – Get Accounts | | Alert – Get FileHashes | Entities – Get FileHashes | | Alert – Get Hosts | Entities – Get Hosts | | Alert – Get IPs | Entities – Get IPs | | Alert – Get URLs | Entities – Get URLs | --- ### Tools SentinelToXDRMigration Docs Qualification Migration Checks (`Tools/SentinelToXDRMigration/docs/qualification/migration-checks.md`) # Sentinel-to-Defender-XDR migration overlay Activate only for a PR migrating Sentinel content to paired `XDR Detections/` or `XDR Workbooks/`, including changes to an existing migration pair. Native XDR content or folder presence alone does not activate this overlay. Apply the [scope and rollup contract](README.md). Validate pairing from provenance and PR intent; ambiguity is unresolved rather than silently skipped. Static `category: validation` checks contribute to the Validation rollup. Live `category: qualification` checks contribute to Qualification only when that profile is explicitly selected. Offline authoring never claims live parity. General validation, privacy, authorization and real-provider gates remain mandatory; this overlay does not duplicate their execution. ## TK-V01 — Complete migration source inventory Keep the full source denominator separate from the acknowledged packaged scope. ```yaml check-definition id: TK-V01 checklist: migration contentType: all title: Complete Migration Inventory and Pairing originalCriterion: Account for every source rule/workbook, target artifact, duplicate or ambiguous identity and unresolved conversion outcome. applicability: Active Sentinel-to-XDR migration; inventory every source and target in the declared migration scope. category: validation howPerformed: Reconcile recursive source inventory and hashes with conversion manifests and target provenance; retain unmigrated/failed/excluded items and reject pairing by list position or name guess. expectedEvidence: [source and target inventories/hashes, conversion manifest ref, explicit pair mappings, duplicate/ambiguity findings, excluded and unresolved outcomes] passCriteria: Migration inventory is complete and every pair is defensible without hidden failures or a reduced denominator. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; scope safety remains non-waivable. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.authoring_contract, migration manifests and planned PR migration detector ``` ## TK-V02 — Source-derived XDR identity Apply `xdr-` only to Sentinel-derived detections, not native XDR IDs. ```yaml check-definition id: TK-V02 checklist: migration contentType: xdr-detections title: Sentinel-derived XDR Identity Validation originalCriterion: Sentinel-derived properties.id is the canonical xdr- prefix plus the source Sentinel GUID with complete source provenance. applicability: Sentinel AR to XDR CD pairs in an active migration. category: validation howPerformed: Validate source GUID, Graph-compatible target identity, canonical xdr- relationship, source path/platform/kind and exact source/query provenance hashes. expectedEvidence: [source and target IDs, provenance field locations, expected canonical ID, observed ID, source/query hash comparisons] passCriteria: Each migrated CD has the required source-derived identity and current provenance. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; no deployed identity change is authorized. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.migration.converter.validate_document and XDR schema validation ``` ## TK-V03 — NRT, lookback and schedule transformation Preserve the distinction between supported transformation and proven behavior. ```yaml check-definition id: TK-V03 checklist: migration contentType: xdr-detections title: NRT and Schedule Migration Contract originalCriterion: Validate Continuous PT0S eligibility, supported cosmetic summarize rewrites and documented PT1H fallback/lookback contracts. applicability: Migrated detection pairs, especially source NRT or changed frequency/lookback. category: validation howPerformed: Inspect source kind and exact transformed query; validate eligible row-level PT0S, supported cosmetic summarize rewrite and recorded PT1H fallback for incompatible behavioral shapes; verify PT4H or other applicable target lookback evidence. expectedEvidence: [source kind/frequency/period, target frequency/lookback, original and rewritten query hashes, eligibility findings, event-time window provenance, review decision refs] passCriteria: Transformation follows the supported contract with timing changes and unresolved assumptions visible; no parity is inferred. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated without changing deployment gates. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.migration.converter NRT eligibility, cosmetic summarize rewrite and schedule validation ``` ## TK-V04 — XDR entity binding integrity Entity labels alone do not establish usable XDR evidence bindings. ```yaml check-definition id: TK-V04 checklist: migration contentType: xdr-detections title: Migrated XDR Entity Binding Integrity originalCriterion: Produce supported asset mappings and require genuine content hash columns for file/process bindings. applicability: Migrated Sentinel AR to XDR CD entity mappings. category: validation howPerformed: Check required Host/Account/Mailbox/IP mappings, identifier translation and output bindings; reject filenames or synthetic/non-content hashes substituted for required SHA1/SHA256 values; preserve supplemental context explicitly. expectedEvidence: [source and target mapping locations, query projection/schema refs, hash-column provenance, expected and observed entity bindings, unresolved review findings] passCriteria: Entity mappings satisfy the supported XDR contract without fabricated bindings or silent context loss. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; this does not prove live entity parity. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.migration.converter entity mapping and hash validation ``` ## TK-V05 — Custom-detail preservation and binding Static limits and actual output-schema verification are separate checks. ```yaml check-definition id: TK-V05 checklist: migration contentType: xdr-detections title: Migrated Custom-detail Integrity originalCriterion: Preserve custom-detail bindings, reject conflicts and missing output columns, and retain documented limits and runtime binding requirements. applicability: Migrated detection pairs with original or supplemental custom details. category: validation howPerformed: Compare source/target key-to-column bindings and final projections, check supported pair/key-size limits and record unresolved output bindings for TK-Q02; never silently truncate or overwrite. expectedEvidence: [source/target customDetails, final query hash and projections, conflict/missing-column findings, limit measurements, runtime verification requirements] passCriteria: Custom-detail preservation and static contracts pass with required runtime binding checks identified. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; schema alone does not verify runtime value size. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.migration.converter custom-detail conversion/validation and authoring.columns ``` ## TK-V08 — Exact migration package selection An explicit empty selection stays empty; excluded files remain visible. ```yaml check-definition id: TK-V08 checklist: migration contentType: solution-package title: Migration Package Selection Integrity originalCriterion: Package only explicit reviewed selections and prevent rescanning folders from reintroducing excluded detections. applicability: Active migration with explicit full or partial AR/CD or workbook package selection. category: validation howPerformed: Compare current selection plan and apply receipt with source data arrays, current artifacts and exact package/ZIP identities; preserve excluded artifacts and original failure evidence. expectedEvidence: [selection plan/hash, author decision and apply receipt refs, source data arrays, included/excluded identities, package and ZIP comparisons] passCriteria: Package matches the explicit current selection with no omitted selection evidence or reintroduced exclusions. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; existing package execution gates are unchanged. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.packaging.packaging_selection and packaging verification ``` ## TK-V09 — Migration coverage and partial-publication contract This toolkit-specific threshold is not the general E2E-V01 inventory rule. ```yaml check-definition id: TK-V09 checklist: migration contentType: all title: Migration Coverage and Partial Publication Accounting originalCriterion: Measure eligible migrated source IDs against the complete original inventory and do not claim selected-subset success as full-inventory success. applicability: Active migration; apply the strict over-50-percent threshold only to the existing partial-detection publication route, not workbook-only or native solutions. category: validation howPerformed: Use unique validated source IDs and complete source denominator; verify source/target evidence eligibility and route-specific threshold; report failed, excluded and untested identities. expectedEvidence: [full denominator and source IDs, eligible numerator and evidence refs, selected workflow route, threshold computation, excluded/failed/untested inventory] passCriteria: Applicable route-specific coverage contract is satisfied without inflated numerator or reduced denominator. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated in this review; it does not authorize a package blocked by existing toolkit gates. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.authoring_contract and solution_manager.packaging.packaging_selection ``` ## MIG-WB-01 — Workbook query transformation and provenance Preserved non-Log-Analytics providers are not silently treated as migrated KQL. ```yaml check-definition id: MIG-WB-01 checklist: migration contentType: xdr-workbooks title: Workbook Migration Query and Provenance Integrity originalCriterion: Bind source and converted workbook/query identities and preserve reviewed provider, parameter and function semantics. applicability: Sentinel workbook to XDR workbook pairs in an active migration. category: validation howPerformed: Reconcile every query/picker pointer and source hash with converted or preserved-provider outcomes; validate reviewed dependencies/functions and public-workbook provenance while keeping private receipts outside content. expectedEvidence: [source/target workbook hashes, query/picker pointer map, provider dispositions, query contracts, dependency decisions, private/public artifact refs] passCriteria: Every source query/picker is accounted for and target contracts and reviewed provenance are current. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; TK-G07 privacy cannot be waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.migration.workbooks and authoring workbook dependency/publication modules ``` ## TK-Q01 — Dual-surface exact runtime KQL Execute both sides on their actual surfaces; one side cannot stand in for both. ```yaml check-definition id: TK-Q01 checklist: migration contentType: all title: Sentinel LA and XDR Advanced Hunting Runtime Validation originalCriterion: Execute original Sentinel and converted Advanced Hunting queries with complete current-hash evidence on their respective surfaces. applicability: Active migration in Qualification for every packaged pair; native/unpaired query tests remain QUAL-RUNTIME-01. category: qualification howPerformed: Execute exact persisted source KQL in the locked Log Analytics workspace and target KQL in the selected XDR Advanced Hunting tenant with approved bindings; reconcile complete paired query inventories. expectedEvidence: [source/target query inventory and hashes, independent provider/target bindings, complete response files, schemas, errors and row counts] passCriteria: Both exact query families execute successfully and completely with current real-provider evidence. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified, never a runtime-pass claim; safety gates remain non-waivable. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.runtime Sentinel SDK and Advanced Hunting execution/import routes ``` ## TK-Q02 — Runtime output-schema binding Zero rows can establish returned schema, but cannot satisfy positive-data or alert-firing checks. ```yaml check-definition id: TK-Q02 checklist: migration contentType: all title: Migrated Runtime Output-schema Binding originalCriterion: Verify actual target query output schema for custom details and required migrated query/entity bindings. applicability: Active migration Qualification with schema-bound detection/workbook outputs, especially custom details. category: qualification howPerformed: Compare actual returned provider schemas for exact current queries with required columns/types and reviewed bindings; retain absent or mismatched columns even when execution itself succeeds. expectedEvidence: [query hashes, actual schema arrays, required binding/type inventory, expected versus observed columns, raw provider response refs] passCriteria: Required runtime columns/types and bindings are proven by actual schemas; column counts or fabricated schemas are insufficient. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; E2E-G01 cannot be waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.runtime and migration.converter runtime custom-detail binding checks ``` ## TK-Q03 — Complete provider families and honest failure classification Retain raw failures and source-waived outcomes. Authoring exceptions are not successful live Qualification. ```yaml check-definition id: TK-Q03 checklist: migration contentType: all title: Provider-family Completeness and Failure Classification originalCriterion: Do not mix providers within a query family or use fallback to hide genuine KQL errors; preserve narrow source-waived evidence. applicability: Active migration runtime validation, fallback/import and reused authoring evidence. category: qualification howPerformed: Check complete-family provider/run/target/query bindings; permit documented complete-family reruns only for eligible provider failures; retain IdentityInfo source-waived raw errors and all unexecuted/failed outcomes. expectedEvidence: [family inventory, provider and dispatch bindings, original and retry response refs, failure classifications, exact source-waiver reason if present, current query hashes] passCriteria: Current paired families have genuine complete successful execution without disguised errors or unsupported fallback. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; raw source-waived status remains distinct and real-provider/target/authorization gates cannot be waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.runtime, reporting.runtime_report_references and migration.source_runtime_waiver ``` ## TK-Q05 — Exact paired workbook fixture assertions Non-ingestible fixture harnesses are supplementary; E2E-Q04 still requires authored queries against injected live data. ```yaml check-definition id: TK-Q05 checklist: migration contentType: xdr-workbooks title: Paired Workbook Exact-result Qualification originalCriterion: Run reviewed workbook fixtures through independent Sentinel and Advanced Hunting providers and compare exact expected results. applicability: Active workbook migration with a supported reviewed paired fixture family in Qualification. category: qualification howPerformed: Execute the hash-bound source/target harness family through independent providers and assert schema, cells, multiplicity and ordering against reviewed expected results. expectedEvidence: [fixture config/provenance, source/target workbook hashes, exact harness query hashes, independent provider response refs, expected and observed result assertions] passCriteria: Every applicable fixture case passes exact assertions with current real-provider evidence; no DCR injection claim follows. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; real-provider safety cannot be waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.workbook_qualification and migration._workbook_core.workbook_fixtures ``` ## TK-Q09 — Shared-fixture marked query parity Do not compare unrelated rows or pair files by their directory order. ```yaml check-definition id: TK-Q09 checklist: migration contentType: xdr-detections title: Shared Scenario Source-row Query Parity originalCriterion: Exact AR and CD queries observe the same unique scenario markers and identical expected matching source-row keys. applicability: Every packaged Sentinel AR to XDR CD pair in active migration Qualification. category: qualification howPerformed: Bind each pair to one reviewed logical mock fixture and coordinated injection receipt; compare exact query outputs by defined stable keys and expected malicious/benign scenarios on both surfaces. expectedEvidence: [provenance pair map, shared fixture hash, injection ledger ref, expected match keys, source/target exact query hashes and result refs, missing/extra keys] passCriteria: Both queries observe the expected marked source records without missing or unexpected scenario matches. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; synthetic sample-row fallback cannot satisfy E2E-G01. sourceChecklist: toolkit-contract automationStatus: automated responsible: Planned real marked-query parity adapter; generic qualification.parity fallback is not acceptable evidence ``` ## TK-Q10 — Strict alert comparison dimensions This comparison is shared with E2E-Q03; do not run a second injection or activation merely to produce another check result. ```yaml check-definition id: TK-Q10 checklist: migration contentType: xdr-detections title: Strict Sentinel and XDR Alert Comparison originalCriterion: Compare alert counts, match keys, expected malicious matches, severity, tactics, entities and decisive evidence. applicability: Each packaged migrated AR/CD pair with real captured alerts. category: qualification howPerformed: Normalize independently captured alerts, reject duplicate or missing detection/match keys, and compare all defined dimensions against nonempty reviewed expected key sets. expectedEvidence: [raw source/target alert capture refs, alert IDs, normalized captures, expected keys, dimension-level comparisons, missing/extra/benign matches] passCriteria: Every strict comparison dimension passes and expected positive alerts exist on both sides; empty-equals-empty is not success. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; live capture provenance and safety cannot be waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.alert_parity.compare_alert_captures with real capture provenance checks ``` ## E2E-Q03 — Complete migrated-pair alert parity This is an overlay over individual firing, not a substitute for E2E-Q02. Unpaired packaged detections still need their individual firing tests. ```yaml check-definition id: E2E-Q03 checklist: migration contentType: xdr-detections title: Sentinel AR to XDR CD End-to-end Alert Parity originalCriterion: Sentinel and XDR detection alerts actually fire for every migrated pair from shared mock injection and satisfy strict alert parity. applicability: Every packaged source-provenance AR/CD pair in active migration Qualification. category: qualification howPerformed: Reconcile full packaged pair inventory with E2E-Q01 injection, E2E-Q02 firing, TK-Q09 row parity, TK-Q10 alert comparison and TK-Q11 controlled lifecycle/cleanup evidence; prohibit separate AR/CD payloads or accidental duplicate ingestion. expectedEvidence: [packaged pair inventory, shared fixture/injection refs, source and target alert IDs, row and alert comparison refs, per-pair coverage results, final disabled-state receipts] passCriteria: Every packaged migrated pair has positive scenario-correlated alerts, strict parity and safe cleanup from the same reviewed injection. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified with failed parity visible; authorization/privacy/target/real-provider and cleanup gates cannot be waived. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned pair-coverage rollup reusing qualification.alert_parity start/complete/abort and real query/alert collectors ``` --- ### Tools SentinelToXDRMigration Docs Qualification Qualification Checks (`Tools/SentinelToXDRMigration/docs/qualification/qualification-checks.md`) # Qualification checklist General solution qualification, not a migration prerequisite for native content. Apply the [result, exception and safety contract](README.md#results-author-evidence-and-rollups). Every live action requires the selected non-production target and its own approved scope. A definition or exception is not execution permission. ## QUAL-00 — Validation checklist passed This is the first check and a mandatory, non-waivable prerequisite. A partial Validation rollup permits only a conditional Qualification rollup. ```yaml check-definition id: QUAL-00 checklist: qualification contentType: all title: Validation checklist passed originalCriterion: Validation checklist rollup must be Validated or Partially Validated before Qualification can pass. applicability: Every Qualification run, including native content and active migration overlays. category: validation howPerformed: Verify the current Validation rollup, check-instance results, acknowledged exclusions and exceptions against the same head/input hashes; include active migration validation checks. expectedEvidence: [Validation rollup artifact ref, head and input hashes, check result inventory, exclusion and exception decision refs, safety gate results] passCriteria: Current Validation rollup is Validated or Partially Validated with all safety gates passed. Partially Validated forces Conditionally Qualified downstream. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; fix the upstream rollup instead. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned PR check runner rollup evaluator ``` ## E2E-G01 — No simulated fallbacks in Qualification mode Non-waivable. Missing real callbacks, dry-run stubs and locally fabricated successes cannot establish deployment, ingestion, query, alert or UI proof. ```yaml check-definition id: E2E-G01 checklist: qualification contentType: all title: No simulated fallbacks in Qualification mode originalCriterion: Verify real Azure, KQL and browser providers were used and block dry-run stubs. applicability: Every live Qualification check and every imported-live evidence artifact. category: validation howPerformed: Verify real provider readiness for every applicable live step and actual provider/collector identity, target, request/query hash, response/capture references and timestamps for every live claim; reject simulation, missing providers and self-reported success booleans without verifiable observations. expectedEvidence: [provider and collector identity, live request/response or browser capture refs, target binding, input/query hashes, execution timestamps, evidence mode] passCriteria: Applicable live steps have verified real providers and all live claims have current verifiable real-provider evidence; imported-live evidence includes collection provenance. Functional waivers make no live claim and cannot excuse missing providers or conceal attempted simulation. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; simulation never yields Qualified or Conditionally Qualified. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned real-provider evidence gate around runtime, ARM, DCR, alert and browser adapters ``` ## TK-G02 — Locked target scope and identity Non-waivable. A workspace in one surface does not establish another surface's tenant or data availability. ```yaml check-definition id: TK-G02 checklist: qualification contentType: all title: Locked Non-production Target Verification originalCriterion: Bind the selected tenant, subscription, workspace and exact resource identities and reject scope mismatches. applicability: Every lab read, deployment, injection, alert action and browser session. category: validation howPerformed: Compare persisted target, authenticated identity, request resource IDs and observed portal/provider identities immediately before dispatch and when accepting results. expectedEvidence: [private target config ref, approved resource scope, authenticated identity binding, expected and observed IDs, target verification receipt] passCriteria: Every operation and observation matches the approved non-production scope. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; wrong or unknown targets block execution and successful rollups. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.core.target_context, runtime and workbook_preflight; planned cross-provider scope adapter ``` ## TK-G03 — Explicit operation authorization Non-waivable. Review acceptance and author exceptions never grant permissions. ```yaml check-definition id: TK-G03 checklist: qualification contentType: all title: Explicit Exact-scope Authorization originalCriterion: Require separate current approvals for reads, browser execution, deployment, ingestion, activation and cleanup. applicability: Every Qualification action; re-evaluate changed or revoked scope. category: validation howPerformed: Verify affirmative actor/reference-bound consent for the exact operation and target, freshness and revocation; require immediate write approval where the canonical route requires it. expectedEvidence: [operation plan ref and hash, approval actor/reference, allowed resources and operations, revocation state, immediate confirmation receipt when required] passCriteria: Applicable current authorization exists before dispatch and remains valid. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; author justification cannot authorize a missing read/write scope. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.runtime_consent, deployment_planning and workbook preflight/UI consent gates ``` ## TK-G04 — Effective permissions and readiness Non-waivable. Cached authentication and declarations are not effective access. ```yaml check-definition id: TK-G04 checklist: qualification contentType: all title: Effective Access and Provider Readiness originalCriterion: Inspect effective permissions, applicable roles and deny assignments separately from approval. applicability: Each selected provider operation, including conditional DCR, DCE or table creation. category: qualification howPerformed: Use approved read preflight to inspect actual access and provider contracts; report query, deployment, ingestion and conditional provisioning permissions separately. expectedEvidence: [preflight response refs, principal/target binding, role and deny observations, per-operation permission results, unresolved provider contract findings] passCriteria: Required access and provider readiness are established for each action. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; denied or unknown access blocks that action. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.graph_readiness, dcr_readiness and workbook_preflight ``` ## QUAL-MOCK-01 — Mock scenario semantic adequacy Validation checks files and schemas; this check establishes whether fixtures exercise each packaged detection and workbook query meaningfully. ```yaml check-definition id: QUAL-MOCK-01 checklist: qualification contentType: all title: Mock Scenario Semantic Qualification originalCriterion: Sample values support all data types and scenarios and represent the product-appliance data experience. applicability: All packaged Sentinel ARs, XDR CDs, Sentinel workbooks and XDR workbooks. category: qualification howPerformed: Review scenario-to-item/query mappings, joins, thresholds, sequences, time windows, baselines, parameter states and positive/benign controls; define expected results before execution. expectedEvidence: [fixture manifest and hashes, item/query coverage map, expected malicious and benign keys, parameter/time bindings, semantic review decisions and evidence refs] passCriteria: Each packaged item has a defensible scenario and independently reviewed expected behavior. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified, not proven semantic coverage. sourceChecklist: master-excel automationStatus: agent-qualified responsible: Mock-data qualification agent with content owner review; existing scenario generation is an input, not the oracle ``` ## TK-Q08 — DCR stream and destination compatibility Do not confuse public table schemas with supported writable input streams. ```yaml check-definition id: TK-Q08 checklist: qualification contentType: all title: DCR Ingestion Contract Compatibility originalCriterion: Prove the exact stream, transform, destination schema and endpoint are compatible with the shared fixture. applicability: Every fixture/table requiring live DCR injection. category: qualification howPerformed: Inspect deployed approved DCR resources and public Custom-* input streams, transformKql, table schema and endpoint; use a solution DCR only when compatibility is proven. expectedEvidence: [DCR resource and immutable ID refs, input stream, transform and destination schema, endpoint ref, resource/contract fingerprints, fixture schema comparison] passCriteria: The real ingestion contract supports the fixture and target without modifying an unrelated connector or substituting a custom lookalike native table. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; scope/access/privacy gates remain non-waivable. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.dcr_readiness and approved registered-contract resolver; solution-first automatic discovery remains planned ``` ## E2E-Q01 — Live DCR injection and table landing Require marker-bound data visibility, not just ingestion acceptance or a background table count. Verify each relevant query surface independently. ```yaml check-definition id: E2E-Q01 checklist: qualification contentType: all title: Live DCR Mock Injection and Table Landing Verification originalCriterion: Mock data is injected into the live lab workspace via DCR and independently verified in the destination tables. applicability: All approved fixtures for packaged detections and workbooks; migration is not required. category: qualification howPerformed: Submit each approved shared payload exactly once through a verified real DCR route; query exact scenario markers and expected row keys/types on required Sentinel and XDR surfaces within bounded visibility windows. expectedEvidence: [payload and contract hashes, approved scope ref, real ingestion response ref, submission/deduplication ledger, marker-filtered query text and hashes, expected and observed row keys/counts, provider/target/timestamps] passCriteria: Ingestion is accepted and complete expected marked data is visible in each required surface. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; no actual injection claim follows and E2E-G01/TK-G02/TK-G03/TK-G07 cannot be waived. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Approved AzureMonitorLogsIngestion route and planned live marker-bound landing adapter; generic mock_data fallback is not evidence ``` ## QUAL-RUNTIME-01 — Every packaged query runs Checklist 2 requires regular rules to be tested when XDR exists. This general check also covers source-only/native content and non-detection queries. ```yaml check-definition id: QUAL-RUNTIME-01 checklist: qualification contentType: all title: Packaged Content Runtime Query Validation originalCriterion: Test regular rules when the PR contains XDR detections; installed hunting and connector queries run without errors. applicability: Every packaged AR/CD and applicable hunting, parser, connector and workbook query, including impacted dependencies. category: qualification howPerformed: Execute exact persisted queries with approved parameter bindings against their actual intended providers; enumerate every required query, record complete responses and compare actual output schemas with required entity/custom-detail, projection and visualization bindings. expectedEvidence: [packaged query inventory, query hashes and bindings, provider/target refs, execution response files, schemas, row counts, per-query errors] passCriteria: Every applicable exact query executes successfully with complete current evidence; zero rows establish execution only, not E2E-Q02 or E2E-Q04. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; safety gates remain mandatory. sourceChecklist: pr-checklist-2-docx automationStatus: automated responsible: solution_manager.qualification.runtime and planned general content query adapters ``` ## E2E-Q02 — Every packaged detection actually fires Require individual firing proof for native/unpaired detections. Migration parity is an additional overlay, never a reason to omit unpaired content. ```yaml check-definition id: E2E-Q02 checklist: qualification contentType: all title: Every Packaged Sentinel and XDR Detection Fires originalCriterion: Every packaged Sentinel AR and XDR CD starts disabled, is enabled, fires at least one scenario-correlated alert from injected mock data and is returned to disabled. applicability: Every packaged Sentinel Analytic Rule and Defender XDR Custom Detection. category: qualification howPerformed: Under TK-Q11 lifecycle control, activate the exact approved detection identities and coordinate fixture timing; capture real expected alerts within schedule-aware windows and correlate them to injected scenario keys. expectedEvidence: [packaged detection IDs, TK-Q11 state/approval receipts, fixture/injection refs, observation window, actual alert IDs and raw capture refs, scenario match keys, final disabled readback] passCriteria: Every packaged detection produces at least one expected scenario-correlated alert and TK-Q11 passes. Firing FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified without claiming it fired; activation safety and disabled-state cleanup cannot be waived. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned all-detection firing adapter reusing solution_manager.qualification.alert_parity lifecycle and real alert collectors ``` ## TK-Q11 — Controlled activation and safe completion Non-waivable safety gate. Reuse E2E-Q02 evidence without duplicating activation. ```yaml check-definition id: TK-Q11 checklist: qualification contentType: all title: Controlled Alert Lifecycle and Disabled-state Cleanup originalCriterion: Verify disabled starting state, enable only reviewed detections and disable every test detection after success or failure. applicability: Every packaged detection participating in live firing or migration parity. category: qualification howPerformed: Verify exact identities and initial disabled state, scoped activation consent, controlled enablement and bounded observation; execute approved complete/abort cleanup and independently read back disabled state. expectedEvidence: [initial state captures, activation/cleanup approvals, exact detection IDs, enablement receipts, completion or abort record, final disabled-state response refs] passCriteria: All test detections start disabled, activation stays within scope and all return to disabled with real readback. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; incomplete safe cleanup blocks a successful rollup. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.alert_parity complete/abort and planned native/unpaired detection lifecycle adapter ``` ## E2E-Q04 — Every workbook query returns injected scenario data Positive row count is necessary but insufficient. A one-row count of zero is not useful scenario data. Keep intended empty negative tests separate. ```yaml check-definition id: E2E-Q04 checklist: qualification contentType: workbooks title: Every Workbook Query Returns Injected Data originalCriterion: Every Workbook query executes against injected data and returns rowCount greater than zero with non-zero scenario values. applicability: Every runnable query in every packaged Sentinel or XDR workbook, including nested, hidden, picker and parameter queries. category: qualification howPerformed: Execute exact authored queries with reviewed parameter/time bindings against real providers after verified injection; compare schema/results with expected scenario values and demonstrate injected-data contribution. expectedEvidence: [complete workbook/query inventory, query and binding hashes, fixture and landing refs, provider responses, rowCount, aggregate/scenario values, expected versus observed results] passCriteria: Each query has a complete successful response with rowCount greater than zero and meaningful non-zero expected scenario values, not a zero-count aggregate or unrelated rows. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; non-ingestible providers are disclosed, not silently passed. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned workbook live-data adapter reusing workbook_qualification execution contracts; offline fixture harnesses do not satisfy this check ``` ## TK-G06 — Approved browser plan and session Non-waivable. Portal traversal is not authorization to configure or save content. ```yaml check-definition id: TK-G06 checklist: qualification contentType: all title: Approved Browser Scope and Session Verification originalCriterion: Use reviewed exact URLs, selectors, expected identities, bounded actions and network scope without Save/Edit or automatic sign-in. applicability: All UI-agent checks for detections, workbooks, connectors and their assets. category: validation howPerformed: Validate browser plan, private session binding, observed tenant/content identities, navigation and read-query requests, timeout bounds and prohibited controls before and during traversal. expectedEvidence: [reviewed browser plan/hash, approval ref, private session hash reference, identity observations, allowed actions/requests, prohibited-control findings] passCriteria: Browser activity remains in the authorized target and read/observe interaction scope with no unapproved writes. FAIL, SKIPPED or BLOCKED cannot become WAIVED_WITH_AUTHOR_EVIDENCE; this is not a browser-wide no-write sandbox guarantee. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.workbook_browser and planned cross-content browser-plan adapter ``` ## E2E-Q05 — Portal traversal of all user-facing content Covers Master solution UI and Checklist 2 Basics/Manage terminology. Enumerate required states rather than claiming an unbounded crawl is complete. ```yaml check-definition id: E2E-Q05 checklist: qualification contentType: all title: UI Agent Traverses All User-facing Portal Content originalCriterion: Traverse Detections, Workbooks and Data Connectors; Basics displays correct counts and Manage lists regular rules and XDR rules. applicability: Every packaged detection, workbook and data connector and associated solution/package UI. category: qualification howPerformed: Use a reviewed content-to-view/state manifest to visit list/detail pages, Basics, Manage, workbook tabs/conditional sections and connector instructions/next steps; wait for lazy content and reconcile inspected versus required states. expectedEvidence: [portal view/state inventory, exact content IDs and target, expected versus observed Basics counts and Manage entries, per-view screenshots/DOM refs, loading/query/JS errors, missing-view list] passCriteria: Every required user-facing view/state is visited and correctly displays expected content without unresolved loading or errors. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; TK-G06 and other safety gates cannot be waived. sourceChecklist: user-e2e-requirement automationStatus: agent-qualified responsible: Planned cross-content portal UI agent reusing reviewed workbook_browser contracts ``` ## E2E-Q06 — Images, logos, diagrams and visualization rendering Use element-appropriate assertions. `naturalWidth` applies to image elements, not inline SVG or canvas; HTTP status applies to fetched assets. ```yaml check-definition id: E2E-Q06 checklist: qualification contentType: all title: UI Images Logos SVG Diagrams and Visualizations Render originalCriterion: All images, logos, SVG diagrams and workbook visualizations render properly with naturalWidth greater than zero, HTTP 200 and zero broken assets. applicability: All assets and visualizations in the E2E-Q05 view/state inventory, including lazy-loaded and conditional content. category: qualification howPerformed: For image elements require successful decode and naturalWidth greater than zero; verify fetched assets return HTTP 200 using a reviewed cache-aware capture; inspect inline SVG/canvas bounds and rendered content plus workbook visualizations and screenshots. expectedEvidence: [asset-to-view inventory, asset URLs or inline element IDs, HTTP status/response refs for fetched assets, image decode and naturalWidth observations, SVG/canvas render assertions, screenshots, broken asset count] passCriteria: All assets render as intended with zero broken assets; image elements have positive naturalWidth, fetched assets have evidenced HTTP 200, and inline diagrams/visualizations have positive render proof. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; safety gates remain mandatory. sourceChecklist: user-e2e-requirement automationStatus: agent-qualified responsible: Planned UI asset/rendering agent and deterministic browser asset probes ``` ## TK-Q06 — Workbook interactions All seven existing workbook interaction kinds remain required, independently of whether a screenshot looks correct. ```yaml check-definition id: TK-Q06 checklist: qualification contentType: workbooks title: Workbook Picker Filter Navigation and Layout Qualification originalCriterion: Verify pickers, dependent refresh, filters, drill-down, reload, layout and conditional visibility with reviewed expectations. applicability: Every packaged interactive Sentinel or XDR workbook; static-only non-applicability needs explicit evidence. category: qualification howPerformed: Execute reviewed non-edit browser steps for each applicable interaction kind and compare DOM values, layout and dependent results with predetermined expectations. expectedEvidence: [workbook identity and hash, seven-kind coverage inventory, reviewed step plan, expected and actual assertions, screenshots and browser trace refs] passCriteria: Every applicable reviewed interaction passes without query/JS errors or prohibited actions. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; browser authorization cannot be waived. sourceChecklist: toolkit-contract automationStatus: agent-qualified responsible: solution_manager.qualification.workbook_qualification and workbook_browser; Sentinel/general adapters planned ``` ## TK-Q07 — Installation, registration and managed updates Template success is not proof of installed content, provenance or update behavior. ```yaml check-definition id: TK-Q07 checklist: qualification contentType: solution-package title: Live Installation Registration and Managed Update Verification originalCriterion: Verify independent installation and registration/provenance readback and repeat-install/update contracts. applicability: Approved live package acceptance, including packaged content and managed XDR workbook/detection updates. category: qualification howPerformed: Perform approved installation and exact resource readback; compare content/package identity/version and selected inventory; verify same-ID managed updates without touching separate personal copies. expectedEvidence: [candidate/package refs, deployment response, independent installed resource and registration readback, identity/version comparisons, repeat-install/update receipts, personal-copy exclusion evidence] passCriteria: Selected content is installed and registered as contracted and applicable managed-update proof passes. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; unverified provider authentication/write scope still blocks dispatch. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.workbook_validation, qualification.arm_acceptance and deployment adapters ``` ## TK-Q04 — Live dependency metadata Input metadata is neither output lineage nor connector installation evidence. ```yaml check-definition id: TK-Q04 checklist: qualification contentType: all title: Live Dependency Metadata Reconciliation originalCriterion: Compare exact-query live input metadata with declared workloads, connectors and data types. applicability: Content with unresolved or changed data dependencies where approved live discovery is selected. category: qualification howPerformed: Capture exact-query input metadata on the intended surface with reviewed bindings; compare declarations, retain partial failures and distinguish documented conservative scanner fallback from observed metadata. expectedEvidence: [query hash and parameter bindings, provider/target ref, raw input metadata, declared versus observed dependencies, fallback classification, review decision refs] passCriteria: Applicable dependencies are reconciled with current complete evidence and explicit decisions. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; privacy and read consent remain non-waivable. sourceChecklist: toolkit-contract automationStatus: agent-qualified responsible: solution_manager.authoring.dependencies, workbook_dependencies and dependency-helper agent ``` ## QUAL-URL-01 — Open short URLs and verify expected destinations Cover mainTemplate and createUiDefinition; the existing hyperlink script's Package exclusions must not hide these checks. ```yaml check-definition id: QUAL-URL-01 checklist: qualification contentType: all title: Short URL and Expected Destination Qualification originalCriterion: Open or hit updated short URLs and verify that they return the expected result. applicability: Short URLs in package files and changed content links, including links affected by moved/deleted targets. category: qualification howPerformed: Follow approved bounded redirects, capture final response/destination and compare content with the expected product, documentation or deployment artifact; do not deploy by following a button. expectedEvidence: [source URL inventory and file locations, old/new URLs, redirect chains, HTTP outcomes, expected and observed destination identity/content, capture refs] passCriteria: Required links resolve to the correct accessible content, not merely an HTTP success or sign-in/error page. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; unsafe navigation or credential disclosure cannot be waived. sourceChecklist: pr-checklist-2-docx automationStatus: agent-qualified responsible: Planned package-inclusive URL qualification adapter and content reviewer ``` ## QUAL-RULE-01 — Rule and hunt semantic review Syntax, successful execution and firing are separate from detecting the intended behavior. This check does not require a migration. ```yaml check-definition id: QUAL-RULE-01 checklist: qualification contentType: all title: Analytic Rule and Hunting Query Semantic Review originalCriterion: Description matches query intent; frequency, period, trigger and threshold match the expected outcome and entity mappings are meaningful. applicability: Packaged or changed analytic rules, XDR detections and hunting queries; apply only kind-appropriate criteria. category: human-review howPerformed: Review attacker behavior, telemetry, query operators/case handling, aggregation/projected context, timing, MITRE meaning, entity coverage and suitability as a rule or hunt using runtime/scenario evidence. expectedEvidence: [content and query refs, expected behavior, positive/negative results, timing/threshold rationale, MITRE/entity rationale, named reviewer decision] passCriteria: The reviewer confirms defensible behavior, appropriate kind-specific settings and understandable context. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; this does not erase technical or safety failures. sourceChecklist: master-excel automationStatus: human-review responsible: Detection/hunting content reviewer assisted by a bounded qualification agent ``` ## QUAL-TI-01 — Threat intelligence matching semantics Cross-apply the five TI Matching criteria; retain historical "Not in TCS scope" as an ownership marker, not an automatic exclusion. ```yaml check-definition id: QUAL-TI-01 checklist: qualification contentType: all title: Threat Intelligence Matching Qualification originalCriterion: Filter inactive indicators, use applicable IP indicators and fields, and select the latest indicator before checking active state. applicability: Rules, hunts, parsers, workbooks or playbooks using threat intelligence matching, including current equivalents of historical TI tables. category: qualification howPerformed: Inspect query ordering and indicator semantics and test active/inactive, duplicate/latest, non-IP and alternate relevant IP-field scenarios against the current TI contract. expectedEvidence: [TI query locations, current table/schema contract, scenario fixtures, expected versus actual indicator matches, reviewer rationale] passCriteria: Applicable TI scenarios and reviewed matching semantics pass without treating stale indicators as current. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; scope/privacy gates remain mandatory. sourceChecklist: master-excel automationStatus: agent-qualified responsible: TI content reviewer and planned scenario qualification adapter ``` ## QUAL-PARSER-01 — Parser deployment, output and ASIM mapping Test actual output fields and filter behavior, not merely function creation. ```yaml check-definition id: QUAL-PARSER-01 checklist: qualification contentType: parsers title: Parser Runtime and ASIM Mapping Qualification originalCriterion: Parser deploys, runs on sample data and returns extended fields; relevant source fields map correctly to ASIM. applicability: Packaged or changed parsers and affected consumers, including parameterized ASIM parsers. category: qualification howPerformed: Use approved installation/read routes, execute representative samples and filter cases, and compare returned schema/values with reviewed generic or ASIM expectations. expectedEvidence: [installed function identity, parser/sample hashes, query and filter cases, expected versus actual fields/types/values, deployment and runtime response refs] passCriteria: Applicable parser deployment, outputs and filter/mapping assertions pass. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; authorization and real-provider requirements cannot be waived. sourceChecklist: master-excel automationStatus: agent-qualified responsible: Existing ASIM validation tools and planned general parser live adapter ``` ## QUAL-DC-01 — Connector runtime and operational behavior Successful invocation alone does not establish correct paging or complete data. ```yaml check-definition id: QUAL-DC-01 checklist: qualification contentType: data-connectors title: Data Connector Operational Qualification originalCriterion: Connector queries work; Azure Function deployment and execution are checked, including authentication, filtering, pagination, batching and invocation logs. applicability: Packaged connectors and changed connector execution/deployment files. category: qualification howPerformed: Under approved scope verify connector visibility and runtime queries, applicable function/custom-connector installation, supported authentication and representative ingestion behavior; inspect full logs and document every warning/error. expectedEvidence: [connector and function IDs, approved scenario, deployment/invocation response refs, expected and observed pages/records, query results, complete log findings] passCriteria: Applicable connector behaviors and expected records are verified with no unexplained failures; a historical 403 exception is not a blanket access waiver. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified. sourceChecklist: master-excel automationStatus: agent-qualified responsible: Planned connector/function qualification adapter and connector reviewer ``` ## QUAL-PLAYBOOK-01 — Playbook deployment and user experience Do not trigger external actions without separate scenario-specific authorization. ```yaml check-definition id: QUAL-PLAYBOOK-01 checklist: qualification contentType: playbooks title: Playbook Deployment and Functional Qualification originalCriterion: Playbooks deploy successfully and appear in Template gallery, Active playbooks, Manual trigger and Automation rule experiences. applicability: Packaged or changed playbooks and their connection resources. category: qualification howPerformed: Verify approved deployment, connection resources and relevant UI surfaces; exercise explicitly approved bounded scenarios and assess errors, authentication setup and Azure/Gov deployment links where supported. expectedEvidence: [template and connection identities, deployment/readback refs, gallery/trigger screenshots, approved action scope, scenario outcomes, connection and error findings] passCriteria: Applicable deployment, visibility and functional scenarios pass with authorized connections/actions. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; authorization/privacy gates remain mandatory. sourceChecklist: master-excel automationStatus: agent-qualified responsible: Planned playbook qualification adapter and playbook reviewer ``` ## QUAL-NOTEBOOK-01 — Notebook quality, execution and exceptions MSTIC approvals for relative data/external code remain explicit human decisions. ```yaml check-definition id: QUAL-NOTEBOOK-01 checklist: qualification contentType: notebooks title: Notebook Usability Runtime and MSTIC Review originalCriterion: Notebook purpose and logic are understandable, handles null results, uses existing capabilities and is tested on AML Notebook VM. applicability: Packaged or changed notebooks and referenced data/code/APIs. category: human-review howPerformed: Review explanations, naming, readable data presentation, msticpy reuse, configuration and API onboarding; inspect approved AML or explicitly reviewed equivalent execution evidence and obtain applicable MSTIC external-file sign-off. expectedEvidence: [notebook/cell refs, approved execution environment, positive/null-result outputs, dependency/configuration review, privacy check ref, MSTIC decision refs where required] passCriteria: Named reviewer accepts usability and applicable runtime evidence, with required external-code/data approvals. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; privacy or execution authorization cannot be waived. sourceChecklist: master-excel automationStatus: human-review responsible: Notebook/MSTIC reviewer assisted by a bounded notebook qualification agent ``` ## QUAL-TEXT-01 — Descriptions, branding and visual usefulness Preserve source requirements such as meaningful descriptions, four workbook charts and 75px logos; explicitly adjudicate legacy/current policy differences. ```yaml check-definition id: QUAL-TEXT-01 checklist: qualification contentType: all title: Content Explanation and Presentation Review originalCriterion: Descriptions are meaningful and grammatically correct, provider/appliance names exist, and images and workbook content are useful. applicability: Changed user-facing descriptions, instructions, previews, charts, logos and product references. category: human-review howPerformed: Review product identity, supported technology descriptions, prerequisites, Preview labels, grammar, appropriate visualizations and legibility using current documentation and UI evidence. expectedEvidence: [text/asset source locations, product/reference evidence, expected and observed presentation, chart inventory, 75px logo capture where applicable, reviewer decision] passCriteria: Reviewer confirms clear accurate content and resolves applicable legacy presentation criteria explicitly. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; privacy and source access restrictions remain non-waivable. sourceChecklist: master-excel automationStatus: human-review responsible: Content review owner assisted by text and UI qualification agents ``` ## QUAL-RELEASE-01 — Ownership, publication and removal decisions Post-merge checks retain their lifecycle phase; do not claim them completed before publication. No publishing or short-link creation is authorized here. ```yaml check-definition id: QUAL-RELEASE-01 checklist: qualification contentType: solution-package title: Ownership Publication and Removal Review originalCriterion: Confirm solution ownership, Partner Center listing requirements, post-merge gallery/shortlinks and Connector team approval before removing a Function App connector. applicability: Relevant new/update releases, Microsoft-supported offer changes, connector removals, ownership changes and post-merge publication verification. category: human-review howPerformed: Verify authorized owner decisions, deletion/conflict-resolution impact, offer identity, Sentinel keyword/CSP settings and listing-update evidence; track gallery and short-link follow-up at the correct release phase. expectedEvidence: [release phase, ownership and team approval refs, deletion impact, Partner Center identity/settings evidence, listing-update receipt, post-merge gallery/link observations or pending obligations] passCriteria: Applicable phase-specific publication and team decisions are evidenced; future obligations remain visible and cannot be claimed passed. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Conditionally Qualified; actual operation authorization cannot be waived. sourceChecklist: master-excel automationStatus: human-review responsible: Solution owner, Connector team, content review team and release manager ``` ## E2E-V04 — Qualification coverage and final evidence reconciliation The full source inventory remains visible even when the packaged subset is smaller. Conditional outcomes must never be presented as full coverage. ```yaml check-definition id: E2E-V04 checklist: qualification contentType: all title: Qualification Coverage and Evidence Reconciliation originalCriterion: Reconcile every required item and check against real evidence; missing, simulated or skipped results are not successful live checks. applicability: Final rollup for every Qualification run, including active migration checks and newly captured privacy-sensitive evidence. category: validation howPerformed: Reconcile packaged detection, workbook/query, portal-view and asset denominators with check instances; recheck safety/privacy, freshness, cleanup and exception records; preserve excluded source items and prior outcomes. expectedEvidence: [full and packaged inventories, expected/tested/passed/missing identities, all check result refs, exception manifest ref, safety and cleanup receipts, final sanitized evidence index] passCriteria: Qualified requires Validated and every applicable check PASS with no exclusions/waivers. Valid author evidence for non-safety FAIL or SKIPPED may yield WAIVED_WITH_AUTHOR_EVIDENCE and Conditionally Qualified; Partially Validated also caps the rollup. Missing safety proof, QUAL-00 failure or unresolved blocks cannot be waived. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned PR check runner rollup/evidence reconciliation adapter ``` --- ### Tools SentinelToXDRMigration Docs Qualification Validation Checks (`Tools/SentinelToXDRMigration/docs/qualification/validation-checks.md`) # Validation checklist Offline/CI and authoring checks for general Sentinel/XDR solution PRs. See the [result and exception contract](README.md#results-author-evidence-and-rollups). Existing CI evidence and externally approved observations may be imported; this checklist does not authorize cloud reads or writes. Active migration validation checks are additional prerequisites, not replacements. ## VAL-CI-01 — Existing PR validations passed Review the exact PR head and the complete applicable official check set. ```yaml check-definition id: VAL-CI-01 checklist: validation contentType: all title: Existing PR Validations Passed originalCriterion: Verify that all validations have passed for the given Pull Request. applicability: Every solution PR, with or without migration. category: validation howPerformed: Consume head-SHA-bound required CI results and official build-and-validate receipts; retain individual failures and skips. expectedEvidence: [PR head SHA, required check inventory, observed conclusions, CI run URLs, local receipt file references] passCriteria: All applicable official checks completed successfully for current inputs. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and a current evidence ref, yielding Partially Validated, never PASS; safety gates cannot be waived. sourceChecklist: pr-checklist-2-docx automationStatus: automated responsible: .script/local-validation/build-and-validate.ps1 and planned PR CI evidence adapter ``` ## E2E-V01 — Content inventory and acknowledged exclusions Count logical content, not duplicate install/registration resources. Acknowledged exclusions permit this check to pass but force a Partially Validated rollup. ```yaml check-definition id: E2E-V01 checklist: validation contentType: all title: Content Inventory and Package Coverage originalCriterion: Package 100% of solution content items or explicitly acknowledge partial inventory exclusions in decision-manifest.json. applicability: Every solution PR; inventory current content, removals, selections and dependencies regardless of migration. category: validation howPerformed: Reconcile recursive logical content inventory with source selections and packaged resources; verify each exclusion has a current explicit author acknowledgement in decision-manifest.json. expectedEvidence: [complete inventory with hashes, packaged item identities, exclusion identities and reasons, author decision records, decision-manifest.json reference, expected and observed counts] passCriteria: Every content item is packaged or explicitly excluded with current author acknowledgement and evidence ref. Acknowledged exclusions force Partially Validated even when this check is PASS. Other FAIL or SKIPPED results may become WAIVED_WITH_AUTHOR_EVIDENCE only with author justification and evidence ref; safety gates cannot be waived. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned PR inventory adapter reusing solution_manager.qualification.deployment_planning and solution_manager.reporting.decision_manifest ``` ## E2E-V02 — Detection mock data existence and schema Cover all packaged Sentinel ARs and XDR CDs. Semantics belong in Qualification. ```yaml check-definition id: E2E-V02 checklist: validation contentType: all title: Detection Mock Data Existence and Schema originalCriterion: Mock data files exist and are schema-valid JSON for all packaged Sentinel Analytic Rules and Defender XDR Custom Detections. applicability: Every packaged detection, including source-only ARs and native XDR CDs; migration is not required. category: validation howPerformed: Map each detection identity to existing nonempty JSON fixtures and an identified schema; parse JSON and validate record shapes and types without inferring scenario semantics. expectedEvidence: [packaged detection inventory, detection-to-fixture mappings, fixture file paths and hashes, schema identity and version, per-file parse and schema results] passCriteria: Every packaged AR and CD maps to existing schema-valid JSON mock data; shared files are allowed with explicit mappings. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; no semantic correctness claim follows. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned fixture inventory/schema adapter; reuse reviewed mock artifacts and ingestion contract schemas ``` ## E2E-V03 — Workbook mock data existence and schema Cover packaged Sentinel and Defender workbooks, including their query data needs. ```yaml check-definition id: E2E-V03 checklist: validation contentType: workbooks title: Workbook Mock Data Existence and Schema originalCriterion: Mock data files exist and are schema-valid JSON for all packaged Workbooks. applicability: Every packaged Sentinel or XDR workbook, regardless of migration. category: validation howPerformed: Map workbook and nested query identities to existing JSON fixtures and declared schemas; validate file existence, JSON shape and types, not expected query behavior. expectedEvidence: [packaged workbook inventory, query-to-fixture mappings, fixture paths and hashes, schema references, observed JSON/schema results] passCriteria: All packaged workbooks have mapped schema-valid JSON mock files with no unaccounted query data dependency. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; live data and semantics remain unverified. sourceChecklist: user-e2e-requirement automationStatus: automated responsible: Planned workbook fixture coverage adapter reusing workbook inventory and fixture metadata ``` ## VAL-PKG-01 — Basics and createUiDefinition component counts Static counts must agree across selection, package and UI declarations. ```yaml check-definition id: VAL-PKG-01 checklist: validation contentType: solution-package title: Package Component Count Validation originalCriterion: Verify that the component counts are present and correct in createUiDefinition.json and Basics. applicability: Solution selection, package or UI definition changes, including dependent component changes. category: validation howPerformed: Count unique logical selected components by type and compare with package resources and createUiDefinition count declarations; keep rendered UI proof separate. expectedEvidence: [source selection reference, logical component inventory, package JSON pointers, UI count locations, expected and actual counts] passCriteria: All required component counts are present and equal, without double-counting registration resources. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: pr-checklist-2-docx automationStatus: automated responsible: Planned count reconciliation adapter reusing solution_manager.packaging.packaging and deployment_planning ``` ## VAL-PKG-02 — Package version increment and consistency The live baseline is evidence, not an assumed local version. Also covers Master Solutions rows 44, 46 and 52; schema versions are separate fields. ```yaml check-definition id: VAL-PKG-02 checklist: validation contentType: solution-package title: Package Version Increment Validation originalCriterion: Ensure that the package version has been incremented from the latest version currently live. applicability: New or updated solution packages; new offers require evidence that no live baseline exists. category: validation howPerformed: Compare canonical release versions against a current imported live-offer snapshot bound to publisher and offer; check ZIP name, package metadata and solution version consistency. expectedEvidence: [publisher and offer identity, live baseline value and observation time, baseline evidence ref, proposed version values and JSON pointers, ZIP filename] passCriteria: Updated release exceeds the proven live baseline and all package version fields agree; new-offer absence is evidenced. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; unavailable baseline is not an inferred pass. sourceChecklist: pr-checklist-2-docx automationStatus: automated responsible: Planned live-baseline comparison adapter; reuse packaging version checks and .script/package-automation/catalogAPI.ps1 evidence ``` ## VAL-META-01 — Solution metadata and branding Review support, author, offer, domain and component references. ```yaml check-definition id: VAL-META-01 checklist: validation contentType: solution-package title: Solution Metadata and Branding Validation originalCriterion: Verify metadata, support information, author/provider details, domains, and Microsoft Sentinel branding. applicability: Solution metadata, solution data or package metadata changes and new solutions. category: validation howPerformed: Apply current metadata/data contracts, identifier casing, support fields, path existence and branding checks; compare imported Content Hub identity evidence for updates. expectedEvidence: [metadata file references, expected contract version, observed field values, field-level findings, Content Hub identity evidence ref when applicable] passCriteria: Required fields and references conform to the applicable contract and supported offers do not use inappropriate personal author/support identities. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: master-excel automationStatus: automated responsible: .script/SolutionValidations, solution-data and solution-metadata instructions, planned contract-gap adapter ``` ## VAL-RULE-01 — Detection and hunting query contracts Apply kind-specific requirements; do not apply Scheduled-rule fields to hunts. Native XDR detection structure is checked here; source-derived identity and transformation contracts remain in the migration overlay. ```yaml check-definition id: VAL-RULE-01 checklist: validation contentType: all title: Detection and Hunting Query Contract Validation originalCriterion: Id is GUID and not duplicated; YAML structure is correct; review names, descriptions, dependencies, tactics, techniques and query fields. applicability: Packaged or changed Analytic Rules, native or migrated XDR detections and Hunting Queries, including dependencies affected by the PR. category: validation howPerformed: Run YAML, identity, detection-schema and KQL checks; apply current kind-specific naming, field, MITRE, entity/custom-detail binding and query contracts; use the appropriate native or source-derived identity contract and record legacy checklist conflicts explicitly. expectedEvidence: [rule or hunt paths and IDs, contract version, schema and KQL results, duplicate-ID search scope, field findings and source locations] passCriteria: Applicable syntax and structural requirements pass with correct kind-specific fields and references. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; semantic intent is assessed separately. sourceChecklist: master-excel automationStatus: automated responsible: .script/local-validation/build-and-validate.ps1, validate-xdr-content.py, detection-schema and KQL tests, planned instruction-to-check mappings ``` ## VAL-DC-01 — Data connector structure and dependencies Covers Master Data Connectors template, naming, permissions, metadata and parser banner checks; provider identity and instruction quality also need human review. ```yaml check-definition id: VAL-DC-01 checklist: validation contentType: data-connectors title: Data Connector Contract Validation originalCriterion: Compare the JSON with the relevant template and ensure properties, data types, permissions and parser dependencies are captured. applicability: Packaged or changed connectors and their affected parser dependencies, regardless of migration. category: validation howPerformed: Validate applicable connector template/schema, IDs, names, datatype formats, permissions, query syntax, metadata and required parser references/banners. expectedEvidence: [connector type and template version, file references, observed fields, query syntax results, parser dependency references, field-level differences] passCriteria: All applicable structural/template checks pass and required dependencies are declared. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; declared permissions do not establish effective access. sourceChecklist: master-excel automationStatus: automated responsible: .script/dataConnectorValidator.ts and checker utilities; planned full template/banner adapter ``` ## VAL-DC-02 — Function App connector artifacts Inspect archives without executing submitted code. ```yaml check-definition id: VAL-DC-02 checklist: validation contentType: data-connectors title: Function App Connector Artifact Validation originalCriterion: Validate deployment and execution files, Function App ZIP contents, function-name bounds and template variables. applicability: Function App connector additions or changes, including changes to Azure deploy files. category: validation howPerformed: Inspect referenced ARM/code/ZIP artifacts, required runtime dependencies such as .python_packages where applicable, name bounds and prohibited hostNameBindings; perform syntax checks without running connector ingestion. expectedEvidence: [deployment and execution file references, ZIP entry inventory, runtime contract, syntax findings, expected and observed template fields] passCriteria: All required artifacts and applicable runtime/template constraints are satisfied without unsafe archive entries. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; unsafe execution or privacy gates cannot be waived. sourceChecklist: master-excel automationStatus: automated responsible: Planned Function App artifact adapter reusing repository JSON/ARM checks ``` ## VAL-SAMPLE-01 — Sample data structure and sanitation Representative scenario behavior is a Qualification responsibility. ```yaml check-definition id: VAL-SAMPLE-01 checklist: validation contentType: all title: Sample Data Structure and Sanitization originalCriterion: Sample values are populated and sanitized for customer or PII information. applicability: Sample Data and packaged mock fixtures, including table catalog metadata. category: validation howPerformed: Apply array/catalog-specific JSON and email sanitation contracts, required field/type checks and the non-waivable TK-G07 privacy check. expectedEvidence: [sample file paths and hashes, applicable schema, record and field counts, sanitized scan findings, privacy check reference] passCriteria: Applicable sample contracts pass and TK-G07 passes. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; PII/privacy failures cannot be waived. sourceChecklist: master-excel automationStatus: automated responsible: .script/utils/sampleDataCheckers/sampleDataCheckers.ts and planned sample-schema adapter ``` ## VAL-LOGO-01 — SVG logo contract Technical validity does not replace the portal rendering check. ```yaml check-definition id: VAL-LOGO-01 checklist: validation contentType: logos title: SVG Logo Technical Validation originalCriterion: Logo needs to be in SVG format and under 5 Kb; prohibited constructs are absent and IDs are uniquely identifiable. applicability: New or changed content logos, including workbook images and referenced package logos. category: validation howPerformed: Inspect SVG format/size, style and cls constructs, embedded PNG, xlink, data-name, title and ID constraints with existing logo checkers; record exact size policy. expectedEvidence: [logo file reference, byte size, format, prohibited-construct findings, ID uniqueness results] passCriteria: Applicable SVG contract checks pass; appearance at 75 pixels remains E2E-Q06. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: master-excel automationStatus: automated responsible: .script/utils/LogoChecker and planned precise source-contract reconciliation ``` ## VAL-PARSER-01 — Parser structure and samples Select generic or ASIM-specific instructions as applicable. ```yaml check-definition id: VAL-PARSER-01 checklist: validation contentType: parsers title: Parser Structure and Test Input Validation originalCriterion: Sample data is available for testing; parser definitions and relevant ASIM schema are identified. applicability: Changed parsers and packaged parser dependencies. category: validation howPerformed: Validate YAML metadata, function names/aliases, versions, query syntax, sample references and applicable ASIM schema/filter contracts without deployment. expectedEvidence: [parser and sample references, generic or ASIM contract identity, metadata findings, KQL syntax results, declared output schema] passCriteria: Applicable structure/query contracts and test-input references are valid. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; returned-field semantics require QUAL-PARSER-01. sourceChecklist: master-excel automationStatus: automated responsible: Repository parser/KQL and ASIM validators; planned parser checklist adapter ``` ## VAL-WB-01 — Workbook structure and metadata Apply Sentinel or XDR workbook schemas, not one schema to both formats. ```yaml check-definition id: VAL-WB-01 checklist: validation contentType: workbooks title: Workbook Structure and Metadata Validation originalCriterion: Workbook metadata, unique workbookKey, dependencies, preview images, version and templateRelativePath are correct. applicability: Packaged or changed Sentinel and XDR workbooks and their metadata/assets. category: validation howPerformed: Validate applicable workbook schema, query syntax, identity and dependency fields, metadata version increment, referenced logos and PNG previews including black/white naming where required. expectedEvidence: [workbook and metadata references, schema identity, nested query inventory, unique IDs, image file references, old and new metadata versions] passCriteria: Applicable workbook contract and references pass; legacy chart-count/layout criteria remain visible for qualification review. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: master-excel automationStatus: automated responsible: .script workbook validators and validate-xdr-content.py; solution_manager workbook validation modules ``` ## VAL-PLAYBOOK-01 — Playbook template and documentation Do not deploy or authorize connections during static review. ```yaml check-definition id: VAL-PLAYBOOK-01 checklist: validation contentType: playbooks title: Playbook Template and Documentation Validation originalCriterion: Validate the Playbooks folder and Readme, templatization, connection variables and Deploy to Azure/Gov buttons. applicability: Packaged or changed playbooks, custom connectors and referenced deployment templates. category: validation howPerformed: Apply ARM/resource/parameter/metadata checks, naming and length rules, connection references, required README sections and deployment-link structure; run TK-G07 for sensitive data. expectedEvidence: [template and README paths, naming findings, ARM/resource results, connection variables, button target references, privacy evidence ref] passCriteria: Applicable template, naming and documentation contracts pass and TK-G07 passes. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: master-excel automationStatus: automated responsible: .script/playbooksValidator.ts and utilities; planned documentation completeness adapter ``` ## VAL-NOTEBOOK-01 — Notebook structure and dependencies Do not execute arbitrary notebook cells during offline validation. ```yaml check-definition id: VAL-NOTEBOOK-01 checklist: validation contentType: notebooks title: Notebook Structure and Dependency Validation originalCriterion: Notebook uses standard header information, documented dependencies and configuration; sample data is provided when possible. applicability: Packaged or changed notebooks and referenced code/data. category: validation howPerformed: Parse notebook JSON, inventory code/data/API dependencies, check required explanatory sections and config.json or msticpyconfig.yaml references, and scan stored outputs through TK-G07. expectedEvidence: [notebook file reference, cell/dependency inventory, configuration and sample refs, missing-section findings, privacy evidence ref] passCriteria: Applicable structural and dependency requirements pass with no privacy failure. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; external-file exceptions require QUAL-NOTEBOOK-01. sourceChecklist: master-excel automationStatus: automated responsible: Planned notebook static adapter ``` ## VAL-URL-01 — Link inventory and static references Actual short-link opening and expected destinations are checked in Qualification. ```yaml check-definition id: VAL-URL-01 checklist: validation contentType: all title: URL and Changed-Reference Inventory originalCriterion: Verify short URLs in mainTemplate.json and createUiDefinition.json and check links when files move or change. applicability: All changed content and package URLs, plus incoming references to moved or deleted files. category: validation howPerformed: Extract URLs including Package files, identify additions/changes and target moves, check syntax/local references and applicable locale/easy-access naming rules; prepare live resolution scope. expectedEvidence: [old and new URLs, source file/line or JSON pointer, incoming reference list, local target existence, static checker results] passCriteria: URL inventory is complete and applicable static rules pass; no reachability claim is made. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: pr-checklist-2-docx automationStatus: automated responsible: .script/documentsLinkValidator.ts plus planned package-inclusive URL inventory adapter ``` ## VAL-RELEASE-01 — Release notes Keep reviewer terminology and exact version/date evidence. ```yaml check-definition id: VAL-RELEASE-01 checklist: validation contentType: solution-package title: Release Notes Format and Coverage originalCriterion: Check release note text and format. applicability: Solution changes requiring release notes under the current release-notes instructions. category: validation howPerformed: Check path, three-column table, canonical versions, descending order, DD-MM-YYYY dates, nonempty history and correspondence to current release. expectedEvidence: [ReleaseNotes.md reference, parsed rows, expected release version, date/ordering findings, changed component inventory] passCriteria: Required release notes are present and structurally valid with current release coverage. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated. sourceChecklist: master-excel automationStatus: automated responsible: solution_manager.packaging.release_notes and planned read-only release-note validation adapter ``` ## VAL-PR-01 — PR review information and deletion accounting A filled template is not proof that a risky removal is approved. ```yaml check-definition id: VAL-PR-01 checklist: validation contentType: all title: PR Review Information and Change Accounting originalCriterion: Ensure the PR review template is filled out; double-check deletions and merge-conflict resolutions. applicability: Every solution PR, especially deletions, renames and conflict resolutions. category: validation howPerformed: Compare PR description with repository template; inventory deletions/renames and expected dependent changes; require explicit unresolved-impact records for human review. expectedEvidence: [PR body and template refs, base/head SHA, deleted/renamed items, dependency impact list, author explanations and evidence refs] passCriteria: Required review information and complete change accounting are present without unrecorded removals. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; required safety authorizations remain separate. sourceChecklist: master-excel automationStatus: automated responsible: Planned PR metadata/diff adapter and content review owner ``` ## VAL-ARTIFACT-01 — XDR artifacts and source protection Missing artifacts must be requested from the author, not silently reconstructed. ```yaml check-definition id: VAL-ARTIFACT-01 checklist: validation contentType: all title: XDR Artifact Completeness and Source Protection originalCriterion: Verify XDR artifacts shared in the PR; if absent, ask the author to provide them. applicability: PRs containing or changing packaged XDR content, including native content. category: validation howPerformed: Verify required content, package and evidence references exist and match current hashes; account for source edits and edited generated files without overwriting them. expectedEvidence: [expected and observed artifact inventory, file/hash references, missing artifact requests, source/generated-file change inventory] passCriteria: Required artifacts are present/current and source changes are explicit. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; unsafe overwrites or privacy failures are not authorized. sourceChecklist: pr-checklist-2-docx automationStatus: automated responsible: Planned PR artifact adapter reusing solution_manager.reporting.evidence and migration conflict checks ``` ## TK-V06 — Workload and connector provenance These general XDR dependencies apply to native as well as migrated content. ```yaml check-definition id: TK-V06 checklist: validation contentType: all title: Workload and Connector Dependency Provenance originalCriterion: Validate workload vocabulary and dependency records; unresolved dependency review blocks readiness. applicability: Packaged XDR detections/workbooks or changed dependency declarations. category: validation howPerformed: Validate record shapes, workload vocabulary, connector/data-type references and review disposition; exclude platform-reference inputs such as Watchlist from telemetry dependency requirements. expectedEvidence: [dependency record locations, declared workloads/connectors, resolved table references, review decision refs, validation findings] passCriteria: Applicable dependency records conform and no required review is unresolved. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated without changing existing deployment gates. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.dependencies and workbook_dependencies ``` ## TK-V07 — Workbook function dependencies Registration and deployment ordering do not prove runtime function availability. ```yaml check-definition id: TK-V07 checklist: validation contentType: workbooks title: Workbook Function Dependency Integrity originalCriterion: Bind required workspace functions to packaged definitions, aliases and current hashes; resolve actual parser installation dependencies. applicability: Packaged Sentinel/XDR workbooks invoking parser or workspace functions. category: validation howPerformed: Check definition paths, aliases, hashes and underlying tables; distinguish built-in ASIM and Watchlist functions; verify unconditional required parser installation and actual saved-search dependsOn references. expectedEvidence: [query/function references, definition hashes, builtin classification, package resource pointers, expected and observed dependency edges] passCriteria: Function provenance and installation dependencies match the applicable workbook contract. FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; runtime availability remains separate. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.workbook_functions and solution_manager.packaging.packaging ``` ## TK-V10 — Package candidate and archive integrity Candidate creation is not package acceptance. ```yaml check-definition id: TK-V10 checklist: validation contentType: solution-package title: Package Candidate and Archive Integrity originalCriterion: Bind candidate inputs and artifacts and verify exact ZIP contents without stale, duplicate or operational entries. applicability: Generated or changed solution packages, including V3.1 hybrid candidates. category: validation howPerformed: Validate selected input hashes, candidate/run identity, exact mainTemplate/createUiDefinition ZIP bytes and safe archive entries; prevent older package nesting. expectedEvidence: [candidate identity, input manifest, artifact hashes, ZIP entry inventory, byte comparison results, package receipt ref] passCriteria: Package artifacts and archive match current selected inputs and the applicable package contract. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; privacy and unsafe execution remain blocking. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.workbook_validation and solution_manager.packaging.packaging ``` ## TK-V11 — XDR package properties and resource identities Also covers Checklist 2's required XDR properties in mainTemplate.json. ```yaml check-definition id: TK-V11 checklist: validation contentType: solution-package title: XDR Package Properties and Registration Integrity originalCriterion: Verify required XDR content properties and exact installation/registration identity, version and dependency contracts. applicability: Packages containing XDR detections or workbooks, whether native or migrated. category: validation howPerformed: Validate required properties, disabled defaults, deployment flags, matching install/registration resources, dependency references and packageId/packageVersion/contentId/contentVersion values. expectedEvidence: [mainTemplate JSON pointers, selected content IDs, deployment flag values, expected and actual identity quartets, resource and dependsOn findings] passCriteria: All applicable XDR package properties and resource contracts pass. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; activation/target authorization is never waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: .script/local-validation/validate-xdr-content.py and solution_manager.packaging.packaging ``` ## TK-V12 — Official receipt and evidence freshness A copied receipt or a skipped validator cannot establish current execution. ```yaml check-definition id: TK-V12 checklist: validation contentType: all title: Official Validation Receipt Integrity originalCriterion: Require complete producer-bound validation receipts for the current candidate and preserve individual statuses. applicability: Any check consuming CI, local validation or package-acceptance evidence. category: validation howPerformed: Verify producer identity, toolkit/content root, run/head/input bindings, artifact hashes and complete validator inventory; reject malformed, conflicting or stale evidence. expectedEvidence: [receipt file refs, producer binding artifact, toolkit and input hashes, complete validator statuses, freshness comparison] passCriteria: Evidence is complete and current for each claimed execution; skips remain skips. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; no waiver upgrades untrusted live evidence under E2E-G01. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.qualification.arm_acceptance and planned generic evidence binding adapter ``` ## TK-V13 — ARM static scope and official template checks Static approval is not live deployment authorization. ```yaml check-definition id: TK-V13 checklist: validation contentType: solution-package title: ARM Template and Bounded Scope Validation originalCriterion: Resolve resource types, parameters, conditions and nested scopes and retain applicable official ARM validation checks. applicability: Package ARM changes or candidate acceptance; selected ARM method determines applicable evidence. category: validation howPerformed: Inspect exact resources, booleans and nested scopes; run/import official ARM-TTK when selected and all remaining applicable official checks; route lab-method execution to Qualification. expectedEvidence: [ARM resource/scope inventory, parameter bindings, official check receipts, selected ARM validation method, unresolved expression findings] passCriteria: Applicable static checks pass with no ambiguous or unauthorized resource scope. Only non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; target scope and permission grants cannot be waived. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.authoring.workbook_validation and .script/local-validation/build-and-validate.ps1 ``` ## TK-G07 — Privacy and safe evidence publication Non-waivable safety gate. Email-only sample scans are not a complete PII review. ```yaml check-definition id: TK-G07 checklist: validation contentType: all title: Privacy and Safe Evidence Export originalCriterion: Keep secrets, private target identifiers, personal data, local paths and operational receipts out of published content and evidence. applicability: All source/sample/package artifacts, decision manifests, screenshots and evidence exports; recheck newly captured Qualification evidence. category: validation howPerformed: Scan and review public projections, samples and captures; separate private receipts, browser sessions and credentials; verify sanitization before release. expectedEvidence: [artifact inventory, sanitized scan findings, redaction review, public/private disposition, sanitized manifest and evidence file refs] passCriteria: All exported artifacts satisfy privacy requirements and private material remains protected. FAIL, SKIPPED or BLOCKED cannot transition to WAIVED_WITH_AUTHOR_EVIDENCE; author justification cannot waive TK-G07 or permit a successful rollup. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.reporting.decision_manifest, reporting.evidence, qualification.readiness_support and privacy reviewer ``` ## VAL-OPS-01 — Durable workflow and accountable decisions Covers toolkit retry, budget, immutable-tooling and reviewer-delivery controls. ```yaml check-definition id: VAL-OPS-01 checklist: validation contentType: all title: Durable Workflow and Decision Integrity originalCriterion: Preserve attempts and failures, use bounded execution, prevent duplicate owners and version consumption, and retain accountable review decisions. applicability: Toolkit-managed authoring or qualification runs and imported workflow evidence. category: validation howPerformed: Inspect run ownership, durable state, budget receipts, retry/reconciliation records and delivered review packets; confirm protected tooling was not modified as a runtime workaround. expectedEvidence: [run/state refs, owner lease, attempt history, budget receipts, review acknowledgement refs, toolkit version/hash, package version action history] passCriteria: Applicable operational controls are satisfied without hidden failures or unapproved execution. Non-safety FAIL or SKIPPED may become WAIVED_WITH_AUTHOR_EVIDENCE with author justification and evidence ref, yielding Partially Validated; authorization/privacy/target/provider safeguards remain non-waivable. sourceChecklist: toolkit-contract automationStatus: automated responsible: solution_manager.core.workflow, coordinator, budget, review_notifications and planned operational receipt adapter ``` --- --- METRICS --- - Files Extracted: 9 - Estimated Token Budget: ~40808 tokens - Canonical Reference: https://codewiki.google/github.com/Azure/Azure-Sentinel