sentry Documentation

Technical Documentation: getsentry/sentry

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (README.md) Copy Section

What's Sentry?

Sentry is the debugging platform that helps every developer detect, trace, and fix issues. Code breaks, fix it faster.

Official Sentry SDKs Copy Chapter

• JavaScript
• Electron
• React-Native
• Python
• Ruby
• PHP
• Laravel
• Go
• Rust
• Java/Kotlin
• Objective-C/Swift
• C#/F#
• C/C++
• Dart/Flutter
• Perl
• Clojure
• Elixir
• Unity
• Unreal Engine
• Godot Engine
• PowerShell

Resources

• Documentation
• Discussions (Bugs, feature requests,
  general questions)
• Discord
• Contributing
• Bug Tracker
• Code
• Transifex (Translate
  Sentry!)

Src Sentry Flags Docs Api (`src/sentry/flags/docs/api.md`) Copy Section

Flags API

Host: https://sentry.io/api/0

Authors.

@cmanallen

How to read this document.

This document is structured by resource with each resource having actions that can be performed against it. Every action that either accepts a request or returns a response must document the full interchange format.

Flag Logs [/organizations//flags/logs/] Copy Chapter

• Parameters
  • flag (optional, string) - The flag name to filter the result by. Can be specified multiple times.
  • start (optional, string) - ISO 8601 format (YYYY-MM-DDTHH:mm:ss.sssZ)
  • end (optional, string) - ISO 8601 format. Required if start is set.
  • statsPeriod (optional, string) - A positive integer suffixed with a unit type.
  • cursor (optional, string)
  • per_page (optional, number)
    Default: 10
  • offset (optional, number)
    Default: 0

Browse Flag Logs [GET] Copy Section

Retrieve a collection of flag logs.

Attributes

• Response 200

  json Copy

  {
    "data": [
      {
        "action": "created",
        "createdAt": "2024-01-01T05:12:33",
        "createdBy": "2552",
        "createdByType": "id",
        "flag": "my-flag-name",
        "id": 1,
        "tags": {
          "environment": "production"
        }
      }
    ]
  }

Flag Log [/organizations//flags/logs//] Copy Chapter

Fetch Flag Log [GET] Copy Section

Retrieve a single flag log instance.

• Response 200

  json Copy

  {
    "data": {
      "action": "updated",
      "createdAt": "2024-11-19T19:12:55",
      "createdBy": "[email protected]",
      "createdByType": "email",
      "flag": "new-flag-name",
      "id": 1,
      "tags": {
        "environment": "development"
      }
    }
  }

Signing Secrets [/organizations//flags/signing-secrets/] Copy Chapter

• Parameters
  • cursor (optional, string)
  • per_page (optional, number)
    Default: 10
  • offset (optional, number)
    Default: 0

Browse Signing Secrets [GET] Copy Section

Browse a list of signing secrets. Secrets are unique per provider. Secrets only show the first six characters; the remainder are redacted.

Attributes

• Response 200

  json Copy

  {
    "data": [
      {
        "createdAt": "2024-12-12T00:00:00+00:00",
        "createdBy": 12345,
        "id": 123,
        "provider": "launchdarkly",
        "secret": "abc123**********"
      }
    ]
  }

Create Signing Secret [POST] Copy Section

Requests from web hook providers can be signed. We use the signing secret to verify the webhook's origin is authentic.

• Request (application/json)

  json Copy

  {
    "provider": "launchdarkly",
    "secret": "d41d7d1adced450d9e2eb7f76dde6a04"
  }

• Response 201

Signing Secret [/organizations//flags/signing-secrets//] Copy Chapter

Delete Signing Secret [DELETE] Copy Section

Delete a signing secret.

• Response 204

Webhooks [/organizations//flags/hooks/provider//] Copy Chapter

Create Generic Flag Log [POST] Copy Section

A flag log event must be emitted after every flag definition change which influences a flag's evaluation. Updates to a flag that do not change a flag's evaluation logic do not need to be emitted to this endpoint. We are only concerned with changes which could have influenced behavior.

Sentry does not currently have a concept of disambiguating flag changes by project or environment. Everything is done at the organization level. Flag changes that are duplicated across projects, environments, or other groupings within the provider, must be de-duplicated. To support this, the posted payload sets a "change_id" field for idempotency. In the presence of duplicate ids, only one audit-log record is written in Sentry.

Data Attributes

Meta Attributes

• Request (application/json)

  json Copy

  {
    "data": [
      {
        "action": "created",
        "created_at": "2024-12-12T00:02:00+00:00",
        "created_by": {
          "id": "[email protected]",
          "type": "email"
        },
        "flag": "hello.world"
      }
    ],
    "meta": {
      "version": 1
    }
  }

• Response 201

Create Provider-Specific Flag Log [POST] Copy Section

The shape of the request object varies by provider. The `` URI parameter informs the server of the shape of the request and it is on the server to handle the provider. The following providers are supported: LaunchDarkly.

Webhooks are signed by their provider. The provider handler must use the secret stored in Sentry to verify the signature of the payload. Failure to do so could lead to unauthorized access.

Any request content-type is acceptable (JSON, XML, binary-formats) so long as the server is capable of decoding the request and mapping it to our object model.

• Request

• Response 201

Src Sentry Workflow Engine Docs Adding Detectors (`src/sentry/workflow_engine/docs/adding-detectors.md`) Copy Section

Adding a Workflow Engine Detector

This guide is a code-oriented checklist for adding a detector that evaluates typed data
and emits Issue Platform occurrences or resolutions. Read the
architecture overview, data model, and
condition contract, then use the execution guide to
trace the surrounding pipelines.

Detector implementations normally live in the product module that owns the data and
Issue Platform group type. Only shared abstractions, processors, and generic source or
condition handling belong in sentry.workflow_engine.

Before coding, identify:

• The owning product module
• The packet payload type
• The source of packet ordering
• Whether one packet contains one value or grouped values
• Trigger and recovery semantics
• The stable issue fingerprint
• The API creation/update requirements
• Expected packet volume and state cardinality

Choose a Handler Abstraction Copy Chapter

flowchart TD
    Start[New detector] --> State{Needs persisted priority, dedupe, thresholds, or resolution?}
    State -->|Yes| Stateful[Inherit StatefulDetectorHandler]
    State -->|No| Default{Does the default stateless evaluation fit?}
    Default -->|Yes| Handler[Inherit DetectorHandler]
    Default -->|No| Custom[Inherit DetectorHandler and override evaluate]
    Stateful --> Hooks[Implement extract_value, extract_dedupe_value, and create_occurrence]
    Handler --> Minimal[Implement extract_value and create_occurrence]
    Custom --> Minimal

Handler hierarchy Copy Section

`BaseDetectorHandler` Copy Section

BaseDetectorHandler is the abstract interface. Do not inherit this directly.

The exception is ErrorDetectorHandler, a placeholder
that associates error issues with a detector. It does not get evaluated in the normal detector evaluation pipeline and its methods are all stubs.

`DetectorHandler` Copy Section

DetectorHandler is the base class for every detector
that does not need persisted state. It loads the detector's trigger condition group and
provides a default stateless evaluate. A concrete subclass implements extract_value
and create_occurrence.

Default stateless evaluation

flowchart TD
    Packet[Receive DataPacket] --> Extract[extract_value]
    Extract --> Group[Normalize to group key and value pairs]
    Group --> Conditions[evaluate_conditions for each group]
    Conditions --> Missing{Condition group missing?}
    Missing -->|Yes| Skip[Skip the group]
    Missing -->|No| Priority{Selected priority is OK?}
    Priority -->|Yes| None[No Issue Platform output]
    Priority -->|No| Occurrence[create_occurrence and build IssueOccurrence]

Override these hooks to adjust the default evaluation:

Override evaluate itself when the detector needs a different flow. See
PreprodSizeAnalysisDetectorHandler for how to achieve this

`StatefulDetectorHandler` Copy Section

Prefer StatefulDetectorHandler for detectors that
evaluate ordered samples and transition among OK, LOW, MEDIUM, and HIGH.

It provides:

• Bulk state loading for grouped values
• Out-of-order and duplicate packet suppression
• Condition-group evaluation
• Priority threshold counters
• Durable priority transitions
• Occurrence creation for non-OK transitions
• Resolution messages for transitions to OK
• Shared metrics and evidence data

Uptime and metric issues are canonical examples.

Design the Packet and Evaluation Value Copy Chapter

DataPacket[T] separates source identity from payload:

@dataclass
class ExamplePacket:
    value: float
    sequence: int


packet = DataPacket(
    source_id=str(source.id),
    packet=ExamplePacket(value=42.0, sequence=123),
)

The payload type is what the producer knows. The evaluation value is what detector
conditions consume. They can differ.

Every handler implements extract_value. A stateful handler also implements
extract_dedupe_value:

def extract_value(
    self, data_packet: DataPacket[ExamplePacket]
) -> float | DetectorGroupValues[float]:
    return data_packet.packet.value


def extract_dedupe_value(self, data_packet: DataPacket[ExamplePacket]) -> int:
    return data_packet.packet.sequence

extract_value can return one value, or DetectorGroupValues mapping group key to value.
Group keys create independent Issue Platform fingerprints for one detector, and independent
detector state for stateful handlers. Any other return value, including a plain dict, is
one ungrouped value.

extract_dedupe_value must return a positive integer whose ordering matches the source
stream. Missing Redis state is read as zero, so a first value of zero or less is skipped.
Good values include a source sequence or integer timestamp with sufficient resolution.
Do not use processing time when the producer can deliver older packets later. The Redis
watermark expires after seven days.

The dedupe value is extracted once per packet and applied to every group key. A fresh
value is committed even when no condition matches. Detectors that need independent
ordering per group or replay of previously non-matching packets require custom
orchestration.

Group-key requirements Copy Section

A DetectorGroupKey is str | None:

• Return one value for an ungrouped detector; the state key is None.
• Return DetectorGroupValues for independent substreams.
• Use non-empty strings no longer than 200 characters. Do not use "": state-key
  construction and the database uniqueness constraint treat it like ungrouped None.
• Keep keys stable across trigger and recovery packets.
• Keep key cardinality bounded. Each key can create durable and Redis state plus an
  Issue Platform group.
• Do not put unbounded IDs or raw user-controlled values into group keys without a
  product-level retention and cardinality plan.
• Return an empty DetectorGroupValues to mean "no groups"; nothing is evaluated.

Implement Occurrence Creation Copy Chapter

Every handler implements create_occurrence with this contract:

def create_occurrence(
    self,
    evaluation: DataConditionGroupEvaluation,
    data_packet: DataPacket[ExamplePacket],
    priority: DetectorPriorityLevel,
) -> tuple[DetectorOccurrence, EventData]:
    ...

The DetectorOccurrence describes the Issue Platform
occurrence. EventData supplies event fields associated with it.

An abbreviated implementation looks like:

def create_occurrence(
    self,
    evaluation: DataConditionGroupEvaluation,
    data_packet: DataPacket[ExamplePacket],
    priority: DetectorPriorityLevel,
) -> tuple[DetectorOccurrence, EventData]:
    occurrence = DetectorOccurrence(
        issue_title=f"{self.detector.name} triggered",
        subtitle="The monitored value crossed its threshold",
        evidence_data={"value": data_packet.packet.value},
        evidence_display=[],
        type=ExampleGroupType,
        level="error",
        culprit="",
        priority=priority,
    )
    event_data: EventData = {"platform": "other", "tags": {}}
    return occurrence, event_data

Use an existing product implementation for the exact event shape. The snippet is
illustrative and omits product-specific evidence. The returned dictionary must be
mutable.

The default evaluations supply defaults for environment, platform, received, and tags

The default evaluations overwrite timestamp and project_id.

Stateless and stateful handlers differ on the event_id field:

• The default DetectorHandler.evaluate uses a random or derived event_id and then generates an occurrence ID from it.
• StatefulDetectorHandler overwrites event_id with a random ID that is also used as
  the occurrence ID.

Evidence Copy Section

Evidence serves two audiences:

• evidence_data stores structured data used by APIs, grouping context, and issue
  rendering.
• evidence_display stores the human-readable evidence shown on the issue.

The stateful handler adds standard detector and condition evidence. Product-specific
evidence should be serializable, bounded in size, and avoid sensitive data.

Fingerprints Copy Section

Default evaluation

The default evaluation uses get_issue_fingerprint(group_key):

detector:<detector_id>
detector:<detector_id>:<group_key>

Overriding get_issue_fingerprint replaces the whole fingerprint.

Stateful evaluation

The stateful implementation always includes its detector state key:

detector:<detector_id>
detector:<detector_id>:<group_key>

Override build_issue_fingerprint only when the product needs additional stable
grouping components. The engine appends the detector state key to trigger and resolution
fingerprints. Currently occurrence creation passes group_key to this override while
resolution calls it without that argument. Do not make custom components depend on the
argument unless resolution is updated and trigger/resolution tests cover the behavior.
Fingerprint changes can split or merge production issues and can break recovery.

Test the exact trigger and resolution fingerprints. For grouped detectors, explicitly
test that one key cannot resolve another key's issue.

Configure Threshold Semantics Copy Chapter

StatefulDetectorHandler.thresholds maps priority to the number of qualifying
evaluations required before transition:

@property
def thresholds(self) -> DetectorThresholds:
    return {
        DetectorPriorityLevel.: self.detector.config["failure_threshold"],
        DetectorPriorityLevel.OK: self.detector.config["recovery_threshold"],
    }

The default threshold is one. For a detectors evaluation, the threshold will say we needs
to see X number of a non-OK evaluations to trigger the detector. An OK
evaluation clears non-OK counters before advancing the OK counter. An evaluation equal
to the current durable state clears accumulated counters. These rules are not simply an
independent consecutive-sample counter for every priority.

A detector that can trigger but never produce OK evaluations will not resolve through
the stateful handler.

For the StatefulDetectorHandler implementation, failing a non-OK condition does not resolve
the detector. To have the detector be considered resolved, the trigger group must resolve
to DetectorPriorityLevel.OK. This normally uses a passing condition with an OK result,
but an empty or passing NONE group can also trigger with no priority-bearing result and
use the default OK priority. A missing detector trigger group is invalid and produces no
state transition.

NOTE: create_occurrence does not receive the current group key. The base handler still adds
the key to DetectorEvaluation.data and the engine fingerprint. If product evidence
must contain the key, include it in the extracted evaluation value or implement and
test the required custom orchestration.

Example Threshold Evaluation Copy Section

Here's an illustration of how a threshold works; specifically how the OK result will
reset the threshold counts. Note that if it the priority is de-escalating, it does not
reset the threshold counts. See

Detector Threshold Settings

@property
def thresholds(self) -> DetectorThresholds:
    return {
        DetectorPriorityLevel.HIGH: 2,
        DetectorPriorityLevel.LOW: 2,
        DetectorPriorityLevel.OK: 1
    }

Example Evaluation Results

Example 1 - Consecutive Triggers

- DetectorPriorityLevel.HIGH
- DetectorPriorityLevel.HIGH
===> Detector is triggered at HIGH

Example 2 - OK resets thresholds

- DetectorPriorityLevel.HIGH
- DetectorPriorityLevel.OK
- DetectorPriorityLevel.HIGH   # Note that the Detector does *not* trigger here
- DetectorPriorityLevel.OK
- DetectorPriorityLevel.HIGH
- DetectorPriorityLevel.HIGH
===> Detector is triggered at HIGH

Example 3 - De-escalation does _not_ reset thresholds

- DetectorPriorityLevel.OK
- DetectorPriorityLevel.HIGH
- DetectorPriorityLevel.LOW
- DetectorPriorityLevel.HIGH
===> Detector is triggered at HIGH

Example 4 - Escalating detectors do _not_ reset thresholds

- DetectorPriorityLevel.LOW
- DetectorPriorityLevel.HIGH
- DetectorPriorityLevel.LOW
===> Detector is triggered at LOW
- DetectorPriorityLevel.HIGH
===> Detector is triggered at HIGH

Define the Issue and Detector Type Copy Chapter

Create or extend a concrete GroupType in the owning
product's grouptype.py:

@dataclass(frozen=True)
class ExampleGroupType(GroupType):
    type_id = 12345  # Allocate the real Issue Platform type ID.
    slug = "example_detector"
    description = "Example detector issue"
    category = GroupCategory.ERROR.value  # Choose the applicable category.
    released = False
    default_priority = PriorityLevel.HIGH

Use the Issue Platform documentation and existing group types to choose the type ID,
category, release controls, priority, auto-resolution, escalation behavior, and
notification configuration.

Register detector settings Copy Section

The group type describes the issue. The detector components live in a
DetectorSettings subclass registered in
detector_settings_registry under the group type's slug:

from sentry.workflow_engine.registry import detector_settings_registry
from sentry.workflow_engine.types import DetectorAPIOperation, DetectorSettings, FeatureGate


@detector_settings_registry.register(ExampleGroupType.slug)
class ExampleDetectorSettings(DetectorSettings):
    handler = ExampleDetectorHandler
    validator = ExampleDetectorValidator
    config_schema = {
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "type": "object",
        "properties": {
            "failure_threshold": {"type": "integer", "minimum": 1},
            "recovery_threshold": {"type": "integer", "minimum": 1},
        },
        "required": ["failure_threshold", "recovery_threshold"],
        "additionalProperties": False,
    }

Register the class, not an instance. The settings can also live in their own module, as long as that module is
imported during application startup.

DetectorSettings fields are:

Settings classes inherit DetectorSettings.DEFAULT_API_AVAILABILITY unless they override
api_availability. This shared mapping is immutable and sets each DetectorAPIOperation
member to True, so ordinary detector implementations need no availability declaration.
Missing entries in custom maps still default to True. The supported enum members are
LIST, GET, POST, PUT, and DELETE.

Each gate is either True, False, or FeatureGate("organizations:feature-name").
FeatureGate checks a registered organization-scoped feature at request time using the
authorized organization and actor. Feature names remain strings, but raw strings are not
valid gate values.

To hide a type from the list and its counts without disabling other operations:

api_availability = {
    **DetectorSettings.DEFAULT_API_AVAILABILITY,
    DetectorAPIOperation.LIST: False,
}

For a rollout across every detector-platform API, assign the same feature gate to all
operations:

api_availability = {
    operation: FeatureGate("organizations:example-detector-api")
    for operation in DetectorAPIOperation
}

LIST additionally requires GET, so a denied GET gate hides a type from both listing
and counts even when LIST is explicitly allowed. Gate checks short-circuit in that order:
implied GET for LIST, then the requested operation.

Generic detector-platform APIs evaluate these gates through get_excluded_detector_types,
which reads registered settings without fetching detector rows. List filtering happens
before pagination and hit counting, including requests that filter by detector ID or type.
Counts use the same LIST policy. GET also controls detail retrieval and type discovery;
LIST alone does not restrict those other reads. Gates are independent of the group type's
released state and the detector's enabled state.

Product-specific APIs, such as anomaly-data retrieval, do not inherit these platform API
exclusions.

Individual requests for a detector excluded from GET, PUT, or DELETE return 404.
Creation of a type excluded from POST, or an update targeting a type excluded from
PUT, returns a 400 type-field validation error before saving.

Bulk updates and deletions filter excluded types before mutation. Query/project selections
operate on the remaining types; explicit ID selections return 400 without changing any
detectors if an excluded ID was requested.

Read exclusions do not restrict creation, updates, or deletion. Write exclusions do not
affect listing, and API exclusions never change detector evaluation.

Both registrations happen at import time. The GroupType subclass registers itself
with the global Issue Platform registry when the class is created. The settings class
registers itself when its decorator runs. Sentry startup imports the root
grouptype.py of every installed Django app through
import_grouptype. Put both classes there, or import a
nested implementation from that file, as
sentry.preprod.grouptype does. Add a test that checks
both the group type and its settings resolve by slug after normal application startup.

Implement API Validation Copy Chapter

Subclass
BaseDetectorTypeValidator in the owning
product or under workflow_engine/endpoints/validators when the behavior is shared.

The base validator provides:

• Common detector fields
• Condition-result priority validation
• Detector config-schema validation
• Transactional creation and update
• Workflow connections
• Audit entries
• Scheduled deletion

Detector-specific validators commonly provide:

• A data_conditions validator
• Product-specific data-source fields and creators
• Cross-field validation
• Quota calculation
• Detector-specific cleanup

If for some reason there's not a consistent data_source or you need to opt-out of the data_source
to detector connections, then set data_source_required = False. This should only be when the detector
is intentionally created and resolved outside the generic data-source graph. Set enforce_single_datasource = True
when multiple sources would make the handler ambiguous.

Do not create the detector graph piecemeal in an endpoint. The base validator's
transaction coordinates condition groups, detector config validation, source mappings,
workflow connections, and audit logging.

Connect a Data Source Copy Chapter

Skip this section only when the validator intentionally sets
data_source_required = False and the producer has another reliable detector lookup.

Reuse or register a source type Copy Section

First determine whether an existing source type represents the producer object. If not,
define a source type constant and implement
DataSourceTypeHandler:

• bulk_get_query_object
• related_model
• get_instance_limit
• get_current_instance_count
• get_relocation_model_name

Register the handler with data_source_type_registry. Update
DataSource.__relocation_dependencies__ when the source references a model not already
listed. Add deletion, quota, relocation, and unknown-type tests appropriate to the new
source.

Create the source graph Copy Section

Native API creation expects validated source data to contain a product-specific creator.
The base validator calls that creator, creates the generic DataSource, and creates the
DataSourceDetector mapping in the detector transaction.

The resulting source identity must match the producer call:

DataSource.type == query_type passed to process_data_packet
DataSource.source_id == DataPacket.source_id

Test this match end to end. A valid detector with a mismatched source type or stringified
ID will never be selected.

Produce Packets Copy Chapter

Use process_data_packet when the producer has a generic
source mapping:

process_data_packet(
    DataPacket(source_id=str(source.id), packet=payload),
    DATA_SOURCE_EXAMPLE,
)

Use process_detectors directly only when the product path
already resolves the exact detector set and a generic lookup would add no value. Explain
and test that ownership boundary in the producer module. process_detectors does not
filter enabled or lifecycle status. Direct callers must define and test both
eligibility checks rather than assuming the generic source path's filtering applies.

Producer requirements:

• Preserve the source's natural ordering value.
• Expect retries and duplicate delivery.
• Avoid one database query per detector or group key.
• Keep packet payloads bounded.
• Record enough product metrics to distinguish missing input from detector rejection.
• Decide whether detector exceptions should fail, retry, or isolate one item in a batch.

Configure Trigger Conditions Copy Chapter

Detector trigger conditions return DetectorPriorityLevel values. The base API
validator rejects results that are not valid priority integers.

Prefer built-in operators when conditions compare the extracted value directly. Add a
DataConditionHandler only when evaluation needs a richer comparison or
event context.

The complete shared evaluation contract and condition-handler extension path live in
Conditions. In particular, handler-group placement is not enforced by
generic backend validation, and stateful Detector triggers do not participate in the
Workflow batched data-acquisition phase.

Rollout and Operational Safety Copy Chapter

Before enabling production issue creation:

• Keep a new GroupType unreleased until Issue Platform ingest, post-process, and UI
  behavior are intentionally enabled.
• Add a feature flag or runtime option when product rollout needs control beyond the
  Issue Platform release controls.
• Estimate packet throughput and group-key cardinality.
• Estimate PostgreSQL DetectorState growth and Redis counter usage.
• Verify occurrence and event payload sizes.
• Ensure fingerprints cannot collide with another detector or an experiment.
• Add metrics with bounded-cardinality tags.
• Verify deletion and missing-record behavior for queued work.
• Document whether old packets can arrive after configuration updates.
• Confirm recovery behavior before triggering real issues.

Detector rollout and Issue Platform type rollout are separate controls. A detector can
run without its resulting type being fully exposed, and a released group type does not
cause a producer to run.

Implementation Checklist Copy Chapter

Design Copy Section

• Confirm Workflow Engine is the correct detection system.
• Define the packet payload and evaluation value types.
• Choose the handler: StatefulDetectorHandler or DetectorHandler.
• For stateful detectors, choose an ordered, integer dedupe value.
• Decide whether results are grouped and bound group-key cardinality.
• Specify trigger, escalation, and recovery transitions.
• Specify stable trigger and resolution fingerprints.

Issue type and handler Copy Section

• Add or update the product GroupType.
• Register a DetectorSettings subclass in detector_settings_registry with
  handler, validator, and strict config schema.
• Implement the selected detector handler abstraction.
• Implement occurrence evidence and event data.
• Implement thresholds and recovery for stateful detectors.
• Verify registry discovery at startup.

API and persistence Copy Section

• Implement the detector-specific validator.
• Validate condition comparisons and detector config.
• Reuse or register a data source type.
• Create DataSource and DataSourceDetector through the validator path.
• Confirm cache invalidation covers every new processing relationship.
• Confirm deletion and relocation behavior.

Producer Copy Section

• Build the typed DataPacket.
• Call process_data_packet or deliberately resolve and call process_detectors.
• Match packet source identity to the persisted DataSource.
• Add bounded metrics and exception handling.

Rollout Copy Section

• Configure Issue Platform release controls.
• Add product flags or options if needed.
• Validate throughput, state cardinality, and payload sizes.
• Plan trigger and recovery monitoring.

Test Matrix Copy Chapter

Use existing factory methods rather than direct model creation in tests. The most useful
shared references are:

• test_base.py
  for the default stateless evaluation
• test_stateful.py
• test_data_packet.py
• test_detector.py
• test_organization_project_detector_index.py
• test_integration.py

Recommended Implementations Copy Chapter

Copy architecture, not product assumptions. Each example has specialized producer,
state, and lifecycle behavior.

Src Sentry Workflow Engine Docs Conditions (`src/sentry/workflow_engine/docs/conditions.md`) Copy Section

Conditions in the Workflow Engine

Detectors and Workflows both store DataConditionGroup
and DataCondition records. Their immediate paths use
process_data_condition_group; delayed
Workflow processing evaluates slow conditions with
evaluate_data_conditions. They do not pass
the same values into that shared layer or interpret its results the same way.

Read this document before adding or changing a condition. It defines the shared
evaluation contract, the Detector and Workflow-specific behavior around it, and the
complete path for adding a condition type and handler.

Architecture Copy Chapter

flowchart TD
    subgraph Shared[Shared condition layer]
        Group[DataConditionGroup]
        Conditions[DataCondition records]
        Process[process_data_condition_group]
        Delayed[evaluate_data_conditions for delayed values]
        Evaluation[DataConditionGroupEvaluation]
        Group --> Conditions
        Conditions --> Process
        Process --> Evaluation
        Conditions --> Delayed
        Delayed --> Evaluation
    end

    Packet[DataPacket] --> Extract[DetectorHandler extracts product value]
    Extract --> Process
    Evaluation --> DetectorResult[Detector interprets priority results]
    DetectorResult --> Issue[IssueOccurrence or StatusChangeMessage]

    Event[WorkflowEventData] --> Process
    Evaluation --> When[Workflow interprets WHEN as a gate]
    When --> If[Workflow evaluates IF groups]
    If --> Actions[Eligible actions]

The paths converge only in the shared condition layer. Detector output later reaches
Workflow processing through Issue Platform ingestion and issue post-processing; a
Detector evaluation does not invoke a Workflow directly. See
process_detectors and
process_workflows.

Shared Condition Layer Copy Chapter

Persistent records Copy Section

DataCondition stores:

DataConditionGroup stores an organization and a
logic type. Detector trigger groups, Workflow WHEN groups, and Workflow IF groups all
use this same model.

Those are intended roles, not exclusive database types. The schema independently
constrains each relation, but does not prevent one group from being referenced through
more than one role-bearing relation. Code that creates groups must preserve role
separation by convention.

Condition dispatch Copy Section

DataCondition.evaluate_value evaluates one condition:

1. eq, gte, gt, lte, lt, and ne use the built-in operators in
   CONDITION_OPS.
2. Every other Condition looks up a class in
   condition_handler_registry and calls
   handler.evaluate_value(value, comparison).
3. A boolean handler/operator result is interpreted as a match:
   • False becomes result=None and triggered=False.
   • True is replaced by get_condition_result(), which converts a valid integer
     priority to DetectorPriorityLevel, preserves floats and booleans, leaves other
     integers unchanged, and rejects unsupported JSON shapes with ConditionError.
4. A non-boolean handler result passes through unchanged and does not receive this
   stored-result normalization.
5. The final evaluation is triggered when result is not None, not when the result is
   truthy.

The last two rules are important. A matched condition whose stored condition_result is
False has triggered=True and result=False. A Detector handler that directly returns
the raw integer 1 does not produce DetectorPriorityLevel.LOW and can be ignored by
the stateful priority filter. Consumers must use .triggered rather than bool(result).

The return value is a
DataConditionEvaluation. It carries the
condition, input data, result, triggered flag, and optional ConditionError.

Group evaluation contract Copy Section

The shared entry point is:

def process_data_condition_group(
    group: DataConditionGroup,
    value: T,
    data_conditions_for_group: list[DataCondition] | None = None,
) -> tuple[DataConditionGroupEvaluation, list[DataCondition]]:
    ...

The first return item is the current group evaluation. The second is the list of slow
conditions that still require evaluation. It is not a list of every condition skipped
by short-circuiting.

If data_conditions_for_group is omitted, the processor uses prefetched conditions or
loads them through get_data_conditions_for_group. Bulk callers should pass already
loaded conditions to avoid per-group queries.

DataConditionGroup.Type has four values:

An existing group with no conditions returns a clean, triggered evaluation for every
logic type. A passing NONE group has no passing condition evaluations in its result
data. An invalid stored logic type returns a non-triggered evaluation with a
ConditionError.

These behaviors are implemented by
evaluate_data_conditions and
evaluate_condition_group_results.

Inline and batched condition data Copy Section

SLOW_CONDITIONS is a manual list of six event-frequency
and session condition types. A slow condition has the same DataConditionHandler and
evaluation-result contract as any other condition. "Slow" describes how its input data
is acquired: the required query is too expensive to run inline for each Workflow, so the
engine batches equivalent queries and gathers the data once for evaluation across the
buffered Workflows.

The classification is not inferred from DataConditionHandler.Group, subgroup, or
handler behavior. It only controls the split between inline evaluation and the batched
data-acquisition phase.

process_data_condition_group splits fast and slow conditions:

1. A group containing only slow conditions returns a provisional non-triggered result
   and all slow conditions.
2. Otherwise, fast conditions run first.
3. A fast result is conclusive when:
   • ANY or ANY_SHORT_CIRCUIT passes, or
   • ALL or NONE fails.
4. A conclusive result discards the remaining slow list.
5. A non-conclusive result returns the remaining slow conditions to the caller.

For event-driven Workflows, the delayed processor batches query generation, obtains the
query values, and calls the same evaluate_data_conditions function used by inline
group processing. This is a scheduling/data-fetch difference, not a second condition
semantics system. Activity-driven Workflows and stateful Detectors do not enqueue this
batched phase.

Errors and taint Copy Section

BaseWorkflowEngineEvaluation keeps result,
triggered, and error separate. An evaluation with an error is tainted: the answer
may have been affected by a failed condition.

The any, all, and none combinators propagate errors only when the failed input
could affect the answer. For example, one clean true input makes ANY clean even when
another input failed; an ANY result of false remains tainted if any input failed.

The condition error boundary is intentionally narrow:

• Operator TypeError, missing handler registration, invalid stored result data, and a
  handler-raised DataConditionEvaluationException become tainted, non-triggered
  condition evaluations.
• An invalid stored condition-type string raises
  DataConditionEvaluationException from DataCondition.evaluate_value.
• Arbitrary exceptions raised by a handler are not converted to ConditionError.

An arbitrary exception therefore escapes the shared layer. Whether it is retried depends
on the Detector producer or Workflow task entrypoint; the condition framework itself
does not retry handler evaluation.

Neither Detector nor Workflow processing automatically suppresses every tainted result.
A tainted group whose .triggered value is decisive can still cause a state transition
or select actions. Handler code should raise DataConditionEvaluationException only for
known evaluation failures and must not rely on taint as a retry mechanism.

Validation is layered Copy Section

BaseDataConditionValidator performs
the complete API validation path:

• Validates the enum value and handler registration
• Validates comparison and result JSON against handler schemas
• Calls handler.validate_comparison(comparison, organization) when organization
  context is available

The model pre_save receiver in
receivers/data_condition.py is weaker. It validates
only comparison JSON for registered, non-operator handlers. It does not enforce result
schemas, contextual validation, or handler placement, and bulk_create bypasses the
signal.

Use the existing validators for API/model graph creation. Do not treat direct model
saves as equivalent validation.

Detector and Workflow Touchpoints Copy Chapter

DataConditionHandler.Group drives the top-level control placement exposed through
OrganizationDataConditionIndexEndpoint.
The intended conceptual split is:

• group identifies whether the control belongs to Detector or Workflow configuration.
• subgroup organizes the UI control within that subsystem, such as trigger or action
  filter placement.

The current enum shape does not represent that hierarchy literally. Group currently
combines subsystem and control phase through DETECTOR_TRIGGER, WORKFLOW_TRIGGER, and
ACTION_FILTER. Subgroup currently refines UI presentation with
ISSUE_ATTRIBUTES, EVENT_ATTRIBUTES, and FREQUENCY. The availability API serializes
those current values as handlerGroup and handlerSubgroup. Subgroup does not affect
evaluation.

Neither field determines the runtime input type or prevents a caller from storing a
handler in the wrong role. Matching the handler's input and result contract to its
placement is a convention-only invariant.

Detectors Copy Chapter

A Detector represents configured detection. A runtime handler
is selected through the DetectorSettings registered for the detector
type in detector_settings_registry.

Handlers that inherit DetectorHandler without overriding evaluate use the
default stateless evaluation. It
selects a priority the same way as the stateful path below and discards slow conditions
the same way, but it has no dedupe or thresholds and produces no output for OK.

The common stateful path is
StatefulDetectorHandler.evaluate:

1. Extract one packet-wide positive integer dedupe value.
2. Extract one evaluation value or a mapping of DetectorGroupKey to value.
3. Load PostgreSQL detector state and Redis dedupe/counter state.
4. Skip a group key when the packet dedupe value is not newer.
5. Enqueue the new dedupe watermark before condition evaluation.
6. Pass each extracted value to process_data_condition_group.
7. From a triggered group, keep triggered results that are actual
   DetectorPriorityLevel values and choose the maximum.
8. Apply priority threshold rules and persist state.
9. Produce an IssueOccurrence for a non-OK transition or a StatusChangeMessage for
   a transition to OK.

Detector-specific consequences:

• A fresh packet advances the Redis watermark even when no condition matches.
• One dedupe value applies to every group key extracted from that packet. Independent
  per-key ordering requires custom orchestration.
• A passing condition with condition_result=DetectorPriorityLevel.OK is the normal
  recovery signal. Failing all non-OK conditions does not imply recovery.
• Stateful Detector processing has no batched data-acquisition phase. Conditions that
  require it are logged as remaining work and discarded, so Detector trigger inputs must
  be available inline.
• A tainted triggered result can still update counters and produce a transition.
• PostgreSQL state, Redis state, and Issue Platform publication are not one transaction.
  DetectorStateManager.commit_state_updates writes PostgreSQL before Redis, and
  process_detectors publishes only after handler evaluation returns. A failure after
  state commit but before publication can cause a retry to be skipped by dedupe.

The default on_complete callback enters Issue Platform through
IssuePlatformOutcomeHandler. Workflows
run later from the resulting issue event; they are not part of the detector state transaction.

Workflows Copy Chapter

A Workflow is connected to detectors through
DetectorWorkflow. Its conditions have two stages:

1. The optional WHEN group decides whether processing continues.
2. Each IF group is evaluated independently and selects the actions attached to that
   group through DataConditionGroupAction.

Both stages pass equivalent WorkflowEventData values to the shared
layer. Each stage creates a dataclass copy with its Workflow environment; the copies
share the event-local cache. The value contains a GroupEvent or Activity, the issue
group, optional state and escalation data, and an optional workflow environment.

process_workflows performs the surrounding lifecycle:

1. Resolve the event-specific and issue-stream detectors.
2. Load enabled Workflows connected to those detectors for the event environment.
3. Evaluate WHEN groups.
4. Evaluate IF groups for Workflows whose WHEN passed or remains pending slow work.
5. Buffer unresolved slow work for eligible GroupEvent processing.
6. Resolve attached actions for passing IF groups.
7. Hand passing action groups to the Workflow frequency, history, active-action,
   deduplication, and asynchronous dispatch path documented in Execution.

Event and activity differences Copy Section

These differences are implemented in tasks/workflows.py,
get_environment_by_event, and
get_preferred_detector.

Adding a Condition Type and Handler Copy Chapter

This walkthrough is verified against
FirstSeenEventConditionHandler and
test_first_seen_event_handler.py.
The example adds a fast Workflow WHEN condition. Differences for Detector triggers and
Workflow IF conditions are called out explicitly.

1. Choose the placement and contract Copy Section

Before editing, decide:

• Detector trigger: What detector-specific value will the handler receive? Should it
  return a priority directly, or a boolean that maps to the stored priority result?
• Workflow WHEN: What property of WorkflowEventData gates the entire Workflow?
• Workflow IF: What property of WorkflowEventData gates one action group?
• Batched Workflow WHEN or IF: Does acquiring the evaluation input require an
  expensive query? If so, the condition handler itself is unchanged, but the delayed
  data-acquisition path must know how to batch and supply that input.

Do not make one handler accept unrelated Detector and Workflow input shapes. The generic
base class cannot enforce the runtime input type.

2. Add the enum value Copy Section

Edit models/data_condition.py:

class Condition(StrEnum):
    # ...
    MY_NEW_CONDITION = "my_new_condition"

The enum feeds model choices and API validation. Storage is a CharField, not a database
enum. Run the repository's migration check rather than assuming whether a model-state
migration is required.

3. Implement and register the handler Copy Section

For a fast Workflow WHEN condition, add
handlers/condition/my_new_condition_handler.py:

from typing import Any

from sentry.workflow_engine.models.data_condition import Condition
from sentry.workflow_engine.registry import condition_handler_registry
from sentry.workflow_engine.types import DataConditionHandler, WorkflowEventData


@condition_handler_registry.register(Condition.MY_NEW_CONDITION)
class MyNewConditionHandler(DataConditionHandler[WorkflowEventData]):
    group = DataConditionHandler.Group.WORKFLOW_TRIGGER
    comparison_json_schema = {"type": "boolean"}
    condition_result_schema = {"type": "boolean"}
    label_template = "The new condition matches"

    @staticmethod
    def evaluate_value(event_data: WorkflowEventData, comparison: Any) -> bool:
        # `group.some_field` is illustrative; use a real Group property.
        return comparison and event_data.group.some_field

Required handler contract from DataConditionHandler:

For a Detector trigger, specialize the generic with the actual detector value type. A
boolean return maps to the stored DetectorPriorityLevel; a direct non-boolean
DetectorPriorityLevel return passes through. Detector API validation requires stored
condition results to be integers convertible to DetectorPriorityLevel.

For a Workflow IF condition, use DataConditionHandler.Group.ACTION_FILTER, continue to
accept WorkflowEventData, and set a subgroup when the automation UI needs to place it
under issue attributes, event attributes, or frequency.

4. Make the registration run at startup Copy Section

Edit handlers/condition/__init__.py:

__all__ = [
    # ...
    "MyNewConditionHandler",
]

from .my_new_condition_handler import MyNewConditionHandler

This import is functional. Config.ready imports the condition package;
the package import executes the registration decorator. An enum and handler class that
are not imported remain unavailable at runtime.

A product-specific condition can live outside this package, but its owning app startup
must import the decorated module. Anomaly detection demonstrates this path through
incidents/handlers/condition and
incidents/grouptype.py.

5. Decide whether manual lists apply Copy Section

Registration does not update these lists automatically:

A condition whose data must be acquired in the batched phase additionally needs a query
handler registered in
slow_condition_query_handler_registry. Implement the
BaseEventFrequencyQueryHandler
data-acquisition contract and register the condition in that module. This handler does
not replace DataConditionHandler; it gathers the values that are later passed through
the same condition evaluator. Without this registration,
generate_unique_queries cannot build a query and
the buffered condition cannot fire.

6. Verify API exposure and serialization Copy Section

Once registered, the handler appears in
OrganizationDataConditionIndexEndpoint
under its group, unless it is in LEGACY_CONDITIONS. The endpoint exposes the type,
group, optional subgroup, and comparison schema.

Stored conditions serialize through
DataConditionSerializer.
Comparison dictionary keys are converted to the API's camelCase representation.

If validate_comparison resolves a user, team, integration, or other organization-owned
object, scope the lookup by the supplied organization. The generic validator does not
make an unscoped handler lookup safe.

7. Add focused backend tests Copy Section

Add tests/sentry/workflow_engine/handlers/condition/test_my_new_condition_handler.py,
using ConditionTestCase.

Cover:

• Passing and non-passing values
• Exact result and .triggered behavior
• Missing or malformed runtime data
• Comparison JSON Schema acceptance and rejection
• Known failure conversion when using DataConditionEvaluationException
• Event versus activity behavior when relevant
• Organization scoping when validate_comparison or render_label loads objects
• Registry discovery through normal application startup

Add an explicit registration assertion without importing the implementation module in
the test:

handler = condition_handler_registry.get(Condition.MY_NEW_CONDITION)
assert handler.__name__ == "MyNewConditionHandler"

Then test the placement:

• Detector trigger: create a detector condition with a priority result and assert the
  detector transition or DetectorEvaluation in
  test_detector.py
  or the owning product's handler test. If the condition needs a new extracted value,
  update and test the owning DetectorHandler.extract_value implementation.
• Workflow WHEN: assert the Workflow stops or proceeds in
  test_workflow.py.
• Workflow IF: in the same processor test, create the
  WorkflowDataConditionGroup/DataConditionGroupAction graph and assert only actions
  attached to passing groups are selected.
• Slow Workflow WHEN or IF: extend
  test_delayed_workflow.py
  for query generation, WHEN gating or IF selection, preservation of passing IF groups,
  result combination, retry behavior, and cleanup.

Detector-specific comparison validation belongs in the owning subclass of
BaseDetectorTypeValidator. The generic
condition handler alone cannot add detector packet data or detector-specific API fields.

8. Add optional compatibility and frontend support Copy Section

These are separate decisions, not part of backend registration.

Production code no longer translates legacy issue-alert conditions. Test factories still do,
through data_condition_translator_mapping in
testutils/helpers/issue_alert_conditions.py.

For the automation builder, add the matching frontend enum and node implementation in a
separate frontend PR:

• static/app/types/workflowEngine/dataConditions.tsx
• static/app/views/automations/components/dataConditionNodes.tsx

The backend comparison schema does not generate the complete editor; current conditions
have explicit frontend nodes, defaults, details, and validation.

File Map Copy Chapter

Conventions and Invariants Copy Chapter

Src Sentry Workflow Engine Docs Data Model (`src/sentry/workflow_engine/docs/data-model.md`) Copy Section

Workflow Engine Data Model

This document describes the persistent graph used by the Workflow Engine. It focuses
on relationships, ownership, and invariants rather than repeating every Django field.
For runtime control flow, see Execution.

Conceptual Graph Copy Chapter

erDiagram
    ORGANIZATION ||--o{ DATA_SOURCE : owns
    PROJECT |o--o{ DETECTOR : owns
    DATA_SOURCE ||--o{ DATA_SOURCE_DETECTOR : maps
    DETECTOR ||--o{ DATA_SOURCE_DETECTOR : evaluates

    DETECTOR ||--o| DETECTOR_TRIGGER_GROUP : owns
    DETECTOR_TRIGGER_GROUP ||--o{ DETECTOR_CONDITION : contains

    DETECTOR ||--o{ DETECTOR_WORKFLOW : connects
    WORKFLOW ||--o{ DETECTOR_WORKFLOW : connects
    ORGANIZATION ||--o{ WORKFLOW : owns

    WORKFLOW ||--o| WHEN_GROUP : owns
    WHEN_GROUP ||--o{ WHEN_CONDITION : contains
    WORKFLOW ||--o{ WORKFLOW_DATA_CONDITION_GROUP : has
    IF_GROUP ||--o| WORKFLOW_DATA_CONDITION_GROUP : connects
    IF_GROUP ||--o{ IF_CONDITION : contains
    IF_GROUP ||--o{ DATA_CONDITION_GROUP_ACTION : gates
    ACTION ||--o{ DATA_CONDITION_GROUP_ACTION : executes

    DETECTOR ||--o{ DETECTOR_STATE : tracks
    DETECTOR ||--o{ DETECTOR_GROUP : creates
    GROUP ||--o| DETECTOR_GROUP : associates

    WORKFLOW ||--o{ WORKFLOW_ACTION_GROUP_STATUS : throttles
    ACTION ||--o{ WORKFLOW_ACTION_GROUP_STATUS : throttles
    GROUP ||--o{ WORKFLOW_ACTION_GROUP_STATUS : throttles

    WORKFLOW ||--o{ WORKFLOW_FIRE_HISTORY : records
    GROUP ||--o{ WORKFLOW_FIRE_HISTORY : records
    DETECTOR |o--o{ WORKFLOW_FIRE_HISTORY : attributes

The diagram is conceptual. Some relationships use join models rather than direct
foreign keys, and compatibility models are omitted from the main graph. The abbreviated
names are condition roles, not model names: WHEN_GROUP/WHEN_CONDITION correspond to
the DataConditionGroup/DataCondition on Workflow.when_condition_group;
IF_GROUP/IF_CONDITION to those on WorkflowDataConditionGroup.condition_group;
DETECTOR_TRIGGER_GROUP to Detector.workflow_condition_group.

Detection Graph Copy Chapter

`DataSource` Copy Section

DataSource belongs to an organization and generically
references a product model that emits detector input. The pair (type, source_id) is
globally unique; the organization is not part of that database constraint.

type selects a registered
DataSourceTypeHandler. The handler provides behavior that cannot be
expressed by the generic reference alone:

• Bulk-loading the referenced product objects
• Deletion relationships
• Organization quota calculation
• Relocation model naming

source_id is text because source models do not share a primary-key type. Callers must
use the registered handler instead of assuming which model it references.

Known examples include Snuba query subscriptions, uptime subscriptions, and cron
monitors. Adding a source type requires both registration and startup import coverage.

`DataSourceDetector` Copy Section

DataSourceDetector is the many-to-many mapping
between sources and detectors. (data_source, detector) is unique.

One source can feed multiple detectors. One detector can also be connected to source
records according to the owning product's model. Detector lookup by source is cached,
so mutations must preserve the receiver-driven invalidation path.

`Detector` Copy Section

Detector is the configured unit of detection. Important
relationships and fields are:

The detector type is registered outside this Django app, in two parts. A concrete
GroupType defines the issue type. A
DetectorSettings subclass, registered in
detector_settings_registry under the group type's slug, can define:

• Runtime handler
• API validator
• Configuration JSON schema
• Query filter for detector visibility

GroupType.detector_settings and Detector.settings read from the registry.

Detector does not enforce a database uniqueness constraint for project and type.
Default detector creation uses short distributed locks and idempotent lookups instead.

Detector trigger conditions Copy Section

The detector's workflow_condition_group points to a
DataConditionGroup. The relation is nullable and
unique independently of the Workflow condition relations.

The field name is historical: this group belongs to detector evaluation even though it
is named workflow_condition_group.

Conditions Copy Chapter

`DataCondition` Copy Section

DataCondition contains:

`DataConditionGroup` Copy Section

DataConditionGroup owns an organization and a
stored logic type. Groups have three intended roles:

• Detector trigger group
• Workflow WHEN group
• Workflow IF/action-filter group

Role exclusivity is not enforced across these relations, and generic validators do not
enforce handler placement. Creation code must keep the roles separate by convention.
See Conditions for the canonical evaluation, validation, and placement
contracts.

Workflow Graph Copy Chapter

`DetectorWorkflow` Copy Section

DetectorWorkflow connects detectors to workflows.
The connection determines which workflows are candidates for an issue event associated
with that detector.

An issue event can select more than one detector for lookup:

• Its specific detector, when known
• The project's issue-stream detector
• An optional organization-wide all-projects issue-stream detector

This allows workflows to target a product-specific detector or the broader issue stream.

`Workflow` Copy Section

Workflow is organization-scoped. Important fields are:

`WorkflowDataConditionGroup` Copy Section

WorkflowDataConditionGroup connects one
workflow to one IF/action-filter condition group. A condition group can belong to only
one workflow through this relationship.

`DataConditionGroupAction` and `Action` Copy Section

DataConditionGroupAction connects actions
to IF groups. Action stores the action type and its handler-
specific configuration and data.

The action type selects a handler from
action_handler_registry. Handlers are commonly implemented in the
owning integration or notification package rather than in workflow_engine.

An action can be reachable from more than one Workflow. The data model does not define
action execution order.

Runtime State and History Copy Chapter

`DetectorState` Copy Section

DetectorState stores durable detector status for a
detector and optional DetectorGroupKey. The group key lets one detector
track independent state for multiple values in a packet, such as separate endpoints.

The unique constraint treats a null group key as an empty value so that a non-grouped
detector has one state row.

Detector state is split across stores:

• PostgreSQL stores the durable priority and triggered status.
• Redis stores dedupe watermarks and priority threshold counters with a TTL.

The split is part of the runtime contract. For StatefulDetectorHandler, dedupe values
must be positive integers that increase with source order. The watermark defaults to
zero when absent and expires from Redis after seven days, so duplicate suppression is
not durable beyond that TTL.

`DetectorGroup` Copy Section

DetectorGroup associates an Issue Platform
Group with the detector that produced or owns it. The detector
foreign key can become null after deletion so existing issue history remains valid.

The association is normally created during Issue Platform ingestion. Compatibility
paths can backfill missing associations. Callers resolving a detector from an issue must
handle deleted or missing detector records.

`WorkflowActionGroupStatus` Copy Section

WorkflowActionGroupStatus records when a
(workflow, action, group) tuple last passed repeat-frequency filtering. It is updated
before event-level deduplication and task execution, so it does not prove that an
external action completed.

`WorkflowFireHistory` Copy Section

WorkflowFireHistory stores a Workflow, issue
group, event ID, notification UUID, and optional detector. It has no action or delivery
outcome field. Rows are created before inactive-action filtering and event-level
deduplication, so they record pre-dispatch intent rather than a scheduled or successfully
delivered action.

Compatibility Models Copy Chapter

Compatibility association models map legacy issue and metric alert records to Detector,
Workflow, condition, action, and Issue Platform records. They are not part of normal
Detector or Workflow evaluation. See Legacy alert API compatibility
for the model list and current mixed read/write behavior.

Lifecycle and Invalidation Copy Chapter

Creation Copy Section

Detector model API creation is coordinated by
BaseDetectorTypeValidator. In one
transaction it creates or connects the detector's condition graph, product-specific
source object, generic data source, source mapping, and workflows.

Default detectors and workflows are created by defaults/ from project
and organization signal receivers. These functions must remain idempotent.

Updates Copy Section

Updates to detectors, workflows, conditions, source mappings, or action mappings can
invalidate cached processing graphs. Receivers in receivers/ generally
schedule invalidation with transaction.on_commit() so a cache cannot be repopulated
from uncommitted data.

Bypassing validators or relationship models can skip schema validation, audit logging,
or cache invalidation. Prefer the existing validator and service paths.

Deletion Copy Section

Detector API deletion marks the detector pending deletion and schedules a cell deletion
task. Related Issue Platform groups and historical fire records may outlive the detector.
Runtime lookup must therefore tolerate missing records.

Action cleanup for control-silo integration changes goes through the hybrid-cloud
ActionService.

Scope and Relocation Copy Chapter

Workflow Engine models live in the cell silo. Models participate in Sentry's relocation
framework according to their declared scopes. DataSource requires special
normalization because its generic source_id must be remapped using the registered
source handler's model name.

Do not infer organization ownership only through an arbitrary related ID. API lookups
must be explicitly scoped to the organization and permitted projects.

Recommended Tests Copy Chapter

• test_organization_project_detector_index.py
  demonstrates transactional detector graph creation.
• test_detectors.py
  covers default creation and locking.
• test_workflows.py
  covers default workflow graphs.
• receivers/ covers validation and
  cache invalidation.
• project_transfer.py contains project-transfer
  cloning and reconnection helpers; these currently have no dedicated processor test.

Src Sentry Workflow Engine Docs Execution (`src/sentry/workflow_engine/docs/execution.md`) Copy Section

Workflow Engine Execution

This document traces data through the Workflow Engine processors and asynchronous
boundaries. For persistent relationships, see Data model. For an
implementation checklist, see Adding detectors.

Two Pipelines Copy Chapter

Detection and workflow processing are connected through Issue Platform, not by a direct
function call. Detector output is published to Issue Platform; the resulting issue event
later enters Workflow processing through post-processing or an activity. The overview
diagram is in the module README.

Existing issue events can enter at post-processing without first running a Workflow
Engine detector. This is how error and issue-stream workflows participate in the same
workflow pipeline.

Detection Pipeline Copy Chapter

1. Build a packet Copy Section

A producer creates a typed DataPacket:

DataPacket(source_id=str(source.id), packet=product_payload)

The packet's payload remains product-specific. The producer also supplies the source
type used to resolve source_id.

2. Resolve detectors Copy Section

process_data_packet is the generic entry point. It
calls process_data_source, which resolves:

(source type, source_id)
    -> DataSource
    -> DataSourceDetector
    -> Detector rows with enabled=True through the default manager

The lookup uses the detector-by-source cache and preloads trigger conditions. Cache
invalidation is connected to source, detector, and mapping updates.

The default manager excludes pending-deletion and deletion-in-progress rows, but this
path does not exclude every non-active status such as plan-level DISABLED.

Not every producer uses this generic lookup. Uptime, processing errors, and preprod
paths can identify detectors through product-specific data and call
process_detectors directly. The handler contract is the
same after detector selection.

3. Evaluate each detector Copy Section

process_detectors obtains detector.detector_handler from the DetectorSettings
registered for the detector's type in the detector_settings_registry. Then, it calls the _evaluate method on the handler,
which delegates to the default or overridden evaluate methods that contain most of the detector's logic.

One packet can produce:

• No evaluation result
• One result for an ungrouped detector
• Multiple results keyed by DetectorGroupKey

A handler returns a mapping of group keys to DetectorEvaluation objects. An evaluation
can contain an IssueOccurrence, a StatusChangeMessage, or None. process_detectors
routes non-null results according to the handler's outcome.

4. Detector orchestration Copy Section

Every detector handler inherits DetectorHandler. Its
default evaluate is stateless; StatefulDetectorHandler replaces it with durable
state and thresholds.

Stateless (default)

flowchart TD
    Packet[Receive DataPacket] --> Extract[Extract evaluation values]
    Extract --> Group[Normalize to group key and value pairs]
    Group --> Conditions[Evaluate detector condition group]
    Conditions --> Priority[Select highest triggered priority]
    Priority --> Ok{Priority is OK?}
    Ok -->|Yes| Stop[No Issue Platform output]
    Ok -->|No| Occurrence[Build IssueOccurrence]

Important semantics:

• Each packet is evaluated independently. There is no dedupe, threshold, or durable
  state.
• OK produces no status-change message; the default path never resolves issues.

Stateful

Most threshold-based detectors inherit
StatefulDetectorHandler:

flowchart TD
    Packet[Receive DataPacket] --> Extract[Extract dedupe value and evaluation values]
    Extract --> Group[Normalize to group key and value pairs]
    Group --> Load[Bulk-load durable and Redis state]
    Load --> Newer{Dedupe value is newer?}
    Newer -->|No| Skip[Skip stale value]
    Newer -->|Yes| Conditions[Evaluate detector condition group]
    Conditions --> Priority[Select highest passing priority]
    Priority --> Counters[Update threshold counters]
    Counters --> Transition{Priority state changed?}
    Transition -->|Non-OK| Occurrence[Build IssueOccurrence]
    Transition -->|OK| Resolution[Build StatusChangeMessage]
    Transition -->|No| Persist[No Issue Platform output]
    Occurrence --> Persist[Persist state, counters, and dedupe]
    Resolution --> Persist

The handler performs state I/O in bulk for grouped values. The exact transaction and
Redis pipeline behavior is implemented by DetectorStateManager in the same module.

Important semantics:

• Dedupe values must be positive integers that increase in event order. Equal or smaller
  values are ignored while the Redis watermark exists. The watermark defaults to zero
  when absent and expires after seven days.
• State is independent for each detector group key.
• Reaching a higher priority also increments applicable lower-priority counters.
• Thresholds describe how many qualifying evaluations, according to the priority
  counter rules, are needed before transition.
• A transition to a non-OK priority creates an occurrence.
• A transition to OK creates a status-change message.
• No state transition means no Issue Platform message.

Failing every non-OK condition does not infer recovery because a non-triggered group
produces no transition. Recovery requires a triggered group whose selected priority
remains OK. This normally comes from a passing condition with an OK result, but an
empty or passing NONE group can also trigger without a priority-bearing result and
therefore use the stateful handler's default OK priority. A missing trigger group is
invalid and produces no transition.

Detectors that need a different flow inherit DetectorHandler and override evaluate,
but then they own this orchestration. BaseDetectorHandler is only the abstract
interface and is not a base for new detectors.

5. Produce detector output Copy Section

process_detectors invokes the detector handler's on_complete lifecycle callback with
the detector and each evaluation. The default callback is
DetectorOutcome.ISSUE_PLATFORM.dispatch; it resolves the registered outcome handler,
whose handle method ignores evaluations without a result and passes the others to Issue
Platform via produce_occurrence_to_kafka. A detector can
assign another supported outcome dispatcher or override on_complete for custom processing.

Supported outcome implementations register their DetectorOutcomeHandler class with
@detector_outcome_registry.add(DetectorOutcome.<KEY>). The closed, typed set of keys
lives in supported_outcomes.py; the registry maps each key to its handler implementation.

Issue Platform ingestion creates or updates a group. The detector ID in occurrence
evidence allows ingestion to create a
DetectorGroup association. Resolution uses the stable
issue fingerprint to find the same group.

The Kafka boundary means detector evaluation succeeding does not imply that group
creation, workflow processing, or action dispatch has completed.

Workflow Ingress Copy Chapter

Event post-processing Copy Section

process_workflow_engine schedules the
process_workflows_event task only for non-reprocessed jobs
with an event whose group is currently unresolved. The task rebuilds a
WorkflowEventData from EventStore and Issue Platform data using
tasks/utils.py. Resolution-driven processing enters through
supported issue activities instead.

Issue activities Copy Section

Issue activities invoke handlers through
invoke_workflow_activity_handlers.
Registered handlers receive an optional detector ID. A handler that schedules
process_workflow_activity must first resolve a concrete
detector and pass its required detector ID with the activity and group.

Event and activity paths use WorkflowEventData, which contains:

• A GroupEvent or Activity
• The issue Group
• Optional group state and escalation information
• Optional workflow environment
• A per-event local cache for repeated lookups

They do not have identical scheduling. Activity processing has no workflow environment,
so it considers only global workflows. Activities do not enter batched condition data
acquisition; an activity whose WHEN or IF result still needs a slow query does not
continue through delayed processing.

Workflow Pipeline Copy Chapter

process_workflows is the main synchronous orchestrator
inside the workflow task.

flowchart TD
    Event[WorkflowEventData] --> Detectors[Resolve event and issue-stream detectors]
    Detectors --> Environment[Resolve event environment]
    Environment --> Workflows[Load connected enabled workflows]
    Workflows --> When[Evaluate WHEN groups]
    When -->|False| Done[Record non-triggered evaluation]
    When -->|True or slow result pending| Filters[Load and evaluate IF groups]
    Filters -->|WHEN or IF slow conditions remain| Delay[Buffer delayed workflow]
    Filters -->|WHEN true and IF groups pass| Select[Select attached actions]
    Select --> Frequency[Apply repeat frequency]
    Frequency --> History[Create fire history]
    History --> Dedup[Filter inactive and deduplicate by get_dedup_key]
    Dedup --> Tasks[Dispatch action tasks]

1. Resolve event detectors Copy Section

get_detectors_for_event_data builds EventDetectors:

• For a GroupEvent with an occurrence, event_detector is resolved only from the
  occurrence's detector_id evidence; a missing or deleted ID yields no event detector.
  For an ordinary error event with no occurrence, the project's default error detector
  is used.
• issue_stream_detectors contains the project issue-stream detector and, when enabled,
  the organization all-projects detector.
• preferred_detector is the event detector when available, otherwise the first
  issue-stream detector.

The preferred detector is later supplied to actions that need detector context.
Detector deletion and older groups can leave associations missing, so lookup paths are
expected to degrade safely.

2. Resolve environment and workflows Copy Section

The processor resolves the event environment, then
get_workflows_by_detectors loads workflows connected to any
selected detector. Eligible workflows must be enabled and either global
(environment=None) or connected to the event's environment.

The workflow cache is keyed by detector and environment. Relationship receivers
invalidate it after committed changes.

3. Evaluate WHEN conditions Copy Section

evaluate_workflow_triggers evaluates each workflow's WHEN
group. A Workflow with no group passes. A conclusively false WHEN stops processing and
reports metrics/context; the returned per-Workflow result map can remain empty.

4. Evaluate IF groups Copy Section

evaluate_workflows_action_filters loads IF groups from the
action-filter cache for workflows whose WHEN group passed or remains pending slow
evaluation. Fast IF results and remaining slow IF groups are retained with a pending
WHEN result so the combined work can be buffered.

Passing groups contribute their attached actions. A Workflow can therefore select
actions from multiple groups. A Workflow with no IF groups selects no actions.

The shared evaluation, fast/slow split, empty-group behavior, taint contract, and
Detector differences are documented once in Conditions.

Delayed Workflow Processing Copy Chapter

The delayed path batches expensive conditions across workflows and issue groups.

sequenceDiagram
    participant Immediate as Immediate workflow processor
    participant Redis as Delayed workflow Redis
    participant Scheduler as Cohort scheduler
    participant Task as Delayed workflow task
    participant Snuba as Snuba
    participant Action as Action processor

    Immediate->>Redis: Store event and remaining condition IDs
    Scheduler->>Redis: Select due project cohorts
    Scheduler->>Task: Schedule project or project batch
    Task->>Redis: Load buffered items
    Task->>Snuba: Execute bulk queries
    Snuba-->>Task: Results by issue group
    Task->>Action: Fire passing groups
    Task->>Redis: Delete successfully processed items

    Note over Task,Snuba: Equivalent condition queries are collapsed into bulk queries;<br/>results are combined with fast results before firing.

Buffering Copy Section

DelayedWorkflowClient stores project IDs in sharded sorted
sets and each project's work in a hash. A hash field is keyed by workflow, issue group,
delayed WHEN and IF groups, and IF groups that already passed. Its value retains the
event or occurrence ID and original evaluation timestamp. Re-enqueuing the same field
overwrites that value, coalescing equivalent pending work rather than preserving every
event.

Scheduling Copy Section

The schedule_delayed_workflows task acquires a global
scheduler lock around process_buffered_workflows, which
divides projects into cohorts and schedules due work. Large project hashes are split into
batches so asynchronous task messages contain identifiers rather than large payloads.

Evaluation Copy Section

process_delayed_workflows:

1. Parses buffered items.
2. Drops references to deleted workflows.
3. Loads remaining conditions.
4. Combines equivalent conditions into bulk queries.
5. Evaluates query results for each issue group.
6. Combines delayed results with fast results retained in the buffer key.
7. Reconstructs representative group events.
8. Sends passing action groups through the normal action path.
9. Deletes successfully processed Redis items.

Snuba rate limits are retried by the task. A missing per-group query result becomes a
tainted non-triggered evaluation. If a slow condition has no registered query handler,
no query is generated; processing can complete and remove the buffered item without
constructing that tainted result.

Malformed hash entries are excluded from the parsed event-key set and are not removed by
normal successful cleanup. They remain until expiry or manual cleanup.

Action Selection and Dispatch Copy Chapter

processors/action.py applies two different controls:

Repeat frequency Copy Section

WorkflowActionGroupStatus tracks when each
workflow, action, and issue group tuple last passed the frequency check. It is updated
before event-level deduplication and task execution. A tuple is suppressed when it
passed within Workflow.config.frequency minutes, even if a later equivalent action was
deduplicated or an external handler failed.

Event-level deduplication Copy Section

get_unique_active_actions first removes actions whose status is not active, then keeps
one concrete action for each Action.get_dedup_key(). This is an implementation key,
not a guarantee that every semantically equivalent external side effect is recognized.
Invocation attribution comes from the concrete action retained for the key and does not
imply Workflow execution order.

History and task dispatch Copy Section

The processor updates frequency statuses and creates
WorkflowFireHistory rows before fire_actions
removes inactive actions and deduplicates by key. It then schedules
trigger_action for retained actions. Fire-history rows describe
scheduling, not successful external delivery, and can exist for actions that are later
filtered or deduplicated.

The action task:

1. Requires exactly one event ID or activity ID.
2. Rebuilds WorkflowEventData.
3. Loads the action and preferred detector.
4. Creates an ActionInvocation.
5. Calls the handler registered for the action type.

External integration calls therefore run outside the workflow evaluation task. Action
handlers must handle deleted integrations and other expected external-state changes.

Processor Reference Copy Chapter

Failure and Consistency Model Copy Chapter

• Database graph mutations and cache invalidation are coordinated with
  transaction.on_commit() where stale repopulation would be unsafe.
• Detector state and Issue Platform publication are not atomic. PostgreSQL and Redis
  state commit before process_detectors publishes, so a failure in between can cause a
  retry to be skipped by the committed dedupe watermark. See
  Conditions for the commit ordering.
• Issue Platform publication is asynchronous. There is no transaction spanning detector
  state, Kafka, group creation, workflow evaluation, and external actions.
• Workflow action frequency and deduplication reduce repeated side effects but do not
  provide a distributed exactly-once guarantee for an external provider.
• Delayed items are removed after successful processing so task retries can recover
  buffered work.
• Models referenced by queued work can be deleted before a task runs. Tasks and
  handlers must treat expected missing records as normal lifecycle races.

Caches Copy Chapter

Treat receivers as part of the model mutation contract. A new relationship that affects
processing usually needs explicit invalidation coverage.

Receivers invalidate supported current relationship keys after commit. Updating a
DataSource type/source ID or a Workflow environment in place does not retain and
invalidate every old cache key; stale entries can therefore survive until TTL expiry.

Observability and Debugging Copy Chapter

The engine emits metrics for detector lookup/evaluation, condition evaluation, workflow
processing, delayed scheduling, caches, and actions. Logging helpers in
utils/log_context.py attach workflow and event context.

emit_evaluations records detector and workflow
evaluations through each EvaluationEmitter, which filters artifacts for its destination.
Runtime options and feature flags control sampling and direct logging.

When organizations:workflow-engine-evaluation-artifacts-eap is enabled,
EAPEmitter publishes artifacts to
SNUBA_ITEMS as TRACE_ITEM_TYPE_WORKFLOW_ENGINE_EVALUATION trace items. These are not
log trace items and do not include log body or severity attributes.

Delivery is best-effort: the producer does not track futures for task completion or apply
backpressure. Serialization and delivery failures are logged without retrying workflow
tasks. Artifacts can be dropped when Kafka is unavailable or its producer queue is full.

The sanitized trigger_evaluation, filter_evaluations, and delayed attributes are
JSON strings because EAP ingestion discards nested protobuf key/value lists. Scalar
attributes and primitive arrays, such as triggered_action_ids, retain their native types.

When tracing a missing action, check the boundaries in order:

1. Was the source mapped to an enabled detector?
2. Did detector state transition and publish an Issue Platform message?
3. Was the resulting issue associated with the expected detector?
4. Did post-processing or activity handling schedule workflow evaluation?
5. Was the workflow connected, enabled, and valid for the environment?
6. Did WHEN and at least one IF group pass?
7. Was a slow condition buffered and later processed?
8. Was the action suppressed by frequency or event-level deduplication?
9. Was fire history created and the action task scheduled?
10. Did the action handler reject missing or invalid external configuration?

Recommended Tests Copy Chapter

• test_integration.py
  covers the full detector and workflow paths.
• test_stateful.py
  covers state transitions and Redis behavior.
• test_detector.py
  covers detector output and event detector selection.
• test_workflow.py
  covers workflow evaluation.
• test_data_condition_group.py
  covers condition logic and fast/slow splitting.
• test_delayed_workflow.py
  covers delayed queries and action firing.
• test_schedule.py
  covers cohort scheduling and batches.
• test_actions.py
  covers action task reconstruction and dispatch.

Src Sentry Workflow Engine Docs Legacy Backport (`src/sentry/workflow_engine/docs/legacy_backport.md`) Copy Section

Legacy Alert API Compatibility

This document covers remaining legacy alert ID compatibility with Workflow Engine
models after the legacy alert APIs were retired. Start with
the Workflow Engine overview, data model, and
execution guide for current architecture.

Current Boundary Copy Chapter

The deprecated legacy alert API routes were removed in
sentry#121879. Clients must use
the detector and workflow APIs to manage alerts. The serializer and issue-alert
POST/PUT rollout flags no longer have consumers and have been retired.

Some compatibility code remains: lookup endpoints expose associations between legacy and
Workflow Engine models, and notification code still uses legacy-shaped data. Endpoint retirement does
not imply that the association tables, serializers, or ID helpers are all unused.

Handling IDs Copy Chapter

Dual-written data Copy Section

Data that was created by the legacy system and migrated (or is being written to both systems simultaneously). Association tables bridge the two:

• AlertRuleDetector — despite the name, this maps either a metric alert (alert_rule_id) or an issue alert (rule_id) to a Detector. Each row carries exactly one of the two (enforced by a check constraint).
• AlertRuleWorkflow — same pattern, mapping alert_rule_id or rule_id to a Workflow.
• IncidentGroupOpenPeriod — maps a legacy incident_id / incident_identifier to a GroupOpenPeriod.

When the workflow engine path receives a real legacy ID (e.g. an alertRule query param), it resolves the corresponding workflow engine object via these tables.

Single-written data Copy Section

Data created exclusively by the workflow engine with no legacy counterpart. These objects have no rows in the association tables. To maintain API compatibility, they are exposed with manufactured IDs. The helpers for this live in src/sentry/incidents/endpoints/serializers/utils.py:

• get_fake_id_from_object_id(obj_id) — used by serializers to manufacture an ID for API responses
• get_object_id_from_fake_id(fake_id) — used by endpoints to recover the real object ID from an incoming parameter. If non-positive, the input wasn't a valid manufactured ID.

Endpoints that accept IDs as input must handle both real legacy IDs (via association tables) and manufactured IDs (via get_object_id_from_fake_id).

Compatibility Endpoints Copy Chapter

Check the registered routes and their implementations when changing compatibility
code. The association lookup routes (alert-rule-workflow, alert-rule-detector, and
incident-groupopenperiod) live in src/sentry/workflow_engine/endpoints/urls.py.
The endpoint-tracking decorator was removed with the deprecated routes.

Unsupported legacy features Copy Chapter

Some legacy features can't or won't be supported in workflow engine models (e.g. AlertRule snapshots). Acknowledge these explicitly in code and tests where appropriate to make it clear which differences are known and intentional and which may be bugs.

Testing Copy Chapter

• Delta tests compare old and new serializer output for dual-written data, with an explicit known_differences set documenting expected divergences.
• Single-write tests verify that workflow engine-only data (no legacy counterpart) is returned correctly.
• Filter tests verify ID-based filters work for both real legacy IDs (via association tables) and manufactured IDs (via extraction).

--- METRICS ---

• Files Extracted: 7
• Estimated Token Budget: ~29824 tokens
• Canonical Reference: https://codewiki.google/github.com/getsentry/sentry