anubis Documentation

File: README.md Copy Chapter

Anubis

Sponsors Copy Chapter

Anubis is brought to you by sponsors and donors like:

Diamond Tier Copy Section

Gold Tier Copy Section

Overview Copy Chapter

Anubis is a Web AI Firewall Utility that weighs the soul of your connection using one or more challenges in order to protect upstream resources from scraper bots.

This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them.

Anubis is a bit of a nuclear response. This will result in your website being blocked from smaller scrapers and may inhibit "good bots" like the Internet Archive. You can configure bot policy definitions to explicitly allowlist them and we are working on a curated set of "known good" bots to allow for a compromise between discoverability and uptime.

In most cases, you should not need this and can probably get by using Cloudflare to protect a given origin. However, for circumstances where you can't or won't use Cloudflare, Anubis is there for you.

If you want to try this out, visit the Anubis documentation site at anubis.techaro.lol.

Support Copy Chapter

If you run into any issues running Anubis, please open an issue. Please include all the information I would need to diagnose your issue.

For live chat, please join the Patreon and ask in the Patron discord in the channel #anubis.

Star History Copy Chapter

Packaging Status Copy Chapter

Contributors Copy Chapter

Made with contrib.rocks.

File: docs/docs/admin/configuration/custom-status-codes.mdx Copy Chapter

Custom status codes for Anubis errors

Out of the box, Anubis will reply with HTTP 200 for challenge and denial pages. This is intended to make AI scrapers have a hard time with your website because when they are faced with a non-200 response, they will hammer the page over and over until they get a 200 response. This behavior may not be desirable, as such Anubis lets you customize what HTTP status codes are returned when Anubis throws challenge and denial pages.

This is configured in the status_codes block of your bot policy file:

status_codes:
  CHALLENGE: 200
  DENY: 200

To match CloudFlare's behavior, use a configuration like this:

status_codes:
  CHALLENGE: 403
  DENY: 403

File: docs/docs/admin/configuration/expressions.mdx Copy Chapter

Expression-based rule matching

Most of the Anubis matchers let you match individual parts of a request and only those parts in isolation. In order to defend a service in depth, you often need the ability to match against multiple aspects of a request. Anubis implements Common Expression Language (CEL) to let administrators define these more advanced rules. This allows you to tailor your approach for the individual services you are protecting.

As an example, here is a rule that lets you allow JSON API requests through Anubis:

- name: allow-api-requests
  action: ALLOW
  expression:
    all:
      - '"Accept" in headers'
      - 'headers["Accept"] == "application/json"'
      - 'path.startsWith("/api/")'

This is an advanced feature and as such it is easy to get yourself in trouble with it. Use this with care.

Common Expression Language (CEL) Copy Chapter

CEL is an expression language made by Google as a part of their access control lists system. As programs grow more complicated and users have the need to express more complicated security requirements, they often want the ability to just run a small bit of code to check things for themselves. CEL expressions are built for this. They are implicitly sandboxed so that they cannot affect the system they are running in and also designed to evaluate as fast as humanly possible.

Imagine a CEL expression as the contents of an if statement in JavaScript or the WHERE clause in SQL. Consider this example expression:

userAgent == ""

This is roughly equivalent to the following in JavaScript:

if (userAgent == "") {
  // Do something
}

Using these expressions, you can define more elaborate rules as facts and circumstances demand. For more information about the syntax and grammar of CEL, take a look at the language specification.

How Anubis uses CEL Copy Chapter

Anubis uses CEL to let administrators create complicated filter rules. Anubis has several modes of using CEL:

• Validating requests against single expressions
• Validating multiple expressions and ensuring at least one of them are true (any)
• Validating multiple expressions and ensuring all of them are true (all)

The common pattern is that every Anubis expression returns true, false, or raises an error.

Single expressions Copy Section

A single expression that returns either true or false. If the expression returns true, then the action specified in the rule will be taken. If it returns false, Anubis will move on to the next rule.

For example, consider this rule:

- name: no-user-agent-string
  action: DENY
  expression: userAgent == ""

For this rule, if a request comes in without a User-Agent string set, Anubis will deny the request and return an error page.

`any` blocks Copy Section

An any block that contains a list of expressions. If any expression in the list returns true, then the action specified in the rule will be taken. If all expressions in that list return false, Anubis will move on to the next rule.

For example, consider this rule:

- name: known-banned-user
  action: DENY
  expression:
    any:
      - remoteAddress == "8.8.8.8"
      - remoteAddress == "1.1.1.1"

For this rule, if a request comes in from 8.8.8.8 or 1.1.1.1, Anubis will deny the request and return an error page.

`all` blocks Copy Section

An all block that contains a list of expressions. If all expressions in the list return true, then the action specified in the rule will be taken. If any of the expressions in the list returns false, Anubis will move on to the next rule.

For example, consider this rule:

- name: go-get
  action: ALLOW
  expression:
    all:
      - userAgent.startsWith("Go-http-client/")
      - '"go-get" in query'
      - query["go-get"] == "1"

For this rule, if a request comes in matching the signature of the go get command, Anubis will allow it through to the target.

Variables exposed to Anubis expressions Copy Chapter

Anubis exposes the following variables to expressions:

Of note: in many languages when you look up a key in a map and there is nothing there, the language will return some "falsy" value like undefined in JavaScript, None in Python, or the zero value of the type in Go. In CEL, if you try to look up a value that does not exist, execution of the expression will fail and Anubis will return an error.

In order to avoid this, make sure the header or query parameter you are testing is present in the request with an all block like this:

- name: challenge-wiki-history-page
  action: CHALLENGE
  all:
    - 'path == "/index.php"'
    - '"title" in query'
    - '"action" in query'
    - 'query["action"] == "history"'

This rule throws a challenge if and only if all of the following conditions are true:

• The URL path is /index.php
• The URL query string contains a title value
• The URL query string contains an action value
• The URL query string's action value is "history"

So given an HTTP request like this:

GET /index.php?title=Index&action=history HTTP/1.1
User-Agent: Mozilla/5.0 Gecko/20100101 Firefox/137.0
Host: wiki.int.techaro.lol
X-Real-Ip: 8.8.8.8

Anubis would return a challenge because all of those conditions are true.

Using the system load average Copy Section

In Unix-like systems (such as Linux), every process on the system has to wait its turn to be able to run. This means that as more processes on the system are running, they need to wait longer to be able to execute. The load average represents the number of processes that want to be able to run but can't run yet. This metric isn't the most reliable to identify a cause, but is great at helping to identify symptoms.

Anubis lets you use the system load average as an input to expressions so that you can make dynamic rules like "when the system is under a low amount of load, dial back the protection, but when it's under a lot of load, crank it up to the mix". This lets you get all of the blocking features of Anubis in the background but only really expose Anubis to users when the system is actively being attacked.

This is best combined with the weight and threshold systems so that you can have Anubis dynamically respond to attacks. Consider these rules in the default configuration file:

## System load based checks.
# If the system is under high load for the last minute, add weight.
- name: high-load-average
  action: WEIGH
  expression: load_1m >= 10.0 # make sure to end the load comparison in a .0
  weight:
    adjust: 20

# If it is not for the last 15 minutes, remove weight.
- name: low-load-average
  action: WEIGH
  expression: load_15m <= 4.0 # make sure to end the load comparison in a .0
  weight:
    adjust: -10

This combination of rules makes Anubis dynamically react to the system load and only kick in when the system is under attack.

Something to keep in mind about system load average is that it is not aware of the number of cores the system has. If you have a 16 core system that has 16 processes running but none of them is hogging the CPU, then you will get a load average below 16. If you are in doubt, make your "high load" metric at least two times the number of CPU cores and your "low load" metric at least half of the number of CPU cores. For example:

Also keep in mind that this does not account for other kinds of latency like I/O latency. A system can have its web applications unresponsive due to high latency from a MySQL server but still have that web application server report a load near or at zero.

Functions exposed to Anubis expressions Copy Chapter

Anubis expressions can be augmented with the following functions:

`missingHeader` Copy Section

Available in bot expressions.

function missingHeader(headers: Record<string, string>, key: string) bool

missingHeader returns true if the request does not contain a header. This is useful when you are trying to assert behavior such as:

# Adds weight to old versions of Chrome
- name: old-chrome
  action: WEIGH
  weight:
    adjust: 10
  expression:
    all:
      - userAgent.matches("Chrome/[1-9][0-9]?\\.0\\.0\\.0")
      - missingHeader(headers, "Sec-Ch-Ua")

`randInt` Copy Section

Available in all expressions.

function randInt(n: int): int;

randInt returns a randomly selected integer value in the range of [0,n). This is a thin wrapper around Go's math/rand#Intn. Be careful with this as it may cause inconsistent behavior for genuine users.

This is best applied when doing explicit block rules, eg:

# Denies LightPanda about 75% of the time on average
- name: deny-lightpanda-sometimes
  action: DENY
  expression:
    all:
      - userAgent.matches("LightPanda")
      - randInt(16) >= 4

It seems counter-intuitive to allow known bad clients through sometimes, but this allows you to confuse attackers by making Anubis' behavior random. Adjust the thresholds and numbers as facts and circumstances demand.

`regexSafe` Copy Section

Available in bot expressions.

function regexSafe(input: string): string;

regexSafe takes a string and escapes it for safe use inside of a regular expression. This is useful when you are creating regular expressions from headers or variables such as remoteAddress.

`segments` Copy Section

Available in bot expressions.

function segments(path: string): string[];

segments returns the number of slash-separated path segments, ignoring the leading slash. Here is what it will return with some common paths:

:::note

If the path ends with a /, then the last element of the result will be an empty string. This is because /users/xe and /users/xe/ are semantically different paths.

:::

This is useful if you want to write rules that allow requests that have no query parameters only if they have less than two path segments:

- name: two-path-segments-no-query
  action: ALLOW
  expression:
    all:
      - size(query) == 0
      - size(segments(path)) < 2

DNS Functions Copy Section

Anubis can also perform DNS lookups as a part of its expression evaluation. This can be useful for doing things like checking for a valid Forward-confirmed reverse DNS (FCrDNS) record.

`arpaReverseIP`

Available in bot expressions.

function arpaReverseIP(ip: string): string;

arpaReverseIP takes an IP address and returns its value in ARPA notation. This can be useful when matching PTR record patterns.

`lookupHost`

Available in bot expressions.

function lookupHost(host: string): string[];

lookupHost performs a DNS lookup for the given hostname and returns a list of IP addresses.

- name: cloudflare-ip-in-host-header
  action: DENY
  expression: '"104.16.0.0" in lookupHost(headers["Host"])'

`reverseDNS`

Available in bot expressions.

function reverseDNS(ip: string): string[];

reverseDNS takes an IP address and returns the DNS names associated with it. This is useful when you want to check PTR records of an IP address.

- name: allow-googlebot
  action: ALLOW
  expression: 'reverseDNS(remoteAddress).endsWith(".googlebot.com")'

::: warning

Do not use this for validating the legitimacy of an IP address. It is possible for DNS records to be out of date or otherwise manipulated. Use verifyFCrDNS instead for a more reliable result.

:::

`verifyFCrDNS`

Available in bot expressions.

function verifyFCrDNS(ip: string): bool;
function verifyFCrDNS(ip: string, pattern: string): bool;

verifyFCrDNS checks if the reverse DNS of an IP address matches its forward DNS. This is a common technique to filter out spam and bot traffic. verifyFCrDNS comes in two forms:

• verifyFCrDNS(remoteAddress) will check that the reverse DNS of the remote address resolves back to the remote address. If no PTR records, returns true.
• verifyFCrDNS(remoteAddress, pattern) will check that the reverse DNS of the remote address is matching with pattern and that name resolves back to the remote address.

This is best used in rules like this:

- name: require-fcrdns-for-post
  action: DENY
  expression:
    all:
      - method == "POST"
      - "!verifyFCrDNS(remoteAddress)"

Here is an another example that allows requests from telegram:

- name: telegrambot
  action: ALLOW
  expression:
    all:
      - userAgent.matches("TelegramBot")
      - verifyFCrDNS(remoteAddress, "ptr\\.telegram\\.org$")

Life advice Copy Chapter

Expressions are very powerful. This is a benefit and a burden. If you are not careful with your expression targeting, you will be liable to get yourself into trouble. If you are at all in doubt, throw a CHALLENGE over a DENY. Legitimate users can easily work around a CHALLENGE result with a proof of work challenge. Bots are less likely to be able to do this.

File: docs/docs/admin/configuration/import.mdx Copy Chapter

Importing configuration rules

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Anubis has the ability to let you import snippets of configuration into the main configuration file. This allows you to break up your config into smaller parts that get logically assembled into one big file.

EG:

bots:
  # Pathological bots to deny
  - # This correlates to data/bots/ai-catchall.yaml in the source tree
    import: (data)/bots/ai-catchall.yaml
  - import: (data)/bots/cloudflare-workers.yaml
  # Import all the rules in the default configuration
  - import: (data)/meta/default-config.yaml

:::note

The (data) pseudo-folder corresponds to a copy of the data as of the time Anubis was compiled. If you want to dump a copy of the (data) folder out of your Anubis binary, run this command:

anubis --extract-resources ./var/anubis-resources

This will extract the contents of the (data) folder compiled into your copy of Anubis as well as the contents of static filesystem used for web assets. For example:

$ tree -L 2 ./var/anubis-resources/
./var/anubis-resources/
├── data
│   ├── apps
│   ├── botPolicies.yaml
│   ├── bots
│   ├── clients
│   ├── common
│   ├── crawlers
│   ├── meta
│   └── services
└── static
    ├── img
    ├── js
    ├── locales
    ├── robots.txt
    └── testdata

The contents of the static folder are not subject to any backwards compatibility promises and may change at any time without warning.

Replace assets in the static folder using reverse proxy configuration at your peril, as doing so may break challenge verification logic.

:::

Of note, a bot rule can either have inline bot configuration or import a bot config snippet. You cannot do both in a single bot rule.

bots:
  - import: (data)/bots/ai-catchall.yaml
    name: generic-browser
    user_agent_regex: >
      Mozilla|Opera
    action: CHALLENGE

This will return an error like this:

config is not valid:
config.BotOrImport: rule definition is invalid, you must set either bot rules or an import statement, not both

Paths can either be prefixed with (data) to import from the the data folder in the Anubis source tree or anywhere on the filesystem. If you don't have access to the Anubis source tree, check /usr/share/docs/anubis/data or in the tarball you extracted Anubis from.

Importing many files at once Copy Chapter

If you want to import policies from many files at once, the import path can contain wildcards. Wildcard matches let you add rules by dropping a file into a folder instead of editing your main configuration file every time.

For example:

bots:
  # Every rule file in the (data)/crawlers folder.
  - import: (data)/crawlers/*.yaml
  # Every rule file you put in /etc/anubis/known-good.
  - import: /etc/anubis/known-good/*.yaml

Supported wildcards Copy Section

Anubis uses the Go standard library function filepath.Match for matching. It supports four wildcards:

Use this Go playground link to test pattern matches.

Matched files load in natural sort order Copy Section

Anubis matches requests top to bottom, similar to Apache's DENY, ALLOW ACL. Anubis uses the first rule that matches the request exactly. A rule that loads early can shadow a rule that loads later.

When Anubis loads files imported with a wildcard match, it sorts the files in natural sort order. To control the order rules are loaded, put a number at the beginning of each filename:

/etc/anubis/known-good/
├── 00-allow-monitoring.yaml
├── 01-allow-git-clients.yaml
├── 02-allow-techaro-ci-bot.yaml
└── 10-weigh-everything-else.yaml

Natural sort compares numbers numerically and words alphabetically. This makes 02-deny.yaml load before 10-deny.yaml.

:::tip

Make sure your indentation is consistent across all files matched in a single import statement. Format your YAML files with yq or configure your editor to format YAML files when you save.

:::

Errors from wildcard imports Copy Section

A wildcard import that fails stops Anubis at startup. These are the common causes:

Patterns that match no files result in errors. The message names the pattern that failed:

glob pattern matched no files: "(data)/bots/*.yml", note that recursive wildcards ("**") and alternation ("{yaml,yml}") are not supported

Importing the default configuration Copy Chapter

If you want to base your configuration off of the default configuration, import (data)/meta/default-config.yaml:

bots:
  - import: (data)/meta/default-config.yaml
  # Write your rules here

This will keep your configuration up to date as Anubis adapts to emerging threats.

How do I exempt most modern browsers from Anubis challenges? Copy Chapter

If you want to exempt most modern browsers from Anubis challenges, import (data)/common/acts-like-browser.yaml:

bots:
  - import: (data)/meta/default-config.yaml
  - import: (data)/common/acts-like-browser.yaml
  # Write your rules here

These rules will allow traffic that "looks like" it's from a modern copy of Edge, Safari, Chrome, or Firefox. These rules used to be enabled by default, however user reports have suggested that AI scraper bots have adapted to conform to these rules to scrape without regard for the infrastructure they are attacking.

Use these rules at your own risk.

Importing from imports Copy Chapter

You can also import from an imported file in case you want to import an entire folder of rules at once.

bots:
  - import: (data)/bots/_deny-pathological.yaml

This lets you import an entire ruleset at once:

# (data)/bots/_deny-pathological.yaml
- import: (data)/bots/cloudflare-workers.yaml
- import: (data)/bots/headless-browsers.yaml
- import: (data)/bots/us-ai-scraper.yaml

Use this with care, you can easily get yourself into a state where Anubis recursively imports things for eternity if you are not careful.

If you need to import many files, please consider using glob matching instead.

Writing snippets Copy Chapter

Snippets can be written in either JSON or YAML, with a preference for YAML. When writing a snippet, write the bot rules you want directly at the top level of the file in a list.

Here is an example snippet that allows IPv6 Unique Local Addresses through Anubis:

- name: ipv6-ula
  action: ALLOW
  remote_addresses:
    - fc00::/7

File: docs/docs/admin/configuration/open-graph.mdx Copy Chapter

id: open-graph title: Open Graph Configuration Copy Chapter

Open Graph Configuration

This page provides detailed information on how to configure Open Graph tag passthrough in Anubis. This enables social previews of resources protected by Anubis without having to exempt each scraper individually.

Configuration Options Copy Chapter

Open Graph settings are configured in the openGraph section of the Policy File.

openGraph:
  # Enables Open Graph passthrough
  enabled: true
  # Enables the use of the HTTP host in the cache key, this enables
  # caching metadata for multiple http hosts at once.
  considerHost: true
  # How long cached OpenGraph metadata should last in memory
  ttl: 24h
  # If set, return these opengraph values instead of looking them up with
  # the target service.
  #
  # Correlates to properties in https://ogp.me/
  override:
    # og:title is required, it is the title of the website
    "og:title": "Techaro Anubis"
    "og:description": >-
      Anubis is a Web AI Firewall Utility that helps you fight the bots
      away so that you can maintain uptime at work!
    "description": >-
      Anubis is a Web AI Firewall Utility that helps you fight the bots
      away so that you can maintain uptime at work!

Configuration flags / envvars (old)

Open Graph passthrough used to be configured with configuration flags / environment variables. Reference to these settings are maintained for backwards compatibility's sake.

Usage Copy Chapter

To configure Open Graph tags, you can set the following environment variables, environment file or as flags in your Anubis configuration:

export OG_PASSTHROUGH=true
export OG_EXPIRY_TIME=1h
export OG_CACHE_CONSIDER_HOST=false

Implementation Details Copy Chapter

When OG_PASSTHROUGH is enabled, Anubis will:

1. Check a local cache for the requested URL's Open Graph tags.
2. If a cached entry exists and is still valid, return the cached tags.
3. If the cached entry is stale or not found, fetch the URL, parse the Open Graph tags, update the cache, and return the new tags.

The cache expiration time is controlled by OG_EXPIRY_TIME.

When OG_CACHE_CONSIDER_HOST is enabled, Anubis will include the host in the cache key for Open Graph tags. This ensures that tags are cached separately for different hosts.

Example Copy Chapter

Here is an example of how to configure Open Graph tags in your Anubis setup:

export OG_PASSTHROUGH=true
export OG_EXPIRY_TIME=1h
export OG_CACHE_CONSIDER_HOST=false

With these settings, Anubis will cache Open Graph tags for 1 hour and pass them through to the challenge page, not considering the host in the cache key.

When to Enable `OG_CACHE_CONSIDER_HOST` Copy Chapter

In most cases, you would want to keep OG_CACHE_CONSIDER_HOST set to false to avoid unnecessary cache fragmentation. However, there are some scenarios where enabling this option can be beneficial:

1. Multi-Tenant Applications: If you are running a multi-tenant application where different tenants are hosted on different subdomains, enabling OG_CACHE_CONSIDER_HOST ensures that the Open Graph tags are cached separately for each tenant. This prevents one tenant's Open Graph tags from being served to another tenant's users.

2. Different Content for Different Hosts: If your application serves different content based on the host, enabling OG_CACHE_CONSIDER_HOST ensures that the correct Open Graph tags are cached and served for each host. This is useful for applications that have different branding or content for different domains or subdomains.

3. Security and Privacy Concerns: In some cases, you may want to ensure that Open Graph tags are not shared between different hosts for security or privacy reasons. Enabling OG_CACHE_CONSIDER_HOST ensures that the tags are cached separately for each host, preventing any potential leakage of information between hosts.

For more information, refer to the installation guide.

File: docs/docs/admin/configuration/redirect-domains.mdx Copy Chapter

title: Redirect Domain Configuration Copy Chapter

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Anubis has an HTTP redirect in the middle of its check validation logic. This redirect allows Anubis to set a cookie on validated requests so that users don't need to pass challenges on every page load.

This flow looks something like this:

sequenceDiagram
  participant User
  participant Challenge
  participant Validation
  participant Backend

  User->>+Challenge: GET /
  Challenge->>+User: Solve this challenge
  User->>+Validation: Here's the solution, send me to /
  Validation->>+User: Here's a cookie, go to /
  User->>+Backend: GET /

However, in some cases a sufficiently dedicated attacker could trick a user into clicking on a validation link with a solution pre-filled out. For example:

sequenceDiagram
  participant Hacker
  participant User
  participant Validation
  participant Evil Site

  Hacker->>+User: Click on example.org with this solution
  User->>+Validation: Here's a solution, send me to evilsite.com
  Validation->>+User: Here's a cookie, go to evilsite.com
  User->>+Evil Site: GET evilsite.com

If this happens, Anubis will throw an error like this:

Redirect domain not allowed

Configuring allowed redirect domains Copy Chapter

By default, Anubis may redirect to any domain which could cause security issues in the unlikely case that an attacker passes a challenge for your browser and then tricks you into clicking a link to your domain.
One can restrict the domains that Anubis can redirect to when passing a challenge by setting up REDIRECT_DOMAINS environment variable.
If you need to set more than one domain, fill the environment variable with a comma-separated list of domain names.
There is also glob matching support. You can pass *.bugs.techaro.lol to allow redirecting to anything ending with .bugs.techaro.lol. There is a limit of 4 wildcards.

:::note

If you are hosting Anubis on a non-standard port (https://example:com:8443, http://www.example.net:8080, etc.), you must also include the port number here.

:::

# anubis.env

REDIRECT_DOMAINS="example.org,secretplans.example.org,*.test.example.org"
# ...

services:
  anubis-nginx:
    image: ghcr.io/techarohq/anubis:latest
    environment:
      REDIRECT_DOMAINS: "example.org,secretplans.example.org,*.test.example.org"
      # ...

- name: anubis
  image: ghcr.io/techarohq/anubis:latest
  env:
    - name: REDIRECT_DOMAINS
      value: "example.org,secretplans.example.org,*.test.example.org"
    # ...

File: docs/docs/admin/configuration/subrequest-auth.mdx Copy Chapter

title: Subrequest Authentication Copy Chapter

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Anubis can act in one of two modes:

1. Reverse proxy (the default): Anubis sits in the middle of all traffic and then will reverse proxy it to its destination. This is the moral equivalent of a middleware in your favorite web framework.
2. Subrequest authentication mode: Anubis listens for requests and if they don't pass muster then they are forwarded to Anubis for challenge processing. This is the equivalent of Anubis being a sidecar service.

:::note

Subrequest authentication requires changing the default policy because nginx interprets the default DENY status code 200 as successful authentication and allows the request.

status_codes:
  CHALLENGE: 200
  DENY: 403

See policy definitions.

:::

Nginx Copy Chapter

Anubis can perform subrequest authentication with the auth_request module in Nginx. In order to set this up, keep the following things in mind:

The TARGET environment variable in Anubis must be set to a space, eg:

# anubis.env

TARGET=" "
# ...

services:
  anubis-nginx:
    image: ghcr.io/techarohq/anubis:latest
    environment:
      TARGET: " "
      # ...

- name: anubis
  image: ghcr.io/techarohq/anubis:latest
  env:
    - name: TARGET
      value: " "
    # ...

In order to configure this, you need to add the following location blocks to each server pointing to the service you want to protect:

location /.within.website/ {
    # Assumption: Anubis is running in the same network namespace as
    # nginx on localhost TCP port 8923
    proxy_pass http://127.0.0.1:8923;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_pass_request_body off;
    proxy_set_header content-length "";
    auth_request off;
}

location @redirectToAnubis {
    return 307 /.within.website/?redir=$scheme://$host$request_uri;
    auth_request off;
}

This sets up /.within.website to point to Anubis. Any requests that Anubis rejects or throws a challenge to will be sent here. This also sets up a named location @redirectToAnubis that will redirect any requests to Anubis for advanced processing.

Finally, add this to your root location block:

location / {
    # diff-add
    auth_request /.within.website/x/cmd/anubis/api/check;
    # diff-add
    error_page 401 = @redirectToAnubis;
}

This will check all requests that don't match other locations with Anubis to ensure the client is genuine.

This will make every request get checked by Anubis before it hits your backend. If you have other locations that don't need Anubis to do validation, add the auth_request off directive to their blocks:

location /secret {
    # diff-add
    auth_request off;

    # ...
}

Here is a complete example of an Nginx server listening over TLS and pointing to Anubis:

Complete example

# /etc/nginx/conf.d/nginx.local.cetacean.club.conf

server {
  listen 443 ssl;
  listen [::]:443 ssl;
  server_name         nginx.local.cetacean.club;
  ssl_certificate     /etc/techaro/pki/nginx.local.cetacean.club/tls.crt;
  ssl_certificate_key /etc/techaro/pki/nginx.local.cetacean.club/tls.key;
  ssl_protocols       TLSv1.2 TLSv1.3;
  ssl_ciphers         HIGH:!aNULL:!MD5;

  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

  location /.within.website/ {
    proxy_pass http://localhost:8923;
    auth_request off;
  }

  location @redirectToAnubis {
    return 307 /.within.website/?redir=$scheme://$host$request_uri;
    auth_request off;
  }

  location / {
    auth_request /.within.website/x/cmd/anubis/api/check;
    error_page 401 = @redirectToAnubis;
    root /usr/share/nginx/html;
    index index.html index.htm;
  }
}

Caddy Copy Chapter

Anubis can be used with the forward_auth directive in Caddy.

First, the TARGET environment variable in Anubis must be set to a space, eg:

# anubis.env

TARGET=" "
# ...

services:
  anubis-caddy:
    image: ghcr.io/techarohq/anubis:latest
    environment:
      TARGET: " "
      # ...

- name: anubis
  image: ghcr.io/techarohq/anubis:latest
  env:
    - name: TARGET
      value: " "
    # ...

Then configure the necessary directives in your site block:

route {
    # Assumption: Anubis is running in the same network namespace as
    # caddy on localhost TCP port 8923
    reverse_proxy /.within.website/* 127.0.0.1:8923
    forward_auth 127.0.0.1:8923 {
        uri /.within.website/x/cmd/anubis/api/check
        trusted_proxies private_ranges
        @unauthorized status 401
        handle_response @unauthorized {
            redir * /.within.website/?redir={uri} 307
        }
    }
}

If you want to use this for multiple sites, you can create a snippet and import it in multiple site blocks.

File: docs/docs/admin/environments/apache.mdx Copy Chapter

Apache

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Anubis is intended to be a filter proxy. The way to integrate this is to break your configuration up into two parts: TLS termination and then HTTP routing. Consider this diagram:

---
title: Apache as tls terminator and HTTP router
---

flowchart LR
    T(User Traffic)
    subgraph Apache 2
        TCP(TCP 80/443)
        US(TCP 3001)
    end

    An(Anubis)
    B(Backend)

    T --> |TLS termination| TCP
    TCP --> |Traffic filtering| An
    An --> |Happy traffic| US
    US --> |whatever you're doing| B

Effectively you have one trip through Apache to do TLS termination, a detour through Anubis for traffic scrubbing, and then going to the backend directly. This final socket is what will do HTTP routing.

:::note

These examples assume that you are using a setup where your Apache configuration is made up of a bunch of files in /etc/httpd/conf.d/*.conf. This is not true for all deployments of Apache. If you are not in such an environment, append these snippets to your /etc/httpd/conf/httpd.conf file.

:::

Configuration Copy Chapter

Assuming you are protecting anubistest.techaro.lol, you need the following server configuration blocks:

1. A block on port 80 that forwards HTTP to HTTPS
2. A block on port 443 that terminates TLS and forwards to Anubis
3. A block on port 3001 that actually serves your websites

/* Detailed source-code truncated for AI context efficiency. */

Make sure to add a separate configuration file for the listener on port 3001:

# /etc/httpd/conf.d/listener-3001.conf

Listen [::1]:3001

In case you are running an IPv4-only system, use the following configuration instead:

# /etc/httpd/conf.d/listener-3001.conf

Listen 127.0.0.1:3001

This can be repeated for multiple sites. Anubis does not care about the HTTP Host header and will happily cope with multiple websites via the same instance.

Then reload your Apache config and load your website. You should see Anubis protecting your apps!

sudo systemctl reload httpd.service

Troubleshooting Copy Chapter

Here are some answers to questions that came in in testing:

I'm running on a Red Hat distribution and Apache is saying "service unavailable" for every page load Copy Section

If you see a "Service unavailable" error on every page load and run a Red Hat derived distribution, you are missing a selinux setting. The exact command will be in a journalctl log message like this:

*****  Plugin catchall_boolean (89.3 confidence) suggests   ******************

If you want to allow HTTPD scripts and modules to connect to the network using TCP.
Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.

Do
setsebool -P httpd_can_network_connect 1

This will fix the error immediately.

File: docs/docs/admin/environments/caddy.mdx Copy Chapter

Caddy

To use Anubis with Caddy, stick Anubis between Caddy and your backend. For example, consider this application setup:

---
title: Caddy with Anubis in the middle
---

flowchart LR
    T(User Traffic)
    TCP(TCP 80/443)
    An(Anubis)
    B(Backend)
    Blocked

    T --> TCP
    TCP --> |Traffic filtering| An
    An --> |Happy traffic| B
    An --> |Malicious traffic| Blocked

Instead of your traffic going directly to your backend, it takes a detour through Anubis. Anubis filters out the "bad" traffic and passes the "good" traffic to the backend.

To set up Anubis with Docker compose and Caddy, start with a docker-compose configuration like this:

services:
  caddy:
    image: caddy:2
    ports:
      - 80:80
      - 443:443
      - 443:443/udp
    volumes:
      - ./conf:/etc/caddy
      - caddy_config:/config
      - caddy_data:/data

  anubis:
    image: ghcr.io/techarohq/anubis:latest
    pull_policy: always
    environment:
      BIND: ":3000"
      TARGET: http://httpdebug:3000

  httpdebug:
    image: ghcr.io/xe/x/httpdebug
    pull_policy: always

volumes:
  caddy_data:
  caddy_config:

And then put the following in conf/Caddyfile:

# conf/Caddyfile

yourdomain.example.com {
  tls [email protected]

  reverse_proxy http://anubis:3000 {
    header_up X-Real-Ip {remote_host}
    header_up X-Http-Version {http.request.proto}
  }
}

If you want to protect multiple services with Anubis, you will need to either start multiple instances of Anubis (Anubis requires less than 32 MB of ram on average) or set up a two-tier routing setup where TLS termination is done with one instance of Caddy and the actual routing to services is done with another instance of Caddy. See the nginx or Apache documentation to get ideas on how you would do this.

File: docs/docs/admin/environments/docker-compose.mdx Copy Chapter

Docker compose

Docker compose is typically used in concert with other load balancers such as Apache or Nginx. Below is a minimal example showing you how to set up an instance of Anubis listening on host port 8080 that points to a static website containing data in ./www:

services:
  anubis:
    image: ghcr.io/techarohq/anubis:latest
    environment:
      BIND: ":8080"
      DIFFICULTY: "4"
      METRICS_BIND: ":9090"
      SERVE_ROBOTS_TXT: "true"
      TARGET: "http://nginx"
      POLICY_FNAME: "/data/cfg/botPolicy.yaml"
      OG_PASSTHROUGH: "true"
      OG_EXPIRY_TIME: "24h"
    healthcheck:
      test: ["CMD", "anubis", "--healthcheck"]
      interval: 5s
      timeout: 30s
      retries: 5
      start_period: 500ms
    ports:
      - 8080:8080
    volumes:
      - "./botPolicy.yaml:/data/cfg/botPolicy.yaml:ro"

  nginx:
    image: nginx
    volumes:
      - "./www:/usr/share/nginx/html"