outline-server Documentation

File: README.md Copy Chapter

Outline Server

Outline Server is the component that provides the Shadowsocks service (via outline-ss-server) and a service management API. You can deploy this server directly following simple instructions in this repository, or if you prefer a ready-to-use graphical interface you can use the Outline Manager.

Components:

• Outline Server (src/shadowbox): The core proxy server that runs and manages outline-ss-server, a Shadowsocks backend. It provides a REST API for access key management.

• Metrics Server (src/metrics_server): A REST service for optional, anonymous metrics sharing.

Join the Outline Community by signing up for the IFF Mattermost!

Shadowsocks and Anti-Censorship Copy Chapter

Outline's use of Shadowsocks means it benefits from ongoing improvements that strengthen its resistance against detection and blocking.

Key Protections:

• AEAD ciphers are mandatory.
• Probing resistance mitigates detection techniques.
• Protection against replayed data.
• Variable packet sizes to hinder identification.

See Shadowsocks resistance against detection and blocking.

Installation Copy Chapter

Prerequisites

• Node LTS (lts/hydrogen, version 18.16.0)
• NPM (version 9.5.1)
• Go 1.21+

1. Install dependencies

   sh Copy

   npm install

2. Start the server

   sh Copy

   ./task shadowbox:start

   Exploring further options:

   • Refer to the README: Find additional configuration and usage options in the core server's README.
   • Learn about the build system: For in-depth build system information, consult the contributing guide.

3. To clean up

   sh Copy

   ./task clean

File: docs/shadowsocks.md Copy Chapter

Shadowsocks Resistance Against Detection and Blocking

Shadowsocks used to be blocked in some countries, and because Outline uses Shadowsocks, there has been skepticism about Outline working in those countries. In fact, people have tried Outline in the past and had their servers blocked.

However, since the second half of 2020 things have changed. The Outline team and Shadowsocks community made a number of improvements that strengthened Shadowsocks beyond the censor's current capabilities.

As shown in the research How China Detects and Blocks Shadowsocks, the censor uses active probing to detect Shadowsocks servers. The probing may be triggered by packet sniffing, but that's not how the servers are detected.

Even though Shadowsocks is a standard, it leaves a lot of room for choices on how it's implemented and deployed.

First of all, you must use AEAD ciphers. The old stream ciphers are easy to break and manipulate, exposing you to simple detection and decryption attacks. Outline has banned all stream ciphers, since people copy old examples to set up their servers. The Outline Manager goes further and picks the cipher for you, since users don't usually know how to choose a cipher, and it generates a long random secret, so you are not vulnerable to dictionary-based attacks.

Second, you need probing resistance. Both shadowsocks-libev and Outline have added that. The research Detecting Probe-resistant Proxies showed that, in the past, an invalid byte would trigger different behaviors whether it was inserted in positions 49, 50 or 51 of the stream, which is very telling. That behavior is now gone, and the censor can no longer rely on that.

Third, you need protection against replayed data. Both shadowsocks-libev and Outline have added such protection, which you may need to enable explicitly on ss-libev, but it's the default on Outline.

Fourth, Outline and clients using shadowsocks-libev now merge the SOCKS address and the initial data in the same initial encrypted frame, making the size of the first packet variable. Before the first packet only had the SOCKS address, with a fixed size, and that was a giveaway.

The censors used to block Shadowsocks, but Shadowsocks has evolved, and in 2021, it was ahead again in the cat and mouse game.

In 2022 China started blocking seemingly random traffic (report). While there is no evidence they could detect Shadowsocks, the protocol ended up blocked.

As a reponse, we added a feature to the Outline Client that allows service managers to specify a connection prefix disguise to be used in the Shadowsocks initialization, which can be used to bypass the blocking in China by making it look like a protocol that is allowed.

Shadowsocks remains our protocol of choice because it's simple, well understood and very performant. Furthermore, it has an enthusiastic community of very smart people behind it.

File: src/shadowbox/integration_test/README.md Copy Chapter

Outline Server Image Integration Test

This folder contains the integration test for the Outline Server image.

To build and test the image:

task shadowbox:integration_test

For development of the test, or to test a specific image, you may prefer calling the test directly, without the build step:

./task shadowbox:test_image IMAGE_NAME=quay.io/outline/shadowbox:stable

If you prefer to use Podman instead of Docker, set the DOCKER=podman environment variable:

DOCKER=podman task shadowbox:integration_test

File: src/shadowbox/README.md Copy Chapter

Outline Server (Shadowbox)

The Outline Server, internal name "Shadowbox," is designed to streamline the setup and sharing of Shadowsocks servers. It includes a user management API and creates Shadowsocks instances when needed. It's managed by the Outline Manager and used as proxy by the Outline Client apps. Shadowbox is also compatible with standard Shadowsocks clients.

Installation Copy Chapter

Self-Hosted Installation Copy Section

1. Run the Installation Script

   sh Copy

   sudo bash -c "$(wget -qO- https://raw.githubusercontent.com/OutlineFoundation/outline-apps/master/server_manager/install_scripts/install_server.sh)"

2. Customize (Optional)

   Add flags for hostname, port, etc. Example:

   sh Copy

   sudo bash -c "$(wget -qO- https://raw.githubusercontent.com/OutlineFoundation/outline-apps/master/server_manager/install_scripts/install_server.sh)" install_server.sh \
     --hostname=myserver.com \
     --keys-port=443

   • Use sudo --preserve-env for environment variables.
   • Use bash -x for debugging.

Running from Source Code Copy Section

Prerequisites

• Docker
• Node LTS (lts/hydrogen, version 18.16.0)
• NPM (version 9.5.1)

[!TIP]
If you use nvm, switch to the correct Node version with nvm use.

1. Build and Run:

   Shadowbox supports running on linux and macOS hosts.

   • Node.js App

     sh Copy

     task shadowbox:start

   • Docker Container

     sh Copy

     task shadowbox:docker:start

     [!TIP]
     Some useful commands when working with Docker images and containers:

     • Debug Image:

       sh Copy

       docker run --rm -it --entrypoint=sh localhost/outline/shadowbox

     • Debug Running Container:

       sh Copy

       docker exec -it shadowbox sh

     • Cleanup Dangling Images:

       sh Copy

       docker rmi $(docker images -f dangling=true -q)

2. Send a Test Request

   sh Copy

   curl --insecure https://[::]:8081/TestApiPrefix/server

Access Keys Management API Copy Chapter

The Outline Server provides a REST API for access key management. If you know the apiUrl of your Outline Server (e.g. https://1.2.3.4:1234/3pQ4jf6qSr5WVeMO0XOo4z), you can directly manage the server's access keys using HTTP requests:

1. Find the Server's apiUrl:

   • Deployed with the Installation Script: Run grep "apiUrl" /opt/outline/access.txt | cut -d: -f 2-

   • Deployed with the Outline Manager: Check the "Settings" tab.

   • Local Deployments from Source: The apiUrl is simply https://[::]:8081/TestApiPrefix

2. API Examples:

   Replace $API_URL with your actual apiUrl.

   • List access keys: curl --insecure $API_URL/access-keys/

   • Create an access key: curl --insecure -X POST $API_URL/access-keys

   • Get an access key (e.g. ID 1): curl --insecure $API_URL/access-keys/1

   • Rename an access key: curl --insecure -X PUT -F 'name=albion' $API_URL/access-keys/2/name

   • Remove an access key: curl --insecure -X DELETE $API_URL/access-keys/1

   • Set a data limit for all access keys: (e.g. limit outbound data transfer access keys to 1MB over 30 days) curl --insecure -X PUT -H "Content-Type: application/json" -d '{"limit": {"bytes": 1000}}' $API_URL/server/access-key-data-limit

   • Remove the access key data limit: curl --insecure -X DELETE $API_URL/server/access-key-data-limit

   • And more...

3. Further Options:

   Consult the OpenAPI spec and documentation for more options.

Testing Copy Chapter

Manual Copy Section

Build and run your image with:

task shadowbox:docker:start

Integration Test Copy Section

The integration test will not only build and run your image, but also run a number of automated tests.

task shadowbox:integration_test

This does the following:

• Sets up three containers (client, shadowbox, target) and two networks.
• Creates a user on shadowbox.
• Connects to target through shadowbox using a Shadowsocks client: client <-> shadowbox <-> target

1. Testing Changes to the Server Config:

If your change includes new fields in the server config which are needed at server start-up time, then you mey need to remove the pre-existing test config:

• Delete Existing Config: rm /tmp/outline/persisted-state/shadowbox_server_config.json

• Manually Edit: You'll need to edit the JSON string within src/shadowbox/Taskfile.yml.

File: src/server_manager/README.md Copy Chapter

Outline Manager

THIS PROJECT HAS MOVED TO A NEW LOCATION: Outline Manager is now part of the Outline Apps repository.

We are keeping this folder to support legacy versions of the app that point to the old server install script.

File: src/sentry_webhook/README.md Copy Chapter

Outline Sentry Webhook

The Outline Sentry webhook is a Google Cloud Run functions that receives a Sentry event and posts it to Salesforce.

Requirements Copy Chapter

• Google Cloud SDK
• Access to Outline's Sentry account.

Build Copy Chapter

task sentry_webhook:build

Deploy Copy Chapter

Authenticate with gcloud:

gcloud auth login

To deploy:

task sentry_webhook:deploy

Configure Sentry Webhooks Copy Chapter

• Log in to Outline's Sentry account
• Select a project (outline-client, outline-client-dev, outline-server, outline-server-dev).
  • Note that this process must be repeated for all Sentry projects.
• Enable the WebHooks plugin at https://sentry.io/settings/outlinevpn//plugins/
• Set the webhook endpoint at https://sentry.io/settings/outlinevpn//plugins/webhooks/
• Configure alerts to invoke the webhook at https://sentry.io/settings/outlinevpn//alerts/
• Create rules to trigger the webhook at https://sentry.io/settings/outlinevpn//alerts/rules/

File: src/metrics_server/README.md Copy Chapter

Outline Metrics Server

The Outline Metrics Server is a Google App Engine project that writes feature and connections metrics to BigQuery, as reported by opted-in Outline servers.

API Copy Chapter

Endpoints Copy Section

The metrics server deploys two services: dev, used for development testing and debugging; and prod, used for production metrics. The dev environment is deployed to https://dev.metrics.getoutline.org; the prod environment is deployed to https://prod.metrics.getoutline.org. Each environment posts metrics to its own BigQuery dataset (see config_[dev|prod].json).

URLs Copy Section

The metrics server supports two URL paths:

• POST /connections: report server data usage broken down by user.

  text Copy

  {
    serverId: string,
    startUtcMs: number,
    endUtcMs: number,
    userReports: [{
        countries: string[],
        asn: number,
        bytesTransferred: number,
        tunnelTimeSec: number,
    }]
  }

• POST /features: report feature usage.

  text Copy

  {
      serverId: string,
      serverVersion: string,
      timestampUtcMs: number,
      dataLimit: {
          enabled: boolean
          perKeyLimitCount: number
      }
  }

Requirements Copy Chapter

• Google Cloud SDK

Build Copy Chapter

task metrics_server:build

Run Copy Chapter

Run a local development metrics server:

task metrics_server:start

Deploy Copy Chapter

• Authenticate with gcloud:
  sh Copy

  gcloud auth login

• To deploy to dev:
  sh Copy

  task metrics_server:deploy:dev

• To deploy to prod:
  sh Copy

  task metrics_server:deploy:prod

  Once deployed, you will need to manually migrate all traffic to the new version via the Google Cloud console.

Test Copy Chapter

• Unit test
  sh Copy

  task metrics_server:test

• Integration test
  sh Copy

  task metrics_server:integration_test

File: third_party/shellcheck/README.md Copy Chapter

Outline Shellcheck Wrapper

This directory is used to lint our scripts using Shellcheck. To ensure consistency across developer systems, the included script

• Attempts to identify the developer's OS (Linux, macOS, or Windows)
• Downloads a pinned version of Shellcheck into ./download
• Checks the archive hash
• Extracts the executable
• Runs the executable

The executable is cached on the developer's system after the first download. To clear the cache, run rm download (or npm run clean in the repository root).