pds Documentation

Technical Documentation: bluesky-social/pds

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`) Copy Section

PDS

Welcome to the repository for the official Bluesky PDS (Personal Data Server). This repository includes container images and documentation designed to assist technical people with hosting a Bluesky PDS.

Head over to the ATProto Touchers Discord to chat with other folks hosting instances and get important updates about the PDS distribution!

Table of Contents Copy Chapter

• PDS
  • Table of Contents
  • FAQ
    • What is Bluesky?
    • What is AT Protocol?
    • Where is the code?
    • What is the current status of federation?
  • Self-hosting a PDS
    • Deploying a PDS onto a VPS
    • Open your cloud firewall for HTTP and HTTPS
    • Configure DNS for your domain
    • Check that DNS is working as expected
    • Installing on Ubuntu 22.04/24.04/26.04 and Debian 11/12/13
    • Verifying that your PDS is online and accessible
    • goat CLI
    • Creating an account
    • Creating an account using an invite code
    • Using the Bluesky app with your PDS
    • Setting up SMTP
      • Common SMTP issues
    • Logging
    • Monitoring and metrics
    • Updating your PDS
    • Environment Variables
    • Customizing the appearance
    • Migrating your PDS
    • Fixing a Relay desync
  • License

FAQ Copy Chapter

What is Bluesky? Copy Section

Bluesky is a social media application built on AT Protocol.

Please visit the Bluesky website for more information.

What is AT Protocol? Copy Section

The Authenticated Transfer Protocol, aka atproto, is a protocol for large-scale distributed social applications.

Please visit the AT Protocol docs for additional information.

Where is the code? Copy Section

• TypeScript code
• Go code

What is the current status of federation? Copy Section

The AT Protocol network is open to federation!

✅ Federated domain handles (e.g. @nytimes.com)

✅ Federated feed generators (custom algorithms)

✅ Federated relays (event firehose)

✅ Federated app views (API service)

✅ Federated data (PDS hosting)

✅ Federated moderation (labeling)

Self-hosting a PDS Copy Chapter

Self-hosting a Bluesky PDS means running your own Personal Data Server that is capable of federating with the wider Bluesky social network.

Deploying a PDS onto a VPS Copy Section

This README provides instructions for deploying a PDS using our install script onto a Virtual Private Server. Digital Ocean and Vultr are two popular choices for VPS hosting.

Ensure that you can ssh to your server and have root access.

Server Requirements

• Public IPv4 address
• Public DNS name
• Public inbound internet access permitted on port 80/tcp and 443/tcp

Server Recommendations

Note: It is a good security practice to restrict inbound ssh access (port 22/tcp) to your own computer's public IP address. You can check your current public IP address using ifconfig.me.

Open your cloud firewall for HTTP and HTTPS Copy Section

One of the most common sources of misconfiguration is not opening firewall ports correctly. Please be sure to double check this step.

In your cloud provider's console, the following ports should be open to inbound access from the public internet.

• 80/tcp (Used only for TLS certification verification)
• 443/tcp (Used for all application requests)

Note: there is no need to set up TLS or redirect requests from port 80 to 443 because the Caddy web server, included in the Docker compose file, will handle this for you.

Configure DNS for your domain Copy Section

From your DNS provider's control panel, set up a domain with records pointing to your server.

Note:

• Replace example.com with your domain name.
• Replace 12.34.56.78 with your server's IP address.
• Some providers may use the @ symbol to represent the root of your domain.
• The wildcard record is required when allowing users to create new accounts on your PDS.
• The TTL can be anything but 600 (10 minutes) is reasonable

Check that DNS is working as expected Copy Section

Use a service like DNS Checker to verify that you can resolve domain names.

Examples to check (record type A):

• example.com
• random.example.com
• test123.example.com

These should all return your server's public IP.

Installing on Ubuntu 22.04/24.04/26.04 and Debian 11/12/13 Copy Section

Note that this script assumes a relatively "fresh" VPS that is not also concurrently hosting a web server or anything else on port 80/443. If you intend to run a PDS alongside an existing webserver on the same VPS, you will not want to use this install script.

On your server, download the install script using curl:

curl https://raw.githubusercontent.com/bluesky-social/pds/main/installer.sh > installer.sh

And then run the installer using bash. You will need sudo permissions to continue:

sudo bash installer.sh

The install script is interactive and will prompt for input during the install process. You will need to provide your public DNS address, an admin email address (which does not need to be from the same domain), and be prompted to create a PDS user account with its own email address and handle. If you plan to reuse an existing AT handle, you can skip user account creation, though if it is your first time deploying a PDS you may want to create an account using your domain like account.your-domain.net for testing purposes.

Upon completion of a successful installation, you'll receive output similar to the following:

========================================================================
PDS installation successful!
------------------------------------------------------------------------

Check service status      : sudo systemctl status pds
Watch service logs        : sudo docker logs -f pds
Backup service data       : /pds
PDS Admin command         : pdsadmin

Required Firewall Ports
------------------------------------------------------------------------
Service                Direction  Port   Protocol  Source
-------                ---------  ----   --------  ----------------------
HTTP TLS verification  Inbound    80     TCP       Any
HTTP Control Panel     Inbound    443    TCP       Any

Required DNS entries
------------------------------------------------------------------------
Name                         Type       Value
-------                      ---------  ---------------
your-domain.net              A          your-ip-address
*.your-domain.net            A          your-ip-address

Detected public IP of this server: your-ip-address

To see pdsadmin commands, run "pdsadmin help"

========================================================================

And, following account creation:

Account created successfully!
-----------------------------
Handle   : handle.your-domain.net
DID      : did:plc:your-did
Password : your-password
-----------------------------
Save this password, it will not be displayed again.

Verifying that your PDS is online and accessible Copy Section

[!TIP]
The most common problems with getting PDS content consumed in the live network usually result from users trying to port the provided Caddy configuration to Nginx, Apache, or other reverse proxies. Getting TLS certificates, WebSockets, and virtual server names provisioned can be challenging. We are not currently providing tech support for other configurations.

After installation, your PDS should be live and accessible on the web. You can check if your server is online and healthy by making a request to https://your-domain.net/xrpc/_health (the healthcheck endpoint). You should see a JSON response with a version, like:

{"version":"0.2.2-beta.2"}

You'll also need to check that WebSockets are working, for the rest of the network to pick up content from your PDS. You can test by installing a tool like wsdump and running a command like:

wsdump "wss://example.com/xrpc/com.atproto.sync.subscribeRepos?cursor=0"

Note that there will be no events output on the WebSocket until they are created in the PDS, so the above command may continue to run with no output immediately post-installation.

goat CLI Copy Section

The PDS image includes goat, our command line tool for performing admin functions. goat can be used locally with a --pds-host parameter or within the container installed by this script.

Creating an account Copy Section

You can run the goat command included in this container with docker exec pds goat.

Use goat pds admin account create to create an account if you haven't already:

docker exec pds goat pds admin account create --admin-password `PDS_ADMIN_PASSWORD` --handle newuser.pds.net --email [email protected] --password new-password

[!NOTE]
You can find PDS_ADMIN_PASSWORD in /pds/pds.env following installation. You can typically skip this arg when running goat from the docker container as PDS_ADMIN_PASSWORD should already be set.

Creating an account using an invite code Copy Section

If needed, use goat to create an invite code:

docker exec pds goat pds admin --admin-password PDS_ADMIN_PASSWORD create-invites

When creating an account using the app, enter this invite code.

Using the Bluesky app with your PDS Copy Section

You can use the Bluesky app to connect to your PDS.

1. Get the Bluesky app
   • Bluesky for Web
   • Bluesky for iPhone
   • Bluesky for Android
2. Enter the URL of your PDS (e.g. https://example.com/)

Note: because the subdomain TLS certificate is created on-demand, it may take 10-30s for your handle to be accessible. If you aren't seeing your first post/profile, wait 30s and try to make another post.

Setting up SMTP Copy Section

To be able to verify users' email addresses and send other emails, you need to set up an SMTP server.

As an alternative to running your own SMTP server, you can use an email service. Resend and SendGrid are two popular choices.

Create an account and API key on an email service, ensure your server allows access on the required ports, and then you can add these configuration variables to /pds/pds.env on your server (example with Resend):

PDS_EMAIL_SMTP_URL=smtps://resend:<your api key here>@smtp.resend.com:465/
[email protected]

If you prefer to use a standard SMTP server (a local one or from your email provider), put your account's username and password in the URL:

PDS_EMAIL_SMTP_URL=smtps://username:[email protected]/

Alternatively, if you're running a local sendmail-compatible mail service like Postfix or Exim on the same host, you can configure the PDS to use the sendmail transport by using such URL:

PDS_EMAIL_SMTP_URL=smtp:///?sendmail=true

Note: Your PDS will need to be restarted with those variables. This varies depending on your setup. If you followed this installation guide, run systemctl restart pds. You might need to restart the server or recreate the container, depending on what you are using.

Common SMTP issues

If you find that your test messages using cURL or other sources go out correctly, but you are not receiving emails from your PDS, you may need to URL encode your username and password on /pds/pds.env and restart the PDS service.

If the username and/or password contain special characters, the special characters will need to be percent encoded. For some email services, the username will contain an extra @ symbol that will also need to be percent encoded. For example, the URL user&name@oci:p@ssword@smtphost:465 after percent encoding for the username and password fields would become user%26name%40oci:p%40ssword@smtphost:465.

If you are migrating an account, Bluesky's UI will ask you to confirm your email address. The confirmation code email is meant to come from your PDS. If you are encountering issues with SMTP and want to confirm the address before solving it, you can find the confirmation code on the email_token table on accounts.sqlite.

Logging Copy Section

By default, logs from the PDS are printed to stdout and end up in Docker's log. You can browse them by running:

[sudo] docker logs pds

Note: these logs are not persisted, so they will be lost after server reboot.

Alternatively, you can configure the logs to be printed to a file by setting LOG_DESTINATION:

LOG_DESTINATION=/pds/pds.log

You can also change the minimum level of logs to be printed (default: info):

LOG_LEVEL=debug

Monitoring and metrics Copy Section

The PDS can report metrics over OpenTelemetry: host-level things like CPU, memory and disk, alongside PDS-level things like accounts created, sign-ins, OAuth grants, and XRPC request rate and latency.

The monitoring/ directory in this repo contains a self-contained Prometheus + Grafana + node_exporter stack and a ready-made Grafana dashboard. It is entirely optional, runs separately from the main PDS stack, and is not affected by pdsadmin update.

curl -sL https://github.com/bluesky-social/pds/archive/refs/heads/main.tar.gz \
  | tar xz --strip-components=1 pds-main/monitoring
cd monitoring && docker compose up --detach

You then add some OTEL_* variables to /pds/pds.env and restart the PDS. Everything binds to 127.0.0.1, so you can reach Grafana over an SSH tunnel rather than opening ports. See monitoring/README.md for a walkthrough.

Updating your PDS Copy Section

It is recommended that you keep your PDS up to date with new versions. This repo sets up watchtower, which handles automatic updates. You can also use the pdsadmin tool to manually update your PDS.

sudo pdsadmin update

Environment Variables Copy Section

Customizing the appearance Copy Section

Your PDS serves the sign-in, sign-up, authorization, and account-management screens. You can brand them with your service's name, logo, colors, and background by adding any of these to pds.env. All are optional. Unset colors fall back to the stock palette in the table below (a #8338ec primary), the service name defaults to <hostname> PDS, and the logo, footer links, and background simply don't appear.

Name, logo & footer links

Colors. Each accepts any CSS color, e.g. #1083fe or rgb(16 131 254).

Background images

The image fills the area behind the card (cover, centered) and is chosen by the visitor's light/dark preference. The card keeps its own opaque surface, so text stays legible over any image.

Tips: Any common web image format works (WebP, AVIF, JPEG, PNG, SVG). Host it at a public HTTPS URL; like PDS_LOGO_URL, the auth pages only load images over https:. It loads on every sign-in and is scaled to fill the screen. Use a well-compressed image around 2000px wide and comfortably under 1 MB (a few hundred KB is ideal). Keep important detail away from the edges, since the image is cropped to fill.

Example

# Appearance
PDS_SERVICE_NAME="Alice's PDS"
PDS_LOGO_URL="https://example.com/logo.png"
PDS_PRIMARY_COLOR="#1083fe"
PDS_BACKGROUND_LIGHT_URL="https://example.com/bg-light.jpg"
PDS_BACKGROUND_DARK_URL="https://example.com/bg-dark.jpg"

Migrating your PDS Copy Section

Once you've deployed a PDS, it will automatically begin broadcasting events to Relay servers.

This means that if you wipe and reinstall your PDS on the same hostname, or move it to a new hostname without performing the appropriate cutover steps, the PDS and Relay can go out of sync.

To avoid this, you should always migrate accounts individually from one PDS to another when updating your PDS host configuration.

Fixing a Relay desync Copy Section

If you become desynced from the Relay due to migration issues — i.e., new records created on your PDS aren't being picked up by other applications — you can fix it with these steps:

1. Look up the last known sequence number for your PDS host. You can use goat to retrieve the last observed cursor from the bsky.network relay:

docker exec pds sh -c 'goat relay host status "$PDS_HOSTNAME" --json'

{"hostname":"justdothings.net","seq":12,"status":"active"}

2. Stop the pds service:

systemctl stop pds

3. Set the cursor sequence on your PDS to the value of seq from step 1, incremented by 1:

sqlite3 /pds/sequencer.sqlite "UPDATE sqlite_sequence SET seq = {new_sequence_number} WHERE name = 'repo_seq';"

4. Restart your pds service:

systemctl start pds

New records should now be indexed as normal.

License Copy Chapter

This project is dual-licensed under MIT and Apache 2.0 terms:

• MIT license (LICENSE-MIT.txt or http://opensource.org/licenses/MIT)
• Apache License, Version 2.0, (LICENSE-APACHE.txt or http://www.apache.org/licenses/LICENSE-2.0)

Downstream projects and end users may choose either license individually, or both together, at their discretion. The motivation for this dual-licensing is the additional software patent assurance provided by Apache 2.0.

Overview (`monitoring/README.md`) Copy Section

Monitoring a PDS

A ready-made Grafana dashboard for a self-hosted PDS, covering both host health
(CPU, memory, disk, network) and PDS activity (accounts, sessions, OAuth grants,
XRPC request rate and latency).

This is entirely optional and completely separate from the main PDS stack. It
does not modify /pds/compose.yaml and is not touched by pdsadmin update.

Check your PDS version first Copy Chapter

Telemetry support was added in @atproto/pds 0.5.26, and your PDS image has to
be new enough to include it. Check this before you change anything: if you
enable telemetry against an older image, the PDS does not start at all. It exits
with ERR_PACKAGE_PATH_NOT_EXPORTED and then crash-loops.

Supported images have a label:

docker image inspect ghcr.io/bluesky-social/pds:0.4 \
  --format '{{index .Config.Labels "social.bsky.pds.telemetry"}}'

That prints otel on a supported image.

How metrics get out of the PDS Copy Chapter

The PDS speaks OpenTelemetry. It pushes metrics
over OTLP rather than exposing a /metrics endpoint to be scraped.

Prometheus v3 can receive OTLP directly, so this stack is just three containers
with no OpenTelemetry Collector in between.

Quick start Copy Chapter

Grab this directory and start the stack:

curl -sL https://github.com/bluesky-social/pds/archive/refs/heads/main.tar.gz \
  | tar xz --strip-components=1 pds-main/monitoring
cd monitoring && docker compose up --detach

Then load the OpenTelemetry SDK and tell the PDS where to send metrics. Add to
/pds/pds.env:

NODE_OPTIONS=--import=@atproto/pds/telemetry
OTEL_SERVICE_NAME=pds
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://localhost:9090/api/v1/otlp/v1/metrics
OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/protobuf
OTEL_METRIC_EXPORT_INTERVAL=15000
OTEL_SEMCONV_STABILITY_OPT_IN=http
OTEL_NODE_RESOURCE_DETECTORS=env,host,os,process,serviceinstance,container

and restart:

sudo systemctl restart pds

Some notes on those variables:

• NODE_OPTIONS=--import=@atproto/pds/telemetry loads the telemetry SDK, and
  is required for a monitoring config. If you already set
  NODE_OPTIONS for something else, append to it rather than replacing it.
• Setting only OTEL_EXPORTER_OTLP_METRICS_ENDPOINT rather than the generic
  OTEL_EXPORTER_OTLP_ENDPOINT is intentional. The PDS enables exactly the
  OTel signals you configure, so this enables metrics and leaves traces and
  logs disabled. For traces and logs, see Going further.
• OTEL_SEMCONV_STABILITY_OPT_IN=http opts into the stable HTTP semantic
  conventions, and the dashboard is built against those names.
• OTEL_NODE_RESOURCE_DETECTORS is not strictly required, but the default
  detector set includes AWS, GCP and Azure detectors that probe cloud metadata
  services at startup. On a plain VPS those attempts fail, and the GCP one
  logs a MetadataLookupWarning every time the PDS boots. The list above keeps
  the detectors that produce something useful and drops the cloud ones.

Reaching Grafana Copy Chapter

Everything binds to 127.0.0.1 and none of it is exposed to the internet by default. Do
not open these ports on your cloud firewall. Use an SSH tunnel:

ssh -L 3001:localhost:3001 you@your-pds-host

Then open http://localhost:3001 and log in with admin / admin. Grafana
will ask you to change the password on first login.

Grafana runs on 3001 because the PDS itself owns port 3000.

The PDS Overview dashboard is provisioned automatically, in a folder named
PDS. It is read-only; to customize it, use "Save as" to make your own copy, or
edit dashboards/pds-overview.json and restart Grafana.

What the dashboard covers Copy Chapter

The PDS reports five business events. Three of them are backed by counters and
show up as metrics; two are only emitted as log records, and so are not on this
dashboard:

The counters have only low-cardinality attributes: source on the account and
session counters, clientFirstParty on the OAuth one. Per-account detail such
as did and clientId is deliberately kept out of the metrics and appears only
in the log records.

Already running Prometheus and Grafana? Copy Chapter

In this case you don't need compose.yaml.

1. Point the PDS at your metrics backend. Set the variables above in
/pds/pds.env, with OTEL_EXPORTER_OTLP_METRICS_ENDPOINT pointed at your own
OTLP endpoint. If your Prometheus runs elsewhere, enable its OTLP receiver
(--web.enable-otlp-receiver) and use
http://your-prometheus:9090/api/v1/otlp/v1/metrics. If you run an
OpenTelemetry Collector, send to that instead.

2. Import the dashboard. In Grafana, Dashboards → New → Import → Upload
JSON file, and pick dashboards/pds-overview.json. It expects a Prometheus
data source; select yours when prompted.

For host metrics, you presumably already run node_exporter. If not, you can
start just that one service from this directory:

docker compose up --detach node-exporter

and add a scrape job to your own Prometheus:

scrape_configs:
  - job_name: pds-node
    static_configs:
      - targets: ['your-pds-host:9100']

Note that compose.yaml binds node_exporter to 127.0.0.1, so a Prometheus on
another machine cannot reach it as-is. Either scrape it over an SSH tunnel or a
private network, or change --web.listen-address, but if you make it listen on
a public interface, firewall it. node_exporter has no authentication.

Retention and disk Copy Chapter

Prometheus keeps 15 days by default (--storage.tsdb.retention.time in
compose.yaml) and stores data in a Docker volume, not under /pds. For a
single PDS this is a small amount of data, but note that it shares the host's
disk with your repos and blobs, and that a /pds backup will not include it.

If a panel is empty Copy Chapter

If every PDS-related panel is empty while the host CPU/disk metrics work, the SDK is not loaded:
check that NODE_OPTIONS=--import=@atproto/pds/telemetry is really in
/pds/pds.env and that the PDS was restarted after you added it.

docker exec pds printenv NODE_OPTIONS

Metric names come from the OpenTelemetry instrumentation and are translated by
Prometheus's OTLP receiver (dots become underscores, and type and unit suffixes
are appended; account.created becomes account_created_total). Names can
change as the instrumentation libraries are upgraded.

To see what your PDS is actually reporting:

curl -s localhost:9090/api/v1/label/__name__/values | tr ',' '\n' | grep -Ei 'account|session|oauth|http_server|nodejs|v8js'

If you see http_server_duration_milliseconds_* rather than
http_server_request_duration_seconds_*, you are missing
OTEL_SEMCONV_STABILITY_OPT_IN=http.

Counters only exist once they have been incremented at least once, so a fresh
PDS legitimately has no account_created_total until somebody signs up.

If a name differs from what a panel queries, edit the panel's query, or open an
issue so the dashboard can be fixed.

Going further: session events and traces Copy Chapter

If you want sign-ins and session refreshes, or traces, Prometheus alone isn't
enough, as it cannot receive either signal. You need something in front that can
route each signal to a different backend: an
OpenTelemetry Collector (or Grafana
Alloy). With one in place you can point the PDS at a single endpoint and let the
collector route each signal:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

The PDS deliberately does
not ship its pino request logs over OTLP. It emits only the five
events above, through the Logs API. Those records use the OTLP event_name
field (account.signed-in, session.refreshed, and so on) with the event detail
in attributes, and they are trace-correlated when traces are also enabled.

From there you have two options. To chart sign-ins and refreshes alongside the
existing panels, have the collector turn those log records into counters with the
count connector,
matching on the record's event name, and export the result to Prometheus. To
explore them as logs instead, send them to Loki and add it as a second Grafana
data source; traces work the same way with Tempo.

--- METRICS ---

• Files Extracted: 2
• Estimated Token Budget: ~7889 tokens
• Canonical Reference: https://codewiki.google/github.com/bluesky-social/pds