websockets Documentation

File: docs/deploy/fly.rst Copy Chapter

Deploy to Fly

This guide describes how to deploy a websockets server to Fly_.

.. _Fly: https://fly.io/

.. admonition:: The free tier of Fly is sufficient for trying this guide.
:class: tip

The `free tier`__ include up to three small VMs. This guide uses only one.

__ https://fly.io/docs/about/pricing/

We're going to deploy a very simple app. The process would be identical for a
more realistic app.

Create application Copy Chapter

Here's the implementation of the app, an echo server. Save it in a file called
app.py:

.. literalinclude:: ../../example/deployment/fly/app.py
:language: python

This app implements typical requirements for running on a Platform as a Service:

• it provides a health check at /healthz;
• it closes connections and exits cleanly when it receives a SIGTERM signal.

Create a requirements.txt file containing this line to declare a dependency
on websockets:

.. literalinclude:: ../../example/deployment/fly/requirements.txt
:language: text

The app is ready. Let's deploy it!

Deploy application Copy Chapter

Follow the instructions__ to install the Fly CLI, if you haven't done that yet.

__ https://fly.io/docs/hands-on/install-flyctl/

Sign up or log in to Fly.

Launch the app — you'll have to pick a different name because I'm already using
websockets-echo:

.. code-block:: console

$ fly launch
Creating app in ...
Scanning source code
Detected a Python app
Using the following build configuration:
    Builder: paketobuildpacks/builder:base
? App Name (leave blank to use an auto-generated name): websockets-echo
? Select organization: ...
? Select region: ...
Created app websockets-echo in organization ...
Wrote config file fly.toml
? Would you like to set up a Postgresql database now? No
We have generated a simple Procfile for you. Modify it to fit your needs and run "fly deploy" to deploy your application.

.. admonition:: This will build the image with a generic buildpack.
:class: tip

Fly can `build images`__ with a Dockerfile or a buildpack. Here, ``fly
launch`` configures a generic Paketo buildpack.

If you'd rather package the app with a Dockerfile, check out the guide to
:ref:`containerize an application <containerize-application>`.

__ https://fly.io/docs/reference/builders/

Replace the auto-generated fly.toml with:

.. literalinclude:: ../../example/deployment/fly/fly.toml
:language: toml

This configuration:

• listens on port 443, terminates TLS, and forwards to the app on port 8080;
• declares a health check at /healthz;
• requests a SIGTERM for terminating the app.

Replace the auto-generated Procfile with:

.. literalinclude:: ../../example/deployment/fly/Procfile
:language: text

This tells Fly how to run the app.

Now you can deploy it:

.. code-block:: console

$ fly deploy

... lots of output...

==> Monitoring deployment

1 desired, 1 placed, 1 healthy, 0 unhealthy [health checks: 1 total, 1 passing]
--> v0 deployed successfully

Validate deployment Copy Chapter

Let's confirm that your application is running as expected.

Since it's a WebSocket server, you need a WebSocket client, such as the
interactive client that comes with websockets.

If you're currently building a websockets server, perhaps you're already in a
virtualenv where websockets is installed. If not, you can install it in a new
virtualenv as follows:

.. code-block:: console

$ python -m venv websockets-client
$ . websockets-client/bin/activate
$ pip install websockets

Connect the interactive client — you must replace websockets-echo with the
name of your Fly app in this command:

.. code-block:: console

$ websockets wss://websockets-echo.fly.dev/
Connected to wss://websockets-echo.fly.dev/.
>

Great! Your app is running!

Once you're connected, you can send any message and the server will echo it,
or press Ctrl-D to terminate the connection:

.. code-block:: console

> Hello!
< Hello!
Connection closed: 1000 (OK).

You can also confirm that your application shuts down gracefully.

Connect an interactive client again — remember to replace websockets-echo
with your app:

.. code-block:: console

$ websockets wss://websockets-echo.fly.dev/
Connected to wss://websockets-echo.fly.dev/.
>

In another shell, restart the app — again, replace websockets-echo with your
app:

.. code-block:: console

$ fly restart websockets-echo
websockets-echo is being restarted

Go back to the first shell. The connection is closed with code 1001 (going
away).

.. code-block:: console

$ websockets wss://websockets-echo.fly.dev/
Connected to wss://websockets-echo.fly.dev/.
Connection closed: 1001 (going away).

If graceful shutdown wasn't working, the server wouldn't perform a closing
handshake and the connection would be closed with code 1006 (abnormal closure).

File: docs/deploy/haproxy.rst Copy Chapter

Deploy behind HAProxy

This guide demonstrates a way to load balance connections across multiple
websockets server processes running on the same machine with HAProxy_.

We'll run server processes with Supervisor as described in :doc:this guide <supervisor>.

.. _HAProxy: https://www.haproxy.org/

Run server processes Copy Chapter

Save this app to app.py:

.. literalinclude:: ../../example/deployment/haproxy/app.py
:language: python

Each server process listens on a different port by extracting an incremental
index from an environment variable set by Supervisor.

Save this configuration to supervisord.conf:

.. literalinclude:: ../../example/deployment/haproxy/supervisord.conf

This configuration runs four instances of the app.

Install Supervisor and run it:

.. code-block:: console

$ supervisord -c supervisord.conf -n

Configure and run HAProxy Copy Chapter

Here's a simple HAProxy configuration to load balance connections across four
processes:

.. literalinclude:: ../../example/deployment/haproxy/haproxy.cfg

In the backend configuration, we set the load balancing method to
leastconn in order to balance the number of active connections across
servers. This is best for long running connections.

Save the configuration to haproxy.cfg, install HAProxy, and run it:

.. code-block:: console

$ haproxy -f haproxy.cfg

You can confirm that HAProxy proxies connections properly:

.. code-block:: console

$ websockets ws://localhost:8080/
Connected to ws://localhost:8080/.
> Hello!
< Hello!
Connection closed: 1000 (OK).

File: docs/deploy/heroku.rst Copy Chapter

Deploy to Heroku

This guide describes how to deploy a websockets server to Heroku_. The same
principles should apply to other Platform as a Service providers.

.. _Heroku: https://www.heroku.com/

.. admonition:: Heroku no longer offers a free tier.
:class: attention

When this tutorial was written, in September 2021, Heroku offered a free
tier where a websockets app could run at no cost. In November 2022, Heroku
removed the free tier, making it impossible to maintain this document. As a
consequence, it isn't updated anymore and may be removed in the future.

We're going to deploy a very simple app. The process would be identical for a
more realistic app.

Create repository Copy Chapter

Deploying to Heroku requires a git repository. Let's initialize one:

.. code-block:: console

$ mkdir websockets-echo
$ cd websockets-echo
$ git init -b main
Initialized empty Git repository in websockets-echo/.git/
$ git commit --allow-empty -m "Initial commit."
[main (root-commit) 1e7947d] Initial commit.

Create application Copy Chapter

Here's the implementation of the app, an echo server. Save it in a file called
app.py:

.. literalinclude:: ../../example/deployment/heroku/app.py
:language: python

Heroku expects the server to listen on a specific port_, which is provided
in the $PORT environment variable. The app reads it and passes it to
:func:~websockets.asyncio.server.serve.

.. _listen on a specific port: https://devcenter.heroku.com/articles/preparing-a-codebase-for-heroku-deployment#4-listen-on-the-correct-port

Heroku sends a SIGTERM signal to all processes when shutting down a dyno_. When the app receives this signal, it closes connections and exits
cleanly.

.. _shutting down a dyno: https://devcenter.heroku.com/articles/dynos#shutdown

Create a requirements.txt file containing this line to declare a dependency
on websockets:

.. literalinclude:: ../../example/deployment/heroku/requirements.txt
:language: text

Create a Procfile to tell Heroku how to run the app.

.. literalinclude:: ../../example/deployment/heroku/Procfile

Confirm that you created the correct files and commit them to git:

.. code-block:: console

$ ls
Procfile         app.py           requirements.txt
$ git add .
$ git commit -m "Initial implementation."
[main 8418c62] Initial implementation.
 3 files changed, 32 insertions(+)
 create mode 100644 Procfile
 create mode 100644 app.py
 create mode 100644 requirements.txt

The app is ready. Let's deploy it!

Deploy application Copy Chapter

Follow the instructions_ to install the Heroku CLI, if you haven't done that
yet.

.. _instructions: https://devcenter.heroku.com/articles/getting-started-with-python#set-up

Sign up or log in to Heroku.

Create a Heroku app — you'll have to pick a different name because I'm already
using websockets-echo:

.. code-block:: console

$ heroku create websockets-echo
Creating ⬢ websockets-echo... done
https://websockets-echo.herokuapp.com/ | https://git.heroku.com/websockets-echo.git

.. code-block:: console

$ git push heroku

... lots of output...

remote: -----> Launching...
remote:        Released v1
remote:        https://websockets-echo.herokuapp.com/ deployed to Heroku
remote:
remote: Verifying deploy... done.
To https://git.heroku.com/websockets-echo.git
 * [new branch]      main -> main

Validate deployment Copy Chapter

Let's confirm that your application is running as expected.

Since it's a WebSocket server, you need a WebSocket client, such as the
interactive client that comes with websockets.

If you're currently building a websockets server, perhaps you're already in a
virtualenv where websockets is installed. If not, you can install it in a new
virtualenv as follows:

.. code-block:: console

$ python -m venv websockets-client
$ . websockets-client/bin/activate
$ pip install websockets

Connect the interactive client — you must replace websockets-echo with the
name of your Heroku app in this command:

.. code-block:: console

$ websockets wss://websockets-echo.herokuapp.com/
Connected to wss://websockets-echo.herokuapp.com/.
>

Great! Your app is running!

Once you're connected, you can send any message and the server will echo it,
or press Ctrl-D to terminate the connection:

.. code-block:: console

> Hello!
< Hello!
Connection closed: 1000 (OK).

You can also confirm that your application shuts down gracefully.

Connect an interactive client again — remember to replace websockets-echo
with your app:

.. code-block:: console

$ websockets wss://websockets-echo.herokuapp.com/
Connected to wss://websockets-echo.herokuapp.com/.
>

In another shell, restart the app — again, replace websockets-echo with your
app:

.. code-block:: console

$ heroku dyno:restart -a websockets-echo
Restarting dynos on ⬢ websockets-echo... done

Go back to the first shell. The connection is closed with code 1001 (going
away).

.. code-block:: console

$ websockets wss://websockets-echo.herokuapp.com/
Connected to wss://websockets-echo.herokuapp.com/.
Connection closed: 1001 (going away).

If graceful shutdown wasn't working, the server wouldn't perform a closing
handshake and the connection would be closed with code 1006 (abnormal closure).

File: docs/deploy/index.rst Copy Chapter

Deployment

.. currentmodule:: websockets

Architecture decisions Copy Chapter

When you deploy your websockets server to production, at a high level, your
architecture will almost certainly look like the following diagram:

.. image:: architecture.svg

The basic unit for scaling a websockets server is "one server process". Each
blue box in the diagram represents one server process.

There's more variation in routing connections to processes. While the routing
layer is shown as one big box, it is likely to involve several subsystems.

As a consequence, when you design a deployment, you must answer two questions:

1. How will I run the appropriate number of server processes?
2. How will I route incoming connections to these processes?

These questions are interrelated. There's a wide range of valid answers,
depending on your goals and your constraints.

Platforms-as-a-Service
......................

Platforms-as-a-Service are the easiest option. They provide end-to-end,
integrated solutions and they require little configuration.

Here's how to deploy on some popular PaaS providers. Since all PaaS use
similar patterns, the concepts translate to other providers.

.. toctree::
:titlesonly:

render
koyeb
fly
heroku

Self-hosted infrastructure
..........................

If you need more control over your infrastructure, you can deploy on your own
infrastructure. This requires more configuration.

Here's how to configure some components mentioned in this guide.

.. toctree::
:titlesonly:

kubernetes
supervisor
nginx
haproxy

Running server processes Copy Chapter

How many processes do I need?
.............................

Typically, one server process will manage a few hundreds or thousands
connections, depending on the frequency of messages and the amount of work
they require.

CPU and memory usage increase with the number of connections to the server.

Often CPU is the limiting factor. If a server process goes to 100% CPU, then
you reached the limit. How much headroom you want to keep is up to you.

Once you know how many connections a server process can manage and how many
connections you need to handle, you can calculate how many processes to run.

You can also automate this calculation by configuring an autoscaler to keep
CPU usage or connection count within acceptable limits.

.. admonition:: Don't scale with threads. Scale only with processes.
:class: tip

Threads don't make sense for a server built with :mod:`asyncio`.

How do I run processes?
.......................

Most solutions for running multiple instances of a server process fall into
one of these three buckets:

1. Running N processes on a platform:

   • a Kubernetes Deployment

   • its equivalent on a Platform as a Service provider

2. Running N servers:

   • an AWS Auto Scaling group, a GCP Managed instance group, etc.

   • a fixed set of long-lived servers

3. Running N processes on a server:

   • preferably via a process manager or supervisor

Option 1 is easiest if you have access to such a platform. Option 2 usually
combines with option 3.

How do I start a process?
.........................

Run a Python program that invokes :func:~asyncio.server.serve or
:func:~asyncio.router.route. That's it!

Don't run an ASGI server such as Uvicorn, Hypercorn, or Daphne. They're
alternatives to websockets, not complements.

Don't run a WSGI server such as Gunicorn, Waitress, or mod_wsgi. They aren't
designed to run WebSocket applications.

Applications servers handle network connections and expose a Python API. You
don't need one because websockets handles network connections directly.

How do I stop a process?
........................

Process managers send the SIGTERM signal to terminate processes. Catch this
signal and exit the server to ensure a graceful shutdown.

Here's an example:

.. literalinclude:: ../../example/faq/shutdown_server.py
:emphasize-lines: 14-16

When exiting the context manager, :func:~asyncio.server.serve closes all
connections with code 1001 (going away). As a consequence:

• If the connection handler is awaiting
  :meth:~asyncio.server.ServerConnection.recv, it receives a
  :exc:~exceptions.ConnectionClosedOK exception. It can catch the exception
  and clean up before exiting.

• Otherwise, it should be waiting on
  :meth:~asyncio.server.ServerConnection.wait_closed, so it can receive the
  :exc:~exceptions.ConnectionClosedOK exception and exit.

This example is easily adapted to handle other signals.

If you override the default signal handler for SIGINT, which raises
:exc:KeyboardInterrupt, be aware that you won't be able to interrupt a
program with Ctrl-C anymore when it's stuck in a loop.

Routing connections to processes Copy Chapter

What does routing involve?
..........................

Since the routing layer is directly exposed to the Internet, it should provide
appropriate protection against threats ranging from Internet background noise
to targeted attacks.

You should always secure WebSocket connections with TLS. Since the routing
layer carries the public domain name, it should terminate TLS connections.

Finally, it must route connections to the server processes, balancing new
connections across them.

How do I route connections?
...........................

Here are typical solutions for load balancing, matched to ways of running
processes:

1. If you're running on a platform, it comes with a routing layer:

   • a Kubernetes Ingress and Service

   • a service mesh: Istio, Consul, Linkerd, etc.

   • the routing mesh of a Platform as a Service

2. If you're running N servers, you may load balance with:

   • a cloud load balancer: AWS Elastic Load Balancing, GCP Cloud Load
     Balancing, etc.

   • A software load balancer: HAProxy, NGINX, etc.

3. If you're running N processes on a server, you may load balance with:

   • A software load balancer: HAProxy, NGINX, etc.

   • The operating system — all processes listen on the same port

You may trust the load balancer to handle encryption and to provide security.
You may add another layer in front of the load balancer for these purposes.

There are many possibilities. Don't add layers that you don't need, though.

How do I implement a health check?
..................................

Load balancers need a way to check whether server processes are up and running
to avoid routing connections to a non-functional backend.

websockets provide minimal support for responding to HTTP requests with the
process_request hook.

Here's an example:

.. literalinclude:: ../../example/faq/health_check_server.py
:emphasize-lines: 7-9,16

File: docs/deploy/koyeb.rst Copy Chapter

Deploy to Koyeb

This guide describes how to deploy a websockets server to Koyeb_.

.. _Koyeb: https://www.koyeb.com

.. admonition:: The free tier of Koyeb is sufficient for trying this guide.
:class: tip

The `free tier`__ include one web service, which this guide uses.

__ https://www.koyeb.com/pricing

We’re going to deploy a very simple app. The process would be identical to a
more realistic app.

Create repository Copy Chapter

Koyeb supports multiple deployment methods. Its quick start guides recommend
git-driven deployment as the first option. Let's initialize a git repository:

.. code-block:: console

$ mkdir websockets-echo
$ cd websockets-echo
$ git init -b main
Initialized empty Git repository in websockets-echo/.git/
$ git commit --allow-empty -m "Initial commit."
[main (root-commit) 740f699] Initial commit.

Render requires the git repository to be hosted at GitHub.

Sign up or log in to GitHub. Create a new repository named websockets-echo.
Don't enable any of the initialization options offered by GitHub. Then, follow
instructions for pushing an existing repository from the command line.

After pushing, refresh your repository's homepage on GitHub. You should see an
empty repository with an empty initial commit.

Create application Copy Chapter

Here’s the implementation of the app, an echo server. Save it in a file
called app.py:

.. literalinclude:: ../../example/deployment/koyeb/app.py
:language: python

This app implements typical requirements for running on a Platform as a Service:

• it listens on the port provided in the $PORT environment variable;
• it provides a health check at /healthz;
• it closes connections and exits cleanly when it receives a SIGTERM signal;
  while not documented, this is how Koyeb terminates apps.

Create a requirements.txt file containing this line to declare a dependency
on websockets:

.. literalinclude:: ../../example/deployment/koyeb/requirements.txt
:language: text

Create a Procfile to tell Koyeb how to run the app.

.. literalinclude:: ../../example/deployment/koyeb/Procfile

Confirm that you created the correct files and commit them to git:

.. code-block:: console

$ ls
Procfile         app.py           requirements.txt
$ git add .
$ git commit -m "Initial implementation."
[main f634b8b] Initial implementation.
 3 files changed, 39 insertions(+)
 create mode 100644 Procfile
 create mode 100644 app.py
 create mode 100644 requirements.txt

The app is ready. Let's deploy it!

Deploy application Copy Chapter

Sign up or log in to Koyeb.

In the Koyeb control panel, create a web service with GitHub as the deployment
method. Install and authorize Koyeb's GitHub app if you haven't done that yet.

Follow the steps to create a new service:

1. Select the websockets-echo repository in the list of your repositories.
2. Confirm that the Free instance type is selected. Click Next.
3. Configure health checks: change the protocol from TCP to HTTP and set the
   path to /healthz. Review other settings; defaults should be correct.
   Click Deploy.

Koyeb builds the app, deploys it, verifies that the health checks passes, and
makes the deployment active.

Validate deployment Copy Chapter

Let's confirm that your application is running as expected.

Since it's a WebSocket server, you need a WebSocket client, such as the
interactive client that comes with websockets.

If you're currently building a websockets server, perhaps you're already in a
virtualenv where websockets is installed. If not, you can install it in a new
virtualenv as follows:

.. code-block:: console

$ python -m venv websockets-client
$ . websockets-client/bin/activate
$ pip install websockets

Look for the URL of your app in the Koyeb control panel. It looks like
https://<app>--.koyeb.app/. Connect the
interactive client — you must replace https with wss in the URL:

.. code-block:: console

$ websockets wss://<app>--.koyeb.app/
Connected to wss://<app>--.koyeb.app/.
>

Great! Your app is running!

Once you're connected, you can send any message and the server will echo it,
or press Ctrl-D to terminate the connection:

.. code-block:: console

> Hello!
< Hello!
Connection closed: 1000 (OK).

You can also confirm that your application shuts down gracefully.

Connect an interactive client again:

.. code-block:: console

$ websockets wss://<app>--.koyeb.app/
Connected to wss://<app>--.koyeb.app/.
>

In the Koyeb control panel, go to the Settings tab, click Pause, and
confirm.

Eventually, the connection gets closed with code 1001 (going away).

.. code-block:: console

$ websockets wss://<app>--.koyeb.app/
Connected to wss://<app>--.koyeb.app/.
Connection closed: 1001 (going away).

If graceful shutdown wasn't working, the server wouldn't perform a closing
handshake and the connection would be closed with code 1006 (abnormal closure).

File: docs/deploy/kubernetes.rst Copy Chapter

Deploy to Kubernetes

This guide describes how to deploy a websockets server to Kubernetes_. It
assumes familiarity with Docker and Kubernetes.

We're going to deploy a simple app to a local Kubernetes cluster and to ensure
that it scales as expected.

In a more realistic context, you would follow your organization's practices
for deploying to Kubernetes, but you would apply the same principles as far as
websockets is concerned.

.. _Kubernetes: https://kubernetes.io/

.. _containerize-application:

Containerize application Copy Chapter

Here's the app we're going to deploy. Save it in a file called
app.py:

.. literalinclude:: ../../example/deployment/kubernetes/app.py

This is an echo server with one twist: every message blocks the server for
100ms, which creates artificial starvation of CPU time. This makes it easier
to saturate the server for load testing.

The app exposes a health check on /healthz. It also provides two other
endpoints for testing purposes: /inemuri will make the app unresponsive
for 10 seconds and /seppuku will terminate it.

The quest for the perfect Python container image is out of scope of this
guide, so we'll go for the simplest possible configuration instead:

.. literalinclude:: ../../example/deployment/kubernetes/Dockerfile

After saving this Dockerfile, build the image:

.. code-block:: console

$ docker build -t websockets-test:1.0 .

Test your image by running:

.. code-block:: console

$ docker run --name run-websockets-test --publish 32080:80 --rm \
    websockets-test:1.0

Then, in another shell, in a virtualenv where websockets is installed, connect
to the app and check that it echoes anything you send:

.. code-block:: console

$ websockets ws://localhost:32080/
Connected to ws://localhost:32080/.
> Hey there!
< Hey there!
>

Now, in yet another shell, stop the app with:

.. code-block:: console

$ docker kill -s TERM run-websockets-test

Going to the shell where you connected to the app, you can confirm that it
shut down gracefully:

.. code-block:: console

$ websockets ws://localhost:32080/
Connected to ws://localhost:32080/.
> Hey there!
< Hey there!
Connection closed: 1001 (going away).

If it didn't, you'd get code 1006 (abnormal closure).

Deploy application Copy Chapter

Configuring Kubernetes is even further beyond the scope of this guide, so
we'll use a basic configuration for testing, with just one Service_ and one
Deployment_:

.. literalinclude:: ../../example/deployment/kubernetes/deployment.yaml

For local testing, a service of type NodePort_ is good enough. For deploying
to production, you would configure an Ingress_.

.. _Service: https://kubernetes.io/docs/concepts/services-networking/service/
.. _Deployment: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
.. _NodePort: https://kubernetes.io/docs/concepts/services-networking/service/#nodeport
.. _Ingress: https://kubernetes.io/docs/concepts/services-networking/ingress/

After saving this to a file called deployment.yaml, you can deploy:

.. code-block:: console

$ kubectl apply -f deployment.yaml
service/websockets-test created
deployment.apps/websockets-test created

Now you have a deployment with one pod running:

.. code-block:: console

$ kubectl get deployment websockets-test
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
websockets-test   1/1     1            1           10s
$ kubectl get pods -l app=websockets-test
NAME                               READY   STATUS    RESTARTS   AGE
websockets-test-86b48f4bb7-nltfh   1/1     Running   0          10s

You can connect to the service — press Ctrl-D to exit:

.. code-block:: console

$ websockets ws://localhost:32080/
Connected to ws://localhost:32080/.
Connection closed: 1000 (OK).

Validate deployment Copy Chapter

First, let's ensure the liveness probe works by making the app unresponsive:

.. code-block:: console

$ curl http://localhost:32080/inemuri
Sleeping for 10s

Since we have only one pod, we know that this pod will go to sleep.

The liveness probe is configured to run every second. By default, liveness
probes time out after one second and have a threshold of three failures.
Therefore Kubernetes should restart the pod after at most 5 seconds.

Indeed, after a few seconds, the pod reports a restart:

.. code-block:: console

$ kubectl get pods -l app=websockets-test
NAME                               READY   STATUS    RESTARTS   AGE
websockets-test-86b48f4bb7-nltfh   1/1     Running   1          42s

Next, let's take it one step further and crash the app:

.. code-block:: console

$ curl http://localhost:32080/seppuku
Terminating

The pod reports a second restart:

.. code-block:: console

$ kubectl get pods -l app=websockets-test
NAME                               READY   STATUS    RESTARTS   AGE
websockets-test-86b48f4bb7-nltfh   1/1     Running   2          72s

All good — Kubernetes delivers on its promise to keep our app alive!

Scale deployment Copy Chapter

Of course, Kubernetes is for scaling. Let's scale — modestly — to 10 pods:

.. code-block:: console

$ kubectl scale deployment.apps/websockets-test --replicas=10
deployment.apps/websockets-test scaled

After a few seconds, we have 10 pods running:

.. code-block:: console

$ kubectl get deployment websockets-test
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
websockets-test   10/10   10           10          10m

Now let's generate load. We'll use this script:

.. literalinclude:: ../../example/deployment/kubernetes/benchmark.py

We'll connect 500 clients in parallel, meaning 50 clients per pod, and have
each client send 6 messages. Since the app blocks for 100ms before responding,
if connections are perfectly distributed, we expect a total run time slightly
over 50 * 6 * 0.1 = 30 seconds.

Let's try it:

.. code-block:: console

$ ulimit -n 512
$ time python benchmark.py 500 6
python benchmark.py 500 6  2.40s user 0.51s system 7% cpu 36.471 total

A total runtime of 36 seconds is in the right ballpark. Repeating this
experiment with other parameters shows roughly consistent results, with the
high variability you'd expect from a quick benchmark without any effort to
stabilize the test setup.

Finally, we can scale back to one pod.

.. code-block:: console

$ kubectl scale deployment.apps/websockets-test --replicas=1
deployment.apps/websockets-test scaled
$ kubectl get deployment websockets-test
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
websockets-test   1/1     1            1           15m

File: docs/deploy/nginx.rst Copy Chapter

Deploy behind nginx

This guide demonstrates a way to load balance connections across multiple
websockets server processes running on the same machine with nginx_.

We'll run server processes with Supervisor as described in :doc:this guide <supervisor>.

.. _nginx: https://nginx.org/

Run server processes Copy Chapter

Save this app to app.py:

.. literalinclude:: ../../example/deployment/nginx/app.py
:language: python

We'd like nginx to connect to websockets servers via Unix sockets in order to
avoid the overhead of TCP for communicating between processes running in the
same OS.

We start the app with :func:~websockets.asyncio.server.unix_serve. Each server
process listens on a different socket thanks to an environment variable set by
Supervisor to a different value.

Save this configuration to supervisord.conf:

.. literalinclude:: ../../example/deployment/nginx/supervisord.conf

This configuration runs four instances of the app.

Install Supervisor and run it:

.. code-block:: console

$ supervisord -c supervisord.conf -n

Configure and run nginx Copy Chapter

Here's a simple nginx configuration to load balance connections across four
processes:

.. literalinclude:: ../../example/deployment/nginx/nginx.conf

We set daemon off so we can run nginx in the foreground for testing.

Then we combine the WebSocket proxying_ and load balancing_ guides:

• The WebSocket protocol requires HTTP/1.1. We must set the HTTP protocol
  version to 1.1, else nginx defaults to HTTP/1.0 for proxying.

• The WebSocket handshake involves the Connection and Upgrade HTTP
  headers. We must pass them to the upstream explicitly, else nginx drops
  them because they're hop-by-hop headers.

  We deviate from the WebSocket proxying_ guide because its example adds a
  Connection: Upgrade header to every upstream request, even if the
  original request didn't contain that header.

• In the upstream configuration, we set the load balancing method to
  least_conn in order to balance the number of active connections across
  servers. This is best for long running connections.

.. _WebSocket proxying: http://nginx.org/en/docs/http/websocket.html
.. _load balancing: http://nginx.org/en/docs/http/load_balancing.html

Save the configuration to nginx.conf, install nginx, and run it:

.. code-block:: console

$ nginx -c nginx.conf -p .

You can confirm that nginx proxies connections properly:

.. code-block:: console

$ websockets ws://localhost:8080/
Connected to ws://localhost:8080/.
> Hello!
< Hello!
Connection closed: 1000 (OK).

File: docs/deploy/render.rst Copy Chapter

Deploy to Render

This guide describes how to deploy a websockets server to Render_.

.. _Render: https://render.com/

.. admonition:: The free plan of Render is sufficient for trying this guide.
:class: tip

However, on a `free plan`__, connections are dropped after five minutes,
which is quite short for WebSocket application.

__ https://render.com/docs/free

We're going to deploy a very simple app. The process would be identical for a
more realistic app.

Create repository Copy Chapter

Deploying to Render requires a git repository. Let's initialize one:

.. code-block:: console

$ mkdir websockets-echo
$ cd websockets-echo
$ git init -b main
Initialized empty Git repository in websockets-echo/.git/
$ git commit --allow-empty -m "Initial commit."
[main (root-commit) 816c3b1] Initial commit.

Render requires the git repository to be hosted at GitHub or GitLab.

Sign up or log in to GitHub. Create a new repository named websockets-echo.
Don't enable any of the initialization options offered by GitHub. Then, follow
instructions for pushing an existing repository from the command line.

After pushing, refresh your repository's homepage on GitHub. You should see an
empty repository with an empty initial commit.

Create application Copy Chapter

Here's the implementation of the app, an echo server. Save it in a file called
app.py:

.. literalinclude:: ../../example/deployment/render/app.py
:language: python

This app implements requirements for zero downtime deploys_:

• it provides a health check at /healthz;
• it closes connections and exits cleanly when it receives a SIGTERM signal.

.. _zero downtime deploys: https://render.com/docs/deploys#zero-downtime-deploys

Create a requirements.txt file containing this line to declare a dependency
on websockets:

.. literalinclude:: ../../example/deployment/render/requirements.txt
:language: text

Confirm that you created the correct files and commit them to git:

.. code-block:: console

$ ls
app.py           requirements.txt
$ git add .
$ git commit -m "Initial implementation."
[main f26bf7f] Initial implementation.
2 files changed, 37 insertions(+)
create mode 100644 app.py
create mode 100644 requirements.txt

Push the changes to GitHub:

.. code-block:: console

$ git push
...
To github.com:/websockets-echo.git
   816c3b1..f26bf7f  main -> main

The app is ready. Let's deploy it!

Deploy application Copy Chapter

Sign up or log in to Render.

Create a new web service. Connect the git repository that you just created.

Then, finalize the configuration of your app as follows:

• Name: websockets-echo
• Start Command: python app.py

If you're just experimenting, select the free plan. Create the web service.

To configure the health check, go to Settings, scroll down to Health & Alerts,
and set:

• Health Check Path: /healthz

This triggers a new deployment.

Validate deployment Copy Chapter

Let's confirm that your application is running as expected.

Since it's a WebSocket server, you need a WebSocket client, such as the
interactive client that comes with websockets.

If you're currently building a websockets server, perhaps you're already in a
virtualenv where websockets is installed. If not, you can install it in a new
virtualenv as follows:

.. code-block:: console

$ python -m venv websockets-client
$ . websockets-client/bin/activate
$ pip install websockets

Connect the interactive client — you must replace websockets-echo with the
name of your Render app in this command:

.. code-block:: console

$ websockets wss://websockets-echo.onrender.com/
Connected to wss://websockets-echo.onrender.com/.
>

Great! Your app is running!

Once you're connected, you can send any message and the server will echo it,
or press Ctrl-D to terminate the connection:

.. code-block:: console

> Hello!
< Hello!
Connection closed: 1000 (OK).

You can also confirm that your application shuts down gracefully when you deploy
a new version. Due to limitations of Render's free plan, you must upgrade to a
paid plan before you perform this test.

Connect an interactive client again — remember to replace websockets-echo
with your app:

.. code-block:: console

$ websockets wss://websockets-echo.onrender.com/
Connected to wss://websockets-echo.onrender.com/.
>

Trigger a new deployment with Manual Deploy > Deploy latest commit. When the
deployment completes, the connection is closed with code 1001 (going away).

.. code-block:: console

$ websockets wss://websockets-echo.onrender.com/
Connected to wss://websockets-echo.onrender.com/.
Connection closed: 1001 (going away).

If graceful shutdown wasn't working, the server wouldn't perform a closing
handshake and the connection would be closed with code 1006 (abnormal closure).

Remember to downgrade to a free plan if you upgraded just for testing this feature.

File: docs/deploy/supervisor.rst Copy Chapter

Deploy with Supervisor

This guide proposes a simple way to deploy a websockets server directly on a
Linux or BSD operating system.

We'll configure Supervisor_ to run several server processes and to restart
them if needed.

.. _Supervisor: http://supervisord.org/

We'll bind all servers to the same port. The OS will take care of balancing
connections.

Create and activate a virtualenv:

.. code-block:: console

$ python -m venv supervisor-websockets
$ . supervisor-websockets/bin/activate

Install websockets and Supervisor:

.. code-block:: console

$ pip install websockets
$ pip install supervisor

Save this app to a file called app.py:

.. literalinclude:: ../../example/deployment/supervisor/app.py

This is an echo server with two features added for the purpose of this guide:

• It shuts down gracefully when receiving a SIGTERM signal;
• It enables the reuse_port option of :meth:~asyncio.loop.create_server,
  which in turns sets SO_REUSEPORT on the accept socket.

Save this Supervisor configuration to supervisord.conf:

.. literalinclude:: ../../example/deployment/supervisor/supervisord.conf

This is the minimal configuration required to keep four instances of the app
running, restarting them if they exit.

Now start Supervisor in the foreground:

.. code-block:: console

$ supervisord -c supervisord.conf -n
INFO Increased RLIMIT_NOFILE limit to 1024
INFO supervisord started with pid 43596
INFO spawned: 'websockets-test_00' with pid 43597
INFO spawned: 'websockets-test_01' with pid 43598
INFO spawned: 'websockets-test_02' with pid 43599
INFO spawned: 'websockets-test_03' with pid 43600
INFO success: websockets-test_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
INFO success: websockets-test_01 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
INFO success: websockets-test_02 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
INFO success: websockets-test_03 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

In another shell, after activating the virtualenv, we can connect to the app —
press Ctrl-D to exit:

.. code-block:: console

$ websockets ws://localhost:8080/
Connected to ws://localhost:8080/.
> Hello!
< Hello!
Connection closed: 1000 (OK).

Look at the pid of an instance of the app in the logs and terminate it:

.. code-block:: console

$ kill -TERM 43597

The logs show that Supervisor restarted this instance:

.. code-block:: console

INFO exited: websockets-test_00 (exit status 0; expected)
INFO spawned: 'websockets-test_00' with pid 43629
INFO success: websockets-test_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Now let's check what happens when we shut down Supervisor, but first let's
establish a connection and leave it open:

.. code-block:: console

$ websockets ws://localhost:8080/
Connected to ws://localhost:8080/.
>

Look at the pid of supervisord itself in the logs and terminate it:

.. code-block:: console

$ kill -TERM 43596

The logs show that Supervisor terminated all instances of the app before
exiting:

.. code-block:: console

WARN received SIGTERM indicating exit request
INFO waiting for websockets-test_00, websockets-test_01, websockets-test_02, websockets-test_03 to die
INFO stopped: websockets-test_02 (exit status 0)
INFO stopped: websockets-test_03 (exit status 0)
INFO stopped: websockets-test_01 (exit status 0)
INFO stopped: websockets-test_00 (exit status 0)

And you can see that the connection to the app was closed gracefully:

.. code-block:: console

$ websockets ws://localhost:8080/
Connected to ws://localhost:8080/.
Connection closed: 1001 (going away).

In this example, we've been sharing the same virtualenv for supervisor and
websockets.

In a real deployment, you would likely:

• Install Supervisor with the package manager of the OS.
• Create a virtualenv dedicated to your application.
• Add environment=PATH="path/to/your/virtualenv/bin" in the Supervisor
  configuration. Then python app.py runs in that virtualenv.