## File: docs/deploy/fly.rst Deploy to Fly ============= This guide describes how to deploy a websockets server to Fly_. .. _Fly: https://fly.io/ .. admonition:: The free tier of Fly is sufficient for trying this guide. :class: tip The `free tier`__ include up to three small VMs. This guide uses only one. __ https://fly.io/docs/about/pricing/ We're going to deploy a very simple app. The process would be identical for a more realistic app. Create application ------------------ Here's the implementation of the app, an echo server. Save it in a file called ``app.py``: .. literalinclude:: ../../example/deployment/fly/app.py :language: python This app implements typical requirements for running on a Platform as a Service: * it provides a health check at ``/healthz``; * it closes connections and exits cleanly when it receives a ``SIGTERM`` signal. Create a ``requirements.txt`` file containing this line to declare a dependency on websockets: .. literalinclude:: ../../example/deployment/fly/requirements.txt :language: text The app is ready. Let's deploy it! Deploy application ------------------ Follow the instructions__ to install the Fly CLI, if you haven't done that yet. __ https://fly.io/docs/hands-on/install-flyctl/ Sign up or log in to Fly. Launch the app — you'll have to pick a different name because I'm already using ``websockets-echo``: .. code-block:: console $ fly launch Creating app in ... Scanning source code Detected a Python app Using the following build configuration: Builder: paketobuildpacks/builder:base ? App Name (leave blank to use an auto-generated name): websockets-echo ? Select organization: ... ? Select region: ... Created app websockets-echo in organization ... Wrote config file fly.toml ? Would you like to set up a Postgresql database now? No We have generated a simple Procfile for you. Modify it to fit your needs and run "fly deploy" to deploy your application. .. admonition:: This will build the image with a generic buildpack. :class: tip Fly can `build images`__ with a Dockerfile or a buildpack. Here, ``fly launch`` configures a generic Paketo buildpack. If you'd rather package the app with a Dockerfile, check out the guide to :ref:`containerize an application `. __ https://fly.io/docs/reference/builders/ Replace the auto-generated ``fly.toml`` with: .. literalinclude:: ../../example/deployment/fly/fly.toml :language: toml This configuration: * listens on port 443, terminates TLS, and forwards to the app on port 8080; * declares a health check at ``/healthz``; * requests a ``SIGTERM`` for terminating the app. Replace the auto-generated ``Procfile`` with: .. literalinclude:: ../../example/deployment/fly/Procfile :language: text This tells Fly how to run the app. Now you can deploy it: .. code-block:: console $ fly deploy ... lots of output... ==> Monitoring deployment 1 desired, 1 placed, 1 healthy, 0 unhealthy [health checks: 1 total, 1 passing] --> v0 deployed successfully Validate deployment ------------------- Let's confirm that your application is running as expected. Since it's a WebSocket server, you need a WebSocket client, such as the interactive client that comes with websockets. If you're currently building a websockets server, perhaps you're already in a virtualenv where websockets is installed. If not, you can install it in a new virtualenv as follows: .. code-block:: console $ python -m venv websockets-client $ . websockets-client/bin/activate $ pip install websockets Connect the interactive client — you must replace ``websockets-echo`` with the name of your Fly app in this command: .. code-block:: console $ websockets wss://websockets-echo.fly.dev/ Connected to wss://websockets-echo.fly.dev/. > Great! Your app is running! Once you're connected, you can send any message and the server will echo it, or press Ctrl-D to terminate the connection: .. code-block:: console > Hello! < Hello! Connection closed: 1000 (OK). You can also confirm that your application shuts down gracefully. Connect an interactive client again — remember to replace ``websockets-echo`` with your app: .. code-block:: console $ websockets wss://websockets-echo.fly.dev/ Connected to wss://websockets-echo.fly.dev/. > In another shell, restart the app — again, replace ``websockets-echo`` with your app: .. code-block:: console $ fly restart websockets-echo websockets-echo is being restarted Go back to the first shell. The connection is closed with code 1001 (going away). .. code-block:: console $ websockets wss://websockets-echo.fly.dev/ Connected to wss://websockets-echo.fly.dev/. Connection closed: 1001 (going away). If graceful shutdown wasn't working, the server wouldn't perform a closing handshake and the connection would be closed with code 1006 (abnormal closure). --- ## File: docs/deploy/haproxy.rst Deploy behind HAProxy ===================== This guide demonstrates a way to load balance connections across multiple websockets server processes running on the same machine with HAProxy_. We'll run server processes with Supervisor as described in :doc:`this guide `. .. _HAProxy: https://www.haproxy.org/ Run server processes -------------------- Save this app to ``app.py``: .. literalinclude:: ../../example/deployment/haproxy/app.py :language: python Each server process listens on a different port by extracting an incremental index from an environment variable set by Supervisor. Save this configuration to ``supervisord.conf``: .. literalinclude:: ../../example/deployment/haproxy/supervisord.conf This configuration runs four instances of the app. Install Supervisor and run it: .. code-block:: console $ supervisord -c supervisord.conf -n Configure and run HAProxy ------------------------- Here's a simple HAProxy configuration to load balance connections across four processes: .. literalinclude:: ../../example/deployment/haproxy/haproxy.cfg In the backend configuration, we set the load balancing method to ``leastconn`` in order to balance the number of active connections across servers. This is best for long running connections. Save the configuration to ``haproxy.cfg``, install HAProxy, and run it: .. code-block:: console $ haproxy -f haproxy.cfg You can confirm that HAProxy proxies connections properly: .. code-block:: console $ websockets ws://localhost:8080/ Connected to ws://localhost:8080/. > Hello! < Hello! Connection closed: 1000 (OK). --- ## File: docs/deploy/heroku.rst Deploy to Heroku ================ This guide describes how to deploy a websockets server to Heroku_. The same principles should apply to other Platform as a Service providers. .. _Heroku: https://www.heroku.com/ .. admonition:: Heroku no longer offers a free tier. :class: attention When this tutorial was written, in September 2021, Heroku offered a free tier where a websockets app could run at no cost. In November 2022, Heroku removed the free tier, making it impossible to maintain this document. As a consequence, it isn't updated anymore and may be removed in the future. We're going to deploy a very simple app. The process would be identical for a more realistic app. Create repository ----------------- Deploying to Heroku requires a git repository. Let's initialize one: .. code-block:: console $ mkdir websockets-echo $ cd websockets-echo $ git init -b main Initialized empty Git repository in websockets-echo/.git/ $ git commit --allow-empty -m "Initial commit." [main (root-commit) 1e7947d] Initial commit. Create application ------------------ Here's the implementation of the app, an echo server. Save it in a file called ``app.py``: .. literalinclude:: ../../example/deployment/heroku/app.py :language: python Heroku expects the server to `listen on a specific port`_, which is provided in the ``$PORT`` environment variable. The app reads it and passes it to :func:`~websockets.asyncio.server.serve`. .. _listen on a specific port: https://devcenter.heroku.com/articles/preparing-a-codebase-for-heroku-deployment#4-listen-on-the-correct-port Heroku sends a ``SIGTERM`` signal to all processes when `shutting down a dyno`_. When the app receives this signal, it closes connections and exits cleanly. .. _shutting down a dyno: https://devcenter.heroku.com/articles/dynos#shutdown Create a ``requirements.txt`` file containing this line to declare a dependency on websockets: .. literalinclude:: ../../example/deployment/heroku/requirements.txt :language: text Create a ``Procfile`` to tell Heroku how to run the app. .. literalinclude:: ../../example/deployment/heroku/Procfile Confirm that you created the correct files and commit them to git: .. code-block:: console $ ls Procfile app.py requirements.txt $ git add . $ git commit -m "Initial implementation." [main 8418c62] Initial implementation.  3 files changed, 32 insertions(+)  create mode 100644 Procfile  create mode 100644 app.py  create mode 100644 requirements.txt The app is ready. Let's deploy it! Deploy application ------------------ Follow the instructions_ to install the Heroku CLI, if you haven't done that yet. .. _instructions: https://devcenter.heroku.com/articles/getting-started-with-python#set-up Sign up or log in to Heroku. Create a Heroku app — you'll have to pick a different name because I'm already using ``websockets-echo``: .. code-block:: console $ heroku create websockets-echo Creating ⬢ websockets-echo... done https://websockets-echo.herokuapp.com/ | https://git.heroku.com/websockets-echo.git .. code-block:: console $ git push heroku ... lots of output... remote: -----> Launching... remote: Released v1 remote: https://websockets-echo.herokuapp.com/ deployed to Heroku remote: remote: Verifying deploy... done. To https://git.heroku.com/websockets-echo.git  * [new branch] main -> main Validate deployment ------------------- Let's confirm that your application is running as expected. Since it's a WebSocket server, you need a WebSocket client, such as the interactive client that comes with websockets. If you're currently building a websockets server, perhaps you're already in a virtualenv where websockets is installed. If not, you can install it in a new virtualenv as follows: .. code-block:: console $ python -m venv websockets-client $ . websockets-client/bin/activate $ pip install websockets Connect the interactive client — you must replace ``websockets-echo`` with the name of your Heroku app in this command: .. code-block:: console $ websockets wss://websockets-echo.herokuapp.com/ Connected to wss://websockets-echo.herokuapp.com/. > Great! Your app is running! Once you're connected, you can send any message and the server will echo it, or press Ctrl-D to terminate the connection: .. code-block:: console > Hello! < Hello! Connection closed: 1000 (OK). You can also confirm that your application shuts down gracefully. Connect an interactive client again — remember to replace ``websockets-echo`` with your app: .. code-block:: console $ websockets wss://websockets-echo.herokuapp.com/ Connected to wss://websockets-echo.herokuapp.com/. > In another shell, restart the app — again, replace ``websockets-echo`` with your app: .. code-block:: console $ heroku dyno:restart -a websockets-echo Restarting dynos on ⬢ websockets-echo... done Go back to the first shell. The connection is closed with code 1001 (going away). .. code-block:: console $ websockets wss://websockets-echo.herokuapp.com/ Connected to wss://websockets-echo.herokuapp.com/. Connection closed: 1001 (going away). If graceful shutdown wasn't working, the server wouldn't perform a closing handshake and the connection would be closed with code 1006 (abnormal closure). --- ## File: docs/deploy/index.rst Deployment ========== .. currentmodule:: websockets Architecture decisions ---------------------- When you deploy your websockets server to production, at a high level, your architecture will almost certainly look like the following diagram: .. image:: architecture.svg The basic unit for scaling a websockets server is "one server process". Each blue box in the diagram represents one server process. There's more variation in routing connections to processes. While the routing layer is shown as one big box, it is likely to involve several subsystems. As a consequence, when you design a deployment, you must answer two questions: 1. How will I run the appropriate number of server processes? 2. How will I route incoming connections to these processes? These questions are interrelated. There's a wide range of valid answers, depending on your goals and your constraints. Platforms-as-a-Service ...................... Platforms-as-a-Service are the easiest option. They provide end-to-end, integrated solutions and they require little configuration. Here's how to deploy on some popular PaaS providers. Since all PaaS use similar patterns, the concepts translate to other providers. .. toctree:: :titlesonly: render koyeb fly heroku Self-hosted infrastructure .......................... If you need more control over your infrastructure, you can deploy on your own infrastructure. This requires more configuration. Here's how to configure some components mentioned in this guide. .. toctree:: :titlesonly: kubernetes supervisor nginx haproxy Running server processes ------------------------ How many processes do I need? ............................. Typically, one server process will manage a few hundreds or thousands connections, depending on the frequency of messages and the amount of work they require. CPU and memory usage increase with the number of connections to the server. Often CPU is the limiting factor. If a server process goes to 100% CPU, then you reached the limit. How much headroom you want to keep is up to you. Once you know how many connections a server process can manage and how many connections you need to handle, you can calculate how many processes to run. You can also automate this calculation by configuring an autoscaler to keep CPU usage or connection count within acceptable limits. .. admonition:: Don't scale with threads. Scale only with processes. :class: tip Threads don't make sense for a server built with :mod:`asyncio`. How do I run processes? ....................... Most solutions for running multiple instances of a server process fall into one of these three buckets: 1. Running N processes on a platform: * a Kubernetes Deployment * its equivalent on a Platform as a Service provider 2. Running N servers: * an AWS Auto Scaling group, a GCP Managed instance group, etc. * a fixed set of long-lived servers 3. Running N processes on a server: * preferably via a process manager or supervisor Option 1 is easiest if you have access to such a platform. Option 2 usually combines with option 3. How do I start a process? ......................... Run a Python program that invokes :func:`~asyncio.server.serve` or :func:`~asyncio.router.route`. That's it! Don't run an ASGI server such as Uvicorn, Hypercorn, or Daphne. They're alternatives to websockets, not complements. Don't run a WSGI server such as Gunicorn, Waitress, or mod_wsgi. They aren't designed to run WebSocket applications. Applications servers handle network connections and expose a Python API. You don't need one because websockets handles network connections directly. How do I stop a process? ........................ Process managers send the SIGTERM signal to terminate processes. Catch this signal and exit the server to ensure a graceful shutdown. Here's an example: .. literalinclude:: ../../example/faq/shutdown_server.py :emphasize-lines: 14-16 When exiting the context manager, :func:`~asyncio.server.serve` closes all connections with code 1001 (going away). As a consequence: * If the connection handler is awaiting :meth:`~asyncio.server.ServerConnection.recv`, it receives a :exc:`~exceptions.ConnectionClosedOK` exception. It can catch the exception and clean up before exiting. * Otherwise, it should be waiting on :meth:`~asyncio.server.ServerConnection.wait_closed`, so it can receive the :exc:`~exceptions.ConnectionClosedOK` exception and exit. This example is easily adapted to handle other signals. If you override the default signal handler for SIGINT, which raises :exc:`KeyboardInterrupt`, be aware that you won't be able to interrupt a program with Ctrl-C anymore when it's stuck in a loop. Routing connections to processes -------------------------------- What does routing involve? .......................... Since the routing layer is directly exposed to the Internet, it should provide appropriate protection against threats ranging from Internet background noise to targeted attacks. You should always secure WebSocket connections with TLS. Since the routing layer carries the public domain name, it should terminate TLS connections. Finally, it must route connections to the server processes, balancing new connections across them. How do I route connections? ........................... Here are typical solutions for load balancing, matched to ways of running processes: 1. If you're running on a platform, it comes with a routing layer: * a Kubernetes Ingress and Service * a service mesh: Istio, Consul, Linkerd, etc. * the routing mesh of a Platform as a Service 2. If you're running N servers, you may load balance with: * a cloud load balancer: AWS Elastic Load Balancing, GCP Cloud Load Balancing, etc. * A software load balancer: HAProxy, NGINX, etc. 3. If you're running N processes on a server, you may load balance with: * A software load balancer: HAProxy, NGINX, etc. * The operating system — all processes listen on the same port You may trust the load balancer to handle encryption and to provide security. You may add another layer in front of the load balancer for these purposes. There are many possibilities. Don't add layers that you don't need, though. How do I implement a health check? .................................. Load balancers need a way to check whether server processes are up and running to avoid routing connections to a non-functional backend. websockets provide minimal support for responding to HTTP requests with the ``process_request`` hook. Here's an example: .. literalinclude:: ../../example/faq/health_check_server.py :emphasize-lines: 7-9,16 --- ## File: docs/deploy/koyeb.rst Deploy to Koyeb ================ This guide describes how to deploy a websockets server to Koyeb_. .. _Koyeb: https://www.koyeb.com .. admonition:: The free tier of Koyeb is sufficient for trying this guide. :class: tip The `free tier`__ include one web service, which this guide uses. __ https://www.koyeb.com/pricing We’re going to deploy a very simple app. The process would be identical to a more realistic app. Create repository ----------------- Koyeb supports multiple deployment methods. Its quick start guides recommend git-driven deployment as the first option. Let's initialize a git repository: .. code-block:: console $ mkdir websockets-echo $ cd websockets-echo $ git init -b main Initialized empty Git repository in websockets-echo/.git/ $ git commit --allow-empty -m "Initial commit." [main (root-commit) 740f699] Initial commit. Render requires the git repository to be hosted at GitHub. Sign up or log in to GitHub. Create a new repository named ``websockets-echo``. Don't enable any of the initialization options offered by GitHub. Then, follow instructions for pushing an existing repository from the command line. After pushing, refresh your repository's homepage on GitHub. You should see an empty repository with an empty initial commit. Create application ------------------ Here’s the implementation of the app, an echo server. Save it in a file called ``app.py``: .. literalinclude:: ../../example/deployment/koyeb/app.py :language: python This app implements typical requirements for running on a Platform as a Service: * it listens on the port provided in the ``$PORT`` environment variable; * it provides a health check at ``/healthz``; * it closes connections and exits cleanly when it receives a ``SIGTERM`` signal; while not documented, this is how Koyeb terminates apps. Create a ``requirements.txt`` file containing this line to declare a dependency on websockets: .. literalinclude:: ../../example/deployment/koyeb/requirements.txt :language: text Create a ``Procfile`` to tell Koyeb how to run the app. .. literalinclude:: ../../example/deployment/koyeb/Procfile Confirm that you created the correct files and commit them to git: .. code-block:: console $ ls Procfile app.py requirements.txt $ git add . $ git commit -m "Initial implementation." [main f634b8b] Initial implementation.  3 files changed, 39 insertions(+)  create mode 100644 Procfile  create mode 100644 app.py  create mode 100644 requirements.txt The app is ready. Let's deploy it! Deploy application ------------------ Sign up or log in to Koyeb. In the Koyeb control panel, create a web service with GitHub as the deployment method. Install and authorize Koyeb's GitHub app if you haven't done that yet. Follow the steps to create a new service: 1. Select the ``websockets-echo`` repository in the list of your repositories. 2. Confirm that the **Free** instance type is selected. Click **Next**. 3. Configure health checks: change the protocol from TCP to HTTP and set the path to ``/healthz``. Review other settings; defaults should be correct. Click **Deploy**. Koyeb builds the app, deploys it, verifies that the health checks passes, and makes the deployment active. Validate deployment ------------------- Let's confirm that your application is running as expected. Since it's a WebSocket server, you need a WebSocket client, such as the interactive client that comes with websockets. If you're currently building a websockets server, perhaps you're already in a virtualenv where websockets is installed. If not, you can install it in a new virtualenv as follows: .. code-block:: console $ python -m venv websockets-client $ . websockets-client/bin/activate $ pip install websockets Look for the URL of your app in the Koyeb control panel. It looks like ``https://--.koyeb.app/``. Connect the interactive client — you must replace ``https`` with ``wss`` in the URL: .. code-block:: console $ websockets wss://--.koyeb.app/ Connected to wss://--.koyeb.app/. > Great! Your app is running! Once you're connected, you can send any message and the server will echo it, or press Ctrl-D to terminate the connection: .. code-block:: console > Hello! < Hello! Connection closed: 1000 (OK). You can also confirm that your application shuts down gracefully. Connect an interactive client again: .. code-block:: console $ websockets wss://--.koyeb.app/ Connected to wss://--.koyeb.app/. > In the Koyeb control panel, go to the **Settings** tab, click **Pause**, and confirm. Eventually, the connection gets closed with code 1001 (going away). .. code-block:: console $ websockets wss://--.koyeb.app/ Connected to wss://--.koyeb.app/. Connection closed: 1001 (going away). If graceful shutdown wasn't working, the server wouldn't perform a closing handshake and the connection would be closed with code 1006 (abnormal closure). --- ## File: docs/deploy/kubernetes.rst Deploy to Kubernetes ==================== This guide describes how to deploy a websockets server to Kubernetes_. It assumes familiarity with Docker and Kubernetes. We're going to deploy a simple app to a local Kubernetes cluster and to ensure that it scales as expected. In a more realistic context, you would follow your organization's practices for deploying to Kubernetes, but you would apply the same principles as far as websockets is concerned. .. _Kubernetes: https://kubernetes.io/ .. _containerize-application: Containerize application ------------------------ Here's the app we're going to deploy. Save it in a file called ``app.py``: .. literalinclude:: ../../example/deployment/kubernetes/app.py This is an echo server with one twist: every message blocks the server for 100ms, which creates artificial starvation of CPU time. This makes it easier to saturate the server for load testing. The app exposes a health check on ``/healthz``. It also provides two other endpoints for testing purposes: ``/inemuri`` will make the app unresponsive for 10 seconds and ``/seppuku`` will terminate it. The quest for the perfect Python container image is out of scope of this guide, so we'll go for the simplest possible configuration instead: .. literalinclude:: ../../example/deployment/kubernetes/Dockerfile After saving this ``Dockerfile``, build the image: .. code-block:: console $ docker build -t websockets-test:1.0 . Test your image by running: .. code-block:: console $ docker run --name run-websockets-test --publish 32080:80 --rm \ websockets-test:1.0 Then, in another shell, in a virtualenv where websockets is installed, connect to the app and check that it echoes anything you send: .. code-block:: console $ websockets ws://localhost:32080/ Connected to ws://localhost:32080/. > Hey there! < Hey there! > Now, in yet another shell, stop the app with: .. code-block:: console $ docker kill -s TERM run-websockets-test Going to the shell where you connected to the app, you can confirm that it shut down gracefully: .. code-block:: console $ websockets ws://localhost:32080/ Connected to ws://localhost:32080/. > Hey there! < Hey there! Connection closed: 1001 (going away). If it didn't, you'd get code 1006 (abnormal closure). Deploy application ------------------ Configuring Kubernetes is even further beyond the scope of this guide, so we'll use a basic configuration for testing, with just one Service_ and one Deployment_: .. literalinclude:: ../../example/deployment/kubernetes/deployment.yaml For local testing, a service of type NodePort_ is good enough. For deploying to production, you would configure an Ingress_. .. _Service: https://kubernetes.io/docs/concepts/services-networking/service/ .. _Deployment: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/ .. _NodePort: https://kubernetes.io/docs/concepts/services-networking/service/#nodeport .. _Ingress: https://kubernetes.io/docs/concepts/services-networking/ingress/ After saving this to a file called ``deployment.yaml``, you can deploy: .. code-block:: console $ kubectl apply -f deployment.yaml service/websockets-test created deployment.apps/websockets-test created Now you have a deployment with one pod running: .. code-block:: console $ kubectl get deployment websockets-test NAME READY UP-TO-DATE AVAILABLE AGE websockets-test 1/1 1 1 10s $ kubectl get pods -l app=websockets-test NAME READY STATUS RESTARTS AGE websockets-test-86b48f4bb7-nltfh 1/1 Running 0 10s You can connect to the service — press Ctrl-D to exit: .. code-block:: console $ websockets ws://localhost:32080/ Connected to ws://localhost:32080/. Connection closed: 1000 (OK). Validate deployment ------------------- First, let's ensure the liveness probe works by making the app unresponsive: .. code-block:: console $ curl http://localhost:32080/inemuri Sleeping for 10s Since we have only one pod, we know that this pod will go to sleep. The liveness probe is configured to run every second. By default, liveness probes time out after one second and have a threshold of three failures. Therefore Kubernetes should restart the pod after at most 5 seconds. Indeed, after a few seconds, the pod reports a restart: .. code-block:: console $ kubectl get pods -l app=websockets-test NAME READY STATUS RESTARTS AGE websockets-test-86b48f4bb7-nltfh 1/1 Running 1 42s Next, let's take it one step further and crash the app: .. code-block:: console $ curl http://localhost:32080/seppuku Terminating The pod reports a second restart: .. code-block:: console $ kubectl get pods -l app=websockets-test NAME READY STATUS RESTARTS AGE websockets-test-86b48f4bb7-nltfh 1/1 Running 2 72s All good — Kubernetes delivers on its promise to keep our app alive! Scale deployment ---------------- Of course, Kubernetes is for scaling. Let's scale — modestly — to 10 pods: .. code-block:: console $ kubectl scale deployment.apps/websockets-test --replicas=10 deployment.apps/websockets-test scaled After a few seconds, we have 10 pods running: .. code-block:: console $ kubectl get deployment websockets-test NAME READY UP-TO-DATE AVAILABLE AGE websockets-test 10/10 10 10 10m Now let's generate load. We'll use this script: .. literalinclude:: ../../example/deployment/kubernetes/benchmark.py We'll connect 500 clients in parallel, meaning 50 clients per pod, and have each client send 6 messages. Since the app blocks for 100ms before responding, if connections are perfectly distributed, we expect a total run time slightly over 50 * 6 * 0.1 = 30 seconds. Let's try it: .. code-block:: console $ ulimit -n 512 $ time python benchmark.py 500 6 python benchmark.py 500 6 2.40s user 0.51s system 7% cpu 36.471 total A total runtime of 36 seconds is in the right ballpark. Repeating this experiment with other parameters shows roughly consistent results, with the high variability you'd expect from a quick benchmark without any effort to stabilize the test setup. Finally, we can scale back to one pod. .. code-block:: console $ kubectl scale deployment.apps/websockets-test --replicas=1 deployment.apps/websockets-test scaled $ kubectl get deployment websockets-test NAME READY UP-TO-DATE AVAILABLE AGE websockets-test 1/1 1 1 15m --- ## File: docs/deploy/nginx.rst Deploy behind nginx =================== This guide demonstrates a way to load balance connections across multiple websockets server processes running on the same machine with nginx_. We'll run server processes with Supervisor as described in :doc:`this guide `. .. _nginx: https://nginx.org/ Run server processes -------------------- Save this app to ``app.py``: .. literalinclude:: ../../example/deployment/nginx/app.py :language: python We'd like nginx to connect to websockets servers via Unix sockets in order to avoid the overhead of TCP for communicating between processes running in the same OS. We start the app with :func:`~websockets.asyncio.server.unix_serve`. Each server process listens on a different socket thanks to an environment variable set by Supervisor to a different value. Save this configuration to ``supervisord.conf``: .. literalinclude:: ../../example/deployment/nginx/supervisord.conf This configuration runs four instances of the app. Install Supervisor and run it: .. code-block:: console $ supervisord -c supervisord.conf -n Configure and run nginx ----------------------- Here's a simple nginx configuration to load balance connections across four processes: .. literalinclude:: ../../example/deployment/nginx/nginx.conf We set ``daemon off`` so we can run nginx in the foreground for testing. Then we combine the `WebSocket proxying`_ and `load balancing`_ guides: * The WebSocket protocol requires HTTP/1.1. We must set the HTTP protocol version to 1.1, else nginx defaults to HTTP/1.0 for proxying. * The WebSocket handshake involves the ``Connection`` and ``Upgrade`` HTTP headers. We must pass them to the upstream explicitly, else nginx drops them because they're hop-by-hop headers. We deviate from the `WebSocket proxying`_ guide because its example adds a ``Connection: Upgrade`` header to every upstream request, even if the original request didn't contain that header. * In the upstream configuration, we set the load balancing method to ``least_conn`` in order to balance the number of active connections across servers. This is best for long running connections. .. _WebSocket proxying: http://nginx.org/en/docs/http/websocket.html .. _load balancing: http://nginx.org/en/docs/http/load_balancing.html Save the configuration to ``nginx.conf``, install nginx, and run it: .. code-block:: console $ nginx -c nginx.conf -p . You can confirm that nginx proxies connections properly: .. code-block:: console $ websockets ws://localhost:8080/ Connected to ws://localhost:8080/. > Hello! < Hello! Connection closed: 1000 (OK). --- ## File: docs/deploy/render.rst Deploy to Render ================ This guide describes how to deploy a websockets server to Render_. .. _Render: https://render.com/ .. admonition:: The free plan of Render is sufficient for trying this guide. :class: tip However, on a `free plan`__, connections are dropped after five minutes, which is quite short for WebSocket application. __ https://render.com/docs/free We're going to deploy a very simple app. The process would be identical for a more realistic app. Create repository ----------------- Deploying to Render requires a git repository. Let's initialize one: .. code-block:: console $ mkdir websockets-echo $ cd websockets-echo $ git init -b main Initialized empty Git repository in websockets-echo/.git/ $ git commit --allow-empty -m "Initial commit." [main (root-commit) 816c3b1] Initial commit. Render requires the git repository to be hosted at GitHub or GitLab. Sign up or log in to GitHub. Create a new repository named ``websockets-echo``. Don't enable any of the initialization options offered by GitHub. Then, follow instructions for pushing an existing repository from the command line. After pushing, refresh your repository's homepage on GitHub. You should see an empty repository with an empty initial commit. Create application ------------------ Here's the implementation of the app, an echo server. Save it in a file called ``app.py``: .. literalinclude:: ../../example/deployment/render/app.py :language: python This app implements requirements for `zero downtime deploys`_: * it provides a health check at ``/healthz``; * it closes connections and exits cleanly when it receives a ``SIGTERM`` signal. .. _zero downtime deploys: https://render.com/docs/deploys#zero-downtime-deploys Create a ``requirements.txt`` file containing this line to declare a dependency on websockets: .. literalinclude:: ../../example/deployment/render/requirements.txt :language: text Confirm that you created the correct files and commit them to git: .. code-block:: console $ ls app.py requirements.txt $ git add . $ git commit -m "Initial implementation." [main f26bf7f] Initial implementation. 2 files changed, 37 insertions(+) create mode 100644 app.py create mode 100644 requirements.txt Push the changes to GitHub: .. code-block:: console $ git push ... To github.com:/websockets-echo.git 816c3b1..f26bf7f main -> main The app is ready. Let's deploy it! Deploy application ------------------ Sign up or log in to Render. Create a new web service. Connect the git repository that you just created. Then, finalize the configuration of your app as follows: * **Name**: websockets-echo * **Start Command**: ``python app.py`` If you're just experimenting, select the free plan. Create the web service. To configure the health check, go to Settings, scroll down to Health & Alerts, and set: * **Health Check Path**: /healthz This triggers a new deployment. Validate deployment ------------------- Let's confirm that your application is running as expected. Since it's a WebSocket server, you need a WebSocket client, such as the interactive client that comes with websockets. If you're currently building a websockets server, perhaps you're already in a virtualenv where websockets is installed. If not, you can install it in a new virtualenv as follows: .. code-block:: console $ python -m venv websockets-client $ . websockets-client/bin/activate $ pip install websockets Connect the interactive client — you must replace ``websockets-echo`` with the name of your Render app in this command: .. code-block:: console $ websockets wss://websockets-echo.onrender.com/ Connected to wss://websockets-echo.onrender.com/. > Great! Your app is running! Once you're connected, you can send any message and the server will echo it, or press Ctrl-D to terminate the connection: .. code-block:: console > Hello! < Hello! Connection closed: 1000 (OK). You can also confirm that your application shuts down gracefully when you deploy a new version. Due to limitations of Render's free plan, you must upgrade to a paid plan before you perform this test. Connect an interactive client again — remember to replace ``websockets-echo`` with your app: .. code-block:: console $ websockets wss://websockets-echo.onrender.com/ Connected to wss://websockets-echo.onrender.com/. > Trigger a new deployment with Manual Deploy > Deploy latest commit. When the deployment completes, the connection is closed with code 1001 (going away). .. code-block:: console $ websockets wss://websockets-echo.onrender.com/ Connected to wss://websockets-echo.onrender.com/. Connection closed: 1001 (going away). If graceful shutdown wasn't working, the server wouldn't perform a closing handshake and the connection would be closed with code 1006 (abnormal closure). Remember to downgrade to a free plan if you upgraded just for testing this feature. --- ## File: docs/deploy/supervisor.rst Deploy with Supervisor ====================== This guide proposes a simple way to deploy a websockets server directly on a Linux or BSD operating system. We'll configure Supervisor_ to run several server processes and to restart them if needed. .. _Supervisor: http://supervisord.org/ We'll bind all servers to the same port. The OS will take care of balancing connections. Create and activate a virtualenv: .. code-block:: console $ python -m venv supervisor-websockets $ . supervisor-websockets/bin/activate Install websockets and Supervisor: .. code-block:: console $ pip install websockets $ pip install supervisor Save this app to a file called ``app.py``: .. literalinclude:: ../../example/deployment/supervisor/app.py This is an echo server with two features added for the purpose of this guide: * It shuts down gracefully when receiving a ``SIGTERM`` signal; * It enables the ``reuse_port`` option of :meth:`~asyncio.loop.create_server`, which in turns sets ``SO_REUSEPORT`` on the accept socket. Save this Supervisor configuration to ``supervisord.conf``: .. literalinclude:: ../../example/deployment/supervisor/supervisord.conf This is the minimal configuration required to keep four instances of the app running, restarting them if they exit. Now start Supervisor in the foreground: .. code-block:: console $ supervisord -c supervisord.conf -n INFO Increased RLIMIT_NOFILE limit to 1024 INFO supervisord started with pid 43596 INFO spawned: 'websockets-test_00' with pid 43597 INFO spawned: 'websockets-test_01' with pid 43598 INFO spawned: 'websockets-test_02' with pid 43599 INFO spawned: 'websockets-test_03' with pid 43600 INFO success: websockets-test_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) INFO success: websockets-test_01 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) INFO success: websockets-test_02 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) INFO success: websockets-test_03 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) In another shell, after activating the virtualenv, we can connect to the app — press Ctrl-D to exit: .. code-block:: console $ websockets ws://localhost:8080/ Connected to ws://localhost:8080/. > Hello! < Hello! Connection closed: 1000 (OK). Look at the pid of an instance of the app in the logs and terminate it: .. code-block:: console $ kill -TERM 43597 The logs show that Supervisor restarted this instance: .. code-block:: console INFO exited: websockets-test_00 (exit status 0; expected) INFO spawned: 'websockets-test_00' with pid 43629 INFO success: websockets-test_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) Now let's check what happens when we shut down Supervisor, but first let's establish a connection and leave it open: .. code-block:: console $ websockets ws://localhost:8080/ Connected to ws://localhost:8080/. > Look at the pid of supervisord itself in the logs and terminate it: .. code-block:: console $ kill -TERM 43596 The logs show that Supervisor terminated all instances of the app before exiting: .. code-block:: console WARN received SIGTERM indicating exit request INFO waiting for websockets-test_00, websockets-test_01, websockets-test_02, websockets-test_03 to die INFO stopped: websockets-test_02 (exit status 0) INFO stopped: websockets-test_03 (exit status 0) INFO stopped: websockets-test_01 (exit status 0) INFO stopped: websockets-test_00 (exit status 0) And you can see that the connection to the app was closed gracefully: .. code-block:: console $ websockets ws://localhost:8080/ Connected to ws://localhost:8080/. Connection closed: 1001 (going away). In this example, we've been sharing the same virtualenv for supervisor and websockets. In a real deployment, you would likely: * Install Supervisor with the package manager of the OS. * Create a virtualenv dedicated to your application. * Add ``environment=PATH="path/to/your/virtualenv/bin"`` in the Supervisor configuration. Then ``python app.py`` runs in that virtualenv.