pingora Documentation

1. Project Overview & Quickstart (cloudflare/pingora) Copy Chapter

File: README.md Copy Chapter

Pingora

What is Pingora Copy Chapter

Pingora is a Rust framework to build fast, reliable and programmable networked systems.

Pingora is battle tested as it has been serving more than 40 million Internet requests per second for more than a few years.

Feature highlights Copy Chapter

• Async Rust: fast and reliable
• HTTP 1/2 end to end proxy
• TLS over OpenSSL, BoringSSL, s2n-tls, or rustls(experimental).
• gRPC and websocket proxying
• Graceful reload
• Customizable load balancing and failover strategies
• Support for a variety of observability tools

Reasons to use Pingora Copy Chapter

• Security is your top priority: Pingora is a more memory safe alternative for services that are written in C/C++
• Your service is performance-sensitive: Pingora is fast and efficient
• Your service requires extensive customization: The APIs Pingora proxy framework provides are highly programmable

Getting started

See our quick starting guide to see how easy it is to build a load balancer.

Our user guide covers more topics such as how to configure and run Pingora servers, as well as how to build custom HTTP servers and proxy logic on top of Pingora's framework.

API docs are also available for all the crates.

Notable crates in this workspace

• Pingora: the "public facing" crate to build networked systems and proxies
• Pingora-core: this crate defines the protocols, functionalities and basic traits
• Pingora-foundations: Foundations telemetry integration for Pingora services
• Pingora-proxy: the logic and APIs to build HTTP proxies
• Pingora-error: the common error type used across Pingora crates
• Pingora-http: the HTTP header definitions and APIs
• Pingora-openssl & pingora-boringssl: SSL related extensions and APIs
• Pingora-ketama: the Ketama consistent algorithm
• Pingora-limits: efficient counting algorithms
• Pingora-load-balancing: load balancing algorithm extensions for pingora-proxy
• Pingora-memory-cache: Async in-memory caching with cache lock to prevent cache stampede
• Pingora-s2n: SSL extensions and APIs related to s2n-tls
• Pingora-timeout: A more efficient async timer system
• TinyUfo: The caching algorithm behind pingora-memory-cache

Note that Pingora proxy integration with caching should be considered experimental, and as such APIs related to caching are currently highly volatile.

System requirements

Systems Copy Chapter

Linux is our tier 1 environment and main focus.

We will try our best for most code to compile for Unix environments. This is for developers and users to have an easier time developing with Pingora in Unix-like environments like macOS (though some features might be missing)

Windows support is preliminary by community's best effort only.

Both x86_64 and aarch64 architectures will be supported.

Rust version Copy Chapter

Pingora keeps a rolling MSRV (minimum supported Rust version) policy of 6 months. This means we will accept PRs that upgrade the MSRV as long as the new Rust version used is at least 6 months old. However, we generally will not bump the highest MSRV across the workspace without a sufficiently compelling reason.

Our current MSRV is 1.85.

Currently not all crates enforce rust-version as it is possible to use some crates on lower versions.

Build Requirements Copy Chapter

Some of the crates in this repository have dependencies on additional tools and
libraries that must be satisfied in order to build them:

• Make sure that Clang is installed on your system (for boringssl)
• Make sure that Perl 5 is installed on your system (for openssl)

Contributing

Please see our contribution guidelines.

License

This project is Licensed under Apache License, Version 2.0.

File: docs/user_guide/conf.md Copy Chapter

Configuration

A Pingora configuration file is a list of Pingora settings in yaml format.

Example

---
version: 1
threads: 2
pid_file: /run/pingora.pid
upgrade_sock: /tmp/pingora_upgrade.sock
user: nobody
group: webusers

Settings Copy Chapter

Offload pools Copy Chapter

The upstream connection offload settings are applied automatically when callers
build ConnectorOptions with ConnectorOptions::from_server_conf().

Downstream TLS handshake offload is applied per TLS listener because each
listener owns its own TlsSettings. Callers that build TLS listeners from a
ServerConf can opt in by applying the config before adding the listener:

use pingora_core::listeners::tls::TlsSettings;
use pingora_core::server::configuration::ServerConf;

fn tls_settings_from_conf(conf: &ServerConf) -> pingora_error::Result<TlsSettings> {
    let mut tls_settings = TlsSettings::intermediate("server.crt", "key.pem")?;
    tls_settings.set_offload_threadpool_from_server_conf(conf);
    Ok(tls_settings)
}

dial9 Copy Chapter

dial9 Tokio runtime telemetry is configured programmatically, not through
the YAML configuration file. This avoids applying experimental telemetry to
every service runtime and lets services provide non-serializable options such
as a pre-built S3 client.

dial9 is only available when Pingora is built with the dial9 feature and
--cfg tokio_unstable. Services can override the global runtime options with
runtime_opts_override():

use pingora::server::{Dial9RuntimeOpts, RuntimeOpts};
use pingora::services::Service;

struct MyService;

impl Service for MyService {
    fn name(&self) -> &str {
        "my-service"
    }

    fn runtime_opts_override(&self, global: &RuntimeOpts) -> Option<RuntimeOpts> {
        let mut opts = global.clone();
        opts.dial9 = Some(
            Dial9RuntimeOpts::new("/var/lib/pingora/dial9/my-service/trace.bin")
                .with_max_file_size(100 * 1024 * 1024)
                .with_max_total_size(512 * 1024 * 1024),
        );
        Some(opts)
    }
}

When built with the dial9-worker-s3 feature, sealed trace segments can also
be uploaded to an S3-compatible bucket:

use pingora::server::{Dial9RuntimeOpts, Dial9S3UploadOpts, RuntimeOpts};
use pingora::services::Service;

struct MyService {
    s3_client: aws_sdk_s3::Client,
}

impl Service for MyService {
    fn name(&self) -> &str {
        "my-service"
    }

    fn runtime_opts_override(&self, global: &RuntimeOpts) -> Option<RuntimeOpts> {
        let mut opts = global.clone();
        opts.dial9 = Some(
            Dial9RuntimeOpts::new("/var/lib/pingora/dial9/my-service/trace.bin")
                .with_s3_upload(
                    Dial9S3UploadOpts::new("my-trace-bucket", "my-service")
                        .with_prefix("traces/my-service")
                        .with_region("us-east-1")
                        .with_client(self.s3_client.clone()),
                ),
        );
        Some(opts)
    }
}

The S3 client is optional. When omitted, dial9 uses the AWS SDK default
configuration chain and its bucket-region detection.

Extension Copy Chapter

Any unknown settings will be ignored. This allows extending the conf file to add and pass user defined settings. See User defined configuration section.

File: docs/user_guide/ctx.md Copy Chapter

Sharing state across phases with `CTX`

Using `CTX` Copy Chapter

The custom filters users implement in different phases of the request don't interact with each other directly. In order to share information and state across the filters, users can define a CTX struct. Each request owns a single CTX object. All the filters are able to read and update members of the CTX object. The CTX object will be dropped at the end of the request.

Example Copy Section

In the following example, the proxy parses the request header in the request_filter phase, it stores the boolean flag so that later in the upstream_peer phase the flag is used to decide which server to route traffic to. (Technically, the header can be parsed in upstream_peer phase, but we just do it in an earlier phase just for the demonstration.)

pub struct MyProxy();

pub struct MyCtx {
    beta_user: bool,
}

fn check_beta_user(req: &pingora_http::RequestHeader) -> bool {
    // some simple logic to check if user is beta
    req.headers.get("beta-flag").is_some()
}

#[async_trait]
impl ProxyHttp for MyProxy {
    type CTX = MyCtx;
    fn new_ctx(&self) -> Self::CTX {
        MyCtx { beta_user: false }
    }

    async fn request_filter(&self, session: &mut Session, ctx: &mut Self::CTX) -> Result {
        ctx.beta_user = check_beta_user(session.req_header());
        Ok(false)
    }

    async fn upstream_peer(
        &self,
        _session: &mut Session,
        ctx: &mut Self::CTX,
    ) -> Result> {
        let addr = if ctx.beta_user {
            info!("I'm a beta user");
            ("1.0.0.1", 443)
        } else {
            ("1.1.1.1", 443)
        };

        let peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string()));
        Ok(peer)
    }
}

Sharing state across requests Copy Chapter

Sharing state such as a counter, cache and other info across requests is common. There is nothing special needed for sharing resources and data across requests in Pingora. Arc, static or any other mechanism can be used.

Example Copy Section

Let's modify the example above to track the number of beta visitors as well as the number of total visitors. The counters can either be defined in the MyProxy struct itself or defined as a global variable. Because the counters can be concurrently accessed, Mutex is used here.

// global counter
static REQ_COUNTER: Mutex = Mutex::new(0);

pub struct MyProxy {
    // counter for the service
    beta_counter: Mutex, // AtomicUsize works too
}

pub struct MyCtx {
    beta_user: bool,
}

fn check_beta_user(req: &pingora_http::RequestHeader) -> bool {
    // some simple logic to check if user is beta
    req.headers.get("beta-flag").is_some()
}

#[async_trait]
impl ProxyHttp for MyProxy {
    type CTX = MyCtx;
    fn new_ctx(&self) -> Self::CTX {
        MyCtx { beta_user: false }
    }

    async fn request_filter(&self, session: &mut Session, ctx: &mut Self::CTX) -> Result {
        ctx.beta_user = check_beta_user(session.req_header());
        Ok(false)
    }

    async fn upstream_peer(
        &self,
        _session: &mut Session,
        ctx: &mut Self::CTX,
    ) -> Result> {
        let mut req_counter = REQ_COUNTER.lock().unwrap();
        *req_counter += 1;

        let addr = if ctx.beta_user {
            let mut beta_count = self.beta_counter.lock().unwrap();
            *beta_count += 1;
            info!("I'm a beta user #{beta_count}");
            ("1.0.0.1", 443)
        } else {
            info!("I'm an user #{req_counter}");
            ("1.1.1.1", 443)
        };

        let peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string()));
        Ok(peer)
    }
}

The complete example can be found under pingora-proxy/examples/ctx.rs. You can run it using cargo:

RUST_LOG=INFO cargo run --example ctx

File: docs/user_guide/daemon.md Copy Chapter

Daemonization

When a Pingora server is configured to run as a daemon, after its bootstrapping, it will move itself to the background and optionally change to run under the configured user and group. The pid_file option comes handy in this case for the user to track the PID of the daemon in the background.

Daemonization also allows the server to perform privileged actions like loading secrets and then switch to an unprivileged user before accepting any requests from the network.

This process happens in the run_forever() call. Because daemonization involves fork(), certain things like threads created before this call are likely lost.

File: docs/user_guide/error_log.md Copy Chapter

Error logging

Pingora libraries are built to expect issues like disconnects, timeouts and invalid inputs from the network. A common way to record these issues are to output them in error log (STDERR or log files).

Log level guidelines Copy Chapter

Pingora adopts the idea behind log. There are five log levels:

• error: This level should be used when the error stops the request from being handled correctly. For example when the server we try to connect to is offline.
• warning: This level should be used when an error occurs but the system recovers from it. For example when the primary DNS timed out but the system is able to query the secondary DNS.
• info: Pingora logs when the server is starting up or shutting down.
• debug: Internal details. This log level is not compiled in release builds.
• trace: Fine-grained internal details. This log level is not compiled in release builds.

The pingora-proxy crate has a well-defined interface to log errors, so that users don't have to manually log common proxy errors. See its guide for more details.

File: docs/user_guide/errors.md Copy Chapter

How to return errors

For easy error handling, the pingora-error crate exports a custom Result type used throughout other Pingora crates.

The Error struct used in this Result's error variant is a wrapper around arbitrary error types. It allows the user to tag the source of the underlying error and attach other custom context info.

Users will often need to return errors by propagating an existing error or creating a wholly new one. pingora-error makes this easy with its error building functions.

Examples Copy Chapter

For example, one could return an error when an expected header is not present:

fn validate_req_header(req: &RequestHeader) -> Result<()> {
    // validate that the `host` header exists
    req.headers()
        .get(http::header::HOST)
        .ok_or_else(|| Error::explain(InvalidHTTPHeader, "No host header detected"))
}

impl MyServer {
    pub async fn handle_request_filter(
        &self,
        http_session: &mut Session,
        ctx: &mut CTX,
    ) -> Result {
        validate_req_header(session.req_header()?).or_err(HTTPStatus(400), "Missing required headers")?;
        Ok(true)
    }
}

validate_req_header returns an Error if the host header is not found, using Error::explain to create a new Error along with an associated type (InvalidHTTPHeader) and helpful context that may be logged in an error log.

This error will eventually propagate to the request filter, where it is returned as a new HTTPStatus error using or_err. (As part of the default pingora-proxy fail_to_proxy() phase, not only will this error be logged, but it will result in sending a 400 Bad Request response downstream.)

Note that the original causing error will be visible in the error logs as well. or_err wraps the original causing error in a new one with additional context, but Error's Display implementation also prints the chain of causing errors.

Guidelines Copy Chapter

An error has a type (e.g. ConnectionClosed), a source (e.g. Upstream, Downstream, Internal), and optionally, a cause (another wrapped error) and a context (arbitrary user-provided string details).

A minimal error can be created using functions like new_in / new_up / new_down, each of which specifies a source and asks the user to provide a type.

Generally speaking:

• To create a new error, without a direct cause but with more context, use Error::explain. You can also use explain_err on a Result to replace the potential error inside it with a new one.
• To wrap a causing error in a new one with more context, use Error::because. You can also use or_err on a Result to replace the potential error inside it by wrapping the original one.

Retry Copy Chapter

Errors can be "retry-able." If the error is retry-able, pingora-proxy will be allowed to retry the upstream request. Some errors are only retry-able on reused connections, e.g. to handle situations where the remote end has dropped a connection we attempted to reuse.

By default a newly created Error either takes on its direct causing error's retry status, or, if left unspecified, is considered not retry-able.

File: docs/user_guide/failover.md Copy Chapter

Handling failures and failover

Pingora-proxy allows users to define how to handle failures throughout the life of a proxied request.

When a failure happens before the response header is sent downstream, users have a few options:

1. Send an error page downstream and then give up.
2. Retry the same upstream again.
3. Try another upstream if applicable.

Otherwise, once the response header is already sent downstream, there is nothing the proxy can do other than logging an error and then giving up on the request.

Retry / Failover Copy Chapter

In order to implement retry or failover, fail_to_connect() / error_while_proxy() needs to mark the error as "retry-able." For failover, fail_to_connect() / error_while_proxy() also needs to update the CTX to tell upstream_peer() not to use the same Peer again.

Safety Copy Section

In general, idempotent HTTP requests, e.g., GET, are safe to retry. Other requests, e.g., POST, are not safe to retry if the requests have already been sent. When fail_to_connect() is called, pingora-proxy guarantees that nothing was sent upstream. Users are not recommended to retry a non-idempotent request after error_while_proxy() unless they know the upstream server enough to know whether it is safe.

Example Copy Section

In the following example we set a tries variable on the CTX to track how many connection attempts we've made. When setting our peer in upstream_peer we check if tries is less than one and connect to 192.0.2.1. On connect failure we increment tries in fail_to_connect and set e.set_retry(true) which tells Pingora this is a retryable error. On retry, we enter upstream_peer again and this time connect to 1.1.1.1. If we're unable to connect to 1.1.1.1 we return a 502 since we only set e.set_retry(true) in fail_to_connect when tries is zero.

pub struct MyProxy();

pub struct MyCtx {
    tries: usize,
}

#[async_trait]
impl ProxyHttp for MyProxy {
    type CTX = MyCtx;
    fn new_ctx(&self) -> Self::CTX {
        MyCtx { tries: 0 }
    }

    fn fail_to_connect(
        &self,
        _session: &mut Session,
        _peer: &HttpPeer,
        ctx: &mut Self::CTX,
        mut e: Box<Error>,
    ) -> Box<Error> {
        if ctx.tries > 0 {
            return e;
        }
        ctx.tries += 1;
        e.set_retry(true);
        e
    }

    async fn upstream_peer(
        &self,
        _session: &mut Session,
        ctx: &mut Self::CTX,
    ) -> Result> {
        let addr = if ctx.tries < 1 {
            ("192.0.2.1", 443)
        } else {
            ("1.1.1.1", 443)
        };

        let mut peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string()));
        peer.options.connection_timeout = Some(Duration::from_millis(100));
        Ok(peer)
    }
}

File: docs/user_guide/graceful.md Copy Chapter

Graceful restart and shutdown

Graceful restart, upgrade, and shutdown mechanisms are very commonly used to avoid errors or downtime when releasing new versions of Pingora servers.

Pingora graceful upgrade mechanism guarantees the following:

• A request is guaranteed to be handled either by the old server instance or the new one. No request will see connection refused when trying to connect to the server endpoints.
• A request that can finish within the grace period is guaranteed not to be terminated.

How to graceful upgrade Copy Chapter

Step 0 Copy Section

Configure the upgrade socket. The old and new server need to agree on the same path to this socket. See configuration manual for details.

Step 1 Copy Section

Start the new instance with the --upgrade CLI option. The new instance will not try to listen to the service endpoint right away. It will try to acquire the listening socket from the old instance instead.

Step 2 Copy Section

Send SIGQUIT signal to the old instance. The old instance will start to transfer the listening socket to the new instance.

Once step 2 is successful, the new instance will start to handle new incoming connections right away. Meanwhile, the old instance will enter its graceful shutdown mode. It waits a short period of time (to give the new instance time to initialize and prepare to handle traffic), after which it will not accept any new connections.

File: docs/user_guide/index.md Copy Chapter

User Guide

In this guide, we will cover the most used features, operations and settings of Pingora.

Running Pingora servers Copy Chapter

• Start and stop
• Graceful restart and graceful shutdown
• Configuration
• Daemonization
• Systemd integration
• Handling panics
• Error logging
• Prometheus

Building HTTP proxies Copy Chapter

• Life of a request: pingora-proxy phases and filters
• Peer: how to connect to upstream
• Sharing state across phases and connections with CTX (including tracking active connection counts)
• How to return errors
• Examples: take control of the request
• Connection pooling and reuse
• Handling failures and failover
• RateLimiter quickstart

Advanced topics (WIP) Copy Chapter

• Pingora internals
• Using BoringSSL
• User defined configuration
• Pingora async runtime and threading model
• Background Service
• Blocking code in async context
• Tracing

File: docs/user_guide/internals.md Copy Chapter

Pingora Internals

(Special thanks to James Munns for writing this section)

Starting the `Server` Copy Chapter

The pingora system starts by spawning a server. The server is responsible for starting services, and listening for termination events.

┌───────────┐
                    ┌─────────>│  Service  │
                    │          └───────────┘
┌────────┐          │          ┌───────────┐
│ Server │──Spawns──┼─────────>│  Service  │
└────────┘          │          └───────────┘
                    │          ┌───────────┐
                    └─────────>│  Service  │
                               └───────────┘

After spawning the services, the server continues to listen to a termination event, which it will propagate to the created services.

Services Copy Chapter

Services are entities that handle listening to given sockets, and perform the core functionality. A service is tied to a particular protocol and set of options.

NOTE: there are also "background" services, which just do stuff, and aren't necessarily listening to a socket. For now we're just talking about listener services.

Each service has its own threadpool/tokio runtime, with a number of threads based on the configured value. Worker threads are not shared cross-service. Service runtime threadpools may be work-stealing (tokio-default), or non-work-stealing (N isolated single threaded runtimes).

┌─────────────────────────┐
│ ┌─────────────────────┐ │
│ │┌─────────┬─────────┐│ │
│ ││  Conn   │  Conn   ││ │
│ │├─────────┼─────────┤│ │
│ ││Endpoint │Endpoint ││ │
│ │├─────────┴─────────┤│ │
│ ││     Listeners     ││ │
│ │├─────────┬─────────┤│ │
│ ││ Worker  │ Worker  ││ │
│ ││ Thread  │ Thread  ││ │
│ │├─────────┴─────────┤│ │
│ ││  Tokio Executor   ││ │
│ │└───────────────────┘│ │
│ └─────────────────────┘ │
│ ┌───────┐               │
└─┤Service├───────────────┘
  └───────┘

Service Listeners Copy Chapter

At startup, each Service is assigned a set of downstream endpoints that they listen to. A single service may listen to more than one endpoint. The Server also passes along any relevant configuration, including TLS settings if relevant.

These endpoints are converted into listening sockets, called TransportStacks. Each TransportStack is assigned to an async task within that service's executor.

/* Detailed source-code truncated for AI context efficiency. */

Downstream connection lifecycle Copy Chapter

Each service processes incoming connections by spawning a task-per-connection. These connections are held open
as long as there are new events to be handled.

┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐

                                  │  ┌───────────────┐   ┌────────────────┐   ┌─────────────────┐    ┌─────────────┐  │
┌────────────────────┐               │ UninitStream  │   │    Service     │   │       App       │    │  Task Ends  │
│                    │            │  │ ::handshake() │──>│::handle_event()│──>│ ::process_new() │──┬>│             │  │
│ Service<ServerApp> │──spawn()──>   └───────────────┘   └────────────────┘   └─────────────────┘  │ └─────────────┘
│                    │            │                                                    ▲           │                  │
└────────────────────┘                                                                 │         while
                                  │                                                    └─────────reuse                │
                                     ┌───────────────────────────┐
                                  └ ─│  Task on Service Runtime  │─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘
                                     └───────────────────────────┘

What is a proxy then? Copy Chapter

Interestingly, the pingora Server itself has no particular notion of a Proxy.

Instead, it only thinks in terms of Services, which are expected to contain a particular implementor of the ServiceApp trait.

For example, this is how an HttpProxy struct, from the pingora-proxy crate, "becomes" a Service spawned by the Server:

┌─────────────┐
│  HttpProxy  │
│  (struct)   │
└─────────────┘
       │
   implements   ┌─────────────┐
       │        │HttpServerApp│
       └───────>│   (trait)   │
                └─────────────┘
                       │
                   implements   ┌─────────────┐
                       │        │  ServerApp  │
                       └───────>│   (trait)   │
                                └─────────────┘
                                       │
                                   contained    ┌─────────────────────┐
                                     within     │                     │
                                       └───────>│ Service<ServiceApp> │
                                                │                     │
                                                └─────────────────────┘

Different functionalities and helpers are provided at different layers in this representation.

┌─────────────┐        ┌──────────────────────────────────────┐
│  HttpProxy  │        │Handles high level Proxying workflow, │
│  (struct)   │─ ─ ─ ─ │   customizable via ProxyHttp trait   │
└──────┬──────┘        └──────────────────────────────────────┘
       │
┌──────▼──────┐        ┌──────────────────────────────────────┐
│HttpServerApp│        │ Handles selection of H1 vs H2 stream │
│   (trait)   │─ ─ ─ ─ │     handling, incl H2 handshake      │
└──────┬──────┘        └──────────────────────────────────────┘
       │
┌──────▼──────┐        ┌──────────────────────────────────────┐
│  ServerApp  │        │ Handles dispatching of App instances │
│   (trait)   │─ ─ ─ ─ │   as individual tasks, per Session   │
└──────┬──────┘        └──────────────────────────────────────┘
       │
┌──────▼──────┐        ┌──────────────────────────────────────┐
│ Service<A>  │        │ Handles dispatching of App instances │
│  (struct)   │─ ─ ─ ─ │  as individual tasks, per Listener   │
└─────────────┘        └──────────────────────────────────────┘

The HttpProxy struct handles the high level workflow of proxying an HTTP connection

It uses the ProxyHttp (note the flipped wording order!) trait to allow customization
at each of the following steps (note: taken from the phase chart doc):

graph TD;
    start("new request")-->request_filter;
    request_filter-->upstream_peer;

    upstream_peer-->Connect{{IO: connect to upstream}};

    Connect--connection success-->connected_to_upstream;
    Connect--connection failure-->fail_to_connect;

    connected_to_upstream-->upstream_request_filter;
    upstream_request_filter --> SendReq{{IO: send request to upstream}};
    SendReq-->RecvResp{{IO: read response from upstream}};
    RecvResp-->upstream_response_filter-->response_filter-->upstream_response_body_filter-->response_body_filter-->logging-->endreq("request done");

    fail_to_connect --can retry-->upstream_peer;
    fail_to_connect --can't retry-->fail_to_proxy--send error response-->logging;

    RecvResp--failure-->IOFailure;
    SendReq--failure-->IOFailure;
    error_while_proxy--can retry-->upstream_peer;
    error_while_proxy--can't retry-->fail_to_proxy;

    request_filter --send response-->logging


    Error>any response filter error]-->error_while_proxy
    IOFailure>IO error]-->error_while_proxy

Zooming out Copy Chapter

Before we zoom in, it's probably good to zoom out and remind ourselves how
a proxy generally works:

┌────────────┐          ┌─────────────┐         ┌────────────┐
│ Downstream │          │    Proxy    │         │  Upstream  │
│   Client   │─────────>│             │────────>│   Server   │
└────────────┘          └─────────────┘         └────────────┘

The proxy will be taking connections from the Downstream client, and (if
everything goes right), establishing a connection with the appropriate
Upstream server. This selected upstream server is referred to as
the Peer.

Once the connection is established, the Downstream and Upstream can communicate
bidirectionally.

So far, the discussion of Server, Services, and Listeners have focused on the LEFT
half of this diagram, handling incoming Downstream connections, and getting it TO
the proxy component.

Next, we'll look at the RIGHT half of this diagram, connecting to Upstreams.

Managing the Upstream Copy Chapter

Connections to Upstream Peers are made through Connectors. This is not a specific type or trait, but more
of a "style".

Connectors are responsible for a few things:

• Establishing a connection with a Peer
• Maintaining a connection pool with the Peer, allowing for connection reuse across:
  • Multiple requests from a single downstream client
  • Multiple requests from different downstream clients
• Measuring health of connections, for connections like H2, which perform regular pings
• Handling protocols with multiple poolable layers, like H2
• Caching, if relevant to the protocol and enabled
• Compression, if relevant to the protocol and enabled

Now in context, we can see how each end of the Proxy is handled:

┌────────────┐          ┌─────────────┐         ┌────────────┐
│ Downstream │       ┌ ─│─   Proxy  ┌ ┼ ─       │  Upstream  │
│   Client   │─────────>│ │           │──┼─────>│   Server   │
└────────────┘       │  └───────────┼─┘         └────────────┘
                      ─ ─ ┘          ─ ─ ┘
                        ▲              ▲
                     ┌──┘              └──┐
                     │                    │
                ┌ ─ ─ ─ ─ ┐         ┌ ─ ─ ─ ─ ─
                 Listeners           Connectors│
                └ ─ ─ ─ ─ ┘         └ ─ ─ ─ ─ ─

What about multiple peers? Copy Chapter

Connectors only handle the connection to a single peer, so selecting one of potentially multiple Peers
is actually handled one level up, in the upstream_peer() method of the ProxyHttp trait.

2. Official Technical Reference & Guides (cloudflare/docs) Copy Chapter

AMPProject.org [](https://travis-ci.org/ampproject/docs)

You're looking at the source code of the official website of the AMP Project.
Even if you don't plan on contributing, explore how we've built the site for
a good example of how to build a canonical, responsive standalone AMP site.

How to build the site Copy Chapter

Install Copy Section

Install Grow and npm dependencies:

$ curl https://install.growsdk.org | bash
$ npm install

Get a GitHub token or app id/secret Copy Section

The docs in the reference section of the AMP Project site are not part of this repository; instead they are read from the amphtml GitHub repository "source of truth." When building the site these docs are fetched via GitHub.

Due to GitHub quotas this fetching will quickly fail unless additional information is provided in the request. This can be either a personal access token or an application id/secret.

Personal access token

1. Request a GitHub personal access token. Since the docs are fetched from a public repository you only need to select the "public_repo" scope when creating the token. (It's a good practice to limit the token to the scope you need.)
2. In your shell export the generated personal access token as AMP_DOC_TOKEN

Application client id and secret

If you prefer not to use a personal access token, you can also create a GitHub application and use its id and secret.

1. Register for a GitHub application
2. In your shell export the application client id as AMP_DOC_ID and the client secret as AMP_DOC_SECRET

Build Copy Section

$ grow build

This will generate a static, complete build of the site into the *build folder.

Develop Copy Section

Note: Be sure to run grow build at least once to pull in reference docs before running the following command.

$ grow run

You can now open http://localhost:8080/ and continue working on the source files, then reload the page to see changes appear.

Support Copy Chapter

If you've found an error or inconsistency, please file an issue:
https://github.com/ampproject/docs/issues

Patches are encouraged, and may be submitted by forking this project and
submitting a pull request through GitHub.

License Copy Chapter

All image and audio files except in folders "source/img/partners",
"source/img/publishers" and "source/img/quotes" (including *.png, *.jpg, *.svg,
*.mp3, *.wav and *.ogg) are licensed under the CC-BY-NC license. Images in the
excluded folders are not licensed.

All other files are licensed under the Apache 2 license.

Copyright 2015 Google, Inc.

Licensed to the Apache Software Foundation (ASF) under one or more contributor
license agreements. See the NOTICE file distributed with this work for
additional information regarding copyright ownership. The ASF licenses this
file to you under the Apache License, Version 2.0 (the "License"); you may not
use this file except in compliance with the License. You may obtain a copy of
the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations under
the License.