## 1. Project Overview & Quickstart (cloudflare/pingora) ## File: README.md # Pingora ## What is Pingora Pingora is a Rust framework to [build fast, reliable and programmable networked systems](https://blog.cloudflare.com/pingora-open-source). Pingora is battle tested as it has been serving more than 40 million Internet requests per second for [more than a few years](https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet). ## Feature highlights * Async Rust: fast and reliable * HTTP 1/2 end to end proxy * TLS over OpenSSL, BoringSSL, s2n-tls, or rustls(experimental). * gRPC and websocket proxying * Graceful reload * Customizable load balancing and failover strategies * Support for a variety of observability tools ## Reasons to use Pingora * **Security** is your top priority: Pingora is a more memory safe alternative for services that are written in C/C++ * Your service is **performance-sensitive**: Pingora is fast and efficient * Your service requires extensive **customization**: The APIs Pingora proxy framework provides are highly programmable # Getting started See our [quick starting guide](./docs/quick_start.md) to see how easy it is to build a load balancer. Our [user guide](./docs/user_guide/index.md) covers more topics such as how to configure and run Pingora servers, as well as how to build custom HTTP servers and proxy logic on top of Pingora's framework. API docs are also available for all the crates. # Notable crates in this workspace * Pingora: the "public facing" crate to build networked systems and proxies * Pingora-core: this crate defines the protocols, functionalities and basic traits * Pingora-foundations: Foundations telemetry integration for Pingora services * Pingora-proxy: the logic and APIs to build HTTP proxies * Pingora-error: the common error type used across Pingora crates * Pingora-http: the HTTP header definitions and APIs * Pingora-openssl & pingora-boringssl: SSL related extensions and APIs * Pingora-ketama: the [Ketama](https://github.com/RJ/ketama) consistent algorithm * Pingora-limits: efficient counting algorithms * Pingora-load-balancing: load balancing algorithm extensions for pingora-proxy * Pingora-memory-cache: Async in-memory caching with cache lock to prevent cache stampede * Pingora-s2n: SSL extensions and APIs related to s2n-tls * Pingora-timeout: A more efficient async timer system * TinyUfo: The caching algorithm behind pingora-memory-cache Note that Pingora proxy integration with caching should be considered experimental, and as such APIs related to caching are currently highly volatile. # System requirements ## Systems Linux is our tier 1 environment and main focus. We will try our best for most code to compile for Unix environments. This is for developers and users to have an easier time developing with Pingora in Unix-like environments like macOS (though some features might be missing) Windows support is preliminary by community's best effort only. Both x86_64 and aarch64 architectures will be supported. ## Rust version Pingora keeps a rolling MSRV (minimum supported Rust version) policy of 6 months. This means we will accept PRs that upgrade the MSRV as long as the new Rust version used is at least 6 months old. However, we generally will not bump the highest MSRV across the workspace without a sufficiently compelling reason. Our current MSRV is 1.85. Currently not all crates enforce `rust-version` as it is possible to use some crates on lower versions. ## Build Requirements Some of the crates in this repository have dependencies on additional tools and libraries that must be satisfied in order to build them: * Make sure that [Clang] is installed on your system (for boringssl) * Make sure that [Perl 5] is installed on your system (for openssl) [Clang]:https://clang.llvm.org/ [Perl 5]:https://www.perl.org/ # Contributing Please see our [contribution guidelines](./.github/CONTRIBUTING.md). # License This project is Licensed under [Apache License, Version 2.0](./LICENSE). --- ## File: docs/user_guide/conf.md # Configuration A Pingora configuration file is a list of Pingora settings in yaml format. Example ```yaml --- version: 1 threads: 2 pid_file: /run/pingora.pid upgrade_sock: /tmp/pingora_upgrade.sock user: nobody group: webusers ``` ## Settings | Key | meaning | value type | | ------------- |-------------| ----| | version | the version of the conf, currently it is a constant `1` | number | | pid_file | The path to the pid file | string | | daemon | whether to run the server in the background | bool | | error_log | the path to error log output file. STDERR is used if not set | string | | upgrade_sock | the path to the upgrade socket. | string | | threads | number of threads per service | number | | user | the user the pingora server should be run under after daemonization | string | | group | the group the pingora server should be run under after daemonization | string | | working_directory | the working directory for the daemonized process | string | | client_bind_to_ipv4 | source IPv4 addresses to bind to when connecting to server | list of string | | client_bind_to_ipv6 | source IPv6 addresses to bind to when connecting to server| list of string | | ca_file | The path to the root CA file | string | | s2n_config_cache_size | The maximum number of unique s2n configs to cache. A value of 0 disables the cache. Default: 10 (s2n-tls only) | number | | work_stealing | Enable work stealing runtime (default true). See Pingora runtime (WIP) section for more info | bool | | runtime_enable_alt_timer | Enable Tokio's experimental alternative timer on work-stealing service runtimes. Requires building with `--cfg tokio_unstable`. Ignored when `work_stealing` is disabled. Default: `false` | bool | | fast_timeout_to_tokio_threshold_seconds | Timeout durations greater than this value use Tokio's native timeout instead of Pingora's fast timeout. Default: `900`. Set to `null` to disable the Tokio fallback. | number | | runtime_metrics_poll_time_histogram | Enable Tokio poll-time histograms on service runtimes. Requires building with `--cfg tokio_unstable`; adds two timestamp reads to every task poll. Default: `false` | bool | | runtime_metrics_poll_time_histogram_scale | Bucket scale for Tokio poll-time histograms. Valid values: `linear`, `log`. Ignored unless `runtime_metrics_poll_time_histogram` is enabled. | string | | runtime_metrics_poll_time_histogram_resolution_micros | Width of the first Tokio poll-time histogram bucket in microseconds. Must be greater than 0. Ignored unless `runtime_metrics_poll_time_histogram` is enabled. | number | | runtime_metrics_poll_time_histogram_buckets | Number of Tokio poll-time histogram buckets. Must be greater than 0 and at most 1024. Memory usage scales with runtimes × workers × buckets. Ignored unless `runtime_metrics_poll_time_histogram` is enabled. | number | | upstream_keepalive_pool_size | The number of idle upstream connections to keep per tokio worker. The pool's effective ceiling is `upstream_keepalive_pool_size × threads`. Eviction is globally consistent across workers. | number | | upstream_connect_offload_threadpools | Number of dedicated thread pools to use for upstream connection establishment. Enabled only when `upstream_connect_offload_thread_per_pool` is also set and both values are greater than 0. | number | | upstream_connect_offload_thread_per_pool | Number of threads per dedicated upstream connection establishment pool. Enabled only when `upstream_connect_offload_threadpools` is also set and both values are greater than 0. | number | | downstream_tls_offload_threadpools | Number of dedicated thread pools to use for downstream TLS handshakes. Enabled only when `downstream_tls_offload_thread_per_pool` is also set and both values are greater than 0. | number | | downstream_tls_offload_thread_per_pool | Number of threads per dedicated downstream TLS handshake pool. Enabled only when `downstream_tls_offload_threadpools` is also set and both values are greater than 0. | number | | daemon_wait_for_ready | When `true` and `daemon` is `true`, the parent process waits for the daemon to signal readiness (via `SIGUSR1`) before exiting. This causes systemd to delay sending `SIGQUIT` to the old process until the new instance is fully bootstrapped. Default: `false` | bool | | daemon_ready_timeout_seconds | How long (in seconds) the parent waits for the daemon to signal readiness when `daemon_wait_for_ready` is `true`. If the daemon does not signal in time the parent exits with a non-zero code, causing systemd to abort the reload. Default: `600` | number | | daemon_notify_timeout_seconds | How long (in seconds) the daemon retries sending `SIGUSR1` to the parent when the attempt fails with a permission error. This covers the brief window after the fork where the parent has not yet dropped its UID to match the daemon. Default: `60` | number | ## Offload pools The upstream connection offload settings are applied automatically when callers build `ConnectorOptions` with `ConnectorOptions::from_server_conf()`. Downstream TLS handshake offload is applied per TLS listener because each listener owns its own `TlsSettings`. Callers that build TLS listeners from a `ServerConf` can opt in by applying the config before adding the listener: ```rust,no_run use pingora_core::listeners::tls::TlsSettings; use pingora_core::server::configuration::ServerConf; fn tls_settings_from_conf(conf: &ServerConf) -> pingora_error::Result { let mut tls_settings = TlsSettings::intermediate("server.crt", "key.pem")?; tls_settings.set_offload_threadpool_from_server_conf(conf); Ok(tls_settings) } ``` ## dial9 dial9 Tokio runtime telemetry is configured programmatically, not through the YAML configuration file. This avoids applying experimental telemetry to every service runtime and lets services provide non-serializable options such as a pre-built S3 client. dial9 is only available when Pingora is built with the `dial9` feature and `--cfg tokio_unstable`. Services can override the global runtime options with `runtime_opts_override()`: ```rust use pingora::server::{Dial9RuntimeOpts, RuntimeOpts}; use pingora::services::Service; struct MyService; impl Service for MyService { fn name(&self) -> &str { "my-service" } fn runtime_opts_override(&self, global: &RuntimeOpts) -> Option { let mut opts = global.clone(); opts.dial9 = Some( Dial9RuntimeOpts::new("/var/lib/pingora/dial9/my-service/trace.bin") .with_max_file_size(100 * 1024 * 1024) .with_max_total_size(512 * 1024 * 1024), ); Some(opts) } } ``` When built with the `dial9-worker-s3` feature, sealed trace segments can also be uploaded to an S3-compatible bucket: ```rust use pingora::server::{Dial9RuntimeOpts, Dial9S3UploadOpts, RuntimeOpts}; use pingora::services::Service; struct MyService { s3_client: aws_sdk_s3::Client, } impl Service for MyService { fn name(&self) -> &str { "my-service" } fn runtime_opts_override(&self, global: &RuntimeOpts) -> Option { let mut opts = global.clone(); opts.dial9 = Some( Dial9RuntimeOpts::new("/var/lib/pingora/dial9/my-service/trace.bin") .with_s3_upload( Dial9S3UploadOpts::new("my-trace-bucket", "my-service") .with_prefix("traces/my-service") .with_region("us-east-1") .with_client(self.s3_client.clone()), ), ); Some(opts) } } ``` The S3 client is optional. When omitted, dial9 uses the AWS SDK default configuration chain and its bucket-region detection. ## Extension Any unknown settings will be ignored. This allows extending the conf file to add and pass user defined settings. See User defined configuration section. --- ## File: docs/user_guide/ctx.md # Sharing state across phases with `CTX` ## Using `CTX` The custom filters users implement in different phases of the request don't interact with each other directly. In order to share information and state across the filters, users can define a `CTX` struct. Each request owns a single `CTX` object. All the filters are able to read and update members of the `CTX` object. The CTX object will be dropped at the end of the request. ### Example In the following example, the proxy parses the request header in the `request_filter` phase, it stores the boolean flag so that later in the `upstream_peer` phase the flag is used to decide which server to route traffic to. (Technically, the header can be parsed in `upstream_peer` phase, but we just do it in an earlier phase just for the demonstration.) ```Rust pub struct MyProxy(); pub struct MyCtx { beta_user: bool, } fn check_beta_user(req: &pingora_http::RequestHeader) -> bool { // some simple logic to check if user is beta req.headers.get("beta-flag").is_some() } #[async_trait] impl ProxyHttp for MyProxy { type CTX = MyCtx; fn new_ctx(&self) -> Self::CTX { MyCtx { beta_user: false } } async fn request_filter(&self, session: &mut Session, ctx: &mut Self::CTX) -> Result { ctx.beta_user = check_beta_user(session.req_header()); Ok(false) } async fn upstream_peer( &self, _session: &mut Session, ctx: &mut Self::CTX, ) -> Result> { let addr = if ctx.beta_user { info!("I'm a beta user"); ("1.0.0.1", 443) } else { ("1.1.1.1", 443) }; let peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string())); Ok(peer) } } ``` ## Sharing state across requests Sharing state such as a counter, cache and other info across requests is common. There is nothing special needed for sharing resources and data across requests in Pingora. `Arc`, `static` or any other mechanism can be used. ### Example Let's modify the example above to track the number of beta visitors as well as the number of total visitors. The counters can either be defined in the `MyProxy` struct itself or defined as a global variable. Because the counters can be concurrently accessed, Mutex is used here. ```Rust // global counter static REQ_COUNTER: Mutex = Mutex::new(0); pub struct MyProxy { // counter for the service beta_counter: Mutex, // AtomicUsize works too } pub struct MyCtx { beta_user: bool, } fn check_beta_user(req: &pingora_http::RequestHeader) -> bool { // some simple logic to check if user is beta req.headers.get("beta-flag").is_some() } #[async_trait] impl ProxyHttp for MyProxy { type CTX = MyCtx; fn new_ctx(&self) -> Self::CTX { MyCtx { beta_user: false } } async fn request_filter(&self, session: &mut Session, ctx: &mut Self::CTX) -> Result { ctx.beta_user = check_beta_user(session.req_header()); Ok(false) } async fn upstream_peer( &self, _session: &mut Session, ctx: &mut Self::CTX, ) -> Result> { let mut req_counter = REQ_COUNTER.lock().unwrap(); *req_counter += 1; let addr = if ctx.beta_user { let mut beta_count = self.beta_counter.lock().unwrap(); *beta_count += 1; info!("I'm a beta user #{beta_count}"); ("1.0.0.1", 443) } else { info!("I'm an user #{req_counter}"); ("1.1.1.1", 443) }; let peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string())); Ok(peer) } } ``` The complete example can be found under [`pingora-proxy/examples/ctx.rs`](../../pingora-proxy/examples/ctx.rs). You can run it using `cargo`: ``` RUST_LOG=INFO cargo run --example ctx ``` --- ## File: docs/user_guide/daemon.md # Daemonization When a Pingora server is configured to run as a daemon, after its bootstrapping, it will move itself to the background and optionally change to run under the configured user and group. The `pid_file` option comes handy in this case for the user to track the PID of the daemon in the background. Daemonization also allows the server to perform privileged actions like loading secrets and then switch to an unprivileged user before accepting any requests from the network. This process happens in the `run_forever()` call. Because daemonization involves `fork()`, certain things like threads created before this call are likely lost. --- ## File: docs/user_guide/error_log.md # Error logging Pingora libraries are built to expect issues like disconnects, timeouts and invalid inputs from the network. A common way to record these issues are to output them in error log (STDERR or log files). ## Log level guidelines Pingora adopts the idea behind [log](https://docs.rs/log/latest/log/). There are five log levels: * `error`: This level should be used when the error stops the request from being handled correctly. For example when the server we try to connect to is offline. * `warning`: This level should be used when an error occurs but the system recovers from it. For example when the primary DNS timed out but the system is able to query the secondary DNS. * `info`: Pingora logs when the server is starting up or shutting down. * `debug`: Internal details. This log level is not compiled in `release` builds. * `trace`: Fine-grained internal details. This log level is not compiled in `release` builds. The pingora-proxy crate has a well-defined interface to log errors, so that users don't have to manually log common proxy errors. See its guide for more details. --- ## File: docs/user_guide/errors.md # How to return errors For easy error handling, the `pingora-error` crate exports a custom `Result` type used throughout other Pingora crates. The `Error` struct used in this `Result`'s error variant is a wrapper around arbitrary error types. It allows the user to tag the source of the underlying error and attach other custom context info. Users will often need to return errors by propagating an existing error or creating a wholly new one. `pingora-error` makes this easy with its error building functions. ## Examples For example, one could return an error when an expected header is not present: ```rust fn validate_req_header(req: &RequestHeader) -> Result<()> { // validate that the `host` header exists req.headers() .get(http::header::HOST) .ok_or_else(|| Error::explain(InvalidHTTPHeader, "No host header detected")) } impl MyServer { pub async fn handle_request_filter( &self, http_session: &mut Session, ctx: &mut CTX, ) -> Result { validate_req_header(session.req_header()?).or_err(HTTPStatus(400), "Missing required headers")?; Ok(true) } } ``` `validate_req_header` returns an `Error` if the `host` header is not found, using `Error::explain` to create a new `Error` along with an associated type (`InvalidHTTPHeader`) and helpful context that may be logged in an error log. This error will eventually propagate to the request filter, where it is returned as a new `HTTPStatus` error using `or_err`. (As part of the default pingora-proxy `fail_to_proxy()` phase, not only will this error be logged, but it will result in sending a `400 Bad Request` response downstream.) Note that the original causing error will be visible in the error logs as well. `or_err` wraps the original causing error in a new one with additional context, but `Error`'s `Display` implementation also prints the chain of causing errors. ## Guidelines An error has a _type_ (e.g. `ConnectionClosed`), a _source_ (e.g. `Upstream`, `Downstream`, `Internal`), and optionally, a _cause_ (another wrapped error) and a _context_ (arbitrary user-provided string details). A minimal error can be created using functions like `new_in` / `new_up` / `new_down`, each of which specifies a source and asks the user to provide a type. Generally speaking: * To create a new error, without a direct cause but with more context, use `Error::explain`. You can also use `explain_err` on a `Result` to replace the potential error inside it with a new one. * To wrap a causing error in a new one with more context, use `Error::because`. You can also use `or_err` on a `Result` to replace the potential error inside it by wrapping the original one. ## Retry Errors can be "retry-able." If the error is retry-able, pingora-proxy will be allowed to retry the upstream request. Some errors are only retry-able on [reused connections](pooling.md), e.g. to handle situations where the remote end has dropped a connection we attempted to reuse. By default a newly created `Error` either takes on its direct causing error's retry status, or, if left unspecified, is considered not retry-able. --- ## File: docs/user_guide/failover.md # Handling failures and failover Pingora-proxy allows users to define how to handle failures throughout the life of a proxied request. When a failure happens before the response header is sent downstream, users have a few options: 1. Send an error page downstream and then give up. 2. Retry the same upstream again. 3. Try another upstream if applicable. Otherwise, once the response header is already sent downstream, there is nothing the proxy can do other than logging an error and then giving up on the request. ## Retry / Failover In order to implement retry or failover, `fail_to_connect()` / `error_while_proxy()` needs to mark the error as "retry-able." For failover, `fail_to_connect() / error_while_proxy()` also needs to update the `CTX` to tell `upstream_peer()` not to use the same `Peer` again. ### Safety In general, idempotent HTTP requests, e.g., `GET`, are safe to retry. Other requests, e.g., `POST`, are not safe to retry if the requests have already been sent. When `fail_to_connect()` is called, pingora-proxy guarantees that nothing was sent upstream. Users are not recommended to retry a non-idempotent request after `error_while_proxy()` unless they know the upstream server enough to know whether it is safe. ### Example In the following example we set a `tries` variable on the `CTX` to track how many connection attempts we've made. When setting our peer in `upstream_peer` we check if `tries` is less than one and connect to 192.0.2.1. On connect failure we increment `tries` in `fail_to_connect` and set `e.set_retry(true)` which tells Pingora this is a retryable error. On retry, we enter `upstream_peer` again and this time connect to 1.1.1.1. If we're unable to connect to 1.1.1.1 we return a 502 since we only set `e.set_retry(true)` in `fail_to_connect` when `tries` is zero. ```Rust pub struct MyProxy(); pub struct MyCtx { tries: usize, } #[async_trait] impl ProxyHttp for MyProxy { type CTX = MyCtx; fn new_ctx(&self) -> Self::CTX { MyCtx { tries: 0 } } fn fail_to_connect( &self, _session: &mut Session, _peer: &HttpPeer, ctx: &mut Self::CTX, mut e: Box, ) -> Box { if ctx.tries > 0 { return e; } ctx.tries += 1; e.set_retry(true); e } async fn upstream_peer( &self, _session: &mut Session, ctx: &mut Self::CTX, ) -> Result> { let addr = if ctx.tries < 1 { ("192.0.2.1", 443) } else { ("1.1.1.1", 443) }; let mut peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string())); peer.options.connection_timeout = Some(Duration::from_millis(100)); Ok(peer) } } ``` --- ## File: docs/user_guide/graceful.md # Graceful restart and shutdown Graceful restart, upgrade, and shutdown mechanisms are very commonly used to avoid errors or downtime when releasing new versions of Pingora servers. Pingora graceful upgrade mechanism guarantees the following: * A request is guaranteed to be handled either by the old server instance or the new one. No request will see connection refused when trying to connect to the server endpoints. * A request that can finish within the grace period is guaranteed not to be terminated. ## How to graceful upgrade ### Step 0 Configure the upgrade socket. The old and new server need to agree on the same path to this socket. See configuration manual for details. ### Step 1 Start the new instance with the `--upgrade` CLI option. The new instance will not try to listen to the service endpoint right away. It will try to acquire the listening socket from the old instance instead. ### Step 2 Send SIGQUIT signal to the old instance. The old instance will start to transfer the listening socket to the new instance. Once step 2 is successful, the new instance will start to handle new incoming connections right away. Meanwhile, the old instance will enter its graceful shutdown mode. It waits a short period of time (to give the new instance time to initialize and prepare to handle traffic), after which it will not accept any new connections. --- ## File: docs/user_guide/index.md # User Guide In this guide, we will cover the most used features, operations and settings of Pingora. ## Running Pingora servers * [Start and stop](start_stop.md) * [Graceful restart and graceful shutdown](graceful.md) * [Configuration](conf.md) * [Daemonization](daemon.md) * [Systemd integration](systemd.md) * [Handling panics](panic.md) * [Error logging](error_log.md) * [Prometheus](prom.md) ## Building HTTP proxies * [Life of a request: `pingora-proxy` phases and filters](phase.md) * [`Peer`: how to connect to upstream](peer.md) * [Sharing state across phases and connections with `CTX`](ctx.md) (including tracking active connection counts) * [How to return errors](errors.md) * [Examples: take control of the request](modify_filter.md) * [Connection pooling and reuse](pooling.md) * [Handling failures and failover](failover.md) * [RateLimiter quickstart](rate_limiter.md) ## Advanced topics (WIP) * [Pingora internals](internals.md) * Using BoringSSL * User defined configuration * Pingora async runtime and threading model * Background Service * Blocking code in async context * Tracing --- ## File: docs/user_guide/internals.md # Pingora Internals (Special thanks to [James Munns](https://github.com/jamesmunns) for writing this section) ## Starting the `Server` The pingora system starts by spawning a *server*. The server is responsible for starting *services*, and listening for termination events. ``` ┌───────────┐ ┌─────────>│ Service │ │ └───────────┘ ┌────────┐ │ ┌───────────┐ │ Server │──Spawns──┼─────────>│ Service │ └────────┘ │ └───────────┘ │ ┌───────────┐ └─────────>│ Service │ └───────────┘ ``` After spawning the *services*, the server continues to listen to a termination event, which it will propagate to the created services. ## Services *Services* are entities that handle listening to given sockets, and perform the core functionality. A *service* is tied to a particular protocol and set of options. > NOTE: there are also "background" services, which just do *stuff*, and aren't necessarily listening to a socket. For now we're just talking about listener services. Each service has its own threadpool/tokio runtime, with a number of threads based on the configured value. Worker threads are not shared cross-service. Service runtime threadpools may be work-stealing (tokio-default), or non-work-stealing (N isolated single threaded runtimes). ``` ┌─────────────────────────┐ │ ┌─────────────────────┐ │ │ │┌─────────┬─────────┐│ │ │ ││ Conn │ Conn ││ │ │ │├─────────┼─────────┤│ │ │ ││Endpoint │Endpoint ││ │ │ │├─────────┴─────────┤│ │ │ ││ Listeners ││ │ │ │├─────────┬─────────┤│ │ │ ││ Worker │ Worker ││ │ │ ││ Thread │ Thread ││ │ │ │├─────────┴─────────┤│ │ │ ││ Tokio Executor ││ │ │ │└───────────────────┘│ │ │ └─────────────────────┘ │ │ ┌───────┐ │ └─┤Service├───────────────┘ └───────┘ ``` ## Service Listeners At startup, each Service is assigned a set of downstream endpoints that they listen to. A single service may listen to more than one endpoint. The Server also passes along any relevant configuration, including TLS settings if relevant. These endpoints are converted into listening sockets, called `TransportStack`s. Each `TransportStack` is assigned to an async task within that service's executor. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Downstream connection lifecycle Each service processes incoming connections by spawning a task-per-connection. These connections are held open as long as there are new events to be handled. ``` ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ ┌───────────────┐ ┌────────────────┐ ┌─────────────────┐ ┌─────────────┐ │ ┌────────────────────┐ │ UninitStream │ │ Service │ │ App │ │ Task Ends │ │ │ │ │ ::handshake() │──>│::handle_event()│──>│ ::process_new() │──┬>│ │ │ │ Service │──spawn()──> └───────────────┘ └────────────────┘ └─────────────────┘ │ └─────────────┘ │ │ │ ▲ │ │ └────────────────────┘ │ while │ └─────────reuse │ ┌───────────────────────────┐ └ ─│ Task on Service Runtime │─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ └───────────────────────────┘ ``` ## What is a proxy then? Interestingly, the `pingora` `Server` itself has no particular notion of a Proxy. Instead, it only thinks in terms of `Service`s, which are expected to contain a particular implementor of the `ServiceApp` trait. For example, this is how an `HttpProxy` struct, from the `pingora-proxy` crate, "becomes" a `Service` spawned by the `Server`: ``` ┌─────────────┐ │ HttpProxy │ │ (struct) │ └─────────────┘ │ implements ┌─────────────┐ │ │HttpServerApp│ └───────>│ (trait) │ └─────────────┘ │ implements ┌─────────────┐ │ │ ServerApp │ └───────>│ (trait) │ └─────────────┘ │ contained ┌─────────────────────┐ within │ │ └───────>│ Service │ │ │ └─────────────────────┘ ``` Different functionalities and helpers are provided at different layers in this representation. ``` ┌─────────────┐ ┌──────────────────────────────────────┐ │ HttpProxy │ │Handles high level Proxying workflow, │ │ (struct) │─ ─ ─ ─ │ customizable via ProxyHttp trait │ └──────┬──────┘ └──────────────────────────────────────┘ │ ┌──────▼──────┐ ┌──────────────────────────────────────┐ │HttpServerApp│ │ Handles selection of H1 vs H2 stream │ │ (trait) │─ ─ ─ ─ │ handling, incl H2 handshake │ └──────┬──────┘ └──────────────────────────────────────┘ │ ┌──────▼──────┐ ┌──────────────────────────────────────┐ │ ServerApp │ │ Handles dispatching of App instances │ │ (trait) │─ ─ ─ ─ │ as individual tasks, per Session │ └──────┬──────┘ └──────────────────────────────────────┘ │ ┌──────▼──────┐ ┌──────────────────────────────────────┐ │ Service │ │ Handles dispatching of App instances │ │ (struct) │─ ─ ─ ─ │ as individual tasks, per Listener │ └─────────────┘ └──────────────────────────────────────┘ ``` The `HttpProxy` struct handles the high level workflow of proxying an HTTP connection It uses the `ProxyHttp` (note the flipped wording order!) **trait** to allow customization at each of the following steps (note: taken from [the phase chart](./phase_chart.md) doc): ```mermaid graph TD; start("new request")-->request_filter; request_filter-->upstream_peer; upstream_peer-->Connect{{IO: connect to upstream}}; Connect--connection success-->connected_to_upstream; Connect--connection failure-->fail_to_connect; connected_to_upstream-->upstream_request_filter; upstream_request_filter --> SendReq{{IO: send request to upstream}}; SendReq-->RecvResp{{IO: read response from upstream}}; RecvResp-->upstream_response_filter-->response_filter-->upstream_response_body_filter-->response_body_filter-->logging-->endreq("request done"); fail_to_connect --can retry-->upstream_peer; fail_to_connect --can't retry-->fail_to_proxy--send error response-->logging; RecvResp--failure-->IOFailure; SendReq--failure-->IOFailure; error_while_proxy--can retry-->upstream_peer; error_while_proxy--can't retry-->fail_to_proxy; request_filter --send response-->logging Error>any response filter error]-->error_while_proxy IOFailure>IO error]-->error_while_proxy ``` ## Zooming out Before we zoom in, it's probably good to zoom out and remind ourselves how a proxy generally works: ``` ┌────────────┐ ┌─────────────┐ ┌────────────┐ │ Downstream │ │ Proxy │ │ Upstream │ │ Client │─────────>│ │────────>│ Server │ └────────────┘ └─────────────┘ └────────────┘ ``` The proxy will be taking connections from the **Downstream** client, and (if everything goes right), establishing a connection with the appropriate **Upstream** server. This selected upstream server is referred to as the **Peer**. Once the connection is established, the Downstream and Upstream can communicate bidirectionally. So far, the discussion of Server, Services, and Listeners have focused on the LEFT half of this diagram, handling incoming Downstream connections, and getting it TO the proxy component. Next, we'll look at the RIGHT half of this diagram, connecting to Upstreams. ## Managing the Upstream Connections to Upstream Peers are made through `Connector`s. This is not a specific type or trait, but more of a "style". Connectors are responsible for a few things: * Establishing a connection with a Peer * Maintaining a connection pool with the Peer, allowing for connection reuse across: * Multiple requests from a single downstream client * Multiple requests from different downstream clients * Measuring health of connections, for connections like H2, which perform regular pings * Handling protocols with multiple poolable layers, like H2 * Caching, if relevant to the protocol and enabled * Compression, if relevant to the protocol and enabled Now in context, we can see how each end of the Proxy is handled: ``` ┌────────────┐ ┌─────────────┐ ┌────────────┐ │ Downstream │ ┌ ─│─ Proxy ┌ ┼ ─ │ Upstream │ │ Client │─────────>│ │ │──┼─────>│ Server │ └────────────┘ │ └───────────┼─┘ └────────────┘ ─ ─ ┘ ─ ─ ┘ ▲ ▲ ┌──┘ └──┐ │ │ ┌ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ Listeners Connectors│ └ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ``` ## What about multiple peers? `Connectors` only handle the connection to a single peer, so selecting one of potentially multiple Peers is actually handled one level up, in the `upstream_peer()` method of the `ProxyHttp` trait. ## 2. Official Technical Reference & Guides (cloudflare/docs) AMPProject.org [](https://travis-ci.org/ampproject/docs) ========================= You're looking at the source code of the official website of the AMP Project. Even if you don't plan on contributing, explore how we've built the site for a good example of how to build a canonical, responsive standalone AMP site. How to build the site --------------------- ### Install Install [Grow](http://grow.io) and npm dependencies: ```sh $ curl https://install.growsdk.org | bash $ npm install ``` ### Get a GitHub token or app id/secret The docs in the reference section of the AMP Project site are not part of this repository; instead they are read from the [amphtml GitHub repository](https://github.com/ampproject/amphtml) "source of truth." When building the site these docs are fetched via GitHub. Due to GitHub quotas this fetching will quickly fail unless additional information is provided in the request. This can be either a personal access token or an application id/secret. #### Personal access token 1. Request a GitHub [personal access token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/). Since the docs are fetched from a public repository you only need to select the "public_repo" scope when creating the token. (It's a good practice to limit the token to the scope you need.) 2. In your shell export the generated personal access token as `AMP_DOC_TOKEN` #### Application client id and secret If you prefer not to use a personal access token, you can also create a GitHub application and use its id and secret. 1. Register for a [GitHub application](https://github.com/settings/applications/new) 2. In your shell export the application client id as `AMP_DOC_ID` and the client secret as `AMP_DOC_SECRET` ### Build ```sh $ grow build ``` This will generate a static, complete build of the site into the **build* folder. ### Develop Note: Be sure to run `grow build` at least once to pull in reference docs before running the following command. ```sh $ grow run ``` You can now open http://localhost:8080/ and continue working on the source files, then reload the page to see changes appear. Support ------- If you've found an error or inconsistency, please file an issue: https://github.com/ampproject/docs/issues Patches are encouraged, and may be submitted by forking this project and submitting a pull request through GitHub. License ------- All image and audio files except in folders "source/img/partners", "source/img/publishers" and "source/img/quotes" (including *.png, *.jpg, *.svg, *.mp3, *.wav and *.ogg) are licensed under the CC-BY-NC license. Images in the excluded folders are not licensed. All other files are licensed under the Apache 2 license. - - - Copyright 2015 Google, Inc. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.