### README (README.md) ## [linkerd.io](linkerd.io) Source code for the linkerd.io website. ### General development instructions 1. Install Hugo: for Mac users: ```bash brew install hugo ``` for Linux users: download the **extended** release of Hugo from [GitHub](https://github.com/gohugoio/hugo/releases), then install the following dependencies: ```bash curl https://htmltest.wjdp.uk | bash npm install markdownlint-cli ``` make sure they are added to `$PATH` 1. From the root `/website` directory, build site and run Hugo in development mode: ```bash hugo serve -s linkerd.io ``` 1. Also in the root directory, run checks on any changes you make: ```bash make check lint ``` You should see the site on localhost:1313, and it should reload automatically upon file write. ### To change the way the site looks #### CSS/HTML Images and static CSS and JavaScript files are located in the `static` directory. These files are served as-is. Some of the site's CSS, however, is generated from [Sass](https://sass-lang.com) sources in `assets/scss` by Hugo. When you change those files, Hugo updates the CSS for the site automatically and refreshes the page. The files in layouts/ are the HTML for the site (including layout/index.html for the front page.) These files are Go templates with a couple extra Hugo goodies thrown in. See [the hugo documentation](http://gohugo.io/templates/overview/) for details. If you're running `hugo server` (see above) then updates should be reflected in your browser window via some fancy javascript magic. ## [run.linkerd.io](run.linkerd.io) Install scripts for linkerd as well as demo applications such as [emojivoto][emojivoto]. ## [versioncheck.linkerd.io](versioncheck.linkerd.io) Location for existing installations to check and see if they're up to date or not. To build and serve against the latest Linkerd2 release: ```bash make serve-versioncheck.linkerd.io ``` ## [api.linkerd.io](api.linkerd.io) API docs for linkerd1. Note: this does not deploy by default as part of `make publish`. It needs to be released separately. ### Updating docs from Linkerd See [slate documentation](slate-linkerd) `./build` will grab whatever's on master from slate-linkerd and add it to the public dir. ### Creating a new release 1. From the [linkerd.io](linkerd.io) directory, run: ```bash ./release-next-version ``` 1. Run `./release-next-version ` in [slate-linkerd](https://github.com/BuoyantIO/slate-linkerd) 1. Then check locally ```bash make serve-api.linkerd.io ``` 1. Finally push to production ```bash make deploy-api.linkerd.io ``` ## Publishing 1. Make sure your gcloud tooling is up to date: ```bash gcloud components update gcloud auth login ``` 1. Do a dry run and make sure everything works: ```bash make publish DRY_RUN=true ``` 1. Update the website: ```bash make publish ``` ### Notes - This does not update api.linkerd.io, see the section for that specifically to update it. - There is no caching in front of run.linkerd.io and versioncheck.linkerd.io. You should see updates there immediately. - There is caching for non-html pages in front of linkerd.io. If you're updating a non-html page for linkerd.io, it might be worth flushing the cache (cloudflare) and waiting awhile. ## If you have to create a new bucket You probably won't have to do this, but if you do, don't forget to do this stuff too to set up the bucket for public serving: ```bash gsutil defacl ch -u AllUsers:R gs://bucketname gsutil web set -m index.html -e 404.html gs://bucketname ``` ## Verifying cache updates Turn off all caching on all files: ```bash gsutil -m setmeta -r -h "Cache-Control: no-cache, no-store, must-revalidate" gs://linkerd.io/ ``` Turn caching back on: ```bash gsutil -m setmeta -r -h "Cache-Control:" gs://linkerd.io/ ``` ## Enable access logs (should only need to be run once) ```bash gsutil logging set on -b gs://linkerd2-access-logs -o AccessLog gs://linkerd.io ``` ## Get access logs ```bash # note: this will download ALL logs. probably not what you want. gsutil -m rsync gs://linkerd2-access-logs logs/ ``` ## Set CORS policy (should only need to be run once) ```bash gsutil cors set versioncheck.linkerd.io.cors.json gs://versioncheck.linkerd.io ``` [emojivoto]: https://github.com/BuoyantIO/emojivoto [slate-linkerd]: https://github.com/BuoyantIO/slate-linkerd/blob/master/BUOYANT-README.md --- ### CONTRIBUTING (CONTRIBUTING.md) # Contributing to the Linkerd website and documentation # :balloon: Thanks for your help improving the project! ## Getting Help ## If you have a question about Linkerd or have encountered problems using it, start by [asking a question in the forums][discourse] or join us on [Slack][slack]. ## Certificate of Origin ## By contributing to this project you agree to the Developer Certificate of Origin (DCO). This document was created by the Linux Kernel community and is a simple statement that you, as a contributor, have the legal right to make the contribution. See the [DCO](DCO) file for details. In practice, just add a line to every git commit message: Signed-off-by: Jane Smith Use your real name (sorry, no pseudonyms or anonymous contributions). If you set your user.name and user.email git configs, you can sign your commit automatically with git commit -s. ## Submitting a Pull Request ## Do you have an improvement? 1. Submit an [issue][issue] describing your proposed change. 2. We will try to respond to your issue promptly. 3. Fork this repo, develop and test your code changes. See the project's [README](README.md) for further information about working in this repository. 4. Submit a pull request against this repo's `master` branch. 5. Your branch may be merged once all configured checks pass, including: - 2 code review approvals, at least 1 of which is from a [linkerd organization member][members]. - The branch has passed tests in CI. ## Committing ## We prefer squash or rebase commits so that all changes from a branch are committed to master as a single commit. All pull requests are squashed when merged, but rebasing prior to merge gives you better control over the commit message. ### Commit messages ### Finalized commit messages should be in the following format: ```text Subject Problem Solution Fixes #[Github issue ID] ``` #### Subject #### - one line, <= 50 characters - describe what is done; not the result - use the active voice - capitalize first word and proper nouns - do not end in a period — this is a title/subject - reference the github issue by number ##### Examples ##### ```text bad: server disconnects should cause dst client disconnects. good: Propagate disconnects from source to destination ``` ```text bad: support tls servers good: Introduce support for server-side TLS (#347) ``` #### Problem #### Explain the context and why you're making that change. What is the problem you're trying to solve? In some cases there is not a problem and this can be thought of as being the motivation for your change. #### Solution #### Describe the modifications you've made. [discourse]: https://discourse.linkerd.io/ [issue]: https://github.com/linkerd/linkerd/issues/new [members]: https://github.com/orgs/linkerd/people [slack]: https://slack.linkerd.io/ --- ### MAINTAINERS (MAINTAINERS.md) The Linkerd maintainers are: * Oliver Gould @olix0r (super-maintainer) * Kevin Lingerfelt @klingerf (super-maintainer) * Thomas Rampelberg @grampelberg: docs --- ### Linkerd.Io/Package.Json (linkerd.io/package.json) { "name": "linkerd.io", "version": "1.0.0", "description": "Linkerd.io Public Website", "main": "index.js", "scripts": { "pa11y": "pa11y-ci --sitemap http://127.0.0.1:1313/sitemap.xml --sitemap-find //localhost:1313 --sitemap-replace http://127.0.0.1:1313", "lint": "prettier --check content/*.* {content,themes}/**/*.{js,css,md}", "markdownlint": "markdownlint content", "autofix": "prettier --write content/*.md content/**/*.md" }, "repository": { "type": "git", "url": "git+https://github.com/linkerdio/website.git" }, "author": "Heneise Consulting (http://heneise.co)", "license": "ISC", "bugs": { "url": "https://github.com/linkerdio/website/issues" }, "homepage": "https://github.com/linkerdio/website#readme", "devDependencies": { "markdownlint-cli": "^0.14.0", "pa11y-ci": "^2.1.1", "prettier": "^1.16.4" }, "prettier": { "semi": false, "singleQuote": true, "printWidth": 80, "proseWrap": "never" } } --- ### Linkerd.Io/.Htmltest.Yml (linkerd.io/.htmltest.yml) IgnoreURLs: - .*localhost.* - https://expedia.com - https://github.com/linkerd/website/edit/master/linkerd.io/content/.* - https://ticketmaster.com - https://douban.com/ DirectoryPath: public --- ### Linkerd.Io/.Markdownlint.Yaml (linkerd.io/.markdownlint.yaml) first-line-h1: level: 2 first-heading-h1: level: 2 line_length: code_blocks: false tables: false no-trailing-punctuation: punctuation: ".,;:" no-duplicate-header: false --- ### Linkerd.Io/Archetypes/Dashboard (linkerd.io/archetypes/dashboard.md) +++ date = "{{ .Date }}" title = "{{ replace .Name "-" " " | title }}" +++ --- ### Linkerd.Io/Content/Choose Your Platform (linkerd.io/content/choose-your-platform.md) +++ title = "Choose Your Platform" layout = "cyp" [sitemap] priority = 1.0 +++ Linkerd comes in two parallel branches of development: 2.x and 1.x. Both are maintained in parallel, but certain platforms are only supported by certain versions. Choose your platform below: --- ### Linkerd.Io/Content/Community (linkerd.io/content/community.md) +++ title = "Community" layout = "community" type = "page" +++ --- ### Linkerd.Io/Content/Enterprise (linkerd.io/content/enterprise.md) +++ title = "Enterprise" description = "Lists companies that provide commercial support for Linkerd." weight = 1 +++ Below is a list of companies that offer commercial support and other enterprise offerings for Linkerd: [][bent] Buoyant is the original creator of Linkerd and provides support, training, and enterprise products. [Learn more »][bent] [bent]: https://buoyant.io/enterprise --- ### Linkerd.Io/Content/Gsoc (linkerd.io/content/gsoc.md) +++ title = "Google Summer of Code" +++ This is the list of ideas for students wishing to apply to Google Summer of Code. For more information on what the program is and how to apply, see the [student guide](https://google.github.io/gsocguides/student/). If you're interested in applying we would love to get to know you more on [Slack](https://slack.linkerd.io/). The most successful projects are often those proposed by the students themselves. The following list represents some of our ideas and wishes for the project. However, suggesting your own idea is always encouraged. Jump over to [Slack](hhttps://slack.linkerd.io/) and chat with us or create an [issue](https://github.com/linkerd/linkerd2/issues)! --- ## Cross-cloud integration testing With the proliferation of managed Kubernetes services on many cloud platforms (GKE, AKS, EKS, Kubernetes on DigitalOcean), the subtle differences between these providers can create hard to debug and understand issues. This project involves building out the tooling to create clusters on multiple providers, interact with those and run the integration test suite on them. It will surface bugs earlier, make it easier to replicate user issues and provide a common framework to build sample workloads on top of. ### Expected outcomes * Tooling that automates creating (and destroying) clusters with specific configuration on each of the major providers. * Configuration with TravisCI that runs the integration tests in parallel. ### Recommended Skills * Bash * TravisCI * Go * Cloud Providers ### Difficulty level Medium ### Issue [https://github.com/linkerd/linkerd2/issues/2213](https://github.com/linkerd/linkerd2/issues/2213) --- ## Auto-Update Linkerd has frequent updates and keeping up with the weekly edge releases can be difficult. This project involves building a Kubernetes operator that can observe the version-check API, auto-update the control plane and replace the Linkerd data plane proxies with the correct version. ### Expected outcomes * An operator that watches the version-check API and triggers an update on changes. * Flexible upgrade plans that are saved for each version to describe ordering and how to move between versions. * Validation that each step of an upgrade completes successfully. * Rolling modification of existing data plane proxies to the latest version based off annotations. ### Recommended Skills * Go * Kubernetes ### Difficulty level Hard ### Issue [https://github.com/linkerd/linkerd2/issues/1903](https://github.com/linkerd/linkerd2/issues/1903) --- ## Conformance Validation Linkerd has an extensive `check` suite that validates a cluster is ready to install Linkerd and that the install was successful. These checks are, unfortunately, static checks. Because of the wide number of ways a Kubernetes cluster can be configured, users want a way to validate their specific install for conformance over time. This project involves building a sample application that exercises all of the features of Linkerd and allows an end user to run it on their own cluster to validate that everything is working and configured correctly over a long period of time time. ### Expected outcomes * Sample application that exercises all Linkerd features. * CLI integration to start/stop validation. * Reporting to describe what is not working for users. ### Recommended Skills * Go * Bash * Kubernetes * gRPC ### Difficulty level Medium ### Issue [https://github.com/linkerd/linkerd2/issues/1096](https://github.com/linkerd/linkerd2/issues/1096) --- ## Alertmanager Integration Linkerd provides rich metrics that are stored in Prometheus out of the box. These are for both the control plane and data plane. The goal is to provide Alertmanager integration that comes out of the box, is configurable with preferred channels (email, slack) and works with ServiceProfiles to easily create alerts that are per-service and per-route. ### Expected outcomes * Alertmanager installation as part of the control plane. * Default alerting policy for the control plane. * Simple configuration for desired alert channel. * Integration with ServiceProfiles to setup rules to alert in Alertmanager. ### Recommended Skills * Go * Prometheus * Grafana ### Difficulty level Easy ### Issue [https://github.com/linkerd/linkerd2/issues/1726](https://github.com/linkerd/linkerd2/issues/1726) --- ## Kafka Introspection HTTP-based traffic is only one type of communication in modern applications. Many use message queues such as Kafka. Getting metrics for consumers/producers/messages is just as critical to application health as metrics for requests and responses in HTTP. The goal of this project is to implement a Kafka codec for the Linkerd proxy that allows it to introspect the Kafka protocol and provide metrics for that protocol. ### Expected outcomes * Proxy instrumentation for the Kafka protocol. * Integration with the Linkerd CLI, dashboards. * Visualization of the topology between HTTP services and Kafka-based consumers. ### Recommended Skills * Go * Rust * Kubernetes * Kafka ### Difficulty level Hard ### Issue [https://github.com/linkerd/linkerd2/issues/2214](https://github.com/linkerd/linkerd2/issues/2214) --- ### Linkerd.Io/Content/1/Administration/ Index (linkerd.io/content/1/administration/_index.md) +++ title = "Overview" description = "Includes detailed information about advanced topics such as service discovery, routing, delegation, writing plugins, and diagnosing network performance issues." weight = 1 aliases = [ "/getting-started/administration", "/getting-started/admin", "/administration" ] [menu.docs] name = "Administration" identifier = "administration" weight = 3 +++ This section describes some of the major features included in Linkerd's admin interface. For a complete list of available endpoints, refer to [the config documentation]({{% linkerdconfig "administrative-interface" %}}). {{% sectiontoc "administration" %}} --- ### Linkerd.Io/Content/1/Administration/Dashboard (linkerd.io/content/1/administration/dashboard.md) +++ title = "Dashboard" description = "Admin UI showing graphs of requests, success rate, latency and more." weight = 2 aliases = [ "/administration/dashboard" ] [menu.docs] parent = "administration" +++ Linkerd runs an administrative web interface on port 9990. If you have Linkerd running locally, simply visit `http://localhost:9990/` to view it. The dashboard displays request volume, success rate, connection information, and latency metrics for all of your configured routers, as well as for all of the clients that Linkerd has built to dynamically route your requests. The graphs update in real time, so you can get an immediate sense of the health of your services. {{< fig src="/images/linkerd-dashboard.png" title="Linkerd admin UI." >}} --- ### Linkerd.Io/Content/1/Administration/Dtab Playground (linkerd.io/content/1/administration/dtab-playground.md) +++ title = "Dtab playground" description = "A web UI that you can use to help debug Dtab rules." weight = 4 aliases = [ "/administration/dtab-playground" ] [menu.docs] parent = "administration" +++ The admin interface also provides a web UI that you can use to help debug [Dtab]({{% ref "/1/advanced/dtabs.md" %}}) rules that are set up on a running Linkerd instance. This provides valuable insight into how Linkerd will route your request. The UI is available at `/delegator`, on the configured admin port. {{< fig src="/images/dtab-playground.png" title="Linkerd admin UI - Dtab playground." >}} --- ### Linkerd.Io/Content/1/Administration/Shutdown (linkerd.io/content/1/administration/shutdown.md) +++ title = "Shutdown" description = "Gracefully shut down Linkerd." weight = 5 aliases = [ "/administration/shutdown" ] [menu.docs] parent = "administration" +++ You can gracefully shut down Linkerd by sending a POST request to `/admin/shutown`. For example: ```bash curl -X POST http://localhost:9990/admin/shutdown ``` --- ### Linkerd.Io/Content/1/Administration/Telemetry (linkerd.io/content/1/administration/telemetry.md) +++ title = "Telemetry" description = "Describes how to handle metrics exported by Linkerd." weight = 3 aliases = [ "/administration/telemetry" ] [menu.docs] parent = "administration" +++ Linkerd also publishes machine-readable versions of its metrics in multiple formats. These metrics are designed to be polled by external metrics-collection utilities and sent to backends such as [Prometheus](https://prometheus.io/), [InfluxDB](https://www.influxdata.com/), and [StatsD](https://github.com/etsy/statsd). All of the collected metrics are available as JSON using the `/admin/metrics.json` endpoint. For instance if you have Linkerd running locally, you can run: ```bash $ curl -s http://localhost:9990/admin/metrics.json?pretty=1 | head -n4 { "clnt/zipkin-tracer/available" : 1.0, "clnt/zipkin-tracer/cancelled_connects" : 0, "clnt/zipkin-tracer/closes" : 294, ... ``` Note that the `pretty=1` param is only required for formatting. To enable additional metrics endpoints, such as Prometheus, InfluxDB, or StatsD, have a look at the [Telemetry section of the Linkerd config]({{% linkerdconfig "telemetry" %}}). ## Prometheus Linkerd provides a metrics endpoint, `/admin/metrics/prometheus`, specifically for exporting stats to Prometheus. To enable the Prometheus telemeter, add this to your Linkerd configuration file: ```yaml telemetry: - kind: io.l5d.prometheus ``` You can configure Prometheus to collect stats automatically from your Linkerd instances by using that endpoint as part of your Prometheus scrape config. For instance: ```yaml global: scrape_interval: 15s scrape_configs: - job_name: 'linkerd' metrics_path: /admin/metrics/prometheus static_configs: - targets: - '1.2.3.4:9990' - '2.3.4.5:9990' - '3.4.5.6:9990' ``` That configuration would scrape metrics from three separate Linkerd instances. ## InfluxDB Linkerd provides a metrics endpoint, `/admin/metrics/influxdb`, specifically for exporting stats in InfluxDB LINE protocol. You can configure [Telegraf](https://github.com/influxdata/telegraf) to automatically collect stats from your Linkerd instances. Have a look at the [InfluxDB section of the linkerd-examples repo](https://github.com/linkerd/linkerd-examples/tree/master/influxdb) for a complete example. ## StatsD Linkerd supports pushing metrics to a StatsD backend. Simply add a StatsD config block to your Linkerd configuration file: ```yaml telemetry: - kind: io.l5d.statsd experimental: true prefix: linkerd hostname: 127.0.0.1 port: 8125 gaugeIntervalMs: 10000 sampleRate: 0.01 ``` --- ### Linkerd.Io/Content/1/Advanced/ Index (linkerd.io/content/1/advanced/_index.md) +++ title = "Overview" description = "Includes detailed information about advanced topics such as service discovery, routing, delegation, writing plugins, and diagnosing network performance issues." weight = 1 aliases = [ "/in-depth/", "/advanced" ] [menu.docs] name = "Advanced" identifier = "advanced" weight = 7 +++ This section of the documentation contains in-depth information about advanced topics pertaining to both Linkerd and namerd. It is divided into the following subsections: {{% sectiontoc "advanced" %}} The [Linkerd Discourse forum](https://discourse.linkerd.io/) also documents: * [Network Performance](https://discourse.linkerd.io/t/debugging-network-performance/54): Common approaches to diagnosing network performance issues that can negatively impact Linkerd. * [Debugging a Linkerd setup](https://discourse.linkerd.io/t/debugging-a-linkerd-setup/52): Common approaches to finding and fixing Linkerd configuration issues that may prevent it from working properly in your setup. --- ### Linkerd.Io/Content/1/Advanced/Deployment (linkerd.io/content/1/advanced/deployment.md) +++ title = "Deployment" description = "Addresses typical deployment models for running Linkerd in your architecture." weight = 40 aliases = [ "/doc/0.1.0/deployment", "/doc/0.2.0/deployment", "/doc/0.2.1/deployment", "/doc/0.3.0/deployment", "/doc/0.3.1/deployment", "/doc/0.4.0/deployment", "/doc/0.5.0/deployment", "/doc/0.6.0/deployment", "/doc/0.7.0/deployment", "/doc/0.7.1/deployment", "/doc/0.7.2/deployment", "/doc/0.7.3/deployment", "/doc/0.7.4/deployment", "/doc/0.7.5/deployment", "/doc/0.8.0/deployment", "/doc/head/deployment", "/doc/latest/deployment", "/doc/deployment", "/in-depth/deployment", "/advanced/deployment" ] [menu.docs] parent = "advanced" +++ There are two common deployment models for Linkerd: per-host, and as a sidecar process. ## Per-host In the per-host deployment model, one Linkerd instance is deployed per host (whether physical or virtual), and all application service instances on that host route traffic through this instance. This model is useful for deployments that are primarily host-based. Each service instance on the host can address its corresponding Linkerd instance at a fixed location (typically, `localhost:4140`), obviating the need for any significant client-side logic. Since this model requires high concurrency of Linkerd instances, a larger resource profile is usually appropriate. In this model, the loss of an individual Linkerd instance is equivalent to losing the host itself. {{< fig src="/images/diagram-per-host-deployment.png" title="Linkerd deployed per host." >}} ## Sidecar In the sidecar deployment model, one Linkerd instance is deployed per instance of every application service. This model is useful for deployments that are primarily instance- or container-based, as opposed to host-based. For example, with a Kubernetes deployment, a Linkerd container can be deployed as part of the Kubernetes "pod", and the service instance can address the Linkerd instance as if it were on the same host, i.e. by connecting to `localhost:4140`. Since this sidecar approach requires many instances of Linkerd, a smaller resource profile is usually appropriate. In this model, the loss of an individual Linkerd instance is equivalent to losing the corresponding service instance. {{< fig src="/images/diagram-sidecar-deployment.png" title="Linkerd deployed as a sidecar process (service-to-linkerd)." >}} There are three configurations for how the application service and Linkerd can talk to each other: service-to-linker, linker-to-service, and linker-to-linker. ### service-to-linker In the service-to-Linkerd configuration, each service instance routes calls through its corresponding Linkerd instance. Each Linkerd will serve in a location known to the matching service instance and will route traffic to remote services. ### linker-to-service In the Linkerd-to-service configuration, application service instances do not serve traffic directly. Instead, the sidecar Linkerd should be registered in service discovery so that incoming traffic is served by Linkerd, which then routes it to the matching service instance. While this configuration misses out on all of the client-side benefits of Linkerd, it does give you server metrics for the application service such as request counts and latency histograms. ### linker-to-linker The Linkerd-to-Linkerd configuration is a combination of the above two configurations and gives you the best of both worlds. The sidecar Linkerd should be registered in service discovery so that incoming traffic is served by the Linkerd, which then routes it to the matching service instance. Then the service instance routes outgoing calls back through Linkerd. This typically requires setting up two routers in the Linkerd config: one for incoming traffic and one for outgoing traffic. --- ### Linkerd.Io/Content/1/Advanced/Dtabs (linkerd.io/content/1/advanced/dtabs.md) +++ title = "Dtabs" description = "Explains delegation tables and delegation rules, the primary mechanism by which Linkerd dynamically routes requests." weight = 30 aliases = [ "/doc/dtabs", "/dtabs", "/in-depth/dtabs", "/advanced/dtabs" ] [menu.docs] parent = "advanced" +++ Delegation tables (dtabs for short) are lists of routing rules that take a "logical path" (e.g., a popular ice cream store) and transform it into to a "concrete name" where that thing is located (e.g., 2790 Harrison St, San Francisco, CA 94110). This is a process we call "resolution" and it happens via a series of prefix rewrites. In addition to this documentation, you can refer to [Finagle's dtab docs](http://twitter.github.io/finagle/guide/Names.html). You can also experiment with the dtab playground feature of a running Linkerd instance by browsing to `http://localhost:9990/delegator`. See the [Administration]({{% ref "/1/administration/dtab-playground.md" %}}) page for more details on the dtab playground. ## Paths The simplest dtab contains a single rule (called a dentry) ```dtab /iceCreamStore => /smitten; ``` This dentry is really only useful for ice cream stores, so the rule does not apply to the path `/shoeStore/windowShop/sandals`. But for the path `/iceCreamStore/try/allFlavors`, the prefix matches the dentry's left-hand side (source) and is replaced with the right-hand side (destination) to create the new path: `/smitten/try/allFlavors` ## Dentries & ordering Dtabs can (and often do) have more than one dentry. For example, we could list several stores: ```dtab /smitten => /USA/CA/SF/Octavia/432; /iceCreamStore => /smitten; /iceCreamStore => /humphrys; ``` When we try to resolve a path that matches more than one prefix, bottom dentries take precedence. So the path `/iceCreamStore/try/allFlavors` would resolve first as `/humphrys/try/allFlavors`. However, if the address for humphrys is unknown (as in this example), we fall back to `/smitten/try/allFlavors`, which ultimately resolves to `/USA/CA/SF/Octavia/432/try/allFlavors`. ---- ### STEP-BY-STEP Example With the dtab: ```dtab /iceCreamStore => /smitten; /smitten/try => /smittenLocation/waitInLine/thenTry; /smittenLocation => /sanfrancisco/octavia/432; /california => /USA/CA; /sanfrancisco => /california/SF; ``` And the path: ```dtab /iceCreamStore/try/allFlavors ``` Here are the resolution steps: `/iceCreamStore/try/allFlavors` first matches the rule `/iceCreamStore => /smitten;` and is rewritten to `/smitten/try/allFlavors` which matches the rule `smitten/try => /smittenLocation/waitInLine/thenTry;` and is rewritten to `/smittenLocation/waitInLine/thenTry/allFlavors` which matches the rule `/smittenLocation => /sanfrancisco/octavia/432;` and is rewritten to `/sanfrancisco/octavia/432/waitInLine/thenTry/allFlavors` which matches the rule `/sanfrancisco => /california/SF;` and is rewritten to `/california/SF/octavia/432/waitInLine/thenTry/allFlavors` which matches the rule `/california => /USA/CA`; and is rewritten to `/USA/CA/SF/octavia/432/waitInLine/thenTry/allFlavors`! ---- Note that every time a prefix match is made, we start with the newly-made path and look through the entire dtab again from bottom to top. This is useful, but also makes it easy to loop accidentally! Consider the following infinite dtab (and don't worry, Finagle exits after too many recursive calls): ```dtab /iceCream => /youScream; /youScream => /weAllScream/for; /weAllScream/for => /iceCream; ``` ## Namers & addresses So far we have only discussed routing on paths. In order for Finagle to successfully route a request, the path must eventually resolve to a concrete name. Most of these concrete names (in Finagle they are called "bound addresses") are defined by or looked up using namers. Finagle provides one such namer called `/$/inet` which interprets the two subsequent path segments as an ip address and port. So the path `/$/inet/127.0.0.1/4140` would resolve to the bound address `127.0.0.1:4140` Linkerd also provides a suite of namers for many different service discovery mechanisms. Some examples are [`/#/io.l5d.consul`]({{% linkerdconfig "consul-service-discovery-experimental" %}}), [`/#/io.l5d.k8s`]({{% linkerdconfig "kubernetes-service-discovery-experimental" %}}), and [`/#/io.l5d.marathon`]({{% linkerdconfig "marathon-service-discovery-experimental" %}}). See more on these and others in the [Linkerd documentation on namers]({{% linkerdconfig "namers" %}}). Once a namer converts a path into a bound address, the routing is considered complete and any residual path segments not used in prefix matching will remain unused. As an example, let's define a namer `/#/routeOnMethod` that takes the next path segment and routes traffic based on if it's a GET or POST. Then for the dtab `/http/1.1 => /#/routeOnMethod;`, the path `/http/1.1/GET/host/users` will be rewritten to `/#/routeOnMethod/GET/host/users` and the prefix `/#/routeOnMethod/GET` will resolve to a bound address. The rest of the segments–`/host` and `/users` –have no bearing on where the traffic is routed. Namers aren't limited to resolving paths, however. At their most basic, namers are functions that operate on the path segments that follow it. Consider the namer `/#/multiply` that multiplies the next two segments together and returns a single number. For the dtab: ```dtab /byNine => /#/multiply/9; /byEight => /#/multiply/8; /bySeven => /#/multiply/7; ``` The path `/byNine/3` will be rewritten to `/#/multiply/9/3` and finally to `/27`. ## Wildcards When receiving paths like `/http/1.1/GET/chocolate/icecream`, we may not be interested in using every path segment when routing the request. If all icecream needs to be routed to `/smitten`, it doesn't matter what flavor it is. One way to write this dtab is to list all possible flavors: ```dtab /http/1.1/GET/chocolate/icecream => /smitten; /http/1.1/GET/vanilla/icecream => /smitten; /http/1.1/GET/rockyroad/icecream => /smitten; /http/1.1/GET/strawberry/icecream => /smitten; /http/1.1/GET/mintchip/icecream => /smitten; ... ``` A simpler and more elegant solution is to replace the flavors segment with a wildcard that will match any string for that segment. ```dtab /http/1.1/GET/*/icecream => /smitten; ``` ## Alternates, unions, & weights When two dentries have the same prefix, we call them alternates. We saw an example of one above. Here it is again: ```dtab /smitten => /USA/CA/SF/Octavia/432; /iceCreamStore => /smitten; /iceCreamStore => /humphrys; ``` Alternates can also be specified using the pipe operator: ```dtab /smitten => /USA/CA/SF/Harrison/2790; /iceCreamStore => /humphrys | /smitten; ``` In both of these examples, humphrys is the first ice cream store address we try to resolve. But if the address is not found we proceed to smitten (and if smitten is not found either, the whole routing operation fails–no one gets ice cream). You can specify any number of alternates `/humphrys | /smitten | /birite | /three-twins` ... Dtabs also support unions with the following syntax `/iceCreamStore => /humphrys & /smitten`. In this example we have an equal chance of routing the path to either store. If we wanted to be more likely to enter one store than another, we can add weights to each path: ```dtab /smitten => 3 * /SF/Octavia/432 & 1 * /SF/California/2404; /iceCreamStore => 0.7 * /humphrys & 0.3 * /smitten; ``` Weights can be decimals or integers. ## Negative, failure, & empty resolutions If a namer isn't able to find a concrete address, it returns a negative resolution. This signals to Finagle that this path was a dud, and if there are any alternate paths to try, now would be a good time. If all paths are negative, Finagle throws an error. This kind of fallback logic can be tested with the symbol `~` which Finagle interprets as a negative resolution. For example: ```dtab /iceCreamStore => ~ | /smitten; ``` If we want to stop before checking any alternate paths, we should use failure instead of negative. Failure is specified using `/$/fail` or even shorter `!`, like in this dtab where we route to smitten or bust: ```dtab /iceCreamStore => /smitten | !; ``` Namers sometimes also return failure resolutions. For example the `/multiply` namer might return a failure for the path `/multiply/cats/dogs`. There is a final resolution called empty. It is invoked via `/$/nil` or `$`, and it is usually only used in test scenarios. --- ### Linkerd.Io/Content/1/Advanced/Namerd (linkerd.io/content/1/advanced/namerd.md) +++ title = "namerd" description = "Introduces namerd as a service that helps route Linkerd requests and centralizes routing decisions to provide global Linkerd control." weight = 20 aliases = [ "/in-depth/namerd", "/advanced/namerd" ] [menu.docs] parent = "advanced" +++ namerd is a service that manages routing for multiple Linkerd instances. It does this by storing [dtabs]({{% ref "/1/advanced/dtabs.md" %}}) and using [namers]({{% ref "/1/advanced/dtabs.md#namers-addresses" %}}) for service discovery. namerd supports the same suite of service discovery backends that Linkerd does, which include services like [ZooKeeper](https://twitter.github.io/commons/apidocs/com/twitter/common/zookeeper/ServerSet.html), [Consul](https://www.consul.io/), [Kubernetes API](http://kubernetes.io/docs/api), and [Marathon](https://mesosphere.github.io/marathon/). Using namerd, individual Linkerds no longer need to talk directly to service discovery or have dtabs hardcoded into their config files. Instead, they ask namerd for any necessary routing information. This provides a number of benefits, which are outlined below. ## Decreased load on service discovery backends Using namerd means that only a small cluster of namerds need to talk directly to the service discovery backends instead of every Linkerd in the fleet. namerd also utilizes caching to further protect the service discovery backend from excessive load. ## Global routing policy By storing dtabs in namerd instead of hardcoding them in the Linkerd configs, it ensures that routing policy is in sync across the fleet and gives you one central source of truth when you need to make changes. ## Dynamic routing policy The other advantage of storing dtabs in namerd is that these dtabs can be updated dynamically using [namerd's API]({{% namerdconfig "http-controller" %}}) or [command-line tool](https://github.com/linkerd/namerctl). This allows you to perform operations like [canary, staging, or blue-green deploy](https://blog.buoyant.io/2016/05/04/real-world-microservices-when-services-stop-playing-well-and-start-getting-real/#dynamic-routing-with-namerd), all without needing to restart any Linkerds. ## More information To learn more about namerd, its setup, and its operation, check out Buoyant's blog post on [dynamic routing](https://blog.buoyant.io/2016/05/04/real-world-microservices-when-services-stop-playing-well-and-start-getting-real/#dynamic-routing-with-namerd). To configure your own namerd, head over to the [namerd config documention]({{% namerdconfig %}}). Also check out [namerctl](https://github.com/linkerd/namerctl), our open source tool for controlling namerd. For a step-by-step walkthrough of running namerd in Kubernetes to facilitate continuous deployment, check out Buoyant's blog post [Continuous deployment via traffic shifting]( https://blog.buoyant.io/2016/11/04/a-service-mesh-for-kubernetes-part-iv-continuous-deployment-via-traffic-shifting/). --- ### Linkerd.Io/Content/1/Advanced/Plugin (linkerd.io/content/1/advanced/plugin.md) +++ title = "Plugins" description = "Sheds light on Linkerd's modular plugin system, and provides a detailed walkthrough for writing your own plugins." weight = 50 aliases = [ "/doc/0.7.0/plugin", "/doc/0.7.1/plugin", "/doc/0.7.2/plugin", "/doc/0.7.3/plugin", "/doc/0.7.4/plugin", "/doc/0.7.5/plugin", "/doc/0.8.0/plugin", "/doc/head/plugin", "/doc/latest/plugin", "/doc/plugin", "/in-depth/plugin", "/advanced/plugin" ] [menu.docs] parent = "advanced" +++ Linkerd is built on a modular plugin system so that individual components may be swapped out without needing to recompile. This also allows anyone to build custom plugins that implement functionality specific to their needs. This guide will show you how to write your own custom Linkerd plugin, how to package it, and how to install it in Linkerd. In this guide we will write a custom [HTTP response classifier]({{% linkerdconfig "http-response-classifiers" %}}). However, the ideas in this guide apply just as well for writing custom [identifiers]({{% linkerdconfig "http-1-1-identifiers" %}}), [namers]({{% linkerdconfig "namers" %}}), [name interpreters]({{% linkerdconfig "interpreter" %}}), [protocols]({{% linkerdconfig "routers" %}}), or any other Linkerd plugin. All the code for the plugin in this guide is available on [GitHub](https://github.com/linkerd/linkerd-examples/tree/master/plugins/header-classifier). I recommend keeping the code open in another window while you read this guide so that you can follow along. ## Overview We will write a custom HTTP response classifier that classifies responses based on a special response header instead of the HTTP response code. If the header's value is "success" we will treat the response as a success, if it is "retry" we will treat it as a retryable failure, and otherwise we will treat it as a non-retryable failure. We will describe how to write the plugin, how to build and package it, and how to install it in Linkerd. ## Writing plugins While Linkerd itself is written in Scala, its plugins can be written in Scala or Java. To demonstrate this, we will write our plugin in Java. For our plugin to be functional, we will need 3 things: the response classifier itself, a config class, and a config initializer. All plugins follow this pattern of having the class that implements the business logic, a config class, and a config initializer. ### Response classifier [HeaderClassifier.java](https://github.com/linkerd/linkerd-examples/blob/master/plugins/header-classifier/src/main/java/io/buoyant/http/classifiers/HeaderClassifier.java) is the response classifier itself. Response classifiers must extend `PartialFunction[ReqRep, ResponseClass]`. Each plugin type has a different interface that it must implement. For example, namer plugins must implement [Namer](https://github.com/twitter/finagle/blob/master/finagle-core/src/main/scala/com/twitter/finagle/Namer.scala#L16) and identifier plugins must implement [Identifier](https://github.com/linkerd/linkerd/blob/master/router/core/src/main/scala/io/buoyant/router/RoutingFactory.scala#L21). ### Config class Next we need a class that defines the structure of the config block for this plugin and constructs the response classifier. We will call this [HeaderClassifierConfig.java](https://github.com/linkerd/linkerd-examples/blob/master/plugins/header-classifier/src/main/java/io/buoyant/http/classifiers/HeaderClassifierConfig.java). Notice that `HeaderClassifierConfig` must implement [`ResponseClassifierConfig`](https://github.com/linkerd/linkerd/blob/master/linkerd/core/src/main/scala/io/buoyant/linkerd/ResponseClassifierInitializer.scala#L14). ResponseClassifierConfigs are deserialized from the response classifier section of the Linkerd config by [Jackson](https://github.com/FasterXML/jackson) and its public members are populated by the corresponding JSON (or YAML) properties. (In Scala this would be a case class.) In our case we have one public member called `headerName`, which defines the name of the response header to use. To satisfy `ResponseClassifierConfig` we must also implement a method called `mk()` which constructs the response classifier. ### Config initializer A config initializer is a special class that Linkerd loads at startup. It tells Linkerd about config classes it can use. We create a config initializer called [HeaderClassifierInitializer.java](https://github.com/linkerd/linkerd-examples/blob/master/plugins/header-classifier/src/main/java/io/buoyant/http/classifiers/HeaderClassifierInitializer.java). This class must define a `configId` and a `configClass`. When Linkerd parses a config block with a `kind` property, it looks for a config initializer with that `configId` and attempts to deserialize the block as an instance of the `configClass`. `HeaderClassifierInitializer` tells Linkerd that when it finds a `responseClassifier` block with `kind: io.buoyant.headerClassifier`, it should deserialize that block as a `HeaderClassifierConfig`. Finally, in order for Linkerd to be able to dynamically load the config initializer at startup, we must register it with the service loader. To do this simply create a resource file called [`META-INF/services/io.buoyant.linkerd.ResponseClassifierConfig`](https://github.com/linkerd/linkerd-examples/blob/master/plugins/header-classifier/src/main/resources/META-INF/services/io.buoyant.linkerd.ResponseClassifierInitializer) and add the fully qualified class name of the config initializer to that file. ## Build & package We use sbt to build our plugin and the assembly sbt plugin to package it into a jar. Here is the [build.sbt](https://github.com/linkerd/linkerd-examples/blob/master/plugins/build.sbt) file for the project. Note that we can mark any Linkerd dependencies as "provided". This means that those dependencies will be provided by Linkerd and do not need to be included in the plugin jar. Similarly, Linkerd will provide the Scala standard libraries, so we can exclude those from the jar as well by setting `includeScala = false`. Build the plugin jar by running: ```bash ./sbt headerClassifier/assembly ``` ## Installing To install this plugin with Linkerd, simply move the plugin jar into Linkerd's plugin directory (`$L5D_HOME/plugins`). Then add a classifier block to the router in your Linkerd config: ```yaml routers: - ... responseClassifier: kind: io.buoyant.headerClassifier headerName: status ``` If you run Linkerd with `-log.level=DEBUG` then you should see a line printed at startup that indicates the HeaderClassifierInitializer has been loaded: ```bash LoadService: loaded instance of class io.buoyant.http.classifiers.HeaderClassifierInitializer for requested service io.buoyant.linkerd.ResponseClassifierInitializer ``` ## Trying it out Now that we have our plugin in the plugins directory, let's try it out. Start Linkerd with this simple config that sends all requests to `localhost:8888`. ```yaml routers: - protocol: http dtab: /svc/* => /$/inet/localhost/8888 responseClassifier: kind: io.buoyant.headerClassifier headerName: status servers: - ip: 0.0.0.0 port: 4140 ``` Then we'll start a simple server on port 8888 that responds with the status header indicating success: ```bash while true; do echo -e "HTTP/1.1 200 OK\r\nstatus: success\r\n" | nc -i 1 -l 8888; done ``` Now let's issue a request: ```bash curl -v localhost:4140 ``` By checking Linkerd's metrics, we can see that this request was classified as a success: ```bash curl -s localhost:9990/admin/metrics.json?pretty=1 | grep -E 'srv.*(success|failure)' "rt/http/srv/0.0.0.0/4140/success" : 1, ``` Now let's restart our server and have it set the header to "failure": ```bash while true; do echo -e "HTTP/1.1 200 OK\r\nstatus: failure\r\n" | nc -i 1 -l 8888; done ``` And issue another request: ```bash curl -v localhost:4140 ``` Now when we check Linkerd's metrics, we'll see that this request was classified as a failure: ```bash curl -s localhost:9990/admin/metrics.json?pretty=1 | grep -E 'srv.*(success|failure)' "rt/http/srv/0.0.0.0/4140/failures" : 1, "rt/http/srv/0.0.0.0/4140/failures/com.twitter.finagle.service.ResponseClassificationSyntheticException" : 1, "rt/http/srv/0.0.0.0/4140/success" : 1, ``` ## More information If you have any questions about using or developing Linkerd plugins, or would like to share what you've created, please drop into [the Linkerd public Slack]( http://slack.linkerd.io). We hope to see you soon! --- ### Linkerd.Io/Content/1/Advanced/Routing (linkerd.io/content/1/advanced/routing.md) +++ title = "Routing" description = "Provides a comprehensive look at how Linkerd routes requests that it receives." weight = 10 aliases = [ "/doc/0.7.0/routing", "/doc/0.7.1/routing", "/doc/0.7.2/routing", "/doc/0.7.3/routing", "/doc/0.7.4/routing", "/doc/0.7.5/routing", "/doc/0.8.0/routing", "/doc/head/routing", "/doc/latest/routing", "/doc/routing", "/in-depth/routing", "/advanced/routing" ] [menu.docs] parent = "advanced" +++ At its core, Linkerd's main job is routing: accepting a request (HTTP, Thrift, Mux, or other protocol) and sending that request to the correct destination. This guide will explain exactly how Linkerd determines where requests should be sent. This process consists of 4 steps: [identification](#identification), [binding](#binding), [resolution](#resolution), and [load balancing](#load-balancing). {{< fig src="/images/routing1.png" title="Linkerd routing" >}} ## Identification Identification is the act of assigning a *name* (also called a *path*) to the request. A name is a slash-delimited string representing the destination of the request. By default, Linkerd uses an **identifier** called `io.l5d.header.token` which assigns names to requests based on the Host header like so: `/svc/`. This means that an HTTP request to `GET http://example/hello` would be assigned the name `/svc/example`. (Note that the path of this URL, `/hello`, is dropped in the name. It will still be proxied as part of the request--the name only determines how the request is routed, not what is sent to the destination service.) Of course, the **identifier** is a pluggable module and can be replaced with a custom **identifier** which assigns names to requests based on any logic you desire. Learn more about Linkerd's built in **identifiers** and how to configure them in the [Linkerd identifier docs]({{% linkerdconfig "http-1-1-identifiers" %}}). The name that the identifier assigns to the request is called the **service name** because it should encode the destination as specified by the application. It typically does not encode information about clusters, zones, environments, or hosts because your application shouldn't need to worry about these concerns. For example, if your application wants to make a request to the "users" service, it could issue an HTTP GET request to Linkerd with "users" as the Host header. The `io.l5d.header.token` **identifier** would assign `/svc/users` as the **service name** of that request. ## Binding Once a **service name** has been assigned to a request, that name undergoes transformations by the dtab (short for delegation table). This is called **binding**. Detailed documentation on how dtab transformations work can be found on the [Dtabs]({{% ref "/1/advanced/dtabs.md" %}}) page. Dtabs encode the routing rules that describe how a **service name** is transformed into a **client name**. A **client name** is the name of a replica set, typically the name of a service discovery entry. Unlike **service names**, **client names** often contain details like cluster, zone, and/or environment. **client names** always begin with `/$` or `/#`. (See below for the distinction between these two prefixes.) Continuing the example, suppose we had the following dtab: ```dtab /env => /#/io.l5d.serversets/discovery /svc => /env/prod ``` The **service name** `/svc/users` would get bound like this: ```dtab /svc/users /env/prod/users /#/io.l5d.serversets/discovery/prod/users ``` and result in `/#/io.l5d.serversets/discovery/prod/users` as the **client name**. ## Resolution **Resolution** is the act of resolving a **client name** into a set of physical endpoints (ip address + port). **Resolution** is done by something called a **namer** which typically does a lookup into some service discovery backend. Linkerd comes with **namers** for most major service discovery implementations built in. Learn more about how to configure them in the [Linkerd namer docs]({{% linkerdconfig "namers" %}}). **client names** that start with `/$` indicate that a **namer** from the classpath should be loaded to bind that name, whereas **client names** that start with `/#` indicate that a **namer** from the Linkerd config file should be loaded to **bind** that name. For example, suppose we have `/#/io.l5d.serversets/discovery/prod/users` as a **client name**. This means that the `io.l5d.serversets` **namer** from the Linkerd config should look up the `/discovery/prod/users` serverset (the result of this lookup is a set of physical addresses). Similarly, the **client name** `/$/inet/users/8888` means to search the classpath for the `inet` **namer**. This **namer** gets the set of addresses by doing a DNS lookup on "users" and using port 8888. ## Load balancing Once Linkerd has a replica set, it uses a [load balancing algorithm](https://blog.buoyant.io/2016/03/16/beyond-round-robin-load-balancing-for-latency/) to determine where to send the request. Because Linkerd does load balancing at the request layer instead of at the connection layer, the load balancing algorithm can take advantage of request latency information to de-weight slow nodes and avoid overloading struggling hosts. --- ### Linkerd.Io/Content/1/Configuration/ Index (linkerd.io/content/1/configuration/_index.md) +++ title = "Overview" description = "Contains versioned configuration references for both projects, including sample config file snippets." weight = 4 aliases = [ "/doc/0.1.0/config", "/doc/0.2.0/config", "/doc/0.2.1/config", "/doc/0.3.0/config", "/doc/0.3.1/config", "/doc/0.4.0/config", "/doc/0.5.0/config", "/doc/head/config", "/doc/latest/config", "/doc/config", "/configuration" ] [menu.docs] name = "Configuration" identifier = "configuration" weight = 6 +++ Both Linkerd and namerd’s configuration is controlled via a configuration file, which must be provided as a command-line argument when the processes are started. Both YAML and JSON config file formats are supported. For the complete, up-to-date configuration reference guides, see: {{% configbuttons %}} --- ### Linkerd.Io/Content/1/Examples/ Index (linkerd.io/content/1/examples/_index.md) +++ title = "Overview" description = "Example applications and tutorials on Linkerd." weight = 1 aliases = [ "/examples" ] [menu.docs] name = "Examples" identifier = "examples" weight = 5 +++ The [linkerd-examples GitHub repo](https://github.com/linkerd/linkerd-examples) contains several examples of how to use Linkerd and namerd in various environments. The [Buoyant blog](https://blog.buoyant.io) also contains several examples and walkthroughs highlighting various Linkerd features. ## Kubernetes For walkthroughs of various Linkerd features with deployable examples, check out [A Service Mesh for Kubernetes blog series](https://buoyant.io/2016/10/04/a-service-mesh-for-kubernetes-part-i-top-line-service-metrics/): * [Top-line service metrics](https://buoyant.io/a-service-mesh-for-kubernetes-part-i-top-line-service-metrics/) * [Pods are great, until they’re not](https://buoyant.io/a-service-mesh-for-kubernetes-part-ii-pods-are-great-until-theyre-not/) * [Encrypting all the things](https://buoyant.io/a-service-mesh-for-kubernetes-part-iii-encrypting-all-the-things/) * [Continuous deployment via traffic shifting](https://buoyant.io/a-service-mesh-for-kubernetes-part-iv-continuous-deployment-via-traffic-shifting/) * [Dogfood environments, ingress, and edge routing](https://buoyant.io/a-service-mesh-for-kubernetes-part-v-dogfood-environments-ingress-and-edge-routing/) * [Staging microservices without the tears](https://buoyant.io/a-service-mesh-for-kubernetes-part-vi-staging-microservices-without-the-tears/) * [Distributed tracing made easy](https://buoyant.io/a-service-mesh-for-kubernetes-part-vii-distributed-tracing-made-easy/) * [Linkerd as an ingress controller](https://buoyant.io/a-service-mesh-for-kubernetes-part-viii-linkerd-as-an-ingress-controller/) * [gRPC for fun and profit](https://buoyant.io/a-service-mesh-for-kubernetes-part-ix-grpc-for-fun-and-profit/) * [The Service Mesh API](https://buoyant.io/a-service-mesh-for-kubernetes-part-x-the-service-mesh-api/) * [Egress](https://buoyant.io/a-service-mesh-for-kubernetes-part-xi-egress/) Other Kubernetes posts: * [The Consolidated Kubernetes Service Mesh Linkerd Config](https://buoyant.io/2017/08/08/a-service-mesh-for-ecs/) * [Using Linkerd with Kubernetes RBAC](https://buoyant.io/2017/07/24/using-linkerd-kubernetes-rbac/) The [linkerd-examples/k8s-daemonset](https://github.com/linkerd/linkerd-examples/tree/master/k8s-daemonset) folder has various Kubernetes examples using different Linkerd features, some of which are referenced in the blogs. ## DC/OS, Mesos * [Linkerd on DC/OS for Service Discovery and Visibility](https://buoyant.io/2016/10/10/linkerd-on-dcos-for-service-discovery-and-visibility/) * [Linkerd on DC/OS: Microservices in Production Made Easy](https://buoyant.io/2016/04/19/linkerd-dcos-microservices-in-production-made-easy/) * [linkerd-examples/dcos](https://github.com/linkerd/linkerd-examples/tree/master/dcos) * [linkerd-examples/mesos-marathon](https://github.com/linkerd/linkerd-examples/tree/master/mesos-marathon) ## Amazon ECS * [A Service Mesh For ECS](https://buoyant.io/2017/08/08/a-service-mesh-for-ecs/) * [linkerd-examples/ecs](https://github.com/linkerd/linkerd-examples/tree/master/ecs) ## Service Mesh * [What’s a service mesh? And why do I need one?](https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-one/) ## Linkerd-tcp * [Introducing Linkerd-tcp](https://buoyant.io/2017/03/29/introducing-linkerd-tcp/) * [linkerd-examples/linkerd-tcp](https://github.com/linkerd/linkerd-examples/tree/master/linkerd-tcp) ## Linkerd features Blogs and example configs highlighting various Linkerd features, including how to use Linkerd to improve latency, load balancing, using failure accrual, http proxying, adding TLS, a metrics pipeline, gRPC and HTTP/2, and more! * [Making Things Faster by Adding More Steps](https://buoyant.io/2017/01/31/making-things-faster-by-adding-more-steps/) * [Making microservices more resilient with circuit breaking](https://buoyant.io/2017/01/13/making-microservices-more-resilient-with-circuit-breaking/) * [HTTP/2, gRPC and Linkerd](https://buoyant.io/2017/01/10/http2-grpc-and-linkerd/) * [Distributed Tracing for Polyglot Microservices](https://buoyant.io/2016/05/17/distributed-tracing-for-polyglot-microservices/) * [Transparent TLS with Linkerd](https://buoyant.io/2016/04/19/linkerd-dcos-microservices-in-production-made-easy/) * [Beyond Round Robin: Load Balancing for Latency](https://buoyant.io/2016/03/16/beyond-round-robin-load-balancing-for-latency/) * [linkerd-examples/getting-started](https://github.com/linkerd/linkerd-examples/tree/master/getting-started) * [linkerd-examples/failure-accrual](https://github.com/linkerd/linkerd-examples/tree/master/failure-accrual) * [linkerd-examples/http-proxy](https://github.com/linkerd/linkerd-examples/tree/master/http-proxy) * [linkerd-examples/influxdb](https://github.com/linkerd/linkerd-examples/tree/master/influxdb) ## Further reading See the [External Resources]({{% ref "/1/examples/external-resources.md" %}}) section for additional articles and tutorials on Linkerd! --- ### Linkerd.Io/Content/1/Examples/External Resources (linkerd.io/content/1/examples/external-resources.md) +++ title = "External resources" description = "Catalogues blogs and talks about Linkerd in practice, from both the perspective of an end user and the perspective of a developer." weight = 2 aliases = [ "/overview/external-resources", "/support/external-resources", "/examples/external-resources" ] [menu.docs] parent = "examples" +++ This page features blogs and talks people have given about Linkerd. Content produced by both Buoyant and Linkerd users is represented below. You can also check out [Buoyant's blog](https://buoyant.io/blog) for more posts! ## Linkerd Users * [Linkerd: the Cloud Native service mesh](https://skillsmatter.com/skillscasts/10912-cloud-native-london-october)
Cloud Native London, October 2017 * [How ForeSee Processes Billions of Events Per Day with Linkerd](https://youtu.be/YRjUuhiXIPc)
Buoyant Webinar, August 2017 * [When Failure is Not an Option](https://youtu.be/wBgBwNZo_EE)
KubeCon EU, March 2017 * [Cloud Native Applications on OpenShift/Azure](https://channel9.msdn.com/Events/TechDaysOnline/MVP-Led-Techdays-Online/Cloud-Native-Applications-on-OpenShiftAzure-)
Microsoft TechDays Online, February 2017 * [Building a Modern Bank Backend](https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend/) (blog)
[Building a Bank with Kubernetes](https://skillsmatter.com/skillscasts/9146-building-a-bank-with-kubernetes) (slides)
Monzo, October 2016 ## Linkerd Developers * [Resilient Communication for Microservice Applications](https://youtu.be/fFgsGKXHKz8)
DevOps Exchange San Francisco, June 2017 * [Keynote: Linkerd Project Update](https://youtu.be/4qeKF5IROSg)
KubeCon EU, March 2017 * [An introduction to Linkerd](https://youtu.be/0xYSy6OmjUM)
Cloud Native Computing Foundation, March 2017 * [Freeing the Whale: How to Fail at Scale](https://www.infoq.com/presentations/twitter-finagle)
InfoQ, March 2017 * [Microservice Staging without the Tears](https://youtu.be/y0D5EAXvUpg)
SF Microservices Meetup, December 2016 * [Linkerd 101: Online Class](https://youtu.be/38DilGa3_Gs)
Buoyant, December 2016 * [Finagle as a Proxy](https://youtu.be/arCUlbuzyow)
Scala By The Bay, November 2016 * [Service Discovery and Visibility for Microservices on DC/OS](https://youtu.be/eC4s-XwUbOk)
Mesosphere, October 2016 * [The Once and Future Layer 5: Resilient Twitter-style Microservices](https://youtu.be/W5hcziNcUig)
VelocityConf NY, September 2016 * [Finagle, Linkerd, and Apache Mesos: Magical Operability Sprinkles for Microservices](https://youtu.be/VGAFFkn5PiE)
MesosCon NA, June 2016 * [Magical Security Sprinkles: Secure, Resilient Microservices on CoreOS and Kubernetes](https://youtu.be/BqepPS7oTMs)
CoreOS Fest SF, May 2016 * [Kubernetes Community Meeting](https://youtu.be/mGdenPQ9jdY)
Kubernetes, March 2016 * [Kubernetes Meets Finagle for Resilient Microservices](https://youtu.be/co7JRxihcdA)
KubeCon EU, March 2016 * [Buoyant Service Discovery](https://youtu.be/ysn8urNTedQ)
Scala By The Bay FinagleCon, August 2015 ## Linkerd In the News * [How Twitter's Fail Whale could save your company](http://www.techrepublic.com/article/how-twitters-fail-whale-could-save-your-company/)
TechRepublic, March 2017 * [Linkerd Celebrates One Year with One Hundred Billion Production Requests](https://www.cncf.io/blog/2017/03/09/linkerd-celebrates-one-year-one-hundred-billion-production-requests/)
Cloud Native Computing Foundation, March 2017 * [Open Source Linkerd Project Celebrates First Anniversary in Quest to Become TCP/IP of Microservices](https://www.infoq.com/news/2017/03/linkerd-celebrates-one-year)
InfoQ, March 2017 * [Cloud Native Computing Foundation adds Linkerd as its fifth hosted project](https://techcrunch.com/2017/01/23/cloud-native-computing-foundation-adds-linkerd-as-its-fifth-hosted-project/)
TechCrunch, January 2017 --- ### Linkerd.Io/Content/1/Features/ Index (linkerd.io/content/1/features/_index.md) +++ title = "Overview" description = "Covers Linkerd's major features, including load balancing, circuit breaking, service discovery, routing, retries, and more." weight = 1 aliases = [ "/features" ] [menu.docs] name = "Features" identifier = "features" weight = 4 +++ Beyond adding reliability through circuit breaking and latency-aware load balancing, Linkerd automatically instruments top-line service metrics such as request volume, success rates, and latency distributions. Linkerd also provides request-level routing and multi-service discovery integration with a powerful language called dtabs. In this section you'll find a rundown of Linkerd's major features. {{% sectiontoc "features" %}} --- ### Linkerd.Io/Content/1/Features/Circuit Breaking (linkerd.io/content/1/features/circuit-breaking.md) +++ title = "Circuit breaking" description = "Linkerd includes automatic circuit breaking that will stop sending traffic to instances that are deemed to be unhealthy, giving them a chance to recover and avoiding cascading failures." weight = 3 aliases = [ "/features/circuit-breaking" ] [menu.docs] parent = "features" +++ Circuit breaking is a mechanism used by Linkerd to remove unhealthy service instances from [load balancing]({{% ref "/1/features/load-balancing.md" %}}). Unhealthy instances can be detected at both the connection level and the request level. By employing circuit breaking, Linkerd can minimize the amount of time spent trying to route requests that ultimately fail, thereby freeing up resources and avoiding a common cause of cascading failures. The two types of circuit breaking that Linkerd provides are described below. They're also addressed more extensively in Finagle's [circuit breaking]( https://twitter.github.io/finagle/guide/Clients.html#circuit-breaking) documentation. ## Fail fast Fail fast circuit breaking happens at the connection level. When fail fast is enabled, if Linkerd sees a connection error when attempting to send a request to one of your service's hosts, Linkerd will remove that connection from its connection pool. In the background, Linkerd attempts to reestablish the connection, without actively trying to send it traffic. Only once the connection is successfully reestablished will it be added back into the pool and start receiving traffic again. Fail fast is disabled by default in Linkerd, since it can be problematic when proxying requests to services with a small number of hosts. If a service has only one host, removing the only connection to that host from the load balancer pool would fail all requests to that service until the connection can be reestablished. In that case it's better to leave the connection in the pool and continue to send requests. For larger services, however, fail fast can be useful, and it can be enabled on a per-router basis by [setting the `failFast` parameter]({{% linkerdconfig "router-parameters" %}}) when configuring routers. ## Failure accrual Failure accrual circuit breaking operates at the request level, based on the number of requests that have failed for a given host. By default, if Linkerd receives 5 consecutive failures from a host, it will temporarily mark the host as dead, giving it a grace period to recover before resending requests. Once a host has been marked dead, Linkerd will attempt to resend traffic to that host based on a backoff interval. Both the threshold for marking a host as dead and the backoff interval are fully configurable by [setting the client `failureAccrual` parameter]({{% linkerdconfig "failure-accrual" %}}). In addition to consecutive failures, other available threshold calculations include observed success rate over a given number of requests, and observed success rate over a given time window. Failure accrual uses response classification to determine which types of responses count as failures. Response classification can be configured on a per-router basis by [setting the router `responseClassifier` parameter]( {{% linkerdconfig "http-response-classifiers" %}}) when configuring routers. ## More information If you'd like to learn more about the performance impacts of various circuit breaking settings, check out Buoyant's blog post on the topic: [Making microservices more resilient with circuit breaking]( https://blog.buoyant.io/2017/01/13/making-microservices-more-resilient-with-circuit-breaking/). --- ### Linkerd.Io/Content/1/Features/Distributed Tracing And Instrumentation (linkerd.io/content/1/features/distributed-tracing-and-instrumentation.md) +++ title = "Distributed tracing" description = "Linkerd supports both distributed tracing and metrics instrumentation, providing uniform observability across all services." weight = 9 aliases = [ "/features/distributed-tracing-and-instrumentation" ] [menu.docs] parent = "features" +++ As the number and complexity of services increases, uniform observability across the data center becomes more critical. Linkerd's tracing and metrics instrumentation is designed to be aggregated, providing broad and granular insight into the health of all services. Linkerd's role as a service mesh makes it the ideal data source for observability information, particularly in a polyglot environment. As requests pass through multiple services, identifying peformance bottlenecks becomes increasingly difficult using traditional debugging techniques. Distributed tracing provides a holistic view of requests transiting through multiple services, allowing for immediate identification of latency issues. With Linkerd, distributed tracing comes for free. Simply configure Linkerd to export tracing data to a backend trace aggregator, such as [Zipkin](http://zipkin.io). This will expose latency, retry, and failure information for each hop in a request. ## Further reading If you're ready to start using distributed tracing in your setup, see the [Tracers section of the Linkerd config]({{% linkerdconfig "tracers" %}}). For a guide on setting up an end-to-end distributed tracing pipeline, see [Distributed Tracing for Polyglot Microservices](https://blog.buoyant.io/2016/05/17/distributed-tracing-for-polyglot-microservices/) on the [Buoyant Blog](https://blog.buoyant.io). --- ### Linkerd.Io/Content/1/Features/Grpc (linkerd.io/content/1/features/grpc.md) +++ title = "gRPC" description = "Linkerd supports both HTTP/2 and TLS, allowing it to route gRPC requests, enabling advanced RPC mechanisms such as bidirectional streaming, flow control, and structured data payloads." weight = 8 aliases = [ "/features/grpc" ] [menu.docs] parent = "features" +++ Linkerd supports configuring gRPC clients and servers, which can be used to introduce gRPC easily into your application. Using Linkerd to route gRPC requests enables resilient distributed communication, as well as support for structured data, bidirectional streaming, flow control, and robust, cross-platform client libraries provided by gRPC and Protocol Buffers. ## Transport The underlying transport used for gRPC is HTTP/2. Linkerd supports [configuring HTTP/2 enabled routers]({{% linkerdconfig "http-2-protocol" %}}), which can also be used to route gRPC requests. When gRPC clients send a request, they include routing information in HTTP/2's `:path` pseudo-header. The path for a gRPC request is prefixed with `/serviceName/methodName` segments, and Linkerd can be configured to read the value ---