# Technical Documentation: docker/Docker > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/docker/Docker) ### Overview (`README.md`) The Moby Project ================ [](https://pkg.go.dev/github.com/moby/moby/v2) [](https://scorecard.dev/viewer/?uri=github.com/moby/moby) [](https://www.bestpractices.dev/projects/10989) Moby is an open-source project created by Docker to enable and accelerate software containerization. It provides a "Lego set" of toolkit components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts and professionals to experiment and exchange ideas. Components include container build tools, a container registry, orchestration tools, a runtime and more, and these can be used as building blocks in conjunction with other tools and projects. ## Principles Moby is an open project guided by strong principles, aiming to be modular, flexible and without too strong an opinion on user experience. It is open to the community to help set its direction. - Modular: the project includes lots of components that have well-defined functions and APIs that work together. - Batteries included but swappable: Moby includes enough components to build fully featured container systems, but its modular architecture ensures that most of the components can be swapped by different implementations. - Usable security: Moby provides secure defaults without compromising usability. - Developer focused: The APIs are intended to be functional and useful to build powerful tools. They are not necessarily intended as end user tools but as components aimed at developers. Documentation and UX is aimed at developers not end users. ## Audience The Moby Project is intended for engineers, integrators and enthusiasts looking to modify, hack, fix, experiment, invent and build systems based on containers. It is not for people looking for a commercially supported system, but for people who want to work and learn with open source code. ## Relationship with Docker The components and tools in the Moby Project are initially the open source components that Docker and the community have built for the Docker Project. New projects can be added if they fit with the community goals. Docker is committed to using Moby as the upstream for the Docker Product. However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed. The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful. The releases are supported by the maintainers, community and users, on a best efforts basis only. For customers who want enterprise or commercial support, [Docker Desktop](https://www.docker.com/products/docker-desktop/) and [Mirantis Container Runtime](https://www.mirantis.com/software/mirantis-container-runtime/) are the appropriate products for these use cases. ## Go modules > [!IMPORTANT] > Starting with Docker v29 (released November 2025), the Go module `github.com/docker/docker` is **deprecated** and won't be updated. The supported public Go modules are: | Module | Description | |---|---| | [`github.com/moby/moby/client`](client/) | Go client for the Docker Engine API | | [`github.com/moby/moby/api`](api/) | API types shared between client and server | The root module `github.com/moby/moby/v2` is the codebase for building container engines based on Moby (such as Docker Engine). It produces binaries only - it is **not intended to be imported as a Go library** and has no API stability guarantees. ### Release tags Docker Engine releases are tagged with a **`docker-`** prefix (e.g. `docker-v29.0.0` for Docker Engine 29.0.0). These tags are only used to build the Docker Engine binary from the root module - they must not be consumed via `go get`. The `client` and `api` modules are versioned independently with their own tags (e.g. `client/v1.x.x`, `api/v1.x.x`). ### Migrating from `github.com/docker/docker` Replace the old import paths: ```diff - import "github.com/docker/docker/client" + import "github.com/moby/moby/client" - import "github.com/docker/docker/api/types" + import "github.com/moby/moby/api/types" ``` Note that v29 includes many breaking API changes (option structs, renamed methods, moved types). See the [v29.0.0 release notes](https://github.com/moby/moby/releases/tag/docker-v29.0.0) for the full list of Go SDK changes. ----- Legal ===== *Brought to you courtesy of our legal counsel. For more context, please see the [NOTICE](https://github.com/moby/moby/blob/master/NOTICE) document in this repo.* Use and transfer of Moby may be subject to certain restrictions by the United States and other governments. It is your responsibility to ensure that your use and/or transfer does not violate applicable laws. For more information, please see https://www.bis.doc.gov Licensing ========= Moby is licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/moby/moby/blob/master/LICENSE) for the full license text. --- ### Overview (`api/README.md`) # Engine API [](https://pkg.go.dev/github.com/moby/moby/api) [](https://scorecard.dev/viewer/?uri=github.com/moby/moby) [](https://www.bestpractices.dev/projects/10989) The Engine API is an HTTP API used by the command-line client to communicate with the daemon. It can also be used by third-party software to control the daemon. It consists of various components in this repository: - `api/swagger.yaml` A Swagger definition of the API. - `api/types/` Types shared by both the client and server, representing various objects, options, responses, etc. Most are written manually, but some are automatically generated from the Swagger definition. See [#27919](https://github.com/moby/moby/issues/27919) for progress on this. - `client/` The Go client used by the command-line client. It can also be used by third-party Go programs. - `daemon/` The daemon, which serves the API. ## Swagger definition The API is defined by the [Swagger](http://swagger.io/specification/) definition in `api/swagger.yaml`. This definition can be used to: 1. Automatically generate documentation. 2. Automatically generate the Go server and client. (A work-in-progress.) 3. Provide a machine readable version of the API for introspecting what it can do, automatically generating clients for other languages, etc. ## Updating the API documentation The API documentation is generated entirely from `api/swagger.yaml`. If you make updates to the API, edit this file to represent the change in the documentation. Documentation for each API version can be found in the [docs directory](docs/README.md), which also provides a [CHANGELOG.md](docs/CHANGELOG.md). The file is split into two main sections: - `definitions`, which defines re-usable objects used in requests and responses - `paths`, which defines the API endpoints (and some inline objects which don't need to be reusable) To make an edit, first look for the endpoint you want to edit under `paths`, then make the required edits. Endpoints may reference reusable objects with `$ref`, which can be found in the `definitions` section. There is hopefully enough example material in the file for you to copy a similar pattern from elsewhere in the file (e.g. adding new fields or endpoints), but for the full reference, see the [Swagger specification](https://github.com/moby/moby/issues/27919). `swagger.yaml` is validated by `hack/validate/swagger` to ensure it is a valid Swagger definition. This is useful when making edits to ensure you are doing the right thing. ## Viewing the API documentation When you make edits to `swagger.yaml`, you may want to check the generated API documentation to ensure it renders correctly. Run `make swagger-docs` and a preview will be running at `http://localhost:9000`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation. The production documentation is generated by vendoring `swagger.yaml` into [docker/docs](https://github.com/docker/docs). --- ### Overview (`client/README.md`) # Go client for the Docker Engine API [](https://pkg.go.dev/github.com/moby/moby/client) [](https://scorecard.dev/viewer/?uri=github.com/moby/moby) [](https://www.bestpractices.dev/projects/10989) The `docker` command uses this package to communicate with the daemon. It can also be used by your own Go applications to do anything the command-line interface does; running containers, pulling or pushing images, etc. For example, to list all containers (the equivalent of `docker ps --all`): ```go package main import ( "context" "fmt" "github.com/moby/moby/client" ) func main() { // Create a new client with "client.FromEnv" (configuring the client // from commonly used environment variables such as DOCKER_HOST and // DOCKER_API_VERSION) and set a custom User-Agent. // // API-version negotiation is enabled by default to allow downgrading // the API version when connecting with an older daemon version. apiClient, err := client.New( client.FromEnv, client.WithUserAgent("my-application/1.0.0"), ) if err != nil { panic(err) } defer apiClient.Close() // List all containers (both stopped and running). result, err := apiClient.ContainerList(context.Background(), client.ContainerListOptions{ All: true, }) if err != nil { panic(err) } // Print each container's ID, status and the image it was created from. fmt.Printf("%s %-22s %s\n", "ID", "STATUS", "IMAGE") for _, ctr := range result.Items { fmt.Printf("%s %-22s %s\n", ctr.ID, ctr.Status, ctr.Image) } } ``` Full documentation is available on [pkg.go.dev](https://pkg.go.dev/github.com/moby/moby/client). --- ### Overview (`contrib/README.md`) The `contrib` directory contains scripts, images, and other helpful things which are not part of the core docker distribution. Please note that they could be out of date, since they do not receive the same attention as the rest of the repository. --- ### Overview (`hack/README.md`) ## About This directory contains a collection of scripts used to build and manage this repository. If there are any issues regarding the intention of a particular script (or even part of a certain script), please reach out to us. It may help us either refine our current scripts, or add on new ones that are appropriate for a given use case. ## DinD (dind.sh) DinD is a wrapper script which allows Docker to be run inside a Docker container. DinD requires the container to be run with privileged mode enabled. ## Generate Authors (generate-authors.sh) Generates AUTHORS; a file with all the names and corresponding emails of individual contributors. AUTHORS can be found in the home directory of this repository. ## Make There are two make files, each with different extensions. Neither are supposed to be called directly; only invoke `make`. Both scripts run inside a Docker container. ### make.ps1 - The Windows native build script that uses PowerShell semantics; it is limited unlike `hack\make.sh` since it does not provide support for the full set of operations provided by the Linux counterpart, `make.sh`. However, `make.ps1` does provide support for local Windows development and Windows to Windows CI. More information is found within `make.ps1` by the author, @jhowardmsft ### make.sh - Referenced via `make test` when running tests on a local machine, or directly referenced when running tests inside a Docker development container. - When running on a local machine, `make test` to run all tests found in `test`, `test-unit`, `test-integration`, and `test-docker-py` on your local machine. The default timeout is set in `make.sh` to 60 minutes (`${TIMEOUT:=60m}`), since it currently takes up to an hour to run all of the tests. - When running inside a Docker development container, `hack/make.sh` does not have a single target that runs all the tests. You need to provide a single command line with multiple targets that performs the same thing. An example referenced from [Run targets inside a development container](/docs/contributing/test.md#run-targets-inside-a-development-container): `root@5f8630b873fe:/go/src/github.com/moby/moby# hack/make.sh dynbinary binary test-unit test-integration test-docker-py` - For more information related to testing outside the scope of this README, refer to [Run tests and test documentation](/docs/contributing/test.md) ## Vendor (vendor.sh) A shell script that is a wrapper around `go mod vendor`. --- ### Overview (`man/README.md`) Docker Engine Documentation =========================== The man pages for Docker Engine are generated from the markdown sources and tooling in this directory. ## Generate the man pages Run `make` from within this directory. A Go toolchain is required. The generated man pages will be placed in man*N* subdirectories, where *N* is the manual section number. ## Install the man pages Run `make install` from within this directory. The make variables `prefix`, `mandir`, `INSTALL`, `INSTALL_DATA` and `DESTDIR` are supported for customizing the installation. ## Add a new man page Create a new Markdown file in this directory with a filename *TITLE*.*SECTION*.md, where *TITLE* is the man page title and *SECTION* is the section number. The Makefile will pick it up automatically. The Makefile ignores Markdown files that do not match the glob `*.*.md`, allowing non-manpage documentation (like this README file) to coexist. --- ### Overview (`pkg/README.md`) pkg/ is a collection of utility packages used by the Moby project without being specific to its internals. Utility packages are kept separate from the moby core codebase to keep it as small and concise as possible. If some utilities grow larger and their APIs stabilize, they may be moved to their own repository under the Moby organization, to facilitate re-use by other projects. However that is not the priority. The directory `pkg` is named after the same directory in the camlistore project. Since Brad is a core Go maintainer, we thought it made sense to copy his methods for organizing Go code :) Thanks Brad! Because utility packages are small and neatly separated from the rest of the codebase, they are a good place to start for aspiring maintainers and contributors. Get in touch if you want to help maintain them! --- ### Overview (`project/README.md`) # Hacking on Docker The `project/` directory holds information and tools for everyone involved in the process of creating and distributing Docker, specifically: ## Guides If you're a *contributor* or aspiring contributor, you should read [CONTRIBUTING.md](../CONTRIBUTING.md). If you're a *maintainer* or aspiring maintainer, you should read [MAINTAINERS](../MAINTAINERS). If you're a *packager* or aspiring packager, you should read [PACKAGERS.md](./PACKAGERS.md). If you're a maintainer in charge of a *release*, you should read [RELEASE-PROCESS.md](./RELEASE-PROCESS.md). ## Roadmap A high-level roadmap is available at [ROADMAP.md](../ROADMAP.md). ## Build tools [hack/make.sh](../hack/make.sh) is the primary build tool for docker. It is used for compiling the official binary, running the test suite, and pushing releases. --- ### Overview (`api/docs/README.md`) # API Documentation This directory contains versioned documents for each version of the API specification supported by this module. While this module provides support for older API versions, support should be considered "best-effort", especially for very old versions. Users are recommended to use the latest API versions, and only rely on older API versions for compatibility with older clients. Newer API versions tend to be backward-compatible with older versions, with some exceptions where features were deprecated. For an overview of changes for each version, refer to [CHANGELOG.md](CHANGELOG.md). The latest version of the API specification can be found [at the root directory of this module](../swagger.yaml) which may contain unreleased changes. For API version v1.24, documentation is only available in markdown format, for later versions [Swagger (OpenAPI) v2.0](https://swagger.io/specification/v2/) specifications can be found in this directory. The Moby project itself primarily uses these swagger files to produce the API documentation; while we attempt to make these files match the actual implementation, the OpenAPI 2.0 specification has limitations that prevent us from expressing all options provided. There may be discrepancies (for which we welcome contributions). If you find bugs, or discrepancies, please open a ticket (or pull request). --- ### Overview (`contrib/otel/README.md`) # Sample stack for testing OTEL functionality with moby To easily test the OTEL functionality present in moby, you can spin up a small demo compose stack that includes: - an OTEL collector container; - a Jaeger container to visualize traces; - an alternative Aspire Dashboard container to visualize traces; The OTEL collector is configured to export Traces to both the Jaeger and Aspire containers. The `contrib/otel` directory contains the compose file with the services configured, along with a basic configuration file for the OTEL collector. ## How can I use it? 1. Export the env var used to override the OTLP endpoint: `export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318` (if running in a devcontainer or in other ways, you might have to change how you pass this env var to the daemon); 2. Start the moby engine you want to get traces from (make sure it gets the env var declared above); 3. Start the otel compose stack by running `docker compose up -d` in the `contrib/otel/` directory; 4. Make some calls to the engine using the Docker CLI to send some traces to the endpoint; 5. Browse Jaeger at `http://localhost:16686` or the Aspire Dashboard at `http://localhost:18888/traces`; 6. To see some traces from the engine, select `dockerd` in the top left dropdown > **Note**: The precise steps may vary based on how you're working on the codebase (building a binary and executing natively, running/debugging in a devcontainer, etc... ) ## Cleanup? Simply run `docker compose down` in the `contrib/otel/` directory. You can also run `unset OTEL_EXPORTER_OTLP_ENDPOINT` to get rid of the OTLP env var from your environment --- ### Overview (`contrib/wireshark/README.md`) Wireshark Dissectors for NetworkDB ================================== `memberlist.lua` is a [Wireshark][] plugin which registers a `memberlist` protocol that can dissect the [memberlist][] TCP and UDP protocols. The `memberlist` protocol can be configured to dissect user data as the protocol named in the `memberlist.userdata_dissector` preference. `moby-networkdb.lua` is a Wireshark plugin which registers a protocol named `networkdbgossip` that dissects NetworkDB gossip messages. As node-to-node communications for NetworkDB are transported as memberlist user messages, the memberlist protocol dissector must be configured to invoke the networkdbgossip protocol as a sub-dissector. (Read: set the preference `memberlist.userdata_dissector:networkdbgossip`) Installation ------------ ### Install Wireshark 4.5 or newer. Wireshark 4.4 has an incomplete msgpack protocol dissector that is unable to properly decode memberlist messages. As of 2025-06-30 Wireshark 4.5 has yet to be released. A [nightly build][ws.dl] may be required. ### Install the plugins Configure Wireshark/Tshark to load `memberlist.lua` and `moby-networkdb.lua`. Refer to [the Wireshark Lua manual][ws.lua.intro] for further instruction. ### Configure the ProtoBuf protocol NetworkDB messages are serialized to protobuf, which is not self-describing. The ProtoBuf Wireshark protocol needs to be supplied with the protobuf IDL definitions of the messages in order to dissect them. 1. Clone [moby/moby][] for the NetworkDB IDL definitions. 2. Clone [protocolbuffers/protobuf][] for the protobuf "standard library" IDL. 3. Configure the ProtoBuf protocol (Preferences -> Protocols -> ProtoBuf) as follows: - ✅ Load .proto files on startup. (`protobuf.reload_protos`) - ✅ Dissect Protobuf fields as Wireshark fields. (`protobuf.pbf_as_hf`) - Add entries to the Protobuf Search Paths table (`uat:protobuf.search_paths`): - path/to/protocolbuffers/protobuf/src - path/to/moby/moby/vendor - path/to/moby/moby - path/to/moby/moby/libnetwork/networkdb (✅ Load all files) - path/to/moby/moby/libnetwork/drivers/overlay (✅ Load all files) Note that it is not sufficient to just grab the .proto files from the repos. The directory structure is necessary for the definitions to load properly. ### Configure the Memberlist protocol Configure memberlist to dissect user data messages as NetworkDB gossip. - In Preferences -> Protocols -> MEMBERLIST, set the User Data Dissector to `networkdbgossip`. (`memberlist.userdata_dissector`) - Optional: set Memberlist TCP+UDP port(s) (`memberlist.ports`) as needed. E.g. a value such as `7946,10000-10999` would be useful for analyzing packet captures from NetworkDB unit tests. Usage Notes ----------- ### Encryption The memberlist protocol dissector supports decryption of encrypted memberlist messages when provided with a file containing the encryption keys used. In Preferences -> Protocols -> MEMBERLIST, set the Encryption Key Logfile Path (`memberlist.keylog`) to a file containing the encryption keys. The logfile should list the encryption keys as hexadecimal strings, delimited by newlines. dockerd may be configured to write the NetworkDB encryption keys to a logfile by setting the environment variable `NETWORKDBKEYLOGFILE` to the path where the file should reside. ### Known Issues The NetworkDB protocol may fail to load with an error when Wireshark is first started: moby-networkdb.lua:4: bad argument #1 to 'new' (Field_new: a field with this name must exist) This is due to [a known issue in Wireshark.](https://gitlab.com/wireshark/wireshark/-/issues/20161) Workaround: reload Lua plugins after Wireshark has been initialized. - Menu: Analyze -> Reload Lua Plugins, or - Keyboard shortcut: Ctrl-Shift-L (Windows/Linux), ⇧⌘L (macOS) ### Limitations - Only memberlist encryption version 1 (AES-GCM 128, no padding) is supported, not version 0 (AES-GCM 128 using PKCS#7 padding). - Labelled messages cannot currently be decrypted. [memberlist]: https://github.com/hashicorp/memberlist [Wireshark]: https://wireshark.org [ws.dl]: https://www.wireshark.org/download/automated/ [ws.lua.intro]: https://www.wireshark.org/docs/wsdg_html_chunked/wsluarm.html#wsluarm_intro [moby/moby]: https://github.com/moby/moby [protocolbuffers/protobuf]: https://github.com/protocolbuffers/protobuf --- ### Overview (`docs/api/README.md`) # The API documentation has moved The API is now implemented in a separate module, and the documentation has moved to [api/docs](../../api/docs/README.md). --- ### Overview (`docs/contributing/README.md`) ### Get set up for Moby development * [README first](who-written-for.md) * [Get the required software](software-required.md) * [Set up for development on Windows](software-req-win.md) * (Optional) [Configure project for IDE](set-up-ide.md) * [Configure Git for contributing](set-up-git.md) * [Work with a development container](set-up-dev-env.md) * [Containerized build and cross compilation](ctn-build.md) * [Run tests and test documentation](test.md) * [Debugging the daemon](debug.md) --- ### Overview (`hack/make/README.md`) This directory holds scripts called by `make.sh` in the parent directory. Each script is named after the bundle it creates. They should not be called directly - instead, pass it as argument to make.sh, for example: ``` ./hack/make.sh binary ubuntu # Or to run all default bundles: ./hack/make.sh ``` To add a bundle: * Create a shell-compatible file here * Add it to $DEFAULT_BUNDLES in make.sh --- ### Overview (`pkg/useragent/README.md`) This package provides helper functions to pack version information into a single User-Agent header. --- ### Overview (`pkg/plugins/pluginrpc-gen/README.md`) Plugin RPC Generator ==================== Generates go code from a Go interface definition for proxying between the plugin API and the subsystem being extended. ## Usage Given an interface definition: ```go type volumeDriver interface { Create(name string, opts opts) (err error) Remove(name string) (err error) Path(name string) (mountpoint string, err error) Mount(name string) (mountpoint string, err error) Unmount(name string) (err error) } ``` **Note**: All function options and return values must be named in the definition. Run the generator: ```bash $ pluginrpc-gen --type volumeDriver --name VolumeDriver -i volume/drivers/extpoint.go -o volume/drivers/proxy.go ``` Where: - `--type` is the name of the interface to use - `--name` is the subsystem that the plugin "Implements" - `-i` is the input file containing the interface definition - `-o` is the output file where the generated code should go **Note**: The generated code will use the same package name as the one defined in the input file Optionally, you can skip functions on the interface that should not be implemented in the generated proxy code by passing in the function name to `--skip`. This flag can be specified multiple times. You can also add build tags that should be prepended to the generated code by supplying `--tag`. This flag can be specified multiple times. ## Annotations `pluginrpc-gen` supports annotations to customize the behavior of the generated code. These annotations are added as comments directly above the interface methods. ### Supported Annotations 1. **`pluginrpc-gen:timeout-type=`** - Specifies the timeout type to use for the method in the generated proxy code. - The `` can be: - `short`: Uses the `shortTimeout` constant (default: 1 minute). - `long`: Uses the `longTimeout` constant (default: 2 minutes). ### Usage To use the `pluginrpc-gen:timeout-type` annotation, place it directly above the method definition in the interface. To use the timeout value annotations, place them directly above the interface definition. #### Example ```go type volumeDriver interface { // pluginrpc-gen:timeout-type=long Create(name string, opts map[string]string) error // pluginrpc-gen:timeout-type=short Remove(name string) error } ``` ### Default Behavior - If no `pluginrpc-gen:timeout-type` annotation is provided, the `shortTimeout` value is used by default. ## Known issues ## go-generate You can also use this with go-generate, which is pretty awesome. To do so, place the code at the top of the file which contains the interface definition (i.e., the input file): ```go //go:generate pluginrpc-gen -i $GOFILE -o proxy.go -type volumeDriver -name VolumeDriver ``` Then cd to the package dir and run `go generate` **Note**: the `pluginrpc-gen` binary must be within your `$PATH` --- ### Overview (`vendor/cloud.google.com/go/README.md`) # Google Cloud Client Libraries for Go [](https://pkg.go.dev/cloud.google.com/go) Go packages for [Google Cloud Platform](https://cloud.google.com) services. ## Installation ```bash go get cloud.google.com/go/firestore@latest # Replace firestore with the package you want to use. ``` **NOTE:** Some of these packages are under development, and may occasionally make backwards-incompatible changes. ## Supported APIs For an updated list of all of our released APIs please see our [reference docs](https://cloud.google.com/go/docs/reference). ## [Go Versions Supported](#supported-versions) Our libraries are compatible with the two most recent major Go releases, the same [policy](https://go.dev/doc/devel/release#policy) the Go programming language follows. This means the currently supported versions are: - Go 1.24 - Go 1.25 ## Authentication By default, each client library will use [Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials) (ADC) to automatically configure the credentials used in calling the API endpoint. When using the libraries in a Google Cloud Platform environment such as Compute Engine, Kubernetes Engine, or App Engine, no additional authentication steps are necessary. See [Authentication methods at Google](https://cloud.google.com/docs/authentication) and [Authenticate for using client libraries](https://cloud.google.com/docs/authentication/client-libraries) for more information. ```go client, err := storage.NewClient(ctx) ``` For applications running elsewhere, such as your local development environment, you can use the `gcloud auth application-default login` command from the [Google Cloud CLI](https://cloud.google.com/cli) to set user credentials in your local filesystem. Application Default Credentials will automatically detect these credentials. See [Set up ADC for a local development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment) for more information. Alternately, you may need to provide an explicit path to your credentials. To authenticate using a [service account](https://cloud.google.com/docs/authentication#service-accounts) key file, either set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path to your key file, or programmatically pass [`option.WithCredentialsFile`](https://pkg.go.dev/google.golang.org/api/option#WithCredentialsFile) to the `NewClient` function of the desired package. For example: ```go client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json")) ``` You can exert even more control over authentication by using the [credentials](https://pkg.go.dev/cloud.google.com/go/auth/credentials) package to create an [auth.Credentials](https://pkg.go.dev/cloud.google.com/go/auth#Credentials). Then pass [`option.WithAuthCredentials`](https://pkg.go.dev/google.golang.org/api/option#WithAuthCredentials) to the `NewClient` function: ```go creds, err := credentials.DetectDefault(&credentials.DetectOptions{...}) ... client, err := storage.NewClient(ctx, option.WithAuthCredentials(creds)) ``` ## Contributing Contributions are welcome. Please, see the [CONTRIBUTING](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md) document for details. Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Contributor Code of Conduct](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md#contributor-code-of-conduct) for more information. ## Links - [Go on Google Cloud](https://cloud.google.com/go/home) - [Getting started with Go on Google Cloud](https://cloud.google.com/go/getting-started) - [App Engine Quickstart](https://cloud.google.com/appengine/docs/standard/go/quickstart) - [Cloud Functions Quickstart](https://cloud.google.com/functions/docs/quickstart-go) - [Cloud Run Quickstart](https://cloud.google.com/run/docs/quickstarts/build-and-deploy#go) --- ### Overview (`vendor/code.cloudfoundry.org/clock/README.md`) # Clock [![Go Report Card](https://goreportcard.com/badge/code.cloudfoundry.org/clock)](https://goreportcard.com/report/code.cloudfoundry.org/clock) [![Go Reference](https://pkg.go.dev/badge/code.cloudfoundry.org/clock.svg)](https://pkg.go.dev/code.cloudfoundry.org/clock) Provides a Clock interface, useful for injecting time dependencies in tests. > \[!NOTE\] > > This repository should be imported as `code.cloudfoundry.org/clock`. # Contributing See the [Contributing.md](./.github/CONTRIBUTING.md) for more information on how to contribute. # Working Group Charter This repository is maintained by [App Runtime Platform](https://github.com/cloudfoundry/community/blob/main/toc/working-groups/app-runtime-platform.md) under `Diego` area. > \[!IMPORTANT\] > > Content in this file is managed by the [CI task > `sync-readme`](https://github.com/cloudfoundry/wg-app-platform-runtime-ci/blob/main/shared/tasks/sync-readme/metadata.yml) > and is generated by CI following a convention. --- ### Overview (`vendor/dario.cat/mergo/README.md`) # Mergo [![GitHub release][5]][6] [![GoCard][7]][8] [![Test status][1]][2] [![OpenSSF Scorecard][21]][22] [![OpenSSF Best Practices][19]][20] [![Coverage status][9]][10] [![Sourcegraph][11]][12] [![FOSSA status][13]][14] [![GoDoc][3]][4] [![Become my sponsor][15]][16] [![Tidelift][17]][18] [1]: https://github.com/imdario/mergo/workflows/tests/badge.svg?branch=master [2]: https://github.com/imdario/mergo/actions/workflows/tests.yml [3]: https://godoc.org/github.com/imdario/mergo?status.svg [4]: https://godoc.org/github.com/imdario/mergo [5]: https://img.shields.io/github/release/imdario/mergo.svg [6]: https://github.com/imdario/mergo/releases [7]: https://goreportcard.com/badge/imdario/mergo [8]: https://goreportcard.com/report/github.com/imdario/mergo [9]: https://coveralls.io/repos/github/imdario/mergo/badge.svg?branch=master [10]: https://coveralls.io/github/imdario/mergo?branch=master [11]: https://sourcegraph.com/github.com/imdario/mergo/-/badge.svg [12]: https://sourcegraph.com/github.com/imdario/mergo?badge [13]: https://app.fossa.io/api/projects/git%2Bgithub.com%2Fimdario%2Fmergo.svg?type=shield [14]: https://app.fossa.io/projects/git%2Bgithub.com%2Fimdario%2Fmergo?ref=badge_shield [15]: https://img.shields.io/github/sponsors/imdario [16]: https://github.com/sponsors/imdario [17]: https://tidelift.com/badges/package/go/github.com%2Fimdario%2Fmergo [18]: https://tidelift.com/subscription/pkg/go-github.com-imdario-mergo [19]: https://bestpractices.coreinfrastructure.org/projects/7177/badge [20]: https://bestpractices.coreinfrastructure.org/projects/7177 [21]: https://api.securityscorecards.dev/projects/github.com/imdario/mergo/badge [22]: https://api.securityscorecards.dev/projects/github.com/imdario/mergo A helper to merge structs and maps in Golang. Useful for configuration default values, avoiding messy if-statements. Mergo merges same-type structs and maps by setting default values in zero-value fields. Mergo won't merge unexported (private) fields. It will do recursively any exported one. It also won't merge structs inside maps (because they are not addressable using Go reflection). Also a lovely [comune](http://en.wikipedia.org/wiki/Mergo) (municipality) in the Province of Ancona in the Italian region of Marche. ## Status Mergo is stable and frozen, ready for production. Check a short list of the projects using at large scale it [here](https://github.com/imdario/mergo#mergo-in-the-wild). No new features are accepted. They will be considered for a future v2 that improves the implementation and fixes bugs for corner cases. ### Important notes #### 1.0.0 In [1.0.0](//github.com/imdario/mergo/releases/tag/1.0.0) Mergo moves to a vanity URL `dario.cat/mergo`. No more v1 versions will be released. If the vanity URL is causing issues in your project due to a dependency pulling Mergo - it isn't a direct dependency in your project - it is recommended to use [replace](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive) to pin the version to the last one with the old import URL: ``` replace github.com/imdario/mergo => github.com/imdario/mergo v0.3.16 ``` #### 0.3.9 Please keep in mind that a problematic PR broke [0.3.9](//github.com/imdario/mergo/releases/tag/0.3.9). I reverted it in [0.3.10](//github.com/imdario/mergo/releases/tag/0.3.10), and I consider it stable but not bug-free. Also, this version adds support for go modules. Keep in mind that in [0.3.2](//github.com/imdario/mergo/releases/tag/0.3.2), Mergo changed `Merge()`and `Map()` signatures to support [transformers](#transformers). I added an optional/variadic argument so that it won't break the existing code. If you were using Mergo before April 6th, 2015, please check your project works as intended after updating your local copy with ```go get -u dario.cat/mergo```. I apologize for any issue caused by its previous behavior and any future bug that Mergo could cause in existing projects after the change (release 0.2.0). ### Donations If Mergo is useful to you, consider buying me a coffee, a beer, or making a monthly donation to allow me to keep building great free software. :heart_eyes: Donate using Liberapay Become my sponsor ### Mergo in the wild Mergo is used by [thousands](https://deps.dev/go/dario.cat%2Fmergo/v1.0.0/dependents) [of](https://deps.dev/go/github.com%2Fimdario%2Fmergo/v0.3.16/dependents) [projects](https://deps.dev/go/github.com%2Fimdario%2Fmergo/v0.3.12), including: * [containerd/containerd](https://github.com/containerd/containerd) * [datadog/datadog-agent](https://github.com/datadog/datadog-agent) * [docker/cli/](https://github.com/docker/cli/) * [goreleaser/goreleaser](https://github.com/goreleaser/goreleaser) * [go-micro/go-micro](https://github.com/go-micro/go-micro) * [grafana/loki](https://github.com/grafana/loki) * [masterminds/sprig](github.com/Masterminds/sprig) * [moby/moby](https://github.com/moby/moby) * [slackhq/nebula](https://github.com/slackhq/nebula) * [volcano-sh/volcano](https://github.com/volcano-sh/volcano) ## Install go get dario.cat/mergo // use in your .go code import ( "dario.cat/mergo" ) ## Usage You can only merge same-type structs with exported fields initialized as zero value of their type and same-types maps. Mergo won't merge unexported (private) fields but will do recursively any exported one. It won't merge empty structs value as [they are zero values](https://golang.org/ref/spec#The_zero_value) too. Also, maps will be merged recursively except for structs inside maps (because they are not addressable using Go reflection). ```go if err := mergo.Merge(&dst, src); err != nil { // ... } ``` Also, you can merge overwriting values using the transformer `WithOverride`. ```go if err := mergo.Merge(&dst, src, mergo.WithOverride); err != nil { // ... } ``` If you need to override pointers, so the source pointer's value is assigned to the destination's pointer, you must use `WithoutDereference`: ```go package main import ( "fmt" "dario.cat/mergo" ) type Foo struct { A *string B int64 } func main() { first := "first" second := "second" src := Foo{ A: &first, B: 2, } dest := Foo{ A: &second, B: 1, } mergo.Merge(&dest, src, mergo.WithOverride, mergo.WithoutDereference) } ``` Additionally, you can map a `map[string]interface{}` to a struct (and otherwise, from struct to map), following the same restrictions as in `Merge()`. Keys are capitalized to find each corresponding exported field. ```go if err := mergo.Map(&dst, srcMap); err != nil { // ... } ``` Warning: if you map a struct to map, it won't do it recursively. Don't expect Mergo to map struct members of your struct as `map[string]interface{}`. They will be just assigned as values. Here is a nice example: ```go package main import ( "fmt" "dario.cat/mergo" ) type Foo struct { A string B int64 } func main() { src := Foo{ A: "one", B: 2, } dest := Foo{ A: "two", } mergo.Merge(&dest, src) fmt.Println(dest) // Will print // {two 2} } ``` ### Transformers Transformers allow to merge specific types differently than in the default behavior. In other words, now you can customize how some types are merged. For example, `time.Time` is a struct; it doesn't have zero value but IsZero can return true because it has fields with zero value. How can we merge a non-zero `time.Time`? ```go package main import ( "fmt" "dario.cat/mergo" "reflect" "time" ) type timeTransformer struct { } func (t timeTransformer) Transformer(typ reflect.Type) func(dst, src reflect.Value) error { if typ == reflect.TypeOf(time.Time{}) { return func(dst, src reflect.Value) error { if dst.CanSet() { isZero := dst.MethodByName("IsZero") result := isZero.Call([]reflect.Value{}) if result[0].Bool() { dst.Set(src) } } return nil } } return nil } type Snapshot struct { Time time.Time // ... } func main() { src := Snapshot{time.Now()} dest := Snapshot{} mergo.Merge(&dest, src, mergo.WithTransformers(timeTransformer{})) fmt.Println(dest) // Will print // { 2018-01-12 01:15:00 +0000 UTC m=+0.000000001 } } ``` ## Contact me If I can help you, you have an idea or you are using Mergo in your projects, don't hesitate to drop me a line (or a pull request): [@im_dario](https://twitter.com/im_dario) ## About Written by [Dario Castañé](http://dario.im). ## License [BSD 3-Clause](http://opensource.org/licenses/BSD-3-Clause) license, as [Go language](http://golang.org/LICENSE). [](https://app.fossa.io/projects/git%2Bgithub.com%2Fimdario%2Fmergo?ref=badge_large) --- ### Overview (`vendor/go.etcd.io/bbolt/README.md`) bbolt ===== [](https://goreportcard.com/report/go.etcd.io/bbolt) [](https://pkg.go.dev/go.etcd.io/bbolt) [](https://github.com/etcd-io/bbolt/releases) [](https://github.com/etcd-io/bbolt/blob/master/LICENSE) bbolt is a fork of [Ben Johnson's][gh_ben] [Bolt][bolt] key/value store. The purpose of this fork is to provide the Go community with an active maintenance and development target for Bolt; the goal is improved reliability and stability. bbolt includes bug fixes, performance enhancements, and features not found in Bolt while preserving backwards compatibility with the Bolt API. Bolt is a pure Go key/value store inspired by [Howard Chu's][hyc_symas] [LMDB project][lmdb]. The goal of the project is to provide a simple, fast, and reliable database for projects that don't require a full database server such as Postgres or MySQL. Since Bolt is meant to be used as such a low-level piece of functionality, simplicity is key. The API will be small and only focus on getting values and setting values. That's it. [gh_ben]: https://github.com/benbjohnson [bolt]: https://github.com/boltdb/bolt [hyc_symas]: https://twitter.com/hyc_symas [lmdb]: https://www.symas.com/symas-embedded-database-lmdb ## Project Status Bolt is stable, the API is fixed, and the file format is fixed. Full unit test coverage and randomized black box testing are used to ensure database consistency and thread safety. Bolt is currently used in high-load production environments serving databases as large as 1TB. Many companies such as Shopify and Heroku use Bolt-backed services every day. ## Project versioning bbolt uses [semantic versioning](http://semver.org). API should not change between patch and minor releases. New minor versions may add additional features to the API. ## Table of Contents - [Getting Started](#getting-started) - [Installing](#installing) - [Opening a database](#opening-a-database) - [Transactions](#transactions) - [Read-write transactions](#read-write-transactions) - [Read-only transactions](#read-only-transactions) - [Batch read-write transactions](#batch-read-write-transactions) - [Managing transactions manually](#managing-transactions-manually) - [Using buckets](#using-buckets) - [Using key/value pairs](#using-keyvalue-pairs) - [Autoincrementing integer for the bucket](#autoincrementing-integer-for-the-bucket) - [Iterating over keys](#iterating-over-keys) - [Prefix scans](#prefix-scans) - [Range scans](#range-scans) - [ForEach()](#foreach) - [Nested buckets](#nested-buckets) - [Database backups](#database-backups) - [Statistics](#statistics) - [Read-Only Mode](#read-only-mode) - [Mobile Use (iOS/Android)](#mobile-use-iosandroid) - [Resources](#resources) - [Comparison with other databases](#comparison-with-other-databases) - [Postgres, MySQL, & other relational databases](#postgres-mysql--other-relational-databases) - [LevelDB, RocksDB](#leveldb-rocksdb) - [LMDB](#lmdb) - [Caveats & Limitations](#caveats--limitations) - [Reading the Source](#reading-the-source) - [Known Issues](#known-issues) - [Other Projects Using Bolt](#other-projects-using-bolt) ## Getting Started ### Installing To start using `bbolt`, install Go and run `go get`: ```sh $ go get go.etcd.io/bbolt@latest ``` This will retrieve the library and update your `go.mod` and `go.sum` files. To run the command line utility, execute: ```sh $ go run go.etcd.io/bbolt/cmd/bbolt@latest ``` Run `go install` to install the `bbolt` command line utility into your `$GOBIN` path, which defaults to `$GOPATH/bin` or `$HOME/go/bin` if the `GOPATH` environment variable is not set. ```sh $ go install go.etcd.io/bbolt/cmd/bbolt@latest ``` ### Importing bbolt To use bbolt as an embedded key-value store, import as: ```go import bolt "go.etcd.io/bbolt" db, err := bolt.Open(path, 0600, nil) if err != nil { return err } defer db.Close() ``` ### Opening a database The top-level object in Bolt is a `DB`. It is represented as a single file on your disk and represents a consistent snapshot of your data. To open your database, simply use the `bolt.Open()` function: ```go package main import ( "log" bolt "go.etcd.io/bbolt" ) func main() { // Open the my.db data file in your current directory. // It will be created if it doesn't exist. db, err := bolt.Open("my.db", 0600, nil) if err != nil { log.Fatal(err) } defer db.Close() ... } ``` Please note that Bolt obtains a file lock on the data file so multiple processes cannot open the same database at the same time. Opening an already open Bolt database will cause it to hang until the other process closes it. To prevent an indefinite wait you can pass a timeout option to the `Open()` function: ```go db, err := bolt.Open("my.db", 0600, &bolt.Options{Timeout: 1 * time.Second}) ``` ### Transactions Bolt allows only one read-write transaction at a time but allows as many read-only transactions as you want at a time. Each transaction has a consistent view of the data as it existed when the transaction started. Individual transactions and all objects created from them (e.g. buckets, keys) are not thread safe. To work with data in multiple goroutines you must start a transaction for each one or use locking to ensure only one goroutine accesses a transaction at a time. Creating transaction from the `DB` is thread safe. Transactions should not depend on one another and generally shouldn't be opened simultaneously in the same goroutine. This can cause a deadlock as the read-write transaction needs to periodically re-map the data file but it cannot do so while any read-only transaction is open. Even a nested read-only transaction can cause a deadlock, as the child transaction can block the parent transaction from releasing its resources. #### Read-write transactions To start a read-write transaction, you can use the `DB.Update()` function: ```go err := db.Update(func(tx *bolt.Tx) error { ... return nil }) ``` Inside the closure, you have a consistent view of the database. You commit the transaction by returning `nil` at the end. You can also rollback the transaction at any point by returning an error. All database operations are allowed inside a read-write transaction. Always check the return error as it will report any disk failures that can cause your transaction to not complete. If you return an error within your closure it will be passed through. #### Read-only transactions To start a read-only transaction, you can use the `DB.View()` function: ```go err := db.View(func(tx *bolt.Tx) error { ... return nil }) ``` You also get a consistent view of the database within this closure, however, no mutating operations are allowed within a read-only transaction. You can only retrieve buckets, retrieve values, and copy the database within a read-only transaction. #### Batch read-write transactions Each `DB.Update()` waits for disk to commit the writes. This overhead can be minimized by combining multiple updates with the `DB.Batch()` function: ```go err := db.Batch(func(tx *bolt.Tx) error { ... return nil }) ``` Concurrent Batch calls are opportunistically combined into larger transactions. Batch is only useful when there are multiple goroutines calling it. The trade-off is that `Batch` can call the given function multiple times, if parts of the transaction fail. The function must be idempotent and side effects must take effect only after a successful return from `DB.Batch()`. For example: don't display messages from inside the function, instead set variables in the enclosing scope: ```go var id uint64 err := db.Batch(func(tx *bolt.Tx) error { // Find last key in bucket, decode as bigendian uint64, increment // by one, encode back to []byte, and add new key. ... id = newValue return nil }) if err != nil { return ... } fmt.Println("Allocated ID %d", id) ``` #### Managing transactions manually The `DB.View()` and `DB.Update()` functions are wrappers around the `DB.Begin()` function. These helper functions will start the transaction, execute a function, and then safely close your transaction if an error is returned. This is the recommended way to use Bolt transactions. However, sometimes you may want to manually start and end your transactions. You can use the `DB.Begin()` function directly but **please** be sure to close the transaction. ```go // Start a writable transaction. tx, err := db.Begin(true) if err != nil { return err } defer tx.Rollback() // Use the transaction... _, err := tx.CreateBucket([]byte("MyBucket")) if err != nil { return err } // Commit the transaction and check for error. if err := tx.Commit(); err != nil { return err } ``` The first argument to `DB.Begin()` is a boolean stating if the transaction should be writable. ### Using buckets Buckets are collections of key/value pairs within the database. All keys in a bucket must be unique. You can create a bucket using the `Tx.CreateBucket()` function: ```go db.Update(func(tx *bolt.Tx) error { b, err := tx.CreateBucket([]byte("MyBucket")) if err != nil { return fmt.Errorf("create bucket: %s", err) } return nil }) ``` You can retrieve an existing bucket using the `Tx.Bucket()` function: ```go db.Update(func(tx *bolt.Tx) error { b := tx.Bucket([]byte("MyBucket")) if b == nil { return errors.New("bucket does not exist") } return nil }) ``` You can also create a bucket only if it doesn't exist by using the `Tx.CreateBucketIfNotExists()` function. It's a common pattern to call this function for all your top-level buckets after you open your database so you can guarantee that they exist for future transactions. To delete a bucket, simply call the `Tx.DeleteBucket()` function. You can also iterate over all existing top-level buckets with `Tx.ForEach()`: ```go db.View(func(tx *bolt.Tx) error { tx.ForEach(func(name []byte, b *bolt.Bucket) error { fmt.Println(string(name)) return nil }) return nil }) ``` ### Using key/value pairs To save a key/value pair to a bucket, use the `Bucket.Put()` function: ```go db.Update(func(tx *bolt.Tx) error { b := tx.Bucket([]byte("MyBucket")) err := b.Put([]byte("answer"), []byte("42")) return err }) ``` This will set the value of the `"answer"` key to `"42"` in the `MyBucket` bucket. To retrieve this value, we can use the `Bucket.Get()` function: ```go db.View(func(tx *bolt.Tx) error { b := tx.Bucket([]byte("MyBucket")) v := b.Get([]byte("answer")) fmt.Printf("The answer is: %s\n", v) return nil }) ``` The `Get()` function does not return an error because its operation is guaranteed to work (unless there is some kind of system failure). If the key exists then it will return its byte slice value. If it doesn't exist then it will return `nil`. It's important to note that you can have a zero-length value set to a key which is different than the key not existing. Use the `Bucket.Delete()` function to delete a key from the bucket: ```go db.Update(func (tx *bolt.Tx) error { b := tx.Bucket([]byte("MyBucket")) err := b.Delete([]byte("answer")) return err }) ``` This will delete the key `answers` from the bucket `MyBucket`. Please note that values returned from `Get()` are only valid while the transaction is open. If you need to use a value outside of the transaction then you must use `copy()` to copy it to another byte slice. ### Autoincrementing integer for the bucket By using the `NextSequence()` function, you can let Bolt determine a sequence which can be used as the unique identifier for your key/value pairs. See the example below. ```go // CreateUser saves u to the store. The new user ID is set on u once the data is persisted. func (s *Store) CreateUser(u *User) error { return s.db.Update(func(tx *bolt.Tx) error { // Retrieve the users bucket. // This should be created when the DB is first opened. b := tx.Bucket([]byte("users")) // Generate ID for the user. // This returns an error only if the Tx is closed or not writeable. // That can't happen in an Update() call so I ignore the error check. id, _ := b.NextSequence() u.ID = int(id) // Marshal user data into bytes. buf, err := json.Marshal(u) if err != nil { return err } // Persist bytes to users bucket. return b.Put(itob(u.ID), buf) }) } // itob returns an 8-byte big endian representation of v. func itob(v int) []byte { b := make([]byte, 8) binary.BigEndian.PutUint64(b, uint64(v)) return b } type User struct { ID int ... } ``` ### Iterating over keys Bolt stores its keys in byte-sorted order within a bucket. This makes sequential iteration over these keys extremely fast. To iterate over keys we'll use a `Cursor`: ```go db.View(func(tx *bolt.Tx) error { // Assume bucket exists and has keys b := tx.Bucket([]byte("MyBucket")) c := b.Cursor() for k, v := c.First(); k != nil; k, v = c.Next() { fmt.Printf("key=%s, value=%s\n", k, v) } return nil }) ``` The cursor allows you to move to a specific point in the list of keys and move forward or backward through the keys one at a time. The following functions are available on the cursor: ``` First() Move to the first key. Last() Move to the last key. Seek() Move to a specific key. Next() Move to the next key. Prev() Move to the previous key. ``` Each of those functions has a return signature of `(key []byte, value []byte)`. You must seek to a position using `First()`, `Last()`, or `Seek()` before calling `Next()` or `Prev()`. If you do not seek to a position then these functions will return a `nil` key. When you have iterated to the end of the cursor, then `Next()` will return a `nil` key and the cursor still points to the last element if present. When you have iterated to the beginning of the cursor, then `Prev()` will return a `nil` key and the cursor still points to the first element if present. If you remove key/value pairs during iteration, the cursor may automatically move to the next position if present in current node each time removing a key. When you call `c.Next()` after removing a key, it may skip one key/value pair. Refer to [pull/611](https://github.com/etcd-io/bbolt/pull/611) to get more detailed info. During iteration, if the key is non-`nil` but the value is `nil`, that means the key refers to a bucket rather than a value. Use `Bucket.Bucket()` to access the sub-bucket. #### Prefix scans To iterate over a key prefix, you can combine `Seek()` and `bytes.HasPrefix()`: ```go db.View(func(tx *bolt.Tx) error { // Assume bucket exists and has keys c := tx.Bucket([]byte("MyBucket")).Cursor() prefix := []byte("1234") for k, v := c.Seek(prefix); k != nil && bytes.HasPrefix(k, prefix); k, v = c.Next() { fmt.Printf("key=%s, value=%s\n", k, v) } return nil }) ``` #### Range scans Another common use case is scanning over a range such as a time range. If you use a sortable time encoding such as RFC3339 then you can query a specific date range like this: ```go db.View(func(tx *bolt.Tx) error { // Assume our events bucket exists and has RFC3339 encoded time keys. c := tx.Bucket([]byte("Events")).Cursor() // Our time range spans the 90's decade. min := []byte("1990-01-01T00:00:00Z") max := []byte("2000-01-01T00:00:00Z") // Iterate over the 90's. for k, v := c.Seek(min); k != nil && bytes.Compare(k, max) <= 0; k, v = c.Next() { fmt.Printf("%s: %s\n", k, v) } return nil }) ``` Note that, while RFC3339 is sortable, the Golang implementation of RFC3339Nano does not use a fixed number of digits after the decimal point and is therefore not sortable. #### ForEach() You can also use the function `ForEach()` if you know you'll be iterating over all the keys in a bucket: ```go db.View(func(tx *bolt.Tx) error { // Assume bucket exists and has keys b := tx.Bucket([]byte("MyBucket")) b.ForEach(func(k, v []byte) error { fmt.Printf("key=%s, value=%s\n", k, v) return nil }) return nil }) ``` Please note that keys and values in `ForEach()` are only valid while the transaction is open. If you need to use a key or value outside of the transaction, you must use `copy()` to copy it to another byte slice. ### Nested buckets You can also store a bucket in a key to create nested buckets. The API is the same as the bucket management API on the `DB` object: ```go func (*Bucket) CreateBucket(key []byte) (*Bucket, error) func (*Bucket) CreateBucketIfNotExists(key []byte) (*Bucket, error) func (*Bucket) DeleteBucket(key []byte) error ``` Say you had a multi-tenant application where the root level bucket was the account bucket. Inside of this bucket was a sequence of accounts which themselves are buckets. And inside the sequence bucket you could have many buckets pertaining to the Account itself (Users, Notes, etc) isolating the information into logical groupings. ```go // createUser creates a new user in the given account. func createUser(accountID int, u *User) error { // Start the transaction. tx, err := db.Begin(true) if err != nil { return err } defer tx.Rollback() // Retrieve the root bucket for the account. // Assume this has already been created when the account was set up. root := tx.Bucket([]byte(strconv.FormatUint(accountID, 10))) // Setup the users bucket. bkt, err := root.CreateBucketIfNotExists([]byte("USERS")) if err != nil { return err } // Generate an ID for the new user. userID, err := bkt.NextSequence() if err != nil { return err } u.ID = userID // Marshal and save the encoded user. if buf, err := json.Marshal(u); err != nil { return err } else if err := bkt.Put([]byte(strconv.FormatUint(u.ID, 10)), buf); err != nil { return err } // Commit the transaction. if err := tx.Commit(); err != nil { return err } return nil } ``` ### Database backups Bolt is a single file so it's easy to backup. You can use the `Tx.WriteTo()` function to write a consistent view of the database to a writer. If you call this from a read-only transaction, it will perform a hot backup and not block your other database reads and writes. By default, it will use a regular file handle which will utilize the operating system's page cache. See the [`Tx`](https://godoc.org/go.etcd.io/bbolt#Tx) documentation for information about optimizing for larger-than-RAM datasets. One common use case is to backup over HTTP so you can use tools like `cURL` to do database backups: ```go func BackupHandleFunc(w http.ResponseWriter, req *http.Request) { err := db.View(func(tx *bolt.Tx) error { w.Header().Set("Content-Type", "application/octet-stream") w.Header().Set("Content-Disposition", `attachment; filename="my.db"`) w.Header().Set("Content-Length", strconv.Itoa(int(tx.Size()))) _, err := tx.WriteTo(w) return err }) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) } } ``` Then you can backup using this command: ```sh $ curl http://localhost/backup > my.db ``` Or you can open your browser to `http://localhost/backup` and it will download automatically. If you want to backup to another file you can use the `Tx.CopyFile()` helper function. ### Statistics The database keeps a running count of many of the internal operations it performs so you can better understand what's going on. By grabbing a snapshot of these stats at two points in time we can see what operations were performed in that time range. For example, we could start a goroutine to log stats every 10 seconds: ```go go func() { // Grab the initial stats. prev := db.Stats() for { // Wait for 10s. time.Sleep(10 * time.Second) // Grab the current stats and diff them. stats := db.Stats() diff := stats.Sub(&prev) // Encode stats to JSON and print to STDERR. json.NewEncoder(os.Stderr).Encode(diff) // Save stats for the next loop. prev = stats } }() ``` It's also useful to pipe these stats to a service such as statsd for monitoring or to provide an HTTP endpoint that will perform a fixed-length sample. ### Read-Only Mode Sometimes it is useful to create a shared, read-only Bolt database. To this, set the `Options.ReadOnly` flag when opening your database. Read-only mode uses a shared lock to allow multiple processes to read from the database but it will block any processes from opening the database in read-write mode. ```go db, err := bolt.Open("my.db", 0600, &bolt.Options{ReadOnly: true}) if err != nil { log.Fatal(err) } ``` ### Mobile Use (iOS/Android) Bolt is able to run on mobile devices by leveraging the binding feature of the [gomobile](https://github.com/golang/mobile) tool. Create a struct that will contain your database logic and a reference to a `*bolt.DB` with a initializing constructor that takes in a filepath where the database file will be stored. Neither Android nor iOS require extra permissions or cleanup from using this method. ```go func NewBoltDB(filepath string) *BoltDB { db, err := bolt.Open(filepath+"/demo.db", 0600, nil) if err != nil { log.Fatal(err) } return &BoltDB{db} } type BoltDB struct { db *bolt.DB ... } func (b *BoltDB) Path() string { return b.db.Path() } func (b *BoltDB) Close() { b.db.Close() } ``` Database logic should be defined as methods on this wrapper struct. To initialize this struct from the native language (both platforms now sync their local storage to the cloud. These snippets disable that functionality for the database file): #### Android ```java String path; if (android.os.Build.VERSION.SDK_INT >=android.os.Build.VERSION_CODES.LOLLIPOP){ path = getNoBackupFilesDir().getAbsolutePath(); } else{ path = getFilesDir().getAbsolutePath(); } Boltmobiledemo.BoltDB boltDB = Boltmobiledemo.NewBoltDB(path) ``` #### iOS ```objc - (void)demo { NSString* path = [NSSearchPathForDirectoriesInDomains(NSLibraryDirectory, NSUserDomainMask, YES) objectAtIndex:0]; GoBoltmobiledemoBoltDB * demo = GoBoltmobiledemoNewBoltDB(path); [self addSkipBackupAttributeToItemAtPath:demo.path]; //Some DB Logic would go here [demo close]; } - (BOOL)addSkipBackupAttributeToItemAtPath:(NSString *) filePathString { NSURL* URL= [NSURL fileURLWithPath: filePathString]; assert([[NSFileManager defaultManager] fileExistsAtPath: [URL path]]); NSError *error = nil; BOOL success = [URL setResourceValue: [NSNumber numberWithBool: YES] forKey: NSURLIsExcludedFromBackupKey error: &error]; if(!success){ NSLog(@"Error excluding %@ from backup %@", [URL lastPathComponent], error); } return success; } ``` ## Resources For more information on getting started with Bolt, check out the following articles: * [Intro to BoltDB: Painless Performant Persistence](http://npf.io/2014/07/intro-to-boltdb-painless-performant-persistence/) by [Nate Finch](https://github.com/natefinch). * [Bolt -- an embedded key/value database for Go](https://www.progville.com/go/bolt-embedded-db-golang/) by Progville ## Comparison with other databases ### Postgres, MySQL, & other relational databases Relational databases structure data into rows and are only accessible through the use of SQL. This approach provides flexibility in how you store and query your data but also incurs overhead in parsing and planning SQL statements. Bolt accesses all data by a byte slice key. This makes Bolt fast to read and write data by key but provides no built-in support for joining values together. Most relational databases (with the exception of SQLite) are standalone servers that run separately from your application. This gives your systems flexibility to connect multiple application servers to a single database server but also adds overhead in serializing and transporting data over the network. Bolt runs as a library included in your application so all data access has to go through your application's process. This brings data closer to your application but limits multi-process access to the data. ### LevelDB, RocksDB LevelDB and its derivatives (RocksDB, HyperLevelDB) are similar to Bolt in that they are libraries bundled into the application, however, their underlying structure is a log-structured merge-tree (LSM tree). An LSM tree optimizes random writes by using a write ahead log and multi-tiered, sorted files called SSTables. Bolt uses a B+tree internally and only a single file. Both approaches have trade-offs. If you require a high random write throughput (>10,000 w/sec) or you need to use spinning disks then LevelDB could be a good choice. If your application is read-heavy or does a lot of range scans then Bolt could be a good choice. One other important consideration is that LevelDB does not have transactions. It supports batch writing of key/values pairs and it supports read snapshots but it will not give you the ability to do a compare-and-swap operation safely. Bolt supports fully serializable ACID transactions. ### LMDB Bolt was originally a port of LMDB so it is architecturally similar. Both use a B+tree, have ACID semantics with fully serializable transactions, and support lock-free MVCC using a single writer and multiple readers. The two projects have somewhat diverged. LMDB heavily focuses on raw performance while Bolt has focused on simplicity and ease of use. For example, LMDB allows several unsafe actions such as direct writes for the sake of performance. Bolt opts to disallow actions which can leave the database in a corrupted state. The only exception to this in Bolt is `DB.NoSync`. There are also a few differences in API. LMDB requires a maximum mmap size when opening an `mdb_env` whereas Bolt will handle incremental mmap resizing automatically. LMDB overloads the getter and setter functions with multiple flags whereas Bolt splits these specialized cases into their own functions. ## Caveats & Limitations It's important to pick the right tool for the job and Bolt is no exception. Here are a few things to note when evaluating and using Bolt: * Bolt is good for read intensive workloads. Sequential write performance is also fast but random writes can be slow. You can use `DB.Batch()` or add a write-ahead log to help mitigate this issue. * Bolt uses a B+tree internally so there can be a lot of random page access. SSDs provide a significant performance boost over spinning disks. * Try to avoid long running read transactions. Bolt uses copy-on-write so old pages cannot be reclaimed while an old transaction is using them. * Byte slices returned from Bolt are only valid during a transaction. Once the transaction has been committed or rolled back then the memory they point to can be reused by a new page or can be unmapped from virtual memory and you'll see an `unexpected fault address` panic when accessing it. * Bolt uses an exclusive write lock on the database file so it cannot be shared by multiple processes. * Be careful when using `Bucket.FillPercent`. Setting a high fill percent for buckets that have random inserts will cause your database to have very poor page utilization. * Use larger buckets in general. Smaller buckets causes poor page utilization once they become larger than the page size (typically 4KB). * Bulk loading a lot of random writes into a new bucket can be slow as the page will not split until the transaction is committed. Randomly inserting more than 100,000 key/value pairs into a single new bucket in a single transaction is not advised. * Bolt uses a memory-mapped file so the underlying operating system handles the caching of the data. Typically, the OS will cache as much of the file as it can in memory and will release memory as needed to other processes. This means that Bolt can show very high memory usage when working with large databases. However, this is expected and the OS will release memory as needed. Bolt can handle databases much larger than the available physical RAM, provided its memory-map fits in the process virtual address space. It may be problematic on 32-bits systems. * The data structures in the Bolt database are memory mapped so the data file will be endian specific. This means that you cannot copy a Bolt file from a little endian machine to a big endian machine and have it work. For most users this is not a concern since most modern CPUs are little endian. * Because of the way pages are laid out on disk, Bolt cannot truncate data files and return free pages back to the disk. Instead, Bolt maintains a free list of unused pages within its data file. These free pages can be reused by later transactions. This works well for many use cases as databases generally tend to grow. However, it's important to note that deleting large chunks of data will not allow you to reclaim that space on disk. For more information on page allocation, [see this comment][page-allocation]. * Removing key/values pairs in a bucket during iteration on the bucket using cursor may not work properly. Each time when removing a key/value pair, the cursor may automatically move to the next position if present. When users call `c.Next()` after removing a key, it may skip one key/value pair. Refer to https://github.com/etcd-io/bbolt/pull/611 for more detailed info. * Bolt db can be corrupted during the initialization phase due to abrupt power failure. - Please note: This issue can only be reproduced during the very first initialization phase, when there is no existing data in bolt database. - In normal production environment, it is difficult to reproduce this. Once the database file has been initialized, it can no longer occur. - Please refer to this issue for more details: https://github.com/etcd-io/etcd/issues/16596. [page-allocation]: https://github.com/boltdb/bolt/issues/308#issuecomment-74811638 ## Reading the Source Bolt is a relatively small code base (<5KLOC) for an embedded, serializable, transactional key/value database so it can be a good starting point for people interested in how databases work. The best places to start are the main entry points into Bolt: - `Open()` - Initializes the reference to the database. It's responsible for creating the database if it doesn't exist, obtaining an exclusive lock on the file, reading the meta pages, & memory-mapping the file. - `DB.Begin()` - Starts a read-only or read-write transaction depending on the value of the `writable` argument. This requires briefly obtaining the "meta" lock to keep track of open transactions. Only one read-write transaction can exist at a time so the "rwlock" is acquired during the life of a read-write transaction. - `Bucket.Put()` - Writes a key/value pair into a bucket. After validating the arguments, a cursor is used to traverse the B+tree to the page and position where the key & value will be written. Once the position is found, the bucket materializes the underlying page and the page's parent pages into memory as "nodes". These nodes are where mutations occur during read-write transactions. These changes get flushed to disk during commit. - `Bucket.Get()` - Retrieves a key/value pair from a bucket. This uses a cursor to move to the page & position of a key/value pair. During a read-only transaction, the key and value data is returned as a direct reference to the underlying mmap file so there's no allocation overhead. For read-write transactions, this data may reference the mmap file or one of the in-memory node values. - `Cursor` - This object is simply for traversing the B+tree of on-disk pages or in-memory nodes. It can seek to a specific key, move to the first or last value, or it can move forward or backward. The cursor handles the movement up and down the B+tree transparently to the end user. - `Tx.Commit()` - Converts the in-memory dirty nodes and the list of free pages into pages to be written to disk. Writing to disk then occurs in two phases. First, the dirty pages are written to disk and an `fsync()` occurs. Second, a new meta page with an incremented transaction ID is written and another `fsync()` occurs. This two phase write ensures that partially written data pages are ignored in the event of a crash since the meta page pointing to them is never written. Partially written meta pages are invalidated because they are written with a checksum. If you have additional notes that could be helpful for others, please submit them via pull request. ## Known Issues - bbolt might run into data corruption issue on Linux when the feature [ext4: fast commit](https://lwn.net/Articles/842385/), which was introduced in linux kernel version v5.10, is enabled. The fixes to the issue are included in stable LTS patchlevels 5.10.94+ and 5.15.17+ (ftruncate tracking), plus 5.15.27+ (ineligible-commit fallback). Linux 5.17 includes these fixes as well, but 5.17 is not an LTS release. Please refer to links below, * [ext4: fast commit may miss tracking unwritten range during ftruncate](https://lore.kernel.org/linux-ext4/20211223032337.5198-3-yinxin.x@bytedance.com/) * [5.10.94 stable backport](https://lore.kernel.org/stable/20220124184041.063143682@linuxfoundation.org/) * [5.15.17 stable backport](https://lore.kernel.org/stable/20220124184125.887304707@linuxfoundation.org/) * [ext4: fast commit may not fallback for ineligible commit](https://lore.kernel.org/lkml/202201091544.W5HHEXAp-lkp@intel.com/T/#ma0768815e4b5f671e9e451d578256ef9a76fe30e) * [5.15.27 stable backport](https://lore.kernel.org/stable/20220307091703.544901888@linuxfoundation.org/) * [ext4 updates for 5.17](https://lore.kernel.org/lkml/YdyxjTFaLWif6BCM@mit.edu/) Please also refer to the discussion in https://github.com/etcd-io/bbolt/issues/562. - Writing a value with a length of 0 will always result in reading back an empty `[]byte{}` value. Please refer to [issues/726#issuecomment-2061694802](https://github.com/etcd-io/bbolt/issues/726#issuecomment-2061694802). ## Other Projects Using Bolt Below is a list of public, open source projects that use Bolt: * [Algernon](https://github.com/xyproto/algernon) - A HTTP/2 web server with built-in support for Lua. Uses BoltDB as the default database backend. * [Bazil](https://bazil.org/) - A file system that lets your data reside where it is most convenient for it to reside. * [bolter](https://github.com/hasit/bolter) - Command-line app for viewing BoltDB file in your terminal. * [boltcli](https://github.com/spacewander/boltcli) - the redis-cli for boltdb with Lua script support. * [BoltHold](https://github.com/timshannon/bolthold) - An embeddable NoSQL store for Go types built on BoltDB * [BoltStore](https://github.com/yosssi/boltstore) - Session store using Bolt. * [Boltdb Boilerplate](https://github.com/bobintornado/boltdb-boilerplate) - Boilerplate wrapper around bolt aiming to make simple calls one-liners. * [BoltDbWeb](https://github.com/evnix/boltdbweb) - A web based GUI for BoltDB files. * [BoltDB Viewer](https://github.com/zc310/rich_boltdb) - A BoltDB Viewer Can run on Windows、Linux、Android system. * [bleve](http://www.blevesearch.com/) - A pure Go search engine similar to ElasticSearch that uses Bolt as the default storage backend. * [bstore](https://github.com/mjl-/bstore) - Database library storing Go values, with referential/unique/nonzero constraints, indices, automatic schema management with struct tags, and a query API. * [btcwallet](https://github.com/btcsuite/btcwallet) - A bitcoin wallet. * [buckets](https://github.com/joyrexus/buckets) - a bolt wrapper streamlining simple tx and key scans. * [Buildkit](https://github.com/moby/buildkit) - concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit * [cayley](https://github.com/google/cayley) - Cayley is an open-source graph database using Bolt as optional backend. * [ChainStore](https://github.com/pressly/chainstore) - Simple key-value interface to a variety of storage engines organized as a chain of operations. * [🌰 Chestnut](https://github.com/jrapoport/chestnut) - Chestnut is encrypted storage for Go. * [Consul](https://github.com/hashicorp/consul) - Consul is service discovery and configuration made easy. Distributed, highly available, and datacenter-aware. * [Containerd](https://github.com/containerd/containerd) - An open and reliable container runtime * [DVID](https://github.com/janelia-flyem/dvid) - Added Bolt as optional storage engine and testing it against Basho-tuned leveldb. * [dcrwallet](https://github.com/decred/dcrwallet) - A wallet for the Decred cryptocurrency. * [drive](https://github.com/odeke-em/drive) - drive is an unofficial Google Drive command line client for \*NIX operating systems. * [event-shuttle](https://github.com/sclasen/event-shuttle) - A Unix system service to collect and reliably deliver messages to Kafka. * [Freehold](http://tshannon.bitbucket.org/freehold/) - An open, secure, and lightweight platform for your files and data. * [Go Report Card](https://goreportcard.com/) - Go code quality report cards as a (free and open source) service. * [GoWebApp](https://github.com/josephspurrier/gowebapp) - A basic MVC web application in Go using BoltDB. * [GoShort](https://github.com/pankajkhairnar/goShort) - GoShort is a URL shortener written in Golang and BoltDB for persistent key/value storage and for routing it's using high performent HTTPRouter. * [gopherpit](https://github.com/gopherpit/gopherpit) - A web service to manage Go remote import paths with custom domains * [gokv](https://github.com/philippgille/gokv) - Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more) * [goraphdb](https://github.com/mstrYoda/goraphdb) - A graph database provides Cypher query, fluent builder and management UI. * [Gitchain](https://github.com/gitchain/gitchain) - Decentralized, peer-to-peer Git repositories aka "Git meets Bitcoin". * [InfluxDB](https://influxdata.com) - Scalable datastore for metrics, events, and real-time analytics. * [ipLocator](https://github.com/AndreasBriese/ipLocator) - A fast ip-geo-location-server using bolt with bloom filters. * [ipxed](https://github.com/kelseyhightower/ipxed) - Web interface and api for ipxed. * [Ironsmith](https://github.com/timshannon/ironsmith) - A simple, script-driven continuous integration (build - > test -> release) tool, with no external dependencies * [Kala](https://github.com/ajvb/kala) - Kala is a modern job scheduler optimized to run on a single node. It is persistent, JSON over HTTP API, ISO 8601 duration notation, and dependent jobs. * [Key Value Access Language (KVAL)](https://github.com/kval-access-language) - A proposed grammar for key-value datastores offering a bbolt binding. * [LedisDB](https://github.com/siddontang/ledisdb) - A high performance NoSQL, using Bolt as optional storage. * [lru](https://github.com/crowdriff/lru) - Easy to use Bolt-backed Least-Recently-Used (LRU) read-through cache with chainable remote stores. * [mbuckets](https://github.com/abhigupta912/mbuckets) - A Bolt wrapper that allows easy operations on multi level (nested) buckets. * [MetricBase](https://github.com/msiebuhr/MetricBase) - Single-binary version of Graphite. * [MuLiFS](https://github.com/dankomiocevic/mulifs) - Music Library Filesystem creates a filesystem to organise your music files. * [NATS](https://github.com/nats-io/nats-streaming-server) - NATS Streaming uses bbolt for message and metadata storage. * [Portainer](https://github.com/portainer/portainer) - A lightweight service delivery platform for containerized applications that can be used to manage Docker, Swarm, Kubernetes and ACI environments. * [Prometheus Annotation Server](https://github.com/oliver006/prom_annotation_server) - Annotation server for PromDash & Prometheus service monitoring system. * [Rain](https://github.com/cenkalti/rain) - BitTorrent client and library. * [reef-pi](https://github.com/reef-pi/reef-pi) - reef-pi is an award winning, modular, DIY reef tank controller using easy to learn electronics based on a Raspberry Pi. * [Request Baskets](https://github.com/darklynx/request-baskets) - A web service to collect arbitrary HTTP requests and inspect them via REST API or simple web UI, similar to [RequestBin](http://requestb.in/) service * [Seaweed File System](https://github.com/chrislusf/seaweedfs) - Highly scalable distributed key~file system with O(1) disk read. * [stow](https://github.com/djherbis/stow) - a persistence manager for objects backed by boltdb. * [Storm](https://github.com/asdine/storm) - Simple and powerful ORM for BoltDB. * [SimpleBolt](https://github.com/xyproto/simplebolt) - A simple way to use BoltDB. Deals mainly with strings. * [Skybox Analytics](https://github.com/skybox/skybox) - A standalone funnel analysis tool for web analytics. * [Scuttlebutt](https://github.com/benbjohnson/scuttlebutt) - Uses Bolt to store and process all Twitter mentions of GitHub projects. * [tentacool](https://github.com/optiflows/tentacool) - REST api server to manage system stuff (IP, DNS, Gateway...) on a linux server. * [torrent](https://github.com/anacrolix/torrent) - Full-featured BitTorrent client package and utilities in Go. BoltDB is a storage backend in development. * [Wiki](https://github.com/peterhellberg/wiki) - A tiny wiki using Goji, BoltDB and Blackfriday. If you are using Bolt in a project please send a pull request to add it to the list. --- ### Overview (`vendor/go.opentelemetry.io/otel/README.md`) # OpenTelemetry-Go [](https://github.com/open-telemetry/opentelemetry-go/actions/workflows/ci.yml) [](https://app.codecov.io/gh/open-telemetry/opentelemetry-go?branch=main) [](https://pkg.go.dev/go.opentelemetry.io/otel) [](https://scorecard.dev/viewer/?uri=github.com/open-telemetry/opentelemetry-go) [](https://www.bestpractices.dev/projects/9996) [](https://issues.oss-fuzz.com/issues?q=project:opentelemetry-go) [](https://app.fossa.com/projects/custom%2B162%2Fgithub.com%2Fopen-telemetry%2Fopentelemetry-go?ref=badge_shield&issueType=license) [](https://slack.cncf.io/) OpenTelemetry-Go is the [Go](https://golang.org/) implementation of [OpenTelemetry](https://opentelemetry.io/). It provides a set of APIs to directly measure performance and behavior of your software and send this data to observability platforms. ## Project Status | Signal | Status | |---------|--------------------| | Traces | Stable | | Metrics | Stable | | Logs | Stable | Progress and status specific to this repository is tracked in our [project boards](https://github.com/open-telemetry/opentelemetry-go/projects) and [milestones](https://github.com/open-telemetry/opentelemetry-go/milestones). Project versioning information and stability guarantees can be found in the [versioning documentation](VERSIONING.md). ### Compatibility OpenTelemetry-Go ensures compatibility with the current supported versions of the [Go language](https://golang.org/doc/devel/release#policy): > Each major Go release is supported until there are two newer major releases. > For example, Go 1.5 was supported until the Go 1.7 release, and Go 1.6 was supported until the Go 1.8 release. For versions of Go that are no longer supported upstream, opentelemetry-go will stop ensuring compatibility with these versions in the following manner: - A minor release of opentelemetry-go will be made to add support for the new supported release of Go. - The following minor release of opentelemetry-go will remove compatibility testing for the oldest (now archived upstream) version of Go. This, and future, releases of opentelemetry-go may include features only supported by the currently supported versions of Go. Currently, this project supports the following environments. | OS | Go Version | Architecture | |----------|------------|--------------| | Ubuntu | 1.27 | amd64 | | Ubuntu | 1.26 | amd64 | | Ubuntu | 1.27 | 386 | | Ubuntu | 1.26 | 386 | | Ubuntu | 1.27 | arm64 | | Ubuntu | 1.26 | arm64 | | macOS | 1.27 | amd64 | | macOS | 1.26 | amd64 | | macOS | 1.27 | arm64 | | macOS | 1.26 | arm64 | | Windows | 1.27 | amd64 | | Windows | 1.26 | amd64 | | Windows | 1.27 | 386 | | Windows | 1.26 | 386 | While this project should work for other systems, no compatibility guarantees are made for those systems currently. ## Getting Started You can find a getting started guide on [opentelemetry.io](https://opentelemetry.io/docs/languages/go/getting-started/). OpenTelemetry's goal is to provide a single set of APIs to capture distributed traces and metrics from your application and send them to an observability platform. This project allows you to do just that for applications written in Go. There are two steps to this process: instrument your application, and configure an exporter. ### Instrumentation To start capturing distributed traces and metric events from your application it first needs to be instrumented. The easiest way to do this is by using an instrumentation library for your code. Be sure to check out [the officially supported instrumentation libraries](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/instrumentation). If you need to extend the telemetry an instrumentation library provides or want to build your own instrumentation for your application directly you will need to use the [Go otel](https://pkg.go.dev/go.opentelemetry.io/otel) package. The [examples](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/examples) are a good way to see some practical uses of this process. ### Export Now that your application is instrumented to collect telemetry, it needs an export pipeline to send that telemetry to an observability platform. All officially supported exporters for the OpenTelemetry project are contained in the [exporters directory](./exporters). | Exporter | Logs | Metrics | Traces | |---------------------------------------|:----:|:-------:|:------:| | [OTLP](./exporters/otlp/) | ✓ | ✓ | ✓ | | [Prometheus](./exporters/prometheus/) | | ✓ | | | [stdout](./exporters/stdout/) | ✓ | ✓ | ✓ | | [Zipkin](./exporters/zipkin/) | | | ✓ | ## Contributing See the [contributing documentation](CONTRIBUTING.md). --- ### Overview (`vendor/go.uber.org/multierr/README.md`) # multierr [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov] `multierr` allows combining one or more Go `error`s together. ## Features - **Idiomatic**: multierr follows best practices in Go, and keeps your code idiomatic. - It keeps the underlying error type hidden, allowing you to deal in `error` values exclusively. - It provides APIs to safely append into an error from a `defer` statement. - **Performant**: multierr is optimized for performance: - It avoids allocations where possible. - It utilizes slice resizing semantics to optimize common cases like appending into the same error object from a loop. - **Interoperable**: multierr interoperates with the Go standard library's error APIs seamlessly: - The `errors.Is` and `errors.As` functions *just work*. - **Lightweight**: multierr comes with virtually no dependencies. ## Installation ```bash go get -u go.uber.org/multierr@latest ``` ## Status Stable: No breaking changes will be made before 2.0. ------------------------------------------------------------------------------- Released under the [MIT License]. [MIT License]: LICENSE.txt [doc-img]: https://pkg.go.dev/badge/go.uber.org/multierr [doc]: https://pkg.go.dev/go.uber.org/multierr [ci-img]: https://github.com/uber-go/multierr/actions/workflows/go.yml/badge.svg [cov-img]: https://codecov.io/gh/uber-go/multierr/branch/master/graph/badge.svg [ci]: https://github.com/uber-go/multierr/actions/workflows/go.yml [cov]: https://codecov.io/gh/uber-go/multierr --- ### Overview (`vendor/go.uber.org/zap/README.md`) # :zap: zap
Blazing fast, structured, leveled logging in Go. [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov]
## Installation `go get -u go.uber.org/zap` Note that zap only supports the two most recent minor versions of Go. ## Quick Start In contexts where performance is nice, but not critical, use the `SugaredLogger`. It's 4-10x faster than other structured logging packages and includes both structured and `printf`-style APIs. ```go logger, _ := zap.NewProduction() defer logger.Sync() // flushes buffer, if any sugar := logger.Sugar() sugar.Infow("failed to fetch URL", // Structured context as loosely typed key-value pairs. "url", url, "attempt", 3, "backoff", time.Second, ) sugar.Infof("Failed to fetch URL: %s", url) ``` When performance and type safety are critical, use the `Logger`. It's even faster than the `SugaredLogger` and allocates far less, but it only supports structured logging. ```go logger, _ := zap.NewProduction() defer logger.Sync() logger.Info("failed to fetch URL", // Structured context as strongly typed Field values. zap.String("url", url), zap.Int("attempt", 3), zap.Duration("backoff", time.Second), ) ``` See the [documentation][doc] and [FAQ](FAQ.md) for more details. ## Performance For applications that log in the hot path, reflection-based serialization and string formatting are prohibitively expensive — they're CPU-intensive and make many small allocations. Put differently, using `encoding/json` and `fmt.Fprintf` to log tons of `interface{}`s makes your application slow. Zap takes a different approach. It includes a reflection-free, zero-allocation JSON encoder, and the base `Logger` strives to avoid serialization overhead and allocations wherever possible. By building the high-level `SugaredLogger` on that foundation, zap lets users *choose* when they need to count every allocation and when they'd prefer a more familiar, loosely typed API. As measured by its own [benchmarking suite][], not only is zap more performant than comparable structured logging packages — it's also faster than the standard library. Like all benchmarks, take these with a grain of salt.[1](#footnote-versions) Log a message and 10 fields: | Package | Time | Time % to zap | Objects Allocated | | :------ | :--: | :-----------: | :---------------: | | :zap: zap | 656 ns/op | +0% | 5 allocs/op | :zap: zap (sugared) | 935 ns/op | +43% | 10 allocs/op | zerolog | 380 ns/op | -42% | 1 allocs/op | go-kit | 2249 ns/op | +243% | 57 allocs/op | slog (LogAttrs) | 2479 ns/op | +278% | 40 allocs/op | slog | 2481 ns/op | +278% | 42 allocs/op | apex/log | 9591 ns/op | +1362% | 63 allocs/op | log15 | 11393 ns/op | +1637% | 75 allocs/op | logrus | 11654 ns/op | +1677% | 79 allocs/op Log a message with a logger that already has 10 fields of context: | Package | Time | Time % to zap | Objects Allocated | | :------ | :--: | :-----------: | :---------------: | | :zap: zap | 67 ns/op | +0% | 0 allocs/op | :zap: zap (sugared) | 84 ns/op | +25% | 1 allocs/op | zerolog | 35 ns/op | -48% | 0 allocs/op | slog | 193 ns/op | +188% | 0 allocs/op | slog (LogAttrs) | 200 ns/op | +199% | 0 allocs/op | go-kit | 2460 ns/op | +3572% | 56 allocs/op | log15 | 9038 ns/op | +13390% | 70 allocs/op | apex/log | 9068 ns/op | +13434% | 53 allocs/op | logrus | 10521 ns/op | +15603% | 68 allocs/op Log a static string, without any context or `printf`-style templating: | Package | Time | Time % to zap | Objects Allocated | | :------ | :--: | :-----------: | :---------------: | | :zap: zap | 63 ns/op | +0% | 0 allocs/op | :zap: zap (sugared) | 81 ns/op | +29% | 1 allocs/op | zerolog | 32 ns/op | -49% | 0 allocs/op | standard library | 124 ns/op | +97% | 1 allocs/op | slog | 196 ns/op | +211% | 0 allocs/op | slog (LogAttrs) | 200 ns/op | +217% | 0 allocs/op | go-kit | 213 ns/op | +238% | 9 allocs/op | apex/log | 771 ns/op | +1124% | 5 allocs/op | logrus | 1439 ns/op | +2184% | 23 allocs/op | log15 | 2069 ns/op | +3184% | 20 allocs/op ## Development Status: Stable All APIs are finalized, and no breaking changes will be made in the 1.x series of releases. Users of semver-aware dependency management systems should pin zap to `^1`. ## Contributing We encourage and support an active, healthy community of contributors — including you! Details are in the [contribution guide](CONTRIBUTING.md) and the [code of conduct](CODE_OF_CONDUCT.md). The zap maintainers keep an eye on issues and pull requests, but you can also report any negative conduct to oss-conduct@uber.com. That email list is a private, safe space; even the zap maintainers don't have access, so don't hesitate to hold us to a high standard.
Released under the [MIT License](LICENSE). 1 In particular, keep in mind that we may be benchmarking against slightly older versions of other packages. Versions are pinned in the [benchmarks/go.mod][] file. [↩](#anchor-versions) [doc-img]: https://pkg.go.dev/badge/go.uber.org/zap [doc]: https://pkg.go.dev/go.uber.org/zap [ci-img]: https://github.com/uber-go/zap/actions/workflows/go.yml/badge.svg [ci]: https://github.com/uber-go/zap/actions/workflows/go.yml [cov-img]: https://codecov.io/gh/uber-go/zap/branch/master/graph/badge.svg [cov]: https://codecov.io/gh/uber-go/zap [benchmarking suite]: https://github.com/uber-go/zap/tree/master/benchmarks [benchmarks/go.mod]: https://github.com/uber-go/zap/blob/master/benchmarks/go.mod --- ### Overview (`vendor/google.golang.org/grpc/README.md`) # gRPC-Go [][API] [](https://goreportcard.com/report/github.com/grpc/grpc-go) [](https://codecov.io/gh/grpc/grpc-go) The [Go][] implementation of [gRPC][]: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information see the [Go gRPC docs][], or jump directly into the [quick start][]. ## Prerequisites - **[Go][]**: any one of the **two latest major** [releases][go-releases]. ## Installation Simply add the following import to your code, and then `go [build|run|test]` will automatically fetch the necessary dependencies: ```go import "google.golang.org/grpc" ``` > **Note:** If you are trying to access `grpc-go` from **China**, see the > [FAQ](#FAQ) below. ## Learn more - [Go gRPC docs][], which include a [quick start][] and [API reference][API] among other resources - [Low-level technical docs](Documentation) from this repository - [Performance benchmark][] - [Examples](examples) - [Contribution guidelines](CONTRIBUTING.md) ## FAQ ### I/O Timeout Errors The `golang.org` domain may be blocked from some countries. `go get` usually produces an error like the following when this happens: ```console $ go get -u google.golang.org/grpc package google.golang.org/grpc: unrecognized import path "google.golang.org/grpc" (https fetch: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1:443: i/o timeout) ``` To build Go code, there are several options: - Set up a VPN and access google.golang.org through that. - With Go module support: it is possible to use the `replace` feature of `go mod` to create aliases for golang.org packages. In your project's directory: ```sh go mod edit -replace=google.golang.org/grpc=github.com/grpc/grpc-go@latest go mod tidy go mod vendor go build -mod=vendor ``` Again, this will need to be done for all transitive dependencies hosted on golang.org as well. For details, refer to [golang/go issue #28652](https://github.com/golang/go/issues/28652). ### Compiling error, undefined: grpc.SupportPackageIsVersion Please update to the latest version of gRPC-Go using `go get google.golang.org/grpc`. ### How to turn on logging The default logger is controlled by environment variables. Turn everything on like this: ```console $ export GRPC_GO_LOG_VERBOSITY_LEVEL=99 $ export GRPC_GO_LOG_SEVERITY_LEVEL=info ``` ### The RPC failed with error `"code = Unavailable desc = transport is closing"` This error means the connection the RPC is using was closed, and there are many possible reasons, including: 1. mis-configured transport credentials, connection failed on handshaking 1. bytes disrupted, possibly by a proxy in between 1. server shutdown 1. Keepalive parameters caused connection shutdown, for example if you have configured your server to terminate connections regularly to [trigger DNS lookups](https://github.com/grpc/grpc-go/issues/3170#issuecomment-552517779). If this is the case, you may want to increase your [MaxConnectionAgeGrace](https://pkg.go.dev/google.golang.org/grpc/keepalive?tab=doc#ServerParameters), to allow longer RPC calls to finish. It can be tricky to debug this because the error happens on the client side but the root cause of the connection being closed is on the server side. Turn on logging on __both client and server__, and see if there are any transport errors. [API]: https://pkg.go.dev/google.golang.org/grpc [Go]: https://golang.org [Go module]: https://github.com/golang/go/wiki/Modules [gRPC]: https://grpc.io [Go gRPC docs]: https://grpc.io/docs/languages/go [Performance benchmark]: https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5180705743044608 [quick start]: https://grpc.io/docs/languages/go/quickstart [go-releases]: https://golang.org/doc/devel/release.html --- ### Overview (`vendor/pgregory.net/rapid/README.md`) # rapid [![PkgGoDev][godev-img]][godev] [![CI][ci-img]][ci] Rapid is a Go library for property-based testing. Rapid checks that properties you define hold for a large number of automatically generated test cases. If a failure is found, rapid automatically minimizes the failing test case before presenting it. ## Features - Imperative Go API with type-safe data generation using generics - Data generation biased to explore "small" values and edge cases more thoroughly - Fully automatic minimization of failing test cases - Persistence and automatic re-running of minimized failing test cases - Support for state machine ("stateful" or "model-based") testing - No dependencies outside the Go standard library *If you like rapid but are looking for a Rust alternative, check out [chaos_theory](https://github.com/flyingmutant/chaos_theory)*. ## Examples Here is what a trivial test using rapid looks like ([playground](https://go.dev/play/p/QJhOzo_BByz)): ```go package rapid_test import ( "slices" "testing" "pgregory.net/rapid" ) func TestSortStrings(t *testing.T) { rapid.Check(t, func(t *rapid.T) { s := rapid.SliceOf(rapid.String()).Draw(t, "s") slices.Sort(s) if !slices.IsSorted(s) { t.Fatalf("unsorted after sort: %v", s) } }) } ``` More complete examples: - `ParseDate` function test: [source code](./example_function_test.go), [playground](https://go.dev/play/p/tZFU8zv8AUl) - `Queue` state machine test: [source code](./example_statemachine_test.go), [playground](https://go.dev/play/p/cxEh4deG-4n) ## Comparison Rapid aims to bring to Go the power and convenience [Hypothesis](https://github.com/HypothesisWorks/hypothesis) brings to Python. Compared to [testing.F.Fuzz](https://pkg.go.dev/testing#F.Fuzz), rapid shines in generating complex structured data, including state machine tests, but lacks coverage-guided feedback and mutations. Note that with [`MakeFuzz`](https://pkg.go.dev/pgregory.net/rapid#MakeFuzz), any rapid test can be used as a fuzz target for the standard fuzzer. Compared to [gopter](https://pkg.go.dev/github.com/leanovate/gopter), rapid provides a much simpler API (queue test in [rapid](./example_statemachine_test.go) vs [gopter](https://github.com/leanovate/gopter/blob/90cc76d7f1b21637b4b912a7c19dea3efe145bb2/commands/example_circularqueue_test.go)), is much smarter about data generation and is able to minimize failing test cases fully automatically, without any user code. As for [testing/quick](https://pkg.go.dev/testing/quick), it lacks both convenient data generation facilities and any form of test case minimization, which are two main things to look for in a property-based testing library. ## FAQ ### What is property-based testing? Suppose we've written arithmetic functions `add`, `subtract` and `multiply` and want to test them. Traditional testing approach is example-based — we come up with example inputs and outputs, and verify that the system behavior matches the examples: ```go func TestArithmetic_Example(t *testing.T) { t.Run("add", func(t *testing.T) { examples := [][3]int{ {0, 0, 0}, {0, 1, 1}, {2, 2, 4}, // ... } for _, e := range examples { if add(e[0], e[1]) != e[2] { t.Fatalf("add(%v, %v) != %v", e[0], e[1], e[2]) } } }) t.Run("subtract", func(t *testing.T) { /* ... */ }) t.Run("multiply", func(t *testing.T) { /* ... */ }) } ``` In comparison, with property-based testing we define higher-level properties that should hold for arbitrary input. Each time we run a property-based test, properties are checked on a new set of pseudo-random data: ```go func TestArithmetic_Property(t *testing.T) { rapid.Check(t, func(t *rapid.T) { var ( a = rapid.Int().Draw(t, "a") b = rapid.Int().Draw(t, "b") c = rapid.Int().Draw(t, "c") ) if add(a, 0) != a { t.Fatalf("add() does not have 0 as identity") } if add(a, b) != add(b, a) { t.Fatalf("add() is not commutative") } if add(a, add(b, c)) != add(add(a, b), c) { t.Fatalf("add() is not associative") } if multiply(a, add(b, c)) != add(multiply(a, b), multiply(a, c)) { t.Fatalf("multiply() is not distributive over add()") } // ... }) } ``` Property-based tests are more powerful and concise than example-based ones — and are also much more fun to write. As an additional benefit, coming up with general properties of the system often improves the design of the system itself. ### What properties should I test? As you've seen from the examples above, it depends on the system you are testing. Usually a good place to start is to put yourself in the shoes of your user and ask what are the properties the user will rely on (often unknowingly or implicitly) when building on top of your system. That said, here are some broadly applicable and often encountered properties to keep in mind: - function does not panic on valid input data - behavior of two algorithms or data structures is identical - all variants of the `decode(encode(x)) == x` roundtrip ### How does rapid work? At its core, rapid does a fairly simple thing: generates pseudo-random data based on the specification you provide, and check properties that you define on the generated data. Checking is easy: you simply write `if` statements and call something like `t.Fatalf` when things look wrong. Generating is a bit more involved. When you construct a `Generator`, nothing happens: `Generator` is just a specification of how to `Draw` the data you want. When you call `Draw`, rapid will take some bytes from its internal random bitstream, use them to construct the value based on the `Generator` specification, and track how the random bytes used correspond to the value (and its subparts). This knowledge about the structure of the values being generated, as well as their relationship with the parts of the bitstream allows rapid to intelligently and automatically minify any failure found. ### What about fuzzing? Property-based testing focuses on quick feedback loop: checking the properties on a small but diverse set of pseudo-random inputs in a fractions of a second. In comparison, fuzzing focuses on slow, often multi-day, brute force input generation that maximizes the coverage. Both approaches are useful. Property-based tests are used alongside regular example-based tests during development, and fuzzing is used to search for edge cases and security vulnerabilities. With [`MakeFuzz`](https://pkg.go.dev/pgregory.net/rapid#MakeFuzz), any rapid test can be used as a fuzz target. ## Usage Just run `go test` as usual, it will pick up also all `rapid` tests. There are a number of optional flags to influence rapid behavior, run `go test -args -h` and look at the flags with the `-rapid.` prefix. You can then pass such flags as usual. For example: ```sh go test -rapid.checks=10_000 ``` Every flag with the `-rapid.` prefix also has a matching `RAPID_` environment variable (replace the dot with an underscore and uppercase the name). For instance, `RAPID_CHECKS=10 go test` sets the default for `-rapid.checks=10`, and an explicit flag still overrides the environment default when both are present. ## Status Rapid is stable: tests using rapid should continue to work with all future rapid releases with the same major version. Possible exceptions to this rule are API changes that replace the concrete type of parameter with an interface type, or other similar mostly non-breaking changes. ## License Rapid is licensed under the [Mozilla Public License Version 2.0](./LICENSE). [godev-img]: https://pkg.go.dev/badge/pgregory.net/rapid [godev]: https://pkg.go.dev/pgregory.net/rapid [ci-img]: https://github.com/flyingmutant/rapid/workflows/CI/badge.svg [ci]: https://github.com/flyingmutant/rapid/actions --- ### Overview (`vendor/resenje.org/singleflight/README.md`) # Singleflight [](https://godoc.org/resenje.org/singleflight) [](https://github.com/janos/singleflight/actions?query=workflow%3AGo) Package singleflight provides a duplicate function call suppression mechanism similar to [golang.org/x/sync/singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight) but with: - support for context cancelation. The context passed to the callback function is a context that preserves all values from the passed context but is cancelled by the singleflight only when all awaiting caller's contexts are cancelled. - support for generics. ## Installation Run `go get resenje.org/singleflight` from command line. --- ### Overview (`vendor/sigs.k8s.io/json/README.md`) # sigs.k8s.io/json [](https://pkg.go.dev/sigs.k8s.io/json) ## Introduction This library is a subproject of [sig-api-machinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery#json). It provides case-sensitive, integer-preserving JSON unmarshaling functions based on `encoding/json` `Unmarshal()`. ## Compatibility The `UnmarshalCaseSensitivePreserveInts()` function behaves like `encoding/json#Unmarshal()` with the following differences: - JSON object keys are treated case-sensitively. Object keys must exactly match json tag names (for tagged struct fields) or struct field names (for untagged struct fields). - JSON integers are unmarshaled into `interface{}` fields as an `int64` instead of a `float64` when possible, falling back to `float64` on any parse or overflow error. - Syntax errors do not return an `encoding/json` `*SyntaxError` error. Instead, they return an error which can be passed to `SyntaxErrorOffset()` to obtain an offset. ## Additional capabilities The `UnmarshalStrict()` function decodes identically to `UnmarshalCaseSensitivePreserveInts()`, and also returns non-fatal strict errors encountered while decoding: - Duplicate fields encountered - Unknown fields encountered ### Community, discussion, contribution, and support You can reach the maintainers of this project via the [sig-api-machinery mailing list / channels](https://github.com/kubernetes/community/tree/master/sig-api-machinery#contact). ### Code of conduct Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md). [owners]: https://git.k8s.io/community/contributors/guide/owners.md [Creative Commons 4.0]: https://git.k8s.io/website/LICENSE --- ### Overview (`vendor/sigs.k8s.io/yaml/README.md`) # YAML marshaling and unmarshaling support for Go [](https://travis-ci.org/kubernetes-sigs/yaml) kubernetes-sigs/yaml is a permanent fork of [ghodss/yaml](https://github.com/ghodss/yaml). ## Introduction A wrapper around [go-yaml](https://github.com/go-yaml/yaml) designed to enable a better way of handling YAML when marshaling to and from structs. In short, this library first converts YAML to JSON using go-yaml and then uses `json.Marshal` and `json.Unmarshal` to convert to or from the struct. This means that it effectively reuses the JSON struct tags as well as the custom JSON methods `MarshalJSON` and `UnmarshalJSON` unlike go-yaml. For a detailed overview of the rationale behind this method, [see this blog post](http://web.archive.org/web/20190603050330/http://ghodss.com/2014/the-right-way-to-handle-yaml-in-golang/). ## Compatibility This package uses [go-yaml](https://github.com/go-yaml/yaml) and therefore supports [everything go-yaml supports](https://github.com/go-yaml/yaml#compatibility). ## Caveats **Caveat #1:** When using `yaml.Marshal` and `yaml.Unmarshal`, binary data should NOT be preceded with the `!!binary` YAML tag. If you do, go-yaml will convert the binary data from base64 to native binary data, which is not compatible with JSON. You can still use binary in your YAML files though - just store them without the `!!binary` tag and decode the base64 in your code (e.g. in the custom JSON methods `MarshalJSON` and `UnmarshalJSON`). This also has the benefit that your YAML and your JSON binary data will be decoded exactly the same way. As an example: ``` BAD: exampleKey: !!binary gIGC GOOD: exampleKey: gIGC ... and decode the base64 data in your code. ``` **Caveat #2:** When using `YAMLToJSON` directly, maps with keys that are maps will result in an error since this is not supported by JSON. This error will occur in `Unmarshal` as well since you can't unmarshal map keys anyways since struct fields can't be keys. ## Installation and usage To install, run: ``` $ go get sigs.k8s.io/yaml ``` And import using: ``` import "sigs.k8s.io/yaml" ``` Usage is very similar to the JSON library: ```go package main import ( "fmt" "sigs.k8s.io/yaml" ) type Person struct { Name string `json:"name"` // Affects YAML field names too. Age int `json:"age"` } func main() { // Marshal a Person struct to YAML. p := Person{"John", 30} y, err := yaml.Marshal(p) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(string(y)) /* Output: age: 30 name: John */ // Unmarshal the YAML back into a Person struct. var p2 Person err = yaml.Unmarshal(y, &p2) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(p2) /* Output: {John 30} */ } ``` `yaml.YAMLToJSON` and `yaml.JSONToYAML` methods are also available: ```go package main import ( "fmt" "sigs.k8s.io/yaml" ) func main() { j := []byte(`{"name": "John", "age": 30}`) y, err := yaml.JSONToYAML(j) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(string(y)) /* Output: age: 30 name: John */ j2, err := yaml.YAMLToJSON(y) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(string(j2)) /* Output: {"age":30,"name":"John"} */ } ``` --- ### Overview (`daemon/libnetwork/cmd/diagnostic/README.md`) --- description: Learn to use the built-in network debugger to debug overlay networking problems keywords: network, troubleshooting, debug title: Debug overlay or swarm networking issues --- **WARNING** This tool can change the internal state of the libnetwork API, be really mindful on its use and read carefully the following guide. Improper use of it will damage or permanently destroy the network configuration. Docker CE 17.12 and higher introduce a network debugging tool designed to help debug issues with overlay networks and swarm services running on Linux hosts. When enabled, a network diagnostic server listens on the specified port and provides diagnostic information. The network debugging tool should only be started to debug specific issues, and should not be left running all the time. Information about networks is stored in the database, which can be examined using the API. Currently the database contains information about the overlay network as well as the service discovery data. The Docker API exposes endpoints to query and control the network debugging tool. CLI integration is provided as a preview, but the implementation is not yet considered stable and commands and options may change without notice. The tool is available into 2 forms: 1) client only: dockereng/network-diagnostic:onlyclient 2) docker in docker version: dockereng/network-diagnostic:17.12-dind The latter allows to use the tool with a cluster running an engine older than 17.12 ## Enable the diagnostic server The tool currently only works on Docker hosts running on Linux. To enable it on a node follow the step below. 1. Set the `network-diagnostic-port` to a port which is free on the Docker host, in the `/etc/docker/daemon.json` configuration file. ```json “network-diagnostic-port”: ``` 2. Get the process ID (PID) of the `dockerd` process. It is the second field in the output, and is typically a number from 2 to 6 digits long. ```bash $ ps aux |grep dockerd | grep -v grep ``` 3. Reload the Docker configuration without restarting Docker, by sending the `HUP` signal to the PID you found in the previous step. ```bash kill -HUP ``` If systemd is used the command `systemctl reload docker` will be enough A message like the following will appear in the Docker host logs: ```none Starting the diagnostic server listening on for commands ``` ## Disable the diagnostic tool Repeat these steps for each node participating in the swarm. 1. Remove the `network-diagnostic-port` key from the `/etc/docker/daemon.json` configuration file. 2. Get the process ID (PID) of the `dockerd` process. It is the second field in the output, and is typically a number from 2 to 6 digits long. ```bash $ ps aux |grep dockerd | grep -v grep ``` 3. Reload the Docker configuration without restarting Docker, by sending the `HUP` signal to the PID you found in the previous step. ```bash kill -HUP ``` A message like the following will appear in the Docker host logs: ```none Disabling the diagnostic server ``` ## Access the diagnostic tool's API The network diagnostic tool exposes its own RESTful API. To access the API, send a HTTP request to the port where the tool is listening. The following commands assume the tool is listening on port 2000. Examples are not given for every endpoint. ### Get help ```bash $ curl localhost:2000/help OK /updateentry /getentry /gettable /leavenetwork /createentry /help /clusterpeers /ready /joinnetwork /deleteentry /networkpeers / /join ``` ### Join or leave the network database cluster ```bash $ curl localhost:2000/join?members=ip1,ip2,... ``` ```bash $ curl localhost:2000/leave?members=ip1,ip2,... ``` `ip1`, `ip2`, ... are the swarm node ips (usually one is enough) ### Join or leave a network ```bash $ curl localhost:2000/joinnetwork?nid= ``` ```bash $ curl localhost:2000/leavenetwork?nid= ``` `network id` can be retrieved on the manager with `docker network ls --no-trunc` and has to be the full length identifier ### List cluster peers ```bash $ curl localhost:2000/clusterpeers ``` ### List nodes connected to a given network ```bash $ curl localhost:2000/networkpeers?nid= ``` `network id` can be retrieved on the manager with `docker network ls --no-trunc` and has to be the full length identifier ### Dump database tables The tables are called `endpoint_table` and `overlay_peer_table`. The `overlay_peer_table` contains all the overlay forwarding information The `endpoint_table` contains all the service discovery information ```bash $ curl localhost:2000/gettable?nid=&tname= ``` ### Interact with a specific database table The tables are called `endpoint_table` and `overlay_peer_table`. ```bash $ curl localhost:2000/?nid=&tname=
&key=[&value=] ``` Note: operations on tables have node ownership, this means that are going to remain persistent till the node that inserted them is part of the cluster ## Access the diagnostic tool's CLI The CLI is provided as a preview and is not yet stable. Commands or options may change at any time. The CLI executable is called `diagnosticClient` and is made available using a standalone container. `docker run --net host dockereng/network-diagnostic:onlyclient -v -net -t sd` The following flags are supported: | Flag | Description | |---------------|-------------------------------------------------| | -t | Table one of `sd` or `overlay`. | | -ip | The IP address to query. Defaults to 127.0.0.1. | | -net | The target network ID. | | -port | The target port. (default port is 2000) | | -a | Join/leave network | | -v | Enable verbose output. | *NOTE* By default the tool won't try to join the network. This is following the intent to not change the state on which the node is when the diagnostic client is run. This means that it is safe to run the diagnosticClient against a running daemon because it will just dump the current state. When using instead the diagnosticClient in the containerized version the flag `-a` MUST be passed to avoid retrieving empty results. On the other side using the `-a` flag against a loaded daemon will have the undesirable side effect to leave the network and so cutting down the data path for that daemon. ### Container version of the diagnostic tool The CLI is provided as a container with a 17.12 engine that needs to run using privileged mode. *NOTE* Remember that table operations have ownership, so any `create entry` will be persistent till the diagnostic container is part of the swarm. 1. Make sure that the node where the diagnostic client will run is not part of the swarm, if so do `docker swarm leave -f` 2. To run the container, use a command like the following: ```bash $ docker container run --name net-diagnostic -d --privileged --network host dockereng/network-diagnostic:17.12-dind ``` 3. Connect to the container using `docker exec -it sh`, and start the server using the following command: ```bash $ kill -HUP 1 ``` 4. Join the diagnostic container to the swarm, then run the diagnostic CLI within the container. ```bash $ ./diagnosticClient ... ``` 4. When finished debugging, leave the swarm and stop the container. ### Examples The following commands dump the service discovery table and verify node ownership. *NOTE* Remember to use the full network ID, you can easily find that with `docker network ls --no-trunc` **Service discovery and load balancer:** ```bash $ diagnostiClient -t sd -v -net n8a8ie6tb3wr2e260vxj8ncy4 -a ``` **Overlay network:** ```bash $ diagnostiClient -port 2001 -t overlay -v -net n8a8ie6tb3wr2e260vxj8ncy4 -a ``` --- ### Overview (`daemon/libnetwork/cmd/ssd/README.md`) # Docker Swarm Service Driller(ssd) ssd is a troubleshooting utility for Docker swarm networks. ### control-plane and datapath consistency check on a node ssd checks for the consistency between docker network control-plane (from the docker daemon in-memory state) and kernel data path programming. Currently the tool checks only for the consistency of the Load balancer (implemented using IPVS). In a three node swarm cluster ssd status for a overlay network `ov2` which has three services running, each replicated to 3 instances. ````bash vagrant@net-1:~/code/go/src/github.com/docker/docker-e2e/tests$ docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/run/docker/netns:/var/run/docker/netns --privileged --net=host sanimej/ssd ov2 Verifying LB programming for containers on network ov2 Verifying container /s2.3.ltrdwef0iqf90rqauw3ehcs56... service s2... OK service s3... OK service s1... OK Verifying container /s3.3.nyhwvdvnocb4wftyhb8dr4fj8... service s2... OK service s3... OK service s1... OK Verifying container /s1.3.wwx5tuxhnvoz5vrb8ohphby0r... service s2... OK service s3... OK service s1... OK Verifying LB programming for containers on network ingress Verifying container Ingress... service web... OK ```` ssd checks the required iptables programming to direct an incoming packet with the : to the right : ### control-plane consistency check across nodes in a cluster Docker networking uses a gossip protocol to synchronize networking state across nodes in a cluster. ssd's `gossip-consistency` command verifies if the state maintained by all the nodes are consistent. ````bash In a three node cluster with services running on an overlay network ov2 ssd consistency-checker shows vagrant@net-1:~/code/go/src/github.com/docker/docker-e2e/tests$ docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/run/docker/netns:/var/run/docker/netns --privileged sanimej/ssd ov2 gossip-consistency Node id: sjfp0ca8f43rvnab6v7f21gq0 gossip hash c57d89094dbb574a37930393278dc282 Node id: bg228r3q9095grj4wxkqs80oe gossip hash c57d89094dbb574a37930393278dc282 Node id: 6jylcraipcv2pxdricqe77j5q gossip hash c57d89094dbb574a37930393278dc282 ```` This is hash digest of the control-plane state for the network `ov2` from all the cluster nodes. If the values have a mismatch `docker network inspect --verbose` on the individual nodes can help in identifying what the specific difference is. --- ### Overview (`extpoints/storage/kv/v0/README.md`) # Persistent record storage Store opaque key/value records without depending on host-local paths. Declare `storagekv.Point.Dependency()` and resolve a namespace during extension initialization: ```go kv, err := storagekv.GetKV(resolver, "org.my.extension") if err != nil { return err } ``` Use the namespace-bound helper for record operations: ```go err = kv.Create(ctx, "jobs/123", data) data, err = kv.Get(ctx, "jobs/123") err = kv.Update(ctx, "jobs/123", updatedData) err = kv.Delete(ctx, "jobs/123") err = kv.DeletePrefix(ctx, "jobs/") page, err := kv.List(ctx, storagekv.ListOptions{Prefix: "jobs/"}) ``` ## Record operations - Keys are logical strings, not paths; consumers own value encoding and schema versions. - `Create` requires an absent key; `Update` requires an existing key; `Delete` tolerates absence. - `DeletePrefix` atomically and durably removes all matching records; a nonempty prefix is required, and no matches (including a missing namespace) succeeds. - Individual writes are durable and atomic per record. `DeletePrefix` is the only multi-record atomic operation; there is no concurrent-update protection. - Listing is paginated, not a snapshot; continue with `NextCursor` using the same prefix. - The helper needs no `Close`; records survive provider restarts and extension removal. Executable consumers register the generated `protogen.ClientPoint` with their SDK client-point registry. See [storage.go](storage.go) for the full contract and size limits. ## Namespace and trust model Namespaces partition records for cooperative use; they are not an authorization boundary. All trusted installed extensions that can reach this provider can access any namespace. v0 performs no identity-based authorization, and namespace names do not authenticate ownership. Any future identity enforcement belongs in host-resolved identity and per-caller storage wiring. Identity enforcement must still allow explicit namespace sharing. ## Persistence and lifecycle Records are not purged automatically when an extension is absent or uninstalled. Each extension owns cleanup of its individual records. v0 stores no namespace ownership metadata and defines no garbage-collection policy; policy is deferred to v1. Do not delete the shared database to remove one extension's data. Corruption of the shared database affects all namespaces, and storage provides no automatic repair or replacement. Storage does not automatically compact the database or reclaim file space. --- ### Overview (`vendor/cloud.google.com/go/auth/README.md`) # Google Auth Library for Go [](https://pkg.go.dev/cloud.google.com/go/auth) ## Install ``` bash go get cloud.google.com/go/auth@latest ``` ## Usage The most common way this library is used is transitively, by default, from any of our Go client libraries. ### Notable use-cases - To create a credential directly please see examples in the [credentials](https://pkg.go.dev/cloud.google.com/go/auth/credentials) package. - To create a authenticated HTTP client please see examples in the [httptransport](https://pkg.go.dev/cloud.google.com/go/auth/httptransport) package. - To create a authenticated gRPC connection please see examples in the [grpctransport](https://pkg.go.dev/cloud.google.com/go/auth/grpctransport) package. - To create an ID token please see examples in the [idtoken](https://pkg.go.dev/cloud.google.com/go/auth/credentials/idtoken) package. ## Contributing Contributions are welcome. Please, see the [CONTRIBUTING](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md) document for details. Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Contributor Code of Conduct](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md#contributor-code-of-conduct) for more information. --- ### Overview (`vendor/cloud.google.com/go/logging/README.md`) # Cloud Logging API [](https://pkg.go.dev/cloud.google.com/go/logging) Go Client Library for Cloud Logging API. ## Install ```bash go get cloud.google.com/go/logging ``` ## Stability The stability of this module is indicated by SemVer. However, a `v1+` module may have breaking changes in two scenarios: * Packages with `alpha` or `beta` in the import path * The GoDoc has an explicit stability disclaimer (for example, for an experimental feature). ### Which package to use? Generated client library surfaces can be found in packages whose import path ends in `.../apivXXX`. The `XXX` could be something like `1` or `2` in the case of a stable service backend or may be like `1beta2` or `2beta` in the case of a more experimental service backend. Because of this fact, a given module can have multiple clients for different service backends. In these cases it is generally recommended to use clients with stable service backends, with import suffixes like `apiv1`, unless you need to use features that are only present in a beta backend or there is not yet a stable backend available. ## Google Cloud Samples To browse ready to use code samples check [Google Cloud Samples](https://cloud.google.com/docs/samples?l=go). ## Go Version Support See the [Go Versions Supported](https://github.com/googleapis/google-cloud-go#go-versions-supported) section in the root directory's README. ## Authorization See the [Authorization](https://github.com/googleapis/google-cloud-go#authorization) section in the root directory's README. ## Contributing Contributions are welcome. Please, see the [CONTRIBUTING](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md) document for details. Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Contributor Code of Conduct](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md#contributor-code-of-conduct) for more information. --- ### Overview (`vendor/cloud.google.com/go/longrunning/README.md`) # Long Running Operations API [](https://pkg.go.dev/cloud.google.com/go/longrunning) Go Client Library for Long Running Operations API. ## Install ```bash go get cloud.google.com/go/longrunning ``` ## Stability The stability of this module is indicated by SemVer. However, a `v1+` module may have breaking changes in two scenarios: * Packages with `alpha` or `beta` in the import path * The GoDoc has an explicit stability disclaimer (for example, for an experimental feature). ### Which package to use? Generated client library surfaces can be found in packages whose import path ends in `.../apivXXX`. The `XXX` could be something like `1` or `2` in the case of a stable service backend or may be like `1beta2` or `2beta` in the case of a more experimental service backend. Because of this fact, a given module can have multiple clients for different service backends. In these cases it is generally recommended to use clients with stable service backends, with import suffixes like `apiv1`, unless you need to use features that are only present in a beta backend or there is not yet a stable backend available. ## Google Cloud Samples To browse ready to use code samples check [Google Cloud Samples](https://cloud.google.com/docs/samples?l=go). ## Go Version Support See the [Go Versions Supported](https://github.com/googleapis/google-cloud-go#go-versions-supported) section in the root directory's README. ## Authorization See the [Authorization](https://github.com/googleapis/google-cloud-go#authorization) section in the root directory's README. ## Contributing Contributions are welcome. Please, see the [CONTRIBUTING](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md) document for details. Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Contributor Code of Conduct](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md#contributor-code-of-conduct) for more information. --- ### Overview (`vendor/github.com/Azure/go-ansiterm/README.md`) # go-ansiterm This is a cross platform Ansi Terminal Emulation library. It reads a stream of Ansi characters and produces the appropriate function calls. The results of the function calls are platform dependent. For example the parser might receive "ESC, [, A" as a stream of three characters. This is the code for Cursor Up (http://www.vt100.net/docs/vt510-rm/CUU). The parser then calls the cursor up function (CUU()) on an event handler. The event handler determines what platform specific work must be done to cause the cursor to move up one position. The parser (parser.go) is a partial implementation of this state machine (http://vt100.net/emu/vt500_parser.png). There are also two event handler implementations, one for tests (test_event_handler.go) to validate that the expected events are being produced and called, the other is a Windows implementation (winterm/win_event_handler.go). See parser_test.go for examples exercising the state machine and generating appropriate function calls. ----- This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. --- ### Overview (`vendor/github.com/Microsoft/go-winio/README.md`) # go-winio [](https://github.com/microsoft/go-winio/actions/workflows/ci.yml) This repository contains utilities for efficiently performing Win32 IO operations in Go. Currently, this is focused on accessing named pipes and other file handles, and for using named pipes as a net transport. This code relies on IO completion ports to avoid blocking IO on system threads, allowing Go to reuse the thread to schedule another goroutine. This limits support to Windows Vista and newer operating systems. This is similar to the implementation of network sockets in Go's net package. Please see the LICENSE file for licensing information. ## Contributing This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [Microsoft CLA](https://cla.microsoft.com). When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. Additionally, the pull request pipeline requires the following steps to be performed before mergining. ### Code Sign-Off We require that contributors sign their commits using [`git commit --signoff`][git-commit-s] to certify they either authored the work themselves or otherwise have permission to use it in this project. A range of commits can be signed off using [`git rebase --signoff`][git-rebase-s]. Please see [the developer certificate](https://developercertificate.org) for more info, as well as to make sure that you can attest to the rules listed. Our CI uses the DCO Github app to ensure that all commits in a given PR are signed-off. ### Linting Code must pass a linting stage, which uses [`golangci-lint`][lint]. The linting settings are stored in [`.golangci.yaml`](./.golangci.yaml), and can be run automatically with VSCode by adding the following to your workspace or folder settings: ```json "go.lintTool": "golangci-lint", "go.lintOnSave": "package", ``` Additional editor [integrations options are also available][lint-ide]. Alternatively, `golangci-lint` can be [installed locally][lint-install] and run from the repo root: ```shell # use . or specify a path to only lint a package # to show all lint errors, use flags "--max-issues-per-linter=0 --max-same-issues=0" > golangci-lint run ./... ``` ### Go Generate The pipeline checks that auto-generated code, via `go generate`, are up to date. This can be done for the entire repo: ```shell > go generate ./... ``` ## Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. ## Special Thanks Thanks to [natefinch][natefinch] for the inspiration for this library. See [npipe](https://github.com/natefinch/npipe) for another named pipe implementation. [lint]: https://golangci-lint.run/ [lint-ide]: https://golangci-lint.run/usage/integrations/#editor-integration [lint-install]: https://golangci-lint.run/usage/install/#local-installation [git-commit-s]: https://git-scm.com/docs/git-commit#Documentation/git-commit.txt--s [git-rebase-s]: https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt---signoff [natefinch]: https://github.com/natefinch --- ### Overview (`vendor/github.com/Microsoft/hcsshim/README.md`) # hcsshim [](https://github.com/microsoft/hcsshim/actions?query=branch%3Amaster) This package contains the Golang interface for using the Windows [Host Compute Service](https://techcommunity.microsoft.com/t5/containers/introducing-the-host-compute-service-hcs/ba-p/382332) (HCS) to launch and manage [Windows Containers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/). It also contains other helpers and functions for managing Windows Containers such as the Golang interface for the Host Network Service (HNS), as well as code for the [guest agent](./internal/guest/README.md) (commonly referred to as the GCS or Guest Compute Service in the codebase) used to support running Linux Hyper-V containers. It is primarily used in the [Moby](https://github.com/moby/moby) and [Containerd](https://github.com/containerd/containerd) projects, but it can be freely used by other projects as well. ## Building While this repository can be used as a library of sorts to call the HCS apis, there are a couple binaries built out of the repository as well. The main ones being the Linux guest agent, and an implementation of the [runtime v2 containerd shim api](https://github.com/containerd/containerd/blob/master/runtime/v2/README.md). ### Linux Hyper-V Container Guest Agent To build the Linux guest agent itself all that's needed is to set your GOOS to "Linux" and build out of ./cmd/gcs. ```powershell C:\> $env:GOOS="linux" C:\> go build .\cmd\gcs\ ``` or on a Linux machine ```sh > go build ./cmd/gcs ``` If you want it to be packaged inside of a rootfs to boot with alongside all of the other tools then you'll need to provide a rootfs that it can be packaged inside of. An easy way is to export the rootfs of a container. ```sh docker pull busybox docker run --name base_image_container busybox docker export base_image_container | gzip > base.tar.gz BASE=./base.tar.gz make all ``` If the build is successful, in the `./out` folder you should see: ```sh > ls ./out/ delta.tar.gz initrd.img rootfs.tar.gz ``` ### Containerd Shim For info on the [Runtime V2 API](https://github.com/containerd/containerd/blob/main/docs/runtime-v2.md). Contrary to the typical Linux architecture of shim -> runc, the runhcs shim is used both to launch and manage the lifetime of containers. ```powershell C:\> $env:GOOS="windows" C:\> go build .\cmd\containerd-shim-runhcs-v1 ``` Then place the binary in the same directory that Containerd is located at in your environment. A default Containerd configuration file can be generated by running: ```powershell .\containerd.exe config default | Out-File "C:\Program Files\containerd\config.toml" -Encoding ascii ``` This config file will already have the shim set as the default runtime for cri interactions. To trial using the shim out with ctr.exe: ```powershell C:\> ctr.exe run --runtime io.containerd.runhcs.v1 --rm mcr.microsoft.com/windows/nanoserver:2004 windows-test cmd /c "echo Hello World!" ``` ### Containerd Shim V2 The V2 shims are the rewrite of the Windows containerd shim. The V1 shim ([`containerd-shim-runhcs-v1`](./cmd/containerd-shim-runhcs-v1)) is a single, monolithic binary that handles LCOW (Linux Containers on Windows), Hyper-V WCOW (Windows Containers on Windows), process-isolated WCOW and host-process containers. In the V2 model that monolith is split into focused, per-platform shims, each backed 1:1 by a sandbox. V2 shims are used with containerd in the same way as the V1 shim, but the API surface they expose is different. The V1 shim implemented only the containerd [Task API](https://github.com/containerd/containerd/blob/main/docs/runtime-v2.md), and used it to manage both the sandbox lifecycle and the container/process (task) lifecycle through a single service. Each V2 shim instead splits these responsibilities across the two APIs that containerd now provides for this purpose: the [Sandbox API](https://github.com/containerd/containerd/blob/main/docs/sandbox-api.md) is used to manage the sandbox, while the [Task API](https://github.com/containerd/containerd/blob/main/docs/runtime-v2.md) is used to manage containers and processes running inside it. Internally each V2 shim implements these as separate sandbox and task services, alongside an auxiliary `shimdiag` service used for diagnostics. All three shims honor the same CRI pod model. A task annotated with `"io.kubernetes.cri.container-type": "sandbox"` is treated as the pause/infra container that creates the pod; sibling workload tasks set `"io.kubernetes.cri.container-type": "container"` and reference their pause via `"io.kubernetes.cri.sandbox-id"`. What a "sandbox" *physically* corresponds to depends on the shim, and is described in each subsection below. #### containerd-shim-lcow-v2 - **Purpose:** Runs Linux Containers on Windows (LCOW) — a Linux utility VM hosting Linux containers. - **Sandbox:** The Linux UVM. Each shim instance is backed 1:1 by a single UVM. This shim supports running *multiple pods in the same UVM*, so a single shim instance may host more than one CRI pods. - **Tasks:** Linux containers and processes running inside the UVM, identified via the same CRI annotations described above. - **Implementation:** [`./cmd/containerd-shim-lcow-v2`](./cmd/containerd-shim-lcow-v2). - **Build Tag:** lcow - **Platform requirement:** Windows Server 2025 (build 26100) or later. #### containerd-shim-wcow-v2 - **Purpose:** Runs Hyper-V isolated Windows containers (WCOW) — a Windows utility VM hosting Process and/or Host Process Containers inside it. - **Sandbox:** The Windows utility VM (UVM). Each shim instance is backed 1:1 by a single UVM. - **Tasks:** the Windows containers and processes running inside the UVM, identified via the standard CRI annotations described above. - **Implementation:** coming soon. - **Build Tag:** wcow #### containerd-shim-process-v2 - **Purpose:** Runs Process-isolated Windows Server containers and Host Process Containers — workloads that execute directly on the host with no utility VM. - **Sandbox:** A *pause container*. The pause container is a minimal, long-lived container that owns the pod's shared resources (such as the network namespace) and keeps them alive while sibling workload containers are started, stopped or replaced. This is the standard Kubernetes pod model: the pause container is the sandbox that the rest of the pod attaches to. - **Tasks:** the actual workload containers belonging to the pod, linked back to the pause via the `io.kubernetes.cri.sandbox-id` annotation. - **Implementation:** coming soon. - **Build Tag:** process ##### Building The V2 shim sources are guarded by the build tags as mentioned above, so the tag must be passed to `go build`. For example the `LCOW` shim has `lcow` tag- ```powershell C:\> $env:GOOS="windows" C:\> go build -tags lcow .\cmd\containerd-shim-lcow-v2 ``` Place the resulting `containerd-shim-lcow-v2.exe` in the same directory as `containerd.exe`, the same as for the V1 shim. ##### Running unit tests The shim's unit tests (and the rest of the tagged packages) are run with the shim specific build tag: ```powershell C:\> go test -tags lcow ./... ``` ##### Running parity tests The repository ships parity tests under [`./test/parity`](./test/parity) that feed identical inputs through the legacy V1 and the new V2 pipelines and assert that the resulting HCS ComputeSystem documents are equivalent. They live in the `test` Go module and are also built with the build tag: ```powershell C:\> cd test C:\> go test -tags lcow ./parity/... ``` ## Contributing This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [Microsoft CLA](https://cla.microsoft.com). When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. We require that contributors sign their commits to certify they either authored the work themselves or otherwise have permission to use it in this project. We also require that contributors sign their commits using using [`git commit --signoff`][git-commit-s] to certify they either authored the work themselves or otherwise have permission to use it in this project. A range of commits can be signed off using [`git rebase --signoff`][git-rebase-s]. Please see [the developer certificate](https://developercertificate.org) for more info, as well as to make sure that you can attest to the rules listed. Our CI uses the [DCO Github app](https://github.com/apps/dco) to ensure that all commits in a given PR are signed-off. ### Linting Code must pass a linting stage, which uses [`golangci-lint`][lint]. Since `./test` is a separate Go module, the linter is run from both the root and the `test` directories. Additionally, the linter is run with `GOOS` set to both `windows` and `linux`. The linting settings are stored in [`.golangci.yaml`](./.golangci.yaml), and can be run automatically with VSCode by adding the following to your workspace or folder settings: ```json "go.lintTool": "golangci-lint", "go.lintOnSave": "package", ``` Additional editor [integrations options are also available][lint-ide]. Alternatively, `golangci-lint` can be [installed][lint-install] and run locally: ```shell # use . or specify a path to only lint a package # to show all lint errors, use flags "--max-issues-per-linter=0 --max-same-issues=0" > golangci-lint run ``` To run across the entire repo for both `GOOS=windows` and `linux`: ```powershell > foreach ( $goos in ('windows', 'linux') ) { foreach ( $repo in ('.', 'test') ) { pwsh -Command "cd $repo && go env -w GOOS=$goos && golangci-lint.exe run --verbose" } } ``` ### Go Generate The pipeline checks that auto-generated code, via `go generate`, are up to date. Similar to the [linting stage](#linting), `go generate` is run in both the root and test Go modules. This can be done via: ```shell > go generate ./... > cd test && go generate ./... ``` ## Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. ## Dependencies This project requires Golang 1.18 or newer to build. For system requirements to run this project, see the Microsoft docs on [Windows Container requirements](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/system-requirements). ## Reporting Security Issues Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) at [secure@microsoft.com](mailto:secure@microsoft.com). You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the [MSRC PGP](https://technet.microsoft.com/en-us/security/dn606155) key, can be found in the [Security TechCenter](https://technet.microsoft.com/en-us/security/default). For additional details, see [Report a Computer Security Vulnerability](https://technet.microsoft.com/en-us/security/ff852094.aspx) on Technet --------------- Copyright (c) 2018 Microsoft Corp. All rights reserved. [lint]: https://golangci-lint.run/ [lint-ide]: https://golangci-lint.run/usage/integrations/#editor-integration [lint-install]: https://golangci-lint.run/usage/install/#local-installation [git-commit-s]: https://git-scm.com/docs/git-commit#Documentation/git-commit.txt--s [git-rebase-s]: https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt---signoff --- ### Overview (`vendor/github.com/RackSec/srslog/README.md`) [](https://travis-ci.org/RackSec/srslog) # srslog Go has a `syslog` package in the standard library, but it has the following shortcomings: 1. It doesn't have TLS support 2. [According to bradfitz on the Go team, it is no longer being maintained.](https://github.com/golang/go/issues/13449#issuecomment-161204716) I agree that it doesn't need to be in the standard library. So, I've followed Brad's suggestion and have made a separate project to handle syslog. This code was taken directly from the Go project as a base to start from. However, this _does_ have TLS support. # Usage Basic usage retains the same interface as the original `syslog` package. We only added to the interface where required to support new functionality. Switch from the standard library: ``` import( //"log/syslog" syslog "github.com/RackSec/srslog" ) ``` You can still use it for local syslog: ``` w, err := syslog.Dial("", "", syslog.LOG_ERR, "testtag") ``` Or to unencrypted UDP: ``` w, err := syslog.Dial("udp", "192.168.0.50:514", syslog.LOG_ERR, "testtag") ``` Or to unencrypted TCP: ``` w, err := syslog.Dial("tcp", "192.168.0.51:514", syslog.LOG_ERR, "testtag") ``` But now you can also send messages via TLS-encrypted TCP: ``` w, err := syslog.DialWithTLSCertPath("tcp+tls", "192.168.0.52:514", syslog.LOG_ERR, "testtag", "/path/to/servercert.pem") ``` And if you need more control over your TLS configuration : ``` pool := x509.NewCertPool() serverCert, err := ioutil.ReadFile("/path/to/servercert.pem") if err != nil { return nil, err } pool.AppendCertsFromPEM(serverCert) config := tls.Config{ RootCAs: pool, } w, err := DialWithTLSConfig(network, raddr, priority, tag, &config) ``` (Note that in both TLS cases, this uses a self-signed certificate, where the remote syslog server has the keypair and the client has only the public key.) And then to write log messages, continue like so: ``` if err != nil { log.Fatal("failed to connect to syslog:", err) } defer w.Close() w.Alert("this is an alert") w.Crit("this is critical") w.Err("this is an error") w.Warning("this is a warning") w.Notice("this is a notice") w.Info("this is info") w.Debug("this is debug") w.Write([]byte("these are some bytes")) ``` If you need further control over connection attempts, you can use the DialWithCustomDialer function. To continue with the DialWithTLSConfig example: ``` netDialer := &net.Dialer{Timeout: time.Second*5} // easy timeouts realNetwork := "tcp" // real network, other vars your dail func can close over dial := func(network, addr string) (net.Conn, error) { // cannot use "network" here as it'll simply be "custom" which will fail return tls.DialWithDialer(netDialer, realNetwork, addr, &config) } w, err := DialWithCustomDialer("custom", "192.168.0.52:514", syslog.LOG_ERR, "testtag", dial) ``` Your custom dial func can set timeouts, proxy connections, and do whatever else it needs before returning a net.Conn. # Generating TLS Certificates We've provided a script that you can use to generate a self-signed keypair: ``` pip install cryptography python script/gen-certs.py ``` That outputs the public key and private key to standard out. Put those into `.pem` files. (And don't put them into any source control. The certificate in the `test` directory is used by the unit tests, and please do not actually use it anywhere else.) # Running Tests Run the tests as usual: ``` go test ``` But we've also provided a test coverage script that will show you which lines of code are not covered: ``` script/coverage --html ``` That will open a new browser tab showing coverage information. # License This project uses the New BSD License, the same as the Go project itself. # Code of Conduct Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. --- ### Overview (`vendor/github.com/agext/levenshtein/README.md`) # A Go package for calculating the Levenshtein distance between two strings [](https://github.com/agext/levenshtein/releases/latest) [](https://godoc.org/github.com/agext/levenshtein)  [](https://travis-ci.org/agext/levenshtein) [](https://coveralls.io/github/agext/levenshtein) [](https://goreportcard.com/report/github.com/agext/levenshtein) This package implements distance and similarity metrics for strings, based on the Levenshtein measure, in [Go](http://golang.org). ## Project Status v1.2.3 Stable: Guaranteed no breaking changes to the API in future v1.x releases. Probably safe to use in production, though provided on "AS IS" basis. This package is being actively maintained. If you encounter any problems or have any suggestions for improvement, please [open an issue](https://github.com/agext/levenshtein/issues). Pull requests are welcome. ## Overview The Levenshtein `Distance` between two strings is the minimum total cost of edits that would convert the first string into the second. The allowed edit operations are insertions, deletions, and substitutions, all at character (one UTF-8 code point) level. Each operation has a default cost of 1, but each can be assigned its own cost equal to or greater than 0. A `Distance` of 0 means the two strings are identical, and the higher the value the more different the strings. Since in practice we are interested in finding if the two strings are "close enough", it often does not make sense to continue the calculation once the result is mathematically guaranteed to exceed a desired threshold. Providing this value to the `Distance` function allows it to take a shortcut and return a lower bound instead of an exact cost when the threshold is exceeded. The `Similarity` function calculates the distance, then converts it into a normalized metric within the range 0..1, with 1 meaning the strings are identical, and 0 that they have nothing in common. A minimum similarity threshold can be provided to speed up the calculation of the metric for strings that are far too dissimilar for the purpose at hand. All values under this threshold are rounded down to 0. The `Match` function provides a similarity metric, with the same range and meaning as `Similarity`, but with a bonus for string pairs that share a common prefix and have a similarity above a "bonus threshold". It uses the same method as proposed by Winkler for the Jaro distance, and the reasoning behind it is that these string pairs are very likely spelling variations or errors, and they are more closely linked than the edit distance alone would suggest. The underlying `Calculate` function is also exported, to allow the building of other derivative metrics, if needed. ## Installation ``` go get github.com/agext/levenshtein ``` ## License Package levenshtein is released under the Apache 2.0 license. See the [LICENSE](LICENSE) file for details. --- ### Overview (`vendor/github.com/anchore/go-struct-converter/README.md`) # Go `struct` Converter A library for converting between Go structs. ```go chain := converter.NewChain(V1{}, V2{}, V3{}) chain.Convert(myV1struct, &myV3struct) ``` ## Details At its core, this library provides a `Convert` function, which automatically handles converting fields with the same name, and "convertable" types. Some examples are: * `string` -> `string` * `string` -> `*string` * `int` -> `string` * `string` -> `[]string` The automatic conversions are implemented when there is an obvious way to convert between the types. A lot more automatic conversions happen -- see [the converter tests](converter_test.go) for a more comprehensive list of what is currently supported. Not everything can be handled automatically, however, so there is also a `ConvertFrom` interface any struct in the graph can implement to perform custom conversion, similar to how the stdlib `MarshalJSON` and `UnmarshalJSON` would be implemented. Additionally, and maybe most importantly, there is a `converter.Chain` available, which orchestrates conversions between _multiple versions_ of structs. This could be thought of similar to database migrations: given a starting struct and a target struct, the `chain.Convert` function iterates through every intermediary migration in order to arrive at the target struct. ## Basic Usage To illustrate usage we'll start with a few basic structs, some of which implement the `ConvertFrom` interface due to breaking changes: ```go // --------- V1 struct definition below --------- type V1 struct { Name string OldField string } // --------- V2 struct definition below --------- type V2 struct { Name string NewField string // this was a renamed field } func (to *V2) ConvertFrom(from interface{}) error { if from, ok := from.(V1); ok { // forward migration to.NewField = from.OldField } return nil } // --------- V3 struct definition below --------- type V3 struct { Name []string FinalField []string // this field was renamed and the type was changed } func (to *V3) ConvertFrom(from interface{}) error { if from, ok := from.(V2); ok { // forward migration to.FinalField = []string{from.NewField} } return nil } ``` Given these type definitions, we can easily set up a conversion chain like this: ```go chain := converter.NewChain(V1{}, V2{}, V3{}) ``` This chain can then be used to convert from an _older version_ to a _newer version_. This is because our `ConvertFrom` definitions are only handling _forward_ migrations. This chain can be used to convert from a `V1` struct to a `V3` struct easily, like this: ```go v1 := // somehow get a populated v1 struct v3 := V3{} chain.Convert(v1, &v3) ``` Since we've defined our chain as `V1` → `V2` → `V3`, the chain will execute conversions to all intermediary structs (`V2`, in this case) and ultimately end when we've populated the `v3` instance. Note we haven't needed to define any conversions on the `Name` field of any structs since this one is convertible between structs: `string` → `string` → `[]string`. ## Backwards Migrations If we wanted to _also_ provide backwards migrations, we could also easily add a case to the `ConvertFrom` methods. The whole set of structs would look something like this: ```go // --------- V1 struct definition below --------- type V1 struct { Name string OldField string } func (to *V1) ConvertFrom(from interface{}) error { if from, ok := from.(V2); ok { // backward migration to.OldField = from.NewField } return nil } // --------- V2 struct definition below --------- type V2 struct { Name string NewField string } func (to *V2) ConvertFrom(from interface{}) error { if from, ok := from.(V1); ok { // forward migration to.NewField = from.OldField } if from, ok := from.(V3); ok { // backward migration to.NewField = from.FinalField[0] } return nil } // --------- V3 struct definition below --------- type V3 struct { Name []string FinalField []string } func (to *V3) ConvertFrom(from interface{}) error { if from, ok := from.(V2); ok { // forward migration to.FinalField = []string{from.NewField} } return nil } ``` At this point we could convert in either direction, for example a `V3` struct could convert to a `V1` struct, with the caveat that there may be data loss, as might need to happen due to changes in the data shapes. ## Contributing If you would like to contribute to this repository, please see the [CONTRIBUTING.md](CONTRIBUTING.md). --- ### Overview (`vendor/github.com/armon/circbuf/README.md`) circbuf ======= This repository provides the `circbuf` package. This provides a `Buffer` object which is a circular (or ring) buffer. It has a fixed size, but can be written to infinitely. Only the last `size` bytes are ever retained. The buffer implements the `io.Writer` interface. Documentation ============= Full documentation can be found on [Godoc](http://godoc.org/github.com/armon/circbuf) Usage ===== The `circbuf` package is very easy to use: ```go buf, _ := NewBuffer(6) buf.Write([]byte("hello world")) if string(buf.Bytes()) != " world" { panic("should only have last 6 bytes!") } ``` --- ### Overview (`vendor/github.com/asaskevich/govalidator/README.md`) govalidator =========== [](https://gitter.im/asaskevich/govalidator?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) [](https://godoc.org/github.com/asaskevich/govalidator) [](https://travis-ci.org/asaskevich/govalidator) [](https://codecov.io/gh/asaskevich/govalidator) [](https://goreportcard.com/report/github.com/asaskevich/govalidator) [](http://go-search.org/view?id=github.com%2Fasaskevich%2Fgovalidator) [](#backers) [](#sponsors) [](https://app.fossa.io/projects/git%2Bgithub.com%2Fasaskevich%2Fgovalidator?ref=badge_shield) A package of validators and sanitizers for strings, structs and collections. Based on [validator.js](https://github.com/chriso/validator.js). #### Installation Make sure that Go is installed on your computer. Type the following command in your terminal: go get github.com/asaskevich/govalidator or you can get specified release of the package with `gopkg.in`: go get gopkg.in/asaskevich/govalidator.v10 After it the package is ready to use. #### Import package in your project Add following line in your `*.go` file: ```go import "github.com/asaskevich/govalidator" ``` If you are unhappy to use long `govalidator`, you can do something like this: ```go import ( valid "github.com/asaskevich/govalidator" ) ``` #### Activate behavior to require all fields have a validation tag by default `SetFieldsRequiredByDefault` causes validation to fail when struct fields do not include validations or are not explicitly marked as exempt (using `valid:"-"` or `valid:"email,optional"`). A good place to activate this is a package init function or the main() function. `SetNilPtrAllowedByRequired` causes validation to pass when struct fields marked by `required` are set to nil. This is disabled by default for consistency, but some packages that need to be able to determine between `nil` and `zero value` state can use this. If disabled, both `nil` and `zero` values cause validation errors. ```go import "github.com/asaskevich/govalidator" func init() { govalidator.SetFieldsRequiredByDefault(true) } ``` Here's some code to explain it: ```go // this struct definition will fail govalidator.ValidateStruct() (and the field values do not matter): type exampleStruct struct { Name string `` Email string `valid:"email"` } // this, however, will only fail when Email is empty or an invalid email address: type exampleStruct2 struct { Name string `valid:"-"` Email string `valid:"email"` } // lastly, this will only fail when Email is an invalid email address but not when it's empty: type exampleStruct2 struct { Name string `valid:"-"` Email string `valid:"email,optional"` } ``` #### Recent breaking changes (see [#123](https://github.com/asaskevich/govalidator/pull/123)) ##### Custom validator function signature A context was added as the second parameter, for structs this is the object being validated – this makes dependent validation possible. ```go import "github.com/asaskevich/govalidator" // old signature func(i interface{}) bool // new signature func(i interface{}, o interface{}) bool ``` ##### Adding a custom validator This was changed to prevent data races when accessing custom validators. ```go import "github.com/asaskevich/govalidator" // before govalidator.CustomTypeTagMap["customByteArrayValidator"] = func(i interface{}, o interface{}) bool { // ... } // after govalidator.CustomTypeTagMap.Set("customByteArrayValidator", func(i interface{}, o interface{}) bool { // ... }) ``` #### List of functions: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` #### Examples ###### IsURL ```go println(govalidator.IsURL(`http://user@pass:domain.com/path/page`)) ``` ###### IsType ```go println(govalidator.IsType("Bob", "string")) println(govalidator.IsType(1, "int")) i := 1 println(govalidator.IsType(&i, "*int")) ``` IsType can be used through the tag `type` which is essential for map validation: ```go type User struct { Name string `valid:"type(string)"` Age int `valid:"type(int)"` Meta interface{} `valid:"type(string)"` } result, err := govalidator.ValidateStruct(User{"Bob", 20, "meta"}) if err != nil { println("error: " + err.Error()) } println(result) ``` ###### ToString ```go type User struct { FirstName string LastName string } str := govalidator.ToString(&User{"John", "Juan"}) println(str) ``` ###### Each, Map, Filter, Count for slices Each iterates over the slice/array and calls Iterator for every item ```go data := []interface{}{1, 2, 3, 4, 5} var fn govalidator.Iterator = func(value interface{}, index int) { println(value.(int)) } govalidator.Each(data, fn) ``` ```go data := []interface{}{1, 2, 3, 4, 5} var fn govalidator.ResultIterator = func(value interface{}, index int) interface{} { return value.(int) * 3 } _ = govalidator.Map(data, fn) // result = []interface{}{1, 6, 9, 12, 15} ``` ```go data := []interface{}{1, 2, 3, 4, 5, 6, 7, 8, 9, 10} var fn govalidator.ConditionIterator = func(value interface{}, index int) bool { return value.(int)%2 == 0 } _ = govalidator.Filter(data, fn) // result = []interface{}{2, 4, 6, 8, 10} _ = govalidator.Count(data, fn) // result = 5 ``` ###### ValidateStruct [#2](https://github.com/asaskevich/govalidator/pull/2) If you want to validate structs, you can use tag `valid` for any field in your structure. All validators used with this field in one tag are separated by comma. If you want to skip validation, place `-` in your tag. If you need a validator that is not on the list below, you can add it like this: ```go govalidator.TagMap["duck"] = govalidator.Validator(func(str string) bool { return str == "duck" }) ``` For completely custom validators (interface-based), see below. Here is a list of available validators for struct fields (validator - used function): ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Validators with parameters ```go "range(min|max)": Range, "length(min|max)": ByteLength, "runelength(min|max)": RuneLength, "stringlength(min|max)": StringLength, "matches(pattern)": StringMatches, "in(string1|string2|...|stringN)": IsIn, "rsapub(keylength)" : IsRsaPub, "minstringlength(int): MinStringLength, "maxstringlength(int): MaxStringLength, ``` Validators with parameters for any type ```go "type(type)": IsType, ``` And here is small example of usage: ```go type Post struct { Title string `valid:"alphanum,required"` Message string `valid:"duck,ascii"` Message2 string `valid:"animal(dog)"` AuthorIP string `valid:"ipv4"` Date string `valid:"-"` } post := &Post{ Title: "My Example Post", Message: "duck", Message2: "dog", AuthorIP: "123.234.54.3", } // Add your own struct validation tags govalidator.TagMap["duck"] = govalidator.Validator(func(str string) bool { return str == "duck" }) // Add your own struct validation tags with parameter govalidator.ParamTagMap["animal"] = govalidator.ParamValidator(func(str string, params ...string) bool { species := params[0] return str == species }) govalidator.ParamTagRegexMap["animal"] = regexp.MustCompile("^animal\\((\\w+)\\)$") result, err := govalidator.ValidateStruct(post) if err != nil { println("error: " + err.Error()) } println(result) ``` ###### ValidateMap [#2](https://github.com/asaskevich/govalidator/pull/338) If you want to validate maps, you can use the map to be validated and a validation map that contain the same tags used in ValidateStruct, both maps have to be in the form `map[string]interface{}` So here is small example of usage: ```go var mapTemplate = map[string]interface{}{ "name":"required,alpha", "family":"required,alpha", "email":"required,email", "cell-phone":"numeric", "address":map[string]interface{}{ "line1":"required,alphanum", "line2":"alphanum", "postal-code":"numeric", }, } var inputMap = map[string]interface{}{ "name":"Bob", "family":"Smith", "email":"foo@bar.baz", "address":map[string]interface{}{ "line1":"", "line2":"", "postal-code":"", }, } result, err := govalidator.ValidateMap(inputMap, mapTemplate) if err != nil { println("error: " + err.Error()) } println(result) ``` ###### WhiteList ```go // Remove all characters from string ignoring characters between "a" and "z" println(govalidator.WhiteList("a3a43a5a4a3a2a23a4a5a4a3a4", "a-z") == "aaaaaaaaaaaa") ``` ###### Custom validation functions Custom validation using your own domain specific validators is also available - here's an example of how to use it: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ###### Loop over Error() By default .Error() returns all errors in a single String. To access each error you can do this: ```go if err != nil { errs := err.(govalidator.Errors).Errors() for _, e := range errs { fmt.Println(e.Error()) } } ``` ###### Custom error messages Custom error messages are supported via annotations by adding the `~` separator - here's an example of how to use it: ```go type Ticket struct { Id int64 `json:"id"` FirstName string `json:"firstname" valid:"required~First name is blank"` } ``` #### Notes Documentation is available here: [godoc.org](https://godoc.org/github.com/asaskevich/govalidator). Full information about code coverage is also available here: [govalidator on gocover.io](http://gocover.io/github.com/asaskevich/govalidator). #### Support If you do have a contribution to the package, feel free to create a Pull Request or an Issue. #### What to contribute If you don't know what to do, there are some features and functions that need to be done - [ ] Refactor code - [ ] Edit docs and [README](https://github.com/asaskevich/govalidator/README.md): spellcheck, grammar and typo check - [ ] Create actual list of contributors and projects that currently using this package - [ ] Resolve [issues and bugs](https://github.com/asaskevich/govalidator/issues) - [ ] Update actual [list of functions](https://github.com/asaskevich/govalidator#list-of-functions) - [ ] Update [list of validators](https://github.com/asaskevich/govalidator#validatestruct-2) that available for `ValidateStruct` and add new - [ ] Implement new validators: `IsFQDN`, `IsIMEI`, `IsPostalCode`, `IsISIN`, `IsISRC` etc - [x] Implement [validation by maps](https://github.com/asaskevich/govalidator/issues/224) - [ ] Implement fuzzing testing - [ ] Implement some struct/map/array utilities - [ ] Implement map/array validation - [ ] Implement benchmarking - [ ] Implement batch of examples - [ ] Look at forks for new features and fixes #### Advice Feel free to create what you want, but keep in mind when you implement new features: - Code must be clear and readable, names of variables/constants clearly describes what they are doing - Public functions must be documented and described in source file and added to README.md to the list of available functions - There are must be unit-tests for any new functions and improvements ## Credits ### Contributors This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)]. #### Special thanks to [contributors](https://github.com/asaskevich/govalidator/graphs/contributors) * [Daniel Lohse](https://github.com/annismckenzie) * [Attila Oláh](https://github.com/attilaolah) * [Daniel Korner](https://github.com/Dadie) * [Steven Wilkin](https://github.com/stevenwilkin) * [Deiwin Sarjas](https://github.com/deiwin) * [Noah Shibley](https://github.com/slugmobile) * [Nathan Davies](https://github.com/nathj07) * [Matt Sanford](https://github.com/mzsanford) * [Simon ccl1115](https://github.com/ccl1115) ### Backers Thank you to all our backers! 🙏 [[Become a backer](https://opencollective.com/govalidator#backer)] ### Sponsors Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [[Become a sponsor](https://opencollective.com/govalidator#sponsor)] ## License [](https://app.fossa.io/projects/git%2Bgithub.com%2Fasaskevich%2Fgovalidator?ref=badge_large) --- ### Overview (`vendor/github.com/aws/smithy-go/README.md`) # Smithy Go [](https://github.com/aws/smithy-go/actions/workflows/go.yml) [](https://github.com/aws/smithy-go/actions/workflows/codegen.yml) [](LICENSE) [Smithy](https://smithy.io/) code generators for [Go](https://go.dev/), and the runtime package the generated code depends on. > [!WARNING] > All interfaces in this repository are subject to change. The client code > generator in particular powers > [aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2), but arbitrary client > generation is in an early stage of development: > * Generated clients are missing certain features that were originally > implemented SDK-side (e.g. retries) > * There may be bugs > * The public APIs of generated clients may be unstable > > If you are interested in using the client code generators, we encourage you > to experiment and share any feedback with us in an issue. --- This repository contains two major components: 1. **Codegen** ([`codegen/`](codegen)): A Java/Gradle-based Smithy build plugin that generates Go client, server, and shape code from Smithy models. 2. **Runtime** (this module, `github.com/aws/smithy-go`): The Go packages that code generated by the plugins above depends on. > [!NOTE] > This repository does not contain any generated clients, such as for S3 or > other AWS services. Rather, these are the tools that facilitate generating > those clients (and non-AWS Smithy clients) from a Smithy model. ## Getting started If this is your first time using Smithy, follow the [Smithy Quickstart guide](https://smithy.io/2.0/quickstart.html) to learn the basics and create a simple Smithy model. The smithy-go runtime requires a minimum version of **Go 1.24**. ### Codegen To generate a Go client from a Smithy model, apply the `go-codegen` plugin in your `smithy-build.json`: ```json { "version": "1.0", "sources": [ "models" ], "maven": { "dependencies": [ "software.amazon.smithy.go:smithy-go-codegen:[0.1.0,2.0)" ] }, "plugins": { "go-codegen": { "service": "example.weather#Weather", "module": "github.com/example/weather", "generateGoMod": true, "goDirective": "1.24" } } } ``` This repository implements the following Smithy build plugins: | ID | GAV prefix | Description | |----|------------|-------------| | `go-codegen` | `software.amazon.smithy.go:smithy-go-codegen` | Go client code generation for Smithy models. | | `go-server-codegen` | `software.amazon.smithy.go:smithy-go-codegen` | Go server code generation for Smithy models (work-in-progress, undocumented). | | `go-shape-codegen` | `software.amazon.smithy.go:smithy-go-codegen` | Go shape code generation (types only) for Smithy models (work-in-progress, undocumented). | #### `go-codegen` settings [`GoSettings`](codegen/smithy-go-codegen/src/main/java/software/amazon/smithy/go/codegen/GoSettings.java) contains all of the settings enabled from `smithy-build.json`. The up-to-date list of top-level properties enabled for `go-codegen` can be found in `GoSettings::from()`. | Setting | Type | Required | Description | |-----------------|---------|----------|-------------------------------------------------------------------------------------------------------------------------------| | `service` | string | yes | The Shape ID of the service for which to generate the client. | | `module` | string | yes | Name of the module in `generated.json` (and `go.mod` if `generateGoMod` is enabled) and `doc.go`. | | `generateGoMod` | boolean | | Whether to generate a default `go.mod` file. The default value is `false`. | | `goDirective` | string | | [Go directive](https://go.dev/ref/mod#go-mod-file-go) of the module. The default value is the minimum supported Go version. | #### Supported protocols The protocol a generated client uses is configured by the `Protocol` field on the client's `Options`. The SDK configures a default based on the protocol traits applied to the modeled service. Each protocol is implemented as its own package under [`transport/http/protocol`](transport/http/protocol). Serialization for these protocols is handled by internal codecs under `protocol/internal` (e.g. `protocol/internal/json`, `protocol/internal/cbor`, `protocol/internal/xml`); these largely supersede the top-level [`encoding/`](encoding) packages of the same name; see [Encoding](#encoding) below. | Protocol | Package | Notes | |----------|---------|-------| | [`smithy.protocols#rpcv2Cbor`](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html) | [`protocol/rpcv2`](transport/http/protocol/rpcv2) | | | [`aws.protocols#restJson1`](https://smithy.io/2.0/aws/protocols/aws-restjson1-protocol.html) | [`protocol/restjson1`](transport/http/protocol/restjson1) | | | [`aws.protocols#restXml`](https://smithy.io/2.0/aws/protocols/aws-restxml-protocol.html) | [`protocol/restxml`](transport/http/protocol/restxml) | | | [`aws.protocols#awsJson1_0`](https://smithy.io/2.0/aws/protocols/aws-json-1_0-protocol.html) | [`protocol/awsjson`](transport/http/protocol/awsjson) | `awsjson.New10` | | [`aws.protocols#awsJson1_1`](https://smithy.io/2.0/aws/protocols/aws-json-1_1-protocol.html) | [`protocol/awsjson`](transport/http/protocol/awsjson) | `awsjson.New11` | | [`aws.protocols#awsQuery`](https://smithy.io/2.0/aws/protocols/aws-query-protocol.html) | [`protocol/awsquery`](transport/http/protocol/awsquery) | | | [`aws.protocols#ec2Query`](https://smithy.io/2.0/aws/protocols/aws-ec2-query-protocol.html) | [`protocol/ec2query`](transport/http/protocol/ec2query) | | These packages are client-only; smithy-go does not currently implement server-side protocol support. Building and testing the codegen plugins is done via Gradle from the `codegen/` directory: ```bash # Build and test codegen cd codegen && ./gradlew build # Publish to local Maven for downstream use cd codegen && ./gradlew publishToMavenLocal ``` See [`codegen/README.md`](codegen/README.md) for local setup instructions. ## Runtime packages All generated clients depend on this module (`github.com/aws/smithy-go`) at runtime. It also includes a few standalone submodules published separately. ### Core - [`middleware`](middleware): The middleware stack that drives the operation pipeline. - [`transport/http`](transport/http): HTTP request/response types and middleware. - [`auth`](auth): Auth identity and scheme interfaces. - [`auth/bearer`](auth/bearer): Bearer token auth. - [`endpoints`](endpoints): Endpoint resolution types. - [`context`](context): Context helpers. - [`logging`](logging): Logging interfaces. - [`tracing`](tracing): Tracing interfaces. - [`metrics`](metrics): Metrics interfaces. ### Encoding Wire format encoders/decoders under [`encoding/`](encoding). > [!NOTE] > Most of these packages are effectively legacy. The protocol > implementations under [`transport/http/protocol`](transport/http/protocol) > (used by current client codegen) have their own internal codecs under > `protocol/internal/*` and generally do not build on these packages. > The `encoding/json`, `encoding/xml`, and `encoding/cbor` packages otherwise > remain in use by [`document/json`](document/json)/[`document/cbor`](document/cbor) > and [`eventstream`](eventstream), and by existing generated SDK code that > predates the newer protocol implementations. - [`encoding/json`](encoding/json): JSON encoding/decoding. - [`encoding/xml`](encoding/xml): XML encoding/decoding. - [`encoding/cbor`](encoding/cbor): CBOR encoding/decoding (used by `rpcv2Cbor`). - [`encoding/httpbinding`](encoding/httpbinding): HTTP binding serde helpers. ### Document - [`document`](document): Smithy document type abstraction. - [`document/json`](document/json): JSON document codec. ### Utilities - [`container`](container): Generic container types. - [`io`](io): I/O helpers. - [`ptr`](ptr): Pointer-to/from-value helpers. - [`time`](time): Smithy timestamp format helpers. - [`rand`](rand): UUID/randomness helpers. - [`testing`](testing): Test assertion helpers for generated protocol tests. ### Standalone modules These are published as separate Go modules, each with its own `go.mod`. - [`aws-http-auth`](aws-http-auth): AWS SigV4/SigV4A HTTP signing. - [`tracing/smithyoteltracing`](tracing/smithyoteltracing): OpenTelemetry tracing adapter. - [`metrics/smithyotelmetrics`](metrics/smithyotelmetrics): OpenTelemetry metrics adapter. ## Development See [CONTRIBUTING](CONTRIBUTING.md) for more information on contributing to this project, including the changelog process for runtime changes. ## License This project is licensed under the Apache-2.0 License. --- ### Overview (`vendor/github.com/bits-and-blooms/bitset/README.md`) # bitset *Go language library to map between non-negative integers and boolean values* [](https://github.com/bits-and-blooms/bitset/actions?query=workflow%3ATest) [](https://pkg.go.dev/github.com/bits-and-blooms/bitset?tab=doc) This library is part of the [awesome go collection](https://github.com/avelino/awesome-go). It is used in production by several important systems. ## Description Package bitset implements bitsets, a mapping between non-negative integers and boolean values. It should be more efficient than map[uint] bool. It provides methods for setting, clearing, flipping, and testing individual integers. But it also provides set intersection, union, difference, complement, and symmetric operations, as well as tests to check whether any, all, or no bits are set, and querying a bitset's current length and number of positive bits. BitSets are expanded to the size of the largest set bit; the memory allocation is approximately Max bits, where Max is the largest set bit. BitSets are never shrunk automatically, but `Shrink` and `Compact` methods are available. On creation, a hint can be given for the number of bits that will be used. Many of the methods, including Set, Clear, and Flip, return a BitSet pointer, which allows for chaining. ### Example use: ```go package main import ( "fmt" "math/rand" "github.com/bits-and-blooms/bitset" ) func main() { fmt.Printf("Hello from BitSet!\n") var b bitset.BitSet // play some Go Fish for i := 0; i < 100; i++ { card1 := uint(rand.Intn(52)) card2 := uint(rand.Intn(52)) b.Set(card1) if b.Test(card2) { fmt.Println("Go Fish!") } b.Clear(card1) } // Chaining b.Set(10).Set(11) for i, e := b.NextSet(0); e; i, e = b.NextSet(i + 1) { fmt.Println("The following bit is set:", i) } if b.Intersection(bitset.New(100).Set(10)).Count() == 1 { fmt.Println("Intersection works.") } else { fmt.Println("Intersection doesn't work???") } } ``` If you have Go 1.23 or better, you can iterate over the set bits like so: ```go for i := range b.EachSet() {} ``` Package documentation is at: https://pkg.go.dev/github.com/bits-and-blooms/bitset?tab=doc ## Serialization You may serialize a bitset safely and portably to a stream of bytes as follows: ```Go const length = 9585 const oneEvery = 97 bs := bitset.New(length) // Add some bits for i := uint(0); i < length; i += oneEvery { bs = bs.Set(i) } var buf bytes.Buffer n, err := bs.WriteTo(&buf) if err != nil { // failure } // Here n == buf.Len() ``` You can later deserialize the result as follows: ```Go // Read back from buf bs = bitset.New() n, err = bs.ReadFrom(&buf) if err != nil { // error } // n is the number of bytes read ``` The `ReadFrom` function attempts to read the data into the existing BitSet instance, to minimize memory allocations. *Performance tip*: When reading and writing to a file or a network connection, you may get better performance by wrapping your streams with `bufio` instances. E.g., ```Go f, err := os.Create("myfile") w := bufio.NewWriter(f) ``` ```Go f, err := os.Open("myfile") r := bufio.NewReader(f) ``` ## Memory Usage The memory usage of a bitset using `N` bits is at least `N/8` bytes. The number of bits in a bitset is at least as large as one plus the greatest bit index you have accessed. Thus it is possible to run out of memory while using a bitset. If you have lots of bits, you might prefer compressed bitsets, like the [Roaring bitmaps](https://roaringbitmap.org) and its [Go implementation](https://github.com/RoaringBitmap/roaring). The `roaring` library allows you to go back and forth between compressed Roaring bitmaps and the conventional bitset instances: ```Go mybitset := roaringbitmap.ToBitSet() newroaringbitmap := roaring.FromBitSet(mybitset) ``` ### Goroutine safety In general, it's not safe to access the same BitSet using different goroutines--they are unsynchronized for performance. Should you want to access a BitSet from more than one goroutine, you should provide synchronization. Typically this is done by using channels to pass the *BitSet around (in Go style; so there is only ever one owner), or by using `sync.Mutex` to serialize operations on BitSets. ## Installation ```bash go get github.com/bits-and-blooms/bitset ``` ## Users Databases & Storage - [milvus-io/milvus](https://github.com/milvus-io/milvus) — popular open-source vector database - [cubefs/cubefs](https://github.com/cubefs/cubefs) — CNCF distributed filesystem (used at JD.com) - [sourcenetwork/defradb](https://github.com/sourcenetwork/defradb) — decentralized document database - [siglens/siglens](https://github.com/siglens/siglens) — open-source observability/log analytics platform - [ByteStorage/FlyDB](https://github.com/ByteStorage/FlyDB) — high-performance KV store Search & Indexing - [blevesearch/bleve](https://github.com/blevesearch/bleve) — full-text search library for Go - [blevesearch/vellum](https://github.com/blevesearch/vellum) — FST library used internally by Bleve - [blugelabs/bluge](https://github.com/blugelabs/bluge) — modern Go search library (successor to Bleve) - [geange/lucene-go](https://github.com/geange/lucene-go) — Lucene port to Go Blockchain & Cryptography - [consensys/gnark](https://github.com/consensys/gnark) — zk-SNARK proving system - [consensys/gnark-crypto](https://github.com/consensys/gnark-crypto) — elliptic curve cryptography for ZK proofs - [NethermindEth/juno](https://github.com/NethermindEth/juno) — StarkNet full node - [onflow/cadence](https://github.com/onflow/cadence) — Flow blockchain smart contract language runtime - [cosmos/cosmos-sdk](https://github.com/cosmos/cosmos-sdk) — Cosmos blockchain SDK (slashing module) - [babylonlabs-io/babylon](https://github.com/babylonlabs-io/babylon) — Bitcoin staking protocol - [bnb-chain/greenfield](https://github.com/bnb-chain/greenfield) — BNB Chain decentralized storage Infrastructure & Networking - [moby/swarmkit](https://github.com/moby/swarmkit) — Docker Swarm orchestration (ID management) - [coredhcp/coredhcp](https://github.com/coredhcp/coredhcp) — pluggable DHCP server (bitmap IP allocation) - [Scalingo/sand](https://github.com/Scalingo/sand) — private networking for containers (IP allocation) - [AliyunContainerService/terway](https://github.com/AliyunContainerService/terway) — Alibaba Cloud K8s CNI - [pomerium/pomerium](https://github.com/pomerium/pomerium) — identity-aware access proxy Distributed Systems & Messaging - [apache/pulsar-client-go](https://github.com/apache/pulsar-client-go) — official Apache Pulsar Go client - [d7y.io/dragonfly](https://github.com/dragonflyoss/Dragonfly2) — CNCF P2P file distribution system - [RoaringBitmap/roaring](https://github.com/RoaringBitmap/roaring) — compressed bitmap library (interops with bitset) - [zhenghaoz/gorse](https://github.com/zhenghaoz/gorse) — Go recommendation engine Machine Learning / Vector Search - [fogfish/hnsw](https://github.com/fogfish/hnsw) — HNSW approximate nearest-neighbor graph - [semafind/semadb](https://github.com/semafind/semadb) — vector database Developer Tools & Security - [bearer/bearer](https://github.com/bearer/bearer) — security code scanner (AST traversal) - [zrepl/zrepl](https://github.com/zrepl/zrepl) — ZFS replication tool - [NVIDIA/dcgm-exporter](https://github.com/NVIDIA/dcgm-exporter) — NVIDIA GPU Prometheus exporter - [NVIDIA/go-dcgm](https://github.com/NVIDIA/go-dcgm) — NVIDIA datacenter GPU management bindings - [gohugoio/hugo](https://github.com/gohugoio/hugo) — the Hugo static site generator - [godoctor/godoctor](https://github.com/godoctor/godoctor) — Go refactoring tool (dataflow analysis) ## Contributing If you wish to contribute to this project, please branch and issue a pull request against master ("[GitHub Flow](https://guides.github.com/introduction/flow/)") ## Running all tests Before committing the code, please check if it passes tests, has adequate coverage, etc. ```bash go test go test -cover ``` ## Further reading

Mastering Programming: From Testing to Performance in Go

--- ### Overview (`vendor/github.com/blang/semver/README.md`) semver for golang [](https://travis-ci.org/blang/semver) [](https://godoc.org/github.com/blang/semver) [](https://coveralls.io/r/blang/semver?branch=master) ====== semver is a [Semantic Versioning](http://semver.org/) library written in golang. It fully covers spec version `2.0.0`. Usage ----- ```bash $ go get github.com/blang/semver ``` Note: Always vendor your dependencies or fix on a specific version tag. ```go import github.com/blang/semver v1, err := semver.Make("1.0.0-beta") v2, err := semver.Make("2.0.0-beta") v1.Compare(v2) ``` Also check the [GoDocs](http://godoc.org/github.com/blang/semver). Why should I use this lib? ----- - Fully spec compatible - No reflection - No regex - Fully tested (Coverage >99%) - Readable parsing/validation errors - Fast (See [Benchmarks](#benchmarks)) - Only Stdlib - Uses values instead of pointers - Many features, see below Features ----- - Parsing and validation at all levels - Comparator-like comparisons - Compare Helper Methods - InPlace manipulation - Ranges `>=1.0.0 <2.0.0 || >=3.0.0 !3.0.1-beta.1` - Wildcards `>=1.x`, `<=2.5.x` - Sortable (implements sort.Interface) - database/sql compatible (sql.Scanner/Valuer) - encoding/json compatible (json.Marshaler/Unmarshaler) Ranges ------ A `Range` is a set of conditions which specify which versions satisfy the range. A condition is composed of an operator and a version. The supported operators are: - `<1.0.0` Less than `1.0.0` - `<=1.0.0` Less than or equal to `1.0.0` - `>1.0.0` Greater than `1.0.0` - `>=1.0.0` Greater than or equal to `1.0.0` - `1.0.0`, `=1.0.0`, `==1.0.0` Equal to `1.0.0` - `!1.0.0`, `!=1.0.0` Not equal to `1.0.0`. Excludes version `1.0.0`. Note that spaces between the operator and the version will be gracefully tolerated. A `Range` can link multiple `Ranges` separated by space: Ranges can be linked by logical AND: - `>1.0.0 <2.0.0` would match between both ranges, so `1.1.1` and `1.8.7` but not `1.0.0` or `2.0.0` - `>1.0.0 <3.0.0 !2.0.3-beta.2` would match every version between `1.0.0` and `3.0.0` except `2.0.3-beta.2` Ranges can also be linked by logical OR: - `<2.0.0 || >=3.0.0` would match `1.x.x` and `3.x.x` but not `2.x.x` AND has a higher precedence than OR. It's not possible to use brackets. Ranges can be combined by both AND and OR - `>1.0.0 <2.0.0 || >3.0.0 !4.2.1` would match `1.2.3`, `1.9.9`, `3.1.1`, but not `4.2.1`, `2.1.1` Range usage: ``` v, err := semver.Parse("1.2.3") range, err := semver.ParseRange(">1.0.0 <2.0.0 || >=3.0.0") if range(v) { //valid } ``` Example ----- Have a look at full examples in [examples/main.go](examples/main.go) ```go import github.com/blang/semver v, err := semver.Make("0.0.1-alpha.preview+123.github") fmt.Printf("Major: %d\n", v.Major) fmt.Printf("Minor: %d\n", v.Minor) fmt.Printf("Patch: %d\n", v.Patch) fmt.Printf("Pre: %s\n", v.Pre) fmt.Printf("Build: %s\n", v.Build) // Prerelease versions array if len(v.Pre) > 0 { fmt.Println("Prerelease versions:") for i, pre := range v.Pre { fmt.Printf("%d: %q\n", i, pre) } } // Build meta data array if len(v.Build) > 0 { fmt.Println("Build meta data:") for i, build := range v.Build { fmt.Printf("%d: %q\n", i, build) } } v001, err := semver.Make("0.0.1") // Compare using helpers: v.GT(v2), v.LT, v.GTE, v.LTE v001.GT(v) == true v.LT(v001) == true v.GTE(v) == true v.LTE(v) == true // Or use v.Compare(v2) for comparisons (-1, 0, 1): v001.Compare(v) == 1 v.Compare(v001) == -1 v.Compare(v) == 0 // Manipulate Version in place: v.Pre[0], err = semver.NewPRVersion("beta") if err != nil { fmt.Printf("Error parsing pre release version: %q", err) } fmt.Println("\nValidate versions:") v.Build[0] = "?" err = v.Validate() if err != nil { fmt.Printf("Validation failed: %s\n", err) } ``` Benchmarks ----- BenchmarkParseSimple-4 5000000 390 ns/op 48 B/op 1 allocs/op BenchmarkParseComplex-4 1000000 1813 ns/op 256 B/op 7 allocs/op BenchmarkParseAverage-4 1000000 1171 ns/op 163 B/op 4 allocs/op BenchmarkStringSimple-4 20000000 119 ns/op 16 B/op 1 allocs/op BenchmarkStringLarger-4 10000000 206 ns/op 32 B/op 2 allocs/op BenchmarkStringComplex-4 5000000 324 ns/op 80 B/op 3 allocs/op BenchmarkStringAverage-4 5000000 273 ns/op 53 B/op 2 allocs/op BenchmarkValidateSimple-4 200000000 9.33 ns/op 0 B/op 0 allocs/op BenchmarkValidateComplex-4 3000000 469 ns/op 0 B/op 0 allocs/op BenchmarkValidateAverage-4 5000000 256 ns/op 0 B/op 0 allocs/op BenchmarkCompareSimple-4 100000000 11.8 ns/op 0 B/op 0 allocs/op BenchmarkCompareComplex-4 50000000 30.8 ns/op 0 B/op 0 allocs/op BenchmarkCompareAverage-4 30000000 41.5 ns/op 0 B/op 0 allocs/op BenchmarkSort-4 3000000 419 ns/op 256 B/op 2 allocs/op BenchmarkRangeParseSimple-4 2000000 850 ns/op 192 B/op 5 allocs/op BenchmarkRangeParseAverage-4 1000000 1677 ns/op 400 B/op 10 allocs/op BenchmarkRangeParseComplex-4 300000 5214 ns/op 1440 B/op 30 allocs/op BenchmarkRangeMatchSimple-4 50000000 25.6 ns/op 0 B/op 0 allocs/op BenchmarkRangeMatchAverage-4 30000000 56.4 ns/op 0 B/op 0 allocs/op BenchmarkRangeMatchComplex-4 10000000 153 ns/op 0 B/op 0 allocs/op See benchmark cases at [semver_test.go](semver_test.go) Motivation ----- I simply couldn't find any lib supporting the full spec. Others were just wrong or used reflection and regex which i don't like. Contribution ----- Feel free to make a pull request. For bigger changes create a issue first to discuss about it. License ----- See [LICENSE](LICENSE) file. --- ### Overview (`vendor/github.com/cilium/ebpf/README.md`) # eBPF [](https://pkg.go.dev/github.com/cilium/ebpf) ebpf-go is a pure Go library that provides utilities for loading, compiling, and debugging eBPF programs. It has minimal external dependencies and is intended to be used in long running processes. See [ebpf.io](https://ebpf.io) for complementary projects from the wider eBPF ecosystem. ## Getting Started Please take a look at our [Getting Started] guide. [Contributions](https://ebpf-go.dev/contributing) are highly encouraged, as they highlight certain use cases of eBPF and the library, and help shape the future of the project. ## Getting Help The community actively monitors our [GitHub Discussions](https://github.com/cilium/ebpf/discussions) page. Please search for existing threads before starting a new one. Refrain from opening issues on the bug tracker if you're just starting out or if you're not sure if something is a bug in the library code. Alternatively, [join](https://ebpf.io/slack) the [#ebpf-go](https://cilium.slack.com/messages/ebpf-go) channel on Slack if you have other questions regarding the project. Note that this channel is ephemeral and has its history erased past a certain point, which is less helpful for others running into the same problem later. ## Packages This library includes the following packages: * [asm](https://pkg.go.dev/github.com/cilium/ebpf/asm) contains a basic assembler, allowing you to write eBPF assembly instructions directly within your Go code. (You don't need to use this if you prefer to write your eBPF program in C.) * [cmd/bpf2go](https://pkg.go.dev/github.com/cilium/ebpf/cmd/bpf2go) allows compiling and embedding eBPF programs written in C within Go code. As well as compiling the C code, it auto-generates Go code for loading and manipulating the eBPF program and map objects. * [link](https://pkg.go.dev/github.com/cilium/ebpf/link) allows attaching eBPF to various hooks * [perf](https://pkg.go.dev/github.com/cilium/ebpf/perf) allows reading from a `PERF_EVENT_ARRAY` * [ringbuf](https://pkg.go.dev/github.com/cilium/ebpf/ringbuf) allows reading from a `BPF_MAP_TYPE_RINGBUF` map * [features](https://pkg.go.dev/github.com/cilium/ebpf/features) implements the equivalent of `bpftool feature probe` for discovering BPF-related kernel features using native Go. * [rlimit](https://pkg.go.dev/github.com/cilium/ebpf/rlimit) provides a convenient API to lift the `RLIMIT_MEMLOCK` constraint on kernels before 5.11. * [btf](https://pkg.go.dev/github.com/cilium/ebpf/btf) allows reading the BPF Type Format. * [pin](https://pkg.go.dev/github.com/cilium/ebpf/pin) provides APIs for working with pinned objects on bpffs. ## Requirements * A version of Go that is [supported by upstream](https://golang.org/doc/devel/release.html#policy) * Linux (amd64, arm64): CI is run against kernel.org LTS releases. >= 4.4 should work but EOL'ed versions are not supported. * Windows (amd64): CI is run against Windows Server 2022. Only the latest eBPF for Windows release is supported. * Other architectures are best effort. 32bit arches are not supported. ## License MIT ### eBPF Gopher The eBPF honeygopher is based on the Go gopher designed by Renee French. [Getting Started]: https://ebpf-go.dev/guides/getting-started/ --- ### Overview (`vendor/github.com/containerd/console/README.md`) # console [](https://pkg.go.dev/github.com/containerd/console) [](https://github.com/containerd/console/actions?query=workflow%3ACI) [](https://goreportcard.com/report/github.com/containerd/console) Golang package for dealing with consoles. Light on deps and a simple API. ## Modifying the current process ```go current := console.Current() defer current.Reset() if err := current.SetRaw(); err != nil { } ws, err := current.Size() current.Resize(ws) ``` ## Project details console is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/errdefs/README.md`) # errdefs A Go package for defining and checking common containerd errors. ## Project details **errdefs** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/fifo/readme.md`) ### fifo [](https://pkg.go.dev/github.com/containerd/fifo) [](https://github.com/containerd/fifo/actions?query=workflow%3ACI) [](https://codecov.io/gh/containerd/fifo) [](https://goreportcard.com/report/github.com/containerd/fifo) Go package for handling fifos in a sane way. ``` // OpenFifo opens a fifo. Returns io.ReadWriteCloser. // Context can be used to cancel this function until open(2) has not returned. // Accepted flags: // - syscall.O_CREAT - create new fifo if one doesn't exist // - syscall.O_RDONLY - open fifo only from reader side // - syscall.O_WRONLY - open fifo only from writer side // - syscall.O_RDWR - open fifo from both sides, never block on syscall level // - syscall.O_NONBLOCK - return io.ReadWriteCloser even if other side of the // fifo isn't open. read/write will be connected after the actual fifo is // open or after fifo is closed. func OpenFifo(ctx context.Context, fn string, flag int, perm os.FileMode) (io.ReadWriteCloser, error) // Read from a fifo to a byte array. func (f *fifo) Read(b []byte) (int, error) // Write from byte array to a fifo. func (f *fifo) Write(b []byte) (int, error) // Close the fifo. Next reads/writes will error. This method can also be used // before open(2) has returned and fifo was never opened. func (f *fifo) Close() error ``` ## Project details The fifo is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/master/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/master/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/master/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/go-cni/README.md`) # go-cni [](https://pkg.go.dev/github.com/containerd/go-cni) [](https://github.com/containerd/go-cni/actions?query=workflow%3ACI) [](https://codecov.io/gh/containerd/go-cni) [](https://goreportcard.com/report/github.com/containerd/go-cni) A generic CNI library to provide APIs for CNI plugin interactions. The library provides APIs to: - Load CNI network config from different sources - Setup networks for container namespace - Remove networks from container namespace - Query status of CNI network plugin initialization - Check verifies the network is still in desired state go-cni aims to support plugins that implement the [Container Network Interface](https://github.com/containernetworking/cni). ## Usage ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Project details The go-cni is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/go-runc/README.md`) # go-runc [](https://pkg.go.dev/github.com/containerd/go-runc) [](https://github.com/containerd/go-runc/actions?query=workflow%3ACI) [](https://app.codecov.io/gh/containerd/go-runc/tree/main) This is a module for consuming the [runc](https://github.com/opencontainers/runc) binary in your Go applications. It tries to expose all the settings and features of the runc CLI. ## Project details **go-runc** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/log/README.md`) # log A Go package providing a common logging interface across containerd repositories and a way for clients to use and configure logging in containerd packages. This package is not intended to be used as a standalone logging package outside of the containerd ecosystem and is intended as an interface wrapper around a logging implementation. In the future this package may be replaced with a common go logging interface. ## Project details **log** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/otelttrpc/README.md`) # ttrpc OpenTelemetry Instrumentation This golang package implements OpenTelemetry instrumentation support for ttrpc. It can be used to automatically generate OpenTelemetry trace spans for RPC methods called on the ttrpc client side and served on the ttrpc server side. # Usage Instrumentation is provided by two interceptors, one to enable instrumentation for unary clients and another for enabling instrumentation for unary servers. These interceptors can be passed as ttrpc.ClientOpts and ttrpc.ServerOpt to ttrpc during client and server creation with code like this: ```golang import ( "github.com/containerd/ttrpc" "github.com/containerd/otelttrpc" ) // on the client side ... client := ttrpc.NewClient( conn, ttrpc.UnaryClientInterceptor( otelttrpc.UnaryClientInterceptor(), ), ) // and on the server side ... server, err := ttrpc.NewServer( ttrpc.WithUnaryServerInterceptor( otelttrpc.UnaryServerInterceptor(), ), ) ``` Once enabled, the interceptors generate trace Spans for all called and served unary method calls. If the rest of the code is properly set up to collect and export tracing data to opentelemetry, these spans should show up as part of the collected traces. For a more complete example see the [sample client](example/client/main.go) and the [sample server](example/server/main.go) code. # Limitations Currently only unary client and unary server methods can be instrumented. Support for streaming interfaces is yet to be implemented. # Project details otelttrpc is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/platforms/README.md`) # platforms A Go package for formatting, normalizing and matching container platforms. This package is based on the Open Containers Image Spec definition of a [platform](https://github.com/opencontainers/image-spec/blob/main/specs-go/v1/descriptor.go#L52). ## Platform Specifier While the OCI platform specifications provide a tool for components to specify structured information, user input typically doesn't need the full context and much can be inferred. To solve this problem, this package introduces "specifiers". A specifier has the format `||/[/]`. The user can provide either the operating system or the architecture or both. An example of a common specifier is `linux/amd64`. If the host has a default runtime that matches this, the user can simply provide the component that matters. For example, if an image provides `amd64` and `arm64` support, the operating system, `linux` can be inferred, so they only have to provide `arm64` or `amd64`. Similar behavior is implemented for operating systems, where the architecture may be known but a runtime may support images from different operating systems. ## Project details **platforms** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/plugin/README.md`) # plugin A Go package providing a common plugin interface across containerd repositories. This package is intended to be imported by the main containerd repository as well as plugin implementations. By sharing a common implementations, plugins can register themselves without needing to import the main containerd repository. This plugin is intended to provide an interface and common functionality, but is not intended to define plugin types used by containerd. Plugins should copy plugin type strings to avoid creating unintended depdenencies. ## Project details **plugin** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/containerd/ttrpc/README.md`) # ttrpc [](https://github.com/containerd/ttrpc/actions/workflows/ci.yml) GRPC for low-memory environments. The existing grpc-go project requires a lot of memory overhead for importing packages and at runtime. While this is great for many services with low density requirements, this can be a problem when running a large number of services on a single machine or on a machine with a small amount of memory. Using the same GRPC definitions, this project reduces the binary size and protocol overhead required. We do this by eliding the `net/http`, `net/http2` and `grpc` package used by grpc replacing it with a lightweight framing protocol. The result are smaller binaries that use less resident memory with the same ease of use as GRPC. Please note that while this project supports generating either end of the protocol, the generated service definitions will be incompatible with regular GRPC services, as they do not speak the same protocol. # Protocol See the [protocol specification](./PROTOCOL.md). # Usage It's recommended to use [`protobuild`](https://github.com/containerd/protobuild) to build the protobufs for this project, but this will work with protoc directly, if required. # Differences from GRPC - The protocol stack has been replaced with a lighter protocol that doesn't require http, http2 and tls. - The client and server interface are identical whereas in GRPC there is a client and server interface that are different. - The Go stdlib context package is used instead. # Status TODO: - [ ] Add testing under concurrent load to ensure - [ ] Verify connection error handling # Project details ttrpc is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE). As a containerd sub-project, you will find the: * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md), * [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS), * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md) information in our [`containerd/project`](https://github.com/containerd/project) repository. --- ### Overview (`vendor/github.com/cpuguy83/tar2go/README.md`) # tar2go tar2go implements are go [fs.FS](https://pkg.go.dev/io/fs#FS) for tar files. Tars are not indexed so by themselves don't really have support for random access. When a request to open/stat a file is made tar2go will scan through the tar, indexing each entry along the way, until the file is found in the tar. A tar file is only ever scanned 1 time and scanning is done lazily (as needed to index the requested entry). tar2go does not support modifying a tar file, however there is support for modifying the in-memory representation of the tar which will show up in the `fs.FS`. You can also write a new tar file with requested modifications. ### Usage ```go f, _ := os.Open(p) defer f.Close() // Entrypoint into this library idx := NewIndex(f) // Get the `fs.FS` implementation goFS := idx.FS() // Do stuff with your fs // ... // Add or replace a file in the index _ := idx.Replace("foo", strings.NewReader("random stuff") data, _ := fs.ReadFile(goFS, "foo") if string(data) != "random stuff") { panic("unexpected data") } // Delete a file in the index _ := idx.Replace("foo", nil) if _, err := fs.ReadFile(goFS, "foo"); !errors.Is(err, fs.ErrNotExist) { panic(err) } // Create a new tar with updated content // First we need to create an `io.Writer`, which is where the updated tar stream will be written to. f, _ := os.CreateTemp("", "updated") idx.Update(f, func(name string, rdr ReaderAtSized) (ReaderAtSized, bool, error) { // Update calls this function for every file in the tar // The returned `ReaderAtSized` is used instead of the content passed in (rdr). // To make no changes just return the same rdr back. // Return true for the bool value if the content is changed. }) ``` --- ### Overview (`vendor/github.com/creack/pty/README.md`) # pty Pty is a Go package for using unix pseudo-terminals. ## Install ```sh go get github.com/creack/pty ``` ## Examples Note that those examples are for demonstration purpose only, to showcase how to use the library. They are not meant to be used in any kind of production environment. If you want to **set deadlines to work** and `Close()` **interrupting** `Read()` on the returned `*os.File`, you will need to call `syscall.SetNonblock` manually. ### Command ```go package main import ( "io" "os" "os/exec" "github.com/creack/pty" ) func main() { c := exec.Command("grep", "--color=auto", "bar") f, err := pty.Start(c) if err != nil { panic(err) } go func() { f.Write([]byte("foo\n")) f.Write([]byte("bar\n")) f.Write([]byte("baz\n")) f.Write([]byte{4}) // EOT }() io.Copy(os.Stdout, f) } ``` ### Shell ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ### Overview (`vendor/github.com/cyphar/filepath-securejoin/README.md`) ## `filepath-securejoin` ## [](https://pkg.go.dev/github.com/cyphar/filepath-securejoin) [](https://github.com/cyphar/filepath-securejoin/actions/workflows/ci.yml) ### Old API ### This library was originally just an implementation of `SecureJoin` which was [intended to be included in the Go standard library][go#20126] as a safer `filepath.Join` that would restrict the path lookup to be inside a root directory. The implementation was based on code that existed in several container runtimes. Unfortunately, this API is **fundamentally unsafe** against attackers that can modify path components after `SecureJoin` returns and before the caller uses the path, allowing for some fairly trivial TOCTOU attacks. `SecureJoin` (and `SecureJoinVFS`) are still provided by this library to support legacy users, but new users are strongly suggested to avoid using `SecureJoin` and instead use the [new api](#new-api) or switch to [libpathrs][libpathrs]. With the above limitations in mind, this library guarantees the following: * If no error is set, the resulting string **must** be a child path of `root` and will not contain any symlink path components (they will all be expanded). * When expanding symlinks, all symlink path components **must** be resolved relative to the provided root. In particular, this can be considered a userspace implementation of how `chroot(2)` operates on file paths. Note that these symlinks will **not** be expanded lexically (`filepath.Clean` is not called on the input before processing). * Non-existent path components are unaffected by `SecureJoin` (similar to `filepath.EvalSymlinks`'s semantics). * The returned path will always be `filepath.Clean`ed and thus not contain any `..` components. A (trivial) implementation of this function on GNU/Linux systems could be done with the following (note that this requires root privileges and is far more opaque than the implementation in this library, and also requires that `readlink` is inside the `root` path and is trustworthy): ```go package securejoin import ( "os/exec" "path/filepath" ) func SecureJoin(root, unsafePath string) (string, error) { unsafePath = string(filepath.Separator) + unsafePath cmd := exec.Command("chroot", root, "readlink", "--canonicalize-missing", "--no-newline", unsafePath) output, err := cmd.CombinedOutput() if err != nil { return "", err } expanded := string(output) return filepath.Join(root, expanded), nil } ``` [libpathrs]: https://github.com/openSUSE/libpathrs [go#20126]: https://github.com/golang/go/issues/20126 ### New API ### [#new-api]: #new-api While we recommend users switch to [libpathrs][libpathrs] as soon as it has a stable release, some methods implemented by libpathrs have been ported to this library to ease the transition. These APIs are only supported on Linux. These APIs are implemented such that `filepath-securejoin` will opportunistically use certain newer kernel APIs that make these operations far more secure. In particular: * All of the lookup operations will use [`openat2`][openat2.2] on new enough kernels (Linux 5.6 or later) to restrict lookups through magic-links and bind-mounts (for certain operations) and to make use of `RESOLVE_IN_ROOT` to efficiently resolve symlinks within a rootfs. * The APIs provide hardening against a malicious `/proc` mount to either detect or avoid being tricked by a `/proc` that is not legitimate. This is done using [`openat2`][openat2.2] for all users, and privileged users will also be further protected by using [`fsopen`][fsopen.2] and [`open_tree`][open_tree.2] (Linux 5.2 or later). [openat2.2]: https://www.man7.org/linux/man-pages/man2/openat2.2.html [fsopen.2]: https://github.com/brauner/man-pages-md/blob/main/fsopen.md [open_tree.2]: https://github.com/brauner/man-pages-md/blob/main/open_tree.md #### `OpenInRoot` #### ```go func OpenInRoot(root, unsafePath string) (*os.File, error) func OpenatInRoot(root *os.File, unsafePath string) (*os.File, error) func Reopen(handle *os.File, flags int) (*os.File, error) ``` `OpenInRoot` is a much safer version of ```go path, err := securejoin.SecureJoin(root, unsafePath) file, err := os.OpenFile(path, unix.O_PATH|unix.O_CLOEXEC) ``` that protects against various race attacks that could lead to serious security issues, depending on the application. Note that the returned `*os.File` is an `O_PATH` file descriptor, which is quite restricted. Callers will probably need to use `Reopen` to get a more usable handle (this split is done to provide useful features like PTY spawning and to avoid users accidentally opening bad inodes that could cause a DoS). Callers need to be careful in how they use the returned `*os.File`. Usually it is only safe to operate on the handle directly, and it is very easy to create a security issue. [libpathrs][libpathrs] provides far more helpers to make using these handles safer -- there is currently no plan to port them to `filepath-securejoin`. `OpenatInRoot` is like `OpenInRoot` except that the root is provided using an `*os.File`. This allows you to ensure that multiple `OpenatInRoot` (or `MkdirAllHandle`) calls are operating on the same rootfs. > **NOTE**: Unlike `SecureJoin`, `OpenInRoot` will error out as soon as it hits > a dangling symlink or non-existent path. This is in contrast to `SecureJoin` > which treated non-existent components as though they were real directories, > and would allow for partial resolution of dangling symlinks. These behaviours > are at odds with how Linux treats non-existent paths and dangling symlinks, > and so these are no longer allowed. #### `MkdirAll` #### ```go func MkdirAll(root, unsafePath string, mode int) error func MkdirAllHandle(root *os.File, unsafePath string, mode int) (*os.File, error) ``` `MkdirAll` is a much safer version of ```go path, err := securejoin.SecureJoin(root, unsafePath) err = os.MkdirAll(path, mode) ``` that protects against the same kinds of races that `OpenInRoot` protects against. `MkdirAllHandle` is like `MkdirAll` except that the root is provided using an `*os.File` (the reason for this is the same as with `OpenatInRoot`) and an `*os.File` of the final created directory is returned (this directory is guaranteed to be effectively identical to the directory created by `MkdirAllHandle`, which is not possible to ensure by just using `OpenatInRoot` after `MkdirAll`). > **NOTE**: Unlike `SecureJoin`, `MkdirAll` will error out as soon as it hits > a dangling symlink or non-existent path. This is in contrast to `SecureJoin` > which treated non-existent components as though they were real directories, > and would allow for partial resolution of dangling symlinks. These behaviours > are at odds with how Linux treats non-existent paths and dangling symlinks, > and so these are no longer allowed. This means that `MkdirAll` will not > create non-existent directories referenced by a dangling symlink. ### License ### `SPDX-License-Identifier: BSD-3-Clause AND MPL-2.0` Some of the code in this project is derived from Go, and is licensed under a BSD 3-clause license (available in `LICENSE.BSD`). Other files (many of which are derived from [libpathrs][libpathrs]) are licensed under the Mozilla Public License version 2.0 (available in `LICENSE.MPL-2.0`). If you are using the ["New API" described above][#new-api], you are probably using code from files released under this license. Every source file in this project has a copyright header describing its license. Please check the license headers of each file to see what license applies to it. See [COPYING.md](./COPYING.md) for some more details. [umoci]: https://github.com/opencontainers/umoci --- ### Overview (`vendor/github.com/digitorus/pkcs7/README.md`) # pkcs7 [](https://godoc.org/go.mozilla.org/pkcs7) [](https://github.com/mozilla-services/pkcs7/actions/workflows/ci.yml?query=branch%3Amaster+event%3Apush) pkcs7 implements parsing and creating signed and enveloped messages. ```go package main import ( "bytes" "crypto/rsa" "crypto/x509" "encoding/pem" "fmt" "os" "go.mozilla.org/pkcs7" ) func SignAndDetach(content []byte, cert *x509.Certificate, privkey *rsa.PrivateKey) (signed []byte, err error) { toBeSigned, err := NewSignedData(content) if err != nil { err = fmt.Errorf("Cannot initialize signed data: %s", err) return } if err = toBeSigned.AddSigner(cert, privkey, SignerInfoConfig{}); err != nil { err = fmt.Errorf("Cannot add signer: %s", err) return } // Detach signature, omit if you want an embedded signature toBeSigned.Detach() signed, err = toBeSigned.Finish() if err != nil { err = fmt.Errorf("Cannot finish signing data: %s", err) return } // Verify the signature pem.Encode(os.Stdout, &pem.Block{Type: "PKCS7", Bytes: signed}) p7, err := pkcs7.Parse(signed) if err != nil { err = fmt.Errorf("Cannot parse our signed data: %s", err) return } // since the signature was detached, reattach the content here p7.Content = content if bytes.Compare(content, p7.Content) != 0 { err = fmt.Errorf("Our content was not in the parsed data:\n\tExpected: %s\n\tActual: %s", content, p7.Content) return } if err = p7.Verify(); err != nil { err = fmt.Errorf("Cannot verify our signed data: %s", err) return } return signed, nil } ``` ## Credits This is a fork of [fullsailor/pkcs7](https://github.com/fullsailor/pkcs7) --- ### Overview (`vendor/github.com/digitorus/timestamp/README.md`) # RFC3161 Time-Stamp Protocol (TSP) package for Go [](https://github.com/digitorus/timestamp/actions?query=workflow%3Abuild-and-test) [](https://github.com/digitorus/timestamp/actions?query=workflow%3Agolangci-lint) [](https://github.com/digitorus/timestamp/actions?query=workflow%3Acodeql) [](https://goreportcard.com/report/github.com/digitorus/timestamp) [](https://codecov.io/gh/digitorus/timestamp) [](https://pkg.go.dev/github.com/digitorus/timestamp) Time-Stamp Protocol (TSP) package for Go #### General The package timestamp implements the Time-Stamp Protocol (TSP) as specified in RFC3161 (Internet X.509 Public Key Infrastructure Time-Stamp Protocol (TSP)). --- ### Overview (`vendor/github.com/dimchansky/utfbom/README.md`) # utfbom [](https://godoc.org/github.com/dimchansky/utfbom) [](https://opensource.org/licenses/Apache-2.0) [](https://travis-ci.org/dimchansky/utfbom) [](https://goreportcard.com/report/github.com/dimchansky/utfbom) [](https://coveralls.io/github/dimchansky/utfbom?branch=master) The package utfbom implements the detection of the BOM (Unicode Byte Order Mark) and removing as necessary. It can also return the encoding detected by the BOM. ## Installation go get -u github.com/dimchansky/utfbom ## Example ```go package main import ( "bytes" "fmt" "io/ioutil" "github.com/dimchansky/utfbom" ) func main() { trySkip([]byte("\xEF\xBB\xBFhello")) trySkip([]byte("hello")) } func trySkip(byteData []byte) { fmt.Println("Input:", byteData) // just skip BOM output, err := ioutil.ReadAll(utfbom.SkipOnly(bytes.NewReader(byteData))) if err != nil { fmt.Println(err) return } fmt.Println("ReadAll with BOM skipping", output) // skip BOM and detect encoding sr, enc := utfbom.Skip(bytes.NewReader(byteData)) fmt.Printf("Detected encoding: %s\n", enc) output, err = ioutil.ReadAll(sr) if err != nil { fmt.Println(err) return } fmt.Println("ReadAll with BOM detection and skipping", output) fmt.Println() } ``` Output: ``` $ go run main.go Input: [239 187 191 104 101 108 108 111] ReadAll with BOM skipping [104 101 108 108 111] Detected encoding: UTF8 ReadAll with BOM detection and skipping [104 101 108 108 111] Input: [104 101 108 108 111] ReadAll with BOM skipping [104 101 108 108 111] Detected encoding: Unknown ReadAll with BOM detection and skipping [104 101 108 108 111] ``` --- ### Overview (`vendor/github.com/distribution/reference/README.md`) # Distribution reference Go library to handle references to container images. [](https://github.com/distribution/reference/actions?query=workflow%3ACI) [](https://pkg.go.dev/github.com/distribution/reference) [](LICENSE) [](https://codecov.io/gh/distribution/reference) [](https://app.fossa.com/projects/custom%2B162%2Fgithub.com%2Fdistribution%2Freference?ref=badge_shield) This repository contains a library for handling references to container images held in container registries. Please see [godoc](https://pkg.go.dev/github.com/distribution/reference) for details. ## Contribution Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute issues, fixes, and patches to this project. ## Communication For async communication and long running discussions please use issues and pull requests on the github repo. This will be the best place to discuss design and implementation. For sync communication we have a #distribution channel in the [CNCF Slack](https://slack.cncf.io/) that everyone is welcome to join and chat about development. ## Licenses The distribution codebase is released under the [Apache 2.0 license](LICENSE). --- ### Overview (`vendor/github.com/docker/distribution/README.md`) # Distribution The Docker toolset to pack, ship, store, and deliver content. This repository provides the Docker Registry 2.0 implementation for storing and distributing Docker images. It supersedes the [docker/docker-registry](https://github.com/docker/docker-registry) project with a new API design, focused around security and performance. [](https://circleci.com/gh/docker/distribution/tree/master) [](https://godoc.org/github.com/docker/distribution) This repository contains the following components: |**Component** |Description | |--------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **registry** | An implementation of the [Docker Registry HTTP API V2](docs/spec/api.md) for use with docker 1.6+. | | **libraries** | A rich set of libraries for interacting with distribution components. Please see [godoc](https://godoc.org/github.com/docker/distribution) for details. **Note**: These libraries are **unstable**. | | **specifications** | _Distribution_ related specifications are available in [docs/spec](docs/spec) | | **documentation** | Docker's full documentation set is available at [docs.docker.com](https://docs.docker.com). This repository [contains the subset](docs/) related just to the registry. | ### How does this integrate with Docker engine? This project should provide an implementation to a V2 API for use in the [Docker core project](https://github.com/docker/docker). The API should be embeddable and simplify the process of securely pulling and pushing content from `docker` daemons. ### What are the long term goals of the Distribution project? The _Distribution_ project has the further long term goal of providing a secure tool chain for distributing content. The specifications, APIs and tools should be as useful with Docker as they are without. Our goal is to design a professional grade and extensible content distribution system that allow users to: * Enjoy an efficient, secured and reliable way to store, manage, package and exchange content * Hack/roll their own on top of healthy open-source components * Implement their own home made solution through good specs, and solid extensions mechanism. ## More about Registry 2.0 The new registry implementation provides the following benefits: - faster push and pull - new, more efficient implementation - simplified deployment - pluggable storage backend - webhook notifications For information on upcoming functionality, please see [ROADMAP.md](ROADMAP.md). ### Who needs to deploy a registry? By default, Docker users pull images from Docker's public registry instance. [Installing Docker](https://docs.docker.com/engine/installation/) gives users this ability. Users can also push images to a repository on Docker's public registry, if they have a [Docker Hub](https://hub.docker.com/) account. For some users and even companies, this default behavior is sufficient. For others, it is not. For example, users with their own software products may want to maintain a registry for private, company images. Also, you may wish to deploy your own image repository for images used to test or in continuous integration. For these use cases and others, [deploying your own registry instance](https://github.com/docker/docker.github.io/blob/master/registry/deploying.md) may be the better choice. ### Migration to Registry 2.0 For those who have previously deployed their own registry based on the Registry 1.0 implementation and wish to deploy a Registry 2.0 while retaining images, data migration is required. A tool to assist with migration efforts has been created. For more information see [docker/migrator](https://github.com/docker/migrator). ## Contribute Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute issues, fixes, and patches to this project. If you are contributing code, see the instructions for [building a development environment](BUILDING.md). ## Support If any issues are encountered while using the _Distribution_ project, several avenues are available for support:
IRC #docker-distribution on FreeNode
Issue Tracker github.com/docker/distribution/issues
Google Groups https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution
Mailing List docker@dockerproject.org
## License This project is distributed under [Apache License, Version 2.0](LICENSE). --- ### Overview (`vendor/github.com/docker/go-events/README.md`) # Docker Events Package [](https://pkg.go.dev/github.com/docker/go-events) [](https://github.com/docker/go-events/actions/workflows/ci.yml) The Docker `events` package implements a composable event distribution package for Go. Originally created to implement the [notifications in Docker Registry 2](https://github.com/docker/distribution/blob/master/docs/notifications.md), we've found the pattern to be useful in other applications. This package is most of the same code with slightly updated interfaces. Much of the internals have been made available. ## Usage The `events` package centers around a `Sink` type. Events are written with calls to `Sink.Write(event Event)`. Sinks can be wired up in various configurations to achieve interesting behavior. The canonical example is that employed by the [docker/distribution/notifications](https://godoc.org/github.com/docker/distribution/notifications) package. Let's say we have a type `httpSink` where we'd like to queue notifications. As a rule, it should send a single http request and return an error if it fails: ```go func (h *httpSink) Write(event Event) error { p, err := json.Marshal(event) if err != nil { return err } body := bytes.NewReader(p) resp, err := h.client.Post(h.url, "application/json", body) if err != nil { return err } defer resp.Body.Close() if resp.Status != 200 { return errors.New("unexpected status") } return nil } // implement (*httpSink).Close() ``` With just that, we can start using components from this package. One can call `(*httpSink).Write` to send events as the body of a post request to a configured URL. ### Retries HTTP can be unreliable. The first feature we'd like is to have some retry: ```go hs := newHTTPSink(/*...*/) retry := NewRetryingSink(hs, NewBreaker(5, time.Second)) ``` We now have a sink that will retry events against the `httpSink` until they succeed. The retry will backoff for one second after 5 consecutive failures using the breaker strategy. ### Queues This isn't quite enough. We we want a sink that doesn't block while we are waiting for events to be sent. Let's add a `Queue`: ```go queue := NewQueue(retry) ``` Now, we have an unbounded queue that will work through all events sent with `(*Queue).Write`. Events can be added asynchronously to the queue without blocking the current execution path. This is ideal for use in an http request. ### Broadcast It usually turns out that you want to send to more than one listener. We can use `Broadcaster` to support this: ```go var broadcast = NewBroadcaster() // make it available somewhere in your application. broadcast.Add(queue) // add your queue! broadcast.Add(queue2) // and another! ``` With the above, we can now call `broadcast.Write` in our http handlers and have all the events distributed to each queue. Because the events are queued, not listener blocks another. ### Extending For the most part, the above is sufficient for a lot of applications. However, extending the above functionality can be done implementing your own `Sink`. The behavior and semantics of the sink can be completely dependent on the application requirements. The interface is provided below for reference: ```go type Sink { Write(Event) error Close() error } ``` Application behavior can be controlled by how `Write` behaves. The examples above are designed to queue the message and return as quickly as possible. Other implementations may block until the event is committed to durable storage. ## Copyright and license Copyright © 2016 Docker, Inc. go-events is licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full license text. --- ### Overview (`vendor/github.com/docker/go-metrics/README.md`) # go-metrics [](https://pkg.go.dev/github.com/docker/go-metrics) This package is small wrapper around the prometheus go client to help enforce convention and best practices for metrics collection in Docker projects. ## Best Practices This packages is meant to be used for collecting metrics. It is not meant to be used as a replacement for the prometheus client but to help enforce consistent naming across metrics collected. If you have not already read the prometheus best practices around naming and labels you can read the naming section in the [prometheus documentation](https://prometheus.io/docs/practices/naming/). The following are a few conventions that help naming metrics in your project. ### Namespace and Subsystem This package provides you with a namespace type that allows you to specify the same namespace and subsystem for your metrics. ```go ns := metrics.NewNamespace("engine", "daemon", metrics.Labels{ "version": daemon.Version, "commit": daemon.GitCommit, }) ``` In the example above we are creating metrics for the daemon package. `engine` would be the namespace in this example where `daemon` is the subsystem or package where we are collecting the metrics. A namespace also allows you to attach constant labels to the metrics such as the git commit and version that it is collecting. ### Declaring your Metrics Try to keep all your metric declarations in one file. This makes it easy for others to see what constant labels are defined on the namespace and what labels are defined on the metrics when they are created. ### Use labels instead of multiple metrics Labels allow you to define one metric such as the time it takes to perform a certain action on an object. If we wanted to collect timings on various container actions such as create, start, and delete then we can define one metric called `container_actions` and use labels to specify the type of action. ```go containerActions = ns.NewLabeledTimer("container_actions", "The number of milliseconds it takes to process each container action", "action") ``` The last parameter is the label name or key. When adding a data point to the metric you can use the `WithValues` function to specify the `action` that you are collecting for. ```go containerActions.WithValues("create").UpdateSince(start) ``` ### Always use a unit The metric name should describe what you are measuring but you also need to provide the unit that it is being measured with. For a timer, the standard unit is seconds and a counter's standard unit is a total. For gauges you must provide the unit. This package provides a standard set of units for use within your projects. ```go Nanoseconds Unit = "nanoseconds" Seconds Unit = "seconds" Bytes Unit = "bytes" Total Unit = "total" ``` If you need to use a unit but it is not defined in the package please open a PR to add it but first try to see if one of the existing units work for your metric, i.e., seconds or nanoseconds vs adding milliseconds. ## HTTP Metrics To instrument a http handler, you can wrap the code like this: ```go namespace := metrics.NewNamespace("api_server", "http", metrics.Labels{"handler": "your_http_handler_name"}) httpMetrics := namespace.NewDefaultHttpMetrics() metrics.Register(namespace) instrumentedHandler = metrics.InstrumentHandler(httpMetrics, unInstrumentedHandler) ``` Note: The `handler` label must be provided when a new namespace is created. ## Additional Metrics Additional metrics are also defined here that are not available in the prometheus client. If you need a custom metrics and it is generic enough to be used by multiple projects, define it here. ## Copyright and license Copyright © 2016 Docker, Inc. All rights reserved, except as follows. Code is released under the Apache 2.0 license. --- ### Overview (`vendor/github.com/docker/go-units/README.md`) [](https://godoc.org/github.com/docker/go-units) # Introduction go-units is a library to transform human friendly measurements into machine friendly values. ## Usage See the [docs in godoc](https://godoc.org/github.com/docker/go-units) for examples and documentation. ## Copyright and license Copyright © 2015 Docker, Inc. go-units is licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full text of the license. --- ### Overview (`vendor/github.com/felixge/httpsnoop/README.md`) # httpsnoop Package httpsnoop provides an easy way to capture http related metrics (i.e. response time, bytes written, and http status code) from your application's http.Handlers. Doing this requires non-trivial wrapping of the http.ResponseWriter interface, which is also exposed for users interested in a more low-level API. [](https://pkg.go.dev/github.com/felixge/httpsnoop) [](https://github.com/felixge/httpsnoop/actions/workflows/main.yaml) ## Usage Example ```go // myH is your app's http handler, perhaps a http.ServeMux or similar. var myH http.Handler // wrappedH wraps myH in order to log every request. wrappedH := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { m := httpsnoop.CaptureMetrics(myH, w, r) log.Printf( "%s %s (code=%d dt=%s written=%d)", r.Method, r.URL, m.Code, m.Duration, m.Written, ) }) http.ListenAndServe(":8080", wrappedH) ``` ## Why this package exists Instrumenting an application's http.Handler is surprisingly difficult. However if you google for e.g. "capture ResponseWriter status code" you'll find lots of advise and code examples that suggest it to be a fairly trivial undertaking. Unfortunately everything I've seen so far has a high chance of breaking your application. The main problem is that a `http.ResponseWriter` often implements additional interfaces such as `http.Flusher`, `http.CloseNotifier`, `http.Hijacker`, `http.Pusher`, and `io.ReaderFrom`. So the naive approach of just wrapping `http.ResponseWriter` in your own struct that also implements the `http.ResponseWriter` interface will hide the additional interfaces mentioned above. This has a high change of introducing subtle bugs into any non-trivial application. Another approach I've seen people take is to return a struct that implements all of the interfaces above. However, that's also problematic, because it's difficult to fake some of these interfaces behaviors when the underlying `http.ResponseWriter` doesn't have an implementation. It's also dangerous, because an application may choose to operate differently, merely because it detects the presence of these additional interfaces. This package solves this problem by checking which additional interfaces a `http.ResponseWriter` implements, returning a wrapped version implementing the exact same set of interfaces. Additionally this package properly handles edge cases such as `WriteHeader` not being called, or called more than once, as well as concurrent calls to `http.ResponseWriter` methods, and even calls happening after the wrapped `ServeHTTP` has already returned. Unfortunately this package is not perfect either. It's possible that it is still missing some interfaces provided by the go core (let me know if you find one), and it won't work for applications adding their own interfaces into the mix. You can however use `httpsnoop.Unwrap(w)` to access the underlying `http.ResponseWriter` and type-assert the result to its other interfaces. However, hopefully the explanation above has sufficiently scared you of rolling your own solution to this problem. httpsnoop may still break your application, but at least it tries to avoid it as much as possible. Anyway, the real problem here is that smuggling additional interfaces inside `http.ResponseWriter` is a problematic design choice, but it probably goes as deep as the Go language specification itself. But that's okay, I still prefer Go over the alternatives ;). ## Performance ``` BenchmarkBaseline-8 20000 94912 ns/op BenchmarkCaptureMetrics-8 20000 95461 ns/op ``` As you can see, using `CaptureMetrics` on a vanilla http.Handler introduces an overhead of ~500 ns per http request on my machine. However, the margin of error appears to be larger than that, therefor it should be reasonable to assume that the overhead introduced by `CaptureMetrics` is absolutely negligible. ## License MIT --- ### Overview (`vendor/github.com/fsnotify/fsnotify/README.md`) fsnotify is a Go library to provide cross-platform filesystem notifications on Windows, Linux, macOS, BSD, and illumos. Go 1.23 or newer is required; the full documentation is at https://pkg.go.dev/github.com/fsnotify/fsnotify --- Platform support: | Backend | OS | Status | | :-------------------- | :--------- | :------------------------------------------------------------------------ | | inotify | Linux | Supported | | kqueue | BSD, macOS | Supported | | ReadDirectoryChangesW | Windows | Supported ([excluding `Chmod` operations][#487]) | | FEN | illumos | Supported | | fanotify | Linux 5.9+ | [Not yet](https://github.com/fsnotify/fsnotify/issues/114) | | FSEvents | macOS | [Needs support in x/sys/unix][fsevents] | | USN Journals | Windows | [Needs support in x/sys/windows][usn] | | Polling | *All* | [Not yet](https://github.com/fsnotify/fsnotify/issues/9) | Linux and illumos should include Android and Solaris, but these are currently untested. [#487]: https://github.com/fsnotify/fsnotify/issues/487 [fsevents]: https://github.com/fsnotify/fsnotify/issues/11#issuecomment-1279133120 [usn]: https://github.com/fsnotify/fsnotify/issues/53#issuecomment-1279829847 Usage ----- A basic example: ```go package main import ( "log" "github.com/fsnotify/fsnotify" ) func main() { // Create new watcher. watcher, err := fsnotify.NewWatcher() if err != nil { log.Fatal(err) } defer watcher.Close() // Start listening for events. go func() { for { select { case event, ok := <-watcher.Events: if !ok { return } log.Println("event:", event) if event.Has(fsnotify.Write) { log.Println("modified file:", event.Name) } case err, ok := <-watcher.Errors: if !ok { return } log.Println("error:", err) } } }() // Add a path. err = watcher.Add("/tmp") if err != nil { log.Fatal(err) } // Block main goroutine forever. <-make(chan struct{}) } ``` Some more examples can be found in [cmd/fsnotify](cmd/fsnotify), which can be run with: % go run ./cmd/fsnotify Further detailed documentation can be found in godoc: https://pkg.go.dev/github.com/fsnotify/fsnotify FAQ --- ### Will a file still be watched when it's moved to another directory? No, not unless you are watching the location it was moved to. ### Are subdirectories watched? No, you must add watches for any directory you want to watch (a recursive watcher is on the roadmap: [#18]). [#18]: https://github.com/fsnotify/fsnotify/issues/18 ### Do I have to watch the Error and Event channels in a goroutine? Yes. You can read both channels in the same goroutine using `select` (you don't need a separate goroutine for both channels; see the example). ### Why don't notifications work with NFS, SMB, FUSE, /proc, or /sys? fsnotify requires support from underlying OS to work. The current NFS and SMB protocols does not provide network level support for file notifications, and neither do the /proc and /sys virtual filesystems. This could be fixed with a polling watcher ([#9]), but it's not yet implemented. [#9]: https://github.com/fsnotify/fsnotify/issues/9 ### Why do I get many Chmod events? Some programs may generate a lot of attribute changes; for example Spotlight on macOS, anti-virus programs, backup applications, and some others are known to do this. As a rule, it's typically best to ignore Chmod events. They're often not useful, and tend to cause problems. Spotlight indexing on macOS can result in multiple events (see [#15]). A temporary workaround is to add your folder(s) to the *Spotlight Privacy settings* until we have a native FSEvents implementation (see [#11]). [#11]: https://github.com/fsnotify/fsnotify/issues/11 [#15]: https://github.com/fsnotify/fsnotify/issues/15 ### Watching a file doesn't work well Watching individual files (rather than directories) is generally not recommended as many programs (especially editors) update files atomically: it will write to a temporary file which is then moved to a destination, overwriting the original (or some variant thereof). The watcher on the original file is now lost, as that no longer exists. The upshot of this is that a power failure or crash won't leave a half-written file. Watch the parent directory and use `Event.Name` to filter out files you're not interested in. There is an example of this in `cmd/fsnotify/file.go`. Platform-specific notes ----------------------- ### Linux When a file is removed a REMOVE event won't be emitted until all file descriptors are closed; it will emit a CHMOD instead: fp := os.Open("file") os.Remove("file") // CHMOD fp.Close() // REMOVE This is the event that inotify sends, so not much can be changed about this. The `fs.inotify.max_user_watches` sysctl variable specifies the upper limit for the number of watches per user, and `fs.inotify.max_user_instances` specifies the maximum number of inotify instances per user. Every Watcher you create is an "instance", and every path you add is a "watch". Reaching the limit will result in a "no space left on device" or "too many open files" error. These are also exposed in `/proc` as `/proc/sys/fs/inotify/max_user_watches` and `/proc/sys/fs/inotify/max_user_instances`. The default values differ per distro and available memory. To increase them you can use `sysctl` or write the value to proc file: sysctl fs.inotify.max_user_watches=200000 sysctl fs.inotify.max_user_instances=256 To make the changes persist on reboot edit `/etc/sysctl.conf` or `/usr/lib/sysctl.d/50-default.conf` (details differ per Linux distro; check your distro's documentation): fs.inotify.max_user_watches=200000 fs.inotify.max_user_instances=256 ### Windows Recursive watching is not currently enabled through fsnotify's public API (see the FAQ "Are subdirectories watched?" above). The notes below describe Windows backend behavior observed when recursive watching is enabled internally (for example, in fsnotify's own tests). They are kept here as a reference for maintainers and contributors who encounter the behavior, since the recursive code path still exists in the backend. When recursive watching is enabled and you watch a directory, you may receive a `Write` event for an intermediate directory whenever a child entry inside it is created, renamed, or removed. For example, with a recursive watch on `/a` and a new file `/a/b/c`, you will receive `Create /a/b/c` and may also receive `Write /a/b`. This happens because, on NTFS-backed volumes, modifying the entries of a directory updates that directory's last-write time, and the Windows backend requests `FILE_NOTIFY_CHANGE_LAST_WRITE` to support `Write` events on files. The same `Write` filter therefore picks up the directory's metadata update. kqueue has the same "directory `Write` = directory contents changed" semantics, so portable code that treats `Write` on a directory as "something inside it changed" works on Windows and BSD/macOS, but not on Linux (inotify uses `Write` only for file-content changes). If you only care about file content, filter out `Write` events whose path refers to a directory. Whether the directory `Write` is actually delivered alongside the child events is not guaranteed: it depends on `ReadDirectoryChangesW` buffering, NTFS metadata update timing, and event coalescing, none of which fsnotify controls. ### kqueue (macOS, all BSD systems) kqueue requires opening a file descriptor for every file that's being watched; so if you're watching a directory with five files then that's six file descriptors. You will run in to your system's "max open files" limit faster on these platforms. The sysctl variables `kern.maxfiles` and `kern.maxfilesperproc` can be used to control the maximum number of open files. --- ### Overview (`vendor/github.com/go-logr/logr/README.md`) # A minimal logging API for Go [](https://pkg.go.dev/github.com/go-logr/logr) [](https://goreportcard.com/report/github.com/go-logr/logr) [](https://securityscorecards.dev/viewer/?platform=github.com&org=go-logr&repo=logr) logr offers an(other) opinion on how Go programs and libraries can do logging without becoming coupled to a particular logging implementation. This is not an implementation of logging - it is an API. In fact it is two APIs with two different sets of users. The `Logger` type is intended for application and library authors. It provides a relatively small API which can be used everywhere you want to emit logs. It defers the actual act of writing logs (to files, to stdout, or whatever) to the `LogSink` interface. The `LogSink` interface is intended for logging library implementers. It is a pure interface which can be implemented by logging frameworks to provide the actual logging functionality. This decoupling allows application and library developers to write code in terms of `logr.Logger` (which has very low dependency fan-out) while the implementation of logging is managed "up stack" (e.g. in or near `main()`.) Application developers can then switch out implementations as necessary. Many people assert that libraries should not be logging, and as such efforts like this are pointless. Those people are welcome to convince the authors of the tens-of-thousands of libraries that *DO* write logs that they are all wrong. In the meantime, logr takes a more practical approach. ## Typical usage Somewhere, early in an application's life, it will make a decision about which logging library (implementation) it actually wants to use. Something like: ``` func main() { // ... other setup code ... // Create the "root" logger. We have chosen the "logimpl" implementation, // which takes some initial parameters and returns a logr.Logger. logger := logimpl.New(param1, param2) // ... other setup code ... ``` Most apps will call into other libraries, create structures to govern the flow, etc. The `logr.Logger` object can be passed to these other libraries, stored in structs, or even used as a package-global variable, if needed. For example: ``` app := createTheAppObject(logger) app.Run() ``` Outside of this early setup, no other packages need to know about the choice of implementation. They write logs in terms of the `logr.Logger` that they received: ``` type appObject struct { // ... other fields ... logger logr.Logger // ... other fields ... } func (app *appObject) Run() { app.logger.Info("starting up", "timestamp", time.Now()) // ... app code ... ``` ## Background If the Go standard library had defined an interface for logging, this project probably would not be needed. Alas, here we are. When the Go developers started developing such an interface with [slog](https://github.com/golang/go/issues/56345), they adopted some of the logr design but also left out some parts and changed others: | Feature | logr | slog | |---------|------|------| | High-level API | `Logger` (passed by value) | `Logger` (passed by [pointer](https://github.com/golang/go/issues/59126)) | | Low-level API | `LogSink` | `Handler` | | Stack unwinding | done by `LogSink` | done by `Logger` | | Skipping helper functions | `WithCallDepth`, `WithCallStackHelper` | [not supported by Logger](https://github.com/golang/go/issues/59145) | | Generating a value for logging on demand | `Marshaler` | `LogValuer` | | Log levels | >= 0, higher meaning "less important" | positive and negative, with 0 for "info" and higher meaning "more important" | | Error log entries | always logged, don't have a verbosity level | normal log entries with level >= `LevelError` | | Passing logger via context | `NewContext`, `FromContext` | no API | | Adding a name to a logger | `WithName` | no API | | Modify verbosity of log entries in a call chain | `V` | no API | | Grouping of key/value pairs | not supported | `WithGroup`, `GroupValue` | | Pass context for extracting additional values | no API | API variants like `InfoCtx` | The high-level slog API is explicitly meant to be one of many different APIs that can be layered on top of a shared `slog.Handler`. logr is one such alternative API, with [interoperability](#slog-interoperability) provided by some conversion functions. ### Inspiration Before you consider this package, please read [this blog post by the inimitable Dave Cheney][warning-makes-no-sense]. We really appreciate what he has to say, and it largely aligns with our own experiences. ### Differences from Dave's ideas The main differences are: 1. Dave basically proposes doing away with the notion of a logging API in favor of `fmt.Printf()`. We disagree, especially when you consider things like output locations, timestamps, file and line decorations, and structured logging. This package restricts the logging API to just 2 types of logs: info and error. Info logs are things you want to tell the user which are not errors. Error logs are, well, errors. If your code receives an `error` from a subordinate function call and is logging that `error` *and not returning it*, use error logs. 2. Verbosity-levels on info logs. This gives developers a chance to indicate arbitrary grades of importance for info logs, without assigning names with semantic meaning such as "warning", "trace", and "debug." Superficially this may feel very similar, but the primary difference is the lack of semantics. Because verbosity is a numerical value, it's safe to assume that an app running with higher verbosity means more (and less important) logs will be generated. ## Implementations (non-exhaustive) There are implementations for the following logging libraries: - **a function** (can bridge to non-structured libraries): [funcr](https://github.com/go-logr/logr/tree/master/funcr) - **a testing.T** (for use in Go tests, with JSON-like output): [testr](https://github.com/go-logr/logr/tree/master/testr) - **github.com/google/glog**: [glogr](https://github.com/go-logr/glogr) - **k8s.io/klog** (for Kubernetes): [klogr](https://git.k8s.io/klog/klogr) - **a testing.T** (with klog-like text output): [ktesting](https://git.k8s.io/klog/ktesting) - **go.uber.org/zap**: [zapr](https://github.com/go-logr/zapr) - **log** (the Go standard library logger): [stdr](https://github.com/go-logr/stdr) - **github.com/sirupsen/logrus**: [logrusr](https://github.com/bombsimon/logrusr) - **github.com/wojas/genericr**: [genericr](https://github.com/wojas/genericr) (makes it easy to implement your own backend) - **logfmt** (Heroku style [logging](https://www.brandur.org/logfmt)): [logfmtr](https://github.com/iand/logfmtr) - **github.com/rs/zerolog**: [zerologr](https://github.com/go-logr/zerologr) - **github.com/go-kit/log**: [gokitlogr](https://github.com/tonglil/gokitlogr) (also compatible with github.com/go-kit/kit/log since v0.12.0) - **bytes.Buffer** (writing to a buffer): [bufrlogr](https://github.com/tonglil/buflogr) (useful for ensuring values were logged, like during testing) ## slog interoperability Interoperability goes both ways, using the `logr.Logger` API with a `slog.Handler` and using the `slog.Logger` API with a `logr.LogSink`. `FromSlogHandler` and `ToSlogHandler` convert between a `logr.Logger` and a `slog.Handler`. As usual, `slog.New` can be used to wrap such a `slog.Handler` in the high-level slog API. ### Using a `logr.LogSink` as backend for slog Ideally, a logr sink implementation should support both logr and slog by implementing both the normal logr interface(s) and `SlogSink`. Because of a conflict in the parameters of the common `Enabled` method, it is [not possible to implement both slog.Handler and logr.Sink in the same type](https://github.com/golang/go/issues/59110). If both are supported, log calls can go from the high-level APIs to the backend without the need to convert parameters. `FromSlogHandler` and `ToSlogHandler` can convert back and forth without adding additional wrappers, with one exception: when `Logger.V` was used to adjust the verbosity for a `slog.Handler`, then `ToSlogHandler` has to use a wrapper which adjusts the verbosity for future log calls. Such an implementation should also support values that implement specific interfaces from both packages for logging (`logr.Marshaler`, `slog.LogValuer`, `slog.GroupValue`). logr does not convert those. Not supporting slog has several drawbacks: - Recording source code locations works correctly if the handler gets called through `slog.Logger`, but may be wrong in other cases. That's because a `logr.Sink` does its own stack unwinding instead of using the program counter provided by the high-level API. - slog levels <= 0 can be mapped to logr levels by negating the level without a loss of information. But all slog levels > 0 (e.g. `slog.LevelWarning` as used by `slog.Logger.Warn`) must be mapped to 0 before calling the sink because logr does not support "more important than info" levels. - The slog group concept is supported by prefixing each key in a key/value pair with the group names, separated by a dot. For structured output like JSON it would be better to group the key/value pairs inside an object. - Special slog values and interfaces don't work as expected. - The overhead is likely to be higher. These drawbacks are severe enough that applications using a mixture of slog and logr should switch to a different backend. ### Using a `slog.Handler` as backend for logr Using a plain `slog.Handler` without support for logr works better than the other direction: - All logr verbosity levels can be mapped 1:1 to their corresponding slog level by negating them. - Stack unwinding is done by the `SlogSink` and the resulting program counter is passed to the `slog.Handler`. - Names added via `Logger.WithName` are gathered and recorded in an additional attribute with `logger` as key and the names separated by slash as value. - `Logger.Error` is turned into a log record with `slog.LevelError` as level and an additional attribute with `err` as key, if an error was provided. The main drawback is that `logr.Marshaler` will not be supported. Types should ideally support both `logr.Marshaler` and `slog.Valuer`. If compatibility with logr implementations without slog support is not important, then `slog.Valuer` is sufficient. ### Context support for slog Storing a logger in a `context.Context` is not supported by slog. `NewContextWithSlogLogger` and `FromContextAsSlogLogger` can be used to fill this gap. They store and retrieve a `slog.Logger` pointer under the same context key that is also used by `NewContext` and `FromContext` for `logr.Logger` value. When `NewContextWithSlogLogger` is followed by `FromContext`, the latter will automatically convert the `slog.Logger` to a `logr.Logger`. `FromContextAsSlogLogger` does the same for the other direction. With this approach, binaries which use either slog or logr are as efficient as possible with no unnecessary allocations. This is also why the API stores a `slog.Logger` pointer: when storing a `slog.Handler`, creating a `slog.Logger` on retrieval would need to allocate one. The downside is that switching back and forth needs more allocations. Because logr is the API that is already in use by different packages, in particular Kubernetes, the recommendation is to use the `logr.Logger` API in code which uses contextual logging. An alternative to adding values to a logger and storing that logger in the context is to store the values in the context and to configure a logging backend to extract those values when emitting log entries. This only works when log calls are passed the context, which is not supported by the logr API. With the slog API, it is possible, but not required. https://github.com/veqryn/slog-context is a package for slog which provides additional support code for this approach. It also contains wrappers for the context functions in logr, so developers who prefer to not use the logr APIs directly can use those instead and the resulting code will still be interoperable with logr. ## FAQ ### Conceptual #### Why structured logging? - **Structured logs are more easily queryable**: Since you've got key-value pairs, it's much easier to query your structured logs for particular values by filtering on the contents of a particular key -- think searching request logs for error codes, Kubernetes reconcilers for the name and namespace of the reconciled object, etc. - **Structured logging makes it easier to have cross-referenceable logs**: Similarly to searchability, if you maintain conventions around your keys, it becomes easy to gather all log lines related to a particular concept. - **Structured logs allow better dimensions of filtering**: if you have structure to your logs, you've got more precise control over how much information is logged -- you might choose in a particular configuration to log certain keys but not others, only log lines where a certain key matches a certain value, etc., instead of just having v-levels and names to key off of. - **Structured logs better represent structured data**: sometimes, the data that you want to log is inherently structured (think tuple-link objects.) Structured logs allow you to preserve that structure when outputting. #### Why V-levels? **V-levels give operators an easy way to control the chattiness of log operations**. V-levels provide a way for a given package to distinguish the relative importance or verbosity of a given log message. Then, if a particular logger or package is logging too many messages, the user of the package can simply change the v-levels for that library. #### Why not named levels, like Info/Warning/Error? Read [Dave Cheney's post][warning-makes-no-sense]. Then read [Differences from Dave's ideas](#differences-from-daves-ideas). #### Why not allow format strings, too? **Format strings negate many of the benefits of structured logs**: - They're not easily searchable without resorting to fuzzy searching, regular expressions, etc. - They don't store structured data well, since contents are flattened into a string. - They're not cross-referenceable. - They don't compress easily, since the message is not constant. (Unless you turn positional parameters into key-value pairs with numerical keys, at which point you've gotten key-value logging with meaningless keys.) ### Practical #### Why key-value pairs, and not a map? Key-value pairs are *much* easier to optimize, especially around allocations. Zap (a structured logger that inspired logr's interface) has [performance measurements](https://github.com/uber-go/zap#performance) that show this quite nicely. While the interface ends up being a little less obvious, you get potentially better performance, plus avoid making users type `map[string]string{}` every time they want to log. #### What if my V-levels differ between libraries? That's fine. Control your V-levels on a per-logger basis, and use the `WithName` method to pass different loggers to different libraries. Generally, you should take care to ensure that you have relatively consistent V-levels within a given logger, however, as this makes deciding on what verbosity of logs to request easier. #### But I really want to use a format string! That's not actually a question. Assuming your question is "how do I convert my mental model of logging with format strings to logging with constant messages": 1. Figure out what the error actually is, as you'd write in a TL;DR style, and use that as a message. 2. For every place you'd write a format specifier, look to the word before it, and add that as a key value pair. For instance, consider the following examples (all taken from spots in the Kubernetes codebase): - `klog.V(4).Infof("Client is returning errors: code %v, error %v", responseCode, err)` becomes `logger.Error(err, "client returned an error", "code", responseCode)` - `klog.V(4).Infof("Got a Retry-After %ds response for attempt %d to %v", seconds, retries, url)` becomes `logger.V(4).Info("got a retry-after response when requesting url", "attempt", retries, "after seconds", seconds, "url", url)` If you *really* must use a format string, use it in a key's value, and call `fmt.Sprintf` yourself. For instance: `log.Printf("unable to reflect over type %T")` becomes `logger.Info("unable to reflect over type", "type", fmt.Sprintf("%T"))`. In general though, the cases where this is necessary should be few and far between. #### How do I choose my V-levels? This is basically the only hard constraint: increase V-levels to denote more verbose or more debug-y logs. Otherwise, you can start out with `0` as "you always want to see this", `1` as "common logging that you might *possibly* want to turn off", and `10` as "I would like to performance-test your log collection stack." Then gradually choose levels in between as you need them, working your way down from 10 (for debug and trace style logs) and up from 1 (for chattier info-type logs). For reference, slog pre-defines -4 for debug logs (corresponds to 4 in logr), which matches what is [recommended for Kubernetes](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md#what-method-to-use). #### How do I choose my keys? Keys are fairly flexible, and can hold more or less any string value. For best compatibility with implementations and consistency with existing code in other projects, there are a few conventions you should consider. - Make your keys human-readable. - Constant keys are generally a good idea. - Be consistent across your codebase. - Keys should naturally match parts of the message string. - Use lower case for simple keys and [lowerCamelCase](https://en.wiktionary.org/wiki/lowerCamelCase) for more complex ones. Kubernetes is one example of a project that has [adopted that convention](https://github.com/kubernetes/community/blob/HEAD/contributors/devel/sig-instrumentation/migration-to-structured-logging.md#name-arguments). While key names are mostly unrestricted (and spaces are acceptable), it's generally a good idea to stick to printable ascii characters, or at least match the general character set of your log lines. #### Why should keys be constant values? The point of structured logging is to make later log processing easier. Your keys are, effectively, the schema of each log message. If you use different keys across instances of the same log line, you will make your structured logs much harder to use. `Sprintf()` is for values, not for keys! #### Why is this not a pure interface? The Logger type is implemented as a struct in order to allow the Go compiler to optimize things like high-V `Info` logs that are not triggered. Not all of these implementations are implemented yet, but this structure was suggested as a way to ensure they *can* be implemented. All of the real work is behind the `LogSink` interface. [warning-makes-no-sense]: http://dave.cheney.net/2015/11/05/lets-talk-about-logging --- ### Overview (`vendor/github.com/go-logr/stdr/README.md`) # Minimal Go logging using logr and Go's standard library [](https://pkg.go.dev/github.com/go-logr/stdr) This package implements the [logr interface](https://github.com/go-logr/logr) in terms of Go's standard log package(https://pkg.go.dev/log). --- ### Overview (`vendor/github.com/go-openapi/analysis/README.md`) # analysis [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- A foundational library to analyze, diff, flatten, merge, and fix OAI specification documents for easier reasoning about the content. ## Announcements * **2025-12-19** : new community chat on discord * a new discord community channel is available to be notified of changes and support users You may join the discord community by clicking the invite link on the discord badge (also above). [![Discord Channel][discord-badge]][discord-url] ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/analysis ``` ## What's inside * An analyzer providing methods to walk the functional content of a specification * A spec flattener producing a self-contained document bundle, while preserving `$ref`s * A spec differ ("diff") to compare two specs and report structural and compatibility changes * A spec merger ("mixin") to merge several spec documents into a primary spec * A spec "fixer" ensuring that response descriptions are non empty ## FAQ * Does this library support OpenAPI 3? > No. > This package currently only supports OpenAPI 2.0 (aka Swagger 2.0). > There is no plan to make it evolve toward supporting OpenAPI 3.x. > This [discussion thread](https://github.com/go-openapi/spec/issues/21) relates the full story. ## Change log See ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/analysis/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/analysis/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/analysis/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/analysis/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/analysis [vuln-scan-badge]: https://github.com/go-openapi/analysis/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/analysis/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/analysis/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/analysis/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fanalysis.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fanalysis [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/analysis [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/analysis [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/analysis [godoc-url]: http://pkg.go.dev/github.com/go-openapi/analysis [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/analysis/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/analysis [goversion-url]: https://github.com/go-openapi/analysis/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/analysis [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/analysis/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/errors/README.md`) # errors [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- Shared errors and error interface used throughout the various libraries found in the go-openapi toolkit. ## Announcements * **2025-12-19** : new community chat on discord * a new discord community channel is available to be notified of changes and support users You may join the discord community by clicking the invite link on the discord badge (also above). [![Discord Channel][discord-badge]][discord-url] ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/errors ``` ## Basic usage ```go const url = "https://www.example.com/#" errGeneric := New(401,"onvalid argument: %s", url) errNotFound := NotFound("resource not found: %s", url) errNotImplemented := NotImplemented("method: %s", url) ``` ## Change log See ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/errors/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/errors/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/errors/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/errors/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/errors [vuln-scan-badge]: https://github.com/go-openapi/errors/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/errors/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/errors/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/errors/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Ferrors.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Ferrors [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/errors [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/errors [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/errors [godoc-url]: http://pkg.go.dev/github.com/go-openapi/errors [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/errors/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/errors [goversion-url]: https://github.com/go-openapi/errors/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/errors [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/errors/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/jsonpointer/README.md`) # jsonpointer [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- An implementation of JSON Pointer for golang, which supports go `struct`. ## Announcements * **2026-07-07** : landing v1.0.0 * stable API pledge * **2026-06-29** : reinsourced external dependency to swag (v0.24.0) * module `github.com/go-openapi/swag/jsonname` is source directly here, so we no longer have any external dependency * `jsonname` was never really used by any other package, so it makes sense to deprecate it away from the `swag` family and retrofit its functionality here. `jsonpointer` no longer get external dependencies, besides test dependencies. * **2026-04-15** : added support for trailing "-" for arrays (v0.23.0) * this brings full support of [RFC6901][RFC6901] * this is supported for types relying on the reflection-based implemented * API semantics remain essentially unaltered. Exception: `Pointer.Set(document any,value any) (document any, err error)` can only perform a best-effort to mutate the input document in place. In the case of adding elements to an array with a trailing "-", either pass a mutable array (`*[]T`) as the input document, or use the returned updated document instead. * types that implement the `JSONSetable` interface may not implement the mutation implied by the trailing "-" * **2026-04-15** : added support for optional alternate JSON name providers * for struct support the defaults might not suit all situations: there are known limitations when it comes to handle untagged fields or embedded types. * the default name provider in use is not fully aligned with go JSON stdlib * exposed an option (or global setting) to change the provider that resolves a struct into json keys * the default behavior is not altered ## Status API is stable and feature-complete. The project continues to receive regular updates, bug fixes and hygiene maintenance (CI, linting, etc). ## Import this library in your project ```cmd go get github.com/go-openapi/jsonpointer ``` ## Basic usage See also some [examples](./examples_test.go) ### Retrieving a value ```go import ( "github.com/go-openapi/jsonpointer" ) var doc any ... pointer, err := jsonpointer.New("/foo/1") if err != nil { ... // error: e.g. invalid JSON pointer specification } value, kind, err := pointer.Get(doc) if err != nil { ... // error: e.g. key not found, index out of bounds, etc. } ... ``` ### Setting a value ```go ... var doc any ... pointer, err := jsonpointer.New("/foo/1") if err != nil { ... // error: e.g. invalid JSON pointer specification } doc, err = p.Set(doc, "value") if err != nil { ... // error: e.g. key not found, index out of bounds, etc. } ``` ## Change log See ## References also known as [RFC6901][RFC6901]. ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). See the license [NOTICE](./NOTICE), which recalls the licensing terms of all the pieces of software on top of which it has been built. ## Limitations * [RFC6901][RFC6901] is now fully supported, including trailing "-" semantics for arrays (for `Set` operations). * Default behavior: JSON name detection in go `struct`s - Unlike go standard marshaling, untagged fields do not default to the go field name and are ignored. - anonymous fields are not traversed if untagged - the above limitations may be overcome by calling `UseGoNameProvider()` at initialization time. - alternatively, users may inject the desired custom behavior for naming fields as an option. ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/jsonpointer/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/jsonpointer/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/jsonpointer/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/jsonpointer/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/jsonpointer [vuln-scan-badge]: https://github.com/go-openapi/jsonpointer/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/jsonpointer/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/jsonpointer/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/jsonpointer/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fjsonpointer.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fjsonpointer [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/jsonpointer [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/jsonpointer [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/jsonpointer [godoc-url]: http://pkg.go.dev/github.com/go-openapi/jsonpointer [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/jsonpointer/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/jsonpointer [goversion-url]: https://github.com/go-openapi/jsonpointer/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/jsonpointer [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/jsonpointer/latest [RFC6901]: https://www.rfc-editor.org/rfc/rfc6901 [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/jsonreference/README.md`) # jsonreference [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- An implementation of JSON Reference for golang. ## Announcements * **2026-07-07** : landing v1.0.0 * stable API pledge ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/jsonreference ``` ## Dependencies * ## Basic usage ```go // Creating a new reference ref, err := jsonreference.New("http://example.com/doc.json#/definitions/Pet") // Fragment-only reference fragRef := jsonreference.MustCreateRef("#/definitions/Pet") // Resolving references parent, _ := jsonreference.New("http://example.com/base.json") child, _ := jsonreference.New("#/definitions/Pet") resolved, _ := parent.Inherits(child) // Result: "http://example.com/base.json#/definitions/Pet" ``` ## Change log See ## References * * ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). See the license [NOTICE](./NOTICE), which recalls the licensing terms of all the pieces of software on top of which it has been built. ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/jsonreference/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/jsonreference/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/jsonreference/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/jsonreference/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/jsonreference [vuln-scan-badge]: https://github.com/go-openapi/jsonreference/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/jsonreference/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/jsonreference/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/jsonreference/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fjsonreference.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fjsonreference [gomod-badge]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fjsonreference.svg [gomod-url]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fjsonreference [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/jsonreference [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/jsonreference [doc-badge]: https://img.shields.io/badge/doc-site-blue?link=https%3A%2F%2Fgoswagger.io%2Fgo-openapi%2F [doc-url]: https://goswagger.io/go-openapi [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/jsonreference [godoc-url]: http://pkg.go.dev/github.com/go-openapi/jsonreference [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/jsonreference/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/jsonreference [goversion-url]: https://github.com/go-openapi/jsonreference/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/jsonreference [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/jsonreference/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/loads/README.md`) # Loads OAI specs [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- Loads OAI v2 API specification documents from local or remote locations. Supports JSON and YAML documents. ## Announcements * **2025-12-19** : new community chat on discord * a new discord community channel is available to be notified of changes and support users You may join the discord community by clicking the invite link on the discord badge (also above). [![Discord Channel][discord-badge]][discord-url] ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/loads ``` ## Basic usage ```go import ( "github.com/go-openapi/loads" ) ... // loads a YAML spec from a http file doc, err := loads.Spec(ts.URL) ... // retrieves the object model for the API specification spec := doc.Spec() ... ``` See also the provided [examples](https://pkg.go.dev/github.com/go-openapi/loads#pkg-examples). ## Security This library does not enforce a security policy of its own: it reads whatever the configured loader is allowed to read. This is deliberate — like `go-openapi/swag/loading`, it is a base utility, and sanitizing or containing untrusted input is the caller's responsibility, just as sanitizing a file name before passing it to `os.ReadFile` is not that function's job. When a spec — its path or its `$ref` contents — may come from an untrusted source, confine loading explicitly (e.g. `loading.WithRoot` for local files and a restricted `loading.WithHTTPClient` for remote URLs, passed via `loads.WithLoadingOptions`). For the common case, the pre-baked `loads.SpecRestricted` / `loads.JSONSpecRestricted` loaders bundle a trusted root with a network-restricted client (`loads.RestrictedHTTPClient`) and apply the confinement to `$ref` resolution as well: ```go doc, err := loads.SpecRestricted(path, trustedRoot) ``` To harden the package-level default in one call — so even callers that rely on the global loader (including cross-package `$ref` resolution via `spec.PathLoader`) are confined, with no unconfined fallback left — use `loads.SetRestrictedLoaders` at startup: ```go loads.SetRestrictedLoaders(trustedRoot) ``` Note that `loads.AddLoader` only *prepends* to the default chain, leaving the unconfined loader reachable; use `loads.SetLoaders` / `loads.SetRestrictedLoaders` to replace it. See the [Security section of the package documentation][security-doc] for the threat model and runnable examples. For the project's vulnerability reporting policy, see [SECURITY.md](./SECURITY.md). ## Change log See ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/loads/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/loads/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/loads/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/loads/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/loads [vuln-scan-badge]: https://github.com/go-openapi/loads/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/loads/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/loads/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/loads/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Floads.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Floads [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/loads [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/loads [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/loads [godoc-url]: http://pkg.go.dev/github.com/go-openapi/loads [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/loads/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/loads [goversion-url]: https://github.com/go-openapi/loads/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/loads [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/loads/latest [security-doc]: https://pkg.go.dev/github.com/go-openapi/loads#hdr-Security [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/runtime/README.md`) # runtime [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![Doc][doc-badge]][doc-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- A runtime for go OpenAPI toolkit. The runtime component for use in code generation or as untyped usage. ## Announcements [**Complete documentation as github pages**][doc-url] **Changes to the API surface in `v0.30.0`**: * utility package `header` has now moved to `github.com/go-openapi/runtime/server-middleware/negotiate/header` > A shim is provided to support existing programs, with a deprecation notice. **Changes in semantics in `v0.30.0`**: Function `negotiate.NegotiateContentType` (available as an alias for backward compatibility as `middleware.NegotiateContentType` now performs a full match considering MIME parameters. The previous behavior (matching in order of appearance after stripping parameters) may be enabled explicitly with option `negotiate.WithIgnoreParameters`. * **2026-05-07** : exposed UI and Spec middleware as a separate, dependency-free module. > Newly available package: `github.com/go-openapi/runtime/server-middleware/docui` that now holds our > UI and spec serve middleware. > > A shim is available in `github.com/go-openapi/runtime/middleware` to bridge the older UI options to the new ones, > with a deprecation notice. > > Methods that were unduly exported and purely used to manipulate options (e.g. `SwaggerUIOpts.EnsureDefaults`) have been > removed. New options in `docui` should be used instead. > Users may reuse this middleware to serve a Redoc, Rapidoc or SwaggerUI documentation without > importing the complete go-openapi scaffolding. * **2026-05-05** : exposed content negotiation methods as a separate, dependency-free module > Users may reuse these utilities to support content-negotiation without extra dependencies. > > Newly available module: `github.com/go-openapi/runtime/server-middleware` > > Newly available packages: `github.com/go-openapi/runtime/server-middleware/negotiate` and > `github.com/go-openapi/runtime/server-middleware/mediatype`. ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/runtime ``` ## Change log See For v0.29.0 release see [release notes](docs/NOTES.md). From that release onwards, changes are tracked in the github release notes. **What coming next?** Moving forward, we want to : * [x] fix a few known issues with some file upload requests (e.g. #286) * [] continue narrowing down the scope of dependencies: * [x] split middleware and other useful utilities as a separate dependency-free module * yaml support in an independent module (v2) * introduce more up-to-date support for opentelemetry as a separate module that evolves independently from the main package (to avoid breaking changes, the existing API will remain maintained, but evolve at a slower pace than opentelemetry). (v2) * [] publish proper documentation and examples ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). See the license [NOTICE](./NOTICE), which recalls the licensing terms of all the pieces of software on top of which it has been built. ## Other documentation * [FAQ](https://go-openapi.github.io/runtime/tutorials/faq/) · [Media-type selection](https://go-openapi.github.io/runtime/tutorials/media-types/) · [Client keep-alive](https://go-openapi.github.io/runtime/tutorials/keep-alive/) * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/runtime/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/runtime/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/runtime/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/runtime/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/runtime [vuln-scan-badge]: https://github.com/go-openapi/runtime/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/runtime/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/runtime/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/runtime/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fruntime.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fruntime [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/runtime [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/runtime [doc-badge]: https://img.shields.io/badge/doc-site-blue?link=https%3A%2F%2Fgo-openapi.github.io%2Fruntime%2F [doc-url]: https://go-openapi.github.io/runtime [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/runtime [godoc-url]: http://pkg.go.dev/github.com/go-openapi/runtime [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/runtime/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/runtime [goversion-url]: https://github.com/go-openapi/runtime/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/runtime [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/runtime/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/spec/README.md`) # spec [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- The object model for OpenAPI v2 specification documents. ## Announcements * **2025-12-19** : new community chat on discord * a new discord community channel is available to be notified of changes and support users You may join the discord community by clicking the invite link on the discord badge (also above). [![Discord Channel][discord-badge]][discord-url] ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/spec ``` ### FAQ * What does this do? > 1. This package knows how to marshal and unmarshal Swagger API specifications into a golang object model > 2. It knows how to resolve $ref and expand them to make a single root document * How does it play with the rest of the go-openapi packages ? > 1. This package is at the core of the go-openapi suite of packages and [code generator](https://github.com/go-swagger/go-swagger) > 2. There is a [spec loading package](https://github.com/go-openapi/loads) to fetch specs as JSON or YAML from local or remote locations > 3. There is a [spec validation package](https://github.com/go-openapi/validate) built on top of it > 4. There is a [spec analysis package](https://github.com/go-openapi/analysis) built on top of it, to analyze, flatten, fix and merge spec documents * Does this library support OpenAPI 3? > No. > This package currently only supports OpenAPI 2.0 (aka Swagger 2.0). > There is no plan to make it evolve toward supporting OpenAPI 3.x. > This [discussion thread](https://github.com/go-openapi/spec/issues/21) relates the full story. > > An early attempt to support Swagger 3 may be found at: * Does the unmarshaling support YAML? > Not directly. The exposed types know only how to unmarshal from JSON. > > In order to load a YAML document as a Swagger spec, you need to use the loaders provided by > github.com/go-openapi/loads > > Take a look at the example there: > > See also * How can I validate a spec? Validation is provided by [the validate package](http://github.com/go-openapi/validate) * Why do we have an `ID` field for `Schema` which is not part of the swagger spec? > We found jsonschema compatibility more important: since `id` in jsonschema influences > how `$ref` are resolved. > This `id` does not conflict with any property named `id`. > > See also ## Change log See ## References ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/spec/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/spec/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/spec/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/spec/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/spec [vuln-scan-badge]: https://github.com/go-openapi/spec/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/spec/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/spec/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/spec/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fspec.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fspec [gomod-badge]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fspec.svg [gomod-url]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fspec [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/spec [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/spec [doc-badge]: https://img.shields.io/badge/doc-site-blue?link=https%3A%2F%2Fgoswagger.io%2Fgo-openapi%2F [doc-url]: https://goswagger.io/go-openapi [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/spec [godoc-url]: http://pkg.go.dev/github.com/go-openapi/spec [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/spec/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/spec [goversion-url]: https://github.com/go-openapi/spec/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/spec [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/spec/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/strfmt/README.md`) # strfmt [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- Golang support for string formats defined by JSON Schema and OpenAPI. ## Announcements * **2026-03-07** : v0.26.0 **dropped dependency to the mongodb driver** * mongodb users can still use this package without any change * however, we have frozen the back-compatible support for mongodb driver at v2.5.0 * users who want to keep-up with future evolutions (possibly incompatible) of this driver can do so by adding a blank import in their program: `import _ "github.com/go-openapi/strfmt/enable/mongodb"`. This will switch the behavior to the actual driver, which remains regularly updated as an independent module. ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/strfmt ``` ## Contents This package exposes a registry of data types to support string formats in the go-openapi toolkit. `strfmt` represents a well known string format such as hostname or email. This package provides a few extra formats such as credit card (US), color, etc. Format types can serialize and deserialize JSON or from a SQL database. BSON is also supported (MongoDB). ### Supported formats `go-openapi/strfmt` follows the swagger 2.0 specification with the following formats defined [here](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#data-types). It also provides convenient extensions to go-openapi users. - [x] JSON-schema draft 4 formats - date-time - email - hostname - ipv4 - ipv6 - uri - [x] swagger 2.0 format extensions - binary - byte (e.g. base64 encoded string) - date (e.g. "1970-01-01") - password - [x] go-openapi custom format extensions - bsonobjectid (BSON objectID) - creditcard - duration (e.g. "3 weeks", "1ms") (aka "duration-human") - hexcolor (e.g. "#FFFFFF") - isbn, isbn10, isbn13 - mac (e.g "01:02:03:04:05:06") - rgbcolor (e.g. "rgb(100,100,100)") - ssn - uuid, uuid3, uuid4, uuid5, uuid7 - cidr (e.g. "192.0.2.1/24", "2001:db8:a0b:12f0::1/32") - ulid (e.g. "00000PP9HGSBSSDZ1JTEXBJ0PW", [spec](https://github.com/ulid/spec)) - [x] JSON-schema draft 2020 formats - duration-iso8601 B(e.g. "P2W") > NOTE: as the name stands for, this package is intended to support string formatting only. > It does not provide validation for numerical values with swagger format extension for JSON types "number" or > "integer" (e.g. float, double, int32...). ## Durations We have 2 very different definitions of the "duration" format: the "human-readable" duration that used to be just "duration", and the new "duration-iso8601". There is no "dual" parser that accepts both formats: types are specialized. To clarify the situation, a new alias for the duration format is introduced "duration-human" (e.g. "1 ms"), as opposed to "duration-iso8601". The `Default` format registry wires "duration-human" as the default mapping for "duration" (preexisting behavior, no breaking change - aligned with Swagger 2.0 which did not define "duration"). A new `JSONSchema2020` registry wires "duration-iso8601" as the default mapping for "duration". ### Type conversion All types defined here are stringers and may be converted to strings with `.String()`. Note that most types defined by this package may be converted directly to string like `string(Email{})`. `Date` and `DateTime` may be converted directly to `time.Time` like `time.Time(Time{})`. Similarly, you can convert `Duration` to `time.Duration` as in `time.Duration(Duration{})` ### Using pointers The `conv` subpackage provides helpers to convert the types to and from pointers, just like `go-openapi/swag` does with primitive types. ### Format types Types defined in `strfmt` expose marshaling and validation capabilities. List of defined types: - Base64 - CreditCard - Date - DateTime - Duration - DurationISO8601 and `ISODuration[P ISODurationPolicy]` (for optional behavior) - Email - HexColor - Hostname - IPv4 - IPv6 - CIDR - ISBN - ISBN10 - ISBN13 - MAC - ObjectId - Password - RGBColor - SSN - URI - UUID - [UUID3](https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-3) - [UUID4](https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-4) - [UUID5](https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-5) - [UUID7](https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-7) - [ULID](https://github.com/ulid/spec) ### Database support All format types implement the `database/sql` interfaces `sql.Scanner` and `driver.Valuer`, so they work out of the box with Go's standard `database/sql` package and any SQL driver. All format types also implement BSON marshaling/unmarshaling for use with MongoDB. By default, a built-in minimal codec is used (compatible with mongo-driver v2.5.0). For full driver support, add `import _ "github.com/go-openapi/strfmt/enable/mongodb"`. > **MySQL / MariaDB caveat for `DateTime`:** > The `go-sql-driver/mysql` driver has hard-coded handling for `time.Time` but does not > intercept type redefinitions like `strfmt.DateTime`. As a result, `DateTime.Value()` sends > an RFC 3339 string (e.g. `"2024-06-15T12:30:45.123Z"`) that MySQL/MariaDB rejects for > `DATETIME` columns. > > Workaround: set `strfmt.MarshalFormat` to a MySQL-compatible format such as > `strfmt.ISO8601LocalTime` and normalize to UTC before marshaling: > > ```go > strfmt.MarshalFormat = strfmt.ISO8601LocalTime > strfmt.NormalizeTimeForMarshal = func(t time.Time) time.Time { return t.UTC() } > ``` > > See [#174](https://github.com/go-openapi/strfmt/issues/174) for details. Integration tests for MongoDB, MariaDB, and PostgreSQL run in CI to verify database roundtrip compatibility for all format types. See [`internal/testintegration/`](internal/testintegration/). ## Change log See ## References ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/strfmt/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/strfmt/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/strfmt/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/strfmt/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/strfmt [vuln-scan-badge]: https://github.com/go-openapi/strfmt/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/strfmt/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/strfmt/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/strfmt/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fstrfmt.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fstrfmt [gomod-badge]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fstrfmt.svg [gomod-url]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fstrfmt [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/strfmt [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/strfmt [doc-badge]: https://img.shields.io/badge/doc-site-blue?link=https%3A%2F%2Fgoswagger.io%2Fgo-openapi%2F [doc-url]: https://goswagger.io/go-openapi [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/strfmt [godoc-url]: http://pkg.go.dev/github.com/go-openapi/strfmt [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/strfmt/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/strfmt [goversion-url]: https://github.com/go-openapi/strfmt/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/strfmt [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/strfmt/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/swag/README.md`) # Swag [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- A bunch of helper functions for go-openapi and go-swagger projects. You may also use it standalone for your projects. > **NOTE** > `swag` is one of the foundational building blocks of the go-openapi initiative. > > Most repositories in `github.com/go-openapi/...` depend on it in some way. > And so does our CLI tool `github.com/go-swagger/go-swagger`, > as well as the code generated by this tool. * [Contents](#contents) * [Dependencies](#dependencies) * [Change log](#change-log) * [Licensing](#licensing) * [Note to contributors](#note-to-contributors) * [Roadmap](#roadmap) ## Announcements * **2025-12-19** : new community chat on discord * a new discord community channel is available to be notified of changes and support users You may join the discord community by clicking the invite link on the discord badge (also above). [![Discord Channel][discord-badge]][discord-url] ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/swag/{module} ``` Or for backward compatibility: ```cmd go get github.com/go-openapi/swag ``` ## Contents `go-openapi/swag` exposes a collection of relatively independent modules. Moving forward, no additional feature will be added to the `swag` API directly at the root package level, which remains there for backward-compatibility purposes. All exported top-level features are now deprecated. Child modules will continue to evolve and some new ones may be added in the future. | Module | Content | Main features | |---------------|---------|---------------| | `cmdutils` | utilities to work with CLIs || | `conv` | type conversion utilities | convert between values and pointers for any types
convert from string to builtin types (wraps `strconv`)
require `./typeutils` (test dependency)
| | `fileutils` | file utilities | | | `jsonname` | JSON utilities (deprecated) | infer JSON names from `go` properties
use `github.com/go-openapi/jsonpointer/jsonname` instead | | `jsonutils` | JSON utilities | fast json concatenation
read and write JSON from and to dynamic `go` data structures
~require `github.com/mailru/easyjson`~
| | `loading` | file loading | load from file or http
require `./yamlutils`
| | `mangling` | safe name generation | name mangling for `go`
| | `netutils` | networking utilities | host, port from address
| | `pools` | utilities to work with sync.Pools | | | `stringutils` | `string` utilities | search in slice (with case-insensitive)
split/join query parameters as arrays
| | `typeutils` | `go` types utilities | check the zero value for any type
safe check for a nil value
| | `yamlutils` | YAML utilities | converting YAML to JSON
loading YAML into a dynamic YAML document
maintaining the original order of keys in YAML objects
require `./jsonutils`
~require `github.com/mailru/easyjson`~
require `go.yaml.in/yaml/v3`
| --- ## Dependencies The root module `github.com/go-openapi/swag` at the repo level maintains a few dependencies outside of the standard library. * YAML utilities depend on `go.yaml.in/yaml/v3` * JSON utilities depend on their registered adapter module: * by default, only the standard library is used * `github.com/mailru/easyjson` is now only a dependency for module `github.com/go-openapi/swag/jsonutils/adapters/easyjson/json`, for users willing to import that module. * integration tests and benchmarks use all the dependencies are published as their own module * other dependencies are test dependencies drawn from `github.com/stretchr/testify` ## Usage **How to explicitly register a dependency at runtime**? The following would maintain how JSON utilities proposed by `swag` used work, up to `v0.24.1`. ```go import ( "github.com/go-openapi/swag/jsonutils/adapters" easyjson "github.com/go-openapi/swag/jsonutils/adapters/easyjson/json" ) func init() { easyjson.Register(adapters.Registry) } ``` Subsequent calls to `jsonutils.ReadJSON()` or `jsonutils.WriteJSON()` will switch to `easyjson` whenever the passed data structures implement the `easyjson.Unmarshaler` or `easyjson.Marshaler` respectively, or fallback to the standard library. For more details, you may also look at our [integration tests](jsonutils/adapters/testintegration/integration_suite_test.go#29). --- ## Note to contributors All kinds of contributions are welcome. This repo is a go mono-repo. See [docs](docs/MAINTAINERS.md). More general guidelines are available [here](.github/CONTRIBUTING.md). ## Roadmap See the current [TODO list](docs/TODOS.md) ## Change log See For pre-v0.26.0 releases, see [release notes](./docs/NOTES.md). **What coming next?** Moving forward, we want to : * provide an implementation of the JSON adapter based on `encoding/json/v2`, for `go1.25` builds. * provide similar implementations for `goccy/go-json` and `jsoniterator/go`, and perhaps some other similar libraries may be interesting too. ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/swag/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/swag/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/swag/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/swag/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/swag [vuln-scan-badge]: https://github.com/go-openapi/swag/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/swag/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/swag/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/swag/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fswag.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fswag [gomod-badge]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fswag.svg [gomod-url]: https://badge.fury.io/go/github.com%2Fgo-openapi%2Fswag [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/swag [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/swag [doc-badge]: https://img.shields.io/badge/doc-site-blue?link=https%3A%2F%2Fgoswagger.io%2Fgo-openapi%2F [doc-url]: https://goswagger.io/go-openapi [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/swag [godoc-url]: http://pkg.go.dev/github.com/go-openapi/swag [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/swag/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/swag [goversion-url]: https://github.com/go-openapi/swag/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/swag [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/swag/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/go-openapi/validate/README.md`) # validate [![Tests][test-badge]][test-url] [![Coverage][cov-badge]][cov-url] [![CI vuln scan][vuln-scan-badge]][vuln-scan-url] [![CodeQL][codeql-badge]][codeql-url] [![Release][release-badge]][release-url] [![CodeFactor Grade][codefactor-badge]][codefactor-url] [![License][license-badge]][license-url] [![GoDoc][godoc-badge]][godoc-url] [![Discord Channel][discord-badge]][discord-url] [![go version][goversion-badge]][goversion-url] ![Top language][top-badge] ![Commits since latest release][commits-badge] --- A validator for OpenAPI v2 specifications and JSON schema draft 4. ## Announcements * **2025-12-19** : new community chat on discord * a new discord community channel is available to be notified of changes and support users You may join the discord community by clicking the invite link on the discord badge (also above). [![Discord Channel][discord-badge]][discord-url] ## Status API is stable. ## Import this library in your project ```cmd go get github.com/go-openapi/validate ``` ## Contents This package provides helpers to validate Swagger 2.0. specification (aka OpenAPI 2.0). * A validator for Swagger specifications * A validator for JSON schemas draft4 * Helper functions to validate individual values (used by code generated by [go-swagger](https://github.com/go-swagger/go-swagger)). * Required, RequiredNumber, RequiredString * ReadOnly * UniqueItems, MaxItems, MinItems * Enum, EnumCase * Pattern, MinLength, MaxLength * Minimum, Maximum, MultipleOf * FormatOf ### FAQ * Does this library support OpenAPI 3? > No. > This package currently only supports OpenAPI 2.0 (aka Swagger 2.0). > There is no plan to make it evolve toward supporting OpenAPI 3.x. > This [discussion thread](https://github.com/go-openapi/spec/issues/21) relates the full story. > > An early attempt to support Swagger 3 may be found at: ## Change log See ## References ## Licensing This library ships under the [SPDX-License-Identifier: Apache-2.0](./LICENSE). ## Other documentation * [All-time contributors](./CONTRIBUTORS.md) * [Contributing guidelines][contributing-doc-site] * [Maintainers documentation][maintainers-doc-site] * [Code style][style-doc-site] ## Cutting a new release Maintainers can cut a new release by either: * running [this workflow](https://github.com/go-openapi/validate/actions/workflows/bump-release.yml) * or pushing a semver tag * signed tags are preferred * The tag message is prepended to release notes [test-badge]: https://github.com/go-openapi/validate/actions/workflows/go-test.yml/badge.svg [test-url]: https://github.com/go-openapi/validate/actions/workflows/go-test.yml [cov-badge]: https://codecov.io/gh/go-openapi/validate/branch/master/graph/badge.svg [cov-url]: https://codecov.io/gh/go-openapi/validate [vuln-scan-badge]: https://github.com/go-openapi/validate/actions/workflows/scanner.yml/badge.svg [vuln-scan-url]: https://github.com/go-openapi/validate/actions/workflows/scanner.yml [codeql-badge]: https://github.com/go-openapi/validate/actions/workflows/codeql.yml/badge.svg [codeql-url]: https://github.com/go-openapi/validate/actions/workflows/codeql.yml [release-badge]: https://badge.fury.io/gh/go-openapi%2Fvalidate.svg [release-url]: https://badge.fury.io/gh/go-openapi%2Fvalidate [codefactor-badge]: https://img.shields.io/codefactor/grade/github/go-openapi/validate [codefactor-url]: https://www.codefactor.io/repository/github/go-openapi/validate [godoc-badge]: https://pkg.go.dev/badge/github.com/go-openapi/validate [godoc-url]: http://pkg.go.dev/github.com/go-openapi/validate [discord-badge]: https://img.shields.io/discord/1446918742398341256?logo=discord&label=discord&color=blue [discord-url]: https://discord.gg/FfnFYaC3k5 [license-badge]: http://img.shields.io/badge/license-Apache%20v2-orange.svg [license-url]: https://github.com/go-openapi/validate/?tab=Apache-2.0-1-ov-file#readme [goversion-badge]: https://img.shields.io/github/go-mod/go-version/go-openapi/validate [goversion-url]: https://github.com/go-openapi/validate/blob/master/go.mod [top-badge]: https://img.shields.io/github/languages/top/go-openapi/validate [commits-badge]: https://img.shields.io/github/commits-since/go-openapi/validate/latest [contributing-doc-site]: https://go-openapi.github.io/doc-site/contributing/contributing/index.html [maintainers-doc-site]: https://go-openapi.github.io/doc-site/maintainers/index.html [style-doc-site]: https://go-openapi.github.io/doc-site/contributing/style/index.html --- ### Overview (`vendor/github.com/gofrs/flock/README.md`) # flock [](https://pkg.go.dev/github.com/gofrs/flock) [](https://github.com/gofrs/flock/blob/main/LICENSE) [](https://goreportcard.com/report/github.com/gofrs/flock) `flock` implements a thread-safe file lock. It also includes a non-blocking `TryLock()` function to allow locking without blocking execution. ## Installation ```bash go get -u github.com/gofrs/flock ``` ## Usage ```go import "github.com/gofrs/flock" fileLock := flock.New("/var/lock/go-lock.lock") locked, err := fileLock.TryLock() if err != nil { // handle locking error } if locked { // do work fileLock.Unlock() } ``` For more detailed usage information take a look at the package API docs on [GoDoc](https://pkg.go.dev/github.com/gofrs/flock). ## License `flock` is released under the BSD 3-Clause License. See the [`LICENSE`](./LICENSE) file for more details. ## Project History This project was originally `github.com/theckman/go-flock`, it was transferred to Gofrs by the original author [Tim Heckman ](https://github.com/theckman). --- ### Overview (`vendor/github.com/gohugoio/hashstructure/README.md`) # hashstructure [](https://godoc.org/github.com/mitchellh/hashstructure) hashstructure is a Go library for creating a unique hash value for arbitrary values in Go. This can be used to key values in a hash (for use in a map, set, etc.) that are complex. The most common use case is comparing two values without sending data across the network, caching values locally (de-dup), and so on. ## Features * Hash any arbitrary Go value, including complex types. * Tag a struct field to ignore it and not affect the hash value. * Tag a slice type struct field to treat it as a set where ordering doesn't affect the hash code but the field itself is still taken into account to create the hash value. * Optionally, specify a custom hash function to optimize for speed, collision avoidance for your data set, etc. * Optionally, hash the output of `.String()` on structs that implement fmt.Stringer, allowing effective hashing of time.Time * Optionally, override the hashing process by implementing `Hashable`. ## Installation Standard `go get`: ``` $ go get github.com/gohugoio/hashstructure ``` ## Usage & Example For usage and examples see the [Godoc](http://godoc.org/github.com/mitchellh/hashstructure). A quick code example is shown below: ```go type ComplexStruct struct { Name string Age uint Metadata map[string]interface{} } v := ComplexStruct{ Name: "mitchellh", Age: 64, Metadata: map[string]interface{}{ "car": true, "location": "California", "siblings": []string{"Bob", "John"}, }, } hash, err := hashstructure.Hash(v, nil) if err != nil { panic(err) } fmt.Printf("%d", hash) // Output: // 2307517237273902113 ``` --- ### Overview (`vendor/github.com/google/btree/README.md`) # BTree implementation for Go This package provides an in-memory B-Tree implementation for Go, useful as an ordered, mutable data structure. The API is based off of the wonderful http://godoc.org/github.com/petar/GoLLRB/llrb, and is meant to allow btree to act as a drop-in replacement for gollrb trees. See http://godoc.org/github.com/google/btree for documentation. --- ### Overview (`vendor/github.com/google/certificate-transparency-go/README.md`) # Certificate Transparency: Go Code [](https://goreportcard.com/report/github.com/google/certificate-transparency-go) [](https://godoc.org/github.com/google/certificate-transparency-go) This repository holds Go code related to [Certificate Transparency](https://www.certificate-transparency.org/) (CT). The repository requires Go version 1.24. - [Repository Structure](#repository-structure) - [Trillian CT Personality](#trillian-ct-personality) - [Working on the Code](#working-on-the-code) - [Running Codebase Checks](#running-codebase-checks) - [Rebuilding Generated Code](#rebuilding-generated-code) ## Support - Slack: https://transparency-dev.slack.com/ ([invitation](https://transparency.dev/slack/)) ## Repository Structure The main parts of the repository are: - Encoding libraries: - `asn1/` and `x509/` are forks of the upstream Go `encoding/asn1` and `crypto/x509` libraries. We maintain separate forks of these packages because CT is intended to act as an observatory of certificates across the ecosystem; as such, we need to be able to process somewhat-malformed certificates that the stricter upstream code would (correctly) reject. Our `x509` fork also includes code for working with the [pre-certificates defined in RFC 6962](https://tools.ietf.org/html/rfc6962#section-3.1). - `tls` holds a library for processing TLS-encoded data as described in [RFC 5246](https://tools.ietf.org/html/rfc5246). - `x509util/` provides additional utilities for dealing with `x509.Certificate`s. - CT client libraries: - The top-level `ct` package (in `.`) holds types and utilities for working with CT data structures defined in [RFC 6962](https://tools.ietf.org/html/rfc6962). - `client/` and `jsonclient/` hold libraries that allow access to CT Logs via HTTP entrypoints described in [section 4 of RFC 6962](https://tools.ietf.org/html/rfc6962#section-4). - `dnsclient/` has a library that allows access to CT Logs over [DNS](https://github.com/google/certificate-transparency-rfcs/blob/master/dns/draft-ct-over-dns.md). - `scanner/` holds a library for scanning the entire contents of an existing CT Log. - CT Personality for [Trillian](https://github.com/google/trillian): - `trillian/` holds code that allows a Certificate Transparency Log to be run using a Trillian Log as its back-end -- see [below](#trillian-ct-personality). - Command line tools: - `./client/ctclient` allows interaction with a CT Log. - `./ctutil/sctcheck` allows SCTs (signed certificate timestamps) from a CT Log to be verified. - `./scanner/scanlog` allows an existing CT Log to be scanned for certificates of interest; please be polite when running this tool against a Log. - `./x509util/certcheck` allows display and verification of certificates - `./x509util/crlcheck` allows display and verification of certificate revocation lists (CRLs). - Other libraries related to CT: - `ctutil/` holds utility functions for validating and verifying CT data structures. - `loglist3/` has a library for reading [v3 JSON lists of CT Logs](https://groups.google.com/a/chromium.org/g/ct-policy/c/IdbrdAcDQto/m/i5KPyzYwBAAJ). ## Trillian CT Personality The `trillian/` subdirectory holds code and scripts for running a CT Log based on the [Trillian](https://github.com/google/trillian) general transparency Log, and is [documented separately](trillian/README.md). ## Working on the Code Developers who want to make changes to the codebase need some additional dependencies and tools, described in the following sections. ### Running Codebase Checks The [`scripts/presubmit.sh`](scripts/presubmit.sh) script runs various tools and tests over the codebase; please ensure this script passes before sending pull requests for review. ```bash # Install golangci-lint go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.1.6 # Run code generation, build, test and linters ./scripts/presubmit.sh # Run build, test and linters but skip code generation ./scripts/presubmit.sh --no-generate # Or just run the linters alone: golangci-lint run ``` ### Rebuilding Generated Code Some of the CT Go code is autogenerated from other files: - [Protocol buffer](https://developers.google.com/protocol-buffers/) message definitions are converted to `.pb.go` implementations. - A mock implementation of the Trillian gRPC API (in `trillian/mockclient`) is created with [GoMock](https://github.com/golang/mock). - Some enums have string-conversion methods (satisfying the `fmt.Stringer` interface) created using the [stringer](https://godoc.org/golang.org/x/tools/cmd/stringer) tool (`go get golang.org/x/tools/cmd/stringer`). Re-generating mock or protobuffer files is only needed if you're changing the original files. The recommended way to do this is by using the Docker image used by the Cloud Build: ```shell docker build -f ./integration/Dockerfile -t ctgo-builder . docker run -it --mount type=bind,src="$(pwd)",target=/src ctgo-builder /bin/bash -c "cd /src; ./scripts/install_deps.sh; go generate -x ./..." ``` These commands first create a docker image from the Dockerfile in this repo, and then launch a container based on this image with the local directory mounted. The correct versions of the tools are determined using the `go.mod` file in this repo, and these tools are installed. Finally, all of the generated files are regenerated and Docker exits. Alternatively, you can install the prerequisites locally: - a series of tools, using `go install` to ensure that the versions are compatible and tested: ``` cd $(go list -f '{{ .Dir }}' github.com/google/certificate-transparency-go); \ go install github.com/golang/mock/mockgen; \ go install google.golang.org/protobuf/proto; \ go install google.golang.org/protobuf/cmd/protoc-gen-go; \ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc; \ go install github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc; \ go install golang.org/x/tools/cmd/stringer ``` - `protoc` tool: you'll need [version 3.20.1](https://github.com/protocolbuffers/protobuf/releases/tag/v3.20.1) installed, and `PATH` updated to include its `bin/` directory. and run the following: ```bash go generate -x ./... # hunts for //go:generate comments and runs them ``` --- ### Overview (`vendor/github.com/google/s2a-go/README.md`) # Secure Session Agent Client Libraries The Secure Session Agent is a service that enables a workload to offload select operations from the mTLS handshake and protects a workload's private key material from exfiltration. Specifically, the workload asks the Secure Session Agent for the TLS configuration to use during the handshake, to perform private key operations, and to validate the peer certificate chain. The Secure Session Agent's client libraries enable applications to communicate with the Secure Session Agent during the TLS handshake, and to encrypt traffic to the peer after the TLS handshake is complete. This repository contains the source code for the Secure Session Agent's Go client libraries, which allow gRPC and HTTP Go applications to use the Secure Session Agent. --- ### Overview (`vendor/github.com/google/uuid/README.md`) # uuid The uuid package generates and inspects UUIDs based on [RFC 4122](https://datatracker.ietf.org/doc/html/rfc4122) and DCE 1.1: Authentication and Security Services. This package is based on the github.com/pborman/uuid package (previously named code.google.com/p/go-uuid). It differs from these earlier packages in that a UUID is a 16 byte array rather than a byte slice. One loss due to this change is the ability to represent an invalid UUID (vs a NIL UUID). ###### Install ```sh go get github.com/google/uuid ``` ###### Documentation [](https://pkg.go.dev/github.com/google/uuid) Full `go doc` style documentation for the package can be viewed online without installing this package by using the GoDoc site here: http://pkg.go.dev/github.com/google/uuid --- ### Overview (`vendor/github.com/gorilla/mux/README.md`) # gorilla/mux [](https://codecov.io/github/gorilla/mux) [](https://godoc.org/github.com/gorilla/mux) [](https://sourcegraph.com/github.com/gorilla/mux?badge) Package `gorilla/mux` implements a request router and dispatcher for matching incoming requests to their respective handler. The name mux stands for "HTTP request multiplexer". Like the standard `http.ServeMux`, `mux.Router` matches incoming requests against a list of registered routes and calls a handler for the route that matches the URL or other conditions. The main features are: * It implements the `http.Handler` interface so it is compatible with the standard `http.ServeMux`. * Requests can be matched based on URL host, path, path prefix, schemes, header and query values, HTTP methods or using custom matchers. * URL hosts, paths and query values can have variables with an optional regular expression. * Registered URLs can be built, or "reversed", which helps maintaining references to resources. * Routes can be used as subrouters: nested routes are only tested if the parent route matches. This is useful to define groups of routes that share common conditions like a host, a path prefix or other repeated attributes. As a bonus, this optimizes request matching. --- * [Install](#install) * [Examples](#examples) * [Matching Routes](#matching-routes) * [Static Files](#static-files) * [Serving Single Page Applications](#serving-single-page-applications) (e.g. React, Vue, Ember.js, etc.) * [Registered URLs](#registered-urls) * [Walking Routes](#walking-routes) * [Graceful Shutdown](#graceful-shutdown) * [Middleware](#middleware) * [Handling CORS Requests](#handling-cors-requests) * [Testing Handlers](#testing-handlers) * [Full Example](#full-example) --- ## Install With a [correctly configured](https://golang.org/doc/install#testing) Go toolchain: ```sh go get -u github.com/gorilla/mux ``` ## Examples Let's start registering a couple of URL paths and handlers: ```go func main() { r := mux.NewRouter() r.HandleFunc("/", HomeHandler) r.HandleFunc("/products", ProductsHandler) r.HandleFunc("/articles", ArticlesHandler) http.Handle("/", r) } ``` Here we register three routes mapping URL paths to handlers. This is equivalent to how `http.HandleFunc()` works: if an incoming request URL matches one of the paths, the corresponding handler is called passing (`http.ResponseWriter`, `*http.Request`) as parameters. Paths can have variables. They are defined using the format `{name}` or `{name:pattern}`. If a regular expression pattern is not defined, the matched variable will be anything until the next slash. For example: ```go r := mux.NewRouter() r.HandleFunc("/products/{key}", ProductHandler) r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler) r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler) ``` The names are used to create a map of route variables which can be retrieved calling `mux.Vars()`: ```go func ArticlesCategoryHandler(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) w.WriteHeader(http.StatusOK) fmt.Fprintf(w, "Category: %v\n", vars["category"]) } ``` And this is all you need to know about the basic usage. More advanced options are explained below. ### Matching Routes Routes can also be restricted to a domain or subdomain. Just define a host pattern to be matched. They can also have variables: ```go r := mux.NewRouter() // Only matches if domain is "www.example.com". r.Host("www.example.com") // Matches a dynamic subdomain. r.Host("{subdomain:[a-z]+}.example.com") ``` There are several other matchers that can be added. To match path prefixes: ```go r.PathPrefix("/products/") ``` ...or HTTP methods: ```go r.Methods("GET", "POST") ``` ...or URL schemes: ```go r.Schemes("https") ``` ...or header values: ```go r.Headers("X-Requested-With", "XMLHttpRequest") ``` ...or query values: ```go r.Queries("key", "value") ``` ...or to use a custom matcher function: ```go r.MatcherFunc(func(r *http.Request, rm *RouteMatch) bool { return r.ProtoMajor == 0 }) ``` ...and finally, it is possible to combine several matchers in a single route: ```go r.HandleFunc("/products", ProductsHandler). Host("www.example.com"). Methods("GET"). Schemes("http") ``` Routes are tested in the order they were added to the router. If two routes match, the first one wins: ```go r := mux.NewRouter() r.HandleFunc("/specific", specificHandler) r.PathPrefix("/").Handler(catchAllHandler) ``` Setting the same matching conditions again and again can be boring, so we have a way to group several routes that share the same requirements. We call it "subrouting". For example, let's say we have several URLs that should only match when the host is `www.example.com`. Create a route for that host and get a "subrouter" from it: ```go r := mux.NewRouter() s := r.Host("www.example.com").Subrouter() ``` Then register routes in the subrouter: ```go s.HandleFunc("/products/", ProductsHandler) s.HandleFunc("/products/{key}", ProductHandler) s.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler) ``` The three URL paths we registered above will only be tested if the domain is `www.example.com`, because the subrouter is tested first. This is not only convenient, but also optimizes request matching. You can create subrouters combining any attribute matchers accepted by a route. Subrouters can be used to create domain or path "namespaces": you define subrouters in a central place and then parts of the app can register its paths relatively to a given subrouter. There's one more thing about subroutes. When a subrouter has a path prefix, the inner routes use it as base for their paths: ```go r := mux.NewRouter() s := r.PathPrefix("/products").Subrouter() // "/products/" s.HandleFunc("/", ProductsHandler) // "/products/{key}/" s.HandleFunc("/{key}/", ProductHandler) // "/products/{key}/details" s.HandleFunc("/{key}/details", ProductDetailsHandler) ``` ### Static Files Note that the path provided to `PathPrefix()` represents a "wildcard": calling `PathPrefix("/static/").Handler(...)` means that the handler will be passed any request that matches "/static/\*". This makes it easy to serve static files with mux: ```go func main() { var dir string flag.StringVar(&dir, "dir", ".", "the directory to serve files from. Defaults to the current dir") flag.Parse() r := mux.NewRouter() // This will serve files under http://localhost:8000/static/ r.PathPrefix("/static/").Handler(http.StripPrefix("/static/", http.FileServer(http.Dir(dir)))) srv := &http.Server{ Handler: r, Addr: "127.0.0.1:8000", // Good practice: enforce timeouts for servers you create! WriteTimeout: 15 * time.Second, ReadTimeout: 15 * time.Second, } log.Fatal(srv.ListenAndServe()) } ``` ### Serving Single Page Applications Most of the time it makes sense to serve your SPA on a separate web server from your API, but sometimes it's desirable to serve them both from one place. It's possible to write a simple handler for serving your SPA (for use with React Router's [BrowserRouter](https://reacttraining.com/react-router/web/api/BrowserRouter) for example), and leverage mux's powerful routing for your API endpoints. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Registered URLs Now let's see how to build registered URLs. Routes can be named. All routes that define a name can have their URLs built, or "reversed". We define a name calling `Name()` on a route. For example: ```go r := mux.NewRouter() r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article") ``` To build a URL, get the route and call the `URL()` method, passing a sequence of key/value pairs for the route variables. For the previous route, we would do: ```go url, err := r.Get("article").URL("category", "technology", "id", "42") ``` ...and the result will be a `url.URL` with the following path: ``` "/articles/technology/42" ``` This also works for host and query value variables: ```go r := mux.NewRouter() r.Host("{subdomain}.example.com"). Path("/articles/{category}/{id:[0-9]+}"). Queries("filter", "{filter}"). HandlerFunc(ArticleHandler). Name("article") // url.String() will be "http://news.example.com/articles/technology/42?filter=gorilla" url, err := r.Get("article").URL("subdomain", "news", "category", "technology", "id", "42", "filter", "gorilla") ``` All variables defined in the route are required, and their values must conform to the corresponding patterns. These requirements guarantee that a generated URL will always match a registered route -- the only exception is for explicitly defined "build-only" routes which never match. Regex support also exists for matching Headers within a route. For example, we could do: ```go r.HeadersRegexp("Content-Type", "application/(text|json)") ``` ...and the route will match both requests with a Content-Type of `application/json` as well as `application/text` There's also a way to build only the URL host or path for a route: use the methods `URLHost()` or `URLPath()` instead. For the previous route, we would do: ```go // "http://news.example.com/" host, err := r.Get("article").URLHost("subdomain", "news") // "/articles/technology/42" path, err := r.Get("article").URLPath("category", "technology", "id", "42") ``` And if you use subrouters, host and path defined separately can be built as well: ```go r := mux.NewRouter() s := r.Host("{subdomain}.example.com").Subrouter() s.Path("/articles/{category}/{id:[0-9]+}"). HandlerFunc(ArticleHandler). Name("article") // "http://news.example.com/articles/technology/42" url, err := r.Get("article").URL("subdomain", "news", "category", "technology", "id", "42") ``` To find all the required variables for a given route when calling `URL()`, the method `GetVarNames()` is available: ```go r := mux.NewRouter() r.Host("{domain}"). Path("/{group}/{item_id}"). Queries("some_data1", "{some_data1}"). Queries("some_data2", "{some_data2}"). Name("article") // Will print [domain group item_id some_data1 some_data2] fmt.Println(r.Get("article").GetVarNames()) ``` ### Walking Routes The `Walk` function on `mux.Router` can be used to visit all of the routes that are registered on a router. For example, the following prints all of the registered routes: ```go package main import ( "fmt" "net/http" "strings" "github.com/gorilla/mux" ) func handler(w http.ResponseWriter, r *http.Request) { return } func main() { r := mux.NewRouter() r.HandleFunc("/", handler) r.HandleFunc("/products", handler).Methods("POST") r.HandleFunc("/articles", handler).Methods("GET") r.HandleFunc("/articles/{id}", handler).Methods("GET", "PUT") r.HandleFunc("/authors", handler).Queries("surname", "{surname}") err := r.Walk(func(route *mux.Route, router *mux.Router, ancestors []*mux.Route) error { pathTemplate, err := route.GetPathTemplate() if err == nil { fmt.Println("ROUTE:", pathTemplate) } pathRegexp, err := route.GetPathRegexp() if err == nil { fmt.Println("Path regexp:", pathRegexp) } queriesTemplates, err := route.GetQueriesTemplates() if err == nil { fmt.Println("Queries templates:", strings.Join(queriesTemplates, ",")) } queriesRegexps, err := route.GetQueriesRegexp() if err == nil { fmt.Println("Queries regexps:", strings.Join(queriesRegexps, ",")) } methods, err := route.GetMethods() if err == nil { fmt.Println("Methods:", strings.Join(methods, ",")) } fmt.Println() return nil }) if err != nil { fmt.Println(err) } http.Handle("/", r) } ``` ### Graceful Shutdown Go 1.8 introduced the ability to [gracefully shutdown](https://golang.org/doc/go1.8#http_shutdown) a `*http.Server`. Here's how to do that alongside `mux`: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Middleware Mux supports the addition of middlewares to a [Router](https://godoc.org/github.com/gorilla/mux#Router), which are executed in the order they are added if a match is found, including its subrouters. Middlewares are (typically) small pieces of code which take one request, do something with it, and pass it down to another middleware or the final handler. Some common use cases for middleware are request logging, header manipulation, or `ResponseWriter` hijacking. Mux middlewares are defined using the de facto standard type: ```go type MiddlewareFunc func(http.Handler) http.Handler ``` Typically, the returned handler is a closure which does something with the http.ResponseWriter and http.Request passed to it, and then calls the handler passed as parameter to the MiddlewareFunc. This takes advantage of closures being able access variables from the context where they are created, while retaining the signature enforced by the receivers. A very basic middleware which logs the URI of the request being handled could be written as: ```go func loggingMiddleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Do stuff here log.Println(r.RequestURI) // Call the next handler, which can be another middleware in the chain, or the final handler. next.ServeHTTP(w, r) }) } ``` Middlewares can be added to a router using `Router.Use()`: ```go r := mux.NewRouter() r.HandleFunc("/", handler) r.Use(loggingMiddleware) ``` A more complex authentication middleware, which maps session token to users, could be written as: ```go // Define our struct type authenticationMiddleware struct { tokenUsers map[string]string } // Initialize it somewhere func (amw *authenticationMiddleware) Populate() { amw.tokenUsers["00000000"] = "user0" amw.tokenUsers["aaaaaaaa"] = "userA" amw.tokenUsers["05f717e5"] = "randomUser" amw.tokenUsers["deadbeef"] = "user0" } // Middleware function, which will be called for each request func (amw *authenticationMiddleware) Middleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { token := r.Header.Get("X-Session-Token") if user, found := amw.tokenUsers[token]; found { // We found the token in our map log.Printf("Authenticated user %s\n", user) // Pass down the request to the next middleware (or final handler) next.ServeHTTP(w, r) } else { // Write an error and stop the handler chain http.Error(w, "Forbidden", http.StatusForbidden) } }) } ``` ```go r := mux.NewRouter() r.HandleFunc("/", handler) amw := authenticationMiddleware{tokenUsers: make(map[string]string)} amw.Populate() r.Use(amw.Middleware) ``` Note: The handler chain will be stopped if your middleware doesn't call `next.ServeHTTP()` with the corresponding parameters. This can be used to abort a request if the middleware writer wants to. Middlewares _should_ write to `ResponseWriter` if they _are_ going to terminate the request, and they _should not_ write to `ResponseWriter` if they _are not_ going to terminate it. ### Handling CORS Requests [CORSMethodMiddleware](https://godoc.org/github.com/gorilla/mux#CORSMethodMiddleware) intends to make it easier to strictly set the `Access-Control-Allow-Methods` response header. * You will still need to use your own CORS handler to set the other CORS headers such as `Access-Control-Allow-Origin` * The middleware will set the `Access-Control-Allow-Methods` header to all the method matchers (e.g. `r.Methods(http.MethodGet, http.MethodPut, http.MethodOptions)` -> `Access-Control-Allow-Methods: GET,PUT,OPTIONS`) on a route * If you do not specify any methods, then: > _Important_: there must be an `OPTIONS` method matcher for the middleware to set the headers. Here is an example of using `CORSMethodMiddleware` along with a custom `OPTIONS` handler to set all the required CORS headers: ```go package main import ( "net/http" "github.com/gorilla/mux" ) func main() { r := mux.NewRouter() // IMPORTANT: you must specify an OPTIONS method matcher for the middleware to set CORS headers r.HandleFunc("/foo", fooHandler).Methods(http.MethodGet, http.MethodPut, http.MethodPatch, http.MethodOptions) r.Use(mux.CORSMethodMiddleware(r)) http.ListenAndServe(":8080", r) } func fooHandler(w http.ResponseWriter, r *http.Request) { w.Header().Set("Access-Control-Allow-Origin", "*") if r.Method == http.MethodOptions { return } w.Write([]byte("foo")) } ``` And an request to `/foo` using something like: ```bash curl localhost:8080/foo -v ``` Would look like: ```bash * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8080 (#0) > GET /foo HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.59.0 > Accept: */* > < HTTP/1.1 200 OK < Access-Control-Allow-Methods: GET,PUT,PATCH,OPTIONS < Access-Control-Allow-Origin: * < Date: Fri, 28 Jun 2019 20:13:30 GMT < Content-Length: 3 < Content-Type: text/plain; charset=utf-8 < * Connection #0 to host localhost left intact foo ``` ### Testing Handlers Testing handlers in a Go web application is straightforward, and _mux_ doesn't complicate this any further. Given two files: `endpoints.go` and `endpoints_test.go`, here's how we'd test an application using _mux_. First, our simple HTTP handler: ```go // endpoints.go package main func HealthCheckHandler(w http.ResponseWriter, r *http.Request) { // A very simple health check. w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusOK) // In the future we could report back on the status of our DB, or our cache // (e.g. Redis) by performing a simple PING, and include them in the response. io.WriteString(w, `{"alive": true}`) } func main() { r := mux.NewRouter() r.HandleFunc("/health", HealthCheckHandler) log.Fatal(http.ListenAndServe("localhost:8080", r)) } ``` Our test code: ```go // endpoints_test.go package main import ( "net/http" "net/http/httptest" "testing" ) func TestHealthCheckHandler(t *testing.T) { // Create a request to pass to our handler. We don't have any query parameters for now, so we'll // pass 'nil' as the third parameter. req, err := http.NewRequest("GET", "/health", nil) if err != nil { t.Fatal(err) } // We create a ResponseRecorder (which satisfies http.ResponseWriter) to record the response. rr := httptest.NewRecorder() handler := http.HandlerFunc(HealthCheckHandler) // Our handlers satisfy http.Handler, so we can call their ServeHTTP method // directly and pass in our Request and ResponseRecorder. handler.ServeHTTP(rr, req) // Check the status code is what we expect. if status := rr.Code; status != http.StatusOK { t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusOK) } // Check the response body is what we expect. expected := `{"alive": true}` if rr.Body.String() != expected { t.Errorf("handler returned unexpected body: got %v want %v", rr.Body.String(), expected) } } ``` In the case that our routes have [variables](#examples), we can pass those in the request. We could write [table-driven tests](https://dave.cheney.net/2013/06/09/writing-table-driven-tests-in-go) to test multiple possible route variables as needed. ```go // endpoints.go func main() { r := mux.NewRouter() // A route with a route variable: r.HandleFunc("/metrics/{type}", MetricsHandler) log.Fatal(http.ListenAndServe("localhost:8080", r)) } ``` Our test file, with a table-driven test of `routeVariables`: ```go // endpoints_test.go func TestMetricsHandler(t *testing.T) { tt := []struct{ routeVariable string shouldPass bool }{ {"goroutines", true}, {"heap", true}, {"counters", true}, {"queries", true}, {"adhadaeqm3k", false}, } for _, tc := range tt { path := fmt.Sprintf("/metrics/%s", tc.routeVariable) req, err := http.NewRequest("GET", path, nil) if err != nil { t.Fatal(err) } rr := httptest.NewRecorder() // To add the vars to the context, // we need to create a router through which we can pass the request. router := mux.NewRouter() router.HandleFunc("/metrics/{type}", MetricsHandler) router.ServeHTTP(rr, req) // In this case, our MetricsHandler returns a non-200 response // for a route variable it doesn't know about. if rr.Code == http.StatusOK && !tc.shouldPass { t.Errorf("handler should have failed on routeVariable %s: got %v want %v", tc.routeVariable, rr.Code, http.StatusOK) } } } ``` ## Full Example Here's a complete, runnable example of a small `mux` based server: ```go package main import ( "net/http" "log" "github.com/gorilla/mux" ) func YourHandler(w http.ResponseWriter, r *http.Request) { w.Write([]byte("Gorilla!\n")) } func main() { r := mux.NewRouter() // Routes consist of a path and a handler function. r.HandleFunc("/", YourHandler) // Bind to a port and pass our router in log.Fatal(http.ListenAndServe(":8000", r)) } ``` ## License BSD licensed. See the LICENSE file for details. --- ### Overview (`vendor/github.com/grpc-ecosystem/go-grpc-prometheus/README.md`) # Go gRPC Interceptors for Prometheus monitoring [](https://travis-ci.org/grpc-ecosystem/go-grpc-prometheus) [](http://goreportcard.com/report/grpc-ecosystem/go-grpc-prometheus) [](https://godoc.org/github.com/grpc-ecosystem/go-grpc-prometheus) [](https://sourcegraph.com/github.com/grpc-ecosystem/go-grpc-prometheus/?badge) [](https://codecov.io/gh/grpc-ecosystem/go-grpc-prometheus) [](LICENSE) [Prometheus](https://prometheus.io/) monitoring for your [gRPC Go](https://github.com/grpc/grpc-go) servers and clients. A sister implementation for [gRPC Java](https://github.com/grpc/grpc-java) (same metrics, same semantics) is in [grpc-ecosystem/java-grpc-prometheus](https://github.com/grpc-ecosystem/java-grpc-prometheus). ## Interceptors [gRPC Go](https://github.com/grpc/grpc-go) recently acquired support for Interceptors, i.e. middleware that is executed by a gRPC Server before the request is passed onto the user's application logic. It is a perfect way to implement common patterns: auth, logging and... monitoring. To use Interceptors in chains, please see [`go-grpc-middleware`](https://github.com/mwitkow/go-grpc-middleware). ## Usage There are two types of interceptors: client-side and server-side. This package provides monitoring Interceptors for both. ### Server-side ```go import "github.com/grpc-ecosystem/go-grpc-prometheus" ... // Initialize your gRPC server's interceptor. myServer := grpc.NewServer( grpc.StreamInterceptor(grpc_prometheus.StreamServerInterceptor), grpc.UnaryInterceptor(grpc_prometheus.UnaryServerInterceptor), ) // Register your gRPC service implementations. myservice.RegisterMyServiceServer(s.server, &myServiceImpl{}) // After all your registrations, make sure all of the Prometheus metrics are initialized. grpc_prometheus.Register(myServer) // Register Prometheus metrics handler. http.Handle("/metrics", promhttp.Handler()) ... ``` ### Client-side ```go import "github.com/grpc-ecosystem/go-grpc-prometheus" ... clientConn, err = grpc.Dial( address, grpc.WithUnaryInterceptor(grpc_prometheus.UnaryClientInterceptor), grpc.WithStreamInterceptor(grpc_prometheus.StreamClientInterceptor) ) client = pb_testproto.NewTestServiceClient(clientConn) resp, err := client.PingEmpty(s.ctx, &myservice.Request{Msg: "hello"}) ... ``` # Metrics ## Labels All server-side metrics start with `grpc_server` as Prometheus subsystem name. All client-side metrics start with `grpc_client`. Both of them have mirror-concepts. Similarly all methods contain the same rich labels: * `grpc_service` - the [gRPC service](http://www.grpc.io/docs/#defining-a-service) name, which is the combination of protobuf `package` and the `grpc_service` section name. E.g. for `package = mwitkow.testproto` and `service TestService` the label will be `grpc_service="mwitkow.testproto.TestService"` * `grpc_method` - the name of the method called on the gRPC service. E.g. `grpc_method="Ping"` * `grpc_type` - the gRPC [type of request](http://www.grpc.io/docs/guides/concepts.html#rpc-life-cycle). Differentiating between the two is important especially for latency measurements. - `unary` is single request, single response RPC - `client_stream` is a multi-request, single response RPC - `server_stream` is a single request, multi-response RPC - `bidi_stream` is a multi-request, multi-response RPC Additionally for completed RPCs, the following labels are used: * `grpc_code` - the human-readable [gRPC status code](https://github.com/grpc/grpc-go/blob/master/codes/codes.go). The list of all statuses is to long, but here are some common ones: - `OK` - means the RPC was successful - `IllegalArgument` - RPC contained bad values - `Internal` - server-side error not disclosed to the clients ## Counters The counters and their up to date documentation is in [server_reporter.go](server_reporter.go) and [client_reporter.go](client_reporter.go) the respective Prometheus handler (usually `/metrics`). For the purpose of this documentation we will only discuss `grpc_server` metrics. The `grpc_client` ones contain mirror concepts. For simplicity, let's assume we're tracking a single server-side RPC call of [`mwitkow.testproto.TestService`](examples/testproto/test.proto), calling the method `PingList`. The call succeeds and returns 20 messages in the stream. First, immediately after the server receives the call it will increment the `grpc_server_started_total` and start the handling time clock (if histograms are enabled). ```jsoniq grpc_server_started_total{grpc_method="PingList",grpc_service="mwitkow.testproto.TestService",grpc_type="server_stream"} 1 ``` Then the user logic gets invoked. It receives one message from the client containing the request (it's a `server_stream`): ```jsoniq grpc_server_msg_received_total{grpc_method="PingList",grpc_service="mwitkow.testproto.TestService",grpc_type="server_stream"} 1 ``` The user logic may return an error, or send multiple messages back to the client. In this case, on each of the 20 messages sent back, a counter will be incremented: ```jsoniq grpc_server_msg_sent_total{grpc_method="PingList",grpc_service="mwitkow.testproto.TestService",grpc_type="server_stream"} 20 ``` After the call completes, its status (`OK` or other [gRPC status code](https://github.com/grpc/grpc-go/blob/master/codes/codes.go)) and the relevant call labels increment the `grpc_server_handled_total` counter. ```jsoniq grpc_server_handled_total{grpc_code="OK",grpc_method="PingList",grpc_service="mwitkow.testproto.TestService",grpc_type="server_stream"} 1 ``` ## Histograms [Prometheus histograms](https://prometheus.io/docs/concepts/metric_types/#histogram) are a great way to measure latency distributions of your RPCs. However, since it is bad practice to have metrics of [high cardinality](https://prometheus.io/docs/practices/instrumentation/#do-not-overuse-labels) the latency monitoring metrics are disabled by default. To enable them please call the following in your server initialization code: ```jsoniq grpc_prometheus.EnableHandlingTimeHistogram() ``` After the call completes, its handling time will be recorded in a [Prometheus histogram](https://prometheus.io/docs/concepts/metric_types/#histogram) variable `grpc_server_handling_seconds`. The histogram variable contains three sub-metrics: * `grpc_server_handling_seconds_count` - the count of all completed RPCs by status and method * `grpc_server_handling_seconds_sum` - cumulative time of RPCs by status and method, useful for calculating average handling times * `grpc_server_handling_seconds_bucket` - contains the counts of RPCs by status and method in respective handling-time buckets. These buckets can be used by Prometheus to estimate SLAs (see [here](https://prometheus.io/docs/practices/histograms/)) The counter values will look as follows: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Useful query examples Prometheus philosophy is to provide raw metrics to the monitoring system, and let the aggregations be handled there. The verbosity of above metrics make it possible to have that flexibility. Here's a couple of useful monitoring queries: ### request inbound rate ```jsoniq sum(rate(grpc_server_started_total{job="foo"}[1m])) by (grpc_service) ``` For `job="foo"` (common label to differentiate between Prometheus monitoring targets), calculate the rate of requests per second (1 minute window) for each gRPC `grpc_service` that the job has. Please note how the `grpc_method` is being omitted here: all methods of a given gRPC service will be summed together. ### unary request error rate ```jsoniq sum(rate(grpc_server_handled_total{job="foo",grpc_type="unary",grpc_code!="OK"}[1m])) by (grpc_service) ``` For `job="foo"`, calculate the per-`grpc_service` rate of `unary` (1:1) RPCs that failed, i.e. the ones that didn't finish with `OK` code. ### unary request error percentage ```jsoniq sum(rate(grpc_server_handled_total{job="foo",grpc_type="unary",grpc_code!="OK"}[1m])) by (grpc_service) / sum(rate(grpc_server_started_total{job="foo",grpc_type="unary"}[1m])) by (grpc_service) * 100.0 ``` For `job="foo"`, calculate the percentage of failed requests by service. It's easy to notice that this is a combination of the two above examples. This is an example of a query you would like to [alert on](https://prometheus.io/docs/alerting/rules/) in your system for SLA violations, e.g. "no more than 1% requests should fail". ### average response stream size ```jsoniq sum(rate(grpc_server_msg_sent_total{job="foo",grpc_type="server_stream"}[10m])) by (grpc_service) / sum(rate(grpc_server_started_total{job="foo",grpc_type="server_stream"}[10m])) by (grpc_service) ``` For `job="foo"` what is the `grpc_service`-wide `10m` average of messages returned for all ` server_stream` RPCs. This allows you to track the stream sizes returned by your system, e.g. allows you to track when clients started to send "wide" queries that ret Note the divisor is the number of started RPCs, in order to account for in-flight requests. ### 99%-tile latency of unary requests ```jsoniq histogram_quantile(0.99, sum(rate(grpc_server_handling_seconds_bucket{job="foo",grpc_type="unary"}[5m])) by (grpc_service,le) ) ``` For `job="foo"`, returns an 99%-tile [quantile estimation](https://prometheus.io/docs/practices/histograms/#quantiles) of the handling time of RPCs per service. Please note the `5m` rate, this means that the quantile estimation will take samples in a rolling `5m` window. When combined with other quantiles (e.g. 50%, 90%), this query gives you tremendous insight into the responsiveness of your system (e.g. impact of caching). ### percentage of slow unary queries (>250ms) ```jsoniq 100.0 - ( sum(rate(grpc_server_handling_seconds_bucket{job="foo",grpc_type="unary",le="0.25"}[5m])) by (grpc_service) / sum(rate(grpc_server_handling_seconds_count{job="foo",grpc_type="unary"}[5m])) by (grpc_service) ) * 100.0 ``` For `job="foo"` calculate the by-`grpc_service` fraction of slow requests that took longer than `0.25` seconds. This query is relatively complex, since the Prometheus aggregations use `le` (less or equal) buckets, meaning that counting "fast" requests fractions is easier. However, simple maths helps. This is an example of a query you would like to alert on in your system for SLA violations, e.g. "less than 1% of requests are slower than 250ms". ## Status This code has been used since August 2015 as the basis for monitoring of *production* gRPC micro services at [Improbable](https://improbable.io). ## License `go-grpc-prometheus` is released under the Apache 2.0 license. See the [LICENSE](LICENSE) file for details. --- ### Overview (`vendor/github.com/hashicorp/errwrap/README.md`) # errwrap `errwrap` is a package for Go that formalizes the pattern of wrapping errors and checking if an error contains another error. There is a common pattern in Go of taking a returned `error` value and then wrapping it (such as with `fmt.Errorf`) before returning it. The problem with this pattern is that you completely lose the original `error` structure. Arguably the _correct_ approach is that you should make a custom structure implementing the `error` interface, and have the original error as a field on that structure, such [as this example](http://golang.org/pkg/os/#PathError). This is a good approach, but you have to know the entire chain of possible rewrapping that happens, when you might just care about one. `errwrap` formalizes this pattern (it doesn't matter what approach you use above) by giving a single interface for wrapping errors, checking if a specific error is wrapped, and extracting that error. ## Installation and Docs Install using `go get github.com/hashicorp/errwrap`. Full documentation is available at http://godoc.org/github.com/hashicorp/errwrap ## Usage #### Basic Usage Below is a very basic example of its usage: ```go // A function that always returns an error, but wraps it, like a real // function might. func tryOpen() error { _, err := os.Open("/i/dont/exist") if err != nil { return errwrap.Wrapf("Doesn't exist: {{err}}", err) } return nil } func main() { err := tryOpen() // We can use the Contains helpers to check if an error contains // another error. It is safe to do this with a nil error, or with // an error that doesn't even use the errwrap package. if errwrap.Contains(err, "does not exist") { // Do something } if errwrap.ContainsType(err, new(os.PathError)) { // Do something } // Or we can use the associated `Get` functions to just extract // a specific error. This would return nil if that specific error doesn't // exist. perr := errwrap.GetType(err, new(os.PathError)) } ``` #### Custom Types If you're already making custom types that properly wrap errors, then you can get all the functionality of `errwraps.Contains` and such by implementing the `Wrapper` interface with just one function. Example: ```go type AppError { Code ErrorCode Err error } func (e *AppError) WrappedErrors() []error { return []error{e.Err} } ``` Now this works: ```go err := &AppError{Err: fmt.Errorf("an error")} if errwrap.ContainsType(err, fmt.Errorf("")) { // This will work! } ``` --- ### Overview (`vendor/github.com/hashicorp/go-immutable-radix/README.md`) go-immutable-radix [](https://circleci.com/gh/hashicorp/go-immutable-radix/tree/master) ========= Provides the `iradix` package that implements an immutable [radix tree](http://en.wikipedia.org/wiki/Radix_tree). The package only provides a single `Tree` implementation, optimized for sparse nodes. As a radix tree, it provides the following: * O(k) operations. In many cases, this can be faster than a hash table since the hash function is an O(k) operation, and hash tables have very poor cache locality. * Minimum / Maximum value lookups * Ordered iteration A tree supports using a transaction to batch multiple updates (insert, delete) in a more efficient manner than performing each operation one at a time. For a mutable variant, see [go-radix](https://github.com/armon/go-radix). Documentation ============= The full documentation is available on [Godoc](http://godoc.org/github.com/hashicorp/go-immutable-radix). Example ======= Below is a simple example of usage ```go // Create a tree r := iradix.New() r, _, _ = r.Insert([]byte("foo"), 1) r, _, _ = r.Insert([]byte("bar"), 2) r, _, _ = r.Insert([]byte("foobar"), 2) // Find the longest prefix match m, _, _ := r.Root().LongestPrefix([]byte("foozip")) if string(m) != "foo" { panic("should be foo") } ``` Here is an example of performing a range scan of the keys. ```go // Create a tree r := iradix.New() r, _, _ = r.Insert([]byte("001"), 1) r, _, _ = r.Insert([]byte("002"), 2) r, _, _ = r.Insert([]byte("005"), 5) r, _, _ = r.Insert([]byte("010"), 10) r, _, _ = r.Insert([]byte("100"), 10) // Range scan over the keys that sort lexicographically between [003, 050) it := r.Root().Iterator() it.SeekLowerBound([]byte("003")) for key, _, ok := it.Next(); ok; key, _, ok = it.Next() { if key >= "050" { break } fmt.Println(key) } // Output: // 005 // 010 ``` --- ### Overview (`vendor/github.com/hashicorp/go-memdb/README.md`) # go-memdb [](https://circleci.com/gh/hashicorp/go-memdb/tree/master) Provides the `memdb` package that implements a simple in-memory database built on immutable radix trees. The database provides Atomicity, Consistency and Isolation from ACID. Being that it is in-memory, it does not provide durability. The database is instantiated with a schema that specifies the tables and indices that exist and allows transactions to be executed. The database provides the following: * Multi-Version Concurrency Control (MVCC) - By leveraging immutable radix trees the database is able to support any number of concurrent readers without locking, and allows a writer to make progress. * Transaction Support - The database allows for rich transactions, in which multiple objects are inserted, updated or deleted. The transactions can span multiple tables, and are applied atomically. The database provides atomicity and isolation in ACID terminology, such that until commit the updates are not visible. * Rich Indexing - Tables can support any number of indexes, which can be simple like a single field index, or more advanced compound field indexes. Certain types like UUID can be efficiently compressed from strings into byte indexes for reduced storage requirements. * Watches - Callers can populate a watch set as part of a query, which can be used to detect when a modification has been made to the database which affects the query results. This lets callers easily watch for changes in the database in a very general way. For the underlying immutable radix trees, see [go-immutable-radix](https://github.com/hashicorp/go-immutable-radix). Documentation ============= The full documentation is available on [Godoc](https://pkg.go.dev/github.com/hashicorp/go-memdb). Example ======= Below is a [simple example](https://play.golang.org/p/gCGE9FA4og1) of usage ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ### Overview (`vendor/github.com/hashicorp/go-metrics/README.md`) go-metrics ========== This library provides a `metrics` package which can be used to instrument code, expose application metrics, and profile runtime performance in a flexible manner. Current API: [](https://godoc.org/github.com/hashicorp/go-metrics) Sinks ----- The `metrics` package makes use of a `MetricSink` interface to support delivery to any type of backend. Currently the following sinks are provided: * StatsiteSink : Sinks to a [statsite](https://github.com/statsite/statsite/) instance (TCP) * StatsdSink: Sinks to a [StatsD](https://github.com/statsd/statsd/) / statsite instance (UDP) * PrometheusSink: Sinks to a [Prometheus](http://prometheus.io/) metrics endpoint (exposed via HTTP for scrapes) * InmemSink : Provides in-memory aggregation, can be used to export stats * FanoutSink : Sinks to multiple sinks. Enables writing to multiple statsite instances for example. * BlackholeSink : Sinks to nowhere In addition to the sinks, the `InmemSignal` can be used to catch a signal, and dump a formatted output of recent metrics. For example, when a process gets a SIGUSR1, it can dump to stderr recent performance metrics for debugging. Labels ------ Most metrics do have an equivalent ending with `WithLabels`, such methods allow to push metrics with labels and use some features of underlying Sinks (ex: translated into Prometheus labels). Since some of these labels may increase the cardinality of metrics, the library allows filtering labels using a allow/block list filtering system which is global to all metrics. * If `Config.AllowedLabels` is not nil, then only labels specified in this value will be sent to underlying Sink, otherwise, all labels are sent by default. * If `Config.BlockedLabels` is not nil, any label specified in this value will not be sent to underlying Sinks. By default, both `Config.AllowedLabels` and `Config.BlockedLabels` are nil, meaning that no tags are filtered at all, but it allows a user to globally block some tags with high cardinality at the application level. Examples -------- Here is an example of using the package: ```go func SlowMethod() { // Profiling the runtime of a method defer metrics.MeasureSince([]string{"SlowMethod"}, time.Now()) } // Configure a statsite sink as the global metrics sink sink, _ := metrics.NewStatsiteSink("statsite:8125") metrics.NewGlobal(metrics.DefaultConfig("service-name"), sink) // Emit a Key/Value pair metrics.EmitKey([]string{"questions", "meaning of life"}, 42) ``` Here is an example of setting up a signal handler: ```go // Setup the inmem sink and signal handler inm := metrics.NewInmemSink(10*time.Second, time.Minute) sig := metrics.DefaultInmemSignal(inm) metrics.NewGlobal(metrics.DefaultConfig("service-name"), inm) // Run some code inm.SetGauge([]string{"foo"}, 42) inm.EmitKey([]string{"bar"}, 30) inm.IncrCounter([]string{"baz"}, 42) inm.IncrCounter([]string{"baz"}, 1) inm.IncrCounter([]string{"baz"}, 80) inm.AddSample([]string{"method", "wow"}, 42) inm.AddSample([]string{"method", "wow"}, 100) inm.AddSample([]string{"method", "wow"}, 22) .... ``` When a signal comes in, output like the following will be dumped to stderr: [2014-01-28 14:57:33.04 -0800 PST][G] 'foo': 42.000 [2014-01-28 14:57:33.04 -0800 PST][P] 'bar': 30.000 [2014-01-28 14:57:33.04 -0800 PST][C] 'baz': Count: 3 Min: 1.000 Mean: 41.000 Max: 80.000 Stddev: 39.509 [2014-01-28 14:57:33.04 -0800 PST][S] 'method.wow': Count: 3 Min: 22.000 Mean: 54.667 Max: 100.000 Stddev: 40.513 --- ### Overview (`vendor/github.com/hashicorp/go-multierror/README.md`) # go-multierror [](https://circleci.com/gh/hashicorp/go-multierror) [](https://pkg.go.dev/github.com/hashicorp/go-multierror) [circleci]: https://app.circleci.com/pipelines/github/hashicorp/go-multierror [godocs]: https://pkg.go.dev/github.com/hashicorp/go-multierror `go-multierror` is a package for Go that provides a mechanism for representing a list of `error` values as a single `error`. This allows a function in Go to return an `error` that might actually be a list of errors. If the caller knows this, they can unwrap the list and access the errors. If the caller doesn't know, the error formats to a nice human-readable format. `go-multierror` is fully compatible with the Go standard library [errors](https://golang.org/pkg/errors/) package, including the functions `As`, `Is`, and `Unwrap`. This provides a standardized approach for introspecting on error values. ## Installation and Docs Install using `go get github.com/hashicorp/go-multierror`. Full documentation is available at https://pkg.go.dev/github.com/hashicorp/go-multierror ### Requires go version 1.13 or newer `go-multierror` requires go version 1.13 or newer. Go 1.13 introduced [error wrapping](https://golang.org/doc/go1.13#error_wrapping), which this library takes advantage of. If you need to use an earlier version of go, you can use the [v1.0.0](https://github.com/hashicorp/go-multierror/tree/v1.0.0) tag, which doesn't rely on features in go 1.13. If you see compile errors that look like the below, it's likely that you're on an older version of go: ``` /go/src/github.com/hashicorp/go-multierror/multierror.go:112:9: undefined: errors.As /go/src/github.com/hashicorp/go-multierror/multierror.go:117:9: undefined: errors.Is ``` ## Usage go-multierror is easy to use and purposely built to be unobtrusive in existing Go applications/libraries that may not be aware of it. **Building a list of errors** The `Append` function is used to create a list of errors. This function behaves a lot like the Go built-in `append` function: it doesn't matter if the first argument is nil, a `multierror.Error`, or any other `error`, the function behaves as you would expect. ```go var result error if err := step1(); err != nil { result = multierror.Append(result, err) } if err := step2(); err != nil { result = multierror.Append(result, err) } return result ``` **Customizing the formatting of the errors** By specifying a custom `ErrorFormat`, you can customize the format of the `Error() string` function: ```go var result *multierror.Error // ... accumulate errors here, maybe using Append if result != nil { result.ErrorFormat = func([]error) string { return "errors!" } } ``` **Accessing the list of errors** `multierror.Error` implements `error` so if the caller doesn't know about multierror, it will work just fine. But if you're aware a multierror might be returned, you can use type switches to access the list of errors: ```go if err := something(); err != nil { if merr, ok := err.(*multierror.Error); ok { // Use merr.Errors } } ``` You can also use the standard [`errors.Unwrap`](https://golang.org/pkg/errors/#Unwrap) function. This will continue to unwrap into subsequent errors until none exist. **Extracting an error** The standard library [`errors.As`](https://golang.org/pkg/errors/#As) function can be used directly with a multierror to extract a specific error: ```go // Assume err is a multierror value err := somefunc() // We want to know if "err" has a "RichErrorType" in it and extract it. var errRich RichErrorType if errors.As(err, &errRich) { // It has it, and now errRich is populated. } ``` **Checking for an exact error value** Some errors are returned as exact errors such as the [`ErrNotExist`](https://golang.org/pkg/os/#pkg-variables) error in the `os` package. You can check if this error is present by using the standard [`errors.Is`](https://golang.org/pkg/errors/#Is) function. ```go // Assume err is a multierror value err := somefunc() if errors.Is(err, os.ErrNotExist) { // err contains os.ErrNotExist } ``` **Returning a multierror only if there are errors** If you build a `multierror.Error`, you can use the `ErrorOrNil` function to return an `error` implementation only if there are errors to return: ```go var result *multierror.Error // ... accumulate errors here // Return the `error` only if errors were added to the multierror, otherwise // return nil since there are no errors. return result.ErrorOrNil() ``` --- ### Overview (`vendor/github.com/hashicorp/go-sockaddr/README.md`) # go-sockaddr ## `sockaddr` Library Socket address convenience functions for Go. `go-sockaddr` is a convenience library that makes doing the right thing with IP addresses easy. `go-sockaddr` is loosely modeled after the UNIX `sockaddr_t` and creates a union of the family of `sockaddr_t` types (see below for an ascii diagram). Library documentation is available at [https://godoc.org/github.com/hashicorp/go-sockaddr](https://godoc.org/github.com/hashicorp/go-sockaddr). The primary intent of the library was to make it possible to define heuristics for selecting the correct IP addresses when a configuration is evaluated at runtime. See the [docs](https://godoc.org/github.com/hashicorp/go-sockaddr), [`template` package](https://godoc.org/github.com/hashicorp/go-sockaddr/template), tests, and [CLI utility](https://github.com/hashicorp/go-sockaddr/tree/master/cmd/sockaddr) for details and hints as to how to use this library. For example, with this library it is possible to find an IP address that: * is attached to a default route ([`GetDefaultInterfaces()`](https://godoc.org/github.com/hashicorp/go-sockaddr#GetDefaultInterfaces)) * is contained within a CIDR block ([`IfByNetwork()`](https://godoc.org/github.com/hashicorp/go-sockaddr#IfByNetwork)) * is an RFC1918 address ([`IfByRFC("1918")`](https://godoc.org/github.com/hashicorp/go-sockaddr#IfByRFC)) * is ordered ([`OrderedIfAddrBy(args)`](https://godoc.org/github.com/hashicorp/go-sockaddr#OrderedIfAddrBy) where `args` includes, but is not limited to, [`AscIfType`](https://godoc.org/github.com/hashicorp/go-sockaddr#AscIfType), [`AscNetworkSize`](https://godoc.org/github.com/hashicorp/go-sockaddr#AscNetworkSize)) * excludes all IPv6 addresses ([`IfByType("^(IPv4)$")`](https://godoc.org/github.com/hashicorp/go-sockaddr#IfByType)) * is larger than a `/32` ([`IfByMaskSize(32)`](https://godoc.org/github.com/hashicorp/go-sockaddr#IfByMaskSize)) * is not on a `down` interface ([`ExcludeIfs("flags", "down")`](https://godoc.org/github.com/hashicorp/go-sockaddr#ExcludeIfs)) * preferences an IPv6 address over an IPv4 address ([`SortIfByType()`](https://godoc.org/github.com/hashicorp/go-sockaddr#SortIfByType) + [`ReverseIfAddrs()`](https://godoc.org/github.com/hashicorp/go-sockaddr#ReverseIfAddrs)); and * excludes any IP in RFC6890 address ([`IfByRFC("6890")`](https://godoc.org/github.com/hashicorp/go-sockaddr#IfByRFC)) Or any combination or variation therein. There are also a few simple helper functions such as `GetPublicIP` and `GetPrivateIP` which both return strings and select the first public or private IP address on the default interface, respectively. Similarly, there is also a helper function called `GetInterfaceIP` which returns the first usable IP address on the named interface. ## `sockaddr` CLI Given the possible complexity of the `sockaddr` library, there is a CLI utility that accompanies the library, also called [`sockaddr`](https://github.com/hashicorp/go-sockaddr/tree/master/cmd/sockaddr). The [`sockaddr`](https://github.com/hashicorp/go-sockaddr/tree/master/cmd/sockaddr) utility exposes nearly all of the functionality of the library and can be used either as an administrative tool or testing tool. To install the [`sockaddr`](https://github.com/hashicorp/go-sockaddr/tree/master/cmd/sockaddr), run: ```text $ go install github.com/hashicorp/go-sockaddr/cmd/sockaddr@latest ``` If you're familiar with UNIX's `sockaddr` struct's, the following diagram mapping the C `sockaddr` (top) to `go-sockaddr` structs (bottom) and interfaces will be helpful: ``` +-------------------------------------------------------+ | | | sockaddr | | SockAddr | | | | +--------------+ +----------------------------------+ | | | sockaddr_un | | | | | | SockAddrUnix | | sockaddr_in{,6} | | | +--------------+ | IPAddr | | | | | | | | +-------------+ +--------------+ | | | | | sockaddr_in | | sockaddr_in6 | | | | | | IPv4Addr | | IPv6Addr | | | | | +-------------+ +--------------+ | | | | | | | +----------------------------------+ | | | +-------------------------------------------------------+ ``` ## Inspiration and Design There were many subtle inspirations that led to this design, but the most direct inspiration for the filtering syntax was OpenBSD's [`pf.conf(5)`](https://www.freebsd.org/cgi/man.cgi?query=pf.conf&apropos=0&sektion=0&arch=default&format=html#PARAMETERS) firewall syntax that lets you select the first IP address on a given named interface. The original problem stemmed from: * needing to create immutable images using [Packer](https://www.packer.io) that ran the [Consul](https://www.consul.io) process (Consul can only use one IP address at a time); * images that may or may not have multiple interfaces or IP addresses at runtime; and * we didn't want to rely on configuration management to render out the correct IP address if the VM image was being used in an auto-scaling group. Instead we needed some way to codify a heuristic that would correctly select the right IP address but the input parameters were not known when the image was created. --- ### Overview (`vendor/github.com/hashicorp/memberlist/README.md`) # memberlist [](https://godoc.org/github.com/hashicorp/memberlist) memberlist is a [Go](http://www.golang.org) library that manages cluster membership and member failure detection using a gossip based protocol. The use cases for such a library are far-reaching: all distributed systems require membership, and memberlist is a re-usable solution to managing cluster membership and node failure detection. memberlist is eventually consistent but converges quickly on average. The speed at which it converges can be heavily tuned via various knobs on the protocol. Node failures are detected and network partitions are partially tolerated by attempting to communicate to potentially dead nodes through multiple routes. ## Building If you wish to build memberlist you'll need Go version 1.2+ installed. Please check your installation with: ``` go version ``` ## Usage Memberlist is surprisingly simple to use. An example is shown below: ```go /* Create the initial memberlist from a safe configuration. Please reference the godoc for other default config types. http://godoc.org/github.com/hashicorp/memberlist#Config */ list, err := memberlist.Create(memberlist.DefaultLocalConfig()) if err != nil { panic("Failed to create memberlist: " + err.Error()) } // Join an existing cluster by specifying at least one known member. n, err := list.Join([]string{"1.2.3.4"}) if err != nil { panic("Failed to join cluster: " + err.Error()) } // Ask for members of the cluster for _, member := range list.Members() { fmt.Printf("Member: %s %s\n", member.Name, member.Addr) } // Continue doing whatever you need, memberlist will maintain membership // information in the background. Delegates can be used for receiving // events when members join or leave. ``` The most difficult part of memberlist is configuring it since it has many available knobs in order to tune state propagation delay and convergence times. Memberlist provides a default configuration that offers a good starting point, but errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage. For complete documentation, see the associated [Godoc](http://godoc.org/github.com/hashicorp/memberlist). ## Protocol memberlist is based on ["SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol"](http://ieeexplore.ieee.org/document/1028914/). However, we extend the protocol in a number of ways: * Several extensions are made to increase propagation speed and convergence rate. * Another set of extensions, that we call Lifeguard, are made to make memberlist more robust in the presence of slow message processing (due to factors such as CPU starvation, and network delay or loss). For details on all of these extensions, please read our paper "[Lifeguard : SWIM-ing with Situational Awareness](https://arxiv.org/abs/1707.00788)", along with the memberlist source. We welcome any questions related to the protocol on our issue tracker. ## Metrics Emission and Compatibility This library can emit metrics using either `github.com/armon/go-metrics` or `github.com/hashicorp/go-metrics`. Choosing between the libraries is controlled via build tags. **Build Tags** * `armonmetrics` - Using this tag will cause metrics to be routed to `armon/go-metrics` * `hashicorpmetrics` - Using this tag will cause all metrics to be routed to `hashicorp/go-metrics` If no build tag is specified, the default behavior is to use `armon/go-metrics`. **Deprecating `armon/go-metrics`** Emitting metrics to `armon/go-metrics` is officially deprecated. Usage of `armon/go-metrics` will remain the default until mid-2025 with opt-in support continuing to the end of 2025. **Migration** To migrate an application currently using the older `armon/go-metrics` to instead use `hashicorp/go-metrics` the following should be done. 1. Upgrade libraries using `armon/go-metrics` to consume `hashicorp/go-metrics/compat` instead. This should involve only changing import statements. All repositories in the `hashicorp` namespace 2. Update an applications library dependencies to those that have the compatibility layer configured. 3. Update the application to use `hashicorp/go-metrics` for configuring metrics export instead of `armon/go-metrics` * Replace all application imports of `github.com/armon/go-metrics` with `github.com/hashicorp/go-metrics` * Instrument your build system to build with the `hashicorpmetrics` tag. Eventually once the default behavior changes to use `hashicorp/go-metrics` by default (mid-2025), you can drop the `hashicorpmetrics` build tag. --- ### Overview (`vendor/github.com/hiddeco/sshsig/README.md`) # sshsig [][godoc] [][goreport] This Go library implements the [`SSHSIG` wire protocol][sshsig-protocol], and can be used to sign and verify messages using SSH keys. Compared to other implementations, this library does all the following: - Accepts an `io.Reader` as input for signing and verifying messages. - Performs simple public key fingerprint and namespace mismatch checks in `Verify`. Malicious input will still fail signature verification, but this provides more useful error messages. - Properly uses `ssh-sha2-512` as signature algorithm when signing with an RSA private key, as [described in the protocol][sshsig-rsa-req]. - Does not accept a `Sign` operation without a `namespace` as [specified in the protocol][sshsig-namespace-req]. - Allows `Verify` operations to be performed without a `namespace`, ensuring compatibility with loose implementations. - Provides `Armor` and `Unarmor` functions to encode/decode the signature to/from an (armored) PEM format. For more information about the use of this library, see the [Go Reference][godoc]. ## Acknowledgements There are several other implementations of the `SSHSIG` protocol in Go, from which this library has borrowed ideas: - [go-sshsig][go-sshsig] by Paul Tagliamonte - [Sigstore Rekor][rekor-ssh] from the Sigstore project [sshsig-protocol]: https://github.com/openssh/openssh-portable/blob/V_9_2_P1/PROTOCOL.sshsig [sshsig-rsa-req]: https://github.com/openssh/openssh-portable/blob/V_9_2_P1/PROTOCOL.sshsig#L69-L72 [sshsig-namespace-req]: https://github.com/openssh/openssh-portable/blob/V_9_2_P1/PROTOCOL.sshsig#L57 [go-sshsig]: https://github.com/paultag/go-sshsig/tree/a684343203bd83859fbe5783fc976948b4413010 [rekor-ssh]: https://github.com/sigstore/rekor/tree/v1.0.1/pkg/pki/ssh [godoc]: https://pkg.go.dev/github.com/hiddeco/sshsig [goreport]: https://goreportcard.com/report/github.com/hiddeco/sshsig --- ### Overview (`vendor/github.com/inconshreveable/mousetrap/README.md`) # mousetrap mousetrap is a tiny library that answers a single question. On a Windows machine, was the process invoked by someone double clicking on the executable file while browsing in explorer? ### Motivation Windows developers unfamiliar with command line tools will often "double-click" the executable for a tool. Because most CLI tools print the help and then exit when invoked without arguments, this is often very frustrating for those users. mousetrap provides a way to detect these invocations so that you can provide more helpful behavior and instructions on how to run the CLI tool. To see what this looks like, both from an organizational and a technical perspective, see https://inconshreveable.com/09-09-2014/sweat-the-small-stuff/ ### The interface The library exposes a single interface: func StartedByExplorer() (bool) --- ### Overview (`vendor/github.com/ishidawataru/sctp/README.md`) Stream Control Transmission Protocol (SCTP) ---- [](https://travis-ci.org/ishidawataru/sctp/builds) Examples ---- See `example/sctp.go` ```go $ cd example $ go build $ # run example SCTP server $ ./example -server -port 1000 -ip 10.10.0.1,10.20.0.1 $ # run example SCTP client $ ./example -port 1000 -ip 10.10.0.1,10.20.0.1 ``` --- ### Overview (`vendor/github.com/klauspost/compress/README.md`) # compress This package provides various compression algorithms. * [zstandard](https://github.com/klauspost/compress/tree/master/zstd#zstd) compression and decompression in pure Go. * [S2](https://github.com/klauspost/compress/tree/master/s2#s2-compression) is a high performance replacement for Snappy. * Optimized [deflate](https://godoc.org/github.com/klauspost/compress/flate) packages which can be used as a dropin replacement for [gzip](https://godoc.org/github.com/klauspost/compress/gzip), [zip](https://godoc.org/github.com/klauspost/compress/zip) and [zlib](https://godoc.org/github.com/klauspost/compress/zlib). * [snappy](https://github.com/klauspost/compress/tree/master/snappy) is a drop-in replacement for `github.com/golang/snappy` offering better compression and concurrent streams. * [lzw](https://github.com/klauspost/compress/tree/master/lzw) is a drop-in replacement for `compress/lzw` with 1.4-4x faster decompression and 1.1-2.7x faster compression, depending on the data. * [huff0](https://github.com/klauspost/compress/tree/master/huff0) and [FSE](https://github.com/klauspost/compress/tree/master/fse) implementations for raw entropy encoding. * [Xpress](https://github.com/klauspost/compress/tree/master/xpress) decompression of the Microsoft XPRESS (MS-XCA) plain LZ77 and LZ77+Huffman formats (the LZ77+Huffman variant is the one used in WIM images and Windows Compact OS / WOF data). * [gzhttp](https://github.com/klauspost/compress/tree/master/gzhttp) Provides client and server wrappers for handling gzipped/zstd HTTP requests efficiently. * [pgzip](https://github.com/klauspost/pgzip) is a separate package that provides a very fast parallel gzip implementation. [](https://pkg.go.dev/github.com/klauspost/compress?tab=subdirectories) [](https://github.com/klauspost/compress/actions/workflows/go.yml) [](https://sourcegraph.com/github.com/klauspost/compress?badge) # package usage Use `go get github.com/klauspost/compress@latest` to add it to your project. This package will support the current Go version and 2 versions back. * Use the `nounsafe` tag to disable all use of the "unsafe" package. * Use the `noasm` tag to disable all assembly across packages. Use the links above for more information on each. # changelog * Jul 1st, 2026 [1.19.0](https://github.com/klauspost/compress/releases/tag/v1.19.0) * zstd: Add true concurrent stream encodingin https://github.com/klauspost/compress/pull/1136 * zstd: arm64 decoder asm by @lizthegrey in https://github.com/klauspost/compress/pull/1160 * flate: Add inflate checkpoints in https://github.com/klauspost/compress/pull/1154 * zstd: avoid unused BuildDict encoder allocation by @snissn in https://github.com/klauspost/compress/pull/1147 * snappy/s2: Limit length of varint in `decodedLen` by @eustas in https://github.com/klauspost/compress/pull/1148 * gzhttp: match qvalue parameter case-insensitively (RFC 7231) by @z9z in https://github.com/klauspost/compress/pull/1149 * zip: add NameDecoder callback for legacy encoding rewrite by @SAY-5 in https://github.com/klauspost/compress/pull/1150 * huff0: Allow building tables from histogram in https://github.com/klauspost/compress/pull/1155 * huff0: Allow building table from oversized histogram in https://github.com/klauspost/compress/pull/1156 * s2sx: Clean symlink targets in https://github.com/klauspost/compress/pull/1163 * Feb 9th, 2026 [1.18.4](https://github.com/klauspost/compress/releases/tag/v1.18.4) * gzhttp: Add zstandard to server handler wrapper https://github.com/klauspost/compress/pull/1121 * zstd: Add ResetWithOptions to encoder/decoder https://github.com/klauspost/compress/pull/1122 * gzhttp: preserve qvalue when extra parameters follow in Accept-Encoding by @analytically in https://github.com/klauspost/compress/pull/1116 * Jan 16th, 2026 [1.18.3](https://github.com/klauspost/compress/releases/tag/v1.18.3) * Downstream CVE-2025-61728. See [golang/go#77102](https://github.com/golang/go/issues/77102). * Dec 1st, 2025 - [1.18.2](https://github.com/klauspost/compress/releases/tag/v1.18.2) * flate: Fix invalid encoding on level 9 with single value input in https://github.com/klauspost/compress/pull/1115 * flate: reduce stateless allocations by @RXamzin in https://github.com/klauspost/compress/pull/1106 * Oct 20, 2025 - [1.18.1](https://github.com/klauspost/compress/releases/tag/v1.18.1) - RETRACTED * zstd: Add simple zstd EncodeTo/DecodeTo functions https://github.com/klauspost/compress/pull/1079 * zstd: Fix incorrect buffer size in dictionary encodes https://github.com/klauspost/compress/pull/1059 * s2: check for cap, not len of buffer in EncodeBetter/Best by @vdarulis in https://github.com/klauspost/compress/pull/1080 * zlib: Avoiding extra allocation in zlib.reader.Reset by @travelpolicy in https://github.com/klauspost/compress/pull/1086 * gzhttp: remove redundant err check in zstdReader by @ryanfowler in https://github.com/klauspost/compress/pull/1090 * flate: Faster load+store https://github.com/klauspost/compress/pull/1104 * flate: Simplify matchlen https://github.com/klauspost/compress/pull/1101 * flate: Use exact sizes for huffman tables https://github.com/klauspost/compress/pull/1103 * Feb 19th, 2025 - [1.18.0](https://github.com/klauspost/compress/releases/tag/v1.18.0) * Add unsafe little endian loaders https://github.com/klauspost/compress/pull/1036 * fix: check `r.err != nil` but return a nil value error `err` by @alingse in https://github.com/klauspost/compress/pull/1028 * flate: Simplify L4-6 loading https://github.com/klauspost/compress/pull/1043 * flate: Simplify matchlen (remove asm) https://github.com/klauspost/compress/pull/1045 * s2: Improve small block compression speed w/o asm https://github.com/klauspost/compress/pull/1048 * flate: Fix matchlen L5+L6 https://github.com/klauspost/compress/pull/1049 * flate: Cleanup & reduce casts https://github.com/klauspost/compress/pull/1050
See changes to v1.17.x * Oct 11th, 2024 - [1.17.11](https://github.com/klauspost/compress/releases/tag/v1.17.11) * zstd: Fix extra CRC written with multiple Close calls https://github.com/klauspost/compress/pull/1017 * s2: Don't use stack for index tables https://github.com/klauspost/compress/pull/1014 * gzhttp: No content-type on no body response code by @juliens in https://github.com/klauspost/compress/pull/1011 * gzhttp: Do not set the content-type when response has no body by @kevinpollet in https://github.com/klauspost/compress/pull/1013 * Sep 23rd, 2024 - [1.17.10](https://github.com/klauspost/compress/releases/tag/v1.17.10) * gzhttp: Add TransportAlwaysDecompress option. https://github.com/klauspost/compress/pull/978 * gzhttp: Add supported decompress request body by @mirecl in https://github.com/klauspost/compress/pull/1002 * s2: Add EncodeBuffer buffer recycling callback https://github.com/klauspost/compress/pull/982 * zstd: Improve memory usage on small streaming encodes https://github.com/klauspost/compress/pull/1007 * flate: read data written with partial flush by @vajexal in https://github.com/klauspost/compress/pull/996 * Jun 12th, 2024 - [1.17.9](https://github.com/klauspost/compress/releases/tag/v1.17.9) * s2: Reduce ReadFrom temporary allocations https://github.com/klauspost/compress/pull/949 * flate, zstd: Shave some bytes off amd64 matchLen by @greatroar in https://github.com/klauspost/compress/pull/963 * Upgrade zip/zlib to 1.22.4 upstream https://github.com/klauspost/compress/pull/970 https://github.com/klauspost/compress/pull/971 * zstd: BuildDict fails with RLE table https://github.com/klauspost/compress/pull/951 * Apr 9th, 2024 - [1.17.8](https://github.com/klauspost/compress/releases/tag/v1.17.8) * zstd: Reject blocks where reserved values are not 0 https://github.com/klauspost/compress/pull/885 * zstd: Add RLE detection+encoding https://github.com/klauspost/compress/pull/938 * Feb 21st, 2024 - [1.17.7](https://github.com/klauspost/compress/releases/tag/v1.17.7) * s2: Add AsyncFlush method: Complete the block without flushing by @Jille in https://github.com/klauspost/compress/pull/927 * s2: Fix literal+repeat exceeds dst crash https://github.com/klauspost/compress/pull/930 * Feb 5th, 2024 - [1.17.6](https://github.com/klauspost/compress/releases/tag/v1.17.6) * zstd: Fix incorrect repeat coding in best mode https://github.com/klauspost/compress/pull/923 * s2: Fix DecodeConcurrent deadlock on errors https://github.com/klauspost/compress/pull/925 * Jan 26th, 2024 - [v1.17.5](https://github.com/klauspost/compress/releases/tag/v1.17.5) * flate: Fix reset with dictionary on custom window encodes https://github.com/klauspost/compress/pull/912 * zstd: Add Frame header encoding and stripping https://github.com/klauspost/compress/pull/908 * zstd: Limit better/best default window to 8MB https://github.com/klauspost/compress/pull/913 * zstd: Speed improvements by @greatroar in https://github.com/klauspost/compress/pull/896 https://github.com/klauspost/compress/pull/910 * s2: Fix callbacks for skippable blocks and disallow 0xfe (Padding) by @Jille in https://github.com/klauspost/compress/pull/916 https://github.com/klauspost/compress/pull/917 https://github.com/klauspost/compress/pull/919 https://github.com/klauspost/compress/pull/918 * Dec 1st, 2023 - [v1.17.4](https://github.com/klauspost/compress/releases/tag/v1.17.4) * huff0: Speed up symbol counting by @greatroar in https://github.com/klauspost/compress/pull/887 * huff0: Remove byteReader by @greatroar in https://github.com/klauspost/compress/pull/886 * gzhttp: Allow overriding decompression on transport https://github.com/klauspost/compress/pull/892 * gzhttp: Clamp compression level https://github.com/klauspost/compress/pull/890 * gzip: Error out if reserved bits are set https://github.com/klauspost/compress/pull/891 * Nov 15th, 2023 - [v1.17.3](https://github.com/klauspost/compress/releases/tag/v1.17.3) * fse: Fix max header size https://github.com/klauspost/compress/pull/881 * zstd: Improve better/best compression https://github.com/klauspost/compress/pull/877 * gzhttp: Fix missing content type on Close https://github.com/klauspost/compress/pull/883 * Oct 22nd, 2023 - [v1.17.2](https://github.com/klauspost/compress/releases/tag/v1.17.2) * zstd: Fix rare *CORRUPTION* output in "best" mode. See https://github.com/klauspost/compress/pull/876 * Oct 14th, 2023 - [v1.17.1](https://github.com/klauspost/compress/releases/tag/v1.17.1) * s2: Fix S2 "best" dictionary wrong encoding https://github.com/klauspost/compress/pull/871 * flate: Reduce allocations in decompressor and minor code improvements by @fakefloordiv in https://github.com/klauspost/compress/pull/869 * s2: Fix EstimateBlockSize on 6&7 length input https://github.com/klauspost/compress/pull/867 * Sept 19th, 2023 - [v1.17.0](https://github.com/klauspost/compress/releases/tag/v1.17.0) * Add experimental dictionary builder https://github.com/klauspost/compress/pull/853 * Add xerial snappy read/writer https://github.com/klauspost/compress/pull/838 * flate: Add limited window compression https://github.com/klauspost/compress/pull/843 * s2: Do 2 overlapping match checks https://github.com/klauspost/compress/pull/839 * flate: Add amd64 assembly matchlen https://github.com/klauspost/compress/pull/837 * gzip: Copy bufio.Reader on Reset by @thatguystone in https://github.com/klauspost/compress/pull/860
See changes to v1.16.x * July 1st, 2023 - [v1.16.7](https://github.com/klauspost/compress/releases/tag/v1.16.7) * zstd: Fix default level first dictionary encode https://github.com/klauspost/compress/pull/829 * s2: add GetBufferCapacity() method by @GiedriusS in https://github.com/klauspost/compress/pull/832 * June 13, 2023 - [v1.16.6](https://github.com/klauspost/compress/releases/tag/v1.16.6) * zstd: correctly ignore WithEncoderPadding(1) by @ianlancetaylor in https://github.com/klauspost/compress/pull/806 * zstd: Add amd64 match length assembly https://github.com/klauspost/compress/pull/824 * gzhttp: Handle informational headers by @rtribotte in https://github.com/klauspost/compress/pull/815 * s2: Improve Better compression slightly https://github.com/klauspost/compress/pull/663 * Apr 16, 2023 - [v1.16.5](https://github.com/klauspost/compress/releases/tag/v1.16.5) * zstd: readByte needs to use io.ReadFull by @jnoxon in https://github.com/klauspost/compress/pull/802 * gzip: Fix WriterTo after initial read https://github.com/klauspost/compress/pull/804 * Apr 5, 2023 - [v1.16.4](https://github.com/klauspost/compress/releases/tag/v1.16.4) * zstd: Improve zstd best efficiency by @greatroar and @klauspost in https://github.com/klauspost/compress/pull/784 * zstd: Respect WithAllLitEntropyCompression https://github.com/klauspost/compress/pull/792 * zstd: Fix amd64 not always detecting corrupt data https://github.com/klauspost/compress/pull/785 * zstd: Various minor improvements by @greatroar in https://github.com/klauspost/compress/pull/788 https://github.com/klauspost/compress/pull/794 https://github.com/klauspost/compress/pull/795 * s2: Fix huge block overflow https://github.com/klauspost/compress/pull/779 * s2: Allow CustomEncoder fallback https://github.com/klauspost/compress/pull/780 * gzhttp: Support ResponseWriter Unwrap() in gzhttp handler by @jgimenez in https://github.com/klauspost/compress/pull/799 * Mar 13, 2023 - [v1.16.1](https://github.com/klauspost/compress/releases/tag/v1.16.1) * zstd: Speed up + improve best encoder by @greatroar in https://github.com/klauspost/compress/pull/776 * gzhttp: Add optional [BREACH mitigation](https://github.com/klauspost/compress/tree/master/gzhttp#breach-mitigation). https://github.com/klauspost/compress/pull/762 https://github.com/klauspost/compress/pull/768 https://github.com/klauspost/compress/pull/769 https://github.com/klauspost/compress/pull/770 https://github.com/klauspost/compress/pull/767 * s2: Add Intel LZ4s converter https://github.com/klauspost/compress/pull/766 * zstd: Minor bug fixes https://github.com/klauspost/compress/pull/771 https://github.com/klauspost/compress/pull/772 https://github.com/klauspost/compress/pull/773 * huff0: Speed up compress1xDo by @greatroar in https://github.com/klauspost/compress/pull/774 * Feb 26, 2023 - [v1.16.0](https://github.com/klauspost/compress/releases/tag/v1.16.0) * s2: Add [Dictionary](https://github.com/klauspost/compress/tree/master/s2#dictionaries) support. https://github.com/klauspost/compress/pull/685 * s2: Add Compression Size Estimate. https://github.com/klauspost/compress/pull/752 * s2: Add support for custom stream encoder. https://github.com/klauspost/compress/pull/755 * s2: Add LZ4 block converter. https://github.com/klauspost/compress/pull/748 * s2: Support io.ReaderAt in ReadSeeker. https://github.com/klauspost/compress/pull/747 * s2c/s2sx: Use concurrent decoding. https://github.com/klauspost/compress/pull/746
See changes to v1.15.x * Jan 21st, 2023 (v1.15.15) * deflate: Improve level 7-9 https://github.com/klauspost/compress/pull/739 * zstd: Add delta encoding support by @greatroar in https://github.com/klauspost/compress/pull/728 * zstd: Various speed improvements by @greatroar https://github.com/klauspost/compress/pull/741 https://github.com/klauspost/compress/pull/734 https://github.com/klauspost/compress/pull/736 https://github.com/klauspost/compress/pull/744 https://github.com/klauspost/compress/pull/743 https://github.com/klauspost/compress/pull/745 * gzhttp: Add SuffixETag() and DropETag() options to prevent ETag collisions on compressed responses by @willbicks in https://github.com/klauspost/compress/pull/740 * Jan 3rd, 2023 (v1.15.14) * flate: Improve speed in big stateless blocks https://github.com/klauspost/compress/pull/718 * zstd: Minor speed tweaks by @greatroar in https://github.com/klauspost/compress/pull/716 https://github.com/klauspost/compress/pull/720 * export NoGzipResponseWriter for custom ResponseWriter wrappers by @harshavardhana in https://github.com/klauspost/compress/pull/722 * s2: Add example for indexing and existing stream https://github.com/klauspost/compress/pull/723 * Dec 11, 2022 (v1.15.13) * zstd: Add [MaxEncodedSize](https://pkg.go.dev/github.com/klauspost/compress@v1.15.13/zstd#Encoder.MaxEncodedSize) to encoder https://github.com/klauspost/compress/pull/691 * zstd: Various tweaks and improvements https://github.com/klauspost/compress/pull/693 https://github.com/klauspost/compress/pull/695 https://github.com/klauspost/compress/pull/696 https://github.com/klauspost/compress/pull/701 https://github.com/klauspost/compress/pull/702 https://github.com/klauspost/compress/pull/703 https://github.com/klauspost/compress/pull/704 https://github.com/klauspost/compress/pull/705 https://github.com/klauspost/compress/pull/706 https://github.com/klauspost/compress/pull/707 https://github.com/klauspost/compress/pull/708 * Oct 26, 2022 (v1.15.12) * zstd: Tweak decoder allocs. https://github.com/klauspost/compress/pull/680 * gzhttp: Always delete `HeaderNoCompression` https://github.com/klauspost/compress/pull/683 * Sept 26, 2022 (v1.15.11) * flate: Improve level 1-3 compression https://github.com/klauspost/compress/pull/678 * zstd: Improve "best" compression by @nightwolfz in https://github.com/klauspost/compress/pull/677 * zstd: Fix+reduce decompression allocations https://github.com/klauspost/compress/pull/668 * zstd: Fix non-effective noescape tag https://github.com/klauspost/compress/pull/667 * Sept 16, 2022 (v1.15.10) * zstd: Add [WithDecodeAllCapLimit](https://pkg.go.dev/github.com/klauspost/compress@v1.15.10/zstd#WithDecodeAllCapLimit) https://github.com/klauspost/compress/pull/649 * Add Go 1.19 - deprecate Go 1.16 https://github.com/klauspost/compress/pull/651 * flate: Improve level 5+6 compression https://github.com/klauspost/compress/pull/656 * zstd: Improve "better" compression https://github.com/klauspost/compress/pull/657 * s2: Improve "best" compression https://github.com/klauspost/compress/pull/658 * s2: Improve "better" compression. https://github.com/klauspost/compress/pull/635 * s2: Slightly faster non-assembly decompression https://github.com/klauspost/compress/pull/646 * Use arrays for constant size copies https://github.com/klauspost/compress/pull/659 * July 21, 2022 (v1.15.9) * zstd: Fix decoder crash on amd64 (no BMI) on invalid input https://github.com/klauspost/compress/pull/645 * zstd: Disable decoder extended memory copies (amd64) due to possible crashes https://github.com/klauspost/compress/pull/644 * zstd: Allow single segments up to "max decoded size" https://github.com/klauspost/compress/pull/643 * July 13, 2022 (v1.15.8) * gzip: fix stack exhaustion bug in Reader.Read https://github.com/klauspost/compress/pull/641 * s2: Add Index header trim/restore https://github.com/klauspost/compress/pull/638 * zstd: Optimize seqdeq amd64 asm by @greatroar in https://github.com/klauspost/compress/pull/636 * zstd: Improve decoder memcopy https://github.com/klauspost/compress/pull/637 * huff0: Pass a single bitReader pointer to asm by @greatroar in https://github.com/klauspost/compress/pull/634 * zstd: Branchless getBits for amd64 w/o BMI2 by @greatroar in https://github.com/klauspost/compress/pull/640 * gzhttp: Remove header before writing https://github.com/klauspost/compress/pull/639 * June 29, 2022 (v1.15.7) * s2: Fix absolute forward seeks https://github.com/klauspost/compress/pull/633 * zip: Merge upstream https://github.com/klauspost/compress/pull/631 * zip: Re-add zip64 fix https://github.com/klauspost/compress/pull/624 * zstd: translate fseDecoder.buildDtable into asm by @WojciechMula in https://github.com/klauspost/compress/pull/598 * flate: Faster histograms https://github.com/klauspost/compress/pull/620 * deflate: Use compound hcode https://github.com/klauspost/compress/pull/622 * June 3, 2022 (v1.15.6) * s2: Improve coding for long, close matches https://github.com/klauspost/compress/pull/613 * s2c: Add Snappy/S2 stream recompression https://github.com/klauspost/compress/pull/611 * zstd: Always use configured block size https://github.com/klauspost/compress/pull/605 * zstd: Fix incorrect hash table placement for dict encoding in default https://github.com/klauspost/compress/pull/606 * zstd: Apply default config to ZipDecompressor without options https://github.com/klauspost/compress/pull/608 * gzhttp: Exclude more common archive formats https://github.com/klauspost/compress/pull/612 * s2: Add ReaderIgnoreCRC https://github.com/klauspost/compress/pull/609 * s2: Remove sanity load on index creation https://github.com/klauspost/compress/pull/607 * snappy: Use dedicated function for scoring https://github.com/klauspost/compress/pull/614 * s2c+s2d: Use official snappy framed extension https://github.com/klauspost/compress/pull/610 * May 25, 2022 (v1.15.5) * s2: Add concurrent stream decompression https://github.com/klauspost/compress/pull/602 * s2: Fix final emit oob read crash on amd64 https://github.com/klauspost/compress/pull/601 * huff0: asm implementation of Decompress1X by @WojciechMula https://github.com/klauspost/compress/pull/596 * zstd: Use 1 less goroutine for stream decoding https://github.com/klauspost/compress/pull/588 * zstd: Copy literal in 16 byte blocks when possible https://github.com/klauspost/compress/pull/592 * zstd: Speed up when WithDecoderLowmem(false) https://github.com/klauspost/compress/pull/599 * zstd: faster next state update in BMI2 version of decode by @WojciechMula in https://github.com/klauspost/compress/pull/593 * huff0: Do not check max size when reading table. https://github.com/klauspost/compress/pull/586 * flate: Inplace hashing for level 7-9 https://github.com/klauspost/compress/pull/590 * May 11, 2022 (v1.15.4) * huff0: decompress directly into output by @WojciechMula in [#577](https://github.com/klauspost/compress/pull/577) * inflate: Keep dict on stack [#581](https://github.com/klauspost/compress/pull/581) * zstd: Faster decoding memcopy in asm [#583](https://github.com/klauspost/compress/pull/583) * zstd: Fix ignored crc [#580](https://github.com/klauspost/compress/pull/580) * May 5, 2022 (v1.15.3) * zstd: Allow to ignore checksum checking by @WojciechMula [#572](https://github.com/klauspost/compress/pull/572) * s2: Fix incorrect seek for io.SeekEnd in [#575](https://github.com/klauspost/compress/pull/575) * Apr 26, 2022 (v1.15.2) * zstd: Add x86-64 assembly for decompression on streams and blocks. Contributed by [@WojciechMula](https://github.com/WojciechMula). Typically 2x faster. [#528](https://github.com/klauspost/compress/pull/528) [#531](https://github.com/klauspost/compress/pull/531) [#545](https://github.com/klauspost/compress/pull/545) [#537](https://github.com/klauspost/compress/pull/537) * zstd: Add options to ZipDecompressor and fixes [#539](https://github.com/klauspost/compress/pull/539) * s2: Use sorted search for index [#555](https://github.com/klauspost/compress/pull/555) * Minimum version is Go 1.16, added CI test on 1.18. * Mar 11, 2022 (v1.15.1) * huff0: Add x86 assembly of Decode4X by @WojciechMula in [#512](https://github.com/klauspost/compress/pull/512) * zstd: Reuse zip decoders in [#514](https://github.com/klauspost/compress/pull/514) * zstd: Detect extra block data and report as corrupted in [#520](https://github.com/klauspost/compress/pull/520) * zstd: Handle zero sized frame content size stricter in [#521](https://github.com/klauspost/compress/pull/521) * zstd: Add stricter block size checks in [#523](https://github.com/klauspost/compress/pull/523) * Mar 3, 2022 (v1.15.0) * zstd: Refactor decoder [#498](https://github.com/klauspost/compress/pull/498) * zstd: Add stream encoding without goroutines [#505](https://github.com/klauspost/compress/pull/505) * huff0: Prevent single blocks exceeding 16 bits by @klauspost in[#507](https://github.com/klauspost/compress/pull/507) * flate: Inline literal emission [#509](https://github.com/klauspost/compress/pull/509) * gzhttp: Add zstd to transport [#400](https://github.com/klauspost/compress/pull/400) * gzhttp: Make content-type optional [#510](https://github.com/klauspost/compress/pull/510) Both compression and decompression now supports "synchronous" stream operations. This means that whenever "concurrency" is set to 1, they will operate without spawning goroutines. Stream decompression is now faster on asynchronous, since the goroutine allocation much more effectively splits the workload. On typical streams this will typically use 2 cores fully for decompression. When a stream has finished decoding no goroutines will be left over, so decoders can now safely be pooled and still be garbage collected. While the release has been extensively tested, it is recommended to testing when upgrading.
See changes to v1.14.x * Feb 22, 2022 (v1.14.4) * flate: Fix rare huffman only (-2) corruption. [#503](https://github.com/klauspost/compress/pull/503) * zip: Update deprecated CreateHeaderRaw to correctly call CreateRaw by @saracen in [#502](https://github.com/klauspost/compress/pull/502) * zip: don't read data descriptor early by @saracen in [#501](https://github.com/klauspost/compress/pull/501) #501 * huff0: Use static decompression buffer up to 30% faster [#499](https://github.com/klauspost/compress/pull/499) [#500](https://github.com/klauspost/compress/pull/500) * Feb 17, 2022 (v1.14.3) * flate: Improve fastest levels compression speed ~10% more throughput. [#482](https://github.com/klauspost/compress/pull/482) [#489](https://github.com/klauspost/compress/pull/489) [#490](https://github.com/klauspost/compress/pull/490) [#491](https://github.com/klauspost/compress/pull/491) [#494](https://github.com/klauspost/compress/pull/494) [#478](https://github.com/klauspost/compress/pull/478) * flate: Faster decompression speed, ~5-10%. [#483](https://github.com/klauspost/compress/pull/483) * s2: Faster compression with Go v1.18 and amd64 microarch level 3+. [#484](https://github.com/klauspost/compress/pull/484) [#486](https://github.com/klauspost/compress/pull/486) * Jan 25, 2022 (v1.14.2) * zstd: improve header decoder by @dsnet [#476](https://github.com/klauspost/compress/pull/476) * zstd: Add bigger default blocks [#469](https://github.com/klauspost/compress/pull/469) * zstd: Remove unused decompression buffer [#470](https://github.com/klauspost/compress/pull/470) * zstd: Fix logically dead code by @ningmingxiao [#472](https://github.com/klauspost/compress/pull/472) * flate: Improve level 7-9 [#471](https://github.com/klauspost/compress/pull/471) [#473](https://github.com/klauspost/compress/pull/473) * zstd: Add noasm tag for xxhash [#475](https://github.com/klauspost/compress/pull/475) * Jan 11, 2022 (v1.14.1) * s2: Add stream index in [#462](https://github.com/klauspost/compress/pull/462) * flate: Speed and efficiency improvements in [#439](https://github.com/klauspost/compress/pull/439) [#461](https://github.com/klauspost/compress/pull/461) [#455](https://github.com/klauspost/compress/pull/455) [#452](https://github.com/klauspost/compress/pull/452) [#458](https://github.com/klauspost/compress/pull/458) * zstd: Performance improvement in [#420]( https://github.com/klauspost/compress/pull/420) [#456](https://github.com/klauspost/compress/pull/456) [#437](https://github.com/klauspost/compress/pull/437) [#467](https://github.com/klauspost/compress/pull/467) [#468](https://github.com/klauspost/compress/pull/468) * zstd: add arm64 xxhash assembly in [#464](https://github.com/klauspost/compress/pull/464) * Add garbled for binaries for s2 in [#445](https://github.com/klauspost/compress/pull/445)
See changes to v1.13.x * Aug 30, 2021 (v1.13.5) * gz/zlib/flate: Alias stdlib errors [#425](https://github.com/klauspost/compress/pull/425) * s2: Add block support to commandline tools [#413](https://github.com/klauspost/compress/pull/413) * zstd: pooledZipWriter should return Writers to the same pool [#426](https://github.com/klauspost/compress/pull/426) * Removed golang/snappy as external dependency for tests [#421](https://github.com/klauspost/compress/pull/421) * Aug 12, 2021 (v1.13.4) * Add [snappy replacement package](https://github.com/klauspost/compress/tree/master/snappy). * zstd: Fix incorrect encoding in "best" mode [#415](https://github.com/klauspost/compress/pull/415) * Aug 3, 2021 (v1.13.3) * zstd: Improve Best compression [#404](https://github.com/klauspost/compress/pull/404) * zstd: Fix WriteTo error forwarding [#411](https://github.com/klauspost/compress/pull/411) * gzhttp: Return http.HandlerFunc instead of http.Handler. Unlikely breaking change. [#406](https://github.com/klauspost/compress/pull/406) * s2sx: Fix max size error [#399](https://github.com/klauspost/compress/pull/399) * zstd: Add optional stream content size on reset [#401](https://github.com/klauspost/compress/pull/401) * zstd: use SpeedBestCompression for level >= 10 [#410](https://github.com/klauspost/compress/pull/410) * Jun 14, 2021 (v1.13.1) * s2: Add full Snappy output support [#396](https://github.com/klauspost/compress/pull/396) * zstd: Add configurable [Decoder window](https://pkg.go.dev/github.com/klauspost/compress/zstd#WithDecoderMaxWindow) size [#394](https://github.com/klauspost/compress/pull/394) * gzhttp: Add header to skip compression [#389](https://github.com/klauspost/compress/pull/389) * s2: Improve speed with bigger output margin [#395](https://github.com/klauspost/compress/pull/395) * Jun 3, 2021 (v1.13.0) * Added [gzhttp](https://github.com/klauspost/compress/tree/master/gzhttp#gzip-handler) which allows wrapping HTTP servers and clients with GZIP compressors. * zstd: Detect short invalid signatures [#382](https://github.com/klauspost/compress/pull/382) * zstd: Spawn decoder goroutine only if needed. [#380](https://github.com/klauspost/compress/pull/380)
See changes to v1.12.x * May 25, 2021 (v1.12.3) * deflate: Better/faster Huffman encoding [#374](https://github.com/klauspost/compress/pull/374) * deflate: Allocate less for history. [#375](https://github.com/klauspost/compress/pull/375) * zstd: Forward read errors [#373](https://github.com/klauspost/compress/pull/373) * Apr 27, 2021 (v1.12.2) * zstd: Improve better/best compression [#360](https://github.com/klauspost/compress/pull/360) [#364](https://github.com/klauspost/compress/pull/364) [#365](https://github.com/klauspost/compress/pull/365) * zstd: Add helpers to compress/decompress zstd inside zip files [#363](https://github.com/klauspost/compress/pull/363) * deflate: Improve level 5+6 compression [#367](https://github.com/klauspost/compress/pull/367) * s2: Improve better/best compression [#358](https://github.com/klauspost/compress/pull/358) [#359](https://github.com/klauspost/compress/pull/358) * s2: Load after checking src limit on amd64. [#362](https://github.com/klauspost/compress/pull/362) * s2sx: Limit max executable size [#368](https://github.com/klauspost/compress/pull/368) * Apr 14, 2021 (v1.12.1) * snappy package removed. Upstream added as dependency. * s2: Better compression in "best" mode [#353](https://github.com/klauspost/compress/pull/353) * s2sx: Add stdin input and detect pre-compressed from signature [#352](https://github.com/klauspost/compress/pull/352) * s2c/s2d: Add http as possible input [#348](https://github.com/klauspost/compress/pull/348) * s2c/s2d/s2sx: Always truncate when writing files [#352](https://github.com/klauspost/compress/pull/352) * zstd: Reduce memory usage further when using [WithLowerEncoderMem](https://pkg.go.dev/github.com/klauspost/compress/zstd#WithLowerEncoderMem) [#346](https://github.com/klauspost/compress/pull/346) * s2: Fix potential problem with amd64 assembly and profilers [#349](https://github.com/klauspost/compress/pull/349)
See changes to v1.11.x * Mar 26, 2021 (v1.11.13) * zstd: Big speedup on small dictionary encodes [#344](https://github.com/klauspost/compress/pull/344) [#345](https://github.com/klauspost/compress/pull/345) * zstd: Add [WithLowerEncoderMem](https://pkg.go.dev/github.com/klauspost/compress/zstd#WithLowerEncoderMem) encoder option [#336](https://github.com/klauspost/compress/pull/336) * deflate: Improve entropy compression [#338](https://github.com/klauspost/compress/pull/338) * s2: Clean up and minor performance improvement in best [#341](https://github.com/klauspost/compress/pull/341) * Mar 5, 2021 (v1.11.12) * s2: Add `s2sx` binary that creates [self extracting archives](https://github.com/klauspost/compress/tree/master/s2#s2sx-self-extracting-archives). * s2: Speed up decompression on non-assembly platforms [#328](https://github.com/klauspost/compress/pull/328) * Mar 1, 2021 (v1.11.9) * s2: Add ARM64 decompression assembly. Around 2x output speed. [#324](https://github.com/klauspost/compress/pull/324) * s2: Improve "better" speed and efficiency. [#325](https://github.com/klauspost/compress/pull/325) * s2: Fix binaries. * Feb 25, 2021 (v1.11.8) * s2: Fixed occasional out-of-bounds write on amd64. Upgrade recommended. * s2: Add AMD64 assembly for better mode. 25-50% faster. [#315](https://github.com/klauspost/compress/pull/315) * s2: Less upfront decoder allocation. [#322](https://github.com/klauspost/compress/pull/322) * zstd: Faster "compression" of incompressible data. [#314](https://github.com/klauspost/compress/pull/314) * zip: Fix zip64 headers. [#313](https://github.com/klauspost/compress/pull/313) * Jan 14, 2021 (v1.11.7) * Use Bytes() interface to get bytes across packages. [#309](https://github.com/klauspost/compress/pull/309) * s2: Add 'best' compression option. [#310](https://github.com/klauspost/compress/pull/310) * s2: Add ReaderMaxBlockSize, changes `s2.NewReader` signature to include varargs. [#311](https://github.com/klauspost/compress/pull/311) * s2: Fix crash on small better buffers. [#308](https://github.com/klauspost/compress/pull/308) * s2: Clean up decoder. [#312](https://github.com/klauspost/compress/pull/312) * Jan 7, 2021 (v1.11.6) * zstd: Make decoder allocations smaller [#306](https://github.com/klauspost/compress/pull/306) * zstd: Free Decoder resources when Reset is called with a nil io.Reader [#305](https://github.com/klauspost/compress/pull/305) * Dec 20, 2020 (v1.11.4) * zstd: Add Best compression mode [#304](https://github.com/klauspost/compress/pull/304) * Add header decoder [#299](https://github.com/klauspost/compress/pull/299) * s2: Add uncompressed stream option [#297](https://github.com/klauspost/compress/pull/297) * Simplify/speed up small blocks with known max size. [#300](https://github.com/klauspost/compress/pull/300) * zstd: Always reset literal dict encoder [#303](https://github.com/klauspost/compress/pull/303) * Nov 15, 2020 (v1.11.3) * inflate: 10-15% faster decompression [#293](https://github.com/klauspost/compress/pull/293) * zstd: Tweak DecodeAll default allocation [#295](https://github.com/klauspost/compress/pull/295) * Oct 11, 2020 (v1.11.2) * s2: Fix out of bounds read in "better" block compression [#291](https://github.com/klauspost/compress/pull/291) * Oct 1, 2020 (v1.11.1) * zstd: Set allLitEntropy true in default configuration [#286](https://github.com/klauspost/compress/pull/286) * Sept 8, 2020 (v1.11.0) * zstd: Add experimental compression [dictionaries](https://github.com/klauspost/compress/tree/master/zstd#dictionaries) [#281](https://github.com/klauspost/compress/pull/281) * zstd: Fix mixed Write and ReadFrom calls [#282](https://github.com/klauspost/compress/pull/282) * inflate/gz: Limit variable shifts, ~5% faster decompression [#274](https://github.com/klauspost/compress/pull/274)
See changes to v1.10.x * July 8, 2020 (v1.10.11) * zstd: Fix extra block when compressing with ReadFrom. [#278](https://github.com/klauspost/compress/pull/278) * huff0: Also populate compression table when reading decoding table. [#275](https://github.com/klauspost/compress/pull/275) * June 23, 2020 (v1.10.10) * zstd: Skip entropy compression in fastest mode when no matches. [#270](https://github.com/klauspost/compress/pull/270) * June 16, 2020 (v1.10.9): * zstd: API change for specifying dictionaries. See [#268](https://github.com/klauspost/compress/pull/268) * zip: update CreateHeaderRaw to handle zip64 fields. [#266](https://github.com/klauspost/compress/pull/266) * Fuzzit tests removed. The service has been purchased and is no longer available. * June 5, 2020 (v1.10.8): * 1.15x faster zstd block decompression. [#265](https://github.com/klauspost/compress/pull/265) * June 1, 2020 (v1.10.7): * Added zstd decompression [dictionary support](https://github.com/klauspost/compress/tree/master/zstd#dictionaries) * Increase zstd decompression speed up to 1.19x. [#259](https://github.com/klauspost/compress/pull/259) * Remove internal reset call in zstd compression and reduce allocations. [#263](https://github.com/klauspost/compress/pull/263) * May 21, 2020: (v1.10.6) * zstd: Reduce allocations while decoding. [#258](https://github.com/klauspost/compress/pull/258), [#252](https://github.com/klauspost/compress/pull/252) * zstd: Stricter decompression checks. * April 12, 2020: (v1.10.5) * s2-commands: Flush output when receiving SIGINT. [#239](https://github.com/klauspost/compress/pull/239) * Apr 8, 2020: (v1.10.4) * zstd: Minor/special case optimizations. [#251](https://github.com/klauspost/compress/pull/251), [#250](https://github.com/klauspost/compress/pull/250), [#249](https://github.com/klauspost/compress/pull/249), [#247](https://github.com/klauspost/compress/pull/247) * Mar 11, 2020: (v1.10.3) * s2: Use S2 encoder in pure Go mode for Snappy output as well. [#245](https://github.com/klauspost/compress/pull/245) * s2: Fix pure Go block encoder. [#244](https://github.com/klauspost/compress/pull/244) * zstd: Added "better compression" mode. [#240](https://github.com/klauspost/compress/pull/240) * zstd: Improve speed of fastest compression mode by 5-10% [#241](https://github.com/klauspost/compress/pull/241) * zstd: Skip creating encoders when not needed. [#238](https://github.com/klauspost/compress/pull/238) * Feb 27, 2020: (v1.10.2) * Close to 50% speedup in inflate (gzip/zip decompression). [#236](https://github.com/klauspost/compress/pull/236) [#234](https://github.com/klauspost/compress/pull/234) [#232](https://github.com/klauspost/compress/pull/232) * Reduce deflate level 1-6 memory usage up to 59%. [#227](https://github.com/klauspost/compress/pull/227) * Feb 18, 2020: (v1.10.1) * Fix zstd crash when resetting multiple times without sending data. [#226](https://github.com/klauspost/compress/pull/226) * deflate: Fix dictionary use on level 1-6. [#224](https://github.com/klauspost/compress/pull/224) * Remove deflate writer reference when closing. [#224](https://github.com/klauspost/compress/pull/224) * Feb 4, 2020: (v1.10.0) * Add optional dictionary to [stateless deflate](https://pkg.go.dev/github.com/klauspost/compress/flate?tab=doc#StatelessDeflate). Breaking change, send `nil` for previous behaviour. [#216](https://github.com/klauspost/compress/pull/216) * Fix buffer overflow on repeated small block deflate. [#218](https://github.com/klauspost/compress/pull/218) * Allow copying content from an existing ZIP file without decompressing+compressing. [#214](https://github.com/klauspost/compress/pull/214) * Added [S2](https://github.com/klauspost/compress/tree/master/s2#s2-compression) AMD64 assembler and various optimizations. Stream speed >10GB/s. [#186](https://github.com/klauspost/compress/pull/186)
See changes prior to v1.10.0 * Jan 20,2020 (v1.9.8) Optimize gzip/deflate with better size estimates and faster table generation. [#207](https://github.com/klauspost/compress/pull/207) by [luyu6056](https://github.com/luyu6056), [#206](https://github.com/klauspost/compress/pull/206). * Jan 11, 2020: S2 Encode/Decode will use provided buffer if capacity is big enough. [#204](https://github.com/klauspost/compress/pull/204) * Jan 5, 2020: (v1.9.7) Fix another zstd regression in v1.9.5 - v1.9.6 removed. * Jan 4, 2020: (v1.9.6) Regression in v1.9.5 fixed causing corrupt zstd encodes in rare cases. * Jan 4, 2020: Faster IO in [s2c + s2d commandline tools](https://github.com/klauspost/compress/tree/master/s2#commandline-tools) compression/decompression. [#192](https://github.com/klauspost/compress/pull/192) * Dec 29, 2019: Removed v1.9.5 since fuzz tests showed a compatibility problem with the reference zstandard decoder. * Dec 29, 2019: (v1.9.5) zstd: 10-20% faster block compression. [#199](https://github.com/klauspost/compress/pull/199) * Dec 29, 2019: [zip](https://godoc.org/github.com/klauspost/compress/zip) package updated with latest Go features * Dec 29, 2019: zstd: Single segment flag condintions tweaked. [#197](https://github.com/klauspost/compress/pull/197) * Dec 18, 2019: s2: Faster compression when ReadFrom is used. [#198](https://github.com/klauspost/compress/pull/198) * Dec 10, 2019: s2: Fix repeat length output when just above at 16MB limit. * Dec 10, 2019: zstd: Add function to get decoder as io.ReadCloser. [#191](https://github.com/klauspost/compress/pull/191) * Dec 3, 2019: (v1.9.4) S2: limit max repeat length. [#188](https://github.com/klauspost/compress/pull/188) * Dec 3, 2019: Add [WithNoEntropyCompression](https://godoc.org/github.com/klauspost/compress/zstd#WithNoEntropyCompression) to zstd [#187](https://github.com/klauspost/compress/pull/187) * Dec 3, 2019: Reduce memory use for tests. Check for leaked goroutines. * Nov 28, 2019 (v1.9.3) Less allocations in stateless deflate. * Nov 28, 2019: 5-20% Faster huff0 decode. Impacts zstd as well. [#184](https://github.com/klauspost/compress/pull/184) * Nov 12, 2019 (v1.9.2) Added [Stateless Compression](#stateless-compression) for gzip/deflate. * Nov 12, 2019: Fixed zstd decompression of large single blocks. [#180](https://github.com/klauspost/compress/pull/180) * Nov 11, 2019: Set default [s2c](https://github.com/klauspost/compress/tree/master/s2#commandline-tools) block size to 4MB. * Nov 11, 2019: Reduce inflate memory use by 1KB. * Nov 10, 2019: Less allocations in deflate bit writer. * Nov 10, 2019: Fix inconsistent error returned by zstd decoder. * Oct 28, 2019 (v1.9.1) ztsd: Fix crash when compressing blocks. [#174](https://github.com/klauspost/compress/pull/174) * Oct 24, 2019 (v1.9.0) zstd: Fix rare data corruption [#173](https://github.com/klauspost/compress/pull/173) * Oct 24, 2019 zstd: Fix huff0 out of buffer write [#171](https://github.com/klauspost/compress/pull/171) and always return errors [#172](https://github.com/klauspost/compress/pull/172) * Oct 10, 2019: Big deflate rewrite, 30-40% faster with better compression [#105](https://github.com/klauspost/compress/pull/105)
See changes prior to v1.9.0 * Oct 10, 2019: (v1.8.6) zstd: Allow partial reads to get flushed data. [#169](https://github.com/klauspost/compress/pull/169) * Oct 3, 2019: Fix inconsistent results on broken zstd streams. * Sep 25, 2019: Added `-rm` (remove source files) and `-q` (no output except errors) to `s2c` and `s2d` [commands](https://github.com/klauspost/compress/tree/master/s2#commandline-tools) * Sep 16, 2019: (v1.8.4) Add `s2c` and `s2d` [commandline tools](https://github.com/klauspost/compress/tree/master/s2#commandline-tools). * Sep 10, 2019: (v1.8.3) Fix s2 decoder [Skip](https://godoc.org/github.com/klauspost/compress/s2#Reader.Skip). * Sep 7, 2019: zstd: Added [WithWindowSize](https://godoc.org/github.com/klauspost/compress/zstd#WithWindowSize), contributed by [ianwilkes](https://github.com/ianwilkes). * Sep 5, 2019: (v1.8.2) Add [WithZeroFrames](https://godoc.org/github.com/klauspost/compress/zstd#WithZeroFrames) which adds full zero payload block encoding option. * Sep 5, 2019: Lazy initialization of zstandard predefined en/decoder tables. * Aug 26, 2019: (v1.8.1) S2: 1-2% compression increase in "better" compression mode. * Aug 26, 2019: zstd: Check maximum size of Huffman 1X compressed literals while decoding. * Aug 24, 2019: (v1.8.0) Added [S2 compression](https://github.com/klauspost/compress/tree/master/s2#s2-compression), a high performance replacement for Snappy. * Aug 21, 2019: (v1.7.6) Fixed minor issues found by fuzzer. One could lead to zstd not decompressing. * Aug 18, 2019: Add [fuzzit](https://fuzzit.dev/) continuous fuzzing. * Aug 14, 2019: zstd: Skip incompressible data 2x faster. [#147](https://github.com/klauspost/compress/pull/147) * Aug 4, 2019 (v1.7.5): Better literal compression. [#146](https://github.com/klauspost/compress/pull/146) * Aug 4, 2019: Faster zstd compression. [#143](https://github.com/klauspost/compress/pull/143) [#144](https://github.com/klauspost/compress/pull/144) * Aug 4, 2019: Faster zstd decompression. [#145](https://github.com/klauspost/compress/pull/145) [#143](https://github.com/klauspost/compress/pull/143) [#142](https://github.com/klauspost/compress/pull/142) * July 15, 2019 (v1.7.4): Fix double EOF block in rare cases on zstd encoder. * July 15, 2019 (v1.7.3): Minor speedup/compression increase in default zstd encoder. * July 14, 2019: zstd decoder: Fix decompression error on multiple uses with mixed content. * July 7, 2019 (v1.7.2): Snappy update, zstd decoder potential race fix. * June 17, 2019: zstd decompression bugfix. * June 17, 2019: fix 32 bit builds. * June 17, 2019: Easier use in modules (less dependencies). * June 9, 2019: New stronger "default" [zstd](https://github.com/klauspost/compress/tree/master/zstd#zstd) compression mode. Matches zstd default compression ratio. * June 5, 2019: 20-40% throughput in [zstandard](https://github.com/klauspost/compress/tree/master/zstd#zstd) compression and better compression. * June 5, 2019: deflate/gzip compression: Reduce memory usage of lower compression levels. * June 2, 2019: Added [zstandard](https://github.com/klauspost/compress/tree/master/zstd#zstd) compression! * May 25, 2019: deflate/gzip: 10% faster bit writer, mostly visible in lower levels. * Apr 22, 2019: [zstd](https://github.com/klauspost/compress/tree/master/zstd#zstd) decompression added. * Aug 1, 2018: Added [huff0 README](https://github.com/klauspost/compress/tree/master/huff0#huff0-entropy-compression). * Jul 8, 2018: Added [Performance Update 2018](#performance-update-2018) below. * Jun 23, 2018: Merged [Go 1.11 inflate optimizations](https://go-review.googlesource.com/c/go/+/102235). Go 1.9 is now required. Backwards compatible version tagged with [v1.3.0](https://github.com/klauspost/compress/releases/tag/v1.3.0). * Apr 2, 2018: Added [huff0](https://godoc.org/github.com/klauspost/compress/huff0) en/decoder. Experimental for now, API may change. * Mar 4, 2018: Added [FSE Entropy](https://godoc.org/github.com/klauspost/compress/fse) en/decoder. Experimental for now, API may change. * Nov 3, 2017: Add compression [Estimate](https://godoc.org/github.com/klauspost/compress#Estimate) function. * May 28, 2017: Reduce allocations when resetting decoder. * Apr 02, 2017: Change back to official crc32, since changes were merged in Go 1.7. * Jan 14, 2017: Reduce stack pressure due to array copies. See [Issue #18625](https://github.com/golang/go/issues/18625). * Oct 25, 2016: Level 2-4 have been rewritten and now offers significantly better performance than before. * Oct 20, 2016: Port zlib changes from Go 1.7 to fix zlib writer issue. Please update. * Oct 16, 2016: Go 1.7 changes merged. Apples to apples this package is a few percent faster, but has a significantly better balance between speed and compression per level. * Mar 24, 2016: Always attempt Huffman encoding on level 4-7. This improves base 64 encoded data compression. * Mar 24, 2016: Small speedup for level 1-3. * Feb 19, 2016: Faster bit writer, level -2 is 15% faster, level 1 is 4% faster. * Feb 19, 2016: Handle small payloads faster in level 1-3. * Feb 19, 2016: Added faster level 2 + 3 compression modes. * Feb 19, 2016: [Rebalanced compression levels](https://blog.klauspost.com/rebalancing-deflate-compression-levels/), so there is a more even progression in terms of compression. New default level is 5. * Feb 14, 2016: Snappy: Merge upstream changes. * Feb 14, 2016: Snappy: Fix aggressive skipping. * Feb 14, 2016: Snappy: Update benchmark. * Feb 13, 2016: Deflate: Fixed assembler problem that could lead to sub-optimal compression. * Feb 12, 2016: Snappy: Added AMD64 SSE 4.2 optimizations to matching, which makes easy to compress material run faster. Typical speedup is around 25%. * Feb 9, 2016: Added Snappy package fork. This version is 5-7% faster, much more on hard to compress content. * Jan 30, 2016: Optimize level 1 to 3 by not considering static dictionary or storing uncompressed. ~4-5% speedup. * Jan 16, 2016: Optimization on deflate level 1,2,3 compression. * Jan 8 2016: Merge [CL 18317](https://go-review.googlesource.com/#/c/18317): fix reading, writing of zip64 archives. * Dec 8 2015: Make level 1 and -2 deterministic even if write size differs. * Dec 8 2015: Split encoding functions, so hashing and matching can potentially be inlined. 1-3% faster on AMD64. 5% faster on other platforms. * Dec 8 2015: Fixed rare [one byte out-of bounds read](https://github.com/klauspost/compress/issues/20). Please update! * Nov 23 2015: Optimization on token writer. ~2-4% faster. Contributed by [@dsnet](https://github.com/dsnet). * Nov 20 2015: Small optimization to bit writer on 64 bit systems. * Nov 17 2015: Fixed out-of-bound errors if the underlying Writer returned an error. See [#15](https://github.com/klauspost/compress/issues/15). * Nov 12 2015: Added [io.WriterTo](https://golang.org/pkg/io/#WriterTo) support to gzip/inflate. * Nov 11 2015: Merged [CL 16669](https://go-review.googlesource.com/#/c/16669/4): archive/zip: enable overriding (de)compressors per file * Oct 15 2015: Added skipping on uncompressible data. Random data speed up >5x.
# deflate usage The packages are drop-in replacements for standard library [deflate](https://godoc.org/github.com/klauspost/compress/flate), [gzip](https://godoc.org/github.com/klauspost/compress/gzip), [zip](https://godoc.org/github.com/klauspost/compress/zip), and [zlib](https://godoc.org/github.com/klauspost/compress/zlib). Simply replace the import path to use them: Typical speed is about 2x of the standard library packages. | old import | new import | Documentation | |------------------|---------------------------------------|-------------------------------------------------------------------------| | `compress/gzip` | `github.com/klauspost/compress/gzip` | [gzip](https://pkg.go.dev/github.com/klauspost/compress/gzip?tab=doc) | | `compress/zlib` | `github.com/klauspost/compress/zlib` | [zlib](https://pkg.go.dev/github.com/klauspost/compress/zlib?tab=doc) | | `archive/zip` | `github.com/klauspost/compress/zip` | [zip](https://pkg.go.dev/github.com/klauspost/compress/zip?tab=doc) | | `compress/flate` | `github.com/klauspost/compress/flate` | [flate](https://pkg.go.dev/github.com/klauspost/compress/flate?tab=doc) | You may also be interested in [pgzip](https://github.com/klauspost/pgzip), which is a drop-in replacement for gzip, which support multithreaded compression on big files and the optimized [crc32](https://github.com/klauspost/crc32) package used by these packages. The packages implement the same API as the standard library, so you can use the original godoc documentation: [gzip](http://golang.org/pkg/compress/gzip/), [zip](http://golang.org/pkg/archive/zip/), [zlib](http://golang.org/pkg/compress/zlib/), [flate](http://golang.org/pkg/compress/flate/). Currently there is only minor speedup on decompression (mostly CRC32 calculation). Memory usage is typically 1MB for a Writer. stdlib is in the same range. If you expect to have a lot of concurrently allocated Writers consider using the stateless compression described below. For compression performance, see: [this spreadsheet](https://docs.google.com/spreadsheets/d/1nuNE2nPfuINCZJRMt6wFWhKpToF95I47XjSsc-1rbPQ/edit?usp=sharing). To disable all assembly add `-tags=noasm`. This works across all packages. # Stateless compression This package offers stateless compression as a special option for gzip/deflate. It will do compression but without maintaining any state between Write calls. This means there will be no memory kept between Write calls, but compression and speed will be suboptimal. This is only relevant in cases where you expect to run many thousands of compressors concurrently, but with very little activity. This is *not* intended for regular web servers serving individual requests. Because of this, the size of actual Write calls will affect output size. In gzip, specify level `-3` / `gzip.StatelessCompression` to enable. For direct deflate use, NewStatelessWriter and StatelessDeflate are available. See [documentation](https://godoc.org/github.com/klauspost/compress/flate#NewStatelessWriter) A `bufio.Writer` can of course be used to control write sizes. For example, to use a 4KB buffer: ```go // replace 'ioutil.Discard' with your output. gzw, err := gzip.NewWriterLevel(ioutil.Discard, gzip.StatelessCompression) if err != nil { return err } defer gzw.Close() w := bufio.NewWriterSize(gzw, 4096) defer w.Flush() // Write to 'w' ``` This will only use up to 4KB in memory when the writer is idle. Compression is almost always worse than the fastest compression level and each write will allocate (a little) memory. # Other packages Here are other packages of good quality and pure Go (no cgo wrappers or autoconverted code): * [github.com/pierrec/lz4](https://github.com/pierrec/lz4) - strong multithreaded LZ4 compression. * [github.com/cosnicolaou/pbzip2](https://github.com/cosnicolaou/pbzip2) - multithreaded bzip2 decompression. * [github.com/dsnet/compress](https://github.com/dsnet/compress) - brotli decompression, bzip2 writer. * [github.com/ronanh/intcomp](https://github.com/ronanh/intcomp) - Integer compression. * [github.com/spenczar/fpc](https://github.com/spenczar/fpc) - Float compression. * [github.com/minio/zipindex](https://github.com/minio/zipindex) - External ZIP directory index. * [github.com/ybirader/pzip](https://github.com/ybirader/pzip) - Fast concurrent zip archiver and extractor. # license This code is licensed under the same conditions as the original Go code. See LICENSE file. --- ### Overview (`vendor/github.com/mdlayher/socket/README.md`) # socket [](https://github.com/mdlayher/socket/actions) [](https://pkg.go.dev/github.com/mdlayher/socket) Package `socket` provides a low-level network connection type which integrates with Go's runtime network poller to provide asynchronous I/O and deadline support. MIT Licensed. This package focuses on UNIX-like operating systems which make use of BSD sockets system call APIs. It is meant to be used as a foundation for the creation of operating system-specific socket packages, for socket families such as Linux's `AF_NETLINK`, `AF_PACKET`, or `AF_VSOCK`. This package should not be used directly in end user applications. Any use of package socket should be guarded by build tags, as one would also use when importing the `syscall` or `golang.org/x/sys` packages. ## Ecosystem Over time, an ecosystem of Go packages has developed around package `socket`. Many of these packages provide access to a specific socket family, such as `AF_NETLINK` or `AF_PACKET`, and act as building blocks for further packages. To have your package included in this diagram, please send a pull request! ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Stability See the [CHANGELOG](./CHANGELOG.md) file for a description of changes between releases. This package only supports the two most recent major versions of Go, mirroring Go's own release policy. Older versions of Go may lack critical features and bug fixes which are necessary for this package to function correctly. --- ### Overview (`vendor/github.com/mdlayher/vsock/README.md`) # vsock [](https://github.com/mdlayher/vsock/actions) [](https://pkg.go.dev/github.com/mdlayher/vsock) [](https://goreportcard.com/report/github.com/mdlayher/vsock) Package `vsock` provides access to Linux VM sockets (`AF_VSOCK`) for communication between a hypervisor and its virtual machines. MIT Licensed. For more information about VM sockets, see my blog about [Linux VM sockets in Go](https://mdlayher.com/blog/linux-vm-sockets-in-go/) or the [QEMU wiki page on virtio-vsock](http://wiki.qemu-project.org/Features/VirtioVsock). ## Stability See the [CHANGELOG](./CHANGELOG.md) file for a description of changes between releases. This package has a stable v1 API and any future breaking changes will prompt the release of a new major version. Features and bug fixes will continue to occur in the v1.x.x series. This package only supports the two most recent major versions of Go, mirroring Go's own release policy. Older versions of Go may lack critical features and bug fixes which are necessary for this package to function correctly. --- ### Overview (`vendor/github.com/miekg/dns/README.md`) [](https://travis-ci.org/miekg/dns) [](https://codecov.io/github/miekg/dns?branch=master) [](https://goreportcard.com/report/miekg/dns) [](https://godoc.org/github.com/miekg/dns) DNS version 2 is now available at . This version should be 2x faster across the board. Further optimizations welcome. The version here will only see specific fixes and nothing else. At some point this repo will be archived. # Alternative (more granular) approach to a DNS library > Less is more. Complete and usable DNS library. All Resource Records are supported, including the DNSSEC types. It follows a lean and mean philosophy. If there is stuff you should know as a DNS programmer there isn't a convenience function for it. Server side and client side programming is supported, i.e. you can build servers and resolvers with it. We try to keep the "master" branch as sane as possible and at the bleeding edge of standards, avoiding breaking changes wherever reasonable. We support the last two versions of Go. # Goals - KISS; - Fast; - Small API. If it's easy to code in Go, don't make a function for it. # Users A not-so-up-to-date-list-that-may-be-actually-current: - https://github.com/coredns/coredns - https://github.com/abh/geodns - https://github.com/baidu/bfe - http://www.statdns.com/ - http://www.dnsinspect.com/ - https://github.com/chuangbo/jianbing-dictionary-dns - http://www.dns-lg.com/ - https://github.com/fcambus/rrda - https://github.com/kenshinx/godns - https://github.com/skynetservices/skydns - https://github.com/hashicorp/consul - https://github.com/DevelopersPL/godnsagent - https://github.com/duedil-ltd/discodns - https://github.com/StalkR/dns-reverse-proxy - https://github.com/tianon/rawdns - https://mesosphere.github.io/mesos-dns/ - https://github.com/fcambus/statzone - https://github.com/benschw/dns-clb-go - https://github.com/corny/dnscheck for - https://github.com/miekg/unbound - https://github.com/miekg/exdns - https://dnslookup.org - https://github.com/looterz/grimd - https://github.com/phamhongviet/serf-dns - https://github.com/mehrdadrad/mylg - https://github.com/bamarni/dockness - https://github.com/fffaraz/microdns - https://github.com/ipdcode/hades - https://github.com/StackExchange/dnscontrol/ - https://www.dnsperf.com/ - https://dnssectest.net/ - https://github.com/oif/apex - https://github.com/jedisct1/dnscrypt-proxy (migrated to v2) - https://github.com/jedisct1/rpdns - https://github.com/xor-gate/sshfp - https://github.com/rs/dnstrace - https://blitiri.com.ar/p/dnss ([github mirror](https://github.com/albertito/dnss)) - https://render.com - https://github.com/peterzen/goresolver - https://github.com/folbricht/routedns - https://domainr.com/ - https://zonedb.org/ - https://router7.org/ - https://github.com/fortio/dnsping - https://github.com/Luzilla/dnsbl_exporter - https://github.com/bodgit/tsig - https://github.com/v2fly/v2ray-core (test only) - https://kuma.io/ - https://www.misaka.io/services/dns - https://ping.sx/dig - https://fleetdeck.io/ - https://github.com/markdingo/autoreverse - https://github.com/slackhq/nebula - https://addr.tools/ - https://dnscheck.tools/ - https://github.com/egbakou/domainverifier - https://github.com/semihalev/sdns - https://github.com/wintbiit/NineDNS - https://linuxcontainers.org/incus/ - https://ifconfig.es - https://github.com/zmap/zdns - https://framagit.org/bortzmeyer/check-soa - https://github.com/jkerdreux-imt/owns Send pull request if you want to be listed here. # Features - UDP/TCP queries, IPv4 and IPv6 - RFC 1035 zone file parsing ($INCLUDE, $ORIGIN, $TTL and $GENERATE (for all record types) are supported - Fast - Server side programming (mimicking the net/http package) - Client side programming - DNSSEC: signing, validating and key generation for DSA, RSA, ECDSA and Ed25519 - EDNS0, NSID, Cookies - AXFR/IXFR - TSIG, SIG(0) - DNS over TLS (DoT): encrypted connection between client and server over TCP - DNS name compression Have fun! Miek Gieben - 2010-2012 - DNS Authors 2012- # Building This library uses Go modules and uses semantic versioning. Building is done with the `go` tool, so the following should work: go get github.com/miekg/dns go build github.com/miekg/dns ## Examples A short "how to use the API" is at the beginning of doc.go (this also will show when you call `godoc github.com/miekg/dns`). Example programs can be found in the `github.com/miekg/exdns` repository. ## Supported RFCs _all of them_ - 103{4,5} - DNS standard - 1183 - ISDN, X25 and other deprecated records - 1348 - NSAP record (removed the record) - 1982 - Serial Arithmetic - 1876 - LOC record - 1995 - IXFR - 1996 - DNS notify - 2136 - DNS Update (dynamic updates) - 2181 - RRset definition - there is no RRset type though, just []RR - 2537 - RSAMD5 DNS keys - 2065 - DNSSEC (updated in later RFCs) - 2671 - EDNS record - 2782 - SRV record - 2845 - TSIG record - 2915 - NAPTR record - 2929 - DNS IANA Considerations - 3110 - RSASHA1 DNS keys - 3123 - APL record - 3225 - DO bit (DNSSEC OK) - 340{1,2,3} - NAPTR record - 3445 - Limiting the scope of (DNS)KEY - 3596 - AAAA record - 3597 - Unknown RRs - 4025 - A Method for Storing IPsec Keying Material in DNS - 403{3,4,5} - DNSSEC + validation functions - 4255 - SSHFP record - 4343 - Case insensitivity - 4408 - SPF record - 4509 - SHA256 Hash in DS - 4592 - Wildcards in the DNS - 4635 - HMAC SHA TSIG - 4701 - DHCID - 4892 - id.server - 5001 - NSID - 5155 - NSEC3 record - 5205 - HIP record - 5702 - SHA2 in the DNS - 5936 - AXFR - 5966 - TCP implementation recommendations - 6605 - ECDSA - 6725 - IANA Registry Update - 6742 - ILNP DNS - 6840 - Clarifications and Implementation Notes for DNS Security - 6844 - CAA record - 6891 - EDNS0 update - 6895 - DNS IANA considerations - 6944 - DNSSEC DNSKEY Algorithm Status - 6975 - Algorithm Understanding in DNSSEC - 7043 - EUI48/EUI64 records - 7314 - DNS (EDNS) EXPIRE Option - 7477 - CSYNC RR - 7828 - edns-tcp-keepalive EDNS0 Option - 7553 - URI record - 7858 - DNS over TLS: Initiation and Performance Considerations - 7871 - EDNS0 Client Subnet - 7873 - Domain Name System (DNS) Cookies - 8080 - EdDSA for DNSSEC - 8490 - DNS Stateful Operations - 8499 - DNS Terminology - 8659 - DNS Certification Authority Authorization (CAA) Resource Record - 8777 - DNS Reverse IP Automatic Multicast Tunneling (AMT) Discovery - 8914 - Extended DNS Errors - 8976 - Message Digest for DNS Zones (ZONEMD RR) - 9460 - Service Binding and Parameter Specification via the DNS - 9461 - Service Binding Mapping for DNS Servers - 9462 - Discovery of Designated Resolvers - 9460 - SVCB and HTTPS Records - 9567 - DNS Error Reporting - 9606 - DNS Resolver Information - 9660 - DNS Zone Version (ZONEVERSION) Option - Draft - Compact Denial of Existence in DNSSEC ## Loosely Based Upon - ldns - - NSD - - Net::DNS - - GRONG - --- ### Overview (`vendor/github.com/mitchellh/copystructure/README.md`) # copystructure copystructure is a Go library for deep copying values in Go. This allows you to copy Go values that may contain reference values such as maps, slices, or pointers, and copy their data as well instead of just their references. ## Installation Standard `go get`: ``` $ go get github.com/mitchellh/copystructure ``` ## Usage & Example For usage and examples see the [Godoc](http://godoc.org/github.com/mitchellh/copystructure). The `Copy` function has examples associated with it there. --- ### Overview (`vendor/github.com/mitchellh/reflectwalk/README.md`) # reflectwalk reflectwalk is a Go library for "walking" a value in Go using reflection, in the same way a directory tree can be "walked" on the filesystem. Walking a complex structure can allow you to do manipulations on unknown structures such as those decoded from JSON. --- ### Overview (`vendor/github.com/moby/extensions/README.md`) # Extensions > [!WARNING] > This project is still a work in progress. Moby extensions provide a common model for extending engine behavior. They are intended to replace the separate plugin systems currently used by network, volume, and logging drivers. An extension can run inside the Host process or as a separate process. In both cases, it implements the same Go interface. See the [authoring guide](./docs/AUTHORING.md) for examples and setup instructions. For details about the architecture, wire protocol, current limitations, and discovery security, see the [design guide](./docs/DESIGN.md). ## How it works Extensions are built around Points. A Point is an interface that describes a capability, such as running a workload. Providers implement Points, and consumers call them. The Host registers these components, connects consumers to providers, and manages their lifecycle. For example, a daemon can expose its workload runtime as a Point. An extension can then use that capability without depending on the daemon's internal runtime implementation. The same mechanism is used for daemon-owned capabilities and communication between extensions. To add an extension: 1. **Define the contract.** A Point assigns a namespaced, versioned identity to a Go interface and its message types. 2. **Declare what the extension provides and depends on.** Providers implement Points. Consumers declare their dependencies using `Point.Dependency()`. 3. **Register the extension with the Host.** The daemon or application decides which extensions to load and supplies any generated wiring required for cross-process calls. Merely importing an extension package does not activate it. 4. **Resolve dependencies during initialization.** In `Declaration.Init`, consumers resolve their declared dependencies using `Point.Single`, `Point.All`, or `Point.ByExtension`. The resolved implementation is not automatically assigned to a field. 5. **Call the Point interface.** Consumers retain and call the resolved interface rather than a concrete Host or daemon type. Calls go directly to an in-process provider or through generated gRPC code when the provider runs in another process. The Host initializes providers before their consumers and shuts them down in the opposite order. A provider must be ready for use when its `Init` method returns. Keep the Host returned by `host.New` and call `Shutdown` when the application exits so that lifecycle resources are released. Breaking changes require a new Point version. The `.v0` suffix identifies an experimental contract. For fan-out calls, provider order is unspecified unless the Point defines its own ordering rules. ## Running extensions A Point defines an interface, not where its implementation runs. In-process consumers call the Go implementation directly. Cross-process consumers use generated gRPC wiring. Because the same contract may cross a process boundary, Point interfaces and messages should not contain Host types, internal backend structures, callbacks, or channels. The `mobyextgen` generator creates the required wire code: * `ServerPoint` exposes a Point implementation over gRPC. * `ClientPoint` presents the remote implementation through the same typed Go interface. The exact setup depends on where the consumer and provider run: * Register in-process extensions with `host.WithExtensions`. * When the Host calls a provider in a launched process, configure the Host with `host.WithClientProviders` and the generated `ClientPoint`. The provider registers its generated `ServerPoint` with the SDK. * When a launched consumer calls a Host provider, configure the Host with `host.WithDependencyProviders` and the generated `ServerPoint`. In the consumer, pass the generated `ClientPoint` to `Server.Depends`. The last case exposes an existing Host provider to the launched process. The consumer must still declare the dependency itself. A dependency callback currently exposes one effective provider for each Point. Do not retain and call that dependency during `Shutdown`, because the SDK closes the connection before shutdown begins. For launched processes, the `ready\n` handshake only confirms that the listener is ready. It does not mean that the extension's `Init` method has completed. See the [authoring guide](./docs/AUTHORING.md) for complete examples covering code generation, SDK registration, dependency resolution, and deployment. ## Publishing services Implementing a Point makes it available within the extension system, but does not automatically expose it to external callers. Publishing a Point requires three things: * The extension offers it using `servicev0.Offer`. * Host policy permits it through `host.WithProviderPolicy`. * The generated server wiring is registered. For an in-process provider, supply the server wiring through `host.WithPointServers`. A provider running in a separate process registers its `ServerPoint` with the SDK. An offer is only a request to publish a Point; it does not grant access or register the transport. If no provider policy is configured, external publication is disabled, while providers remain available to internal consumers. Published services are responsible for enforcing their own access control. See the [publication examples](./docs/AUTHORING.md#publishing-an-ordinary-point) for complete configurations and generated client usage. ## Glossary ### Point A named and versioned Go interface, together with its message types. It describes a capability independently of its implementation or location. ### Extension A component that can provide Points, depend on other Points or extensions, and perform initialization and shutdown work. It may run inside the Host or in a separate process. ### Host The framework component embedded in a daemon or application. It registers extensions, resolves their dependencies, and manages their lifecycle. ### Provider An extension that implements a Point. ### Consumer Code that resolves and calls a Point. An extension may be both a consumer and a provider. ### Dependency A declared requirement for a Point provider or another extension. The Host uses dependencies to verify that requirements are available and to determine initialization order. ### Host capability Functionality owned by the daemon or application and exposed as a Point through an in-process extension. ### ClientPoint / ServerPoint Generated code used to call a Point across a process boundary. `ClientPoint` presents the Point's Go interface to the consumer, while `ServerPoint` forwards incoming calls to the implementation. ### Offer A request from an extension to make one or more of its Points available to external callers. ### Publication The Host exposing an offered Point to external callers after applying its policy and registering the required server wiring. ## License Licensed under the [Apache License, Version 2.0](LICENSE). --- ### Overview (`vendor/github.com/moby/ipvs/README.md`) # ipvs - networking for containers [](https://godoc.org/github.com/moby/ipvs) [](https://goreportcard.com/report/github.com/moby/ipvs) ipvs provides a native Go implementation for communicating with IPVS kernel module using a netlink socket. #### Using ipvs ```go import ( "log" "github.com/moby/ipvs" ) func main() { handle, err := ipvs.New("") if err != nil { log.Fatalf("ipvs.New: %s", err) } svcs, err := handle.GetServices() if err != nil { log.Fatalf("handle.GetServices: %s", err) } } ``` ## Contributing Want to hack on ipvs? [Docker's contributions guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md) apply. ## Copyright and license Copyright 2015 Docker, inc. Code released under the [Apache 2.0 license](LICENSE). --- ### Overview (`vendor/github.com/moby/locker/README.md`) Locker ===== locker provides a mechanism for creating finer-grained locking to help free up more global locks to handle other tasks. The implementation looks close to a sync.Mutex, however, the user must provide a reference to use to refer to the underlying lock when locking and unlocking, and unlock may generate an error. If a lock with a given name does not exist when `Lock` is called, one is created. Lock references are automatically cleaned up on `Unlock` if nothing else is waiting for the lock. ## Usage ```go package important import ( "sync" "time" "github.com/moby/locker" ) type important struct { locks *locker.Locker data map[string]interface{} mu sync.Mutex } func (i *important) Get(name string) interface{} { i.locks.Lock(name) defer i.locks.Unlock(name) return i.data[name] } func (i *important) Create(name string, data interface{}) { i.locks.Lock(name) defer i.locks.Unlock(name) i.createImportant(data) i.mu.Lock() i.data[name] = data i.mu.Unlock() } func (i *important) createImportant(data interface{}) { time.Sleep(10 * time.Second) } ``` For functions dealing with a given name, always lock at the beginning of the function (or before doing anything with the underlying state), this ensures any other function that is dealing with the same name will block. When needing to modify the underlying data, use the global lock to ensure nothing else is modifying it at the same time. Since name lock is already in place, no reads will occur while the modification is being performed. --- ### Overview (`vendor/github.com/moby/policy-helpers/README.md`) Work in progress policy helpers for Moby & BuildKit. --- ### Overview (`vendor/github.com/moby/term/README.md`) # term - utilities for dealing with terminals [](https://godoc.org/github.com/moby/term) [](https://goreportcard.com/report/github.com/moby/term) term provides structures and helper functions to work with terminal (state, sizes). #### Using term ```go package main import ( "log" "os" "github.com/moby/term" ) func main() { fd := os.Stdin.Fd() if term.IsTerminal(fd) { ws, err := term.GetWinsize(fd) if err != nil { log.Fatalf("term.GetWinsize: %s", err) } log.Printf("%d:%d\n", ws.Height, ws.Width) } } ``` ## Contributing Want to hack on term? [Docker's contributions guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md) apply. ## Copyright and license Code and documentation copyright 2015 Docker, inc. Code released under the Apache 2.0 license. Docs released under Creative commons. --- ### Overview (`vendor/github.com/morikuni/aec/README.md`) # aec [](https://godoc.org/github.com/morikuni/aec) Go wrapper for ANSI escape code. ## Install ```bash go get github.com/morikuni/aec ``` ## Features ANSI escape codes depend on terminal environment. Some of these features may not work. Check supported Font-Style/Font-Color features with [checkansi](./checkansi). [Wikipedia](https://en.wikipedia.org/wiki/ANSI_escape_code) for more detail. ### Cursor - `Up(n)` - `Down(n)` - `Right(n)` - `Left(n)` - `NextLine(n)` - `PreviousLine(n)` - `Column(col)` - `Position(row, col)` - `Save` - `Restore` - `Hide` - `Show` - `Report` ### Erase - `EraseDisplay(mode)` - `EraseLine(mode)` ### Scroll - `ScrollUp(n)` - `ScrollDown(n)` ### Font Style - `Bold` - `Faint` - `Italic` - `Underline` - `BlinkSlow` - `BlinkRapid` - `Inverse` - `Conceal` - `CrossOut` - `Frame` - `Encircle` - `Overline` ### Font Color Foreground color. - `DefaultF` - `BlackF` - `RedF` - `GreenF` - `YellowF` - `BlueF` - `MagentaF` - `CyanF` - `WhiteF` - `LightBlackF` - `LightRedF` - `LightGreenF` - `LightYellowF` - `LightBlueF` - `LightMagentaF` - `LightCyanF` - `LightWhiteF` - `Color3BitF(color)` - `Color8BitF(color)` - `FullColorF(r, g, b)` Background color. - `DefaultB` - `BlackB` - `RedB` - `GreenB` - `YellowB` - `BlueB` - `MagentaB` - `CyanB` - `WhiteB` - `LightBlackB` - `LightRedB` - `LightGreenB` - `LightYellowB` - `LightBlueB` - `LightMagentaB` - `LightCyanB` - `LightWhiteB` - `Color3BitB(color)` - `Color8BitB(color)` - `FullColorB(r, g, b)` ### Color Converter 24bit RGB color to ANSI color. - `NewRGB3Bit(r, g, b)` - `NewRGB8Bit(r, g, b)` ### Builder To mix these features. ```go custom := aec.EmptyBuilder.Right(2).RGB8BitF(128, 255, 64).RedB().ANSI custom.Apply("Hello World") ``` ## Usage 1. Create ANSI by `aec.XXX().With(aec.YYY())` or `aec.EmptyBuilder.XXX().YYY().ANSI` 2. Print ANSI by `fmt.Print(ansi, "some string", aec.Reset)` or `fmt.Print(ansi.Apply("some string"))` `aec.Reset` should be added when using font style or font color features. ## Example Simple progressbar. ```go package main import ( "fmt" "strings" "time" "github.com/morikuni/aec" ) func main() { const n = 20 builder := aec.EmptyBuilder up2 := aec.Up(2) col := aec.Column(n + 2) bar := aec.Color8BitF(aec.NewRGB8Bit(64, 255, 64)) label := builder.LightRedF().Underline().With(col).Right(1).ANSI // for up2 fmt.Println() fmt.Println() for i := 0; i <= n; i++ { fmt.Print(up2) fmt.Println(label.Apply(fmt.Sprint(i, "/", n))) fmt.Print("[") fmt.Print(bar.Apply(strings.Repeat("=", i))) fmt.Println(col.Apply("]")) time.Sleep(100 * time.Millisecond) } } ``` ## License [MIT](./LICENSE) --- ### Overview (`vendor/github.com/opencontainers/cgroups/README.md`) # OCI Project Template Useful boilerplate and organizational information for all OCI projects. * README (this file) * [The Apache License, Version 2.0](LICENSE) * [A list of maintainers](MAINTAINERS) * [Maintainer guidelines](MAINTAINERS_GUIDE.md) * [Contributor guidelines](CONTRIBUTING.md) * [Project governance](GOVERNANCE.md) * [Release procedures](RELEASES.md) --- ### Overview (`vendor/github.com/opencontainers/go-digest/README.md`) # go-digest [](https://godoc.org/github.com/opencontainers/go-digest) [](https://goreportcard.com/report/github.com/opencontainers/go-digest) [](https://travis-ci.org/opencontainers/go-digest) Common digest package used across the container ecosystem. Please see the [godoc](https://godoc.org/github.com/opencontainers/go-digest) for more information. # What is a digest? A digest is just a [hash](https://en.wikipedia.org/wiki/Hash_function). The most common use case for a digest is to create a content identifier for use in [Content Addressable Storage](https://en.wikipedia.org/wiki/Content-addressable_storage) systems: ```go id := digest.FromBytes([]byte("my content")) ``` In the example above, the id can be used to uniquely identify the byte slice "my content". This allows two disparate applications to agree on a verifiable identifier without having to trust one another. An identifying digest can be verified, as follows: ```go if id != digest.FromBytes([]byte("my content")) { return errors.New("the content has changed!") } ``` A `Verifier` type can be used to handle cases where an `io.Reader` makes more sense: ```go rd := getContent() verifier := id.Verifier() io.Copy(verifier, rd) if !verifier.Verified() { return errors.New("the content has changed!") } ``` Using [Merkle DAGs](https://en.wikipedia.org/wiki/Merkle_tree), this can power a rich, safe, content distribution system. # Usage While the [godoc](https://godoc.org/github.com/opencontainers/go-digest) is considered the best resource, a few important items need to be called out when using this package. 1. Make sure to import the hash implementations into your application or the package will panic. You should have something like the following in the main (or other entrypoint) of your application: ```go import ( _ "crypto/sha256" _ "crypto/sha512" ) ``` This may seem inconvenient but it allows you replace the hash implementations with others, such as https://github.com/stevvooe/resumable. 2. Even though `digest.Digest` may be assemblable as a string, _always_ verify your input with `digest.Parse` or use `Digest.Validate` when accepting untrusted input. While there are measures to avoid common problems, this will ensure you have valid digests in the rest of your application. 3. While alternative encodings of hash values (digests) are possible (for example, base64), this package deals exclusively with hex-encoded digests. # Stability The Go API, at this stage, is considered stable, unless otherwise noted. As always, before using a package export, read the [godoc](https://godoc.org/github.com/opencontainers/go-digest). # Contributing This package is considered fairly complete. It has been in production in thousands (millions?) of deployments and is fairly battle-hardened. New additions will be met with skepticism. If you think there is a missing feature, please file a bug clearly describing the problem and the alternatives you tried before submitting a PR. ## Code of Conduct Participation in the OpenContainers community is governed by [OpenContainer's Code of Conduct][code-of-conduct]. ## Security If you find an issue, please follow the [security][security] protocol to report it. # Copyright and license Copyright © 2019, 2020 OCI Contributors Copyright © 2016 Docker, Inc. All rights reserved, except as follows. Code is released under the [Apache 2.0 license](LICENSE). This `README.md` file and the [`CONTRIBUTING.md`](CONTRIBUTING.md) file are licensed under the Creative Commons Attribution 4.0 International License under the terms and conditions set forth in the file [`LICENSE.docs`](LICENSE.docs). You may obtain a duplicate copy of the same license, titled CC BY-SA 4.0, at http://creativecommons.org/licenses/by-sa/4.0/. [security]: https://github.com/opencontainers/org/blob/master/security [code-of-conduct]: https://github.com/opencontainers/org/blob/master/CODE_OF_CONDUCT.md --- ### Overview (`vendor/github.com/package-url/packageurl-go/README.md`) # packageurl-go [](https://github.com/package-url/packageurl-go/actions?query=workflow%3Atest) [](https://coveralls.io/github/package-url/packageurl-go) [](https://pkg.go.dev/github.com/package-url/packageurl-go) [](https://goreportcard.com/report/github.com/package-url/packageurl-go) Go implementation of the package url spec. ## Install ``` go get -u github.com/package-url/packageurl-go ``` ## Versioning The versions will follow the spec. So if the spec is released at ``1.0``. Then all versions in the ``1.x.y`` will follow the ``1.x`` spec. ## Usage ### Create from parts ```go package main import ( "fmt" "github.com/package-url/packageurl-go" ) func main() { instance := packageurl.NewPackageURL("test", "ok", "name", "version", nil, "") fmt.Printf("%s", instance.ToString()) } ``` ### Parse from string ```go package main import ( "fmt" "github.com/package-url/packageurl-go" ) func main() { instance, err := packageurl.FromString("test:ok/name@version") if err != nil { panic(err) } fmt.Printf("%#v", instance) } ``` ## Test Testing using the normal ``go test`` command. Using ``make test`` will pull the test fixtures shared between all package-url projects and then execute the tests. ``` curl -Ls https://raw.githubusercontent.com/package-url/purl-spec/master/test-suite-data.json -o testdata/test-suite-data.json go test -v -cover ./... === RUN TestFromStringExamples --- PASS: TestFromStringExamples (0.00s) === RUN TestToStringExamples --- PASS: TestToStringExamples (0.00s) === RUN TestStringer --- PASS: TestStringer (0.00s) === RUN TestQualifiersMapConversion --- PASS: TestQualifiersMapConversion (0.00s) PASS github.com/package-url/packageurl-go coverage: 90.7% of statements ok github.com/package-url/packageurl-go 0.004s coverage: 90.7% of statements ``` ## Fuzzing Fuzzing is done with standard [Go fuzzing](https://go.dev/doc/fuzz/), introduced in Go 1.18. Fuzz tests check for inputs that cause `FromString` to panic. Using `make fuzz` will run fuzz tests for one minute. To run fuzz tests longer: ``` go test -fuzztime=60m -fuzz . ``` Or omit `-fuzztime` entirely to run indefinitely. --- ### Overview (`vendor/github.com/pelletier/go-toml/README.md`) # go-toml Go library for the [TOML](https://toml.io/) format. This library supports TOML version [v1.0.0-rc.3](https://toml.io/en/v1.0.0-rc.3) [](https://pkg.go.dev/github.com/pelletier/go-toml) [](https://github.com/pelletier/go-toml/blob/master/LICENSE) [](https://dev.azure.com/pelletierthomas/go-toml-ci/_build/latest?definitionId=1&branchName=master) [](https://codecov.io/gh/pelletier/go-toml) [](https://goreportcard.com/report/github.com/pelletier/go-toml) [](https://app.fossa.io/projects/git%2Bgithub.com%2Fpelletier%2Fgo-toml?ref=badge_shield) ## Development status **ℹ️ Consider go-toml v2!** The next version of go-toml is in [active development][v2-dev], and [nearing completion][v2-map]. Though technically in beta, v2 is already more tested, [fixes bugs][v1-bugs], and [much faster][v2-bench]. If you only need reading and writing TOML documents (majority of cases), those features are implemented and the API unlikely to change. The remaining features will be added shortly. While pull-requests are welcome on v1, no active development is expected on it. When v2.0.0 is released, v1 will be deprecated. 👉 [go-toml v2][v2] [v2]: https://github.com/pelletier/go-toml/tree/v2 [v2-map]: https://github.com/pelletier/go-toml/discussions/506 [v2-dev]: https://github.com/pelletier/go-toml/tree/v2 [v1-bugs]: https://github.com/pelletier/go-toml/issues?q=is%3Aissue+is%3Aopen+label%3Av2-fixed [v2-bench]: https://github.com/pelletier/go-toml/tree/v2#benchmarks ## Features Go-toml provides the following features for using data parsed from TOML documents: * Load TOML documents from files and string data * Easily navigate TOML structure using Tree * Marshaling and unmarshaling to and from data structures * Line & column position data for all parsed elements * [Query support similar to JSON-Path](query/) * Syntax errors contain line and column numbers ## Import ```go import "github.com/pelletier/go-toml" ``` ## Usage example Read a TOML document: ```go config, _ := toml.Load(` [postgres] user = "pelletier" password = "mypassword"`) // retrieve data directly user := config.Get("postgres.user").(string) // or using an intermediate object postgresConfig := config.Get("postgres").(*toml.Tree) password := postgresConfig.Get("password").(string) ``` Or use Unmarshal: ```go type Postgres struct { User string Password string } type Config struct { Postgres Postgres } doc := []byte(` [Postgres] User = "pelletier" Password = "mypassword"`) config := Config{} toml.Unmarshal(doc, &config) fmt.Println("user=", config.Postgres.User) ``` Or use a query: ```go // use a query to gather elements without walking the tree q, _ := query.Compile("$..[user,password]") results := q.Execute(config) for ii, item := range results.Values() { fmt.Printf("Query result %d: %v\n", ii, item) } ``` ## Documentation The documentation and additional examples are available at [pkg.go.dev](https://pkg.go.dev/github.com/pelletier/go-toml). ## Tools Go-toml provides three handy command line tools: * `tomll`: Reads TOML files and lints them. ``` go install github.com/pelletier/go-toml/cmd/tomll tomll --help ``` * `tomljson`: Reads a TOML file and outputs its JSON representation. ``` go install github.com/pelletier/go-toml/cmd/tomljson tomljson --help ``` * `jsontoml`: Reads a JSON file and outputs a TOML representation. ``` go install github.com/pelletier/go-toml/cmd/jsontoml jsontoml --help ``` ### Docker image Those tools are also available as a Docker image from [dockerhub](https://hub.docker.com/r/pelletier/go-toml). For example, to use `tomljson`: ``` docker run -v $PWD:/workdir pelletier/go-toml tomljson /workdir/example.toml ``` Only master (`latest`) and tagged versions are published to dockerhub. You can build your own image as usual: ``` docker build -t go-toml . ``` ## Contribute Feel free to report bugs and patches using GitHub's pull requests system on [pelletier/go-toml](https://github.com/pelletier/go-toml). Any feedback would be much appreciated! ### Run tests `go test ./...` ### Fuzzing The script `./fuzz.sh` is available to run [go-fuzz](https://github.com/dvyukov/go-fuzz) on go-toml. ## Versioning Go-toml follows [Semantic Versioning](http://semver.org/). The supported version of [TOML](https://github.com/toml-lang/toml) is indicated at the beginning of this document. The last two major versions of Go are supported (see [Go Release Policy](https://golang.org/doc/devel/release.html#policy)). ## License The MIT License (MIT) + Apache 2.0. Read [LICENSE](LICENSE). --- --- METRICS --- - Files Extracted: 115 - Estimated Token Budget: ~125683 tokens - Canonical Reference: https://codewiki.google/github.com/docker/Docker