## File: README.md # `ko`: Easy Go Containers [](https://github.com/ko-build/ko/actions?query=workflow%3ABuild) [](https://godoc.org/github.com/google/ko) [](https://goreportcard.com/report/ko-build/ko) [](https://slsa.dev/images/gh-badge-level3.svg) --- > 🎉 Google has applied for `ko` to join the Cloud Native Computing Foundation as a Sandbox project! Learn more [here](https://opensource.googleblog.com/2022/10/ko-applies-to-become-a-cncf-sandbox-project.html)! `ko` is a simple, fast container image builder for Go applications. It's ideal for use cases where your image contains a single Go application without any/many dependencies on the OS base image (e.g., no cgo, no OS package dependencies). `ko` builds images by effectively executing `go build` on your local machine, and as such doesn't require `docker` to be installed. This can make it a good fit for lightweight CI/CD use cases. `ko` makes [multi-platform builds](https://ko.build/features/multi-platform/) easy, produces [SBOMs](https://ko.build/features/sboms/) by default, and includes support for simple YAML templating which makes it a powerful tool for [Kubernetes applications](https://ko.build/features/k8s/). # [Install `ko`](https://ko.build/install/) and [get started](https://ko.build/get-started/)! ### Acknowledgements This work is based heavily on experience from having built the [Docker](https://github.com/bazelbuild/rules_docker) and [Kubernetes](https://github.com/bazelbuild/rules_k8s) support for [Bazel](https://bazel.build). That work was presented [here](https://www.youtube.com/watch?v=RS1aiQqgUTA). ### Discuss Questions? Comments? Ideas? Come discuss `ko` with us in the `#ko-build` channel on the [Kubernetes Slack](https://slack.k8s.io)! See you there! ### Community Meetings You can find all the necessary details about the community meetings in this [page](https://ko.build/community). --- ## File: docs/advanced/faq.md # Frequently Asked Questions ## How can I set `ldflags`? [Using -ldflags](https://blog.cloudflare.com/setting-go-variables-at-compile-time/) is a common way to embed version info in go binaries (In fact, we do this for `ko`!). You can pass ldflags to `go build` with the `--ldflags` flag (repeatable). When set, `--ldflags` takes precedence over ldflags in `.ko.yaml`: ```sh ko build --ldflags "-X=main.version=1.2.3" --ldflags "-s -w" . ``` You can also use the `GOFLAGS` environment variable: ```sh GOFLAGS="-ldflags=-X=main.version=1.2.3" ko build . ``` `GOFLAGS` has a limitation that does not allow setting multiple arguments in `ldflags`. For more complex cases, use the [`builds` section](./../configuration.md) in the `.ko.yaml` file. ## Why are my images all created in 1970? In order to support [reproducible builds](https://reproducible-builds.org), `ko` doesn't embed timestamps in the images it produces by default. However, `ko` does respect the [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/) environment variable, which will set the container image's timestamp accordingly. Similarly, the `KO_DATA_DATE_EPOCH` environment variable can be used to set the _modtime_ timestamp of the files in `KO_DATA_PATH`. For example, you can set the container image's timestamp to the current timestamp by executing: ```sh export SOURCE_DATE_EPOCH=$(date +%s) ``` or set the timestamp of the files in `KO_DATA_PATH` to the latest git commit's timestamp with: ```sh export KO_DATA_DATE_EPOCH=$(git log -1 --format='%ct') ``` ## Can I build Windows containers? Yes, but support for Windows containers is new, experimental, and tenuous. Be prepared to file bugs. 🐛 The default base image does not provide a Windows image. You can try out building a Windows container image by [setting the base image](./../configuration.md) to a Windows base image and building with `--platform=windows/amd64` or `--platform=all`: For example, to build a Windows container image, update your `.ko.yaml` to set the base image: ```plaintext defaultBaseImage: mcr.microsoft.com/windows/nanoserver:ltsc2022 ``` And build for `windows/amd64`. ```sh ko build ./ --platform=windows/amd64 ``` ### Known issues 🐛 - Symlinks in `kodata` are ignored when building Windows images; only regular files and directories will be included in the Windows image. ## Does `ko` support autocompletion? Yes! `ko completion` generates a Bash/Zsh/Fish/PowerShell completion script. You can get how to load it from help document. ```sh ko completion [bash|zsh|fish|powershell] --help ``` Or, you can source it directly: ```bash source <(ko completion) ``` ## Does `ko` work with [Kustomize](https://kustomize.io/)? Yes! `ko resolve -f -` will read and process input from stdin, so you can have `ko` easily process the output of the `kustomize` command. ```sh kustomize build config | ko resolve -f - ``` ## Does `ko` integrate with other build and development tools? Oh, you betcha. Here's a partial list: - `ko` support in [Skaffold](https://skaffold.dev/docs/pipeline-stages/builders/ko/) - `ko` support for [goreleaser](https://goreleaser.com/customization/ko/) - `ko` task in the [Tekton catalog](https://github.com/tektoncd/catalog/tree/main/task/ko/) - `ko` support in [Carvel's `kbld`](https://carvel.dev/kbld/docs/latest/config/#ko) - `ko` extension for [Tilt](https://github.com/tilt-dev/tilt-extensions/tree/master/ko) ## Does `ko` work with [OpenShift Internal Registry](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.11/html/registry/registry-overview#registry-integrated-openshift-registry_registry-overview)? Yes! Follow these steps: 1. [Connect to your OpenShift installation](https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/getting-started-cli.html#cli-logging-in_cli-developer-commands) 1. [Expose the OpenShift Internal Registry](https://docs.openshift.com/container-platform/latest/registry/securing-exposing-registry.html) so you can push to it: 1. Export your token to `$HOME/.docker/config.json`: ```sh oc registry login --to=$HOME/.docker/config.json ``` 1. Create a namespace where you will push your images, i.e: `ko-images` 1. Execute this command to set `KO_DOCKER_REPO` to publish images to the internal registry. ```sh export KO_DOCKER_REPO=$(oc registry info --public)/ko-images ``` --- ## File: docs/advanced/go-packages.md # Go Packages `ko`'s functionality can be consumed as a library in a Go application. To build an image, use [`pkg/build`](https://pkg.go.dev/github.com/google/ko/pkg/build), and publish it with [`pkg/publish`](https://pkg.go.dev/github.com/google/ko/pkg/publish). This is a minimal example of using the packages together, to implement the core subset of `ko`'s functionality: ```go package main import ( "context" "fmt" "log" "github.com/google/go-containerregistry/pkg/authn" "github.com/google/go-containerregistry/pkg/name" "github.com/google/go-containerregistry/pkg/v1/remote" "github.com/google/ko/pkg/build" "github.com/google/ko/pkg/publish" ) const ( baseImage = "cgr.dev/chainguard/static:latest" targetRepo = "example.registry/my-repo" importpath = "github.com/my-org/miniko" commitSHA = "deadbeef" ) func main() { ctx := context.Background() b, err := build.NewGo(ctx, ".", build.WithPlatforms("linux/amd64"), // only build for these platforms. build.WithBaseImages(func(ctx context.Context, _ string) (name.Reference, build.Result, error) { ref := name.MustParseReference(baseImage) base, err := remote.Index(ref, remote.WithContext(ctx)) return ref, base, err })) if err != nil { log.Fatalf("NewGo: %v", err) } r, err := b.Build(ctx, importpath) if err != nil { log.Fatalf("Build: %v", err) } p, err := publish.NewDefault(targetRepo, // publish to example.registry/my-repo publish.WithTags([]string{commitSHA}), // tag with :deadbeef publish.WithAuthFromKeychain(authn.DefaultKeychain)) // use credentials from ~/.docker/config.json if err != nil { log.Fatalf("NewDefault: %v", err) } ref, err := p.Publish(ctx, r, importpath) if err != nil { log.Fatalf("Publish: %v", err) } fmt.Println(ref.String()) } ``` --- ## File: docs/advanced/lambda.md # AWS Lambda `ko` can build images that can be deployed as AWS Lambda functions, using [Lambda's container support](https://docs.aws.amazon.com/lambda/latest/dg/images-create.html). For best results, use the [Go runtime interface client](https://docs.aws.amazon.com/lambda/latest/dg/go-image.html#go-image-clients) provided by the [`lambda` package](https://pkg.go.dev/github.com/aws/aws-lambda-go/lambda). For example: ```go package main import ( "context" "fmt" "github.com/aws/aws-lambda-go/lambda" ) type Event struct { Name string `json:"name"` // TODO: add other request fields here. } func main() { lambda.Start(func(ctx context.Context, event Event) (string, error) { return fmt.Sprintf("Hello %s!", event.Name), nil }) } ``` See AWS's [documentation](https://docs.aws.amazon.com/lambda/latest/dg/golang-handler.html) for more information on writing Lambda functions in Go. To deploy to Lambda, you must push to AWS Elastic Container Registry (ECR): ```sh KO_DOCKER_REPO=[account-id].dkr.ecr.[region].amazonaws.com/my-repo image=$(ko build ./cmd/app) ``` Then, create a Lambda function using the image in ECR: ```sh aws lambda create-function \ --function-name hello-world \ --package-type Image \ --code ImageUri=${image} \ --role arn:aws:iam::[account-id]:role/lambda-ex ``` See AWS's [documentation](https://docs.aws.amazon.com/lambda/latest/dg/go-image.html) for more information on deploying Lambda functions using Go container images, including how to configure push access to ECR, and how to configure the IAM role for the function. The base image that `ko` uses by default supports both x86 and Graviton2 architectures. You can also use the [`ko` Terraform provider](./terraform.md) to build and deploy Lambda functions as part of your IaC workflow, using the [`aws_lambda_function` resource](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function.html). See the [provider example](https://github.com/ko-build/terraform-provider-ko/tree/main/provider-examples/lambda) to get started. --- ## File: docs/advanced/limitations.md # Limitations `ko` works best when your application has no dependencies on the underlying image. This means `ko` is ideal when you don't require [cgo](https://pkg.go.dev/cmd/cgo), and builds are executed with `CGO_ENABLED=0` by default. To install other OS packages, make those available in your [configured base image](../../configuration). `ko` only supports Go applications. For a similar tool targeting Java applications, try [Jib](https://github.com/GoogleContainerTools/jib). For other languages, try [apko](https://github.com/chainguard-dev/apko) and [melange](https://github.com/chainguard-dev/melange). --- ## File: docs/advanced/linux-capabilities.md # Linux Capabilities In Linux, capabilities are a way to selectively grant privileges to a running process. Docker provides `--cap-add` and `--cap-drop` [run options](https://docs.docker.com/engine/containers/run/#runtime-privilege-and-linux-capabilities) to tweak container capabilities, e.g: ``` docker run --cap-add bpf hello-world ``` If container runs as a non-root user, capabilities are narrowed by intersecting with *file* capabilities of the application binary. When building images with a Dockerfile, one typically uses `setcap` tool to modify file capabilities, e.g: `setcap FILE bpf=ep`. To set file capabilities with `ko`, specify `linux_capabilities` in builds configuration section in your `.ko.yaml`. Use `setcap` syntax: ```yaml builds: - id: caps linux_capabilities: bpf=ep ``` ## Alternative spelling ```yaml builds: - id: caps linux_capabilities: - cap1 - cap2 - cap3 ``` A list of capability names is equivalent to `cap1,cap2,cap3=p`. ## Improving UX in capability-reliant apps A capability can be *permitted* (`=p`), or both *permitted* and *effective* (`=ep`). Effective capabilities are used for permission checks. A program can promote permitted capability to effective when needed. ```yaml builds: - id: caps linux_capabilities: bpf,perfmon,net_admin=ep ``` Initially, `=ep` might look like a good idea. There's no need to explicitly promote *permitted* capabilities. Application takes advantage of *effective* capabilities right away. There is a catch though. ``` $ docker run --cap-add bpf ko.local/caps.test-4b8f7bca75c467b3d2803e1c087a3287 exec /ko-app/caps.test: operation not permitted ``` When run options request fewer capabilities than specified in file capabilities, container fails to start. It is hard to tell what went wrong since `operation not permitted` is a generic error. (Docker is unlikely to improve diagnostics for this failure case since the check is implemented in Linux kernel.) We suggest to use `=p` instead. This option puts application in charge of verifying and promoting permitted capabilities to effective. It can produce much better diagnostics: ``` $ docker run --cap-add bpf ko.local/caps.test-4b8f7bca75c467b3d2803e1c087a3287 current capabilities: cap_bpf=p activating capabilities: cap_net_admin,cap_perfmon,cap_bpf=ep: operation not permitted ``` [Sample code](https://go.dev/play/p/uPMzyotkNHg). --- ## File: docs/advanced/migrating-from-dockerfile.md # Migrating from Dockerfile If your `Dockerfile` looks like either of the examples in the [official tutorial for writing a Dockerfile to containerize a Go application](https://docs.docker.com/language/golang/build-images/), you can easily migrate to use `ko` instead. Let's review the best practice multi-stage Dockerfile in that tutorial first: ```Dockerfile ## Build FROM golang:1.16-buster AS build WORKDIR /app COPY go.mod ./ COPY go.sum ./ RUN go mod download COPY *.go ./ RUN go build -o /docker-gs-ping ## Deploy FROM gcr.io/distroless/base-debian10 WORKDIR / COPY --from=build /docker-gs-ping /docker-gs-ping EXPOSE 8080 USER nonroot:nonroot ENTRYPOINT ["/docker-gs-ping"] ``` This `Dockerfile`: 1. pulls the `golang:1.16` image 1. `COPY`s your local source into the container environment (`COPY`ing `go.mod` and `go.sum` first and running `go mod download`, to cache dependencies in the container environment) 1. `RUN`s `go build` on your source, inside the container, to produce an executable 1. `COPY`s the executable built in the previous step into a new image, on top of a minimal [distroless](https://github.com/GoogleContainerTools/distroless) base image. The result is a Go application built on a minimal base image, with an optimally cached build sequence. After running `docker build` on this `Dockerfile`, don't forget to push that image to the registry so you can deploy it. --- ## Migrating to `ko` If your Go source is laid out as described in the tutorial, and you've [installed](../../install) and [set up your environment](../../get-started), you can simply run `ko build ./` to build and push the container image to your registry. You're done. You can delete your `Dockerfile` and uninstall `docker`. `ko` takes advantage of your local [Go build cache](../../features/build-cache) without needing to be told to, and it sets the `ENTRYPOINT` and uses a nonroot distroless base image by default. To build a multi-arch image, simply add `--platform=all`. Compare this to the [equivalent Docker instructions](https://docs.docker.com/desktop/multi-arch/). --- ## File: docs/advanced/root-ca-certificates.md # Root CA Certificates To install a [root certificate](https://en.wikipedia.org/wiki/Root_certificate) into your container built using `ko`, you can use one of the following methods. ## incert [`incert`](https://github.com/chainguard-dev/incert) allows you to append CA certificates to an image and push the modified image to a specified registry. `incert` can be run after `ko build` to build your Go application container image with custom root CA certificates. ### Example 1. Build and push your Go application container image using `ko build` ```sh KO_DOCKER_REPO=mycompany/myimage:latest ko build . ``` 2. Append the built image with your custom CA certificate(s) using `incert` ```sh incert -image-url=mycompany/myimage:latest -ca-certs-file=/path/to/cacerts.pem -dest-image-url=myregistry/myimage:latest ``` ## Custom Base Image New root certificates can be [installed into a custom image](https://stackoverflow.com/questions/42292444/how-do-i-add-a-ca-root-certificate-inside-a-docker-image) using standard OS packages. Then, this custom image can be used [to override the base image for `ko`](https://ko.build/configuration/#overriding-base-images). Once the Go application container image is built using `ko` with the custom base image, the root certificates installed on the base image will be trusted by the Go application. ### Example 1. Make a custom container image with your new root certificates ```dockerfile # Dockerfile FROM alpine RUN apk update RUN apk add ca-certificates ADD new-root-ca.crt /usr/local/share/ca-certificates/new-root-ca.crt RUN chmod 644 /usr/local/share/ca-certificates/new-root-ca.crt RUN update-ca-certificates ``` 2. Build and push the custom container image to a container registry ```sh docker build . -t docker.io/ko-build/image-with-new-root-certs docker push docker.io/ko-build/image-with-new-root-certs ``` 3. Configure `ko` to [override the default base image](https://ko.build/configuration/#overriding-base-images) with the custom image ```yaml # .ko.yaml defaultBaseImage: docker.io/ko-build/image-with-new-root-certs ``` **OR** ```sh export KO_DEFAULTBASEIMAGE=docker.io/ko-build/image-with-new-root-certs ``` 4. Build the Go app container image with `ko` ```sh ko build . ``` ## Static Assets Alternatively, root certificates can be installed into the Go application container image using a combination of [`ko` static assets](https://ko.build/features/static-assets/) and [overriding the default system location for SSL certificates](https://pkg.go.dev/crypto/x509#SystemCertPool). Using `ko`'s support for static assets, root certificates can be stored in the `/kodata` directory (either checked into the repository, or injected dynamically by a CI pipeline). After running `ko build`, the certificate files are then bundled into the built image at the path `$KO_DATA_PATH`. To enable the Go application to trust the bundled certificate(s), the container runtime or orchestrator (Docker, Kubernetes, etc) must set the environment variable `SSL_CERT_DIR` to the same value as `KO_DATA_PATH`. Go [uses `SSL_CERT_DIR` to determine the directory to check for SSL certificate files](https://go.dev/src/crypto/x509/root_unix.go). Once this variable is set, the Go application will trust the bundled root certificates in `$KO_DATA_PATH`. ### Example 1. Copy the root certificate(s) to the `/kodata/` directory ```sh # $(pwd) assumed to be at for this example mkdir -p kodata cp $CERT_FILE_DIR/*.crt kodata/ ``` 2. Build the Go application container image ```sh KO_DOCKER_REPO=docker.io/ko-build/static-assets-certs ko build . ``` 3. Run the Go application container image with `SSL_CERT_DIR` equal to `/var/run/ko` (the default value for `$KO_DATA_PATH`) ```sh docker run -e SSL_CERT_DIR=/var/run/ko docker.io/ko-build/static-assets-certs ``` A functional client-server example for this can be seen [here](https://github.com/kosamson/ko-private-ca-test). --- ## File: docs/advanced/terraform.md # Terraform Provider In addition to the CLI, `ko`'s functionality is also available as a Terraform provider. This allows `ko` to be integrated with your Infrastructure-as-Code (IaC) workflows, and makes building your code a seamless part of your deployment process. Using the Terraform provider is as simple as adding a `ko_build` resource to your Terraform configuration: ```hcl // Require the `ko-build/ko` provider. terraform { required_providers { ko = { source = "ko-build/ko" } } } // Configure the provider to push to your repo. provider "ko" { repo = "example.registry/my-repo" // equivalent to KO_DOCKER_REPO } // Build your code. resource "ko_build" "app" { importpath = "github.com/example/repo/cmd/app" } // TODO: use the `ko_build.app` resource elsewhere in your Terraform configuration. // Report the build image's digest. output "image" { value = ko_build.app.image_ref } ``` See the [`ko-build/ko` provider on the Terraform Registry](https://registry.terraform.io/providers/ko-build/ko/latest) for more information, and the [GitHub repo](https://github.com/ko-build/terraform-provider-ko) for more examples. --- ## File: docs/features/build-cache.md # Build Cache Because `ko` just runs `go build` in your normal development environment, it automatically reuses your [`go build` cache](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching) from previous builds, making iterative development faster. `ko` also avoids pushing blobs to the remote image registry if they're already present, making pushes faster. You can make `ko` even faster by setting the `KOCACHE` environment variable. This tells `ko` to store a local mapping between the `go build` inputs to the image layer that they produce, so `go build` can be skipped entirely if the layer is already present in the image registry.