ko Documentation

File: README.md Copy Chapter

`ko`: Easy Go Containers

🎉 Google has applied for ko to join the Cloud Native Computing Foundation as a Sandbox project! Learn more here!

ko is a simple, fast container image builder for Go applications.

It's ideal for use cases where your image contains a single Go application
without any/many dependencies on the OS base image (e.g., no cgo, no OS package
dependencies).

ko builds images by effectively executing go build on your local machine,
and as such doesn't require docker to be installed. This can make it a good
fit for lightweight CI/CD use cases.

ko makes multi-platform builds easy, produces SBOMs by default, and includes support for simple YAML templating which makes it a powerful tool for Kubernetes applications.

[Install `ko`](https://ko.build/install/) and [get started](https://ko.build/get-started/)!

Acknowledgements Copy Section

This work is based heavily on experience from having built the Docker and Kubernetes support for Bazel.
That work was presented here.

Discuss Copy Section

Questions? Comments? Ideas?
Come discuss ko with us in the #ko-build channel on the Kubernetes Slack!
See you there!

Community Meetings Copy Section

You can find all the necessary details about the community meetings in this page.

File: docs/advanced/faq.md Copy Chapter

Frequently Asked Questions

How can I set `ldflags`? Copy Chapter

Using -ldflags is a common way to embed version info in go binaries (In fact, we do this for ko!).
You can pass ldflags to go build with the --ldflags flag (repeatable). When set, --ldflags takes precedence over ldflags in .ko.yaml:

ko build --ldflags "-X=main.version=1.2.3" --ldflags "-s -w" .

You can also use the GOFLAGS environment variable:

GOFLAGS="-ldflags=-X=main.version=1.2.3" ko build .

GOFLAGS has a limitation that does not allow setting multiple arguments in ldflags.
For more complex cases, use the builds section in the .ko.yaml file.

Why are my images all created in 1970? Copy Chapter

In order to support reproducible builds, ko doesn't embed timestamps in the images it produces by default.

However, ko does respect the SOURCE_DATE_EPOCH environment variable, which will set the container image's timestamp accordingly.

Similarly, the KO_DATA_DATE_EPOCH environment variable can be used to set the modtime timestamp of the files in KO_DATA_PATH.

For example, you can set the container image's timestamp to the current timestamp by executing:

export SOURCE_DATE_EPOCH=$(date +%s)

or set the timestamp of the files in KO_DATA_PATH to the latest git commit's timestamp with:

export KO_DATA_DATE_EPOCH=$(git log -1 --format='%ct')

Can I build Windows containers? Copy Chapter

Yes, but support for Windows containers is new, experimental, and tenuous. Be prepared to file bugs. 🐛

The default base image does not provide a Windows image.
You can try out building a Windows container image by setting the base image to a Windows base image and building with --platform=windows/amd64 or --platform=all:

For example, to build a Windows container image, update your .ko.yaml to set the base image:

defaultBaseImage: mcr.microsoft.com/windows/nanoserver:ltsc2022

And build for windows/amd64.

ko build ./ --platform=windows/amd64

Known issues 🐛 Copy Section

• Symlinks in kodata are ignored when building Windows images; only regular files and directories will be included in the Windows image.

Does `ko` support autocompletion? Copy Chapter

Yes! ko completion generates a Bash/Zsh/Fish/PowerShell completion script.
You can get how to load it from help document.

ko completion [bash|zsh|fish|powershell] --help

Or, you can source it directly:

source <(ko completion)

Does `ko` work with [Kustomize](https://kustomize.io/)? Copy Chapter

Yes! ko resolve -f - will read and process input from stdin, so you can have ko easily process the output of the kustomize command.

kustomize build config | ko resolve -f -

Does `ko` integrate with other build and development tools? Copy Chapter

Oh, you betcha. Here's a partial list:

• ko support in Skaffold
• ko support for goreleaser
• ko task in the Tekton catalog
• ko support in Carvel's kbld
• ko extension for Tilt

Does `ko` work with [OpenShift Internal Registry](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.11/html/registry/registry-overview#registry-integrated-openshift-registry_registry-overview)? Copy Chapter

Yes! Follow these steps:

1. Connect to your OpenShift installation
2. Expose the OpenShift Internal Registry so you can push to it:
3. Export your token to $HOME/.docker/config.json:

oc registry login --to=$HOME/.docker/config.json

1. Create a namespace where you will push your images, i.e: ko-images
2. Execute this command to set KO_DOCKER_REPO to publish images to the internal registry.

export KO_DOCKER_REPO=$(oc registry info --public)/ko-images

File: docs/advanced/go-packages.md Copy Chapter

Go Packages

ko's functionality can be consumed as a library in a Go application.

To build an image, use pkg/build, and publish it with pkg/publish.

This is a minimal example of using the packages together, to implement the core subset of ko's functionality:

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/google/go-containerregistry/pkg/authn"
	"github.com/google/go-containerregistry/pkg/name"
	"github.com/google/go-containerregistry/pkg/v1/remote"
	"github.com/google/ko/pkg/build"
	"github.com/google/ko/pkg/publish"
)

const (
	baseImage  = "cgr.dev/chainguard/static:latest"
	targetRepo = "example.registry/my-repo"
	importpath = "github.com/my-org/miniko"
	commitSHA  = "deadbeef"
)

func main() {
	ctx := context.Background()

	b, err := build.NewGo(ctx, ".",
		build.WithPlatforms("linux/amd64"), // only build for these platforms.
		build.WithBaseImages(func(ctx context.Context, _ string) (name.Reference, build.Result, error) {
			ref := name.MustParseReference(baseImage)
			base, err := remote.Index(ref, remote.WithContext(ctx))
			return ref, base, err
		}))
	if err != nil {
		log.Fatalf("NewGo: %v", err)
	}
	r, err := b.Build(ctx, importpath)
	if err != nil {
		log.Fatalf("Build: %v", err)
	}

	p, err := publish.NewDefault(targetRepo,                 // publish to example.registry/my-repo
		publish.WithTags([]string{commitSHA}),               // tag with :deadbeef
		publish.WithAuthFromKeychain(authn.DefaultKeychain)) // use credentials from ~/.docker/config.json
	if err != nil {
		log.Fatalf("NewDefault: %v", err)
	}
	ref, err := p.Publish(ctx, r, importpath)
	if err != nil {
		log.Fatalf("Publish: %v", err)
	}
	fmt.Println(ref.String())
}

File: docs/advanced/lambda.md Copy Chapter

AWS Lambda

ko can build images that can be deployed as AWS Lambda functions, using Lambda's container support.

For best results, use the Go runtime interface client provided by the lambda package.

For example:

package main

import (
	"context"
	"fmt"
	"github.com/aws/aws-lambda-go/lambda"
)

type Event struct {
	Name string `json:"name"`
	// TODO: add other request fields here.
}

func main() {
	lambda.Start(func(ctx context.Context, event Event) (string, error) {
		return fmt.Sprintf("Hello %s!", event.Name), nil
	})
}

See AWS's documentation for more information on writing Lambda functions in Go.

To deploy to Lambda, you must push to AWS Elastic Container Registry (ECR):

KO_DOCKER_REPO=[account-id].dkr.ecr.[region].amazonaws.com/my-repo
image=$(ko build ./cmd/app)

Then, create a Lambda function using the image in ECR:

aws lambda create-function \
  --function-name hello-world \
  --package-type Image \
  --code ImageUri=${image} \
  --role arn:aws:iam::[account-id]:role/lambda-ex

See AWS's documentation for more information on deploying Lambda functions using Go container images, including how to configure push access to ECR, and how to configure the IAM role for the function.

The base image that ko uses by default supports both x86 and Graviton2 architectures.

You can also use the ko Terraform provider to build and deploy Lambda functions as part of your IaC workflow, using the aws_lambda_function resource. See the provider example to get started.

File: docs/advanced/limitations.md Copy Chapter

Limitations

ko works best when your application has no dependencies on the underlying image.

This means ko is ideal when you don't require cgo, and builds are executed with CGO_ENABLED=0 by default.

To install other OS packages, make those available in your configured base image.

ko only supports Go applications.
For a similar tool targeting Java applications, try Jib.
For other languages, try apko and melange.

File: docs/advanced/linux-capabilities.md Copy Chapter

Linux Capabilities

In Linux, capabilities are a way to selectively grant privileges to a running process.

Docker provides --cap-add and --cap-drop
run options
to tweak container capabilities, e.g:

docker run --cap-add bpf hello-world

If container runs as a non-root user,
capabilities are narrowed by intersecting with file capabilities of the
application binary. When building images with a Dockerfile, one
typically uses setcap tool to modify file capabilities, e.g:
setcap FILE bpf=ep.

To set file capabilities with ko, specify linux_capabilities
in builds configuration section in your .ko.yaml. Use setcap syntax:

builds:
- id: caps
  linux_capabilities: bpf=ep

Alternative spelling Copy Chapter

builds:
- id: caps
  linux_capabilities:
  - cap1
  - cap2
  - cap3

A list of capability names is equivalent to cap1,cap2,cap3=p.

Improving UX in capability-reliant apps Copy Chapter

A capability can be permitted (=p), or both permitted and effective (=ep).
Effective capabilities are used for permission checks.
A program can promote permitted capability to effective when needed.

builds:
- id: caps
  linux_capabilities: bpf,perfmon,net_admin=ep

Initially, =ep might look like a good idea.
There's no need to explicitly promote permitted capabilities.
Application takes advantage of effective capabilities right away.

There is a catch though.

$ docker run --cap-add bpf ko.local/caps.test-4b8f7bca75c467b3d2803e1c087a3287
exec /ko-app/caps.test: operation not permitted

When run options request fewer capabilities than specified in file capabilities,
container fails to start. It is hard to tell what went wrong since
operation not permitted is a generic error. (Docker is unlikely to improve diagnostics
for this failure case since the check is implemented in Linux kernel.)

We suggest to use =p instead. This option puts application in charge of verifying and
promoting permitted capabilities to effective. It can produce much better diagnostics:

$ docker run --cap-add bpf ko.local/caps.test-4b8f7bca75c467b3d2803e1c087a3287
current capabilities: cap_bpf=p
activating capabilities: cap_net_admin,cap_perfmon,cap_bpf=ep: operation not permitted

Sample code.

File: docs/advanced/migrating-from-dockerfile.md Copy Chapter

Migrating from Dockerfile

If your Dockerfile looks like either of the examples in the official tutorial for writing a Dockerfile to containerize a Go application, you can easily migrate to use ko instead.

Let's review the best practice multi-stage Dockerfile in that tutorial first:

## Build
FROM golang:1.16-buster AS build

WORKDIR /app

COPY go.mod ./
COPY go.sum ./
RUN go mod download

COPY *.go ./

RUN go build -o /docker-gs-ping

## Deploy
FROM gcr.io/distroless/base-debian10

WORKDIR /

COPY --from=build /docker-gs-ping /docker-gs-ping

EXPOSE 8080

USER nonroot:nonroot

ENTRYPOINT ["/docker-gs-ping"]

This Dockerfile:

1. pulls the golang:1.16 image
2. COPYs your local source into the container environment (COPYing go.mod and go.sum first and running go mod download, to cache dependencies in the container environment)
3. RUNs go build on your source, inside the container, to produce an executable
4. COPYs the executable built in the previous step into a new image, on top of a minimal distroless base image.

The result is a Go application built on a minimal base image, with an optimally cached build sequence.

After running docker build on this Dockerfile, don't forget to push that image to the registry so you can deploy it.

Migrating to `ko` Copy Chapter

If your Go source is laid out as described in the tutorial, and you've installed and set up your environment, you can simply run ko build ./ to build and push the container image to your registry.

You're done. You can delete your Dockerfile and uninstall docker.

ko takes advantage of your local Go build cache without needing to be told to, and it sets the ENTRYPOINT and uses a nonroot distroless base image by default.

To build a multi-arch image, simply add --platform=all.
Compare this to the equivalent Docker instructions.

File: docs/advanced/root-ca-certificates.md Copy Chapter

Root CA Certificates

To install a root certificate into your container built using ko, you can use one of the following methods.

incert Copy Chapter

incert allows you to append CA certificates to an image and push the modified image to a specified registry.

incert can be run after ko build to build your Go application container image with custom root CA certificates.

Example Copy Section

1. Build and push your Go application container image using ko build

KO_DOCKER_REPO=mycompany/myimage:latest ko build .

2. Append the built image with your custom CA certificate(s) using incert

incert -image-url=mycompany/myimage:latest -ca-certs-file=/path/to/cacerts.pem -dest-image-url=myregistry/myimage:latest

Custom Base Image Copy Chapter

New root certificates can be installed into a custom image using standard OS packages. Then, this custom image can be used to override the base image for ko. Once the Go application container image is built using ko with the custom base image, the root certificates installed on the base image will be trusted by the Go application.

Example Copy Section

1. Make a custom container image with your new root certificates

# Dockerfile
FROM alpine

RUN apk update
RUN apk add ca-certificates

ADD new-root-ca.crt /usr/local/share/ca-certificates/new-root-ca.crt
RUN chmod 644 /usr/local/share/ca-certificates/new-root-ca.crt
RUN update-ca-certificates

2. Build and push the custom container image to a container registry

docker build . -t docker.io/ko-build/image-with-new-root-certs
docker push docker.io/ko-build/image-with-new-root-certs

3. Configure ko to override the default base image with the custom image

# .ko.yaml
defaultBaseImage: docker.io/ko-build/image-with-new-root-certs

**OR**

export KO_DEFAULTBASEIMAGE=docker.io/ko-build/image-with-new-root-certs

4. Build the Go app container image with ko

ko build .

Static Assets Copy Chapter

Alternatively, root certificates can be installed into the Go application container image using a combination of ko static assets and overriding the default system location for SSL certificates.

Using ko's support for static assets, root certificates can be stored in the /kodata directory (either checked into the repository, or injected dynamically by a CI pipeline). After running ko build, the certificate files are then bundled into the built image at the path $KO_DATA_PATH.

To enable the Go application to trust the bundled certificate(s), the container runtime or orchestrator (Docker, Kubernetes, etc) must set the environment variable SSL_CERT_DIR to the same value as KO_DATA_PATH. Go uses SSL_CERT_DIR to determine the directory to check for SSL certificate files. Once this variable is set, the Go application will trust the bundled root certificates in $KO_DATA_PATH.

Example Copy Section

1. Copy the root certificate(s) to the /kodata/ directory

# $(pwd) assumed to be at  for this example
mkdir -p kodata
cp $CERT_FILE_DIR/*.crt kodata/

2. Build the Go application container image

KO_DOCKER_REPO=docker.io/ko-build/static-assets-certs ko build .

3. Run the Go application container image with SSL_CERT_DIR equal to /var/run/ko (the default value for $KO_DATA_PATH)

docker run -e SSL_CERT_DIR=/var/run/ko docker.io/ko-build/static-assets-certs

A functional client-server example for this can be seen here.

File: docs/advanced/terraform.md Copy Chapter

Terraform Provider

In addition to the CLI, ko's functionality is also available as a Terraform provider.

This allows ko to be integrated with your Infrastructure-as-Code (IaC) workflows, and makes building your code a seamless part of your deployment process.

Using the Terraform provider is as simple as adding a ko_build resource to your Terraform configuration:

// Require the `ko-build/ko` provider.
terraform {
  required_providers {
    ko = { source = "ko-build/ko" }
  }
}

// Configure the provider to push to your repo.
provider "ko" {
  repo = "example.registry/my-repo" // equivalent to KO_DOCKER_REPO
}

// Build your code.
resource "ko_build" "app" {
  importpath = "github.com/example/repo/cmd/app"
}

// TODO: use the `ko_build.app` resource elsewhere in your Terraform configuration.

// Report the build image's digest.
output "image" {
  value = ko_build.app.image_ref
}

See the ko-build/ko provider on the Terraform Registry for more information, and the GitHub repo for more examples.

File: docs/features/build-cache.md Copy Chapter

Build Cache

Because ko just runs go build in your normal development environment, it automatically reuses your go build cache from previous builds, making iterative development faster.

ko also avoids pushing blobs to the remote image registry if they're already present, making pushes faster.

You can make ko even faster by setting the KOCACHE environment variable.
This tells ko to store a local mapping between the go build inputs to the image layer that they produce, so go build can be skipped entirely if the layer is already present in the image registry.