cadvisor Documentation

Analyzes resource usage and performance characteristics of running containers.

19,298 stars Go Go 1 file · ~5,849 tokens
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy cadvisor on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run) 
docker run -d --name cadvisor --restart always google/cadvisor:latest

File: README.md

cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers. Specifically, for each container it keeps resource isolation parameters, historical resource usage, histograms of complete historical resource usage and network statistics. This data is exported by container and machine-wide.

cAdvisor has native support for Docker containers and should support just about any other container type out of the box. We strive for support across the board so feel free to open an issue if that is not the case. cAdvisor's container abstraction is based on lmctfy's so containers are inherently nested hierarchically.

Quick Start: Running cAdvisor in a Docker Container

To quickly tryout cAdvisor on your machine with Docker, we have a Docker image that includes everything you need to get started. You can run a single cAdvisor to monitor the whole machine. Simply run:

text
VERSION=0.55.1 # use the latest release version from https://github.com/google/cadvisor/releases
sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  --privileged \
  --device=/dev/kmsg \
  ghcr.io/google/cadvisor:$VERSION # for versions prior to v0.53.0, use gcr.io/cadvisor/cadvisor instead

cAdvisor is now running (in the background) on http://localhost:8080. The setup includes directories with Docker state cAdvisor needs to observe.

Note: If you're running on CentOS, Fedora, or RHEL (or are using LXC), take a look at our running instructions.

We have detailed instructions on running cAdvisor standalone outside of Docker. cAdvisor running options may also be interesting for advanced usecases. If you want to build your own cAdvisor Docker image, see our deployment page.

For Kubernetes users, cAdvisor can be run as a daemonset. See the instructions for how to get started, and for how to kustomize it to fit your needs.

Building and Testing

See the more detailed instructions in the build page. This includes instructions for building and deploying the cAdvisor Docker image.

Exporting stats

cAdvisor supports exporting stats to various storage plugins. See the documentation for more details and examples.

Web UI

cAdvisor exposes a web UI at its port:

http://<hostname>:/

See the documentation for more details.

Remote REST API & Clients

cAdvisor exposes its raw and processed stats via a versioned remote REST API. See the API's documentation for more information.

There is also an official Go client implementation in the client directory. See the documentation for more information.

Roadmap

cAdvisor aims to improve the resource usage and performance characteristics of running containers. Today, we gather and expose this information to users. In our roadmap:

  • Advise on the performance of a container (e.g.: when it is being negatively affected by another, when it is not receiving the resources it requires, etc).
  • Auto-tune the performance of the container based on previous advise.
  • Provide usage prediction to cluster schedulers and orchestration layers.

Community

Contributions, questions, and comments are all welcomed and encouraged! cAdvisor developers hang out on Slack in the #sig-node channel (get an invitation here). We also have discuss.kubernetes.io.

Please reach out and get involved in the project, we're actively looking for more contributors to bring on board!

Core Team

Frequent Collaborators

Emeritus

File: client/v2/README.md

Example REST API Client

This is an implementation of a cAdvisor REST API in Go. You can use it like this:

go
client, err := client.NewClient("http://192.168.59.103:8080/")

Obviously, replace the URL with the path to your actual cAdvisor REST endpoint.

MachineInfo

go
client.MachineInfo()

There is no v2 MachineInfo API, so the v2 client exposes the v1 MachineInfo

text
(*v1.MachineInfo)(0xc208022b10)({
 NumCores: (int) 4,
 MemoryCapacity: (int64) 2106028032,
 Filesystems: ([]v1.FsInfo) (len=1 cap=4) {
  (v1.FsInfo) {
   Device: (string) (len=9) "/dev/sda1",
   Capacity: (uint64) 19507089408
  }
 }
})

You can see the full specification of the MachineInfo struct in the source

VersionInfo

go
client.VersionInfo()

This method returns the cAdvisor version.

Attributes

go
client.Attributes()

This method returns a cadvisor/info/v2/Attributes struct with all the fields filled in. Attributes includes hardware attributes (as returned by MachineInfo) as well as software attributes (eg. software versions). Here is an example return value:

text
(*v2.Attributes)({
 KernelVersion: (string) (len=17) "3.13.0-44-generic"
 ContainerOsVersion: (string) (len=18) "Ubuntu 14.04.1 LTS"
 DockerVersion: (string) (len=9) "1.5.0-rc4"
 CadvisorVersion: (string) (len=6) "0.10.1"
 NumCores: (int) 4,
 MemoryCapacity: (int64) 2106028032,
 Filesystems: ([]v2.FsInfo) (len=1 cap=4) {
  (v2.FsInfo) {
   Device: (string) (len=9) "/dev/sda1",
   Capacity: (uint64) 19507089408
  }
 }
})

You can see the full specification of the Attributes struct in the source

File: client/README.md

Example REST API Client

This is an implementation of a cAdvisor REST API in Go. You can use it like this:

go
client, err := client.NewClient("http://192.168.59.103:8080/")

Obviously, replace the URL with the path to your actual cAdvisor REST endpoint.

MachineInfo

go
client.MachineInfo()

This method returns a cadvisor/v1.MachineInfo struct with all the fields filled in. Here is an example return value:

text
(*v1.MachineInfo)(0xc208022b10)({
 NumCores: (int) 4,
 MemoryCapacity: (int64) 2106028032,
 Filesystems: ([]v1.FsInfo) (len=1 cap=4) {
  (v1.FsInfo) {
   Device: (string) (len=9) "/dev/sda1",
   Capacity: (uint64) 19507089408
  }
 }
})

You can see the full specification of the MachineInfo struct in the source

ContainerInfo

Given a container name and a ContainerInfoRequest, will return all information about the specified container. See the ContainerInfoRequest struct in the source for the full specification.

go
request := v1.ContainerInfoRequest{NumStats: 10}
sInfo, err := client.ContainerInfo("/docker/d9d3eb10179e6f93a...", &request)

Returns a ContainerInfo struct

SubcontainersInfo

Given a container name and a ContainerInfoRequest, will recursively return all info about the container and all subcontainers contained within the container. See the ContainerInfoRequest struct in the source for the full specification.

go
request := v1.ContainerInfoRequest{NumStats: 10}
sInfo, err := client.SubcontainersInfo("/docker", &request)

Returns a ContainerInfo struct with the Subcontainers field populated.

File: cmd/internal/storage/bigquery/README.md

BigQuery Storage Driver

[EXPERIMENTAL] Support for BigQuery backend as cAdvisor storage driver.
The current implementation takes bunch of BigQuery specific flags for authentication.
These will be merged into a single backend config.

To run the current version, following flags need to be specified:

text
# Storage driver to use.
 -storage_driver=bigquery
 
 # Information about server-to-server Oauth token.
 # These can be obtained by creating a Service Account client id under `Google Developer API`
 
 # service client id
 -bq_id="XYZ.apps.googleusercontent.com"
 
 # service email address
 -bq_account="[email protected]"
 
 # path to pem key (converted from p12 file)
 -bq_credentials_file="/path/to/key.pem"
 
 # project id to use for storing datasets.
 -bq_project_id="awesome_project"

See Service account Authentication for Oauth related details.

File: deploy/kubernetes/README.md

cAdvisor Kubernetes Daemonset

cAdvisor uses Kustomize to manage Kubernetes manifests. See the Install Kustomize for installation instructions, and for a description of how it works.

Deploy

Pick a cAdvisor release

text
VERSION=v0.42.0

Deploy to Kubernetes cluster with remote build:

text
kustomize build "https://github.com/google/cadvisor/deploy/kubernetes/base?ref=${VERSION}" | kubectl apply -f -

Usage

To update the image version(reference):

text
cd deploy/kubernetes/base && kustomize edit set image ghcr.io/google/cadvisor:${VERSION} && cd ../../..

To generate the base daemonset:

text
kubectl kustomize deploy/kubernetes/base

To apply the base daemonset to your cluster:

text
kubectl kustomize deploy/kubernetes/base | kubectl apply -f -

To generate the daemonset with example patches applied:

text
kubectl kustomize deploy/kubernetes/overlays/examples

To apply the daemonset to your cluster with example patches applied:

text
kubectl kustomize deploy/kubernetes/overlays/examples | kubectl apply -f -

cAdvisor with perf support on Kubernetes

Example of modifications needed to deploy cAdvisor with perf support is provided in overlays/examples_perf directory (modification to daemonset and configmap with perf events configuration).

To generate and apply the daemonset with patches for cAdvisor with perf support:

text
kubectl kustomize deploy/kubernetes/overlays/examples_perf | kubectl apply -f -

Kustomization

On your own fork of cAdvisor, create your own overlay directory with your patches. Copy patches from the example folder if you intend to use them, but don't modify the originals. Commit your changes in your local branch, and use git to manage them the same way you would any other piece of code.

To run the daemonset with your patches applied:

text
kubectl kustomize deploy/kubernetes/overlays/<my_custom_overlays> | kubectl apply -f -

To get changes made to the upstream cAdvisor daemonset, simply rebase your fork of cAdvisor on top of upstream. Since you didn't make changes to the originals, you won't have any conflicts.

File: docs/development/build.md

Building and Testing cAdvisor

Note: cAdvisor only builds on Linux since it uses Linux-only APIs.

Installing Dependencies

cAdvisor is written in the Go programming language. If you haven't set up a Go development environment, please follow these instructions to install go tool and set up GOPATH. Note that the version of Go in package repositories of some operating systems is outdated, so please download the latest version.

Note: cAdvisor requires Go 1.14 to build.

After setting up Go, you should be able to go get cAdvisor as expected (we use -d to only download):

text
$ go get -d github.com/google/cadvisor

Building from Source

At this point you can build cAdvisor from the source folder:

text
$GOPATH/src/github.com/google/cadvisor $ make build

or run only unit tests:

text
$GOPATH/src/github.com/google/cadvisor $ make test

For integration tests, see the integration testing page.

Non-volatile Memory Support

cAdvisor can be linked against libipmctl library that allows to gather information about Intel® Optane™ DC Persistent memory. If you want to build cAdvisor with libipmctl support you must meet following requirements:

  • libipmctl-devel must be installed on build system.
  • libipmctl must be installed on all systems where cAdvisor is running.

Detailed information about building libipmctl can be found in the project's README. Make sure to use the most up to date released version. Functionality that relies on libipmctl was tested against version 02.00.00.3820 of the library.

To enable libipmctl support GO_FLAGS variable must be set:

text
$GOPATH/src/github.com/google/cadvisor $ GO_FLAGS="-tags=libipmctl,netgo" make build

Perf Support

cAdvisor can be linked against libpfm4 library that allows to gather information about performance monitoring events.
If you want to build cAdvisor with libpfm4 support you must meet following requirements:

  • libpfm4-dev must be installed on build system.
  • libpfm4 must be installed on all systems where cAdvisor is running.

libpfm4 packages are available in Debian- and RHEL-derivatives distributions.

libpfm4 can be installed using apt package manager:

text
apt-get install libpfm4 libpfm4-dev

or yum package manager:

text
yum install libpfm libpfm-devel

To enable libpfm4 support GO_FLAGS variable must be set:

text
$GOPATH/src/github.com/google/cadvisor $ GO_FLAGS="-tags=libpfm,netgo" make build

Running Built Binary

Now you can run the built binary:

text
$GOPATH/src/github.com/google/cadvisor $ sudo ./cadvisor

Perf Support

It is required to include perf config (examplary config is available here) to run cAdvisor with performance monitoring events:

text
$GOPATH/src/github.com/google/cadvisor $ sudo ./cadvisor -perf_events_config=perf/testing/perf-non-hardware.json

File: docs/development/integration_testing.md

Integration Testing cAdvisor

Docker-based tests

The cAdvisor integration tests are run per-pr using Github Actions. Workflow configuration can be found at .github/workflows/test.yml. Tests are executed in Docker containers run on MS Azure virtual machines.

To run them locally Docker must be installed on your machine. Following command allows you to execute default suite of integration tests:

text
make docker-test-integration

Build scripts take care of building cAdvisor and integration tests, and executing them against running cAdvisor process.

In order to run non-default tests suites (e.g. such that rely on third-party C libraries) you must source one of the files available at build/config, e.g.:

text
source build/config/libpfm4.sh && make docker-test-integration

All the necessary packages will be installed, build flags will be applied and additional parameters will be passed to cAdvisor automatically. Configuration is performed using shell environment variables.

VM-base tests (legacy)

The cAdvisor integration tests are run per-pr using the kubernetes node-e2e testing framework on GCE instances. To make use of this framework, complete the setup of GCP described in the node-e2e testing framework, clone k8s.io/kubernetes, and from that repository run:

text
$ make test-e2e-node TEST_SUITE=cadvisor REMOTE=true

This will create a VM, build cadvisor, run integration tests on that VM, retrieve logs, and will clean up the test afterwards. See the node-e2e testing documentation for more running options.

To simply run the tests against an existing cAdvisor:

text
$ go test github.com/google/cadvisor/integration/tests/... -host=HOST -port=PORT

Note that HOST and PORT default to localhost and 8080 respectively.
Today We only support remote execution in Google Compute Engine since that is where we run our continuous builds.

File: docs/development/issues.md

GitHub Issue tracking cAdvisor

This document outlines the process around GitHub issue tracking for cAdvisor at https://github.com/google/cadvisor/issues

Labels

A brief explanation of what issue labels mean. Most labels also apply to pull requests, but for pull
requests which reference an issue, it is not necessary to copy the same labels to the PR.

  • area/API - For issues related to the API.
  • area/UI - For issues related to the web UI.
  • area/documentation - For issues related to the documentation (inline comments or markdown).
  • area/performance - For issues related to cAdvisor performance (speed, memory, etc.).
  • area/storage - For issues related to cAdvisor storage plugins.
  • area/testing - For issues related to testing (integration tests, unit tests, jenkins, etc.)
  • closed/duplicate - For issues which have been closed as duplicates of another issue. The final
    comment on the issue should hold a reference the duplicate issue.
  • closed/infeasible - For issues which cannot be resolved (e.g. a request for a feature we cannot
    or do not want to add).
  • community-assigned - For issues which are being worked on by a community member (when github won't let us assign the issue to them).
  • kind/bug - For issues referring to a bug in the existing implementation.
  • kind/enhancement - For issues proposing an enhancement or new feature.
  • kind/support - For issues which might just be user confusion / environment setup. If support
    issue ends up requiring a PR, it should probably be relabeled (for example, to bug). Many
    support issues may indicate a shortcoming of the documentation.
  • help wanted - For issues which have been highlighted as a good place to contribute to
    cAdvisor. help wanted issues could be enhancements that the core team is unlikely to get to in
    the near future, or small projects which might be a good starting point. Lack of a help wanted
    label does not mean we won't accept contributions, it only means it was not identified as a
    candidate project for community contributions.

File: docs/development/README.md

The development directory holds documentation for cAdvisor developers and contributors. If you
are looking for development using cAdvisor (as opposed to development of cAdvisor), then these
documents probably don't apply to you.

File: docs/development/releasing.md

cAdvisor Release Instructions

1. Send Release PR

Example: https://github.com/google/cadvisor/pull/1281

Add release notes to CHANGELOG.md

  • Tip: Use a github PR search to find changes since the last release
    is:pr is:merged merged:>2016-04-21

2. Create the release tag

2.a Create the release branch (only for major/minor releases)

Skip this step for patch releases.

text
# Example version
VERSION=v0.23
PATCH_VERSION=$VERSION.0
# Sync to HEAD, or the commit to branch at
git fetch upstream && git checkout upstream/master
# Create the branch
git branch release-$VERSION
# Push it to upstream
git push [email protected]:google/cadvisor.git release-$VERSION

2.b Tag the release (for all releases)

text
# Example patch version
VERSION=v0.23
PATCH_VERSION=$VERSION.0
# Checkout the release branch
git fetch upstream && git checkout upstream/release-$VERSION
# Tag the release commit. If you aren't signing, ommit the -s
git tag -s -a $PATCH_VERSION
# Push it to upstream
git push [email protected]:google/cadvisor.git $PATCH_VERSION

3. Build release artifacts

Command: make release

  • Make sure your git client is synced to the release cut point
  • Use the same go version as kubernetes: dependencies.yaml
  • Tip: use https://github.com/moovweb/gvm to manage multiple go versions.
  • Try to build it from the release branch, since we include that in the binary version
  • Verify the ldflags output, in particular check the Version, BuildUser, and GoVersion are expected

Once the build is complete, copy the output after Release info... and save it to use in step 5

Example:

text
Multi Arch Container:
gcr.io/cadvisor/cadvisor:v0.44.1-test-8

Architecture Specific Containers:
gcr.io/cadvisor/cadvisor-arm:v0.44.1-test-8
gcr.io/cadvisor/cadvisor-arm64:v0.44.1-test-8
gcr.io/cadvisor/cadvisor-amd64:v0.44.1-test-8

Binaries:
SHA256 (cadvisor-v0.44.1-test-8-linux-arm64) = e5e3f9e72208bc6a5ef8b837473f6c12877ace946e6f180bce8d81edadf66767
SHA256 (cadvisor-v0.44.1-test-8-linux-arm) = 7d714e495a4f50d9cc374bd5e6b5c6922ffa40ff1cc7244f2308f7d351c4ccea
SHA256 (cadvisor-v0.44.1-test-8-linux-amd64) = ea95c5a6db8eecb47379715c0ca260a8a8d1522971fd3736f80006c7f6cc9466

4. Check that the Containers for the release work

The only argument to the script is the tag of the Multi Arch Container from step
3. To verify that the container images for the release were built successfully,
use the check_container.sh script. The script will start each cadvisor image and
curl the /healthz endpoint to confirm that it is working.

Running this script requires that you have installed qemu-user-static and
configured qemu as a binary interpreter.

text
$ sudo apt install qemu-user-static
$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

The only argument to the script is the tag of the Multi Arch Container from step
3.

sh
build/check_container.sh gcr.io/tstapler-gke-dev/cadvisor:v0.44.1-test-8

5. Cut the release

Go to https://github.com/google/cadvisor/releases and click "Draft a new
release"

  • "Tag version" and "Release title" should be preceded by 'v' and then the
    version. Select the tag pushed in step 2.b
  • Copy an old release as a template (e.g.
    github.com/google/cadvisor/releases/tag/v0.23.1)
  • Body should start with release notes (from CHANGELOG.md)
  • Next are the docker images and binary hashes you copied (from step 3).
  • Upload the binaries build in step 3, they are located in the _output
    directory.
  • If this is a minor version release, mark the release as a "pre-release"
  • Click publish when done

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →