## File: README.md

cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers. Specifically, for each container it keeps resource isolation parameters, historical resource usage, histograms of complete historical resource usage and network statistics. This data is exported by container and machine-wide.

cAdvisor has native support for [Docker](https://github.com/docker/docker) containers and should support just about any other container type out of the box. We strive for support across the board so feel free to open an issue if that is not the case. cAdvisor's container abstraction is based on [lmctfy](https://github.com/google/lmctfy)'s so containers are inherently nested hierarchically.

#### Quick Start: Running cAdvisor in a Docker Container

To quickly tryout cAdvisor on your machine with Docker, we have a Docker image that includes everything you need to get started. You can run a single cAdvisor to monitor the whole machine. Simply run:

```
VERSION=0.55.1 # use the latest release version from https://github.com/google/cadvisor/releases
sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  --privileged \
  --device=/dev/kmsg \
  ghcr.io/google/cadvisor:$VERSION # for versions prior to v0.53.0, use gcr.io/cadvisor/cadvisor instead
```

cAdvisor is now running (in the background) on `http://localhost:8080`. The setup includes directories with Docker state cAdvisor needs to observe.

**Note**: If you're running on CentOS, Fedora, or RHEL (or are using LXC), take a look at our [running instructions](docs/running.md).

We have detailed [instructions](docs/running.md#standalone) on running cAdvisor standalone outside of Docker. cAdvisor [running options](docs/runtime_options.md) may also be interesting for advanced usecases. If you want to build your own cAdvisor Docker image, see our [deployment](docs/deploy.md) page.

For [Kubernetes](https://github.com/kubernetes/kubernetes) users, cAdvisor can be run as a daemonset. See the [instructions](deploy/kubernetes) for how to get started, and for how to [kustomize](https://github.com/kubernetes-sigs/kustomize#kustomize) it to fit your needs.

## Building and Testing

See the more detailed instructions in the [build page](docs/development/build.md). This includes instructions for building and deploying the cAdvisor Docker image.

## Exporting stats

cAdvisor supports exporting stats to various storage plugins. See the [documentation](docs/storage/README.md) for more details and examples.

## Web UI

cAdvisor exposes a web UI at its port:

`http://<hostname>:/`

See the [documentation](docs/web.md) for more details.

## Remote REST API & Clients

cAdvisor exposes its raw and processed stats via a versioned remote REST API. See the API's [documentation](docs/api.md) for more information.

There is also an official Go client implementation in the [client](client/) directory. See the [documentation](docs/clients.md) for more information.

## Roadmap

cAdvisor aims to improve the resource usage and performance characteristics of running containers. Today, we gather and expose this information to users. In our roadmap:

- Advise on the performance of a container (e.g.: when it is being negatively affected by another, when it is not receiving the resources it requires, etc).
- Auto-tune the performance of the container based on previous advise.
- Provide usage prediction to cluster schedulers and orchestration layers.

## Community

Contributions, questions, and comments are all welcomed and encouraged! cAdvisor developers hang out on [Slack](https://kubernetes.slack.com) in the #sig-node channel (get an invitation [here](http://slack.kubernetes.io/)). We also have [discuss.kubernetes.io](https://discuss.kubernetes.io/).

Please reach out and get involved in the project, we're actively looking for more contributors to bring on board!

### Core Team
* [@bobbypage, Google](https://github.com/bobbypage)
* [@iwankgb, Independent](https://github.com/iwankgb)
* [@creatone, Independent](https://github.com/creatone)
* [@dims, VMWare](https://github.com/dims)
* [@mrunalp, RedHat](https://github.com/mrunalp)

### Frequent Collaborators
* [@haircommander, RedHat](https://github.com/haircommander)

### Emeritus
* [@dashpole, Google](https://github.com/dashpole)
* [@dchen1107, Google](https://github.com/dchen1107)
* [@derekwaynecarr, RedHat](https://github.com/derekwaynecarr)

---

## File: client/v2/README.md

# Example REST API Client

This is an implementation of a cAdvisor REST API in Go.  You can use it like this:

```go
client, err := client.NewClient("http://192.168.59.103:8080/")
```

Obviously, replace the URL with the path to your actual cAdvisor REST endpoint.


### MachineInfo

```go
client.MachineInfo()
```

There is no v2 MachineInfo API, so the v2 client exposes the [v1 MachineInfo](../../info/v1/machine.go#L131)

```
(*v1.MachineInfo)(0xc208022b10)({
 NumCores: (int) 4,
 MemoryCapacity: (int64) 2106028032,
 Filesystems: ([]v1.FsInfo) (len=1 cap=4) {
  (v1.FsInfo) {
   Device: (string) (len=9) "/dev/sda1",
   Capacity: (uint64) 19507089408
  }
 }
})
```

You can see the full specification of the [MachineInfo struct in the source](../../info/v1/machine.go#L131)

### VersionInfo

```go
client.VersionInfo()
```

This method returns the cAdvisor version.

### Attributes

```go
client.Attributes()
```

This method returns a [cadvisor/info/v2/Attributes](../../info/v2/machine.go#L24) struct with all the fields filled in. Attributes includes hardware attributes (as returned by MachineInfo) as well as software attributes (eg. software versions). Here is an example return value:

```
(*v2.Attributes)({
 KernelVersion: (string) (len=17) "3.13.0-44-generic"
 ContainerOsVersion: (string) (len=18) "Ubuntu 14.04.1 LTS"
 DockerVersion: (string) (len=9) "1.5.0-rc4"
 CadvisorVersion: (string) (len=6) "0.10.1"
 NumCores: (int) 4,
 MemoryCapacity: (int64) 2106028032,
 Filesystems: ([]v2.FsInfo) (len=1 cap=4) {
  (v2.FsInfo) {
   Device: (string) (len=9) "/dev/sda1",
   Capacity: (uint64) 19507089408
  }
 }
})
```

You can see the full specification of the [Attributes struct in the source](../../info/v2/machine.go#L24)

---

## File: client/README.md

# Example REST API Client

This is an implementation of a cAdvisor REST API in Go.  You can use it like this:

```go
client, err := client.NewClient("http://192.168.59.103:8080/")
```

Obviously, replace the URL with the path to your actual cAdvisor REST endpoint.


### MachineInfo

```go
client.MachineInfo()
```

This method returns a cadvisor/v1.MachineInfo struct with all the fields filled in.  Here is an example return value:

```
(*v1.MachineInfo)(0xc208022b10)({
 NumCores: (int) 4,
 MemoryCapacity: (int64) 2106028032,
 Filesystems: ([]v1.FsInfo) (len=1 cap=4) {
  (v1.FsInfo) {
   Device: (string) (len=9) "/dev/sda1",
   Capacity: (uint64) 19507089408
  }
 }
})
```

You can see the full specification of the [MachineInfo struct in the source](../info/v1/machine.go#L131)

### ContainerInfo

Given a container name and a [ContainerInfoRequest](../info/v1/container.go#L101), will return all information about the specified container.  See the [ContainerInfoRequest struct in the source](../info/v1/container.go#L101) for the full specification.

```go
request := v1.ContainerInfoRequest{NumStats: 10}
sInfo, err := client.ContainerInfo("/docker/d9d3eb10179e6f93a...", &request)
```
Returns a [ContainerInfo struct](../info/v1/container.go#L128)

### SubcontainersInfo

Given a container name and a [ContainerInfoRequest](../info/v1/container.go#L101), will recursively return all info about the container and all subcontainers contained within the container.  See the [ContainerInfoRequest struct in the source](../info/v1/container.go#L101) for the full specification.

```go
request := v1.ContainerInfoRequest{NumStats: 10}
sInfo, err := client.SubcontainersInfo("/docker", &request)
```

Returns a [ContainerInfo struct](../info/v1/container.go#L128) with the Subcontainers field populated.

---

## File: cmd/internal/storage/bigquery/README.md

BigQuery Storage Driver
=======

[EXPERIMENTAL] Support for BigQuery backend as cAdvisor storage driver.
The current implementation takes bunch of BigQuery specific flags for authentication.
These will be merged into a single backend config.

To run the current version, following flags need to be specified:
```
 # Storage driver to use.
 -storage_driver=bigquery
 
 # Information about server-to-server Oauth token.
 # These can be obtained by creating a Service Account client id under `Google Developer API`
 
 # service client id
 -bq_id="XYZ.apps.googleusercontent.com"
 
 # service email address
 -bq_account="ABC@developer.gserviceaccount.com"
 
 # path to pem key (converted from p12 file)
 -bq_credentials_file="/path/to/key.pem"
 
 # project id to use for storing datasets.
 -bq_project_id="awesome_project"
```

See [Service account Authentication](https://developers.google.com/accounts/docs/OAuth2) for Oauth related details.

---

## File: deploy/kubernetes/README.md

# cAdvisor Kubernetes Daemonset

cAdvisor uses [Kustomize](https://github.com/kubernetes-sigs/kustomize) to manage Kubernetes manifests. See the [Install Kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/) for installation instructions, and for a description of how it works.

## Deploy

Pick a [cAdvisor release](https://github.com/google/cadvisor/releases)
```
VERSION=v0.42.0
```

Deploy to Kubernetes cluster with [remote build](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md):
```
kustomize build "https://github.com/google/cadvisor/deploy/kubernetes/base?ref=${VERSION}" | kubectl apply -f -
```

## Usage

To update the image version([reference](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/image.md)):
```
cd deploy/kubernetes/base && kustomize edit set image ghcr.io/google/cadvisor:${VERSION} && cd ../../..
```

To generate the base daemonset:
```
kubectl kustomize deploy/kubernetes/base
```

To apply the base daemonset to your cluster:
```
kubectl kustomize deploy/kubernetes/base | kubectl apply -f -
```

To generate the daemonset with example patches applied:
```
kubectl kustomize deploy/kubernetes/overlays/examples
```

To apply the daemonset to your cluster with example patches applied:
```
kubectl kustomize deploy/kubernetes/overlays/examples | kubectl apply -f -
```

### cAdvisor with perf support on Kubernetes

Example of modifications needed to deploy cAdvisor with perf support is provided in [overlays/examples_perf](overlays/examples_perf) directory (modification to daemonset and configmap with perf events configuration).

To generate and apply the daemonset with patches for cAdvisor with perf support:
```
kubectl kustomize deploy/kubernetes/overlays/examples_perf | kubectl apply -f -
```

## Kustomization

On your own fork of cAdvisor, create your own overlay directory with your patches.  Copy patches from the example folder if you intend to use them, but don't modify the originals.  Commit your changes in your local branch, and use git to manage them the same way you would any other piece of code.

To run the daemonset with your patches applied:
```
kubectl kustomize deploy/kubernetes/overlays/<my_custom_overlays> | kubectl apply -f -
```

To get changes made to the upstream cAdvisor daemonset, simply rebase your fork of cAdvisor on top of upstream.  Since you didn't make changes to the originals, you won't have any conflicts.

---

## File: docs/development/build.md

# Building and Testing cAdvisor

**Note**: cAdvisor only builds on Linux since it uses Linux-only APIs.

## Installing Dependencies

cAdvisor is written in the [Go](http://golang.org) programming language. If you haven't set up a Go development environment, please follow [these instructions](http://golang.org/doc/code.html) to install go tool and set up GOPATH. Note that the version of Go in package repositories of some operating systems is outdated, so please [download](https://golang.org/dl/) the latest version.

**Note**: cAdvisor requires Go 1.14 to build.

After setting up Go, you should be able to `go get` cAdvisor as expected (we use `-d` to only download):

```
$ go get -d github.com/google/cadvisor
```

## Building from Source

At this point you can build cAdvisor from the source folder:

```
$GOPATH/src/github.com/google/cadvisor $ make build
```

or run only unit tests:

```
$GOPATH/src/github.com/google/cadvisor $ make test
```

For integration tests, see the [integration testing](integration_testing.md) page.

### Non-volatile Memory Support

cAdvisor can be linked against [libipmctl](https://github.com/intel/ipmctl) library that allows to gather information about Intel® Optane™ DC Persistent memory. If you want to build cAdvisor with libipmctl support you must meet following requirements:
* `libipmctl-devel` must be installed on build system.
* `libipmctl` must be installed on all systems where cAdvisor is running.

Detailed information about building `libipmctl` can be found in the project's [README](https://github.com/intel/ipmctl#build). Make sure to use the most up to date released version. Functionality that relies on `libipmctl` was tested against version 02.00.00.3820 of the library.

To enable `libipmctl` support `GO_FLAGS` variable must be set:

```
$GOPATH/src/github.com/google/cadvisor $ GO_FLAGS="-tags=libipmctl,netgo" make build
```

### Perf Support

cAdvisor can be linked against [libpfm4](http://perfmon2.sourceforge.net/) library that allows to gather information about performance monitoring events.
If you want to build cAdvisor with libpfm4 support you must meet following requirements:
* `libpfm4-dev` must be installed on build system.
* `libpfm4` must be installed on all systems where cAdvisor is running.

libpfm4 packages are available in Debian- and RHEL-derivatives distributions.

libpfm4 can be installed using apt package manager:
```
apt-get install libpfm4 libpfm4-dev
```
or yum package manager:
```
yum install libpfm libpfm-devel
```

To enable `libpfm4` support `GO_FLAGS` variable must be set:

```
$GOPATH/src/github.com/google/cadvisor $ GO_FLAGS="-tags=libpfm,netgo" make build
```

## Running Built Binary

Now you can run the built binary:

```
$GOPATH/src/github.com/google/cadvisor $ sudo ./cadvisor
```

### Perf Support

It is required to include perf config (examplary config is available [here](../../perf/testing/perf-non-hardware.json)) to run cAdvisor with performance monitoring events:
```
$GOPATH/src/github.com/google/cadvisor $ sudo ./cadvisor -perf_events_config=perf/testing/perf-non-hardware.json

```

---

## File: docs/development/integration_testing.md

# Integration Testing cAdvisor 

## Docker-based tests

The cAdvisor integration tests are run per-pr using [Github Actions](https://help.github.com/en/actions). Workflow configuration can be found at [.github/workflows/test.yml](.github/workflows/test.yml). Tests are executed in Docker containers run on MS Azure virtual machines.

To run them locally Docker must be installed on your machine. Following command allows you to execute default suite of integration tests:

```
make docker-test-integration
```

Build scripts take care of building cAdvisor and integration tests, and executing them against running cAdvisor process.

In order to run non-default tests suites (e.g. such that rely on third-party C libraries) you must source one of the files available at [build/config](build/config), e.g.:

```
source build/config/libpfm4.sh && make docker-test-integration
```

All the necessary packages will be installed, build flags will be applied and additional parameters will be passed to cAdvisor automatically. Configuration is performed using shell environment variables.

## VM-base tests (legacy)

The cAdvisor integration tests are run per-pr using the [kubernetes node-e2e testing framework](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-node-tests.md) on GCE instances.  To make use of this framework, complete the setup of GCP described in the node-e2e testing framework, clone `k8s.io/kubernetes`, and from that repository run:
```
$ make test-e2e-node TEST_SUITE=cadvisor REMOTE=true
```
This will create a VM, build cadvisor, run integration tests on that VM, retrieve logs, and will clean up the test afterwards.  See the [node-e2e testing documentation](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-node-tests.md) for more running options.

To simply run the tests against an existing cAdvisor:

```
$ go test github.com/google/cadvisor/integration/tests/... -host=HOST -port=PORT
```

Note that `HOST` and `PORT` default to `localhost` and `8080` respectively.
Today We only support remote execution in Google Compute Engine since that is where we run our continuous builds.

---

## File: docs/development/issues.md

# GitHub Issue tracking cAdvisor

This document outlines the process around GitHub issue tracking for cAdvisor at https://github.com/google/cadvisor/issues

## Labels

A brief explanation of what issue labels mean. Most labels also apply to pull requests, but for pull
requests which reference an issue, it is not necessary to copy the same labels to the PR.

- `area/API` - For issues related to the API.
- `area/UI` - For issues related to the web UI.
- `area/documentation` - For issues related to the documentation (inline comments or markdown).
- `area/performance` - For issues related to cAdvisor performance (speed, memory, etc.).
- `area/storage` - For issues related to cAdvisor storage plugins.
- `area/testing` - For issues related to testing (integration tests, unit tests, jenkins, etc.)
- `closed/duplicate` - For issues which have been closed as duplicates of another issue. The final
  comment on the issue should hold a reference the duplicate issue.
- `closed/infeasible` - For issues which cannot be resolved (e.g. a request for a feature we cannot
  or do not want to add).
- `community-assigned` - For issues which are being worked on by a community member (when github won't let us assign the issue to them).
- `kind/bug` - For issues referring to a bug in the existing implementation.
- `kind/enhancement` - For issues proposing an enhancement or new feature.
- `kind/support` - For issues which might just be user confusion / environment setup. If support
  issue ends up requiring a PR, it should probably be relabeled (for example, to `bug`). Many
  support issues may indicate a shortcoming of the documentation.
- `help wanted` - For issues which have been highlighted as a good place to contribute to
  cAdvisor. `help wanted` issues could be enhancements that the core team is unlikely to get to in
  the near future, or small projects which might be a good starting point. Lack of a `help wanted`
  label does not mean we won't accept contributions, it only means it was not identified as a
  candidate project for community contributions.

---

## File: docs/development/README.md

The [development](./) directory holds documentation for cAdvisor developers and contributors. If you
are looking for development using cAdvisor (as opposed to development of cAdvisor), then these
documents probably don't apply to you.

---

## File: docs/development/releasing.md

# cAdvisor Release Instructions

## 1. Send Release PR

Example: https://github.com/google/cadvisor/pull/1281

Add release notes to [CHANGELOG.md](../../CHANGELOG.md)

- Tip: Use a github PR search to find changes since the last release
  `is:pr is:merged merged:>2016-04-21`

## 2. Create the release tag

### 2.a Create the release branch (only for major/minor releases)

Skip this step for patch releases.

```
# Example version
VERSION=v0.23
PATCH_VERSION=$VERSION.0
# Sync to HEAD, or the commit to branch at
git fetch upstream && git checkout upstream/master
# Create the branch
git branch release-$VERSION
# Push it to upstream
git push git@github.com:google/cadvisor.git release-$VERSION
```

### 2.b Tag the release (for all releases)

```
# Example patch version
VERSION=v0.23
PATCH_VERSION=$VERSION.0
# Checkout the release branch
git fetch upstream && git checkout upstream/release-$VERSION
# Tag the release commit. If you aren't signing, ommit the -s
git tag -s -a $PATCH_VERSION
# Push it to upstream
git push git@github.com:google/cadvisor.git $PATCH_VERSION
```

## 3. Build release artifacts

Command: `make release`

- Make sure your git client is synced to the release cut point
- Use the same go version as kubernetes: [dependencies.yaml](https://github.com/kubernetes/kubernetes/blob/master/build/dependencies.yaml#L101)
- Tip: use https://github.com/moovweb/gvm to manage multiple go versions.
- Try to build it from the release branch, since we include that in the binary version
- Verify the ldflags output, in particular check the Version, BuildUser, and GoVersion are expected

Once the build is complete, copy the output after `Release info...` and save it to use in step 5

Example:

```
Multi Arch Container:
gcr.io/cadvisor/cadvisor:v0.44.1-test-8

Architecture Specific Containers:
gcr.io/cadvisor/cadvisor-arm:v0.44.1-test-8
gcr.io/cadvisor/cadvisor-arm64:v0.44.1-test-8
gcr.io/cadvisor/cadvisor-amd64:v0.44.1-test-8

Binaries:
SHA256 (cadvisor-v0.44.1-test-8-linux-arm64) = e5e3f9e72208bc6a5ef8b837473f6c12877ace946e6f180bce8d81edadf66767
SHA256 (cadvisor-v0.44.1-test-8-linux-arm) = 7d714e495a4f50d9cc374bd5e6b5c6922ffa40ff1cc7244f2308f7d351c4ccea
SHA256 (cadvisor-v0.44.1-test-8-linux-amd64) = ea95c5a6db8eecb47379715c0ca260a8a8d1522971fd3736f80006c7f6cc9466
```

## 4. Check that the Containers for the release work

The only argument to the script is the tag of the Multi Arch Container from step
3. To verify that the container images for the release were built successfully,
use the check_container.sh script. The script will start each cadvisor image and
curl the `/healthz` endpoint to confirm that it is working.

Running this script requires that you have installed `qemu-user-static` and
configured qemu as a binary interpreter.

```
$ sudo apt install qemu-user-static
$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
```

The only argument to the script is the tag of the Multi Arch Container from step
3.

```sh
build/check_container.sh gcr.io/tstapler-gke-dev/cadvisor:v0.44.1-test-8
```

## 5. Cut the release

Go to https://github.com/google/cadvisor/releases and click "Draft a new
release"

- "Tag version" and "Release title" should be preceded by 'v' and then the
  version. Select the tag pushed in step 2.b
- Copy an old release as a template (e.g.
  github.com/google/cadvisor/releases/tag/v0.23.1)
- Body should start with release notes (from CHANGELOG.md)
- Next are the docker images and binary hashes you copied (from step 3).
- Upload the binaries build in step 3, they are located in the `_output`
  directory.
- If this is a minor version release, mark the release as a "pre-release"
- Click publish when done