## 1. Project Overview & Quickstart (parca-dev/CONTRIBUTING) # CONTRIBUTING Open-source repository parca-dev/CONTRIBUTING ### Repository Details - **Repository:** [parca-dev/CONTRIBUTING](https://github.com/parca-dev/CONTRIBUTING) - **Primary Language:** Code *Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.* ## 2. Official Technical Reference & Guides (parca-dev/docs) ## File: README.md # Website This website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator. ## Installation ```console yarn install ``` ## Local Development ```console yarn start ``` This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server. ## Build ```console yarn build ``` This command generates static content into the `build` directory and can be served using any static contents hosting service. ## Deployment ```console GIT_USER= USE_SSH=true yarn deploy ``` If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch. --- ## File: docs/agent-binary.mdx # Parca Agent from Binary import WithVersions from '@site/src/components/WithVersions'; import CodeBlock from '@theme/CodeBlock'; import BrowserWindow from '@site/src/components/BrowserWindow'; You can download the latest agent binary release for your architecture from our [releases page](https://github.com/parca-dev/parca-agent/releases). { versions => curl -sL https://github.com/parca-dev/parca-agent/releases/download/{versions.agent}/parca-agent_{versions.agent.substring(1)}_`uname -s`_`uname -m` -o parca-agent && \{'\n'} chmod +x parca-agent } We also produce Docker images that can be pulled: { versions => docker pull ghcr.io/parca-dev/parca-agent:{versions.agent} } To start Parca Agent, run the following command with the flag: ```shell sudo parca-agent --log-level=debug --node=binary-test --remote-store-address=localhost:7070 --remote-store-insecure ``` :::info The Agent needs to run as a privileged user to load necessary eBPF programs to Kernel. Please check [our FAQ for further information](/docs/faq#since-parca-agent-has-to-run-as-root-for-ebpf-what-are-the-security-considerations). ::: The command in the above configuration targets the Parca server that is presumably running on your system. And it uses the `systemd` service discovery to find the cgroups that have been running on your system. :::caution Note that Parca Agent's `systemd` metadata service discovery supports cgroup v1 and v2. This is only used to enrich the metadata of collected profiles. ::: You should see with some log lines: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Now we can view the active profilers by visiting `http://localhost:7071`: And all the discovered processes: Once Parca and Parca Agent are both running, you can navigate to the web interface on the browser. You should shortly see the `Select profile...` dropdown menu populate with the profiles that Parca is retrieving from itself and receiving from the Agent. Selecting `parca_agent_cpu_sample_count` as profile types and clicking the `Search` button will retrieve the profiles from Parca Agent for the time selection (default Last Hour). --- ## File: docs/agent-server-snap-services.mdx # Parca and Parca Agent as snap services import BrowserWindow from "@site/src/components/BrowserWindow" :::tip You can configure both services quickly with a simple [cloud-init](https://cloudinit.readthedocs.io/en/latest/index.html) file: ```yaml #cloud-config snap: commands: - [install, parca] - [install, parca-agent, --classic] - [set, parca-agent, remote-store-insecure=true] - [set, parca-agent, remote-store-address=localhost:7070] - [start, parca] - [start, parca-agent] ``` ::: ## Setting up Parca You can install Parca using the [snap](https://snapcraft.io/about) package. ```shell sudo snap install parca sudo snap start parca ``` This will start the Parca server on port `7070` and configure it to retrieve profiles from itself every 1 second automatically. Once Parca is running, you can navigate to the web interface on the browser. ## Setting up Parca Agent You can install Parca Agent using the [snap](https://snapcraft.io/about) package. ```shell sudo snap install --classic parca-agent sudo snap set parca-agent remote-store-insecure=true sudo snap set parca-agent remote-store-address=localhost:7070 sudo snap start parca-agent ``` The `systemd` service will be collecting profiles from `docker.service`, `parca.service` and `parca-agent.service` that have been running on your system. --- ## File: docs/binary.mdx # Parca from Binary import WithVersions from "@site/src/components/WithVersions"; import CodeBlock from "@theme/CodeBlock"; import BrowserWindow from "@site/src/components/BrowserWindow"; You can download the latest binary release for your architecture from our [releases page](https://github.com/parca-dev/parca/releases). {(versions) => ( curl -sL https://github.com/parca-dev/parca/releases/download/ {versions.server}/parca_{versions.server.substring(1)}_`uname -s`_`uname -m`.tar.gz | tar xvfz - parca )} We also produce Docker images that can be pulled: {(versions) => ( docker pull ghcr.io/parca-dev/parca:{versions.server} )} We provide an example configuration file that you can use to get Parca running, and the scrape configuration section should look familiar to anyone familiar with Prometheus. ```yaml object_storage: bucket: type: "FILESYSTEM" config: directory: "./tmp" scrape_configs: - job_name: "default" scrape_interval: "2s" static_configs: - targets: ["127.0.0.1:7070"] ``` You can download from our repository: {(versions) => ( curl -sL https://raw.githubusercontent.com/parca-dev/parca/ {versions.server}/parca.yaml > parca.yaml )} To start Parca, pass this config file with the `config-path` flag: ```shell parca --config-path="parca.yaml" ``` You should be greeted with the splash and some log lines: ```shell ooooooooo. `888 `Y88. 888 .d88' .oooo. oooo d8b .ooooo. .oooo. 888ooo88P' `P )88b `888""8P d88' `"Y8 `P )88b 888 .oP"888 888 888 .oP"888 888 d8( 888 888 888 .o8 d8( 888 o888o `Y888""8o d888b `Y8bod8P' `Y888""8o {"caller":"log.go:124","level":"info","msg":"loading bucket configuration","name":"parca","ts":"2021-09-16T01:13:18.874904009Z"} {"caller":"log.go:124","level":"debug","msg":"Starting provider","name":"parca","provider":"static/0","subs":"[default]","ts":"2021-09-16T01:13:18.876115335Z"} {"addr":":7070","caller":"server.go:84","level":"info","msg":"starting server","name":"parca","ts":"2021-09-16T01:13:18.876178645Z"} {"caller":"log.go:124","level":"debug","msg":"Discoverer channel closed","name":"parca","provider":"static/0","ts":"2021-09-16T01:13:18.876192954Z"} ``` This will start the Parca server on port `7070` and configure it to retrieve profiles from itself every 1 second automatically. Other flags can be found using the `help` flag. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Once Parca is running, you can navigate to the web interface on the browser. You should shortly see the `Select profile...` dropdown menu populate with the profiles that Parca is retrieving from itself. Selecting one of these profile types and clicking the `Search` button will retrieve the profiles of that type for the time selection (default Last Hour) This should result in a time series based on the profile that is interactable. Clicking anywhere on the line graph should then bring up an Flame Graph for the profile that you've selected. You can then interact with the Flame Graph to better understand how Parca is behaving. --- ## File: docs/binary-filtering.mdx # Binary Filtering import BrowserWindow from "@site/src/components/BrowserWindow"; Parca allows filtering profiles by their associated binary. This is useful to narrow down the scope of a profile to a specific binary or set of binaries. ### How to use it To use the binary filter feature, you need to be using a color profile that allows for binary filtering. Click on the "Preferences" button to load the preferences modal. In the preferences modal, you will see a section called "Flamegraph Color Profile". Ensure any of the color profiles except the "Neutral" option is selected. The "Neutral" option does not support binary filtering. To see the list of the binaries in the profile, click on the "Show Legend" button to show the legend. The legend shows the list of binaries in the profile. You can now click on any binary name to filter the profile by the binary. The example above specifically works for when you want to only see data in the Flame Graph for a particular binary. So how about when you want to see data in the Flame Graph excluding a particular binary? You can do this by right-clicking on the frame in the Flame Graph and then click on the "Hide Binary" option. This will hide the data for the binary you right-clicked on. This is especially useful for Python workloads that call out to native extensions, so just viewing interpreted frames is not quite right, but just hiding the Python runtime. This means other native extensions like libcuda or pytorch would still be visible. In addition to that, you can hide multiple binaries at a time! Using the same Flame Graph as above, we also want to hide the *libc.so.6* frames. To get the Flame Graph back to its initial state, there's a "Reset Legend" button to reset the Flame Graph. --- ## File: docs/commercial.md # Commercial Solutions [Polar Signals](https://polarsignals.com), the original creator of the Parca project, is the only commercial provider of Parca-compatible APIs as part of the Polar Signals Cloud product. Polar Signals also offers paid support for Parca open-source users. --- ## File: docs/community.md # Community ## Code of Conduct Parca follows [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). That includes everything conducted under the umbrella of the project, including but not limited to the Office hours, interactions on GitHub, mailing lists, and Slack. Please contact the Parca maintainers at parca-team@googlegroups.com to report any Code of Conduct violations. ## Contact You can reach out to the rest of the Parca community including the core developers on our [Discord](https://discord.com/invite/ZgUpYgpzXy). ## Contributing If you'd like to contribute, we'd love to have you do so! For major changes, please open an issue first to discuss what you would like to change. If you are not entirely sure about this, you can discuss this on the [Parca Discord](https://discord.gg/ZgUpYgpzXy) server as well. If there is an existing issue you would like to work on, please comment on the issue. For detailed instructions, please refer to the `CONTRIBUTING.md` files in [parca](https://github.com/parca-dev/parca/blob/main/CONTRIBUTING.md) and [parca-agent](https://github.com/parca-dev/parca-agent/blob/main/CONTRIBUTING.md) respectively. We use RFCs to document all things architecture and design for the Parca project. You can find an index of the RFCs [here](https://docs.google.com/document/d/171XgH4l_gxvGnETVKQBddo75jQz5aTSDOqO0EZ7LLqE/edit?usp=share_link). Parca uses [GitHub](https://github.com/parca-dev/) to manage reviews, issues and pull requests. Contributions to Parca documentation and this website can be made [here](https://github.com/parca-dev/parca.dev). --- ## File: docs/concepts.mdx # Concepts import BrowserWindow from "@site/src/components/BrowserWindow"; ## Data model Parca's multi-dimensional data-model originates from the [Prometheus](https://prometheus.io/docs/concepts/data_model/) project. It allows for concise and easy to use label-selectors to slice and dice the data when [querying](./querying-parca). ## Stable links Parca has a strong philosophy on having stable links. Observability projects work best when they can be shared with teams and so Parca does its best to represent the state of the frontend via the URL to be shared with other people via chat such as Discord, Slack or issue trackers like GitHub or GitLab. ## Pprof Parca ingests profiles using the open-source pprof profile.proto format. More can be read about pprof and the format [here](https://github.com/google/pprof/blob/master/doc/README.md). #### How is this different than the pprof utility? The pprof utility offers a robust set of tools for exploring profiles. Parca specifically tries to solve the storage and retrieval of these profiles, and allows you to explore these profiles across time. ## Flame Graphs Often to better understand how your application is behaving you need to compare two profiles. Parca allows you to make selections of two points in time that you want to compare and will generate an Flame Graph of the difference between the two selected profiles. If a span is larger in the second profile, you'll see that span colored in red, and will be darker shade the larger the difference is. This allows one to quickly visually identify where a program is growing in usage scope. You'll see in the image below with the profile selections reversed from the previous example, that the spans that are now smaller in the second selection are colored green. This allows you to quickly visually identify where improvements to usage are being made. If a span remains blue, it means there was no significant change between the two compared profiles for that section of code. ## Cumulative and Diff values When comparing two profiles, it's not unusual to see a `Cumulative` value and `Diff` value in the tooltip that displays when you hover over the Flame Graph. As an example, a stack trace like `main() > foo() > bar()` where `main` calls `foo` that calls `bar`. In one profile, we observe this stack trace running at 500ns, so its cumulative value will be 500 and then later in another profile, we see this stack trace but it was running for only 300ns, thus it's cumulative value will be 300. However, if we compare the latest stack trace that was running at 300ns to the former one running 500ns, we see that the diff is 300-500 = -200. It means that compared to the first profile we've seen this same stack trace 200ns less. --- ## File: docs/configuration.md # Configuration Parca obtains configuration information via a YAML file. The overall structure of a config file for Parca is as follows: ```yaml object_storage: bucket: type: "FILESYSTEM" config: directory: "./tmp" scrape_configs: # Section name, mandatory - job_name: "pr" # Set of query parameters with which the target is scraped. params: # Name of keys with value(s) - key_1: ['value1', 'value2'] # How frequently to scrape the targets of this scrape config. scrape_interval: "30s" # Timeout for scraping targets of this config. scrape_timeout: "40s" # URL scheme with which to fetch metrics from targets. scheme: https # Config for adding custom scrape endpoints. profiling_config: pprof_config: config_1: # Name of the profile. enabled: true path: /path/to/scrape # Add a query parameter '?seconds=' to the profile. delta: true config_2: enabled: true path: /another/path # String prefix to add to all paths specified in the pprof config. path_prefix: /debug # static_configs: - targets: [ '127.0.0.1:7070' ] # Configuration for relabeling of target label sets. relabel_configs: - source_labels: [__address__] target_label: instance ``` See [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) for details about relabelling. ## S3 configuration Parca supports using an S3-compatible bucket for storing profile data. The s3 configuration uses [the Thanos config format](https://thanos.io/tip/thanos/storage.md/#s3). For example: ```yaml object_storage: bucket: type: S3 config: bucket: "bucky", endpoint: "some.s3.endpoint:8000", access_key: "bar", secret_key: "foo", insecure: true ``` --- ## File: docs/debuginfo-cli.mdx # Debuginfo CLI import WithVersions from '@site/src/components/WithVersions'; import CodeBlock from '@theme/CodeBlock'; import BrowserWindow from '@site/src/components/BrowserWindow'; The `parca-debuginfo` CLI is an auxiliary tool that helps you to upload debug information for your binaries. It is useful when you do not want to deploy binaries with debug information to your production environments, yet you want to utilize Parca Agent. By using the `parca-debuginfo` tool, you can extract and upload the debug information from your binaries to your Parca server and then strip the debug information from the binaries before you deploy them. Read the docs on [symbolization](https://www.parca.dev/docs/symbolization) to understand the process. You can download the latest `parca-debuginfo` binary release for your architecture from our [releases page](https://github.com/parca-dev/parca-debuginfo/releases). { versions => curl -sLO https://github.com/parca-dev/parca-debuginfo/releases/download/{versions.debuginfo_cli}/parca-debuginfo_{versions.debuginfo_cli.substring(1)}_`uname -s`_`uname -m`.tar.gz } { versions => curl -sL https://github.com/parca-dev/parca-debuginfo/releases/download/{versions.debuginfo_cli}/checksums.txt | shasum --ignore-missing -a 256 --check } { versions => tar -xzf parca-debuginfo_{versions.debuginfo_cli.substring(1)}_`uname -s`_`uname -m`.tar.gz parca-debuginfo } If you want to move the CLI to a path in your `$PATH` you can move the binary: ```shell mv parca-debuginfo /usr/local/bin/parca-debuginfo ``` The `parca-debuginfo` tool has to commands: ```shell Usage: parca-debuginfo Flags: -h, --help Show context-sensitive help. --log-level="info" Log level. Commands: upload --store-address=STRING ... Upload debug information files. extract ... Extract debug information. buildid Extract buildid. Run "parca-debuginfo --help" for more information on a command. ``` `Upload` command to directly extract and upload the debug information to a Parca server: ```shell Usage: main upload --store-address=STRING ... Upload debug information files. Arguments: ... Paths to upload. Flags: -h, --help Show context-sensitive help. --log-level="info" Log level. --store-address=STRING gRPC address to sends symbols to. --bearer-token=STRING Bearer token to authenticate with store. --bearer-token-file=STRING File to read bearer token from to authenticate with store. --insecure Send gRPC requests via plaintext instead of TLS. --insecure-skip-verify Skip TLS certificate verification. ``` ```shell parca-debuginfo upload --store-address=localhost:7070 --insecure /path/to/your/binary ``` ```shell level=info ts=2021-10-07T12:28:30.710169775Z caller=debuginfo.go:109 msg="debug information uploaded successfully" buildid=324367736569664b3941566c467a727543434d622f576479554f6838735973614362395a71307441742f61754d4a7357616c32695a734e576b77337256472f3836554d764a506b7846656d333936726d417653 file=/path/to/your/binary level=info ts=2021-10-07T12:28:30.710224803Z caller=main.go:154 msg=done! ``` And another command `Extract` to extract the debug information for local investigations. ```shell Usage: main extract ... Extract debug information. Arguments: ... Paths to extract debug information. Flags: -h, --help Show context-sensitive help. --log-level="info" Log level. --output-dir="out" Output directory path to use for extracted debug information files. ``` ```shell parca-debuginfo extract --insecure /path/to/your/binary ``` ```shell level=info ts=2021-10-07T12:30:27.549546915Z caller=main.go:138 msg="debug information extracted" file=out/324367736569664b3941566c467a727543434d622f576479554f6838735973614362395a71307441742f61754d4a7357616c32695a734e576b77337256472f3836554d764a506b7846656d333936726d417653 level=info ts=2021-10-07T12:30:27.549605633Z caller=main.go:154 msg=done! ``` By default, you can find extracted information in `out` directory where you executed the tool: ```shell ls out 324367736569664b3941566c467a727543434d622f576479554f6838735973614362395a71307441742f61754d4a7357616c32695a734e576b77337256472f3836554d764a506b7846656d333936726d417653 ``` If all you want to do is extract the buildid of a binary, you can use the `buildid` command: ```shell Usage: main buildid Extract buildid. Arguments: Paths to extract buildid. Flags: -h, --help Show context-sensitive help. --log-level="info" Log level. ``` For example for your own binary use: ```shell parca-debuginfo buildid /path/to/your/binary ``` ```shell Build ID: 324367736569664b3941566c467a727543434d622f576479554f6838735973614362395a71307441742f61754d4a7357616c32695a734e576b77337256472f3836554d764a506b7846656d333936726d417653 level=info msg=done! ```