buf Documentation

Technical Documentation: bufbuild/buf

ℹ️ Provenance: Hybrid Fusion: bufbuild/buf (README + 2 In-Tree Chapters) · CodeWiki Reference · Recency: Active (< 180 days)

1. Project Overview & Quickstart (bufbuild/buf) Copy Chapter

Buf

Buf is the modern toolchain for Protobuf. It replaces day-to-day protoc use with a fast compiler, module-aware workspaces, formatting, linting, breaking-change detection, code generation, dependency management, API calls, and a client for the Buf Schema Registry.

If you are still driving Protobuf with shell scripts around protoc -I ..., Buf is the upgrade you want: the same schema language, the same generated-code plugin model, fewer moving parts, and a direct path from local .proto files to governed, versioned APIs.

Start Copy Chapter

Install buf with Homebrew:

brew install bufbuild/buf/buf

Initialize a workspace and run the checks you should expect every Protobuf repository to pass:

buf config init
buf build
buf format -w
buf lint
buf breaking --against '.git#branch=main'

Generate code from a checked-in buf.gen.yaml instead of a hand-maintained shell command:

buf generate

For a guided walkthrough from an empty workspace to a working Connect service, run the Buf CLI quickstart.

Why Buf wins Copy Chapter

Core CLI features work without a BSR account. Signing in to the registry adds distribution, remote plugins, generated SDKs, hosted docs, dependency resolution for private modules, and server-side checks when you need them.

Core workflow Copy Chapter

Buf treats a directory tree of .proto files as a module, and a project as a workspace. A small buf.yaml is enough to make build, lint, breaking-change detection, generation, dependency resolution, and publishing agree on the same input.

version: v2
modules:
  - path: proto
lint:
  use:
    - STANDARD
breaking:
  use:
    - FILE

From there, the useful commands are deliberately boring:

buf build
buf format -w
buf lint
buf breaking --against '.git#branch=main'
buf generate
buf push

buf build compiles the workspace. buf lint catches API-shape problems while the author is still editing. buf breaking compares the current schema against a previous version and flags source, JSON, or wire-format incompatibilities. buf generate runs protoc plugins from a checked-in template. buf push publishes named modules to the BSR.

Code generation Copy Chapter

buf generate is compatible with the normal protoc plugin model, but it moves generation into versioned configuration. This example generates Go Protobuf types and ConnectRPC handlers from proto/ using remote plugins hosted on the BSR:

version: v2
clean: true
managed:
  enabled: true
  override:
    - file_option: go_package_prefix
      value: github.com/acme/weather/gen/go
plugins:
  - remote: buf.build/protocolbuffers/go
    out: gen/go
    opt: paths=source_relative
  - remote: buf.build/connectrpc/gosimple
    out: gen/go
    opt:
      - paths=source_relative
      - simple
inputs:
  - directory: proto

Remote plugins remove the need to install and maintain generator binaries on every developer machine or CI runner. Managed mode lets API producers keep language-specific file options out of .proto files while consumers still get correct generated package names for their target language.

Local plugins work too. If a plugin speaks the standard Protobuf plugin protocol, Buf can run it.

Breaking changes Copy Chapter

Protobuf compatibility is not one thing. Renaming a field can break generated source code while preserving the binary wire format; changing a field from int32 to string breaks every existing serialized message. buf breaking makes that distinction explicit with rule categories for FILE, PACKAGE, WIRE_JSON, and WIRE compatibility.

buf breaking --against '.git#branch=main'

--against accepts a Git branch, a BSR module, a tarball, a zip file, a local directory, or a prebuilt Buf image. That matters in real repositories: the same command works on a laptop, in CI, and in release automation.

Buf Schema Registry Copy Chapter

Buf Schema Registry is a Protobuf-aware registry. It stores modules, verifies they compile, renders documentation, resolves dependencies, hosts remote plugins, produces generated SDKs, and can enforce schema checks before a breaking change reaches consumers.

buf push

Pushing a module to the BSR gives your organization a source of truth for Protobuf APIs. Consumers can depend on the schema as a BSR module, install generated SDKs from their normal package manager, or use the BSR docs to inspect services, messages, fields, enums, references, and historical commits.

Related projects Copy Chapter

Buf is most useful when schemas drive more than code generation. ConnectRPC uses Protobuf schemas to build simple HTTP APIs that support Connect, gRPC, and gRPC-Web without separate service definitions. Protobuf-ES gives JavaScript and TypeScript users a modern Protobuf runtime and generator. Protovalidate puts validation rules in the schema and runs them consistently across languages.

One contract should drive the whole workflow: compile, lint, compatibility checks, generated clients and servers, validation, API calls, package publishing, and governed changes.

Installation Copy Chapter

Homebrew installs the buf, protoc-gen-buf-breaking, and protoc-gen-buf-lint binaries, plus shell completion scripts for Bash, Fish, PowerShell, and zsh.

brew install bufbuild/buf/buf

Other supported installation methods include npm, Windows, Docker, binary downloads, tarballs, source builds, and minisign verification. See the installation docs for the full list.

CLI stability Copy Chapter

Buf CLI releases do not make breaking changes within a major version. Since buf reached v1.0, you can expect no breaking changes until v2.0. We have no plans to release v2.0.

This policy does not apply to commands behind the buf beta gate. Expect breaking changes for beta commands until they are promoted.

Documentation Copy Chapter

• Buf CLI
• CLI quickstart
• Modules and workspaces
• Code generation
• Linting
• Breaking-change detection
• Formatting
• Calling APIs with buf curl
• Buf Schema Registry
• Generated SDKs
• Remote plugins
• Schema checks
• Migrating from protoc

Community Copy Chapter

For help and discussion around Protobuf, best practices, and Buf, join us on Slack.

For bugs, feature requests, and technical questions, open an issue in this repository or email [email protected]. For general inquiries, email [email protected].

2. In-Tree Documentation Chapters (bufbuild/buf) Copy Chapter

File: README.md Copy Chapter

Buf

Buf is the modern toolchain for Protobuf. It replaces day-to-day protoc use with a fast compiler, module-aware workspaces, formatting, linting, breaking-change detection, code generation, dependency management, API calls, and a client for the Buf Schema Registry.

If you are still driving Protobuf with shell scripts around protoc -I ..., Buf is the upgrade you want: the same schema language, the same generated-code plugin model, fewer moving parts, and a direct path from local .proto files to governed, versioned APIs.

Start Copy Chapter

Install buf with Homebrew:

brew install bufbuild/buf/buf

Initialize a workspace and run the checks you should expect every Protobuf repository to pass:

buf config init
buf build
buf format -w
buf lint
buf breaking --against '.git#branch=main'

Generate code from a checked-in buf.gen.yaml instead of a hand-maintained shell command:

buf generate

For a guided walkthrough from an empty workspace to a working Connect service, run the Buf CLI quickstart.

Why Buf wins Copy Chapter

Core CLI features work without a BSR account. Signing in to the registry adds distribution, remote plugins, generated SDKs, hosted docs, dependency resolution for private modules, and server-side checks when you need them.

Core workflow Copy Chapter

Buf treats a directory tree of .proto files as a module, and a project as a workspace. A small buf.yaml is enough to make build, lint, breaking-change detection, generation, dependency resolution, and publishing agree on the same input.

version: v2
modules:
  - path: proto
lint:
  use:
    - STANDARD
breaking:
  use:
    - FILE

From there, the useful commands are deliberately boring:

buf build
buf format -w
buf lint
buf breaking --against '.git#branch=main'
buf generate
buf push

buf build compiles the workspace. buf lint catches API-shape problems while the author is still editing. buf breaking compares the current schema against a previous version and flags source, JSON, or wire-format incompatibilities. buf generate runs protoc plugins from a checked-in template. buf push publishes named modules to the BSR.

Code generation Copy Chapter

buf generate is compatible with the normal protoc plugin model, but it moves generation into versioned configuration. This example generates Go Protobuf types and ConnectRPC handlers from proto/ using remote plugins hosted on the BSR:

version: v2
clean: true
managed:
  enabled: true
  override:
    - file_option: go_package_prefix
      value: github.com/acme/weather/gen/go
plugins:
  - remote: buf.build/protocolbuffers/go
    out: gen/go
    opt: paths=source_relative
  - remote: buf.build/connectrpc/gosimple
    out: gen/go
    opt:
      - paths=source_relative
      - simple
inputs:
  - directory: proto

Remote plugins remove the need to install and maintain generator binaries on every developer machine or CI runner. Managed mode lets API producers keep language-specific file options out of .proto files while consumers still get correct generated package names for their target language.

Local plugins work too. If a plugin speaks the standard Protobuf plugin protocol, Buf can run it.

Breaking changes Copy Chapter

Protobuf compatibility is not one thing. Renaming a field can break generated source code while preserving the binary wire format; changing a field from int32 to string breaks every existing serialized message. buf breaking makes that distinction explicit with rule categories for FILE, PACKAGE, WIRE_JSON, and WIRE compatibility.

buf breaking --against '.git#branch=main'

--against accepts a Git branch, a BSR module, a tarball, a zip file, a local directory, or a prebuilt Buf image. That matters in real repositories: the same command works on a laptop, in CI, and in release automation.

Buf Schema Registry Copy Chapter

Buf Schema Registry is a Protobuf-aware registry. It stores modules, verifies they compile, renders documentation, resolves dependencies, hosts remote plugins, produces generated SDKs, and can enforce schema checks before a breaking change reaches consumers.

buf push

Pushing a module to the BSR gives your organization a source of truth for Protobuf APIs. Consumers can depend on the schema as a BSR module, install generated SDKs from their normal package manager, or use the BSR docs to inspect services, messages, fields, enums, references, and historical commits.

Related projects Copy Chapter

Buf is most useful when schemas drive more than code generation. ConnectRPC uses Protobuf schemas to build simple HTTP APIs that support Connect, gRPC, and gRPC-Web without separate service definitions. Protobuf-ES gives JavaScript and TypeScript users a modern Protobuf runtime and generator. Protovalidate puts validation rules in the schema and runs them consistently across languages.

One contract should drive the whole workflow: compile, lint, compatibility checks, generated clients and servers, validation, API calls, package publishing, and governed changes.

Installation Copy Chapter

Homebrew installs the buf, protoc-gen-buf-breaking, and protoc-gen-buf-lint binaries, plus shell completion scripts for Bash, Fish, PowerShell, and zsh.

brew install bufbuild/buf/buf

Other supported installation methods include npm, Windows, Docker, binary downloads, tarballs, source builds, and minisign verification. See the installation docs for the full list.

CLI stability Copy Chapter

Buf CLI releases do not make breaking changes within a major version. Since buf reached v1.0, you can expect no breaking changes until v2.0. We have no plans to release v2.0.

This policy does not apply to commands behind the buf beta gate. Expect breaking changes for beta commands until they are promoted.

Documentation Copy Chapter

• Buf CLI
• CLI quickstart
• Modules and workspaces
• Code generation
• Linting
• Breaking-change detection
• Formatting
• Calling APIs with buf curl
• Buf Schema Registry
• Generated SDKs
• Remote plugins
• Schema checks
• Migrating from protoc

Community Copy Chapter

For help and discussion around Protobuf, best practices, and Buf, join us on Slack.

For bugs, feature requests, and technical questions, open an issue in this repository or email [email protected]. For general inquiries, email [email protected].

File: private/pkg/protosourcepath/README.md Copy Chapter

`protosourcepath`

protosourcepath is a simple package that takes a Protobuf source path and
returns a list of associated source paths. A Protobuf source path is a
SourceCodeInfo.Location path, which is an array of integers of a variable length
that identifies a Protobuf definition. Each element of the source path represents a field
number from a descriptor proto or an index, and they form a path from the FileDescriptorProto
to the definition itself. An index is needed for any repeated types on the descriptor proto,
such as messages, enums, services, and extensions on FileDescriptorProto.

For example, let's say we have the following source path:

[4, 0, 2, 0, 1]

This path represents .message_type(0).field(0).name, which is the name of field at index
0 for the message at index 0. We can break down the source path by following the numbers,
starting from FileDescriptorProto:

• 4 is the field number of message_type on FileDescriptorProto, which
  is a repeated field representing message declarations in the file. 0 is the index of the
  message for this path.
• 2 is the field number of field on DescriptorProto, which is a repeated field
  representing field declarations of a message. 0 is the index of the field for this path.
• 1 is the field number of name on FieldDescriptorProto, which is a field
  representing the name of field declarations.

All source paths start from FileDescriptorProto and end at the Protobuf definition they
are pointing to. More details on source paths can be found in descriptor.proto.

Source paths are useful because they can be used to retrieve the SourceCodeInfo
of Protobuf definitions from their FileDescriptorProto's. SourceCodeInfo provides
location-based metadata of Protobuf definitions, including comments attached to the Protobuf
definition, which we use to look for comment ignores for lint checks. And in the case of comment
ignores, a list of associated paths would allow us to check "associated locations" as potential
sites for defining a comment ignore (e.g. for a lint rule that checks fields, we want users to
be able to define a comment ignore on a specific field, but also the message a field belongs
to, since they could use that to ignore this rule for all fields for that message instead of
defining individual comment ignores for each field).

Associated paths Copy Chapter

Associated paths are source paths that we consider "associated" with the given source path,
which we define as parent paths or child paths.

Parent paths are valid source paths to "complete" Protobuf declarations that are equal or "closer"
to the FileDescriptorProto than the given source path. A "complete" Protobuf declaration starts
from either the keyword (e.g. message or enum) or label/name (e.g. fields may or may not
have a label, and enum values would start at the name) and terminates at the opening brace
or semicolon respectively. For example, the path we looked at earlier, [4, 0, 2, 0, 1], one of
the parent paths would be [4, 0, 2, 0], which is the complete field declaration of message_type(0).field(0)
(which starts at the label of the field and terminates at the semicolon). Parent paths are
always "complete" Protobuf declarations. The following is a breakdown of what we consider as parent paths:

• For each top-level declaration (e.g. messages, enums, services, extensions, options), we consider the
  complete declaration as a parent path. This means that a given path can be one of its parent paths,
  (e.g. if the given path is [4, 0, 2, 0], then [4, 0, 2, 0] would be considered one of
  the parent paths).
• For each nested declaration (e.g. field; enum values; options; nested messages, enums, and
  extensions), we consider the complete declaration of the Protobuf definition as a parent path,
  and the paths of the complete declarations of all parent types as parent paths (e.g. if the
  given path is [4, 0, 3, 2], which is a path to the complete declaration of a nested message,
  then the path itself, [4, 0, 3, 2] is considered a parent path and [4, 0], the path of the complete
  declaration of the parent message is considered a parent path).
• For each specific attribute (e.g. name, label, field number, enum value number, etc.), we consider the complete
  declaration of the Protobuf definition as a parent path (as illustrated in the initial example,
  [4, 0, 2, 0, 1], given the path to a field name, the complete declaration of the field
  and the complete declaration of the message would be considered parent paths).

Child paths are valid source paths that are not complete Protobuf declarations that are
equal or "closer" to the FileDescriptorProto than the given source path. Going back to our
example path, [4, 0, 2, 0, 1], a path to a field name, it would be considered its own child
path, since it is not a complete Protobuf declaration, and other associated child paths would
include the field number, label, type, and type name. In addition, the associated child paths
of its parent type would also be considered associated child paths, in this case, the path
to the message name.

Details examples for associated paths can be found through the tests.

API Copy Chapter

There is a single function, GetAssociatedSourcePaths, that takes a protoreflect.SourcePath
and returns a list of associated paths.

func GetAssociatedSourcePaths(
	sourcePath protoreflect.SourcePath,
) ([]protoreflect.SourcePath, error)

We expect there always to be at least one associated path, the path itself.

Future Copy Chapter

We are currently returning all associated source paths, but we have the option to exclude
child paths/Protobuf declarations that are not complete, since our use-case is primarily to
get comments, which should only be attached to complete Protobuf declarations. However, it
is currently inexpensive to check all associated source paths, so we have not exposed that
functionality on the exported function.

--- METRICS ---

• Files Extracted: 3
• Estimated Token Budget: ~7041 tokens
• Recency Window: Active (< 180 days)
• Canonical Reference: https://codewiki.google/github.com/bufbuild/buf