## 1. Project Overview & Quickstart (protocolbuffers/protobuf-editions-design-features)
# protobuf-editions-design-features
Open-source repository protocolbuffers/protobuf-editions-design-features
### Repository Details
- **Repository:** [protocolbuffers/protobuf-editions-design-features](https://github.com/protocolbuffers/protobuf-editions-design-features)
- **Primary Language:** Code
*Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.*
## 2. Official Technical Reference & Guides (protocolbuffers/protocolbuffers.github.io)
## File: content/best-practices/_index.md
+++
title = "Proto Best Practices"
weight = 90
description = "An overview of best practices topics."
type = "docs"
no_list = "true"
+++
Best practices content for defining and using protos exists in the following
topics:
* [Proto Best Practices](/best-practices/dos-donts)
* [1-1-1 Rule](/best-practices/1-1-1)
* [Avoid Cargo Culting](/best-practices/no-cargo-cults)
---
## File: content/best-practices/1-1-1.md
+++
title = "1-1-1 Best Practice"
weight = 105
linkTitle = "1-1-1 Best Practice"
description = "All proto definitions should have one top-level element and build target per file."
type = "docs"
aliases = "/programming-guides/1-1-1"
+++
The "1-1-1" best practice advocates structuring definitions with one top-level
entry point (exported message, exported enum, service, or extension) per
`.proto` file, corresponding to a single `proto_library` build rule. This
approach promotes small, modular proto definitions. Key benefits include
simplified refactoring, potentially improved build times, and smaller binary
sizes due to minimized transitive dependencies.
## Rationale {#rationale}
The 1-1-1 best practice is to keep every `proto_library` and `.proto` file as
small as is reasonable, with the ideal ratio being:
* One `proto_library` build rule
* One source `.proto` file
* One top-level entry point (`export message`, `export enum`, `service`, or
`extend`)
Having the fewest number of exported messages, enums, extensions, and services
as you reasonably can makes refactoring easier. Moving files when they're
separated is much easier than extracting messages from a file with other
messages.
Following this practice can help build times and binary size by reducing the
size of your transitive dependencies in practice: when some code only needs to
use one enum, under a 1-1-1 design it can depend just on the `.proto` file that
defines that enum and avoid incidentally pulling in a large set of transitive
dependencies that may only be used by another message defined in the same file.
There are cases where the 1-1-1 ideal is not possible (circular dependencies),
not ideal (extremely conceptually coupled messages which have readability
benefits by being co-located), or where some of the downsides don't apply (when
a `.proto` file has no imports, then there are no technical concerns about the
size of transitive dependencies). As with any best practice, use good judgment
for when to diverge from the guideline.
## Symbol Visibility in Edition 2024 and Later {#2024-later}
Starting in `edition = "2024"`, Protobuf introduces
[Symbol Visibility](/programming-guides/symbol_visibility)
with `export` and `local` keywords
and the `features.default_symbol_visibility` option.
> **Scope Note:** Symbol visibility controls **only** the Protobuf compiler
> (`protoc`) import behavior when resolving references between `.proto` files.
> It has **no impact** on language-specific generated code or consumers of proto
> descriptors.
### Recommended Setting: `STRICT` {#strict}
For all new `.proto` files in Edition 2024 and later file-option
`features.default_symbol_visibility` should be set to STRICT. This is the
default in edition 2026:
```proto
option features.default_symbol_visibility = STRICT;
```
In `STRICT` mode:
* All symbols default to `local`.
* Only top-level `message` and `enum` declarations intended as entry points
may be marked with `export`.
* Nested symbols cannot use `export` and are enforced `local` (preventing
accidental imports of nested types across file boundaries).
Top-level messages are designed to be **atomic** with regards to reuse. Nested
messages and enums are inherently private to their parent container. Attempting
to use another message's nested types as reusable vocabulary subverts the "use
all or none" design of `.proto`
files. Types intended for independent reuse should be
defined as top-level symbols in dedicated `.proto` files.
### Relaxation for `local` Helper Symbols {#relaxation}
Symbol visibility relaxes strict 1-1-1 for internal helper symbols: a file may
contain multiple `local message` or `local enum` definitions, provided they
support a single top-level entry point (forming a single cluster of `local`
symbols supporting one entry point). Because `local` symbols cannot be imported
into other `.proto` files, they do not introduce transitive dependency bloat to
importers.
### Service Request and Response Messages {#req-res}
Co-locating a `service` and its request/response `message`s in a single file is
acceptable in Edition 2024 if those request and response messages are marked
`local` (or default to `local` under `STRICT` mode). This prevents them from
being imported into other `.proto` files and avoids pulling the `service`
definition and its gRPC dependencies into unrelated build
targets.
However, if a request or response message ever needs to be shared or imported
outside of the service file (for example, for logging, data retention, or reuse
in another RPC), it should be defined as a top-level `export message` in its own
dedicated `.proto` file.
## Modular Schema Example {#example}
One place that modularity of proto schema files is important is when
creating gRPC definitions. The following set of proto
files shows modular structure in Edition 2024.
**student_id.proto**
```proto
edition = "2024";
package my.package;
option features.default_symbol_visibility = STRICT;
export message StudentId {
string value = 1;
}
```
**full_name.proto**
```proto
edition = "2024";
package my.package;
option features.default_symbol_visibility = STRICT;
export message FullName {
string family_name = 1;
string given_name = 2;
}
```
**student.proto**
```proto
edition = "2024";
package my.package;
option features.default_symbol_visibility = STRICT;
import "student_id.proto";
import "full_name.proto";
export message Student {
StudentId id = 1;
FullName name = 2;
}
```
**get_student_request.proto**
```proto
edition = "2024";
package my.package;
option features.default_symbol_visibility = STRICT;
import "student_id.proto";
import "external/expensive/dependency.proto";
// Export and stand-alone because it is used in both
// student_request_log.proto AND student_service.proto.
export message GetStudentRequest {
StudentId id = 1;
expensive.external.OtherMessage message = 2;
}
```
**student_request_log.proto**
```proto
edition = "2024";
package my.package;
option features.default_symbol_visibility = STRICT;
import "get_student_request.proto";
// log any requests made.
message StudentRequestLog {
string requesting_user = 1;
GetStudentRequest request = 2;
}
```
**student_service.proto**
```proto
edition = "2024";
package my.package;
option features.default_symbol_visibility = STRICT;
// imported because we log this message in student_request_log.proto meaning the
// message needs to be in its own file and `export`.
import "get_student_request.proto";
local message GetStudentResponse {
Student student = 1;
}
local message CreateStudentRequest {
FullName name = 1;
}
local message CreateStudentResponse {
Student student = 1;
}
service StudentService {
rpc CreateStudent(CreateStudentRequest) returns (CreateStudentResponse);
rpc GetStudent(GetStudentRequest) returns (GetStudentResponse);
}
```
The service definition and 3 of its 4 message definitions are defined in
`student_service.proto`. Those messages are all marked `local` making it safe to
co-locate with the `service` definition without introducing unnecessary
dependencies. `GetStudentRequest` is shared by the `StudentService` and
`StudentRequestLog`, and so it should be in its own file.
In this example, `Student`, `StudentId`, and `FullName` are public domain types
that are reusable in logs, requests, responses and external protos.
If you later need to add a `middle_name` field to the `FullName` message, you
won't need to update every individual top-level message with that new field.
Likewise, if you need to update `Student` with more information, all the
requests and responses get the update. Further, `StudentId` might update to be a
multi-part ID.
Lastly, having even simple types like `StudentId` wrapped as a message means
that you have created a type that has semantics and consolidated documentation.
For something like `FullName` you'll need to be careful with where this PII gets
logged; this is another advantage of not repeating these fields in multiple
top-level messages. You can tag those fields in one place as sensitive
and exclude them from logging.
---
## File: content/best-practices/dos-donts.md
+++
title = "Proto Best Practices"
weight = 90
description = "Shares vetted best practices for authoring Protobuf schemas."
type = "docs"
aliases = "/programming-guides/dos-donts"
+++
Clients and servers are never updated at exactly the same time - even when you
try to update them at the same time. One or the
other may get rolled back. Don’t assume that you can make a breaking change and
it'll be okay because the client and server are in sync.
## **Don't** Re-use a Tag Number {#reuse-number}
Never re-use a tag number. It messes up deserialization. Even if you think no
one is using the field, don’t re-use a tag number. If the change was live ever,
there could be serialized versions of your proto in a log
somewhere. Or there could be old code in another server that will break.
## **Do** Reserve Tag Numbers for Deleted Fields {#reserve-tag-numbers}
When you delete a field that's no longer used, reserve its tag number so that no
one accidentally re-uses it in the future. Just `reserved 2, 3;` is enough. No
type required (lets you trim dependencies!). You can also reserve names to avoid
recycling now-deleted field names: `reserved foo, bar;`. See the docs on
[reserved field numbers](/programming-guides/editions#fieldreserved)
and
[reserved field names](/programming-guides/editions#reserved-field-names).
## **Do** Reserve Numbers for Deleted Enum Values {#reserve-deleted-numbers}
When you delete an enum value that's no longer used, reserve its number so that
no one accidentally re-uses it in the future. Just `reserved 2, 3;` is enough.
You can also reserve names to avoid recycling now-deleted value names: `reserved
FOO, BAR;`. See also
[enum reserved values](/programming-guides/editions#reserved)
## **Do** Put New Enum Aliases Last {#enum-aliases-last}
When you add a new enum alias, put the new name last to give
services time to pick it up.
To safely remove the original name (if it's being used for interchange, which it
[shouldn't](#text-format-interchange)), you must do the following:
* Add the new name below the old name and deprecate the old one (serializers
will continue to use the old name)
* After every parser has the schema rolled out, swap the order of the two
names (serializers will begin using the new name, parsers accept both)
* After every serializer has that version of the schema, you can delete the
deprecated name.
> **Note:** While in theory clients shouldn't be using the old name for
> interchange, it's still polite to follow the above steps, especially for
> widely-used enum names.
## **Don't** Change the Type of a Field {#change-type}
Almost never change the type of a field; it'll mess up deserialization, same as
re-using a tag number. The
[Protobuf docs](/programming-guides/editions#updating)
outline cases where old binary encoded data may successfully parse under a
different type (for example, going between `int32`, `uint32`, `int64` and
`bool`). However, changing a field's type can be difficult to roll out safely
even when the new schema can successfully parse old data.
## **Don't** Add a Required Field {#add-required}
Never add a required field, instead add `// required` to document the API
contract. Required fields are considered harmful by so many they were
removed from proto3 completely. Make all fields
optional or repeated. You never know how long a message type is going to last
and whether someone will be forced to fill in your required field with an empty
string or zero in four years when it’s no longer logically required but the
proto still says it is.
For proto3 there are no `required` fields, so this advice does not apply.
## **Don't** Make a Message with Lots of Fields {#lots-of-fields}
Don't make a message with “lots” (think: hundreds) of fields. In C++ every field
adds roughly 65 bits to the in-memory object size whether it’s populated or not
(8 bytes for the pointer and, if the field is declared as optional, another bit
in a bitfield that keeps track of whether the field is set). When your proto
grows too large, the generated code may not even compile (for example, in Java
there is a hard limit on the size of a method
).
## **Do** Include an Unspecified Value in an Enum {#unspecified-enum}
Enums should include a default `FOO_UNSPECIFIED` value as the first value in the
declaration.
When new values are added to an enum, old clients will see the field as unset
and the getter will return the default value or the first-declared value if no
default exists . For consistent behavior with [proto enums][proto-enums],
the first declared enum value should be a default `FOO_UNSPECIFIED` value and
should use tag 0. It may be tempting to declare this default as a semantically
meaningful value but as a general rule, do not, to aid in the evolution of your
protocol as new enum values are added over time. All enum values declared under
a container message are in the same C++ namespace, so prefix the unspecified
value with the enum’s name to avoid compilation errors. If you'll never need
cross-language constants, an `int32` will preserve unknown values and generates
less code. Note that [proto enums][proto-enums] require the first value to be
zero and can round-trip (deserialize, serialize) an unknown enum value.
[example-unspecified]: http://cs/#search/&q=file:proto%20%22_UNSPECIFIED%20=%200%22&type=cs
[proto-enums]: /programming-guides/editions#enum
## **Don't** Use C/C++ Macro Constants for Enum Values {#macro-constants}
Using words that have already been defined by the C++ language - specifically,
in its headers such as `math.h`, may cause compilation errors if the `#include`
statement for one of those headers appears before the one for `.proto.h`. Avoid
using macro constants such as "`NULL`," "`NAN`," and "`DOMAIN`" as enum values.
## **Do** prefer extensions over `Any` where possible {#avoid-any}
If you plan to use `Any`, first double-check if extensions can satisfy the use
case instead. If yes, prefer using extensions.
`Any` was created in Proto3 to replace extensions. However, it has a number of
design flaws. For most use cases, prefer using extensions.
The main use case that `Any` fulfills is the uncommon scenario in which
horizontal infrastructure needs to propagate completely arbitrary messages, and
declaring an extension for each possible legal types would be infeasible.
In the future, Protobuf may introduce a new concept that would satisfy the `Any`
use cases without the design flaws, but we have no concrete plans at this time.
## **Do** Use Well-Known Types and Common Types {#well-known-common}
Using the following common, shared types is strongly encouraged. E.g., do not
use `int32 timestamp_seconds_since_epoch` or `int64 timeout_millis` in your code
when a perfectly suitable common type already exists!
* [`duration`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/duration.proto)
is a signed, fixed-length span of time (for example, 42s).
* [`timestamp`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/timestamp.proto)
is a point in time independent of any time zone or calendar (for example,
2017-01-15T01:30:15.01Z).
* [`interval`](https://github.com/googleapis/googleapis/blob/master/google/type/interval.proto)
is a time interval independent of time zone or calendar (for example,
2017-01-15T01:30:15.01Z - 2017-01-16T02:30:15.01Z).
* [`date`](https://github.com/googleapis/googleapis/blob/master/google/type/date.proto)
is a whole calendar date (for example, 2005-09-19).
* [`month`](https://github.com/googleapis/googleapis/blob/master/google/type/month.proto)
is a month of year (for example, April).
* [`dayofweek`](https://github.com/googleapis/googleapis/blob/master/google/type/dayofweek.proto)
is a day of week (for example, Monday).
* [`timeofday`](https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto)
is a time of day (for example, 10:42:23).
* [`field_mask`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/field_mask.proto)
is a set of symbolic field paths (for example, f.b.d).
* [`postal_address`](https://github.com/googleapis/googleapis/blob/master/google/type/postal_address.proto)
is a postal address (for example, 1600 Amphitheatre Parkway Mountain View,
CA 94043 USA).
* [`money`](https://github.com/googleapis/googleapis/blob/master/google/type/money.proto)
is an amount of money with its currency type (for example, 42 USD).
* [`latlng`](https://github.com/googleapis/googleapis/blob/master/google/type/latlng.proto)
is a latitude/longitude pair (for example, 37.386051 latitude and
-122.083855 longitude).
* [`color`](https://github.com/googleapis/googleapis/blob/master/google/type/color.proto)
is a color in the RGBA color space.
**Note:** While the "Well-Known Types" (such as `Duration` and `Timestamp`) are
included with the Protobuf compiler, the "Common Types" (such as `Date` and
`Money`) are not. To use the Common Types, you may need to add a dependency on
the [googleapis repository](https://github.com/googleapis/googleapis).
## **Do** Define Message Types in Separate Files {#separate-files}
When defining a proto schema, you should have a single top-level entry point
(exported message, exported enum, service, or extension) per file. This makes
refactoring easier. Moving files when they're separated is much easier than
extracting messages from a file with other messages. Following this practice
also helps to keep the proto schema files smaller, which enhances
maintainability and reduces transitive dependency bloat.
Symbol visibility (`export` / `local`) relaxes strict separation for internal
helper types: a `.proto` file may contain multiple `local message` or `local
enum` definitions supporting a single top-level entry point (such as RPC
request/response messages supporting a `service`), provided those helper types
remain local and are not imported into other files.
Messages and enums are intended to be **atomic** with regards to reuse. Nested
types (messages or enums defined inside another message) are inherently private
to their parent container. With `STRICT` symbol visibility nested entities
cannot be exported or imported by other `.proto` files at all. You should not
use another message's nested types as reusable vocabulary. Any symbol intended
for independent reuse outside its parent file should be defined as a top-level
`export message` or `export enum` in its own `.proto` file.
If a type will be widely used outside of your project, consider putting them in
their own file with no dependencies. Then it's easy for anyone to use those
types without introducing the transitive dependencies in your other proto files.
For more on this topic, see
[1-1-1 Rule](/best-practices/1-1-1).
## **Don't** Change the Default Value of a Field {#change-default-value}
Almost never change the default value of a proto field. This causes version skew
between clients and servers. A client reading an unset value will see a
different result than a server reading the same unset value when their builds
straddle the proto change. Proto3 removed the ability to
set default values.
## **Don't** Go from Repeated to Scalar {#repeated-to-scalar}
Although it won't cause crashes, you'll lose data. For JSON, a mismatch in
repeatedness will lose the whole *message*. For numeric proto3 fields and proto2
`packed` fields, going from repeated to scalar will lose all data in that
*field*. For non-numeric proto3 fields and un-annotated proto2 fields, going
from repeated to scalar will result in the last deserialized value "winning."
Going from scalar to repeated is OK in proto2 and in proto3 with
`[packed=false]` because for binary serialization the scalar value becomes a
one-element list.
## **Do** Follow the Style Guide for Generated Code {#follow-style-guide}
Proto generated code is referred to in normal code. Ensure that options in
`.proto` file do not result in generation of code which violate the style guide.
For example:
* `java_outer_classname` should follow
https://google.github.io/styleguide/javaguide.html#s5.2.2-class-names
* `java_package` and `java_alt_package` should follow
https://google.github.io/styleguide/javaguide.html#s5.2.1-package-names
* `package`, although used for Java when `java_package` is not present, always
directly corresponds to C++ namespace and thus should follow
https://google.github.io/styleguide/cppguide.html#Namespace_Names.
If these style guides conflict, use `java_package` for Java.
* `ruby_package` should be in the form `Foo::Bar::Baz` rather than
`Foo.Bar.Baz`.
## **Don't** use Text Format Messages for Interchange {#text-format-interchange}
Text-based serialization formats like text format and
JSON represent fields and enum values as strings. As a result, deserialization
of Protobuf messages in these formats using old code will fail when a field or
enum value is renamed, or when a new field or enum value or extension is added.
Use binary serialization when possible for data interchange, and use text format
for human editing and debugging only.
If you use protos converted to JSON in your API or for storing data, you may not
be able to safely rename fields or enums at all.
## **Never** Rely on Serialization Stability Across Builds {#serialization-stability}
The stability of proto serialization is not guaranteed across binaries or across
builds of the same binary. Do not rely on it when, for example, building cache
keys.
## **Don't** Generate Java Protos in the Same Java Package as Other Code {#generate-java-protos}
Generate Java proto sources into a separate package from your hand-written Java
sources. The `package`, `java_package` and `java_alt_api_package` options
control
[where the generated Java sources are emitted](/reference/java/java-generated#package).
Make sure hand-written Java source code does not also live in that same package.
A common practice is to generate your protos into a `proto` subpackage in your
project that **only** contains those protos (that is, no hand-written source
code).
## **Do** Derive Java Package from the .proto Package (if overridden) {#derive-java-package}
Setting the `java_package` can introduce fully-qualified name collisions in
generated code that did not exist in the .proto semantics. For example, these
two files may create collisions in the generated code despite the
fully-qualified names not colliding in the original schema:
```proto
package x;
option java_package = "com.example.proto";
message Abc {}
```
```proto
package y;
option java_package = "com.example.proto";
message Abc {}
```
To avoid these problems, you should never set the same `java_package` in two
files that have different .proto packages set.
The best practice is to establish a local naming pattern where the package name
is derived from the .proto package. For example, a best practice file with
`package y` might consistently set `option java_package =
"com.example.proto.y"`.
This guidance also applies to any other language-specific options where package
overrides are possible.
## Avoid Using Language Keywords for Field Names {#avoid-keywords}
If the name of a message, field, enum, or enum value is a keyword in the
language that reads from/writes to that field, then Protobuf may change the
field name, and may have different ways to access them than normal fields. For
example, see
[this warning about Python](/reference/python/python-generated#keyword-conflicts).
You should also avoid using keywords in your file paths, as this can also cause
problems.
## **Do** Use Different Messages For RPC APIs and Storage {#separate-types-for-storage}
Reusing the same messages for APIs and long-term storage may seem convenient,
reducing boilerplate and overhead of conversion between messages.
However, the needs of long-term storage and live RPC services tend to later
diverge. Using separate types even if they are largely duplicative initially
gives freedom to change your storage format without impacting your external
clients. Layer your code so that modules deal either with client protos, storage
protos, or translation.
There is a cost in maintaining the translation layer, but it quickly pays off
once you have clients and have to do your first storage changes.
## **Don't** Use Booleans for Something That Has Two States Now, but Might Have More Later {#bad-bools}
If you are using boolean for a field, make sure that the field is indeed
describing just two possible states (for all time, not just now and the near
future). The future flexibility of using an enum is often worth it, even if it
only has two values when it is first introduced.
```
message Photo {
// Bad: True if it's a GIF.
optional bool gif;
// Good: File format of the referenced photo (for example, GIF, WebP, PNG).
optional PhotoType type;
}
```
## **Do** Use java_outer_classname (before Edition 2024) {#java-outer-classname}
Every proto schema definition file should set option `java_outer_classname` to
the `.proto` file name converted to TitleCase with the '.' removed. For example,
the file `student_record_request.proto` should set:
```java
option java_outer_classname = "StudentRecordRequestProto";
```
The default behavior of Edition 2024 files has been aligned with this
recommendation, so no option should be set when using Edition 2024 or later.
---
## File: content/best-practices/no-cargo-cults.md
+++
title = "Avoid Cargo Culting"
weight = 90
description = "Avoid using features where they are not needed."
type = "docs"
+++
Do not
[cargo cult](https://en.wikipedia.org/wiki/Cargo_cult_programming)
settings in proto files. If you are creating a new proto file based on existing
schema definitions, don't apply option settings except for those that you
understand the need for.
## Best Practices Specific to Editions {#editions}
Avoid applying [editions features](/editions/features)
except when they're actually necessary. Features in `.proto` files signal the
use of either experimental future behaviors or deprecated past behaviors. Best
practices for the latest edition will always be the default. New proto schema
definition content should remain feature-free, except if you want to early-adopt
a feature for future behavior that's being rolled out.
Copying-forward feature settings without understanding why they are set can lead
to unexpected behaviors in your code.
---
## File: content/design-decisions/_index.md
+++
title = "Protobuf Team Design Decisions"
weight = 88
description = "Covers the design decisions the Protobuf team has made"
type = "docs"
no_list = "true"
linkTitle = "Design Decisions"
+++
This section contains some positions of the Protobuf team that inform our design
and implementation decisions.
These positions were taken after careful consideration and won't be overturned
on a whim, but are open to being revisited as we gain new information, and as
things develop in the broader ecosystem context.
* [No Nullable
Setters/Getters](/design-decisions/nullable-getters-setters)
* [Opaque APIs](/design-decisions/opaque-apis)
* [Implicit
Presence](/design-decisions/implicit-presence)
* [Insertion Points and Generic
Services](/design-decisions/insertion-points-and-generic-services)
---
## File: content/design-decisions/nullable-getters-setters.md
+++
title = "No Nullable Setters/Getters Support"
weight = 89
description = "Covers why Protobuf doesn't support nullable setters and getters"
type = "docs"
aliases = "/programming-guides/nullable-getters-setters/"
+++
We have heard feedback that some folks would like protobuf to support nullable
getters/setters in their null-friendly language of choice (particularly Kotlin,
C#, and Rust). While this does seem to be a helpful feature for folks using
those languages, the design choice has tradeoffs which have led to the Protobuf
team choosing not to implement them.
Explicit presence is not a concept that directly maps to the traditional notion
of nullability. It is subtle, but explicit presence philosophy is closer to "the
fields are not nullable, but you can detect if the field was explicitly assigned
a value or not. Normal access will see some default value if it is not assigned,
but you can check if the field was actively written to or not, when needed."
The biggest reason not to have nullable fields is the intended behavior of
default values specified in a `.proto` file. By design, calling a getter on an
unset field will return the default value of that field.
**Note:** C# does treat *message* fields as nullable. This inconsistency with
other languages stems from the lack of immutable messages, which makes it
impossible to create shared immutable default instances. Because message fields
can't have defaults, there's no functional problem with this.
As an example, consider this `.proto` file:
```proto
message Msg { Child child = 1; }
message Child { Grandchild grandchild = 1; }
message Grandchild { int32 foo = 1 [default = 72]; }
```
and corresponding Kotlin getters:
```kotlin
// With our API where getters are always non-nullable:
msg.child.grandchild.foo == 72
// With nullable submessages the ?. operator fails to get the default value:
msg?.child?.grandchild?.foo == null
// Or verbosely duplicating the default value at the usage site:
(msg?.child?.grandchild?.foo ?: 72)
```
and corresponding Rust getters:
```rust
// With our API:
msg.child().grandchild().foo() // == 72
// Where every getter is an Option, verbose and no default observed
msg.child().map(|c| c.grandchild()).map(|gc| gc.foo()) // == Option::None
// For the rare situations where code may want to observe both the presence and
// value of a field, the _opt() accessor which returns a custom Optional type
// can also be used here (the Optional type is similar to Option except can also
// be aware of the default value):
msg.child().grandchild().foo_opt() // Optional::Unset(72)
```
If a nullable getter existed, it would necessarily ignore the user-specified
defaults (to return null instead) which would lead to surprising and
inconsistent behavior. If users of nullable getters want to access the default
value of the field, they would have to write their own custom handling to use
the default if null is returned, which removes the supposed benefit of
cleaner/easier code with null getters.
Similarly, we do not provide nullable setters as the behavior would be
unintuitive. Performing a set and then get would not always give the same value
back, and calling a set would only sometimes affect the has-bit for the field.
Note that message-typed fields are always explicit presence fields (with
hazzers). Proto3 defaults to scalar fields having implicit presence (without
hazzers) unless they are explicitly marked `optional`, while Proto2 does not
support implicit presence. With
[Editions](/editions/features#field_presence), explicit
presence is the default behavior unless an implicit presence feature is used.
With the forward expectation that almost all fields will have explicit presence,
the ergonomic concerns that come with nullable getters are expected to be more
of a concern than they may have been for Proto3 users.
Due to these issues, nullable setters/getters would radically change the way
default values can be used. While we understand the possible utility, we have
decided it’s not worth the inconsistencies and difficulty it introduces.
---
## File: content/editions/_index.md
+++
title = "Protobuf Editions"
weight = 41
description = "Topics related to the Protobuf Editions functionality."
type = "docs"
+++
* [Protobuf Editions Overview](/editions/overview)
* [Feature Settings for Editions](/editions/features)
* [Implementing Editions Support](/editions/implementation)
---
## File: content/editions/features.md
+++
title = "Feature Settings for Editions"
weight = 43
description = "Protobuf Editions features and how they affect Protobuf behavior."
type = "docs"
+++
This topic provides an overview of the features that are included in the
released edition versions. Subsequent editions' features will be added to this
topic. We announce new editions in
[the News section](/news).
Before configuring feature settings in your new schema definition content, make
sure you understand why you are using them. Avoid
[cargo-culting](/best-practices/no-cargo-cults) with
features.
## Prototiller {#prototiller}
Prototiller is a command-line tool that updates proto schema configuration files
between syntax versions and editions. It hasn't been released yet, but is
referenced throughout this topic.
## Features {#features}
The following sections include all of the behaviors that are configurable using
features in editions. [Preserving proto2 or proto3 Behavior](#preserving) shows
how to override the default behaviors so that your proto definition files act
like proto2 or proto3 files. For more information on how editions and features
work together to set behavior, see
[Protobuf Editions Overview](/editions/overview).
Feature settings apply at different levels:
**File-level:** These settings apply to all elements (messages, fields, enums,
and so on) that don't have an overriding setting.
**Non-nested:** Messages, enums, and services can override settings made at the
file level. They apply to everything within them (message fields, enum values)
that aren't overridden, but don't apply to other parallel messages and enums.
**Nested:** Oneofs, messages, and enums can override settings from the message
they're nested in.
**Lowest-level:** Fields, extensions, enum values, extension ranges, and methods
are the lowest level at which you can override settings.
Each of the following sections has a comment that states what scope the feature
can be applied to. The following sample shows a mock feature applied to each
scope:
```proto
edition = "2024";
// File-level scope definition
option features.bar = BAZ;
enum Foo {
// Enum (non-nested scope) definition
option features.bar = QUX;
A = 1;
B = 2;
}
message Corge {
// Message (non-nested scope) definition
option features.bar = QUUX;
message Garply {
// Message (nested scope) definition
option features.bar = WALDO;
string id = 1;
}
// Field (lowest-level scope) definition
Foo A = 1 [features.bar = GRAULT];
}
```
In this example, the setting "`GRAULT"` in the lowest-level scope feature
definition overrides the non-nested-scope "`QUUX`" setting. And within the
Garply message, "`WALDO`" overrides "`QUUX`."
### `features.default_symbol_visibility` {#symbol-vis}
This feature enables setting the default visibility for messages and enums,
making them available or unavailable when imported by other protos. Use of this
feature will reduce dead symbols in order to create smaller binaries.
> **Scope Note:** Symbol visibility controls **only** the Protobuf compiler
> (`protoc`) import behavior when resolving references between `.proto` files.
> It has **no impact** on language-specific generated code or consumers of proto
> descriptors.
In addition to setting the defaults for the entire file, you can use the `local`
and `export` keywords to set per-field behavior. Read more about this at
[`export` / `local` Keywords](/editions/overview#export-local).
**Values available:**
* `EXPORT_ALL`: This is the default prior to Edition 2024. All messages and
enums are exported by default. Do not set this value in Edition 2024 and
beyond; it exists only for compatibility with previous proto editions.
* `EXPORT_TOP_LEVEL`: All top-level symbols default to export; nested default
to local.
* `LOCAL_ALL`: All symbols default to local.
* `STRICT`: All symbols local by default. Nested types cannot be exported,
except for a special-case caveat for `message { enum {} reserved 1 to max;
}`.
**Applicable to the following scope:** file
**Added in:** Edition 2024
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | ------------------
2026 | `STRICT`
2024 | `EXPORT_TOP_LEVEL`
2023 | `EXPORT_ALL`
proto3 | `EXPORT_ALL`
proto2 | `EXPORT_ALL`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following sample shows how you can apply the feature to elements in your
proto schema definition files:
```proto
// foo.proto
edition = "2024";
// Symbol visibility defaults to EXPORT_TOP_LEVEL in Edition 2024. Setting
// default_symbol_visibility overrides this default.
option features.default_symbol_visibility = LOCAL_ALL;
// Setting default_symbol_visibility to LOCAL_ALL makes all symbols local by
// default. Applying the export keyword explicitly overrides this.
export message ExportedMessage {
int32 baz = 1;
// Nested symbols also default to local under LOCAL_ALL (and under Edition
// 2024 defaults). Without an explicit export keyword, this enum remains local.
enum LocalNestedEnum {
UNKNOWN_LOCAL_NESTED_ENUM_VALUE = 0;
}
}
// bar.proto
edition = "2024";
import "foo.proto";
message ImportedMessage {
// The following is valid because ExportedMessage explicitly overrides
// the LOCAL_ALL visibility setting in foo.proto.
ExportedMessage bar = 1;
// The following is not valid because LocalNestedEnum is local by default,
// even though its parent message is exported.
// ExportedMessage.LocalNestedEnum qux = 2;
}
```
### `features.enforce_naming_style` {#enforce_naming}
Introduced in Edition 2024, this feature enables strict naming style enforcement
as defined in
[the style guide](/programming-guides/style) to ensure
protos are round-trippable by default with a feature value to opt-out to use
legacy naming style.
**Values available:**
* `STYLE2026`: Enforces strict adherence to the 2026 style guide for naming to
discourage common field-naming collisions.
* `STYLE2024`: Enforces strict adherence to the style guide for naming.
* `STYLE_LEGACY`: Applies the pre-Edition 2024 level of style guide
enforcement.
**Applicable to the following scopes:** file, extension range, message, field,
oneof, enum, enum value, service, method
**Added in:** Edition 2024
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | --------------
2026 | `STYLE2026`
2024 | `STYLE2024`
2023 | `STYLE_LEGACY`
proto3 | `STYLE_LEGACY`
proto2 | `STYLE_LEGACY`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows an Edition 2023 file:
Edition 2023 defaults to `STYLE_LEGACY`, so a non-conformant field name is fine:
```proto
edition = "2023";
message Foo {
// A non-conforming field name is not a problem
int64 bar_1 = 1;
}
```
Edition 2024 defaults to `STYLE2024`, so an override is needed to keep the
non-conformant field name:
```proto
edition = "2024";
// To keep the non-conformant field name, override the STYLE2024 setting
option features.enforce_naming_style = STYLE_LEGACY;
message Foo {
int64 bar_1 = 1;
}
```
The following code sample shows an Edition 2024 file:
Edition 2024 defaults to `STYLE2024`, so collision avoidance is not enforced:
```proto
edition = "2024";
// Field name collision avoidance is not enforced
message Bar {
int64 x = 1;
int64 has_x = 2;
}
```
Edition 2026 defaults to `STYLE2026`, so an override is needed to keep the
colliding field names:
```proto
edition = "2024";
option features.enforce_naming_style = STYLE2024;
// To keep the colliding field names, override the STYLE2026 setting
message Bar {
int64 x = 1;
int64 has_x = 2;
}
```
### `features.enforce_proto_limits` {#enforce_proto_limits}
Introduced in Edition 2026, this feature enables structure size limit
enforcement in the Protobuf compiler when parsing `.proto` files to prevent
oversize message and enum definitions.
**Values available:**
* `PROTO_LIMITS2026`: Enforces maximum limit checks:
* Up to 1500 fields per message
* Up to 1000 oneofs per message
* Up to 1200 fields per oneof
* Up to 1700 values per enum
* `LEGACY_NO_EXPLICIT_LIMITS`: Opts out of limit checking.
**Applicable to the following scope:** message, oneof, enum
**Added in:** Edition 2026
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | ---------------------------
2026 | `PROTO_LIMITS2026`
2024 | `LEGACY_NO_EXPLICIT_LIMITS`
2023 | `LEGACY_NO_EXPLICIT_LIMITS`
proto3 | `LEGACY_NO_EXPLICIT_LIMITS`
proto2 | `LEGACY_NO_EXPLICIT_LIMITS`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows an Edition 2024 file:
Edition 2024 defaults to `LEGACY_NO_EXPLICIT_LIMITS`, so a message with 1501
fields is fine:
```proto
edition = "2024";
message Foo {
// A message with more than 1500 fields
int64 one = 1;
int64 two = 2;
int64 three = 3;
# ...
int64 one_thousand_five_hundred = 1500;
int64 one_thousand_five_hundred_and_one = 1501;
}
```
Edition 2026 defaults to `PROTO_LIMITS2026`, so an override is needed to keep
oversized messages:
```proto
edition = "2026";
message Foo {
// Must be set at the level of the oversized message, enum, or oneof as this
// feature is not allowed at the file level.
option features.enforce_proto_limits = PROTO_LIMITS2026;
int64 one = 1;
int64 two = 2;
int64 three = 3;
# ...
int64 one_thousand_five_hundred = 1500;
int64 one_thousand_five_hundred_and_one = 1501;
}
```
### `features.enum_type` {#enum_type}
This feature sets the behavior for how enum values that aren't contained within
the defined set are handled. See
[Enum Behavior](/programming-guides/enum) for more
information on open and closed enums.
This feature doesn't impact proto3 files, so this section doesn't have a before
and after of a proto3 file.
**Values available:**
* `CLOSED:` Closed enums store enum values that are out of range in the
unknown field set.
* `OPEN:` Open enums parse out of range values into their fields directly.
**Applicable to the following scopes:** file, enum
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | --------
2026 | `OPEN`
2024 | `OPEN`
2023 | `OPEN`
proto3 | `OPEN`
proto2 | `CLOSED`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
enum Foo {
A = 2;
B = 4;
C = 6;
}
```
After running [Prototiller](#prototiller), the equivalent code might look like
this:
```proto
edition = "2024";
enum Foo {
// Setting the enum_type feature overrides the default OPEN enum
option features.enum_type = CLOSED;
A = 2;
B = 4;
C = 6;
}
```
### `features.field_presence` {#field_presence}
This feature sets the behavior for tracking field presence, or the notion of
whether a Protobuf field has a value.
**Values available:**
* `LEGACY_REQUIRED`: The field is required for parsing and serialization.
Any explicitly-set value is
serialized onto the wire (even if it is the same as the default value).
* `EXPLICIT`: The field has explicit presence tracking. Any explicitly-set
value is serialized onto the wire (even if it is the same as the default
value). For singular primitive fields, `has_*` functions are generated for
fields set to `EXPLICIT`.
* `IMPLICIT`: The field has no presence tracking. The default value is not
serialized onto the wire (even if it is explicitly set). `has_*` functions
are not generated for fields set to `IMPLICIT`.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | -----------
2026 | `EXPLICIT`
2024 | `EXPLICIT`
2023 | `EXPLICIT`
proto3 | `IMPLICIT`*
proto2 | `EXPLICIT`
\* proto3 is `IMPLICIT` unless the field has the `optional` label, in which case
it behaves like `EXPLICIT`. See
[Presence in Proto3 APIs](/programming-guides/field_presence#presence-proto3)
for more information.
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
message Foo {
required int32 x = 1;
optional int32 y = 2;
repeated int32 z = 3;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
message Foo {
// Setting the field_presence feature retains the proto2 required behavior
int32 x = 1 [features.field_presence = LEGACY_REQUIRED];
int32 y = 2;
repeated int32 z = 3;
}
```
The following shows a proto3 file:
```proto
syntax = "proto3";
message Bar {
int32 x = 1;
optional int32 y = 2;
repeated int32 z = 3;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
// Setting the file-level field_presence feature matches the proto3 implicit default
option features.field_presence = IMPLICIT;
message Bar {
int32 x = 1;
// Setting the field_presence here retains the explicit state that the proto3
// field has because of the optional syntax
int32 y = 2 [features.field_presence = EXPLICIT];
repeated int32 z = 3;
}
```
Note that the `required` and `optional` labels no longer exist in Editions, as
the corresponding behavior is set explicitly with the `field_presence` feature.
### `features.json_format` {#json_format}
This feature sets the behavior for JSON parsing and serialization.
This feature doesn't impact proto3 files, so this section doesn't have a before
and after of a proto3 file. Editions behavior matches the behavior in proto3.
**Values available:**
* `ALLOW`: A runtime must allow JSON parsing and serialization. Checks are
applied at the proto level to make sure that there is a well-defined mapping
to JSON.
* `LEGACY_BEST_EFFORT`: A runtime does the best it can to parse and serialize
JSON. Certain protos are allowed that can result in unspecified behavior at
runtime (such as many:1 or 1:many mappings).
**Applicable to the following scopes:** file, message, enum
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | --------------------
2026 | `ALLOW`
2024 | `ALLOW`
2023 | `ALLOW`
proto3 | `ALLOW`
proto2 | `LEGACY_BEST_EFFORT`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
message Foo {
// Warning only
string bar = 1;
string bar_ = 2;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
option features.json_format = LEGACY_BEST_EFFORT;
message Foo {
string bar = 1;
string bar_ = 2;
}
```
### `features.message_encoding` {#message_encoding}
This feature sets the behavior for encoding fields when serializing.
This feature doesn't impact proto3 files, so this section doesn't have a before
and after of a proto3 file.
Depending on the language, fields that are "group-like" may have some unexpected
capitalization in generated code and in text-format, in order to provide
backwards compatibility with proto2. Message fields are "group-like" if all of
the following conditions are met:
* Has `DELIMITED` message encoding specified
* Message type is defined in the same scope as the field
* Field name is exactly the lowercased type name
**Values available:**
* `LENGTH_PREFIXED`: Fields are encoded using the LEN wire type described in
[Message Structure](/programming-guides/encoding#structure).
* `DELIMITED`: Message-typed fields are encoded as
[groups](/programming-guides/proto2#groups).
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | -----------------
2026 | `LENGTH_PREFIXED`
2024 | `LENGTH_PREFIXED`
2023 | `LENGTH_PREFIXED`
proto3 | `LENGTH_PREFIXED`
proto2 | `LENGTH_PREFIXED`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
message Foo {
group Bar = 1 {
optional int32 x = 1;
repeated int32 y = 2;
}
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
message Foo {
message Bar {
int32 x = 1;
repeated int32 y = 2;
}
Bar bar = 1 [features.message_encoding = DELIMITED];
}
```
### `features.repeated_field_encoding` {#repeated_field_encoding}
This feature is what the proto2/proto3
[`packed` option](/programming-guides/encoding#packed)
for `repeated` fields has been migrated to in Editions.
**Values available:**
* `PACKED`: `Repeated` fields of a primitive type are encoded as a single LEN
record that contains each element concatenated.
* `EXPANDED`: `Repeated` fields are each encoded with the field number for
each value.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | ----------
2026 | `PACKED`
2024 | `PACKED`
2023 | `PACKED`
proto3 | `PACKED`
proto2 | `EXPANDED`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
message Foo {
repeated int32 bar = 6 [packed=true];
repeated int32 baz = 7;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
option features.repeated_field_encoding = EXPANDED;
message Foo {
repeated int32 bar = 6 [features.repeated_field_encoding=PACKED];
repeated int32 baz = 7;
}
```
The following shows a proto3 file:
```proto
syntax = "proto3";
message Foo {
repeated int32 bar = 6;
repeated int32 baz = 7 [packed=false];
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
message Foo {
repeated int32 bar = 6;
repeated int32 baz = 7 [features.repeated_field_encoding=EXPANDED];
}
```
### `features.utf8_validation` {#utf8_validation}
This feature sets how strings are validated. It applies to all languages except
where there's a language-specific `utf8_validation` feature that overrides it.
See [`features.(pb.java).utf8_validation`](#java-utf8_validation) for the
Java-language-specific feature.
**Important:** In Java, `map` fields always undergo UTF-8
validation, even when `utf8_validation = NONE` is configured. Because Java keys
will be represented as `java.lang.String` objects, disabling validation would
force the runtime to convert invalid byte sequences *lossily* using replacement
characters. This lossy conversion can result in unexpected behavior, such as key
collisions. As a wire-compatible workaround, consider using a `repeated` field
with a custom "map entry" message instead.
This feature doesn't impact proto3 files, so this section doesn't have a before
and after of a proto3 file.
**Values available:**
* `VERIFY`: The runtime should verify UTF-8. This is the default proto3
behavior.
* `NONE`: The field behaves like an unverified `bytes` field on the wire.
Parsers may handle this type of field in an unpredictable way, such as
replacing invalid characters. This is the default proto2 behavior.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | --------
2026 | `VERIFY`
2024 | `VERIFY`
2023 | `VERIFY`
proto3 | `VERIFY`
proto2 | `NONE`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
message MyMessage {
string foo = 1;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
message MyMessage {
string foo = 1 [features.utf8_validation = NONE];
}
```
## Language-specific Features {#lang-specific}
Some features apply to specific languages, and not to the same protos in other
languages. Using these features requires you to import the corresponding
*_features.proto file from the language's runtime. The examples in the following
sections show these imports.
### `features.(pb.go).api_level` {#go-api_level}
**Languages:** Go
The `api_level` feature enables you to select which API version the Go Protobuf
plugin should generate code for. The Opaque API is the latest version of the
Protobuf implementation for the Go programming language. The previous version is
now called Open Struct API. See the
[Go Protobuf: Releasing the Opaque API](https://go.dev/blog/protobuf-opaque)
blog post for an introduction.
**Values available:**
* `API_OPEN`: The Open Struct API generates struct types that are open to
direct access.
* `API_HYBRID`: Hybrid is a step between Open and Opaque: The Hybrid API also
includes accessor methods (so you can update your code), but still exports
the struct fields as before. There is no performance difference; this API
level only helps with the migration.
* `API_OPAQUE`: With the Opaque API, the struct fields are hidden and can no
longer be directly accessed. Instead, the new accessor methods allow for
getting, setting, or clearing a field.
**Applicable to the following scopes:** message, file
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | ------------
2026 | `API_OPAQUE`
2024 | `API_OPAQUE`
2023 | `API_OPEN`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
You can set the `api_level` feature starting in edition 2023:
```proto
edition = "2023";
import option "third_party/golang/protobuf/v2/src/google/protobuf/go_features.proto";
// Remove this line after migrating the code to the Opaque API.
option features.(pb.go).api_level = API_HYBRID;
```
NOTE: `import option` feature was added to reduce any impact that these added
imports may have on runtime reflection behaviors or other descriptor processing.
Always use `import option` if you are importing another `.proto` file only to
use it in options.
See also:
[Opaque API: Migration](/reference/go/opaque-migration)
### `features.(pb.cpp).enum_name_uses_string_view` {#enum-name-string-view}
**Languages:** C++
Before Edition 2024, all generated enum types provide the following function to
obtain the label out of an enum value, which has some overhead to construct the
`std::string` instances at runtime:
```cpp
const std::string& Foo_Name(int);
```
The default feature value in Edition 2024 changes this signature to return
`absl::string_view` to allow for better storage decoupling and potential
memory/CPU savings. If you aren't ready to migrate, yet, you can override this
to set it back to its previous behavior. See
[string_view return type](/support/migration#string_view-return-type)
in the migration guide for more on this topic.
**Values available:**
* `true`: The enum uses `string_view` for its values.
* `false`: The enum uses `std::string` for its values.
**Applicable to the following scopes:** file, enum
**Added in:** Edition 2024
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | -------
2026 | `true`
2024 | `true`
2023 | `false`
proto3 | `false`
proto2 | `false`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
### `features.(pb.cpp).repeated_type` {#cpp-repeated_type}
**Languages:** C++
When set to `PROXY`, repeated field accessors return proxy types
(`RepeatedFieldProxy`) instead of pointers or references to `RepeatedField` or
`RepeatedPtrField`. Proxies allow for a more efficient in-memory representation
of repeated fields in messages.
Importing this feature requires `import option "google/protobuf/cpp_features.proto";`
**Values available:**
* `PROXY`: Accessors return proxy types.
* `LEGACY`: Accessors return legacy `RepeatedField` or `RepeatedPtrField`
pointers/references.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2026
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | --------
2026 | `LEGACY`
2024 | `LEGACY`
2023 | `LEGACY`
proto3 | `LEGACY`
proto2 | `LEGACY`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
### `features.(pb.java).large_enum` {#java-large_enum}
**Languages:** Java
This language-specific feature enables you to adopt new functionality that
handles large enums in Java without causing compiler errors. Note that this
feature replicates enum-like behavior but has some notable differences. For
example, switch statements are not supported.
**Values available:**
* `true`: Java enums will use the new functionality.
* `false`: Java enums will continue to use Java enums.
**Applicable to the following scopes:** file, enum
**Added in:** Edition 2024
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | -------
2026 | `false`
2024 | `false`
2023 | `false`
proto3 | `false`
proto2 | `false`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
### `features.(pb.cpp/pb.java).legacy_closed_enum` {#legacy_closed_enum}
**Languages:** C++, Java
This feature determines whether a field with an open enum type should be behave
as if it was a closed enum. This allows editions to reproduce
[non-conformant behavior](/programming-guides/enum) in
Java and C++ from proto2 and proto3.
This feature doesn't impact proto3 files, and so this section doesn't have a
before and after of a proto3 file.
**Values available:**
* `true`: Treats the enum as closed regardless of [`enum_type`](#enum_type).
* `false`: Respect whatever is set in the `enum_type`.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | -------
2026 | `false`
2024 | `false`
2023 | `false`
proto3 | `false`
proto2 | `true`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
import "myproject/proto3file.proto";
message Msg {
myproject.proto3file.Proto3Enum name = 1;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
import "myproject/proto3file.proto";
import option "third_party/protobuf/cpp_features.proto";
import option "third_party/java/protobuf/java_features.proto";
message Msg {
myproject.proto3file.Proto3Enum name = 1 [
features.(pb.cpp).legacy_closed_enum = true,
features.(pb.java).legacy_closed_enum = true
];
}
```
### `features.(pb.java).nest_in_file_class` {#java-nest_in_file}
**Languages:** Java
This feature controls whether the Java generator will nest the generated class
in the Java generated file class. Setting this option to `NO` is the equivalent
of setting `java_multiple_files = true` in proto2/proto3/edition 2023.
The default outer classname is also updated to always be the camel-cased .proto
filename suffixed with Proto by default (for example, `foo/bar_baz.proto`
becomes `BarBazProto`). You can still override this using the
`java_outer_classname` file option and replace the pre-Edition 2024 default of
`BarBaz` or `BarBazOuterClass` depending on the presence of conflicts.
**Values available:**
* `NO`: Do not nest the generated class in the file class.
* `YES`: Nest the generated class in the file class.
* Legacy: Nesting behavior controlled by `java_multiple_files` option (see
go/java-proto-names#immutable-api-message-names).
**Applicable to the following scopes:** message, enum, service
**Added in:** Edition 2024
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | -------
2026 | `NO`
2024 | `NO`
2023 | Legacy
proto3 | Legacy
proto2 | Legacy
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
### `features.(pb.cpp).string_type` {#string_type}
**Languages:** C++
This feature determines how generated code should treat string fields. This
replaces the `ctype` option from proto2 and proto3, and offers a new
`string_type` feature. In Edition 2023, you can specify either `ctype` or
`string_type` on a field, but not both. In Edition 2024, the `ctype` option is
removed.
**Values available:**
* `VIEW`: Generates `string_view` accessors for the field.
* `CORD`: Generates `Cord` accessors for the field. Not supported on extension
fields.
* `STRING`: Generates `string` accessors for the field.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | --------
2026 | `VIEW`
2024 | `VIEW`
2023 | `STRING`
proto3 | `STRING`
proto2 | `STRING`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
message Foo {
optional string bar = 6;
optional string baz = 7 [ctype = CORD];
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
import option "third_party/protobuf/cpp_features.proto";
message Foo {
string bar = 6 [features.(pb.cpp).string_type = STRING];
string baz = 7 [features.(pb.cpp).string_type = CORD];
}
```
The following shows a proto3 file:
```proto
syntax = "proto3"
message Foo {
string bar = 6;
string baz = 7 [ctype = CORD];
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
import option "third_party/protobuf/cpp_features.proto";
message Foo {
string bar = 6 [features.(pb.cpp).string_type = STRING];
string baz = 7 [features.(pb.cpp).string_type = CORD];
}
```
### `features.(pb.java).utf8_validation` {#java-utf8_validation}
**Languages:** Java
This language-specific feature enables you to override the file-level settings
at the field level for Java only.
This feature doesn't impact proto3 files, and so this section doesn't have a
before and after of a proto3 file.
**Values available:**
* `DEFAULT`: The behavior matches that set by
[`features.utf8_validation`](#utf8_validation).
* `VERIFY`: Overrides the file-level `features.utf8_validation` setting to
force it to `VERIFY` for Java only.
**Applicable to the following scopes:** file, field
**Added in:** Edition 2023
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | ---------
2026 | `DEFAULT`
2024 | `DEFAULT`
2023 | `DEFAULT`
proto3 | `DEFAULT`
proto2 | `DEFAULT`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
The following code sample shows a proto2 file:
```proto
syntax = "proto2";
option java_string_check_utf8=true;
message MyMessage {
string foo = 1;
string bar = 2;
}
```
After running Prototiller, the equivalent code might look like this:
```proto
edition = "2024";
import "google/protobuf/java_features.proto";
option features.utf8_validation = NONE;
option features.(pb.java).utf8_validation = VERIFY;
message MyMessage {
string foo = 1;
string bar = 2;
}
```
### `features.(pb.go).strip_enum_prefix` {#go-strip_enum_prefix}
**Languages:** Go
Enum values are not scoped by their containing enum name, so
[prefixing every value with the enum name is recommended](/programming-guides/style#enums):
```proto
edition = "2024";
enum Strip {
STRIP_ZERO = 0;
STRIP_ONE = 1;
}
```
However, the generated Go code will now contain two prefixes!
```go
type Strip int32
const (
Strip_STRIP_ZERO Strip = 0
Strip_STRIP_ONE Strip = 1
)
```
The language-specific `strip_enum_prefix` feature determines whether the Go code
generator strips the repetitive prefix or not.
**Values available:**
* `STRIP_ENUM_PREFIX_KEEP`: Keep the name as-is, even if repetitive.
* `STRIP_ENUM_PREFIX_GENERATE_BOTH`: Generate both, a full name and a stripped
name (to help with migrating your Go code).
* `STRIP_ENUM_PREFIX_STRIP`: Strip the enum name prefix from enum value names.
**Applicable to the following scopes:** file, enum, enum value
**Added in:** Edition 2024
**Default behavior per syntax/edition:**
Syntax/edition | Default
-------------- | ------------------------
2026 | `STRIP_ENUM_PREFIX_KEEP`
2024 | `STRIP_ENUM_PREFIX_KEEP`
**Note:** Feature settings on different schema elements
[have different scopes](#cascading).
You can set the `strip_enum_prefix` feature in edition 2024 (or newer) .proto
files:
```proto
edition = "2024";
import "google/protobuf/go_features.proto";
option features.(pb.go).strip_enum_prefix = STRIP_ENUM_PREFIX_STRIP;
enum Strip {
STRIP_ZERO = 0;
STRIP_ONE = 1;
}
```
The generated Go code will now strip the `STRIP` prefix:
```go
type Strip int32
const (
Strip_ZERO Strip = 0
Strip_ONE Strip = 1
)
```
## Preserving proto2 or proto3 Behavior in Edition 2023 {#preserving}
You may want to move to the editions format but not deal with updates to the way
that generated code behaves yet. This section shows the changes that the
Prototiller tool makes to your .proto files to make edition-2023-based protos
behave like a proto2 or proto3 file.
After these changes are made at the file level, you get the proto2 or proto3
defaults. You can override at lower levels (message level, field level) to
consider additional behavior differences (such as
[required, proto3 optional](#caveats)) or if you want your definition to only be
*mostly* like proto2 or proto3.
We recommend using Prototiller unless you have a specific reason not to. To
manually apply all of these instead of using Prototiller, add the content from
the following sections to the top of your .proto file.
### Proto2 Behavior {#proto2-behavior}
The following shows the settings to replicate proto2 behavior with Edition 2023.
```proto
edition = "2023";
import "google/protobuf/cpp_features.proto";
import "google/protobuf/java_features.proto";
option features.field_presence = EXPLICIT;
option features.enum_type = CLOSED;
option features.repeated_field_encoding = EXPANDED;
option features.json_format = LEGACY_BEST_EFFORT;
option features.utf8_validation = NONE;
option features.(pb.cpp).legacy_closed_enum = true;
option features.(pb.java).legacy_closed_enum = true;
```
### Proto3 Behavior {#proto3-behavior}
The following shows the settings to replicate proto3 behavior with Edition 2023.
```proto
edition = "2023";
import "google/protobuf/cpp_features.proto";
import "google/protobuf/java_features.proto";
option features.field_presence = IMPLICIT;
option features.enum_type = OPEN;
// `packed=false` needs to be transformed to field-level repeated_field_encoding
// features in Editions syntax
option features.json_format = ALLOW;
option features.utf8_validation = VERIFY;
option features.(pb.cpp).legacy_closed_enum = false;
option features.(pb.java).legacy_closed_enum = false;
```
### Edition 2023 to 2024 {#2023-2024}
The following shows the settings to replicate Edition 2023 behavior with
Edition 2024.
```proto
// foo/bar_baz.proto
edition = "2024";
import option "third_party/protobuf/cpp_features.proto";
import option "third_party/java/protobuf/java_features.proto";
import option "third_party/golang/protobuf/v2/src/google/protobuf/go_features.proto";
// If previously relying on edition 2023 default java_outer_classname.
option java_outer_classname = "BarBaz" // or BarBazOuterClass
option features.(pb.cpp).string_type = STRING;
option features.enforce_naming_style = STYLE_LEGACY;
option features.default_symbol_visibility = EXPORT_ALL;
option features.(pb.cpp).enum_name_uses_string_view = false;
option features.(pb.go).api_level = API_OPEN;
message MyMessage {
option features.(pb.java).nest_in_file_class = YES;
}
```
### Edition 2024 to 2026 {#2024-2026}
The following shows the settings to replicate Edition 2024 behavior with
Edition 2026.
```proto
edition = "2026";
option features.enforce_naming_style = STYLE2024;
option features.default_symbol_visibility = EXPORT_TOP_LEVEL;
```
### Caveats and Exceptions {#caveats}
This section shows the changes that you'll need to make manually if you choose
not to use Prototiller.
Setting the file-level defaults shown in the previous section sets the default
behaviors in most cases, but there are a few exceptions.
**Edition 2023 and later**
* `optional`: Remove all instances of the `optional` label and change the
[`features.field_presence`](#field_presence) to `EXPLICIT` if the file
default is `IMPLICIT`.
* `required`: Remove all instances of the `required` label and add the
[`features.field_presence=LEGACY_REQUIRED`](#field_presence) option at the
field level.
* `groups`: Unwrap the `groups` into a separate message and add the
`features.message_encoding = DELIMITED` option at the field level. See
[`features.message_encoding`](#message_encoding) for more on this.
* `java_string_check_utf8`: Remove this file option and replace it with the
[`features.(pb.java).utf8_validation`](#java-utf8_validation). You'll need
to import Java features, as covered in
[Language-specific Features](#lang-specific).
* `packed`: For proto2 files converted to editions format, remove the `packed`
field option and add `[features.repeated_field_encoding=PACKED]` at the
field level when you don't want the `EXPANDED` behavior that you set in
[Proto2 Behavior](#proto2-behavior). For proto3 files converted to editions
format, add `[features.repeated_field_encoding=EXPANDED]` at the field level
when you don't want the default proto3 behavior.
**Edition 2024 and later**
* (C++) `ctype`: Remove all instances of the `ctype` option and set the
[`features.(pb.cpp).string_type`](#string_type) value.
* (C++ and Go) `weak`: Remove weak
imports. Use
[`import option`](/editions/overview#import-option)
instead.
* (Java) `java_multiple_files`: Remove `java_multiple_files` and use
[`features.(pb.java).nest_in_file_class`](#java-nest_in_file) instead.
**Edition 2026 and later**
* (C++) Remove Language-specific options (`cc_api_version`,
`cc_utf8_verification`, `cc_enable_arenas`).
* Set `features.enforce_proto_limits` to `LEGACY_NO_EXPLICIT_LIMITS` within
1. Messages that have either more than 1500 fields or more than 1000 oneofs
1. Oneofs that have more than 1200 fields
1. Enums that have more than 1700 values
---
## File: content/editions/implementation.md
+++
title = "Implementing Editions Support"
weight = 43
description = "Instructions for implementing Editions support in runtimes and plugins."
type = "docs"
+++
This topic explains how to implement editions in new runtimes and generators.
## Overview {#overview}
### Edition 2023 {#edition-2023}
The first edition released is Edition 2023, which is designed to unify proto2
and proto3 syntax. The features we’ve added to cover the difference in behaviors
are detailed in
[Feature Settings for Editions](/editions/features).
### Feature Definition {#feature-definition}
In addition to *supporting* editions and the global features we've defined, you
may want to define your own features to leverage the infrastructure. This will
allow you to define arbitrary features that can be used by your generators and
runtimes to gate new behaviors. The first step is to claim an extension number
for the `FeatureSet` message in descriptor.proto above 9999. You can send a
pull-request to us in GitHub, and it will be included in our next release (see,
for example, [#15439](https://github.com/protocolbuffers/protobuf/pull/15439)).
Once you have your extension number, you can create your features proto (similar
to
[cpp_features.proto](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/cpp_features.proto)).
These will typically look something like:
```proto
edition = "2023";
package foo;
import "google/protobuf/descriptor.proto";
extend google.protobuf.FeatureSet {
MyFeatures features = ;
}
message MyFeatures {
enum FeatureValue {
FEATURE_VALUE_UNKNOWN = 0;
VALUE1 = 1;
VALUE2 = 2;
}
FeatureValue feature_value = 1 [
targets = TARGET_TYPE_FIELD,
targets = TARGET_TYPE_FILE,
feature_support = {
edition_introduced: EDITION_2023,
edition_deprecated: EDITION_2024,
deprecation_warning: "Feature will be removed in 2025",
edition_removed: EDITION_2025,
},
edition_defaults = { edition: EDITION_LEGACY, value: "VALUE1" },
edition_defaults = { edition: EDITION_2024, value: "VALUE2" }
];
}
```
Here we’ve defined a new enum feature `foo.feature_value` (currently only
boolean and enum types are supported). In addition to defining the values it can
take, you also need to specify how it can be used:
* **Targets** - specifies the type of proto descriptors this feature can be
attached to. This controls where users can explicitly specify the feature.
Every type must be explicitly listed.
* **Feature support** - specifies the lifetime of this feature relative to
edition. You must specify the edition it was introduced in, and it will not
be allowed before then. You can optionally deprecate or remove the feature
in later editions.
* **Edition defaults** - specifies any changes to the default value of the
feature. This must cover every supported edition, but you can leave out any
edition where the default didn’t change. Note that `EDITION_PROTO2` and
`EDITION_PROTO3` can be specified here to provide defaults for the “legacy”
editions (see [Legacy Editions](#legacy_editions)).
#### What is a Feature? {#what-feature}
Features are designed to provide a mechanism for ratcheting down bad behavior
over time, on edition boundaries. While the timeline for actually removing a
feature may be years (or decades) in the future, the desired goal of any feature
should be eventual removal. When a bad behavior is identified, you can introduce
a new feature that guards the fix. In the next edition (or possibly later), you
would flip the default value while still allowing users to retain their old
behavior when upgrading. At some point in the future you would mark the feature
deprecated, which would trigger a custom warning to any users overriding it. In
a later edition, you would then mark it removed, preventing users from
overriding it anymore (but the default value would still apply). Until support
for that last edition is dropped in a breaking release, the feature will remain
usable for protos stuck on older editions, giving them time to migrate.
Flags that control optional behaviors you have no intention of removing are
better implemented as
[custom options](/programming-guides/proto2/#customoptions).
This is related to the reason we’ve restricted features to be either boolean or
enum types. Any behavior controlled by a (relatively) unbounded number of values
probably isn’t a good fit for the editions framework, since it’s not realistic
to eventually turn down that many different behaviors.
One caveat to this is behaviors related to wire boundaries. Using
language-specific features to control serialization or parsing behavior can be
dangerous, since any other language could be on the other side. Wire-format
changes should always be controlled by global features in `descriptor.proto`,
which can be respected by every runtime uniformly.
### Generators {#generators}
Generators written in C++ get a lot for free, because they use the C++ runtime.
They don’t need to handle [Feature Resolution](#feature_resolution) themselves,
and if they need any feature extensions they can register them in
`GetFeatureExtensions` in their CodeGenerator. They can generally use
`GetResolvedSourceFeatures` for access to resolved features for a descriptor in
codegen and `GetUnresolvedSourceFeatures` for access to their own unresolved
features.
Plugins written in the same language as the runtime they generate code for may
need some custom bootstrapping for their feature definitions.
#### Explicit Support {#explicit-support}
Generators must specify exactly which editions they support. This allows you to
safely add support for an edition after it’s been released, on your own
schedule. Protoc will reject any editions protos sent to generators that don’t
include `FEATURE_SUPPORTS_EDITIONS` in the `supported_features` field of their
`CodeGeneratorResponse`. Additionally, we have `minimum_edition` and
`maximum_edition` fields for specifying your precise support window. Once you’ve
defined all of the code and feature changes for a new edition, you can bump
`maximum_edition` to advertise this support.
#### Codegen Tests {#codegen-tests}
We have a set of codegen tests that can be used to lock down that Edition 2023
produces no unexpected functional changes. These have been very useful in
languages like C++ and Java, where a significant amount of the functionality is
in gencode. On the other hand, in languages like Python, where the gencode is
basically just a collection of serialized descriptors, these are not quite as
useful.
This infrastructure is not reusable yet, but is planned to be in a future
release. At that point you will be able to use them to verify that migrating to
editions doesn’t have any unexpected codegen changes.
### Runtimes {#runtimes}
Runtimes without reflection or dynamic messages should not need to do anything
to implement Editions. All of that logic should be handled by the code
generator.
Languages *with* reflection but *without* dynamic messages need resolved
features, but may optionally choose to handle it in their generator only. This
can be done by passing both resolved and unresolved feature sets to the runtime
during codegen. This avoids re-implementing
[Feature Resolution](#feature_resolution) in the runtime with the main downside
being efficiency, since it will create a unique feature set for every
descriptor.
Languages with dynamic messages must fully implement Editions, because they need
to be able to build descriptors at runtime.
#### Syntax Reflection {#syntax_reflection}
The first step in implementing Editions in a runtime with reflection is to
remove all direct checks of the `syntax` keyword. All of these should be moved
to finer-grained feature helpers, which can continue to use `syntax` if
necessary.
The following feature helpers should be implemented on descriptors, with
language-appropriate naming:
* `FieldDescriptor::has_presence` - Whether or not a field has explicit
presence
* Repeated fields *never* have presence
* Message, extension, and oneof fields *always* have explicit presence
* Everything else has presence iff `field_presence` is not `IMPLICIT`
* `FieldDescriptor::is_required` - Whether or not a field is required
* `FieldDescriptor::requires_utf8_validation` - Whether or not a field should
be checked for utf8 validity
* `FieldDescriptor::is_packed` - Whether or not a repeated field has packed
encoding
* `FieldDescriptor::is_delimited` - Whether or not a message field has
delimited encoding
* `EnumDescriptor::is_closed` - Whether or not a field is closed
{{% alert title="Note" color="note" %}} In
most languages, the message encoding feature is still currently signaled by
`TYPE_GROUP` and required fields still have `LABEL_REQUIRED` set. This is not
ideal, and was done to make downstream migrations easier. Eventually, these
should be migrated to the appropriate helpers and
`TYPE_MESSAGE/LABEL_OPTIONAL`.{{% /alert %}}
Downstream users should migrate to these new helpers instead of using syntax
directly. The following class of existing descriptor APIs should ideally be
deprecated and eventually removed, since they leak syntax information:
* `FileDescriptor` syntax
* Proto3 optional APIs
* `FieldDescriptor::has_optional_keyword`
* `OneofDescriptor::is_synthetic`
* `Descriptor::*real_oneof*` - should be renamed to just “oneof” and the
existing “oneof” helpers should be removed, since they leak information
about synthetic oneofs (which don’t exist in editions).
* Group type
* The `TYPE_GROUP` enum value should be removed, replaced with the
`is_delimited` helper.
* Required label
* The `LABEL_REQUIRED` enum value should be removed, replaced with the
`is_required` helper.
There are many classes of user code where these checks exist but *aren’t*
hostile to editions. For example, code that needs to handle proto3 `optional`
specially because of its synthetic oneof implementation won’t be hostile to
editions as long as the polarity is something like `syntax == "proto3"` (rather
than checking `syntax != "proto2"`).
If it’s not possible to remove these APIs entirely, they should be deprecated
and discouraged.
#### Feature Visibility {#visibility}
As discussed in
[editions-feature-visibility](https://github.com/protocolbuffers/protobuf/blob/main/docs/design/editions/editions-feature-visibility.md),
feature protos should remain an internal detail of any Protobuf implementation.
The *behaviors* they control should be exposed via descriptor methods, but the
protos themselves should not. Notably, this means that any options that are
exposed to the users need to have their `features` fields stripped out.
The one case where we permit features to leak out is when serializing
descriptors. The resulting descriptor protos should be a faithful representation
of the original proto files, and should contain *unresolved features* inside of
the options.
#### Legacy Editions {#legacy_editions}
As discussed more in
[legacy-syntax-editions](https://github.com/protocolbuffers/protobuf/blob/main/docs/design/editions/legacy-syntax-editions.md),
a great way to get early coverage of your editions implementation is to unify
proto2, proto3, and editions. This effectively migrates proto2 and proto3 to
editions under the hood, and makes all of the helpers implemented in
[Syntax Reflection](#syntax_reflection) use features exclusively (instead of
branching on syntax). This can be done by inserting a *feature inference* phase
into [Feature Resolution](#feature_resolution), where various aspects of the
proto file can inform what features are appropriate. These features can then be
merged into the parent’s features to get the resolved feature set.
While we provide reasonable defaults for proto2/proto3 already, for edition 2023
the following additional inferences are required:
* required - we infer `LEGACY_REQUIRED` presence when a field has
`LABEL_REQUIRED`
* groups - we infer `DELIMITED` message encoding when a field has `TYPE_GROUP`
* packed - we infer `PACKED` encoding when the `packed` option is true
* expanded - we infer `EXPANDED` encoding when a proto3 field has `packed`
explicitly set to false
#### Conformance Tests {#conformance-tests}
Editions-specific conformance tests have been added, but need to be opted-in to.
A `--maximum_edition 2023` flag can be passed to the runner to enable these. You
will need to configure your testee binary to handle the following new message
types:
* `protobuf_test_messages.editions.proto2.TestAllTypesProto2` - Identical to
the old proto2 message, but transformed to edition 2023
* `protobuf_test_messages.editions.proto3.TestAllTypesProto3` - Identical to
the old proto3 message, but transformed to edition 2023
* `protobuf_test_messages.editions.TestAllTypesEdition2023` - Used to cover
edition-2023-specific test cases
### Feature Resolution {#feature-resolution}
Editions use lexical scoping to define features, meaning that any non-C++ code
that needs to implement editions support will need to reimplement our *feature
resolution* algorithm. However, the bulk of the work is handled by protoc
itself, which can be configured to output an intermediate `FeatureSetDefaults`
message. This message contains a “compilation” of a set of feature definition
files, laying out the default feature values in every edition.
For example, the feature definition above would compile to the following
defaults between proto2 and edition 2025 (in text-format notation):
```
defaults {
edition: EDITION_PROTO2
overridable_features { [foo.features] {} }
fixed_features {
// Global feature defaults…
[foo.features] { feature_value: VALUE1 }
}
}
defaults {
edition: EDITION_PROTO3
overridable_features { [foo.features] {} }
fixed_features {
// Global feature defaults…
[foo.features] { feature_value: VALUE1 }
}
}
defaults {
edition: EDITION_2023
overridable_features {
// Global feature defaults…
[foo.features] { feature_value: VALUE1 }
}
}
defaults {
edition: EDITION_2024
overridable_features {
// Global feature defaults…
[foo.features] { feature_value: VALUE2 }
}
}
defaults {
edition: EDITION_2025
overridable_features {
// Global feature defaults…
}
fixed_features { [foo.features] { feature_value: VALUE2 } }
}
minimum_edition: EDITION_PROTO2
maximum_edition: EDITION_2025
```
Global feature defaults are left out for compactness, but they would also be
present. This object contains an ordered list of every edition with a unique set
of defaults (some editions may end up not being present) within the specified
range. Each set of defaults is split into *overridable* and *fixed* features.
The former are supported features for the edition that can be freely overridden
by users. The fixed features are those which haven’t yet been introduced or have
been removed, and can’t be overridden by users.
We provide a Bazel rule for compiling these intermediate objects:
```
load("@com_google_protobuf//editions:defaults.bzl", "compile_edition_defaults")
compile_edition_defaults(
name = "my_defaults",
srcs = ["//some/path:lang_features_proto"],
maximum_edition = "PROTO2",
minimum_edition = "2024",
)
```
The output `FeatureSetDefaults` can be embedded into a raw string literal in
whatever language you need to do feature resolution in. We also provide an
`embed_edition_defaults` macro to do this:
```
embed_edition_defaults(
name = "embed_my_defaults",
defaults = ":my_defaults",
output = "my_defaults.h",
placeholder = "DEFAULTS_DATA",
template = "my_defaults.h.template",
)
```
Alternatively, you can invoke protoc directly (outside of Bazel) to generate
this data:
```
protoc --edition_defaults_out=defaults.binpb --edition_defaults_minimum=PROTO2 --edition_defaults_maximum=2023
```
Once the defaults message is hooked up and parsed by your code, feature
resolution for a file descriptor at a given edition follows a simple algorithm:
1. Validate that the edition is in the appropriate range [`minimum_edition`,
`maximum_edition`]
2. Binary-search the ordered `defaults` field for the highest entry less than
or equal to the edition
3. Merge `overridable_features` into `fixed_features` from the selected
defaults
4. Merge any explicit features set on the descriptor (the `features` field in
the file options)
From there, you can recursively resolve features for all other descriptors:
1. Initialize to the parent descriptor’s feature set
2. Merge any explicit features set on the descriptor (the `features` field in
the options)
For determining the “parent” descriptor, you can reference our
[C++ implementation](https://github.com/protocolbuffers/protobuf/blob/27.x/src/google/protobuf/descriptor.cc#L1129).
This is straightforward in most cases, but extensions are a bit surprising
because their parent is the enclosing scope rather than the extendee. Oneofs
also need to be considered as the parent of their fields.
#### Conformance Tests {#conformance-tests}
In a future release, we plan to add conformance tests to verify feature
resolution cross-language. Until then, our regular
[conformance tests](#conformance_tests) do give partial coverage, and our
[example inheritance unit tests](https://github.com/protocolbuffers/protobuf/blob/27.x/python/google/protobuf/internal/descriptor_test.py#L1386)
can be ported to provide more comprehensive coverage.
### Examples {#examples}
Below are some real examples of how we implemented editions support in our
runtimes and plugins.
#### Java {#java}
* [#14138](https://github.com/protocolbuffers/protobuf/pull/14138) - Bootstrap
compiler with C++ gencode for Java features proto
* [#14377](https://github.com/protocolbuffers/protobuf/pull/14377) - Use
features in Java, Kotlin, and Java Lite code generators, including codegen
tests
* [#15210](https://github.com/protocolbuffers/protobuf/pull/15210) - Use
features in Java full runtimes covering Java features bootstrap, feature
resolution, and legacy editions, along with unit-tests and conformance
testing
#### Pure Python {#python}
* [#14546](https://github.com/protocolbuffers/protobuf/pull/14546) - Setup
codegen tests in advance
* [#14547](https://github.com/protocolbuffers/protobuf/pull/14547) - Fully
implements editions in one shot, along with unit-tests and conformance
testing
#### 𝛍pb {#upb}
* [#14638](https://github.com/protocolbuffers/protobuf/pull/14638) - First
pass at editions implementation covering feature resolution and legacy
editions
* [#14667](https://github.com/protocolbuffers/protobuf/pull/14667) - Added
more complete handling of field label/type, support for upb’s code
generator, and some tests
* [#14678](https://github.com/protocolbuffers/protobuf/pull/14678) - Hooks up
upb to the Python runtime, with more unit tests and conformance tests
#### Ruby {#ruby}
* [#16132](https://github.com/protocolbuffers/protobuf/pull/16132) - Hook up
upb/Java to all four Ruby runtimes for full editions support
---
## File: content/editions/overview.md
+++
title = "Protobuf Editions Overview"
linkTitle = "Overview"
weight = 42
description = "An overview of the Protobuf Editions functionality."
type = "docs"
+++
Protobuf Editions replace the proto2 and proto3 designations that we have used
for Protobuf. Instead of adding `syntax = "proto2"` or `syntax = "proto3"` at
the top of proto definition files, you use an edition number, such as `edition =
"2024"`, to specify the default behaviors your file will have. Editions enable
the language to evolve incrementally over time.
Instead of the hardcoded behaviors that older versions have had, editions
represent a collection of [features](/editions/features)
with a default value (behavior) per feature. Features are options on a file,
message, field, enum, and so on, that specify the behavior of protoc, the code
generators, and Protobuf runtimes. You can explicitly override a behavior at
those different levels (file, message, field, ...) when your needs don't match
the default behavior for the edition you've selected. You can also override your
overrides. The [section later in this topic on lexical scoping](#scoping) goes
into more detail on that.
**NOTE:** The latest released edition is 2024.
## Lifecycle of a Feature {#lifecycles}
Editions provide the fundamental increments for the lifecycle of a
feature. Features have an expected lifecycle:
introducing it, changing its default behavior, deprecating it, and then removing
it. For example:
1. Edition 2031 creates `feature.amazing_new_feature` with a default value of
`false`. This value maintains the same behavior as all earlier editions.
That is, it defaults to no impact. Not all new features will default to the
no-op option, but for the sake of this example, `amazing_new_feature` does.
2. Developers update their .proto files to `edition = "2031"`.
3. A later edition, such as edition 2033, switches the default of
`feature.amazing_new_feature` from `false` to `true`. This is the desired
behavior for all protos, and the reason that the Protobuf team created the
feature.
Using the Prototiller tool to migrate earlier versions of proto files to
edition 2033 adds explicit `feature.amazing_new_feature = false` entries as
needed to continue to retain the previous behavior. Developers remove these
newly-added settings when they want the new behavior to apply to their
.proto files.
4. At some point, `feature.amazing_new_feature` is marked deprecated in an
edition and removed in a later one.
When a feature is removed, the code generators for that behavior and the
runtime libraries that support it may also be removed. The timelines will be
generous, though. Following the example in the earlier steps of the
lifecycle, the deprecation might happen in edition 2034 but not be removed
until edition 2036, roughly two years later. Removing a feature will always
initiate a major version bump.
You will have the full
window of the Google migration plus the deprecation window to upgrade your code.
The preceding lifecycle example used boolean values for the features, but
features may also use enums. For example, `features.field_presence` has values
`LEGACY_REQUIRED`, `EXPLICIT`, and `IMPLICIT.`
## Migrating to Protobuf Editions {#migrating}
Editions won't break existing binaries and don't change a message's binary,
text, or JSON serialization format. Edition 2023 was as minimally disruptive as
possible. It established the baseline and combined proto2 and proto3 definitions
into a new single definition format.
As more editions are released, default behaviors for features may change. You
can have Prototiller do a no-op transformation of your .proto file or you can
choose to accept some or all of the new behaviors. Editions are planned to be
released roughly once a year.
### Proto2 to Editions {#proto2-migration}
This section shows a proto2 file, and what it might look like after running the
Prototiller tool to change the definition files to use Protobuf Editions syntax.
#### Proto2 Syntax {.new-tab}
```proto
// proto2 file
syntax = "proto2";
package com.example;
message Player {
// in proto2, optional fields have explicit presence
optional string name = 1 [default = "N/A"];
// proto2 still supports the problematic "required" field rule
required int32 id = 2;
// in proto2 this is not packed by default
repeated int32 scores = 3;
enum Handed {
HANDED_UNSPECIFIED = 0;
HANDED_LEFT = 1;
HANDED_RIGHT = 2;
HANDED_AMBIDEXTROUS = 3;
}
// in proto2 enums are closed
optional Handed handed = 4;
reserved "gender";
}
```
#### Editions Syntax {.new-tab}
```proto
// Edition version of proto2 file
edition = "2024";
package com.example;
option features.utf8_validation = NONE;
option features.enforce_naming_style = STYLE_LEGACY;
// NOTE: You should generally avoid setting EXPORT_ALL; it is used here only to
// illustrate how options can be used to match previous proto2 defaults.
option features.default_symbol_visibility = EXPORT_ALL;
// Sets the default behavior for C++ strings
option features.(pb.cpp).string_type = STRING;
message Player {
// fields have explicit presence, so no explicit setting needed
string name = 1 [default = "N/A"];
// to match the proto2 behavior, LEGACY_REQUIRED is set at the field level
int32 id = 2 [features.field_presence = LEGACY_REQUIRED];
// to match the proto2 behavior, EXPANDED is set at the field level
repeated int32 scores = 3 [features.repeated_field_encoding = EXPANDED];
export enum Handed {
// this overrides the default editions behavior, which is OPEN
option features.enum_type = CLOSED;
HANDED_UNSPECIFIED = 0;
HANDED_LEFT = 1;
HANDED_RIGHT = 2;
HANDED_AMBIDEXTROUS = 3;
}
Handed handed = 4;
reserved gender;
}
```
### Proto3 to Editions {#proto3-migration}
This section shows a proto3 file, and what it might look like after running the
Prototiller tool to change the definition files to use Protobuf Editions syntax.
#### Proto3 Syntax {.new-tab}
```proto
// proto3 file
syntax = "proto3";
package com.example;
message Player {
// in proto3, optional fields have explicit presence
optional string name = 1;
// in proto3 no specified field rule defaults to implicit presence
int32 id = 2;
// in proto3 this is packed by default
repeated int32 scores = 3;
enum Handed {
HANDED_UNSPECIFIED = 0;
HANDED_LEFT = 1;
HANDED_RIGHT = 2;
HANDED_AMBIDEXTROUS = 3;
}
// in proto3 enums are open
optional Handed handed = 4;
reserved "gender";
}
```
#### Editions Syntax {.new-tab}
```proto
// Editions version of proto3 file
edition = "2024";
package com.example;
option features.utf8_validation = NONE;
option features.enforce_naming_style = STYLE_LEGACY;
// NOTE: Avoid setting EXPORT_ALL; it is used here only to
// illustrate how options can be used to match previous proto3 defaults.
option features.default_symbol_visibility = EXPORT_ALL;
// Sets the default behavior for C++ strings
option features.(pb.cpp).string_type = STRING;
message Player {
// fields have explicit presence, so no explicit setting needed
string name = 1 [default = "N/A"];
// to match the proto3 behavior, IMPLICIT is set at the field level
int32 id = 2 [features.field_presence = IMPLICIT];
// PACKED is the default state, and is provided just for illustration
repeated int32 scores = 3 [features.repeated_field_encoding = PACKED];
export enum Handed {
HANDED_UNSPECIFIED = 0;
HANDED_LEFT = 1;
HANDED_RIGHT = 2;
HANDED_AMBIDEXTROUS = 3;
}
Handed handed = 4;
reserved gender;
}
```
### Lexical Scoping {#scoping}
Editions syntax supports lexical scoping, with a per-feature list of allowed
targets. For example, in Edition 2023, features can be specified at only the
file level or the lowest level of granularity. The implementation of lexical
scoping enables you to set the default behavior for a feature across an entire
file, and then override that behavior at the message, field, enum, enum value,
oneof, service, or method level. Settings made at a higher level (file, message)
apply when no setting is made within the same scope (field, enum value). Any
features not explicitly set conform to the behavior defined in the edition
version used for the .proto file.
The following code sample shows some features being set at the file, field, and
enum level.
```proto {highlight="lines:3,7,16"}
edition = "2024";
option features.enum_type = CLOSED;
message Person {
string name = 1;
int32 id = 2 [features.field_presence = IMPLICIT];
enum Pay_Type {
PAY_TYPE_UNSPECIFIED = 1;
PAY_TYPE_SALARY = 2;
PAY_TYPE_HOURLY = 3;
}
enum Employment {
option features.enum_type = OPEN;
EMPLOYMENT_UNSPECIFIED = 0;
EMPLOYMENT_FULLTIME = 1;
EMPLOYMENT_PARTTIME = 2;
}
Employment employment = 4;
}
```
In the preceding example, the presence feature is set to `IMPLICIT`; it would
default to `EXPLICIT` if it wasn't set. The `Pay_Type` `enum` will be `CLOSED`,
as it applies the file-level setting. The `Employment` `enum`, though, will be
`OPEN`, as it is set within the enum.
### Prototiller {#prototiller}
When the Prototiller tool is launched, we
will provide both a migration guide and migration tooling to ease the migration
to and between editions. The tool will enable you to:
* convert proto2 and proto3 definition files to the new editions syntax, at
scale
* migrate files from one edition to another
* manipulate proto files in other ways
### Backward Compatibility {#compatibility}
We are building Protobuf Editions to be as minimally disruptive as possible. For
example, you can import proto2 and proto3 definitions into editions-based
definition files, and vice versa:
```proto
// file myproject/foo.proto
syntax = "proto2";
enum Employment {
EMPLOYMENT_UNSPECIFIED = 0;
EMPLOYMENT_FULLTIME = 1;
EMPLOYMENT_PARTTIME = 2;
}
```
```proto
// file myproject/edition.proto
edition = "2024";
import "myproject/foo.proto";
```
While the generated code changes when you move from proto2 or proto3 to
editions, the wire format does not. You'll still be able to access proto2 and
proto3 data files or file streams using your editions-syntax proto definitions.
### Grammar Changes {#syntax}
There are some grammar changes in editions compared to proto2 and proto3.
#### Syntax Description {#syntax-descrip}
Instead of the `syntax` element, you use an `edition` element:
```proto
syntax = "proto2";
syntax = "proto3";
edition = "2028";
```
#### Reserved Names {#reserved-names}
You no longer put field names and enum value names in quotation marks when
reserving them:
```proto
reserved foo, bar;
```
#### Group Syntax {#group-syntax}
Group syntax, available in proto2, is removed in editions. The special
wire-format that groups used is still available by using `DELIMITED` message
encoding.
#### Required Label {#required-label}
The `required` label, available only in proto2, is unavailable in editions. The
underlying functionality is still
available by using `features.field_presence=LEGACY_REQUIRED`.
#### `import option` {#import-option}
Edition 2024 added support for option imports using the syntax `import option`.
Option imports must come after any other `import` statements.
Unlike normal `import` statements, `import option` only imports custom options
defined in a `.proto` file, without importing other symbols.
This means that messages and enums are excluded from the option import. In the
following example, the `Bar` message cannot be used as a field type in
`foo.proto`, but options with type `Bar` can still be set.
```proto
// bar.proto
edition = "2024";
import "google/protobuf/descriptor.proto";
message Bar {
bool bar = 1;
}
extend proto2.FileOptions {
bool file_opt1 = 5000;
Bar file_opt2 = 5001;
}
// foo.proto:
edition = "2024";
import option "bar.proto";
option (file_opt1) = true;
option (file_opt2) = {bar: true};
message Foo {
// Bar bar = 1; // This is not allowed
}
```
Option imports do not require generated code for its symbols and should thus be
provided as `option_deps` in `proto_library` instead of `deps`. This avoids
generating unreachable code.
```proto
proto_library(
name = "foo",
srcs = ["foo.proto"],
option_deps = [":custom_option_proto"]
)
```
Option imports and `option_deps` are strongly recommended when importing
Protobuf language features and other custom options to avoid generating
unnecessary code.
This replaces `import weak`, which was removed in Edition 2024.
#### `export` / `local` Keywords {#export-local}
`export` and `local` keywords were added in Edition 2024 as modifiers for the
symbol visibility of importable symbols, from the default behavior specified by
[`features.default_symbol_visibility`](/editions/features#symbol-vis).
This controls which symbols can be imported from other proto files, but does not
affect code-generation.
In Edition 2024, these can be set on all `message` and `enum` symbols by
default. However, some values of the `default_symbol_visibility` feature further
restrict which symbols are exportable.
Example:
```proto
// Top-level symbols are exported by default in Edition 2024
message LocalMessage {
int32 baz = 1;
// Nested symbols are local by default in Edition 2024; applying the `export`
// keyword overrides this
export enum ExportedNestedEnum {
UNKNOWN_EXPORTED_NESTED_ENUM_VALUE = 0;
}
}
// The `local` keyword overrides the default behavior of exporting messages
local message AnotherMessage {
int32 foo = 1;
}
```
#### `import weak` and Weak Field Option {#import-weak}
As of Edition 2024, weak imports are no longer allowed.
If you previously relied on `import weak` to declare a "weak
dependency"—to import custom options without generated code for C++ and
Go—you should instead migrate to use `import option`.
See [`import option`](/editions/overview#import-option)
for more details.
#### `ctype` Field Option {#ctype}
As of Edition 2024, `ctype` field option is no longer allowed. Use the
`string_type` feature instead.
See
[`features.(pb.cpp).string_type`](/editions/features#string_type)
for more details.
#### `java_multiple_files` File Option {#java-mult-files}
As of Edition 2024, the `java_multiple_files` file option no longer available.
Use the
[`features.(pb.java).nest_in_file_class`](/editions/features#java-nest_in_file)
Java feature, instead.
---
## File: content/getting-started/_index.md
+++
title = "Tutorials"
weight = 200
description = "Each tutorial in this section shows you how to implement a simple application using Protobuf in your favourite language, introducing you to the language's Protobuf API as well as showing you the basics of creating and using .proto files."
type = "docs"
+++
The complete sample code for each application is also provided.
The tutorials don't assume that you know anything about Protobuf, but do assume
that you are comfortable writing code in your chosen language, including using
file I/O.
---
## File: content/getting-started/cpptutorial.md
+++
title = "Protobuf Basics: C++"
weight = 210
linkTitle = "C++"
description = "A basic C++ programmers introduction to working with Protobuf."
type = "docs"
+++
This tutorial provides a basic C++ programmers introduction to working with
Protobuf. By walking through creating a simple example application, it shows you
how to
- Define message formats in a `.proto` file.
- Use the Protobuf compiler.
- Use the C++ Protobuf API to write and read messages.
This isn't a comprehensive guide to using Protobuf in C++. For more detailed
reference information, see the
[Protobuf Language Guide](/programming-guides/editions),
the [C++ API Reference](/reference/cpp/api-docs), the
[C++ Generated Code Guide](/reference/cpp/cpp-generated),
and the
[Encoding Reference](/programming-guides/encoding).
## The Problem Domain {#problem-domain}
The example we're going to use is a very simple "address book" application that
can read and write people's contact details to and from a file. Each person in
the address book has a name, an ID, an email address, and a contact phone
number.
How do you serialize and retrieve structured data like this? There are a few
ways to solve this problem:
- The raw in-memory data structures can be sent/saved in binary form. Over
time, this is a fragile approach, as the receiving/reading code must be
compiled with exactly the same memory layout, endianness, etc. Also, as
files accumulate data in the raw format and copies of software that are
wired for that format are spread around, it's very hard to extend the
format.
- You can invent an ad-hoc way to encode the data items into a single
string -- such as encoding 4 ints as "12:3:-23:67". This is a simple and
flexible approach, although it does require writing one-off encoding and
parsing code, and the parsing imposes a small run-time cost. This works best
for encoding very simple data.
- Serialize the data to XML. This approach can be very attractive since XML is
(sort of) human readable and there are binding libraries for lots of
languages. This can be a good choice if you want to share data with other
applications/projects. However, XML is notoriously space intensive, and
encoding/decoding it can impose a huge performance penalty on applications.
Also, navigating an XML DOM tree is considerably more complicated than
navigating simple fields in a class normally would be.
Instead of these options, you can use Protobuf. Protobuf is the flexible,
efficient, automated solution to solve exactly this problem. With Protobuf, you
write a `.proto` description of the data structure you wish to store. From that,
the Protobuf compiler creates a class that implements automatic encoding and
parsing of the Protobuf data with an efficient binary format. The generated
class provides getters and setters for the fields that make up a Protobuf
message and takes care of the details of reading and writing the message as a
unit. Importantly, the Protobuf format supports the idea of extending the format
over time in such a way that the code can still read data encoded with the old
format.
## Where to Find the Example Code {#example-code}
The example code is included in the source code package, under the
["examples" directory](https://github.com/protocolbuffers/protobuf/tree/main/examples).
## Defining Your Protocol Format {#protocol-format}
To create your address book application, you'll need to start with a `.proto`
file. The definitions in a `.proto` file are simple: you add a *message* for
each data structure you want to serialize, then specify a name and a type for
each field in the message. Here is the `.proto` file that defines your messages,
`addressbook.proto`.
```proto
edition = "2023";
package tutorial;
message Person {
string name = 1;
int32 id = 2;
string email = 3;
enum PhoneType {
PHONE_TYPE_UNSPECIFIED = 0;
PHONE_TYPE_MOBILE = 1;
PHONE_TYPE_HOME = 2;
PHONE_TYPE_WORK = 3;
}
message PhoneNumber {
string number = 1;
PhoneType type = 2;
}
repeated PhoneNumber phones = 4;
}
message AddressBook {
repeated Person people = 1;
}
```
As you can see, the syntax is similar to C++ or Java. Let's go through each part
of the file and see what it does.
The `.proto` file starts with an `edition` declaration. Editions replace the
older `syntax = "proto2"` and `syntax = "proto3"` declarations and provide a
more flexible way to evolve the language over time.
Next is a package declaration, which helps to prevent naming conflicts between
different projects. In C++, your generated classes will be placed in a namespace
matching the package name.
Following the package declaration are your message definitions. A message is
just an aggregate containing a set of typed fields. Many standard simple data
types are available as field types, including `bool`, `int32`, `float`,
`double`, and `string`. You can also add further structure to your messages by
using other message types as field types -- in the above example the `Person`
message contains `PhoneNumber` messages, while the `AddressBook` message
contains `Person` messages. You can even define message types nested inside
other messages -- as you can see, the `PhoneNumber` type is defined inside
`Person`. You can also define enum types if you want one of your fields to have
one of a predefined list of values -- here you want to specify that a phone
number can be one of several types.
The " = 1", " = 2" markers on each element identify the unique field number that
field uses in the binary encoding. Field numbers 1-15 require one less byte to
encode than higher numbers, so as an optimization you can decide to use those
numbers for the commonly used or repeated elements, leaving field numbers 16 and
higher for less-commonly used elements.
Fields can be one of the following:
* singular: By default, fields are optional, meaning the field may or may not
be set. If a singular field is not set, a type-specific default is used:
zero for numeric types, the empty string for strings, false for bools, and
the first defined enum value for enums (which must be 0). Note that you
cannot explicitly set a field to `singular`. This is a description of a
non-repeated field.
* **`repeated`**: The field may be repeated any number of times (including
zero). The order of the repeated values will be preserved. Think of repeated
fields as dynamically sized arrays.
In older versions of Protobuf, a `required` keyword existed, but it has been
found to be brittle and is not supported in modern versions of Protobuf (though
editions does have a feature you can use to enable it, for backward
compatibility).
You'll find a complete guide to writing `.proto` files -- including all the
possible field types -- in the
[Protobuf Language Guide](/programming-guides/editions).
Don't go looking for facilities similar to class inheritance, though -- Protobuf
doesn't do that.
## Compiling Your Protobuf Schema {#compiling-protocol-buffers}
Now that you have a `.proto`, the next thing you need to do is generate the
classes you'll need to read and write `AddressBook` (and hence `Person` and
`PhoneNumber`) messages. To do this, you need to run the Protobuf compiler
`protoc` on your `.proto`:
1. If you haven't installed the compiler, follow the instructions in
[Protobuf Compiler Installation](/installation/).
2. Now run the compiler, specifying the source directory (where your
application's source code lives -- the current directory is used if you
don't provide a value), the destination directory (where you want the
generated code to go; often the same as `$SRC_DIR`), and the path to your
`.proto`. In this case:
```shell
protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/addressbook.proto
```
Because you want C++ classes, you use the `--cpp_out` option -- similar
options are provided for other supported languages.
This generates the following files in your specified destination directory:
- `addressbook.pb.h`, the header which declares your generated classes.
- `addressbook.pb.cc`, which contains the implementation of your classes.
## The Protobuf API {#protobuf-api}
Let's look at some of the generated code and see what classes and functions the
compiler has created for you. If you look in `addressbook.pb.h`, you can see
that you have a class for each message you specified in `addressbook.proto`.
Looking closer at the `Person` class, you can see that the compiler has
generated accessors for each field. For example, for the `name`, `id`, `email`,
and `phones` fields, you have these methods:
```cpp
// name
bool has_name() const; // Only for explicit presence
void clear_name();
const ::std::string& name() const;
void set_name(const ::std::string& value);
::std::string* mutable_name();
// id
bool has_id() const;
void clear_id();
int32_t id() const;
void set_id(int32_t value);
// email
bool has_email() const;
void clear_email();
const ::std::string& email() const;
void set_email(const ::std::string& value);
::std::string* mutable_email();
// phones
int phones_size() const;
void clear_phones();
const ::google::protobuf::RepeatedPtrField< ::tutorial::Person_PhoneNumber >& phones() const;
::google::protobuf::RepeatedPtrField< ::tutorial::Person_PhoneNumber >* mutable_phones();
const ::tutorial::Person_PhoneNumber& phones(int index) const;
::tutorial::Person_PhoneNumber* mutable_phones(int index);
::tutorial::Person_PhoneNumber* add_phones();
```
As you can see, the getters have exactly the name as the field in lowercase, and
the setter methods begin with `set_`. There are also `has_` methods for singular
fields that have explicit presence tracking, which return true if that field has
been set. Finally, each field has a `clear_` method that un-sets the field back
to its default state.
While the numeric `id` field just has the basic accessor set described above,
the `name` and `email` fields have a couple of extra methods because they're
strings -- a `mutable_` getter that lets you get a direct pointer to the string,
and an extra setter. Note that you can call `mutable_email()` even if `email` is
not already set; it will be initialized to an empty string automatically. If you
had a repeated message field in this example, it would also have a `mutable_`
method but not a `set_` method.
Repeated fields also have some special methods -- if you look at the methods for
the repeated `phones` field, you'll see that you can
- check the repeated field's `_size` (in other words, how many phone numbers
are associated with this `Person`).
- get a specified phone number using its index.
- update an existing phone number at the specified index.
- add another phone number to the message which you can then edit (repeated
scalar types have an `add_` that just lets you pass in the new value).
For more information on exactly what members the protocol compiler generates for
any particular field definition, see the
[C++ generated code reference](/reference/cpp/cpp-generated).
### Enums and Nested Classes {#enums-nested-classes}
The generated code includes a `PhoneType` enum that corresponds to your `.proto`
enum. You can refer to this type as `Person::PhoneType` and its values as
`Person::PHONE_TYPE_MOBILE`, `Person::PHONE_TYPE_HOME`, and
`Person::PHONE_TYPE_WORK` (the implementation details are a little more
complicated, but you don't need to understand them to use the enum).
The compiler has also generated a nested class for you called
`Person::PhoneNumber`. If you look at the code, you can see that the "real"
class is actually called `Person_PhoneNumber`, but a typedef defined inside
`Person` allows you to treat it as if it were a nested class. The only case
where this makes a difference is if you want to forward-declare the class in
another file -- you cannot forward-declare nested types in C++, but you can
forward-declare `Person_PhoneNumber`.
### Standard Message Methods {#standard-message-methods}
Each message class also contains a number of other methods that let you check or
manipulate the entire message, including:
- `bool IsInitialized() const;`: checks if all required fields have been set.
- `string DebugString() const;`: returns a human-readable representation of
the message, particularly useful for debugging.
- `void CopyFrom(const Person& from);`: overwrites the message with the given
message's values.
- `void Clear();`: clears all the elements back to the empty state.
These and the I/O methods described in the following section implement the
`Message` interface shared by all C++ Protobuf classes. For more info, see the
[complete API documentation for `Message`](/reference/cpp/api-docs/google.protobuf.message#Message).
### Parsing and Serialization {#parsing-serialization}
Finally, each Protobuf class has methods for writing and reading messages of
your chosen type using the Protobuf
[binary format](/programming-guides/encoding). These
include:
- `bool SerializeToString(string* output) const;`: serializes the message and
stores the bytes in the given string. Note that the bytes are binary, not
text; we only use the `string` class as a convenient container.
- `bool ParseFromString(const string& data);`: parses a message from the given
string.
- `bool SerializeToOstream(ostream* output) const;`: writes the message to the
given C++ `ostream`.
- `bool ParseFromIstream(istream* input);`: parses a message from the given
C++ `istream`.
These are just a couple of the options provided for parsing and serialization.
See the
[`Message` API reference](/reference/cpp/api-docs/google.protobuf.message#Message)
for a complete list.
{{% alert title="Important" color="warning" %}} **Protobuf and Object Oriented Design**
Protobuf classes are basically data holders (like structs in C) that don't
provide additional functionality; they don't make good first class citizens in
an object model. If you want to add richer behavior to a generated class, the
best way to do this is to wrap the generated Protobuf class in an
application-specific class. Wrapping Protobuf messages is also a good idea if
you don't have control over the design of the .proto file (if, say, you're
reusing one from another project). In that case, you can use the wrapper class
to craft an interface better suited to the unique environment of your
application: hiding some data and methods, exposing convenience functions, etc.
**You cannot add behavior to the generated classes by inheriting from them**, as
they are final. This prevents breaking internal mechanisms and is not good
object-oriented practice anyway.
{{% /alert %}}
## Writing a Message {#writing-a-message}
Now let's try using your Protobuf classes. The first thing you want your address
book application to be able to do is write personal details to your address book
file. To do this, you need to create and populate instances of your Protobuf
classes and then write them to an output stream.
Here is a program that reads an `AddressBook` from a file, adds one new `Person`
to it based on user input, and writes the new `AddressBook` back out to the file
again. The parts which directly call or reference code generated by the protocol
compiler are highlighted.
```cpp
#include
#include
#include
#include "addressbook.pb.h"
using namespace std;
// This function fills in a Person message based on user input.
void PromptForAddress(tutorial::Person& person) {
cout << "Enter person ID number: ";
int id;
cin >> id;
person.set_id(id);
cin.ignore(256, '\n');
cout << "Enter name: ";
getline(cin, *person.mutable_name());
cout << "Enter email address (blank for none): ";
string email;
getline(cin, email);
if (!email.empty()) {
person.set_email(email);
}
while (true) {
cout << "Enter a phone number (or leave blank to finish): ";
string number;
getline(cin, number);
if (number.empty()) {
break;
}
tutorial::Person::PhoneNumber* phone_number = person.add_phones();
phone_number->set_number(number);
cout << "Is this a mobile, home, or work phone? ";
string type;
getline(cin, type);
if (type == "mobile") {
phone_number->set_type(tutorial::Person::PHONE_TYPE_MOBILE);
} else if (type == "home") {
phone_number->set_type(tutorial::Person::PHONE_TYPE_HOME);
} else if (type == "work") {
phone_number->set_type(tutorial::Person::PHONE_TYPE_WORK);
} else {
cout << "Unknown phone type. Using default." << endl;
}
}
}
// Main function: Reads the entire address book from a file,
// adds one person based on user input, then writes it back out to the same
// file.
int main(int argc, char* argv[]) {
// Verify that the version of the library that we linked against is
// compatible with the version of the headers we compiled against.
GOOGLE_PROTOBUF_VERIFY_VERSION;
if (argc != 2) {
cerr << "Usage: " << argv[0] << " ADDRESS_BOOK_FILE" << endl;
return -1;
}
tutorial::AddressBook address_book;
{
// Read the existing address book.
fstream input(argv[1], ios::in | ios::binary);
if (!input) {
cout << argv[1] << ": File not found. Creating a new file." << endl;
} else if (!address_book.ParseFromIstream(&input)) {
cerr << "Failed to parse address book." << endl;
return -1;
}
}
// Add an address.
PromptForAddress(*address_book.add_people());
{
// Write the new address book back to disk.
fstream output(argv[1], ios::out | ios::trunc | ios::binary);
if (!address_book.SerializeToOstream(&output)) {
cerr << "Failed to write address book." << endl;
return -1;
}
}
// Optional: Delete all global objects allocated by libprotobuf.
google::protobuf::ShutdownProtobufLibrary();
return 0;
}
```
Notice the `GOOGLE_PROTOBUF_VERIFY_VERSION` macro. It is good practice -- though
not strictly necessary -- to execute this macro before using the C++ Protobuf
library. It verifies that you have not accidentally linked against a version of
the library which is incompatible with the version of the headers you compiled
with. If a version mismatch is detected, the program will abort. Note that every
`.pb.cc` file automatically invokes this macro on startup.
Also notice the call to `ShutdownProtobufLibrary()` at the end of the program.
All this does is delete any global objects that were allocated by the Protobuf
library. This is unnecessary for most programs, since the process is just going
to exit anyway and the OS will take care of reclaiming all of its memory.
However, if you use a memory leak checker that requires that every last object
be freed, or if you are writing a library which may be loaded and unloaded
multiple times by a single process, then you may want to force Protobuf to clean
up everything.
## Reading a Message {#reading-a-message}
Of course, an address book wouldn't be much use if you couldn't get any
information out of it! This example reads the file created by the above example
and prints all the information in it.
```cpp
#include
#include
#include
#include "addressbook.pb.h"
using namespace std;
// Iterates though all people in the AddressBook and prints info about them.
void ListPeople(const tutorial::AddressBook& address_book) {
for (const tutorial::Person& person : address_book.people()) {
cout << "Person ID: " << person.id() << endl;
cout << " Name: " << person.name() << endl;
if (!person.has_email()) {
cout << " E-mail address: " << person.email() << endl;
}
for (const tutorial::Person::PhoneNumber& phone_number : person.phones()) {
switch (phone_number.type()) {
case tutorial::Person::PHONE_TYPE_MOBILE:
cout << " Mobile phone #: ";
break;
case tutorial::Person::PHONE_TYPE_HOME:
cout << " Home phone #: ";
break;
case tutorial::Person::PHONE_TYPE_WORK:
cout << " Work phone #: ";
break;
case tutorial::Person::PHONE_TYPE_UNSPECIFIED:
default:
cout << " Phone #: ";
break;
}
cout << phone_number.number() << endl;
}
}
}
// Main function: Reads the entire address book from a file and prints all
// the information inside.
int main(int argc, char* argv[]) {
// Verify that the version of the library that we linked against is
// compatible with the version of the headers we compiled against.
GOOGLE_PROTOBUF_VERIFY_VERSION;
if (argc != 2) {
cerr << "Usage: " << argv[0] << " ADDRESS_BOOK_FILE" << endl;
return -1;
}
tutorial::AddressBook address_book;
{
// Read the existing address book.
fstream input(argv[1], ios::in | ios::binary);
if (!address_book.ParseFromIstream(&input)) {
cerr << "Failed to parse address book." << endl;
return -1;
}
}
ListPeople(address_book);
// Optional: Delete all global objects allocated by libprotobuf.
google::protobuf::ShutdownProtobufLibrary();
return 0;
}
```
## Extending a Protobuf Message {#extending-a-protobuf}
Sooner or later after you release the code that uses your Protobuf message, you
will undoubtedly want to "improve" the message's definition. If you want your
new messages to be backwards-compatible, and your old messages to be
forward-compatible -- and you almost certainly do want this -- then there are
some rules you need to follow. In the new version of the message:
* you *must not* change the field numbers of any existing fields.
* you *may* delete singular or repeated fields.
* you *may* add new singular or repeated fields but you must use fresh field
numbers (that is, field numbers that were never used in this message, not
even by deleted fields).
(There are
[some exceptions](/programming-guides/editions#updating)
to these rules, but they are rarely used.)
If you follow these rules, old code will happily read new messages and simply
ignore any new fields. To the old code, fields that were deleted will simply
have their default value, and deleted repeated fields will be empty. New code
will also transparently read old messages. However, keep in mind that new fields
will not be present in old messages, so you will need to check for their
presence by checking if they have the default value (e.g., an empty string)
before use.
## Optimization Tips {#optimization}
The C++ Protobuf library is extremely heavily optimized. However, proper usage
can improve performance even more. Here are some tips for squeezing every last
drop of speed out of the library:
- **Use Arenas for memory allocation.** When you create many Protobuf messages
in a short-lived operation (like parsing a single request), the system's
memory allocator can become a bottleneck. Arenas are designed to mitigate
this. By using an arena, you can perform many allocations with low overhead,
and a single deallocation for all of them at once. This can significantly
improve performance in message-heavy applications.
To use arenas, you allocate messages on a `google::protobuf::Arena` object:
```cpp
google::protobuf::Arena arena;
tutorial::Person* person = google::protobuf::Arena::Create(&arena);
// ... populate person ...
```
When the arena object is destroyed, all messages allocated on it are freed.
For more details, see the [Arenas guide](/arenas).
- **Reuse non-arena message objects when possible.** Messages try to keep
around any memory they allocate for reuse, even when they are cleared. Thus,
if you are handling many messages with the same type and similar structure
in succession, it is a good idea to reuse the same message object each time
to take load off the memory allocator. However, objects can become bloated
over time, especially if your messages vary in "shape" or if you
occasionally construct a message that is much larger than usual. You should
monitor the sizes of your message objects by calling the `SpaceUsed` method
and delete them once they get too big.
Reusing arena messages can lead to unbounded memory growth. Reusing heap
messages is safer. Even with heap message, though, you can still experience
issues with the high water mark of fields. For example, if you see messages:
```none
a: [1, 2, 3, 4]
b: [1]
```
and
```none
a: [1]
b: [1, 2, 3, 4]
```
and reuse the messages, then both fields will have enough memory for the
largest they have seen. So if each input only had 5 elements, the reused
message will have memory for 8.
- Your system's memory allocator may not be well-optimized for allocating lots
of small objects from multiple threads. Try using
[Google's TCMalloc](https://github.com/google/tcmalloc) instead.
## Advanced Usage {#advanced-usage}
Protobuf has uses that go beyond simple accessors and serialization. Be sure to
explore the [C++ API reference](/reference/cpp) to see
what else you can do with it.
One key feature provided by protocol message classes is *reflection*. You can
iterate over the fields of a message and manipulate their values without writing
your code against any specific message type. One very useful way to use
reflection is for converting protocol messages to and from other encodings, such
as XML or JSON. A more advanced use of reflection might be to find differences
between two messages of the same type, or to develop a sort of "regular
expressions for protocol messages" in which you can write expressions that match
certain message contents. If you use your imagination, it's possible to apply
Protobuf to a much wider range of problems than you might initially expect!
Reflection is provided by the
[`Message::Reflection` interface](/reference/cpp/api-docs/google.protobuf.message#Reflection).