Day 114 — Code Generation with `protoc` & `buf`¶

Month 5 · Week 1 · ⬅ Day 113 · Day 115 ➡ · Journal index

🎯 Learning Objective¶

Turn a .proto schema into idiomatic Go using both raw protoc and the modern buf workflow, and understand exactly which files each plugin emits and why.

📚 Topics¶

protoc + plugins (protoc-gen-go, protoc-gen-go-grpc), paths=source_relative
buf (buf.yaml, buf.gen.yaml, buf lint, buf breaking, buf generate)

📖 Reading / Sources¶

📝 Notes¶

protoc is just a parser/driver: it parses .proto into a descriptor set and hands it to plugins over stdin/stdout. The plugins, not protoc, write Go → [[code-generation]].
Two separate plugins for Go gRPC: protoc-gen-go emits message types into *.pb.go; protoc-gen-go-grpc emits the client/server stubs into *_grpc.pb.go. Install both with go install and make sure $(go env GOPATH)/bin is on PATH.
The go_package option in the .proto decides the import path of the generated package — it is required for Go. The ;name suffix sets the package name: option go_package = ".../gen/userv1;userv1";.
paths=source_relative writes output next to the proto's package path instead of recreating the full go_package directory tree under --go_out. It's the option you almost always want.
The generated *_grpc.pb.go embeds an UnimplementedXxxServer struct. Embed it in your server so adding new RPCs to the schema doesn't break compilation (forward compatibility) → [[grpc-server]].
buf replaces brittle protoc shell lines: buf.yaml declares the module + lint/breaking rules; buf.gen.yaml declares plugins + outputs; buf generate runs them. buf lint enforces style, and buf breaking diffs against a baseline to catch wire-incompatible changes in CI → [[buf]].
Re-generation is deterministic and idempotent: generated files are build artifacts. Check them in for easy diffs, or .gitignore them and generate in CI — pick one and be consistent.

💻 Code Examples¶

# buf.gen.yaml (v2) — plugins + where their output lands
version: v2
managed:
  enabled: true            # let buf manage go_package etc.
plugins:
  - local: protoc-gen-go
    out: gen
    opt: paths=source_relative
  - local: protoc-gen-go-grpc
    out: gen
    opt: paths=source_relative

# Install the two Go plugins (once):
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

# buf workflow:
buf lint && buf generate

# Equivalent raw protoc invocation:
protoc --go_out=gen       --go_opt=paths=source_relative \
       --go-grpc_out=gen  --go-grpc_opt=paths=source_relative \
       proto/user/v1/user.proto

No stdlib example here — codegen needs the real toolchain. The output of this step (wire bytes) is what examples/month-05/protowire decodes by hand. Run: go run ./examples/month-05/protowire

🏋️ Exercises / Practice¶

Exercise	Status	Link
Inspect generated varint bytes vs. `encoding/binary`	✅	exercises/month-05/week-1/varint

🐛 Mistakes Made¶

Ran protoc and got program not found: protoc-gen-go — the plugins were installed but $(go env GOPATH)/bin wasn't on PATH. Added it.
Forgot option go_package → protoc-gen-go refused to run. It's mandatory for Go.

❓ Open Questions¶

When is managed mode in buf worth it vs. hand-writing go_package in every file? (Leaning: managed mode for multi-file modules so the import prefix lives in one place.)

🧠 Active Recall (answer without looking)¶

Q: Which file holds the message structs and which holds the service stubs?
A

*.pb.go (from protoc-gen-go) holds the message types and accessors; *_grpc.pb.go (from protoc-gen-go-grpc) holds the client interface and server registration/stubs. They come from two different plugins. 2. Q: Why embed UnimplementedUserServiceServer in your server type?

A

Forward compatibility: it provides default "Unimplemented" methods so that when a new RPC is added to the .proto, existing servers still compile (and return codes.Unimplemented) instead of failing to satisfy the interface.

🪶 Feynman Reflection¶

protoc/buf is a compiler whose backend you choose. You feed it a .proto, it parses it once, then streams a descriptor to each plugin which writes language-specific code. For Go that's two plugins: one for data, one for RPC. buf just wraps the whole thing with linting and breaking-change detection so the contract stays safe.

🕳️ Knowledge Gaps¶

The Buf Schema Registry (remote plugins, buf.build/... deps) — revisit when sharing protos across teams.
Generating gRPC-Gateway / OpenAPI stubs alongside the Go code.

✅ Summary¶

I can install the Go protobuf/gRPC plugins, generate code with both protoc and buf, explain the two output files, and use buf lint/buf breaking to keep the schema safe.

⏭️ Next Steps / Prep for Tomorrow¶

Day 115: implement and call a unary RPC against the generated stubs.

Time spent	Difficulty	Confidence
90 min	🟦🟦⬜⬜⬜	🟦🟦🟦⬜⬜

Suggested commit: docs(journal): protoc and buf code generation (day 114)

Day 114 — Code Generation with protoc & buf¶