Skip to content

Day 150 — Image hardening & distroless

Month 6 · Week 2 · ⬅ Day 149 · Day 151 ➡ · Journal index

🎯 Learning Objective

Shrink the attack surface of a Go container: choose scratch vs distroless, run as non-root, mount a read-only root filesystem, and drop Linux capabilities.

📚 Topics

  • scratch vs gcr.io/distroless/static — what each gives you
  • Non-root USER, read-only rootfs, dropped capabilities, no shell
  • TLS roots, timezone data, and /etc/passwd in minimal images

📖 Reading / Sources

📝 Notes

  • Smaller image = smaller attack surface. No shell, no package manager, no libc means far fewer CVEs and no sh for an attacker to pivot into → [[image-hardening]].
  • scratch is empty: no CA certs, no /etc/passwd, no tzdata. distroless static adds exactly those few essentials (ca-certificates, /etc/passwd with a nonroot user, tzdata) but still no shell → [[distroless]]. Use static for CGO_ENABLED=0 Go binaries; base if you need glibc.
  • A Go program calling HTTPS needs CA roots; on scratch you must COPY /etc/ssl/certs/ca-certificates.crt from the builder, or x509.SystemCertPool returns nothing and TLS fails. distroless includes them.
  • Run as non-root. Set USER nonroot:nonroot (distroless ships UID 65532) or a numeric UID. A root process that escapes the container is root on issues; a UID 65532 process is far less dangerous.
  • Harden at runtime too: --read-only root filesystem (+ a tmpfs for /tmp if needed), --cap-drop=ALL, and --security-opt=no-new-privileges. In Kubernetes these map to securityContext fields (runAsNonRoot, readOnlyRootFilesystem, capabilities.drop).
  • If you embed timezone data, import _ "time/tzdata" so the zoneinfo is compiled in and you don't depend on the OS tzdata at all → [[tzdata]].
  • Pin image tags by digest (@sha256:...) for reproducibility; :latest drifts.

💻 Code Examples

# syntax=docker/dockerfile:1
FROM golang:1.22 AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -ldflags="-s -w" -o /out/app ./cmd/app

# distroless: CA certs + tzdata + a nonroot user, but NO shell or libc.
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /out/app /app
USER nonroot:nonroot          # UID 65532, set by the base image
ENTRYPOINT ["/app"]
# run with:  docker run --read-only --cap-drop=ALL \
#            --security-opt=no-new-privileges --tmpfs /tmp myimg

The Go side can verify it actually has TLS roots at startup:

// Fail fast if the image shipped without CA certificates.
pool, err := x509.SystemCertPool()
if err != nil || pool == nil {
    log.Fatal("no system CA roots: TLS to external services will fail")
}

🏋️ Exercises / Practice

Exercise Status Link
Resolve secrets from mounted files (non-root friendly) ✅ exercises/month-06/week-2/secretsource

🐛 Mistakes Made

  • HTTPS calls failed with "x509: certificate signed by unknown authority" on a scratch image — no CA bundle. Switched to distroless static, which bundles ca-certificates.
  • Logged the time in UTC only on a scratch image because there was no tzdata. Added import _ "time/tzdata" to embed it.

❓ Open Questions

  • distroless :debug ships a busybox shell for kubectl exec debugging — is it safe to use only in staging? (Yes; keep it out of prod images.)

🧠 Active Recall (answer without looking)

  1. Q: Your scratch-based Go service can't make HTTPS calls. Most likely cause?
    A

No CA root certificates in the image. scratch is empty, so x509.SystemCertPool finds nothing. Copy /etc/ssl/certs/ca-certificates.crt from the builder, or use distroless static, which bundles them. 2. Q: Name three runtime hardening flags and what they prevent.

A

--read-only (no writes to the rootfs → no dropping tools/persistence), --cap-drop=ALL (remove Linux capabilities like NET_RAW), --security-opt=no-new-privileges (block setuid privilege escalation). Plus running as non-root USER.

🪶 Feynman Reflection

A hardened image is a sealed room with one door (the binary) and nothing else inside — no shell to run, no tools to grab, no root keys lying around. If an attacker gets in through the app, there's nowhere to go and nothing to use.

🕳️ Knowledge Gaps

  • Image signing/provenance (cosign, SLSA) — comes up with the CI pipeline on [[day-151]].

✅ Summary

I can pick the right minimal base, run non-root with a read-only rootfs and dropped capabilities, and make sure the image still carries CA roots and tzdata my Go code needs.

⏭️ Next Steps / Prep for Tomorrow

  • Day 151: automate build → test → vulnerability scan in GitHub Actions.
Time spent Difficulty Confidence
90 min 🟦🟦⬜⬜⬜ 🟦🟦🟦⬜⬜

Suggested commit: docs(journal): image hardening & distroless (day 150)