Skip to content

Weekly Review — Month 5 · Week 4 (Days 134–140)

Journal index · Roadmap › this week

📅 The Week in One Line

Built the capstone microservice end to end: a gRPC skeleton, hexagonal layering, a Redis-backed job queue with a worker pool, resilient retries + dead-letter, race-clean tests + metrics, and docs/ADRs — then tagged v0.5.0.

✅ What I Completed

  • Day 134 — gRPC service skeleton: cmd/+internal/ layout, wired server, graceful shutdown via signal.NotifyContext
  • Day 135 — Hexagonal layering: domain-owned ports, error-translating adapters, compile-time interface checks
  • Day 136 — Redis job queue: LPUSH/BRPOPLPUSH, worker pool, backpressure, graceful drain
  • Day 137 — Retries & dead-letter: overflow-safe exponential backoff + full jitter, retry budget, DLQ, idempotency
  • Day 138 — Tests & metrics: fakes + bufconn, go test -race, Prometheus-style labeled counters
  • Day 139 — Docs & ADRs: godoc/doc.go, task-oriented README, append-only ADRs
  • Day 140 — Month review + recall + tag v0.5.0
  • Stdlib examples: jobqueue, workerpool, retry, metrics
  • Exercises solved: 3 (backoff, deadletter, metrics) — all go test green, metrics race-clean

💡 Lessons Learned

  • GracefulStop() drains in-flight RPCs; Stop() kills them. Drive shutdown from signal.NotifyContext + GracefulStop.
  • Dependencies point inward: the core declares the Queue/Repository interfaces; adapters implement them and translate infra errors (redis.Nil/sql.ErrNoRows) to domain errors.
  • var _ Port = (*Adapter)(nil) is a free compile-time conformance test.
  • Redis BRPOP is at-most-once; BRPOPLPUSH into a processing list gives recoverable, at-least-once delivery.
  • Only the producer closes the jobs channel, exactly once; buffered capacity is the backpressure.
  • Compute backoff overflow-safe (double + cap in-loop), add full jitter to avoid thundering herds, and classify transient vs permanent before spending the retry budget.
  • At-least-once ⇒ handlers must be idempotent (dedupe on job ID).
  • Sort label keys before building a metric series key; never use unbounded label values; Snapshot returns a copy.
  • A passing test without -race proves little for concurrent code.

💪 Strengths (what clicked)

  • The worker-pool + channel-ownership model transferred straight from Month 3 concurrency.
  • Hexagonal layering made the core trivially unit-testable with in-memory fakes.
  • Error wrapping (%w) + errors.Is made the DLQ records inspectable.

🧩 Weaknesses (what's still fuzzy)

  • Reclaiming stuck jobs from the :processing list (a reaper with TTL).
  • Choosing retry budgets / backoff caps from real SLOs rather than guessing.
  • A clean metrics interceptor that updates RED metrics without per-handler boilerplate.

🔁 Spaced-Repetition Re-quiz (topics from earlier weeks)

  1. Q: (Day 120) In grpc.ChainUnaryInterceptor(a, b, c), which interceptor is outermost?
    Aa — the first argument wraps everything; it runs first on the way in and last on the way out.
  2. Q: (Day 122) Why is InsecureSkipVerify: true a trap, and what's the right fix?
    AIt disables certificate verification entirely (MITM-vulnerable). The fix is to add the signing CA to the client's RootCAs so the chain verifies.
  3. Q: (Month 3 channels) What does a receive from a closed channel return?
    AThe element type's zero value immediately, with ok == false in the comma-ok form. Close is a broadcast.
  4. Q: (Month 1 errors) How do you check for a sentinel error through wrapping, and how do you extract a typed one?
    Aerrors.Is(err, ErrTarget) for sentinels and errors.As(err, &target) for typed errors; wrap with %w so the chain is walkable.
  5. Q: (Day 118) A handler returns codes.DeadlineExceeded — what HTTP status does grpc-gateway map it to?
    A504 Gateway Timeout.

🎯 Action Items

  • Add a reaper that re-queues :processing entries older than a visibility TTL.
  • Wrap processing in a metrics interceptor emitting jobs_processed_total{status} + a duration histogram.
  • Add a bufconn test that submits a job and asserts it lands on the queue.
  • Write a DLQ re-drive (replay) command.

🚀 Next Week Goals

  • Month 6: production & observability — structured logging, metrics in depth, and distributed tracing across the queue boundary.
  • Streaming RPCs end-to-end; load testing the queue.
  • Containerize and deploy the capstone.

📊 Metrics

Hours Days hit Exercises Commits Avg confidence
10.5 7/7 3 7 3.⅗

Suggested commit: docs(journal): week 4 review