Day 076 — errgroup & Rate Limiting

Month 3 · Week 3 · ⬅ Day 075 · Day 077 ➡ · Journal index

🎯 Learning Objective

Run a group of goroutines, propagate the first error while cancelling the rest with errgroup.Group, and pace work with stdlib and token-bucket rate limiting.

📚 Topics

• golang.org/x/sync/errgroup: Go, Wait, WithContext, SetLimit
• First-error semantics + automatic context cancellation
• Rate limiting: time.Ticker pacing (stdlib) vs token bucket (x/time/rate)

📖 Reading / Sources

• golang.org/x/sync/errgroup docs
• golang.org/x/time/rate docs
• time.Ticker docs

📝 Notes

• errgroup.Group is "WaitGroup + error handling". g.Go(func() error {...}) launches a task; g.Wait() blocks for all and returns the first non-nil error (others are discarded) → [[errgroup]].
• errgroup.WithContext(ctx) returns a group and a derived context that is cancelled the moment any task errors (or when Wait returns). Tasks should select on that ctx so siblings stop early → [[context-cancellation]].
• g.SetLimit(n) bounds concurrency: g.Go blocks until a slot frees, turning the group into a built-in semaphore (Day 074) → [[bounded-concurrency]]. g.TryGo is the non-blocking variant.
• errgroup lives in golang.org/x/sync (an external module), so there's no stdlib-only runnable example — today's exercise re-implements the mechanism with sync + context to prove I understand it → [[taskgroup]].
• Rate limiting caps how often work happens (events/sec), distinct from bounded parallelism which caps how many at once. The two compose.
• Stdlib pacing: a time.Ticker fires every d; receive from ticker.C before each unit of work to enforce a steady rate. Always defer ticker.Stop() to free it → [[time-ticker]].
• A token bucket allows short bursts: tokens refill at rate r up to capacity b; each event spends a token. golang.org/x/time/rate.Limiter is the canonical implementation (Wait, Allow, Reserve). A Ticker is the no-burst (b == 1) special case.
• Don't conflate the two limits: an errgroup with SetLimit bounds in-flight work; a Limiter bounds the start rate. A scraper often needs both → [[rate-limiting]].

💻 Code Examples

// errgroup: fan out, cancel-on-first-error, bounded to 4 concurrent tasks.
// import "golang.org/x/sync/errgroup"  (external module — not stdlib)
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(4)
for _, u := range urls {
    u := u
    g.Go(func() error {
        req, _ := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return fmt.Errorf("fetch %s: %w", u, err) // first error cancels ctx
        }
        return resp.Body.Close()
    })
}
if err := g.Wait(); err != nil { // first error across all tasks
    log.Println(err)
}

// Stdlib rate limiting (no burst): pace work to one unit per tick.
ticker := time.NewTicker(200 * time.Millisecond) // 5 events/sec
defer ticker.Stop()
for _, job := range jobs {
    <-ticker.C // block until the next tick, enforcing the rate
    process(job)
}

No runnable example today: errgroup and the token-bucket rate.Limiter are in golang.org/x/..., outside the stdlib. The exercise taskgroup re-implements errgroup with only sync+context.

🏋️ Exercises / Practice

Exercise	Status	Link
Stdlib mini-errgroup (Go/Wait/SetLimit/WithContext)	✅	exercises/month-03/week-3/taskgroup/

🐛 Mistakes Made

• Returned the first error from Wait but forgot tasks keep running unless they watch the group's ctx — wired each task to ctx.Done() so a failure actually cancels the rest.
• Created a time.Ticker and never called Stop() → leaked the underlying timer in a long-running loop. Added defer ticker.Stop().

❓ Open Questions

• When is rate.Limiter (burst) preferable to a Ticker (steady)? (Bursty clients/APIs that allow a buffer of requests; revisit in the API month.)

🧠 Active Recall (answer without looking)

1. Q: What does errgroup.Group.Wait return when three tasks fail, and what happens to the others after the first failure?
   

   A

   It returns the **first** non-nil error only (the rest are dropped). With `WithContext`, the first error cancels the derived context, so the other tasks — if they select on that ctx — stop early.

2. Q: Rate limiting vs bounded parallelism — what does each cap?
   

   A

   Rate limiting caps *how often* events start (events/sec, e.g. a `Ticker` or token bucket). Bounded parallelism caps *how many* run at once (a semaphore / `SetLimit`). They're orthogonal and often used together.

🪶 Feynman Reflection

errgroup is a WaitGroup that also carries a clipboard: the first worker to fail writes the reason and flips a master switch (cancels the context) so everyone else can down tools. Rate limiting is a separate metronome — it doesn't care how many workers there are, only that a new beat (and thus a new task start) happens no faster than the set tempo.

🕳️ Knowledge Gaps

• rate.Limiter's Reserve/Wait cancellation semantics under load — hands-on in the API month.

✅ Summary

I understand errgroup (first-error + cancel-the-rest + SetLimit bounding), proved it with a stdlib re-implementation, and can rate-limit with a time.Ticker (steady) or a token bucket (bursty), knowing rate vs concurrency are different limits.

⏭️ Next Steps / Prep for Tomorrow

• Day 077: Week 3 review + closed-book recall over all the patterns.

---

Time spent	Difficulty	Confidence
90 min	🟦🟦⬜⬜⬜	🟦🟦🟦⬜⬜

Suggested commit: feat(exercises): stdlib mini-errgroup + rate-limiting notes (day 076)