Day 072 — Fan-Out / Fan-In

Month 3 · Week 3 · ⬅ Day 071 · Day 073 ➡ · Journal index

🎯 Learning Objective

Name and build the two halves of parallel processing: fan-out (many goroutines read one channel) and fan-in (merge many channels into one), with a correct single-close discipline.

📚 Topics

• Fan-out: multiple receivers on one source channel
• Fan-in: a merge that multiplexes N channels into one
• Closing the merged channel exactly once with a WaitGroup + closer goroutine

📖 Reading / Sources

• Go blog — Go Concurrency Patterns: Pipelines and cancellation (the "Fan-out, fan-in" section)
• Effective Go — Channels
• sync.WaitGroup docs

📝 Notes

• Fan-out = start several goroutines that all receive from the same channel. Because a channel hands each value to exactly one receiver, the source is spread across workers with zero dispatch logic → [[fan-out]].
• Fan-in (a.k.a. multiplexing) = combine several input channels into one output channel a single consumer can range. The merge starts one forwarding goroutine per input → [[fan-in]].
• The merge's hard part is closing out exactly once, only after every forwarder finishes. Use a WaitGroup: each forwarder wg.Done()s; a separate closer goroutine does wg.Wait(); close(out) → [[close-once-closer]].
• Closing out from inside a forwarder is wrong: the first one to finish would close while others still send → send-on-closed panic.
• Fan-out + fan-in is the worker pool from [[worker-pool]] expressed as composable stages — each stage returns a <-chan so stages snap together.
• Output order is non-deterministic under fan-in; if order matters, carry an index or sort downstream.
• In Go 1.22 the per-iteration loop variable is fresh, but inside a merge it's still clearest (and portable) to pass the channel as a parameter to the forwarder goroutine → [[loop-var-capture]].

💻 Code Examples

// merge multiplexes any number of input channels into one output channel,
// closing it exactly once after every input has drained.
func merge[T any](cs ...<-chan T) <-chan T {
    out := make(chan T)
    var wg sync.WaitGroup
    wg.Add(len(cs))
    for _, c := range cs {
        go func(c <-chan T) {
            defer wg.Done()
            for v := range c {
                out <- v
            }
        }(c)
    }
    go func() { wg.Wait(); close(out) }() // closer: close once, after all forwarders
    return out
}

Full code: examples/month-03/fanin-fanout/main.go · Run: go run ./examples/month-03/fanin-fanout

🏋️ Exercises / Practice

Exercise	Status	Link
Order-preserving bounded Map (fan-out + index reorder)	✅	exercises/month-03/week-3/boundedmap/
Counting semaphore (caps the fan-out width)	✅	exercises/month-03/week-3/semaphore/

🐛 Mistakes Made

• Called close(out) inside each forwarder → second forwarder hit send-on-closed panic. Moved the close into a single closer goroutine guarded by wg.Wait().
• Captured the range variable c directly in the goroutine on an older Go mental model; passing it as a parameter makes intent obvious and is version-proof.

❓ Open Questions

• When the consumer stops early, who tells the fan-out workers to quit? (Answer: a done/context channel — Day 073/075.)

🧠 Active Recall (answer without looking)

1. Q: Why must the merged channel be closed by a closer goroutine and not by the forwarders?
   

   A

   Each forwarder finishes at a different time; if one closes `out` while another is still sending, the still-running send panics. A single closer that runs `wg.Wait()` first guarantees all sends are done before the one close.

2. Q: How does fan-out balance work without any dispatcher?
   

   A

   All workers receive from the same channel, and a channel delivers each value to exactly one ready receiver — so a free worker naturally grabs the next item.

🪶 Feynman Reflection

Fan-out splits one stream of work across many hands; fan-in funnels their results back into one stream. The only subtlety is the funnel's "we're all done" sign: you raise it once, after counting that every hand has put its tools down (wg.Wait), then close the channel.

🕳️ Knowledge Gaps

• Bounding the number of fan-out goroutines when the source is huge — semaphores (Day 074).

✅ Summary

I can articulate and implement fan-out (shared-channel workers) and fan-in (merge with a WaitGroup + closer goroutine), and I know why the single-close discipline matters.

⏭️ Next Steps / Prep for Tomorrow

• Day 073: chain stages into a pipeline and add cancellation.

---

Time spent	Difficulty	Confidence
90 min	🟦🟦⬜⬜⬜	🟦🟦🟦⬜⬜

Suggested commit: feat(examples): fan-out/fan-in merge (day 072)