Day 166 — Capstone: Integration Tests

Month 6 · Week 4 · ⬅ Day 165 · Day 167 ➡ · Journal index

🎯 Learning Objective

Prove linkr works end to end against real infrastructure: spin up Postgres and Redis with testcontainers, exercise the REST/gRPC APIs with httptest/a real client, and gate it all behind a build tag.

📚 Topics

• Test pyramid: unit (fast, fakes) → integration (real DB/cache) → e2e
• testcontainers-go · net/http/httptest · build tags (//go:build integration) · t.Cleanup

📖 Reading / Sources

• Go blog — Subtests and Sub-benchmarks
• testcontainers-go
• net/http/httptest docs
• Go testing — build constraints / tags

📝 Notes

• Test pyramid: most tests are unit tests over the in-memory adapter (the linkstore/lrucache exercises) — fast, no Docker. Fewer integration tests hit a real Postgres/Redis. A couple of e2e tests drive the running binary → [[test-pyramid]].
• Integration tests are real, not mocked. testcontainers-go starts an ephemeral Postgres container, runs migrations against it, and the test exercises the actual pgRepo. This catches SQL typos and driver-mapping bugs that fakes can't → [[integration-testing]].
• Build tags isolate slow tests. //go:build integration at the top means go test ./... stays fast; CI runs them with go test -tags=integration ./.... The blank line after the tag line is mandatory.
• httptest.Server mounts the real router on a loopback port and hands you a configured *http.Client — full HTTP semantics, no real network → [[httptest]].
• t.Cleanup registers teardown that runs even if the test fails (LIFO). Use it to Terminate() containers and close pools — more robust than defer across helpers → [[t-cleanup]].
• Table-driven across the API: one slice of {name, method, body, wantStatus} cases looped with t.Run → readable failures and -run filtering.
• Golden path + edges: create→resolve→hit-count, plus duplicate code (409/AlreadyExists), missing code (404/NotFound), and bad body (400).
• Run the race detector on integration tests too (-race) — concurrency bugs love real I/O timing.

💻 Code Examples

Build-tagged integration test driving the real router (the runnable restapi example uses this exact httptest shape):

//go:build integration

package rest_test

func TestShortenResolve(t *testing.T) {
    pool := startPostgres(t)              // testcontainers; t.Cleanup terminates it
    srv := httptest.NewServer(newRouter(link.NewService(postgres.New(pool))))
    t.Cleanup(srv.Close)
    client := srv.Client()

    cases := []struct {
        name, body string
        want       int
    }{
        {"create", `{"code":"go","url":"https://go.dev"}`, http.StatusCreated},
        {"duplicate", `{"code":"go","url":"https://go.dev"}`, http.StatusConflict},
        {"bad-body", `{"oops":true}`, http.StatusBadRequest},
    }
    for _, tc := range cases {
        t.Run(tc.name, func(t *testing.T) {
            resp, err := client.Post(srv.URL+"/links", "application/json", strings.NewReader(tc.body))
            if err != nil {
                t.Fatal(err)
            }
            defer resp.Body.Close()
            if resp.StatusCode != tc.want {
                t.Errorf("status = %d, want %d", resp.StatusCode, tc.want)
            }
        })
    }
}

The unit layer of this pyramid is fully runnable: exercises/month-06/week-4/linkstore and the restapi example's httptest pattern → go test ./exercises/month-06/week-4/...

testcontainers Postgres bootstrap (third-party — snippet only):

func startPostgres(t *testing.T) *pgxpool.Pool {
    ctx := context.Background()
    c, err := postgres.Run(ctx, "postgres:16-alpine",
        postgres.WithDatabase("linkr"),
        testcontainers.WithWaitStrategy(wait.ForListeningPort("5432/tcp")),
    )
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { _ = c.Terminate(ctx) }) // runs even on failure
    dsn, _ := c.ConnectionString(ctx)
    pool, _ := pgxpool.New(ctx, dsn)
    migrate(t, pool)
    return pool
}

🏋️ Exercises / Practice

Exercise	Status	Link
linkstore — unit tests incl. -race concurrent counter	✅	exercises/month-06/week-4/linkstore
lrucache — TTL via injectable clock (deterministic)	✅	exercises/month-06/week-4/lrucache

🐛 Mistakes Made

• Captured the loop variable in a t.Run goroutine (pre-Go-1.22 habit) — on older toolchains all subtests saw the last case. Go 1.22 fixed loop-var semantics, but I still copy tc := tc defensively in shared snippets.
• Used defer container.Terminate() inside a helper → it ran when the helper returned, killing the DB before the test used it. t.Cleanup (tied to the test, not the function) is the fix.
• Forgot the blank line after //go:build integration → the line was treated as a plain comment and the tag did nothing.

❓ Open Questions

• Share one Postgres container across a package's tests (faster) vs one per test (isolated)? TestMain with a package-level container is the usual compromise.

🧠 Active Recall (answer without looking)

1. Q: Why prefer t.Cleanup over defer for tearing down a test container?
   A

t.Cleanup is registered with the *testing.T, so it runs when the test finishes (LIFO), even across helper functions and even if the test fails or calls t.Fatal. A defer inside a helper runs when the helper returns, which can destroy a resource the test still needs. 2. Q: What does the //go:build integration tag buy you, and what's the syntax gotcha?

A

It excludes slow, infra-dependent tests from the default go test ./..., so the fast unit suite stays fast; CI opts in with -tags=integration. Gotcha: the //go:build line must be at the very top of the file, followed by a blank line before the package clause, or it's ignored.

🪶 Feynman Reflection

Testing is buying confidence at the cheapest price. Unit tests over fakes are pennies — run them constantly. Integration tests over real Postgres and Redis cost seconds and a Docker daemon, so I run fewer of them, but they catch the lies my fakes tell (a wrong column name, an error I forgot to map). The build tag is the toggle that keeps the cheap tests cheap while the expensive ones still guard the real wiring.

🕳️ Knowledge Gaps

• Contract testing between the gRPC client and server as they version independently — not needed for a solo capstone, but real in production.

✅ Summary

linkr has a real test pyramid: fast unit tests over the in-memory adapter, build-tagged integration tests against ephemeral Postgres/Redis via testcontainers, and httptest-driven API tests — all table-driven and race-checked.

⏭️ Next Steps / Prep for Tomorrow

• Day 167: write the README, record a demo, and capture the key decisions as ADRs.

---

Time spent	Difficulty	Confidence
90 min	🟦🟦🟦⬜⬜	🟦🟦🟦⬜⬜

Suggested commit: test(linkr): integration tests with testcontainers + httptest behind a build tag (day 166)