Day 109 — Project: Integration Tests (testcontainers)

Month 4 · Week 4 · ⬅ Day 108 · Day 110 ➡ · Journal index

🎯 Learning Objective

Write integration tests that boot the whole service against a real, throwaway Postgres (via testcontainers) and exercise it like a client with net/http/httptest — proving the layers cooperate, not just that units pass in isolation.

📚 Topics

• net/http/httptest: NewRecorder (in-process) vs NewServer (real socket)
• testcontainers-go: spin up Postgres per test run, get a DSN, terminate after
• TestMain, build tags (//go:build integration), and short-mode skips
• Test isolation: fresh schema/transaction per test

📖 Reading / Sources

• net/http/httptest package docs
• testcontainers-go — Postgres module
• testing.Short + go test -short
• Go blog — Testable examples / table-driven tests

📝 Notes

• Two test scopes: unit tests use a fake repo (fast, no Docker); integration tests use the real repo against a container (slower, high-fidelity). Both matter → [[test-isolation]].
• httptest.NewServer(handler) binds a real loopback port and gives you srv.URL; you hit it with a normal http.Client, exercising routing + middleware + transport. httptest.NewRecorder skips the socket and calls ServeHTTP directly — fastest for handler tests → [[httptest]].
• testcontainers starts a Docker container from Go, waits for a readiness signal (e.g. log line "ready to accept connections"), and hands back a connection string. t.Cleanup/defer container.Terminate(ctx) tears it down → [[integration-tests]].
• Gate slow tests behind a build tag (//go:build integration) or if testing.Short() { t.Skip() }, so go test -short ./... stays fast in the inner loop and CI runs the full set.
• Isolation: run each test in its own transaction rolled back at the end, or recreate the schema, so tests don't see each other's rows. Shared mutable state across tests is the classic flaky-test cause.
• Use t.Cleanup instead of bare defer for resources created in helpers — it runs even when the helper returns early and composes across sub-tests.
• The container is third-party, but the driving loop — boot the app, send requests, assert on responses — is identical to the stdlib examples/month-04/httptest demo. Only the database backing it differs.

💻 Code Examples

//go:build integration

func TestUsersAPI_Integration(t *testing.T) {
    ctx := context.Background()
    pg, err := postgres.Run(ctx, "postgres:16-alpine",
        postgres.WithDatabase("app"),
        postgres.WithUsername("test"), postgres.WithPassword("test"),
        testcontainers.WithWaitStrategy(
            wait.ForLog("database system is ready to accept connections").
                WithOccurrence(2)),
    )
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { _ = pg.Terminate(ctx) }) // always torn down

    dsn, _ := pg.ConnectionString(ctx, "sslmode=disable")
    app := mustBuildApp(t, dsn) // migrate + wire repo→service→handler

    srv := httptest.NewServer(app) // hit the real stack like a client
    defer srv.Close()

    resp, _ := http.Post(srv.URL+"/users", "application/json",
        strings.NewReader(`{"email":"ada@example.com"}`))
    if resp.StatusCode != http.StatusCreated {
        t.Fatalf("status = %d, want 201", resp.StatusCode)
    }
}

Stdlib version (recorder vs server, no Docker): examples/month-04/httptest/main.go · Run: go run ./examples/month-04/httptest

🏋️ Exercises / Practice

Exercise	Status	Link
Handler tested with recorder + httptest.Server	✅	exercises/month-04/week-4/httpapi
(Concept) testcontainers Postgres integration test	✅	snippet above

🐛 Mistakes Made

• Reused one container row state across sub-tests → order-dependent failures. Switched to a per-test transaction rolled back in t.Cleanup.
• Forgot the wait strategy and got "connection refused" — Postgres logs "ready" twice (init + restart), so WithOccurrence(2) is needed.

❓ Open Questions

• One shared container for the whole package (faster) vs one per test (cleaner)? Leaning shared container + per-test transaction.

🧠 Active Recall (answer without looking)

1. Q: When would you reach for httptest.NewServer instead of httptest.NewRecorder?

A

When you need the full transport path over a real socket — a genuine `http.Client`, redirects, TLS, connection handling. The recorder is faster but calls `ServeHTTP` directly and skips the network.

1. Q: How do you keep slow container-backed tests out of the fast feedback loop?

A

Gate them behind a build tag like `//go:build integration` (run explicitly with `go test -tags=integration`) or skip them under `testing.Short()` so `go test -short` ignores them.

🪶 Feynman Reflection

Unit tests check each musician can play their part; integration tests put the band on stage with real instruments and listen to the whole song. testcontainers rents the stage (a real Postgres) just for the show and dismantles it after. httptest is the microphone — sometimes plugged straight into the mixing board (recorder), sometimes into a real PA system on a real port (server).

🕳️ Knowledge Gaps

• Parallelising integration tests (t.Parallel) safely with a shared container — schema-per-test vs transaction-per-test trade-offs.

✅ Summary

I can drive the whole service in tests with httptest, and stand up a real Postgres per run with testcontainers, gated so the fast loop stays fast.

⏭️ Next Steps / Prep for Tomorrow

• Day 110: publish the API contract as an OpenAPI document and serve interactive docs.

---

Time spent	Difficulty	Confidence
90 min	🟦🟦🟦⬜⬜	🟦🟦🟦⬜⬜

Suggested commit: test(project): httptest + testcontainers integration tests (day 109)