Lesson 34 — Integration testing

Stack: Django/DRF · Project: TicketFlow Status: Published — real Postgres, no SQLite pretending Prerequisite: Lesson 33 — Unit testing done well

Objectives

1. Set up real Postgres/Redis for tests with containers and test what unit tests can't: transactions, locks, constraints and the outbox.
2. Isolate tests from each other (transactions, truncation) with no step ordering and no shared state.
3. Know when integration and when unit: the practical boundary that keeps the suite fast AND honest.

1. Why real Postgres (and not SQLite)

manage.py test's SQLite lies about everything this part of the course built: select_for_update is a no-op on SQLite (the heart of 10!); partial indexes (condition=Q(...), 25) don't exist; the JSONField types are TEXT with parsed JSON (no @> operators nor GIN, 12); the UniqueConstraint(condition=...) constraints are ignored. A suite green on SQLite and red on Postgres is staging's surprise (47). The project's rule: integration = the same engine, same major version (Postgres 16 in prod, 16 in test). The minor version may vary; the major, never.

# docker-compose.test.yml
services:
  db:
    image: postgres:16-alpine
    environment: {POSTGRES_USER: tf, POSTGRES_PASSWORD: tf, POSTGRES_DB: tf_test}
    tmpfs: [/var/lib/postgresql/data]     # in RAM: tests wipe and create without disk I/O
    ports: ["5433:5432"]
  redis:
    image: redis:7-alpine
    ports: ["6380:6379"]

tmpfs for the test DB: the disk is not the suite's bottleneck. The shifted port (5433) avoids stepping on your dev Postgres.

2. What gets tested here (and what doesn't)

Integration tests the seams with real infrastructure: 10's transaction with lock (two concurrent transactions, one waits), the outbox's partial index and constraint (25), the advisory lock (31), JSONB with GIN (12), select_related/prefetch on real queries (09), the resumable saga (32). It does NOT test: pure logic (that's unit, 33), nor the full HTTP flow (E2E, 35), nor performance (36). This layer's signature test:

@pytest.mark.django_db(transaction=True)     # transaction=True: real TXs (not TestCase's wrapper)
def test_dos_reservas_concurrentes_un_ganador(evento, asientos, dos_conexiones):
    with ThreadPoolExecutor(max_workers=2) as pool:
        f1 = pool.submit(reservar, u1, evento, ["A1"], clock=clock_fijo)
        f2 = pool.submit(reservar, u2, evento, ["A1"], clock=clock_fijo)
    resultados = [f.result() for f in (f1, f2)]
    exitosos = [r for r in resultados if not isinstance(r, Exception)]
    assert len(exitosos) == 1                 # the lock decides; the other: SeatUnavailable

This test is IMPOSSIBLE on SQLite and on TestCase (which wraps everything in a TX): it needs transaction=True and real connections. It is the test validating TicketFlow's entire inventory guarantee: one seat, one winner.

3. Isolation: every test its own world

The isolation hierarchy in pytest-django: @pytest.mark.django_db (a transaction per test, rollback at the end — fast, covers 90%), transaction=True (real TXs inside, truncation at the end — for the above), and django_db_blocker for the rare ones that create the DB. Rules: no test writes outside its fixture; migrations run ONCE per session (--create-db controls it); and test files/Redis carry a prefix (test:cache:{...} — the tests' Redis uses DB 15, NEVER the dev one).

# integration conftest.py
@pytest.fixture(scope="session")
def db_url():
    return os.environ["DATABASE_URL"]         # from the test compose: 27 applies here too

@pytest.fixture(autouse=True)
def _redis_test_db(settings):
    settings.CACHES["default"]["LOCATION"] = "redis://localhost:6380/15"

Execution order does NOT exist as a concept: every test raises its world (baker/fixtures) and tears it down with rollback. If a test "only works when it runs after another", that is a test bug, not an optimization.

4. The practical unit/integration boundary

The assignment criterion: is the thing under test the infrastructure interaction itself? (locks, constraints, SQL: integration). Or is it logic with infrastructure as a side detail? (unit with fakes). The reservar() service yields TWO tests: the unit one (fakes: rules and state machine, 3 ms) and the integration one (real Postgres: lock and outbox, 150 ms). That is not duplication: they test DIFFERENT contracts — "the rule works" and "the rule survives real concurrency". The whole suite: units <10 s on every save; integration <2 min on every push; E2E <10 min on PR (35).

The classic assignment mistake: EVERYTHING to integration "because it's more real" (a 30-min suite nobody runs, the feedback loop dies) or EVERYTHING to unit with mocks (green in test, red in prod: nothing tested the locks). The project's balance: 70/25/5 (unit/integration/e2e) with integrations concentrated on the guarantees that sell the product (concurrency, money, outbox).

5. TicketFlow's integration list

This table is the layer's contract: each row is a guarantee the business buys (09-10-25-31-32 built them) and the integration test is its policy. When something breaks in production (47), first reflex: which row of this table failed? And add the test that would have caught it.

Self-assessment

1. Which four things does SQLite lie about regarding what lessons 10-25 built, and what would the staging consequence be?
2. @pytest.mark.django_db vs transaction=True: what does each wrap, and when is the second mandatory?
3. Why is the "one seat, one winner" test not a duplication of reservar()'s unit test?
4. The unit/integration assignment criterion in one sentence, and the mistake of both extremes (all-integration / all-mocked).
5. How does §5's guarantees table relate to a postmortem (47)?

Continue with the exercises. The solutions only after trying it yourself.