Lesson 35 — E2E and contract testing

Exercise 1 — The catalog

1. TicketFlow's catalog (8 stories):

1. The reassigned ones: "email without @ → 400" (serializer unit), "staff can't reserve across orgs" (permissions integration), "the refresh token rotates" (18's integration), "the partial index works" (integration, 34), "the rate limit trips" (23's integration). None needs the whole system: pushing them to E2E is paying 20 s for what costs 3 ms.

1. pytest -m e2e --collect-only → 8 tests × ~20 s = 160 s ≈ 3 min: inside the PR budget. The catalog is a living list: a new critical story enters, and something LEAVES (the budget is hard).

Exercise 2 — The stories

1. The tragic one:

def test_declinacion_compensa(client_api, pasarela_declina, mailbox):
    ref = reservar_a1(client_api)
    r = pagar(client_api, ref)
    assert r.status_code == 402 and r.json()["type"].endswith("payment-declined")
    drenar_cola()
    assert estado_de(ref) == "CANCELLED"
    assert asiento_libre("A1")
    assert len(mailbox) == 1          # failure email
    assert not webhook_saliente_enviado()   # the third party never sees the failed sale

The webhook-NOT-sent assert is what turns this story into a policy: the saga (32) decides who notifies; the test pins it.

1. The dispute:

def test_un_asiento_un_ganador(client_a, client_b, evento):
    with ThreadPoolExecutor(2) as pool:
        f1 = pool.submit(reservar, client_a, evento, ["A1"])
        f2 = pool.submit(reservar, client_b, evento, ["A1"])
    refs = [f.result() for f in (f1, f2)]      # one 201, one 409
    pagar_solo_el_ganador(...)
    drenar_cola()
    assert suma_de_ingresos == precio_evento    # and not 2×

With the two REAL HTTP clients (the pool's concurrency — 34 tested it at service level; here, at story level): the entire guarantee "we never sell the same seat twice" is under contract.

Exercise 3 — The flaky

1. The cure:

# before: time.sleep(1); assert saga.estado == "CONFIRMED"     # red on slow CI
# after:
wait_for(lambda: saga.refresh_from_db() or saga.estado == "CONFIRMED",
         timeout=5, msg=f"saga {saga.saga_id} did not confirm")

--count=10 stable green: the sleep failed when the worker took >1 s (shared CI); the wait_for waits for WHAT'S NEEDED with a deadline — 10/10 green.

1. The dump that diagnoses:

AssertionError: saga 8a2f no confirmó: dump={
  "saga": {"estado": "PENDING", "intentos": 1},
  "cola_pendiente": ["cobrar(int_9f2c)"],
  "outbox_ultimos": ["ReservationCreated", "PaymentPending"]
}

The dump states the diagnosis: the cobrar task is still PENDING in the eager queue (not drained — on_commit didn't fire) or it failed. Without opening the IDE: the test is its own postmortem (47).

1. The CONTRIBUTING rule: quarantine max 2, mandatory issue with the cause, 2-week expiry (either it gets cured or deleted — an eternal flaky is noise with a flag). CI's automatic re-run marks the report ("passed after retry") so the statistics don't lie.

Exercise 4 — The contract

1. The contract break: red in test_el_front_puede[POST /api/v1/reservations] in <5 s (it is a shape unit test, fast). The dual compatibility (14):

"total": "int|obj{amount,currency}"    # the consumer accepts both during the transition sprint

and on the provider, the dual response with an X-API-Version header or negotiated content — at the sprint's end, the front's test demands the new format (the contract has versions like the API).

1. The error contract:

"errores": {"4xx": {"type": "uri", "title": "str", "status": "==HTTP", "detail": "str?"},
            "409-seat": {"type": "…/seat-unavailable", "extra": ["conflicting_seats"]}}

The dispute's 409 validates it: the front programs against type+conflicting_seats (26) without parsing prose — the error contract is the one that prevents the most integration fights.

Exercise 5 — The engine room

1. The reset numbers: truncating 12 tables + flushing Redis DB 15 ≈ 400 ms; compose down/up ≈ 25-40 s. With 8 tests: reset-in-place saves ~4 min per run — and in CI (running 30×/day) that is 2 hours of daily compute. The compose's healthcheck (pg_isready, redis-cli ping) avoids the app container's false start.

1. The auth fixture:

@pytest.fixture
def comprador_autenticado(db):
    user = baker.make(User, email="buyer@tf.test")
    token = str(RefreshToken.for_user(user).access_token)
    return ApiClient(HTTP_AUTHORIZATION=f"Bearer {token}")

1. The metric: 8 tests × 20 runs = 160 executions, 2 no-cause failures (exercise 3's sleep before the cure) → flaky_rate 1.25% → after the cure: 0.6%. Under 1%: the suite is credible; the day it exceeds 2%, the team stops trusting and the pyramid sinks — the metric is the engine room's early alarm.

Professor's summary

• E2E = the full story over HTTP with real infrastructure and doubles ONLY at the business boundary; the catalog is ≤8 stories, hard limit.
• The flaky gets cured with active wait + a diagnostic dump; quarantine with expiry avoids the re-run-everything culture.
• The CONSUMER defines the contract: breaks show up in seconds in the pipeline, not on the buyer's phone.