Module 12 · Enterprise level (capstone)

Lesson 57 — Capstone project

Defense of the complete backend and a simulated postmortem — the final exam of the craft.

Published
In this lesson
  1. Exercise 1 — The material
  2. Exercise 2 — The defense
  3. Exercise 3 — The drill
  4. Exercise 4 — The pitch
  5. Exercise 5 — The close
  6. Professor's summary

Exercise 1 — The material

  1. The honest inventory: tests green (the whole suite <15 min), pipeline (41's gate), ADRs (12 indexed), runbook (47), the compliance map 9/10 (the access review in issue). The gap: 54's game day only ran 2 of the 5 incidents — the fix: exercise 3's drill closes the inventory.
  2. The randomly drawn guarantees: 3 (money not doubled), 7 (the peak's degradation), 10 (GDPR): 7 was the weakest (the incomplete game day) — the policy exists (54), the full evidence arrived with the drill. The exam's lesson: the guarantee without a demo is a claim.

Exercise 2 — The defense

  1. The REQUEST's script (excerpt): "The POST /pay enters through the contract guard (35)... the payments service validates the intent (14)... the saga decides: the charge OUTSIDE the transaction (32: 39's pool breathes)... the result arrives by signed webhook (17) or reconciliation (32)... the CONFIRMED transition is the state machine (00b)... the outbox publishes the event (25) after the commit... the email leaves queued (29) with dedup... the client sees the problem+json if something fails (26) with the trace_id following the whole thread (45). Every leg has its test: the names say it."
  2. The FAILURE (excerpt): "When the gateway takes 5 s: 46's SLO pages the burn... 54's timeout cuts at 8 s... the breaker opens and the pool breathes... the user sees 503 with Retry-After and their seat held (32)... when it returns, the retry drains and the saga converges... and 47's postmortem closes with actions". The defense by what hurts.
  3. The weapon answers: K8s (43: the ADR with the PCI/volume trigger; "the modular boundary is the pre-payment"), Kafka (30: "150k events/day: Redis's stream covers it; the trigger is the ×10 volume — the ADR dates it"), partial DDD (52: "the VOs and the Reservation aggregate pure; CRUD in the ORM; 3 weeks of full DDD are not paid by any line of 51's map — the trigger: the second service").

Exercise 3 — The drill

  1. The 5 chronologies (summary):
(a) Latency 5s:      SLO burn pages (T+2m) → timeout doesn't cut (5s<8s: functional-slow) → breaker doesn't open (no failures) → the cure: the SLO alert is the ONLY net for the functional-slow → action: the k6 mix adds the 10% slow
(b) Poller dead:     46's heartbeat pages (T+5m: not running) → outbox_pending climbs → restart → drain → convergence. The INVISIBLE incident without the heartbeat (zero errors)
(c) Deploy's N+1:    listing p95 climbs after deploy (46's annotation) → py-spy (37) → the N+1 → rollback → regression test (33)
(d) Redis full:      key evictions → the cold cache (browse p95 climbs) → the lock: 38's test → maxmemory-policy + the memory alert → the proven FLUSHALL holds
(e) Duplicated charge: the ledger (08) shows ONE charge + the intent's reconciliation (14): the "double" was the FRONT's timeout retrying the POST (the user's browser): the answer to the customer with evidence; the action: the front receives the idempotent 409 (14) — the bug was the client's, the fix the contract's
  1. The verdict on (e): the ledger says ONE charge (intent_id unique, the gateway confirms it); the "double" was the frontend's RETRY at the timeout (the user saw 2 spinners, 1 charge). The answer to the customer (51): "we checked the gateway's and the bank's records: you were charged once; the second charge you see is the same operation shown twice by your browser — here is the receipt and the bank's identifier". The lesson: the money's complaint gets investigated with the ledger, not with faith — and the contract's fix (the idempotent 409) prevents the next call.
  2. The worst chronology (the first attempt's (b)): without the heartbeat's alert, the dead poller got discovered BY THE USER'S TICKET (T+40m instead of T+5m). The fix: the heartbeat's alert (46) + the stuck-poller runbook — the difference between 5 min and 40 min IS one alert.

Exercise 4 — The pitch

  1. The pitch (excerpt): "We sell the digital box office: the organizer publishes, we sell and hand them their money for a commission. What they bought: the guarantee that a seat is never sold twice (even when 5,000 buy at once), that every payment gets confirmed or refunded — never in limbo —, and that if something breaks we know what happened before they call. The decisions that bought the most value: the event queue that never loses confirmations, the presale shifts that hold the big Friday, and the modular boundaries that let us grow without rewriting."
  2. The answers: "it costs ~120 €/month with the peak paid per use (42)"; "if it goes down: an alert sounds in 2 min, the user sees a clear message with their seat held, and in 15 min it is diagnosed (46/47)"; "it scales to 12k reservations/min measured today (36), the growth plan is written with triggers (55)".

Exercise 5 — The close

  1. The letter (excerpt): "I'd start with ADR-0001 on day 1 (48); I'd make availability read-only from the start (38's cache cost me an incident); and I'd extract the logic from the mechanism from the first line (28: the expiration job was born as a human cron and I paid for it 29 lessons later)".
  2. The next project's startup: phase 1 (2 d): the pure domain (52) + the business's VOs + the unit suite; phase 2 (3 d): the modular boundaries with the boundary test (53) + the outbox (25); phase 3 (2 d): 41's pipeline + ADR-0001. The rest of the course: the toolbox rung by rung.
  3. The final checklist (10 items): fast suite on every save · refactor with a net · ADR before the decision · measure before optimizing · the blameless postmortem · the pitch without jargon · the quarterly game day · the trigger before fashion · the business in the backlog · the method over memory.

The course ends here: 58 lessons, 13 modules, a complete system with 10 demonstrable guarantees. The craft continues: the method stays installed.


Professor's summary

  • The defense walks the REQUEST, the FAILURE and the 3 argued ADRs: every guarantee with its demo — the guarantee without a demo is a claim.
  • The simulated postmortem trains the 5 incidents with their weapons: the SLO for the slow, the heartbeat for the silent, the ledger for the money.
  • The craft is the method: "I don't know, but here's how I'd find out" is the senior answer. The course's index is the lifelong map.