The Professor
Index Course index

Module 6 · Async processing and messaging

Lesson 29 — Queues and background jobs

Celery in TicketFlow: reservation expiry stops being a human cron.

Published
In this lesson
  1. Exercise 1 — The local worker
  2. Exercise 2 — The expiration with a new transport
  3. Exercise 3 — on_commit or the ghost email
  4. Exercise 4 — Task idempotency
  5. Exercise 5 — Task classification
  6. Submit

Celery on top of the logic extracted in 28. No solutions.md before submitting.

Exercise 1 — The local worker

  1. Bring up the stack: Redis (broker), worker (celery -A ticketflow worker -l info --concurrency=2) and beat (celery -A ticketflow beat). Write the dev docker-compose.override.yml if you use compose (40 formalizes it).
  2. Create the saludo(name) task returning "hola, {name}" and launch it with saludo.delay("TicketFlow") from the shell. Verify the result with AsyncResult.
  3. Kill the worker WHILE it processes a slow task (sleep 30) and bring it back up: with acks_late, did the task re-execute? Paste the log evidence.

Exercise 2 — The expiration with a new transport

  1. Turn expirar_reservas (28) into a task with autoretry_for=(OperationalError,), retry_backoff=True, retry_jitter=True, max_retries=5.
  2. Schedule beat every minute. Verify across 3 consecutive minutes that only ONE run touches reservations (the logs must show refs [] in the two minutes without expirations — no double-marking).
  3. Test with eager mode: override_settings(CELERY_TASK_ALWAYS_EAGER=True) and FakeClock — the task runs in the test's own process and the assert counts the expired refs.

Exercise 3 — on_commit or the ghost email

  1. Write the test that DEMONSTRATES the bug: enqueue enviar_email_confirmacion.delay() inside a transaction that then fails (forced duplicate seat); in eager mode, the fake email went out anyway. Paste it red.
  2. Switch to transaction.on_commit(...) and verify: rollback → zero emails; commit → one email.
  3. Apply on_commit to ALL the project's .delay() calls: git grep -n "\.delay(" | grep -v on_commit must return empty.

Exercise 4 — Task idempotency

  1. Add Redis dedup to enviar_email_confirmacion (email:{ref}:confirmed, 7-day TTL, cache.add). Test: launch the task 3 times in a row (simulating the broker's at-least-once) → ONE email in the SMTP fake's outbox.
  2. The charge task: pass intent.id as the Idempotency-Key to the fake gateway (14). Test: the fake gateway receives 2 calls with the same intent → 1 charge registered, the same response both times.

Exercise 5 — Task classification

  1. Build TicketFlow's task table: confirmation email, 24h-before reminder email, outgoing webhooks (17), expiration, monthly commission settlement, availability. Mark each: critical (infinite retry + alert) vs disposable (3 retries then DLQ), tolerated latency, and idempotency mechanism.
  2. Implement "infinite retry with alert" for the critical ones: max_retries=None + a task header that after 10 attempts publishes to the task_retry_total{task} metric (46 formalizes the alert).

Submit

Paste the acks_late evidence, the on_commit red→green test, the dedup test and the classification table. Next: Lesson 30 — Message brokers.

^ top