Lesson 03 — Concurrency and asynchrony

Stack: Django + DRF · Project: TicketFlow Status: Published — taught after Lesson 02 is corrected Prerequisite: Lesson 02 — Networking basics

Objectives

By the end of this lesson you will be able to:

1. Explain the difference between process, thread and event loop — and which model your stack uses at each layer.
2. Demonstrate a race condition with code (the same one that can sell a seat twice).
3. Reason about Python's GIL: what it blocks and what it doesn't, and why Django still scales with threads.
4. Decide when an async view (ASGI) helps and when it is noise.
5. Recognize a deadlock before creating one, with lock-acquisition rules.

1. Concurrency is not parallelism (start with the vocabulary)

• Concurrency: the program manages several tasks at once (interleaving them). A coffee machine with two queues and one barista.
• Parallelism: the program executes several tasks at the same physical instant. Two baristas.

A web server always needs concurrency (while your view waits for the DB, another request must be servable) and parallelism when there is real CPU work (parsing a large CSV).

The three models that exist:

Key: the models combine. Gunicorn: 4 processes × 2 threads. Uvicorn: 1 process with an event loop + workers for the blocking parts. Celery: processes with prefork concurrency or threads with --pool=threads.

2. The GIL: the rule of the game in CPython

The GIL (Global Interpreter Lock) allows only one thread to execute Python bytecode at a time. Consequences:

• Two Python threads do not parallelize CPU work (even with 8 cores).
• I/O operations (network, disk, DB) release the GIL while waiting: that is why threads do work for web servers, whose time goes into waiting.
• C extensions (psycopg2, lxml, numpy) release the GIL in their heavy parts.

So, how does Django scale if the GIL forbids per-thread parallelism? With the right combination at each layer:

Gunicorn --workers 4        ← real parallelism (4 processes, one GIL each)
         threads/async      ← cheap concurrency inside each process
         DB and Redis out   ← shared state lives in another process (and that's why you must synchronize!)

Interview key: "the GIL prevents CPU parallelism between threads, not I/O concurrency". And for CPU: processes or finer workers. (CPython 3.13 experiments with the free-threaded build; in 2026 it is still the exception, not the production standard.)

3. The race condition: the bug you never see in a demo

A race happens when the result depends on the interleaving of two flows. The classic check-then-act pattern:

def reserve(seat_id, user):
    seat = Seat.objects.get(pk=seat_id)
    if seat.is_reserved:                      # CHECK
        raise Conflict("taken")
    ReservationItem.objects.create(           # ACT
        seat=seat, reservation=..., price_at_purchase=...)

Between the CHECK and the ACT there is a window of milliseconds. Two concurrent requests can both read is_reserved=False and both create their item. On your laptop with one user it never happens; with 500 users on concert day, it happens hourly.

Controlled demonstration (you will do it in the exercises):

import threading

counter = 0

def increment():
    global counter
    for _ in range(100_000):
        counter += 1          # read, add, write: three steps, not one

t1 = threading.Thread(target=increment)
t2 = threading.Thread(target=increment)
t1.start(); t2.start(); t1.join(); t2.join()
print(counter)                # 200_000? Run it several times...

counter += 1 is not atomic: it is three operations (read, add, write) and the scheduler can interrupt between them. With two threads, the result comes in below 200,000 and differs on every run. That is a race: neither deterministic nor reliably reproducible.

The defenses, from cheapest to strongest:

1. No shared mutable state (isolate: one reservation is processed by a single worker).
2. In-memory locks (threading.Lock) — only works inside one process: with 4 Gunicorn workers it does not protect you. Classic mistake.
3. DB atomicity: atomic updates, unique constraints, transactions with locking (Lesson 10).
4. Queues: serialize access (one Celery job per reservation; the producer-consumer does the rest).

Key: TicketFlow's invariant I1 (one seat, one active reservation) is already guaranteed by partial constraints (Lesson 00b, ADR-0004). This lesson teaches you to recognize all the windows where code can interleave, even though the DB covers the most expensive ones.

4. Deadlocks: the deadly embrace

A deadlock needs four simultaneous conditions (Coffman): mutual exclusion, hold and wait, no preemption and circular wait. The typical DB case:

Transaction A: UPDATE seats WHERE id=1 → then UPDATE seats WHERE id=2
Transaction B: UPDATE seats WHERE id=2 → then UPDATE seats WHERE id=1

Each holds its first lock and waits for the one the other holds: nobody advances until the DB kills one (PostgreSQL raises Deadlock detected and rolls back one transaction; your code must retry it).

Rules of the craft to avoid creating them:

1. Global acquisition order: all resources (rows, locks) are taken in the same order (for example, always by ascending id). It breaks the circular wait.
2. Short transactions: inside a transaction, no network or slow calls (never call the payment gateway inside the reservation transaction — the serious version arrives in Lesson 54).
3. One lock if that suffices: group the acquisition (coarse-grained lock when contention is low).
4. Timeout + retry: assume it can happen and respond with idempotent retries (Lesson 14).

5. Async in Django: what it solves and what it doesn't

Django 5 supports async def views served by Uvicorn/Daphne (ASGI). The event loop serves thousands of connections with a single thread if the waits are asynchronous (await): a view that waits for the DB with sync_to_async or async drivers does not block the others.

It pays off when: your view does parallelizable I/O (calling two external APIs at once with asyncio.gather, webhooks, payment aggregators) or you have tons of slow connections (SSE/WebSockets — Lesson 16).

It does not pay off when: the view is CPU (the event loop is a single thread: it blocks everyone), or you use synchronous libraries without wrapping them (a requests.get in an async view blocks the whole event loop: worse than sync).

sync (Gunicorn):    1 request → 1 busy worker/thread
async (Uvicorn):    1 request → free to "await" → the loop serves others

Practical rule for TicketFlow: the project lives happily with Gunicorn sync + Celery. Async enters when availability streaming arrives (Lesson 16) or parallel gateway calls. Choose by measurable need, not by fashion.

6. TicketFlow's concurrency map (closing the circle)

Each layer synchronizes its own thing. The architectural mistake is mixing them: a Python lock does not protect against another process; a DB constraint does not speed up a queue; a Redis rate limit does not replace a transaction.

Self-assessment (answer me in the chat)

1. With 4 Gunicorn workers, does a threading.Lock protect you against two concurrent requests incrementing the same counter? Why not, and what would you use?
2. Why does the check-then-act pattern fail even when the CHECK is the first statement? Which part of the system guarantees the missing atomicity?
3. An async view calls requests.get(...) (synchronous). What happens to the event loop and to the other requests? How is it fixed?
4. Design the correct acquisition order for an operation that touches seat row 1051 and user row 7. Which rule do you apply?
5. When would you say "TicketFlow is due for ASGI"? Name the project's concrete trigger.

Continue with the exercises. The solutions only after trying it yourself.