The Professor
Index Course index

Module 8 · Performance and caching

Lesson 39 — Pools, pagination and compression

Connections, listings and lighter responses without cutting content.

Published
In this lesson
  1. Exercise 1 — The pool's arithmetic
  2. Exercise 2 — The cursor
  3. Exercise 3 — The client's limits
  4. Exercise 4 — Compression
  5. Exercise 5 — The edges report
  6. Submit

The performance of the edges. No solutions.md before submitting.

Exercise 1 — The pool's arithmetic

  1. Compute your connection budget: gunicorn (workers × threads × replicas), Celery workers (concurrency), beat, commands. Write it as a table with the total and compare against SHOW max_connections.
  2. Configure CONN_MAX_AGE + psycopg 3's pool (min 2 / max 10) and measure with pg_stat_activity before and after the k6 plateau (36): how many connections idle? How many active?
  3. The controlled disaster: in a test environment, lower max_connections to 20 and launch the 200-VU plateau. Which error appears and where (app, worker, both)? Document the EXACT symptom for 47's runbook.

Exercise 2 — The cursor

  1. Implement cursor pagination on the events listing (encode/decode of the uuid, limit with a ceiling of 100) and the contract test: 3 consecutive pages with no overlap, next_cursor: null at the end, stable order.
  2. The honest measurement: with a 2M-row seed, measure (EXPLAIN ANALYZE, 09) page 1 vs page 1000 with offset and with cursor: paste both plans. How much does deep offset cost?
  3. The moving window: insert 5 events between reading page 2 and requesting page 3 with offset: does anything repeat or get skipped? Repeat with the cursor: what happens? Paste both evidences.
  4. The removed total: rewrite the front contract (35) without total and add the test that the response NO LONGER carries it. Which endpoint DOES keep a total and why (the admin with filters? cached, 38?)?

Exercise 3 — The client's limits

  1. Add the limit ceiling (min(limit, 100)) and the test: ?limit=100000 returns 100 items WITHOUT an error (or a 400? decide and justify: silent or explicit, 13).
  2. The bandwidth attack: with k6 (36), launch 50 VUs continuously requesting limit=100 and compare p95/DB bandwidth against limit=20. Which ceiling protects more? Write the decision in the API docs.

Exercise 4 — Compression

  1. Measure the 400-event listing: bytes uncompressed, with gzip 4, with gzip 9, and the CPU time of each level (37's cProfile or timeit). Where is the trade-off zone?
  2. Configure gzip in nginx (or gunicorn if there is no proxy): gzip_comp_level 4, gzip_min_length 1024, gzip_types application/json. Test with curl --compressed: the Content-Encoding header and the real bytes.
  3. The cache and Vary: with 38's ETag active, serve one compressed response and one uncompressed for the same resource: does the cache confuse them without Vary: Accept-Encoding? Demonstrate it with the 304 test (the client without gzip support must receive it UNcompressed).

Exercise 5 — The edges report

  1. Write docs/perf/2027-09-bordes.md: the pool's arithmetic, the cursor's plan (before/after of page 1000), the limit ceiling, and the compression decision with numbers.
  2. The project's rule for EVERY new listing: cursor by default, limit ceiling, no COUNT, gzip at the proxy. Where do you make that rule live (a mixin/base view? a parametrized contract test over all listings?)? Implement it.

Submit

Paste the connections table, the EXPLAIN plans of page 1000, the moving window's evidence and the edges report. Next: Lesson 40 — Docker (module 9).

^ top