Skip to main content

Measurement discipline

advanced16 min readLesson 130 of 169

Baseline, attribute, change one thing, verify ā€” with countable metrics.

Measure first: the discipline

Every performance claim in this module is graded the same way professionals verify: count the work, don't guess at the clock. Wall-clock on a shared machine is noisy; call counts, query counts, and allocation counts are facts.

The workflow:

  1. Establish a baseline ā€” the current numbers (latency, throughput, calls to the expensive thing).
  2. Attribute ā€” find where the work actually goes (next lesson: cProfile).
  3. Hypothesize ā€” "the O(nĀ²) membership test dominates", "we re-fetch the same rows 50 times".
  4. Change one thing.
  5. Verify ā€” re-measure the SAME metric. Keep the win, or revert.

Premature optimization has a precise meaning here: changing code before step 1 exists. timeit is the tool for micro-benchmarks (it disables GC and repeats enough times to beat noise); time.perf_counter() is the wall-clock tool for in-process timing; but for graded correctness of an optimization, count operations ā€” a memoization win is calls: 13529 ā†’ 39, not a fuzzy x2.