Software Design Reference Master Designer · System Trade-Offs

Lesson 0034 · Master Designer · Module 2

Performance vs Maintainability

The fast version is usually the ugly version. The Master's move: keep the system clean, measure, and hide the few hot paths behind deep modules — so the ugliness has a small address.

Reading ≈ 8 minPractice ≈ 25 min

Mission tie-in: the last System Trade-Off — where premature optimization and cargo-cult caching both get their verdicts.

Knowledge: clean until measured

The principle behind the principle

The two forces are machine cost — latency, throughput, the monthly compute bill — and human cost — the minutes each future reader spends understanding this before they may safely change it. The unit of decision is the hot path: a specific code path with a measured share of the time, not a module and never a codebase.

The mechanism is an asymmetry in how the two costs are distributed. Execution time is wildly uneven: a small fraction of the code accounts for nearly all of it, and Amdahl's law puts a ceiling on what optimising the rest can ever return — make a component that is 5% of runtime infinitely fast and you have bought 5%, forever, no matter how clever the code is. Comprehension cost is distributed evenly: every reader pays for every line they must read to make a change, hot or cold. Optimise cold code and you buy roughly nothing at full price. That ratio, not taste, is why "measure first" is true.

It stops being true where the distribution flattens. In a numeric kernel, a game loop, a codec or a hardware-constrained device, the hot path is the program: there is no cold majority to keep clean, and the data layout has to be chosen first because it cannot be hidden behind an interface later. It also stops being true when speed is a correctness requirement — a real-time deadline, a payment authorisation window — because then performance is an architecture driver to design for, not an optimisation to earn.

The sequence that keeps both forces happy:

  1. Build clean. Correct, clear, well-bounded code (everything Phase I taught). Performance is not yet a requirement — it's a guess.
  2. Measure. A profiler, a load test, a real metric — not "this feels slow". The requirement is a number: "the checkout must return in 300ms p95".
  3. Find the hot path. Usually 2–3% of the code does 95% of the work. Only those places trade maintainability for speed.
  4. Hide the trade behind a deep module. The fast implementation lives behind the same clean interface — caches, indexes, and cleverness contained where no one else must read them.
# the interface stays clean:
class OrderRepository(Protocol):
    def get(self, order_id: str) -> Order: ...

# the hot path gets its speed, hidden:
class CachedOrderRepository:
    def __init__(self, inner: OrderRepository, cache: Cache) -> None:
        self._inner = inner
        self._cache = cache

    def get(self, order_id: str) -> Order:
        cached = self._cache.get(order_id)
        if cached is not None:
            return cached
        order = self._inner.get(order_id)
        self._cache.set(order_id, order)
        return order

The domain never learns about the cache (lesson 0023's ports at work). The fast-but-complex code has one address: the adapter. Knuth's verdict stands: premature optimization is the root of much evil — because it spends maintainability before the measurement exists. But the measured optimization, contained behind a boundary, spends a little maintainability for a lot of performance, deliberately.

Worked: the same optimisation, two opposite verdicts

A nightly reconciliation job takes four hours on eight vCPUs. An engineer proposes rewriting its inner loop — three days of work, and the result is code that only they will enjoy reading — for a measured 20% saving.

The first thing a lead does is convert both sides into the same unit. Twenty per cent of four hours is 48 minutes a night: 8 vCPU × 0.8 h = 6.4 vCPU-hours, at roughly €0.04 per vCPU-hour, about $0.26 a night, $93 a year. Three days of loaded engineering time is roughly €1,920, plus a permanent readability tax on whoever debugs the job at 2 a.m. Payback: about twenty years. The proposal is not wrong about the speed-up; it is wrong about the scale, and no amount of profiling would have revealed that, because the profiler measures the job, not the business.

Now change one fact. The same 20% applied to a fleet of 400 always-on instances is 3,504,000 instance-hours a year; at €0.09 an hour that is €315,360, so 20% is about €63,000 a year. The same three days now pays for itself in under a fortnight, and a lead who refused it on principle ("premature optimisation") would be wrong by two orders of magnitude. Identical code change, identical measurement, opposite verdict — decided entirely by the multiplier the code sits behind.

The alternative a lead rejects in both cases is the generalising one: "the ORM is slow, let's drop it across the service". The profile said 78% of one endpoint's time went into serialising a 5,000-row response. That finding does not travel: the other 140 endpoints return a handful of rows each, so raw SQL buys them nothing measurable while charging every one of their future readers. Fix the one path, behind the one interface, and leave the other 140 alone.

The consequence over time is the thing seniors actually protect. Maintainability is what buys the option to optimise later: an implementation hidden behind a port can be replaced in an afternoon, whereas a data model that has leaked into forty call sites cannot be replaced at all without a migration project. "We will optimise it later" is only true if someone kept the boundary that makes later possible.

† The corollary for AI: AI will "optimize" on request — producing clever code with no measurement behind it. Require the number first; then let the cleverness be contained in an adapter.
Field notes · every optimization owes you a number
The “optimization”The number that would justify itIf there is no number
a hand-rolled cachep95 latency, and the share of it spent in the cached call.Delete it — the hidden state costs more than it saves.
a denormalized columnThe query plan before and after, at production row counts.Revert; a covering index is cheaper to maintain.
__slots__ on every classMeasured memory, at the object count you actually create.Remove it; it constrains subclasses for nothing.
a batch size of 1000Throughput measured at 100, 1 000 and 10 000.It is a guess wearing a constant’s clothes.
What people get wrong after reading this
"Premature optimisation is evil, so performance is a later problem."
Knuth was talking about micro-tuning individual statements. The decisions that actually determine performance — the data layout, the chattiness of a boundary, whether a loop crosses the network — are architectural and are the most expensive things to change afterwards. Guessing at line level is premature; thinking about shape at design time is not.
"The profiler tells you what to fix."
It tells you where time went under the workload you gave it. Profile with 50 rows and you will confidently optimise the wrong function for the production case with 50,000. The workload is the measurement; the tool is just arithmetic.
"Clean code is slow code."
Layers cost nanoseconds. Real slowness is almost always shape: an N+1 query, a serialisation of everything, a chatty call inside a loop, a missing index. Those are easier to see, not harder, when the code is clean — which is why the fastest teams profile clean systems.
"We can always optimise it later."
Only if a boundary survives to optimise behind. The maintainability you keep is not sentimentality; it is the option you will exercise on the day the number finally arrives.

When this is the wrong move

"Clean first, measure, then optimise the few hot paths" assumes a cold majority exists. Three situations where it does not: a compute kernel, codec or simulation loop where essentially all of the time is in one place and the memory layout must be chosen before the first line is written; an embedded or battery-constrained target where the budget is fixed by physics rather than by preference; and any deadline that is part of the contract — a fraud check that must answer inside the authorisation window is not slow-but-correct, it is wrong. In those cases performance is a quality-attribute scenario with a number attached from day one (lesson 0043), and the design is built around it rather than cleaned up afterwards.

In the lab: the companion Phase 3 exercise puts a hand-rolled cache and a per-row lookup into the same returns service. You profile it against a realistic case volume, find which of the two the numbers actually accuse, and either delete the cache or hide the fix behind the existing interface — with the measurement recorded next to the change.

Skill: where does the speed live?

What justifies spending maintainability on speed?

The optimized implementation belongs:

"Premature optimization is evil" because it:

Practice on your own code

Find the "optimization" in your codebase — the cache, the micro-optimized loop, the clever data structure. Ask: what number justified it? If there is no number, delete it or write the number that would justify it. If there is a number, confirm the fast code is hidden behind a clean interface.

Reveal: a measured verdict

A report module had a hand-rolled LRU cache added "because reports were slow". The number: an instrumented run showed the bottleneck was a N+1 query, not the formatting the cache wrapped. The cache was deleted; the query was fixed in the repository adapter. Result: faster, and one less hidden state (lesson 0019's greps came out clean).

Teach it back
  1. In two sentences, explain to a junior why making a component that is 5% of the runtime twice as fast is almost never worth three days.
  2. Find one optimisation in your own codebase and convert both sides into one unit: what does it save per year in money or milliseconds, and what did it cost in engineering time plus every future reader's minutes?
  3. Predict the one-year state if unmeasured cleverness keeps landing: which part of the system will nobody volunteer to change, and what will the team call that reluctance instead of "we cannot read it"?

Your win

You can demand a number before any optimization, contain the measured hot paths behind deep modules, and audit existing "optimizations" for the evidence they never had.

Read and watch deeper

Bring a cache or clever loop to your agent-teacher and hunt for its justifying number.