Lesson 0032 · Master Designer · Module 2

Consistency vs Availability

When the network fails, a distributed system must choose: answer with possibly-stale data, or wait for the truth. The Master's skill is choosing per operation — not per database.

Mission tie-in: the second System Trade-Off — where lesson 0011's invariants meet the real world of partitions and timeouts.

Knowledge: the choice under failure

CAP says: during a partition (network failure between nodes), you choose between consistency (every read sees the last write) and availability (every request gets an answer). The catch — Kleppmann's correction: almost no system is purely one or the other, and CAP is a framing, not a product picker. The real question per operation: what does a stale answer cost, and what does a timeout cost?

Operation	Stale read costs…	Timeout costs…	Lean
Account balance	Overspending, legal exposure	Frustration, retry	Consistency
Product stock display	A wrong "in stock" hint	Lost sale, empty page	Availability
Order status	Confused customer	Retry-able annoyance	Availability
Fraud decision	Chargeback	Blocked purchase	Consistency

The engineering that makes this a design instead of a gamble:

Decide at the data boundary: what must be strongly consistent (a balance, a reservation) vs what tolerates eventual consistency (a stock hint, a cache, a search index).
Make staleness visible: "as of 2 minutes ago" — an honest UI beats a silently stale one.
Prefer constraints that work offline: an idempotency key on writes (charge once, even if retried) buys a lot of the safety without sacrificing availability.
Contain the strong-consistency core: the smaller the always-consistent surface (lesson 0012's aggregates), the less the system pays for it.

Both of the middle two are code you can point at. The idempotency key buys availability without giving up write safety:

def charge(self, amount: Money, token: str, key: IdempotencyKey) -> ChargeId:
    existing = self._charges.find_by_key(key)
    if existing is not None:
        return existing.id            # the retry returns the first answer
    charge_id = self._gateway.charge(amount, token, key)
    self._charges.record(key, charge_id)
    return charge_id

The detail that makes this work: the key is generated by the caller and reused across every retry of the same intent. A client that mints a fresh key per attempt gets two charges, and the mechanism has bought nothing — the safety lives in the caller's discipline as much as in the server's table.

And staleness, made visible instead of silent:

@dataclass(frozen=True)
class StockHint:
    quantity: int
    as_of: datetime              # the UI can say "as of 2 minutes ago"

A number with a timestamp is an honest answer. The same number without one is a promise the system cannot keep — and the customer discovers that at checkout instead of on the product page.

† The framer's trap: saying "our database is CP" and moving on. Products have configurable knobs (sync vs async replication, quorum sizes); the per-operation choice is yours, and most operations in most systems can afford availability with an idempotent write path.

Skill: which way does this operation lean?

A cached product-stock hint may be stale. The cost analysis favors:

An account balance read during a partition should:

An idempotency key on charges buys safety by:

Practice on your own code

List your system's read operations. For each, write the two cost sentences: "a stale read costs…" and "a timeout costs…". Mark the lean. Then check: where staleness is served, is it visible to the user?

Reveal: an operation ledger

A booking system: seat availability — stale costs a bad hint, timeout costs the sale → cache with "live availability" refresh. Booking confirmation — stale costs double-booking → strict, with an idempotency key per booking attempt. Price display — stale costs a wrong quote → cache with a visible "as of" timestamp. Three reads, three leans, all written down.

Your win

You can write the two-cost analysis for any read operation, lean per operation instead of per database, and use idempotency keys to buy availability without losing write safety.

Read and watch deeper

"Please stop calling databases CP or AP", Kleppmann — the framing this lesson's per-operation choice comes from.
Designing Data-Intensive Applications, Kleppmann — ch. 5 (replication), 8 (the trouble with distributed systems), 9 (consistency and consensus).
Release It!, Nygard — failure modes and degradation strategies under partial outage.
Watch: ArjanCodes YouTube — search "CAP theorem" or "eventual consistency".
Next: lesson 0033 — Coupling vs Coordination.
Reference: Trade-offs — consistency vs availability; glossary — eventual consistency, idempotency key.

Bring your read-operation ledger to your agent-teacher and cost the failures together.

Mark lesson complete