Downloadable pack · blank template and completed example
The Initiative Artifact Pack
Thirteen artifacts that carry an initiative from an ambiguous brief to a measured, operated outcome. Each one appears twice: a blank template you can copy, and the same artifact filled in for the returns-resolution initiative that runs through lessons 0039–0060.
How to use the pack
Every artifact here answers a question that would otherwise be answered by accident. The blank version is the copy; the completed version shows what a filled-in answer looks like when it is specific enough to argue with. Nothing in the pack is mandatory. If an artifact does not support a decision in your initiative, delete it — a playbook that becomes document bureaucracy has failed at the thing it was built to prevent.
The Markdown files sit next to this page and are meant to be committed alongside the code the initiative changes, so the reasoning and the system age together. The pack README lists all thirteen with their source lessons; the narrative treatment of the same material is the Initiative Playbook.
| Need to decide | Artifact | Ready when |
|---|---|---|
| Why act, and what counts? | Initiative frame | Outcome, guardrails, scope, unknowns, and owners are arguable. |
| What might be false? | Assumption map | High-impact, weak-evidence beliefs have tests, owners, and dates. |
| How do we decide with a room? | Workshop agenda | One decision question, criteria agreed before options, a named decider. |
| How does work really happen? | Workflow board | Rules, waits, actors, failures, language, and external systems are visible. |
| What is the smallest useful path? | Story map | One vertical slice can produce evidence end to end. |
| What must the design be good at? | Driver table | Five to seven scenarios have measures and evidence mechanisms. |
| Do the diagrams expose the decision? | C4 review checklist | Every boundary has a driver and every driver has a visible response. |
| Who owns which fact? | State and contract review | Transitions, writers, failure semantics, and evolution are written. |
| Which option, on what evidence? | Options, spike, ADR | Forces, rejected options, accepted costs, dissent, and a revisit trigger exist. |
| In what order do we retire risk? | Delivery map | Milestones produce evidence; dependencies have owners and fallbacks. |
| How do eight people act as one? | Team charter | Decision rights, WIP, sources of truth, cadence, and escalation are explicit. |
| How do we expose this safely? | Rollout contract | Tests, telemetry, gates, rehearsed reversal, and an operational owner exist. |
| What changed, and when is it done? | Review and closeout | Evidence changes decisions, and temporary machinery leaves. |
The traceability test
The pack succeeds by coherence, not by weight. Take one line and trace it end to end: outcome → assumption → driver → decision → rollout. On the returns case that line reads: median 72 h → under 4 h · assumption A3, “the carrier can create a label inside the request” · driver D1, survive a 30-minute carrier outage and drain 99% of labels within 15 minutes of recovery · ADR 0003, which superseded an already-accepted ADR 0002 after a four-hour spike · a rollout whose every stage rehearses the disable path that decision created — and whose first rehearsal found the path did not work.
Run it backwards too. Take any artifact, milestone, or recurring meeting and ask which outcome, assumption, driver, risk, or rollout decision it supports. If the honest answer is “we always produce this one”, it is unnecessary work — delete it.
Printing. This page is designed to print: the masthead, navigation, and drawer drop out, and every blank template prints as a form you can write on. Completed examples live inside collapsed disclosures, so open the ones you want on paper before printing — or print the Markdown file instead, which is the complete version.
The completed example is one coherent initiative across all thirteen artifacts, not thirteen unrelated samples — the same initiative worked at greater length in the capstone packet. The same numbers recur throughout: 72 hours, under 4 hours, 41% → 31%, 40 requests/second, 800 ms, 50 referrals a day, fraud loss of 2.4 per 1,000, $3.42 per resolved return, 30 September. It includes a four-hour spike that killed an already-accepted decision, an assumption that turned out false, an inconclusive result reported as inconclusive, dissent that survived the decision and was later tested, a changed decision taken on evidence rather than under deadline pressure, a missed target reported as a miss, a guardrail that held on a measure too under-powered to detect a breach, and debt left behind deliberately with a repayment trigger that had already fired.
1 · Initiative frame
One page, written by product, design, and engineering together, before anyone estimates anything. It exists to turn a requested output into a decision-ready problem. Version it as facts change; a changed frame is learning, not failure.
Download Markdown · taught in lesson 0039
- Why now
- The trigger, the evidence behind it, and the cost of doing nothing this quarter.
- Problem
- For [cohort] in [situation], [observed condition] causes [impact]. Baseline [number], source [system], measured [date range].
- Outcome
- Move [measure] from [baseline] to [target] for [cohort] by [date]. A behavior change, not a shipped feature.
- Guardrails
- Measures that must not degrade. Each needs a threshold and an owner.
- First boundary
- The first cohort and situation served end to end. Narrow enough to finish.
- Non-goals
- Each one prevents a specific plausible misunderstanding, not a generic “out of scope”.
- Constraints
- Facts the design cannot negotiate. If asking could change it, it is a preference.
- Assumptions
- Top value, usability, feasibility, and viability beliefs, each with its evidence strength.
- Risks
- Impact × likelihood, the earliest signal, mitigation, owner, review date.
- Decision rights
- Final call on outcome/scope, experience, architecture, rollout — and who must be asked first.
- Next evidence
- What the team will learn before the next expensive commitment, and by when.
Quality check. Give the page to someone outside the initiative. They should be able to answer: why now, for whom, what measurable change, what is deliberately excluded, what cannot move, what might be false, and who decides. If they can only describe the proposed features, this is still a brief.
Completed example · returns resolution pilot
- Why now
- Returns are the second-largest source of support contact volume, and returns volume rises about 40% from mid-November. No support headcount is approved for the season. Doing nothing means agents spend the season chasing status instead of handling exceptions.
- Problem
- For signed-in domestic customers returning an unopened item within 30 days, a request waits a median of 72 hours for an agent decision, and 41% produce at least one support contact. Baseline: returns queue export joined to the contact log, 1 June – 31 July, n = 4,812; query owned by Priya and re-runnable.
- Outcome
- Median time from request submitted to decision communicated: 72 hours → under 4 hours. Support-contact rate: 41% → 31% or lower, a 25% relative reduction. Pilot open to real customers by 30 September.
- Guardrails
- Fraud loss ≤ 2.4 per 1,000 eligible returns (Risk). Manual review queue ≤ 50/day (Nils, support ops). WCAG 2.2 AA on four critical flows (Marco). Carrier cost ≤ $3.42 per resolved return (Priya). Personal data retained ≤ 30 days (Ola, privacy).
- First boundary
- Signed-in domestic customer, unopened item, delivered 30 days ago or less, refund to the original payment method.
- Non-goals
- Exchanges. International shipping. Guest checkout. Redesigning warehouse intake. Replacing the agent tool. Each is here because someone plausibly assumed it was in scope.
- Constraints
- The order platform stays authoritative for order truth. The carrier contract is fixed for the pilot term. No payment card data enters the returns module. The team is seven people and does not grow. The 30 September date is fixed by the season, not by preference.
- Risks
- R1 automatic decisions raise fraud loss (signal: exposed cohort vs. matched control; Risk). R2 the referred cohort reads a referral as a refusal and contacts support anyway, harming the very measure (signal: contact rate inside the referred segment; Marco). R4 guest orders are 29% of demand and stay outside the pilot, so the headline outcome overstates the business effect (Priya, decision 15 November).
- Decision rights
- Outcome, scope, cohort: Priya. Experience: Marco. Architecture and technical risk: Dana. Rollout advance: Dana with Nils. Stop authority: any on-call engineer, acting first and explaining afterwards.
- Next evidence
- Comprehension retest at n = 12 (Marco, 15 September). Fraud rate in the 1% cohort against a matched control after 120 exposed requests (Risk, from 29 September). Guest-order sizing (Priya, by 15 November).
The version history is the part most frames delete. v0.1 (25 Aug) defined success as “automatic approval for at least 90% of eligible requests”. v0.2 (4 Sep) changed the outcome event from automatic approval to decision communicated, because the 200-case policy sample showed 15% of requests are genuine judgment calls — an outcome defined on automation would have scored the correct product behavior, routing a judgment call to a human quickly, as failure. v0.3 (2 Oct) moved guest orders from “non-goal” to “deferred opportunity, sized, decision date 15 November”, because “non-goal” kept being re-argued as “refused forever”.
Full frame with every field and the version history: initiative-frame.md.
2 · Assumption map
Every plan rests on beliefs. This artifact separates the beliefs that would be expensive to get wrong from the ones that would not, and buys evidence only where the answer would change a decision. Place each belief on two axes — impact if false, and evidence strength — and test the high-impact, weak-evidence corner first.
Download Markdown · taught in lesson 0040
| Column | What goes in it |
|---|---|
| Assumption | Written so it can be false. “Customers want self-service” is a slogan; “at least half of eligible customers complete a return without contacting support” is a belief. |
| Type | Value (will they choose it?), usability (can they complete it?), feasibility (can we build and run it?), viability (can the business support its policy, cost, and operation?). |
| Impact if false | Name the decision or artifact that would have to change — not a severity word alone. |
| Evidence today | None, anecdote, sample, or measurement. A vendor claim and an internal opinion are both “none”. |
| Test | The cheapest thing that could change your mind. A sample, five interviews, a four-hour spike. Not “research”. |
| Owner / by | One person, one date. An unowned test is a wish. |
| Result | The number or finding, and which artifact it changed. |
Completed example · eight beliefs, three of which got a budget
| Assumption | Impact if false | Test (cost) | Result |
|---|---|---|---|
| A1 · At least 90% of eligible requests match a codifiable policy rule (feasibility, evidence: one estimate) | High — the 50/day queue guardrail breaks and the automation premise weakens | Classify 200 closed agent decisions. 2 analyst-days | Falsified. 63% codifiable, 22% need a photo rule outside pilot scope, 15% are judgment calls. Changed the outcome statement and created the referral state. |
| A2 · Customers trust an automatic decision enough not to contact support (value, evidence: inference only) | High — the contact-rate half of the outcome fails even if the latency half succeeds | Five-participant moderated comprehension test. 3 designer-days | Inconclusive. 2 of 5 read “your label is being prepared” as a refusal — a strong-sounding fraction of a sample too small to separate a wording problem from two participants. Retest at n = 12 booked. |
| A3 · The carrier is fast enough to sit inside the request (feasibility, evidence: vendor documentation) | High — it decides the failure model of the whole flow | Four-hour sandbox spike: 200 creates for p50/p95/p99, then 20 induced timeouts retried, ten with a client reference and ten without | Falsified. p50 640 ms, p95 1.9 s, p99 11.4 s. Duplicates: 0 of 10 with a client reference, 2 of 10 without. Superseded ADR 0002 with ADR 0003. |
| A4 · Warehouse receipt events are reliable enough to trigger a refund automatically | Medium — a missed event strands money; a duplicate double-refunds | Replay 30 days of events. 1 engineer-day | Confirmed with a caveat: no missing or duplicate events, 0.4% arrive more than 24 hours late. The refund SLA now starts at the event, not the scan. |
| A6 · Fraud loss does not rise when the decision becomes automatic | High — a breach stops the initiative regardless of the outcome measures | No pre-test bought — fraud responds to the presence of the new path, not to a prototype | Converted into a rollout guardrail with a matched control, a stop threshold, and Risk holding unilateral stop authority. |
| A8 · Customers would rather self-serve than call, for this class of request | High | None bought, deliberately — 71% of baseline contacts are status chasing, so the belief is already evidenced | Held as a stated premise and re-checked at closeout. Buying evidence for a well-evidenced belief is a way of looking busy. |
Two habits are worth stealing. The test is sized to the decision, not to the assumption: A3 got a four-hour spike because a number would change a structural choice, while A7 (do policy rules change often?) got two hours of reading because the change log already existed. And “no test” is a legitimate recorded choice — a map where every row has a test is usually a map nobody costed.
All eight rows with owners, dates, and evidence strength: assumption-map.md.
3 · Discovery and brainstorming workshop
A workshop is a decision-making instrument, not a room booking. Unstructured brainstorming rewards the fastest speaker, the most senior person, and the first plausible idea — after which the group spends an hour decorating an anchor. Design the conversation before you send the invitation.
Download Markdown · taught in lesson 0040
- Decision question
- One sentence, visible throughout. If you cannot write it, cancel the meeting and go find it.
- Decider
- The named person who closes the choice after advice. Not “the group”.
- Participants
- Invited for knowledge, not representation: user and domain knowledge, product, design, engineering, and whoever carries the operational consequence.
- Pre-read
- Frame, evidence, constraints, glossary — a day ahead, with facts, interpretations, and open questions marked separately.
- Criteria
- Outcome impact, confidence, effort, reversibility, guardrail risk. Agreed before anyone sees a favored option.
- Closing artifact
- Named in advance: an assumption map with owners, an option table, a decision plus its dissent.
| Minutes | Mode | Prompt | Output |
|---|---|---|---|
| 0–10 | frame | Read outcome, evidence, constraints. Clarifying questions only. | Shared decision frame |
| 10–20 | silent diverge | Facts, assumptions, questions — one per note, written alone. | Independent raw material |
| 20–35 | share and cluster | Round-robin. Clarify without defending. Name themes. | Named opportunity themes |
| 35–50 | silent sketch | At least three mechanisms, including a manual or no-build option. | Three genuinely different options |
| 50–65 | converge | Score against the agreed criteria; each person argues against their own preference. | Trade-offs and recorded dissent |
| 65–75 | close | Direction, dissent, unknowns, tests, owners, decision date. | Discovery actions |
A dot vote reveals sentiment; it does not make the decision. The board is not the output — the closing paragraph is.
Completed example · returns option workshop, 26 August
- Decision question
- What is the smallest intervention that can cut eligible resolution time from 72 hours to under 4 during the pilot?
- Direction
- Test automated eligibility and proactive status before committing to a full portal. Do not build the portal shell first.
- Dissent
- Nils (support operations) believes guest orders will dominate demand and that a signed-in-only pilot will measure the wrong population. Recorded. The evidence that would settle it is the guest share of return-eligible orders.
- Unknowns
- The codifiable-policy rate; carrier label latency; whether customers understand an automated refusal; whether Risk will accept an automatic decision at all.
- Next evidence
- Priya samples 200 closed agent decisions · Marco runs a five-participant comprehension test · Dana runs a four-hour carrier latency spike · Priya sizes the guest share. Four tests, four owners, four dates inside two weeks.
- Decision date
- The technical direction in two weeks, once the spike lands.
The most useful contribution came from Nils's silent-writing notes: most eligible returns are unopened domestic items that agents approve without real deliberation, and most of the 72 hours is queue time rather than decision time. In an open discussion anchored on “build a portal”, that fact does not get said.
Five mechanisms were sketched, four genuinely different: automated eligibility; proactive status with no automation at all; agent tooling that removes re-keying; a full self-serve portal; and adding two agents for the season. The workshop deliberately did not produce an architecture — choosing a mechanism before the codifiable-policy rate was known would have been an opinion with a diagram attached, and the sample later came back at 63%, not the 90% everyone in the room had assumed.
And the dissent was right. Guest orders turned out to be 29% of demand. The cohort did not change, but the outcome statement did — it now names the signed-in cohort explicitly, so nobody quotes the headline number as if it covered everyone.
Preparation, block-by-block notes, and facilitator failure modes: workshop-agenda.md.
4 · Workflow and EventStorming board
The fastest route to a bad architecture is turning nouns in a brief into tables and services. Map one real recent case with the people who do the work — including the unofficial steps, because those usually contain the actual domain rules.
Download Markdown · taught in lesson 0041
- 1 · Domain events
- Facts that happened, past tense, left to right, for one real recent case.
- 2 · Commands and actors
- What caused each event, and who issued it — including people whose work is invisible in the system.
- 3 · Policies
- “When [event], [decide or do] if [condition].” These are the domain rules, and most are unwritten.
- 4 · Reality
- External systems with their owning team or vendor, waiting time with its cause, manual tools, and failures.
- 5 · Hotspots
- Where language, rules, responsibility, or confidence changes. Mark them; do not resolve them in the room.
- Hotspot record
- Example and counterexample, who knows, the open question, and the next evidence with a date.
- Glossary
- Per contested word: precise local meaning, the context that owns it, and the phrase to stop using.
- Candidate boundaries
- Name, the evidence for it (language, invariant, change cadence, or ownership — not “it is a noun”), and what it would own. Model boundaries, not deployables.
- Target workflow
- The spine rewritten with waiting, duplicate entry, and safely automatable decisions removed. Do not automate every current step.
Map the current state before the target state. Skip it and the team will faithfully automate invented steps and miss real constraints.
Completed example · returns workflow, hotspots, and boundaries
return requested → order verified → evidence requested → evidence received → policy interpreted → eligibility decided → label created → customer notified → parcel received → refund authorized → refund settled
Actors: customer, support agent, policy owner, warehouse clerk, finance clerk. External: order platform, carrier, payments, warehouse system.
Two facts fell out of the timings immediately. 68 of the 72 baseline hours sit in two queue waits — order verification and policy interpretation — and neither is work; both are the request waiting for a human to reach it. And the 19-hour evidence round-trip only happens for opened items, which the pilot excludes. That is why the pilot's narrow scope is not timidity: it removes the second-largest wait by construction.
| Hotspot | What it turned out to be |
|---|---|
| “Approved” means three different things | Support: policy eligibility passed. Warehouse: inspection passed. Finance: money settled. One global status field would have collapsed three models into one lie. |
| Policy interpretation varies by agent | Three agents given the same 31-day case: two refused, one approved “because the customer was polite about it”. The sample then said 63% codifiable, 22% needing a photo rule, 15% judgment. |
| Gift orders have no rule and no owner | The purchaser is not the recipient, so a refund to the original payment method surprises the recipient. An owner was assigned and the rule deferred to slice 3 rather than smuggled into the pilot. |
| Nobody owns “the request that went quiet” | A request with no reply simply ages. No queue, no alert, no age limit. Became the 21-day expiry transition and a daily reconciliation sweep — a detector with an owner, because it is not an invariant. |
| Boundary | Language | Change cadence | Invariant | Ownership |
|---|---|---|---|---|
| Eligibility | “Decision”, “reason”, “override” exist nowhere else | 9 policy changes a year, 4 mid-quarter | One open decision per return | One named policy owner |
| Return journey | “Intent”, “progress”, “notification” | Changes on design's beat | One label intent per accepted return | Design owns the customer-visible steps |
| Payments (external) | “Settled”, “captured” | Payment-rail cadence | Refund never exceeds the captured amount | Another team; financial truth |
All four kinds of evidence point the same way, which is the only kind of boundary evidence worth acting on. Target workflow: two waits deleted, one manual step deliberately kept (warehouse receipt, about 40 confirmations a day, with a stated removal trigger), and one new state added — referral — because the sample said 15% of cases need a human and pretending otherwise would have produced bad automatic refusals.
Every hotspot record, the full glossary, and the target-state table: workflow-and-eventstorming-board.md.
5 · Story map and example card
A flat backlog hides the journey and rewards component work. A story map lets the team cut the smallest end-to-end path that can teach something; example cards make its behavior specific before anyone implements it.
Download Markdown · taught in lesson 0042
- Backbone
- User activities across the whole journey, left to right, in the user's language.
- Depth
- Alternatives and rule variations beneath each activity.
- Release line
- One narrow row that completes the journey. State which assumption or outcome it tests.
- Manual seam
- Which uncertain automation can a person safely perform for the first cohort? Record capacity, owner, and removal trigger.
- Slice test
- Deployable independently, demonstrable in user language, observable with a named measure, and usable to decide what to do next. If the demo is “the endpoint returns 200”, it is a task bundle.
- Rule
- The behavior being clarified, in one sentence of domain language.
- Happy
- Given [specific state with real values], when [action], then [observable result].
- Boundary
- The exact edge or counterexample, and the reason the user is given.
- Failure
- A dependency timeout, retry, conflict, or partial outcome — and what the caller may safely do.
- Guardrail
- The accessibility, security, privacy, performance, or operational case.
- Telemetry
- The event or measure that will show this behavior happening in production.
- Open question
- What the three amigos could not answer, and who will.
Specificity matters more than Given/When/Then syntax. “Delivered 12 days ago” teaches; “a recent order” does not.
Completed example · pilot slice and two example cards
| Start request | Check eligibility | Choose resolution | Return parcel | Track outcome |
|---|---|---|---|---|
| signed-in order | domestic / unopened / ≤ 30 days | refund to original method | carrier label, drop-off | status page and email |
| gift order | referred to an agent | refund with restocking fee | QR drop-off | SMS |
| guest order | opened / damaged, photo evidence | exchange | home collection | agent contact / appeal |
Release line: the first row only. What it proves: both halves of the outcome on 1% of eligible demand, against a matched control, at n ≥ 120 requests. Manual seam: warehouse receipt matching — about 40 confirmations a day inside the clerk's existing workload, removed after 14 consecutive days with under 0.1% late receipt events and zero mismatches.
What the pilot slice is not. It is not “the eligibility service” and not “the returns database”. Both could be finished without a single customer being served.
- Rule
- An unopened item bought by the signed-in customer and delivered 30 days ago or less is accepted automatically. The window is evaluated on the delivery date in the store's timezone, not on the submission timestamp.
- Happy
- Given an unopened domestic item delivered 12 days ago and a signed-in purchaser, when they submit a return, then the decision is accepted, exactly one label intent is recorded in the same transaction, and the decision reaches the customer within 800 ms at p95.
- Boundary
- Delivered 31 days ago → refused with reason code outside_return_window, stating the delivery date it used and offering an appeal path. Delivered exactly 30 days ago at 23:50 store time, submitted at 00:10 the next day → accepted: 30 days is inclusive and measured in whole days. That second example exists because the first implementation used UTC timestamps and refused it.
- Failure
- A timed-out carrier call retried with the same client reference returns the original label and creates no second one. A 30-minute carrier outage still accepts eligible requests, with 99% of pending labels created within 15 minutes of recovery.
- Guardrail
- A keyboard-only customer completes the happy path; every refusal reason is announced by a screen reader and never conveyed by color alone. Editing the order id in the URL returns 404, logs the attempt with the actor, and leaks no field of that order into any response, log line, or error message.
- Telemetry
- eligibility_decision_total{outcome, reason_code} and eligibility_decision_seconds as a histogram. A shift in the reason-code mix is the earliest signal that a policy change did something unintended.
A second card covers the fraud hold, and its most important line is a guardrail rather than a rule: no customer-visible text, email, or API response distinguishes a fraud hold from any other referral, asserted by a negative test that fails if the internal reason code ever appears in a customer-facing payload.
Both cards in full, plus the release-slice table: story-map-and-example-card.md.
6 · Architecture-driver table
Most requirements do not change a structure. Drivers are the few that do. A driver is ready when two plausible architectures would respond to it differently, the team can say why it matters to the outcome, and a check can prove whether the design meets it. If nothing decides and nothing tests, it is context, not a driver.
Download Markdown · taught in lesson 0043
When [source] causes [stimulus] during [environment], [artifact] must [response], measured by [number with a unit].
- Why it matters
- The outcome, guardrail, regulatory, or operational consequence. If you cannot name one, this is context.
- Design pressure
- Which structural choices this scenario rules in or out. This is the test of whether it is a driver at all.
- Evidence
- The test, fitness function, drill, or production measure that can fail.
- Owner and date
- One name, one date.
Select five to seven, ranked by business impact, uncertainty, and structural reach. Then write the conflicts between them, and separate constraints (facts you cannot change inside this initiative) from preferences wearing a constraint's clothes.
Completed example · five drivers, three conflicts, five rejections
| # | Scenario | Design pressure | Evidence mechanism |
|---|---|---|---|
| D1 · Availability | The carrier becomes unavailable for 30 minutes during peak. Keep accepting and deciding eligible requests, and create 99% of queued labels within 15 minutes of recovery. | Rules out creating the label inside the request. Requires persisted intent, an idempotent retry, and a customer-visible pending state. | Recovery drill with the sandbox forced to 100% timeouts, before every exposure stage. |
| D2 · Performance | At 40 requests/second at seasonal peak, return an eligibility decision within 800 ms at p95, excluding label creation. | Separates the fast decision from the slow vendor call; forces the label into its own observable state. | Load test at 40 rps against a production-shaped dataset; a production histogram with a p95 alert. |
| D3 · Security and privacy | A signed-in customer substitutes another customer's order identifier, in any environment including staging. Disclose no field of that order or return — in responses, errors, or logs — and record the attempt. | Object-level authorization at the domain boundary, not route-level authentication; identifiers out of log lines. | Negative authorization tests on every read path; a log-scanning fitness function that fails the build on an unredacted order id. |
| D4 · Modifiability | The policy owner adds one eligibility rule, mid-quarter, nine times a year. One engineer releases it within one working day, changing no adapter, no UI component, and no API contract. | A cohesive eligibility module with a stable decision-plus-reasons result; no policy conditionals in adapters or templates. | Import-rule fitness function in the build; measured lead time on the last three policy changes. |
| D5 · Operability | Label errors exceed 2% for five minutes during any exposure stage. On-call identifies the carrier as the cause and disables label attempts within ten minutes, without refusing new returns and without leaving the worker retrying. | A vendor-scoped metric, an actionable alert, a flag with a safe off-state, a runbook — and “pending label” as a real state rather than an error. | The reversal rehearsal before each stage, which is where a real defect was found. |
| Conflict | Resolution |
|---|---|
| D2's 800 ms budget vs. a label the customer can use | The spike measured the carrier at p99 11.4 s. Return the decision quickly; make label creation separately observable with its own measure. Recorded in ADR 0003 rather than discovered in sprint 3. The cost is a two-step mental model for the customer — which is exactly what the comprehension test exists to measure. |
| D3 (nothing identifying in logs) vs. D5 (on-call needs detail) | Correlation ids in logs; identifiers only behind an access-controlled lookup. Both drivers keep their measure. |
| D4 (fast policy change) vs. D3 (authorization strictness) | Policy decides eligibility, never visibility. Authorization sits outside the policy module and cannot be widened by a rule change — enforced by D4's own import rule. |
Five candidates were rejected as drivers, each with a reason: “the system must scale” (no load, operation, or horizon — and no option responds differently to it); “must handle 10× volume” (beyond the demand horizon, and untestable); “must be cloud-native / event-driven” (mechanisms, not qualities — they belong in the option table where they can be rejected); “must be maintainable” (replaced by D4, which can fail a check); and “99.99% uptime” (a number with no operation, which would have hidden the distinction that label creation may degrade but request acceptance may not).
Also caught: “use our standard message broker” was classified as a preference, not a constraint — one database transaction already gives all-or-nothing across return, decision, and intent. And “build inside the existing deployable” was classified as a decision, not a constraint, so its reasoning stayed visible and arguable.
All five drivers with why-they-are-ranked-there and owners: architecture-driver-table.md.
7 · C4 review checklist
The valuable diagram is not the most detailed one; it is the smallest view that lets its audience inspect a decision. Use this as a review instrument, with the driver table on the wall: every comment must point at a driver, a workflow, or a risk.
Download Markdown · taught in lesson 0044
- Title
- What it shows, in the audience's language.
- Scope
- System, container, or component — and which part.
- Audience
- Who is reading it, and what they must decide.
- Version and date
- Diagrams rot silently. Date them.
- Legend
- What a box, a dashed line, and a color mean.
- Drivers addressed
- The numbered drivers this view answers.
| View | Check |
|---|---|
| Context | Every external system names its owning team or vendor · every trust boundary is drawn, including between your own systems · every actor is a role someone performs, including operational ones · PII crossings are marked. |
| Container | Every arrow is labeled with the interaction and the protocol · every data store names its single authoritative writer · every deployable has an owner and an on-call route · every deployable beyond the first completes the extraction sentence below. |
| Dynamic | The path drawn is the risky one, not the easy one · at least one failure path is drawn · timeouts, retries, and idempotency keys are visible · the point of no return for external side effects is clear. |
| Module | Boundaries follow evidence — language, change cadence, invariants, ownership — not nouns from the brief · dependency direction is enforceable by a test, not only by intent · technology sits at the edges: create_label(intent), not carrier_sdk.shipments.post(payload). |
| Whole packet | Every driver has a visible response somewhere · every box and arrow has a justification · no diagram serves two audiences at once. |
The service-extraction sentence. “We accept network failure, versioning, deployment, observability, security, and on-call overhead because independent [scaling / isolation / availability / release cadence / technology / ownership] is required by driver [n].” If the blank says “cleanliness” or “maybe later”, keep a module seam and postpone the distributed system.
Completed example · what one review actually found
| Finding | Action |
|---|---|
| An unlabeled arrow from the worker to the carrier | Labeled “create label · HTTPS/JSON · client reference = label intent id · 5 s timeout”. The timeout was news to two people in the room. |
| The first draft drew a separate “Returns Service” deployable | The extraction sentence could not be completed — no driver required independent scaling, availability, release cadence, or ownership. Reverted to modules inside the existing deployable, with the accepted cost (no independent scaling or release of eligibility) recorded in ADR 0001. |
| A proposed message broker between the API and the worker | Removed. One database transaction already gives all-or-nothing across return, decision, and intent; a broker would have added a distributed failure mode to buy nothing. An outbox row in the same schema does the job. |
| The dynamic view showed only the happy path | Added the carrier-timeout view. Drawing it is what made the duplicate-label problem visible on paper rather than in production. |
| An arrow from eligibility to the carrier adapter | Removed. Eligibility decides and performs no I/O. The build now fails if that import returns. |
| The privacy driver had no visible response anywhere | Added the personal-data column to the context view — which is where the team noticed the carrier payload carried the return reason. It now carries recipient name and address only. |
The one boundary that was justified is smaller than a service: the label worker is a separate process in the same artifact, sharing the same schema, because D1 requires the carrier call to survive a 30-minute outage without blocking request acceptance. That is the smallest boundary that satisfies the driver, and no network hop was introduced where a transaction would do.
The three “authoritative:” labels on the context view — order truth, settled money, parcel received — are the most load-bearing text in the packet. They say the returns system holds no independent opinion about any of them, which is what makes the state machine enforceable rather than aspirational.
Full checklist, the packet contents, and the module import table: c4-review-checklist.md.
8 · State and contract review
Most integration bugs are unanswered state questions wearing API syntax. The state machine answers what is legal; data ownership answers who may establish each fact. Prefer one authoritative writer per piece of state — if two modules can both set it, its invariant lives nowhere.
Download Markdown · taught in lesson 0045
From · command · to · actor · precondition · invariant protected · authoritative owner · fact emitted · audit need. If a transition is not in the table, the code refuses it.
- Race
- Two commands arrive for the same entity at once. What is the expected end state?
- Retry
- The same command arrives twice with the same intent key. What is returned the second time?
- Stale
- A command refers to a state the entity has already left. Reject, ignore, or reconcile?
- Reordering
- Facts arrive out of order. Which are safe to apply late?
- Compensation
- Which external side effects cannot be undone by a code rollback, and what reverses them?
- Meaning
- Command, fact, or query? What business promise does success make?
- Identity
- What identifies the resource, the caller's intent (idempotency key), and the ordering?
- Validation vs. authorization
- Which edge rejects malformed input, and which domain rule refuses valid-but-forbidden behavior?
- Failures
- Rejected, conflicted, unavailable, timed out, partial — and the safe caller action for each.
- Evolution
- Required fields, tolerant-reader rule, what may be added, deprecation window, migration owner.
- Data
- Classification, minimum fields, trust crossing, encryption, retention, deletion, who may read.
- Proof
- Producer and consumer contract tests, plus the operational signal that shows it working.
The contract review test. Ask the producer and the consumer separately what success, timeout, duplicate, missing field, new field, and deleted user mean. Any different answer is already a production defect; the code has merely not executed it yet.
Completed example · the return journey machine, its rules, and two contracts
| From | Command | To | Rule and authoritative owner |
|---|---|---|---|
| — | SubmitReturn | requested | No other open return for this order line; the order line belongs to the actor. Return journey. |
| requested | AssessEligibility | accepted / refused / referred | Exactly one decision per return. A refusal or referral without both a reason code and human text is not a decision. Eligibility. |
| referred | AgentDecides | accepted / refused | Agent identity recorded; contradicting a policy rule requires an override reason. Eligibility. |
| accepted | AttachLabel | awaiting_parcel | Exactly one carrier label per return — a unique constraint, not a code check. Return journey. |
| awaiting_parcel | Expire | expired | 21 days with no receipt; the customer is notified exactly once. Reconciliation job. |
| received | RequestRefund | refund_requested | Never sets settlement; carries the return id as its idempotency key. Return journey. |
| refund_requested | — | refunded | Payments only. No other actor may write this state, so “a refund never exceeds the captured amount” lives where the captured amount lives. |
Two rows carry most of the design. Only Payments may write “refunded” — the returns module requests and observes, and physically cannot break that invariant. And the refund row has no recovery, written down rather than hidden behind the word “rollback”. A code deploy can be reverted; a settled refund cannot. That single admission is what produced a settlement_disputed state and the rule that we never auto-compensate.
Not everything called an invariant is one. “Every accepted return reaches a terminal state within 21 days” looks like an invariant and is not: it is false for up to 21 days by design. Enforcing it as one would mean refusing legal states. It needs a daily detector with an owner and an escalation — a completely different piece of work, and one that only got built because the classification exercise happened.
The customer-facing contract makes one careful choice worth copying: 201 means the return exists and a decision has been made — it does not promise a label exists. That distinction is the whole point of ADR 0003. And the closed/open enum split matters more than it looks: reason_code is an open enum with mandatory human text, so a new reason ships freely; state is a closed enum consumers branch on, so a new state needs a version bump.
The carrier contract's key row came from measurement, not instinct: a timed-out create retried without a client reference produced a second label in 2 of 10 attempts, and with one, in 0 of 10. The client reference is not a nicety; it is what makes “exactly one label per return” achievable at all.
All thirteen transitions, the rule classification, both contracts, and the idempotency-key table: state-and-contract-review.md.
9 · Option table, spike brief, and ADR
Senior judgment is not having the first answer faster. It is exposing credible alternatives, buying evidence where uncertainty matters, and leaving a decision a future teammate can reconstruct. Produce at least two genuinely different designs — “Kafka” versus “managed Kafka” is one mechanism with two suppliers.
Download Markdown · taught in lesson 0046
Per option: mechanism in one sentence · drivers it satisfies · drivers it harms · delivery and operational cost · reversibility · largest unknown · evidence already held. Include “do nothing”, manual, or buy whenever they are credible.
- Decision blocked
- Which choice cannot be made until this is answered.
- Question
- One falsifiable uncertainty, phrased so a number can answer it.
- Timebox
- Fixed hours or days, and one named investigator.
- Method
- The smallest experiment; production-like only where the question requires it.
- Result format
- A number, a trace, a compatibility finding — or an explicit “inconclusive”.
- Disposal
- Throw the code away, or review it separately before it goes near production.
“Explore the carrier API” is not a spike. It is an unbounded implementation prelude.
- Title, status, date, decider
- Proposed, accepted, or superseded by ADR [n]. Name the decider and the advisers.
- Context
- Facts that are true today, and the decision required. No solution here.
- Forces
- Named drivers, constraints, risks, and the owners affected.
- Options
- Each credible alternative and, in one line, why it was not chosen.
- Decision
- Active voice, specific enough to constrain implementation.
- Consequences
- Benefits, the costs accepted, follow-up work, and who operates the result.
- Dissent and advice
- The material concern, who holds it, and what evidence would prove it right.
- Revisit when
- A measurable change in context. Then supersede — never rewrite.
Compression test. A teammate arriving in six months should be able to answer: what was true, what mattered, which alternatives were real, who decided, what was chosen, what price was accepted, and which changed fact should reopen it.
Completed example · three options, one four-hour spike, ADR 0003
Decision: what happens to an accepted return request when the carrier is slow or down? The team's provisional design called the carrier inside the request, and it had already been written up and accepted as ADR 0002 on 1 September.
| Option | Buys | Costs | Verdict |
|---|---|---|---|
| A · Synchronous call | Least code; the customer sees a label immediately, which helps comprehension | D1 and D2 both fail: at p99 11.4 s the 800 ms budget is impossible, and a 30-minute outage refuses every request | Rejected on measured evidence |
| B · Persist intent, drain from a worker | D1, D2, D5. An accepted request survives an outage; the vendor is separately observable and separately disableable | A two-step customer experience, a worker with a backlog to watch, a flag to own and delete. About 6 days | Chosen |
| C · An agent buys the label | Near-zero engineering; human judgment on odd cases | One manual purchase per accepted return — roughly 96/day at full exposure, on top of 34/day of referrals. The 50/day guardrail breaks around 30% exposure | Rejected as the everyday mechanism; retained as the fallback for outages longer than four hours |
Option C was not a straw man. It lost as the everyday mechanism because its cost scales with success, and it won as the emergency mechanism because it needs no new code and no vendor. Keeping a rejected option alive in a narrower role is often the most useful outcome of an option table.
- Question
- At the pilot's request rate, is the carrier's label-creation p95 under 500 ms, and does a retry after a timeout create a second label?
- Method
- Carrier sandbox: 200 sequential creates recording p50/p95/p99; then 20 creates with the socket timeout forced to 1 s, each retried once — ten with a client reference, ten without.
- Result
- p50 640 ms, p95 1.9 s, p99 11.4 s. Duplicates: 0 of 10 with a client reference, 2 of 10 without. The answer to the question is no, and yes.
- Disposal
- The harness was deleted. The one thing kept was the assertion “exactly one carrier label per intent”, which became a production reconciliation check rather than a script on a laptop.
- Caveat recorded
- The sandbox is not production. The p99 figure is directionally trusted, not contractually — which is why the availability driver's evidence is a recurring drill rather than a one-off number.
Four hours removed the option the team preferred, eleven days after it had been accepted as an ADR. That is the entire economic case for spikes.
- Supersedes
- ADR 0002 (“Create the carrier label inside the request”, accepted 1 September). ADR 0002 stays in the log, unedited, marked superseded — it is a historical fact about what the team believed on 1 September.
- Context
- A 4-hour median, 40 rps at peak, an 800 ms p95 decision path, and a fixed carrier contract measured at p50 640 ms, p95 1.9 s, p99 11.4 s, creating duplicates on timeout-retries without a client reference. Seven people, one deployable, a fixed 30 September date.
- Decision
- Commit the return, the decision, and a label intent in one transaction; return a decision with label_state = pending. A worker process in the same deployable drains intents and calls the carrier with the intent id as the client reference. A unique constraint makes a second label unstorable. Label attempts sit behind a flag readable inside the worker loop, disableable in under ten minutes.
- Accepted costs
- A two-step customer experience, on an assumption still unsettled at n = 5. A worker, a backlog, and an aging metric to watch — new on-call surface for a team of seven. A pending state visible in the API, so removing it later is a contract change. A flag that is temporary machinery with a deletion owner and date.
- Dissent
- Sam (engineer): “For a pilot of about 50 requests a day we are buying a queue, a backlog metric, a flag and an on-call surface to solve a problem that option C solves with a runbook.” Not resolved by argument. What would prove it right: by the 10% stage, no incident the manual path would not also have produced, and a manual queue under 20/day.
- Revisit when
- Sustained volume exceeds the worker's measured drain rate of 12 labels/minute; or the carrier publishes a durable asynchronous API; or the queue misses its 15-minute recovery target twice in a quarter; or the comprehension retest shows the two-step flow costs more contacts than the outage risk it avoids.
How the dissent ended. At closeout, referrals alone reached 41/day at peak, so option C's extra label purchases would have breached the 50/day guardrail: Sam's main point was recorded, tested, and not upheld. His second point — that the flag carries ongoing cost — was upheld and put on the removal list with a date. A dissent is not a vote to be defeated; it is a prediction to be checked.
Full option table, spike record, ADR text, and the five-entry decision log: option-table-spike-brief-and-adr.md.
10 · Walking skeleton and delivery map
A delivery plan should retire uncertainty, not arrange component tasks. Connect the real path early, then grow behavior through thin, demonstrable slices. Sequence by four forces: learning, technical risk, dependency, and value.
Download Markdown · taught in lesson 0047
From [real source branch or build], move one [controlled user action] through [real deployment, real boundaries, representative storage, one representative adapter] and prove it with [telemetry or demo].
Then list every fake or hard-coded behavior in it. The business behavior may be deliberately tiny; the delivery path must be real. An undeclared fake is how a skeleton becomes a demo that cannot grow.
- Milestone
- Named after what becomes true, with the evidence produced, a binary exit condition, and the risk or assumption it retires. Never “percent complete”.
- Dependency row
- External owner · the exact deliverable or decision · needed-by date · readiness evidence today · fallback · escalation trigger and recipient. “Waiting on platform” is not a plan.
- Forecast
- Range and confidence · what changed since last time · what is fixed (outcome, date, scope, team) · the one movable variable · the smallest scope trade that preserves the outcome · the date the forecast next changes.
Keep enabling work attached to the slice it enables. A free-floating “architecture runway” becomes infrastructure nobody has yet earned.
Completed example · a skeleton that found three surprises in week 2
From the main branch, deploy one internal-user return request through the production pipeline; persist its intent, cross the carrier port with a fake adapter, render “pending label” on the real status page, and find the complete request by correlation id in the real logging stack.
Real: the build, pipeline, deployable, database and its migration, module boundaries and their import check, the outbox, the worker process, correlation ids, the telemetry pipeline, the status page. Fake, stated rather than implied: eligibility always returns accepted; the carrier adapter returns a fixed label id after 100 ms; order lookup returns one fixture; no notifications; no authorization beyond “is signed in”; warehouse receipt is a button on an internal page.
What it proved in week 2 — three things nobody predicted. The storefront's session cookie was not available to the worker process, so correlation ids broke across that boundary. The migration needed a two-step deploy to avoid locking a hot table. And the platform's log retention was 3 days, not the 30 the privacy review had assumed. All three were cheap in week 2 and would have been expensive in week 9.
| Milestone | Exit condition | Retires |
|---|---|---|
| Path walks | Repeatable from a clean checkout; a forced failure is visible in the logging stack within one minute | Integration and deployment risk |
| Policy works | The policy owner accepts the decisions and the reasons on 200 sampled historical cases; disagreements are logged as rules, not bugs | The codifiable-policy assumption, at production fidelity |
| Team uses it | No severity-one usability, accessibility, or authorization defect open | The comprehension assumption in part, and the security driver |
| Pilot learns | The pre-agreed continue / change / stop thresholds can actually be evaluated at n ≥ 120 exposed requests | The fraud guardrail, and the outcome itself |
| Scale earns | Every rollout gate passes and the disable path has been exercised twice | Availability, performance, operability, and operational ownership |
| Dependency | Fallback | Escalation |
|---|---|---|
| Carrier sandbox credentials and rate limit (platform team) | The fake adapter behind the real port — the skeleton is designed to proceed without them | Not delivered by Wednesday → platform lead |
| Written confirmation of the refund idempotency key (payments team) | Refund requests queue for an agent; the pilot still measures the decision outcome | No answer by end of week 3 → engineering director |
| Warehouse receipt event access | Manual confirmation, which the pilot uses anyway | Access refused → escalate as a scope trade, not as a delay |
| Privacy and security sign-off | None. This is a gate, not a risk. Exposure does not start without it | Not signed five days before the stage → CISO |
Note the third row: when a dependency has a genuine fallback, its failure is a scope trade to be decided, not a delay to be reported. Writing the fallback in advance converts an escalation from a complaint into a decision request.
Forecast. Pilot open to 1% between 26 September and 3 October, about 70% confidence — not “30 September”, because a single date at that range is a claim the evidence does not support. Fixed: the date and the team. Movable: scope. The smallest trade, identified in week 1 and taken in week 3 on evidence rather than under deadline pressure, was to drop the in-product appeal flow to a later slice: about five days, and it touches neither outcome measure — which is the test a scope trade has to pass. What would not be traded: the reversal rehearsal, the authorization negative tests, and the control cohort, because trading those buys days by removing the ability to know whether the pilot worked.
All five milestones, the full dependency map, and the forecast fields: delivery-and-dependency-map.md.
11 · Team charter
Team structure is part of system design: communication paths, decision rights, and ownership become boundaries and coupling in the software. Write this together in about 45 minutes, and revise it when it stops describing what the team actually does.
Download Markdown · taught in lesson 0048
- Purpose
- The outcome and the first cohort, copied from the frame rather than paraphrased.
- People
- Accountabilities, availability as a fraction of the week, missing expertise, and the consultation path for it.
- Decision rights
- Who has the final call on outcome and scope, experience, architecture, implementation, quality, rollout — and who must be asked first. Advice improves a decision; it does not create a consensus veto.
- Flow
- WIP limit, swarm rule, review and pairing expectations, and the trigger for calling an item blocked.
- Sources of truth
- One location per fact: why, scope, flow, decisions, risk, evidence. Duplicated status creates conflicting truth.
- Cadence
- Per meeting: purpose, inputs, the artifact it changes, and the rule for canceling it.
- Escalation
- Objective triggers, the DRI, the format of the ask, the recipient, and the response-needed-by date.
- Health
- Sustainable pace, focus protection, knowledge redundancy, and one team experiment at a time.
The meeting test. Every recurring meeting needs an inspectable input, a decision or learning purpose, a source of truth it changes, and a shorter asynchronous alternative for weeks with no decision. If the output is “everyone is informed”, a written update is the deeper module.
Completed example · seven people, and the two rows that matter most
| Accountability | Final call on | Advice from |
|---|---|---|
| Product | Outcome definition, priority, cohort, scope trades | Design, tech lead, support ops, policy, risk |
| Design | Experience coherence, interaction, research method, accessibility | Customers, product, engineers, the accessibility reviewer |
| Tech lead | Technical integrity, architecture decisions, technical risk acceptance | Engineers, product, design, privacy, on-call |
| Engineers | Implementation decisions inside agreed boundaries | Peers, and any owner affected by a contract change |
| Risk | Stopping exposure on a fraud guardrail breach — unilaterally | Nobody |
| Tech lead with support ops | Advancing an exposure stage | Risk, privacy, product |
| Any on-call engineer | Stopping exposure, acting first and explaining afterwards | Nobody |
The asymmetry in the last two rows is the single most useful line in the charter. Advancing exposure requires two named people to agree; stopping it requires one person to act. A team that needs permission to stop will not stop in time.
Flow. WIP limit of two slices, which feels wrong for about a week and then stops feeling wrong. Swarm when a slice has not moved for half a day rather than starting a third. Any engineer may approve a pull request; the tech lead reviews only changes crossing a module boundary, a contract, or an authorization path — the three places where a mistake is expensive. That is why the tech lead is not a throughput bottleneck, and it is also what makes the one-day policy release achievable at all.
Escalation is triggered objectively: a dependency misses its needed-by date; a driver cannot be met inside the agreed scope; a decision crosses another team's ownership; forecast confidence drops below the agreed range; a guardrail breaches, in which case whoever notices stops first and talks afterwards.
Health. No planned weekend work — the date moves scope, not hours. Two protected no-meeting mornings a week, protected by the product manager including from the product manager. Every module touched by at least two people before the pilot gate: checked, not assumed.
Full charter with people, sources of truth, and the cadence table: team-charter.md.
12 · Readiness and rollout contract
Done is not “merged”, and launch is not one event. A release is a controlled experiment: every stage has an entry condition, something to observe, an advance threshold, and a rehearsed way back. A green error dashboard does not prove that returns resolve faster.
Download Markdown · taught in lesson 0049
- Risk-to-test map
- Risk or driver → test type (example, integration or contract, end-to-end, quality, exploratory charter, recovery drill) → what it proves → owner → where the evidence lives. Test according to risk, not a universal pyramid quota.
- Telemetry map
- Outcome and guardrails by cohort · system health and saturation · business-state distribution and age · dependency health · data quality and privacy · manual and support load. Every measure needs an owner and a decision it can change.
- Exposure stages
- Per stage: cohort or percentage · entry conditions · what to observe with segments and minimum volume and duration · advance threshold and named decider · pause or reverse trigger, authority, action, recovery time, communication · what happens to data and in-flight side effects.
- Dashboards exist, and someone has looked at them when nothing was wrong.
- Every alert names a threshold, an owner, and a runbook step.
- Structured events carry a correlation id across every boundary, including the worker.
- The runbook covers the known failure states, with the disable path first.
- A support brief written by someone who will answer the contacts.
- On-call and steady-state service owner named — a person, not a team alias.
- Access, capacity assumption, and data recovery documented.
- The disable path has been exercised, not merely written.
- Every temporary flag has a deletion owner and date.
Then run a 30-minute failure drill before the first customer stage: the dependency returns timeouts, the worker backlog grows, one request is duplicated. Can the team detect, diagnose, limit, recover, and explain?
Completed example · five stages, a holdback control, and what the rehearsal broke
| Stage | Entry | Observe | Advance | Pause or reverse |
|---|---|---|---|---|
| 1 · Team only | All tests pass; a support path exists; a first-draft runbook | All flows, 2 days | No critical security or accessibility defect open | Any critical defect · anyone |
| 2 · Staff accounts | Runbook exercised; alerts route to a human; privacy sign-off; reversal rehearsal passed | Errors, retries, comprehension; 50 real cases | Zero duplicate labels; decision p95 within 800 ms | Any duplicate label · tech lead |
| 3 · 1% of eligible demand, with a 10% holdback control unexposed for the whole pilot | Support briefed; reversal rehearsed again; the control cohort instrumented | Outcome and guardrails; 3 days and at least 120 exposed requests | Median improved against control; fraud not worse; queue under 50/day | Fraud above control by more than 1.0 per 1,000 · Risk, unilaterally. Contact rate worse than control · product |
| 4 · 10%, then 50% | The previous gate signed off by two named people | Capacity, referral age, manual queue, segments by device and decision outcome; 3 days each | All stage-3 thresholds still hold at the larger n | Referral SLA missed for 4 hours, or queue above 45/day · support ops |
| 5 · 100%, minus the holdback | A steady-state owner is named and has accepted the runbook | Continuous; weekly review | Outcome and guardrails hold for two consecutive weeks | Any guardrail breach · that guardrail's owner |
The holdback is the point of stage 3. “It improved after we launched” is not a finding. A control cohort that stays unexposed for the whole pilot is what turns the outcome numbers into evidence rather than into a coincidence with a date on it.
Reversal means “stop the intake”, never “undo the week”. Disabling exposure stops new returns entering the new path; returns already in flight continue, because abandoning them would strand customers mid-journey. Labels already bought are not voided. Refunds already requested are not recalled. The contract says so, rather than leaving the team to discover it at 2 a.m.
The rehearsal that found a real defect. First run: the disable flag did not do what everyone believed. It was checked when a request created an intent, so it correctly stopped new label attempts — but the worker read the flag once at start-up and kept retrying the 63 intents already in its backlog against a failing carrier. The stopwatch said “disabled in 3 minutes”; the carrier's dashboard said we were still calling it eleven minutes later. The fix: read the flag inside the worker loop, with a test asserting the worker stops within one iteration. Second run: disable worked in 90 seconds, and the gap was the support message — nobody could say what a customer with a pending label should be told. It is now a stage-2 entry condition.
The rehearsal runs before every stage, not once, because the disable path is code and code changes. An untested control is a belief; thirty minutes found a defect that would otherwise have surfaced at 02:00 during a real outage.
Risk-to-test map, telemetry map, and the operational readiness checklist: rollout-contract.md.
13 · Weekly review and closeout checklist
Leadership is keeping the truth cheap to see. Review evidence while options remain, change the plan without disguising the change, and finish the operational and organizational work after launch. Shipping is a transition in the lifecycle, not the finish line.
Download Markdown · taught in lesson 0050
- Outcome
- Baseline, current, target, cohort, and confidence — from the dashboard, not from memory.
- Guardrails
- Each one with its threshold, and any segment moving the wrong way.
- Learned
- Which assumption became stronger, weaker, or false — and what decision that changes.
- Delivered
- Integrated, demonstrable behavior. Not activity, not percentages.
- Flow
- What is aging or blocked, and where the team should swarm.
- Risk and dependency
- Which signal changed, the owner, the action, and whether it escalates.
- Decisions
- Made, waiting, and the dissent carried. Who owns the next evidence, and when.
- Forecast
- Range and confidence, what changed, and the smallest scope trade that preserves the outcome.
- Ask
- One explicit decision or resource, with the options and their consequences.
Continue when outcome movement and guardrails support the hypothesis. Change when the opportunity remains but the evidence rejects this mechanism. Stop when the opportunity is too small, the intervention cannot move it economically, a guardrail cannot be protected, or strategy changed. Stopping after cheap evidence is capital allocation, not team failure.
- Outcome and guardrails reported against baseline, including the segments that did not benefit and any cost that moved elsewhere.
- Steady-state ownership accepted by name: service, on-call, vendor, data, support knowledge, roadmap.
- Release flags, temporary adapters, manual pilot paths, elevated access, pilot dashboards, and test data removed — or each recorded as debt with its interest and a repayment trigger.
- Architecture views and contracts updated; superseded decisions superseded rather than rewritten; fitness functions still running.
- Learning review completed: expected, observed, surprised, next time. At most two improvements, each owned and dated.
- Continue, change, stop, or hand over decided explicitly and communicated.
Completed example · the week the evidence arrived, and the closeout
- Outcome
- Not yet measurable — no customer is exposed. First reading due at the 1% stage. Reporting “on track” here would be a claim with no source.
- Guardrails
- One moved: the manual-queue projection is now 34 referrals/day at full exposure — 68% of the 50/day cap — against a week-1 estimate of “about 12”. The headroom is nine a day, not thirty-eight.
- Learned
- The carrier is far slower and less safe under retry than its documentation says. The referral volume implied by the policy sample is nearly three times the week-1 estimate. Customer comprehension remains unknown: the n = 5 test was inconclusive, not positive.
- Delivered
- The walking skeleton runs end to end from a clean build, including the worker and correlation ids across it. The 200-case policy suite runs; 186 of 200 pass, and all 14 failures are gift orders — a missing rule, not a defect.
- Flow
- Two slices in progress, at the WIP limit. The appeal flow has not moved in three days. It is the item the swarm rule applies to, and the item the scope trade below removes.
- Decisions
- Two made today. One waiting: whether the gift-order rule enters a later slice or is refused for the pilot, with a named decider and a date.
- Forecast
- 1% exposure between 26 September and 3 October, about 70% confidence — down from about 80% last week, because the worker is six days that were not in last week's plan. Today's scope trade restores about five of them.
- Ask
- One decision on log retention by Monday, because the audit trail's design depends on it. No staffing change requested.
Three results, three different responses. The carrier spike reversed a decision: ADR 0002 superseded by ADR 0003, with every affected artifact updated the same day — the ADR log, the container view, a new dynamic view, the carrier-failure example, the delivery map, and the rollout contract. The referral volume model re-ordered the plan, taking the scope trade the week-1 forecast had pre-identified; the designer dissented, and the threshold that would prove him right was instrumented before exposure opened, adding a telemetry measure whose only job is to adjudicate a disagreement. And the comprehension test changed nothing, on purpose: two of five is too small a sample to separate a wording problem from two participants, so the wording changed (cheap and reversible) and a retest was commissioned, but no expensive decision moved. A review in which every piece of evidence conveniently supports the existing plan is a review that is being performed rather than run.
| Closeout, after an eight-week pilot | Result |
|---|---|
| Median submission → decision communicated | 3.1 h exposed against 69 h in the holdback control; target under 4 h. n = 6,480 exposed, 694 control. Met. |
| Support-contact rate within 7 days | 31.6% exposed against 40.2% control — a 23% relative reduction against a 25% target. Missed by 0.6 points, and reported as a miss rather than rounded into a win. |
| Fraud loss per 1,000 eligible returns | 2.2 exposed against a 2.4 baseline. Held on the agreed measure — but the control cohort of 694 requests contains fewer than two expected events, so it could not have detected a breach of any plausible size. A defect in the evidence, not in the system, and the first thing the steady-state owner should fix. |
| Manual review queue · carrier cost · accessibility · retention · duplicate labels | Mean 26/day, peak 41 · $3.39 against a $3.42 cap · zero open AA issues at the external audit, two of the three found by manual keyboard testing rather than the automated check · zero records past 30 days · zero duplicate labels in 6,480 returns. |
| Who did not benefit | Guest customers, 29% of demand, got nothing — they were never in scope, and the outcome is quoted with that clause attached everywhere. Refused customers contacted support at 47%, above the overall baseline: being refused quickly is still being refused. Mobile started 11 points below desktop and closed to 4. |
| Cost that moved elsewhere | Front-line contact volume fell; specialist referral work rose to 26/day. Work moved rather than disappearing, and two agents were rostered permanently to carry it. |
| Machinery removed | The release flag and its fallback branch deleted with both configurations' tests. The holdback machinery deleted — a holdback nobody reads is a cohort receiving a worse product for no reason. Elevated access revoked. The label-attempt disable flag kept, because it is an operational control rather than release machinery. |
| Debt kept deliberately | Manual warehouse receipt confirmation. Its removal trigger had already fired and was not acted on. Interest: about 40 clerk actions a day, roughly 1.5 hours of someone's day, indefinitely. Owner named, reviewed quarterly. The decision it came from was not superseded, because superseding it would have hidden the debt. |
| Learning | Expected speed to drive the contact reduction; observed that visibility did. Expected policy to be 90% codifiable; it was 63%, and the gap was undocumented rules held by two people. Expected the carrier to be the easy integration; it reshaped the architecture. Expected the disable flag to work because it existed; it half-worked. Two improvements owned and dated — buy a domain sample before writing an outcome statement, and power-check every guardrail at design time. |
| The decision | Hand over and continue in a new initiative. Not “phase two”: the remaining work needs a different fraud model and a different cohort, and continuing under this frame would have let the outcome statement drift to cover work it was never evidence for. |
Full review page and closeout checklist: weekly-review-and-closeout-checklist.md.
Turning your own filled-in pack into evidence
A completed pack is the raw material for a promotion case, an interview narrative, or an architecture review — but only after the confidential parts are removed and the reasoning is separated from your employer's implementation detail. The portfolio guide covers anonymization, a one-page-per-initiative portfolio template, an interview narrative, executive and technical-review versions of the same decision, and how to describe your own contribution without overclaiming your team's work.
- Take the last initiative you led or worked on. Which of the thirteen artifacts existed in some form, even informally? Which decision went wrong in a way one of the missing ones would have caught?
- Run the traceability test backwards on a document your organization produces every time. Which outcome, assumption, driver, risk, or rollout decision does it support? If you cannot answer, you have found the first thing to delete.
- Find one number in your current plan that has no source, and one “must” that has no owner. Those two circles are your next discovery backlog.
Related: Initiative Playbook (the same material as prose) · worked capstone (a complete packet, plus an intentionally weak version and reviewer annotations) · capstone rubric (how a reviewer scores it).