Assessment · Phase IV
Phase 4 assessment · Initiative Lead
A new brief from the same company, so nothing here can be answered by remembering the returns case. Six scenarios, then three artifacts: a frame, a decision record, and a delivery plan whose milestones are evidence rather than output. Lessons 0039–0050.
The three tasks are one initiative, in order: frame it, decide the hard thing, plan the delivery. Do them in sequence and your Task 3 will depend on your Task 1 — which is the point. If the dependency does not appear, the frame was not doing any work.
Part 1 · Six scenarios
Somebody writes into the frame: “customers will accept a self-service decision without speaking to an agent.”
In the initiative frame, that belief belongs under:
A discovery workshop opens with the tech lead describing the design they have in mind, then asking the room for input.
What has that opening cost the session?
Which of these is a vertical slice of the returns pilot?
A draft driver: “when the carrier API is unavailable, the system accepts eligible returns and issues labels once the carrier recovers.”
What must be added before this driver can be tested?
One diagram shows the customer, the returns service, its PostgreSQL database, the
EligibilityPolicy class, the carrier, and a Kafka topic.
The problem with that diagram is:
The pilot is live for 5% of eligible customers. The dashboard shows error rate, p95 latency and CPU, all green.
The gate cannot be decided, because the dashboard is missing:
Part 2 · Three written tasks
One initiative, three artifacts, roughly eighty minutes. Blank templates for all three are in the artifact pack.
Task 1 · The brief becomes a frame
What to write
Turn the brief below into an initiative frame: problem with a baseline, outcome with a number and a date, guardrails, first slice, non-goals, constraints, unknowns, and decision owners. About twenty-five minutes, one page.
Then do the part that matters most: circle every number without a source and every “must” without an owner. List what those circles become — they are your first discovery backlog.
The brief, as it arrived
“Let customers change their delivery address and remove items from an order before it is dispatched. Sales says we are losing orders because people cancel outright rather than contact support. Ship it this quarter.”
What you also know: the order system owns order truth and is not yours. Dispatch runs in three waves a day from two warehouses. Support handles roughly 300 amendment calls a week; nobody has measured how many end in a cancellation. Payment is captured at dispatch, not at checkout.
Self-review — answer these before you open the reveal
- Read your outcome sentence. Does it describe a change in someone’s behavior, or the existence of a feature? “Customers can amend orders” is an output.
- Do you have a baseline? If not, say so explicitly and put “measure it” in week one, rather than writing a target that nothing can be compared against.
- Every guardrail should be a measure with a threshold. Count how many of yours are adjectives.
- Your first slice: which single customer, in which single situation, gets the whole path? If it names a component, re-cut it.
- How many circles did you draw? A frame from a brief this thin with fewer than five unsourced claims has been written too confidently.
- Payment is captured at dispatch. Did that fact change anything in your frame, or did you copy it into constraints and move on?
Now score what you actually wrote, not what you meant. Each dimension links to its level descriptions.
Use of measurable architecture drivers
Product, design, and engineering collaboration
Technical communication and decision clarity
Not scored yet.
Reveal: a strong answer, a weak one, and the difference
Strong
Problem. Customers who want to change an order before dispatch have one route: contacting support. About 300 do so weekly. We do not know how many cancel instead of asking, and that number is the whole premise — it is unsourced, and it is circled.
Outcome. For signed-in customers with an undispatched order in the UK, reduce order cancellations attributable to an unmet amendment request by 40% within eight weeks of pilot start. Baseline: unknown. Week one is a support-ticket sample of 100 amendment contacts, coded for what the customer wanted and what they did next. If the honest answer turns out to be 30 orders a quarter, this initiative should stop, and it is cheaper to learn that in week one.
Guardrails. No order amended after its dispatch wave has picked (zero tolerance; each occurrence is an incident). Payment capture stays correct to the cent. Support contacts about orders must not rise. Amendment-related manual interventions stay under 20 a day. WCAG 2.2 AA on the amendment flow.
First slice. A signed-in UK customer changes the delivery address on an order that has not yet entered a picking wave, from one warehouse, and sees the new address confirmed. Address only — not item removal, which touches payment.
Non-goals. Item removal in the first slice; adding items; changing payment method; amendments after picking; international orders; anything in the warehouse’s own systems.
Constraints. The order system owns order truth and belongs to another team; we integrate, we do not fork. Dispatch is three waves a day from two warehouses, so “before dispatch” is a discrete, knowable moment rather than a continuous one — that fact is the most useful thing in the brief. Payment is captured at dispatch, which means an address change before capture is cheap and an item removal before capture changes the amount to capture.
Unknowns, in priority order. (1) How many cancellations are really caused by this — the premise. (2) Whether the order system can expose the picking state early enough to be useful, and how the other team feels about being asked. (3) What proportion of amendment requests arrive after picking has begun, because if it is most of them, the product is a notification rather than an amendment. (4) Whether removing an item before capture is a change or a partial cancellation in finance’s vocabulary.
Decision owners. Scope and outcome: product. Experience: design, with support as a stakeholder. Architecture and the integration contract: me, with the order team’s lead as a required reviewer — they can veto anything touching their write path. Rollout: me and the on-call owner jointly. Evidence that reopens scope: unknown (1) landing below 100 orders a quarter.
The circles. “We are losing orders” (no measurement). “This quarter” (no owner named it, and no evidence says why). “300 calls a week” (source is a support lead’s recollection; needs the ticket query). “Before dispatch” (nobody has said whether that means before the wave, before picking, or before the van leaves — three different systems). Four circles, four discovery tasks, all of them cheaper than a week of building.
Weak — and plausible
Problem: customers cannot amend orders and have to call support. Outcome: ship a self-service order amendment feature this quarter that lets customers change their address and remove items before dispatch, reducing support calls and improving customer satisfaction. Guardrails: the system must be secure, performant and accessible, and must not break the order system. Scope: address changes and item removal for all customers. Constraints: the order system owns order truth; payment is captured at dispatch. Assumptions: customers want this. Owners: the whole team decides together.
The outcome is the brief with a verb attached. “Ship a feature… reducing support calls” names no baseline, no target, no cohort and no date, so on 30 September nobody can say whether it worked. The strong answer commits to a number it does not yet have, and says where the number will come from — which is the honest version of not knowing.
Three adjectives are standing where guardrails should be. Secure, performant and accessible cannot get worse in any measurable sense, so they cannot protect anything. “Zero orders amended after their wave has picked” can, and it is the one that would stop a release.
“The whole team decides together” means nobody decides. It also hides the decision that actually matters here: the order system belongs to another team, so somebody outside this initiative holds a veto. Naming that early is what turns it into a dependency with a date instead of a surprise in week six.
The strong answer’s best move is the smallest one: noticing that dispatch runs in three waves, so “before dispatch” is a discrete moment. That single observation turns a race condition into a deadline, and it came from reading the constraints rather than the feature list.
Task 2 · Two options, one spike, one decision record
What to write
The hard part of the amendment initiative is the race: an amendment can arrive while the warehouse is picking. Three credible designs:
- Lock. The amendment flow asks the order system to lock the order, applies the change, unlocks. Requires a new write operation from a team that is not yours.
- Optimistic with compensation. Accept the amendment, apply it, and if the order was already picked, compensate: intercept at packing, or let it ship and offer a free return.
- Hard cut-off. Amendments close 90 minutes before each wave. No race exists, because the window closes before picking starts.
In about thirty minutes: compare them against drivers you name; identify the single largest unknown; write a spike brief for it (question, timebox, evidence format, discard rule); then write the one-page ADR — context, forces, options, decision, consequences, dissent, revisit trigger.
Self-review — answer these before you open the reveal
- Are your drivers measurable, or are they adjectives? An option table scored against “simplicity” has been scored against taste.
- Which option loses on your top driver and wins on your third? If none does, the drivers are not ranked yet.
- Your spike: could someone else run it and get a number you would both accept? If it needs your judgement to interpret, it is not a spike, it is a design session.
- Does your spike have a discard rule? Say what happens to the code afterwards, in writing.
- In the ADR, is the consequence section written from the point of view of the people who carry it — the warehouse, support, the on-call engineer — or only from the code’s?
- Is there a dissent line? If everybody agreed, either the options were not different enough or somebody stayed quiet.
- Your revisit trigger: a number, a threshold, an owner. Not “if it becomes a problem”.
Now score what you actually wrote, not what you meant. Each dimension links to its level descriptions.
Quality of alternatives and trade-off reasoning
Use of measurable architecture drivers
Technical communication and decision clarity
Product, design, and engineering collaboration
Not scored yet.
Reveal: a strong answer, a weak one, and the difference
Strong
Drivers, ranked. (1) No order ships to the wrong address — zero tolerance, it is a guardrail. (2) Amendment succeeds within 5 seconds at p95, because the customer is waiting. (3) No new synchronous dependency on the order team’s write path before the pilot, because their next available slot is in six weeks. (4) Warehouse process unchanged in the pilot.
| Option | Buys | Costs | Against the drivers |
|---|---|---|---|
| Lock | correctness with no compensation logic | a new write operation from another team; lock timeouts become our incidents | wins (1), fails (3) outright — six weeks we do not have |
| Optimistic + compensation | no dependency; widest amendment window | a packing interception or a free return; warehouse process change; support script | wins (3), risks (1), fails (4) |
| Hard cut-off | the race cannot happen; no new write path; nothing changes downstream | customers who ask inside 90 minutes get nothing; the window may cover most requests | wins (1), (3), (4); loses on coverage, which is not a driver but is the outcome |
Largest unknown. Not the mechanism — the distribution. What proportion of amendment requests arrive inside the 90 minutes before a wave? If it is 5%, the cut-off is obviously right and the other two options are engineering we do not need. If it is 60%, the cut-off ships a product that says no to most people who use it, and the compensation option becomes worth its cost. Everything else is downstream of that one number.
Spike brief
Question: what proportion of support amendment contacts arrive within 90 minutes of the order’s next dispatch wave, and how does that split by warehouse and by weekday?
Timebox: one day, one analyst plus two hours of an engineer.
Evidence format: a histogram of contact time minus wave time for the last 8 weeks of amendment tickets, and the single percentage inside the window.
Discard rule: the query is pasted into the ADR; the notebook is deleted. No code from this spike goes near the product.
Decision rule agreed in advance: under 20% inside the window, take the cut-off and stop. Over 40%, the cut-off cannot carry the outcome and we choose between lock and compensation. Between 20% and 40%, ship the cut-off for the pilot and put compensation on the roadmap with this number attached.
ADR 0007 · Close amendments 90 minutes before each dispatch wave
Status: Accepted, 14 May, pending the spike result. Owner: returns and amendments tech lead.
Context: Amendments race with picking. Dispatch runs in three waves a day from two warehouses. The order system belongs to another team whose next integration slot is six weeks out; the pilot is due in four.
Forces: zero wrong-address shipments; 5 s p95; no new dependency on another team’s write path before the pilot; no warehouse process change during the pilot.
Options: order lock; optimistic with compensation; hard cut-off. All three were costed; the lock was rejected on schedule rather than on design, which is worth recording because it may become the right answer later.
Decision: hard cut-off at 90 minutes, with the closed window shown in the interface as a countdown rather than as a refusal after the fact.
Consequences: Engineering: no distributed transaction, no compensation path, roughly two weeks saved. Customers: some requests get a “too late” that a human could have honored; design owns making that non-hostile, and support keeps its manual override. Operations: nothing new on call. Organization: the order team is not blocked on, but is informed, because we now depend on their wave schedule being accurate. Finance: unchanged, since payment capture is untouched.
Dissent: the support lead believes the 90-minute window will feel arbitrary and that most calls come late. Recorded; the spike measures exactly this, and if the answer exceeds 40%, this record is superseded rather than argued about.
Revisit when: requests refused by the cut-off exceed 15% of amendment attempts for two consecutive weeks, or the order team offers a lock operation. Owner: the tech lead, checked at the weekly review.
Weak — and plausible
I recommend the optimistic approach with compensation, because it gives customers the best experience — they can amend right up until dispatch — and it does not require us to depend on the order team, which would slow us down. The lock option is too coupled and introduces a distributed transaction, which is an anti-pattern. The hard cut-off is the simplest but provides a worse user experience.
Consequences: we will need to handle the case where the order has already been picked, which we can do by intercepting at packing. Revisit if this becomes a problem.
The recommendation arrives before any driver has been named, so the comparison is between adjectives: “best experience”, “too coupled”, “an anti-pattern”. Nothing in it could be checked, and nothing in it could lose.
“We can intercept at packing” is a change to somebody else’s process, written as a subordinate clause. That is the most expensive sentence in the weak answer: it commits the warehouse to a new step, during a pilot, without their having been asked. The strong answer treats “warehouse process unchanged” as a driver precisely so that this cannot happen by accident.
No spike, so the decision rests on a distribution nobody has looked at. One day of a query answers whether the simplest option is also the right one, and the strong answer commits to the thresholds before seeing the number — which is what stops the result being read to suit the design already chosen.
“Revisit if this becomes a problem” is not a trigger. Nobody owns it, nothing measures it, and by the time it is a problem the decision has been load-bearing for a year.
Task 3 · The walking skeleton and the slice list
What to write
Continue the amendment initiative. In about twenty-five minutes write:
- The walking skeleton: the thinnest end-to-end path that runs in a production-shaped environment, and what it proves that nothing else can.
- Four to six slices, in order. For each: the assumption or risk it retires, and the evidence that says it is done. Not “code complete”.
- The dependency map: what you need from outside the team, when you need it, and what you do if it is late.
- One forecast with a range, and the single variable that would move it.
Self-review — answer these before you open the reveal
- Does your skeleton actually deploy? A skeleton that runs on a laptop proves the code and none of the integration, which is where the risk was.
- For each slice, write the evidence sentence. If it is “the feature works”, you have written a task list with milestones in it.
- Which slice retires the biggest unknown? If it is not near the front, say what forced it later — a dependency, a person, a season — and whether that is really true.
- Your dependency on the order team: what is the fallback, and on what date does the fallback have to be triggered rather than hoped past?
- Your forecast range: is it a range because you widened a single number, or because you named a variable that could genuinely go either way?
- Where does design work sit in your slice list? If it is absent, the flow is being designed by whoever writes the endpoint.
Now score what you actually wrote, not what you meant. Each dimension links to its level descriptions.
Delivery sequencing and uncertainty reduction
Product, design, and engineering collaboration
Operational ownership and initiative closeout
Testability, observability, security, and reversibility
Not scored yet.
Reveal: a strong answer, a weak one, and the difference
Strong
Walking skeleton. A signed-in customer opens an existing order in the real staging environment, sees a real countdown to the real next wave (read from the order system’s actual schedule), submits an address change, and the change appears in the order system’s staging instance. One customer, one warehouse, address only, no validation beyond the country check, no design polish. It proves the two things nothing else can: that we can read wave timing early enough for a countdown to be truthful, and that our write is accepted by a system we do not own. Both are integration facts, and both are cheaper to be wrong about in week one.
Slices.
| Slice | Retires | Evidence it is done |
|---|---|---|
| 0 · The measurement | the premise: are we losing orders to unmet amendments at all? | 100 coded support tickets and one percentage, reviewed by product and support together |
| 1 · Skeleton, above | can we read wave timing and write an address to a system we do not own? | an address changed end to end in staging, and the round-trip latency measured |
| 2 · The window, in front of ten real customers | is the 90-minute cut-off comprehensible, or does it read as arbitrary? | ten moderated sessions; the count who could say when their window closes |
| 3 · Address change, one warehouse, 5% of traffic | does the race really not happen, and does anyone amend at all? | two weeks with zero post-pick amendments, and an amendment volume worth continuing for |
| 4 · Both warehouses, all UK traffic | does the wave schedule hold for the second warehouse, whose waves are not fixed? | countdown accuracy within 5 minutes across a fortnight of real waves |
| 5 · Item removal | does removing an item before capture reconcile cleanly with finance? | a fortnight of amended orders reconciled to the cent by finance, not by us |
Slice 0 can stop the initiative, which is the point of putting it first. Slice 5 is last because it is the only one that touches money, and every earlier slice makes it cheaper to get right.
Dependency map. From the order team: read access to wave schedules (needed for slice 1; they have offered week 2; fallback is polling the existing dispatch export, which is 15 minutes stale — acceptable for a 90-minute window, and the fallback must be triggered by the end of week 3 rather than waited on). From design: the countdown pattern and the too-late state, needed for slice 2, and it is the input to slice 2 rather than a decoration afterwards. From finance: the reconciliation rule for a removed item before capture, needed for slice 5, requested in week 1 because their turnaround is three weeks and that is why slice 5 is last, not because it is hard.
Forecast. Address amendment live for UK traffic in 7 to 10 weeks. The variable is slice 4: whether the second warehouse’s waves are predictable enough for a truthful countdown. If they are not, the countdown becomes a conservative fixed cut-off, which costs coverage and no time; if they are unpredictable and product refuses the coverage loss, the estimate goes to 14 weeks because we then need the order team’s lock after all. That is the one variable, and it is knowable in week 4.
Weak — and plausible
The walking skeleton is a basic version of the amendment flow: a simple API endpoint and a form that can change an address. Then the slices are: (1) build the amendment service and data model, (2) build the front-end flow, (3) integrate with the order system, (4) add item removal, (5) testing and hardening, (6) launch. Each slice is done when the code is complete and reviewed. We depend on the order team for API access, which we will request early. Estimated at 8 weeks.
The slice list is a build order, and the risk is in slice 3. Integrating with a system you do not own is the thing that can invalidate everything before it, and it is scheduled after two slices of work that assume it will go well. The strong answer puts the integration in the skeleton, in week one, because that is the week in which being wrong is cheap.
“Done when the code is complete and reviewed” is a milestone with no information in it. After slice 2 nobody knows anything they did not know before. Evidence milestones are what turn a plan into a way of finding out you are wrong.
There is no slice 0. The brief’s premise — that orders are being lost — is unmeasured, and the weak plan spends eight weeks before anyone checks it. One day of ticket analysis is the highest-value work in the whole initiative, and it is the work most likely to be skipped because it does not look like building.
“Which we will request early” is not a dependency plan. No date, no fallback, no trigger. The strong answer names the stale-export fallback and the day it must be chosen — which is what stops a dependency from quietly becoming a delay.
Part 3 · What to practice next
| If you missed | What it points at | What to practice next |
|---|---|---|
| p4q1 or p4q2 | Framing and facilitation — separating beliefs from constraints, and protecting divergence. | 0039 and 0040. |
| p4q3 | Slicing — component completion still feels like progress. | 0042, then 0047. |
| p4q4 | Quality-attribute scenarios — a driver without a measure decides nothing. | 0043, then the lab’s fitness functions. |
| p4q5 | Architecture communication — one diagram trying to serve three audiences. | 0044, then the technical-influence supplement. |
| p4q6 | Telemetry that measures the system and not the outcome. | 0049 and 0050. |
| Task 1 scored below Practicing | Outputs written as outcomes, or adjectives standing in for guardrails. | 0039 and the playbook’s frame template. |
| Task 2 scored below Practicing | Options compared on taste, or a spike with no discard rule and no decision thresholds. | 0046, then the decision-change simulations. |
| Task 3 scored below Practicing | A build order wearing slice numbers, or dependencies with no fallback and no trigger date. | 0047 and lab checkpoint 4. |
Exit capability for Phase IV. Given a feature-shaped brief, you can produce a frame somebody outside the initiative could argue with, name the decision that is expensive to reverse, buy evidence for it with a timeboxed spike whose thresholds were agreed in advance, and sequence delivery so the largest unknown is retired first. The worked capstone is the full version of all three on the returns initiative — read it after you have written your own, not before.
Next: the Phase 5 assessment, after lessons 0051–0060, which designs a system from the domain outward.