Rank the fix, not the plausibility
When a question describes a system that keeps producing a bad outcome, every option will be defensible. Sort them by how far upstream they act:
- Removes the cause. Restructure the flow, change the default to fail-closed, create a canonical upstream artifact, fix the protocol or the boundary. This is almost always the credited answer.
- Catches it downstream. A repair pass, a review step, a reconciliation job. This is the instinct most working engineers reach for, and it is the most common wrong answer on the exam.
- Makes it rarer. Retries, tighter timeouts, more alerting, a bigger retrieval window.
- Documents it. A changelog, a disclaimer, a rationale log.
Before you commit, ask one question: does my answer still let the failure keep being generated? If it does, look again. Something higher in the list is usually sitting right there.
This applies to architecture questions as much as governance ones. On pure design items, "add a stage" is nearly always the wrong shape. The credited answer tends to move or remove a stage instead: a durable queue with a dead-letter path rather than retries with backoff, one structured-output call rather than a chain of twelve calls stitched together for consistency.
Step zero: is anything actually failing?
The ranking only applies when the question describes a failure that keeps happening. If the problem is anticipated rather than occurring — future scale, a competitor did it, someone is worried — then the simplest design wins and the impressive-sounding restructure is the trap.
The numbers in the question pick the side. Nine stable tools with two changes a year does not need a discovery layer. Forty-five tools with dead weight does. Read the numbers before you reach for a pattern.
Read the last sentence first
These questions front-load context. You get a paragraph of scenario and then, at the very end, the actual ask. Read that final sentence first, then read the scenario knowing what you are looking for.
It changes what you extract. Who consumes the output. Which numbers matter. Whether the question wants a diagnosis, a fix, or a legal obligation. Read in order and the context steers you before you know what you are steering toward.
Match the answer to the question that was asked
"Why is X inadequate?" wants the mechanism by which X fails. An option proposing a better alternative is structurally wrong before you evaluate whether it is a good idea, because it does not answer the question. The same holds for combined "diagnosis and fix" items: a wrong diagnosis sinks an otherwise workable fix.
Mechanism beats human diligence
If an option only works when people voluntarily do something — re-run a script, read a changelog, heed a confidence score, respond to an alert — it loses to the option that makes the right behavior automatic and verifiable.
The sharpest version of this: never accept a model's self-report as a control. A model's stated confidence or its own rationale is output, not oversight.
Safety-critical means fail-closed
When a guardrail exists to prevent harm, the credited answer is no verdict, no release: degrade to a safe fallback and stop. Retries, alerts and caches are availability engineering. They change how often the guardrail is unavailable. They never change what happens while it is.
Related, and easy to miss: a pre-registered guardrail metric that regresses significantly blocks a rollout outright. Extending the test window, blaming external factors, and shipping to a smaller percentage are all ways of shipping the harm more slowly.
The exception: when the review step is the broken thing
The ranking downranks review steps, so candidates learn to reject them reflexively. That is a trap of its own.
If the question describes a rubber-stamp review — approval rates near 100%, review times collapsing — then fixing the review is cause removal, not a downstream patch. Blind-first review on risk-routed cases, audits with planted defects, independent context for the reviewer. Do not auto-reject review-shaped options on governance items.
ROI means net unit economics against a measured baseline
Gross savings, meaning time saved multiplied by wage, is a planted trap and it appears often. So is any single line item, like token spend.
The credited answer compares the full loaded cost of the new process — inference, plus the review that still happens, plus maintenance — against a baseline somebody actually measured, with sensitivity analysis on the assumptions.
The same discipline applies when defending a cost increase. Lead with cost per outcome, not usage. "Spend rose because volume went from 900 to 2,100 while unit cost fell from $240 to $95 per report" is an answer. "Usage is up, so it is working" measures engagement and concedes the argument. A hard spend cap with throttling is also wrong, because it makes the pipeline the constraint on revenue.
Patterns need triggering conditions
Routers, discovery layers, and multi-agent fan-outs earn their complexity at scale. If the scenario's numbers are small and stable, the boring static design wins. Complexity is not a demonstration of expertise, and the exam knows it.
A few that come up more than you would expect
- Retrieval: match small, ground big. Retrieval granularity and grounding granularity are different decisions. Embed and match precise chunks, then expand each hit to its enclosing section before assembling context. Any option that leaves the model with fragments loses.
- Machine-to-machine handoffs get validated at the boundary. When one system's output feeds another automatically, enforce the contract deterministically at the handoff and re-prompt or queue for repair on failure. Better prompts and few-shot examples only lower the error rate.
- A component changed and things broke immediately? The contract broke, not the consumer. Re-validate against the new component and enforce structure mechanically. Patching the parser treats the symptom.
- Latency budgets come from actual consumption. If a report is read tomorrow morning, a four-minute pipeline is free, and a fast provisional path nobody reads is pure cost plus the risk of someone acting on a draft.
- Eval scores are only as good as their labels. If inter-rater agreement sits in the seventies, the eval measures one person's contested judgment. Fix it with adjudicated labels and a tightened rubric, not by collapsing the taxonomy or relabeling more items with the same broken criteria.
- Watch for self-reinforcing label loops. When a system's own decisions generate the outcomes it later trains on, more data entrenches the bias. The fix is ground truth gathered independently, such as randomized audits of the path the system disfavours.
Where the ranking does not apply
Regulatory questions suspend it. "What does the law require?" is a different question from "what fixes the system?", and the legally correct answer is usually an assessment performed before processing plus rights granted to the affected person — exactly the paperwork-shaped option the ranking otherwise pushes down.
Learn the common myth distractors too: data residency mandates that the regulation does not actually impose, regulator pre-approval, consent as the only lawful basis, and anonymise-everything.