Decision Quality Is Not a Scorecard

Decision quality is the degree to which a decision rests on assumptions that have been surfaced, tested for significance, and either accepted or reduced. It is not a maturity model or six elements on a whiteboard. It is the discipline the Universal Decision-Making Method produces when followed honestly: knowing what your decision depends on and whether you know enough to proceed.

Decision quality is the state of a choice resting on assumptions you have made visible and checked before you commit to it.

What decision quality actually means

The phrase has been captured by a cottage industry. Stanford's Strategic Decisions Group, now SmartOrg, popularised a framework of six elements: appropriate frame, creative alternatives, relevant information, clear values, sound reasoning, commitment to action. Clean model. Useless in the room where the decision is being made.

I, Grant Purdy, have spent fifty years advising organisations on consequential decisions. Not one failed because the Decider lacked a six-element checklist. They failed because the assumptions underneath went unexamined. The frame looked right. The alternatives seemed reasonable. The information appeared sufficient. Nobody asked: "What is this decision resting on, and do we know enough about those things to proceed?"

That question, and the discipline of answering it, is what decision quality means in practice. It is a live test applied before the commitment is made, not a retrospective score or an audit trail. It begins with Purpose: what is this decision meant to achieve? Until that is clear, every evaluation of quality is measuring the wrong thing.

The apparatus that replaces judgment with documentation

The decision quality industry has produced an impressive apparatus. Risk committees with their own Chief Officers, maturity assessments grading organisations from "initial" to "optimised," and risk registers cataloguing uncertainty in spreadsheet form. ISO 31000 alone contained 29 special labels for ordinary words, as though renaming the furniture would change what happened in the room.

None of this changed how decisions were actually made. I have put the following question to clients struggling to make the machinery work: "If risk management is the answer, what was your question?" The response is genuine puzzlement, followed by the slow realisation that they had been starting with an answer and trying to fit it to an undefined problem.

The apparatus is a belief system. Organisations adopt it because other organisations adopt it, because regulators require it, because consultants sell it, and because visible compliance is safer than admitting nobody knows whether it helps. The groups sustaining the industry, regulators and consultants chief among them, each pull the levers available to them. More consulting work preparing for certification; more work fixing where the client fell short. Not a virtuous circle, but certainly a lucrative one. Most decision-making frameworks were not designed to test whether the original rationale still holds. They were designed to document that the process was followed.

Click to expand

Enron Corporation was the seventh-largest company in the United States by early 2001. Within months, its stock dropped from $100 to cents and the company went bankrupt. Enron had a risk management committee, a Chief Risk Officer, formal risk reporting, and external oversight from Arthur Andersen, who had praised Enron's risk management. Arthur Andersen also collapsed. The full apparatus was in place and it detected nothing, because it was not designed to ask whether the assumptions underlying the business model were sound. It was designed to demonstrate that the process of documenting risks had been followed.

Boeing's 737 MAX program killed 346 people across two crashes. Both the manufacturer's safety management system and the FAA's delegated oversight had the complete machinery: committees, reports, assessments. Neither had a process to surface the assumption that a single-sensor MCAS system was adequate, or that pilots could override it within the time available. The apparatus certified the process while the untested assumption killed the people.

Australia's banking Royal Commission found that institutions with risk committees, Chief Risk Officers, risk appetite statements, and formal risk reporting had been charging customers for services never provided. The apparatus failed across every institution. And none of the organisations I work with reached for their risk register when they had to decide how to respond to COVID-19. None of them consulted their risk appetite statement, either.

Where decisions actually go wrong

Decisions fail through specific mechanisms, not general carelessness. Two do the most damage in organisational settings; a third compounds them both.

Anchoring is what happens when a model, a manual, or a prior commitment becomes the decision. In 2023, Silicon Valley Bank's management anchored on an interest-rate model that assumed low rates would persist, even as the Federal Reserve signalled aggressive tightening. The model was the process and the process was the investment; within days the bank collapsed. In 2008, an internal TEPCO study estimated a 15.7-metre tsunami against the Fukushima plant's 5.7-metre design basis. The study was suppressed because it conflicted with the existing model. Three years later, the tsunami arrived.

Framing determines what you see before you start looking. In 2014, Flint, Michigan framed its water-supply decision around cost savings rather than public health. The frame determined what was measured (budget line items) and what was ignored (lead contamination rising to dangerous levels). Nobody asked whether what the city was doing matched what it existed to do. In 2022, the FDA shut down Abbott's infant formula plant on contamination concerns, chasing maximum certainty on one dimension. The result was a nationwide shortage that harmed the consumers the shutdown was meant to protect: maximum certainty on safety detracted from the overarching Purpose of feeding infants.

Confirmation bias is quieter but equally destructive. Boeing's own safety management system confirmed existing assumptions about design adequacy rather than testing them. The FAA's delegated oversight confirmed the manufacturer's conclusions. Each layer of the apparatus confirmed what the layer below had already assumed. Nobody asked what the decision rested on; they asked whether the documentation was complete.

Five steps that produce decision quality

The Universal Decision-Making Method that Roger Estall and I developed addresses each quality failure through its structure, not through additional apparatus bolted on top.

Frame the decision. Not the project, not the initiative: the decision. What is its Purpose? What outcome would serve the organisation? Most organisations skip this and wonder later why the answer did not help. The hospital exists to improve patient outcomes. The statutory body exists to fulfil its legislative function. If the frame is wrong, everything downstream is noise, and no amount of risk-register entries will unpoison it.

Develop options. Not "go or no-go," which is a forced binary that frames rejection as failure. Two or three genuine alternatives, each tested against the Purpose. Boards love a single recommendation with a green light; that is not deciding, it is ratifying. Multiple options prevent premature commitment to the first idea that seemed adequate.

Recognise assumptions. This is the core act. Every option rests on assumptions about the future. "The market will recover." "The model is valid for these conditions." "The manual covers this scenario." Surface them, rank them by influence and confidence, and the critical ones become visible. Even facts carry an implicit assumption that they will remain facts over the life of the decision. The classification Grant Purdy and Roger Estall set out in Deciding makes this concrete: Critical assumptions sit where influence is high and confidence is low. Those are where the work happens.

Most of those assumptions in decision making are mistaken for facts until someone asks two questions.

Sufficient certainty. The FDA's Abbott shutdown chased maximum certainty on contamination and caused a nationwide formula shortage that harmed the infants it was meant to protect. The question is not "are we certain?" but "have we identified the assumptions that matter, assessed their significance, and either accepted them or reduced them?" If yes, decide. If no, you have specific work to do. That work is testing assumptions, not "analysing more."

The stopping rule for having enough information to make a decision is assumption coverage, not data volume.

Design monitoring. Specify what to watch before the decision is implemented, when the Decider still has the assumptions in view. Two failure modes matter most: conditions changing after the decision is made, and implementation diverging from intent. The Brisbane dam, Enron, and Boeing cases all demonstrate monitoring failure, or more precisely, the absence of anything worth monitoring.

Sufficient certainty is the quality test

Decision quality is commonly measured by outcome. This is backwards. A sound decision can produce a poor outcome when unforeseeable conditions change. A poor decision can produce a good outcome through luck. Evaluating quality by outcome confuses the lottery with the ticket-buying strategy.

A good decision can produce a bad outcome and still be the right call. The quality test was always in the assumptions, not the ending.

The test that matters is assumption coverage. Were the critical assumptions surfaced? Tested? Accepted with eyes open, or reduced through investigation? Was monitoring specified for the ones that could change?

NASA's handling of the Starliner thruster anomalies in 2024 is the test working. Billions invested in the program. Two astronauts in orbit. Thruster data that did not demonstrate sufficient certainty about safe re-entry. NASA chose to bring its astronauts home on a different vehicle. The sunk cost did not enter the decision. That is what the discipline looks like: the willingness to ask what the decision rests on, even when the answer is expensive.

If decision quality is not a maturity score, how do you know whether your organisation produces it?

The decision record names the critical assumptions, not the risk register or the business case but the record of what was actually decided, what it depended on, and what would trigger a review. If the record does not contain these, the decision was not made with sufficient certainty. It was made with sufficient paperwork.

Monitoring is specified before implementation, not retrofitted after problems emerge. When an organisation designs monitoring at the point of decision, it demonstrates that the Decider understood what the decision rested on. When monitoring appears only after something goes wrong, it demonstrates the opposite.

The decision autopsy asks "what assumptions failed" rather than "what went wrong." A post-mortem focused on outcomes teaches you nothing about the quality of the process. A review focused on assumptions, whether they were identified, tested, and monitored, teaches you everything. I have written about what standard post mortem analysis misses and the one box every template leaves blank.

The same blind spot runs through the standard after action review: the meeting replays the chronology and grades performance, but the original approval stays untouched because reopening it would mean questioning the people who signed it. The archive grows, but the same bad assumption reappears in the next commitment. I have written about why lessons learned fail when nobody traces the outcome back to the belief that produced it. The same assumption goes unexamined when an organisation treats its existing model as settled fact; innovation decisions stall because the incumbent never had to pass the test the new idea faces.

---

Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.