Your Data Driven Decision Making Framework Starts Too Late

A chief executive I worked with came into a board meeting with a 42-page pack and a financing decision already deferred twice. The pack was full of numbers, yet the room still had not named the assumption that would decide whether the call was safe. Everyone said they wanted a data driven decision making framework. What they had was a way to delay judgement with evidence.

That distinction matters because most organisations already swim in data. The failure is not a shortage of evidence. It is the absence of a prior question: evidence of what, for which choice, judged against what threshold? Without that question, the team collects whatever is available, formats it attractively, and presents it to a room that still cannot act.

I have spent forty years watching competent people confuse collection with rigour. The pattern is always the same. Someone says we need to be more data-driven. The analysts build a better dashboard. The dashboard arrives, everyone admires it, and the decision still drifts because nobody stated what would count as enough. That pattern repeats every time an organisation tries to build a data-driven culture without first asking what the dashboards are supposed to test.

A data driven decision making framework is a method for deciding which evidence matters to a choice and whether it gives enough certainty to act.

Most Data Driven Decision Making Frameworks Start Too Late

Most of what is sold under this label is a collection routine dressed up as a framework. IBM turns the idea into better reporting. Atlan turns it into a collection sequence. Both sound tidy. Neither answers the only question that matters: what decision is the evidence supposed to support?

Once a framework starts with available data, it has already smuggled in the hard part. It assumes the decision is clear, and it assumes the evidence speaks to that choice. I have watched boards spend weeks refining a report when the real dispute sat inside one hidden bet about timing or customer behaviour. That is the broader problem behind data-driven decision making as it is usually practised: the report grows while the choice stays blurry, which lets people sound careful while avoiding the call.

Mixed evidence is where these frameworks usually collapse. One chart says demand is softening. Another says the margin will recover. The room responds by asking for more collection, because collection looks responsible. It is easier to extend the report than to name the assumption doing the real work and judge whether it is still believable.

A Data Driven Decision Making Framework Begins With the Decision

I start with Frame the decision, because until the room is clear on the purpose and the consequence of being wrong, analysis is just scavenging. The first job is to put the live choice into plain English before anyone opens the spreadsheet. From there, Develop options and Recognise assumptions stop the evidence wandering away from the choice and attaching itself to whatever is easiest to measure.

The AQuA Book at least admits analysis should fit the decision, and GovS 010 warns against false certainty, which is a low bar most vendor frameworks still fail to clear because they mistake collection for competence.

In practice, framing the decision first changes the shape of the evidence the team collects. A utility company I worked with spent three months building a demand model before asking what they were deciding. The answer turned out to be whether to delay a capital programme by eighteen months. Once the decision was plain, only one variable mattered: whether the regulator would accept the revised timeline. The model was elegant and irrelevant. Three months of analyst time produced an artefact that tested nothing the Decider needed tested. That is not a failure of analytics. It is a failure of sequence.

Roger Estall and I wrote Deciding to make competent decision practice explicit, not to add another layer of ceremony. The Universal Decision-Making Method keeps asking the same irritating question: what is this number evidence of? That question sounds simple until someone has to answer it in front of a board or an investment committee. It is also why numbers arrive carrying someone's judgement. Data comes with a source and a definition. If either is shaky, the confidence is borrowed.

A Stopping Rule Turns Data Into a Decision

A data driven decision making framework is useless if it cannot tell the Decider when to act. Most vendor frameworks slide from analysis to decision as if the last chart settles the matter. Real choices do not behave like that. Evidence is often incomplete, and the crucial assumption is often about the future, which means it cannot be proved before the call.

This is where Sufficient certainty matters. I am not looking for perfect information, because perfect information is not on offer. I am asking whether the evidence now on hand makes the outcome certain enough to justify the call. If it does not, stop pretending the answer is one more round of analysis. Change the decision, or go after the assumption still carrying the weight. That is why enough information to make a decision is a better question than how much more data the team can pile up.

Teams often call this rigour. Most of the time it is fear with a spreadsheet attached. The stopping rule matters because it forces somebody to own the remaining uncertainty. Without it, every request for another model or another month of data sounds prudent, even when it is only a way of staying uncommitted.

The OECD's report on public-sector AI gives a clean example from Sweden's labour-market system, where caseworkers found negative recommendations hard to override. Once the override rule is vague, the machine gets the authority and the human keeps the blame. That is what happens when a framework cannot say, in advance, what evidence is enough and who may overrule the tool.

Conflicting data does not excuse indecision. It tells you where the assumption is weak. A real framework makes that weakness visible and forces the room to decide whether the option still stands, or whether the call now needs to change.

Monitoring Belongs Inside the Framework

A decision is unfinished until someone owns the signal that would reopen it. That is why the last step is Design monitoring, and why I treat monitoring as part of the decision, not the tidy-up after it. A data driven decision making framework that ends at approval has abandoned the hard part, because approval is only the start of exposure to reality.

I have seen organisations carry forty metrics into a board cycle and still miss the one signal that mattered, because nobody had tied the numbers back to the assumption they were meant to test. The U.S. Government Accountability Office found the same weakness in federal IT oversight. It reviewed 738 major IT investments and found 160 medium- or high-risk investments where agencies had not fully considered active risks when scoring them. The dashboard existed. The judgement around it was thin.

The common failure is treating monitoring as a compliance exercise rather than an active test. I have reviewed governance arrangements where the board received a traffic-light dashboard every quarter but nobody had written down which light would trigger a re-examination of the original approval. Green meant comfortable, amber meant watch, and red meant argue about definitions. The missing piece was always the same: a named assumption, a signal that would weaken it, and a person obligated to act. Without those three elements written into the decision record at the point of approval, the monitoring apparatus measures activity, not exposure. It tells the board what happened without telling the Decider whether the basis for the original call still holds.

Write down the assumption under test and the signal that would expose it. Name the person who must pull the decision back into the room when that signal turns. Without that discipline, the dashboard comforts observers and gives very little protection to Deciders.

When I hear the phrase data-driven, I want two things on the page: the choice and the bet it depends on. If nobody is named to reopen the call when that bet weakens, the data is not driving judgement. It is giving cover to a guess.

---

Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.