Which Numbers Matter Before You Make the Call? Most Do Not

Most reporting packs carry dozens of numbers. Typically one to three of them test what the decision actually rests on. The rest are there for atmosphere. Narrowing a pack to the numbers that matter before the call is how evidence starts doing work instead of filling pages.

A reporting pack lands on your desk: revenue forecasts, cost variances. Forty numbers across a dozen pages. Somewhere in that pack is the evidence you need before you approve or reject the decision in front of you. The problem is finding it, because most of those numbers are not relevant to this particular decision; they are relevant to running the organisation.

That is the distinction almost every guide to data-driven decision making ignores. They promise you the right metrics to track. For any single decision, only one to three numbers are load-bearing, and the rest are decoration.

Decision-relevant numbers are the small subset of available data that directly test the assumptions a specific decision depends on, rather than the broader metrics an organisation routinely monitors.

Frame the Decision Before You Count

The first thing that goes wrong is the sequence. Teams collect data, build dashboards, and then try to work out what to decide. The Universal Decision-Making Method that Roger Estall and I developed works the other way: frame the decision first, then determine what information bears on it.

Framing means identifying the purpose of the decision and the context that could change during its life. Without that frame, every number in the building looks potentially relevant; the team drowns in reporting rather than deciding. I have watched committees spend entire meetings debating operational metrics that had nothing to do with the strategic question on the agenda, because nobody had asked which numbers mattered before the discussion started.

Framing is not a creative exercise. It answers three questions: what is the purpose of this decision, what outcome would satisfy it, and what context could change while the decision is live? Those three answers shrink the reporting pack from forty numbers to the handful that bear on the decision's purpose. Everything else is organisational monitoring: useful in its own right, irrelevant to this particular call. Without the frame, a committee defaults to reviewing whatever data it was given, which is usually whatever data was easy to collect.

The taxi industry had access to every signal that made Uber possible: satellite location data and smartphone adoption. Each signal was individually visible. The failure was not missing data; it was the absence of a decision frame that would have connected those signals to a question about the industry's future. That is what happens when you start with the data instead of the decision: you monitor everything and act on nothing.

Click to expand

Which Numbers Test the Assumptions?

Once the decision is framed, the question changes. It is no longer "what numbers do we have?" It becomes "what assumptions is this decision resting on?"

Every tentative decision rests on assumptions about the future. A hiring decision assumes the role will still exist in eighteen months; a capital expenditure assumes the technology will not be superseded before the investment pays back. These are the assumptions that matter, and the numbers that matter are the ones that test them.

A distinction that most reporting misses: a number can be factual today and still become an assumption over the life of the decision. A cost projection that was accurate at sign-off becomes a guess six months later if nobody is monitoring the inputs it depended on. The question is not whether the data was good when you collected it; the question is whether it will still be good when the decision depends on it.

Before trusting any number as decision evidence, check whether it is complete and reliable. A clean audit report or a green dashboard reading is not a decision. That is evidence about part of the process, gathered through instruments that may themselves have drifted. I have seen organisations take comfort from dashboards that measured the right things two years ago and have been recycled ever since, without anyone asking whether they still test what the current decision rests on.

The Challenger disaster remains one of the clearest examples of this failure. Engineers had temperature data showing O-ring erosion increased at lower temperatures, and the forecast for launch day was well below any previous flight. That single number was decision-relevant. It was buried under layers of operational reporting, and the decision-makers never received it in a form that connected it to the launch decision.

Once you have identified the assumptions, rank them by two criteria: how much influence does this assumption have on the outcome, and how confident are you that it will hold? A number matters when it reduces the significance of a high-influence, low-confidence assumption. If it does not do that, it is a reporting metric, not a decision input. Charles Goodhart observed that when a measure becomes a target, it ceases to be a good measure; the corollary for decisions is that when a metric becomes a dashboard fixture, it often stops testing anything at all.

Name the three assumptions your next sign-off depends on, then count how many numbers in the pack actually test them. Start the Walk →

Design the Monitoring Before You Sign Off

If the evidence supports the decision and the team has sufficient certainty to act, one task remains before sign-off: designing the monitoring. This means specifying which assumptions will be tracked after the decision is live, who owns each check, and what threshold triggers a review. Without this design, the decision floats on evidence that decays from fact into assumption with nobody watching.

I have sat in post-implementation reviews where the question "who was monitoring this?" produced silence. The decision had been approved without a monitoring design, and nobody owned the follow-through. That is the cost of treating sign-off as the finish line rather than the starting gun for a new set of obligations.

A practical discipline at sign-off: reduce the monitoring to three questions for each key assumption. What number are we watching? Who checks it? At what threshold do we reconvene? If the team cannot answer those questions before the decision is approved, the decision is going live without a mechanism to detect when the world has moved past its assumptions. Insisting on those three answers before approval changes the quality of the discussion more than any additional report.

If certainty is still insufficient after testing the key assumptions, the options are specific: obtain more information about the assumption that concerns you, or modify the decision to reduce its dependence on that assumption. Commissioning another report is not a third option; it is a version of the first, justified only when the additional information would actually change the call. Repeated without that test, it becomes analysis paralysis dressed as diligence.

Confirmation bias makes this harder than it sounds. Teams naturally seek data that supports the decision they have already made, not data that would challenge it. The discipline of naming assumptions before looking for evidence guards against this, because it forces you to specify what would change your mind before you start looking for reasons not to change it.

The conversation about which numbers matter before you make the call is dominated by promises of more metrics and smarter analytics. In my experience, the decisions that fail are rarely short on data. They are short on clarity about which data tests the assumptions the decision rests on. Before the next sign-off, ask one question: which numbers, if they moved, would force us to reconsider? If the answer is none of them, you are monitoring for comfort, not for decisions.

---

Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.