Prioritisation Matrix: Who Chose the Weights?

A prioritisation matrix is most useful when it does what JCVI's 26 February 2021 statement did: make the trade-off plain. JCVI had to choose how phase 2 of the COVID-19 vaccination programme would run under hard supply and delivery-capacity constraints. This was not a classroom exercise.

Different criteria would favour different groups, and delay itself would change the value of every option. It chose age-based ordering over the rival pull of transmission reduction and occupational targeting, and it said why in public: rapid deployment and simpler delivery mattered more than the apparent neatness of a different ranking, because a slower ordering would have cost public-health benefit in the real world of doses, appointments, and record systems.

That is why I distrust the way most organisations use the grid. It usually arrives before the question is properly framed, then the meeting behaves as if the arithmetic has settled something. Roger and I wrote Deciding after watching too many rooms give the score more authority than the Decider who would carry the consequences, especially in capital allocation and project meetings where the spreadsheet arrives dressed as neutrality.

Once that happens, the meeting stops examining a choice and starts hiding behind a format. Most advice pages on this subject never get past the drawing of the grid itself. They show the boxes, explain the scoring, and leave the reader alone with the most important question in the room: whose judgement has just been smuggled into the model?

The evidence does not support the certainty that tool sellers and template writers project. I have found no rigorous study showing that a prioritisation matrix improves decisions. That is not surprising, because Gettinger and colleagues explain that this kind of model has no external right answer apart from the Decider's own preferences, and I have found no validation study for RICE, ICE, WSJF, MoSCoW or the Eisenhower matrix.

Most of the literature is easier on the tool than real decisions are: it measures confidence or perceived quality, not whether the ranking led to a better call. That absence is part of the argument, because it tells you where the authority really sits.

The grid can organise a discussion, but where the literature is blunt is on instability: rankings shift when the model or its weights change, while certainty rises faster than justification. That is why so much matrix advice sounds confident yet leaves so little behind that a Decider could defend under challenge.

A prioritisation matrix compares options against chosen criteria and weights, but its ranking hides the assumptions inside those weights, and those assumptions decide whether the choice is defensible.

Click to expand

What is a prioritisation matrix?

It is a comparison device, not a decision-maker. The useful part is the forced comparison: which option best serves the Purpose, in this Context, under this budget, for this Decider. Useful tools do that. Bad meetings forget the second half and pretend the comparison has become a verdict. The danger is that a tidy ranking makes an untested judgement look objective, especially once the room starts speaking as if a score were a fact rather than a compressed argument.

People use the same family of tools for different jobs. Sometimes the aim is to rank projects. Sometimes it is to force a vague disagreement into the open.

Those are related tasks, but they are not the same task, and a matrix built for one can mislead badly when it is smuggled into another with the same columns and a different decision underneath. The same diagram can therefore serve as a rough conversation aid in one room and as false authority in another.

The American Society for Quality cannot even keep the label stable. On one page it defines a prioritization matrix as an L-shaped pairwise-comparison tool; on another it defines a decision matrix as weighted rating against criteria and treats it as a variation of the same family.

It also files the Pugh matrix under that umbrella even though Cambridge IfM and Frey and his co-authors describe Pugh's method as deliberately non-numeric and resistant to weighting. When a professional body cannot say consistently what family of tool it means, the reader should be careful about treating the label as settled.

So I do not spend much time arguing over whether a team is holding a prioritisation grid, a scoring model, or a weighted matrix. I want to know what decision it is meant to support and what assumptions have been buried inside the ranking.

If the room cannot answer that in plain language, the matrix is already doing theatre. That is also why extractable definitions matter less to me than written reasons. And if someone starts saying the tool had told them, I assume the real decision has gone missing behind the apparatus.

How do you create a prioritisation matrix that can survive challenge?

Start before the criteria. In the Universal Decision-Making Method, I frame the decision first, then develop the real options and recognise the assumptions carrying each one.

Only after that do I ask whether the case is sufficient and what must be monitored after commitment. By then the scoring can serve the decision instead of replacing it. A grid built earlier has no anchor, so the criteria end up reflecting habit, politics, or whoever spoke first.

Keeney's 1994 article made this point when he said most people put the cart before the horse by identifying alternatives before articulating values. Later work found that decision makers often leave out objectives they later judge personally relevant. People do not merely forget minor objectives. They leave out objectives they later admit mattered.

That is a quiet but devastating result, because it means the criteria list in a scoring model is often incomplete before the first number is typed, and an incomplete criteria list does not become sound because the scoring looks tidy.

That is why I start with Purpose and desired outcome rather than with criteria. Teams that skip that step can spend an hour debating weights without first stating what a good result would look like or how long it must hold. That is how a room ends up arguing over percentages when it has not even named the decision horizon.

Criterion	Assumption inside the weight	What would reopen the decision
Delivery speed	The operating gain depends on launch this quarter, not next quarter	Implementation dates slip far enough that the timing advantage disappears
Service resilience	The chosen option can absorb peak demand without extra support cost	Stress testing or early live data shows the support load is materially higher
Regulatory exposure	The approval route and reporting burden stay within the current compliance plan	A regulator changes the timetable, scope, or evidence requirement
Adoption risk	Front-line teams will use the new process without a second change programme	Pilot feedback shows workarounds, rejection, or training demand above plan

A defensible matrix therefore records more than scores. It records the decision and the real options, then leaves a clear note of the assumptions carrying the apparent winner, the life of the call, and what will be watched after commitment. Without that record, a later challenge has to guess what the original meeting thought it was buying.

The Universal Decision-Making Method gives that sequence directly, and the wider family of decision-making frameworks only becomes useful once the decision itself has been framed properly. A ranking that cannot survive challenge from finance, operations, or the board was not ready to guide action in the first place.

Surface the assumption your top-ranked option depends on and decide whether it has been tested. Start the Walk →

Why the weights matter more than the score

Weights are choices, not discoveries. By the time a score appears, someone has already decided what matters more, whose loss matters less, and which kind of error the organisation is prepared to live with.

That choice deserves a name, a reason, and a record. A total score cannot repair the omission. Most matrices get none of the three.

The weighted decision matrix looks objective only after that hidden choice has been turned into arithmetic. Ask a room who chose the weights and you often get a shrug, a story about consensus, or a claim that the numbers were obvious.

They were not obvious. They were somebody's view of importance, frozen into the sheet and then treated as if the sheet had produced it. When the team contests those weights, the argument is usually about rank, not numbers. RICE prioritization builds the same flaw into its confidence score, a percentage picked by feel with nothing to check it against.

The technical literature is enough to kill the innocence. Pöyhönen and Hämäläinen showed that the ranking reflects a weighting choice someone made, even when people are working on their own decisions. Fischer's test showed that the common meeting-room habit of assigning importance weights by feel fails a basic technical test. That is not a side issue. It means the authority people hear in the total score is often borrowed from an assumption nobody wrote down or tested.

The Community Sport Infrastructure Program made the same point in public. Applications were scored out of 100 against published merit criteria, and the full $100 million could have been allocated to projects scoring 74 or more. The Minister's Office inserted electorate-status coding into the sheet, did not rank by score, and 63 per cent of round-two selections scored below 74.

What are the four quadrants of a prioritisation matrix really saying?

Quadrants do not think. Someone chose the axes, and the boxes inherit that choice.

Use urgent and important, or growth and share, and the picture starts to look more authoritative than the argument that produced it. That visual discipline can make a weak argument look settled. The eye sees order and forgets to ask who drew the frame.

The so-called Eisenhower matrix is the clearest example. Eisenhower's 19 August 1954 speech contains the line, "The urgent are not important, and the important are never urgent", but it contains no grid and credits the thought to an unnamed former college president.

The four-box Time Management Matrix appears in Covey's 1989 book, and no earlier urgent-important grid has been identified. The authority people borrow from Eisenhower belongs to a line of speech, not to a historical matrix.

The BCG matrix is not stronger just because it looks more commercial. Bruce Henderson justified the market-share axis in 1970 as a matter of common observation, which is a thin warrant for a tool that governed capital allocation for years. That matters because capital allocation tools are often defended by pedigree long after their warrant has thinned out.

45, 64, 87 per cent is the sequence Armstrong and Brodie reported for controls, those exposed to the BCG matrix, and those who used it choosing the less profitable investment; the finding was challenged in the same journal issue, but the baseline remains the point. A quadrant is somebody's theory about what deserves priority. If the theory is thin, the neatness of the boxes will not rescue it.

What to do when the matrix picks the wrong option

When the matrix picks the wrong option, do not fiddle the numbers in private until your preferred answer wins. A wrong-feeling result usually means the model has exposed something the room was trying not to name: a missing criterion, a political weight, or an assumption being treated as fact.

The mistake at that point is to preserve appearances by retuning the model until it flatters the preferred answer. That discomfort is useful because it tells you where the argument broke, not where the spreadsheet became inconvenient.

Oregon's Commission proved that a scoring model can run correctly and still produce a ranking the Commission rejects. Its cost-utility formula pushed inexpensive treatment for minor conditions above treatment for serious conditions, so the Commission abandoned the formula and rebuilt the list around ranked categories of care because the judgement behind it was wrong.

It later published explicit category weights ranging from 100 down to 1, along with unequal scoring scales, which is another way of admitting that the real judgement was never in the original formula.

If the matrix produces a winner the room does not trust, that is a decision quality problem that has to be surfaced, not an embarrassment to conceal. A matrix that cannot help a leader say no at work has not prioritised anything.

Keep the number if you like, but treat it as evidence of where the model broke and ask which assumption, omitted option, or hidden loss is doing the real work. If the room cannot say that plainly, it still has not decided. What it has is competing priorities that no reordering of the queue will settle.

Why project prioritisation fails when support costs sit outside the grid

Project prioritisation fails when the grid prices only the launch and leaves the support outside the frame. The real option is the project plus whatever secondary elements are needed to make it work. The option being compared is never just the shiny initiative on the front page of the business case.

If those supports sit in another budget, the ranking is under-costed before the comparison even begins. A cheap-looking option can therefore be nothing more than a project whose real operating demands have been hidden in somebody else's column.

The Magnox decommissioning competition shows how badly that can mislead. CFP won on 86.48 against 85.42.

The High Court found that CFP should have been disqualified against two threshold requirements and that transparent reasoning was not always provided for particular scores.

The upwards of £122m the NAO later put on the cost to the taxpayer is what hidden support and threshold failures look like once the ranking meets reality.

This is where portfolio ranking becomes fantasy in a hurry. A project prioritisation matrix that ignores monitoring burden, contingency commitments, or threshold requirements hides cost in another column. Sooner or later the operating budget or the taxpayer pays for the omission.

The budget is the real priority list, and resource allocation tells the truth long after the scorecard has been forgotten. If the organisation would never approve the support package in the same meeting, it has no business pretending it approved the project itself. The column nobody scores changes the ranking.

How priorities go stale after the meeting

Scores go stale faster than committees admit. A ranking only describes the world as the room believed it to be at that moment. Yesterday's assumptions do not become today's priorities by sitting in a spreadsheet. Once the market shifts, the regulation changes, or delivery slows, the order can decay long before anyone remembers to revisit it. This is the part most template pages ignore, because a static grid is easier to teach than a live decision that needs watching.

The manufacturer considering a new product line in Deciding is making that exact kind of bet. The payoff depends on future conditions, just as a government tax change depends on the economy responding in the way ministers hope.

The problem is not that the first ranking was irrational. The problem is that conditions move. In both cases the option does not fail because the original ranking was foolish; it fails because the assumptions behind the ranking stopped being true.

That is why I treat Design monitoring as part of the decision, not as a postscript. Roger and I put it into the method because time changes the meaning of a score.

Monitoring is what stops a live decision from turning into archived wishful thinking. Even WSJF only makes sense because waiting changes the economics of the choice. A ranking stays alive only while the assumptions that supported it still resemble the world outside the meeting.

---

Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.