Serendipity in Decision Making Can Be Dangerous

A cancer researcher and a cosmologist were working in the same Durham building in 2019. Their disciplines had nothing obvious in common until one realised the other's signal-extraction methods could improve treatment decisions for triple-negative breast cancer. That meeting later fed Concr's nested-sampling crossover and then a GBP 1 million AI-VISION study backed by Innovate UK. People call that serendipity in decision making. I call it a frame that had been too tight until an outsider walked through the door.

I have seen rooms make the opposite mistake just as often. A chance introduction or a pilot result arrives late, and the room treats surprise as proof. That is sloppy judgment wearing the costume of open-mindedness. Serendipity is valuable only when it tells you the frame was too narrow. The moment it starts doing the job of evidence, it begins to corrupt the decision.

Serendipity in decision making is the useful discovery of a relevant idea or connection that was not being deliberately sought when the decision first began.

Serendipity in decision making is a framing problem first

Most writing on this subject treats serendipity as luck with good public relations. Wendy Ross's cognitive model of serendipity is more useful because it isolates the judgment step: an accident happens, and someone in the room decides it is relevant to the question they are already working on. The accident is common. The judgment is what matters, and it depends entirely on how wide the frame was when the decision began.

I have watched committees define Context as the people already in the room and the data already in circulation. In the Universal Decision-Making Method, Context is specific to the decision, not to the organisation in general. Useful surprise usually exposes a relevance judgment the room got wrong at the start, because someone decided which disciplines belonged before anyone asked what the decision actually required. The wider discipline of problem framing exists precisely to catch that mistake before it compounds.

Committees dislike that conclusion because it offends tidy governance. They want relevance settled before the meeting starts. The serious mistake is thinking a frame is mature because it is stable. A stable frame can still be stupid. I have seen executives exclude the only useful outsider simply because the outsider did not fit the org chart. Later they describe the discovery as lucky, as if the luck, rather than the exclusion, were the interesting part.

Click to expand

Chance helps one team and does nothing for another

Alexander Krauss, reviewing 761 major discoveries since 1900, found that important breakthroughs increasingly ride on new methods imported from elsewhere. A separate study of 2.4 million papers confirmed the pattern: accidental exposure to unfamiliar work did not lift everybody equally. The scientists who benefited were the ones already open to less familiar methods. I have seen the same thing in decision rooms. Two executives hear the same outsider, and only one realises the outsider is speaking directly to the assumption that carries the whole paper. The other smiles politely and moves on, because the outsider was not on the agenda. Good problem framing techniques widen the frame enough for that input to be heard before the committee hardens around its home discipline.

The teams that benefit from chance are not more intelligent. They are less defended. One room allows a strange input to interrupt the draft. Another treats anything from outside the tribe as charming but inadmissible, and the chairs who run that second room rarely suffer for the exclusion. By the time anyone acknowledges the outsider was right, the capital is committed and the governance record reads as if every relevant discipline was always present.

Serendipity in decision making turns dangerous when coincidence impersonates evidence

David Spiegelhalter's Cambridge Coincidence Collection has gathered more than 5,000 reported coincidences since 2011. I do not admire a lucky story merely because it feels uncanny. People experience striking coincidences constantly, which is why a coincidence should make a board more skeptical, not more romantic. A board that treats each one as strategic evidence is not being imaginative. It is surrendering judgment to narrative.

I have watched executives retell the one lucky introduction that worked and forget the ten that led nowhere. That is survivorship bias with better tailoring. One surprise deserved investigation, and most did not, but career folklore does not retell the cases where the surprise was noise. A chance encounter should reopen the question, not settle it. If the surprise matters, the room still has to say what assumption moved and why the evidence now clears the bar. That is the discipline behind reframing problems when the original question has turned out to be too small.

I design for surprise, then force it to earn its keep

The Francis Crick Institute says openly that it was designed to promote chance encounters and scientific exchange. I like that because it treats serendipity as a context design question, not a prayer. You can widen a decision by changing who meets whom and which neighbouring discipline gets invited before the budget is committed. That is a better use of structure than pretending structure and surprise are enemies.

Roger and I learned long ago that surprise earns its place only after two hard questions. Our names are on the cover of Deciding, but the practical discipline inside it is even plainer. What assumption does this affect? What would we do differently if it proves true? That is how the Universal Decision-Making Method handles surprise: make it earn the right to change the decision.

I do not want surprise turned into spectacle. When a fresh method appears from outside the field, the team should test it against the live Purpose and the current assumptions. If it strengthens the case, fine. If it merely makes a better story, throw it out. Organisations that cannot do that swing between over-managed routine and sentimental faith in luck. The people who resist the test are usually the ones whose authority depends on narrative flair rather than evidence.

Serendipity in decision making is useful only when the room treats it as a prompt to reopen the frame. When luck is allowed to stand in for judgment, the people who benefit are the ones who never wanted the decision examined in the first place.

---

Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.