AI decision making ethics covers the principles and processes that determine whether a human being remains accountable when an algorithm shapes a consequential decision. The landscape is well mapped: bias, transparency, fairness, oversight. In fifty years of working with organisations on how they decide, the gap I see is not in the principles. It is in the process.
Each of the cases below illustrates a different dimension of that gap. Algorithmic bias is real and demands correction, but bias is only one failure mode. Accountability vanishes when no person owns the output. Transparency is absent when operators are not told the system exists. And principles, however well articulated, produce nothing until a defined process connects them to the person who must decide.
I have spent nearly fifty years working with organisations on how they make decisions. The pattern I see with AI is identical to the one I saw with risk matrices, enterprise risk management frameworks, and consultant reports: a tool gets promoted from decision aid to decision substitute. The people who should be deciding stop deciding, and when the consequences arrive the tool gets blamed (as designed); the people who deferred to it say they were following best practice. The apparatus of due diligence grows; the quality of decisions does not.
AI decision making ethics is the field concerned with how organisations assign accountability, maintain human oversight, and govern the use of algorithmic outputs in consequential decisions.
Algorithmic Bias and the COMPAS Case
In 2016, ProPublica published an analysis of 18,610 COMPAS recidivism risk scores from Broward County, Florida. They found that Black defendants were nearly twice as likely to be labelled high risk without reoffending, while white defendants were more likely to be labelled low risk and then reoffend. That disparity was real and well documented, and it rightly drew public attention to algorithmic bias in the criminal justice system.
Bias in algorithmic systems is a genuine ethical concern, and the COMPAS analysis demonstrated it in measurable terms. But the case also revealed a second failure. Across multiple US states, judges treated COMPAS scores as sentencing decisions rather than as one input among many. In Wisconsin, Judge Scott Horne sentenced Eric Loomis to eight and a half years, citing that Loomis "had been identified, through the COMPAS assessment, as an individual who is at high risk to the community." The algorithm's creator, Northpointe, insisted the tool was designed as a decision aid, not a decision-maker (a claim that would have carried more weight had the product included any mechanism to enforce the distinction).
Correcting the bias in COMPAS was necessary. But even a perfectly calibrated score would not have solved the ethical problem if judges continued to treat the output as the decision. The bias was in the model. The accountability failure was in the courtroom.
A structured decision process would have required each judge to state what level of certainty they needed, what assumptions the score rested on, and whether those assumptions held for the defendant in front of them. COMPAS was not designed to answer those questions. That responsibility remained with the judges, and in courtroom after courtroom it went unexercised.
Name the person who will decide when your next AI recommendation conflicts with professional judgment, and check whether they know. Start the Walk →
The Accountability Gap
In December 2024, the United Nations General Assembly voted 166 to 3 to demand human control over lethal autonomous weapons systems. The resolution centres on "meaningful human control": not whether an algorithm performs better than a human at target identification, but whether a human being retains the authority and ability to decide whether to fire.
The military context is extreme, but it strips the abdication problem to its clearest form. When a weapon system "decides" to engage a target, who is accountable for that decision? The algorithm has no moral agency and cannot bear accountability. The operator did not make the decision. The developer built a tool, not a firing order. Accountability does not transfer to some other party; it evaporates entirely. That is what happens when the human steps out of the decision and nobody notices until the consequences arrive.
The same disappearance occurs in corporate settings, with lower stakes and identical mechanics. The IAPP AI Governance Survey in 2025 found that only 28% of organisations have formally defined oversight roles for AI governance. In my experience, even that figure overstates the reality; having a defined role and having the authority to override an algorithm when it conflicts with professional judgment are different things entirely. Organisations are assigning accountability for AI outcomes without assigning the authority to shape those outcomes. An executive becomes "responsible for AI ethics" in the same way a chief risk officer becomes responsible for risks the organisation has already decided to take. The title exists. The decision-making authority does not.
Oversight by Design
Boeing's MCAS system on the 737 MAX was engineered to push the aircraft's nose down automatically when sensors detected an excessive angle of attack. The system relied on a single sensor with no redundancy. Pilots were not told it existed; Boeing excluded it from training materials and flight manuals. When a faulty sensor fed incorrect data, the pilots could not diagnose the problem and could not override the system effectively. Two crashes killed 346 people. Boeing's initial response was to blame the pilots who had been structurally excluded from the decision the system was making on their behalf.
I wrote about the 737 MAX in Deciding because it illustrates a principle that Roger Estall and I considered foundational: a model is not a decision. MCAS was a model of what the aircraft should do under specific sensor conditions; it worked correctly when its input data was correct. The ethical failure was not in the algorithm's logic. It was in the organisational decision to remove pilot judgment from a critical flight-control function, and then to deny that removal had occurred. That is the template now being replicated with AI decision systems across hiring, lending, and criminal sentencing: build a system that removes human judgment without disclosure, then blame the operator when it fails.
What AI Decision Making Ethics Actually Requires
The standard response to these failures is more principles. Fairness, accountability, transparency, human oversight: every AI ethics framework published in the past five years contains some version of this list. The values are fine. Values are always fine. The question is whether anyone has connected them to a decision. I sat across from a chief risk officer who could not name a single decision the risk framework had changed (the apparatus was immaculate). The Australian banking Royal Commission exposed the same pattern at national scale. No principle, however well articulated, substitutes for a defined process that tells a specific person what to do when the principle is tested.
If an AI ethics framework is the answer, what was the question? In the organisations I work with, the question is always the same: who decides when the algorithm's output conflicts with professional judgment, and what process they follow to reach that decision.
An AI decision making ethics policy that specifies "maintain human oversight" without defining what oversight means in practice is a statement of aspiration, not a process. Oversight requires a named person with the authority to override, a trigger that initiates review, and a process for reaching a decision when the algorithm and the human disagree. Building the operational process is not the hard part. The hard part is that naming a decider threatens the comfortable anonymity the current apparatus provides, and nobody with a title wants to be the person who was wrong.
The Universal Decision-Making Method was built around exactly this kind of structured thinking, and the method predates the current AI conversation by decades. Its five steps apply to any consequential decision, including the decision to accept or reject an algorithmic output. For AI outputs specifically, the critical discipline is surfacing the assumptions the model rests on and determining whether those assumptions give the decider sufficient certainty to act. The discipline does not change because the input comes from a neural network rather than a consultant's report. The practical shape of that discipline, three steps for using AI to make decisions without outsourcing judgement, applies identically to any algorithmic output.
AI decision making ethics is ultimately a question about whether the person who owns the outcome retains the ability and the authority to decide. The moment an organisation treats an algorithm as the decider, it has not solved an ethics problem. It has created one.
You could adopt the ethics framework and still leave nobody authorised to override the output.
Work through your decisionNo sign-up. Just pick your decision and start.
Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.