Most companies evaluating an AI system that will say yes or no to customers ask one question before they approve it: can the model decide this correctly enough? Dr. Jonah Tebaa argues that's the wrong starting point. In his work advising operations and risk leaders on AI go-live decisions, the question that actually predicts how much damage a bad quarter does is different: when the system is wrong, who hears about it, and how fast can it be undone?
His framework for answering that resolves to two axes, scored before launch, not discovered after it. Reversibility — can a wrong decision be undone cheaply and completely? And blast radius — does the damage from a wrong decision stay contained to one transaction, or does it compound financially, reputationally, or across a relationship the company can't easily rebuild? Cross the two and three tiers emerge.
The Three Tiers
Tier 1 covers decisions that are reversible and low stakes — a small denied refund, correctable with a message and a credit. Dr. Tebaa's position is that formal appeal infrastructure here is overbuilding; a simple "reply if you disagree" is already proportional to the risk.
Tier 2 covers decisions that are technically reversible but carry real friction — the customer has to notice the error, complain, and wait before anything is corrected. This is the tier he flags as the most commonly mismanaged, because companies assume "reversible" means "self-correcting," and it doesn't. Nobody reverses a decision no one is watching.
Tier 3 covers decisions that are irreversible or high stakes — account termination, employee-facing outcomes, anything with legal or safety exposure. His guidance here is unambiguous: the human review has to happen before the decision ships, not after, because there is no cheap undo once it lands.
The tiering is Tebaa's own, but in some jurisdictions the floor beneath it is not discretionary. Article 22(3) of the EU's General Data Protection Regulation (Regulation (EU) 2016/679) requires a controller relying on solely automated decision-making to implement safeguards including "the right to obtain human intervention on the part of the controller, to express his or her point of view and to contest the decision." Set against his framework, that legal minimum lands at roughly Tier 2 — a reachable human, and a route for the affected person to argue — which is exactly the tier he identifies as the one companies most often skip.
The Case That Makes the Framework Concrete
To illustrate the cost of skipping this classification, Dr. Tebaa points to a composite pattern drawn from several AI deployments he's observed rather than any single client: a mid-size retailer running an AI refund system across a ninety-day pilot. Of 15,200 refund decisions, 14,000 were auto-approved without incident. The remaining 1,200 were denials, and of those, 40 were flatly wrong — a returned item that had never left the warehouse, a duplicate charge mistaken for a duplicate order, a loyalty exception the policy engine didn't account for.
The detail Dr. Tebaa emphasizes isn't the error rate — 96.7 percent denial accuracy is a strong pilot number by most standards, and the review board that approved the launch had every reason to sign off on it. It's how the 40 errors were discovered. Not by the system. Not by the team that built it. Every one surfaced because a customer posted publicly — a comment, a tag, a screenshot attached to an escalating support thread. The system had accuracy. It had no way to hear that it was wrong.
That, in his framing, is precisely a Tier 2 failure: reversible mistakes with no appeal mechanism built to catch them before they went public.
Building an Appeal Path Proportional to the Tier
For Tier 2 decisions, Dr. Tebaa's specification is deliberately narrow — not a platform, but three concrete commitments. A named reviewer, so an appeal doesn't dead-end in a queue. A stated service-level agreement, communicated to the customer, not an internal target nobody sees. And a pattern-level escalation trigger: if two complaints land against the same denial reason within a week, that flags the underlying policy for review, not just the individual case. Applied to the retailer example, a trigger set at that threshold would have caught the systemic issue by the fifteenth or twentieth wrong denial rather than the fortieth.
The economic argument he makes to skeptical leadership is comparative rather than abstract: forty wrong denials carry the lost lifetime value of forty aggrieved customers, the support hours spent managing each complaint after it goes public, and whatever reputational cost attaches to a public dispute showing up in search results. Against that, a lightweight Tier 2 appeal path — one named reviewer, a 48-hour SLA, a two-complaint policy trigger — typically costs less to stand up in its first week than the support team would otherwise spend handling the complaints it prevents the following quarter.
Recourse Belongs on the Pre-Deployment Checklist
His broader argument is that recourse design belongs on the pre-deployment checklist, alongside accuracy, not on the post-incident response plan. Before a customer-facing AI decision ships, he asks teams to answer five questions:
- What tier is this decision, scored on reversibility and blast radius?
- Who hears the appeal, and within what stated SLA?
- What complaint volume on a repeated reason triggers a policy review rather than a case review?
- What does the appeal path cost now, versus after the first public complaint?
- Who owns the answer if none of this was decided before go-live?
Teams that can't answer all five, in his view, aren't ready to ship.
