What Dr. Jonah Tebaa Says to Check Before an AI Launch: Who Takes the Exceptions?
Before an AI launch, Dr. Jonah Tebaa says leaders must verify who handles fallout by using his Launch-Week Queue Sheet framework. Because automation shifts routine tasks into an exception queue, leaders must agree on projected arrivals, timed service minutes, and an age limit defining the longest an item may wait. In an illustrative composite distributor case, an unplanned 14% exception rate created multi-day backlogs, showing why teams must explicitly pair staffing capacity with incoming exception volumes.
A dashboard can say that 86% of a distributor's orders were handled automatically while a new order landing in the exception folder faces a wait of more than five working days and a customer has already phoned twice. Dr. Jonah Tebaa uses that contrast to open one of his practical arguments about launching AI-assisted processes: the number leadership approves and the experience customers have are produced by different parts of the operation.
The problem: the exceptions need people
Dr. Tebaa argues that an AI process does not remove work on launch day. It converts routine work into an exception queue, and the people who receive that queue are the real capacity limit. Firms usually approve a launch on the strength of a success rate for routine cases. What they rarely do is multiply the remaining share by daily volume, by handling time, and by the number of staff who exist to do the handling.
In his work, Tebaa illustrates the gap with a constructed example that he labels an illustrative composite, not a client engagement. A distributor receives 480 orders a day. The launch plan assumed 6% would be exceptions, about 29 a day. Two clerks, each able to close roughly 20 a day after other duties, can clear 40, so the plan looked comfortable. The actual day-one rate was 14%, about 67 exceptions a day. Twenty-seven more arrive than leave each day, so the backlog reaches 135 by day five and 270 by day ten. With items served in order, an exception arriving on day five waits roughly 3.4 working days behind the backlog, and one arriving on day ten waits 6.8.
Tebaa stresses that no single day in this sequence looks like a crisis. The daily shortfall is a small slice of intake, which explains why it passes unnoticed until a customer calls.
His framework: the Launch-Week Queue Sheet
The remedy he proposes is deliberately small: three numbers on one page, agreed and signed before go-live by whoever owns the outcome and whoever owns the staffing.
- Arrivals. A range rather than a single figure. He suggests using double the pilot's exception rate for the first two weeks, followed by a stated decay, because real customers and unforeseen cases show up only after launch.
- Service. Minutes per exception taken from a timed sample of 20 real items, then reduced for the time people spend on other work.
- Age limit. The longest an exception may wait, together with what happens when that point is reached.
Of the three, Tebaa treats the age limit as the commitment and the other two as inputs that show whether it can be kept. The consequence at the limit might be that a named senior person takes the item, that the customer receives a holding message, or that the earlier manual route is used. What matters is that it is chosen in advance.
The five-item checklist
Tebaa pairs the sheet with a checklist to complete before anything goes live:
- Write the three numbers on one page.
- Staff to the week-one peak using borrowed people, each with a dated release.
- Tag every exception with a single cause, so the rate can fall as causes are fixed.
- Display the age of the oldest item on the manager's screen, in place of the average.
- Define an exit trigger: borrowed staff are released only after three consecutive days under an agreed rate.
The fourth item reflects a recurring theme in his writing. An average smooths over the one customer who has waited longest, whereas the oldest item cannot be averaged away.
He also addresses the case where the queue grows despite good preparation. His advice is to choose between two honest options: reduce the volume routed to the new process, or pay for temporary capacity. What he warns against is letting the age limit drift without a decision. In the composite, a one-day limit would be breached on day two, which is when the choice should be made.
Who benefits
The sheet is aimed at operations directors, COOs, owners and department heads whose process goes live within the next one to three months. Order desks, claims handling, invoicing, customer service and onboarding all share the same structure: automation takes the routine cases and passes the awkward ones to a small team.
Firms that have already launched can use it as well. Tebaa suggests starting with a single calculation: take the exception rate in the original plan, multiply it by daily volume, and compare the result with what the team can truly clear in a day. If the first figure is larger, the backlog is already forming, and the sheet gives a way to stop it before customers feel it.
Related evidence: AWS's SageMaker documentation describes shadow testing: a new model or serving setup is run alongside the one already deployed and its performance compared, so configuration errors and performance issues are caught before they impact end users. (AWS's guide to shadow testing a new model before it reaches users)