A validation report is usually the last document anyone reads before an AI system gets a green light. It has a number on it, the number looks strong, and the conversation moves on to launch logistics. Dr. Jonah Tebaa has spent considerable time arguing that this is precisely the point where most of the risk in an AI rollout actually sits — not in the model itself, but in the unexamined question of where that number's underlying data came from.
A Validation Score Is Only as Honest as Its Source
Tebaa's position is that an accuracy figure is not a property of a model in the abstract. It is a statement about how well a model performs on one specific sample, and the sample almost always tells a story of its own — one that gets skipped in the rush to approve. In his review of AI systems across support, sales, and operations functions, he keeps encountering the same structural error: teams validate a new system against a historical archive because the archive is convenient, well-organized, and already labeled. What goes unexamined is whether that archive resembles, in any structural sense, the traffic the system will actually process once it goes live.
Standards bodies state the same condition in a single line. The NIST AI Risk Management Framework (AI RMF 1.0) holds that "Accuracy measurements should always be paired with clearly defined and realistic test sets – that are representative of conditions of expected use – and details about test methodology." On Tebaa's reading, the load-bearing phrase there is conditions of expected use. A resolved-ticket archive documents the conditions that exist after a trained agent has already interpreted the message, which is a different set of conditions from the one the classifier was about to meet.
He points to a case, drawn from an anonymized MENA-region support operation of roughly forty agents handling both WhatsApp and web chat, as a clean illustration. A ticket classifier built to route incoming messages into refund, shipping, product, and complaint queues was tested against five hundred historical tickets pulled from the helpdesk's "resolved" archive. It scored ninety-four percent and was approved without further scrutiny.
Where the Test Set Quietly Diverged From the Real Channel
The flaw, according to Tebaa's account of the case, was not in the model's architecture or training. It was in what those five hundred archived records actually were. They were not the customers' original messages. They were the closing summaries agents had written when tickets were resolved — tidy, single-language, grammatically complete sentences composed by a human for internal record-keeping. The model had been tested on a translated, cleaned-up version of the problem it would eventually be asked to solve.
Two weeks after launch, a manual audit of two hundred genuine incoming messages put live accuracy at sixty-one percent — a thirty-three point collapse from the number that had gotten the system approved. Tebaa's analysis of the gap identifies five categories of real input that were structurally absent from the archive-based test set:
- Arabizi and mixed-script messages, where Arabic and Latin characters and numerals appear in the same sentence
- One- to three-word fragments such as "refund??" or "wain package," with no surrounding context
- Transcription artifacts from voice notes converted to text, carrying recognition errors the original speaker never made
- Forwarded threads in which several layers of quoted history arrive bundled into what the system reads as a single message
- Submissions consisting only of an emoji or an image, with no text content for a classifier to work from
None of these appear in a closing summary written by a trained agent after the fact, because the agent's job was precisely to translate that messiness into something clean enough to file. The archive had, in effect, already done the hard part of the classification problem before the model ever saw it — which is why testing against it produced a number that measured the agent's editing, not the system's judgment.
What Rebuilding the Test Set Actually Changed
Tebaa is careful, in discussing this case, not to frame the fix as a modeling problem. The team did not swap in a larger or more capable model. Over roughly three weeks, they rebuilt the test set entirely from raw, unedited input pulled directly from the live channel — the same fragments, mixed scripts, transcription errors, and image-only submissions the original archive had filtered out. Against that honest sample, the same system scored seventy-nine percent before any additional tuning. That number, unlike the first one, held once the system went back into production.
The fifteen-point gap between ninety-four and seventy-nine percent, in Tebaa's telling, is the real cost of validating against a cleaned dataset — not a rounding error, but the entire distance between a number that flatters a decision and a number a team can actually plan around.
A Different Question to Ask Before Approval
The broader argument Tebaa makes from this case is not specific to ticket classifiers, WhatsApp support, or any one industry. It applies to any system whose test data was assembled from records a human had already cleaned, summarized, or standardized before the model ever touched them — call transcripts smoothed into meeting minutes, sales conversations logged as CRM notes, customer feedback rewritten into survey categories. In each of those situations, an impressive-looking accuracy score may be measuring how well a person once tidied the data, rather than how well the system handles it in its native, disorderly form.
His recommendation, before anyone signs off on a system based on a validation report, is straightforward: pull a small, entirely unedited sample directly from the channel the system will actually operate in, and check whether it structurally resembles what the model was tested against. If it does not, the number on the report is not wrong exactly — it is simply answering a different question than the one the business is asking. For Tebaa, closing that gap before launch, rather than discovering it through a customer complaint two weeks in, is the difference between an AI system that is merely approved and one that is actually ready.
