How Should a Business Find Out Why Staff Are Working Around a New AI Tool?

Most firms judge a new AI tool by whether people sign in. Dr. Jonah Tebaa, an AI strategy and implementation adviser, argues that this is the wrong instrument. In his work on the weeks after go-live, the question that separates a launch that lasts from one that fades is simpler: what are people doing instead?

The scene Dr. Tebaa uses

He describes an illustrative composite, not a single client. A building-materials distributor launches a tool that drafts customer quotes. By week three every salesperson has logged in. Yet of the last 120 quotes sent, 38 were built in an Excel sheet kept on the sales manager's desktop, the very sheet the tool was meant to replace.

The owner faces a choice. A mandate is quick: everyone must use the tool from Monday. A diagnosis is slower: learn why a third of the quotes went elsewhere. Dr. Tebaa notes that the mandate usually improves the dashboard and leaves the cause untouched, because the work simply moves somewhere less visible.

A workaround is an unwritten requirement

The central claim is that a side spreadsheet is the cheapest and most honest requirements document a new tool will produce. Each one records a job the official system does not do: a handshake discount for a loyal customer, a supplier price changed by phone that morning, a unit conversion done by habit.

Dr. Tebaa contrasts this with surveys and workshops, where people tend to be polite. Behaviour is not polite. It shows exactly where the tool costs more effort than the old way.

This is also why he treats login counts with suspicion. Access is not use. The honest adoption measure is output share: of the work that actually went out, how much passed through the tool?

The audit in five moves

He runs the review over thirty days, one team at a time, and says it needs a notebook and one afternoon a week rather than technical skill.

• Count the leak. For a week, tally finished outputs that bypassed the tool. A number gives the team something to improve.
• Sit beside people. Ask a few staff to show the last thing they made the old way, and watch for the step they will not give up.
• Name each job. Phrase every workaround as "I use this because the tool cannot do that," in the employee's own words.
• Sort and cap. Decide whether each item is added to the tool, handled by a process change, or left alone deliberately. He caps the list at three fixes per fortnight, since small firms have limited capacity to change.
• Re-measure. Count again at day 14 and day 30, and retire the side sheet openly only when its share is close to zero.

What changed in the composite

In Dr. Tebaa's illustration, an afternoon beside three salespeople revealed three gaps. Customer discount tiers were missing. There was no way to enter a price that had changed that day. The first screen asked for nine fields where the sales manager used four.

The fixes were to pre-load the tiers, add a "price changed today" field that expires, and shorten the opening screen to four fields. By week six, side-sheet use in the example had fallen from 38 of 120 quotes to 9 of 120, and the remaining nine were a conscious choice rather than a leak. No one had been lectured or retrained.

Mistakes he warns against

Dr. Tebaa names three. Banning the spreadsheet on day one removes both the evidence and the working method. Counting logins measures access and flatters the launch. Asking staff "why aren't you using it?" invites defensive answers; asking "show me the last quote you built" lets the screen speak.

Underneath is a point about culture. If a workaround is handled as a discipline problem, employees learn to conceal it. If it is handled as a requirement, they learn that naming a gap gets it fixed, and they report the next gap sooner.

The takeaway for owners

The practical advice is modest. Choose one team, count the outputs that bypass the tool for a week, and ask to see the last item built the old way. Within days, Dr. Tebaa suggests, an owner will know which few fixes decide whether the launch survives. He explores adoption at this stage further in his books, Applied AI and The E-mployee Operating Model.

Frequently asked questions

What does a business lose when staff keep a parallel spreadsheet after an AI tool launches?

According to Dr. Jonah Tebaa, the firm loses visibility first. Work happens outside the tool, so the data the launch was meant to produce never accumulates, and the investment looks successful on a login dashboard while delivering little. The sheet also hides the unmet needs that the tool should have covered.

Who in a small company should run the first-month review of a new AI tool?

In Dr. Tebaa's approach, an operations lead or the owner runs it, with a notebook and one afternoon a week. It needs no technical skill, but it does need someone staff will show their real working method to, and who has the authority to approve a few small fixes.

Which signals separate a training gap from a missing feature?

Dr. Tebaa points to repetition. If several people give the same reason, such as a discount tier the tool cannot hold, the feature is missing. If one person struggles with a step others complete easily, it is more likely a training gap. Watching the task done reveals which.

What should be fixed first when several workarounds appear at once?

He advises grouping them by root cause, since two to four causes usually explain most of the leakage, then fixing the cluster behind the largest share of bypassed outputs. He caps changes at three per fortnight so the team can absorb them, and re-counts to confirm the leak shrank.