Dr. Jonah Tebaa presents a comprehensive approach to implementing artificial intelligence in business operations, focusing on the often-overlooked "last mile" between a useful AI demo and a fully functional workflow. This critical phase is where many AI projects stall, as the transition from a controlled demo environment to a real-world workflow can be challenging. Dr. Tebaa argues that the key to success lies in creating a robust and operational workflow that can handle real inputs, exceptions, and handoffs.
The Last-Mile AI Shipping Checklist
Dr. Tebaa's framework consists of seven essential steps to ensure a smooth transition from a useful AI demo to a functional workflow. These steps are designed to address the common pitfalls that can occur when implementing AI in business operations.
The first step is to define the exact workflow moment, avoiding broad labels and instead focusing on specific moments such as a new inbound client request or a consultation note that needs to be turned into follow-up tasks. This precision is crucial in creating a clearly bounded workflow step that AI can operate within effectively.
The second step involves specifying the input contract, which includes determining what the AI needs to produce a useful result. This includes required fields, source documents, customer context, prior conversation history, formatting rules, and minimum detail. By defining these expectations, the workflow can be designed to handle incomplete or imperfect input, reducing the risk of failure.
The third step is to decide where the output lands, ensuring that the AI output has a clear destination and is designed for that specific location. This could be a CRM, task system, shared document, draft email, ticket, spreadsheet, or review queue. The output should be tailored to its destination, taking into account the required structure, fields, and level of polish.
The fourth step involves creating an exception path, which is essential for handling edge cases and imperfect input. This path should be designed to keep exceptions from contaminating the main workflow, ensuring that the process remains maintainable and efficient. A simple exception path could involve moving uncertain cases to a review queue, where they can be handled in batches.
The fifth step is to test the workflow with messy real examples, including incomplete forms, long messages, contradictory notes, and unusual requests. This testing phase helps to identify potential issues and improve the implementation quality, ensuring that the workflow is robust and can handle real-world friction.
The sixth step involves adding a review sample for early runs, which creates a feedback loop that allows for practical questions to be asked about the workflow's performance. This review sample can help identify areas for improvement, such as output usability, landing in the right place, requiring rework, or missing fields.
The seventh and final step is to set a manual fallback, ensuring that the business can continue to operate even when the AI step is unavailable, slow, or unsuitable for a particular case. This manual fallback provides a safety net, allowing the workflow to continue functioning even in the event of AI failure.
Planning for that kind of failure mode is standard advice in AI risk-management guidance, not an idiosyncratic caution. The NIST AI Risk Management Framework "is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems" — trustworthiness that a manual fallback is one concrete way of protecting when an AI step underperforms.
Key Takeaways
Dr. Tebaa's framework provides a comprehensive approach to implementing AI in business operations, focusing on the critical "last mile" between a useful demo and a functional workflow. The key takeaways from this approach include:
- Defining the exact workflow moment to create a clearly bounded workflow step
- Specifying the input contract to handle incomplete or imperfect input
- Deciding where the output lands to ensure a clear destination and tailored output
- Creating an exception path to handle edge cases and imperfect input
- Testing with messy real examples to improve implementation quality
- Adding a review sample for early runs to create a feedback loop
- Setting a manual fallback to ensure business continuity in the event of AI failure
By following these steps and considering the key takeaways, businesses can ensure a successful implementation of AI in their operations, creating a robust and efficient workflow that can handle real inputs, exceptions, and handoffs.
