What I check before adding an AI feature

1. Define the job without saying AI

Imagine a fictional team that processes customer documents and wants a feature that “uses AI to read everything automatically.” Before choosing a model, describe the job using ordinary verbs: identify the document type, locate required fields, compare them with a checklist and route uncertain items for review.

This matters because “reading” is not a single observable outcome. A model may extract a date accurately while missing that it belongs to a different person. It may classify a document correctly but invent a missing field. The product must define which errors are tolerable, which are costly and when to ask a human.

Write the intended user, source material, permitted actions and output format. Then list what the system is not authorised to do. In the document example, a sensible boundary might be: suggest extracted fields and exceptions, but never approve an identity decision or send an external message without a human-controlled action.

2. Compare against a real baseline

“The model is 90% accurate” is not enough to decide whether a feature is useful. Accurate at what, against which dataset, for which people and under what cost of error? Instead, define a baseline workflow and compare it with the proposed assisted workflow on the same representative tasks.

Suppose reviewers now check every submitted document manually. A pilot should measure time per completed task, number of corrections, genuinely resolved cases and the frequency of reviews that still require full manual inspection. If the model creates more checking work than it removes, a promising extraction score will not create value.

Build the comparison using a mixture of common and difficult material. Include images with poor lighting, ambiguous names, nonstandard layouts and documents that do not contain the requested field. A system designed for Indian users may also need to handle mixed scripts or transliterated names, depending on its actual audience.

3. Evaluate failure modes by severity

Averages hide the cases a product team most needs to understand. Create a small evaluation matrix separating the type of failure from its consequence:

For each test case, keep the input, expected result, actual result, reviewer decision and model or prompt version. A simple reproducible record is more valuable than an impressive screenshot because it allows the team to see whether a later change improved the system or merely changed the examples that happen to work.

Use severity-weighted reporting. A wrong punctuation mark and a wrong account number should not cancel each other out inside a single accuracy percentage. Critical failures should have explicit thresholds and escalation procedures.

4. Build the human workflow and permission boundary

Automation does not remove responsibility for the outcome. Decide who can accept a model suggestion, who can override it and how a user discovers the evidence behind a recommendation. The interface should distinguish source text, extracted values, model interpretations and final human decisions.

Prevent a tool-using model from obtaining more access than a typical task requires. If it only needs to read a document and suggest corrections, do not provide write access to unrelated customer records. Approval gates, scoped credentials and action logs are product requirements, not optional infrastructure decorations.

Plan what happens when the service is slow or unavailable. A good fallback may allow manual completion, preserve the partially entered form and explain why the AI suggestion is missing. An automated component that blocks the entire workflow during an outage can make the user worse off than before.

For privacy, document where input data is sent, how long it is retained, whether vendors can use it for their own purposes, what deletion controls exist and what users are told. These questions must be answered for the actual architecture, not filled in from marketing claims.

5. Decide using complete task economics

Model pricing is only one cost. A realistic pilot accounts for infrastructure, processing retries, latency, reviewer time, exception handling, support and monitoring. An AI tool can look inexpensive per request but become costly per successful task if outputs often require manual correction.

Here is a deliberately hypothetical calculation. Suppose 1,000 documents arrive per week. The system proposes results for all 1,000, but only 600 are accepted without correction. If the team evaluates cost per document submitted, the system may look efficient. If it evaluates cost per correctly completed task and includes the remaining 400 reviews, the picture can change significantly. Replace these illustrative counts with measured pilot data before using the calculation to make a product decision.

Equally important is the alternative. Sometimes clear form validation, better search, a rule-based parser or redesigned information collection can remove the problem more reliably. A responsible product manager should be able to choose not to use AI when a simpler system performs better.

A lightweight pilot exit gate

• The user task and existing baseline are written down.
• The evaluation set represents realistic inputs and difficult exceptions.
• Critical failure categories have explicit thresholds.
• Human review, permission scope and fallback paths are designed.
• Vendor data handling and retention assumptions are verified.
• Task-level costs and review effort are measured, not guessed.
• The team has a rollback plan and a named owner for incidents.

The test for a successful AI feature is not whether the model can produce an answer. It is whether a person can complete their real task more reliably, with fewer avoidable risks, at a cost the product can sustain.