In 30 seconds

Do not judge an AI step only by how often it is right. Judge it by what happens when it is wrong: how bad, how visible, and how reversible the mistake is. That decides how much control the step needs.

Why this comes up

A step that is right 95 percent of the time sounds good until you ask about the other 5 percent. If a wrong result means a slightly awkward draft, that is fine. If it means a wrong payment or a lost customer, 95 percent is nowhere near good enough.

The cost of being wrong, not just the rate, should drive the design.

Ask three questions about errors

How bad is a wrong result?

Sort the possible mistakes by consequence. A wrong classification that a person reviews is minor. A wrong action on a customer's account is major. The worst plausible error, not the average, sets the safety bar.

How visible is it?

An error you can see is far safer than one you cannot. If wrong results are obvious and caught quickly, you can tolerate more of them. If they blend in and surface weeks later, even rare errors are dangerous. Errors that are hard to detect are a strong reason for caution.

How reversible is it?

A mistake you can undo is much cheaper than one you cannot. Prefer designs where consequential actions are reversible, or gated behind a person, so a wrong result can be corrected before it does harm.

Match control to cost

Once you know the cost, choose the control:

  • Low cost, visible, reversible: the AI can act with light oversight.
  • Higher cost or hard to reverse: require approval before action. See draft, approve, execute.
  • Severe and hard to detect: reconsider whether AI belongs here at all, or keep a person firmly in charge.

A small example

An AI drafted and, in one version, sent customer emails automatically. A wrong email is hard to unsend and can damage a relationship, a high, low-visibility, low-reversibility cost. Moving to draft-then-approve kept the time savings while making every mistake catchable before it reached a customer.

What this does not solve

Understanding error cost tells you how much control a step needs; it does not tell you the actual error rate. You measure that when you build a test set and evaluate real cases. Cost and rate together decide whether the step is safe.

Good enough to move on when

  • You have listed the plausible error types for each AI step.
  • You know how bad, how visible and how reversible each is.
  • The worst plausible error is acceptable given the controls.
  • Control matches cost: approval and reversibility where it matters.