Human–AI Interaction Designer

A designer runs usability testing on an AI feature and every session goes well. At launch, complaints arrive within hours. The interface tested cleanly because the model happened to produce good output during those eight sessions. What was never tested was the interaction that occurs when the model is wrong — which is a design surface, and one that did not exist in any classical product this designer had shipped.

What actually changes

Output is probabilistic, so the interface has more than two states. Deterministic software succeeds or errors. An AI feature can be right, subtly wrong, confidently wrong, uncertain, refusing, or slow — and each needs designing. The confidently wrong state deserves the most attention and usually gets the least.

Users express intent, not commands. Classical interfaces expose features the user selects among. AI interfaces accept an intent the user must articulate, which shifts the design problem from "make the option findable" to "help the user say what they want" — a much harder problem, because users often do not know until they see a result.

The system's capabilities are invisible and inconsistent. A button either exists or does not. A model can handle a request beautifully and fail on a near-identical one, which makes it impossible for users to build a reliable mental model through use. Much of the design work is compensating for this.

Where classic heuristics run out

Established usability principles remain necessary. They stop short in two specific places.

Visibility of system status assumes the system knows its status. A model that is confidently wrong reports success, so the honest signal the heuristic asks for is not available without deliberate work — calibrated confidence, source attribution, or verification the interface performs on the user's behalf.

Error prevention assumes errors are user mistakes to be designed out. Here the system produces errors while the user does everything right, so the design goal shifts from prevention to detectability and repair: making wrong output easy to notice and cheap to fix.

Failure modes

Happy-path testing — evaluation sessions where the model behaved. Signal: usability scores that do not survive launch.

Undesigned wrong state — no specification for confidently incorrect output. Signal: users discovering errors downstream, in their own work.

Capability opacity — no way to learn what the feature handles well. Signal: users abandoning after two failures, having generalised from a small sample.

Blank-canvas paralysis — an empty input with no indication of what is possible. Signal: high abandonment before first submission.

Explaining this to stakeholders

On why this is not ordinary UX: classical design assumes the system does what it says. Here the system is usually right and occasionally wrong while sounding identical, so we are designing for a capable colleague who does not know when they are mistaken.

On testing: showing the feature working proves it can work. What we need to test is what happens when it does not, because that is the experience that determines whether people keep using it.

On the empty box: an open input looks like freedom and reads as an exam. People do not know what to type, so they type nothing and leave.