Article 14 of the EU AI Act, the section governing human oversight of high-risk systems, does something statutes rarely do. It names a psychological failure by name. Deployers have to ensure the people overseeing a system stay aware of their own tendency to over-rely on its output. Read that as what it is. The law is not asking you to install a human. It is telling you that installing one is the easy part and does not work on its own.
Most of what has been written about this lands on attention. Automation bias: when a system is right most of the time, people stop genuinely checking it. Decision fatigue: a reviewer working a queue applies less scrutiny at item two hundred than at item two. Both are real, both are documented across aviation and medicine going back decades. And both point at the same family of fix, which amounts to helping the human pay better attention.
That framing is too generous. It assumes the reviewer would catch the error if they were sharper or less tired. Often they would not. Not complacent. Structurally unable.
Here is the question that settles it, and I would ask it about every approval gate you run. What would this person actually have to do to catch a wrong answer?
Walk it through for one gate, concretely. Sometimes there is a real procedure at the end of that question: pull the source record, recompute the figure, call the customer and ask. Sometimes the honest answer is that there is nothing they could do. They have the agent's output and their own intuition, and intuition against a fluent, confident, usually-correct system is not a check. It is a coin flip wearing a lanyard.
So the useful distinction is not attentive reviewer versus inattentive one. It is whether the person has an independent path to the answer. Could they have reached a conclusion another way, without the agent telling them first? If yes, their approval means something, because their agreement is a second opinion. If no, their approval means only that they were present. The paperwork records a human decision either way, which is exactly the problem. Researchers have a name for a person placed in a system mainly to absorb blame for it: a moral crumple zone. That is what you get when you put a reviewer in front of a decision they cannot evaluate.
Two things have to be true for a check to be a check. The reviewer needs a way to reach the answer independently, and they need enough time and access to actually walk it. Miss the first and you have a signature. Miss the second and you have a signature with better intentions.
Two fixes are worth more than a vigilance reminder.
Make the human commit first. Have the reviewer record their own read before the agent's answer is visible, then show it. The research on this calls it a cognitive forcing function, and it outperforms telling people to be careful, because it converts a review into an independent judgment you can compare. Where the two disagree, you have found something worth looking at. Where you cannot ask the human to commit first, you have learned that they had no independent path, which is its own answer.
Measure the catch rate. Seed known errors into the queue and track how many the loop finds. If you cannot say what fraction of planted mistakes your reviewers catch, you do not know whether you have oversight. You have a hope with a timestamp on it. The number will be uncomfortable the first time. That discomfort is the point, and it is cheaper to learn it from a planted error than from a real one.
Then the part nobody wants to hear. Sometimes you run the test and the honest conclusion is that no reviewer can evaluate this decision at the speed and volume you need it evaluated. The answer then is not a better reviewer. It is to narrow what the agent is allowed to do alone until what remains is something a person can genuinely check.
That is a harder conversation than adding an approval step. It is also the only version of human oversight that survives contact with someone asking how it works.