Ask how a company plans to adopt AI responsibly and you will probably hear one sentence: "There will always be a human in the loop." It signals responsibility, and it tells teams they'll be fine. The EU AI Act has even made it a requirement for high-risk systems, with Article 14 dedicated to human oversight.
We have no quarrel with human review. It catches mistakes, and that is reason enough to keep it. The problem appears when a safeguard quietly becomes the blueprint for the whole company.
Designing around what AI can't do yet
Most plans follow the same logic. List what AI can't do yet, then define human work as whatever fills those gaps. Foundation Capital's "The Great Reorg" sketches human roles built around accountability, workflow design, relationships, and validating what AI produces.
In Harvard Business Review, Sangeet Paul Choudary challenges that picture in "The 'Human-in-the-Loop' Fallacy." Under this model, he argues, the company of the future is today's company minus the automated tasks, plus a layer of people checking the machine's output. Those people filter. They don't add new capability.
And the line they stand on keeps moving. What the machine can't do shifts every few months, so any role defined by that line comes with an expiration date.
Why oversight alone falls short
In 1983, Lisanne Bainbridge published "Ironies of Automation." Her argument has aged well: the more reliable a system becomes, the less its operator practices, and the harder it is to take over when something breaks. Air France Flight 447 in 2009 is often cited here. When the autopilot disconnected, the crew struggled to handle the situation, as the BEA's final report documents.
Generative AI makes this worse. Its wrong answers are as fluent as its right ones. A reviewer who trusts the feeling that something "sounds right" is grading coherence, and a perfectly coherent paragraph can be wrong from the first sentence to the last.
Then there is a second effect. Each human correction can become training data, which lowers the need for the next correction. The reviewer spends the day teaching the system the exact work that justifies their job today.
When a constraint falls, the work moves
A better starting question: when something that used to be expensive or slow becomes cheap and fast, what can the company do now that it couldn't before?
Banking offers the classic case. When ATMs arrived, many predicted the end of the bank teller. Yet the number of tellers in the US didn't fall for decades. Branches got cheaper to run, banks opened more of them, and tellers moved from counting bills to serving customers and selling products.
A 2023 NBER study by Erik Brynjolfsson and coauthors followed customer support agents who began using an AI assistant. Productivity went up on average, and the biggest gains went to the least experienced agents. The tool captured what the best agents knew and passed it along to newer colleagues. Here AI widened what people could do. It didn't sit on top of them as a review layer.
Klarna shows the opposite path. In 2024 the company said its AI assistant was doing the work of hundreds of customer service agents. In 2025, according to press reports, it admitted that the cost cuts had hurt quality and began hiring people again. The savings arrived quickly and dominated the story. Nobody seemed to ask what else customer service could do.
How to reorganize in practice
Four moves help companies get past the oversight model.
Change the audit question. The usual question is which tasks the machine can take over and how many hours that saves. Ask instead what wasn't worth doing before and is worth doing now. Maybe it's a customer group you never served because the numbers didn't work. Maybe it's an analysis that took so long nobody bothered asking for it.
Put a second number in every business case. Next to the hours saved, the team names a problem the initiative now makes solvable. If that column stays empty, the initiative is reshuffling tasks. Sometimes that's the right call. Just make it on purpose.
Build governance into the work. Reviews at the end of the line tend to fail, because the reviewer is tired and has too much in front of them. Tests and quality criteria work better inside the process, each with a named person accountable for it. They belong in the product design from the first version, while changes are still cheap.
Revisit the split between people and machines on a schedule. The technology changes in months, and a fixed job description can't keep pace. Check regularly what people do better, what the machine already handles well, and where each makes the other stronger.
The human role
As execution gets cheaper, human value concentrates in deciding what to produce, for whom, and which problems deserve the effort. Accountability for outcomes stays with people too. No tool can carry it.
That is why we see the human role as more than gap-filling. People gain new decision rights and put together combinations of work that didn't exist before. Companies that never make that shift stay stuck supervising a shrinking list of leftovers.
Where to start
At your next AI portfolio review, write two things next to each initiative: the hours it saves and the new problem it lets you solve. Anything with only the first column filled in deserves a candid conversation.
If you're planning the next phase of AI at your company and want an outside view on that review, talk to the VX Technology team.
Sources
Ready to accelerate your business with innovative software solutions?
Get in touch to discover how our custom software solutions can digitally transform your business.
Let's talk?