Does smarter AI generate more human errors?

Businesses are handing more and more decisions to AI, but some of these tools can overthink simple problems and make worse choices than humans

Featured faculty: Chen Zhi
Written by Ellis Ng

Today’s AI chatbots are fueled by large language models (LLMs) trained on vast expanses of human data, from books, news articles, company reports, and social media conversations. However, as corporate reliance on these tools grows, so does the threat of embedded biases and critical errors.

An October 2025 report from the BBC found that nearly half of AI-generated responses contain inaccuracies. Last March, an engineer at Meta accidentally leaked sensitive data after following a suggestion from an AI agent. These high-profile cases prompt a critical question: What if these powerful AIs, trained on everything humans have ever created, also pick up human flaws?

“LLMs don’t just mirror human biases, but also often amplify them,” says Chen Zhi, Associate Professor in the Department of Decisions, Operations and Technology at the Chinese University of Hong Kong (CUHK) Business School. “Our experiments show that LLMs consistently replicate the classic too-low or too-high ordering bias well-documented in humans, ordering too little in high-margin scenarios and too much in low-margin ones.”

A study Professor Chen co-authored with Liu Jifei and Zhong Yuanguang of South China University of Technology, Large language newsvendor: Decision biases and cognitive mechanisms, tests LLMs on the “newsvendor problem”, a decision-making problem in which an AI must stock a resource before knowing the actual demand. The results show that AI often overshot the mark and performed worse than humans. The smarter the model, the more likely it was to act irrationally.

The paradox of intelligence

Professor Chen and the team ran multi-round experiments with 15 decision rounds each: LLaMA-8B, GPT-4, and GPT-4o. Each AI had to decide how much product to order, just as a store owner would try to guess what customers would buy and how much. After each round, the AI received feedback on what customers actually demanded and how much profit it made.

The entire process was repeated for three scenarios with varying customer demand and conditions: one with no guidance, one with a mathematical formula for calculating the optimal order quantity, and the last with a special “no-risk” scenario in which every order placed was guaranteed to turn profits.

“The most sophisticated model by the time we conduct our study, GPT-4, actually showed the greatest irrationality,” Professor Chen says. “It would correctly compute the optimal order quantity in its reasoning, then talk itself out of it through elaborate ‘risk management’ adjustments, even in settings with zero financial risk.”

“The most sophisticated model by the time we conduct our study, GPT-4, actually showed the greatest irrationality.”

Professor Chen Zhi

Each model failed for a different reason. For instance, GPT-4 over-ordered by 70 per cent more than humans in low-margin scenarios. These errors are not due to the AI being cautious and overthinking, but stemming from deep-seated information-processing, as its deliberation undermined its own best decisions, even when there was no chance of losing money.

Meanwhile, LLaMA-8B lacked the computational power to consistently apply rules, leading to erratic choices. GPT-4o was built for speed, stuck closely to simple formulas, and made steady decisions. “The three models illustrate three distinct failures: over-analysis (GPT-4), rigid heuristic adherence (GPT-4o), and fundamental computational limitations (LLaMA-8B),” Professor Chen adds.

The researchers call this the paradox of intelligence, where more brainpower does not guarantee better results. All models were anchored by their first scenario, carrying the bias even after many rounds, but also fixated on the most recent orders. In the most volatile scenario, LLMs reacted to the latest data point by changing their orders all the time. By comparison, humans did this fewer than four times out of ten.

Such errors can be traced back to how the models read and process text. LLMs take in information one piece at a time, and this sequential reading spills into decision-making. The models either cling to whatever they encountered first or overreact to whatever arrived last.

Giving AI clear rules and more adjustments

There are several practical strategies to curb these biases, according to Professor Chen. Providing clear and structured prompting is highly effective. When GPT-4o was given the optimal formula and clear rules, it applied them directly and achieved near-perfect results with minor deviations.

“Providing explicit optimal formulas significantly improved performance,” he adds. “Well-designed prompts can constrain bias without requiring any modification to the models.”

Another way is to choose LLMs carefully by matching them to the task rather than defaulting to the most powerful option. “Model selection matters enormously,” he says. “Efficiency-optimised models can outperform more complex ones on well-defined optimisation tasks, so managers should match an AI model to the task rather than defaulting to the most capable option.”

Fine-tuning also offers a promising solution. Professor Chen explains that fine-tuning LLM, where the model is further trained on a more specific dataset to adapt to a particular task, could improve performance. “Finally, human-in-the-loop oversight remains essential, especially for detecting cases where a model computes the right answer but then overrides it with heuristic adjustments,” he adds.

Do we really need smarter AIs?

Professor Chen and his fellow researchers have continued testing newer LLMs and expanded their experiments to other scenarios. He notes that biases persist in newer LLMs, but the driving factors are not always the same as those seen in the initial study. Some models have continued to overthink like GPT-4, but in different ways, and new kinds of errors have appeared.

“Firms should not assume that a more advanced or expensive model will produce better operational decisions,” he says. “AI biases are systematically shaped by their underlying architecture, and the specific contours of these biases evolve as model architectures change.”

When models compute the right answer but then talk themselves out of it, humans can intervene before those decisions get implemented and compare the model’s reasoning against its final output. Clear decision checkpoints, where AI outputs pass through rules-based checks before implementation, can also serve as a practical safeguard.

“Since LLMs can intensify human biases, the economic consequences of unsupervised deployment can exceed what we would expect from human decision-makers alone, so organisations need systematic monitoring for bias amplification,” he adds. “The goal is not to eliminate AI from the decision process but to design systems that leverage AI’s computational strengths while constraining its vulnerabilities.”