The Myth of the Objective Product Review

Growing Concerns Over Bias

Yet just how objective are the reviews of expert critics, which 85 percent of consumers regularly seek out when thinking of buying something, according to Nielsen market research?

There have been growing concerns in recent years about the increase in biased recommendations by expert reviewers. In the world of finance, for example, it has been found that brokerage analysts’ recommendations are severely biased when their company can benefit from higher prices of the stocks that it underwrites. In February, in the world of e-commerce, the BBC reported that 10 websites had been found selling fake positive reviews “in bulk” to retailers offering their goods on Amazon’s Marketplace platform.

Now a group of researchers has carried out one of the first studies into review objectivity and the marketplace influence of expert reviews.

“The growth of the internet has made it that much easier for people to find expert opinion on products or services, and that’s why in recent years there’s been increasing discussion on the myth of the so-called ‘objective’ product review,” says Kim Keehyung, Assistant Professor at the Department of Decision Sciences and Managerial Economics at The Chinese University of Hong Kong Business School , and the co-author of a new study.

“Our goal was to examine whether the opinions provided by product reviewers can be distorted by their social ties and financial penalties for any bias,” he says.

His research, published in the paper titled Social Structures and Reputation in Expert Review Systems, was carried out jointly with Prof. Kevin Chung at University of Wisconsin–Madison, and Prof. Noah Lim of the National University of Singapore.

Ties Encourage Favouritism

Their study featured a series of experiments centred around an incentive-driven laboratory review game. The participants – a group of 188 undergraduates recruited at a large research university in the U.S. – were randomly assigned the role of either companies with products or services that were to be reviewed, or as expert reviewers.

Prof. Kim and his colleagues carefully restricted the social ties between players during the study. Before the start, they allowed each of the two reviewers to establish social ties by briefly meeting and introducing themselves – either to different reviewees, or only the same reviewee.

The findings of the study showed that during the game, reviewers overwhelmingly favoured those reviewees they had met when they were assessing the proposed quality levels, even if it meant they received a financial penalty for bias.

Prof. Kim says they were also surprised to find that when both reviewers were aware they had ties with the same reviewee the likelihood of them displaying any bias dropped significantly – suggesting they were reluctant to show signs of favouritism and damage their own reputations.

“People say that the term ‘objective product review’ itself is an oxymoron, and it could very well be that subjective points of view are important because they infuse product reviews with tone and information that you cannot read from facts alone,” Prof. Kim says.

“However, at the very least, consumers and regulators need to be aware that the objectivity of so-called ‘expert’ reviewers can and in most cases are likely to be influenced by the ties that bind or other non-financial incentives.

“That just makes it that much more crucial to ensure there are sufficient penalties in place to punish reviewers if and when they are found to provide biased ratings.”

He says changes were needed because it was often difficult to detect biased ratings and impose penalties on “errant” expert reviewers. For example, although the finance industry recommends stock analysts disclose any conflict of interest, they can easily evade direct associations with the product or services they strongly endorse.

Difficult to Prove Bias

In some specialised industries there are also few expert reviewers, so it is likely that the reviewers and the companies they are rating will know each other well, Prof. Kim says. It can also be difficult to prove a recommendation is biased, even if a reviewer falls under suspicion, because the assessment may involve subjective judgements and a level of expertise may be needed to detect any bias.

Prof. Kim says the findings of the study clearly show that reviewers derive significant psychological benefits from helping reviewees with whom they have social ties, despite the fact that doing so leads to them incurring financial penalties.

“At the end of the day, it’s important to remember that even where a reviewer tries their best to be objective in crafting a review, they still operate within a social context and social ties can invariably affect their judgement either consciously or subconsciously,” he says.

“What our paper shows is that not only social ties between expert reviewers and the companies that produce goods or services, but also the relationships between expert reviewers themselves, can affect the objectivity of the resulting product review.”

He said further studies into the objectivity of expert review systems could focus more specifically on the decisions of consumers, which could reveal interesting behavioural dynamics and provide valuable insights.