FreshNetworks Blog

Ratings and reviews lie. Simple, subtle lies, but lies all the same. And I suspect most people will never know.

This is Part2 in a series about Ratings and Reviews. To read Part1 click on: Introduction to Ratings and Reviews

“Professional” reviews

The first time I realized what a con ratings could be was when I visited Dubai. I was travelling with a group of friends and one of them booked the hotels. It was quite a surprise to hear we’d booked into a 5-star hotel for what seemed like a 3-star price. It transpired that this was not because my friend had negotiated a great deal, but because hotel stars in Dubai are dispersed as liberally as banking bonuses. Dubai is the land of the 7-star hotel and I’m afraid that has nothing to do with the Burj Al Arab being better than the 5-star Carlyle in New York or Cipriani in Venice.

Thankfully, one of the great things about online consumer-generated Ratings and Reviews is their potential to overwhelm organisational bias assuming a large number of “real people” comment. However even apparently unbiased people give biased reviews. Alongside a trick of averages, these biases help cloud the validity of ratings and reviews. 

So how do online ratings and reviews lie?

There are four key ways in which ratings and reviews lie. There may be more but these are the ones that jump out at me.

1. There is no “zero” score.
Ask most people “When is a product good or bad?” and they’ll say above 2.5 is good and below 2.5 is bad. The assumption is that 2.5 is the mean average score that can be awarded. However that’s simply not the case. The vast majority of five-point scales force you to allocate 1,2,3,4,or 5 stars. If you thought a product or service was rubbish, you cannot give it zero; that does not count as a rating. As a result the mean score that can be awarded is 3 not 2.5. This is great for retailers who have ratings and reviews on their site as most items will appear to be better than average even when they are not.

2. Self-selection bias in ratings and reviews
There are three kinds of purchase bias that add to “the lie”. The first is self-selection bias. Ask me to rate the restaurant behind my office and I won’t. I’ve never eaten there because I don’t like the way it looks. As a result I have selected myself out of being able to review its food or service.

Thus, ratings and reviews’ second lie comes from the self-selection bias inherent in the need for people to have experienced a product or service before rating it. I know that restaurants which sell all you can eat buffets for £3.50 serve poor quality meat, so I’m not going to buy and I’m not going to review. The people who do review are those who for a given product or service had a reasonable expectation that it would fulfill their needs when making the purchase. Hence the group of potential reviewers is biased from the outset.

3. Choice-supportive bias
The second type of purchase bias comes post-purchase. Choice-supportive bias describes our tendency to recall positive feelings or memories to the choice we made. We tend to remember the positive things about the options we chose more than we remember the positive attributes of the alternatives we did not select.

4. Post-purchase rationalization
The third and final purchase bias is post-purchase rationalisation. This bias comes from our tendency to retrospectively justify our decisions as rational ones. We humans prefer to feel that we made good selections not poor ones. So if you ask me whether or not I liked a product that I just spent my hard earned money on, you’re going to get a positive review more frequently than you get a negative one – we like to feel our past choices have been rational and well made.

It is the sum of these biases that results in an average five-star review of 4.3. That’s a long distance from 2.5.

Despite the lies, Ratings and Reviews are great

I’ve had a bit of a go at Ratings and Reviews, but that does not stop me from liking them. They are excellent tools for online retailers and they do a great service to customers. Just like a review on the back-cover of a book or a politician’s statistics, we should always treat claims with care. And so long as that’s done everyone still stands to benefit. For one thing, if ratings and reviews are considered on a relative basis (as is often the case) then the absolute number does not matter in the least.

And let’s be honest, right now, anything that encourages people to buy more is a good thing for the economy and for all of us.

What do you think?