A series of tubes, leading to Lake Wobegon

According to a recent article in the Wall Street Journal, on a five-star scale, the average rating given by a reviewer on the World Wide Web is 4.3. Despite the Web's reputation for rudeness and hostility, the average rating given to goods and services is above average.

Thinking to my experiences as an Epinions.com contributor and ChefMoz editor, 4.3 sounds about right. If people like something, they tend to give it maximum or near maximum rating.

That there's a filtering step keeping most truly lousy products out of the hands of Web reviewers would seem at first to explain this, but it isn't compelling. Yes, consumers--unlike newspaper and magazine writers--tend only to review products they buy, and that they have thus already researched, but this should drive expectations down, certainly not up. Consider that the filtering-by-research means that the average product a savvy customer sees is better than the average product being sold. If we knock all of the one-star products out, the scale we're left with centers on 3.5 stars. If we expect product ratings to follow a Gaussian ("normal") distribution up to the limits posed by the discretization and boundaries, this will bias slightly to the left of 3.5, but the point still stands. Filtering can't explain pervasive overrating.

The U.S. school grading custom, in which scores between 70 and 80 points out of 100 are supposed to be the average, may have something to do with it. Products may start out in people's minds with 5-star ratings, with stars being knocked off for lousiness. That's quite a different thought process than considering 3 stars as average, 2 stars as below average, one star as lousy, etc. Perhaps, as is speculated at the end of the WSJ, the ability to be meaningfully negative--or just to shed Lake Wobegon Syndrome--may come with experience; willingness to give five-star ratings to mediocre goods and services could be simple naivety.

Before adjusting a few Epinions ratings (downward, very slightly), I computed my average rating: 3.18 on a scale from 1-5. That's above 3, probably statistically significantly so, but not by much, and I've been reviewing "great stuff" lately. That the number is close to 3 is reassuring. Then again, when clicking the Haloscan stars on 'blogs, I'm so consistently a downrater that I didn't need much reassurance at all.

If you don't mean to say that something is great--not good, not "great!" like "pizza tonight?--Great!, I want anchovies...", but better than good, one of the best in its area, a real standout, please don't give it five stars.



HT: Mark Stevens