Ratings are Overrated

I’ve been thinking about this post for a while now, and something that happened recently hit home hard enough to really send me over the edge.

So let me go ahead and put it out there:  I have a problem with user ratings.

I have been a member of Amazon for 10 years.  I am completely loyal to them and probably spend a minimum of $1,000 per year with them.  I have had similar relationships with other sites that have soured, while Amazon continues to stand tall (no pun intended).

One of my reasons for being so loyal to Amazon is that their member rating system is beyond compare.  When you read user reviews of books for example, you are pretty much looking at the real deal.  For the most part, people say what they think.

Sure, there are authors who will try to artificially increase a particular book’s rating by having all their friends and neighbors and parents and fans go on to the site and give it good ratings.  However, I’d like to believe the law of averages will eventually take care of that and besides, most readers are savvy enough to see through these games.

Now, contrast this with another site, (which I won’t mention but if you email me I’ll tell you privately), that lauds itself in its marketing materials because “our vendors get all 4.5 star or above ratings.”

Can someone explain to me how this is useful?  What does it say about this company?  That any vendor below 4.5 stars gets summarily kicked off their site?

NO!  What it says is that they “encourage” their raters to give good ratings.  So please, tell me how this helps me, as a consumer, to decide whether to use this service or not?  In other words, if everyone gets 5 stars, what is the point???

It reminds me of the 60-Minutes article a few years about about the Millenials, who feel that everyone in the game deserves a trophy whether they won or not.  Which is ridiculous when you think about it.  A game isn’t a game if someone doesn’t win.  Giving trophies to the losers sets these kids up for a lifetime of unreasonable expectations.

In addition, how does the losing team (or poorly performing vendor) ever learn of their weaknesses, in order to try to improve them?  Not only is it not fair to the readers, it’s not fair to the vendors!

Here’s another example:  A couple of months ago I ordered two exercise tapes from a seller on a popular auction site.  The condition was listed as “Like New” which to me means the box is open and maybe the tape has been played once or twice but it’s otherwise indistinguishable from a new item.

I received the tapes and after watching them both, found that one of them was in only fair condition.  The color was off, the viewing was scratchy, and the sound was inconsistent, which is indicative of a VHS tape that has either been played too many times or sat in someone’s trunk for a half a year.

The tapes were cheap and I did not want to send them back, I was happy enough with  my purchase, but I wanted to make a point.  So I gave the seller a “neutral” rating and stated that one of the tapes was not in the advertised condition.

An interesting thing happened.  The auction site REALLY did not want me to leave this less than perfect rating.  It discouraged me *strongly* and made me agree to a list of statements by checking off a series of boxes, before it would finalize my rating.  Huh???

Now, for the icing on the cake… In last week’s post I mentioned that I had hired two VAs.  One of them delivered substandard work, and I felt that I paid her for 10 hours to do something I could have done in about two.  So I politely told her I didn’t need her any more and ended her assignment, and generously gave her a 4-star rating.  She had all 5-star ratings previously, which honestly stumped me a bit given the poor quality of the work I got from her.

She emailed me and asked me to please change my rating to all five stars, because she “cares about her reputation.”  She mentioned that she had given me five stars in return and so would I please reconsider my rating.  Well, I did not respond to her because what I really wanted to tell her was that she was lucky I didn’t give her two stars.

Whatever happened to “Gee why didn’t you like my work? What could I have done to make you happy? Can I make it up to you?”

I have had several of these WTF moments over the past several months regarding ratings.  I have thought a lot about it, and I have come up with a list of possible reasons for why this trend may be taking place.  These really are guesses, I claim no expertise in this field except for a healthy dose of insight into human nature.

• People don’t like conflict and giving someone a poor rating “to their face” is hard to do.  Amazon’s products are very impersonal, which makes it easier to be honest.  Hiring a freelancer to do work for you on the other hand, is very personal.
• Most people don’t know how to give constructive criticism.
• Sites that support ratings systems have come to believe for some reason that having better overall ratings makes their site or product more desirable.
• People would rather not give a rating at all if they have a bad experience, which skews the ratings (but they WILL tell their friends about it).
• People believe that they deserve a good rating automatically, without having to earn it.
• Somewhat related to the previous point, but worth its own mention is the whole concept of the entitlement generation.  Urban legends about millenials whose mothers call their bosses when they don’t get a favorable annual review come to mind.

I’d love to hear what others think of this, whether you agree or disagree.  If you agree, can you offer your own suggestions for what might be happening here?  If you disagree with me, why?  Really, I want to know.

* * *

Do you enjoy reading these posts? Why not sign-up to receive Grouvia’s e-newsletter? You’ll get the latest news delivered to your inbox and you can participate in the Grouvia development process. It’s free. Sign up at www.grouvia.com.