Life with Alacrity: Collective Choice: Rating Systems is an article examining the mechanics, implementations, and ultimately failures of rating systems online. While I appreciate the effort towards education and improvement, for me the article created more questions than it answered (which of course, is the hallmark of a good article).In the article, eBay’s rating system is held up as an example of a failure :
Unfortunately, as one of the first in this field, eBay made many mistakes which now leave their ratings system only slightly helpful. However, its failures can also provide us with insights in creating new rating systems on the Internet.
According to the article, eBay’s Feedback system is a failure for the following reasons:
Overall, eBay has a few major problems with their rating system:
- It’s non-granular, with only two options (positive/negative), or more recently three (positive/negative/neutral).
- It’s non-distinct, with no useful guidelines on what behaviors should result in each rating.
- It’s non-statistical, and thus ends up showing only a gross number of sales, not a real subjective measure.
- It’s bilateral, with buyers and sellers rating each other simultaneously, and thus people are afraid to give bad ratings lest they get them in return.
- It’s meaningless, because there are no good tools to control who bids on an auction based on Feedback numbers. (Technically it may be legitimate to ban low feedback bidders from an auction, then cancel their bids if they enter the auction, but this is neither obvious, automatic, nor simple.)
True, sort of. eBay’s Feedback system certainly has these ‘flaws’. But I think to characterize it as a failure is missing the point. The Feedback system tends to produce many more positive than negative ratings. This serves to give people a sense of security when making eBay transactions. When eBay was first struggling to get a foothold consumers were only just beginning to get a taste of conducting business transactions online. Many people still felt like giving a credit card number to a website would almost certainly lead to $1000s of dollars of illegal charges, or worse. For people to trust a website and another, anonymous person to represent their item truthfully, and actually ship it or to actually provide payment was quite a leap of faith. It still is and this is after many people have figured out that not everyone on the net is a criminal trying to game you. Having lots of positive feedback ratings was probably a large reason that eBay was able to work at all.
So am I saying that it is good to give people a false sense of security? No, but Im not sure what eBay is doing is false. For example, can you imagine what ratings might look like if eBay made them granular, distinct, non-bilateral and statistical?
ebayuser123: Item: bowl Rating 6.3(+ or – 1.5 points); cause: 2 days late shipping, marginal packaging, item had a few scratches. Buyer Comment “I was scared to death I was never going to get my item. When it arrived I was like, ‘This package is so ugly.’ Also, the item had some scratches on it, not what I wanted.”
Lets put this rating into perspective looking at the beginnings of eBay. You have people that are already really suspicous and freaked out about doing business transactions over the Internet. Also, many of these people are only used to dealing with brick and mortar merchants or with online retailers and their expectations for shipping promptness, packaging quality, item quality etc. have been formed based on those experiences. So when an eBay user views this review, what are they going to think? Something along the lines of: “Oh my God! This guy could have a rating as low as a 4.8, he sounds really sketchy, he basically sent the woman a damaged product in a horrible looking package. etc. No way.”
From using eBay, we all know that it is a used marketplace and we also know that this behavior isnt really all that horrible. We expect a used product to possibly have a couple of scratches (depending on the description) and the packaging is probably going to be a little homebrew. Two days late? Maybe that wasnt even the fault of the seller. Of course you can say my sample review doesnt resemble the ideal review, maybe you could have checkbox categories instead of freeform descriptions, etc.
My point is, by making a binary rating system that scews towards rating positively, eBay’s Feedback system acted as a sort of Valium for an over anxious user base. Most people are good and have good intentions. When you ‘open the floodgates’ and allow people to pick apart the eBay experience, people will tend to heavily criticize and forget the true value of 1) being able to sell their stuff online and 2) being able to bid on used stuff for cheap.
Now that eBay is established, perhaps they can begin to think about making some changes, but I believe that back in the day the Feedback system could have generated a super-paranoid user base, which is to say, no user base at all. Despite eBay’s ‘failed’ rating system, eBay is a success; I suppose the question is “To what degree does eBay owe its success to its rating system?”
Filed under: Uncategorized | Leave a Comment
No Responses Yet to “eBay: A Failed Rating Experiment?”