Parsing Amazon reviews
January 24, 2022 12:37 AM   Subscribe

How am I supposed to assign credibility and compare products using Amazon reviews?

A product will have 5000 ratings and an average of 4.3 stars. The 1 star reviews will mention the product breaking after a week, getting jammed/stuck, etc. The 5 star reviews will rave about what a pleasure it is to use. I can't tell how long any of the 5 star reviewers have had the product. How reliable is that 4.3 star rating? How does it compare to a 4.7 rating on a product with 300 reviews? What's the difference between a product whose ratings are almost entirely split between 5 and 1, vs one with a spread of 5, 4, 3, 2, and 1 star votes?

How on earth are people choosing between multiple seemingly very similar products?
posted by Cozybee to Shopping (17 answers total) 4 users marked this as a favorite
 
I think that the whole crowd-sourced rating system in general is broken. It looked like it worked for a while there, maybe in an earlier & more idealistic version of the internet, but after a large enough number of reviews accumulate it gets taken over by irrelevant axe-grinding or paid-for reviews or whatever, and becomes useless. If you ever need a bitter laugh at humanity's expense, have a read through the user reviews for your favourite movie on IMDB, and see by what enormous margin all the one-star reviewers have missed the entire point.

So I guess you fall back on ostensibly more objective sources like a dedicated review site, or you go with your gut & rely on the returns policy.
posted by rd45 at 1:38 AM on January 24, 2022 [3 favorites]


There is a mathematically correct way of averaging rating scores, unfortunately Amazon and a lot of other web retailers choose not to use it.

It is always worth running FakeSpot* analyzer to see if a lot of reviews are from suspicious accounts or a lot of reviews have been deleted.

I put a lot more faith into just reading the oldest reviews rather than trusting the star numbers, if you sort the listings by 'most recent', and then go to the last page. Amazon don't make this easy but you can go up the the address bar and adjust the &pageNumber=1 to something like &pageNumber=20 to find the oldest review.

Also notice that they always tell you the number of 'global ratings' but that's not the number of reviews, about 70% of people rate things but don't bother to write a review.
posted by Lanark at 1:56 AM on January 24, 2022 [16 favorites]


Honestly, I wouldn't bother. The effort you'd need to put in, to overcome the many flaws of the review system, isn't worth the benefit. I'd, instead, find a completely different way of judging possible purchases, for example:
  • use a well-established seller such as a department store, who you trust to sell good-quality stock
  • only buy goods made by a manufacturer with a solid reputation e.g. VW or Bosch
  • subscribe to an independent research organisation e.g. Which magazine or Wirecutter

posted by vincebowdren at 2:33 AM on January 24, 2022 [7 favorites]


Another problem with the reviews is that Amazon conflates reviews (and stock) from different sellers, which often change over time and sometimes sell counterfeits or entirely different products, or expired goods, under the same listing. So it's completely possible that one reviewer received a good product and another reviewer received something different, and you have no way of knowing which you'll get.

More and more, it feels like buying on Amazon is only for things where quality isn't actually important and taking a risk is fine.

One concrete factor you can pay attention to: the seller's return policy. It feels like an increasing number of sellers either don't allow returns, or don't allow free returns through the usual Amazon system, which was a surprise to me.
posted by trig at 3:02 AM on January 24, 2022 [18 favorites]


The method I've hit on is to refer to the bar graph showing the number of reviews/ratings at each star level.

What I want to see is a funnel-shaped profile, with the largest number of ratings being five-star, fewer four-star, even fewer three-star, etc. down to the point where one-star reviews are the smallest cohort.

I avoid products that have hourglass-shaped review profiles, where there are more one and two-star reviews than three-star. Even if the aggregate score is very high, a spike in one and two-star reviews suggests problems more likely than I care to waste time risking.

This approach seems fairly resistant to dishonest gaming of the system, because re-establishing a funnel-shaped profile would require a seller/producer to post large numbers of middling reviews, bringing the average (or "average" as Lanark points out) down in the process.

On preview, trig makes an important point. Amazon is a wild-western mess that the sheriff isn't able to govern. The star ratings are still useful, but they're not sufficient by themselves.
posted by jon1270 at 3:30 AM on January 24, 2022 [10 favorites]


It’s a crapshoot. I mostly use external reviews/recommendations from sources I trust, and then use the Amazon rating as a “sanity check” to filter out anything with a rating of less than 4 stars or so. Sometimes I read a sampling of the one-star reviews to see whether there’s a particular issue that crops up often, and whether that concern is something I care about. But ultimately, yeah, the Amazon reviews on their own aren’t enough to ensure you’re making a good purchase.
posted by mekily at 4:26 AM on January 24, 2022


I look at the reviews that include pictures. I don't even read the reviews most of the time, just look at the pictures.

Then I read a few of the 3 and 4 star reviews. There's no reason to fake a 3 star review. And you also don't rate 3 stars when you're knee jerk pissed off. 3 star reviews are more likely to contain some truth, and then I weigh that against the price and the photos and the return policy to decide if I'm buying.
posted by phunniemee at 5:31 AM on January 24, 2022 [11 favorites]




Seconding FakeSpot Analyzer. Last couple times I've used it it immediately highlighted that many of the reviews were for entirely different products (I don't know what you call it but it's when one product builds up a good review history then they swap an entirely different product into the listing and it inherits the reviews). It's not the be-all end-all but it's a useful tool.
posted by achrise at 6:05 AM on January 24, 2022 [2 favorites]


I pay attention to the photos, to what the complaints are, to the length of time people have owned stuff (sometimes a factor, but not always), and tend to sample a few of the reviews from each star level. In recent months, however, Amazon seems to have been limiting how many back reviews you can see, which I don't like at all.

If I'm checking on reviews at Amazon, however, it's usually because I'm already planning to buy the thing from some other store, usually ebay, which has no reviews. For years now, I've tried to limit if not entirely eliminate my spending at Amazon. Lots of big boxes have poor quality control, but Amazon is the only place I've ever heard of with obvious fire hazards and used products being sent to buyers who'd paid for something new. There is no quality control at most big boxes, but Bezos is far more ruthless than most.
posted by Violet Blue at 6:06 AM on January 24, 2022 [2 favorites]


I also look mostly at three-star reviews. They aren't going to be fake, and they aren't going to be people who give a blender one star because it arrived a week late. I find the three-star reviews are generally much more nuanced.

I also check for reviews on sites like Wirecutter. And searching Ask or posting an Ask can work really well sometimes.

I'm also avoiding Amazon as much as possible because there is no way to know whether you'll get something from Amazon or some random person somewhere (who may be selling fakes or stolen goods).
posted by FencingGal at 6:19 AM on January 24, 2022 [4 favorites]


How on earth are people choosing between multiple seemingly very similar products?

I think with a lot of stuff there's no real way to tell. Like, if there are 14 different nearly identical-looking showerheads or shoe insoles or phone cases with brand names that sound like they came out of some random syllable combination engine* or where the item name is just a long description of its attributes, there is just no guarantee that even the broad specs are going to stay consistent from month to month, never mind the quality control. You're gambling - you may have to return the item or you may have to put up with it being kind of crap.

For actual brands that are available from places other than Amazon/Wish/AliExpress I think Amazon reviews are generally fine and I follow the recommendations above.


* I have a HOPOPRO shower head, a GBLUCCI watch band, and a PHUNAYA wall mount for my bike - none of them are great but the are basically adequate.
posted by mskyle at 6:27 AM on January 24, 2022


Seconding (or thirding?) a few things:

-3 star views are more likely to not be fake, and give an honest pros/cons review. If the cons are not things I care about, then I feel confident about buying.

-avoid anything that is mostly 5's and 1's - like jon1270 says, you want something that is mostly 5s/4s, then 3s, with 1s/2s relatively low.

-only buy from legit companies/vendors for anything remotely important
posted by coffeecat at 6:55 AM on January 24, 2022


IMO one can't trust any of the reviews because there isn't any assurance they are even talking about the same product category. Eg: Clifford Stoll got his Amazon store hijacked.
posted by Mitheral at 8:05 AM on January 24, 2022 [1 favorite]


I agree that customer reviews are fundamentally broken. People who have had bad experiences are much more likely to write about them (I've heard 7x as likely), but a lot of the people who have a problem with the product are complaining about something extrinsic to the product itself: late delivery, or that they didn't understand what the product is supposed to do, etc. Positive reviews are often fake (I suspect some negative reviews are too), or are written after very little experience with the product, eg "arrived on time, was the right color, ★★★★★." The written reviews can be helpful, but need to be taken with a grain of salt.

Also, consider someone buying a $30 coffeemaker vs someone buying a $300 coffeemaker. The person spending $30 has much lower expectations—if the machine makes coffee without leaking everywhere, that's probably a five-star coffeemaker. The person spending $300 has very high expectations and is probably a total coffee nerd, and will knock off points for insufficient blooming time or whatever. Does that make it a worse coffeemaker?
posted by adamrice at 8:24 AM on January 24, 2022 [2 favorites]


brand names that sound like they came out of some random syllable combination engine

I'd been wondering about that phenomenon for a while, and finally stumbled on this article which gives some insights a while back. I wish it was from a more recognized source, but it's the best I've been able to find. Marketplace Pulse: Short-Lived Brands on Amazon
posted by commander_fancypants at 8:34 AM on January 24, 2022


If there are multiple products that are genuinely identical, then I assume that an original design has been ripped off in The Country Of Manufacture and is being re-sold by multiple drop-shippers. This usually happens when the design is popular and well-established (as mskyle said, above).

Some of these knock-offs are good, and then it's a coin flip; some are dreadful -- but the reviews are either carefully scrubbed of all but 5-stars, or else pictures of failed goods will give them game away -- and I avoid those.

The only thing that saves Amazon is its good return policy, and so I make many selections by checking for 3-star reviews, negative reviews, and pictures -- and the return policy will make or break the choice.
posted by wenestvedt at 11:19 AM on January 24, 2022


« Older How can I say I prefer in-office work over WFH...   |   How to navigate bureaucracy Newer »
This thread is closed to new comments.


Tags

Share