Online User Reviews: Can They Be Trusted?

— -- Anyone can write a product review, and everybody reads them. But can you trust them? I refer, of course, to reader or user reviews, the kind you find on Amazon, Buy.com, Epinions, PC World, Yelp, and even the sites of tech product manufacturers, such as Dell. They're everywhere.

But it's the fraudulent reviews--positive reviews contributed by "readers" paid by the company being evaluated--that worry critics and advocates alike.

In an October 2007 poll conducted by the PR firm Burson-Marsteller, 1000 savvy Web consumers (dubbed "e-fluentials" by some wordsmith who evidently was unfamiliar with the term "effluent") were clearly convinced that fake reviews are endemic--and could result in a backlash from online consumers.

The numbers tell the tale: 48% (up from 39% in 2001) believe that fake reviews are being planted on consumer sites. 57% say they won't buy a product if the reader reviews seem suspect. And a whopping 76% claim to double-check what they read online. All are signs of a healthy skepticism.

So, how pervasive are falsified reviews?

Beau Brendler, Director of Consumer Reports' WebWatch site, says that the bottom line is: "[Fake reviews] happen all the time--but proving it, quantifying it--is very hard."

WebWatch--whose motto is "Look Before You Click"--says on its site that its credibility campaign has led more than 170 sites, including CNN, CNet, The New York Times, Travelocity, and Orbitz to agree to uphold WebWatch's credibility guidelines.

Barbara Kasser, author of Online Shopping Directory For Dummies and Internet Shopping Yellow Pages, says: "There's no way to check the reviewer's veracity or if they're on the take--they're anonymous." Another concern: the reviewer might not be competent. "How did [the reviewer] use the product? Did they use it properly? Did they follow the manufacturer's directions? There's no way to know," she points out.

Many ordinary people consider reviews written by consumers to be more reliable, more critical, and ultimately, more useful than many other sources of information. At least that's what they told The Nielsen Company in a survey conducted in April 2007. The top three most trusted sources: "Recommendations from consumers" (78%), "Newspapers" (63%), and "Consumer opinions posted online" (61%). (In a story that PC World posted in 2003, we generally agreed with the above perceptions--but we're a bit more cynical now.)

Certainly, reader reviews have come a long way since the era of Usenet and reader forums. Depending on the site and its readers, you may find pithy commentary, long-winded rants, numeric ratings, pros and cons, graphs, and even reviewer videos.

But Mitch Meyerson, author of the book Guerilla Marketing on the Internet, thinks that "influenced" reviews (paid for or not) are pretty common. For example, says Meyerson, "authors often enlist friends, colleagues, and clients to review their books on Amazon."

According to Blogging Tips founder and Web developer Kevin Muldoon, "tech sites usually have fair, accurate [reader] reviews...but there are definitely more fake reviews [on sites] covering cosmetics and hotels." Read Muldoon's blog entry on his own guidelines for how he reviews products.

The purveyors of this shadowy trade wouldn't talk to us on the record. But their methods--confirmed by many of the sites we talked to--are pretty clear.

Gaming the system involves more than just planting a review or two. It means creating multiple personalities and voices, crafting realistic conversations among those personalities, and using other tricks honed by stealth marketers and paid bloggers. (See Dan Tynan's "This Blog for Hire" for more information on how this works.)

"[These reviewers] are pretty fringe", says Ben McConnell, cofounder of the Church of the Customer Blog. "They come and go, change their names. They're like roaches, scuttling away when the lights are turned on."

Not surprisingly, the vendors, resellers, and meta-opinion sites we interviewed maintain that fake reviews are a very minor problem. There are safeguards to prevent most fake reviews from getting through, they say--and besides, readers aren't stupid.

"There's no way to vet the thousands of reviewers on Amazon," says Patty Smith, the company's director of corporate communications. "But we don't need to. When readers see 25 negative reviews and one glowing one--well, they can figure it out."

Screening Suspect Reviews

Still, most big sites have automatic and manual ways of screening out iffy reviews. Yelp's algorithms, for example, are on the lookout for suspicious patterns--such as a person who creates five new accounts and posts a positive review of the same restaurant from each account. Suspect reviews can be automatically suppressed, the reviewer's IP address blocked, and so on. Yelp caused a ruckus this summer when it yanked a number of reviews. At the time, Yelp said it was disturbed about positive reviews that some business owners had swapped with other owners.

Epinions has a panel of 10 to 15 readers in the topic area that screens reviews before they're published. "These core members will pelt the reviewer with questions and ask them to flesh out the review," says Alisa Weiner, Epinions' vice president of online comparison shopping. "They have a very high bar for what they consider a useful review."

The real secret weapon many sites rely on? Community policing. Namely, dedicated (dare we say obsessed?) readers who whomp on any review that's suspicious. "It's hard to game our core base of reviewers--they're very protective," says Jeremy Stoppelman, Yelp's CEO and founder. But he admits policing can sometimes turn vigilante.

If a dubious review appears--say, a glowing review for a known lousy restaurant--Yelpers will sometimes pile on with one-star reviews. "And of course," sighs Stoppelman, "we have to wade in and undo the mess."

Beyond community policing, what about real policing? "Fake reader reviews would violate section 255.5 of the FTC guidelines on the use of endorsements and testimonials in advertising," says Frank Dorman of the Federal Trade Commission. Such deceptive practices could conceivably land the "reviewer" in hot water. But none of the sites we talked to, from Amazon to Yelp, have ever pursued such legal action. Self-policing and filftering technology, in their view, are more than enough.

What's a consumer to do? Here's some advice from online shopping pros, flinty-eyed reviewers, and others in the know.

Beware of illogical positivism: If a review is excessively positive (especially compared to others) and has contains only one very minor quibble--usually at the end--for balance, there is a strong possibilty that it's bogus. "You can usually tell a shill by their complete lack of balance and fact, and unusual enthusiasm for something very minor," says Nick Merritt, Editor in Chief of the UK-based TechRadar.

Consider the source: For example, on Dell's site, customers give the Dell Vostro 1310 notebook 4.1 stars out of 5. But on CNet, readers give it 3 out of 5. Says Barbara Kasser bluntly, "Don't look at reviews on a vendor's site--there's a good chance they're planted."

Weigh the balance: Look for reviews that provide both positive and negative feedback and include salient details, says one longtime professional tech reviewer. "That way you know the reviewer has actually seen and used the product."

Check for consensus: Check multiple sites to see whether there's any consensus. "You've got to look at four different sources at least," says computer curmudgeon Steve Bass. "That'll give you a broader sense of how good or bad the product is." Adds online business author Frank Fiore, "I like sites, like Amazon, that give you an aggregate score and show a history of the reviewer. That way you can get a handle on whether they're plants or have a personal agenda."

Check to see who's talking: Look for sites that identify reviewers in some fashion. "I take only 'real name' reviews on Amazon seriously" says Kasser. McConnell likes how eBay requires users to disclose a lot of verifiable information, from full physical address to e-mail to phone number. Many review sites barely ask for your e-mail address.

Caveats aside, reader reviews have their place. Even WebWatch's Brendler admits that an expert review isn't always enough. "The passionate reader review can be a plus...it can tell you how the leather seats in that car feel."

Better yet, says Brendler, reader reviews encourage "frank, open dialogue between consumers and companies." To keep that dialogue open, he says, it's in a company's best interest not to plant phony reviews.

Robert Luhn is the former editor in chief of Computer Currents magazine.