Ratings Sites Like Yelp and Angie's List in the Hot Seat

Sept. 18, 2013 — -- Millions of consumers visit ratings sites like Yelp before they choose whether to patronize a business, but with a myriad of ratings sites, who's judging the judges?

Consumer Reports investigated the business models of five sites including Yelp and Angie's LIst.

The magazine said that advertisers on some sites could have a greater influence on user tools like search result rankings.

Consumer Reports also said a customer's take on a business can vary widely from one ratings site to another. One plumbing business in the San Francisco Bay area received an "F" on Angie's List, "A+" on the Better Business Bureau and 2.5 stars out of five on Yelp.

Read More: Yelp Sues Firm Over Allegedly Posting Fake Reviews

"No opinion is monolithic," Consumer Reports' senior writer Jeff Blyskal told ABC News. "If you're just looking at one of those, you're going to get a different decision than when you look at all of them."

Blyskal's concern about reviews on Yelp is the small percentage of reviews that may be fraudulent.

"There's an incentive to game the system," Blyskal said.

Yelp says it has systems in place to protect consumers from fake reviews and to make sure the most useful information is displayed prominently.

Yelp also offers business owners a set of tools to provide accurate information, such as communicating with users privately or publicly, said Kristen Whisenand, a company spokeswoman. Another distinguishing feature of Yelp, she notes, is that it's a community review website, though users don't have to sign up for an account to read comments.

"We really rely on that community of users," she said. "We find the most helpful reviews are people who are continuing to return to the site."

Read More: $10,000 Fridge Not Worth the Cold Cash, Says Consumer Reports

Darnell Holloway, Yelp's manager of local business outreach, said the company is "very transparent about the fact that we don't highlight all the reviews submitted to Yelp."

As of last quarter, there were 42 million reviews on the site. Yelp is displaying about 75 percent of those reviews, while the remainder of those reviews are picked up by a review filter, he said.

The site had 108 million average monthly visitors in the three months ending June 30.

Businesses that are caught, say, buying fake reviews, are flagged on the website and users are notified of that when visiting that Yelp page.

Those situations are few and far between," Whisenand said. "Our review filter is working day in and day out."

Blyskal expressed some concerns that advertisers on Angie's List are reportedly told they can get 12 times more profile views than companies that don't buy ads. Angie's List costs consumers $46 a year to use the service.

"When [companies] get two reviews and a B average or better, and there are no alerts about them, they can pay to advertise and must offer discount coupons that 'position your business to rotate on page 1 of search results'," Consumer Reports writes.

He is also concerned that the ability of A- and B-rated companies to be displayed at the top of the default search results skews outcomes.

Angie Hicks, co-founder and chief marketing officer of Angie's List, said in a statement, "Angie's List is built on a foundation of fairness and transparency. Everything at Angie's List starts with the consumers."

She explained companies' A-F ratings are the average of the consumer reviews. Companies that with A-B ratings are eligible to advertise with Angie's List, but they must offer a discount with that advertisement. If their grades fall below a B, we pull their advertising, she said.

"Over the years, our members have continuely told us that they want to see discounts from highly rated service companies first, so that is the default sort order," she explained. "Members can change the default to various orders including, ratings, number of reviews, alphabetical, etc. We're very transparent with our members on how advertising is displayed. We describe the sort order logic in several places on our site in particular on our search results page and also include an explanation in our magazine to our members each month."

She also said the company sets itself apart with an outside audit of its data handling process every year.

"That audit includes an examination of whether companies that advertise are treated differently than those that do not," she said. "The audits have always found Angie's List reviews are a fair, impartial and trustworthy resource."