For months, bloggers have been speculating on the future of Digg, a news-aggregation site that allows members to vote on the popularity of online news stories. A recent post from the popular blog TechCrunch predicts that Google will soon buy the startup for $200 million. But more recently, Valleywag, a Silicon Valley gossip rag, refuted the claim, writing that the deal was completely off.
While neither Google nor Digg would discuss any deal talks, the chatter highlights the fact that Google News is a weak spot in the Google empire: many experts in the field believe that it lags behind other online news services in providing timely updates on big stories. For instance, it took Google News about an hour to report on Tim Russert's death from when the news first hit the wires. Google publicly claims that this was due to a glitch, but even when everything is running smoothly, its speed still isn't impressive: it took the site about 25 minutes to post a story on Donald Rumsfeld's resignation two years ago, while the news hit the front page of Digg within minutes.
Google News and Digg both offer the service of spotlighting popular news stories around the Web, but they do so in different ways. Digg relies completely on its community of citizen editors to find interesting news stories and submit them to the site to be voted on by others. It employs algorithms to determine when a story is gaining enough votes to be bumped to the Digg front page, where it explodes in popularity because it's viewed by millions of visitors to the site.
Google's system is more automated, and it doesn't benefit from the wisdom of human crowds. Josh Cohen, a business product manager at Google, says that the company's algorithms wade through the headlines and stories of 4,500 English-language news sources (and thousands of others in 21 different languages). Next, another algorithm scans the story for keywords and groups them according to category. Articles are then ranked in two ways. First, Google's algorithms look at the location of the story on the news site's page, which is a hint to its importance. Second, the story is ranked within its cluster. Duplicates are ranked lower than original content, for instance. And the sources that people click on more often are ranked higher, says Cohen.
It might not make sense for Google to redesign its news aggregator to be more like Digg, says Greg Sterling, founder of Sterling Market Intelligence, a consultancy. Google is traditionally conservative when it comes to changing how people interact with its services, although recently, it has experimented with allowing journalists and sources related to news stories to comment on the articles that it collects from other sites. Instead of overhauling Google News, it would make more sense, says Sterling, for Google to employ data collected from Digg users to improve the speed and utility of Google News algorithms.
If Google were to buy Digg or gain access to Digg's intellectual property or the data that it has on its community, it would only be a shortcut to getting something done fast, says Dan Gillmor, director of the Knight Center for Digital Media Entrepreneurship at Arizona State University's Walter Cronkite School of Journalism and Mass Communication. But he's hesitant to read too much into the rumors, and he believes that Google is capable of improving on its own. "Even though I don't think they've done that well with Google News so far...it's not like they're out of this," says Gillmor.
And while the speed at which news stories are posted matters, another important challenge for online news aggregators is disambiguating stories that are high quality from those that are merely popular. Neither Digg nor Google News does this consistently well, says Gillmor. "Google should be working on ways to combine reputation, but not just reputation of some editorial product: reputation with popularity," he says. "Everyone in this field should be doing that." Gillmor adds that there's enormous possibility out there for the company that gets that right, but "we're a long way from seeing it sorted out."
Google's Cohen claims not to be interested in data from Digg that highlights how it spotlights and promotes stories. But he does say that engineers are constantly trying to improve Google News by looking at different types of data, including the type of news links that people click on. "There's a whole host of signals that you can use," he says, including community voting signals. "It's a question of what you want to get out of it."