Amazon reviews are just as likely to give an accurate summary of a book’s quality as those of professional newspapers, according to a study from Harvard Business School.
Professor Michael Luca and his co-authors analysed the top 100 reviews from 40 media outlets, including the New York Times, and the Washington Post, between 2004 and 2007 for their paper. The academics used data from reviews aggregator metacritic.com, which summarises professional reviews and then awards ratings, if not given, based on content. They also looked at Amazon reviews for each title.
Although the study points out that there is “virtually no quality assurance” in Amazon’s consumer reviews, which can also be “gamed” by publishers or competitors submitting false reviews, they found that, nevertheless, experts and consumers agreed in aggregate about the quality of a book.
Another piece of research looking at the reliability of information shared through social media. Earlier in the year, a report was published of researchers who looked at mining Twitter to predict the success of movies. The study was not so positive in this case:
Overall, the study found no clear evidence that shows a direct link between Twitter hype, ratings and box office sales.
“The most surprising finding was that Twitter data may not be representative enough of the total population, so it is somewhat risky to use the site for forecasting,” Sen said. “More sophisticated techniques may be needed to understand the applicability of such data sets, such as the metrics we developed to understand the extent of the difference between Twitter users and other online rating side users.”
Others, like The Economist magazine, still see potential:
search-volume forecasts will help spot consumer trends of this sort with increased precision. But the improvements they bring will be incremental. Sophisticated methods based on natural-language analysis of tweets, blogs, or Facebook pages, by contrast, hold greater disruptive potential. As users of social media grow accustomed to sharing highly personal information, apparently unfazed by market-research outfits like WiseWindow watching their every step, the feelings and intentions of hundreds of millions of people are there for data-hungry computers to see.