In a very entertaining article for FT.com, Tim Harford writes:
Cheerleaders for big data have made four exciting claims, each one reflected
in the success of Google Flu Trends: that data analysis produces uncannily
accurate results; that every single data point can be captured, making old
statistical sampling techniques obsolete; that it is passé to fret about
what causes what, because statistical correlation tells us what we need to
know; and that scientific or statistical models aren’t needed because, to
quote “The End of Theory”, a provocative essay published in Wired in 2008,
“with enough data, the numbers speak for themselves”.
Unfortunately, these four articles of faith are at best optimistic
- I’m very sure that the first wheel was not a Pirelli.
- If I’m yelling that I want a blue unicorn, I’m pretty sure sooner rather than later
a bunch of people will try to sell me one.
✚ As you’d expect the Hacker News thread is also highly entertaining:
Another conclusion to draw from this article (which I
really enjoyed, by the way) is that Big Data has been
turned into one of the most abstract buzzwords ever. You
thought “cloud” was bad? “Big Data” is far worse in its
Original title and link: Big data: are we making a big mistake?