ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

Big Data Is Falling Into the Trough of Disillusionment

Svetlana Sicular for Gartner:

Last several weeks show that big data is falling into the trough of disillusionment. I realized it earlier today, when I was describing a recent Elephant Riders meetup to my colleagues at Gartner. MapR, HortonWorks and Cloudera were debating the state of Hadoop. And I heard from the very core of the Hadoop movement that MapReduce has always been Hadoop’s bottleneck or that Hadoop is “primitive and old-fashioned.” […] Meanwhile, my most advanced with Hadoop clients are also getting disillusioned. They do not realize that they are ahead of others and think that someone else is successful while they are struggling. These organizations have fascinating ideas, but they are disappointed with a difficulty of figuring out reliable solutions. Their disappointment applies to more advanced cases of sentiment analysis, which go beyond traditional vendor offerings. Difficulties are also abundant when organizations work on new ideas, which depend on factors that have been traditionally outside of their industry competence, e.g. linking a variety of unstructured data sources. Several days ago, a financial industry client told me that framing a right question to express a game-changing idea is extremely challenging: first, selecting a question from multiple candidates; second, breaking it down to many sub-questions; and, third, answering even one of them reliably. It is hard.

I’ve recently wrote in a post about the state of the Hadoop ecosystem:

The current state of affairs in the Hadoop space is that pretty much everything is possible, but the complexity of getting results varies way too much.

Anyways:

  1. while Hadoop is one of the most important tool of the Big Data toolchain, Hadoop and Big Data are not equivalent
  2. according to the Hype cycle definitions, the Trough of disillusionment is defined as the failure of technologies to meet expectations thus becoming unfashionable. But I think it would be a mistake to generalize such a definition to include unrealistic expectations and stories and promises of silver bullets.

    To expect that a technology which is still in its very early stages will be able to not only solve all previously unsolved problems, but do this fast and very simple is unrealistic. I am not aware of any young software technology that has ever done this before.

I’m not in measure to suggest changes to Gartner’s Hype Cycle, but my feeling is that for Big Data and Hadoop the Trough of Disillusionment and Slope of Enlightenment phases may coalesce. I don’t know if there are other historical examples to confirm this hypothesis in the software industry or outside it.

Original title and link: Big Data Is Falling Into the Trough of Disillusionment (NoSQL database©myNoSQL)

via: http://blogs.gartner.com/svetlana-sicular/big-data-is-falling-into-the-trough-of-disillusionment/