This post by Daniel Abadi dates from July 2012, but it’s so right that I wonder why I didn’t link to it before:

Many people don’t realize that Hadoop and parallel relational databases have an extremely similar design. Both are capable of storing large data sets by breaking the data into pieces and storing them on multiple independent (“shared-nothing”) machines in a cluster. Both scale processing over these large data sets by parallelizing the processing of the data over these independent machines. Both do as much independent processing as possible across individual partitions of data, in order to reduce the amount of data that must be exchanged between machines. Both store data redundantly in order to increase fault tolerance.  The algorithms for scaling operations like selecting data, projecting data, grouping data, aggregating data, sorting data, and even joining data are the same. If you squint, the basic data processing technology of Hadoop and parallel database systems are identical.

There is absolutely no technical reason why there needs to be two separate systems doing the exact same type of parallel processing.

Someone trying to defend his position could counter this perspective by mentioning that Daniel Abadi’s product Hadapt is competing on this exact segment of the market. But by doing so, that someone would just prove how right Abadi is.

Original title and link: Why Database-To-Hadoop Connectors Are Fundamentally Flawed and Entirely Unnecessary (NoSQL database©myNoSQL)

via: http://hadapt.com/why-database-to-hadoop-connectors-are-flawed/

Monte Zweben interviewed by Gery Menegaz for ZDNet:

“The NoSQL community threw out the baby with the bath water. They got it right with flexible schemas and distributed, auto-sharded architectures, but it was a mistake to discard SQL,”

After reading the above, I was ready to file this together with The Time for NoSQL Standards Is Now; which by the way dates from July 2012.

Then it is only the final paragraph that clarifies what Monte Zweben is talking about: a SQL engine for Hadoop. I don’t know if the order makes any difference, but before taking about thrown babies:

1. this is not SQL for NoSQL databases
2. there’s already Hive for SQL on Hadoop
3. there’s already Spire from Drawn to Scale for ANSI SQL on Hadoop
4. there’s already Hadapt for SQL and Hadoop

Original title and link: The NoSQL Community Threw Out the Baby With the Bath Water (NoSQL database©myNoSQL)

via: http://www.zdnet.com/the-nosql-community-threw-out-the-baby-with-the-bath-water-7000010361/

LinkedIn’s Adam Silberstein and Daniel Tunkelang provide a fantastic summary of a presentation I wish I could attend: Daniel Abadi’s “Improving Hadoop Performance by (up to) 1000x”.

Overly simplified, Daniel Abadi’s proposal is to create an analytical platform by using the best of two worlds: Hadoop and row-based or column-based relational database storage and query engines.

Hadapt, the company founded by Daniel Abadi, is in my list of the 8 most interesting companies for Hadoop’s future because I think that an interesting product can be built by combining the long optimized and tested storage and query engines of relational databases with Hadoop’s fault tolerance, scalability, and power, topped with a resource management layer.

Original title and link: Improving Hadoop Performance by (Up To) 1000x (NoSQL database©myNoSQL)

via: http://engineering.linkedin.com/hadoop/recap-improving-hadoop-performance-1000x

Speaking of funding, Hadapt, the company founded by Daniel Abadi, announced that it has closed a $9.5 million Series A round of financing led by Norwest Venture Partners (NVP) and Bessemer Venture Partners.

I haven’t heard much of Hadapt since the initial announcement, so I hope things will change.

Original title and link: Hadapt Raises $9.5m Series a Financing (NoSQL database©myNoSQL)

via: http://www.prnewswire.com/news-releases/hadapt-secures-95mm-series-a-financing-132431973.html

Daniel Abadi’s^[1] reasons for backing his HadoopDB research with the Hadapt startup:

If it wasn’t for the fact that I spent the majority of the last decade soaking up the wisdom of Mike Stonebraker, I might not have chosen option (3). But I watched as my PhD thesis on C-Store was commercialized by Vertica (which was sold last month to HP), and another one of my research projects (H-Store) was commercialized by VoltDB. Thanks to Stonebraker and the first-class engineers at Vertica, I can claim that my PhD research is in use today by Groupon, Verizon, Twitter, Zynga, and hundreds of other businesses. When I come up for tenure, I want to be able to make similar claims about my research at Yale on HadoopDB. So I’m taking the biggest gamble of my career to see that happen.

While this subject would fit better in an entrepreneurship or startups blog, I felt Daniel’s decision reflects the passion of the people involved in the NoSQL and BigData space.

@justinsheehy

Original title and link: Hadapt and Why I’m doing a start-up pre-tenure (NoSQL databases © myNoSQL)

via: http://dbmsmusings.blogspot.com/2011/03/why-im-doing-start-up-pre-tenure.html