ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

SSDs and MapReduce performance

Conclusions of comparing SSDs and HDDs for different cluster scenarios from the cost perspective of performance and storage capacity:

  • For a new cluster, SSDs deliver up to 70 percent higher MapReduce performance compared to HDDs of equal aggregate IO bandwidth.
  • For an existing HDD cluster, adding SSDs lead to more gains if configured properly.
  • On average, SSDs show 2.5x higher cost-per-performance, a gap far narrower than the 50x difference in cost-per-capacity.

The post offers many details of the tests run and also various results. But the 3 bullets above should be enough to drive your decision.

Original title and link: SSDs and MapReduce performance (NoSQL database©myNoSQL)

via: http://blog.cloudera.com/blog/2014/03/the-truth-about-mapreduce-performance-on-ssds/


CouchDB - a short review

Pretty good summary of what’s good and what you need to pay attention to when using CouchDB:

During one of our last projects we had a small 2-year adventure with Apache CouchDB NoSQL database. Here, I’m going to briefly present its strong points as well as drawbacks. […] CouchDB was chosen based on requirements and assumptions in the project. Especially, easy multi-master replication seemed to be attractive in the context of the project, which was supposed to be a distributed document database without any relations and rather unstructured data. Unfortunately, as we were going deeper into the project those assumptions turned out not to be 100% correct, and sometimes using this technology was a bit painful.

✚ It’s been quite a while since I last read a post about CouchDB. I won’t conclude based on a single article that CouchDB is still doing well, but it was nice to see it mentioned again.

Original title and link: CouchDB - a short review (NoSQL database©myNoSQL)

via: http://www.future-processing.pl/blog/couchdb-short-review/


Cloudera Search Interface: Inside Cloudera's customer support Enterprise Data Hub

Great use of their own technologies to better server the customer:

This application goes way beyond simple indexing and searching. We are using Cloudera Search, HBase, and MapReduce to process, store, and visualize stack traces that wouldn’t be possible with just a search index. How Monocle Stack Trace integrates with the larger CSI application goes way beyond that, though. It’s a great feeling when you are able to execute a search in Monocle Stack Trace that links directly to a point in time in a customer log file that an Impala query returned after churning through tens of GBs of data — done interactively from a Web UI on the order of a second or two.

I can easily see this becoming a real product used by software companies that offer direct customer support.

Original title and link: Cloudera Search Interface: Inside Cloudera’s customer support Enterprise Data Hub (NoSQL database©myNoSQL)

via: http://blog.cloudera.com/blog/2014/02/secrets-of-cloudera-support-inside-our-own-enterprise-data-hub/


The Forrester Wave for Hadoop market

Update: I’d like to thank the people that pointed out in the comment thread that I’ve messed up quite a few aspects in my comments about the report. I don’t believe in taking down posts that have been out for a while, so please be warned that basically this article can be ignored.

Thank you and my apologies for those comments that were a misinterpretation of the report..


This is the Q1 2014 Forrester Wave for Hadoop:

Forrester wave for Hadoop

A couple of thoughts:

  1. Cloudera, Hortonworks, MapR are positioned very (very) close.

    1. Hortonworks is position closer to the top right meaning they report more customers/larger install base
    2. MapR is higher on the vertical axis meaning that MapR’s strategy is slightly better.

      For me, MapR’s strategy can be briefly summarized as:

      1. address some of the limitations in the Hadoop ecosystem
      2. provide API-compatible products for major components of the Hadoop ecosystem
      3. use these Apache product (trade marked) names to advertise their products

      I think the 1st point above explains the better positioning of MapR’s current offering.

    3. Even if Cloudera has been the first pure-play Hadoop distribution it’s positioned behind behind both Hortonworks and MapR.

  2. IBM has the largest market presence. That’s a big surprise as I’m very rarely hearing clear messages from IBM.

  3. IBM and Pivotal Software are considered to have the strongest strategy. That’s another interesting point in Forrester’s report. Except the fact that IBM has a ton of data products and that Pivotal Software is offering more than Hadoop, I don’t know what exactly explains this position.

    The Forrester report Strategy positioning is based on quantifying the following categories: Licensing and pricing, Ability to execute, Product road map, Customer support. IBM and Pivotal are ranked the first in all these categories (with maximum marks for the last 3). As a comparison Hortonworks has 3/5 for Ability to execute — this must be related only to budget; Cloudera has 3/5 for both Ability to execute and Customer support.

    Pivotal is the 3rd last in terms of current offering. I guess my hypothesis for ranking Pivotal as 1st in terms of strategy is wrong.

  4. Microsoft who through the collaboration with Hortonworks came up with HDInsight, which basically enabled Hadoop for Excel and its data warehouse offering, it positioned the 2nd last on all 3 axes.

    No one seems to love Microsoft anymore.

  5. While not a pure Hadoop player, DataStax has been offering the DataStax Enterprise platform that includes support for analytics through Hadoop and search through Solr for at least 2 years. That’s actually way before anyone else from the group of companies in the Forrester’s report had anything similar1.

    This report focuses only on “general-purpose Hadoop solutions based on a differentiated, commercial Hadoop distribution”.

You can download the report after registering on Hortonwork’s site: here.


  1. DataStax is my employer. But what I wrote is a pure fact. 

Original title and link: The Forrester Wave for Hadoop market (NoSQL database©myNoSQL)


Hortonworks raises $100M to grow engineering and company's ecosystem globally

Derrick Harris for GigaOm has the scoop:

Hadoop vendor Hortonworks has raised $100 million in a new round of venture capital led by BlackRock and Passport Capital. The company’s existing investors — Dragoneer, Tenaya Capital, Benchmark, Index Ventures and Yahoo — also participated in the latest round. Hortonworks CEO Rob Bearden said in an interview that the new funding will help Hortonworks scale its engineering efforts, grow the company’s ecosystem and scale its global operations.

Last week’s round E for Cloudera turned up to be $160 instead of the Bloomberg rumored $200.

These big rounds raised by the Hadoop pure-players are a confirmation of the Hadoop market. But I also think they can be explained by the tough competition Cloudera and Hortonworks are facing from large corporations like IBM, Teradata, Oracle, Microsoft. At least in terms of budget.

✚ While some of the above mentioned companies are partnering with at least one pure-play Hadooper — Cloudera, Hortonworks, MapR — that doesn’t mean they are not keeping an eye on the prize.

Original title and link: Hortonworks raises $100M to grow engineering and company’s ecosystem globally (NoSQL database©myNoSQL)

via: http://gigaom.com/2014/03/24/hortonworks-raises-100m-to-scale-its-hadoop-business/


The NoSQL Family Tree

NoSQL-Family-Tree

Even if it includes just a handful of NoSQL databases, it’s still a nice visualization.

Original title and link: The NoSQL Family Tree (NoSQL database©myNoSQL)

via: https://cloudant.com/blog/the-nosql-family-tree/


Examples of analytics applications across industries

A great matrix of the different analytics use cases across industries in Hortonworks’s post “Enterprise Hadoop and the Journey to a Data Lake“:

Anaylitcs use cases

The data type column section covers multiple dimensions of data. And the authors took a conservative approach for the structured and unstructured categories (in the sense that they marked very few categories as unstructured).

A couple of interesting exercises that can be done using this matrix as an input:

  1. figure out how adding data from different categories to a specific use case would benefit it. One obvious example is: how would Telecom companies benefit from adding to their infrastructure analysis social data?

    Building on the above, decide what tools exist to help with this extra scenario.

  2. can one use case from an industry be applied to a different industry to disrupt it?

    What would be the quickest road to accomplish it?

Original title and link: Examples of analytics applications across industries (NoSQL database©myNoSQL)


Cassandra hits 1 million writes per second on Google Compute Engine

Google using Cassandra to show the performance and cost efficiency of the Google Compute Engine:

  • sustain one million writes per second to Cassandra with a median latency of 10.3 ms and 95% completing under 23 ms
  • sustain a loss of 1/3 of the instances and volumes and still maintain the 1 million writes per second (though with higher latency)
  • scale up and down linearly so that the configuration described can be used to create a cost effective solution
  • go from nothing in existence to a fully configured and deployed instances hitting 1 million writes per second took just 70 minutes. A configured environment can achieve the same throughput in 20 minutes.

Make sure you check the charts and get to the conclusion part. The other conclusion I’d suggest is: based on the real benchmarks I’ve seen over the years, Cassandra is the only system that was proven to scale lineary and provide top performance1.


  1. Before saying that I’m biased, make sure you are reading at least this story and Netflix’s post

Original title and link: Cassandra hits 1 million writes per second on Google Compute Engine (NoSQL database©myNoSQL)

via: http://googlecloudplatform.blogspot.co.uk/2014/03/cassandra-hits-one-million-writes-per-second-on-google-compute-engine.html


4 Reasons Perfect Market chose MongoDB

A team from Perfect Market about choosing MongoDB for their Digital Publishing Suite:

There are many NoSQL products out there, why did we bet on MongoDB? There are four major reasons: great performance, great features, ease of use and great support. Of course not every day with MongoDB is a sunshine day. Some tradeoffs we made are shared at the end of this post.

  1. I’m sure Perfect Market would get great support from almost every NoSQL database vendor — that’s what I’ve always heard in this market segment.
  2. By great performance I’ll assume Perfect Market got the numbers they needed. While presented as the top reason for choosing MongoDB, I think this was more in line with: “considering these other features, is MongoDB’s performance good enough for us?”.

    MongoDB is not the fastest NoSQL database.

  3. Great features and ease of use. Nobody can deny that, at least at the first glance, MongoDB’s feature set is very compelling. And they’ve absolutely nailed the user experience part.

    My hypothesis for MongoDB’s adoption rate has always been that it’s mostly due to it looking familiar to people with relational db experience and also removing most of the strict constraints of these. This is echoed in this post too:

    Althought MongoDB is a NoSQL document DBMS, it bears resemblance to RDBMS’s.

Original title and link: 4 Reasons Perfect Market chose MongoDB (NoSQL database©myNoSQL)

via: http://perfectmarket.com/four-reasons-perfect-market-bets-on-mongodb/


VoltDB raises $8M in Series B

WSJ:

VoltDB has raised $8 million from Sigma Ventures, Kepha Partners and three other “strategic investors”, bringing total venture capital investment to $18.7 million, said its CEO, Bruce Reading. Sigma and Kepha participated in an earlier round, in 2012, through which it raised $5.7 million.

I assume some will say it’s a small round. I’ll say congrats to the VoltDB team.

Original title and link: VoltDB raises $8M in Series B (NoSQL database©myNoSQL)

via: http://blogs.wsj.com/cio/2014/03/17/voltdb-raises-another-8m/


Bloomberg says Cloudera raises at least $200m in new round

Dina Bass and Serena Saitto (Bloomberg):

Cloudera Inc. is raising at least $200 million in a new round of financing from investors including Intel Corp., according to people with knowledge of the situation.

Not confirmed yet.

Original title and link: Bloomberg says Cloudera raises at least $200m in new round (NoSQL database©myNoSQL)

via: http://www.bloomberg.com/news/2014-03-18/cloudera-said-to-raise-at-least-200-million-in-funding.html


A simple distributed algorithm for small idempotent information

In this blog post I’m going to describe a very simple distributed algorithm that is useful in different programming scenarios. The algorithm is useful when you need to take some kind of information synchronized among a number of processes. The information can be everything as long as it is composed of a small number of bytes, and as long as it is idempotent, that is, the current value of the information does not depend on the previous value, and we can just replace an old value, with the new one.

While reading this post from Salvatore Sanfilippo all I was visualizing were the diagrams in James Micken’s “The saddest moment” paper.

Original title and link: A simple distributed algorithm for small idempotent information (NoSQL database©myNoSQL)

via: http://antirez.com/news/71