NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



Cloudera: All content tagged as Cloudera in NoSQL databases and polyglot persistence

Forbes Top 10 Most Funded Big Data Startups

  • MongoDB (formerly 10gen) $231m Document-oriented database
  • Mu Sigma $208m Data-Science-as-a-Service
  • Cloudera $141m Hadoop-based software, services and training
  • Opera Solutions $114 Data-Science-as-a-Service
  • Hortonworks $98 Hadoop-based software, services and training
  • Guavus $87 Big data analytics solution
  • DataStax $83.7 Cassandra-based big data platform
  • GoodData $75.5 Cloud-based platform and big data apps
  • Talend $61.6 App and business process integration platform
  • Couchbase $56 Document-oriented database

I’m not really sure there are any conclusions one could make based only on this data.

Original title and link: Forbes Top 10 Most Funded Big Data Startups (NoSQL database©myNoSQL)


Cloudera Announces Support for Apache Accumulo - what, how, why

Cloudera, the leader in enterprise analytic data management powered byApache Hadoop™, today announced its formal support for, and integration with, Apache Accumulo, a highly distributed, massively parallel processing database that is capable of analyzing structured and unstructured data and delivers fine-grained user access control and authentication. Accumulo uniquely enables system administrators to assign data access at the cell- level, ensuring that only authorized users can view and manipulate individual data points. This increased control allows a database to be accessed by a maximum number of users, while remaining compliant with data privacy and security regulations.

What about HBase?

Mike Olson:

It offers a strong complement to HBase, which has been part of our CDH offering since 2010, and remains the dominant high-performance delivery engine for NoSQL workloads running on Hadoop. However, Accumulo was expressly built to augment sensitive data workloads with fine-grained user access and authentication controls that are of mission-critical importance for federal and highly regulated industries.

The way I read this is: if you don’t need security go with HBase. If you need advanced security features you go with Accumulo.


While there aren’t any details about what formal support means, I assume Cloudera will start offering Accumulo as an alternative to HBase.


I might be wrong though about Accumulo being a replacement for HBase. I’d love to learn how and why the 2 would co-exist.


The obvious reason is that Cloudera wants to get into government and super-regulated markets contracts where security is a top requirement.

Another reason might be that Cloudera is continuing to expand its portfolio to catch as many customers as possible. Something à la Oracle or IBM. The alternative would be to stay focused. Like Teradata.

Original title and link: Cloudera Announces Support for Apache Accumulo (NoSQL database©myNoSQL)


Hadoop Security and Cloudera’s new Role Based Access Control Sentry project

Security is an enterprise feature

At Hadoop Summit, Merv Adrian (VP Gartner) has shown data about Hadoop’s adoption in the enterprise space over the last 2 years and the numbers were great (actually they weren’t even good).

Hadoop vendors are becoming more aggressive in adding features that would make Hadoop enterprise ready. In some sectors (e.g. government, financial and health services) data security is regulated and this makes security features a top priority for adopting Hadoop in these spaces.

The state of Hadoop Security

Tony Baer1 has a nice guest post on ZDNet summarizing the current state of Hadoop security.

There’s a mix of activity on the open source and vendor proprietary sides for addressing the void. There are some projects at incubation stage within Apache, or awaiting Apache approval, for providing LDAP/Active Directory linked gateways (Knox), data lifecycle policies (Falcon), and APIs for processor-based encryption (Rhino). There’s also an NSA-related project for adding fine-grained data security (Accumulo) based on Google BigTable constructs. And Hive Server 2 will add the LDAP/AD integration that’s current missing.

What’s interesting to note is that many big vendors have been focusing on adding proprietary security and auditing features to Hadoop.

Cloudera’s post introducing Sentry also provides a short overview of security in Hadoop, by looking at 4 areas:

  1. Perimeter security: network security, firewall, and Kerberos authentication
  2. Data security: encryption and masking currently available through a combination of recent work in the Hadoop community and vendor solutions.
  3. Access security: fine grained ACL
  4. Visibility: monitoring access and auditing

Sentry: Role-based Access Control for Hadoop

Cloudera has announced Sentry a fine grained role-based access control solution for Hadoop meant to simplify and augment the current course-grained HDFS-level authorization available in Hadoop.

Sentry architecture

Sentry architecture

Sentry comprises a core authorization provider and a binding layer. The core authorization provider contains a policy engine, which evaluates and validates security policies, and a policy provider, which is responsible for parsing the policy. The binding layer provides a pluggable interface that can be leveraged by a binding implementation to talk to the policy engine. (Note that the policy provider and the binding layer both provide pluggable interfaces.)

At this time, we have implemented a file-based provider that can understand a specific policy file format.

According to the post, right now only Impala and Hive have bindings for Sentry. This makes me wonder how Sentry is deployed in a Hadoop cluster so other layers could take advantage of the Sentry ACL. I see such a security feature implemented very close to HDFS so it would basically work with all types of access to data stored.

For more details about Sentry, read the official post With Sentry, Cloudera Fills Hadoop’s Enterprise Security Gap.

There are also numerous rewrites of the announcement:

  1. Tony Baer is a principal analyst covering Big Data at Ovum. 

Original title and link: Hadoop Security and Cloudera’s new Role Based Access Control Sentry project (NoSQL database©myNoSQL)

With New Product Packaging, Adopting the Platform for Big Data is Even Easier

In addition, by choosing Cloudera Enterprise, you open the door to add other capabilities to your subscription as you wish – powerful tools like:

  • Cloudera Enterprise RTD (Real Time Delivery) – Support for HBase
  • Cloudera Enterprise RTQ (Real Time Query) – Support for Impala
  • Cloudera Enterprise BDR (Backup and Disaster Recovery) - Support for BDR
  • Cloudera Navigator – Data management for your Cloudera Enterprise deployment

And when Cloudera Search (beta) becomes generally available, you’ll be able to add:

  • RTS (Real Time Search) – Support for Cloudera Search

Isn’t this called nickel-and-diming?

Original title and link: With New Product Packaging, Adopting the Platform for Big Data is Even Easier (NoSQL database©myNoSQL)


Announcing Open Source, Interactive Search on Hadoop

Announced through a webinar with all big name analysts listening, Cloudera announced Cloudera Search:

Cloudera Search brings full-text, interactive search and scalable indexing to your data in Hadoop. Cloudera Search adds to and extends the value of Apache Solr™, the enterprise standard for open source search. With Cloudera’s 100% open source Big Data platform, CDH, Cloudera Search gains the same fault tolerance, scale, visibility, and flexibility provided to other workloads, like MapReduce, Apache Hive™, and Cloudera Impala.

You know who did this first, right? DataStax. And it was over a year ago.

Original title and link: Announcing Open Source, Interactive Search on Hadoop (NoSQL database©myNoSQL)


Cloudera Announces Cloudera Developer Kit, Enabling Developers to Build Hadoop Apps Faster

I didn’t know what to think of this announcement after reading the WSJ title . After checking the project GitHub page, I still don’t know what to make of it.

Original title and link: Cloudera Announces Cloudera Developer Kit, Enabling Developers to Build Hadoop Apps Faster (NoSQL database©myNoSQL)

Cloudera Impala 1.0 Release Notes and A Couple of Questions

This is what I’ve been looking for since posting about Impala 1.0: the release notes. From the new features list:

  • support for ALTER TABLE
  • REFRESH for a single table
  • Hints for specifying particular join strategies
  • Dynamic resource management, allowing high concurrency for Impala queries

Question: if I remember correctly Impala uses a single process on each machine to execute queries.

  1. is it multi-threaded?
  2. does it do any memory/CPU management so one query is not completely exhausting any of these resources?
  3. what happens with the queries executing when this process fails?

Original title and link: Cloudera Impala 1.0 Release Notes and A Couple of Questions (NoSQL database©myNoSQL)

Impala 1.0 - That was fast

Cloudera announces Impala 1.0 GA release.

That was fast—I guess this is one of the (little) advantages of having Hortonworks working on Stinger, Pivotal on HAWQ, Qubole offering Hive, Pig and Sqoop as-a-Service

Original title and link: Impala 1.0 - That was fast (NoSQL database©myNoSQL)

MapR Raises $30mil in Series C

Where is MapR today?

  1. MapR raised a total of $59mil.
  2. According to John Schroeder (CEO) “92% of MapR customers pay primarely for licenses and not for ancillary services and support”.
  3. According to Wikibon, MapR had $23mil. revenue in 2012, 49% of which coming from services (nb: this seem to contradict the above point)
  4. Support for MapR installations is offered by Accenture and Booz Allen Hamilton

How will MapR use the new capital?

With the new funding, the company plans to invest in research & development, and expand into Asia.

How is MapR seeing its competitors?

John Schroeder (CEO):

“Our competitors’ model is very cash intensive and you have to wonder whether or not they’ll ever be cash-flow positive”.

Cloudera has raised until now $141mil:

  1. Series A: $5mil
  2. Series B: $6mil
  3. Series C: $25mil
  4. Series D: $40mil
  5. Series E: $65mil

According to this, Cloudera raised $36mil in the first 3 rounds. I couldn’t find any official data about the capital raised by Hortonworks, but the number I’ve seen in a couple of places is $50mil. So far MapR raised $59mil.

Sources for these bits:

Original title and link: MapR Raises $30mil in Series C (NoSQL database©myNoSQL)

How Does MapR Compare to Cloudera?

Staying in the MapR land, the question of comparing MapR to Cloudera is answered by people from all sides (MapR, Cloudera and Hortonworks). My summary: “cool proprietary technology addressing some of the current limitations of the Hadoop, but also missing some of the features the Hadoop community has come up with”.

Original title and link: How Does MapR Compare to Cloudera? (NoSQL database©myNoSQL)


Parquet - Columnar Storage Format for Hadooop by Twitter and Cloudera

Announced 2 hours ago, by Twitter’s analytics infrastructure engineer Dmitriy Ryaboy, here comes Parquet:

We created Parquet to make the advantages of compressed, efficient columnar data representation available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model, or programming language.

The Parquet format page describes the details of the Apache Thrift metadata encoding, supported types, Thrift definitions, etc.

Original title and link: Parquet - Columnar Storage Format for Hadooop by Twitter and Cloudera (NoSQL database©myNoSQL)


How Many Hadoops?

The short answer is there is only one Apache Hadoop distribution.

The long answer is that there are many distributions that include Apache Hadoop or are claiming compatibility with Apache Hadoop.

The oldest and probably most popular: Cloudera’s Distribution of Hadoop (CDH)

The 100% open source: Hortonworks Data Platform.

The prioprietary: MapR.

The blue one: IBM InfoSphere BigInsights.

The latest: WANdisco Hadoop WDD, Intel Distribution of Hadoop and Pivotal HD from EMC Greenplum.

There’s also the version Facebook’s running on their cluster which includes Facebook Corona: a different approach to job scheduling and resource management.

But this list is not complete as it doesn’t include appliances featuring Hadoop. In this category we have:

  1. Oracle’s Big Data appliance featuring Cloudera’s Distribution of Hadoop
  2. Netapp’s Hadooplers
  3. EMC Greenplum DCA
  4. Teradata Aster Discovery Platform featuring Hortonworks’s Hadoop Data Platform
  5. Data Direct Networks (DDN)

I hope I didn’t miss any important ones1. As a conclusion for this list, my question is: who is actually benefiting from all these distributions?

  1. I left aside for now Hadoop-as-a-Service.  

Original title and link: How Many Hadoops? (NoSQL database©myNoSQL)