ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

Cloudera: All content tagged as Cloudera in NoSQL databases and polyglot persistence

Intel kills a Hadoop and feeds another

I seriously doubt you could have missed the 2nd part of this, but here’s the shortest executive summary:

  1. Intel has killed its own distribution of Hadoop — is there anyone that would disagree this is a good idea?
  2. Intel has invested $740mil in Cloudera (for 18%) — there’s no typo. 740 millions.

The main questions:

  1. where will Cloudera put the $900mil raised in the last round(s)?
  2. why Intel invested so much?

These questions were also asked by Dan Primack for CNN Money and after looking at different angles he comes out empty.

So let’s check other sources:

  1. TechCrunch has initially speculated that much of the investment went to existing shareholders.

    The post was later updated with a comment from Cloudera’s VP of marketing stating that the majority of the money went to the company. But no word on how they’ll be used.

  2. Reuters writes that Intel made the investment to ensure their leading position in server processors:

    Intel hopes that encouraging more companies to leap into Big Data analysis will lead to higher sales of its high- end Xeon server processors. The chipmaker believes that hitching its wagon to Cloudera’s version of Hadoop, instead of pushing its own version, will make that happen faster.

    Still no word on how Cloudera will be using the money.

  3. Derrick Harris for GigaOm writes that the deal makes a lot of sense for both companies1:

    Cloudera needs capital and Intel’s huge sales force to keep up its engineering efforts and grow the company internationally.

    As part of the deal, Cloudera will be an early adopter of Intel gear and will optimize its Hadoop software to run on Intel’s latest technologies. Intel will port some of its work into the Cloudera distribution and will maintain its own Hadoop engineering team that will work alongside Cloudera’s engineers to help unite the two company’s goals.

  4. Jeff Kelly for SiliconAngle emphasizes the same channel advantages:

    Cloudera’s biggest reseller partner is Oracle. Based on my reading of the Intel announcement, the deal is not an official reseller partnership, but Intel will “market and promote CDH and Cloudera Enterprise to its customers as its preferred Hadoop platform.” Not quite as nice as having the Intel salesforce closing deals for it, but Cloudera stands to gain significant new business from the arrangement.


So how about this short list on how this round will be used by Cloudera:

  1. a part goes for international expansion
  2. a larger part goes to early shareholders
  3. the largest part goes into acquisitions

As for Intel, what if this investment also sealed an exclusive deal for Hadoop-centric Cloudera-supported Intel-powered appliance?


  1. Insert snarky comment here about a $740m deal that would not make sense to one of the parties. How about not making sense to any of them? 

Original title and link: Intel kills a Hadoop and feeds another (NoSQL database©myNoSQL)


Cloudera Search Interface: Inside Cloudera's customer support Enterprise Data Hub

Great use of their own technologies to better server the customer:

This application goes way beyond simple indexing and searching. We are using Cloudera Search, HBase, and MapReduce to process, store, and visualize stack traces that wouldn’t be possible with just a search index. How Monocle Stack Trace integrates with the larger CSI application goes way beyond that, though. It’s a great feeling when you are able to execute a search in Monocle Stack Trace that links directly to a point in time in a customer log file that an Impala query returned after churning through tens of GBs of data — done interactively from a Web UI on the order of a second or two.

I can easily see this becoming a real product used by software companies that offer direct customer support.

Original title and link: Cloudera Search Interface: Inside Cloudera’s customer support Enterprise Data Hub (NoSQL database©myNoSQL)

via: http://blog.cloudera.com/blog/2014/02/secrets-of-cloudera-support-inside-our-own-enterprise-data-hub/


The Forrester Wave for Hadoop market

Update: I’d like to thank the people that pointed out in the comment thread that I’ve messed up quite a few aspects in my comments about the report. I don’t believe in taking down posts that have been out for a while, so please be warned that basically this article can be ignored.

Thank you and my apologies for those comments that were a misinterpretation of the report..


This is the Q1 2014 Forrester Wave for Hadoop:

Forrester wave for Hadoop

A couple of thoughts:

  1. Cloudera, Hortonworks, MapR are positioned very (very) close.

    1. Hortonworks is position closer to the top right meaning they report more customers/larger install base
    2. MapR is higher on the vertical axis meaning that MapR’s strategy is slightly better.

      For me, MapR’s strategy can be briefly summarized as:

      1. address some of the limitations in the Hadoop ecosystem
      2. provide API-compatible products for major components of the Hadoop ecosystem
      3. use these Apache product (trade marked) names to advertise their products

      I think the 1st point above explains the better positioning of MapR’s current offering.

    3. Even if Cloudera has been the first pure-play Hadoop distribution it’s positioned behind behind both Hortonworks and MapR.

  2. IBM has the largest market presence. That’s a big surprise as I’m very rarely hearing clear messages from IBM.

  3. IBM and Pivotal Software are considered to have the strongest strategy. That’s another interesting point in Forrester’s report. Except the fact that IBM has a ton of data products and that Pivotal Software is offering more than Hadoop, I don’t know what exactly explains this position.

    The Forrester report Strategy positioning is based on quantifying the following categories: Licensing and pricing, Ability to execute, Product road map, Customer support. IBM and Pivotal are ranked the first in all these categories (with maximum marks for the last 3). As a comparison Hortonworks has 3/5 for Ability to execute — this must be related only to budget; Cloudera has 3/5 for both Ability to execute and Customer support.

    Pivotal is the 3rd last in terms of current offering. I guess my hypothesis for ranking Pivotal as 1st in terms of strategy is wrong.

  4. Microsoft who through the collaboration with Hortonworks came up with HDInsight, which basically enabled Hadoop for Excel and its data warehouse offering, it positioned the 2nd last on all 3 axes.

    No one seems to love Microsoft anymore.

  5. While not a pure Hadoop player, DataStax has been offering the DataStax Enterprise platform that includes support for analytics through Hadoop and search through Solr for at least 2 years. That’s actually way before anyone else from the group of companies in the Forrester’s report had anything similar1.

    This report focuses only on “general-purpose Hadoop solutions based on a differentiated, commercial Hadoop distribution”.

You can download the report after registering on Hortonwork’s site: here.


  1. DataStax is my employer. But what I wrote is a pure fact. 

Original title and link: The Forrester Wave for Hadoop market (NoSQL database©myNoSQL)


Bloomberg says Cloudera raises at least $200m in new round

Dina Bass and Serena Saitto (Bloomberg):

Cloudera Inc. is raising at least $200 million in a new round of financing from investors including Intel Corp., according to people with knowledge of the situation.

Not confirmed yet.

Original title and link: Bloomberg says Cloudera raises at least $200m in new round (NoSQL database©myNoSQL)

via: http://www.bloomberg.com/news/2014-03-18/cloudera-said-to-raise-at-least-200-million-in-funding.html


A guide to write and run Giraph jobs on Hadoop

A good setup guide by Mirko Kämpf:

In this how-to, you will learn how to use Giraph 1.0.0 on top of CDH 4.x using a simple example dataset, and run example jobs that are already implemented in Giraph. You will also learn how to set up your own Giraph- based development environment. The end result will be a setup (not intended for production) for writing and testing Giraph jobs, or just for playing around with Giraph and small sample datasets.

giraph

Anatomy of the Giraph data flow

Original title and link: A guide to write and run Giraph jobs on Hadoop (NoSQL database©myNoSQL)

via: http://blog.cloudera.com/blog/2014/02/how-to-write-and-run-giraph-jobs-on-hadoop/


Cloudera shipped a mountain... what can you read between the lines

Cloudera Engineering (@ClouderaEng) shipped a mountain of new product (production-grade software, not just technical previews): Cloudera Impala, Cloudera Search, Cloudera Navigator, Cloudera Development Kit (now Kite SDK), new Apache Accumulo packages for CDH, and several iterative releases of CDH and Cloudera Manager. (And, the Cloudera Enterprise 5 Beta release was made available to the world.). Furthermore, as always, a ton of bug fixes and new features went upstream, with the features notably but not exclusively HiveServer2 and Apache Sentry (incubating).

How many things can you read in this paragraph?

  1. a not that subtle stab at Hortonwork’s series of technical previews.
  2. more and more projects brought under the CDH umbrella. Does more ever become too much? (I cannot explain why, but my first thought was “this feels so Oracle-style”)
  3. Cloudera’s current big bet is Impala. SQL and low latency querying. A big win for the project, but not necessarily a direct financial win for Cloudera, was its addition as a supported service on Amazon Elastic MapReduce.

Original title and link: Cloudera shipped a mountain… what can you read between the lines (NoSQL database©myNoSQL)

via: http://blog.cloudera.com/blog/2014/01/this-month-and-year-in-the-ecosystem-december-2013/


Integrating R with Cloudera Impala for Real-Time queries on Hadoop

A very long tutorial by Istvan Szegedi on how to integrate R with Cloudera Impala, through the ODBC driver:

Cloudera Impala is an exciting new technology to provide real-time, interactive queries in Hadoop environment. It supports ODBC connectors and this makes it possible to integrate it with many popular BI tools and statistical software such as R. Together R and Impala provide an excellent combination for data analyst to process massive data sets efficiently and they can also support graphical representation of the result sets.

Original title and link: Integrating R with Cloudera Impala for Real-Time queries on Hadoop (NoSQL database©myNoSQL)

via: http://bighadoop.wordpress.com/2013/11/25/integrating-r-with-cloudera-impala-for-real-time-queries-on-hadoop/


Forbes Top 10 Most Funded Big Data Startups

  • MongoDB (formerly 10gen) $231m Document-oriented database
  • Mu Sigma $208m Data-Science-as-a-Service
  • Cloudera $141m Hadoop-based software, services and training
  • Opera Solutions $114 Data-Science-as-a-Service
  • Hortonworks $98 Hadoop-based software, services and training
  • Guavus $87 Big data analytics solution
  • DataStax $83.7 Cassandra-based big data platform
  • GoodData $75.5 Cloud-based platform and big data apps
  • Talend $61.6 App and business process integration platform
  • Couchbase $56 Document-oriented database

I’m not really sure there are any conclusions one could make based only on this data.

Original title and link: Forbes Top 10 Most Funded Big Data Startups (NoSQL database©myNoSQL)

via: http://www.forbes.com/sites/gilpress/2013/10/30/top-10-most-funded-big-data-startups-updated/


Cloudera Announces Support for Apache Accumulo - what, how, why

Cloudera, the leader in enterprise analytic data management powered byApache Hadoop™, today announced its formal support for, and integration with, Apache Accumulo, a highly distributed, massively parallel processing database that is capable of analyzing structured and unstructured data and delivers fine-grained user access control and authentication. Accumulo uniquely enables system administrators to assign data access at the cell- level, ensuring that only authorized users can view and manipulate individual data points. This increased control allows a database to be accessed by a maximum number of users, while remaining compliant with data privacy and security regulations.

What about HBase?

Mike Olson:

It offers a strong complement to HBase, which has been part of our CDH offering since 2010, and remains the dominant high-performance delivery engine for NoSQL workloads running on Hadoop. However, Accumulo was expressly built to augment sensitive data workloads with fine-grained user access and authentication controls that are of mission-critical importance for federal and highly regulated industries.

The way I read this is: if you don’t need security go with HBase. If you need advanced security features you go with Accumulo.

How?

While there aren’t any details about what formal support means, I assume Cloudera will start offering Accumulo as an alternative to HBase.

CE_diagram

I might be wrong though about Accumulo being a replacement for HBase. I’d love to learn how and why the 2 would co-exist.

Why?

The obvious reason is that Cloudera wants to get into government and super-regulated markets contracts where security is a top requirement.

Another reason might be that Cloudera is continuing to expand its portfolio to catch as many customers as possible. Something à la Oracle or IBM. The alternative would be to stay focused. Like Teradata.

Original title and link: Cloudera Announces Support for Apache Accumulo (NoSQL database©myNoSQL)

via: http://www.cloudera.com/content/cloudera/en/about/press-center/press-releases/release.html?ReleaseID=1859607


Hadoop Security and Cloudera’s new Role Based Access Control Sentry project

Security is an enterprise feature

At Hadoop Summit, Merv Adrian (VP Gartner) has shown data about Hadoop’s adoption in the enterprise space over the last 2 years and the numbers were great (actually they weren’t even good).

Hadoop vendors are becoming more aggressive in adding features that would make Hadoop enterprise ready. In some sectors (e.g. government, financial and health services) data security is regulated and this makes security features a top priority for adopting Hadoop in these spaces.

The state of Hadoop Security

Tony Baer1 has a nice guest post on ZDNet summarizing the current state of Hadoop security.

There’s a mix of activity on the open source and vendor proprietary sides for addressing the void. There are some projects at incubation stage within Apache, or awaiting Apache approval, for providing LDAP/Active Directory linked gateways (Knox), data lifecycle policies (Falcon), and APIs for processor-based encryption (Rhino). There’s also an NSA-related project for adding fine-grained data security (Accumulo) based on Google BigTable constructs. And Hive Server 2 will add the LDAP/AD integration that’s current missing.

What’s interesting to note is that many big vendors have been focusing on adding proprietary security and auditing features to Hadoop.

Cloudera’s post introducing Sentry also provides a short overview of security in Hadoop, by looking at 4 areas:

  1. Perimeter security: network security, firewall, and Kerberos authentication
  2. Data security: encryption and masking currently available through a combination of recent work in the Hadoop community and vendor solutions.
  3. Access security: fine grained ACL
  4. Visibility: monitoring access and auditing

Sentry: Role-based Access Control for Hadoop

Cloudera has announced Sentry a fine grained role-based access control solution for Hadoop meant to simplify and augment the current course-grained HDFS-level authorization available in Hadoop.

Sentry architecture

Sentry architecture

Sentry comprises a core authorization provider and a binding layer. The core authorization provider contains a policy engine, which evaluates and validates security policies, and a policy provider, which is responsible for parsing the policy. The binding layer provides a pluggable interface that can be leveraged by a binding implementation to talk to the policy engine. (Note that the policy provider and the binding layer both provide pluggable interfaces.)

At this time, we have implemented a file-based provider that can understand a specific policy file format.

According to the post, right now only Impala and Hive have bindings for Sentry. This makes me wonder how Sentry is deployed in a Hadoop cluster so other layers could take advantage of the Sentry ACL. I see such a security feature implemented very close to HDFS so it would basically work with all types of access to data stored.

For more details about Sentry, read the official post With Sentry, Cloudera Fills Hadoop’s Enterprise Security Gap.

There are also numerous rewrites of the announcement:


  1. Tony Baer is a principal analyst covering Big Data at Ovum. 

Original title and link: Hadoop Security and Cloudera’s new Role Based Access Control Sentry project (NoSQL database©myNoSQL)


With New Product Packaging, Adopting the Platform for Big Data is Even Easier

In addition, by choosing Cloudera Enterprise, you open the door to add other capabilities to your subscription as you wish – powerful tools like:

  • Cloudera Enterprise RTD (Real Time Delivery) – Support for HBase
  • Cloudera Enterprise RTQ (Real Time Query) – Support for Impala
  • Cloudera Enterprise BDR (Backup and Disaster Recovery) - Support for BDR
  • Cloudera Navigator – Data management for your Cloudera Enterprise deployment

And when Cloudera Search (beta) becomes generally available, you’ll be able to add:

  • RTS (Real Time Search) – Support for Cloudera Search

Isn’t this called nickel-and-diming?

Original title and link: With New Product Packaging, Adopting the Platform for Big Data is Even Easier (NoSQL database©myNoSQL)

via: http://blog.cloudera.com/blog/2013/06/adopting-cloudera-platform-even-easier/


Announcing Open Source, Interactive Search on Hadoop

Announced through a webinar with all big name analysts listening, Cloudera announced Cloudera Search:

Cloudera Search brings full-text, interactive search and scalable indexing to your data in Hadoop. Cloudera Search adds to and extends the value of Apache Solr™, the enterprise standard for open source search. With Cloudera’s 100% open source Big Data platform, CDH, Cloudera Search gains the same fault tolerance, scale, visibility, and flexibility provided to other workloads, like MapReduce, Apache Hive™, and Cloudera Impala.

You know who did this first, right? DataStax. And it was over a year ago.

Original title and link: Announcing Open Source, Interactive Search on Hadoop (NoSQL database©myNoSQL)

via: http://app.go.cloudera.com/e/es.aspx?s=1465054361&e=9583&elq=2a81ee10fb714c3c9afc2225da89700c