As I was hoping and expecting, Datameer updated the chart visualizing Hadoop’s business side ecosystem:

It shouldn’t be a surprise to anyone that the top most connected companies in the Hadoop space are Cloudera and Hortonworks. They outrank the IT industry mammoths: IBM, HP, Microsoft, Oracle, SAP, etc.

Original title and link: Hadoop Business Ecosystem as of January 2013 (NoSQL database©myNoSQL)

via: http://www.datameer.com/blog/perspectives/hadoop-ecosystem-as-of-january-2013-now-an-app.html

Deveraj Das’s post on Hortonworks blog details the current and future work on HBase:

1. Reliability and High Availability (all data always available, and recovery from failures is quick)
2. Autonomous operation (minimum operator intervention)
3. Wire compatibility (to support rolling upgrades across a couple of versions at least)
4. Cross data-center replication (for disaster recovery)
5. Snapshots and backups (be able to take periodic snapshots of certain/all tables and be able to restore them at a later point if required)
6. Monitoring and Diagnostics (which regionserver is hot or what caused an outage)

Future:

1. Better and improved clients (asynchronous clients, and, in multiple languages)
2. Cell-level security (access control for every cell in a table)
3. Multi-tenancy (HBase becomes a viable shared platform for multiple applications using it)
4. Secondary indexing functionality

Current work=reliability. Future work=usability.

Original title and link: HBase Roadmap (NoSQL database©myNoSQL)

via: http://hortonworks.com/blog/hbase-futures/

Although Pig is designed as a data flow language, it supports all the functionalities required by TPC-H; thus it makes sense to use TPC-H to benchmark Pig’s performance. Below is the final result.

Original title and link: Pig Performance and Optimization Analysis (NoSQL database©myNoSQL)

via: http://hortonworks.com/blog/pig-performance-and-optimization-analysis/

Just a new HTTP interface for Hadoop file system. The main differences between HttpFS and WebHDFS are that this one is created by Cloudera, not Hortonworks (on top of their previos Hoop library) and:

HttpFs is a proxy so, unlike WebHDFS, it does not require clients be able to access every machine in the cluster. This allows clients to to access a cluster that is behind a firewall via the WebHDFS REST API.

Question is: if they are API compatible and both open source, why not unifying them?

Original title and link: HttpFS: Another Hadoop File System Over HTTP (NoSQL database©myNoSQL)

via: http://www.cloudera.com/blog/2012/08/httpfs-for-cdh3-the-hadoop-filesystem-over-http/

I’d normally wouldn’t link to a pat on the back post, but Hortonworks’ presence in the Hadoop market has accelerated its evolution and adoption. Plus the promises Hortonworks made 1 year ago represents a very good list of the shortcomings new adopters of Hadoop are still facing:

• make Apache Hadoop easier to install, manage, and use
• make Apache Hadoop more robus
• make Apache Hadoop easier to integrate and extend
• deliver an ever-increasing array of services aimed at improving the Hadoop experience and support in the growing needs of enterprises, systems integrators and technology vendors

Original title and link: Hortonworks at 1 Year: Promises and Achievements (NoSQL database©myNoSQL)

via: http://hortonworks.com/blog/happy-birthday-hortonworks/

Excellent infographic about the relationships in the Hadoop market created with Datameer:

A while ago I’ve created a Google Spreadsheet in which I’ve tried to track all these relationships, but going through PR announcements wasn’t really my thing. Now there’s a CSV file with all this data.

Original title and link: The Hadoop Ecosystem Relationships (NoSQL database©myNoSQL)

via: http://www.cloudera.com/blog/2012/07/the-hadoop-ecosystem-visualized-in-datameer/

Compare the title, which is the subtitle of the InformationWeek post, with this paragraph which reflects the reality:

Both Cloudera and Hortonworks will be distributing open source software from Apache’s Hadoop 2.3 release, which includes upgrades aimed at high-availability and improved security. The release includes a hot-failover for the NameNode (metadata server) of the Hadoop Distributed File System (HDFS), which has long been a single point of failure.

Cloudera is indeed one of the biggest Hadoop contributors and a company that have helped a lot proving and thus popularizing Hadoop through their packaging of open source Hadoop ecosystem components paired with their management tool (Cloudera Manager). But NameNode high availability and security improvements are part of the Apache Hadoop source code.

Original title and link: Looking to Stay Ahead of Hortonworks and MapR in the Hadoop Market, Cloudera Delivers High Availability, Better Security, and Easier System Management (NoSQL database©myNoSQL)

via: http://www.informationweek.com/news/software/info_management/240001574

A Hortonworks post listing the 7 key drivers for the Big Data market from the business, technical, and financial perspective:

Original title and link: Big Data: Transactions Plus Interactions Plus Observations (NoSQL database©myNoSQL)

via: http://hortonworks.com/blog/7-key-drivers-for-the-big-data-market/

Curt Monash shares what he heard from his customers:

• Over half of Cloudera’s customers (nb 100 subscription customers) use HBase
• Hortonworks thinks a typical enterprise Hadoop cluster has 20-50 nodes, with 50-100 already being on the large side.
• There are huge amounts of Elastic MapReduce/Hadoop processing in the Amazon cloud. Some estimates say it’s the majority of all Amazon Web Services processing.

Original title and link: Notes on the Hadoop and HBase Markets (NoSQL database©myNoSQL)

via: http://www.dbms2.com/2012/04/24/notes-on-the-hadoop-and-hbase-markets/