ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

Hadoiop: All content tagged as Hadoiop in NoSQL databases and polyglot persistence

Building a self-serve platform for Hadoop

What big users, in this case Pinterest, would get, ideally, from Hadoop:

Though Hadoop is a powerful processing and storage system, it’s not a plug and play technology. Because it doesn’t have cloud or elastic computing, or non-technical users in mind, its original design falls short as a self-serve platform. Fortunately there are many Hadoop libraries/applications and service providers that offer solutions to these limitations. Before choosing from these solutions, we mapped out our Hadoop setup requirements.

If you go through the 7 items listed in this post, you’ll have to agree that none sounds unreasonable. Some of these requirements might be Pinterest specific, or at least derived from their size, but I can see how each of them would simplify things. On the other hand, I’m not aware of work being done in any of these areas (nb: security is a hairy topic and everyone wants exactly what they are using).

Original title and link: Building a self-serve platform for Hadoop (NoSQL database©myNoSQL)

via: http://engineering.pinterest.com/post/92742371919/powering-big-data-at-pinterest