ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

What’s New and Upcoming in HDFS

Great retrospective with many architecture details of the improvements added to HDFS in 2012 and what is planned for this year by Todd Lipcon.

For a quick overview:

  • 2012: HDFS 2.0
    • HA (in 2 phases)
    • Performance improvements:
      • for Impala: faster libhdfs, APIs for spindle-based scheduling
      • for HBase and Accumulo: direct reads from block files in secure environments, application level checksums and IOPS elimintation
    • on-the-wire encryption
    • rolling upgrades and wire compatibility
  • 2013:
    • HDFS snapshots
    • better storage density and file formats
    • caching and hierarchical storage management

Original title and link: What’s New and Upcoming in HDFS (NoSQL database©myNoSQL)