A Key-Value Cache for Flash Storage: Facebook's McDipper and What Preceded It
A post on Facebook Engineering’s blog:
The outgrowth of this was McDipper, a highly performant flash-based cache server that is Memcache protocol compatible. The main design goals of McDipper are to make efficient use of flash storage (i.e. to deliver performance as close to that of the underlying device as possible) and to be a drop-in replacement for Memcached. McDipper has been in active use in production at Facebook for nearly a year.
I know at least 3 companies that have attacked this problem with different approaches and different results:
- Couchbase (ex-Membase, ex-NorthScale) started as a persistent clustered Memcached implementation. It was not optimized for Flash storage though. Today’s Couchbase product is still based on the memcache protocol, but it adding new features inspired by CouchDB.
- RethinkDB, a YC company and the company that I work for, has worked and released in 2011 a Memcache compatible storage engine optimized for SSDs. Since then, RethinkDB has been building and released an enhanced product, a distributed JSON store with advanced data manipulation support.
- Aerospike (ex Citrusleaf) sells a storage engine for flash drives. Its API is not Memcache compatible though.
People interested in this market segment have something to learn from this.
Original title and link: A Key-Value Cache for Flash Storage: Facebook’s McDipper and What Preceded It (©myNoSQL)
