Just in case you thought someone made up the whole thing about the status of CouchDB being confusing:

Found in Koji Kawamura‘s Introduction of CouchDB JP slides .

On the other hand I’m still trying to figure out if things in CouchDB land were more confusing than the various Hadoop versions out there. If you compare the two genealogy trees you’ll notice a reversed pattern.

Original title and link: History of Couch Projects (NoSQL database©myNoSQL)

If you take a look at the topic of security in the NoSQL context, you’ll notice that things are far from being perfect. So, any contributions in this area are welcome. Patrik Karlsoon added a couple of network exploration Nmap scripts for Riak, Redis, and Memcached. And while these will not help much with security they might proove useful for managing your NoSQL deployments:

• Added the script riak-http-info that lists version and statistics information from the Basho Riak distributed database.

• Added the script memcached-info that lists version and statistics information from the distributed memory object caching service memcached

• Added the script redis-info that lists version and statistic information gathered from the Redis network key-value store.

• Added the redis library and the script redis-brute that performs brute force password guessing against the Redis network key-value store.

Original title and link: Nmap Scripts for Riak, Redis, Memcached (NoSQL database©myNoSQL)

ServerFault Q&A:

Q: Copy MySQL to RAM as a poor man’s memcached replacement?

A: Use the the MEMORY storage engine on a read only slave to do your reads from, is exactly what you really want and a sane setup. Forget “dumping it to disk” (?!) or other strange things.

You can even put the slave as another instance on your existing server if you can’t afford to setup a dedicated slave, but properly tuning the MySQL parameters for mostly read workloads will bring a significant performance enhancement too!

Jiminy

Original title and link: MySQL MEMORY as Poor Man’s Memcached Replacement (NoSQL database©myNoSQL)

A new great article from Todd Hoff dissecting the DataSift architecture:

Click for a larger image

In terms of data store, DataSift architecture includes:

• MySQL (Percona server) on SSD drives
• HBase cluster (currently, ~30 hadoop nodes, 400TB of storage)
• Memcached (cache)
• Redis (still used for some internal queues, but probably going to be dismissed soon)

Leave whatever you were doing and go read it now.

Original title and link: DataSift Using MySQL, HBase, Memcached to Deal With Twitter Firehose (NoSQL database©myNoSQL)

Earlier today when writing about a benchmark of MongoDB, Redis, Memcached-based Rails caches, I’ve refered to eviction policies and their possible impact on the cache performance. But as I wasn’t sure about how Memcached works in a couple of areas, I did a bit of research and found a tweet from Tim de Pater pointing to 2 great articles about Memcached.

Joshua Thijssen’s post covers 4 topics: Memcached operations Big-O, LRU eviction policy, memory allocation, consistent hashing.

Now, in order to combat this “malloc()” problem, memcache does its own memory management by default (you can let memcache use the standard malloc() function, but that would not be advisable). Memcache’s memory manager will allocate the maximum amount of memory from the operating system that you have set (for instance, 64Mb, but probably more) through one malloc() call. From that point on, it will use its own memory manager system called the slab allocator.

Then this post describes in great detail how Memcached eviction policy works:

The other day I was chatting with a colleague about Memcached.  Eviction policy came up, and I casually mentioned that Memcache isn’t strictly LRU.  But a quick Bing search said Memcache is LRU, like this Wikipedia entry.  Hmm, I was 99.9% sure Memcache is not LRU, something to do with how it manages memory, but maybe I was wrong all these years.  After reading through some Danga mailing lists and documentation, the answer is, Memcached is LRU per slab class, but not globally LRU.  

Original title and link: Memcached Internals: Memory Allocation, Eviction Policy, Consistent Hashing (NoSQL database©myNoSQL)

A couple of Rails caching solutions—file, memcached, MongoDB, and Redis—benchmarked firstly here by Steph Skardal and then here by Thomas W. Devol. Thomas W. Devol concludes:

Though it looks like mongo-store demonstrates the best overall performance, it should be noted that a mongo server is unlikely to be used solely for caching (the same applies to redis), it is likely that non-caching related queries will be running concurrently on a mongo/redis server which could affect the suitability of these benchkmarks.

I’m not a Rails user, so please take these with a grain of salt:

• without knowing the size of the cached objects, at 20000 iterations most probably neither MongoDB, nor Redis have had to persist to disk.

  This means that all three of memcached, MongoDB, Redis stored data in memory only^[1]

• if no custom object serialization is used by any of the memcached, MongoDB, Redis caches, then the performance difference is mostly caused by the performance of the driver

• it should not be a surprise to anyone that the size of the cached objects can and will influence the results of such benchmarks

• there doesn’t seem to be any concurrent access to caches. Concurrent access and concurrent updates of caches are real-life scenarios and not including them in a benchmark greatly reduces the value of the results

• none of these benchmarks doesn’t seem to contain code that measure the performance of cache eviction

Original title and link: Rails Caching Benchmarked: MongoDB, Redis, Memcached (NoSQL database©myNoSQL)