memcached: All content tagged as memcached in NoSQL databases and polyglot persistence
A new great article from Todd Hoff dissecting the DataSift architecture:
In terms of data store, DataSift architecture includes:
- MySQL (Percona server) on SSD drives
- HBase cluster (currently, ~30 hadoop nodes, 400TB of storage)
- Memcached (cache)
- Redis (still used for some internal queues, but probably going to be dismissed soon)
Leave whatever you were doing and go read it now.
Original title and link: DataSift Using MySQL, HBase, Memcached to Deal With Twitter Firehose ( ©myNoSQL)
Earlier today when writing about a benchmark of MongoDB, Redis, Memcached-based Rails caches, I’ve refered to eviction policies and their possible impact on the cache performance. But as I wasn’t sure about how Memcached works in a couple of areas, I did a bit of research and found a tweet from Tim de Pater pointing to 2 great articles about Memcached.
Joshua Thijssen’s post covers 4 topics: Memcached operations
Big-O, LRU eviction policy, memory allocation, consistent hashing.
Now, in order to combat this “malloc()” problem, memcache does its own memory management by default (you can let memcache use the standard malloc() function, but that would not be advisable). Memcache’s memory manager will allocate the maximum amount of memory from the operating system that you have set (for instance, 64Mb, but probably more) through one malloc() call. From that point on, it will use its own memory manager system called the slab allocator.
Then this post describes in great detail how Memcached eviction policy works:
The other day I was chatting with a colleague about Memcached. Eviction policy came up, and I casually mentioned that Memcache isn’t strictly LRU. But a quick Bing search said Memcache is LRU, like this Wikipedia entry. Hmm, I was 99.9% sure Memcache is not LRU, something to do with how it manages memory, but maybe I was wrong all these years. After reading through some Danga mailing lists and documentation, the answer is, Memcached is LRU per slab class, but not globally LRU.
Original title and link: Memcached Internals: Memory Allocation, Eviction Policy, Consistent Hashing ( ©myNoSQL)
Though it looks like mongo-store demonstrates the best overall performance, it should be noted that a mongo server is unlikely to be used solely for caching (the same applies to redis), it is likely that non-caching related queries will be running concurrently on a mongo/redis server which could affect the suitability of these benchkmarks.
I’m not a Rails user, so please take these with a grain of salt:
without knowing the size of the cached objects, at 20000 iterations most probably neither MongoDB, nor Redis have had to persist to disk.
This means that all three of memcached, MongoDB, Redis stored data in memory only
if no custom object serialization is used by any of the memcached, MongoDB, Redis caches, then the performance difference is mostly caused by the performance of the driver
it should not be a surprise to anyone that the size of the cached objects can and will influence the results of such benchmarks
there doesn’t seem to be any concurrent access to caches. Concurrent access and concurrent updates of caches are real-life scenarios and not including them in a benchmark greatly reduces the value of the results
none of these benchmarks doesn’t seem to contain code that measure the performance of cache eviction
Except the case where any of these forces a disk write ↩
Original title and link: Rails Caching Benchmarked: MongoDB, Redis, Memcached ( ©myNoSQL)
While many will find this new service useful, it is a bit of a disappointement that Amazon took the safe route and went with pure Memcached. The only notable feature of Amazon ElastiCache is automatic failure detection and recovery. But compared with Membase (and the soon to be released Couchbase 2.0) it is missing clustering, replication, support for virtual nodes, etc. Even if advertising a push-button scaling, ElastiCache will lose cached data on adding or removing instances.
The pace at which Amazon is launching new services is indeed impressive. I’m wondering what will be the first NoSQL database that will get official Amazon support.
Original title and link: Memcached in the Cloud: Amazon ElastiCache ( ©myNoSQL)