NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



Redis: All content tagged as Redis in NoSQL databases and polyglot persistence

Redis Partial Resyncs and Synchronous Replication

Speaking about what I think to be a wrong decision in MongoDB replication, I’ve found the following in a post about Redis replication:

If a slave lost the connection, it connects again, see if the master RUNID is the same, and asks to continue from a given offset. If this is possible, we continue, nothing is lost, and a full resynchronization is not needed. Otherwise if the offset is about data we no longer have in the backlog, we full resync.

The last part made me think that the behavior of Redis’s replication is exactly the same as the one described by the OP when losing data on the secondary node after setting up replication. But it is the first part that makes the difference: the slave checks first if the master is the same.

Original title and link: Redis Partial Resyncs and Synchronous Replication (NoSQL database©myNoSQL)


Reliable Delivery Message Queues With Redis

A very detailed article about creating a message queue (nb: not Pub/Sub) using Redis:

One particular combination has recently allowed us to implement a moderately (read: good enough) reliable message delivery mechanism for multiple consumers consuming messages at their own pace.

Redis allowed for all of the implementation characteristics we were looking for, namely:

  • Concurrency: Because all operations in Redis are atomic, supporting concurrency without too much of a hassle is straightforward.
  • Persistence: Configured properly, we can ensure persistence of our queues to disk using one of the supported Redis persistence strategies.
  • Lightweight: Using Redis from any language/platform is extremely simple and provisioning it / maintaining it on a production server is dead easy.

I’m always saying to use the right tool for the job, but maintaining the simplicity of a system architecture is an art too.

Original title and link: Reliable Delivery Message Queues With Redis (NoSQL database©myNoSQL)


Powered by Redis: Animated Gifs

I know many of you will really, really like this: an animated Gifs site powered by Redis:

Oh yes

It’s Node.js and Redis and the source code is on GitHub.

Sam Breed

Original title and link: Powered by Redis: Animated Gifs (NoSQL database©myNoSQL)

Modeling a Simple Social App Using SQL and Redis

Felix Lin sent me a link to the slides he presented at NoSQL Taiwan meetup. There are 105 of them!

The deck covers:

  • how to build a simple social site using SQL
  • what are the performance issues with SQL
  • how to use the data structures in Redis for getting the same features
  • how to solve the performance issues in SQL by using Redis

Check them up after the break:

Redis 2.6 Released. Finally

Salvatore Sanfilippo:

Scripting, bitops, and all the big features are good additions but my feeling is that Redis 2.6 is especially significative as a step forward in the maturity of the Redis implementation. This does not mean that’s bug free, it’s new code and we’ll likely discover bugs especially in the early days as with every new release that starts to be adopted more and more.

Redis 2.6 is the release I’ve written about so many times that is tempting to just say finally.

Original title and link: Redis 2.6 Released. Finally (NoSQL database©myNoSQL)


Redis and Memcached Benchmark on Amazon Cloud

Garantia Data, providers of Redis and Memcached as-a-Service-in-the-Amazon-Cloud, published the results of a throughput and latency benchmark for different AWS deployment models:

The first thing we looked at when putting together our benchmark was the various architectural alternatives we wanted to compare. Users typically choose the most economical AWS instance based on the initial size estimate of their dataset, however, it’s crucial to also keep in mind that other AWS users might share the same physical server that runs your data (as nicely explained by Adrian Cockcroft here). This is especially true if you have a small-to-medium dataset, because instances between m1.small and m1.large are much more likely to be shared on a physical server than large instances like m2.2xlarge and m2.4.xlarge, which typically run on a dedicated physical server. Your “neighbours” may become “noisy” once they start consuming excess I/O and CPU resources from your physical server. In addition, small-to-medium instances are by nature weaker in processing power than large instances.

Only two comments:

  1. it’s not clear if there were multiple instances of Redis used per machine when the chosen instances had multi-cores
  2. I would have really liked to also have a pricing comparison in the conclusion section

Original title and link: Redis and Memcached Benchmark on Amazon Cloud (NoSQL database©myNoSQL)


Python Whoosh With Redis Storage

Zohaib Sibt-e-Hassan:

For me Redis can turn out to be an excellent inverted index storage system due to its available data-structures. I won’t be implementing anything from scratch or anything; but just to kick off and get some more minds into circle, I took whoosh (no particular reason) and tried to modify its storage for Redis, so that it becomes a semi equivalent of RamStorage.

Data structures and efficient specialized operations can indeed be the building blocks for implementing inverted index storages.

Original title and link: Python Whoosh With Redis Storage (NoSQL database©myNoSQL)


Redis Live: Real Time Dashboard for Redis


Redis Live is a dashboard application with a number of useful widgets. At it’s heart is a monitoring script that periodically issues INFO and MONITOR command to the redis instances and stores the data for analytics.

Redis Live

Built in Python with tornado and available on GitHub.

Original title and link: Redis Live: Real Time Dashboard for Redis (NoSQL database©myNoSQL)


Demoing the Python-Based Map-Reduce R3 Against GitHub Data

A nice demo of the recently announced MapReduce engine written on Python r3 library1 against commit histories from GitHub:

It is pretty simple to get r3 to do some cool calculations for us. I got the whole sample in a very short amount of time. It took me more time to write this post than to make r3 calculate the commiter percentages.

  1. r3 is a Python-based map-reduce engine using Redis as a backend 

Original title and link: Demoing the Python-Based Map-Reduce R3 Against GitHub Data (NoSQL database©myNoSQL)


Pros and Cons of Redis-Resque and Amazon SQS

Eric Lubow published a comparison of Resque, a Redis-based queue system, and Amazon SQS:

Resque has to be run locally (meaning within your environment). And because it’s native to your architecture, it can be incredibly fast in comparison. It’s durability comes into question where even though Redis allows you to dump your data to disk under varying circumstances (say once per second) or have a master/slave architecture, ultimately you are still bound by the potential loss of a single machine (aka a single point of failure candidate). While this may only be the case until Redis Cluster is released, comparisons have been made with the tools at hand. With SQS, it is much more durable. They also have the notion of in-flight messages. This means that the message is pulled off the queue but never deleted until the delete command is sent for that message id. So if you lose your worker mid-processing of the event, that event isn’t lost for good. The message will be timed out after being in-flight for 5 minutes and then dropped back onto the available queue. While this functionality could be written into Resque, it just wasn’t part of the fundamental design.

Related to Redis durability, you should read Redis persistence demystified. The conclusion might surprise those that associate Redis is a pure in-memory solution.

Original title and link: Pros and Cons of Redis-Resque and Amazon SQS (NoSQL database©myNoSQL)


Using Apache to Save Data in Redis

Using a bash script and redis-cli to write Apache stats directly into Redis:

In one of my projects, I was using redis database to collect some statistics and I thought of saving data into it at apache level. This would considerably enhance the speed of saving data as it would not require the interception of grails to save data.

Original title and link: Using Apache to Save Data in Redis (NoSQL database©myNoSQL)


Should We Namespace Redis?

Bruno Michel:

But Redis doesn’t let you have several databases with this naming schema. So my teammates have used something else for their projects: redis-namespace. The idea is to prefix all keys with a given string (foobar_development in my previous example) and a separator (:) by subclassing the Redis library for Ruby.

I’m a bit confused as I knew Redis support multiple databases per server—indeed they don’t have a nice name, but just an index. Still it should work just fine for this scenario.

Original title and link: Should We Namespace Redis? (NoSQL database©myNoSQL)