NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



SQS: All content tagged as SQS in NoSQL databases and polyglot persistence

Pros and Cons of Redis-Resque and Amazon SQS

Eric Lubow published a comparison of Resque, a Redis-based queue system, and Amazon SQS:

Resque has to be run locally (meaning within your environment). And because it’s native to your architecture, it can be incredibly fast in comparison. It’s durability comes into question where even though Redis allows you to dump your data to disk under varying circumstances (say once per second) or have a master/slave architecture, ultimately you are still bound by the potential loss of a single machine (aka a single point of failure candidate). While this may only be the case until Redis Cluster is released, comparisons have been made with the tools at hand. With SQS, it is much more durable. They also have the notion of in-flight messages. This means that the message is pulled off the queue but never deleted until the delete command is sent for that message id. So if you lose your worker mid-processing of the event, that event isn’t lost for good. The message will be timed out after being in-flight for 5 minutes and then dropped back onto the available queue. While this functionality could be written into Resque, it just wasn’t part of the fundamental design.

Related to Redis durability, you should read Redis persistence demystified. The conclusion might surprise those that associate Redis is a pure in-memory solution.

Original title and link: Pros and Cons of Redis-Resque and Amazon SQS (NoSQL database©myNoSQL)