NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



Resque: All content tagged as Resque in NoSQL databases and polyglot persistence

Parallelizing Work: From Single-Thread to Redis

An old post from Santosh Kumar describes the sequence of steps to parallelize processing from single threaded apps to using Redis and Resque and going through forking and process pools. Santosh has written extensively about using Redis for concurrency or as a Message oriented middleware (MOM).

The next time you need to get some background job action going, stop yourself from just grabbing a library. Instead, toy around with redis lists a little. You’ll be surprised by how much you can accomplish with just straight redis primitives.

While I agree with the usefulness of Redis[1], none of his posts say what comes after it. And that’s specialized solutions.

  1. Last weekend I’ve built a tool for myself using Redis.  

Original title and link: Parallelizing Work: From Single-Thread to Redis (NoSQL database©myNoSQL)


Screencast: Background Jobs With Resque

A Resque, the Redis backed background jobs tool, screencast by the Railscasts peeps. The 12 minutes video is available after the break and you can download the source code from the Railscast website.

Ruby On Rails: Cron Job Scheduling using Redis, Resque, and Rufus

The 5 R: Ruby on Rails, Redis, Resque, Rufus.

Now the question is what’s the best way to run scheduled tasks in a Rails environment? Generally, Rails developers can use an application specific crontab to run application tasks. […] Downsides of Cron are:

  • Cron works well on a single server but what if you need to scale to multiple app servers? You’ll need to introduce some kind of lock to avoid concurrency problems. Also, you need to maintain these shared locks.
  • The other problem with Cron is that they are difficult to debug.
  • Cron is for scheduling things, not doing them.

This can almost always be better to place jobs in queue and place the worker system to perform or to execute those jobs. Luckily there is a very good gem named ‘resque‘ available in Ruby on Rails.

Original title and link: Ruby On Rails: Cron Job Scheduling using Redis, Resque, and Rufus (NoSQL databases © myNoSQL)


Simple Ruby Workers with MongoDB and Resque

After some research, we decided to use resque from github, but to adapt it to use mongodb instead of redis. The advantage here is that we already have a significant investment in mongodb, so we would not be introducing a new type of server to our infrastructure. Mongodb also has some features that redis does not, and we used those to build some interesting new features into resque.

After reading the post, I couldn’t identify those additional MongoDB features mentioned above. But I do agree you shouldn’t bring in new infrastructure requirements with every new tools you are using. Very heterogeneous platforms are difficult and expensive to maintain.

Original title and link: Simple Ruby Workers with MongoDB and Resque (NoSQL databases © myNoSQL)