Currently, we are using MySQL and delayed_job as a background job system, but it gets “iffy” under high load. We are looking to replace it with Redis in conjunction with Resque. [editor’s note: The creator of Resque, @defunkt, will be at POSSDEV.]
This makes a lot of sense considering Redis Queues are hot!
We are also looking at MongoDB to possibly replace MySQL. Mostly because with our level of data it is difficult to add columns and joins are getting so expensive that we are forced to denormalize in the name of performance anyway. So, we ask ourselves why are we using a relational database. MongoDB is so much easier to use.
I am not sure denormalization is a good enough reason to move from MySQL to MongoDB, so there might be probably other aspects that are not presented in the interview. Plus, it looks like scaling out MongoDB is at least as complicated as scaling MySQL.
I have personally been playing with CouchDB and see several applications for it. For example, if you need easy replication between a laptop and headquarters or between New York and London. If you do it right, you don’t need a application server, just the database. The internal architecture is amazing.
Not to mention that replication in CouchDB got even better in 0.11.
Looking to the future, projects like Cassandra and Riak have the right architecture. You can “just add nodes” without micro-managing master/slave relationships or sharding.
- Both Cassandra and Riak distributed architecture is based on the ☞ Dynamo paper, which currently seems to be the most validated approach.
- Project Voldemort is working hard to get at the same level of ease of scaling out and down. When compared with Riak, both show interesting performance (nb the benchmark is not scientific).