The architecture of a fault-tolerant ad network built on top of HAProxy, Apache with mod_wsgi and Python, Redis, a bit of PostgreSQL and ActiveMQ deployed on AWS:

The real workhorse of our ad targeting platform was Redis. Each box slaved from a master Redis, and on failure of the master (which happened once), a couple “slaveof” calls got us back on track after the creation of a new master. A combination of set unions/intersections with algorithmically updated targeting parameters (this is where experimentation in our setup was useful) gave us a 1 round-trip ad targeting call for arbitrary targeting parameters. The 1 round-trip thing may not seem important, but our internal latency was dominated by network round-trips in EC2. The targeting was similar in concept to the search engine example I described last year, but had quite a bit more thought regarding ad targeting. It relied on the fact that you can write to Redis slaves without affecting the master or other slaves. Cute and effective. On the Python side of things, I optimized the redis-py client we were using for a 2-3x speedup in network IO for the ad targeting results.

Original title and link: Building an Ad Network Ready for Failure (NoSQL database©myNoSQL)

via: http://dr-josiah.blogspot.com/2011/06/building-ad-network-ready-for-failure.html

Little trick to get access to your logs:

This is a little python log handler that lets you stream your log data over a redis pub/sub channel, so you can monitor your system in real time from any redis client.[…] I am also interested in mapping the python logging names (foo.bar.baz) to redis channels that would enable you to listen to arbitrary applications, or subscribe to a collection using the redis wildcard subscriptions.

On GitHub.

Original title and link: Watch System Logs in Real Time with Redis Pub/Sub (NoSQL databases © myNoSQL)

via: http://jedp.posterous.com/a-python-logging-handler-for-redis-pubsub

CouchOne has published another CouchDB success story:

The team felt a relational database and full web application framework would be overkill and inflexible. With these factors in mind they chose to use Flask and CouchDB. An additional factor that appealed to the developers was CouchDB’s native storage of JSON documents. They communicate and collect various data with the Facebook Graph API, which speaks JSON — so it just made sense to dump that data straight into CouchDB.

The final application was delivered in aproximately one week — though the article forgets to mention the size of team that worked on the project.

Original title and link: CouchDB for Timely and Budget Limited Projects (NoSQL databases © myNoSQL)

via: http://www.couchone.com/case-study-wwf

redis_graph is a graph database implemented in Python. It shows how awesome Redis is as the implementation is under 40 lines of code.

The perfomance should be excellent, while scaling it might be an issue. I would not recommend using it if you are storing nodes in the millions range.

Except being fun — as fun as storing a social graph in Redis or implementing a social graph using Redis — I’d say using a real graph database is probably a better approach.

Original title and link: redis_graph: Redis-based Graph Database for Python (NoSQL databases © myNoSQL)

via: http://amix.dk/blog/post/19592

For Pylons users:

About a year ago, Chris Moos came up with a nice tutorial on how to integrate Pylons with CouchDB.  Well, times have changed: Pylons is now 1.0 (thus, syntactical differences), and I’m into MongoDB for my datastore. This document is revised/edited/rewritten/and updated for MongoDB.

Django users seem to be a bit more active in the NoSQL space though.

Original title and link: Pylons & MongoDB: User Registration & Login (NoSQL databases © myNoSQL)

via: http://devinfee.com/blog/2010/11/04/pylons-mongodb-user-registration-login/

A story of using Hadoop at Yelp and migrating it to Amazon Elastic MapReduce:

We used to do what a lot of companies do, which is run a Hadoop cluster. We had a dozen or so machines that we otherwise would have gotten rid of, and whenever we pushed our code to our webservers, we’d push it to the Hadoop machines.

It was also not so cool. You couldn’t really tell if a job was going to work at all until you pushed it to production. But the worst part was, most of the time our cluster would sit idle, and then every once in a while, a really beefy job would come along and tie up all of our nodes, and all the other jobs would have to wait.

Yelp has released their Python library for running MapReduce jobs on Hadoop or Amazon Elastic MapReduce on ☞ GitHub.

Original title and link: Hadoop and Elastic MapReduce at Yelp (NoSQL databases © myNoSQL)

via: http://engineeringblog.yelp.com/2010/10/mrjob-distributed-computing-for-everybody.html

The first reason that bring me away from the great mongoengine API is that’s there’s no way to easily manage recursive trees.

Sounds like someone agrees with me.

Original title and link: MongoDB: Designing Trees using mongodm (NoSQL databases © myNoSQL)

via: http://dev-solutions.fr/post/designing-trees-in-mongodb-using-mongodm

In the days the news are about instant searches and auto complete, Salvatore Sanfilippo (@antirez) shows how to use Redis sorted sets and corresponding commands (ZRANGE, ZRANK) to implement autocompletion:

The initial code in Ruby:

already got ported to Python:

and ☞ Java and ☞ PHP.

As Ilya Grigorik (@igrigorik) commented, this is building a ☞ Trie with Redis.

Original title and link: Redis: Implementing Auto Complete or How to build Trie on Redis (NoSQL databases © myNoSQL)

via: http://antirez.com/post/autocomplete-with-redis.html