Bitly’s new asynchronous Amazon DynamoDB Python client:

Asyncdynamo requires Boto and Tornado to be installed, and must be run with Python 2.7. It replaces Boto’s synchronous calls to Dynamo and to Amazon STS (to retrieve session tokens) with non-blocking Tornado calls. For the end user its interface seeks to mimic that of Boto Layer1, with each method now requiring an additional callback parameter.

Available on GitHub.

Original title and link: Asyncdynamo: Amazon DynamoDB Async Python Library by Bitly (NoSQL database©myNoSQL)

via: http://word.bitly.com/post/18861837158/introducing-asyncdynamo

Sreejith K describing his riak_multi_query Python library for muti-indeces-based queries:

One of the advantage of using LevelDB with Riak is that they support Secondary Indexes. […] I wrote a Python wrapper that allows multiple index queries using Secondary indexes and MapReduce. The basic idea is as follows:

• Query Multiple Indexes and get the associated keys
• Pass the keys to a MapReduce job where Multiple filters are again evaluated. The map phase applies all the conditions to individual keys.

Now imagine this library would run those queries in parallel.

Original title and link: Multiple Index Queries in Riak Using Python (NoSQL database©myNoSQL)

via: http://foobarnbaz.com/2012/02/25/multi-index-queries-in-riak/

A tutorial on PyMongo from Rick Copeland covering:

• configuration options for MongoDB
• documents structure, inserts and batch inserts
• querying and indexing
• deleting
• updating

One thing that’s nice about the pymongo connection is that it’s automatically pooled. What this means is that pymongo maintains a pool of connections to the mongodb server that it reuses over the lifetime of your application. This is good for performance since it means pymongo doesn’t need to go through the overhead of establishing a connection each time it does an operation. Mostly, this happens automatically. you do, however, need to be aware of the connection pooling, however, since you need to manually notify pymongo that you’re “done” with a connection in the pool so it can be reused.

Original title and link: Quick Guide to MongoDB and Python With PyMongo (NoSQL database©myNoSQL)

via: http://blog.pythonisito.com/2012/01/moving-along-with-pymongo.html

The architecture of a fault-tolerant ad network built on top of HAProxy, Apache with mod_wsgi and Python, Redis, a bit of PostgreSQL and ActiveMQ deployed on AWS:

The real workhorse of our ad targeting platform was Redis. Each box slaved from a master Redis, and on failure of the master (which happened once), a couple “slaveof” calls got us back on track after the creation of a new master. A combination of set unions/intersections with algorithmically updated targeting parameters (this is where experimentation in our setup was useful) gave us a 1 round-trip ad targeting call for arbitrary targeting parameters. The 1 round-trip thing may not seem important, but our internal latency was dominated by network round-trips in EC2. The targeting was similar in concept to the search engine example I described last year, but had quite a bit more thought regarding ad targeting. It relied on the fact that you can write to Redis slaves without affecting the master or other slaves. Cute and effective. On the Python side of things, I optimized the redis-py client we were using for a 2-3x speedup in network IO for the ad targeting results.

Original title and link: Building an Ad Network Ready for Failure (NoSQL database©myNoSQL)

via: http://dr-josiah.blogspot.com/2011/06/building-ad-network-ready-for-failure.html

Little trick to get access to your logs:

This is a little python log handler that lets you stream your log data over a redis pub/sub channel, so you can monitor your system in real time from any redis client.[…] I am also interested in mapping the python logging names (foo.bar.baz) to redis channels that would enable you to listen to arbitrary applications, or subscribe to a collection using the redis wildcard subscriptions.

On GitHub.

Original title and link: Watch System Logs in Real Time with Redis Pub/Sub (NoSQL databases © myNoSQL)

via: http://jedp.posterous.com/a-python-logging-handler-for-redis-pubsub

CouchOne has published another CouchDB success story:

The team felt a relational database and full web application framework would be overkill and inflexible. With these factors in mind they chose to use Flask and CouchDB. An additional factor that appealed to the developers was CouchDB’s native storage of JSON documents. They communicate and collect various data with the Facebook Graph API, which speaks JSON — so it just made sense to dump that data straight into CouchDB.

The final application was delivered in aproximately one week — though the article forgets to mention the size of team that worked on the project.

Original title and link: CouchDB for Timely and Budget Limited Projects (NoSQL databases © myNoSQL)

via: http://www.couchone.com/case-study-wwf

redis_graph is a graph database implemented in Python. It shows how awesome Redis is as the implementation is under 40 lines of code.

The perfomance should be excellent, while scaling it might be an issue. I would not recommend using it if you are storing nodes in the millions range.

Except being fun — as fun as storing a social graph in Redis or implementing a social graph using Redis — I’d say using a real graph database is probably a better approach.

Original title and link: redis_graph: Redis-based Graph Database for Python (NoSQL databases © myNoSQL)

via: http://amix.dk/blog/post/19592

For Pylons users:

About a year ago, Chris Moos came up with a nice tutorial on how to integrate Pylons with CouchDB.  Well, times have changed: Pylons is now 1.0 (thus, syntactical differences), and I’m into MongoDB for my datastore. This document is revised/edited/rewritten/and updated for MongoDB.

Django users seem to be a bit more active in the NoSQL space though.

Original title and link: Pylons & MongoDB: User Registration & Login (NoSQL databases © myNoSQL)

via: http://devinfee.com/blog/2010/11/04/pylons-mongodb-user-registration-login/

A story of using Hadoop at Yelp and migrating it to Amazon Elastic MapReduce:

We used to do what a lot of companies do, which is run a Hadoop cluster. We had a dozen or so machines that we otherwise would have gotten rid of, and whenever we pushed our code to our webservers, we’d push it to the Hadoop machines.

It was also not so cool. You couldn’t really tell if a job was going to work at all until you pushed it to production. But the worst part was, most of the time our cluster would sit idle, and then every once in a while, a really beefy job would come along and tie up all of our nodes, and all the other jobs would have to wait.

Yelp has released their Python library for running MapReduce jobs on Hadoop or Amazon Elastic MapReduce on ☞ GitHub.

Original title and link: Hadoop and Elastic MapReduce at Yelp (NoSQL databases © myNoSQL)

via: http://engineeringblog.yelp.com/2010/10/mrjob-distributed-computing-for-everybody.html

The first reason that bring me away from the great mongoengine API is that’s there’s no way to easily manage recursive trees.

Sounds like someone agrees with me.

Original title and link: MongoDB: Designing Trees using mongodm (NoSQL databases © myNoSQL)

via: http://dev-solutions.fr/post/designing-trees-in-mongodb-using-mongodm