ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

python: All content tagged as python in NoSQL databases and polyglot persistence

Another Redis-Based Queue for Python: Introducing RQ

Vincent Driessen creates RQ as an alternative to Celery inspired by Resque:

I wanted a solution that was lightweight, easy to adopt, and easy to grasp. So I devised a simple queueing library for Python, and dubbed it RQ.

Welcome to the world of a thousand Redis-based queues.

Original title and link: Another Redis-Based Queue for Python: Introducing RQ (NoSQL database©myNoSQL)

via: http://nvie.com/posts/introducing-rq/


In-Memory Key-Value Store in C, Go and Python

Graham King:

On paternity leave for my second child, I found myself writing an in-memory hashmap (a poor-man’s memcached), in Go, Python and C. I was wondering how hard it would be to replace memcached, if we wanted to do something unusual with our key-value store. I also wanted to compare the languages, and, well, I get bored easily!

Actually it’s very easy and doesn’t require any coding at all. Plus you’ll get a bit more than what you’d expect.

Original title and link: In-Memory Key-Value Store in C, Go and Python (NoSQL database©myNoSQL)

via: http://www.darkcoding.net/software/in-memory-key-value-store-in-c-go-and-python/


Asyncdynamo: Amazon DynamoDB Async Python Library by Bitly

Bitly’s new asynchronous Amazon DynamoDB Python client:

Asyncdynamo requires Boto and Tornado to be installed, and must be run with Python 2.7. It replaces Boto’s synchronous calls to Dynamo and to Amazon STS (to retrieve session tokens) with non-blocking Tornado calls. For the end user its interface seeks to mimic that of Boto Layer1, with each method now requiring an additional callback parameter.

Available on GitHub.

Original title and link: Asyncdynamo: Amazon DynamoDB Async Python Library by Bitly (NoSQL database©myNoSQL)

via: http://word.bitly.com/post/18861837158/introducing-asyncdynamo


Multiple Index Queries in Riak Using Python

Sreejith K describing his riak_multi_query Python library for muti-indeces-based queries:

One of the advantage of using LevelDB with Riak is that they support Secondary Indexes. […] I wrote a Python wrapper that allows multiple index queries using Secondary indexes and MapReduce. The basic idea is as follows:

  • Query Multiple Indexes and get the associated keys
  • Pass the keys to a MapReduce job where Multiple filters are again evaluated. The map phase applies all the conditions to individual keys.

Now imagine this library would run those queries in parallel.

Original title and link: Multiple Index Queries in Riak Using Python (NoSQL database©myNoSQL)

via: http://foobarnbaz.com/2012/02/25/multi-index-queries-in-riak/


Quick Guide to MongoDB and Python With PyMongo

A tutorial on PyMongo from Rick Copeland covering:

  • configuration options for MongoDB
  • documents structure, inserts and batch inserts
  • querying and indexing
  • deleting
  • updating

One thing that’s nice about the pymongo connection is that it’s automatically pooled. What this means is that pymongo maintains a pool of connections to the mongodb server that it reuses over the lifetime of your application. This is good for performance since it means pymongo doesn’t need to go through the overhead of establishing a connection each time it does an operation. Mostly, this happens automatically. you do, however, need to be aware of the connection pooling, however, since you need to manually notify pymongo that you’re “done” with a connection in the pool so it can be reused.

Original title and link: Quick Guide to MongoDB and Python With PyMongo (NoSQL database©myNoSQL)

via: http://blog.pythonisito.com/2012/01/moving-along-with-pymongo.html


The Stories of the Revamped Riak Java Client and Improvements in Python Client

If you read the story of the MongoDB Erlang driver, you’ll probably enjoy reading about Riak’s revamped Java client or the improvements in the Riak’s Python client .

Original title and link: The Stories of the Revamped Riak Java Client and Improvements in Python Client (NoSQL database©myNoSQL)


Building an Ad Network Ready for Failure

The architecture of a fault-tolerant ad network built on top of HAProxy, Apache with mod_wsgi and Python, Redis, a bit of PostgreSQL and ActiveMQ deployed on AWS:

The real workhorse of our ad targeting platform was Redis. Each box slaved from a master Redis, and on failure of the master (which happened once), a couple “slaveof” calls got us back on track after the creation of a new master. A combination of set unions/intersections with algorithmically updated targeting parameters (this is where experimentation in our setup was useful) gave us a 1 round-trip ad targeting call for arbitrary targeting parameters. The 1 round-trip thing may not seem important, but our internal latency was dominated by network round-trips in EC2. The targeting was similar in concept to the search engine example I described last year, but had quite a bit more thought regarding ad targeting. It relied on the fact that you can write to Redis slaves without affecting the master or other slaves. Cute and effective. On the Python side of things, I optimized the redis-py client we were using for a 2-3x speedup in network IO for the ad targeting results.

Original title and link: Building an Ad Network Ready for Failure (NoSQL database©myNoSQL)

via: http://dr-josiah.blogspot.com/2011/06/building-ad-network-ready-for-failure.html


Watch System Logs in Real Time with Redis Pub/Sub

Little trick to get access to your logs:

This is a little python log handler that lets you stream your log data over a redis pub/sub channel, so you can monitor your system in real time from any redis client.[…] I am also interested in mapping the python logging names (foo.bar.baz) to redis channels that would enable you to listen to arbitrary applications, or subscribe to a collection using the redis wildcard subscriptions.

On GitHub.

Original title and link: Watch System Logs in Real Time with Redis Pub/Sub (NoSQL databases © myNoSQL)

via: http://jedp.posterous.com/a-python-logging-handler-for-redis-pubsub


CouchDB for Timely and Budget Limited Projects

CouchOne has published another CouchDB success story:

The team felt a relational database and full web application framework would be overkill and inflexible. With these factors in mind they chose to use Flask and CouchDB. An additional factor that appealed to the developers was CouchDB’s native storage of JSON documents. They communicate and collect various data with the Facebook Graph API, which speaks JSON — so it just made sense to dump that data straight into CouchDB.

The final application was delivered in aproximately one week — though the article forgets to mention the size of team that worked on the project.

Original title and link: CouchDB for Timely and Budget Limited Projects (NoSQL databases © myNoSQL)

via: http://www.couchone.com/case-study-wwf


redis_graph: Redis-based Graph Database for Python

redis_graph is a graph database implemented in Python. It shows how awesome Redis is as the implementation is under 40 lines of code.

The perfomance should be excellent, while scaling it might be an issue. I would not recommend using it if you are storing nodes in the millions range.

Except being fun — as fun as storing a social graph in Redis or implementing a social graph using Redis — I’d say using a real graph database is probably a better approach.

Original title and link: redis_graph: Redis-based Graph Database for Python (NoSQL databases © myNoSQL)

via: http://amix.dk/blog/post/19592


Pylons & MongoDB: User Registration & Login

For Pylons users:

About a year ago, Chris Moos came up with a nice tutorial on how to integrate Pylons with CouchDB.  Well, times have changed: Pylons is now 1.0 (thus, syntactical differences), and I’m into MongoDB for my datastore. This document is revised/edited/rewritten/and updated for MongoDB.

Django users seem to be a bit more active in the NoSQL space though.

Original title and link: Pylons & MongoDB: User Registration & Login (NoSQL databases © myNoSQL)

via: http://devinfee.com/blog/2010/11/04/pylons-mongodb-user-registration-login/