Tornado, the non-blocking web server and tools open sourced by FriendFeed before their acquisition, seems to get some NoSQL activity. While Django is leading the way in the Python world, judging by the NoSQL projects happening around Node.js, one could say that Tornado, with its non-blocking architecture, may be an interesting alternative.

Thomas Pelletier has ☞ a blog post about a simple websocket + Tornado + Redis Pub/Sub protocol integration:

The principle is very simple: when your user loads the page, she is automatically added to a list of “listeners”. An independent thread is running: it listens for messages from Redis with the subscribe command, and send a message through Websocket to every registered ”listener”. In this example, the user can send a message to herself with a simple AJAX-powered form, which calls a view with a payload (the message), and the view publish it via the publish command of Redis.

This is basically a web chat! If you want to have fun, you can then add a roster, with a presence system, authentication etc…

There’s also a ☞ GitHub project called Trombi:

Trombi is an asynchronous CouchDB client for Tornado.

And I’m pretty sure there are other projects I’ve missed (but you can leave a comment to add them to the list).

Original title and link for this post: Tornado Sees Some NoSQL Activity (published on the NoSQL blog: myNoSQL)

Recently, in the Django and NoSQL databases revisited, I’ve covered the coordinated efforts for making Django a NoSQL friendly framework. Alex Gaynor, the main person behind this initiative having the support of the Django community, has ☞ published the final report of the GSOC project:

With this past week GSOC has officially come to it’s close, and I’m here to report on the status of the query-refactor. The original purpose of this branch was to do refactorings to the internals of the ORM, and produce a prototype backend for a non-relational database to demonstrate that this was a viable option. At this time far more work has gone into the latter half of the project, I have developed a fully functioning MongoDB backend, that demonstrates that possibility of using the ORM, almost unmodified on non-relational databases. However, some of the larger refactors that I was originally hoping to do have ultimately not happened, on the other hand they are evidentially not necessary for a functioning backend. At this time there are a number of outstanding tasks, such as: porting the ListField to work on Postgres, and completing the work on embedded documents.
However the largest open question is what of this work should be merged into trunk, and what should live external. My recommendation would be for any changes in Django itself to be merged, including the new form fields, but for the MongoDB backend (and, indeed, any future backends) to live external to Django, until such a time as it obtains a user base anywhere approaching our current backends, as well as a more individuals dedicated to maintaining it.

The guys over ☞ All Buttons Pressed also commented on the outcome of this project:

The biggest design issue (in my opinion) is how to handle AutoField. In the GSoC branch, non-relational model code would always need a manually added NativeAutoField(primary_key=True) because many NoSQL DBs use string-based primary keys. As you can see in Django-nonrel, a NativeAutoField is unnecessary. The normal AutoField already works very well and it has the advantage that you can reuse existing Django apps unmodified and you don’t need a special NativeAutoField definition in your model. Hopefully this issue will get fixed before official NoSQL support is merged into trunk.

Original title and link for this post: Django and NoSQL Databases Latest Status Update (published on the NoSQL blog: myNoSQL)

Django decided long time ago that Ruby on Rails cannot be the only framework where people can have fun integrating with all NoSQL databases. During this year DjangoCon Europe there were several session dedicated to Django and NoSQL databases:

• Alex Gaynor: What NoSQL support in the Django ORM looks like, and how do we get there
• Peter Bengtsson: Using MongoDB in your app
• Benoît Chesneau: Relax your project with CouchDB
• Tobias Ivarsson: Django and Neo4j: Domain Modeling that Kicks Ass
• Django and NoSQL Panel

What NoSQL support in the Django ORM looks like, and how do we get there

Alex Gaynor speaks about what needs to change in Django ORM to make it more NoSQL friendly:

Reinout van Rees has a summary of the talk ☞ here.

Using MongoDB in your app

Peter Bengtsson talks about his experience of passing from using ZODB for the last 10 years to MongoDB

Some notes from the talk are available ☞ here.

Relax your project with CouchDB

Benoît Chesneau talks about what makes CouchDB appealing to python developers. He also covers the CouchDBkit python framework.

Django and Neo4j: Domain Modeling that Kicks Ass

Not coming from DjangoCon, but still about Django and Neo4j, is Tobias Ivarsson’s presentation: “Django and Neo4j - Domain modeling that kicks ass”:

Derek Stainer summarizes the slide deck ☞ here.

Django and NoSQL Panel

A fantastic panel on the future of Django and NoSQL databases that you can watch over ☞ blip.tv. Reinout van Rees published a transcript of the panel ☞ here.

All in all a lot of NoSQL excitement in the Django world! Or should it be the opposite?

Update: Here is the latest Django and NoSQL Databases status update

Django and NoSQL Databases Revisited originally posted on the NoSQL blog: myNoSQL

These are the slides and video of Tobias Ivarsson (@thobe) presenting at PyCon on Neo4j with a Python flavor.

I really liked this slide in particular:

Python code starts at slide 23. A couple of my comments:

• I am not really sure I understand how the Python scripts are accessing the Neo4j storage when using CPython (Neo4j is supposed to run in a JVM)
• traversals in graph databases are somewhat synonymous to queries
• having the traversal implemented like classes extending neo4j.Traversal doesn’t really look Pythonic
• Django and Neo4j can work together

Interested in Python, Django and MongoDB? Then I hope you’ll find these posts interesting:

• Mike Dirolf’s presentation on MongoDB and Python
• MultiDB support in Django

And then there is this fresh screencast from Kevin Fricovsky talking Django and MongoDB integration. You can read about it ☞ here, but as a quick summary, the screencast will introduce you to mongoengine and then using Django-Mumblr, a NoSQL-based blog engine it will dive deeper into the details of Django and MongoDB integration.

Update: Just found a couple of more MongoDB Django tricks that you may find interesting.

The first one is a solution that provides access to MongoDB document _id’s from Django templates. The ☞ solution is based on a custom Django filter and using it as in {{ object|mongo_id }}

I find the solution pretty odd, not to mention that using a filter for accessing such an important document information seems convoluted. I’d much prefer to have the _id accessible directly on an object through either a field or at least a special property. Behavior for an unsaved document might be as simple as returning None or raising an exception.

The second trick fixes a problem with using Django’s FileWrapper while working with MongoDB’s GridFS. I’d probably be tempted to call this a bug, so before getting it fixed you can read the details ☞ here.

Aside Twitter applications, blogs are another darling of NoSQL projects. So, I’ve put together a list of NoSQL powered blog projects.

CouchDB

SemanticJournal ☞

A Rails and CouchDB blog. Code on ☞ GitHub

@RicRoberts

OikoNote ☞

A CouchDB-based blog built in “one day” with Django (nb so far I couldn’t find the source code, so any leads are appreciated).

Neo4j

jo4neoblog ☞

A simple blog built using neo4j, jo4neo and Stripes. You can read more about it ☞ here and get the code from ☞ Google code.

Couple of comments:

• I don’t really like the fact that the model is neo4j aware, but that’s similar to what JPA is doing too
• I like the indexing annotation though, but I am not sure if it uses neo4j Lucene full text indexing

@tcowan

subwiz ☞

A cli-application blog built using neo4j. Code available on ☞ Google code.

@subwiz

MongoDB

MongoBlog ☞

A lightweight blogging engine written in C++ and using MongoDB. Code available on ☞ GitHub

@mich181189

Update: thanks to the comments, I have added two more NoSQL-based blog engines.

Django-Mumblr ☞

Django-Mumblr is a basic Django tumblelog application that uses MongoDB. Source code can be found on ☞ GitHub

mmmblog ☞

mmmblog is a blog engine based on Rails, mongomapper and MongoDB, providing feeds, OpenID comments and a simple admin interface. Code is available on ☞ Gitorious

I am pretty sure there are more out there, so please send them over!

Django is one of the most popular Python frameworks, the one that Google picked to integrate with their Google App Engine PaaS. Thanks to a GSOC project, Django has added now ☞ support for multiple databases and that includes NoSQL stores:

Multiple TYPES of databases. This is the one I’m most excited about. This is going to enable people to use some of the NoSQL databases […]

The multi-database support is right now only in the development trunk — documentation can be found ☞ here — so it might take a while until a Django release will include it and I’m not sure this feature will be backported. But this is definitely just another validation for the NoSQL world.

As far as I know, even before this announcement, there were some efforts to integrate NoSQL solutions with Django and the one I know about is ☞ Neo4j for Django:

The way that the integration between Django and Neo4j is implemented is in the Model layer. Since Neo4j does not have a SQL engine it would not have been efficient or practical to implement the support as a database layer for Django. Google did their implementation in the same way when they integrated BigTable with Django for App Engine. This means that there will be some minor modifications needed in your code compared to using PostgreSQL or MySQL. Just as with BigTable on App Engine, you will have to use a special library for defining your models when working with Neo4j, but the model definition is very similar to Djangos built in ORM.

Now, you might wonder why I do believe that getting NoSQL support in (popular) frameworks is an important step. For the last year or so, the NoSQL stores have been under scrutiny for their technical value and I’d say that, by now, this phase is almost over. Next came the business validation and there are good signs that NoSQL world sees some traction there too.

So, what is left? The simple answer is bringing NoSQL to the people. And by this I mean making it easy to adopt one or the other NoSQL solution and having seamless and standardized integration with existing frameworks and tools. Making it easy to switch from classical RDBMS based solutions to NoSQL solutions or even hybrid SQL-NoSQL solutions is both important and critical for adoption. Ease of adoption will bring us a lot of new use cases and that will make further adoption even easier. And that is the guarantee for a bright NoSQL future.

Update: Even if there were ways to use NoSQL solutions with Django (take for example this intro to using CouchDB with Django), I think the new integration layer will make things feel more natural. I read that here is already a CouchDB Django database adapter and also a demo of using MongoDB with Django. And I bet more will come, so keep an eye on our list of NoSQL libraries.