Bookmark this for the time you’ll be looking into RavenDB or when you’ll have around 6 hours to watch Oren Eini (Ayende Rahien) talk on NoSQL and RavenDB.

• ☞ NoSQL in .NET
• ☞ Introduction to RavenDB
• ☞HERDING CODE 83: AYENDE RAHIEN ON RAVENDB

Embedded below are the slides from Introduction to RavenDB:

Interesting to note that some VoltDB don’ts from the paper ☞ Do’s and Don’ts (pdf) are validating some major assumptions in the NoSQL space:

Don’t create tables with very large rows (that is, lots of columns or large VARCHAR columns). Several smaller tables with a common partitioning key are better.

Basically both wide-column stores (i.e. Cassandra, HBase, Hypertable) with their column-families and document databases (i.e. CouchDB, MongoDB, RavenDB, Terrastore) with their schema-less approach are addressing this issue.

1. Don’t use ad hoc SQL queries as part of a production application.

Firstly this points to the mindset change required by the NoSQL space when doing data modeling: think about data access patterns.

Secondly, it pretty much validates CouchDB and RavenDB approaches of having queries defined upfront making their reads extremely fast.

Recently I have noticed that Doctrine^[1], a PHP library focused on persistence services, has been working on ☞ defining a new query language for document databases.

So, I couldn’t stop asking myself is there a need for a (new) document query language?

To be able to answer this question, I thought I should firstly review what are the existing solutions/approaches.

1. CouchDB doesn’t allow running dynamic queries against the store, but you can define views with the help of Javascript-based mapreduce functions.
2. MongoDB allows dynamic and pretty complex queries, but it is using a custom query API.
3. RavenDB, the latest addition to the document database space, has chosen the route of Linq^[2] for defining indexes.
4. Terrastore supports predicate (XPath currently) and range queries offering a mapreduce-like solution. You can read more about these in the Terrastore 0.5.0 article
5. Last, but not least, XML databases are using XPath for querying.

Simply put, it looks like each solution comes with its own approach. While it will probably make sense to create a unified query language for document databases, I see only two possible solutions:

• either make all document databases sign up to use this query language (note: this might be quite difficult)
• or provide it through a framework that works will all of the existing document stores (note: this might not be possible)

But do not create a new query language in a framework that works only with a single document store.

Oren Eini has an interesting ☞ post emphasizing the main differences between document databases (f.e. CouchDB, MongoDB, etc.) and key-value stores (f.e. Redis, Project Voldemort, Tokyo Cabinet):

The major benefit of using a document database comes from the fact that while it has all the benefits of a key/value store, you aren’t limited to just querying by key.

One of the main advantages of data transparency (as opposed to opaque data) is that the engine will be able to perform additional work without having to translate the data into an intermediary or a format that it understands. Querying by non primary key is such an example. The various document stores provide different implementation flavors depending on index creation time, index update strategy, etc. Oren goes on and covers the query behavior for CouchDB, Raven and MongoDB:

In the first case (nb indexes prepared ahead of time), you define an indexing function (in Raven’s case, a Linq query, in CouchDB case, a JavaScript function) and the server will run it to prepare the results, once the results are prepared, they can be served to the client with minimal computation. CouchDB and Raven differs in the method they use to update those indexes, Raven will update the index immediately on document change, and queries to indexes will never wait. […] With CouchDB, a view is updated at view query time, which may lead to a long wait time on the first time a view is accessed if there were a lot of changes in the meanwhile. […]

Note that in both CouchDB and Raven’s cases, indexes do not affect write speed, since in both cases this is done at a background task.

MongoDB, on the other hand, allows ad-hoc querying, and relies on indexes defined on the document values to help it achieve reasonable performance when the data size grows large enough. MongoDB’s indexes behave in much the same way RDBMS indexes behave, that is, they are updated as part or the insert process, so large number of indexes is going to affect write performance.

Another good resource explaining the differences between MongoDB and CouchDB queries is Rick Osbourne’s ☞ article.

After RavenDB made his appearance in the NoSQL space we’ll probably have to compare it to the existing CouchDB and MongoDB features.

This is not to say that some of this functionality cannot be achieved with pure key-value stores, but these seem to be focused mainly on single/multi key lookups and most probably you’ll have to build this additional layer by yourself.

Lately I’ve been hearing about a couple of newcomers in the NoSQL space so here is your chance to find out about them and why not provide other readers with some additional feedback about each of them:

kumofs ☞

Kumofs is a distributed key-value store built on top of Tokyo Cabinet and using the memcached protocol.

According to the ☞ project homepage:

• Data is partitioned and replicated over multiple servers.
• Extreme single node performance; comparable with memcached.
• Both read and write performance got improved as servers added.
• Servers can be added without stopping the system.
• Servers can be added without modifying any configuration files.
• The system does not stop even if one or two servers crashed.
• The system does not stop to recover crashed servers.
• Automatic rebalancing support with a consistency control algorithm.
• Safe CAS operation support.
• memcached protocol support.

There already seems to be ☞ success story of kumofs, but you’ll have to take that with a grain of salt as the same company was announcing a success story with Redis for the same problem.

Raven DB ☞

Raven DB is a dual licensed document database for the .NET platform with a RESTful API.

According to the ☞ project homepage:

• Scalable infrastructure: Raven builds on top of existing, proven and scalable infrastructure
• Simple Windows configuration: Raven is simple to setup and run on windows as either a service or IIS7 website
• Transactional: Raven support System.Transaction with ACID transactions. If you put data in it, that data is going to stay there
• Map/Reduce: Easily define map/reduce indexes with Linq queries
• .NET Client API: Raven comes with a fully functional .NET client API which implements Unit of Work and much more
• RESTful: Raven is built around a RESTful API

Orient ☞

Orient comes in two flavors: OrientDB: a document database and OrientKV a key-value store, both running on the Java platform.

According to the ☞ project homepage, OrientDB is a

scalable Document based DBMS that uses the features of the Graph Databases to handle links. It’s the basic engine of all the Orient products. It can work in schema-less mode, schema-full or a mix of both. Supports advanced features, such as indexing, fluent and SQL-like queries. It handles natively JSON and XML documents.

The OrientKV is a:

Key/Value Server based on the Document Database technology and accessible as embedded repository via Java APIs or via HTTP using a RESTful API. Orient K/V uses a new algorithm called RB+Tree, derived from the Red-Black Tree and from the B+Tree. Orient Key/Value server can run in high availability mode using a cluster of multiple nodes partitioned.

Have you tried any of these?

Pincaster ☞

According to the GitHub page, Pincaster is a persistent NoSQL database to store geographic data and key/value pairs, with a HTTP/JSON interface. Unfortunately there’s almost no documentation about the project so I don’t think there’s anything else I could add for now.

Update: hint received from @fs111