Ayende Rahien:

We stream the results to the server directly, so while the client is still sending results, we are already flushing them to disk.

To make things even more interesting, we aren’t using standard GZip compression over the whole request. Instead, each batch is compressed independently, which means we don’t have a dependency on the internals of the compression routine internal buffering system, etc. It also means that we get each batch much faster.

There are, of course, rate limits built in, to protect ourselves from flooding the buffers, but for the most part, you will have hard time hitting them.

Bulk inserts and data import are two interesting topics in the world of NoSQL databases where there are no ACID guarantees. What is the state of the databases if data stream is cut midway? What is the state of the database if the import fails midway? What is the state of the database if some insert/update operations fail? I’m not aware of any good answers for these possible issues.

Original title and link: RavenDB Bulk Inserts: Implementation Details (NoSQL database©myNoSQL)

via: http://ayende.com/blog/160547/implementation-details-ravendb-bulk-inserts

Daniel Lang:

One of the main differences between relational databases and document databases is the lack of native joining capabilities, right? This is no longer true for RavenDB.

This wasn’t the case for RethinkDB¹ which launched with support for JOINs. But it’s great to see others doing it too.

Original title and link: NoSQL and JOINs: RavenDB and RethinkDB (NoSQL database©myNoSQL)

via: http://daniellang.net/joining-documents-in-ravendb-2-0/

Good overview of main characteristics of RavenDB replication by John Bennett:

1. one-way
2. push-based
3. asynchronous
4. secure
5. batched

Original title and link: An Overview of RavenDB Replication (NoSQL database©myNoSQL)

via: http://jtbennett.com/blog/2013/01/ravendb-replication-an-overview

Daniel Lang:

The question which database to choose obviously depend on your concrete scenario, the skills of your team, your environment (existing licenses), etc. but here is what I think could help you:

We choose RavenDB when…

• we can think of our data in terms of aggregates with mostly independent chunks of data (e.g. customer, order, product, etc.)
• we need to have good performance on aggregation and calculation queries
• we need to have complex searching (full-text, facets, etc.)
• we need to be able to scale
• we need high availability at low costs

We choose SQL Server when…

• when we need to support user generated reports and highly dynamical data analysis
• we have to deal with mostly relation data (e.g. accounting, statistics)
• we want to use Windows Azure
• our customer definitely wants us to choose sql server without knowing better

My additional 2 cents:

1. the easy part: don’t choose one or another based on feature lists. Feature lists should be used only in apples-to-apples comparisons.
2. the more complicated part: don’t use a relational database just because you’ve always used one. Don’t use a NoSQL database just because it’s the shiny new toy you need on your portfolio/resumé. Don’t use both just because it might be fun.

Original title and link: RavenDB vs MSSQL: Which to Choose? (NoSQL database©myNoSQL)

via: http://daniellang.net/ravendb-vs-mssql/

There may be a need to query the various kinds of documents as if they were the same. […] One possibility is to execute 3 separate queries (one for each entity type) and then aggregate the result sets for the end user to view. That may work, but it wouldn’t be very efficient nor elegant. Another possibility is to create a multi-map index that could be queried. Let’s take a look at that approach.

Nice feature. I assume something similar could be achieved in NoSQL databases supporting MapReduce by mapping only the items of interest from the various collections in the database.

Original title and link: RavenDB Polymorphic Queries Using Multi-Map Indices (NoSQL database©myNoSQL)

via: http://ravendb.net/kb/11/querying-unlike-documents-using-a-multi-map-index

Ayende about RavenDB’s HiLO id generator:

this is perfect for most scenarios. Except one very common scenario, bulk import. When you need to load a lot of data into RavenDB, you will very quickly note that most of the time is actually spent just getting new ranges. More time than actually saving the new documents takes, in fact.

But, notice what happens as times goes by (and not that much time) before RavenDB recognizes that you need bigger ranges, and it gives you them.

Neat.

Original title and link: RavenDB HiLo Id Generation for Bulk Imports (NoSQL database©myNoSQL)

via: http://ayende.com/blog/154337/ravendb-self-optimizing-ids?key=9dd327cf-4678-483a-91dc-28950a4634d0

Following my comments about RavenDB sharding, I had a very interesting email exchange with Matt Warren about RavenDB HiLo id generation. Matt pointed me to Rob Ashton’s post which covers in more details what is behind RavenDB HiLo:

1. Waiting until SaveChanges to get ids for saved entities makes writing logic against those entities troublesome
2. Calling SaveChanges every time a new entity is created makes transactions troublesome
3. Calling SaveChanges to get the entity id means a call across the wire just to get an entity id, which is expensive
4. Simply assigning a Guid to the Id makes accessing documents via REST an unpleasant experience
5. You can’t just assign a random integer, because you’d just get collisions as other clients did the same and tried to save their entities
6. HiLo provides a method of creating incremental integer based ids for entities in a fashion that is safe in concurrent environments

I do agree with points 2 to 5 above, but I see those as consequences or alternatives to the first one. Assuming that the first point represents indeed a real issue, the trade-off of RavenDB HiLo is offering is the following:

1. one network round trip to get the Hi value
2. some locking server side to ensure the next Hi value generation is truly atomic
3. some more locking client side for generating the Lo values

Hopefully I’m missing something as this doesn’t look to me either less complicated or a good trade-off. Or maybe the hypothesis is wrong.

Original title and link: RavenDB HiLo ID Generation: What, How, Why (NoSQL database©myNoSQL)

via: http://codeofrob.com/entries/ravendb---the-hilo-what-how-and-why.html

One of those getting started guides that can be very helpful when first trying new technologies:

Today we are going to look at how we can use it as the only backing store of an ASP.NET MVC application. In this article, we will see how to build an ASP.NET MVC app using RavenDB Embedded and then publish it to a hosted source.

Using RavenDB as a backing store is actually very easy. In a development environment, all you have to do is install using Nuget package manager and start the Raven.Server.exe from the packages folder. However in a production environment, you may or may not be able to run an additional HTTP server for yourself. If you have a hosting provider where your site is shared with multiple other sites, you may not have the access to run a separate HTTP server. Beauty of RavenDB is that even in such a constrained environment, you can easily run RavenDB and not require SQL Server backend. RavenDB has special embedded variant that allows you to run RavenDB in medium trust.

Original title and link: RavenDB Tutorial: Building an ASP.NET MVC App Using RavenDB as a Backing Store (NoSQL database©myNoSQL)

via: http://www.dotnetcurry.com/ShowArticle.aspx?ID=787