Thoughts from NoSQL Evening in Palo Alto

About:

I should start by saying that I love new technology as long as it proves to be useful and that it works. For example I’ve been using LiveScribe to record the great NoSQL Evening in Palo Alto event organized by Tim Anglade with InfiniteGraph support, just to discover later that it decided to lose everything. So, instead of being able to quote things from the event, I’ll have to rely on my memory, which is really really bad.

After Berlin Buzzwords, this was the largest NoSQL event I’ve participated to. With InifiniteGraph’s people support, Tim Anglade did a great job in organizing this event and gathered together in a panel quite a few leaders of the NoSQL market. Unfortunately there were a few notable absences too; Redis, HBase, Project Voldemort, Neo4j, and RavenDB being the ones I’ve missed. Anyways, knowing how difficult is to put something like this together, this is understandable.

Before getting to what I’m remembering from the event, I have to tell you that I’ve been impressed with the fact that InfiniteGraph has not pushed for their product during the event. They have been great hosts and I had enjoyable discussions with many of their people, especially Darren Wood the lead architect.

Now it is time to test how bad my memory is. In case I got things wrong, please feel free to correct me.

What triggered so much activity in the persistence space?

cloud computing
failings do lead to innovation
the changing nature of the applications
old ideas reoccurring in customer work

Simplicity has different meanings for different people

NoSQL databases try to be:

developer friendly
ops friendly
user friendly

As a side note, it looks like there is still a myth out there related to NoSQL databases not needing DBAs. While the title doesn’t need to be the same, every NoSQL database will need someone…

What is the market size?

The relational database market size is estimated at $27bil. Noone wanted to go on record with their NoSQL databases market estimation. The only number I’ve heard mentioned that “70% usecases can fit NoSQL solutions” (Roger Bodamer, 10gen)

(New) Data models

The data model determines the access model. This discussion continued over the dinner, when people tried to answer the question how connected is SQL to the relational model.

I’ve also seem to remember some interesting remarks about indexes:

indexes are a different data model that enable different access models
indexes will be orders of magnitude larger than real data

RAM is the new disk

There are many products out there which believe in RAM being the new disk (i.e. VoltDB, elastic caching solutions, etc.). Darren Wood (InfiniteGraph) mentioned that “graph analytics is a counter-example of using RAM storage for all data”.

Licensing

Many NoSQL databases have chosen to use some form or another of open source licensing models. The reasons for doing it:

ease market penetration
there’ll always be companies willing to pay for software, support, streams of patches, etc.
open source gives great hiring resources

Some other disparate notes:

Many of the solutions are old, but the wrapping/packaging is new. For example, MarkLogic is a good proof document databases work, even if it is not one of the “cool” products.
Not all NoSQL databases are about size
OLTP is for knowledge classification; OLAP is for knowledge discovery
Will we have multi-purpose NoSQL databases?
Cold data should be on disk
Using the right protocol can help you skip supporting specific features. CouchDB is HTTP friendly, so it doesn’t need to directly have a caching layer
Are key-value stores offering too little compared to file systems?
SQLite has a great distribution model: it is basically everywhere.

For more accurate coverage of the event you can read:

Update: now we’ve got also the video!