NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



nosql databases: All content tagged as nosql databases in NoSQL databases and polyglot persistence

The Future is Polyglot Persistence

Marting Fowler and Pramod Sadalage in an infographic promoting their upcoming book (PDF):

Polyglot persistence will occur over the enterprise as different applications use different data storage technologies. It will also occur within a single application as different parts of an application’s data store have different access characteristics.

There are over 2 years since I’ve begun evangelizing polyglot persistence. By now, most thought leaders agree it is the future. Next on my agenda is having the top relational vendors sign off too. Actually, I’m almost there: Oracle is promoting an Oracle NoSQL Database and Microsoft is offering both relational and non-relational solutions with Azure. They just need to say it.

Original title and link: The Future is Polyglot Persistence (NoSQL database©myNoSQL)

Data Grid or NoSQL? What are the common points? The main differences?

A great post by Olivier Mallassi on a topic that comes up very often: how do data grids and NoSQL databases compare?

  • Data Grids enable you controlling the way data is stored. They all have default implementation (Gigaspaces offers RDBMS by default, Gemfire offers file and disk based storage by default….) but in all cases, you can choose the one that fits your needs: do you need to store data, do you need to relieve the existing databases….
  • In order to minimize the latency, data grids enable you to store data synchronously (write-through) or asynchronously (write-behind) on disk. You can also define overflow strategies. In that case, data is store in memory up to a treshold where data is flushed on disk (following algorithms like LRU …). NoSQL solutions have not been designed to provide these features.
  • Data grids enable you developing Event Driven Architecture.
  • Querying is maybe the point on which pure NoSQL solutions and data grids are merging.
  • Data grids enable near-cache topologies.

Taking a step back you’ll notice that there are actually more similarities than differences. While Oliver Mallasi lists the above points as features that prove data grids as being more configurable and so more adaptable, some of these do exist also in the NoSQL databases taking different forms:

  1. pluggable storage backends. Not many of the NoSQL databases have this feature,but Riak and Project Voldemort are offering different solutions that are optimized for specific scenarios.
  2. replicated and durable writes. Not the same as synchronous vs asynchronous writes, but a different perspective on writes.
  3. Notification mechanisms. Once again not all of the NoSQL databases support notification mechanisms, but a couple of them have offer some interesting approaches:
    1. CouchDB: _changes feed with filters
    2. Riak: pre-commit and post-commit hooks
    3. HBase coprocessors
  4. Most of the NoSQL database have local per-node caches.

With these, I’ve probably made things even blurrier. But let me try to draw a line between data grids and NoSQL databases. Data grids are optimized for handling data in memory. Everything that spills over is secondary. On the other hand, NoSQL databases are for storing data. Thus they focus on how they organize data (on disk or in memory) and optimize access to it. Data grids are a processing/architectural model. NoSQL databases are storage solutions.

Original title and link: Data Grid or NoSQL? What are the common points? The main differences? (NoSQL database©myNoSQL)

The time for NoSQL is now

Andrew C. Oliver:

The transition to NoSQL databases will take time. We still don’t have TOAD, Crystal Reports, query language standardization and other essential tools needed for mass adoption. There will be missteps (i.e. I may need a different type of database for reporting than for my operational system), but I truly think this is one technology that isn’t just marketing.

This coming from someone that was happy to discover back in 1998 all the knobs in Oracle.

Original title and link: The time for NoSQL is now (NoSQL database©myNoSQL)


What does DynamoDB mean for the enteprise world?

Tim Moreton1:

I expect not much in the short term, until some key issues are solved. First, the sources and consumers of data are still on-site. These guys are tackling a specific technical limitation, not necessarily looking to re-architect their wider systems, which are often complex and inter-dependent. Second, security and regulatory concerns may need addressing. Third, the TCO needs to stack up. A quick and dirty back of the envelope calculation suggests that although it’s free to get started with DynamoDB, for the sort of deployment sizes we’re seeing, DynamoDB works out considerably more expensive than alternatives like Acunu deployed on hardware (even after accounting for typical full costing for outsourced data centers).

Keeping my eyes on all things NoSQL for more than 2 years, I’d say that NoSQL databases in general do not mean much for the enterprise world2. Yet.

Original title and link: What does DynamoDB mean for the enteprise world? (NoSQL database©myNoSQL)


Why NoSQL Databases Are Not Just For Google and Amazon?

Oren Eini1:

Why the history lesson, you ask? Why, to give you some perspective on the design choices that led to the victory of the relational databases. Space was at a premium, the interaction between the user and the application closely modeled the physical layout of the data in the database. That made sense, because there really were no other alternatives given the environment that existed at the time.

In my company, we are using RavenDB as the backend database for everything from a blog, our ordering and purchasing systems, the daily build server and many more. The major advantages that we found weren’t the ability to scale (although that exists), it is the freedom that it gives us in terms of modeling our data and changing our minds.

The Googles, Facebooks, and Amazons told the story of this was not our relational database vendors’ fault. Jan Lehnardt2 said a while back that NoSQL is about choice. I said that NoSQL databases are a departure from having just good enough solutions. And Oren Eini is emphasizing the benefits of other data models.

  1. Oren Eini is the creator and main developer of the RavenDB document database. 

  2. Jan Lehnardt: Apache CouchDB committer, Couchbase engineer 

Original title and link: Why NoSQL Databases Are Not Just For Google and Amazon? (NoSQL database©myNoSQL)


The History of NoSQL: This Was Not Our Technology Vendors’ Fault

Werner Vogels in the post about Amazon DynamoDB:

We had been pushing the scalability of commercially available technologies to their limits and finally reached a point where these third party technologies could no longer be used without significant risk. This was not our technology vendors’ fault; Amazon’s scaling needs were beyond the specs for their technologies and we were using them in ways that most of their customers were not. A number of outages at the height of the 2004 holiday shopping season can be traced back to scaling commercial technologies beyond their boundaries.

Here is what I wrote about the history behind NoSQL databases:

Providing decent solutions, up to a point, to a wide range of problems and covering more scenarios than alternative storage solutions existing at that time, made relational databases the de facto storage for the last 30 years. But during the last years, more and more problems crossed the boundaries of what could have been considered decent solutions leading to the need for specialized, better than good enough alternative solutions. And thus NoSQL databases.

It feels rewarding to get such confirmation from people that are at the forefront of NoSQL.

Original title and link: The History of NoSQL: This Was Not Our Technology Vendors’ Fault (NoSQL database©myNoSQL)

Reinforcing Couchbase's Commitment to Open Source

Bob Wiederhol, the CEO of Couchbase:

We’re 100% committed to open source and all of our code is available under the Apache 2.0 license.


Original title and link: Reinforcing Couchbase’s Commitment to Open Source (NoSQL database©myNoSQL)


NoSQL Databases’ Adoption in the Enteprise World

There are two guaranteed ways that NoSQL databases will penetrate the enterprise world. I’m not referring here to small departamental experiments, but big budget and long-term penetration.

The first way is by seemless integration with Hadoop[1]. It’s a fact that Hadoop isn’t anymore just a tool for startups or internet companies. It is seeing widely adoption and with the release of Hadoop 1.0.0—which changes the way Hadoop’s maturity in perceived—things will only get better. Being able to augment a Hadoop-centric architecture will definitely be a huge adoption driver.

The second way is by being first class citizens of widely adopted frameworks. In the Java world that could be JEE and Spring—there’s is already Spring Data and the Enterprise integration patterns Spring framework Spring Integration which features Redis and MongoDB. In the Python world that could be Django—Django played nice with NoSQL databases and even considered including support for NoSQL databases in the trunk. In Ruby, that could be Ruby on Rails which has almost always adopted the latest technologies. And the list could go on[2]. Keep in mind though that extensions and plugins are just the first step, but they do not represent first class citizenship.

  1. Integration with Hadoop can take different forms, so unfortunately we will see more and more claims of seemless integration.  

  2. What are the major frameworks in the .NET world?  

Original title and link: NoSQL Databases’ Adoption in the Enteprise World (NoSQL database©myNoSQL)

NoSQL Applications Panel Video

Hey, it looks like the NoSQL applications panel I’ve moderated at QCon SF 2011 went live minutes ago on InfoQ. Featuring Andy Gross (Basho), Frank Weigel (Couchbase), Matt Pfeil (DataStax), Michael Stack (StumbleUpon), Jared Rosoff (10gen), and yours truly.

Drop everything and start watching it now! I promise you’ll love every second of it[1].

  1. It misses my opening jokes though  

Original title and link: NoSQL Applications Panel Video (NoSQL database©myNoSQL)

Are Some NoSQL Technologies Going NoWHERE?

Conor O’Mahony, Program Director for Database Software, IBM, prediction for 2012:

This apparent challenge from NoSQL is not the first time that the relational database has been challenged. A few years ago, many predicted that object databases would conquer the relational database. However, the relational database added stored procedures, user-defined functions, and a number of other object-like features, and it has gone from strength-to-strength, and object databases are now just a bit player in the overall database market.

I predict that the major relational database vendors will, where it makes sense, add certain NoSQL capabilities to their products. For instance, this makes sense for both name-value pair and graph-store capabilities. Of course, this has already happened for XML data, which the major relational products support.

While I can see ways to adapt and optimize a relational database to behave like a key-value store or document database, I would aplaud any relational database vendor that would be able to transform or add an engine that would behave like Cassandra, or HBase, or a graph database. Add on top of that support for multi-datacenter deployments and seemless integration with Hadoop and that would be a fabulous product.

To me things look like this: in one corner of the ring we will have the experience accummulated in the field by NoSQL databases and their creators and in the opposite corner the experience of the marketing and sales departments from relational databases vendors.

Original title and link: Are Some NoSQL Technologies Going NoWHERE? (NoSQL database©myNoSQL)

NoSQL Databases: 6 Business and Technical Reasons for a New Set of Requirements

Here are some technical reasons why traditional relational database vendors are getting challenged:

  1. Scale: The biggest driver of the new generation of data management systems was unprecedented scale.
  2. Disk Capacities have exploded and Random Disk Accesses have gotten cheaper
  3. Fixed Schema is too rigid
  4. First Normal Form is too restrictive/rich
  5. Columnar Stores: It may sometimes be desirable to store together related columns which are accessed together.
  6. Programmatic Access is also important

Great post with a somehow bogus conclusion though.

Original title and link: NoSQL Databases: 6 Business and Technical Reasons for a New Set of Requirements (NoSQL database©myNoSQL)


How Polyglot Persistence and Having Data Storage Options Changes Things

No 2012 predictions. Just facts.

GigaOm’s Barb Darrow commenting on Oracle’s Q2 financial results:

This is bad news for the company which pinned its cloud hopes on specialized data center appliances — the Exadata database machine, Exalogic middleware/application appliance, Exalytics analytics engine as well as a proposed ”Big Data Appliance.” This is the third consecutive quarter where Oracle posted hardware sales declines but this is probably more painful because the company is finally now fully engaged in its big data center appliance push.

An anonymous commenter:

As a low to mid range customer, I was disenfranchised by oracle, and we have gotten rid of segments of oracle in our data center. Just recently they have shown interest in us again so perhaps the low margin equipment will be back. Sun was not focused enough, and Oracle was too focused. Hopefully they are finding an profitable and useful middle ground.

Kent Langley:

So, NoSQL database #9999 there are many other equally usable solutions that are far more transparent in the way they do business and foster community around their products.  This isn’t about paying money.  This is about trust.  So, sorry NoSQL #9999, but I’ll not be entering your sales cycle in this fashion or evaluating your product at this time.  Moving along now…

Original title and link: How Polyglot Persistence and Having Data Storage Options Changes Things (NoSQL database©myNoSQL)