ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close
Thursday, 28 April 2011
‹ Back

Introducing AODBM

About:
,
,

What’s the point in writing a new dbm style DBMS when so many exist already? Surely there is nothing new that anyone can add to this long line of projects?

Having done exactly that, I’m going to dedicate the rest of this post to convincing you that I haven’t lost my sanity.

I became interested in databases about six months ago when I started studying databases that spanned more than one machine. I found that it seems to be the accepted norm that if one ACID compliant database server can’t handle the number of writes that you need it to then you’ll have to loosen your reliability guarantees.

This can manifest in a number of ways:

  • Amongst the most frequent is manually sharding your database. Using two or more databases is a sure fire way to loose your atomicity and isolation guarantees for cross database operations.

  • Forsaking durability is common. It is the founding principle of database systems like Redis.

  • Settling for eventual consistency is the latest craze. CouchDB is heading the wave.

    Of course, this isn’t always a bad thing. CouchDB, for instance, was designed for environments where partitioning is common and in these circumstances eventual consistency is a logical choice. ACID guarantees aren’t always a deal-breaker and when you can afford to give them up in order to gain performance, you definitely should.

However, I couldn’t find a database implementation that would allow me to scale without loosing multi-record atomic updates, durability or immediate consistency. I set about designing my perfect DBMS.

Following several false starts, I settled upon a design. AODBM (Append Only Database Manager) is the storage backend for this future DBMS. However, AODBM has merit on its own! It is a simple, hackable, open-source implementation of an append-only B+Tree written in C. It uses MVCC. It has an easy to use API (for C and Python) and it is designed to be fast for heavy, highly-concurrent loads.

In practice, this means that if you find a need to update a key-value store from many threads and you need ACID compliance, then I have a solution for you.

AODBM is currently in (early) beta and I am pushing for the first release. Early performance figures look promising; on my five year old desktop I can do 8500 insertions per second on a single thread from Python. I have implemented get, set and delete; iteration is on it’s way. I’m hoping to release the first version by the end of this month.

I hope I’ve succeeded in persuading you that my sanity is still intact. Have fun.

Daniel Waterworth

The AODBM project source code is on GitHub and it has a page on SourceForge.

Original title and link: Introducing AODBM (NoSQL databases © myNoSQL)