Todd Hoff extracts and expands on some thoughts about BASE vs ACID from Eric Brewer’s NoSQL: Past, Present, Future published on InfoQ:

Consistency it turns out is not the Holy Grail. What trumps consistency is:

• Auditing
• Risk Management
• Availability

But the cornerstone of the availability vs consistency conversation is:

Availability correlates with revenue and consistency generally does not.

✚ Over time Michael Stonebraker has been the most prominent supporter of exactly the opposite argument.

✚ Remember Emin Gün Sirer’s The NoSQL Partition Tolerance Myth? He used the bank example too.

Original title and link: Banks and the Ethernal Consistency Example or What trumps consistency (NoSQL database©myNoSQL)

via: http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on-why-banks-are-base-not-acid-availability.html

Interesting new direction of TokuDB pushing their storage engine based on Fractal Tree Indexes to MongoDB:

Running MongoDB with Fractal Tree Indexes (used today in the MySQL storage engine TokuDB) is fully transactional. Each statement is transactional. If an update is to modify ten rows, then either all rows are modified, or none are. Queries use multi-versioning concurrency control (MVCC) to return results from a snapshot of the system, thereby not being affected by write operations that may happen concurrently.

Original title and link: MongoDB Transactions With TokuDB’s Fractal Tree Indexes Engine (NoSQL database©myNoSQL)

via: http://www.tokutek.com/2013/04/mongodb-transactions-yes/#gsc.tab=0

Learning from Peter Bailis:

While the CAP Theorem is fairly well understood, the relationship between CAP and ACID transactions is not. If we consider the current lack of highly available systems providing arbitrary multi-object operations with ACID-like semantics, it appears that CAP and transactions are incompatible. This is partly due to the historical design of distributed database systems, which typically chose consistency over high availability. Standard database techniques like two-phase locking and multi-version concurrency control do not typically perform well in the event of partial failure, and the master-based (i.e., master-per-shard) and overlapping quorum-based techniques often adopted by many distributed database designs are similarly unavailable if users are partitioned from the anointed primary copies.

There’s also a paper (PDF) authored by Peter Bailis, Alan Fekete, Ali Ghodsi, Joseph m. Hellerstein, Ion Stoica. These names should tell you something.

Original title and link: Introducing Highly Available Transactions: The Relationship Between CAP and ACID Transactions (NoSQL database©myNoSQL)

via: http://www.bailis.org/blog/hat-not-cap-introducing-highly-available-transactions/

But not everything we do in a database needs guaranteed transactional consistency.

Imagine you are charged with designing a system to collect data on temperature, air flow and electricity use in a building every few minutes from hundreds of locations. The system will be used to make the building more energy efficient. Now imagine you lose a few data points every day.  The cause isn’t important but it could be a glitch with a sensor, a dropped packet, or an incomplete write operation in the database.

Do you care?

It depends from what angle I’m looking at this question. If I’m the producer of the sensor, I do care if it has a glitch. If I’m a network administrator I do care there are dropped packets. And if I am a database system I do care if I’m dropping write operations. And I also have to tell whoever is using me if I am able to receive operations—am I available when I’m needed?

Original title and link: Cloud Computing Lets Us Rethink How We Use Data (NoSQL database©myNoSQL)

via: http://www.tomsitpro.com/articles/mapreduce-hadoop-cloud_computing-acid-relational_database,1-165.html

Lars Hofhansl:

HBase employs a kind of MVCC. And HBase has no mixed read/write transactions. […] When a write transaction (a set of puts or deletes) starts it retrieves the next highest transaction number. In HBase this is called a WriteNumber. When a read transaction (a Scan or Get) starts it retrieves the transaction number of the last committed transaction. HBase calls this the ReadPoint.

Understanding the behavior of read and write operations in HBase is definitely useful. Learning that an upcoming HBase version will support atomic multi operations (HBASE-3584) and even multi-row local transactions (HBASE-5229) is priceless.

For HBase atomic multi-operations:

 Delete d = new Delete(ROW);
 Put p = new Put(ROW);
 ...
 AtomicRowMutation arm = new AtomicRowMutation(ROW);
 arm.add(p);
 arm.add(d);
 myHtable.mutateAtomically(arm);

and HBase multi-row local transactions is implemented as mutateRowsWithLocks method in HRegion and can be used by coprocessors only (no client API).

Original title and link: ACID in HBase: Row Level Operations Explained. Plus Something New (NoSQL database©myNoSQL)

via: http://hadoop-hbase.blogspot.com/2012/03/acid-in-hbase.html?spref=tw

Dave Engberg has published on the Evernote Techblog a post explaining why the Atomicity, Consistency, and Durability characteristics of a single replicated MySQL/InnoDB deployment are essential to the way Evernote operates.

While it’s difficult to argue about a technical decision with so little details available, I still wanted to point out a couple of things:

1. Atomicity: most of the NoSQL databases offer atomic operation at the level of a single record. For distributed systems that do not want to rely on 2PC, it is the multi-row atomic operations that are not supported.

   The example presented in the post does not require multi-row transactions, but rather guaranteed client operation ordering. This is achievable in most NoSQL databases.

2. Consistency: the post talks about data consistency from the perspective of data integrity guarantees through usage of foreign keys.

   In the world of NoSQL similar behavior could be achieved by different data modeling solutions. Using Cassandra as an example for the notebook deletion scenario, one could store all the notes of a notebook in a single Cassandra row, thus making the delete operation safe.

   It’s also worth mentioning that many of the eventually consistent NoSQL databases offer different consistent read and write operations.

3. Durability: with just a few known exceptions, most NoSQL databases offer strong durability guarantees.
   

In conclusion, based only on the few details of the post, one could easily argument that a NoSQL database would fit the bill. But most of the time the reality behind is much different, making technical decisions a tad more complicated.

Original title and link: WhySQL: MySQL/InnoDB ACID Guarantees for Evernote (NoSQL database©myNoSQL)

via: http://blog.evernote.com/tech/2012/02/23/whysql/

Santosh Kumar:

As you work more with Redis you soon start finding yourself building out workflows, i.e. small pieces of code that talk to each other via Redis. For someone familiar with a Service Oriented approach to building systems this should feel like deja-vu. Except, instead of using a protocol (HTTP, TCP, UDP, AMQP, ZeroMQ) we are going back to CS101 using a good old queue datastructure.

I think instead of workflows and Service Oriented, the right term is message services or MOM. Indeed, Redis’ blocking queues, the corresponding commands, and PUB/SUB support provides one with the basic building blocks of message services. But these are useful only if you don’t need specialized solutions (RabbitMQ, ActiveMQ, etc.) which will provide solutions for more complicated scenarios.

Later in the post, Kumar mentions implementing transactions for the scenario he picked, payment processing, using Redis’ multi-exec. If things haven’t changed radically lately, I’d underline the fact that Redis MULTI/EXEC/DISCARD is just a batched serialized executor and not ACID transactions.

Original title and link: Redis-Based MOM: Redis for Processing Payments (NoSQL database©myNoSQL)

via: http://santosh-log.heroku.com/2011/08/19/redis-for-processing-payments/

Dan Weinreb comments on Michael Stonebraker’s Urban Myths about SQL (PDF) :

Dr. Michael Stonebraker recently posted a presentation entitled “Urban Myths about NoSQL”. Its primary point is to defend SQL, i.e. relational, database systems against the claims of the new “NoSQL” data stores. Dr. Stonebraker is one of the original inventors of relational database technology, and has been one of the most eminent database researchers and practitioners for decades.

In fact, Michael Stonebraker bashes everything that is not his current product—this GigaOm interview is the latest example.

For now, I’m filing this away until VoltDB is sold.

Original title and link: Comments on Urban Myths About NoSQL (NoSQL database©myNoSQL)

via: http://danweinreb.org/blog/657

Excellent list:

1. Understand how ACID compares with BASE (Basically Available, Soft-state, Eventually Consistent)
2. Understand persistence vs non-persistence, i.e., some NoSQL technologies are entirely in-memory data stores
3. Recognize there are entirely different data models from traditional normalized tabular formats: Columnar (Cassandra) vs key/value (Memcached) vs document-oriented (CouchDB) vs graph oriented (Neo4j)
4. Be ready to deal with no standard interface like JDBC/ODBC or standarized query language like SQL; every NoSQL tool has a different interface
5. Architects: rewire your brain to the fact that web-scale/large-scale NoSQL systems are distributed across dozens to hundreds of servers and networks as opposed to a shared database system
6. Get used to the possibly uncomfortable realization that you won’t know where data lives (most of the time)
7. Get used to the fact that data may not always be consistent; ‘eventually consistent’ is one of the key elements of the BASE model
8. Get used to the fact that data may not always be available
9. Understand that some solutions are partition-tolerant and some are not

Print it out and distribute it among your colleagues.

Original title and link: 9 Things to Acknowledge about NoSQL Databases (NoSQL databases © myNoSQL)

via: http://www.evidentsoftware.com/nosql-basics-for-the-rdbms-savvy/