Alexander Sicular:

The focus of this post is to show you how to do the equivalent of the sql “SORT BY date DESC” using Riak’s map/reduce interface. Due to Riak’s schemaless, document focused nature Riak lacks internal indexing and by extension, native sorting capabilities.

Complete code included (and embedded below):

A couple of links you’ll probably find useful before/after reading the article:

• Riak has improved the fetching of keys in a bucket, that making MapReduce on buckets directly not so expensive
• Even if there are some saying MapReduce is complicated, take a look at how to translate SQL to MapReduce or this MapReduce explanation in simple terms
• A complete guide to MapReduce with Riak

Original title and link for this post: Riak: Sort by with MapReduce (published on the NoSQL blog: myNoSQL)

Over this week I’ve been working on a proof of concept to see if it’s possible to use Clojure as the map/reduce language for Riak, in the same way now we can use Javascript and Erlang for that purpose. To accomplish that I needed a way to call Clojure code from Erlang. So I set up a very simple server in Clojure that runs as an Erlang node using Closerl.

Theoretically nice… practically I’d say there is a fundamental problem with this idea (different than the ones listed in the article). Map and reduce functions are supposed to run on the nodes hosting the data^[1]. If you need to wire this data is like implementing mapreduce on your application so the data locality property is lost. Not to mention that adding another variable to the equation (the JVM) your distributed system will become more sensible to failures.

Original title and link for this post: Riak Map/Reduce Queries in Clojure (published on the NoSQL blog: myNoSQL)

Even if CouchDB benefits of probably one of the most sophisticated and cool replication mechanisms that doesn’t make it horizontally scalable. I’ve already covered the different solutions for scaling CouchDB, but what Cloudant promises seems to be the missing part:

All of these features — distributed, horizontally scalable, durable, consistent — happen with little or no change required in applications that have been written for CouchDB. A cluster looks just like a stand-alone CouchDB, and API compliance has been our goal from the beginning. Granted, there are a few extra options like overriding quorum constant defaults and there are a few vagaries, like views always performing rereduce due to the views being distributed. But on the whole, the extras in Cloudant are transparent to the application.

Now I’m wondering how Cloudant CouchDB scaling compares with running CouchDB with a Riak backend, Riak offering also a Dynamo-like distributed system.

CouchDB: Horizontal Scalability from Cloudant originally posted on the NoSQL blog: myNoSQL

From Basho:

Web applications built with Ruby on Rails have lots of ways to take advantage of scalable, distributed storage systems like Riak. These resources can help you get started.

Video and slides below:

Riak and Rails: 6 Steps for Getting Started originally posted on the NoSQL blog: myNoSQL

Pure awesomeness:

To make CouchDB store documents remotely, we only have to replace the implementation of the two functions listed above. For our remote storage let’s use Riak as our Key-Value store (because it’s awesome). CouchDB persists Erlang terms to disk and Riak persists Erlang terms to disk. We get to remove redundant code from CouchDB since Riak is converting terms for us. Riak also automatically replicates everything we store, easily handles adding more machines to increase capacity, and deals with failures transparently.

Not only has Matt seen seen the connection between CouchDB and Riak, but he made it ☞ work.

CouchDB with a Riak Backend originally posted on the NoSQL blog: myNoSQL

From installation to using the Clojure library for Riak ☞ clj-riak including MapReduce with Riak:

This brief introduction leaves many aspects of Riak unaddressed. For example, we have not looked at throughput, scalability, fault tolerance, conflict resolution, or production operations – all critical to a complete understanding of the datastore.

Quick Guide for Riak with Clojure originally posted on the NoSQL blog: myNoSQL

A couple of confusing things in this post:

The nice thing about Cassandra was the data model. Super columns allowed us to store metadata for a resource as needed. […] Concurrency issues were also not a bother. We could do simultaneous updates to columns and super columns and not worry about data consistency issues. […] When looking for alternatives Riak was our first choice primarily because of it being in Erlang and since it had a map-reduce option which looked seriously promising.

I don’t see any connection between these. Going from a granular data model supporting column level operations to an key-value store with opaque values doesn’t really add up.

Of the back-ends available this has worked best for us giving a consistent performance along with being reasonable on the resource usage.

This seems a bit contradictory with what was said about the new default Riak storage Bitcask in the Innostore and Bitcask comparison.

Anyone able to clarify these? (nb I’m not saying something is wrong, but I’d like to better understand the details). For now, Mozilla story Cassandra, HBase, Riak: Choosing the Right Solution seems to be much better documented.

Update: Thanks to Jebu Ittiachen things are a clearer now:

My issues with Cassandra and with Bitcask under Riak were with how they behaved in terms of their memory consumption. In the presence of ever increasing number of keys like the tweets which keep coming in both of them would eat up all the memory available on my servers. Cassandra I guess because of its per SSTable cache of keys and Bitcask because it maintains all keys in memory. This initially being the reason for me looking out for a different store than Cassandra. I should mention that in addition to tweets other data is also managed in Cassandra / Riak.

What I was trying to convey is how something that was easily modeled in Cassandra could still be mapped into Riak and possibly be to an advantage given the map-reduce infrastructure.

My preference of innostore over bitcask has purely been seeing how they behave in real use. Bitcask is definitely faster but high in memory usage on the servers. Innostore on the other hand is steady on the memory usage over time.

From Cassandra to Riak at inagist.com originally posted on the NoSQL blog: myNoSQL