Announced a while back and ☞ not quite here yet, Riak Search is Basho’s solution to the full text indexing problem.

While waiting for the release of Riak Search, I think that you can already start doing full text indexing using one of the existing indexing solutions (Lucene^[1], Solr^[2], ElasticSearch^[3], etc.) and Riak post-commit hooks.

Simply put, all you’ll have to do is to create a Riak post-commit hook that feeds data into your indexing system.

The downside of this solution is that:

1. you’ll still have to make sure that your indexing system is scalable, elastic, etc.
2. you’ll not be able to use indexed data directly from Riak mapreduce functions, a feature that will be available through Riak Search.

Anyways, until Riak Search is out, why not having some fun!

Update: Embedded below a presentation on Riak Search providing some more details about this upcoming Basho product:

Update: Looks like the other presentation is not available anymore, so here is another on Riak search:

Sounds like quite a few NoSQL projects are externalizing the full text indexing to either Lucene or Solr (take for example CouchDB integration with Lucene or Neo4j integration with Lucene and Solr).

Now even if there are some basic ways (see ^[1] and ^[2]) to achieve this with MongoDB alone, people are still looking for more scalable solutions as shown by this thread ☞ covering Solr integration with MongoDB. The thread also mentions a couple of existing Ruby or Rails plugins for this integration.

One concern that I’ve expressed about the integration with Lucene alone is that you’ll have to deal with its scalability. Solr is one way to do that automatically. Lately I have heard of a new solution for scalable search: ☞ ElasticSearch which sounds quite interesting (nb: I haven’t yet gone through its docs or played with it, but the creator of the project has a long search/indexing history behind. You can find more details about Elastic Search here^[3]).

In a previous post, I was writing that Neo4j, as CouchDB, is using Lucene for full text indexing. While agreeing that this is definitely better than reinventing the wheel, I was also raising my concern about the complexity and scalability of this approach.

Now it looks like there is some work to integrate Neo4j with Solr, the standalone full-text search server based on Lucene ^[1]. This would definitely address the issue I have raised. Anyway it is not yet clear from the original message ^[2] how this integration will work though (it sounds like a two-way integration, but I may be misinterpreting the details). The code is availalbe on Neo4j ☞ SVN.