myNoSQL

Cassandra at SocialFlow with Drew Robb - Powered by NoSQL

Sun, 05 Feb 2012 12:14:05 +0200

To alternate a bit after yesterday’s educational CQL: SQL for Cassandra in the Cassandra NYC 2011 video series from DataStax, today’s video is Drew Robb covering Cassandra usage at SocialFlow for capturing real-time data from Twitter and Bit.ly.

For watching more videos from this event follow the Cassandra NYC 2011 tag.

Original title and link: Cassandra at SocialFlow with Drew Robb - Powered by NoSQL (NoSQL database©myNoSQL)

XFS: the filesystem of the future?

Sat, 04 Feb 2012 19:12:05 +0200

Jonathan Corbet summarizing a presentation about the present and future of XFS by Dave Chinner:

XFS is often seen as the filesystem for people with massive amounts of data. It serves that role well, Dave said, and it has traditionally performed well for a lot of workloads. Where things have tended to fall down is in the writing of metadata; support for workloads that generate a lot of metadata writes has been a longstanding weak point for the filesystem. In short, metadata writes were slow, and did not really scale past even a single CPU.

After the break the video of Dave Chinner’s presentation, “XFS: Recent and Future Adventures in Filesystem scalability”.

Even if it’s very long, make sure you check the comment thread.

Jonathan Ellis

Original title and link: XFS: the filesystem of the future? (NoSQL database©myNoSQL)

CQL: SQL for Cassandra with Eric Evans - NoSQL videos

Sat, 04 Feb 2012 12:14:05 +0200

The fine folks from DataStax have made available the presentations from their Cassandra NYC 2011 event.

The first video to post here is Eric Evans’s presentation on Cassandra Query Language.

CQL: SQL for Cassandra

For watching more videos from this event follow the Cassandra NYC 2011 tag.

Original title and link: CQL: SQL for Cassandra with Eric Evans - NoSQL videos (NoSQL database©myNoSQL)

A bit of history around Hadoop Companies

Fri, 03 Feb 2012 15:10:05 +0200

A bit of history around Hadoop Companies:

Imagine this story 5 years from now… with all the scars from the ~~battle~~ competition with the other companies trying to monetize Hadoop and … the millions in the bank.

Original title and link: A bit of history around Hadoop Companies (NoSQL database©myNoSQL)

Designing HBase Schema to Best Support Specific Queries

Fri, 03 Feb 2012 12:30:05 +0200

Designing HBase Schema to Best Support Specific Queries:

Real scenario, very good analysis of different data access requirements, and three possible solutions. What’s your pick?

The problem is fairly simple - I am storing “notifications” in hbase, each of which has a status (“new”, “seen”, and “read”). Here are the API’s I need to provide:

Get all notifications for a user

Get all “new” notifications for a user

Get the count of all “new” notifications for a user

Update status for a notification

Update status for all of a user’s notifications

Get all “new” notifications accross the database

Notifications should be scannable in reverse chronological order and allow pagination.

Original title and link: Designing HBase Schema to Best Support Specific Queries (NoSQL database©myNoSQL)

Modeling A/B Tests With Cassandra

Fri, 03 Feb 2012 12:15:05 +0200

Modeling A/B Tests With Cassandra:

A Cassandra data modeling session around a real-life scenario: tracking data for A/B tests:

With most things in life data modeling in Cassandra can be compared to learning to ride a bike. It can be scary, you might fall off, but in the end once you learn a few fundamental concepts everything will be easier to do. The goal of this article is to get you comfortable with a basic data modeling scenario that you will likely see in the real world.

Original title and link: Modeling A/B Tests With Cassandra (NoSQL database©myNoSQL)

What's the big deal about Big Data?

Fri, 03 Feb 2012 10:00:05 +0200

What's the big deal about Big Data?:

Roger Ehrenberg (Founder and Managing Partner of IA Ventures):

Every so often a term becomes so beloved by media that it moves from “instructive” to “hackneyed” to “worthless,” and Big Data is one of those terms. […] But since this time the term Big Data has become diluted. Very diluted. So much so that it is almost totally meaningless. Does Big Data mean new kinds of databases? Sure. Does it mean innovative ways to visualize data to create actionable intelligence? Absolutely. Can it be applied to the health care sector? Without question. Has it contributed to the rise of the Data Scientist? Mos def.

And I thought I was off.

Original title and link: What’s the big deal about Big Data? (NoSQL database©myNoSQL)

The time for NoSQL is now

Fri, 03 Feb 2012 09:51:06 +0200

The time for NoSQL is now:

Andrew C. Oliver:

The transition to NoSQL databases will take time. We still don’t have TOAD, Crystal Reports, query language standardization and other essential tools needed for mass adoption. There will be missteps (i.e. I may need a different type of database for reporting than for my operational system), but I truly think this is one technology that isn’t just marketing.

This coming from someone that was happy to discover back in 1998 all the knobs in Oracle.

Original title and link: The time for NoSQL is now (NoSQL database©myNoSQL)

What does DynamoDB mean for the enteprise world?

Fri, 03 Feb 2012 01:48:17 +0200

What does DynamoDB mean for the enteprise world?:

Tim Moreton¹:

I expect not much in the short term, until some key issues are solved. First, the sources and consumers of data are still on-site. These guys are tackling a specific technical limitation, not necessarily looking to re-architect their wider systems, which are often complex and inter-dependent. Second, security and regulatory concerns may need addressing. Third, the TCO needs to stack up. A quick and dirty back of the envelope calculation suggests that although it’s free to get started with DynamoDB, for the sort of deployment sizes we’re seeing, DynamoDB works out considerably more expensive than alternatives like Acunu deployed on hardware (even after accounting for typical full costing for outsourced data centers).

Keeping my eyes on all things NoSQL for more than 2 years, I’d say that NoSQL databases in general do not mean much for the enterprise world². Yet.

Tim Moreton: CEO Acunu ↩
DataStax’s CEO, Billy Bosworth seems to agree. ↩

Original title and link: What does DynamoDB mean for the enteprise world? (NoSQL database©myNoSQL)

DataStax's CEO thoughts on the NoSQL Market and Competition

Fri, 03 Feb 2012 01:36:44 +0200

DataStax's CEO thoughts on the NoSQL Market and Competition:

Billy Bosworth¹:

Personally, I have never believed that other post-relational (aka NoSQL/Hadoop) database companies were our primary competition. The brute fact of the matter is that if you put us all together, we are still not statistically relevant compared to the overall DBMS market.

I had only one real personal fear coming into this market: That I would sink a big portion of my life into something that would never take hold in the mainstream. I suspect that would be a truly awful ending for all of us in this space. But thanks to companies like Amazon and Oracle, that feels highly unlikely now, and that is a great thing.

Just to play the devil advocate for a second: Oracle won’t lose much in the NoSQL market if things don’t work out well and Amazon’s DynamoDB is part of a larger plan. But for all the NoSQL database companies it is an all-or-nothing game².

Billy Bosworth: CEO DataStax ↩
An all-or-nothing game is not the same with a winner-takes-all game ↩

Original title and link: DataStax’s CEO thoughts on the NoSQL Market and Competition (NoSQL database©myNoSQL)

Get them by the data

Fri, 03 Feb 2012 01:27:00 +0200

Get them by the data:

Gavin Clarke and Chris Mellor about AWS Storage Gateway:

Once you’ve got them by the data, of course, their hearts and minds will follow, and Amazon’s using the AWS Storage Gateway beta as a sampler for the rest of its compute cloud.

The Storage Gateway is another piece, together with S3, DynamoDB, SimpleDB, Elastic MapReduce, in Amazon’s great strategical puzzle of a complete polyglot platform.

Original title and link: Get them by the data (NoSQL database©myNoSQL)

MongoDB Tips and Tricks: You Only Wish MongoDB Wasn't Relational

Fri, 03 Feb 2012 01:20:06 +0200

MongoDB Tips and Tricks: You Only Wish MongoDB Wasn't Relational:

So when you read that MongoDB is a document store, you might get the wonderful idea to store your relationships in a big document. Since mongo lets you reach into objects, you can query against them, right?

Several times, we’ve excitedly begun a schema this way, only to be forced to pull the nested documents out into their own collection. I’ll show you why, and why it’s not a big deal.

I’m no MongoDB expert, but the suggested solution requires: 1) two network roundtrips; 2) an additional index.

The way I’d look at this problem is:

what is the most frequent operation: reading a blog post and all its comments or displaying all the comments of a specific user?
assuming the answer is reading a blog post and all its comments, I’d ask myself how frequent is the other operation.
1. if it’s something that I need to perform just once in a while, I’d consider using MongoDB MapReduce, even if that would be suboptimal.
2. if it’s a frequent operation, then I’d consider adding a separate collection for comments. Even better, I’d add a per user collection for comments.

Original title and link: MongoDB Tips and Tricks: You Only Wish MongoDB Wasn’t Relational (NoSQL database©myNoSQL)

Thoughts on SimpleDB, DynamoDB and Cassandra

Fri, 03 Feb 2012 01:05:28 +0200

Thoughts on SimpleDB, DynamoDB and Cassandra:

Adrian Cockcroft:

So the lesson here is that for a first step into NoSQL, we went with a hosted solution so that we didn’t have to build a team of experts to run it, and we didn’t have to decide in advance how much scale we needed. Starting again from scratch today, I would probably go with DynamoDB. It’s a low “friction” and developer friendly solution.

You can look at this in two ways: 1) a biased opinion of someone that has already betted on Amazon with the infrastructure of a multi-billion business; 2) the opinion of someone that has accumulated a ton of experience in the NoSQL space and that is successfully¹ running the infrastructure of a multi-billion business on NoSQL solutions. I’d strongly suggest you to think of it as the latter.

Netflix was one of the few companies that continued to operate during Amazon’s EBS major failure. ↩

Original title and link: Thoughts on SimpleDB, DynamoDB and Cassandra (NoSQL database©myNoSQL)

Neo4j 1.6 GA Release: Heroku, Cypher, Lucene 3.5

Thu, 02 Feb 2012 22:40:04 +0200

Announced last week, Jörn Kniv aka Neo4j 1.6 features:

Improved Cypher (the query language)
Web admin - Full Neo4j Shell commands, including versioned Cypher syntax.
Kernel improvements
Upgraded Lucene version to 3.5.

Also the Neo guys have been pushing quite a bit their public beta Heroku add-on.

NoSQL Market from Couchbase Perspective

Thu, 02 Feb 2012 17:53:11 +0200

NoSQL Market from Couchbase Perspective:

James Philips (Couchbase) for Curt Monash:

MongoDB is the big competition. He believes Couchbase has an excellent win rate vs. 10gen for actual paying accounts.

DataStax/Cassandra wins over Couchbase only when multi-data-center capability is important. Naturally, multi-data-center capability is planned for Couchbase. (Indeed, that’s one of the benefits of swapping in CouchDB at the back end.)

Redis has “dropped off the radar”, presumably because there’s no particular persistence strategy for it.

Riak doesn’t show up much.

I assume this is sort of a pre-sales/sales department 100k feet overview.

MongoDB Tips & Tricks: Using MongoDB ObjectIds as created-on timestamps

Thu, 02 Feb 2012 17:32:42 +0200

MongoDB Tips & Tricks: Using MongoDB ObjectIds as created-on timestamps:

One of my favorite MongoDB tricks is the ability to use an ObjectId (the default type for MongoDB’s _id primary key) as a timestamp for when a document was created. Here’s how it works:
  $ import pymongo
  $ db = pymongo.Connection().test
  $ db.test.insert({'hello': 'world'})
  ObjectId('4f202e64e6fb1b56ff000000')
  $ doc = db.test.find_one()
  $ doc['_id'].generation_time
  datetime.datetime(2012, 1, 25, 16, 31, 32, tzinfo

Mike Dirolf used to work for 10gen so he probably knows quite a few such MongoDB tips & tricks.

Original title and link: MongoDB Tips & Tricks: Using MongoDB ObjectIds as created-on timestamps (NoSQL database©myNoSQL)

Hadoop and Seismic Data Processing

Thu, 02 Feb 2012 17:13:05 +0200

Hadoop and Seismic Data Processing:

Geophysicists have been pushing the limits of high-performance computing for more than three decades; they were early adopters of the first Cray supercomputers as well as the massively parallel Connection Machine. Today, the most challenging seismic data processing tasks are performed on custom compute clusters that take advantage of multiple GPUs per node, high-performance networking and storage systems for fast data access.

How many fields we’ve never heard of have handcrafted over years their own solutions to deal with big data that would fit so nicely in Hadoop today?

NoSQL tutorials: Storing User Preference in Amazon DynamoDB using the Mobile SDKs

Thu, 02 Feb 2012 14:03:06 +0200

NoSQL tutorials: Storing User Preference in Amazon DynamoDB using the Mobile SDKs:

Just a CRUD tutorial for DynamoDB but based on a scenario that makes sense and demoing the API with two languages (Objective-C and Java):

The sample mobile application described here demonstrates how to store user preferences in Amazon DynamoDB. Because more and more people are using multiple mobile devices, connecting these devices to the cloud and storing user preferences in the cloud enables developers to provide a more uniform cross-device experience for their users.

This article shows sample code for both the iOS and Android platforms.

Original title and link: NoSQL tutorials: Storing User Preference in Amazon DynamoDB using the Mobile SDKs (NoSQL database©myNoSQL)

Doug Cutting About Hadoop's Adoption

Thu, 02 Feb 2012 13:46:11 +0200

Doug Cutting About Hadoop's Adoption:

Doug Cutting expressing his suprise with Hadoop’s growth in an interview with Audrey Watters over O’Reillly Radar:

Yes. I didn’t expect Hadoop to become such a central component of data processing. I recognized that Google’s techniques would be useful to other search engines and that open source was the best way to spread these techniques. But I did not realize how many other folks had big data problems nor how many of these Hadoop applied to.

Hadoop is not Doug Cutting’s first widely successful open source project, so I’m tempted to think this is just pure modesty.

Getting off the CouchDB... or Lessons Learned while Experimenting in Production

Thu, 02 Feb 2012 09:36:05 +0200

Getting off the CouchDB... or Lessons Learned while Experimenting in Production:

The move to CouchDB went well. Pages in our web application that would occasionally time out were now loading in a couple of seconds. And, our MySQL database was much, much happier. We liked CouchDB so much that we started planning a feature that would make heavy use of CouchDB’s schema-less nature.

And that’s when the wheels came off.

Word of caution: this is not the “CouchDB sucks so we went with MongoDB” type of post. It’s more of “we thought CouchDB can solve one of our problems, but then got confused and thought it can solve world hunger. So we decided to throw a bunch of data to it to see if it sticks. Surprise! It didn’t.”

Just to be clear, I’m not defending CouchDB and everything John Wood writes about it is correct. It’s just that experimenting with CouchDB in a non-production environment or at least reading myNoSQL would have already offered all those answers.

Original title and link: Getting off the CouchDB… or Lessons Learned while Experimenting in Production (NoSQL database©myNoSQL)