ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

video: All content tagged as video in NoSQL databases and polyglot persistence

Amazon SimpleDB, MongoDB, CouchDB, and RavenDB Compared

I wanted to share this before the weekend is over: Jesse Wolgamott’s video: “Battle of NoSQL starts: Amazon’s SDB vs MongoDB vs CouchDB vs RavenDB” from September’s Lone Star Ruby Conference.

You can download the video from the Confreaks site for watching offline.

Original title and link: Amazon SimpleDB, MongoDB, CouchDB, and RavenDB Compared (NoSQL databases © myNoSQL)


HBase at Facebook: The Underlying Technology of Messages

There have been lots of discussions and speculations after the announcement that Facebook is using HBase for the new messaging system. In case you missed it, here are the most important bits:

  • Kannan Muthukkaruppan: The underlying Technology of Messages (facebook.com)

    We spent a few weeks setting up a test framework to evaluate clusters of MySQL, Apache Cassandra, Apache HBase, and a couple of other systems. We ultimately chose HBase. MySQL proved to not handle the long tail of data well; as indexes and data sets grew large, performance suffered. We found Cassandra’s eventual consistency model to be a difficult pattern to reconcile for our new Messages infrastructure.

  • Quora.com: How does HBase write performance differ from write performance in Cassandra with consistency level ALL

    While setting the a write consistency level of ALL with a read level of ONE in Cassandra provides a strong consistency model similar to what HBase provides (and in fact using quorum writes and reads would as well), the two operations are actually semantically different and lead to different durability and availability guarantees.

  • Cassandra mailing list: Facebook messaging and choice of HBase over Cassandra

  • Todd Hoff: Facebook’s New Real-Time Messaging System: HBase to Store 135+ Billion Messages a Month (highscalability.com)

    HBase is a scaleout table store supporting very high rates of row-level updates over massive amounts of data. Exactly what is needed for a Messaging system. HBase is also a column based key-value store built on the BigTable model. It’s good at fetching rows by key or scanning ranges of rows and filtering. Also what is needed for a Messaging system. Complex queries are not supported however. Queries are generally given over to an analytics tool like Hive, which Facebook created to make sense of their multi-petabyte data warehouse, and Hive is based on Hadoop’s file system, HDFS, which is also used by HBase.

  • Jeremiah Peschka: Facebook messaging - HBase Comes of Age (facility9.com)

    Existing expertise: The technology behind HBase – Hadoop and HDFS – is very well understood and has been used previously at Facebook. […] Since Hive makes use of Hadoop and HDFS, these shared technologies are well understood by Facebook’s operations teams. As a result, the same technology that allows Facebook to scale their data will be the technology that allows Facebook to scale their Social Messaging feature. The operations team already understands many of the problems they will encounter.

  • Quora.com: What version of HBase is Facebook using for its new messaging platform?

    Facebook has an internal branch of HBase which periodically updates from the Apache SVN. As far as I know, the current version in production is very similar to the 0.89.20100924 development release with a couple more patches pulled in from trunk.

    Facebook engineers continue to actively contribute to the open source trunk, though - it’s not an internal “fork”

    Todd Lipcon (HBase committer)

The engineering team behind Facebook’s new messaging system has posted now a video talking more about their choice of HBase. You can watch the a bit over 1 hour long video here.

The engineering team behind Facebook Messages spent the past year building out a robust, scalable infrastructure. We shared some details about the technology on our Engineering Blog (http://fb.me/95OQ8YaD2rkb3r). This tech talk digs deeper into some of the twenty different infrastructure services we created for the project as well as how we’re using Apache HBase.

I’m still watching the video, so my notes will follow.

Why HBase?

Choosing HBase at Facebook

  • Strong consistency model
  • Automatic failover
  • Multiple shards per server for load balancing

  • Prevents cascading failures

  • Compression: save disk space and network bandwidth

  • Read-modify-write operation support, like counter increment
  • Map Reduce supported out of the box

I’m still not sure why one needs a strong consistency model for messages (and that’s the part missing from all these articles).

As a side note, I feel like the decission was based not on some major facts, but rather a sum of small but important features that HBase was offering compared to other solutions (i.e. consistent increments, perfect integration with Hadoop, etc.)

Original title and link: HBase at Facebook: The Underlying Technology of Messages (NoSQL databases © myNoSQL)


The Beauty of Data Visualization

David McCandless talking at TED about data visualization:

Data science is the future and there cannot be data science without data visualization and vice versa.

Or in Bundy’s Frank Sinatra words: You can’t have one without the other.

Original title and link: The Beauty of Data Visualization (NoSQL databases © myNoSQL)


Redis Cluster Explained

For Redis users, it shouldn’t be a surprise that Salvatore Sanfilippo (@antirez) has been working on Redis clustering lately. In the videos embedded below, Salvatore explains the details of Redis clustering .


Redis 2.2: What’s New with Pieter Noordhuis

Pieter Noordhuis (@pnoordhuis) talks about the upcoming Redis 2.2:


Membase Short Video Series

Membase has started a series of short videos on setting and getting up to speed with Membase. I think it’s a good idea, but calling the series “Membase for Dummies”, not so much!

So far, they’ve published:

Original title and link: Membase Short Video Series (NoSQL databases © myNoSQL)


MongoDB Videos from MongoSV

It’s weekend again and so it’s time for some NoSQL videos. For MongoDB fans, 10gen has published ☞ here videos recorded at their latest MongoSV event.

There are two case studies:

  • Jeremy Zawodny (Craigslist): MongoDB and Craigslist’s Data Storage Evolution ☞ video
  • Kenny Gorman (Shutterfly): Sharding Life’s Joy using MongoDB: A Shutterfly Case Study ☞ video

and then a series on various MongoDB aspects:

  • Eliot Horowitz (10gen): Scaling with MongoDB ☞ video
  • Richard Kreuter (10gen): Deployment strategies ☞ video
  • Mathias Stearn (10gen): Administration ☞ video
  • Alvin Richards: Schema Design: Data as documents ☞ video
  • Richard Kreuter (10gen): Indexing and Query Optimizer ☞ video
  • Brendan McAdams (10gen): Map/Reduce, geospatial indexing, and other cool features ☞ video

Original title and link: MongoDB Videos from MongoSV (NoSQL databases © myNoSQL)


OpenTSDB Lightning Talk with Benoit “tsuna” Sigoure

Benoit “tsuna” Sigoure talking OpenTSDB:

☞ OpenTSDB is a distributed, scalable Time Series Database written on top of HBase announced and open sourced by StumpleUpon.

Original title and link: OpenTSDB Lightning Talk with Benoit “tsuna” Sigoure (NoSQL databases © myNoSQL)


Non-canonical Use Case for MongoDB for schools.chicagotribute.com

Christopher Groskopf[1] presenting a not so typical use case for MongoDB for building schools.chicagotribune.com, sharing their experience with schema-less design, search faceting, and geo-queries :


  1. Christopher Groskopf: News Application Developer, Chicago Tribune  ()

Original title and link: Non-canonical Use Case for MongoDB for schools.chicagotribute.com (NoSQL databases © myNoSQL)


To SQL or not to SQL Panel at CODEBITS IV

A panel discussion on NoSQL, NoSQL databases, and relational databases, featuring Salvatore Sanfilippo[1], Lenz Grimmer[2], Filipe David Borba Manana[3], and a forth person from SAPO whose name I couldn’t spell:


  1. Salvatore Sanfilippo: creator and main developer of Redis  ()
  2. Lenz Grimmer: MySQL community relations team  ()
  3. Filipe David Borba Manana: CouchDB committer  ()

Original title and link: To SQL or not to SQL Panel at CODEBITS IV (NoSQL databases © myNoSQL)


CouchDB and PHP Video Tutorial

Besides LAMP, you can now learn about LAP(hp)-CouchDB-:

PHP developers generally use MySQL, but the rise of NoSQL databases has opened up new options for creating web apps. CouchDB’s unique replication feature set and easy API make it a great fit for PHP developers looking to move into a NoSQL database.

Original title and link: CouchDB and PHP (NoSQL databases © myNoSQL)


CouchDB for .NET Developers

A presentation from the Norwegian Developer Conference 2010 covering the basics of CouchDB and how to interact with CouchDB from .NET applications. Video available ☞ here (nb: I had to start the player and get back to watching it after a bit so it’d have time to buffer).

Original title and link: CouchDB for .NET Developers (NoSQL databases © myNoSQL)