NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



devops: All content tagged as devops in NoSQL databases and polyglot persistence

Migrating Live Memcached Servers

Ian Winter describes the possible approaches to migrate their memcached server farm:

  1. Big Bang. Update all the configuration files and simply push that code live. This has a huge hit on performance as all the new instances are cold, it is however the quickest option.
  2. Migration instance by instance. Each instance would be moved and relevant configuration files pushed live. This way only 1 or X instances is “cold”. This limits impact, but, also takes time as only 1 instance can be done at any one time.
  3. Write to both. The application code could be altered to make all writes to both the existing set of instances and the new. This isn’t great as we’d need a full QA cycle to validate the code works. It’s also something we’d have to implement then pull back out down the line.
  4. Mirror traffic. Similar to the above, but, this time lower down the stack. Essentially duplicate all the TCP level traffic so it warms in parallel and keeps both instances in sync meaning new writes, deletes occur on both existing and new sets.

Can you tell which approach they used?

PS: For the first two approaches I’d use different names: Stop the World and Rolling upgrade respectively.

Original title and link: Migrating Live Memcached Servers (NoSQL database©myNoSQL)


MongoDB vs MySQL: A DevOps point of view

Pierre Bailet and Mathieu Poumeyrol of fotopedia (a French photo site) share their experience of operating a small MongoDB cluster since Sep.2009 compared to a MySQL cluster.

Some details about fotopedia:

  • fotopedia is 100% on AWS
  • Amazon RDS for MySQL
  • 4 nodes MongoDB cluster
  • 150mil. photo views

MongoDB advantages:

  • no alter table
  • background index creation
  • data backup & restoration
    • note: as far as I can tell MySQL is able to do the same
  • replica sets
  • hardware migration
    • note: the same procedure can be used for MySQL

Before leaving you with the slides, here is an interesting accepted trade-off:

Quietly losing seconds of writes is preferable to:

  • weekly minutes-long maintenance periods
  • minutes-long unscheduled downtime and manual failover in case of hardware failures

Hadoop and Cassandra in the Top 10 Most Important Open Source Projects of 2011

From the Big Data and NoSQL space: Hadoop and Cassandra. And related to this space: OpenStack and Puppet.

So to judge importance, I looked at projects that are influential, gaining in popularity, and/or technical standouts in new areas. In other words, projects that are even more noteworthy than the other noteworthy projects. This means that many projects that are crucial didn’t make the list.

On my list of projects left out: Redis, Riak, HBase, and MongoDB. By looking at the explanation of why Android didn’t make the list, I could understand why not including Redis, Riak, and MongoDB. But I’d keep HBase.

Original title and link: Hadoop and Cassandra in the Top 10 Most Important Open Source Projects of 2011 (NoSQL database©myNoSQL)


Four Operational Essentials: Rock-Solid MongoDB Hosting

Great slides from mongolab‘s Todd O. Dampier about operating MongoDB:

  1. Stay up => high availability
  2. Stay fast => performance & scale
  3. Take good care of your data => data durability
  4. Always know what’s going on => monitoring & alerting

Whatever database you are planning to use for a busy application, someone that specialized in operating it will give you the best insights. They’ve seen the best and worst of it.

The Top 5 Reasons to Use Chef

Bryan Berry:

  1. Writing reams of documentation sucks. Chef drastically reduces the amount of documentation you have to write.
  2. Bash doesn’t scale. Seriously.
  3. Technical Awesomeness
  4. Chef grows with you
  5. You can stop reinventing the wheel

There’s no large scale without automation. Just ask Netflix about their army of monkeys.

Original title and link: The Top 5 Reasons to Use Chef (NoSQL database©myNoSQL)


Monitoring MongoDB

The guys from Boxed Ice have already published a lot about their experience running MongoDB in production, plus a series of posts advising on MongoDB monitoring. Actually they are offering a hosted service for MongoDB monitoring: Server Density .

If you still don’t have MongoDB monitoring in place—unfortunately Foursquare had to learn what this means the hard waycheck this talk from David Mytton[1].

  1. The video quality is very low and I couldn’t embed it here.  

Original title and link: Monitoring MongoDB (NoSQL database©myNoSQL)

The Server Architecture Debate Rages On

Big processors or little processors, scale-up or scale-out, on-premise or in the cloud […] The plethora of choices for application architecture and delivery model are great if you like variety, but I don’t envy anyone tasked with choosing which system on which to spend their limited budget dollars.

Too little options is bad[1]. Too many options are paralizing[2]. Then what’s the solution? I think the only answer is to build experience. By trying, failing, learning, and sharing with everyone else.

Original title and link: The Server Architecture Debate Rages On (NoSQL database©myNoSQL)


Limit MongoDB memory use on Windows without Virtualization

If you are using MongoDB on a dedicated server then you generally want it to use all the memory it can but if you want to use it on a server shared with other processes then you will want to put a cap on how much it uses to ensure memory is kept available for the other processes.

So is it possible if you are not on a virtualized environment? Yes and we’ll explore how

The question of limiting MongoDB memory pops up quite often and this is an unexpected solution. But I’m not sure how many will start using Windows 2003 or Windows 2008 on shared servers just to have memory resource management.

Original title and link: Limit MongoDB memory use on Windows without Virtualization (NoSQL databases © myNoSQL)


MongoDB for DevOps

For some of us it is already weekend, so here is the video to enjoy in your free time. Last week, it was VoltDB.

Are there things that Foursquare could have used?

Original title and link: MongoDB for DevOps (NoSQL databases © myNoSQL)