Ian Winter describes the possible approaches to migrate their memcached server farm:
- Big Bang. Update all the configuration files and simply push that code live. This has a huge hit on performance as all the new instances are cold, it is however the quickest option.
- Migration instance by instance. Each instance would be moved and relevant configuration files pushed live. This way only 1 or X instances is “cold”. This limits impact, but, also takes time as only 1 instance can be done at any one time.
- Write to both. The application code could be altered to make all writes to both the existing set of instances and the new. This isn’t great as we’d need a full QA cycle to validate the code works. It’s also something we’d have to implement then pull back out down the line.
- Mirror traffic. Similar to the above, but, this time lower down the stack. Essentially duplicate all the TCP level traffic so it warms in parallel and keeps both instances in sync meaning new writes, deletes occur on both existing and new sets.
Can you tell which approach they used?
PS: For the first two approaches I’d use different names: Stop the World and Rolling upgrade respectively.
Original title and link: Migrating Live Memcached Servers ( ©myNoSQL)
Pierre Bailet and Mathieu Poumeyrol of fotopedia (a French photo site) share their experience of operating a small MongoDB cluster since Sep.2009 compared to a MySQL cluster.
Some details about fotopedia:
- fotopedia is 100% on AWS
- Amazon RDS for MySQL
- 4 nodes MongoDB cluster
- 150mil. photo views
- no alter table
- background index creation
- data backup & restoration
note: as far as I can tell MySQL is able to do the same
- replica sets
- hardware migration
note: the same procedure can be used for MySQL
Before leaving you with the slides, here is an interesting accepted trade-off:
Quietly losing seconds of writes is preferable to:
- weekly minutes-long maintenance periods
- minutes-long unscheduled downtime and manual failover in case of hardware failures
Continue to read ➤
From the Big Data and NoSQL space: Hadoop and Cassandra. And related to this space: OpenStack and Puppet.
So to judge importance, I looked at projects that are influential, gaining in popularity, and/or technical standouts in new areas. In other words, projects that are even more noteworthy than the other noteworthy projects. This means that many projects that are crucial didn’t make the list.
On my list of projects left out: Redis, Riak, HBase, and MongoDB. By looking at the explanation of why Android didn’t make the list, I could understand why not including Redis, Riak, and MongoDB. But I’d keep HBase.
Original title and link: Hadoop and Cassandra in the Top 10 Most Important Open Source Projects of 2011 ( ©myNoSQL)
Great slides from mongolab‘s Todd O. Dampier about operating MongoDB:
- Stay up => high availability
- Stay fast => performance & scale
- Take good care of your data => data durability
- Always know what’s going on => monitoring & alerting
Whatever database you are planning to use for a busy application, someone that specialized in operating it will give you the best insights. They’ve seen the best and worst of it.
Continue to read ➤
- Writing reams of documentation sucks. Chef drastically reduces the amount of documentation you have to write.
- Bash doesn’t scale. Seriously.
- Technical Awesomeness
- Chef grows with you
- You can stop reinventing the wheel
There’s no large scale without automation. Just ask Netflix about their army of monkeys.
Original title and link: The Top 5 Reasons to Use Chef ( ©myNoSQL)
The guys from Boxed Ice have already published a lot about their experience running MongoDB in production, plus a series of posts advising on MongoDB monitoring. Actually they are offering a hosted service for MongoDB monitoring: Server Density .
If you still don’t have MongoDB monitoring in place—unfortunately Foursquare had to learn what this means the hard way—check this talk from David Mytton.
Original title and link: Monitoring MongoDB ( ©myNoSQL)
Big processors or little processors, scale-up or scale-out, on-premise or in the cloud […] The plethora of choices for application architecture and delivery model are great if you like variety, but I don’t envy anyone tasked with choosing which system on which to spend their limited budget dollars.
Too little options is bad. Too many options are paralizing. Then what’s the solution? I think the only answer is to build experience. By trying, failing, learning, and sharing with everyone else.
Original title and link: The Server Architecture Debate Rages On ( ©myNoSQL)
If you are using MongoDB on a dedicated server then you generally want it to use all the memory it can but if you want to use it on a server shared with other processes then you will want to put a cap on how much it uses to ensure memory is kept available for the other processes.
So is it possible if you are not on a virtualized environment? Yes and we’ll explore how
The question of limiting MongoDB memory pops up quite often and this is an unexpected solution. But I’m not sure how many will start using Windows 2003 or Windows 2008 on shared servers just to have memory resource management.
Original title and link: Limit MongoDB memory use on Windows without Virtualization (NoSQL databases © myNoSQL)
For some of us it is already weekend, so here is the video to enjoy in your free time. Last week, it was VoltDB.
Are there things that Foursquare could have used?
Original title and link: MongoDB for DevOps (NoSQL databases © myNoSQL)