Deepak Bala sharing his recommendations for running Riak on EC2 based on his own experience:

There are a couple of problems to field when deploying Riak.

1. The EC2 instances that are provisioned by default change the following on restart.

   • Private IP address
   • Public IP address
   • Private DNS
   • Public DNS
   • EBS instances provide stable durable storage while Ephemeral storage provides for better predictable performance at the cost of losing data on restarts.

2. Performance.

Original title and link: Deploying Riak on EC2 - What to Pick? (NoSQL database©myNoSQL)

via: http://deepakbala.me/2013/02/08/deploying-riak-on-ec2/

Way too much to be learned from this fantastic post.

Original title and link: Benchmarking EC2 I/O: An Extensive Analysis by Scalyr (NoSQL database©myNoSQL)

via: http://blog.scalyr.com/2012/10/16/a-systematic-look-at-ec2-io/

Put together by the team at Altoros Systems Inc., this time run in the Amazon EC2 and including Cassandra, HBase, MongoDB, MySQL Cluster, sharded MySQL and Riak:

After some of the results had been presented to the public, some observers said MongoDB should not be compared to other NoSQL databases because it is more targeted at working with memory directly. We certainly understand this, but the aim of this investigation is to determine the best use cases for different NoSQL products. Therefore, the databases were tested under the same conditions, regardless of their specifics.

Teaser: HBase got the best results in most of the benchmarks (with flush turned off though). And I’m not sure the setup included the latest HBase read improvements from Facebook.

Original title and link: YCSB Benchmark Results for Cassandra, HBase, MongoDB, MySQL Cluster, and Riak (NoSQL database©myNoSQL)

via: http://www.networkworld.com/cgi-bin/mailto/x.cgi?pagetosend=/news/tech/2012/102212-nosql-263595.html&pagename=/news/tech/2012/102212-nosql-263595.html&pageurl=http://www.networkworld.com/news/tech/2012/102212-nosql-263595.html&site=printpage&nsdr=n

James Hamilton puts in perspective the last two new I/O related features coming from Amazon: the high performance I/O EC2 instances and EBS provisioned IOPS together with EBS-optimized EC2 instances:

With the announcement today, EC2 customers now have access to two very high performance storage solutions. The first solution is the EC2 High I/O Instance type announced last week which delivers a direct attached, SSD-powered 100k IOIPS for $3.10/hour. In today’s announcement this direct attached storage solution is joined by a high-performance virtual storage solution. This new type of EBS storage allows the creation of striped storage volumes that can reliably delivery 10,000 to 20,000 IOPS across a dedicated virtual storage network.

I’ve already said it, but this confirms it once again that Amazon is addressing most of the complains of running I/O intensive applications on EC2 and EBS.

Original title and link: I/O Intensive Apps and Amazon Cloud Improvements: EBS Provisioned IOPS & Optimized Instance Types (NoSQL database©myNoSQL)

via: http://perspectives.mvdirona.com/2012/08/01/EBSProvisionedIOPSOptimizedInstanceTypes.aspx

Jeff Barr:

In order to meet this need, we are introducing a new family of EC2 instances¹ that are designed to run low-latency, I/O-intensive applications, and are an exceptionally good host for NoSQL databases such as Cassandra and MongoDB.

Many complains about running databases on EC2 instances were about the I/O. I guess Amazon has been hearing this loud and clear.

Original title and link: Amazon Introduces High I/O SSD-backed EC2 Instances (NoSQL database©myNoSQL)

via: http://aws.typepad.com/aws/2012/07/new-high-io-ec2-instance-type-hi14xlarge.html

Adrian Cockcroft:

The SSD based system running the same workload had plenty of IOPS left over and could also run compaction operations under full load without affecting response times. The overall throughput of the 12-instance SSD based system was CPU limited to about 20% less than the existing system, but with much lower mean and 99th percentile latency. This sizing exercise indicated that we could replace the 48 m2.4xlarge and 36 m2.xlarge with 15 hi1.4xlarge to get the same throughput, but with much lower latency.

Tons of details and data about the benchmarks Netflix ran against the new high I/O SSD-backed EC2 instances. Results are even more impressive than the IOPS numbers in Werner Vogel’s High performance I/O instances for EC2.

Original title and link: Benchmarking High Performance I/O With SSD for Cassandra on AWS (NoSQL database©myNoSQL)

via: http://techblog.netflix.com/2012/07/benchmarking-high-performance-io-with.html

The story of how Wordnik changed a monolithic platform to one based on Micro Services and the implications at the data layer (MongoDB):

To address this, we made a significant architectural shift. We have split our application stack into something called Micro Services — a term that I first heard from the folks at Netflix. […] This translates to the data tier as well. We have low cost servers, and they work extremely well when they stay relatively small. Make them too big and things can go sour, quickly. So from the data tier, each service gets its own data cluster. This keeps services extremely focused, compact, and fast — there’s almost no fear that some other consumer of a shared data tier is going to perform some ridiculously slow operation which craters the runtime performance. Have you ever seen what happens when a BI tool is pointed at the runtime database? This is no different.

Original title and link: Wordnik: Migrating From a Monolythic Platform to Micro Services (NoSQL database©myNoSQL)

via: http://blog.wordnik.com/with-software-small-is-the-new-big

A complete and detailed guide for setting up a Hadoop cluster using MongoDB by Arten Yankov. It uses the MongoDB Hadoop adapter mongo-hadoop , which provides input and output adapters, support for InputSplits, and write-only Pig.

What is covered in the tutorial:

• Creating an AMI with the custom settings (installed hadoop and mongo-hadoop)
• Launching a hadoop cluster on EC2
• Adding more nodes to the cluster
• Running some sample jobs

Original title and link: NoSQL Tutorial: Setting Up a Hadoop Cluster with MongoDB Support on EC2 (NoSQL database©myNoSQL)

via: http://artemyankov.com/post/16717104998/how-to-set-up-a-hadoop-cluster-with-mongo-support-on

Using MongoDB replicate sets:

At Famigo, we house all of our valuable data in MongoDB and we also serve all requests from Amazon EC2 instances. We’ve devoted many mental CPU cycles to finding the right architecture for our data in the cloud, focusing on 3 main factors: cost, reliability, and performance.

Original title and link: Reliable, Scalable, and Kinda Sorta Cheap: A Cloud Hosting Architecture for MongoDB (NoSQL database©myNoSQL)

via: http://www.codypowell.com/taods/2011/08/a-cloud-hosting-architecture-for-mongodb.html

Zachary Witte:

When you have the instance basically set, go back into the AWS control panel, right click the instance and choose Create Image. You can start up any number of these for the replica set, but you need to change the /etc/hostname and /etc/hosts file to reflect the individual IP address and hostname of the bot (db1, db2, db3, etc.)

Before you set up MongoDB on EC2 make sure you understand the various aspects of running MongoDB in the Amazon cloud:

• MongoDB on EC2: The Basic Setup
• MongoDB in the Amazon Cloud
• MongoDB on Amazon EC2 with EBS volumes
  

Original title and link: Setting Up MongoDB Replica Sets on Amazon EC2 (NoSQL database©myNoSQL)

via: http://www.zacwitte.com/how-to-set-up-ubuntu-w-mongodb-replica-sets-on-amazon-ec2