cloud computing: All content tagged as cloud computing in NoSQL databases and polyglot persistence
Cadir Lee (CTO Zynga) quoted in a VentureBeat post:
It’s not the amount of hardware that matters. It’s the architecture of the application. You have to work at making your app architecture so that it takes advantage of Amazon. You have to have complete fluidity with the storage tier, the web tier. We are running our own data centers. We are looking more at doing our own data centers with more of a private cloud.
Couple of thoughts:
- Zynga is going the opposite direction than Netflix. While Netflix is focusing (by using Amazon for most of their infrastructure), Zynga is diversifying (building their own data centers) .
- Zynga’s applications are great examples of where fully distributed NoSQL databases fit. Availability is key.
- My answer to the question: “how many Zyngas are out there” would be: “enough to ensure some good business for the most reliable and scalable distributed databases”
- Zynga has contributed and is an investor in Membase, the company that merged with CouchOne to form Couchbase. But Zynga was using a custom version of Membase.
- Zynga also operates a large MySQL cluster.
- Zynga processes over 15 terabytes of game data every day (according to their SEC filing ). That’s Hadoop sweet spot.
PS: I’d love to talk to someone from Zynga about their data storage approach. If you have any connections I’d really appreciate an introduction.
Original title and link: Zynga, Data Centers, Polyglot Persistence, and Big Data ( ©myNoSQL)
Do you remember the 5 lessons Netflix learned while using the Amazon Web Services—judging by how much Netflix shared about their experience in the cloud including Amazon SimpleDB I’d say these 5 are only the tip of the iceberg—where they talked about the Chaos Monkey?
One of the first systems our engineers built in AWS is called the Chaos Monkey. The Chaos Monkey’s job is to randomly kill instances and services within our architecture. If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most – in the event of an unexpected outage.
Hadoop provides a similar framework: Fault Injection Framework :
The idea of fault injection is fairly simple: it is an infusion of errors and exceptions into an application’s logic to achieve a higher coverage and fault tolerance of the system. Different implementations of this idea are available today. Hadoop’s FI framework is built on top of Aspect Oriented Paradigm (AOP) implemented by AspectJ toolkit.
As a sidenote, this is one of the neatest usages of AspectJ I’ve read about.
Update: Abhijit Belapurkar says that Fault injection using AOP was part of Recovery Oriented Computing research at Stanford/UCB many years ago: JAGR: An Autonomous Self-Recovering Application Server.
Original title and link: Hadoop Chaos Monkey: The Fault Injection Framework ( ©myNoSQL)
The workshop takes you through creating a Sinatra application using sample code from here . Once the Sinatra application which leverages Twitter is working, the workshop then takes you through adding Redis to your application. Finally the workshop ends after taking you through scaling your application instances up and then back down.
Only 15 minutes to get it up and running:
VMWare’s Cloud Foundry has the potential to become the preferred PaaS solution. It bundles together a set of services that it took years for other PaaS providers (Google App Engine, Microsoft Azure) to offer. And it seems that Cloud Foundry has much less (or none at all) vendor lock in.
From a storage perspective, Cloud Foundry is encouraging polyglot persistence right from the start offering access to a relational database (MySQL), a super-fast smart key-value store (Redis), and a popular document database (MongoDB). The only bit missing is a graph database.
I think the first graph database to get there will see an immediate bump in its adoption.
Original title and link: Cloud Foundry, NoSQL Databases, and Polyglot Persistence (NoSQL databases © myNoSQL)
There’s a bit of CouchDB in the project:
We’ve built an open-source product that automatically deploys, scales and distributes VoIP calls across the Internet on commodity or virtualized servers. It fully utilizes Erlang for VoIP logic as well as relies on other Erlang products like CouchDB and RabbitMQ. It’s got an awesome set of APIs and some other nifty features.