Ron Miller (TechCrunch):
According to Matt Trifiro, SVP at Mesosphere, this is possible because of
containerization technology developed by Google. Building on Google’s
concept, Mesos allows system administrators to take complex applications
that run at scale and use the resources of the entire datacenter as a single
unit, using containerization to isolate the processes.
Meanwhile, last week at its first conference, Docker announced the 1.0 version and Google was pretty quick to announce both support and additional tools for it.
Original title and link: Apache Mesos company Mesosphere raises $10M
Edward Capriolo asks some very good questions about the use and advantages of Apache Hadoop YARN and Mesos: :
- Can a technology like YARN or Mesos be used together with puppet or chef?
- What at the best practices when using these two things together?
- In YARNs case. How many current software packages can YARN manage outside hadoop?
- Then what?
- Aren’t YARN/Mesos just sneaky forms of devops/noops?
- With clusters spinning up and falling on command how do we monitor this environment and guarantee quality of service?
- Couldn’t AWS/open stack do this on a more general scale?
- Shouldn’t we just all be using solaris zones?
When I first learned about YARN—I still need to get more familiar with Mesos beyond Jay Kreps’s YARN and Mesos comparison—I had a much simpler question: How exactly would you use YARN?
I still don’t have a good answer to my question, but now we have a couple more specific ones. Maybe someone could help us out.
Original title and link: What Can You Do With YARN and Mesos? ( ©myNoSQL)
Jay Kreps (LinkedIn) provides an excellent response to the question in the title on a Quora thread:
- Java vs C++
- Memory scheduling vs both memory and CPU scheduling
- Unix processes vs Linux container groups
- Pull vs push resource request model
- 3x more code in YARN
- YARN integrated pluggable schedulers vs Mesos’ upcoming hierarchical scheduling
- YARN integrates with Kerberos and inherits Hadoop security
- YARN provides rach and machine locality out of the box vs Mesos allowing to implement these
- YARN is still under work vs Mesos being a mature project
- YARN is the next generation of Hadoop MapReduce so you’ll be able to use on your Hadoop cluster
- YARN is written by Yahoo/HortonWorks which has shown to be experienced with multi-tenancy and very large-scale cluster computing. But YARN still needs work and testing.
- Mesos ships with a number of out-of-the-box frameworks.
You can find out more about Mesos here and about YARN here. And if you have a Quora account go upvote Jay’s answer.
Original title and link: How Does YARN (NextGen MapReduce) and Mesos Compare? ( ©myNoSQL)