How Does YARN (NextGen MapReduce) and Mesos Compare?
Jay Kreps (LinkedIn) provides an excellent response to the question in the title on a Quora thread:
- Java vs C++
- Memory scheduling vs both memory and CPU scheduling
- Unix processes vs Linux container groups
- Pull vs push resource request model
- 3x more code in YARN
- YARN integrated pluggable schedulers vs Mesos’ upcoming hierarchical scheduling
- YARN integrates with Kerberos and inherits Hadoop security
- YARN provides rach and machine locality out of the box vs Mesos allowing to implement these
- YARN is still under work vs Mesos being a mature project
- YARN is the next generation of Hadoop MapReduce so you’ll be able to use on your Hadoop cluster
- YARN is written by Yahoo/HortonWorks which has shown to be experienced with multi-tenancy and very large-scale cluster computing. But YARN still needs work and testing.
- Mesos ships with a number of out-of-the-box frameworks.
You can find out more about Mesos here and about YARN here. And if you have a Quora account go upvote Jay’s answer.
Original title and link: How Does YARN (NextGen MapReduce) and Mesos Compare? (©myNoSQL)
