ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

Mesos: All content tagged as Mesos in NoSQL databases and polyglot persistence

What Can You Do With YARN and Mesos?

Edward Capriolo asks some very good questions about the use and advantages of Apache Hadoop YARN and Mesos: :

  1. Can a technology like YARN or Mesos be used together with puppet or chef?
    1. What at the best practices when using these two things together?
  2. In YARNs case. How many current software packages can YARN manage outside hadoop?
    1. MPI?
    2. Then what?
  3. Aren’t YARN/Mesos just sneaky forms of devops/noops?
  4. With clusters spinning up and falling on command how do we monitor this environment and guarantee quality of service?
  5. Couldn’t AWS/open stack do this on a more general scale?
  6. Shouldn’t we just all be using solaris zones?

When I first learned about YARN—I still need to get more familiar with Mesos beyond Jay Kreps’s YARN and Mesos comparison—I had a much simpler question: How exactly would you use YARN?

I still don’t have a good answer to my question, but now we have a couple more specific ones. Maybe someone could help us out.

Original title and link: What Can You Do With YARN and Mesos? (NoSQL database©myNoSQL)

via: http://www.edwardcapriolo.com/roller/edwardcapriolo/entry/trying_to_find_a_fit


How Does YARN (NextGen MapReduce) and Mesos Compare?

Jay Kreps (LinkedIn) provides an excellent response to the question in the title on a Quora thread:

  1. Java vs C++
  2. Memory scheduling vs both memory and CPU scheduling
  3. Unix processes vs Linux container groups
  4. Pull vs push resource request model
  5. 3x more code in YARN
  6. YARN integrated pluggable schedulers vs Mesos’ upcoming hierarchical scheduling
  7. YARN integrates with Kerberos and inherits Hadoop security
  8. YARN provides rach and machine locality out of the box vs Mesos allowing to implement these
  9. YARN is still under work vs Mesos being a mature project
  10. YARN is the next generation of Hadoop MapReduce so you’ll be able to use on your Hadoop cluster
  11. YARN is written by Yahoo/HortonWorks which has shown to be experienced with multi-tenancy and very large-scale cluster computing. But YARN still needs work and testing.
  12. Mesos ships with a number of out-of-the-box frameworks.

You can find out more about Mesos here and about YARN here. And if you have a Quora account go upvote Jay’s answer.

Original title and link: How Does YARN (NextGen MapReduce) and Mesos Compare? (NoSQL database©myNoSQL)