Big Data at Torbit: Custom MapReduce-like System
Tylor Arndt about Torbit’s “build-your-own-MapReduce”:
The final system begins with a web-service against which client systems interface. To ensure resiliency, an instance of the web- service runs on each cluster host. When a client request arrives the web-service creates a MapReduce job to fulfill client requests. The reducer function component of the MapReduce job runs within the web- service handling the request.
The requirements listed in the post are too high level to understand why building their own solutions was better. But if it works for them, that’s OK. Just keep in mind that NIH and distributed systems don’t always mix well.
Original title and link: Big Data at Torbit: Custom MapReduce-like System (©myNoSQL)
