A paper authored by a team from MIT CSAIL whose goal is to identify various scalability issues in the Linux kernel:
This paper analyzes the scalability of seven system applications (Exim,
memcached, Apache, PostgreSQL, gmake, Psearchy, and MapReduce) running on
Linux on a 48- core computer. Except for gmake, all applications trigger
scalability bottlenecks inside a recent Linux kernel. Us-ing mostly
standard parallel programming techniques—this paper introduces one new
technique, sloppy counters—these bottlenecks can be removed from the
kernel or avoided by changing the applications slightly. Modifying the
kernel required in total 3002 lines of code changes. A speculative
conclusion from this analysis is that there is no scalability reason to give
up on traditional operating system organizations just yet.
Interesting choice of tools. Note that the team used an in-memory file system to eliminate the disk-related bottlenecks.
Original title and link: Paper: An Analysis of Linux Scalability to Many Cores ( ©myNoSQL)
Goldmacher estimated that YouTube consumption—user uploads of 48 hours of video a minute and 3 billion videos a day along with roughly 45 petabytes of viewed videos a day—would require at least 9 full-rack Exadata machines at $1.5 million each. There would be at least 18 Exadata machines to handle spikes. Those machines would add up to 14 Exalogic devices to serve data at $1.1 million per system. The software stack under Oracle would include WebLogic middleware, Oracle databases, Exadata optimized storage and Oracle as operating system. The open source comparison included JBoss middleware, MySQL, Hadoop and Red Hat Enterprise Linux as the OS.
Credit Peter Goldmacher (Cowen & Co. analyst)
Two comments (the only I have):
- what advantages would the enterprise stack offer to justify a 5x cost?
- in case all numbers are completely wrong, what’s the advantage of the enterprise stack?
Original title and link: Enterprise Big Data Stack vs Open Source Big Data Stack ( ©myNoSQL)
The installation details are for Ubuntu, but who knows maybe somebody can get this to work on Mac OS too.