bigdata: All content about bigdata in NoSQL databases and polyglot persistence
Wednesday, 15 May 2013
Hadoop for graphs - GraphLab picks up $6.75m from Madrona and NEA
Robin Wauters for TNW:
Seattle startup GraphLab claims it is building the “fastest machine-learning analytics engine for graph datasets”, based on the popular open-source distributed graph computation framework with the same name, and it has just raised capital to come through on its promise.
Good luck to GraphLab’s team.
✚ Here’s a short list of MapReduce implementations for graphs.
Original title and link: Hadoop for graphs - GraphLab picks up $6.75m from Madrona and NEA (©myNoSQL)
via: http://thenextweb.com/insider/2013/05/14/graphlab-funding/
Tuesday, 14 May 2013
Hadoop, Moh's Law and Corollaries
Robert Novak’s proposes Moh’s law and Rob’s corollaries to Hadoop and Big Data:
- Hadoop is hard.
- Make sure your’re measuring what you think you’re measuring.
- Make sure you’re measuring what you need to be measuring.
For the first, I’m somehow confident that Cloudera and Hortonworks and others will finally solve it over time. But for the latter you are the only responsible. Not even a SaaS can save you.
Original title and link: Hadoop, Moh’s Law and Corollaries (©myNoSQL)
via: https://rsts11.wordpress.com/2013/05/14/mohs-law-and-big-data-rsts11/
This is why big data is the sweet spot for SaaS … and here are 5 reasons why it is Not
Derrick Harris in an article about some SEO-as-a-Service company I haven’t heard about:
People often ask me where the smart money is in big data. I often tell them that’s a foolish question, because I’m not an investor — but if I were, I’d look to software as a service.
There are two primary reasons why, the first of which is obvious: Companies are tired of managing applications and infrastructure, so something that optimizes a common task using techniques they don’t know on servers they don’t have to manage is probably compelling. It’s called cloud computing.
The other reason is that the big part of big data really is important if you want to get a really clear picture of what’s happening in any given space. While no single end-user company can (or likely would) address search-engine optimization, for example, by building a massive store comprised of data from hundreds or thousands of companies as well as the entire web, a cloud service dedicated to that specific task can.
These are obvious advantages of moving the responsibility to a third party service. But I don’t believe SaaS is the future of big data and here’s why big data is not the sweet spot of SaaS:
- a SaaS solution is good at a particular job, but it’s rarely the case that particular job is answering all your company questions and reveal the insights in your data. SaaS solutions will tell you want they, not you, think is important about your data.
- the promise of a SaaS solution to give you access to more aggregate data sounds wrong. Big data is mostly about your data and each customer will have access to their own slices. Indeed a SaaS solution could augment your data with open data or extra data you’d need to pay for.
- transporting your data to each SaaS to answer every question your company has is extremely expensive. If possible.
- the nature and form of the questions big data tries to answer is changing. SaaS services will not adapt as fast as you want to the range and depth you need.
- having your data in different SaaS solutions is just equivalent to having it in different internal silos. Except you’d pay someone else to protect the silo. The costs of breaking these silos will be much, much higher, so long term you might actually find a real reason why you cannot analyze your data.
Big Data is about agility. It’s about experiments. It’s trial and error. SaaS is about none of these when speaking years and years of data.
Original title and link: This is why big data is the sweet spot for SaaS … and here are 5 reasons why it is Not (©myNoSQL)
Monday, 13 May 2013
Even web giants like Facebook and Yahoo generally aren’t dealing with big data
Even web giants like Facebook and Yahoo generally aren’t dealing with big data, and the application of Google-style tools is inappropriate.
Facebook and Yahoo run their own giant, in-house “clusters”—collections of powerful servers—for crunching data. The necessity of these clusters is one of the hallmarks of big data. After all, data isn’t all that “big” if you could chew through it on your PC at home. The necessity of breaking problems into many small parts, and processing each on a large array of computers, characterizes classic big data problems like Google’s need to compute the rank of every single web page on the planet.
But it appears that for both Facebook and Yahoo, those same clusters are unnecessary for many of the tasks which they’re handed.
I guess we need some sort of “big journalism” sooner rather than later.
Original title and link: Even web giants like Facebook and Yahoo generally aren’t dealing with big data (©myNoSQL)
via: http://qz.com/81661/most-data-isnt-big-and-businesses-are-wasting-money-pretending-it-is/
What Open Source Hadoop Coming to Windows Means to IT
This will open up Hadoop to a large number of organizations that have no in- house Linux skills. Shaun Connolly, vice president of Corporate Strategy at Hortonworks, explains the thinking behind moving HDP to Windows in this way: “Essentially it’s a market-driven decision,” he says. “Hadoop is built for the scaleout commodity hardware market, and the commodity hardware market is 70% Windows by install base and expertise.”
Employees in Windows-only companies will be able to make use of Hadoop easily because Excel can be used as a business intelligence tool to view the results of Hadoop Big Data analysis (whether Hadoop is running on Windows or Linux). “Ideally we want Microsoft users to be oblivious to the fact that everything is coming from Hadoop,” says Connolly. “If end users can consume data without any learning curve, thanks to tools like Excel, then they get more value.”
Either the data or the logic above is not sound:
- those Windows machines that make up the 70% of the market are probably running Excel
- those 70% of the market Windows machines are not going to run Hadoop
Based on this sort of market-share decisions, tomorrow we should see Hadoop for iOS and Android and Nokia. Sometime soon Microsoft will release Excel for iOS and maybe Android.
Original title and link: What Open Source Hadoop Coming to Windows Means to IT (©myNoSQL)
via: http://www.cio.com/article/733260/What_Open_Source_Hadoop_Coming_to_Windows_Means_to_IT
Cloudera Announces Cloudera Developer Kit, Enabling Developers to Build Hadoop Apps Faster
I didn’t know what to think of this announcement after reading the WSJ title . After checking the project GitHub page, I still don’t know what to make of it.
Original title and link: Cloudera Announces Cloudera Developer Kit, Enabling Developers to Build Hadoop Apps Faster (©myNoSQL)
Tuesday, 30 April 2013
Hadoop Drives Down Costs
Darryl K. Taft reporting the experience of using Hadoop at UC Irvine Medical Center:
Because they were bleeding money, the team wanted a cost-effective solution. “Our target was $500 per terabyte. We were at $100,000 per terabyte with the old system,” Peterson said. “With our Hadoop cluster, we’re now at $900 per terabyte.”
How are these costs calculated?
- Fixed costs: hardware, any one time licenses
- Recurring costs: hardware replacement, energy, HR
Is this all?
Original title and link: Hadoop Drives Down Costs (©myNoSQL)
Impala 1.0 - That was fast
Cloudera announces Impala 1.0 GA release.
That was fast—I guess this is one of the (little) advantages of having Hortonworks working on Stinger, Pivotal on HAWQ, Qubole offering Hive, Pig and Sqoop as-a-Service
Original title and link: Impala 1.0 - That was fast (©myNoSQL)
Hadoop Virtualization
Roberto V. Zicari interviewing Joe Russell1 about Hadoop virtualization with Serengeti:
A common misconception when virtualizing Hadoop clusters is that we decouple the data nodes from the physical infrastructure. This is not necessarily true. When users virtualize a Hadoop cluster using Project Serengeti, they separate data from compute while preserving data locality. By preserving data locality, we ensure that performance isn’t negatively impacted, or essentially making the infrastructure appear as static. Additionally, it creates true multi-tenancy within more layers of the Hadoop stack, not just the name node.
I’m not 100% sure I get this, but the way I explained it to myself to actually make sense this would mean that HDFS lives directly on the physical hardware and only the compute part is virtualized. Is that what he means?
-
Joe Russell is Product Line Marketing Manager at VMware. ↩
Original title and link: Hadoop Virtualization (©myNoSQL)
via: http://www.odbms.org/blog/2013/04/on-virtualize-hadoop-interview-with-joe-russell/
A Value Definition of Big Data
Jim Walker:
Last year, Shaun Connolly, Hortonworks VP of Corporate Strategy came up with this definition…
Big Data = Transactions + Interactions + Observations.
Well, give me an example of any data system that doesn’t satisfy this definition.
Here’s my proposal for yet another definition of Big Data: a buzzword that we’ll never have a real definition so we’d be better moving over.
Original title and link: A Value Definition of Big Data (©myNoSQL)
via: http://hortonworks.com/blog/big-data-defined-part-deux-value-definition/
Monday, 29 April 2013
Project Savanna: Hadoop and OpenStack
Timothy Prickett Morgan for The Register about Project Savanna, a collaboration between Mirantis, Hortonworks, and Red Hat:
Batman and Robin. Peanut butter and chocolate. OpenStack and Hadoop. These are things that go together, with the latter pairing being something that commercial OpenStack distie Mirantis, commercial Hadoop distie Hortonworks, and commercial KVM and Linux distie (and soon to be OpenStack commercializer) Red Hat are putting together under a new OpenStack effort dubbed Project Savanna.
Hadoop is at the age where everyone tries to package it and claim they’ll be the Red Hat of the Hadoop ecosystem. I cannot really dot the i-s and cross the t-s, but my gut feeling is that right now all these are actually more similar to the attempts of bringing Linux to the desktop.
We know how successful these have been so far.
Original title and link: Project Savanna: Hadoop and OpenStack (©myNoSQL)
via: http://www.theregister.co.uk/2013/04/18/project_savanna_hadoop_on_openstack/
Boundary for Splunk app for correlating alerts
Alex Williams for TechCrunch:
Boundary‘s application performance monitoring technology is now integrated into Splunk‘s enterprise platform, providing a window into apps that increasingly are distributed across cloud and on-premise virtualized environments.
At first I thought this means Boundary will use Splunk as the backend for the data. But Boundary is a service so that’s not the case. Plus Splunk can already be used for network management and monitoring.
According to the post, “Splunk real-time alerts are tagged as annotations in Boundary’s time-series graphs. Customers can then correlate alerts against application flow and performance data.” So basically this is monitoring your monitoring system, right?
Original title and link: Boundary for Splunk app for correlating alerts (©myNoSQL)
