ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

Debating usefulness of log data - That needle in the haystack of useful big data may be smaller than we thought

Barb Darrow for GigaOm citing a report looking into how much useful information can be extracted from log data:

When it comes to application log data, you might be surprised at how little of it really matters, says Logentries. […] Out of 22 billion log events across 6,000 Heroku applications, just 0.18 percent held information that a developer (or a devops pro) would actually need to know to prevent a failure or boost performance, according to Logentries.

The trick with such conclusions (and reports) is in defining what is the useful information. To exemplify:

  1. consider a constant traffic unchanged web application deployed on a stable platform. Indeed logs will contain little information. But change any of the above (deploy a new version of the app, change the hardware, or suppose there is a spike in traffic). The information you’ll find in the logs will become critical.
  2. consider a hacked application. The logs will become your initial source of tracing this down. The more you have the better chances you’ll have to figure things out.

Original title and link: Debating usefulness of log data - That needle in the haystack of useful big data may be smaller than we thought (NoSQL database©myNoSQL)

via: http://gigaom.com/2013/10/31/that-needle-in-the-haystack-of-useful-big-data-may-be-smaller-than-we-thought/