Hadoop Tutorial Part 3: Custom Stats from Hadoop

In case you have been following this Hadoop tutorial you’ve already have it installed and configured Hadoop and you’ve probably already played with some custom partitioning solutions. So, it is time to see if you can gather some runtime stats.

It is very handy to define your own counters to track any kind of statistics about the records you are manipulating in the mapper and the reducer.

[…] One easy way to define your own counters from your Java code is:

• Declaring an enum representing your counters. The enum name is the group of the counter, and each field of the enum is the name of the counter that will be reported in this same group
• Incrementing the desired counters from your map and reduce methods through the Context of your mapper or reducer (in previous hadoop version it was through the Reporter.incrCounter() method, but the reporter no longer exists in hadoop 0.20)

via: http://philippeadjiman.com/blog/2010/01/07/hadoop-tutorial-series-issue-3-counters-in-action/