RevoScaleR: R for BigData
Dave Rosenberg:
the new package will allow users to process, visualize, and model terabyte-class data sets in a matter of seconds, and it leverages many popular data processors and storage mechanisms, including the popular Apache Hadoop framework and countless NoSQL databases, for complex statistical analysis.
The RevoScaleR package introduces a number of new features, including:
- a new binary ‘Big Data’ file format—XDF—with an interface to the R language that provides high-speed access to arbitrary rows, blocks, and columns of data
- a collection of the most common statistical algorithms optimized for big data, including high-performance implementations of summary statistics, linear regression, binomial logistic regression, and crosstabs
- data reading and transformation tools to prepare large data sets for analysis
