RStorm: Developing and Testing Streaming Algorithms in R

Authors

Maurits Kaptein, Tilburg University

Date of this Version

6-2014

Document Type

Article

Citation

The R Journal (June 2014) 6(1); Editor: Deepayan Sarkar

Comments

Copyright 2014, The R Foundation. Open access material. License: CC BY 3.0 Unported

Abstract

Streaming data, consisting of indefinitely evolving sequences, are becoming ubiquitous in many branches of science and in various applications. Computer scientists have developed streaming applications such as Storm and the S4 distributed stream computing platform1 to deal with data streams. However, in current production packages testing and evaluating streaming algorithms is cumbersome. This paper presents RStorm for the development and evaluation of streaming algorithms analogous to these production packages, but implemented fully in R. RStorm allows developers of streaming algorithms to quickly test, iterate, and evaluate various implementations of streaming algorithms. The paper provides both a canonical computer science example, the streaming word count, and examples of several statistical applications of RStorm.