An Experimentation Workbench for Replayable Networking Research

Eric Eide, Leigh Stoller, and Jay Lepreau
{eeide,stoller,lepreau}@cs.utah.edu

University of Utah, School of Computing
www.emulab.net
Flux Technical Note FTN–2006–03
December 11, 2006

A revised version of this paper was published in Proceedings of the Fourth USENIX Symposium on Networked Systems Design and Implementation (NSDI '07), Cambridge, MA, Apr. 2007. Please read and cite the published NSDI '07 paper in preference to this report.

Abstract

The network and distributed systems research communities have an increasing need for "replayable" research, but our current experimentation resources fall short of reaching this goal. Replayable activities are those that can be re-executed, either as-is or in modified form, yielding new results that can be compared to previously obtained results. Replayability requires complete records of experiment processes and data, of course, but it also requires facilities that allow those processes to actually be examined, repeated, modified, and reused for new studies.

We are now evolving Emulab, our popular network testbed management system, to be the basis of a new "experimentation workbench" in support of realistic, large-scale, replayable networking research. We have implemented a new model of testbed-based experiments, based on scientific workflow, that allows people to move forward and backward through their experimentation processes. Integrated tools in the workbench help researchers manage their activities (both planned and unplanned), software artifacts, data, and analyses. In this paper we present the workbench, describe its implementation, and report how it has been used by early adopters. Our initial case studies, using PlanetLab nodes, cluster PCs, 802.11 wireless, and software radio, have highlighted both the utility of the current workbench and additional usability challenges that must be addressed.

• PDF