Computer Science Department
School of Computer Science, Carnegie Mellon University
Dynamic Function Placement in Active Storage Clusters
Khalil Amiri, David Petrou, Gregory R. Ganger, Garth A. Gibson
Keywords: Distributed systems, performance of systems, management,
software configuration management, operating systems, storage management,
filesystems management, organization and design
Optimally partitioning application and filesystem functionality within a
cluster of clients and servers is a difficult problem due to dynamic
variations in application behavior, resource availability and workload mixes.
This paper presents ABACUS, a run-time system that monitors and dynamically
changes function placement for applications that manipulate large data sets.
Several examples of data-intensive workloads are used to show the importance
of proper function placement and its dependence on dynamic run-time
characteristics, with performance differences frequently reaching 2-10X. We
evaluate how well the ABACUS prototype adapts to run-time system behavior,
including both long-term variation (e.g., filter selectivity) and short-term
variation (e.g., multi-phase applications and inter-application resource
contention). Our experiments with ABACUS indicate that it is possible to adapt
in all of these situations and that the adaptation converges most quickly in
those cases where the performance impact is most significant.