Energy efficiency for MapReduce workloads: an in-depth study

TY - GEN

T1 - Energy efficiency for MapReduce workloads

T2 - 23rd Australasian Database Conference

AU - Feng, Boliang

AU - Lu, Jiaheng

AU - Zhou, Yongluan

AU - Yang, Nan

N1 - Conference code: 23

PY - 2012/1

Y1 - 2012/1

N2 - Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processingmo del for parallel computingin data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an indepth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key findingis that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.

AB - Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processingmo del for parallel computingin data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an indepth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key findingis that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.

M3 - Article in proceedings

T3 - Conferences in Research and Practice in Information Technology

SP - 61

EP - 70

BT - Proceedings of the Twenty-Third Australasian Database Conference

A2 - Zhang, Rui

A2 - Zhang, Yanchun

PB - Australian Computer Society

Y2 - 31 January 2012 through 3 February 2012

ER -