Energy efficiency for MapReduce workloads: an in-depth study

Boliang Feng; Jiaheng Lu; Yongluan Zhou; Nan Yang

Energy efficiency for MapReduce workloads: an in-depth study

Boliang Feng, Jiaheng Lu, Yongluan Zhou, Nan Yang

15 Citations (Scopus)

Abstract

Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processingmo del for parallel computingin data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an indepth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key findingis that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.

Original language	English
Title of host publication	Proceedings of the Twenty-Third Australasian Database Conference
Editors	Rui Zhang, Yanchun Zhang
Number of pages	10
Publisher	Australian Computer Society
Publication date	Jan 2012
Pages	61-70
ISBN (Electronic)	978-1-921770-05-0
Publication status	Published - Jan 2012
Externally published	Yes
Event	23rd Australasian Database Conference - Melbourne, Australia Duration: 31 Jan 2012 → 3 Feb 2012 Conference number: 23

Conference

Conference	23rd Australasian Database Conference
Number	23
Country/Territory	Australia
City	Melbourne
Period	31/01/2012 → 03/02/2012

Series	Conferences in Research and Practice in Information Technology
Volume	124
ISSN	1445-1336

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

http://crpit.com/confpapers/CRPITV124Feng.pdf

Cite this

Energy efficiency for MapReduce workloads : an in-depth study. / Feng, Boliang; Lu, Jiaheng; Zhou, Yongluan et al.

Proceedings of the Twenty-Third Australasian Database Conference. ed. / Rui Zhang; Yanchun Zhang. Australian Computer Society, 2012. p. 61-70 (Conferences in Research and Practice in Information Technology, Vol. 124).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Feng, B, Lu, J, Zhou, Y & Yang, N 2012, Energy efficiency for MapReduce workloads: an in-depth study. in R Zhang & Y Zhang (eds), Proceedings of the Twenty-Third Australasian Database Conference. Australian Computer Society, Conferences in Research and Practice in Information Technology, vol. 124, pp. 61-70, 23rd Australasian Database Conference, Melbourne, Victoria, Australia, 31/01/2012. <http://crpit.com/confpapers/CRPITV124Feng.pdf>

@inproceedings{6266b28ab5bb4e13a6db6c1502e26fa3,

title = "Energy efficiency for MapReduce workloads: an in-depth study",

abstract = "Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processingmo del for parallel computingin data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an indepth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key findingis that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.",

author = "Boliang Feng and Jiaheng Lu and Yongluan Zhou and Nan Yang",

note = "Best papers; 23rd Australasian Database Conference, ADC 2012 ; Conference date: 31-01-2012 Through 03-02-2012",

year = "2012",

month = jan,

language = "English",

series = "Conferences in Research and Practice in Information Technology",

publisher = "Australian Computer Society",

pages = "61--70",

editor = "Rui Zhang and Yanchun Zhang",

booktitle = "Proceedings of the Twenty-Third Australasian Database Conference",

}

TY - GEN

T1 - Energy efficiency for MapReduce workloads

T2 - 23rd Australasian Database Conference

AU - Feng, Boliang

AU - Lu, Jiaheng

AU - Zhou, Yongluan

AU - Yang, Nan

N1 - Conference code: 23

PY - 2012/1

Y1 - 2012/1

N2 - Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processingmo del for parallel computingin data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an indepth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key findingis that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.

AB - Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processingmo del for parallel computingin data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an indepth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key findingis that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.

M3 - Article in proceedings

T3 - Conferences in Research and Practice in Information Technology

SP - 61

EP - 70

BT - Proceedings of the Twenty-Third Australasian Database Conference

A2 - Zhang, Rui

A2 - Zhang, Yanchun

PB - Australian Computer Society

Y2 - 31 January 2012 through 3 February 2012

ER -

Energy efficiency for MapReduce workloads: an in-depth study

Abstract

Conference

UN SDGs

Access to Document

Fingerprint

Cite this