Entropy: a consolidation manager for clusters

Fabien Hermenier; Xavier Lorca; Jean-Marc Menaud; Gilles Muller; Julia Lawall

doi:10.1145/1508293.1508300

Entropy: a consolidation manager for clusters

Fabien Hermenier, Xavier Lorca, Jean-Marc Menaud, Gilles Muller, Julia Lawall

Department of Computer Science

374 Citations (Scopus)

Abstract

Clusters provide powerful computing environments, but in practice much of this power goes to waste, due to the static allocation of tasks to nodes, regardless of their changing computational requirements. Dynamic consolidation is an approach that migrates tasks within a cluster as their computational requirements change, both to reduce the number of nodes that need to be active and to eliminate temporary overload situations. Previous dynamic consolidation strategies have relied on task placement heuristics that use only local optimization and typically do not take migration overhead into account. However, heuristics based on only local optimization may miss the globally optimal solution, resulting in unnecessary resource usage, and the overhead for migration may nullify the benefits of consolidation.

In this paper, we propose the Entropy resource manager for homogeneous clusters, which performs dynamic consolidation based on constraint programming and takes migration overhead into account. The use of constraint programming allows Entropy to find mappings of tasks to nodes that are better than those found by heuristics based on local optimizations, and that are frequently globally optimal in the number of nodes. Because migration overhead is taken into account, Entropy chooses migrations that can be implemented efficiently, incurring a low performance overhead.

Original language	English
Title of host publication	VEE'09 : Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, Washington, D.C., USA, March 11-13, 2009
Publisher	Association for Computing Machinery
Publication date	2009
Pages	41-50
ISBN (Print)	978-1-60558-375-4
DOIs	https://doi.org/10.1145/1508293.1508300
Publication status	Published - 2009
Event	ACM SIGPLAN/SIGOPS International Conference On Virtual Execution Environments - Washington, DC, United States Duration: 11 Mar 2009 → 13 Mar 2009 Conference number: 5

Conference

Conference	ACM SIGPLAN/SIGOPS International Conference On Virtual Execution Environments
Number	5
Country/Territory	United States
City	Washington, DC
Period	11/03/2009 → 13/03/2009

Access to Document

10.1145/1508293.1508300

Cite this

Entropy: a consolidation manager for clusters. / Hermenier, Fabien; Lorca, Xavier; Menaud, Jean-Marc et al.

VEE'09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, Washington, D.C., USA, March 11-13, 2009. Association for Computing Machinery, 2009. p. 41-50.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Hermenier, F, Lorca, X, Menaud, J-M, Muller, G & Lawall, J 2009, Entropy: a consolidation manager for clusters. in VEE'09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, Washington, D.C., USA, March 11-13, 2009. Association for Computing Machinery, pp. 41-50, ACM SIGPLAN/SIGOPS International Conference On Virtual Execution Environments, Washington, DC, United States, 11/03/2009. https://doi.org/10.1145/1508293.1508300

@inproceedings{cd6e2a401c3f11deb43e000ea68e967b,

title = "Entropy: a consolidation manager for clusters",

abstract = "Clusters provide powerful computing environments, but in practice much of this power goes to waste, due to the static allocation of tasks to nodes, regardless of their changing computational requirements. Dynamic consolidation is an approach that migrates tasks within a cluster as their computational requirements change, both to reduce the number of nodes that need to be active and to eliminate temporary overload situations. Previous dynamic consolidation strategies have relied on task placement heuristics that use only local optimization and typically do not take migration overhead into account. However, heuristics based on only local optimization may miss the globally optimal solution, resulting in unnecessary resource usage, and the overhead for migration may nullify the benefits of consolidation.In this paper, we propose the Entropy resource manager for homogeneous clusters, which performs dynamic consolidation based on constraint programming and takes migration overhead into account. The use of constraint programming allows Entropy to find mappings of tasks to nodes that are better than those found by heuristics based on local optimizations, and that are frequently globally optimal in the number of nodes. Because migration overhead is taken into account, Entropy chooses migrations that can be implemented efficiently, incurring a low performance overhead.",

author = "Fabien Hermenier and Xavier Lorca and Jean-Marc Menaud and Gilles Muller and Julia Lawall",

year = "2009",

doi = "10.1145/1508293.1508300",

language = "English",

isbn = "978-1-60558-375-4",

pages = "41--50",

booktitle = "VEE'09",

publisher = "Association for Computing Machinery",

note = "ACM SIGPLAN/SIGOPS International Conference On Virtual Execution Environments ; Conference date: 11-03-2009 Through 13-03-2009",

}

TY - GEN

T1 - Entropy: a consolidation manager for clusters

AU - Hermenier, Fabien

AU - Lorca, Xavier

AU - Menaud, Jean-Marc

AU - Muller, Gilles

AU - Lawall, Julia

N1 - Conference code: 5

PY - 2009

Y1 - 2009

N2 - Clusters provide powerful computing environments, but in practice much of this power goes to waste, due to the static allocation of tasks to nodes, regardless of their changing computational requirements. Dynamic consolidation is an approach that migrates tasks within a cluster as their computational requirements change, both to reduce the number of nodes that need to be active and to eliminate temporary overload situations. Previous dynamic consolidation strategies have relied on task placement heuristics that use only local optimization and typically do not take migration overhead into account. However, heuristics based on only local optimization may miss the globally optimal solution, resulting in unnecessary resource usage, and the overhead for migration may nullify the benefits of consolidation.In this paper, we propose the Entropy resource manager for homogeneous clusters, which performs dynamic consolidation based on constraint programming and takes migration overhead into account. The use of constraint programming allows Entropy to find mappings of tasks to nodes that are better than those found by heuristics based on local optimizations, and that are frequently globally optimal in the number of nodes. Because migration overhead is taken into account, Entropy chooses migrations that can be implemented efficiently, incurring a low performance overhead.

AB - Clusters provide powerful computing environments, but in practice much of this power goes to waste, due to the static allocation of tasks to nodes, regardless of their changing computational requirements. Dynamic consolidation is an approach that migrates tasks within a cluster as their computational requirements change, both to reduce the number of nodes that need to be active and to eliminate temporary overload situations. Previous dynamic consolidation strategies have relied on task placement heuristics that use only local optimization and typically do not take migration overhead into account. However, heuristics based on only local optimization may miss the globally optimal solution, resulting in unnecessary resource usage, and the overhead for migration may nullify the benefits of consolidation.In this paper, we propose the Entropy resource manager for homogeneous clusters, which performs dynamic consolidation based on constraint programming and takes migration overhead into account. The use of constraint programming allows Entropy to find mappings of tasks to nodes that are better than those found by heuristics based on local optimizations, and that are frequently globally optimal in the number of nodes. Because migration overhead is taken into account, Entropy chooses migrations that can be implemented efficiently, incurring a low performance overhead.

U2 - 10.1145/1508293.1508300

DO - 10.1145/1508293.1508300

M3 - Article in proceedings

SN - 978-1-60558-375-4

SP - 41

EP - 50

BT - VEE'09

PB - Association for Computing Machinery

T2 - ACM SIGPLAN/SIGOPS International Conference On Virtual Execution Environments

Y2 - 11 March 2009 through 13 March 2009

ER -