Implementing Parallel Google Map-Reduce in Eden

Jost Berthold; Mischa Dieterle; Rita Loogen

doi:10.1007/978-3-642-03869-3

Implementing Parallel Google Map-Reduce in Eden

Jost Berthold, Mischa Dieterle, Rita Loogen

Department of Computer Science

14 Citations (Scopus)

Abstract

Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden's specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.

Original language	English
Title of host publication	Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Number of pages	13
Publisher	Springer
Publication date	2009
Pages	990-1002
ISBN (Print)	978-3-642-03868-6
DOIs	https://doi.org/10.1007/978-3-642-03869-3
Publication status	Published - 2009
Event	Euro-Par 2009 - Delft, Netherlands Duration: 25 Aug 2009 → 28 Aug 2009 Conference number: 15

Conference

Conference	Euro-Par 2009
Number	15
Country/Territory	Netherlands
City	Delft
Period	25/08/2009 → 28/08/2009

Series	Lecture notes in computer science
Volume	5704
ISSN	0302-9743

Access to Document

10.1007/978-3-642-03869-3

Cite this

Berthold, J., Dieterle, M., & Loogen, R. (2009). Implementing Parallel Google Map-Reduce in Eden. In Proceedings of the 15th International Euro-Par Conference on Parallel Processing: Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden’s specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons. (pp. 990-1002). Springer. Lecture notes in computer science Vol. 5704 https://doi.org/10.1007/978-3-642-03869-3

Implementing Parallel Google Map-Reduce in Eden. / Berthold, Jost; Dieterle, Mischa; Loogen, Rita.

Proceedings of the 15th International Euro-Par Conference on Parallel Processing: Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden’s specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.. Springer, 2009. p. 990-1002 (Lecture notes in computer science, Vol. 5704).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Berthold, J, Dieterle, M & Loogen, R 2009, Implementing Parallel Google Map-Reduce in Eden. in Proceedings of the 15th International Euro-Par Conference on Parallel Processing: Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden’s specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.. Springer, Lecture notes in computer science, vol. 5704, pp. 990-1002, Euro-Par 2009, Delft, Netherlands, 25/08/2009. https://doi.org/10.1007/978-3-642-03869-3

Berthold J, Dieterle M, Loogen R. Implementing Parallel Google Map-Reduce in Eden. In Proceedings of the 15th International Euro-Par Conference on Parallel Processing: Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden’s specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.. Springer. 2009. p. 990-1002. (Lecture notes in computer science, Vol. 5704). doi: 10.1007/978-3-642-03869-3

Berthold, Jost ; Dieterle, Mischa ; Loogen, Rita. / Implementing Parallel Google Map-Reduce in Eden. Proceedings of the 15th International Euro-Par Conference on Parallel Processing: Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden’s specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.. Springer, 2009. pp. 990-1002 (Lecture notes in computer science, Vol. 5704).

@inproceedings{51e780a0fae311de825d000ea68e967b,

title = "Implementing Parallel Google Map-Reduce in Eden",

abstract = "Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden's specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.",

author = "Jost Berthold and Mischa Dieterle and Rita Loogen",

year = "2009",

doi = "10.1007/978-3-642-03869-3",

language = "English",

isbn = "978-3-642-03868-6",

series = "Lecture notes in computer science",

publisher = "Springer",

pages = "990--1002",

booktitle = "Proceedings of the 15th International Euro-Par Conference on Parallel Processing",

note = "Euro-Par 2009 ; Conference date: 25-08-2009 Through 28-08-2009",

}

TY - GEN

T1 - Implementing Parallel Google Map-Reduce in Eden

AU - Berthold, Jost

AU - Dieterle, Mischa

AU - Loogen, Rita

N1 - Conference code: 15

PY - 2009

Y1 - 2009

N2 - Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden's specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.

AB - Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden's specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.

U2 - 10.1007/978-3-642-03869-3

DO - 10.1007/978-3-642-03869-3

M3 - Article in proceedings

SN - 978-3-642-03868-6

T3 - Lecture notes in computer science

SP - 990

EP - 1002

BT - Proceedings of the 15th International Euro-Par Conference on Parallel Processing

PB - Springer

T2 - Euro-Par 2009

Y2 - 25 August 2009 through 28 August 2009

ER -

Implementing Parallel Google Map-Reduce in Eden

Abstract

Conference

Access to Document

Fingerprint

Cite this