Practical cloud evaluation from a Nordic eScience user perspective

TY - GEN

T1 - Practical cloud evaluation from a Nordic eScience user perspective

AU - Edlund, Åke

AU - Koopmans, Maarten

AU - Shah, Zeeshan Ali

AU - Livenson, Ilja

AU - Orellana, Frederik

AU - Kommeri, Jukka

AU - Tuisku, Miika

AU - Lehtovuori, Pekka

AU - Hansen, Klaus Marius

AU - Neukirchen, Helmut

AU - Hvannberg, Ebba

N1 - Conference code: 5

PY - 2011

Y1 - 2011

N2 - In this paper, we describe the findings of the NEON project - a cross-Nordic - Sweden, Norway, Denmark, Finland and Iceland - project evaluating the usefulness of private versus public cloud services for HPC users. Our findings are briefly that private cloud technology is not mature enough yet to provide a transparent user experience. It is expected that this will be the case mid 2012. The cost efficiency of both public and private cloud should be continuously monitored as there is a strong downward trend. This conclusion is supported by NEON experimenting as well as larger initiatives e.g. StratusLab reports. Public cloud technology is mature enough but lacks certain features that will be necessary to include cloud resources in a transparent manner in a national infrastructure like the Norwegian NOTUR (www.notur.no) case, e.g. with respect to quota management. These features are expected to emerge in 2011 via third party management software and in the best of breed public cloud services. Public clouds are competitive in the low end for non-HPC jobs (low memory, low number of cores) on price. A significant fraction (ca. 20%) of the jobs running on the current Nordic supercomputer infrastructure are potentially suitable for cloud-like technology. This holds in particular for singlethreaded or single-node jobs with small/medium memory requirements and non-intensive I/O. There is a backlog of real supercomputer jobs that suffers from the non-HPC jobs on the supercomputer infrastructure. Off-loading these non-HPC jobs to a public cloud would effectively add supercomputing capacity. Another finding is that available storage capacity is not accessible in a user-friendly way; most storage clouds are only accessible via programmable interfaces. A number of experiments and piloting are presented to support these claims.

AB - In this paper, we describe the findings of the NEON project - a cross-Nordic - Sweden, Norway, Denmark, Finland and Iceland - project evaluating the usefulness of private versus public cloud services for HPC users. Our findings are briefly that private cloud technology is not mature enough yet to provide a transparent user experience. It is expected that this will be the case mid 2012. The cost efficiency of both public and private cloud should be continuously monitored as there is a strong downward trend. This conclusion is supported by NEON experimenting as well as larger initiatives e.g. StratusLab reports. Public cloud technology is mature enough but lacks certain features that will be necessary to include cloud resources in a transparent manner in a national infrastructure like the Norwegian NOTUR (www.notur.no) case, e.g. with respect to quota management. These features are expected to emerge in 2011 via third party management software and in the best of breed public cloud services. Public clouds are competitive in the low end for non-HPC jobs (low memory, low number of cores) on price. A significant fraction (ca. 20%) of the jobs running on the current Nordic supercomputer infrastructure are potentially suitable for cloud-like technology. This holds in particular for singlethreaded or single-node jobs with small/medium memory requirements and non-intensive I/O. There is a backlog of real supercomputer jobs that suffers from the non-HPC jobs on the supercomputer infrastructure. Off-loading these non-HPC jobs to a public cloud would effectively add supercomputing capacity. Another finding is that available storage capacity is not accessible in a user-friendly way; most storage clouds are only accessible via programmable interfaces. A number of experiments and piloting are presented to support these claims.

KW - Faculty of Science

KW - computer

U2 - 10.1145/1996121.1996129

DO - 10.1145/1996121.1996129

M3 - Article in proceedings

SP - 29

EP - 37

BT - Proceedings of the 5th international workshop on Virtualization technologies in distributed computing

PB - Association for Computing Machinery

T2 - 5th International Workshop on Virtualization Technologies in Distributed Computing

Y2 - 8 June 2011 through 8 June 2011

ER -