On GPU-based nearest neighbor queries for large-scale photometric catalogs in astronomy

TY - GEN

T1 - On GPU-based nearest neighbor queries for large-scale photometric catalogs in astronomy

AU - Heinermann, Justin

AU - Kramer, Oliver

AU - Polsterer, Kai Lars

AU - Gieseke, Fabian

N1 - Conference code: 36

PY - 2013

Y1 - 2013

N2 - Nowadays astronomical catalogs contain patterns of hundreds of millions of objects with data volumes in the terabyte range. Upcoming projects will gather such patterns for several billions of objects with peta-and exabytes of data. From a machine learning point of view, these settings often yield unsupervised, semi-supervised, or fully supervised tasks, with large training and huge test sets. Recent studies have demonstrated the effectiveness of prototype-based learning schemes such as simple nearest neighbor models. However, although being among the most computationally efficient methods for such settings (if implemented via spatial data structures), applying these models on all remaining patterns in a given catalog can easily take hours or even days. In this work, we investigate the practical effectiveness of GPU-based approaches to accelerate such nearest neighbor queries in this context. Our experiments indicate that carefully tuned implementations of spatial search structures for such multi-core devices can significantly reduce the practical runtime. This renders the resulting frameworks an important algorithmic tool for current and upcoming data analyses in astronomy.

AB - Nowadays astronomical catalogs contain patterns of hundreds of millions of objects with data volumes in the terabyte range. Upcoming projects will gather such patterns for several billions of objects with peta-and exabytes of data. From a machine learning point of view, these settings often yield unsupervised, semi-supervised, or fully supervised tasks, with large training and huge test sets. Recent studies have demonstrated the effectiveness of prototype-based learning schemes such as simple nearest neighbor models. However, although being among the most computationally efficient methods for such settings (if implemented via spatial data structures), applying these models on all remaining patterns in a given catalog can easily take hours or even days. In this work, we investigate the practical effectiveness of GPU-based approaches to accelerate such nearest neighbor queries in this context. Our experiments indicate that carefully tuned implementations of spatial search structures for such multi-core devices can significantly reduce the practical runtime. This renders the resulting frameworks an important algorithmic tool for current and upcoming data analyses in astronomy.

U2 - 10.1007/978-3-642-40942-4_8

DO - 10.1007/978-3-642-40942-4_8

M3 - Article in proceedings

SN - 978-3-642-40941-7

T3 - Lecture notes in computer science

SP - 86

EP - 97

BT - KI 2013: Advances in Artificial Intelligence

A2 - Timm, Ingo J.

A2 - Thimm, Matthias

PB - Springer

T2 - 36th Annual German Conference on Artificial Intelligence

Y2 - 16 September 2013 through 20 September 2013

ER -