Direct training of dynamic observation noise with UMarineNet

TY - GEN

T1 - Direct training of dynamic observation noise with UMarineNet

AU - Oehmcke, Stefan

AU - Zielinski, Oliver

AU - Kramer, Oliver

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Accurate uncertainty predictions are crucial to assess the reliability of a model, especially for neural networks. Part of this uncertainty is the observation noise, which is dynamic in our marine virtual sensor task. Typically, dynamic noise is not trained directly, but approximated through terms in the loss function. Unfortunately, this noise loss function needs to be scaled by a trade-off-parameter to achieve accurate uncertainties. In this paper we propose an upgrade to the existing architecture, which increases interpretability and introduces a novel direct training procedure for dynamic noise modelling. To that end, we train the point prediction model and the noise model separately. We present a new loss function that requires Monte Carlo runs of the model to directly train for the uncertainty prediction accuracy. In an experimental evaluation, we show that in most tested cases the uncertainty prediction is more accurate than the manually tuned trade-off-parameter. Because of the architectural changes we are able to analyze the importance of individual parts of the time series of our prediction.

AB - Accurate uncertainty predictions are crucial to assess the reliability of a model, especially for neural networks. Part of this uncertainty is the observation noise, which is dynamic in our marine virtual sensor task. Typically, dynamic noise is not trained directly, but approximated through terms in the loss function. Unfortunately, this noise loss function needs to be scaled by a trade-off-parameter to achieve accurate uncertainties. In this paper we propose an upgrade to the existing architecture, which increases interpretability and introduces a novel direct training procedure for dynamic noise modelling. To that end, we train the point prediction model and the noise model separately. We present a new loss function that requires Monte Carlo runs of the model to directly train for the uncertainty prediction accuracy. In an experimental evaluation, we show that in most tested cases the uncertainty prediction is more accurate than the manually tuned trade-off-parameter. Because of the architectural changes we are able to analyze the importance of individual parts of the time series of our prediction.

KW - CNN

KW - LSTM

KW - Predictive uncertainty

KW - Time series

UR - http://www.scopus.com/inward/record.url?scp=85054878493&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01418-6_13

DO - 10.1007/978-3-030-01418-6_13

M3 - Article in proceedings

AN - SCOPUS:85054878493

SN - 9783030014179

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 123

EP - 133

BT - Artificial Neural Networks and Machine Learning – ICANN 2018 - 27th International Conference on Artificial Neural Networks, 2018, Proceedings

A2 - Kurkova, Vera

A2 - Hammer, Barbara

A2 - Manolopoulos, Yannis

A2 - Iliadis, Lazaros

A2 - Maglogiannis, Ilias

PB - Springer Verlag,

T2 - 27th International Conference on Artificial Neural Networks, ICANN 2018

Y2 - 4 October 2018 through 7 October 2018

ER -