Differential geometry and stochastic dynamics with deep learning numerics

TY - JOUR

T1 - Differential geometry and stochastic dynamics with deep learning numerics

AU - Kühnel, Line

AU - Sommer, Stefan

AU - Arnaudon, Alexis

PY - 2019

Y1 - 2019

N2 - With the emergence of deep learning methods, new computational frameworks have been developed that mix symbolic expressions with efficient numerical computations. In this work, we will demonstrate how deterministic and stochastic dynamics on manifolds, as well as differential geometric constructions can be implemented in these modern frameworks. In particular, we use the symbolic expression and automatic differentiation features of the python library Theano, originally developed for high-performance computations in deep learning. We show how various aspects of differential geometry and Lie group theory, connections, metrics, curvature, left/right invariance, geodesics and parallel transport can be formulated with Theano using the automatic computation of derivatives of any orders. We will also show how symbolic stochastic integrators and concepts from non-linear statistics can be formulated and optimized with only a few lines of code. We will then give explicit examples on low-dimensional classical manifolds for visualization and demonstrate how this approach allows both a concise implementation and efficient scaling to high dimensional problems. With this paper and its accompanying code, we hope to stimulate the use of modern symbolic and numerical computation frameworks for experimental applications in mathematics, for computations in applied mathematics, and for data analysis by showing how the resulting code allows for flexibility and simplicity in implementing many experimental mathematics endeavors.

AB - With the emergence of deep learning methods, new computational frameworks have been developed that mix symbolic expressions with efficient numerical computations. In this work, we will demonstrate how deterministic and stochastic dynamics on manifolds, as well as differential geometric constructions can be implemented in these modern frameworks. In particular, we use the symbolic expression and automatic differentiation features of the python library Theano, originally developed for high-performance computations in deep learning. We show how various aspects of differential geometry and Lie group theory, connections, metrics, curvature, left/right invariance, geodesics and parallel transport can be formulated with Theano using the automatic computation of derivatives of any orders. We will also show how symbolic stochastic integrators and concepts from non-linear statistics can be formulated and optimized with only a few lines of code. We will then give explicit examples on low-dimensional classical manifolds for visualization and demonstrate how this approach allows both a concise implementation and efficient scaling to high dimensional problems. With this paper and its accompanying code, we hope to stimulate the use of modern symbolic and numerical computation frameworks for experimental applications in mathematics, for computations in applied mathematics, and for data analysis by showing how the resulting code allows for flexibility and simplicity in implementing many experimental mathematics endeavors.

KW - Automatic differentiation

KW - Deep learning numerics

KW - Differential geometry

KW - Non-linear statistics

KW - Theano

UR - http://www.scopus.com/inward/record.url?scp=85063748437&partnerID=8YFLogxK

U2 - 10.1016/j.amc.2019.03.044

DO - 10.1016/j.amc.2019.03.044

M3 - Journal article

AN - SCOPUS:85063748437

SN - 0096-3003

VL - 356

SP - 411

EP - 437

JO - Applied Mathematics and Computation

JF - Applied Mathematics and Computation

ER -