Motion compensated frame interpolation with a symmetric optical flow constraint

TY - GEN

T1 - Motion compensated frame interpolation with a symmetric optical flow constraint

AU - Rakêt, Lars Lau

AU - Roholm, Lars

AU - Bruhn, Andrés

AU - Weickert, Joachim

N1 - Conference code: 8

PY - 2012

Y1 - 2012

N2 - We consider the problem of interpolating frames in an image sequence. For this purpose accurate motion estimation can be very helpful. We propose to move the motion estimation from the surrounding frames directly to the unknown frame by parametrizing the optical flow objective function such that the interpolation assumption is directly modeled. This reparametrization is a powerful trick that results in a number of appealing properties, in particular the motion estimation becomes more robust to noise and large displacements, and the computational workload is more than halved compared to usual bidirectional methods. The proposed reparametrization is generic and can be applied to almost every existing algorithm. In this paper we illustrate its advantages by considering the classic TV-L 1 optical flow algorithm as a prototype. We demonstrate that this widely used method can produce results that are competitive with current state-of-the-art methods. Finally we show that the scheme can be implemented on graphics hardware such that it becomes possible to double the frame rate of 640 x 480 video footage at 30 fps, i.e. to perform frame doubling in realtime.

AB - We consider the problem of interpolating frames in an image sequence. For this purpose accurate motion estimation can be very helpful. We propose to move the motion estimation from the surrounding frames directly to the unknown frame by parametrizing the optical flow objective function such that the interpolation assumption is directly modeled. This reparametrization is a powerful trick that results in a number of appealing properties, in particular the motion estimation becomes more robust to noise and large displacements, and the computational workload is more than halved compared to usual bidirectional methods. The proposed reparametrization is generic and can be applied to almost every existing algorithm. In this paper we illustrate its advantages by considering the classic TV-L 1 optical flow algorithm as a prototype. We demonstrate that this widely used method can produce results that are competitive with current state-of-the-art methods. Finally we show that the scheme can be implemented on graphics hardware such that it becomes possible to double the frame rate of 640 x 480 video footage at 30 fps, i.e. to perform frame doubling in realtime.

U2 - 10.1007/978-3-642-33179-4_43

DO - 10.1007/978-3-642-33179-4_43

M3 - Article in proceedings

SN - 978-3-642-33178-7

VL - Part I

T3 - Lecture notes in computer science

SP - 447

EP - 457

BT - Advances in Visual Computing

A2 - Bebis, George

A2 - Boyle, Richard

A2 - Parvin, Bahram

A2 - Koracin, Darko

A2 - Fowlkes, Charless

A2 - Wang, Sen

A2 - Choi, Min-Hyng

A2 - Mantler, Stephan

A2 - Schulze, Jürgen

A2 - Acevedo, Daniel

A2 - Mueller, Klaus

A2 - Papka, Michael

PB - Springer

T2 - 8th International Symposium on Visual Computing

Y2 - 16 July 2012 through 18 July 2012

ER -