Javier Sanchez* and Jean-Michel Morel** * Universidad de Las Palmas - PowerPoint PPT Presentation

MOTION SMOOTHING STRATEGIES FOR VIDEO STABILIZATION Javier Sanchez* and Jean-Michel Morel** * Universidad de Las Palmas Gran Canarias ** Ecole Normale Supérieure Paris-Saclay

In this work, we propose a unified mathematical analysis and classification of motion smoothing strategies. We adopt the classic assumption that the apparent motion of the scene is mainly induced by the camera's motion and can therefore be captured through a set of global parametric models. We show that the best choice yields a scale-space of the camera's ego-motion parameters. Displaying this scale space on examples, we show how it is highly characteristic of the camera's path, permitting to detect for example periodic ego-motions like walk or run.

Current stabilization approaches employ key-point feature tracking and linear motion estimation in the form of 2D transformations, or use Structure from Motion (SfM) to estimate the original camera path. From this original shaky camera path, a new smooth camera path is estimated by either smoothing the linear motion models [1] to suppress high frequency jitter, or fitting linear camera paths [2] augmented with smooth changes in velocity to avoid sudden jerks. If SfM is used to estimate the 3D path of the camera, more sophisticated smoothing and linear fits for the 3D motion may be employed [3]. L1-optimal camera paths to generate stabilized. In [4] the method computes camera paths that are composed of constant, linear and parabolic segments mimicking the camera motions employed by professional cinematographers by minimizing the first, second, and third derivatives of the resulting camera path. [1] Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H.-Y. Shum. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell., 2006. [2] M. L. Gleicher and F. Liu. Re-cinematography: Improving the camerawork of casual video. ACM Trans. Mult. Comput. Commun. Appl., 2008. [3] F. Liu, M. Gleicher, H. Jin, and A. Agarwala. Content-preserving warps for 3d video stabilization. In ACM SIGGRAPH, 2009. [4] M. Grundmann V. Kwatra I. Essa. Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths, preprint.

Questions: How to chose the parametric model How to smooth the camera path How to fix boundary conditions How to use the stabilization transform for motion analysis F. Liu, M. Gleicher, H. Jin, and A. Agarwala. Content-preserving warps for 3d video stabilization. In ACM SIGGRAPH, 2009. ( 3D reconstruction by structure from motion )

Formalization

Formalization Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths: uses only affine (or similarity, but how to decide?) Matthias Grundmann Vivek Kwatra Irfan Essa

Formalization