Optimizing a single path Data term: blue should match red Smoothness term: t blue at time t should match the (60) frames around t min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term
Detour: bilateral filter Slides adapted from Sylvain Paris
Objective of bilateral filtering • Smooth texture • Preserve edges
Illustration in 1D
Illustration in 1D 1D image = line of pixels
Illustration in 1D 1D image = line of pixels Better visualized as a plot pixel intensity pixel position
Definition
Definition Gaussian blur p I p = ∑ G σ s ( ∥ p − q ∥ ) I q q q only spatial distance, intensity ignored space
Definition Gaussian blur p I p = ∑ G σ s ( ∥ p − q ∥ ) I q q q only spatial distance, intensity ignored space Bilateral filter p [Aurich 95, Smith 97, Tomasi 98] I p = 1 range W p ∑ G σ s ( ∥ p − q ∥ ) G σ r ( | I p − I q | ) I q q q spatial and range distances space
Example on a real image Ä Ä Ä
Bilateral filter is not just for pixel values!
Bilateral filter is not just for pixel values! Back to stabilization…
Optimizing a single path min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term
Optimizing a single path min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term ω t , r = G t ( ∥ r − t ∥ ) ⋅ G m ( ∥ C ( r ) − C ( t ) ∥ )
Optimizing a single path min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term ω t , r = G t ( ∥ r − t ∥ ) ⋅ G m ( ∥ C ( r ) − C ( t ) ∥ ) distance between frames
Optimizing a single path min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term distance between camera poses ω t , r = G t ( ∥ r − t ∥ ) ⋅ G m ( ∥ C ( r ) − C ( t ) ∥ ) distance between frames
Optimizing a single path min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term setting the weights λ t
Optimizing a single path min ∑ ∥ P ( t ) − C ( t ) ∥ 2 + λ t ∑ ω t , r ( C ) ⋅ ∥ P ( t ) − P ( r ) ∥ 2 r ∈Ω t t data term smoothness term setting the weights λ t Run optimization with global weight For each frame While too much cropping or distortion Decrease weight and re-run
Optimizing bundled paths
Optimizing bundled paths single path smoothness between neighboring paths min ∑ O ({ P i ( t )}) + ∑ ∑ ∥ P i ( t ) − P j ( t ) ∥ 2 j ∈ N ( i ) i t i N(i)
Calculate relation Smooth relation Create frames using Detect features between photos between photos smoothed relation Input Output frames frames warping-based adaptive space-time motion path smoothing representation
Evaluation & Results
Comparison to previous methods
Comparison to commercial products
User study
Recommend
More recommend
