Outline Outline Digital Matting Digital Matting Introduction to Digital Matting Introduction to Digital Matting 1. 1. Bayesian Matting Bayesian Matting 2. 2. Poisson Matting Poisson Matting 3. 3. A Closed Form Solution to Matting A Closed Form Solution to Matting 4. 4. Presenting: Presenting: Alon Gamliel Gamliel, Tel , Tel- -Aviv University, May 2006 Aviv University, May 2006 Alon Introduction to Digital Matting Introduction to Digital Matting Introduction to Digital Matting Introduction to Digital Matting Problem: Intricate boundaries – – hair hair Problem: Intricate boundaries Goal: Separate a foreground object strands and fur. strands and fur. from the background Original Photo Object with new background Compositing equation: Introduction to Digital Matting Introduction to Digital Matting Solution: Opacity Solution: Opacity Alpha Matte Image = * + * C = α * F + (1- α ) * B 1
Blue Screen Matting Compositing equation: CR = α * FR + ( 1 – α ) * BR C = α F + (1- α )B CG = α * FG + ( 1 – α ) * BG CB = α * FB + ( 1 – α ) * BB Solve: Given C, find (F,B, α ) for each pixel The Problem: 3 equations, 7 unknowns Chroma Keying, “Blue Screen” Matting Statistical Methods Berman et al. 2000, Ruzon Berman et al. 2000, Ruzon and and Tomasi Tomasi 2000, 2000, Chuang et al. 2001 Chuang et al. 2001 � Known Background Known Background � Input User Supplied Alpha Matte Compositing Trimap Image � Controlled Scene Controlled Scene � Statistical Methods Color Model Acquisition - Summary 1. gather statistical information from user 1. gather statistical information from user trimap trimap Calculate P(B) Given image Calculate P(F) 2. solve for F, B, 2. solve for F, B, α α in the unknown region in the unknown region Unknown region Rely on good separation in color space Rely on good separation in color space Blue Screen Natural Matting Natural Matting Natural Matting Geometric Model Geometric Model Probabilistic Model Bayesian Model 2
Bayesian Matting Bayesian Matting Overview Bayesian Matting Overview “ “ Video Matting of Complex Scenes Video Matting of Complex Scenes ” ” , SIGGRAPH 2002 , SIGGRAPH 2002 � 1. Start with a user 1. Start with a user trimap trimap � “ A Bayesian Approach to Digital Matting “ A Bayesian Approach to Digital Matting ” ” , CVPR 2001 , CVPR 2001 Chuang, , Agarwala Agarwala, , Curless Curless, , Salesin Salesin, , Szeliski Szeliski Chuang � 2. Solve for boundaries of the unknown region � Estimate F,B, α using probabilistic framework, relying on nearest pixels from trimap � 3. Refine trimap � 4. Back to (2) Bayesian Matting Bayesian Matting – Details � Estimating L( Estimating L(C| C|F,B, F,B, α α ) ) Find most likely Find most likely � � � α ) values for each α = α (F, B, α (F, B, ) values for each ( , , ) arg max ( , , | ) F B P F B C pixel. pixel. α F , B , = α α arg max ( | , , ) ( ) ( ) ( ) / ( ) Apply Bayes Bayes’ ’ Rule Rule P C F B P F P B P P C Apply � � α , , F B P(C) is constant with P(C) is constant with = α α � � arg max P ( C | F , B , ) P ( F ) P ( B ) P ( ) F,B, α α � This log-likelihood models error in the measurement F,B, α , , F B of C and corresponds to a gaussian probability = α + + + α arg max L ( C | F , B , ) L ( F ) L ( B ) L ( ) Use log- -likelihood likelihood Use log � � distribution centered at μ = α F+(1- α )B with standard α , , F B deviation σ C Estimating Color Correspondence Bayesian Matting – Details Nearest known F points L(F) (weighted) � The probability that color c agree with The probability that color c agree with gaussian gaussian � color model, parameterized with μ μ , , Σ Σ color model, parameterized with � Estimating L(F),L(B) Estimating L(F),L(B) � ( ) ( ) ⎛ ⎞ � How well estimated F,B How well estimated F,B − μ T Σ − − μ 1 � 1 c c μ Σ = ⎜ − ⎟ f c ( , ) exp correspond to F/B color correspond to F/B color ⎜ ⎟ ( ) 2 π d Σ ⎝ ⎠ 2 det( ) model model � Taking the maximum logarithm (in respect to c) Taking the maximum logarithm (in respect to c) � ( ) ( ) μ Σ = − − μ Σ − − μ T 1 L c ( , ) c c 3
Bayesian Matting – Details Bayesian Matting – Results α ) � Eliminating L( Eliminating L( α ) � � With no other good assumption, assume L( With no other good assumption, assume L( α α ) is ) is � α constant for every possible constant for every possible α � Solving: Solving: � � Derive from Derive from argmax( argmax(L( L(C| C|F,B F,B, , α α )+L(F)+L(B)) )+L(F)+L(B)) � two sets of linear equations two sets of linear equations α � Solve iteratively, alternating the sets, assuming that Solve iteratively, alternating the sets, assuming that α � or F/B are constant each iteration F/B are constant each iteration or Video Matting Optical Flow Video Matting Optical Flow � Problem Problem � � Bayesian matting requires a trimap for each frame Bayesian matting requires a trimap for each frame � � Drawing Drawing trimaps trimaps is labor is labor- -intensive intensive � � Solution Solution � � Draw Draw trimaps trimaps only for key only for key- -frames frames � � Interpolate Interpolate trimaps trimaps between key between key- -frames frames � = + � Optical Flow � How to interpolate? How to interpolate? � Optical Flow ( ) ( ) C C � x x u + i 1 i � C(x C(x) : color at pixel ) : color at pixel coord coord. x . x � – called � u(x u(x) : velocity of the pixel at x ) : velocity of the pixel at x – called flow field flow field � Video Result Video Result Poisson Matting Poisson Matting “ “ Poisson Matting Poisson Matting ” ” , SIGGRAPH 2004 , SIGGRAPH 2004 Sun, Sun, Jia Jia, Tang, Shum , Tang, Shum 4
Poisson Matting Overview Initializing Iterative Process Poisson Matting Overview Initializing Iterative Process � Assumption: The matting gradients mimic the image Assumption: The matting gradients mimic the image � Start from a user Start from a user- -defined defined trimap trimap � � gradients gradients Ω � 1. Start with a user 1. Start with a user trimap trimap � Ω F � 2. Estimate alpha values in unknown area, based on 2. Estimate alpha values in unknown area, based on � Ω B image gradients and known alpha values image gradients and known alpha values � 3. Refine 3. Refine trimap trimap � � 4. Back to (2) 4. Back to (2) � For each p in Ω Ω , measure F&B from the nearest pixels � For each p in , measure F&B from the nearest pixels � in Ω Ω F & Ω Ω B in F & B Estimating Matting Field Calculating α Estimating Matting Field Calculating α � Ω Ω – – unknown area I = α F + (1 + (1- - α )B unknown area I = α F α )B � � Ω Ω F F – – trimap trimap foreground area foreground area � ∇ = − ∇ α + α ∇ + − α ∇ ( ) ( 1 ) � Ω Ω B B – – trimap I F B F B trimap background area background area � � Assume smooth foreground & background Assume smooth foreground & background � Solve: Solve: � � α ∇ + − α ∇ << − ∇ α F ( 1 ) B ( F B ) � We get: We get: � 1 are taken from nearest pixels at Ω Ω F / Ω Ω B � F F p p /B /B p p are taken from nearest pixels at F / ∇ α ≈ ∇ � B I − F B Calculating α Calculating Refinement Refinement α � Minimizing is equivalent to solving the Poisson Minimizing is equivalent to solving the Poisson � equation: equation: � Ω Ω F+ Ω , = { p in Ω F+ = { p in , s.t s.t. . α α P P > 0.95 (I > 0.95 (I P P ~ F ~ F P P ) } ) } � ∂ α ∂ α ⎛ ∇ ⎞ � Ω Ω B+ Ω , 2 2 = { p in Ω I B+ = { p in , s.t s.t. . α α P P < 0.05 (I < 0.05 (I P P ~ B ~ B P P ) } ) } � Δ α = + = ⎜ ⎟ div ∂ ∂ − 2 2 ⎝ ⎠ x y F B � Generate a new Generate a new trimap trimap � � Assuming Assuming Dirichlet Dirichlet boundary condition boundary condition � � Iterate for Iterate for “ “few few” ” iterations iterations � � F, B and are measured in grayscale F, B and are measured in grayscale ∇ � I � Solve linear system Solve linear system � 5
Recommend
More recommend
