Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew - PowerPoint PPT Presentation

Old and New Algorithms for Blind Deconvolution Yair Weiss Hebrew University of Jerusalem joint work with Anat Levin, WIS (thanks to Meir Feder, EE, TAU)

Blind Deblurring y = x * k

Much Recent Progress (Fergus et al. 06, Cho et al. 07, Jia 07, Joshi et al. 08, Whyte et al. 10, Shan et al. 07, Harmeling et al. 10, Sroubek and Milanfar 11 · · · )

MAP using sparse derivative prior 5 10 log counts 0 10 −100 −50 0 50 100 dx | x i | α + λ � k ∗ x − y � 2 � log P ( x , k | y ) = C + i • MAP xk : ( x ∗ , k ∗ ) = arg max P ( x , k | y ) . Guaranteed to fail with global optimization. Often works well in practice. • MAP k : ( k ∗ ) = arg max k � x P ( x , k | y ) . Guaranteed to succeed if images sampled from prior. Can be difficult to optimize. (Levin et al. 09)

This talk: • Old algorithms for blind deconvolution from communication systems. � = MAP . Rigorous correctness proofs. • Can explain success of some MAP x , k algorithms.

Blind Deconvolution in Communication • x ( t ) the transmitted signal. • y ( t ) = � τ k τ x ( t − τ ) received signal. • y=x * k

Blur makes signals Gaussians (Central Limit Theorem) Orig. Signal (IID) Histogram ( κ =1) 600 500 400 300 200 100 0 −1 −0.5 0 0.5 1 Blurred Histogram ( κ = 2 . 683) 50 40 30 20 10 0 0 1 2 3 4 5 κ ( y ) = 1 � y 4 ¯ i , ¯ y i = y i / std ( y ) N i (Shalvi and Weinstein 1990)

Blur makes signals Gaussians (Central Limit Theorem) Orig. Signal (IID) Histogram ( κ =26.43) 400 300 200 100 0 −4 −2 0 2 4 Blurred Histogram ( κ = 5 . 45) 100 80 60 40 20 0 −4 −2 0 2 4

Simple Blind Deconvolution Algorithm (Shalvi and Weinstein 1980) Assume x i IID. y = x ∗ k . Solve for the “inverse filter” e such that e ∗ y = x . • x i sub-Gaussian ( κ < 3): e ∗ = arg min e κ ( e ∗ y ) (Godard 1970) • x i super-Gaussian ( κ > 3): e ∗ = arg max κ ( e ∗ y ) e Claim: Guaranteed to find the correct e : e ∗ ∗ y = x . No local minima.

Simple Proof Assume x i IID. x i super-Gaussian ( κ > 3): y = x ∗ k . e ∗ = arg max κ ( e ∗ y ) e Proof: ˆ x = e ∗ y = e ∗ k ∗ x = ( e ∗ k ) ∗ x ⇒ Unless e ∗ k = δ , ˆ x is more Gaussian than x so κ (ˆ x ) < κ ( x )

Proof by pictures: Orig. Signal (IID) Histogram ( κ =26.43) 400 300 200 100 0 −4 −2 0 2 4 Blurred Histogram ( κ = 5 . 45) 100 80 60 40 20 0 −4 −2 0 2 4 deblurred (wrong e ) Histogram ( κ = 14 . 14) 180 160 140 120 100 80 60 40 20 0 −3 −2 −1 0 1 2 3

Blind Deconvolution by Maximizing non Gaussianity Assume x i IID. x i super-Gaussian. ( κ > 3): y = x ∗ k . e ∗ = arg max κ ( e ∗ y ) e • Universal proof of correcteness (don’t need to know Pr ( x i ) , just IID and sub/super Gaussianity). • Proofs of global convergence of iterative algorithms. • Used in microwave radio transmissions (1980s), cable set-top boxes (mid 1990s) and wireless communication (late 1990s-today) (Johnson et al. 1998) • Can we use this for blind image deblurring?

Blur makes derivatives more Gaussian Sharp image dx Histogram ( κ =20.02) 5 10 0 10 −100 −50 0 50 100 Histogram ( κ = 17 . 99) Blurred image dx 5 10 blurred sharp 0 10 −100 −50 0 50 100

New class of image blind deblurring alglrithms y : blurred (deriv) image Histogram ( κ = 17 . 99) 5 10 0 10 −40 −20 0 20 40 x sharp (deriv) image Histogram ( κ =20.02) 5 10 0 10 −100 −50 0 50 100 Find ( x ∗ , k ∗ ) such that: • y = x ∗ ∗ k ∗ • x ∗ as non Gaussian as possible. If derivatives are IID, this algorithm provably finds the correct blur kernel.

Measuring non Gaussianity using normalized moments 1 � x α ¯ i , ¯ x i = x i / std ( x ) N i 0 0 0 −1 −0.5 −5 log prob log prob −2 log prob −1 −10 −1.5 −3 −15 −2 −4 −20 −2.5 −5 −25 −5 0 5 −5 0 5 −5 0 5 x x x x 4 x 4 x 4 i ¯ i ¯ i ¯ � i = 25 . 73 � i = 5 . 8 � i = 3 x 1 / 2 x 1 / 2 x 1 / 2 i ¯ i ¯ i ¯ � = 0 . 60 � = 0 . 74 � = 0 . 81 i i i

A new(?) algorithm y : Blurred image Histogram ( κ = 17 . 99) 5 10 0 10 −40 −20 0 20 40 x i | α + λ � k ∗ x − y � 2 � | ¯ ( x ∗ , k ∗ ) = arg min x , k i ¯ x i = x i / std ( x ) Guaranteed to succeed.

MAP x , k algorithm y : Blurred image Histogram ( κ = 17 . 99) 5 10 0 10 −40 −20 0 20 40 | x i | α + λ � k ∗ x − y � 2 � ( x ∗ , k ∗ ) = arg min x , k i Guaranteed to fail.

Normalized Sparsity (Krishnan et al. 11) x , k λ � x ∗ k − y � 2 + � x � 1 min + Ψ � k � 1 � x � 2 Can be seen as special case of “new” algorithm.

MAP xk with bilateral filtering (Cho and Lee 09, Hirsch et al. 11) ¯ x ← BilateralFilter ( x ) � y − k ∗ ¯ k ← arg min x � k λ � y − k ∗ x � 2 + sparsity ( x ) x ← arg min x Can be seen as approximating special case of “new” algorithm.

A new(?) algorithm y : Blurred image Histogram ( κ = 17 . 99) 5 10 0 10 −40 −20 0 20 40 x i | α + λ � k ∗ x − y � 2 � | ¯ ( x ∗ , k ∗ ) = arg min x , k i ¯ x i = x i / std ( x ) Guaranteed to succeed.

So what does this buy us? ⇔ • Understanding recent algorithms. Proof of success. • Improving recent algorithms. Filters with better independence properties. Iterative algorithms with global convergence.

Conclusions • MAP algorithms for blind image deconvolution. May work even when not supposed to. • Old algorithms for blind deconvolution in communications. Universal guarantees, global convergence, used in millions of devices. • Can help us understand and improve image deblurring algorithms.