A Gentle Introduction A Gentle Introduction to Bilateral Filtering to Bilateral Filtering and its Applications and its Applications 07/10: Novel Variants of the Bilateral Filter Jack Tumblin – EECS, Northwestern University
Review: Bilateral Filter Review: Bilateral Filter A 2- -D filter window: weights vary with intensity D filter window: weights vary with intensity A 2 Range Range c c f(x) f(x) x x Domain Domain 2 Gaussian Weights: 2 Gaussian Weights: product = product = s s ellisoidal footprint footprint ellisoidal Normalize weights to Normalize weights to always sum to 1.0 always sum to 1.0
Review: Bilateral Filter Review: Bilateral Filter Range Range Why it works: graceful segmentation Why it works: graceful segmentation f(x) f(x) • Smoothing for ‘ ‘similar similar’ ’ parts parts ONLY ONLY • Smoothing for x x Domain Domain • Range Gaussian s s acts as a acts as a ‘ ‘filtered region filtered region’ ’ finder finder • Range Gaussian c c s s c c s s
Bilateral Filter Variants Bilateral Filter Variants • before the ‘Bilateral’ name : – Yaroslavsky (1985): T.D.R.I.M. – Smith & Brady (1997): SUSAN And now, a growing set of named variants: • ‘Trilateral’ Filter (Choudhury et al., EGSR 2003) • Cross-Bilateral (Petschnigg04, Eisemann04) • NL-Means (Buades 05) And more coming: application driven…
Who was first? Many Pioneers Who was first? Many Pioneers • Elegant, Simple, Broad Idea � � ‘Invented’ several times • Different Approaches, Increasing Clarity – Tomasi & Manduchi(1998): ‘Bilateral Filter’ – Smith & Brady (1995): ‘SUSAN’ “Smallest Univalue Segment Assimilating Nucleus” – Yaroslavsky(1985) ‘Transform Domain Image Restoration Methods’
New Idea! New Idea! 1985 Yaroslavsky Yaroslavsky: : 1985 A 2- -D filter window: D filter window: A 2 weights vary with intensity ONLY weights vary with intensity ONLY Range Range f(x) f(x) x x Domain Domain c c Square neighborhood, Square neighborhood, Gaussian Weighted Gaussian Weighted s s ‘similarity similarity’ ’ ‘ Normalize weights to Normalize weights to always sum to 1.0 always sum to 1.0
New Idea! New Idea! 1995 Smith: ‘ ‘SUSAN SUSAN’ ’ Filter Filter 1995 Smith: A 2- -D filter window: weights vary with intensity D filter window: weights vary with intensity A 2 Range Range c c f(x) f(x) x x Domain Domain 2 Gaussian Weights: 2 Gaussian Weights: product = product = s s ellisoidal footprint footprint ellisoidal Normalize weights to Normalize weights to always sum to 1.0 always sum to 1.0
Background: ‘ ‘Unilateral Unilateral’ ’ Filter Filter Background: e.g. traditional, linear, FIR filters e.g. traditional, linear, FIR filters Key Idea: Convolution Convolution Key Idea: - Output(x) = local weighted avg. of inputs. Output(x) = local weighted avg. of inputs. - - Weights vary within a Weights vary within a ‘ ‘window window’ ’ of nearby x of nearby x - • Smoothes away details, Smoothes away details, BUT BUT blurs result blurs result • Note that weights Note that weights always sum to 1.0 always sum to 1.0 c c weight(x) weight(x)
Bilateral Filter: Strengths Strengths Bilateral Filter: Piecewise smooth result Piecewise smooth result – averages local small details, ignores outliers averages local small details, ignores outliers – – preserves steps, large preserves steps, large- -scale ramps, and curves,... scale ramps, and curves,... – • Equivalent to anisotropic diffusion and robust statistics • Equivalent to anisotropic diffusion and robust statistics [Black98,Elad02,Durand02] [Black98,Elad02,Durand02] • Simple & Fast (esp. w/ • Simple & Fast (esp. w/ [Durand02] [Durand02] FFT FFT- -based speedup) based speedup) c c s s
Bilateral Filter: 3 Difficulties 3 Difficulties Bilateral Filter: • Poor Smoothing in Poor Smoothing in • High Gradient Regions High Gradient Regions c c • Smoothes and blunts Smoothes and blunts • cliffs, valleys & ridges cliffs, valleys & ridges s s • Can combine disjoint Can combine disjoint • signal regions signal regions Output at is Output at is average of a average of a tiny region tiny region
Bilateral Filter: 3 Difficulties 3 Difficulties Bilateral Filter: c • Poor Smoothing in Poor Smoothing in • s s High Gradient Regions High Gradient Regions • Smoothes and blunts Smoothes and blunts • cliffs, valleys & ridges cliffs, valleys & ridges • Can combine disjoint Can combine disjoint • signal regions signal regions
’ � � Weak Halos ‘Blunted Corners Blunted Corners’ Weak Halos ‘ Bilateral : Bilateral :
’ � � Weak Halos ‘Blunted Corners Blunted Corners’ Weak Halos ‘ ‘Trilateral Trilateral’ ’: : ‘
Bilateral Filter: 3 Difficulties 3 Difficulties Bilateral Filter: • Poor Smoothing in Poor Smoothing in • High Gradient Regions High Gradient Regions c • Smoothes and blunts Smoothes and blunts • cliffs, valleys & ridges cliffs, valleys & ridges s s • Disjoint regions Disjoint regions • can blend together can blend together
New Idea! New Idea! Trilateral Filter (Choudhury Choudhury 2003) 2003) Trilateral Filter ( Goal: Goal: Piecewise linear smoothing, not piecewise constant Piecewise linear smoothing, not piecewise constant Method: Method: Extensions to the Bilateral Filter Extensions to the Bilateral Filter Intensity Intensity EXAMPLE: remove noise from a piecewise linear remove noise from a piecewise linear scanline scanline EXAMPLE: Position Position
� Trilateral Filter Outline: Bilateral � Trilateral Filter Outline: Bilateral Three Key Ideas: Three Key Ideas: Tilt the filter window • the filter window • Tilt c c s according to bilaterally- - s according to bilaterally smoothed gradients smoothed gradients • Limit the filter window the filter window • Limit to connected regions to connected regions of similar smoothed gradient. of similar smoothed gradient. • Adjust Parameters Parameters • Adjust from measurements from measurements of the windowed signal of the windowed signal
� Trilateral Filter Outline: Bilateral � Trilateral Filter Outline: Bilateral Key Ideas: Key Ideas: Tilt the filter window • the filter window • Tilt c c s according to bilaterally- - s according to bilaterally smoothed gradients smoothed gradients • Limit the filter window the filter window • Limit to connected regions to connected regions of similar smoothed gradient. of similar smoothed gradient. • Adjust Parameters Parameters • Adjust from measurements from measurements of the windowed signal of the windowed signal
� Trilateral Filter Outline: Bilateral � Trilateral Filter Outline: Bilateral Key Ideas: Key Ideas: Tilt the filter window • the filter window • Tilt c c s according to bilaterally- - s according to bilaterally smoothed gradients smoothed gradients • Limit the filter window the filter window • Limit to connected regions to connected regions of similar smoothed gradient. of similar smoothed gradient. • Adjust Parameters Parameters • Adjust from measurements from measurements of the windowed signal of the windowed signal
Comparisons: Skylight Details Comparisons: Skylight Details . . Bilateral Bilateral
Comparisons: Skylight Details Comparisons: Skylight Details . . Trilateral Trilateral
. . • , , •
Trilateral Filter (Choudhury Choudhury 2003) 2003) Trilateral Filter ( • Strengths Strengths • – Sharpens Sharpens corners corners – – Smoothes similar Smoothes similar gradients gradients – – Automatic Automatic parameter parameter setting setting – – 3 3- -D D mesh de mesh de- -noising noising, too! , too! – • Weaknesses Weaknesses • – S S- -L L- -O O- -W; W; very costly connected very costly connected- -region finder region finder – – Shares Shares Bilateral Bilateral’ ’s s ‘ ‘Single Single- -pixel region pixel region’ ’ artifacts artifacts – – Noise Tolerance Noise Tolerance limits; disrupts limits; disrupts ‘ ‘tilt tilt’ ’ estimates estimates –
NEW IDEA : ‘Joint’ or ‘Cross’ Bilateral’ NEW IDEA : ‘Joint’ or ‘Cross’ Bilateral’ Petschnigg(2004) and Eisemann(2004) Petschnigg(2004) and Eisemann(2004) Bilateral � two kinds of weights NEW : get them from two kinds of images. • Smooth image A pixels locally, but • Limit to ‘similar regions’ of image B Why do this? To get ‘best of both images’
Ordinary Bilateral Filter Ordinary Bilateral Filter Bilateral � two kinds of weights, one image A : ( ) 1 ( ) ∑ = − − [ ] || || | | BF A G G A A A p q σ σ p p q q W s r ∈ S q p Image A: c c Range Range s s f(x) f(x) x x Domain Domain
‘Joint’ or ‘Cross’ Bilateral Filter ‘Joint’ or ‘Cross’ Bilateral Filter NEW: two kinds of weights, two images ( ) 1 ( ) ∑ = − − [ ] || || | | BF A G G B B A p q σ σ p p q q W s r ∈ S q p B: Clean,strong A: Noisy, dim (Flash image) (ambient image) c c c c s s s s
Recommend
More recommend
