Texture Characterization via Joint Statistics of Wavelet Coef cient - PowerPoint PPT Presentation

Texture Characterization via Joint Statistics of Wavelet Coef Þ cient Magnitudes Eero Simoncelli Center for Neural Science, and Courant Institute of Mathematical Sciences New York University Javier Portilla Instituto de Optica Consejo Superior de Investigaciones Cienti Þ cas Madrid, SPAIN

Example Texture Types structured random periodic 2nd-order Can we derive a statistical model (and sampling technique) to rep- resent all of these? 10/98 1

Synthesis-by-Analysis Analysis Example measure Transform Texture statistics Synthesis Random Statistical Synthesized Sampler Seed Texture measure Transform statistics � Choice of statistical measurements crucial � Proper transform can simplify statistics � Most algorithms are iterative 10/98 2

Recent Inspirational Approaches � Portilla et. al. (1996): Adaptive Gabor transform, constrained subband auto-correlation. Weakness: structures. � Heeger & Bergen (1995): Steerable pyramid, constrained sub- band marginals (histograms). Weakness: periodicity, extended structures. � Zhu, Wu & Mumford (1996): Small set of Þ lters, constrained sub- band marginals, Gibbs sampling (maximal entropy). Weakness: extended structures, ef Þ ciency. � DeBonet & Viola (1997): Laplacian pyramid, coarse-to- Þ ne boot- strap sampling from the scale-conditional empirical neighbor- hood statistics. Weakness: random textures, no parameteriza- tion. 10/98 3

Complex Steerable Pyramid Representation Fourier spectra of 4-orientation 3-scale complex analytic Steerable pyramid. 10/98 4

Matched Pixel Marginals 10/98 5

Matched Subband Variances Captures smoothed distribution of energy in frequency domain. 10/98 6

Matched Subband Autocorrelation Captures periodicity. 10/98 7

Matched Subband Marginals Captures some local structure. 10/98 8

Subband Magnitudes Correlated or anti-correlated magnitudes capture important struc- ture. 10/98 9

Texture Model Parameters � Coef Þ cient magnitude correlations: E � j c j � j c j � i j � Raw coef Þ cient auto-correlation: E � c � � c x � � x � Pixel statistics: mean, variance, skew, kurtosis, min, max. � neighborhoods � � � � � � � � � orientations � ��� parameters � � � � � � scales � � � � � 10/98 10

Texture Synthesis System Analysis estimate mag statistics build Sample complex Texture pyramid estimate statistics estimate moments Synthesis impose mag statistics build collapse Gaussian complex real part of Noise pyramid pyramid impose statistics mag impose moments 10/98 11

Projection onto Constraint Surfaces � Joint magnitudes � : match correlation of local (spatial position, orientation, scale) magnitudes . Find linear transformation A � � � � minimizing: � E jj � Q jj Q A T � � � � � � � � subject to: T T A . E A Q Q A � E Q Q � � � 10/98 12

Synthesis Results Arti Þ cial textures. 10/98 13

Synthesis Results Natural textures, random. 10/98 14

Synthesis Results Natural textures, structured. 10/98 15

Synthesis Failures 10/98 16

Spatial Extrapolation 10/98 17

Scale Extrapolation 10/98 18