9/14/2015 Announcements • Reminder: A1 due this Friday Texture Tues, Sept 15 Kristen Grauman UT Austin Review Today: Texture • Edge detection: – Filter for gradient – Threshold gradient magnitude, thin • Chamfer matching to compare shapes (in terms of edge points) • Binary image analysis – Thresholding What defines a texture? – Morphological operators to “clean up” – Connected components to find regions Includes: more regular patterns Includes: more random patterns 1
9/14/2015 Texture-related tasks Scale and texture • Shape from texture – Estimate surface orientation or shape from image texture Shape from texture Texture-related tasks • Use deformation of texture from point to point to • Shape from texture estimate surface shape – Estimate surface orientation or shape from image texture • Segmentation/classification from texture cues – Analyze, represent texture – Group image regions with consistent texture • Synthesis – Generate new texture patches/images given some examples Pics from A. Loh: http://www.csse.uwa.edu.au/~angie/phdpics1.html Analysis vs. Synthesis Why analyze texture? Images:Bill Freeman, A. Efros 2
9/14/2015 http://animals.nationalgeographic.com/ What kind of response will we get with an edge detector for these images? …and for this image? Images from Malik and Perona, 1990 Image credit: D. Forsyth Psychophysics of texture Why analyze texture? • Some textures distinguishable with preattentive Importance to perception: perception – without scrutiny, eye movements • Often indicative of a material’s properties [Julesz 1975] • Can be important appearance cue, especially if shape is similar across objects • Aim to distinguish between shape, boundaries, Same or different? and texture Technically: • Representation-wise, we want a feature one step above “building blocks” of filters, edges. 3
9/14/2015 Capturing the local patterns with image measurements [Bergen & Adelson, Nature 1988] Scale of patterns influences discriminability Size-tuned linear filters Texture representation Texture representation: example • Textures are made up of repeated local patterns, so: mean mean – Find the patterns d/dx d/dy value value • Use filters that look like patterns (spots, bars, raw Win. #1 4 10 patches…) • Consider magnitude of response – Describe their statistics within each local window, e.g., original image … • Mean, standard deviation • Histogram • Histogram of “prototypical” feature occurrences statistics to summarize patterns derivative filter in small windows responses, squared 4
9/14/2015 Texture representation: example Texture representation: example mean mean mean mean d/dx d/dy d/dx d/dy value value value value Win. #1 4 10 Win. #1 4 10 Win.#2 18 7 Win.#2 18 7 original image original image … … statistics to statistics to derivative filter summarize patterns derivative filter summarize patterns in small windows in small windows responses, squared responses, squared Texture representation: example Texture representation: example Dimension 2 (mean d/dy value) mean mean mean mean d/dx d/dy d/dx d/dy value value value value Win. #1 4 10 Win. #1 4 10 Win.#2 18 7 Win.#2 18 7 … … Win.#9 20 20 Win.#9 20 20 original image … … Dimension 1 (mean d/dx value) statistics to statistics to summarize patterns summarize patterns derivative filter in small windows in small windows responses, squared Texture representation: example Texture representation: example Windows with primarily horizontal Both edges Dimension 2 (mean d/dy value) mean mean d/dx d/dy value value Win. #1 4 10 Win.#2 18 7 … Win.#9 20 20 original image … Dimension 1 (mean d/dx value) visualization of the assignment to texture “types” Windows with Windows with statistics to small gradient in primarily vertical both directions summarize patterns edges derivative filter in small windows responses, squared 5
9/14/2015 Texture representation: example Texture representation: example Dimension 2 (mean d/dy value) 2 2 D ( a , b ) ( a b ) ( a b ) 1 1 2 2 mean mean a d/dx d/dy 2 value value Dimension 2 Far: dissimilar 2 ( , ) ( ) D a b a b Win. #1 4 10 i i textures b i 1 Win.#2 18 7 Close: similar … textures Win.#9 20 20 Dimension 1 Dimension 1 (mean d/dx value) … statistics to summarize patterns in small windows Texture representation: Texture representation: example window scale • We’re assuming we know the relevant window a a size for which we collect these statistics. Dimension 2 b Possible to perform scale b selection by looking for window scale where texture description not changing. Dimension 1 b Distance reveals how dissimilar texture from window a is from texture in window b. Filter banks Filter banks orientations • Our previous example used two filters, and resulted in a 2-dimensional feature vector to “Edges” “Bars” scales describe texture in a window. – x and y derivatives revealed something about local “Spots” structure. • We can generalize to apply a collection of • What filters to put in the bank? multiple ( d ) filters: a “filter bank” – Typically we want a combination of scales • Then our feature vectors will be d -dimensional. and orientations, different types of patterns. – still can think of nearness, farness in feature space Matlab code available for these examples: http://www.robots.ox.ac.uk/~vgg/research/texclass/filters.html 6
9/14/2015 Multivariate Gaussian Filter bank 10 5 9 0 16 0 5 5 0 9 0 9 Showing magnitude of responses Image from http://w w w.texasexplorer.com/austincap2.jpg 7
9/14/2015 8
9/14/2015 9
9/14/2015 You try: Can you match the texture to Representing texture by mean abs the response? response Filters A Filters B 1 2 C 3 Mean abs responses Derek Hoiem Mean abs responses Derek Hoiem 10
9/14/2015 d -dimensional features Euclidean distance (L 2 ) d 2 D ( a , b ) ( a b ) i i i 1 [r1, r2, …, r38] We can form a feature vector from the list of responses at . . . each pixel. 3d 2d Classifying materials, “stuff” Example uses of texture in vision: analysis Figure by Varma & Zisserman Texture features Characterizing for image retrieval scene categories by texture L. W. Renninger and J. Malik. When is scene identification Y . Rubner, C. Tomasi, and L. J. Guibas. The earth mover's distance as a just texture metric for image retrieval. International Journal of Computer Vision , recognition? Vision 40(2):99-121, November 2000, Research 44 (2004) 2301 – 2311 11
9/14/2015 Texture-related tasks • Shape from texture – Estimate surface orientation or shape from image texture Segmenting • Segmentation/classification from texture cues aerial imagery – Analyze, represent texture by textures – Group image regions with consistent texture • Synthesis – Generate new texture patches/images given some examples http://www.airventure.org/2004/gallery/images/073104_satellite.jpg Texture synthesis The Challenge • Goal: create new samples of a given texture • Many applications: virtual environments, hole- filling, texturing surfaces • Need to model the whole repeated spectrum: from repeated to stochastic texture stochastic Alexei A. Efros and Thomas K. Leung, “T exture Synthesis by Non- parametric Sampling,” Proc. International Conference on Computer Vision (ICCV), 1999. Both? Markov Chains Markov Chain Example: Text “A dog is a man’s best friend. It’s a dog eat dog world out there.” Markov Chain • a sequence of random variables 2/3 1/3 a dog 1/3 1/3 1/3 • is the state of the model at time t is 1 man’s 1 best 1 friend 1 • Markov assumption : each state is dependent only on the it’s 1 previous one eat 1 – dependency given by a conditional probability : world 1 out 1 there 1 • The above is actually a first-order Markov chain . 1 • An N’th -order Markov chain: a dog is man’s best it’s . friend eat out there world Source S. Seitz Source: S. Seitz 12
Recommend
More recommend
