texture
play

Texture CS 419 Slides by Ali Farhadi What is a Texture? Texture - PowerPoint PPT Presentation

Texture CS 419 Slides by Ali Farhadi What is a Texture? Texture Spectrum Steven Li, James Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu, CVPR 06 Texture scandals!! Two crucial algorithmic points Nearest neighbors again and again


  1. Texture CS 419 Slides by Ali Farhadi

  2. What is a Texture?

  3. Texture Spectrum Steven Li, James Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu, CVPR 06

  4. Texture scandals!!

  5. Two crucial algorithmic points • Nearest neighbors • again and again and again • Dynamic programming • likely new; we’ll use this again, too

  6. Texture Synthesis Efros & Leung ICCV99

  7. How to paint this pixel? ? p Input texture Efros & Leung ICCV99

  8. Ask Neighbors p p • What is the conditional probability distribution of p, given it’s neighbors? Efros & Leung ICCV99

  9. Input image • Don’t bother to model the distribution • It’s already there, in the image Efros & Leung ICCV99

  10. Efros & Leung Algorithm non-parametric sampling p Input image Synthesizing a pixel Efros & Leung ICCV99

  11. Concerns • Distance metric • Neighborhood size • Order to paint

  12. Distance metric • Normalized sum of squared distances • Not all the neighbors worth the same • Gaussian mask • Preserve the local structure • Pick among reasonably similar neighborhoods

  13. Neighborhood size input Efros & Leung ICCV99

  14. Varying Window Size Increasing window size Efros & Leung ICCV99

  15. The Order matters

  16. Some Results Efros & Leung ICCV99

  17. More Results Efros & Leung ICCV99

  18. More Results french canvas rafia weave Efros & Leung ICCV99

  19. wood granite More Results Efros & Leung ICCV99

  20. white bread brick wall More Results Efros & Leung ICCV99

  21. Growing Regions Hole Filling Efros & Leung ICCV99

  22. Hole Filling Efros & Leung ICCV99

  23. Extrapolation Efros & Leung ICCV99

  24. Failure Cases Growing garbage Verbatim copying Efros & Leung ICCV99

  25. Pros and Cons • Very simple • Easy to implement • Promising results • Very sloooooooowwwwwww • Idea: • Patches instead of pixels

  26. Patch based non-parametric sampling B Synthesizing a block Input image • Observation • neighbouring pixels are highly correlated • Idea: • unit of synthesis = block Efros & Freeman SIGGRAPH01

  27. block Input texture B1 B2 B1 B1 B2 B2 Random placement Neighboring blocks Minimal error of blocks constrained by overlap boundary cut Efros & Freeman SIGGRAPH01

  28. Minimal error boundary overlapping blocks vertical boundary 2 _ = overlap error min. error boundary Efros & Freeman SIGGRAPH01

  29. Dynamic Programming S T

  30. Dynamic Programming S T

  31. B1 B2 B1 B1 B2 B2 Random placement Neighboring blocks Minimal error of blocks constrained by overlap boundary cut Efros & Freeman SIGGRAPH01

  32. More Results Efros & Freeman SIGGRAPH01

  33. More Results Efros & Freeman SIGGRAPH01

  34. Efros & Freeman SIGGRAPH01

  35. Efros & Freeman SIGGRAPH01

  36. Efros & Freeman SIGGRAPH01

  37. Efros & Freeman SIGGRAPH01

  38. Failures Efros & Freeman SIGGRAPH01

  39. Texture Transfer • Take the texture from on object and paint it on another object = + Decomposing shape and texture Very challenging Walk around Add some constraint to the search Efros & Freeman SIGGRAPH01

  40. Destination Source Texture Destination Map Source Map

  41. Texture Transfer + = Efros & Freeman SIGGRAPH01

  42. + = Efros & Freeman SIGGRAPH01

  43. Efros & Freeman SIGGRAPH01

  44. parmesan + = rice + = Efros & Freeman SIGGRAPH01

  45. Image Analogies ? Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  46. Image Analogies Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  47. Image Analogies Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  48. Image Analogies Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  49. Training Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  50. :: : B B’ Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  51. Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  52. :: : B B’ Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  53. Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  54. Learn to Blur Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  55. Texture by Numbers Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  56. Colorization Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  57. Super-resolution A A’ Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  58. Super-resolution (result!) B’ B Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  59. Training images Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  60. Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

  61. Inpainting Criminisi et.al. CVPR03

  62. Order of inpainting matters Criminisi et al, 04

  63. Choosing the order Criminisi et al 03

  64. Constraining the match region • We don’t have to look for matches in the whole image • idea: allow user to “paint” good sources of matches on top of the image

  65. Nearest Neighbor search The core of most of the patch based methods Very slow Smarter neighborhood search Barnes et.al. SIGGRAPH09

  66. Inpainting Barnes et.al. SIGGRAPH09

  67. Applications Barnes et.al. SIGGRAPH09

  68. Retargeting • Make an image bigger or smaller in one direction • eg change aspect ratio • Traditional • cut off pixels • difficulty: lousy results • Strategy • cut out a curve of pixels that “doesn’t matter much” • low energy at pixels • many energy functions, eg

  69. Finding a seam=DP Avidan, Shamir, SIGGRAPH07

  70. • Different energies give different results • e1 = abs gradient (as above) • ehog = (look for gradients in patch) • eentropy = (entropy of patch) • eseg = (segment image, e1 in segments, 0 on boundaries)

  71. Retargeting Seam removal Scaling Cropping Avidan, Shamir, SIGGRAPH07

  72. Retargeting Avidan, Shamir, SIGGRAPH07

  73. Avidan, Shamir, SIGGRAPH07

  74. Can use constraints in retargeting Barnes et.al. SIGGRAPH09

  75. Constrained retargeting Barnes et.al. SIGGRAPH09

  76. Local scale editing Barnes et.al. SIGGRAPH09

  77. reshuffling Barnes et.al. SIGGRAPH09

  78. Barnes et.al. SIGGRAPH09

Recommend


More recommend