cragl
play

CraGL Creativity and Graphics Lab Background: Digital Painting - PowerPoint PPT Presentation

Decomposing Time-Lapse Paintings into Layers Jianchao Tan George Mason University Marek Dvoro k Czech Technical University in Prague Daniel S kora Czech Technical University in Prague Yotam Gingold George Mason University CraGL


  1. Decomposing Time-Lapse Paintings into Layers Jianchao Tan George Mason University Marek Dvoro žň ák Czech Technical University in Prague Daniel S ý kora Czech Technical University in Prague Yotam Gingold George Mason University CraGL Creativity and Graphics Lab

  2. Background: Digital Painting [Angela Sasser, https://www.artstation.com/artwork/nariko-heavenly-guardian]

  3. Background: Digital Painting Layers [Angela Sasser, https://www.artstation.com/artwork/nariko-heavenly-guardian]

  4. Background: Digital Painting Layers are RGBA images [Angela Sasser, https://www.artstation.com/artwork/nariko-heavenly-guardian]

  5. Background: Digital Painting [Angela Sasser, https://www.artstation.com/artwork/nariko-heavenly-guardian]

  6. Motivation • Physical paintings are hard to edit.

  7. Motivation • What if we have a time lapse video?

  8. Motivation • What if we have a time lapse video?

  9. Goal • Decompose a time-lapse painting video into layers

  10. Goal • Decompose a time-lapse painting video into layers

  11. Goal • Decompose a time-lapse painting video into layers

  12. Challenges • Preprocessing:

  13. Challenges • Preprocessing: painter

  14. Challenges • Preprocessing: painter shadows

  15. Challenges color shift • Preprocessing: painter shadows

  16. Challenges color shift lighting • Preprocessing: painter shadows

  17. Challenges • Recovering paint layers before after

  18. Challenges color change • Recovering paint layers before after

  19. Related Work • Interacting with editing history Su et al. [2009], VisTrails [2009], McCann and Pollard [2009; 2012], Grossman et al. [2010], Noris • et al. [2012], Denning and Pellacini [2013] , Chen et al. [2014], Matzen and Snavely [2014], Karsch et al. [2014]. Chronicle [Grossman et al. 2010]

  20. Related Work • Decomposing edits Xu et al. [2006], Amati and Brostow [2010], Fu et al. [2011], Hu et al. [2013], Richardt et al. [2014]. • Inverse Image Editing [Hu et al. 2013]

  21. Related Work • Image matting Smith and Blinn [1996], Zongker et al. [1999], Farid and Adelson [1999], Szeliski et al. [2000], • Levin et al. [2006; 2007] Blue Screen Matting [Smith and Blinn 1996]

  22. Pipeline Input Preprocess Extract Layers Edit

  23. Pipeline Input Preprocess Extract Layers Edit

  24. Pipeline Preprocess Extract Layers Edit Input

  25. Pipeline Preprocess Extract Layers Edit Input

  26. Pipeline Input Extract Layers Edit Preprocess

  27. Pipeline Input Extract Layers Edit Preprocess

  28. Pipeline Input Preprocess Edit Extract Layers

  29. Pipeline Input Preprocess Edit Extract Layers

  30. Pipeline Input Preprocess Extract Layers Edit

  31. Pipeline Input Preprocess Extract Layers Edit

  32. Pipeline Input Preprocess Extract Layers Edit

  33. Pipeline Input Preprocess Extract Layers Edit

  34. Preprocessing Overview

  35. Preprocessing Overview

  36. Preprocessing Overview time

  37. Preprocessing Overview time

  38. Preprocessing Overview The value of an unblocked pixel should be piecewise constant in time ( stable ) time

  39. Preprocessing Overview The value of an unblocked pixel should be piecewise constant in time ( stable ) time Identical sequences of stable frames provide checkpoints for the painting progress

  40. Preprocessing time

  41. Preprocessing time

  42. Preprocessing time

  43. Preprocessing time

  44. Preprocessing • See paper for: • illumination • color shift • noise removal 1D L 0 smoothing • and 
 bilateral filtering

  45. Preprocessing • See paper for: • illumination • color shift • noise removal 1D L 0 smoothing • and 
 bilateral filtering

  46. Recovering Layers ? + = before after

  47. Recovering Layers ? + = before after opaque solution our solution

  48. Recovering Layers

  49. Recovering Layers Model Porter-Duff (1983) Kubelka-Munk (1931)

  50. Recovering Layers Model Porter-Duff (1983) Kubelka-Munk (1931) The standard for: digital compositing physical compositing

  51. Recovering Layers Model Porter-Duff (1983) Kubelka-Munk (1931) The standard for: digital compositing physical compositing Compositing operation: Linear Non-linear

  52. Recovering Layers Model Porter-Duff (1983) Kubelka-Munk (1931) The standard for: digital compositing physical compositing Compositing operation: Linear Non-linear Occasionally Used in graphics: Almost everywhere Lu et al. [2014], …

  53. Porter-Duff Model • “Over” operator: After = Before · (1 − α ) + Paint · α

  54. Porter-Duff Model • “Over” operator: After = Before · (1 − α ) + Paint · α Before

  55. Porter-Duff Model • “Over” operator: After = Before · (1 − α ) + Paint · α Before Paint

  56. Porter-Duff Model • “Over” operator: After = Before · (1 − α ) + Paint · α Before Paint After

  57. Porter-Duff Model • “Over” operator: After = Before · (1 − α ) + Paint · α unknown Before Paint After

  58. Porter-Duff Model before after

  59. Porter-Duff Model before after RGB Color Space

  60. Porter-Duff Model after before before after RGB Color Space

  61. Porter-Duff Model D V AL I N I V A LI D after after I NV A L I D before before before after RGB Color Space

  62. Porter-Duff Model D V AL I N I V A LI D after after I NV A L I D before before before after RGB Color Space Find solution that minimizes alpha

  63. Porter-Duff Model D D V AL I V AL I N N I I paint V A LI D V A LI D after after after I NV A L I D I NV A L I D before before before before after RGB Color Space Find solution that minimizes alpha

  64. Porter-Duff Model + = Layer (RGBA) before after

  65. Kubelka-Munk Model • Layer model (mixing model can be found in paper) before after

  66. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance canvas : thickness canvas Transmittance canvas :

  67. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance canvas : before thickness canvas Transmittance canvas :

  68. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance paint : ? Reflectance canvas : before paint ? Transmittance paint : thickness canvas Transmittance canvas :

  69. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance overall : paint thickness canvas

  70. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance overall : after paint thickness canvas

  71. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance paint : ? paint Transmittance paint : ?

  72. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance paint : ? paint Transmittance paint : ? Find solution that maximizes Transmittance paint

  73. Kubelka-Munk Model • Layer model (mixing model can be found in paper) Reflectance paint : ? recovered paint Transmittance paint : ? recovered Find solution that maximizes Transmittance paint

  74. Kubelka-Munk Model before after Reflectance Transmittance Layer (on white canvas)

  75. Results Overview

  76. Results Overview

  77. Editing • Temporal-Spatial Selection:

  78. Editing • Coloring using Time Gradient :

  79. Editing

  80. Editing

  81. Editing

  82. Conclusion • A preprocessing method to get a clean, albedo video

  83. Conclusion • A preprocessing method to get a clean, albedo video

  84. Conclusion • Two types of solutions for extracting translucent layers

  85. Conclusion • Two types of solutions for extracting translucent layers

  86. Conclusion • Useful layers for editing

  87. Conclusion • Useful layers for editing

  88. Future Work

  89. Future Work • Camera and canvas calibration.

  90. Future Work • Camera and canvas calibration.

  91. Future Work • Camera and canvas calibration. • Single image layer extraction?

  92. Future Work • Camera and canvas calibration. • Single image layer extraction? • Apply layer data into more systems. WetPaint [Bonanni et al. 2009] • Chronicle [Grossman et al. 2010] • … •

  93. Future Work • Camera and canvas calibration. • Single image layer extraction? • Apply layer data into more systems. WetPaint [Bonanni et al. 2009] • Chronicle [Grossman et al. 2010] • … • • Apply our technique to art education.

  94. Thank You! • Contact Information • Jianchao Tan: jtan8@gmu.edu • Marek Dvoro žň ák: dvoromar@fel.cvut.cz • Daniel S ý kora: sykorad@fel.cvut.cz • Yotam Gingold: ygingold@gmu.edu • Project Website: https://cs.gmu.edu/~ygingold/timemap/ • Artists: Marcello Barenghi, Matyá š Vesel ý , Dani Jones, semisecretsoftware (YouTube) • Sponsors: • United States National Science Foundation, Google. • Technology Agency of the Czech Republic, Czech Science Foundation, Grant Agency of the Czech Technical University in Prague

  95. P-D and K-M Comparison Layers P-D K-M Input

Recommend


More recommend