Vectorising Bitmaps into Semi-Transparent Gradient Layers 1 - PowerPoint PPT Presentation

Vectorising Bitmaps into Semi-Transparent Gradient Layers 1 Christian Richardt 1,2 2 Jorge Lopez-Moreno 1,3 Adrien Bousseau 1 3 Maneesh Agrawala 4 4 George Drettakis 1 1

photos drawings vector art 2

Vector art representations 3

Vector art representations single layer 3

Vector art representations single layer multiple layers 3

Image vectorisation Ardeco Gradient meshes Diffusion curves [Lecot & Lévy 2006] [Sun+ 2007] [Orzan+ 2008] 4

Our interactive workflow Shutterstock/George Dolgikh 5

Our interactive workflow Shutterstock/George Dolgikh 5

Our interactive workflow Shutterstock/George Dolgikh 5

Our interactive workflow 6

Vectorised result 7

Editing result 8

Similarity to matting + = I = α · F + (1 − α ) · B compositing equation [Porter & Duff 1984] 9

The matting problem [Smith & Blinn 1996] I = α · F + (1 − α ) · B we have 3 equations: one each for R, G, B 10

The matting problem [Smith & Blinn 1996] I = α · F + (1 − α ) · B we have 3 equations: one each for R, G, B know solve for # unknowns 7 underconstrained α F B I 4 underconstrained α F I B 10

Solving the matting problem [Smith & Blinn 1996] I 1 = α · F + (1 − α ) · B 1 we know: I 2 = α · F + (1 − α ) · B 2 I 1 I 2 B 1 B 2 solve for: α F 6 equations, B 1 B 2 4 unknowns great! 11

Image decompositions Alpha matting [e.g. Smith+ 1996, Chuang+ 2001, Levin+ 2008] Reflection separation [e.g. Levin+ 2004/2007, Kim+ 2013, Li & Brown 2014] Intrinsic images [e.g. Bousseau+ 2009, Carroll+ 2011] 12

Decompositing 13

Decompositing 13

Decompositing I 1 13

Decompositing B 1 I 1 13

Decompositing B 1 I 1 I 2 B 2 13

Decompositing 14

Decompositing 14

Decompositing 14

Decompositing 15

Decompositing 15

Decompositing 15

Parametric gradient functions f = c � g = c ( g ( x , θ ) , θ ) 16

Parametric gradient functions f = c � g = c ( g ( x , θ ) , θ ) g ( x , θ ) gradient function Linear gradient g linear ( x , θ ) = x · θ v k θ v k 2 + θ o 0 1 Radial gradient g radial ( x , θ ) = k x � θ p k 0 1 θ r 16

Parametric gradient functions f = c � g = c ( g ( x , θ ) , θ ) c ( β , θ ) colour function Two-stop gradient � c1 � c2 c 2 ( β , θ ) = mix( θ c1 , θ c2 , β ) � s1 � s2 � 0 1 8 ⇣ ⌘ β Three-stop gradient mix θ c1 , θ c2 , β ≤ θ s2 < � c1 � c2 � c3 θ s2 c 3 ( β , θ ) = � s1 � s2 � s3 ⇣ ⌘ θ c2 , θ c3 , β − θ s2 mix β > θ s2 : � 1 − θ s2 0 1 mix( a , b , t )=(1 − t ) · a + t · b 17

Foreground estimation I = α · F + (1 � α ) · B = F � B I ( x ) = F ( x ) � B ( x ) I ( x ) = f ( x , θ ) � B ( x ) � 2 X � arg min I ( x ) � f ( x , θ ) � B ( x ) θ , B x ∈ R pixel position x gradient parameters θ selected image region R 18

Foreground estimation � 2 X � arg min I ( x ) � f ( x , θ ) � B ( x ) θ , B x ∈ R I ( b ( x )) I ( x ) pixel position x gradient parameters θ selected image region R background sample b 18

Foreground estimation � 2 X � arg min I ( x ) � f ( x , θ ) � I ( b ( x )) θ x ∈ ∂ R I ( b ( x )) ≈ B ( x ) I ( x ) pixel position x gradient parameters θ selected image region R background sample b region boundary ∂ R 18

Foreground estimation � 2 X � arg min I ( x ) � f ( x , θ ) � I ( b ( x )) θ x ∈ ∂ R pixel position x gradient parameters θ selected image region R background sample b region boundary ∂ R 18

Background estimation Shutterstock/Picsfive Input photo (slightly blurred) 19

Background estimation Shutterstock/Picsfive Input photo (slightly blurred) 19

Background estimation Shutterstock/Picsfive Input photo (slightly blurred) 19

Background estimation Shutterstock/Picsfive with hard region boundary 20

Background estimation Shutterstock/Picsfive hard region boundary 21

Background estimation Shutterstock/Picsfive trimap from hard region boundary 22

Background estimation Shutterstock/Picsfive matted region boundary 23

Background estimation Shutterstock/Picsfive with matted region boundary 24

Background estimation Shutterstock/Picsfive plus TV-smoothed region 25

Background estimation Shutterstock/Picsfive plus Poisson blending 26

Background estimation Shutterstock/Picsfive plus Poisson blending 26

Summary 1. joint decompositing and vectorisation of foreground: solving the matting problem around region boundary strong prior on foreground + user input 2. background estimation by optimisation: similar to inversion of compositing equation additional terms to remove residuals: TV smoothness + Poisson blending 27

Input photo Shutterstock/Givaga 28

Vectorised result 29

Editing result 30

Input photo Flickr/squinza ( CC BY - SA 2.0) 31

Vectorised result 32

Input drawing Spencer Nugent 33

Vectorised result 34

Editing result 35

Limitation: few iso-contours foreground background input image ground truth our decomposition 36

Limitation: background textures input photo our recomposited result 37

Limitation: background textures input photo estimated background 37

Future work more complex semi-transparent vector primitives automatic segmentation and decompositing extract Vector Shade Trees [Lopez-Moreno+ 2013] from exemplar materials 38

Conclusion key insight: complex images can often be explained by stacking simple layers first approach creating layered vector art from bitmaps: opaque and semi-transparent gradient layers produces a simple, editable stack of vector layers valuable for professionals and novices alike We thank: Inria CRISP associate team, ANR-12-JS02-003-01 DRAO, research donation from Adobe. 39