Seam Carving: problem: target many devices How to Rescale scaling? - PDF document

Changing Image Size …while keeping content intact Seam Carving:  problem: target many devices How to “Rescale”  scaling? cropping? seam carving! P Pictures CS 176 Winter 2011 CS 176 Winter 2011 1 2 Basic Idea Seams in Action Remove inconspicuous pixels  one pixel from each (and every) row (or column) CS 176 Winter 2011 CS 176 Winter 2011 3 4 Seams in Action Finding the Seam? CS 176 Winter 2011 CS 176 Winter 2011 5 6

The Optimal Seam Optimal Path What is a valid path?  monotonic, continuous  cost is evaluated per pixel  e g centered difference  e.g., centered difference  other cost measures…  find by dynamic programming  optimal substructure…  lots over overlapping sub problems CS 176 Winter 2011 CS 176 Winter 2011 7 8 Construction Typical Seams Top to bottom (for vertical seams)  from second to last row 5 8 9 3 9 9 2 2 3 3 9 9 7 3 4 2 4 5 7 8 CS 176 Winter 2011 CS 176 Winter 2011 9 10 Image Enlargement Image Enlargement Run “in reverse” Run “in reverse”  insert seams as interpolants of  insert seams as interpolants of neighbors by increasing energy neighbors by increasing energy CS 176 Winter 2011 CS 176 Winter 2011 11 12

Both Dimensions Object Removal Retargeting horizly. and vertly. Additional weighting possible  what order for given size?  lower or higher energy on demand  dynamic programming again  binary array of decisions CS 176 Winter 2011 CS 176 Winter 2011 13 14 Object Removal Refinements Additional weighting possible Forward and backward energy  lower or higher energy on demand  energy after removing seam?  could go up! CS 176 Winter 2011 CS 176 Winter 2011 15 16 Artifacts Changes in Image CS 176 Winter 2011 CS 176 Winter 2011 (c) ariel shamir 17 18

Changes in Image Changes in Image CS 176 Winter 2011 CS 176 Winter 2011 19 20 Energy Inserted/Removed Changes in Energy Energy Reduced Energy Increased CS 176 Winter 2011 CS 176 Winter 2011 21 22 Tracking Energy Pixel P i,j : Left Seam p i-1,j-1 p i-1,j-1 p i-1,j p i-1,j+1 p i-1,j-1 p i-1,j-1 p i-1,j p i-1,j+1 p i,j-1 p i,j p i,j p i,j+1 p i,j-1 p i,j p i,j p i,j+1 CS 176 Winter 2011 CS 176 Winter 2011 23 24

Pixel P i,j : Right Seam Pixel P i,j : Vert. Seam p i-1,j-1 p i-1,j p i-1,j+1 p i-1,j+1 p i-1,j-1 p i-1,j p i-1,j p i-1,j+1 p i,j-1 p i,j p i,j p i,j+1 p i,j-1 p i,j p i,j p i,j+1 CS 176 Winter 2011 CS 176 Winter 2011 25 26 New Energy Function Backward (SIG 07) CS 176 Winter 2011 CS 176 Winter 2011 27 28 Forward (SIG 08) Backward CS 176 Winter 2011 CS 176 Winter 2011 29 30

Forward Backward CS 176 Winter 2011 CS 176 Winter 2011 31 32 Forward Backward Expand CS 176 Winter 2011 CS 176 Winter 2011 33 34 Forward Expand As Graph Cut Problem Motivated by video…  set up graph so that optimal seam is optimal cut of graph O(VE 2 )  or LP  or LP  expensive…  need other tricks to make practical  we’ll ignore that for now CS 176 Winter 2011 CS 176 Winter 2011 35 36

Challenge Constraints Seams need to be… How to Define a Seam from a Cut?  connected Kwatra et al. Siggraph 2003, Graph cut textures  monotonic Piecewise Connected CS 176 Winter 2011 CS 176 Winter 2011 37 38 Piecewise vs.Connected Standard Construction  x  x p i,j p i,j+1 p i j p i,j p i,j+1 p i j+1 T S  y  y  y  y p i+1,j p i+1,j+1  x  x p i+1,j p i+1,j+1 CS 176 Winter 2011 CS 176 Winter 2011 39 40 Monotonicity Seam from Cut Conditions  x  monotonic p i,j p i,j+1 ∞  connected T S  y  y  x p i+1,j p i+1,j+1 ∞ CS 176 Winter 2011 CS 176 Winter 2011 41 42

Seam from Cut Conditions  monotonic  connected CS 176 Winter 2011 43