A Color Tree of Shapes with Illustrations on Filtering, Simplification, and Segmentation Edwin Carlinet 1,2 , Thierry G´ eraud 1 1 EPITA Research and Development Laboratory (LRDE) 2 Laboratoire d’Informatique Gaspard-Monge (LIGM) firstname . lastname @lrde.epita.fr May 2015, 27 th
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Context About morphological tree representations: • versatile and efficient → many apps; • (very) easy to compute/manipulate [5, 2, 4], • implicit multiscale analysis, • some of them feature (very) desirable properties: • contrast change invariance, • self-duality. . . Not convinced? Let’s see. . . A Color Tree of Shapes 2/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Grain filters [3](1/2) Method overview 16 16 6 1 8 6 8 1 Tree pruning 3 2 7 7 3 2 λ < 4 4 1 1 4 1 1 1. Compute the size attribute over the tree. 2. Threshold and collapse. A Color Tree of Shapes 3/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Grain filters (2/2): Document layout extraction A Color Tree of Shapes 4/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Interactive segmentation (1/2) Method overview Color ToS Computation Markers on Tree Node Image the tree Classification Classification Markers (User Input) A Color Tree of Shapes 5/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Interactive segmentation (2/2) A Color Tree of Shapes 6/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Document detection in videos (ICDAR SmartDoc’15) Energy 10 10 Shape computation selection 1 4 7 1 4 7 6 3 2 6 3 2 5 9 8 5 9 8 Text Text 1. Valuate an energy adpated to the object to detect. 2. Retrieve the shape with the lowest energy. A Color Tree of Shapes 7/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Document detection in videos (ICDAR SmartDoc’15) Leakages Shadows Specular effects A Color Tree of Shapes 8/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Natural Image Simplification[7] Principle. Mumford-Shah energy optimization constrained to the tree. ∆ Energy Iteration 1 Iteration n + ∞ + ∞ + ∞ 1 · · · -13 1 3 1 3 1 3 3 3 -3 -5 -9 2 4 -9 4 2 2 2 -11 2 4 -11 2 4 2 4 A Color Tree of Shapes 9/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Natural Image Simplification[7] Image simplification: the simplified images have less than 100 nodes (original: ∼ 80k nodes) A Color Tree of Shapes 10/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Document Image Simplification[7] (a) Original (113k nodes). # nodes ÷ 100 # nodes ÷ 1000 (b) Strong simplification (1000 nodes). (c) Drastic simplification (285 nodes). A Color Tree of Shapes 11/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni These applications use a single image representation: The Color Tree of Shapes A Color Tree of Shapes 12/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Outline What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion A Color Tree of Shapes 13/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni What is the Tree of Shapes? (1/2) As the fusion of the min- and max- trees ≥ 0 Ω < 4 A < 2 C F D B A B C ≥ 2 < 1 E D E F < 2 ≥ 2 ≥ 2 The Tree of shapes (ToS) of u , formed by cavity-filled connected components of the min- and max- trees (self-dual representation) A Color Tree of Shapes 14/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni What is the Tree of Shapes? (2/2) As the inclusion tree of the level lines u and its level lines (every 5 levels) • The ToS also encodes the inclusion of the image level lines, • They are the contours of shapes. A Color Tree of Shapes 15/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Properties of the ToS We have: • Invariance by contrast change : T ( g ( u )) = T ( u ) for any increasing function g → it handles low-contrasted objects • Invariance by contrast inversion : T ( ∁ u ) = T ( u ) → it represents light objects over dark background and the contrary, in a symmetric way • A way to get self-dual connected operators: → they do not shift object boundaries A Color Tree of Shapes 16/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni We would like the same “kind” of properties for color images. → Yet, the ToS requires a total order on colors (does one make sense?) A Color Tree of Shapes 17/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Outline What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion A Color Tree of Shapes 18/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Local contrast change Independant Marginal contrast change & inversion. What do these images have in common? They share an exact same representation: the Color Tree of Shapes A Color Tree of Shapes 19/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni General Overview What do we want? • Given M = {S 1 , S 2 , . . . , S n } , where ( S i , ⊆ ) is a tree, we note S = � S i the primary shape set. • We aim at defining a new set of shapes S such that: (P1) Tree structure: every two shapes are either nested or disjoint. (P2) Maximal shape preservation: any shape that does not overlap with any other shape should exist in the final shape set. It implies the Scalar ToS equivalence if u is scalar. (P3) Marginal contrast change/inversion invariance: invariant to any strictly monotonic functions applied independently to u ’s channels. (Q) A “well-formed” tree: # nodes ≃ # pixels and not degenerated. A Color Tree of Shapes 20/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni General Overview Scheme of the method Graph of Shapes Construction Tree Extraction Graph of Shapes ρ computation on G Hole-filled ToS T 1 G + ω reconstruction maxtree of ω 40 66 46 ToS T 2 3 20 86 67 96 86 ToS T 3 1. Get the primary shape set S from the marginal ToS. 2. Compute the Graph of Shapes G = ( S , ⊆ ) A Color Tree of Shapes 21/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni General Overview Scheme of the method Graph of Shapes Construction Tree Extraction Graph of Shapes ρ computation on G Hole-filled ToS T 1 G + ω reconstruction maxtree of ω 91 8 92 ToS T 2 46 54 10 78 17 27 ToS T 3 1. Compute the depth attribute ρ over G , 2. Reconstruct the attribute map ω (in the image space), 3. Compute the cavity-filled maxtree of ω A Color Tree of Shapes 22/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni T 1 T 2 G Ω Ω Ω 0 A D A 1 A B E F B 2 D 2 B D C F C C 3 F 3 E E 4 (a) Input u (b) Graph of Shapes + ρ T ω Ω A 1 2 B ∪ D 2 3 3 4 C F E (c) ω map (d) Cavity-filled Maxtree T ω A Color Tree of Shapes 23/ 31 E. Carlinet, T. G´ eraud
What for? Why is a Color ToS challenging? Proposal for a Color ToS Comparison and Conclusion Bibliography Boni Justification There is no magic! In gray level: The ToS of u is related to the maxtree of the depth map (cf. paper). Furthermore. . . It fulfills the properties. (Proofs in an upcoming paper) You can get effective results. (you’ve already seen that!) A Color Tree of Shapes 24/ 31 E. Carlinet, T. G´ eraud
Recommend
More recommend
