Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Open tools and methods for large scale segmentation of Very High Resolution satellite images Julien Michel (CNES) , Manuel Grizonnet (CNES), Arnaud Jaen (CS-SI), Sebastien Harasse (CS-SI), Luc Hermitte (CS-SI), Jonathan Guinet (CS-SI), Julien Malik (CS-SI), Mickaël Savinaud (CS-SI) November 19, 2012
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Outline Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Outline Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Introduction OBIA and Spatial Reasonning vs. segmentation ◮ In VHR satellite imagery, OBIA 1 and Spatial Reasoning widely studied and used ◮ But . . . segmentation pre-processing step: ◮ Lacks of quality and stability across large scenes ◮ Is hardly scalable to those scenes anyway (at least in free software) This talk ◮ Tries to address the unscalability ◮ Presents a new framework available as a part of the OTB 2 1 Object-Based Image Analysis 2 Orfeo ToolBox, Satellite Image open source software developed by CNES
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Why is it difficult ? On the image processing side (what this talk is not about) ◮ VHR optical imagery is not a natural imagery (blur, noise, pan-sharpening . . . ) ◮ Scene is VERY large: wide swath with varying conditions (shadows, specular reflections . . . ) ◮ Objects (urban ones) are often small wrt spatial resolution On the computer science side (what this talk is about) ◮ Data is VERY large: the data does not fit into memory ◮ Segmentation does not cope well with piecewise computation ◮ Results are hard to store and manipulate efficiently
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Outline Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Binary and labeled rasters Principle ◮ Image of integer labels ◮ Unique labels for each segment ◮ Very close to the image processing perspective Why it can not meet scalability ◮ Billions of segments: how many bits for unique labels ? ◮ Getting segment pixels or boundaries: intensive image parsing !
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Label maps Principle ◮ Run Length Encoding: from the data compression field ◮ A run (raw of pixels): (start index, length) ◮ A segment : collection of runs ◮ A segmentation : map (dictionary) of (label, segment) Why it can not meet scalability ◮ Almost images . . . But how to store them ? ◮ Some efficient operations, other are nightmares ◮ Poorer segmentation means poorer memory efficiency
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Vectors Principle ◮ Store segments as polygons Why it can meet scalability ◮ Plenty of file formats from the GIS world ◮ Even Databases (PostGIS . . . ) ◮ Size grows linearly with the amount of segments ◮ No explicit unique labels required ◮ Direct compatibility with GIS world
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Outline Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion The segmentation algorithms ◮ For now, available algorithms are: ◮ Simple Connected Components with custom criterion ◮ Watershed ◮ Mean-Shift 3 ◮ Any segmentation algorithm can be fitted in the framework, if: ◮ It outputs a (partial) lableled raster output ◮ It is written (or encapsulated) as an OTB filter ◮ . . . that’s all: no need for parallel computation compatibility for instance We do not want strong constraints on the algorithms! 3 D. Comaniciu and P . Meer, “Mean shift: A robust approach toward feature space analysis,” IEEE Transactions on pattern analysis and machine intelligence, pp. 603–619, 2002.
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion (New) components in Orfeo ToolBox ◮ OGR encapsulation for large vector file handling ◮ Filters for exact conversion between labeled raster and vector ◮ Based on GDAL and OGR capabilities ◮ Most segmentation algorithms procude labeled raster ◮ New filter for large scale segmentation ◮ Tile based segmentation and vectorization ◮ Geometries written to disk for each tile ◮ Genericity on segmentation algorithms (templates) ◮ An application to handle both ◮ Standard segmentation with labeled raster output ◮ Large-scale segmentation with vector output
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Workflow of the Segmentation application in OTB
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Examples: raster mode (Connected components) distance<30 or (intensity_p1>400 and intensity_p2 > 400 and distance<50) or(intensity_p1 >1000 and intensity_p2>1000 or (p1b4 <150 and p2b4<150))
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Examples: raster mode (Watershed)
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Examples: raster mode (Mean-Shift)
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Examples: raster mode (Morphological profiles 4 , WIP) 4 Martino Pesaresi and Jon Alti Benediktsson, Member, IEEE: A new approach for the morphological segmentation of high resolution satellite imagery. IEEE Transactions on geoscience and remote sensing, vol. 39, NO. 2, February 2001, p. 309-320.
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Examples: vector mode (Mean-Shift) ◮ Larger image : 4096 by 4096 pixels, 4 bands ◮ Requires more than 2 Go of RAM for processing in raster mode ◮ Requires around 500 Mo for processing in vector mode ◮ In less than 3 minutes ◮ Produces 52799 polygons ◮ An image of 16384 by 16384 pixels would require 48 minutes of processing ◮ But would still require around 500 Mo of RAM
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Examples: vector mode (Mean-Shift, QGis export)
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Outline Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion This is great but . . . ◮ Did you notice the tiling effects in the results ? ◮ Do I really need to segment everything in the scene ? ◮ How can I control the size of the output vector file ?
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Stitching ◮ Simple rule to stitch together polygons split by tiling ◮ We look for neighboring polygons on both side of the split ◮ We merge polygons based on overlap surface ◮ We process horizontal splits first, and then vertical ones
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Stitching (example)
Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Masking and small objects rejection We usually do not want small objects ◮ They are almost pure noise ◮ They are hard to classify in OBIA ◮ They grow the size of the output file We might also want to discard some parts of the image ◮ Based on land-cover or image features ◮ An input mask can be provided ◮ Masked pixels will be discarded during vectorization step
Recommend
More recommend
