Towards a fully automated, high-performance pipeline for stereo reconstruction from earth observing satellites Dave Kelbe, PhD 1 Jeff Nichols, PhD 2 David Page, PhD 1 Devin White, PhD 1 Andrew Hardin 3 Lawrence MacIntyre 2 1 Geographic Data Sciences Team Geographic Information Science and Technology Group 2 Cyber and Information Security Group 3 Scalable and High Performance Geocomputation Team Geographic Information Science and Technology Group Oak Ridge National Laboratory ORNL is managed by UT-Battelle for the US Department of Energy
GTC 2017 Kelbe
Scientific Foundation: Random dot stereograms Normal Viewing Conditions Focus Convergence GTC 2017 Kelbe
Scientific Foundation: Random dot stereograms Cross-Eyed Viewing Conditions Focus Convergence GTC 2017 Kelbe
Stereo reconstruction from earth observing satellites • Scientific Foundation: 3D illusions • Challenge and necessity of obtaining 3D surface data • Historical and technological motivation • Scientific foundation – Development Concepts – Hardware – Pipeline • Technological solution: kernels • Current system performance GTC 2017 Kelbe
Challenge: High-resolution, global surface data are scarce Shuttle Radar Global Coverage ORNL Fortuitous Topography Mission Stereo Pipeline (SRTM) WorldDEM ASTER GDEM (Stereo) Low Resolution High Resolution Airborne laser scanning UAV Structure from Motion Limited coverage SRTM GTC 2017 Kelbe
High-resolution global surface data are foundational • Photogrammetry is a 3-dimensional problem s l High-resolution surface Low-resolution surface Z True ortho Available terrain estimate Terrain ortho X GTC 2017 Kelbe
Isn’t this a solved problem? Wild A7 Stereoplotter (1952-1972) ”Easy Button” GPU Computing Software (2010’s - ) GTC 2017 Kelbe
Isn’t this a solved problem? GTC 2017 Kelbe
Scientific Foundation: Core development concepts • Use open source , in-house, or government-furnished s/w libraries, • Leverage well-established photogrammetric and computer vision techniques to reduce risk , • Expose components as services , • Use high performance computing (HPC) architectures, • Strictly adhere to and take full advantage of metadata standards to enable interoperability , and • Keep the system as flexible as possible through extensive use of modular frameworks. GTC 2017 Kelbe
Technological Solution: System Hardware CPU/GPU hybrid architecture • Dell C4130 HPC Node • Each node has: • Dual Intel Xeon processors: • 24 physical cores • 48 hyperthreaded • 256GB of RAM • 4 Tesla K80s • Used as 8 GPUs with 12GB each GTC 2017 Kelbe
Scientific Foundation: Pipeline Left image Right image Registration ? Epipolar Rectification Semiglobal Matching Matching cost computation Cost Aggregation Disparity computation Disparity refinement Stereo intersection Point Cloud GPU-accelerated GTC 2017 Kelbe
CPU: Matching Cost Computation: Naïve for(pixel){ for(disparity){ costs(i,j)[d] = abs(left(i,j) – right(i-d,j)) } } pixel j j j costs left right i-d i i i abs( - ) = GTC 2017 Kelbe
GPU: Matching Cost Computation: Naïve // Divide image into 2D grid pattern THREADS_PER_BLOCK.x = sqrt(1024); threadIdx.x THREADS_PER_BLOCK.y = sqrt(1024); blockIdx.x 0 1 2 3 0 1 2 3 … 31 N_BLOCKS.x = sizex + THREADS_PER_BLOCK.X − 1 ; THREADS_PER_BLOCK.X 0 1 2 blockIdx.y N_BLOCKS.y = sizey + THREADS_PER_BLOCK.Y − 1 pixel THREADS_PER_BLOCK. ! j // Index into i and j i = blockIdx.x * blockDim.x + threadIdx.x j = blockIdx.y * blockDim.y + threadIdx.y i left GTC 2017 Kelbe
CPU: Matching Cost Computation: Census with Hamming for(pixel){ for(disparity){ for(windowSize){ } } } pixel j j j costs left right i-d i i i sum( ) xor( , ) = GTC 2017 Kelbe
CPU: Cost Aggregation for(pixel){ for(disparity){ for(disparity){ }}} + min( ) j j current local prior costs accumulated i i GTC 2017 Kelbe
GPU: Cost Aggregation for(pixel){ for(disparity){ for(disparity){ }}} disparityRange … j j current local prior … costs accumulated i i // Divide image into 2D grid pattern THREADS_PER_BLOCK.x = 1024/dispartyRange; THREADS_PER_BLOCK.y = disparityRange; GTC 2017 Kelbe
Performance Comparison Pipeline Timings 16 14 Computation Time (Seconds) 12 Startup 10 Cost Calculation 8 Cost Aggregation Disparity Computation 6 GPU Overhead 4 * GPU accelerated 2 0 48 CPU 8 GPU GTC 2017 Kelbe
Our solution: Fortuitous stereo 3D reconstruction GTC 2017 Kelbe
GTC 2017 Kelbe
Conclusions • Motivation: Global, high resolution surface data are critical for photogrammetric preprocessing • Scientific Foundation: Computationally intensive, but class-leading Semi-Global Matching algorithm (SGM) • Technological Solution : – GPU acceleration makes processing of rapid streams of satellite imagery viable GTC 2017 Kelbe
Thank you Dave Kelbe, PhD Oak Ridge National Laboratory kelbedj@ornl.gov GTC 2017 Kelbe
Recommend
More recommend
