Occluded Bilateral EPI Regularization 2nd Workshop on Light Fields - PowerPoint PPT Presentation

Occluded Bilateral EPI Regularization 2nd Workshop on Light Fields for Computer Vision July 26, 2017

Overview Overview standard minimization approach → data term, smoothness term, solver some twists for crisper and smoother results some (over) simplifications OBER - Hendrik Schilling - July 26, 2017 1/15

Key Insight 3D Reconstruction as Minimization Problem inverse problem underconstrained non-convex OBER - Hendrik Schilling - July 26, 2017 2/15

Key Insight Key Insights The Problem is inherently non-differentiable due to occlusions → enforcing differentiabiliy will result in sub-optimal results Secondary Intermediate steps impair occlusion border & fine detail performance → formulate error metrics in input domain OBER - Hendrik Schilling - July 26, 2017 3/15

Method Data T erm OBER - Hendrik Schilling - July 26, 2017 4/15

Method Smoothness T erm candidate d = candidate d = 0.2 0.2 0.1 0.7 0.6 0.5 0.3 0.9 0.8 0.8 0.6 0.4 0.8 1.0 1.0 0.7 0.2 0.7 0.5 0.3 0.5 0.8 0.5 0.2 0.2 0.8 0.8 0.3 modified bilateral filter evaluate both color and disparity hard thresholds → crisp occlusion boundaries OBER - Hendrik Schilling - July 26, 2017 5/15

Method Solver Algorithm 1 Randomized Solver 1: for 20 iterations do for Every disp map pixel do 2: Calc error for current depth 3: Calc error for depth candidates 4: Keep best result 5: end for 6: 7: end for OBER - Hendrik Schilling - July 26, 2017 6/15

Method Depth Candidates random change random guess random (large range) neighbour direct neighbour OBER - Hendrik Schilling - July 26, 2017 7/15

Method Propagation pixel processing order neighbour candidates even iteration odd OBER - Hendrik Schilling - July 26, 2017 8/15

Results ground center truth view disparity disparity BadPix(0.07) mesh of OBER result (as viewed from above) OBER-cross+ANP SPO-MO 2 nd best BadPix(0.07) OBER - Hendrik Schilling - July 26, 2017 9/15

Results OBER BadPix(0.01) OBER-cross+ANP OFSY_330/DNR MAE BadPix(0.03) PS_RF Planes RM3DE SPO-MO 74.03 24.20 23.56 MAE 55.52 Contin. BadPix(0.07) 18.15 17.67 Surfaces 40.47 37.01 12.04 12.10 11.78 30.35 9.03 20.23 18.51 6.02 10.12 3.01 19.19 14.39 9.59 4.80 0.00 1.76 3.52 5.27 7.03 Fine MSE 11.17 0.27 Thinning 22.35 0.42 0.53 33.52 0.80 16.86 0.94 44.70 0.85 1.07 25.28 1.41 1.27 Fine 33.71 1.88 Q25 Fattening 1.69 Bumpiness Discontinuities Contin. Surfaces Bumpiness Planes OBER - Hendrik Schilling - July 26, 2017 10/15

Results OBER - Hendrik Schilling - July 26, 2017 11/15

Results reference result re (OFSY_330/DNR) OBER-cross+ANP (half-size imgs) OBER - Hendrik Schilling - July 26, 2017 12/15

Outlook Outlook A lot of things to improve: scale space, un-occlusions, heuristics → NN, disp map → mesh Main strength: very flexible, per-pixel adaptable solver OBER - Hendrik Schilling - July 26, 2017 13/15

Outlook OBER - Hendrik Schilling - July 26, 2017 14/15

The End The End OBER - Hendrik Schilling - July 26, 2017 15/15