Monocular Depth Estimation Using Atrous Convolutions Group 5 - Faraz Saeedan Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Introduction Experiments Results Discussion Conclusion 1 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Monocular Depth Estimation ◮ Monodepth: Estimate depth from a single image at test-time (instead of stereo pair) ◮ Existing approaches treat this as a supervised regression problem ◮ Godard et al., CVPR 2017: Depth estimation as a stereo reconstruction problem Figure: Godard et al. 2017 2 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Atrous Convolutions Figure: Chen et al., 2017 3 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Project Objective ◮ Get Monodepth baseline to run ◮ Design and implement encoder-decoder network using atrous convolutions ◮ Consider memory and runtime in architectural decisions ◮ Quantitative comparison of proposed architectures to baseline 4 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Our Architecture with ASPP Atrous Spatial Pyramid Pooling 1x1 Conv Input 3x3 Conv rate=6 Conv1 Block 1 Block 2 Block 3 Block 4 + Pool1 3x3 Conv rate=12 3x3 Conv Output 4 8 16 16 rate=18 Stride Image Pooling Concat + Prediction 1x1 Conv 5 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Remember last time? Sky Artifacts 6 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt How we solved it: Reimplementation 7 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt How we solved it: Reimplementation Differences of their implementation compared to standard ResNet: ◮ No batch-normalization ◮ Nearest-neighbor instead of bilinear interpolation ◮ Order of convolution strides in ResNet switched + Fixed bug in author’s implementation of loss function 7 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt New Disparity Maps 8 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt General Experimental Setup ◮ General setup ◮ Batch Size 16 ◮ Learning Rate 2e-4 ◮ 50 epochs ◮ KITTI for training ◮ Rescaled to 256 x 512 px ◮ KITTI Stereo 2015 for testing ◮ Sparse ground truth 9 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Output Strides Issues Output stride 64 with Atrous rate > 1 F e a t u r e s a f t e r E n c o d e r Convolution Kernel 10 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Output Strides 11 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Output Strides Stride Abs. Rel Runtime Memory Params (M) 64 0.1120 NaN 6115MiB 58.4 32 0.1048 NaN 8871MiB 58.4 16 0.1041 NaN 8883MiB 58.4 8 NaN NaN 10609MiB 58.4 11 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt ASPP vs. no ASPP (with Skip Connections) 12 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt ASPP vs. no ASPP (without Skip Connections) 13 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
0.110 stride 32 stride 16 0.105 Abs.Rel. 0.100 0.095 Baseline 1,1,1,1 1,2,3,4 1,3,5,7 ASPP rates Technische Universität Darmstadt Experiments: Atrous Rates Atrous Spatial Pyramid Pooling 1x1 Conv 3x3 Conv rate=6 3x3 Conv rate=12 3x3 Conv rate=18 Image Pooling 14 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates 0.110 stride 32 Atrous Spatial stride 16 Pyramid Pooling 0.105 1x1 Conv Abs.Rel. 3x3 Conv 0.100 rate=6 3x3 Conv rate=12 0.095 3x3 Conv rate=18 Image Pooling Baseline 1,1,1,1 1,2,3,4 1,3,5,7 ASPP rates 14 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
70 0.110 60 Million Parameters 50 0.105 40 Abs.Rel. 30 0.100 20 10 0.095 0 Baseline 1 2 3 4 5 ASPP modules Technische Universität Darmstadt Experiments: Different number of modules Atrous Spatial Pyramid Pooling 1x1 Conv 3x3 Conv rate=1 3x3 Conv rate=1 Image Pooling 15 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Different number of modules 70 0.110 60 Atrous Spatial Million Parameters 50 Pyramid Pooling 0.105 40 1x1 Conv Abs.Rel. 30 3x3 Conv 0.100 rate=1 20 10 3x3 Conv 0.095 rate=1 0 Image Pooling Baseline 1 2 3 4 5 ASPP modules 15 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Convolutions in Encoder Atrous Spatial Pyramid Pooling 1x1 Conv Input 3x3 Conv rate=6 Conv1 Block 1 Block 2 Block 3 Block 4 + Pool1 3x3 Conv rate=12 3x3 Conv rate=18 Image Pooling No improvement with atrous convolutions in ResNet blocks 16 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: ASPP in ResNet Atrous Spatial Pyramid Pooling Input 1x1 Conv 3x3 Conv rate=6 ..... Conv1 Block 1 Block 2 + Pool1 3x3 Conv rate=12 3x3 Conv rate=18 Image Pooling No improvement with ASPP module between ResNet blocks 17 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Final Results Pretrained on CityScapes & finetuned on KITTI Output-stride ASPP Abs.Rel. Sq.Rel. RMSE Log RMSE a1 a2 a3 Params (M) ∆ Abs. Rel. (%) 64 - Godard - 0.0970 0.8960 5.093 0.176 0.962 0.962 0.986 58.4 – 16 - Ours 1-1-1-1 58.4 0.0927 0.8132 4.865 0.168 0.888 0.967 0.987 4.43 32 - Ours 1 0.0941 0.8196 4.910 0.173 0.882 0.963 0.986 44.1 2.99 32 - Ours 1-2-3-4 0.0936 0.8281 4.941 0.172 0.884 0.963 0.987 58.4 3.50 18 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Discussion ◮ Loss hyperparameters ◮ Learning rate ◮ Output stride 8 ◮ Decoder architecture ◮ Robustness of predictions 19 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Recap ◮ Reproduced baseline ◮ Experiments ◮ Output stride ◮ Skip connections ◮ ASPP rates ◮ Number of ASPP modules 20 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Conclusion ◮ Atrous convolutions . . . ◮ do not improve monocular depth estimation ◮ need a decreased output stride, which harms runtime and memory ◮ Channel reduction after encoder . . . ◮ decreases parameter count and improves runtime ◮ without losing predictive power 21 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt References ◮ C. Godard, O. Aodha and G. Brostow: Unsupervised Monocular Depth Estimation with Left- Right Consistency, CVPR 2017. ◮ L. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam: Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. arXiv 2018. ◮ M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele: The cityscapes dataset for semantic urban scene understanding. CVPR. 2016. ◮ M. Menze and A. Geiger: Object Scene Flow for Autonomous Vehicles. CVPR. 2015. 22 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (Visual Comparison) 23 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (Visual Comparison) 24 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (Visual Comparison) 25 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (Visual Comparison) 26 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (Squared relative error) 1.10 stride 32 stride 16 1.05 Sq.Rel. 1.00 0.95 0.90 Baseline 1,1,1,1 1,2,3,4 1,3,5,7 ASPP rates 27 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (RMSE) stride 32 5.6 stride 16 5.4 RMSE 5.2 5.0 4.8 Baseline 1,1,1,1 1,2,3,4 1,3,5,7 ASPP rates 28 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Technische Universität Darmstadt Experiments: Atrous Rates (RMSE of log) log.pdf log.pdf 0.20 stride 32 stride 16 0.19 RMSE log 0.18 0.17 Baseline 1,1,1,1 1,2,3,4 1,3,5,7 ASPP rates 29 Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019
Recommend
More recommend
![Tsinghua University Monocular Depth-Pose Prediction [R, t] Depth and Pose RGB PoseNet](https://c.sambuz.com/691712/tsinghua-university-monocular-depth-pose-prediction-s.jpg)
