Depth Prediction and RGBD Images for Recognition Yihui He, Metehan - PowerPoint PPT Presentation

Depth Prediction and RGBD Images for Recognition Yihui He, Metehan Ozten yihuihe@foxmail.com, m ozten@umail.ucsb.edu May 25, 2016 Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Related work and motivation Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

overview our project: depth estimation & Classification on RGBD images implement previous work Go further (2) Build a RGBD CIFAR10 based on indoor depth knowledge (3) Compare RGBD and RGB label = f ( RGBD ) label = f ( RGB ) Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

first part: implement previous work infer depth from RGB image Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Infer depth from RGB image: Loss defination At training time, we combine two objective function 1 1 regress to groud truth depth image(Kinect, PrimeSense) y p ) 2 , p stands for pixel. Σ p ( y p − ˆ k =1 β k S ( k ) 2 Similarity between superpixels. R pq = � K pq β is trainable weight. S is similarity function. 1 Fayao Liu, Chunhua Shen, and Guosheng Lin. “Deep convolutional neural fields for depth estimation from a single image”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2015, pp. 5162–5170. Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Architecture: Deep convolutional Neural Field Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Infer depth from RGB image: Supervised part using traditional CNN. Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Compare performance with original paper Error Accuracy Method (lower is better) (higher is better) δ < 1 . 25 2 δ < 1 . 25 3 rel log10 rms δ < 1 . 25 Our implementation 0.252 0.103 0.860 0.544 0.861 0.943 Original paper 0.230 0.883 0.095 0.824 0.614 0.971 Table: Sanity check ( Bold is better) Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

second part: go further Classification on RGBD images Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

build RGBD CIFAR dataset 32 x 32 x 3 400 x 400 x 1 400 x 400 x 3 32 x 32 x 4 Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

architecture 32 x 32 x 4 Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

R vs G vs B vs D: training time 0.8 0.7 0.6 training accuracy 0.5 0.4 0.3 0.2 0.1 0 20 40 60 80 100 120 140 160 180 Epoch Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

R vs G vs B vs D: testing time 0.50 0.45 0.40 validation accuracy 0.35 0.30 0.25 0.20 0 20 40 60 80 100 120 140 160 180 Epoch Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

RGBD vs RGB: training time 1.0 0.9 0.8 0.7 training accuracy 0.6 0.5 0.4 0.3 0.2 0.1 0 50 100 150 200 250 Epoch Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

RGBD vs RGB: testing time 0.6 0.5 val accuracy 0.4 0.3 0.2 0 50 100 150 200 250 Epoch Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

architecture Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

results Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

our contribution 1 reproduce previous work on depth estimation 2 create the first RGBD CIFAR10 dataset 3 define a new metric for depth prediction problem 4 prove that depth channel has a better feature representation 5 show that training on RGBD images can somehow improve accuracy Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition

questions? 2 2 code, references, report and slides can be access here: https://github.com/yihui-he/Depth-estimation-with-neural-network Yihui He, Metehan Ozten Depth Prediction and RGBD Images for Recognition