Branch&Rank: Efficient, Non-Linear Object Detection Alain D. - PowerPoint PPT Presentation

− Image Classification Task "BBox" of Hypotheses Set Λ Ranking Condition (Multiple) Inter mediate Tasks − + f( ) Λ < f( ) Λ some elements of Λ − Object Detection Task T4 T3 T2 + T1 "BBox" of Hypotheses Set Λ Branch&Rank: Efficient, Non-Linear Object Detection Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Computer Vision Laboratory, ETH Zurich, Switzerland 30 August, 2011 Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Detection means to localise and categorise objects input annotated output car car Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Appearance variations make it difficult intra-class variations sophisticated different views/poses expensive illumination changes models occlusions, etc . Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Localise objects among thousands of hypotheses Search space size > 10 ′ 000 locations > 1 ′ 000 classes avoid exhaustive � �� � enumeration ⇓⇓⇓⇓⇓⇓ Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Efficient detection by ranking sub-images Runtime = (classifier cost) × (#calls) ◮ reduce cost: cascades [Viola et al . 04, Vedaldi et al . 09] exhaustive search → not scalable ◮ reduce calls: branch&bound [Lampert et al . 08, Lehmann et al . 09] bounds not tight enough → not effective Image Classification Task Ranking: ”learn the bound” (Multiple) Inter mediate Tasks ◮ branch, but not bound Object Detection Task ◮ often < 100 classifier calls → non-linear SVMs T4 ◮ classification for detection T3 T2 T1 Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Outline 1 Detection: best-first search 2 Training: ranking hypothesis sets 3 Multi-tasks aspects 4 Results and conclusion Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning Sets of hypothesis exploit correlations split promising sets

Efficiency by means of adaptive partitioning ? ? ? ? ? Sets of hypothesis exploit correlations split promising sets Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Efficiency by means of adaptive partitioning ? ? ? ? ? ? ? ? ? ? Ranking function f prioritises Sets of hypothesis     exploit correlations f  > f  � �  split promising sets �� � �� � contains obj . no object correspond to subimages supersedes upper bounds Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Training with sets for increased efficiency − "BBox" of Hypotheses Set Λ Ranking Condition − + f( ) Λ f( ) Λ < some elements of Λ − + "BBox" of Hypotheses Set Λ Structured SVM ranking [Tsochantaridis et al . 04, Blaschko et al . 08] � w ,ξ i ≥ 0 � w � 2 + C ◮ bag-of-words descriptor φ (Λ) min ξ i ◮ kernelize with RBF- χ 2 kernel i f (Λ + i ) − f (Λ − ) ≥ ∆(Λ − ) − ξ i ◮ Λ + : generate with oracle ◮ Λ − : delayed constraint generation with f (Λ) = � w , φ (Λ) � Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

From image classification to object categorisation Large sets Image Classification Task ◮ object somewhere (Multiple) Inter mediate Tasks ◮ image classification Object Detection Task Small sets ◮ object centred T4 T3 T2 T1 ◮ object categorisation Task-adapted ranking f (Λ) = � w q (Λ) , φ (Λ) � ◮ task mapping q (Λ) ◮ exploit context ◮ improved AP by ≈ 10% ◮ leverage set information Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Branch&rank detects in often < 50 iterations 1.0 prec 0.5 recall 0.0 200 'Horse' detections heat map true positive iterations till detection 150 avg. #iterations more efficient prec=recal 100 50 0 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 detection score more confident detections Dataset: PASCAL VOC 2007 (Horses) [Everingham et al ., 2007] non-linear RBF- χ 2 SVMs costly classifier no cascade approximations feasible Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

More results Cow Cat Horse 1.0 1.0 1.0 prec prec prec 0.5 0.5 0.5 recall recall recall 0.0 0.0 0.0 200 200 200 'Horse' detections heat map 'Cow' detections heat map 'Cat' detections heat map true positive true positive true positive iterations till detection iterations till detection iterations till detection 150 150 150 avg. #iterations avg. #iterations avg. #iterations prec=recal prec=recal prec=recal 100 100 100 50 50 50 0 0 0 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 detection score detection score detection score branch&rank part-based detector best in challenge [Lehmann et al . 2011] [Felzenszwalb et al . 2008] [Everingham et al . 2007] Horse 36.8% 30.1% 33.5% better Cow 10.8% 16.5% 14.0% worse Cat 17.6% 11.0% 24.0% in-between Future work combine multiple features use task-adapted features Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection

Conclusion Branch&rank is efficient − "BBox" of Hypotheses Set Λ Ranking Condition less than 100 classifier calls − + f( ) Λ < f( ) Λ some elements of Λ − non-linear SVMs feasible Process hypothesis sets + "BBox" of Hypotheses Set Λ during detection and training “learn the bound” Image Classification Task (Multiple) Inter mediate Tasks Multiple task Object Detection Task combine classification and detection T4 T3 T2 T1 Alain D. Lehmann, Peter V. Gehler, and Luc Van Gool Branch&Rank: Efficient, Non-Linear Object Detection