Program 13:00 Welcome and introduction 13:20 Research progress on - PDF document

26/05/2016 Program • 13:00 Welcome and introduction • 13:20 Research progress on RGB+LWIR pedestrian IWT-Tetra project detection • 14:20 Hardware update and geometrical calibration issues and solutions • 14:45 Rule-based reasoning with real-life application demo • 15:30 Discussion and planning of in-the-field tests • 16:00 Conclusions and future work User group meeting 13 May 2016 Updated industrial users group Project abstract • Camera-based safety and security systems • Real-time reaction on incidents? Manual monitoring o Automatic processing and incident o detection • Needed components: Very reliable detection of persons 1. in camera images Reasoning system that can decide 2. if an alarm must be generated SPINOFF Enabling factors Project idea • State-of-the-art person detection algorithms show • Making people detection reliable, astonishing results also in difficult circumstances (fog, smoke, rain, dust, motion blur, …): o Accuracy great on standard benchmark data sets o Combine RGB and LWIR o EAVISE succeeded in running these in real-time on camera limited hardware o Adapt state-of-the-art person o Both open source and commercial-grade detection algorithms for this implementations available sensor combination • Price of LWIR-cameras descends steeply, with increasing • Use probabilistic KR for analysis resolution of situation: must an alarm be • Knowledge-representation based probabilistic reasoning generated? offers potential to analyse each situation 1

26/05/2016 People@VIPER Project goals Developing a sensor combination and software for ultra- o reliably detecting people in real-time Composing a real-life reference image database for Prof. Joost Kristof Van Dr. Floris De (Dr.) Kristof Prof. Toon Andy New o Vennekens Engeland Smedt Van Beeck Goedemé Warrens employee evaluating person detection techniques in difficult circumstances Studying techniques for automatic analysis of the observed o situation and classification as normal or abnormal Studying the certification procedure for camera-based o safety and security systems The demonstration and dissemination of the project o results via 5 real-life user cases Supporting industrial companies to adopt the developed o techniques in their products and services Work packages and progress Planning WP1: Hardware WP2: Person detection WP3: Alarm system 1.A Study on sensors 2.A Study on algorithms 3.A Study on AI Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 3.B Learning of ranking WP1.A: Study on sensors 1.B Hardware imple- 2.B Person detection WP1.B: Hardware realisation and calibration mentation & calibration SW implementation 3.C Online learning WP1.C: Benchmark database MP8 WP2.A: Study on algoriths for person detection 1.C Benchmark 2.C Evaluation on 3.D Evaluation WP2.B: Implementation algorithm person detection MP6 database Benchmark database WP2.C: Evaluation on benchmark database MP7 WP3.A: Study on learning alarm system WP3.B: Learning of ranking WP3.C: Online learning WP4: Evaluation and dissemination WP3.D: Evaluation on benchmark database MP7 MP1 MP2 MP3 MP4 MP5 WP4.A: User cases 4.B Evaluation and documentation MP7 WP4.B: Evaluation and documentation 4.A MP9 WP4.C: Certification & legal aspects User 4.C Study on certification and legal aspects MP9 WP4.D: Broad dissemination and networking Cases 4.D Broad dissimination Program Program • 13:00 Welcome and introduction • 13:00 Welcome and introduction • 13:20 Research progress on RGB+LWIR pedestrian • 13:20 Research progress on RGB+LWIR pedestrian detection detection • 14:20 Hardware update and geometrical calibration issues • 14:20 Hardware update and geometrical calibration issues and solutions and solutions • 14:45 Rule-based reasoning with real-life application demo • 14:45 Rule-based reasoning with real-life application demo • 15:30 Discussion and planning of in-the-field tests • 15:30 Discussion and planning of in-the-field tests • 16:00 Conclusions and future work • 16:00 Conclusions and future work 2

26/05/2016 Overview • How does pedestrian detection work? • KAIST dataset • Performed experiments: comparitive study • ACF – ACF+ Research progress on RGB- • Amount of training data LWIR pedestrian detection • Trained model size • Resolution of LWIR-image (simulate lower resolution sensor) • Training set - Testing set Pedestrian detection approach • Create a model for pedestrians o Examples of positives (pedestrians) o Examples of negatives (non-pedestrians) o Convert to feature representation How does pedestrian • Good distinction between pedestrians and background detection work? • Robust for scene changes (e.g. illumination) o Train a model • Machine Learning: Adaboost, Support Vector Machines, Neural networks, … • Distinction between “Pedestrian” and “Background” • Intra-class variation: pedestrians can have many appearances 16 Pedestrian detection approach Pedestrian detection approach Search the model in the image features (Sliding Window): • Non-Maximum-Suppression At every location…and multiple scales  sliding window o Sliding window results in clusters of detections … around pedestrians o NMS reduces this to only 87.81 the highest scoring detection 68.71 68.46 26.89 of each cluster 8.405 • Calculate features at multiple scales  Feature pyramid • Similarity between the model and the features forms the certainty of a pedestrian at that location • A threshold defines the boundary between “background” and “detection” 17 3

26/05/2016 Influence of the threshold value Measure accuracy Miss rate vs. False Positives per Image Precision vs. Recall Low threshold • More pedestrians Miss Rate: The share of found Recall: share of pedestrians • More mistakes pedestrians that is not found found Precision: share of detections FPPI: Average number of that is a pedestrian false detections (non- High threshold pedestrian) per image • Less pedestrians Best point: top right found • Less mistakes Best point: bottom left 19 20 Used detector • Channel based detectors o Use both gradient and color information o Feature values are calculated as the sum of pixel values in rectangles o AdaBoost Machine Learning • Integral Channel Features [1] KAIST dataset o 30 000 random rectangles inside model window o Each stage (2000) is a decision tree of features • Aggregate Channel Features [2] o Approximation of the features at most scales o All possible squares of a specific size inside the model window 21 [1] “Integral Channel Features”, Dollàr, Tum Perona and Belongie, BMCV 2009 [2] “Fast Feature Pyramids for Object Detection”, Dollàr, Appel, Belongie and Perona, PAMI 2014 IR Dataset: KAIST Previous work o Color and LWIR • Results from literature o “Multispectral pedestrian detection: Benchmark dataset and baseline” CVPR 2015 o Add channels calculated on LWIR o for both day and night conditions • Pixel information • Gradient magnitude • Gradient orientations o Night experiments • Reasonable • ≥ 50px • ≥ 65% visible • Each 30 th image 24 4

26/05/2016 RGB to LWIR extension • Easy extension of existing RGB detectors to LWIR o Avoids retraining o Based on [3] Performed experiments A. Warrens [3] “Far infra-red based pedestrian detection for driver-assistance systems based on candidate filters, Gradient-based filters and multi-frame approval matching”, Wang and Liu, December 2015 RGB to LWIR extension RGB to LWIR extension • Start from an existing RGB-based detector o ACF • Classify as pedestrian if a peak in intensity takes place on the LWIR-image 73% reduction in false positives RGB to LWIR extension • This approach can not improve the recall, only the precision o Very limited range of ACF-detections used o A larger range will increase processing time! Performed • “Resolution is not sufficient to distinguish pedestrians from road” (A. Warrens) experiments • We need a stronger approach than “Thresholding”! Dr. F. De Smedt 5