Analysis of Viewer and View Ashish Tawari, Andreas Moegelmose, - PowerPoint PPT Presentation

Attention Estimation by Simultaneous Analysis of Viewer and View Ashish Tawari, Andreas Moegelmose, Sujitha Martin* , Thomas B. Moselund and Mohan M. Trivedi Oct. 14 th , 2014 Qingdao, China 1

2 Introduction [Video] • What is the driver observing?

3 Introduction • 2012 Pedestrian Traffic Safety Facts: – 4,743 pedestrians died in traffic crashes … a 6% increase from 2011 – 88% of pedestrian fatalities occur during normal weather conditions (clear/cloudy) - NHTSA

4 Important Safety Reminders “Look out for pedestrians everywhere, at all times. Very often pedestrians are not walking where they should be .” - NHTSA’s Safety Countermeasures Division We propose an Advanced Driver Assistance Systems (ADAS) with simultaneous analysis of the VIEWER and VIEW … … to determine which pedestrians the driver has seen and has not seen.

5 Related Work • Wearable cameras have commonly been used to capture first person perspective. • Z. Lu and K. Grauman , “Story - Driven Summarization for Egocentric Video,” CVPR 2013. • Face-looking cameras have been used for gaze estimation before. • A. Tawari , K. H. Chen and M. M. Trivedi, “Where is the Driver Looking: Analysis of Head, Eye and Iris for Robust Gaze Zone Estimation,” ITSC 2014. Our work is the first time combining the two modalities for driver attention analysis

6 The Approach: Hardware setup • The view from first person perspective – Google Glass – 1280 x 720 pixel resolution • The viewer from a spatially distributed camera setup – GigE cameras – 960 x 1280 pixel resolution

7 The View: Salient Objects • Pedestrians are detected using HOG/SVM trained on the Inria dataset • Challenges in first person view

8 The View: Salient Objects • To decrease false positive detection – Determine the region of interest – Limit the size of the pedestrians Raw pedestrian detection Filtered pedestrian detection

9 The Viewer • Gaze-surrogate – Coarse estimation of the gaze direction – Requires location of iris center and eye corners

10 The Viewer • Gaze-surrogate – Coarse estimation of the gaze direction – Requires location of iris center and eye corners

11 Attended Object Determination • We combine the estimated gaze with salient zones (pedestrians) from the wearable camera perspective to estimate exactly what the driver is looking at.

12 Experimental Evaluation • The dataset: – Multiple drivers – 40 minutes of data in total – Naturalistic on-road driving with a focus on intersections – Ground truth: • 410 frames of 1413 annotated pedestrians in the first person view • 300 frames of where the driver is looking in the first person view

13 Experimental Evaluation Performance of the attention estimator compared with the center-bias as baseline Estimator Mean gaze Median gaze Attended pedestrian accuracy error (in error (in (%) pixels) pixels) Manually Full system annotated pedestrians Center-bias 148.3 127.0 55.9 37.0 based (baseline) Proposed 54.1 32.2 79.4 46.0

14 Analysis of the View and the Viewer

15 Conclusion • A new approach to analyze driver’s attention state has been shown • We fused the first-person video data with the face video of the same person to infer attention • Evaluated our work on naturalistic driving data • The framework can easily accommodate any object of interest or even a low-level saliency model to estimate the focus of attention.

16 Thank you! • Please contact us for any questions and we will respond to you within 24 hours. Ashish Tawari: ashish.tawari@gmail.com Andreas Moegelmose: andreas@moegelmose.com Sujitha Martin: sujitha.martin@gmail.com Our website: http://cvrr.ucsd.edu