Balancing Privacy and Safety: Protecting Driver Identity in - PowerPoint PPT Presentation

Balancing Privacy and Safety: Protecting Driver Identity in Naturalistic Driving Video Data Sujitha Martin, Ashish Tawari and Mohan M. Trivedi Laboratory of Intelligent and Safe Automobiles September 19 th , 2014 1

2 Question: Can you tell what the driver is doing?

3 Question: Can you identify the driver?

4 Outline • Motivation • Related Works • De-Identification Filter – Requirements and Challenges – Design: identity protection and gaze preservation • Case Study – Experiment Design – Performance: face recognition and gaze zone estimation • Concluding Remarks

5 Motivation • Why “Naturalistic” Driving Data?

6 Motivation • The 100-Car Naturalistic Driving Study Data (NDS) has collected data of – 2 million miles, 42 000 hours – 82 crashes, 761 near crashes, 8295 critical incidents • The SHRP-2 NDS has been collecting data of – 3000 subjects, 1 million hours, 5 million trips, 33 million miles, 4 billion GPS points.

7 Motivation • Public access to raw data of NDS, including data from 100-Car and SHRP-2, is NOT available because of personal identifiable information • Need ways to protect the privacy of drivers • But also preserve sufficient details to infer driver behavior

8 Related Works Privacy protection, by de-identifying people, is typically needed for one of two reasons…

9 Related Works Privacy protection, by de-identifying people, is typically needed for one of two reasons… 1) Person is not intended to be in the image, or Flores, A., and Belongie, S., “Removing pedestrians from google street view images,” In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, IEEE (2010).

10 Related Works Privacy protection, by de-identifying people, is typically needed for one of two reasons… 1) Person is not intended to be in the image, or 2) Presence and action of person is intended but not their identity Agrawal, P., and Narayanan, P., “Person de - identification in videos,” Circuits and Systems for Video Technology, IEEE Transactions on 21, 3 (2011), 299 – 31.

11 Related Works Research Approach Preserve Context Sample Evaluation Study Cheng & Trivedi, Voxel reconstruction of Action Looking inside N/A 2005 the scene the vehicle Schiff, Meingast Solid ellipsoidal overlays Scene and Surveillance Hand labeled: & Mulligan, 2009 on faces action false positives and false negatives Nodari et al., Replacing pedestrians Scene and Google street Algorithmic 2012 with similar pedestrians action view detection, from controlled dataset segmentation, matching and replacing results Our work Isolated segmentation of Gaze Looking inside User study of face eyes with and without the vehicle recognition and face mask gaze estimation

13 De-Identification Filter

14 De-Identification Filter • How to protect identity and preserve gaze? – Isolated segmentation of facial regions

15 De-Identification Filter • How to protect identity and preserve gaze? – Isolated segmentation of facial regions – Background distortion Sujitha Martin, Ashish Tawari and Mohan M. Trivedi, “Towards Privacy Protecting Safety Systems for Naturalistic Driving Videos,” IEEE Transactions on Intelligent Transportation Systems, 2014.

16 Experimentation Evaluation • Case Study of Face Recognition – 10 participants – ~80 de-identified images – Given a de-identified image, participants choose one of the 12 candidates that best matches.

17 Experimental Evaluation • Case Study of Gaze Zone Estimation – Gaze zones of interest: Left, front, right, rear-view, inside

18 Experimental Evaluation • Case Study of Gaze Zone Estimation – Gaze zones of interest: Left, front, right, rear-view, inside – 10 participants – 150 de-identified images (2 drivers, 5 images per gaze zone, 3 types of de-identification)

19 Experimental Evaluation De-Identification Recognition Gaze-zone Estimation Method Rate Accuracy (chance =8.3%) One-Eye 5% 65% Two-Eyes 8% 71% Mask with Two-Eyes 8%* 85% (b) (c) (a) Two- Eyes Mask with One- Eye Two- Eyes

20 Demo [Video]

21 Concluding Remarks • Public access to raw data of NDS is NOT available because of personal identifiable information. • Need privacy protection while preserving sufficient details to infer driver behavior. • Case study on three de-identification scheme: One-eye, Two- eyes and Mask with two-eyes. • Achieved 85% accuracy in gaze-zone estimation with face recognition below chance.

22 Thank You! Any Questions?