Towards Reversible De-Identification in Video Sequences Using 3D - PowerPoint PPT Presentation

6th COST Action IC1206 Meeting, WG2 Towards Reversible De-Identification in Video Sequences Using 3D Avatars and Steganography Karla Brkić University of Zagreb Faculty of Electrical Engineering and Computing HR-10000 Zagreb, Croatia Collaborators: Martin Blažević, Tomislav Hrkać Karla Brkić November 3, 2015 1

Outline o Reversibly de-identifying people in videos and images – Challenges – Possible approaches o A 3D avatar-based de-identification pipeline o Preliminary results (using the Kinect sensor) o Conclusions Karla Brkić November 3, 2015 2

Our goal: de-identifying people in videos and images o concealing identities of people while maintaining informa Karla Brkić November 3, 2015 3

How to do proper de-identification? Identity revealing features: o biometric – e.g. face o soft biometric – e.g. tattoos, scars, birthmarks o non-biometric – e.g. dressing style, hairstyle We aim to de-identify soft and non-biometric features. Karla Brkić November 3, 2015 4

Additionally, we are interested in … o concealing identities of people while maintaining information on the action and its context o reversible de-identification Input: image or Person video sequence detection Identifier Identifier segmentation segmentation De-identifying De-identifying Output: de- transformation transformation identified content REVERSIBLE! Karla Brkić November 3, 2015 5

Possible approaches (I) o Find and de-identify individual soft and non-biometric identifiers find tattoos, blur them find hair, change its color find birthmarks, blur them find clothes, change their style Karla Brkić November 3, 2015 6

Possible approaches (II) o De-identification of the whole body – replace the person with a virtual avatar Karla Brkić November 3, 2015 7

Our envisioned de-identification pipeline (I) Karla Brkić November 3, 2015 8

Our envisioned de-identification pipeline (II) o focus of this work: the last two stages of the pipeline – replacing the person in the video with a 3D model – steganographically encoding the replaced image in the new image o prerequisite: pose estimation – a hard problem – to investigate close-to-best scenario, we use Microsoft Kinect Karla Brkić November 3, 2015 9

Recovering the 3D skeleton from Kinect o Kinect features: – standard VGA color camera – multi-array microphone – depth sensor o capabilities: Hierarchy of joints defined by the skeletal tracking system – motion capture – face gestures – voice detection Karla Brkić November 3, 2015 10

Rendering the 3D model o the avatar consists of two parts: – hierarchical set of interconnected bones – skeleton or rig – surface representation used to draw the character – skin or mesh Karla Brkić November 3, 2015 11

Concealing the original image (I) o steganography: the science of hiding information Receiver Sender o 𝑔 𝐹 : insertion o carrier : data file into which the −1 : extraction o 𝑔 𝐹 hidden data is hidden o key: steganographic key o message : secret message that o stego: steganographic file needs to be hidden Karla Brkić November 3, 2015 12

Concealing the original image (II) o LSB insertion steganographic algorithm Karla Brkić November 3, 2015 13

Experiments – model rendering Karla Brkić November 3, 2015 14

Experiments – steganographic encoding (I) (a) 1 hidden bit (b) 2 hidden bits Karla Brkić November 3, 2015 15

Experiments – steganographic encoding (II) (c) 4 hidden bits (d) 7 hidden bits Karla Brkić November 3, 2015 16

Conclusions o Our solution enables: – detecting and tracking persons in videos – reversible hiding of their identity o Limitations: – dependent on the Kinect sensor – the 3D model cannot perform a broad range of movements – errors in tracking, pertaining to the Kinect device o Future work: – more natural human models – finer capture of joint dana and body movements Karla Brkić November 3, 2015 17

Thank you! o Questions? Karla Brkić November 3, 2015 18