Situational Awareness for Smart City: Opportunities and Challenges - PowerPoint PPT Presentation

Situational Awareness for Smart City: Opportunities and Challenges Hao Lu | hao.lv@yitu-inc.com

About YITU AI Public Safety Retail Health Finance

Face Recognition Price Challenge 2017 Winner In Face Identification �������� ��������� �� ��������� ��� ����������

<1 billionth false match rate

<1 billionth false match rate But, what does it mean?

0.20 0.20 percentage 0.15 0.10 0.05 0.00 30 40 50 60 70 80 90 100 score

1/1k 1/10k 1/100k 1/1M 1/10M 1/100M 1/1B False Alarm Rate 40 60 66 75 85 90 95 Similarity Score 0.20 0.20 percentage 0.15 0.10 0.05 0.00 30 40 50 60 70 80 90 100 score

1/1k 1/10k 1/100k 1/1M 1/10M 1/100M 1/1B False Alarm Rate 40 60 66 75 85 90 95 Similarity Score 0.20 0.20 percentage 0.15 Twins 0.10 0.05 0.00 30 40 50 60 70 80 90 100 score

1/1k 1/10k 1/100k 1/1M 1/10M 1/100M 1/1B False Alarm Rate 40 60 66 75 85 90 95 Similarity Score 0.20 0.20 percentage 0.15 0.10 0.05 0.00 30 40 50 60 70 80 90 100 score

1/1k 1/10k 1/100k 1/1M 1/10M 1/100M 1/1B False Alarm Rate 40 60 66 75 85 90 95 Similarity Score 0.20 0.20 percentage 0.15 Cost 0.10 0.05 0.00 30 40 50 60 70 80 90 100 score

Towards Situational Awareness Opportunity: City can better understand how infrastructure is serving citizens

Towards Situational Awareness Opportunity: Shops can have a deeper understanding about their customers

One common theme: re-identify a person

One common theme: re-identify a person ID Video

Re-identify a person Capture Cluster Identify Across cameras; Across time; At large scale

City Scale 10 million people 100k cameras

City Scale 10 million people 100k cameras 1 billion faces per day

Capture faces Detection Tracking Filter Alignment Quality SSD w/ VGG VGG VGG

Capture faces Detection Tracking Filter Alignment Quality SSD w/ VGG VGG VGG

Face Quality Data Detection Tracks of 100k faces Tracking Good and hours of 1:1 Bad Frames videos Filter

Capture faces Video streams from 100k cameras = lot of bandwidth!

Capture faces on the camera end Feature/headshot batch + With capture TX2 Box

Cluster the same faces • Hierarchical Clustering on CPU • With heuristics: e.g., cluster same camera first

Identify against clusters • ANN on GPU • Using a few thousands of Tesla P4

Early results - Performance

Early results - accuracy • #Clusters to #people: 3 to 1 • Long tail

Discussion • Camera angles can be di ff erent • Poses can be di ff erent

Going forward • Leverage more attributes, e.g., wearing glasses, same outwear • Leverage history data

Thanks