C LUSTER -B ASED D ISTRIBUTED F ACE T RACKING IN C AMERA N ETWORKS Josiah Yoder Robot Vision Lab — Purdue University 9 September 2011 1/51
I NTRODUCTION — C AMERA N ETWORKS 2/51
A C AMERA N ODE Vision Camera Computer Network Communication Interface 3/51
O VERVIEW Wired Networks Wireless Networks Medeiros et al. (2008) 4/51
O VERVIEW Face Pose Tracking Distributed Face Tracking Wired Networks Wireless Networks Medeiros et al. (2008) 5/51
O VERVIEW Face Pose Tracking Distributed Familiar Face Face Recognition Tracking Wired Networks Wireless Networks Unconstrained Face Tracking Medeiros et al. (2008) 6/51
C LUSTER - BASED T RACKING IN W IRELESS N ETWORKS Face Pose Tracking Distributed Familiar Face Face Recognition Tracking Wired Networks Wireless Networks Unconstrained Face Tracking Medeiros et al. (2008) 7/51
C LUSTER - BASED T RACKING IN W IRELESS N ETWORKS Developed by Medeiros et al. (2008) Addresses challenges of wireless networks: Limited communication range Cameras tracking same target may not be able to communicate 8/51
V ISION G RAPHS & C OMMUNICATION G RAPHS Vision Graph Communication Graph 9/51
C LUSTER L EADER E LECTION 1 2 5 3 4 10/51
C LUSTER C OALESCENCE 2 2 Coalesce 3 3 4 4 11/51
C LUSTER F RAGMENTATION 1 1 Fragment 2 2 3 3 12/51
T RACKING IN W IRED N ETWORKS Face Pose Tracking Distributed Familiar Face Face Recognition Tracking Wired Networks Wireless Networks Unconstrained Face Tracking Medeiros et al. (2008) 13/51
T RACKING IN W IRED N ETWORKS Vision Graph Communication Graph 14/51
E STABLISHING C ORRESPONDING D ETECTIONS How can cameras determine they have detected the same target? Detect objects Extract visual features Apply similarity criterion Objects “match” if criterion passes a decision threshold Many variations in features and matching criteria Color Histograms HoG features SIFT features Point clouds Face pose estimates Gabor jets . . . 15/51
T RACKING G RAPHS Edge between two cameras if their detected 1 3 4 Target objects pass the matching Target object A object B criterion Cameras may participate 2 5 in more than one tracking 1 4 graph 5 2 3 Dynamic: Changes as objects move 16/51
T RACKING G RAPHS : C LUSTER F ORMATION 1 May have missing edges in the tracking graph Target Cameras can only object establish correspondence 2 3 through other cameras For rapid cluster 1 formation, cameras join clusters only with 2 3 immediate neighbors 4 4 17/51
T RACKING G RAPHS : C LUSTER F ORMATION 1 2 3 May have false edges in tracking graph Leads to a single cluster tracking multiple targets Fixed by cluster 1 3 2 fragmentation during propagation 18/51
A F RAMEWORK FOR M ULTI -C AMERA F ACE T RACKING Face Pose Tracking Distributed Familiar Face Face Recognition Tracking Wired Networks Wireless Networks Unconstrained Face Tracking Medeiros et al. (2008) 19/51
A F RAMEWORK FOR M ULTI -C AMERA F ACE T RACKING A framework for face detection, pose estimation, and tracking: Allows generic single-camera methods to be incorporated into a multi-camera method Representation of face position as a coherent 6-DOF quantity 20/51
T HE 6-DOF W ORLD AND I MAGE -B ASED P OSES � �� � ] T p i = [ u , v , s � �� � , α , β , γ , � �� � ] T p w = [ x , y , z ���� , θ , φ , ψ = [ x T r T ] T i , i = [ x T r T ] T w , w 21/51
Coordinating Agent Face Camera 1 Camera 2 Camera N Detection Face Face Face Method Detection Detection Detection Observation Matching Orient. Orient. Orient. Orientation Estimation Estimation Estimation Estimation Method Evidence Integration Kalman Filtering
T RANSFORMATION OF P OSITION AND R OTATION We can transform observations from image to world coordinates, through an invertible function f p w = f ( p i ) � x w � � � f x ( u , v , s ) p w = = r w f r ( u , v , α , β , γ ) θ x x w = r w = φ y z ψ f x ( u , v , s ) – [Iwaki et al. 2008] f r ( u , v , α , β , γ ) – [Murphy-Chutorian and Trivedi 2008] 23/51
T RANSFORMATION OF P OSITION (u,v,1) (0,0,0) � K s � ˆ x w = f x ( x i ) = w R c + w t c d c s � u 2 + v 2 + 1 ) d c = ( u ˆ ˆ i c + v ˆ j c + ˆ k c ) / ( 24/51
T RANSFORMATION OF R OTATION r w = f r ( x i , r i ) = [ w R c c R i [ r i ] 3 × 3 ] 3 × 1 w R c — Camera Rotation c R i — Murphy-Chutorian & Trivedi Rotation [] 3 × 3 — Conversion to Rotation Matrix [] 3 × 1 — Conversion to Roll-Pitch-Yaw M URPHY -C HUTORIAN & T RIVEDI R OTATION (2008) rotation about the axis ˆ k c × ˆ d c by the angle cos − 1 ( ˆ k c · ˆ d c ) 25/51
U NCERTAINTY M ODELING We represent observations as Gaussian distributions Image-based observation: p i ∼ N ( p i , C p , i ) World observation: p w ∼ N ( p w , C p , w ) Transform from p i to p w using the Unscented Transform 26/51
C OMPARING O BSERVATIONS We compute the distance between j th and k th observation using the Mahalanobis distance d ( p j w , p k w ) = ( p j w − p k w ) T ( C j p , w + C k p , w ) − 1 ( p j w − p k w ) Distributed: We declare two observations consistent if d ( p j w , p k w ) < T Centralized: We employ a feature clustering algorithm based on the distance d ( p j w , p k w ) − T clique 27/51
I NTEGRATING O BSERVATIONS We use a minimum-variance estimator to integrate world observations into a more accurate estimate � − 1 p k � p w ]) ∑ C k E [ ˆ p w ] = ( Cov [ ˆ p , w w p k w ∈ E � � − 1 � � − 1 C k ∑ Cov [ ˆ p w ] = p , w p k w ∈ E 28/51
Coordinating Agent Face Camera 1 Camera 2 Camera N Detection Face Face Face Method Detection Detection Detection Observation Matching Orient. Orient. Orient. Orientation Estimation Estimation Estimation Estimation Method Evidence Integration Kalman Filtering
D ISTRIBUTED C LUSTER - BASED F ACE P OSE T RACKING Face Pose Tracking Distributed Familiar Face Face Recognition Tracking Wired Networks Wireless Networks Unconstrained Face Tracking Medeiros et al. (2008) 30/51
D ISTRIBUTED C LUSTER - BASED F ACE P OSE T RACKING Here we combine the two systems Multi-camera face pose tracking framework Cluster-based tracking protocol 31/51
S YSTEM A RCHITECTURE Camera Node 1 6-DOF Face Pose Estimation Clustering Module 1 Object Manager Matching Clustering Module Module 2 Integration ... Module Clustering Module k Network Camera Node 2 ... Camera Node N 32/51
F ACE P OSE AS I DENTIFYING F EATURE Cluster-based protocol makes use of a feature to distinguish targets Real-time unconstrained face recognition not available We use current face pose for this feature 33/51
E XPERIMENTS 270 280 290 300
E XPERIMENTS 200 40 20 150 ψ (degrees) X (cm) 0 100 −20 50 −40 0 0 100 200 300 400 500 0 100 200 300 400 500 frames frames 200 40 150 20 φ (degrees) Y (cm) 0 100 −20 50 −40 0 0 100 200 300 400 500 0 100 200 300 400 500 frames frames 200 40 150 20 θ (degrees) Z (cm) 0 100 −20 50 −40 0 0 100 200 300 400 500 0 100 200 300 400 500 frames frames
E XPERIMENTS Comparison with a centralized system Both use same synchronization of collaboration period use 6-DOF face pose estimation framework detect faces in the individual cameras In centralized system 6-DOF observations are sent to a central server Correspondences are computed based on all pairwise matches 36/51
E XPERIMENTS 200 150 Distributed X (cm) 100 50 0 0 50 100 150 200 250 300 350 400 450 500 frames 200 150 X (cm) Centralized 100 50 0 0 50 100 150 200 250 300 350 400 450 500 frames 37/51
E XPERIMENTS TP FP rmse T ( cm ) rmse R ( ◦ ) Centralized 95 (95 % ) 12 (12 % ) 5.8 20.8 Distributed 94 (94 % ) 4 (4 % ) 6.1 18.7 38/51
F ACE R ECOGNITION Cluster-based protocol can also be used for other activities Here, we perform distributed face recognition Face recognition useful for multi-camera tracking Associate observations from multiple cameras Associate multiple tracks with the same person Restore lost tracks Many other applications Each camera performs PCA face recognition Project face images into PCA space Select nearest neighbor from training set (gallery) of faces Send vote for that person to cluster leader Cluster leader counts votes and declares overall winner 39/51
E XPERIMENTS : D ISTRIBUTED F ACE R ECOGNITION Tracking Recognition TP / FP TP / FP 92.4% / 7.6% 87.9% / 9.9% Completely distributed: No central server, single point of failure Scalable — Load on each link or node does not increase with network size Only using frontal faces . . . uncommon in camera networks 40/51
H UMAN F AMILIARITY -B ENCHMARKED F ACE R ECOGNITION D ATABASE Face Pose Tracking Distributed Familiar Face Face Recognition Tracking Wired Networks Wireless Networks Unconstrained Face Tracking Medeiros et al. (2008) 41/51
Recommend
More recommend
