Estimation of Automotive Pitch, Yaw, and Roll using Enhanced Phase Correlation on Multiple Far-field Windows M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester Visual Sensorics and Information Processing Lab Goethe-University, Frankfurt and Computer Vision Laboratory Link¨ oping University, Sweden July 22, 2015 M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 1
Motion estimation From far-field sub-images at two instants t − 1 and t , the module determines: Global 2D translational motion Rate changes in the camera angle (roll/pitch/yaw) Rotation matrix between t − 1 and t The motion is analysed for each pair of corresponding windows at instants t − 1 and t M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 2
System Overview M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 3
Structure test The motion is only measured on those cells with enough structure information. To find those, we compute the windowed average M 1 � m k = ˆ x i w T ( x i ) · w T ( x i ) · s( x i ) (1) � i =1 and the variance: M 1 σ 2 � m k ) 2 ˆ k = x i w T ( x i ) · w T ( x i ) · (s( x i ) − ˆ (2) � i =1 Further computations are performed only on cells with sufficient structure. too weak structure < σ 2 ˆ T var (3) > k sufficient structure M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 4
Determining the translation motion We use a regularized version of Phase-only-Correlation. S( f ) · G ∗ ( f ) P( f ) = (4) | S( f ) · G ∗ ( f ) | + λ Where lamba performs a soft thresholding of the FFT, leaving almost intact the votes from amplitudes larger than lambda and giving small weight to the rest. The displacement d between cells is determined as the location of the maximum of the inverse Fourier transform of P( f ): p( x ) = F − 1 { P( f ) } ≃ δ ( x − d ) (5) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 5
Thresholding the back-projected spectrum Due to non-pure translation motion, the back-projected spectrum might not concentrate in one peak. We compute the variance of the noise n = 1 σ 2 M � p( x ) � 2 (6) F and then we select a subset P of pixel coordinates belonging to the pixels of the peaks that passed the test ! ! p( x ) > 5 · σ n ∧ p( x ) > 0 . 6 · max(p( x )) (7) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 6
Checking the compactness of the peaks To find the peak compactness, the image intensity is normalized p( x ) / � p( x ) , x ∈ P h( x ) = (8) x ∈P 0 , else We first determine the centroids µ = ( µ 1 , µ 2 ) of h( x ) � µ = h( x ) · x (9) x and then the compactness K is computed as the sum of the 2nd order moments in both directions � h( x ) · � x − µ � 2 K = (10) 2 x We test the compactness K and only the compact cells are used afterwards compact < K (11) T comp > not compact M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 7
Estimating the roll angle We define the matrix A with the centres of the windows and matrix B with position after applying the displacement d Both sets of points are aligned to the origin of coordinates by k k A ′ = A − 1 B ′ = B − 1 � � c ′ c i (12) i k k i =1 i =1 M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 8
Estimating the roll angle We find the rotation matrix R that minimizes the element-wise difference of the points in A ′ and the rotated points in B ′ : min � A ′ R − B ′ � F (13) From the sought rotation matrix R : � cos(Φ) � − sin(Φ) R = (14) sin(Φ) cos(Φ) We obtain the roll angle Φ according to: Φ = arctan ( R 2 , 1 , R 1 , 1 ) (15) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 9
Estimating the roll angle Additionally we determine the average translation ¯ d between the point sets A and B , for latter estimating the pitch and yaw angles: k k d = 1 i − c i ) = 1 ¯ � ( c ′ � d i (16) k k i =1 i =1 The transformation T = ( R , ¯ d ) maps the B points into A . M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 10
Testing the validity of the mapping We validate if the mapping of the point sets is correct: k valid 1 < � i � 2 � T · c i − c ′ (17) T dist > 2 k not valid i =1 If the distance is too large, we iteratively repeat the previous process using a leave-one-out strategy and we take the motion vector with the smallest distance M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 11
Finding pitch and yaw angles f x x -p 1 α α H 1 -p 2 -p 2 H 2 f -p 1 f H The pitch θ and yaw Ψ are computed as: � c x − p ′ � c x − p ′ � � 2 x 1 x θ = arctan − arctan (18) f x f x � c y − p ′ � c y − p ′ � � 2 y 1 y Ψ = arctan − arctan (19) f y f y M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 12
Motion results Estimation of camera angles for Kitti sequence 3 rel. roll rate [deg/frame] 1 gt estim. 0.5 0 −0.5 50 100 150 200 250 frame number rel. pitch rate [deg/frame] 1 gt estim. 0.5 0 −0.5 50 100 150 200 250 frame number rel. yaw rate [deg/frame] 4 gt estim. 2 0 −2 50 100 150 200 250 frame number M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 13
Motion results Estimation of camera angles for Kitti sequence 7 rel. roll rate [deg/frame] 0.5 gt estim. 0 −0.5 −1 800 850 900 950 1000 1050 frame number rel. pitch rate [deg/frame] 0.5 gt estim. 0 −0.5 −1 800 850 900 950 1000 1050 frame number rel. yaw rate [deg/frame] 2 gt estim. 0 −2 −4 800 850 900 950 1000 1050 frame number M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 14
Motion results Table with results for the KITTI odometry sequences Roll Pi/Ya Pitch Yaw Seq Valid Est. E Corr Valid Est. E Corr Est. E Corr 0 96.19 0.121 0.631 99.98 0.044 0.963 0.133 0.991 1 99.18 0.078 0.577 99.91 0.024 0.949 0.068 0.99 2 93.84 0.159 0.5 99.98 0.053 0.951 0.126 0.987 3 97.99 0.086 0.754 99.87 0.038 0.968 0.098 0.985 4 94.78 0.142 0.395 99.63 0.04 0.873 0.121 0.193 5 98.37 0.108 0.509 99.96 0.033 0.966 0.114 0.989 6 98.45 0.089 0.503 99.91 0.026 0.968 0.105 0.994 7 97.63 0.1 0.608 99.91 0.03 0.965 0.151 0.922 8 95.82 0.104 0.62 99.98 0.034 0.965 0.123 0.991 9 94.08 0.136 0.524 99.94 0.04 0.953 0.125 0.987 10 93.41 0.139 0.537 99.92 0.04 0.974 0.121 0.986 M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 15
Conclusions What we achieved: A reliable method for determining 2D global motion vectors and rotation angles for a pair of consecutive frames of an automotive sequence. An algorithm which can run on real-time (20-25fps) and assist on the prediction of tracked keypoints. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 16
Future work Boost the accuracy of the 2D motion estimates by using sub-pixel measurements. Choose among multiple hypotheses from phase correlation. Measure the motion in more windows, and use depth estimates from SfM of previous frame. Use more complex consistency checks to reduce outliers still further. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 17
Contact M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester barnada,conrad,bradler,ochs,mester@vsi.cs.uni-frankfurt.de http://www.vsi.cs.uni-frankfurt.de http://www.cvl.isy.liu.se/ M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 18
Recommend
More recommend
