Motion Estimation by Affine Transforms Motion Estimation by Affine - PowerPoint PPT Presentation

Motion Estimation by Affine Transforms Motion Estimation by Affine Transforms Motion Estimation by Affine Transforms Motion Estimation by Affine Transforms Based on Based on Codirectionality of Movements Codirectionality of Movements Codirectionality of Movements Codirectionality of Movements Mohrekesh, S. Samavi, N. Karimi, S. Shirani, P. Behnamfar Department of Electrical and Computer Engineering Department of Electrical and Computer Engineering Isfahan Isfahan University of Technology, Isfahan, Iran University of Technology, Isfahan, Iran McMaster McMaster University, Hamilton, McMaster McMaster University, Hamilton, University, Hamilton, Canada University, Hamilton, Canada Canada Canada University of British Colombia, Canada University of British Colombia, Canada

Outline Outline • Introduction • Motion vectors • Block based • Mesh based • ABC ABC • Results • Conclusion • Conclusion

Importance of Video Compression • Improvements in Video Processing • Video Applications pp • Machine vision • Medical imaging • Video conferencing Vid f i • Remote learning • Information storage/transmission in • Information storage/transmission in limited memory/bandwidth � Impossible storage/transmission of � Impossible storage/transmission of raw data � Definite need for compression � Definite need for compression

Introduction: Video Compression • Video Compression Basis • Motion estimation/compensation • Time redundancy elimination • Differences Between Methods D fferences Between Methods • Complexity • Accuracy Accuracy • Time for calculation

Block Based Motion Estimation • Best mach of current block in the reference frame f f • Current block displacement from reference block = − f ( x , y ) x u ⎡ ⎤ u i i � Motion Vector ⎢ ⎥ = − v v g g ( ( x x , y y ) ) y y v v ⎣ ⎣ ⎦ ⎦ i i i Reference Frame Current Frame

Error Criteria 2 − − ( ( ) ) N 1 N 1 1 ∑∑ ∑∑ = − MSE C R ij j ij j 2 2 N N = = i 0 j 0 − − 1 N 1 N 1 ∑∑ = − MAE C R ij ij 2 N = = = = i i 0 0 j j 0 0 − − N 1 N 1 ∑∑ ∑∑ = − SAD C R ij ij = = i 0 j 0

Block Based Shortcomings • Just Translation • Unable to identify motions such as: • Rotation • Shearing • Zoom in/out • Disability in Codirectionality • Equal Motions for Pixels of a Block • Equal Motions for Pixels of a Block • Reconstructed Frame Discontinuities

Mesh Based Motion Estimation • Various Motions Modeling • Various Motions Modeling • Using Transforms • Current Frame � Mesh F h • Mesh • Regular: less accurate • Irregular: more complex g p

Transforms = + + f ( x , y ) a x a y a • Affine • Affine i 1 i 2 i 3 = + + g ( x , y ) a x a y a i 4 i 5 i 6 • Ability to Model Different Motions y • Disability in Codirectionality • More Complex • More Complex

Proposed Method: ABC • Affine transform Based on Codirectionality • Current Frame Partitioning • Assuming Block for Nodes g • Block Matching Reference Frame Current Frame

ABC • Finding Triangles Motion Vectors • Triangle Partitioning (if needed) Triangle Partitioning (if needed)

ABC • Transforms • Affine • Affine • Bilinear vector interpolation interpolation • Translation

ABC: Transform Selection Criteria • Affine Domain • Rotation • Zoom in • Zoom out Almost equilateral triangles produced � Vector difference lengths almost equal � Selection based on closeness of vector differences

ABC: Finding Different Vector a b c One of differences smaller than half of average of the others smaller than half of average of the others

for each Δ ABC { } ∀ ∈ α , β A,B,C ABC ABC ABC ABC ( ) ( ) ( ) ( ) 1 1 = − + − 2 2 d α , β x x y y 2 ( ) ( ) ( ) ( ) MV α MV β MV α MV β { { } } ( ( ) ) ( ) ( ) ( ( ) ) = ∀ ∈ ≠ N ,N d γ α , β α β arg min γ γ 1 2 ( ) ( ) ∑ < 1 if d N ,N d γ 1 2 10 γ { { } { } { } } = − F A,B,C N ,N 1 2 + + F N F N ′ ′ = = N N 1 2 , 1 2 2 2 2 2 ( ( ) ) ′ ′ translate Δ F N N ,MV F 1 2 ( ( ) ) ′ ′ − interpolat interpolat e e Δ FN Δ FN N N Δ F Δ F N N N N 1 1 2 2 1 1 2 2 else ( ( ( ) ( ) ( ) ( ) ( ) ( ) ) ) affine ff Δ ABC,MV A ,MV B ,MV C end if end for

Results Results Paris Paris ABC

Results Results Mobile Mobile ABC ABC

Results Results Foreman Foreman ABC ABC

Results Results Hall Monitor Hall Monitor ABC ABC

Results Results Original frame frame MFMB result result ABC result

Conclusion Conclusion • Video Compression Importance Vid C i I t • Time redundancy • Motion Estimation • Block based method • Mesh based method • Proposed ABC Method Proposed ABC Method • Better performance in codirectionality • Higher PSNR Higher PSNR • Results