Yet a Faster Motion Estimation Algorithm with Directional Search - PDF document

15 th International Conference on Digital Signal Processing (DSP’2007), July 2007, Cardiff, UK. Yet a Faster Motion Estimation Algorithm with Directional Search Strategies Speaker: Prof. W.C. Siu Ying Zhang+*, Wan-Chi Siu* and Tingzhi Shen+* *The Hong Kong Polytechnic University, Hong Kong + Beijing Institute of Technology, Beijing W.C. Siu Outline Introduction 1 Proposed algorithm 2 Experimental results 3 Conclusion 4 2 W.C. Siu 1

1. Introduction of Motion Estimation • Motion estimation (ME) is a popular technique for hybrid video coding, and has been adopted in video-coding standards to exploit the temporal redundancy existing between frames. • The most common approach of the ME is the block matching algorithm (BMA). • It divides each frame into a number of non-overlapping macroblocks (MBs), and to predict the motion activities of each of these MBs by searching for the most similar block from a reference frame which is the previous decoded frame usually. 3 W.C. Siu Introduction of Motion Estimation • The motion of a MB is represented by a motion vector (MV) which is the spatial displacement between the original position of the MB and the position of the reference block. • Exhaustive full search (FS) method is the most basic method of the BMA. • In the FS, a MB will find its best match in the reference frame from all positions within a range called search window which is centered at the position of the current MB. Motion Estimation Error Signal Extraction 4 W.C. Siu 2

Introduction Motion Estimation and Fast Motion Estimation Algorithms Full Search ME Pixel Decimation Fast Full Search Search by Pattern: Fast search TSS, DS, BBGDS, .. Search by Scheme: MVFAST, PMVFAST, FAME, EPZS .. (with patterns) Proposed: by Scheme Jump to Terms 5 W.C. Siu Proposed algorithm － Search patterns The search pattern: 1. the star (diamond) based search pattern 2. the block based search pattern -4 -3 -2 -1 0 1 2 3 4 -4 -3 -2 -1 0 1 2 3 4 Figure 1. The star based search pattern 6 W.C. Siu 3

Proposed algorithm － Search patterns The search pattern: 1. the star based search pattern 2. the block based search pattern -4 -3 -2 -1 0 1 2 3 4 -4 -3 -2 -1 0 1 2 3 4 Figure 2. The block based search pattern 7 W.C. Siu Proposed algorithm － Use of search pattern A sample of search procedure with star based search pattern. the sum of absolute differences (SAD) -4 4 -3 -2 -1 0 1 2 3 The Distortion value is decreasing , -4 -3 Step in the future -2 Step3 -1 Current best Step2 0 position 1 Step1 Original 2 position 3 4 Figure 3. Sample Search Procedure 8 W.C. Siu 4

Proposed algorithm － Block classification In the previous frame : Predicted MV Final MV = Predicted MV + additional MV MV MV prediction Search Stage1 Stage2 If additional MV of the block=0, Classify the block into the ordinary block group; Else Classify the block into the special block group. 9 W.C. Siu Proposed algorithm － Search pattern selection In the current frame: Predicted MV Final MV MV MV prediction Search Search pattern selection Stage1 Stage2 If the current block is prejudged as a ordinary block, Choose the star based search pattern; Else Choose the block based search pattern. 1 0 W.C. Siu 5

Proposed algorithm － The directional information The directional information: 1. the latest directional vectors 2. the general directional vectors -4 -3 -2 -1 0 1 2 3 4 -4 -3 SAD: 982 -2 SAD:1745 SAD:2231 -1 SAD:1008 SAD:1823 Current best 0 position Latest directional set 1 right 2 top 3 4 Figure 4. The latest directional vectors 1 1 W.C. Siu Proposed algorithm － The directional information The directional information: 1. the latest directional vectors 2. the general directional vectors -4 4 -3 -2 -1 0 1 2 3 -4 -3 Step in the future -2 Step3 -1 Current best Step2 0 position 1 Step1 Original 2 position 3 General directional set right 4 top Figure 5. The general directional vectors 1 2 W.C. Siu 6

Proposed algorithm － The directional information The number of search points required can be reduced with the directional information. Step 3, Latest directional set -4 -3 -2 -1 0 1 2 3 4 right -4 top -3 Original position Step4 -2 General directional set Step3 -1 right 0 top 1 Policy: The Union of 2 these two directional sets Step1 Step2 (‘Latest|General’) 3 Step 4, 4 Candidate points set Figure 6. Sample Search Procedure Right, top points 1 3 W.C. Siu Proposed algorithm － The flow chart Classify blocks into Ordinary or Special blocks, & find the threshold for early termination. Find SAD of the initial point. Obtain the predicted motion vectors and choose a new center for the search window. Yes Note1 Early termination Next block No Search the surrounding area from the current best point with the selected search pattern. Note2 Last block? No Yes Calculate the statistical motion information. Note1: MV=predicted MV, for SAD < threshold Note2: MV=predicted MV+  MV 1 4 W.C. Siu 7

Some further Details: Early termination approach: We make use of the statistics of the previous frame, etc. to determine a threshold, for early termination. We assume that the numbers of stationary blocks between successive frames are similar. To look for the centre of the search window: motion vector = (0,0) with reference to the block location, (i,j), medium motion vector of (MV(i-1,j,t), MV(i,j-1,t) and MV(i+1,j-1,t)), MV(i,j,t-1), and MV(i+1,j+1) 1 5 W.C. Siu Experimental results Table I: Performance comparison of different algorithms PSNR : the average PSNR value of the luminance component of the decoded frames. SpUp : the speed-up on the average number of SAD checking points VS full search. SpUp t : the average speed up on real time measurement VS full search algorithm. Sequences Hall Container Flower PSNR SpUp SpUp t PSNR SpUp SpUp t PSNR SpUp SpUp t 37.21 1 1 35.64 1 1 23.87 1 1 FS TSS 37.22 31 27 35.58 31 28 23.36 31 25 37.26 70 62 35.57 75 68 23.91 59 53 DS BBGDS 37.25 94 80 35.58 103 79 23.92 73 67 37.27 181 158 35.52 192 155 23.86 151 128 CDS 37.28 147 145 35.56 170 134 23.95 109 92 MVFAST FAME 36.64 103 80 35.00 91 74 24.08 89 71 37.34 445 273 35.75 786 370 23.96 157 118 OurAlg. bitrate=380K bit/sec. 1 6 W.C. Siu 8

Experimental results PSNR(dB) 32.5 Proposed 32.4 MVFAST 32.3 DS 32.2 BBGDS 32.1 TSS FAME 32.0 CDS Further 31.9 Results 31.8 20 40 60 80 100 120 140 160 Speed up (real-time) 1 7 W.C. Siu Conclusion The important points of this algorithm: 1. The block classification; 2. The directional information. Simplicity, low memory, fast realisation speed and low error are our objectives. Simulation results: 1. Our approach is faster than all algorithms available to us in the literature; 2. It also has similar or even better performance on image Jump to quality and bit-rates. the End 1 8 W.C. Siu 9

Future Work: Sub-pixel motion estimation using directional information: Elementary results obtained: a good trade-off between quality and realisation speed is obtained. Motion Estimation with RDO setting: 1. Simply using the Lagrangian Optimzation equation in H.264 (12.2) 2. Similar results obtained as discussed Further work on the simplification of the process to look for the centre of the search window, etc.: motion vector = (0,0) with reference to the block location, (i,j), medium motion vector of (MV(i-1,j,t), MV(i,j-1,t) and MV(i+1,j-1,t)), MV(i,j,t-1), and MV(i+1,j+1) and the simplification 1 9 W.C. Siu The End Thank you! W.C. Siu 10

Additional slices: Performance comparison (with RD control) --the average value of the results obtained with 18 sequences PSNR Bits/pixel Sequence (dB) FS 32.58 0.1653 Proposed 32.41 0.1653 MVFAST 32.35 0.1658 EPZS 32.26 0.1657 FAME 32.08 0.1661 BBGDS 32.12 0.1680 DS With RD 32.21 0.1681 CDS 31.91 0.1679 TSS 31.93 0.1686 2 1 W.C. Siu Additional slices: Performance comparison (with RD control) --the average value of the results obtained with 18 sequences 32.6 PSNR (dB) Proposed MVFAST 32.5 EPZS CDS 32.4 BBGDS DS FAME 32.3 TSS 32.2 32.1 32.0 31.9 0.1655 0.1660 0.1665 0.1670 0.1675 0.1680 0.1685 Return Bits/pixel 2 2 W.C. Siu 11

Experimental results bits/sec 380000 (for rate control) 760000 (for rate control) PSNR Bits/pixel PSNR Bits/pixel FS 32.46982 0.1659 34.98632 0.3127 Proposed 32.40655 0.1656 34.88912 0.3126 MVFAST 32.33186 0.1659 34.85154 0.3128 32.27764 0.1658 EPZS 34.80252 0.3128 FAME 32.01032 0.1664 34.65904 0.3150 BBGDS 32.08266 0.1683 34.54872 0.3148 DS 32.15516 0.1679 34.63003 0.3143 CDS 31.88913 0.1688 34.39547 0.3156 TSS 32.06936 0.1681 34.57621 0.3149 2 3 W.C. Siu 12