外文翻譯--基于相鄰幀的差別預測的多分辨率小波域運動估計和補償

1、<p><b>　　文獻翻譯</b></p><p>　　基于相鄰幀的差別預測的多分辨率小波域運動估計和補償</p><p>　　2011年3月26日</p><p>　院（系）名稱信息工程學院</p><p>　專業(yè)名稱計算機科學與技術(shù)</p><p>　學生姓名</p>

2、<p>　指導教師</p><p><b>　　附：英文原文</b></p><p>　　Multi-resolution Motion Estimation and Compensation based on Adjacent Prediction of Frame Difference in Wavelet Domain</p><

3、p>　　Tang Guowei</p><p>　　Abstract：Aiming at the higher bit—rate occupation of motion vector encoding and more time load of full—searching strategies, a multi-resolution motion estimation and compensation

4、algorithm based on adjacent prediction of frame difference was proposed. Differential motion detection was employed to image sequences and proper threshold was ad opted to identify the connected region. Then the motion r

5、egion was extracted to carry out motion estimation and motion compensation on it．The experiment resul</p><p>　　Key words: Motion estimation; Motion compensation; Multi-resolution analysis; Video coding</p

6、><p>　　I. Introduction</p><p>　　For the excellent properties of time-frequency localization, wavelet analysis is widely used in the field of image/video coding. As to image sequences, motion estima

7、tion and motion compensation can effectively reduce the time-relativity and improve the encoding efficiency. But the traditional motion compensation wavelet coding, taking the structure of motion estimation plus intra-fr

8、ame still image encoding, can not make full use of the advantages of the inherent multi—resolution characteristics </p><p>　　By using comparatively small searching window and matching block, this method can

9、reduce the amount of operation effectively, get rid of the blocking artifacts and be easy to achieve video scalable encoding suitable for human visual system and progressive transmission. But in the MRME algorithm there

10、exist the defaults of discontinuous motion vector and the inconformity of real object border with block border, which leads to the increase of high frequency components in transform coefficients and</p><p>　

11、　Zan Jinwen proposed multi resolution motion estimation through median filtering which can produce more smooth motion fields and result in a better estimation performance. But median filtering brings about a quite negati

12、ve effect on the unsmooth motion of high frequency sub-band in high resolution. Y.C.Su made a theoretical deduction and a deep research to interpolation algorithm of wavelet coefficients and proposed half-pixel multi-res

13、olution motion compensation which improves the accuracy of mo</p><p>　　This paper presents a Frame Difference adjacency prediction MRME algorithm (FDMRME), which adopts differential motion detection for imag

14、e sequence and extracts the motion region to carry out motion estimation and compensation. This method reduces the complexity of motion estimation, improves the encoding efficiency of motion vector and raises the quality

15、 of the reconstruction image at the same bit-rate as MRME.</p><p>　　II. Motion Detection Based On Frame Difference</p><p>　　1. Three frame difference method </p><p>　　Motion detecti

16、on includes optical flow algorithm, background elimination algorithm, adjacent frame algorithm and three frame difference algorithm. By using continuous three images to make difference operation and carry out AND operati

17、on to the difference results, the three frame difference algorithm can quickly detect the motion region from image sequences. The detection procedure is shown in Fig.1.</p><p>　　Fig.1 The procedure of motion

18、 object detection using three frame difference method</p><p>　　g1(x ,y)is the motion variation image of the first two frames, and g2(x ,y )is that of the later two frames. The motion information is included

19、in both of g1(x ,y)and g2(x ,y). Binarize the two motion variation images and make AND operation for them to obtain the motion objects. </p><p>　　2. Threshold selection of difference images</p><p&

20、gt;　　In order to extract motion objects. It is necessary to select proper threshold T for the frame difference images gl(x ,y)and g2(x ,y)by using a gray feature based approach and then to binarize the frame difference i

21、mages by T . The process of thres- hold selection consists of 4 steps:</p><p>　　(1) Among the three frame images. Divide the first and the second frames to 2x2 blocks．Add the 4 pixels of each blocks in secon

22、d frame to get a sum a , and add the 4 pixels of each corresponding blocks in first frame to get bi.</p><p>　　Here m．n are respectively the length and the width of the image, and k is the number of the 2

23、5;2 blocks.</p><p>　　(2) To binarize the frame difference image of the second and the first frames by using threshold T = 1.1S and get the binarization image. In this step a rough threshold is obtained on wh

24、ich a fined threshold can be obtained.</p><p>　　(3) Compute the mean value of the pixels less than the threshold T in the frame difference image.</p><p>　　Here q is the number of the pixels less

25、 than the threshold T in the frame difference image. Take M as the threshold of the frame difference image of the second and the first frames and then binarize the frame difference image.</p><p>　　(4) To bin

26、arize the frame difference image of the third and second frames by threshold M. And then compute the mean value of the pixels less than M in the frame difference image. Take the mean value as the threshold of the frame d

27、ifference image of the third and second frames and then binarize the frame difference image.</p><p>　　3. Identification of the connected region and extraction of the coordinates of motion region. </p>

28、<p>　　To label the AND operation images to get the coordinates of the motion region by using object clustering approach, which includes 2 steps:</p><p>　　Label each object pixels </p><p>　

29、　Scan the AND operation images from left to right and from top to bottom. When a pixel (gray-level is 1) belonging to object region is met, detect the 8 neighbors of this pixe1. If they are not labeled, label them with a

30、 new label number (start from 1), and each time the number increases by 1. Otherwise label the current pixel with the smallest label number of the 8 neighboring pixels.</p><p>　　(2) Cluster each object regio

31、n</p><p>　　Scan the motion object image from top to bottom line by line (from left to right and then from right to left).when an object pixel is met, detect the 8 neighbors of this pixe1. If the smallest lab

32、el number of them is greater than that of the object pixe1. It is substituted for the smallest labe1 number of the 8 neighboring pixels. When the whole image has been scanned., scan the image once more from bottom to top

33、 in the same way until all the labels of the object pixels stop changing.</p><p>　　For head-shoulder images there might be some irregular regions after the AND difference image has been processed by the obje

34、ct clustering procedure. Denoise these irregular regions and save them as the motion regions of the image.</p><p>　　III．Difference Adjacent Block Prediction Motion Estimation and Compensation</p><

35、p>　　Step1 Make 3 level wavelet decomposition to the image sequences and conduct motion detection to the lowest frequency sub-band LL3 by using three frame difference method. Extract the motion regions and divide them

36、into 2×2 blocks. Suppose the motion vector of each block as V3( x ,y)，and the other 3 sub-band motion vectors are also ( x ,y). Define a reliability flag R. If not all the pixels in theblock share the same state, th

37、en it can be determined that this block lies in the border of the moveme</p><p>　　Step 2 Check the corresponding flags of the pixels of the current block in 1ow frequency sub-band. If they belong to still re

38、gion then R = 1. And the motion vectors are 0, which do not need to be estimated. And the motion vectors of the corresponding position in other sub-bands are also 0. </p><p>　　Step 3 If all the current block

39、s of LL3 belong to motion region. Then R = 1. Predict according to the motion of the adjacent bocks. The relation between the block at( x ,y)with the adjacent bocks are shown in Fig.2.The motion estimation values are:<

40、;/p><p>　　Here the value of a1 is 0 or 1. If the corresponding block is located at the same region with the current block, then the value is 1. Otherwise it is 0. Determine whether the prediction value exceeds

41、the bound or not. If not, then start searching by</p><p>　　taking the prediction value as the center. In order to promote the consistency, the Mean Absolute Difference (MAD) is taken as the matching criterio

42、n.</p><p>　　Current block</p><p>　　Fig.2 The sketch of current block R=1 and the adjacencies</p><p>　　Step 4 If there axe both motion pixels and still pixels in the current block. T

43、hen it can be inferred that the current block is located at the border of the motion region. The motion is less reliable and it can not be processed as the reliable block．In order to promote the reliability of the predic

44、tion, more information needs to be obtained. So it will be processed later. W hen the first scanning is finished, all the motion vectors of reliable blocks have been achieved. Then the reliable motion </p><p&g

45、t;　　Here a1 has the same meaning as above.</p><p>　　Current block</p><p>　　Fig.3 The sketch of current block R=0 and the adjacencies</p><p>　　Step 5 Estimate the corresponding bloc

46、ks in other sub-bands by using the block motion vectors of the reference frame LL3 sub-band. For the level m(m <3) sub-images, the initial value of the motion vector of the block at(x ,y)is. Repeat Steps 3 and 4,the p

47、rediction value of motion estimation can be obtained as. Then the prediction value of the motion compensation is.</p><p>　　Step 6 The final motion vectors can be obtained for every block at(x ,y)in each sub-

48、images. So the motion compensation prediction of any pixel(x ,y)in a block is not only decided by the motion vector of this block but also by the motion vector in the adjacent blocks. At a fixed bit-rate, More bits can b

49、e allocated to the residual information to improve the quality of the reconstruction images. Meanwhile as the estimation is conducted only for the motion region, the time consumed in estimation is</p><p>　　I

50、V. Analysis of the Experiment Results</p><p>　　The condition of the experiment is shown in Tab.1, and 100 frames of Claire and Miss America are tested respectively.</p><p>　　Tab.2 shows the byte

51、s (represented by motion vector, Bytes/frame) for encoding the motion vector and the total time (represented by total time, seconds) consumed in motion estimation and compensation in the MRME/MRMC approach and the FDMRME

52、 approach. By making use of the sub-band orientation selectivity of wavelet decomposition in the FDMRME algorithm, the accuracy of the base motion vectors in low frequency sub-bands is promoted, and the error for motion

53、vectors of each sub- bands is decreased </p><p>　　Tab.1 Testing condition</p><p>　　Tab.2 Contrast of motion estimation performance</p><p>　　Tab.3 shows the encoding results of the F

54、DMRME method to the testing sequences. Here the PSNR represents the quality of the reconstruction image, motion vector is the same as that of the above, ER (Bytes/frame) represents the number of bytes for transmitting th

55、e error image after motion compensation, TOTAL (Bytes/frame) represents the total number of bytes of ER an d motion vector．From Tab.3, at a certain bit-rate, the bits for encoding the motion vector in the FDMRME algorith

56、m are much less, so</p><p>　　Tab.3 Contrast of encoding results</p><p>　　V. Conclusion</p><p>　　The variable block multi-resolution motion estimation and compensation is an importan

57、t approach to achieve high efficiency video encoding in wavelet based video encoding domain. By analyzing the problem of the MRME method, this paper proposed that segment the motion region through motion detection and gu

58、ide the process of motion estimation with it, the consistency and the accuracy of the motion vectors are promoted, the encoding efficiency is improved, and meanwhile the complexity of motion esti</p><p>　　Fr

59、om: Tang Guowei, Multi-resolution Motion Estimation and Compensation based on Adjacent Prediction of Frame Difference in Wavelet Domain, Journal of Electronics, May 2009.</p><p><b>　　英語譯文</b><

60、/p><p>　　基于相鄰幀的差別預測的多分辨率小波域運動估計和補償</p><p><b>　　唐國偉</b></p><p>　　摘要：針對高比特率的運動矢量編碼和占用更多時間負載的全部搜索策略，提出了基于相鄰幀的差別預測的多分辨率小波域運動估計和運動補償算法。圖像序列使用微分運動檢測方法，并選擇適當?shù)拈撝祦泶_定相連區(qū)域。接下來提取出運動的區(qū)域

61、來進行運動估計和運動補償。實驗結(jié)果表明，該算法提高了運動矢量編碼的效率，降低了運動估計的復雜度，在相同的比特率的情況下，重構(gòu)圖像具有更好的質(zhì)量，多分辨率運動估計算法（MRME）得以改善。</p><p>　　關(guān)鍵詞：運動估計；運動補償；多分辨率分析；視頻編碼</p><p><b>　　1、引 言</b></p><p>　　由于小波分析的時頻

62、局部化性能，被廣泛應用于圖像/視頻編碼領(lǐng)域。正如圖像序列，運動估計和運動補償可以有效地減少相對的時間并且有效地改進編碼。然而傳統(tǒng)的運動補償小波編碼，以其運動估測的結(jié)構(gòu)再加上幀的內(nèi)部靜止圖像編碼，不能充分利用固有的多分辨率特性的優(yōu)勢。在1992年，Yaqin Zhang和S.Zafar提出可變方塊多分辨率運動估測的fMRME1視頻壓縮算法，即1aid的運動估測和運動補償基礎(chǔ)是在在小域中。</p><p>　　通過使

63、用相對較小的搜索窗口和匹配塊，這種方法可以有效的減少運算量，擺脫方塊效應，并有易于使視頻可擴展編碼適合人類視覺系統(tǒng)并實現(xiàn)進步的過渡。但是在MRME算法中，存在著不連續(xù)的運動矢量的默認值和實物邊界和方塊邊界的不一致這導致了高頻率在系數(shù)變換中的成分增加，并影響位移幀的差[DFD]的編碼。</p><p>　　Zan Jinwen提出通過多分辨率運動估測中值濾波，可產(chǎn)生更平穩(wěn)的運動領(lǐng)域，產(chǎn)生一個更好估計性能。但中值濾波

64、會在不平穩(wěn)高分辨率的子帶運動中帶來相當負面的影響Su141作了理論推斷，深入研究小波系數(shù)的插補算法并提出半像素多分辨率運動補償，有效提高運動估計的準確性。為了克服了離散的小波變換不斷轉(zhuǎn)移的屬性。Zhang提出基于2聲道高通訊濾波和子帶的小波域運動估測適應中央搜索點預測.此方法有一個相當?shù)偷挠嬎銖碗s度，但編碼器的性能也從PSNR值的數(shù)據(jù)中減少了。Cagnazzo 所研究的基于視頻編碼的小波理論性優(yōu)化準則，提出一個最佳運動估計和補償?shù)姆椒ǎ?/p>

65、但是此方法是以擴展一個復雜性的成本為代價的法，但在一個擴展的COM成本復雜性。</p><p>　　本文提出了一種幀差的鄰接預測MRME算法（即FDMRME）它采用差動檢測圖像序列，并提取出運動區(qū)域進行運動估計和補償。這種方法減少運動估測的復雜性，提高了運動矢量的編碼的高效性，并提出在同一比特率情況下，作為MRME的圖像重建的質(zhì)量。</p><p>　　2、基于幀差的運動檢測</p&

66、gt;<p><b>　?。?） 三幀差算法</b></p><p>　　運動檢測包括光流的算法，背景消除算法，相鄰三幀差法。利用連續(xù)三幅圖像，使差異化操作并進行AND運算導致出不同的結(jié)果，三幀差算法可以快速從圖像序列中檢測出運動區(qū)域。檢測過程如圖1所示。</p><p>　　圖1 用三幀差算法對運動對象的檢測過程</p><p>

67、;　　g1(x，y)是前兩個幀的運動圖像的變化，g2(x，y)是后兩個幀。g1(x，y)和g2(x，y)都包含有運動的信息。使兩個運動變化圖像二值化，并為他們進行AND運算以獲得運動對象。</p><p>　　（2） 不同圖像的閾值選擇</p><p>　　為了提取運動目標，有必要選擇合適的閾值T作為框架。不同的圖像g1(x，y)和 g2(x，y) 通過使用基于灰度特征的方法，然后通過閾值

68、T使幀差圖像二值化。</p><p>　　閾值選擇的過程包含4個步驟。</p><p>　　第一、在三幀圖像中，將第一和第二幀分成2X2的塊。在第二幀上每塊增加4個像素以獲得總和a，然后每個相應的塊加上的第一幀的4個像素以獲得bi。</p><p>　　這里m，n分別代表圖像的長和寬，k是2×2個塊的數(shù)量。</p><p>　　第二

69、、通過使用閾T=1.1S，使第一第二幀的幀差圖像二值化，得到二值化圖像。</p><p>　　在這一步驟中，獲得一個粗略的閾值，進而可以得到一個被罰閾值。</p><p>　　第三、在幀差圖像中，計算像素的平均值小于閾值T。</p><p>　　q是指在幀差圖像中像素數(shù)量小于閾值T。以M為第二和第一幀的幀差圖像的閾值，然后使幀差圖像二值化。</p>&

70、lt;p>　　第四、通過閾值M使第三和第二幀的幀差圖像二值化然后在幀差圖像中計算出像素的平均值小于閾值M。以平均值作為第三和第二幀的幀差圖像的閾值，然后二值化幀差圖像。</p><p>　?。?） 識別所連接的區(qū)域和運動區(qū)域的坐標的提取</p><p>　　標注出AND運算圖像以通過使對象聚類的方法得出運動區(qū)域的坐標，包括以下2個步驟。</p><p>　　第

71、一、標注出每個對象的像素</p><p>　　從左到右，從上到下掃描AND運算的圖像。每當遇到一個屬于目標區(qū)域的像素（灰度級為1），就檢測該像素的相鄰的8個像素。如果這8個像素沒有被標注出來，就用和一個新的標簽號碼標注他們（從1開始），每一次的數(shù)量增加1。否則就把當前像素的標簽號碼標注成這8個相鄰像素中最小的編號。</p><p>　　第二、群集每個對象區(qū)域</p><

72、p>　　從頭到尾一行一行的掃描運動物體圖像（從左到右，然后由右至左）。每當遇到一個對象的像素，檢測該像素的8個相鄰的像素。如果這8個中最小的標簽號比該像素的標簽號大，它就會被這8個中小的標簽號所取代。當整個圖象已經(jīng)掃描完之后，再從后到前以同樣的方式再掃描一下的圖像，直到所有對象像素的標簽不再變化為止。</p><p>　　當AND差異圖像已被對象聚類過程處理過之后，頭肩圖像可能有一些異常區(qū)域。降噪這些異常區(qū)

73、域并把它們作為圖像的運動區(qū)域保存下來。</p><p>　　3、差異相鄰塊運動估計和補償</p><p>　　第1步，把3級小波分解到圖像序列，并進行運動檢測，通過使用三幀差法，以得出最低頻率子帶LL3。提取運動區(qū)域，并把他們分成2× 2個方塊。假設(shè)每塊的運動矢量是V3（x，y），其他3子帶的運動矢量為也（x，y）。定義一個可靠的標記R。如果不是所有的像素在方塊所共享的同一狀態(tài)，

74、那么就可以斷定這個塊所在的運動物體邊界是不可靠的運動估計。</p><p>　　第2步，檢查當前方塊低頻率子帶相應的像素標志,如果他們屬于靜止區(qū)域則R= 1且運動矢量是0，這并不需要估計。而在其他子帶對應位置的運動矢量也為0。</p><p>　　第3步，如果所有的LL3當前方塊屬于運動區(qū)域。則R= 1，據(jù)相鄰方塊運動的預測，塊（x，y）與相鄰方塊之間的關(guān)系如圖2所示。 </p>

75、;<p>　　a1的值是0或1。如果相應的方塊位于同一地區(qū)的當前塊，那么這個值是1。否則則是0。確定是否超過預測值，綁定與否。如果不是，那么就把預測值當成中心開始搜索。為了保持一致性，把平均絕對差（MAD）作為匹配準則。</p><p><b>　　當前塊</b></p><p>　　圖2 當前塊R=1以及相鄰方塊的略圖</p><p

76、>　　第4步，如果有一個對象，它的像素即是它的運動像素，同時也是當前方塊的像素。那么可以推斷當前方塊位于運動區(qū)域邊界處。該運動是不可靠，它不能被當成是可靠的方塊來進行處理。為了促進了預測的可靠性，需要知道更多的信息。所以這些隨后會來處理。當?shù)谝粋€掃描完成后，所有可靠的方塊運動矢量已實現(xiàn)。然后，毗鄰不可靠方塊的可靠方塊的運動矢量可以被用來于作出預測。當前方塊R = 0時與相鄰方塊之間的位置關(guān)系如圖3所示。</p>&l

77、t;p><b>　　當前塊</b></p><p>　　圖3 當前塊R=0以及相鄰方塊的略圖</p><p>　　第5步，通過參照幀LL3子帶使用方塊運動矢量，在其他子帶中估計相應的方塊。對于m(m <3)子圖像，方塊（x，y）的運動矢量的初始值是 。重復第3步和第4步，可以得出運動估計預測值是 。</p><p>　　第6步，因為

78、每個方塊（x，y）在每個子圖像中，所以最后的運動矢量 的可以得出來。因此，在方塊中，任何像素（x，y）的運動補償預測不僅取決于這個方塊的運動矢量，同時也取決于相鄰方塊的運動矢量。在固定的比特率的情況下，更多的比特可以分配到剩余的信息，以提高重建圖像的質(zhì)量。與此同時。由于估計只針對運動區(qū)域， 用于估計的時間消耗要少得多。對于運動區(qū)域這是成正比的，而不是對于整個圖像。</p><p><b>　　4、實驗結(jié)

79、果分析</b></p><p>　　該實驗條件如表1所示。</p><p>　　克萊爾100幀 和美國小姐分別進行了測試。表.2顯示運動矢量的字節(jié)（由運動矢量，字節(jié)/幀表示）和在MRME/ MRMC補償法和FDMRME方法中所消耗的運動估計的總時間（表示總時間，秒）通過利用的子帶取向小波分解的選擇性，使用FDMRME算法，改進低頻率子帶的基運動矢量的準確性。每個子帶的運動矢量的

80、錯誤率下降，這些證明了運動矢量的編碼的有效性是有目共睹的。在CIF序列中美國小姐所消耗的時間比在QCIF序列中克萊爾所消耗的時間少。通過使用運動檢測，視頻幀分為運動區(qū)域，靜止區(qū)域。編碼器不會使運動估計和補償成為靜止區(qū)域，所以消耗的時間是由運動區(qū)域而不是由整個圖像決定的，這導致系統(tǒng)效率的改進。</p><p><b>　　表1 實驗條件</b></p><p>　　表2

81、 運動估計值對比</p><p>　　表3顯示了FDMRME方法的編碼結(jié)果，用來測試序列。這里PSNR表示重建圖像的質(zhì)量，運動矢量代表的意思和上面的一樣。ER（字節(jié)/幀）表示經(jīng)過運動補償之后，用于傳輸錯誤圖像的字節(jié)數(shù)。TOTAL（字節(jié)/幀）表示ER字節(jié)和運動矢量的總字節(jié)數(shù)。從表3得出，在一定的比特率情況下，在FDMRME算法中，用于編碼的運動矢量的比特要少得多。所以更多的比特可以分配到運動補償后的殘差，以提高重建

82、圖像的質(zhì)量。</p><p><b>　　表3 編碼結(jié)果對比</b></p><p><b>　　5、結(jié)論</b></p><p>　　變量方塊多分辨率運動估計和補償是基于視頻編碼領(lǐng)域?qū)崿F(xiàn)小波視頻編碼高效率的重要途徑。通過分析MRME方法出現(xiàn)的問題，本文提出了通過運動檢測的運動矢量部分以及運動估計過程的指導，促進了運動矢量