參看論文：Fast and robust 3D ultrasound registration – Block and game theoretic matching

期刊水平：Medical imaging analysis (MIA)

投稿單位：伊拉斯謨醫學院 計算醫學中心

文章作者設計了一種全域性稠密塊匹配的跟蹤演算法，原理是序列配準，核心是幾何約束下的outliers reject策略。

1. 摘要

Real-time 3D US has potential for image guidance in minimally invasive liver interventions. However, motion caused by patient breathing makes it hard to visualize a localized area, and to maintain alignment with pre-operative information. In this work we develop a fast affine registration framework to compensate in real-time for liver motion/displacement due to breathing. The affine registration of two consecutive ultrasound volumes in time is performed using block-matching. For a set of evenly distributed points in one volume and their correspondences in the other volume, we propose a robust outlier rejection method to reject false matches. The inliers are then used to determine the affine transformation. The approach is evaluated on 13 4D ultrasound sequences acquired from 8 subjects. For 91 pairs of 3D ultrasound volumes selected from these sequences, a mean registration error of 1.8 mm is achieved. A graphics processing unit (GPU) implementation runs the 3D US registration at 8 Hz.

在微創肝臟介入治療中，實時的3D超聲有潛力成為影像導航工具。然而，由於患者呼吸引發的運動使得觀察一個固定點、並與術前的資訊匹配變得很難。該工作中，作者開發了一個快速的彷射框架實時的補償由於呼吸運動引發的肝臟運動偏移。兩個連續的3D立體之間的的彷射配準是通過塊匹配完成的。對於兩個3D立體中均勻分佈的點集，作者提出了一種魯棒性的離群排斥方法，拒絕錯誤的匹配資訊。inliers（也就是正確的匹配資訊）被用於確定仿射變換矩陣。作者從8位受試者中採集到了13個超聲序列，從這些序列中選擇了91對3D立體，實現了1.8mm的配準誤差。此外，作者還採用了GPU對3D超聲之間的配準進行加速，最終實現了8Hz的速度。

2. 背景

1.1 臨床動機

Replacing classical surgical interventions by minimally invasive alternatives is beneficial for the patient and the health care system, as it has large potential for reducing complication rates, minimizing surgical trauma, and reducing hospital stay. The minimally invasive character, however, makes these interventions challenging for the clinician. There is no direct eyesight on the target region and conventional interventional imaging modalities have limited capabilities. Furthermore the user interfacing and interaction with the equipment involved often is not ergonomically well-designed, and does not match the interventional work flow well. Image guidance is crucial in minimally invasive interventions. Image guidance can be based on preoperative imaging data (mostly magnetic resonance imaging and computed tomography) or intraoperative imaging data (X-ray, ultrasound). Hybrid approaches can also be useful, in which the diagnostic quality of preoperative images can be combined with the real-time nature of intraoperative images (also known as fusion imaging).

Four dimensional (4D) ultrasound (US) is a relatively novel imaging modality that currently is mainly used for diagnosis.
用微創取代傳統的手術干預對患者和健康監護系統非常有利，因為微創手術可以降低病症的發生率、最小化手術創傷和減少住院時間。然而，由於微創手術本身的原因，使得這種介入方式在臨床上面臨挑戰。醫生並不能看見目標區域，傳統的介入成像模式更是能力有限。此外，用於介入手術治療的器械也與手術流程不相匹配。影象導航在微創介入手術中非常重要。影象導航可以基於術前的影象資料（主要是MR和CT）以及術中影像資料（X-ray，超聲）。混合影像的方法也是非常有用的，將術前影像和術中影像聯合使用（就是影象配準+影象融合）。4D超聲是一種非常新的成像模態，當前主要用於診斷中。

Radiofrequency ablation (RFA) and the transjugular intrahepatic portosystemic shunt (TIPS) procedure are examples of percutaneous minimally invasive image-guided interventions which are used more and more as alternative to surgical procedures. 4D US has large potential in assisting the clinicians in these procedures, as it provides real-time three dimensional (3D) vision. During these procedures the clinician often holds the US probe steady to visualize a localized area in the liver US volume. Breathing shifts the region of interest and makes it difficult to constantly focus on a region of interest, more so in the presence of a catheter. The purpose of our work is to develop a technique suited for fast 3D ultrasound registration during image guided minimally invasive intervention to compensate breathing motion. In addition, our approach would help in keeping the registration up to date in US fusion imaging.

1.2 相關工作

理解一下影象配準技術：

Image registration is the process of aligning two or more frames of the same or similar scene. The basic input data to the
registration process are two images: the fixed image and the moving image.These approaches are either feature-based
or intensity-based. Intensity-based methods compare intensity patterns in images via similarity metrics, registering either images or sub images. If sub images are registered, centers of corresponding sub images are treated as corresponding feature points. 影象配準是指對齊兩個或多個具有相同或相似的場景的幀。配準過程需要輸入兩張影象，參考影象和移動影象。這一類方法可以分為基於特徵和基於灰度的配準方法。基於灰度的方法需要通過相似性度量計算兩個影象灰度模式之間的距離。配準可以再影象和影象子塊之間進行。

• 相似性度量

誤差平方和 Sum of Square Different SSD; 絕對誤差和 Sum of absolut Different, SAD; 歸一化互相關 Normalization Cross-correlation. 互資訊 Mutual Information, MI.

互資訊提出文獻：Mattes, D., Haynor, D.R., Vesselle, H., Lewellyn, T.K., Eubank, W., 2001. Nonrigid multimodality image registration, pp. 1609–1620. http://dx.doi.org/10.1117/12.431046.

互資訊最適合用於US-US配準：Kaar, M., Figl, M., Hoffmann, R., Birkfellner, W., Hummel, J., Stock, M., Georg, D., Goldner, G., 2013. Automatic patient alignment system using 3D ultrasound. Med. Phys. 40. http://dx.doi.org/10.1118/1.4795129.

Vijayan研究了4D超聲序列離線分段配準的工作：Vijayan, S., Klein, S., Hofstad, E., Lindseth, F., Ystgaard, B., Lang, T., 2013. Validation of a non-rigid registration method for motion compensation in 4D ultrasound of the liver. In: IEEE ISBI2013. Work done in collaboration with NTNU/SINTEF, Trondheim, Norway.

• 特徵描述

基於屬性向量：Foroughi, P., Abolmaesumi, P., Hashtrudi-Zaad, K., 2006. Intra-subject elastic registration of 3D ultrasound images. Med. Image Anal. 10, 713–725.

基於資訊理論的特徵描述子：Wang, Z.W., Slabaugh, G.G., Unal, G.B., Fang, T., 2007. Registration of ultrasound images using an information-theoretic feature detector. In: ISBI, pp. 736–739.

3D SIFT描述子：

Ni, D., Chui, Y.P., Qu, Y., Yang, X.S., Qin, J., Wong, T.T., Ho, S.S.H., Heng, P.A., 2009. Reconstruction of volumetric ultrasound panorama based on improved 3D sift. Comp. Med. Imag. Graph. 33, 559–566.

Skibbe, H., Reisert, M., Schmidt, T., Brox, T., Ronneberger, O., Burkhardt, H., 2012. Fast rotation invariant 3D feature computation utilizing efficient local neighborhood operators. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1563–1575.

區域性相位描述子：Grau, V., Becher, H., Noble, J.A., 2007. Registration of multiview real-time 3-D echocardiographic sequences. IEEE Trans. Med. Imag. 26, 1154–1165.
基於混合特徵配準：Cifor, A., Risser, L., Chung, D., Anderson, E., Schnabel, J., 2013. Hybrid feature-based diffeomorphic registration for tumor tracking in 2-D liver ultrasound images. IEEE Trans. Med. Imag. 32, 1647–1656. http://dx.doi.org/10.1109/ TMI.2013.2262055.

在最近幾年基於灰度強度的配準方法在所有的基於特徵配準裡面是最火的。作者認為，從資訊理論角度來看，只要是特徵提取就一定會存在資訊損失。但是，如果考慮到資料噪聲比較嚴重或者目標發生了形變，開發抵抗這種不變性的特徵描述子還是非常重要的。有效的特徵可以提升不同目標的判別能力，同時降低資料的維度。目前也存在一些實時3D-3D配準的工作。

基於小波特徵的多視角融合：Rajpoot, K., Noble, J.A., Grau, V., Szmigielski, C., Becher, H., 2009. Multiview RT3D echocardiography image fusion. In: FIMH, pp. 134–143.

SSD+灰度特徵：Øye, O.K., Wein, W., Ulvang, D.M., Matre, K., Viola, I., 2012. Real time image-based tracking of 4D ultrasound data. In: MICCAI (1), pp. 447–454.

特徵對齊：Schneider, R.J., Perrin, D.P., Vasilyev, N.V., Marx, G.R., del Nido, P.J., Howe, R.D., 2012. Real-time image-based rigid registration of three-dimensional ultrasound. Med. Image Anal. 16, 402–414.

2.3 靈感來源及貢獻

Our ultrasound registration approach is motivated by methods described in ultrasound speckle tracking literature, see Harris et al. (2010). A speckle pattern contains densely positioned targets created by the interaction of ultrasonic beams and the tissue.
In an ideal scenario, if all the speckle patterns are tracked accurately, the speckle pattern correspondences can be used to estimate the transformation. However in practice not all of the speckle patterns will be tracked well, e.g. because of acoustic shadowing or due to motion decorrelation。

在理想環境中，如果斑點模式可以準確的被跟蹤，我們可以利用斑點間的相關性直接進行變換矩陣估計。然而，在實際中，並不是所有的斑點都可以很好的跟蹤，特別是在聲影和運動去相關環境下。

Liang, T., Yung, L.S., Yu, W., 2013. On feature motion decorrelation in ultrasound speckle tracking. IEEE Trans. Med. Imag. 32, 435–448.

To address these issues and to remove false matches, we employ a matching approach, inspired by game theory, to retain only pairs that have been matched correctly. This outlier rejection is formulated in a game theoretic framework. Speed is an important aspect of our application, and a matching strategy reduces over reliance on selecting the speckle patterns.

為了解決這個問題（去掉由於聲影和運動去相關造成的誤匹配）。作者受到博弈論的啟發，僅保留正確的塊匹配對。並採用博弈論框架剔除誤匹配情況。在實際應用過程中，速度也很重要，作者也採用了一個匹配策略減少了對匹配策略的依賴。

In this work, we propose a novel, fast solution to the 3D ultrasound liver registration problem. Our contributions are fourfold: first, we integrate a fast outlier rejection approach to improve the result of a block-matching approach, second, we develop a method to use both the geometric consistency and the appearance information from block-matching to reject outliers, third, the (non-homogeneous quadratic) optimization function of the outlier rejection module is mapped to a homogeneous quadratic function to be solved efficiently using replicator dynamics, and fourth, we perform an extensive evaluation on real 3D imaging data. Finally, we demonstrate that the method is able to perform registrations at 8 Hz.
這篇文章，作者對3D超聲肝臟配準問題提出了一個全新的、快速的解決方案。作者說他們的貢獻集中在四個方面：

1.作者的演算法集成了一個快速的離群排斥方法來提高塊匹配的結果；

2.作者採用形貌約束和幾何約束設計了離群排斥方法；

3.離群排斥策略中的非齊次二次函式被對映到齊次二次優化問題，並使用 Replicator Dynamics進行求解；

4.作者在真實的3D資料上進行了大量的實驗。

Comment:其實吧，核心問題就一個...如何設計離群排斥演算法。

3. 作者的測試結果

先看一下作者的測試結果：

圖1： 左邊一列是配準之後的結果；中間的是參考影象；右邊的是浮動影象。我們能夠初步得出幾條結論：

1.配準的效果還是有的.... 

2.出現錯誤匹配的情況還確實不在少數...

3.基於塊匹配的結果必然面臨嚴重的邊界效應...