Rui Liao’s work on patient-specific 3-D model guidance for interventional and hybrid-operating-room applications

Abstract

Compared to surgery, interventional and hybrid-operating-room (OR) approaches diagnose or treat pathology with the most minimally invasive techniques possible. By minimizing the physical trauma to the patient, peripheral or hybrid approaches can reduce infection rates and recovery time as well as shorten hospital stays. Minimally invasive approaches therefore are the trend and often the preferred choice, and may even be the only option for the patients associated with high surgery risks. Common interventional imaging modalities include 2-D X-ray fluoroscopy and ultrasound. However, fluoroscopic images do not display the anatomic structures without a contrast agent, which on the other hand, needs to be minimized for patients’ safety. Ultrasound images suffer from relatively low image quality and tissue contrast problems. To augment the doctor’s view of the patient’s anatomy and help doctors navigate the devices to the targeted area with more confidence and a higher accuracy, high-resolution pre-operative volumetric data such as computed tomography and/or magnetic resonance can be fused with intra-operative 2-D images during interventions. A seamless workflow and accurate 2-D/3-D registration as well as cardiac and/or respiratory motion compensation are the key components for a successful image guidance system using a patient-specific 3-D model. Dr. Liao’s research has been focused on developing methods and systems of 3-D model guidance for various interventions and hybrid-OR applications. Dr. Liao’s work has led to several Siemens products with high clinical and/or market impact and a good number of scientific publications in leading journals/conferences on medical imaging.

Key Words: Angiography and fluoroscopy, Augmented reality, Hybrid-operating-room applications, Image-guided interventions, Motion compensation, 2-D/3-D fusion, 2-D/3-D registration, X-ray

INTRODUCTION AND EDUCATIONAL EXPERIENCE

Dr. Rui Liao is a senior research scientist at Siemens Corporate Research, Princeton, NJ USA. She received a B.Sc in Electrical Engineering and Automation from Tianjin University (formerly known as Bei Yang University, the first university in China) in 1997, with the highest honor. She obtained her MSc from Nanyang Technological University (Singapore) in 2000, and her PhD from Duke University (USA) in 2004, both in Electrical Engineering (minor in Biomedical Engineering). Dr. Liao was then employed as a research scientist and later a senior research scientist by Siemens Corporate Research, Princeton, NJ, USA. Her research interests are in the broad area of signal/image processing and biomedical imaging, with the focus on image-guided interventions and hybrid-OR applications.

Dr. Liao was interviewed by Siemens Picture of The Future (PoF) Magazine (http://www.siemens.com/innovation/en/publications/pof_fall_2010/demographic_change/cardio.htm) for her instrumental contribution to the Aortic ValveGuide technology that won EACTS Techno-College Innovation Award 2010 and Siemens Top+ Innovation Award 2010 (http://www.ehealthnews.eu/siemens/2259-siemens-wins-2010-techno-college-innovation-award). Dr. Liao has co-authored over 60 journal/conference papers and patent applications, and received the Young Investigator Award on ISBI 2004, and co-authored the runner-up of Young Scientist Award on MICCAI 2009. Dr. Liao is currently on the editorial board of The Open Biomedical Engineering Journal and World Journal of Radiology, and has consistently been on the reviewing committee for major journals and conferences in medical imaging and image processing.

ACADEMIC STRATEGY AND GOALS

Over the last 6 years, Dr. Liao’s research has been focused on image processing and medical imaging for image-guided interventions and hybrid-OR applications. During interventions, patient-specific medical images are acquired intra-operatively to monitor the procedures, and to provide road maps that allow the interventional radiologist to insert the interventional devices via small incisions in the patient and navigate them through the body to the target area with disease. Minimally invasive approaches have rapidly gained popularity in recent years. For example, percutaneous stenting has now largely replaced coronary artery bypass grafts (once the gold standard), and there is a huge movement from open heart valve surgery toward minimally invasive trans-catheter heart valve repair. Dr. Liao proposed multiple innovative 2-D/3-D registration and motion compensation technologies to bring patient-specific 3-D models into the operating room, including learning-based, spine-based, curve-based, graph-based, location constraint-based, lasso-based, motion model-based, and hybrid models. The proposed methods significantly improved the robustness and accuracy of 2-D/3-D fusion for various applications, which in turn provides more accurate anatomical details and further enhances physicians’ confidence in device navigation and deployment during interventions. Dr. Liao has led and/or participated in multiple 2-D/3-D fusion prototyping projects for X-ray and magnetic resonance (MR)-based interventions and hybrid-OR applications, including trans-catheter aortic valve implantation (TAVI), abdominal aortic aneurysm (AAA) stenting, electrophysiology radio-frequency catheter ablation (RFCA), and chronicle total occlusion (CTO) revascularization.

ACADEMIC ACHIEVEMENTS

2-D/3-D fusion and motion compensation for TAVI

Aortic valve disease affects 1.8% of the global population and is the most frequent heart valve disease in Western countries, leading to 60 000 surgical aortic valve replacements every year in Europe and even more in the United States[1]. TAVI is a new and breakthrough minimally invasive surgery. During trans-apical TAVI, an antegrade access is used where the catheter and the prosthesis are inserted via small incisions in the chest and the apex of the left ventricle. During trans-femoral TAVI, the catheter is inserted retrogradely via the femoral artery and the aortic arch. Both approaches require X-ray angiographic and fluoroscopic imaging to guide the procedure.

Prior to implantation it is important to angulate the C-arm of the interventional X-ray machine with respect to the patient’s aortic root anatomy. Usually, an appropriate angulation is achieved with iterated C-arm angulations, each followed by an angiogram with approximately 15 mL contrast agent to double-check the aortic root position. Further angiograms are needed later on for correct prosthesis positioning and for functional control after implantation. Recently, we proposed to support TAVI procedures using 3-D models from interventional C-arm computed tomography (CT) volume by overlaying the 3-D aortic model onto fluoroscopy to provide anatomical details and more automatic and accurate C-arm angulation for optimal valve deployment[2].

A hybrid 2-D/3-D registration method to align the 3-D model and 2-D angiography for TAVI procedures was proposed in[3] (Figure 1). The proposed hybrid method incorporates the segmentation and landmark information of the 3-D aortic root into intensity-based registration for highly accurate and robust 2-D/3-D alignment of the aorta. Both the 3-D volume and the 2-D images are captured with contrast injection showing the patient’s aortic root. 2-D angiographic images are first pre-processed to remove the background and/or devices such as the catheter and TEE probe. 3-D aorta segmentation and coronary ostia landmark detection is performed on the 3-D C-arm CT volume using the learning-based method presented in[4]. Aorta segmentation is then used to produce clean digitally reconstructed radiographs (DRRs) that show only the aorta and excludes all the peripheral structures such as the spine. Landmarks representing the left and right coronary ostia are further utilized in an integrated fashion with the intensity-based method. A multi-layer and multi-resolution optimization strategy is finally deployed to find the optimal registration.

Figure 1 Registration of 3-D volume and 2-D angiography of a patient’s aorta[3]. A: The original 3-D volume; B: Aorta segmentation and coronary ostia landmark detection; C: Aortic angiography; D: Overlay of aortic angiography with the contour of the 3-D model after registration.

Our system needs to be set up in the complex environment of a hybrid operating room and used by a physician during a highly complicated hybrid-OR application that involves a large number of staff, equipment and steps. Therefore, it is crucial for the acceptance of such a system to be fast, to minimize the user interaction and to allow table-side control. Fast and automatic contrast detection in the aortic root on X-ray images facilitates a seamless workflow to utilize the 3-D models by triggering 2-D/3-D registration automatically when motion compensation is needed. In[5], we proposed a novel method for automatic detection of contrast injection in the aortic root on fluoroscopic and angiographic sequences. The proposed method is based on histogram analysis and likelihood ratio test, and is robust to variations in the background, the density and volume of the injected contrast, and the size of the aorta. The performance of the proposed algorithm was evaluated on 26 sequences from 5 patients and 3 clinical sites, with 16 out of 17 contrast injections correctly detected and zero false detections.

2-D/3-D Fusion and motion compensation for RFCA

Atrial fibrillation (AFIB) is a leading cause of stroke and one of the most common heart rhythm disorders. RFCA has become an accepted option for treating AFIB in today’s electrophysiology (EP) labs, especially if drug treatment has become ineffective. In order to guide the process of finding the site of origin where the excess firing of cells occur, 2-D X-ray fluoroscopy has been routinely used to provide real-time monitoring of EP procedures. Unfortunately X-ray projection images cannot distinguish soft tissue well. To address this issue, fused visualization combining pre-operative high-resolution 3-D atrial CT and/or MR volumes with the fluoroscopic images has been developed (fluoro overlay image guidance)[6].

Registration between 3-D volumes and 2-D fluoroscopic images for EP is a challenging task due to the fact that there are few discernable features in a typical EP fluoroscopic image. Rhode et al[7] proposed to use multimodality fiducial skin markers to register MR volumes with X-ray images. This method, however, requires specialized markers as well as a specialized workflow to guarantee that the markers are applied at the same position between the pre- and intra-operative data. We further proposed to utilize the coronary sinus (CS) and the catheter placed inside the CS as a location constraint to perform 2-D/3-D registration[8]. This method is not applicable when a CS catheter is not used during the procedure. In[9] we designed an accurate and workflow-friendly spine-based 2-D/3-D registration technique to align C-Arm CT volume and fluoroscopic images to compensate for patient movement in EP (Figure 2). The spine is a natural landmark with rich geometric variations that can be used for accurate registration. The proposed method can be applied when C-Arm CT volume is available for bringing the pre-operative CT/MR volume into alignment with the patient via 3-D/3-D registration, or when C-Arm CT is used in place of CT/MR for interventional guidance. To our knowledge, this is the first fully-automatic intensity-based 2-D/3-D registration method that is designed and tested for EP, without the requirement for contrast agent injection.

Figure 2 Spine-based 2-D/3-D registration examples for electrophysiology[9]. A: Spine-enhanced fluoroscopy with the edge contours depicted; B: Digitally reconstructed radiographs (DRRs) before registration overlaid with the edge contours from fluoroscopy; C: DRRs after registration overlaid with the edge contours from fluoroscopy; D: 2-D/3-D overlay used for electrophysiology procedures. The 3-D mesh model is generated from the segmentation of the left atrium and pulmonary veins, and is overlaid with fluoroscopy by the proposed spine-based 2-D/3-D registration technique.

Static overlay techniques do not follow the heart while it beats and moves through the breathing cycle. This effect may impair the utility of fluoroscopic overlays for catheter navigation. By synchronizing the fluoroscopic images with an electrocardiogram (ECG), cardiac motion can be eliminated, and respiratory motion can be isolated for the fused visualization. While it has been widely recognized that motion compensation is crucial for fluoroscopic overlays, image-based 3-D motion-compensation methods designed for EP applications are very challenging because there are few discernible features in EP fluoroscopic images. Motion compensated navigation for coronary intervention using magnetic tracking was suggested in[10], but it requires special catheters equipped with an electromagnetic sensor at increased cost. Only vertical motion in the imaging plane was compensated in[11] and[12] for liver embolization and hepatic artery catheterization, respectively. In general, though, this is not sufficient for EP breathing motion compensation because the left atrium undergoes three dimensional motions during respiration, as shown in[13].

In[14-16] we described an image-based method to detect and compensate respiratory motion in 3-D for EP procedures using a biplane C-arm fluoroscopy device (Figure 3). Such an X-ray system has two imaging planes commonly referred to as A-plane and B-plane, respectively. To perform motion estimation, we track a commonly available EP catheter, the ring-shaped lasso catheter. Since this device is often used for pulmonary vein (PV) ablations, there is no need for additional instruments or fiducial markers. In addition, the lasso catheter is a unique shape and is one of the most prominent structures shown in EP fluoroscopy, representing a good feature for robust tracking. During isolation of the four PVs during AFIB ablation, the lasso catheter is typically fixed at the ostium of the PV that is to be ablated. By tracking the lasso catheter, we can obtain a motion estimate right at the ablation site. The motion estimation takes place directly in 3-D and not in 2-D. Once an estimate of the 3-D motion is available, we can apply it to the static fluoroscopic overlay to generate an animated representation. To the best of our knowledge, this is the first image-based method that is specifically designed for the challenging task of 3-D breathing motion compensation during EP applications.

Figure 3 Workflow of image processing and 2-D/3-D registration for lasso catheter-based motion compensation for electrophysiology radio-frequency catheter ablation[15].

A biplane system is less common than a monoplane system due to its much higher cost, and involves increased radiation for both the patient and the clinician[34]. In[17], we further investigated the feasibility of tracking in the monoplane fluoroscopy the circumferential lasso catheter using a B-spline-based method. The tracking results are then used for the estimation of the 3-D breathing motion trajectory and for subsequent motion compensation in a monoplane setup. The performance of the proposed tracking algorithm was evaluated on 340 monoplane frames with an average error of 0.68 ± 0.36 mm. Our contributions to this work are two-fold. First and foremost, we showed how to design an effective, practical, and workflow-friendly 3-D motion compensation scheme for EP procedures in a monoplane setup. In addition, we developed an efficient and accurate method for model-based tracking of the circumferential lasso catheter in low-dose EP fluoroscopy.

2-D/3-D fusion and motion compensation for CTO revascularization

Percutaneous coronary intervention (PCI) of coronary arteries is a common procedure in cardiology intended to provide functional revascularization of stenosed (narrowed) or occluded coronary vessels[18]. Although PCI is commonly performed on patients diagnosed with coronary diseases, many cases remain highly challenging due to the complex 3-D topology of the coronary arteries (e.g. tortuous vessels) and their limited visibility due to stenosed and/or heavily calcified vessels on X-ray fluoroscopy. Hence, navigation in the coronary pathways can be very difficult, since the vessel’s depth is mentally estimated by the cardiologist from a monoplane 2-D image. Biplane fluoroscopic systems are actually proposed to overcome vessel foreshortening and to provide accurate navigation guidance, but their availability in today’s clinical setting and the additional radiation exposure for the patient are two major inconveniences. A third inconvenience during PCI is that contrast agent injection has to be used to highlight the targeted vessel. Ideally, this should be minimized during the overall procedure and used only at key steps of the intervention. Consequently, guidance during PCI relies extensively on the cardiologist’s experience to advance a guide wire without perforating the boundary of the vessel wall, using a monoplane fluoroscopic view.

Preoperative soft tissue imaging is evolving rapidly and becoming more accepted prior to PCI, to provide insights about the vessel 3-D morphology. Multi-slice computed tomography (MSCT) is proposed to delineate in 3-D the vessel centerline, vessel wall and most importantly, the MSCT volumetric intensities corresponding to plaque or calcification. An intra-operative overlay of this information onto fluoroscopy could provide valuable guidance cues to cardiologists currently unavailable on fluoroscopic images.

Two categories of 2-D/3-D registration techniques were proposed for coronary registration: landmark-based and intensity-based registration. Intensity-based registration was previously proposed to align a full coronary artery tree from a biplane system to fuse CT or MR to angiograms[19]. This method can be computationally expensive and more importantly lack the flexibility to target specific structures to align. Landmarks can be introduced to focus the registration process on a specific anatomical structure, but are subject to the number, spatial location and proper identification of landmarks. For instance, bifurcation points of the coronary artery tree are generally used for that purpose[20], but are not always visible due to vessel foreshortening.

In[21], we proposed a novel registration workflow involving specific coronary artery branches to align MSCT acquisition with X-ray fluoroscopy (Figure 4). Since coronary vessels are long, continuous and elongated structures, a curve-based technique is appealing for geometric representation of coronary vessels and has been previously proposed for coronary 3-D reconstruction[22]. In addition, our proposed method incorporates automatic ECG gating for cardiac motion compensation, and respiratory motion is further compensated by rigid body registration of the vessels. The coronary vessels are first segmented from a MSCT volume and corresponding vessel segments are identified on a single gated 2-D fluoroscopic frame. Registration can be explicitly constrained using one or multiple branches of a contrast-enhanced vessel tree or the outline of the guide wire used to navigate during the procedure. Finally, the alignment problem is solved by the Iterative Closest Point algorithm. To be computationally efficient, a distance transform is computed from the 2-D identification of each vessel such that distance is zero on the centerline of the vessel and increases away from the centerline.

Figure 4 Curve-based 2-D/3-D registration for chronicle total occlusion revascularization[21]. A: Curves of the LCx (green) and LAD (red) identified on fluoroscopic images (in bold) using a semi-automatic tool based on the vesselness measure, fast marching, and B-Spline fitting. The 2-D projection of the corresponding curve segments from the 3-D multi-slice computed tomography (MSCT) is shown with an offset (in dashed). The image frame used for registration is selected to match the acquisition cardiac phase of the MSCT volume, expressed by a percentage of the R-R interval from the electrocardiogram; B, C: The distance transform image obtained from the curve identified on the LCx and the LAD; D: Registration results obtained by selecting the LCx and LAD. The mesh model of the segmented MSCT is overlaid on top of the fluoroscopic image after registration.

In[23], we further proposed and validated a PCA-based respiratory motion model for more robust and efficient breathing motion compensation during CTO revascularization. In a preparatory training phase, a preoperative 3-D segmentation of the coronary arteries is automatically registered with a cardiac gated biplane cineangiogram, and then used to build a respiratory motion model. This motion model is subsequently used as a prior within the intra-operative registration process for motion compensation to restrict the search space. Our hypothesis is that the use of this model-constrained registration increases the robustness and registration accuracy, especially for weak data constraints such as low signal-to-noise ratio, the lack of contrast information, or an intra-operative monoplane setting. This allows for reducing radiation exposure without compromising on registration accuracy. Experimentally, we were able to significantly accelerate the intra-operative registration with a 3-D error of less than 2 mm for both monoplane images and intra-procedure settings with missing contrast information based on 2-D guide wire tracking, which makes it feasible for motion correction in clinical procedures.

3-D reconstruction of the coronary artery tree from a rotational X-ray angiography

The gold standard for diagnosis of cardiovascular disease is X-ray coronary angiography, which gives 2-D projection images of the complex coronary artery tree. Therefore, a significant amount of 3-D information of the underlying coronary arteries is lost by the 2-D projection simplification. Traditionally, several angiographic views are taken from different angles and then are selected subjectively based on the cardiologist’s experience, to minimize foreshortening and vessel overlap. However, this is associated with a high intra-observer variability and increased radiation/contrast injection. Green et al[24] found that vessel shortening was markedly greater in expert-recommended views than in computer-generated views.

3-D symbolic reconstruction of coronary arteries from angiographic images was investigated using two projections[25,26], either from a fixed biplane system, or from different projections on a monoplane C-Arm. However, 3-D reconstruction of the complex 3-D topology of coronary vessels using only two 2-D projections is often not sufficient to obtain a precise reconstruction. The extracted 2-D vessel centerlines may not be complete or accurate due to the flow of the dye, low image contrast, or superposition of vessels in a certain view. In addition, whereas typically it is preferred to have two projections orthogonal to each other, this angular distance may not be achievable in a clinical setup when foreshortening and vessel overlap need to be minimized for the whole coronary artery tree in both projections. Another limitation is that corresponding vessels between the two views can be established typically only through user interaction.

The goal of our papers[27,28] is to describe an efficient and robust method for 3-D model generation of the coronary artery tree using all the projections of a rotational X-ray sequence that correspond to the same cardiac phase (Figure 5). A rotational X-ray sequence is acquired by rotating the C-arm with a constant source to intensifier distance, a constant cranio/caudal angle, and a varying left/right anterior oblique angle. Depending on the rotation speed and the heart rate of the patient, a rotational X-ray sequence may cover 5 to 8 cardiac cycles. 2-D coronary artery centerlines are extracted automatically from X-ray projection images using an enhanced multi-scale analysis. For difficult data with low vessel contrast, a semi-automatic tool based on a fast marching method is implemented to allow manual correction of automatically-extracted 2-D centerlines. We then formulated the 3-D symbolic reconstruction of coronary arteries from multiple views as an energy minimization problem incorporating a soft epipolar line constraint and a smoothness term evaluated in 3-D. The proposed formulation results in the robustness of the reconstruction to the imperfectness in 2-D centerline extraction, as well as the reconstructed coronary artery tree being inherently smooth in 3-D. We further proposed to solve the energy minimization problem using α-expansion moves of Graph Cuts[29], a powerful optimization technique that yields a local minimum in a strong sense at a relatively low computational complexity.

Figure 5 Two examples of 3-D reconstruction of the coronary artery tree from an X-ray rotational angiography viewed at different angles[28].

We showed experimental results on a synthetic coronary phantom, a porcine data set and 11 patient data sets. For the coronary phantom, results obtained using a different number of views were presented. 3-D reconstruction error evaluated by the mean plus one standard deviation is below one millimeter when 4 or more views are used. For real data, reconstruction using 4 to 5 views and 256 depth labels averaged around 12 s on a computer with 2.13 GHz Intel Pentium M and achieved a mean 2-D back-projection error of 1.18 mm (ranging from 0.84 to 1.71 mm) in 12 cases. With significant improvements in efficiency, our method is promising in terms of clinical usability. The most straightforward application of our method in a clinical setup would be to provide the reconstructed 3-D coronary model to the interventional cardiologists at the beginning of the PCI procedures. The 3-D model visualization will help clinicians to choose optimal angiographic views to minimize vessel foreshortening, select proper stent type and size, and properly plan the stent placement 3-D locations.

Advanced 2-D/3-D registration: Deformable and learning-based

AAA is the local expansion of the abdominal aorta. There is a risk of rupture of the aneurysm if the expansion becomes large enough, and the chance of post-rupture survival for such patients is low. AAA is currently ranked as the 13th leading cause of death in the U.S. In recent years, minimal-invasive interventional repairs are rapidly emerging as an alternative to open surgeries for the treatment of AAA, especially for patients at increased surgical risk due to age or other medical conditions. Due to the deformable nature of the abdominal organs, there typically exist elastic deformations between pre-operative volumes and intra-operative X-ray images. The insertion of the medical devices into the artery during AAA procedures can further introduce significant deformations to the target vessel that needs to be aligned. Therefore, a rigid-body 2-D/3-D registration scheme is typically not sufficient for AAA stenting applications, and an advanced deformable 2-D/3-D registration method needs to be developed here.

In[30], we proposed to formulate the deformable 2-D/3-D registration problem on a 3-D graph, and further provided an efficient numerical solution to the graph-based formulation (Figure 6). In particular, a 3-D graph is generated from the abdominal aorta segmented from the pre-operative MSCT data, and a 2-D distance map is generated from each of the 2-D X-ray images used for registration. A distance map is a smooth shape encoding of the underlying structures, and by utilizing a distance map, explicit establishment of corresponding points between 2-D and 3-D graphs can be avoided during the optimization. This reduces the optimization space to a much lower dimension. In addition, smoothness calculation is defined on the 3-D graph, the derivative of which can be calculated efficiently using the well-known Laplacian matrix of a graph.

Figure 6 Graph-based deformable 2-D/3-D registration for abdominal aortic aneurysm stenting[30]. A: An example of patient computed tomography data, with abdominal aortic aneurysm segmented and shown in mesh; B: The 3-D graph and the ground truth with synthesized deformation for patient one; C: Registration results of (B); D: The 3-D graph and the ground truth with natural deformation for patient two; E: Registration result of (D).

Specific to the anatomy of the abdominal aorta, a hierarchical registration scheme was further deployed. In particular, the 3-D graph is divided into three segments, renal arteries, iliac arteries, and abdominal aorta. A piecewise rigid-body transformation is first applied individually to the three segments while their connectivity is maintained. Local deformation is then estimated for the complete graph comprising all three segments. We further compared the registration accuracy achieved by using one and two views. We showed that even with the incorporation of a length-preserving term, a single projection image alone can only produce accurate registration in the imaging plane, while two views are required in order to achieve accurate registration in 3-D. The method was validated using both phantom and clinical datasets, and achieved an average error of < 1 mm within 0.1 s. The extremely high efficiency further opens the door to many potential applications. For example, the weight for each term in the energy functional is data-dependent. Our runtime of 0.1 s makes it feasible to obtain the optimal combination of weights interactively for a given dataset, even during interventions. The proposed method is of general form and has the potential to be applied in a wide range of applications requiring efficient deformable 2-D/3-D registration of vascular structures.

In recent years, learning-based methods have been suggested for general medical registration to impose prior knowledge to achieve more robust and reliable registration[31]. Learning-based methods were further extended to 2-D non-rigid image registration in[32] where the Kullback-Leibler divergence (KLD) w.r.t., a prior joint distribution, was minimized together with the maximization of the mutual information measure.

In[33], we proposed a Jensen-Shannon divergence (JSD)-based method for learning-based 2-D/3-D registration (Figure 7). Unlike KLD, JSD is symmetric, bounded, and a true metric, which has triggered its popularity in various applications in recent years, ranging from statistical language analysis, image edge detection and segmentation, to DNA sequence analysis. Therefore, JSD provides a more suitable measure than KLD in quantifying histogram discrepancy because some histogram bins may vanish for the training data but not for the observed data or vice versa, in which case KLD is undefined. Furthermore, JSD is upper-bounded and symmetric, facilitating its easy use as one of the factors in the driving force for registration.

Figure 7 Learning-based 2-D/3-D registration[33]. DRR: Digitally reconstructed radiograph.

Other advantages of our method include: (1) Depending on how well the a priori represents the observed data, the registration process is driven by a compounding effect of the statistical consistency of the observed joint histogram to the learned prior, and the statistical dependence between the individual intensity distributions of the images being registered; (2) There is no requirement on image segmentation and labeling, whose error can lead to further errors in subsequent registration. Instead, automatic nonlinear histogram mapping is done iteratively during the matching process to handle the intensity discrepancy between the observed data and the training data; and (3) An intensity-based histogram is used, which is supposed to result in a higher registration accuracy than the case where a class-based histogram was used in the argument of a higher computational efficiency in the generation of DRRs. Instead we deployed highly efficient GPU-based DRR generation for a speed-up. Experimental results demonstrated that a combination of the prior knowledge and the low-level similarity measure between the images being registered led to a more robust and accurate registration in comparison with the cases where either of the two factors was used alone as the driving force for registration.

ACKNOWLEDGMENTS

I am grateful to all past and present colleagues for their insightful discussion and indispensable contributions to the work presented here. I am thankful to Siemens Corporate Research for providing me with a productive platform to perform all the interesting research presented in the paper.