Development of tomographic reconstruction for three-dimensional optical imaging: From the inversion of light propagation to artificial intelligence

Abstract

Optical molecular tomography (OMT) is an imaging modality which uses an optical signal, especially near-infrared light, to reconstruct the three-dimensional information of the light source in biological tissue. With the advantages of being low-cost, noninvasive and having high sensitivity, OMT has been applied in preclinical and clinical research. However, due to its serious ill-posedness and ill-condition, the solution of OMT requires heavy data analysis and the reconstruction quality is limited. Recently, the artificial intelligence (commonly known as AI)-based methods have been proposed to provide a different tool to solve the OMT problem. In this paper, we review the progress on OMT algorithms, from conventional methods to AI-based methods, and we also give a prospective towards future developments in this domain.

Key Words: Optical molecular tomography, Deep learning, Artificial intelligence, Light propagation based algorithm, Tomographic reconstruction

Core Tip: Most of the existing review articles about optical molecular tomography (OMT) focus on the traditional light propagation model-based algorithm, which possesses ill-posedness and ill-condition and the reconstruction result is unsatisfactory. The emergence of deep learning has brought OMT into the era of artificial intelligence, which can obtain a highly accurate reconstruction result. This article systematically reviews the development of tomographic reconstruction for OMT, which involves the light propagation model-based OMT algorithm and machine learning-based OMT algorithm. The challenges and perspectives of these machine learning-based algorithms are given at the end of the article.

INTRODUCTION

Optical molecular imaging (OMI) is the technology of using optical imaging instruments to detect biological tissues in organisms. In the time since Roger Yonchien Tsien reported that the tumor of a mouse could be resected under the guidance of fluorescence microscopy, winning the Nobel Prize in 2008, OMI has achieved rapid development, especially in recent years. With the advantages of high imaging sensitivity, tissue specificity, relatively short acquisition time and low cost, OMI has been successfully applied to many research fields, including - but not limited to - gene expression, tumor detection, drug development, and therapy evaluation[1-12]. However, OMI can only provide a two-dimensional image, which lacks deeper information and cannot describe the 3D distribution of the optical signal in an imaging object. Thus, researchers have proposed a series of 3D imaging methods, which can be named as optical molecular tomography (OMT).

In fact, OMT can be further divided into several subtypes, such as bioluminescence tomography, Cerenkov luminescence tomography (CLT), fluorescence molecular tomography (FMT), diffuse optical tomography, X-ray luminescence computed tomography (commonly referred to as XLCT), and so on[13-18]. The main difference between them is the means of producing the optical signal. For example, in CLT, the optical signal is emitted during the decay of a radionuclide probe, and in XLCT, high energy X-ray photons are used to excite X-ray excitable nanophosphors which emit the optical signal. Although the way of producing light signal varies, the reconstruction methods for these modalities can be concluded as one unified framework, as shown in Figure 1. It should be noted that anatomical information is essential for OMT, and in most cases, it is provided by X-ray computed tomography or magnetic resonance imaging[19-21]. Finally, the 3D distribution of the optical signal in the imaging object can be obtained, and the light source can then be located based on the reconstruction result. It is obvious that the core component of the framework is the OMT algorithm, which can determine the quality of the final reconstruction result.

Figure 1 Main flowchart of optical molecular tomography. 3D: Three-dimensional; MRI: Magnetic resonance imaging; OMT: Optical molecular tomography; XCT: X-ray computed tomography.

In this review, we summarize recent progress on the OMT algorithm in two aspects: the traditional light propagation model-based way and machine learning-based way. Subsequently, we will provide a prospect towards future developments in a machine learning-based way for OMT.

LIGHT PROPAGATION MODEL-BASED OMT ALGORITHM

The accuracy of the traditional OMT algorithm is dependent on the description of photon propagation in biological tissue. The most popular light transfer model for OMT is the radiative transfer equation (RTE) from Maxwell’s equations[22-25]. Although RTE can accurately depict photon propagation in diffusive media, it is a complicated integro-differential equation, and the computational time and memory requirements are extremely expensive. As a result, RTE is commonly simplified as the diffusion equation (DE, the lower-order approximation of the RTE)[26-28] and N^rd order is a simplified spherical harmonics function (SP3, the high-order approximation of RTE, and in most cases, N equals 3)[29-31]. After introducing the boundary condition, the simplified RTE can be solved using the finite element method[32-34] and the OMT problem can be linearized as the following weight matrix equation[28,25-37]: AX = Φ^measure, where X ≥ 0 (Eq. 1) "where A" denotes the optical transport system matrix, X is the unknown distribution of the optical source and Φ^measure represents the luminous flux of the vertices. As Φ^measure can only be collected on the surface of an imaging object, the goal of OMT can be regarded as the determination of the 3D luminescence source distribution X from boundary measurements Φ^measure based on the formulation of Eq. 1, and this is a typically inverse problem. It should be noted that the number of measurements is often substantially less than the number of unknowns, making the inverse reconstruction an ill-conditioned problem.

Up until now, many methods have been developed to address the limitation mentioned above to make the OMT algorithm strong and robust. These methods can be roughly divided into two categories. The first one is the priori information-based method. In these methods, a priori information is first inferred according to the surface light power distribution and the heterogeneous structure of the imaging object, and then is used as the permissible source region. The aim of using a priori information is to constrain the unknown sources in the region where the sources may exist, resulting in the reduction of the amount of unknown source locations. Many numerical and in vivo experiments have been conducted, and the results indicate that the size of the permissible source region can significantly affect the reconstruction quality[37-46]. It is obvious that the smaller the permissible source region, the more stable the reconstruction results. The main obstacle of the priori information-based methods is that the prior information about the permissible source region cannot always be obtained in advance, especially for the early diseased tissue which cannot be distinguished from anatomical information. Figure 2 shows the reconstruction results with a priori information[47].

Figure 2 Reconstruction results with a priori information. A and B: The axial and sagittal views of single photon emission computed tomography/computed tomography imaging, and an implanted light source is inserted into a mouse; C: The axial-view result of the reconstructed source. These images are reproduced from[47].

The second one is the posteriori information-based method. In these methods, the whole object is used as the initial permissible source region, and the permissible region is updated by selecting the elements where the reconstructed energy is relatively higher than others[48-53]. As the posteriori information-based method avoids the segmentation of the permissible source region from anatomical information, it has superior generalization performance than the priori information-based method, and most of the recent studies are focused on optimizing it[54-58]. Besides the above methods for OMT, the reconstruction accuracy can also be improved by increasing the number of detectable measurements[46,59-66]. For example, in FMT, the quality of the reconstructed results can be improved with the increasing number of measurement data. In CLT, multispectral images can be acquired using a group of filters and the result can be improved significantly. The drawback of this method is that the more optical signal data are acquired, the more time is consumed. However, these traditional light propagation model-based methods are still limited to their reconstruction accuracy, and the main reason is that the simplified RTE cannot accurately describe the process of photon propagation. Thus, more effective methods to improve the reconstruction quality of OMT are still required. Figure 3 shows the reconstruction results with posteriori information[62,67].

Figure 3 Reconstruction results with posteriori information. A: The luminescence distribution in the body; B and C: The three-dimensional results and the results of the local enlarged image in the local area of the liver; C: The images of the capillary acquired using six filters; D, E: The trans-axial multispectral-Cerenkov luminescence tomography reconstructed image of the capillary filled with ³²P-ATP at a 9 mm depth. A and B are reproduced from[67], while C and D are reproduced from[62].

MACHINE LEARNING-BASED OMT ALGORITHM

With the development of artificial intelligence (AI), machine learning algorithms, especially deep learning-based technologies, have gained stunning successes at solving difficult and previously unsolved computational problems in many fields, such as computer vision, natural language processing, speech recognition, and so on[68-71]. The great success of AI has also attracted the attention of researchers in the field of OMT. Based on multilayer perceptrons (commonly known as MLPs), Gao et al[72] proposed a data-driven-based strategy for OMT. As the machine learning-based method requires large amounts of data to train the network, Molecular Optical Simulation Environment software[73] is adopted to produce the simulation data. The experimental results showed the proposed method can greatly improve the reconstruction quality compared with conventional approaches. Subsequently, based on the convolutional neural network and recurrent neural network, Guo et al[74] proposed a framework for FMT reconstruction. The input of this method is two-dimensional fluorescent images, which can avoid errors caused by mesh registration in conventional methods. Zhang et al[75] used MLPs to solve the CLT problem, and the complex relationship between the surface optical signal and the true photon source has been learned by the network. Meng et al[76] constructed a K-nearest neighbor-based locally connected network (KNN-LCN) for FMT. In their work, KNN-LCN cascades a fully connected (referred to as FC) sub-network with a locally connected (referred to as LC) sub-network, where the FC part provides a coarse reconstruction result and the LC part fine-tunes the morphological quality of the reconstructed result. Compared to the traditional light propagation model-based methods, the biggest advantage of the machine learning-based method is that it can directly fit the nonlinear relationship between an object surface optical density and its internal luminescence source. Figure 4 shows the structure of the networks used in OMT reconstruction[72,74-76].

Figure 4 Structure of the networks used in optical molecular tomography reconstruction. A: Multilayer perceptron-based bioluminescence tomography reconstruction network reproduced from[72]; B: Convolutional neural network-recurrent neural network-based fluorescence molecular tomography (FMT) reconstruction framework reproduced from[74]; C: Multilayer fully-connected neural network based on Cerenkov luminescence tomography reproduced from[75]; D: K-nearest neighbor-based locally connected network based on FMT reproduced from[76].

CONCLUSION

Although the machine learning-based OMT algorithm can obtain a more accurate reconstruction result than the traditional light propagation model-based algorithm, further application is still limited and requires more theoretical research. One reason is that the network trained for one object cannot be used for others, and if the object is changed, another network with different parameters should be built and its training will cost a lot of time. Another reason is that there is no ideal method that can explain the mechanism of such a neural network. The solution to the above two limitations is the development direction for future research. In addition, there are many environmental, dietary, and other factors that influence the microbiome, immune system, and pathogenic mechanisms. The recent studies on molecular pathological epidemiology have provided a powerful tool which can pathologically, epidemiologically investigate those factors in relation to molecular pathologies, immunity, and clinical outcomes[77], and it is believed that the molecular pathological epidemiology research can be a promising direction and in which OMT can take a big role.