Chytas D, Nikolaou VS. Mixed reality for visualization of orthopedic surgical anatomy. World J Orthop 2021; 12(10): 727-731 [PMID: 34754828 DOI: 10.5312/wjo.v12.i10.727]
Corresponding Author of This Article
Vasileios S Nikolaou, MD, MSc, PhD, Associate Professor, 2nd Department of Orthopedics, National and Kapodistrian University of Athens, 21 Dimitriou Ralli Str, Marousi, Athens 15124, Greece. firstname.lastname@example.org
Checklist of Responsibilities for the Scientific Editor of This Article
This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Author contributions: Chytas D collected the data and wrote the original draft; Nikolaou VS critically reviewed and edited the manuscript.
Conflict-of-interest statement: The authors declare that they have no competing interests
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Vasileios S Nikolaou, MD, MSc, PhD, Associate Professor, 2nd Department of Orthopedics, National and Kapodistrian University of Athens, 21 Dimitriou Ralli Str, Marousi, Athens 15124, Greece. email@example.com
Received: March 18, 2021 Peer-review started: March 18, 2021 First decision: June 7, 2021 Revised: June 16, 2021 Accepted: August 30, 2021 Article in press: August 30, 2021 Published online: October 18, 2021
In the modern era, preoperative planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing the safety and accuracy of surgical interventions. In the field of orthopedic surgery, the increase in safety and accuracy improves treatment quality and orthopedic patient outcomes. Artificial reality technologies, which include virtual reality (VR), augmented reality (AR), and mixed reality (MR), use digital images obtained from computed tomography or magnetic resonance imaging. VR replaces the user’s physical environment with one that is computer generated. AR and MR have been defined as technologies that permit the fusing of the physical with the virtual environment, enabling the user to interact with both physical and virtual objects. MR has been defined as a technology that, in contrast to AR, enables users to visualize the depth and perspective of the virtual models. We aimed to shed light on the role that MR can play in the visualization of orthopedic surgical anatomy. The literature suggests that MR could be a valuable tool in orthopedic surgeon’s hands for visualization of the anatomy. However, we remark that confusion exists in the literature concerning the characteristics of MR. Thus, a more clear description of MR is needed in orthopedic research, so that the potential of this technology can be more deeply understood.
In the modern era, surgical planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing safety and accuracy. Among artificial reality technologies, virtual reality (VR) has been defined as a technology that completely replaces the user’s physical environment with one that is computer generated. Augmented reality (AR) enables specific devices, to fuse digital models with physical objects and allow for interaction with both. MR, like AR, permits fusing of physical with virtual environment, but in contrast to AR, enables users to visualize depth and perspective in the virtual models[2,4]. The models are derived from preoperative images, obtained by computed tomography (CT) or magnetic resonance imaging. Of these technologies, VR and AR allow for adequate visualization of orthopedic surgical anatomy, thus facilitating the performance of several types of orthopedic interventions. The technologies provide surgeons with the ability to visualize patient data in real time, improve preoperative planning, and offer accuracy in performance of interventions, thus leading to upgrades of treatment quality and orthopedic patient outcomes. We aimed to shed light on the role that MR can play in the perception of orthopedic surgical anatomy. We consider that, in contrast with VR and AR, the confusion that exists in the literature impedes the understanding of the value of this technology for the visualization of anatomy in orthopedic surgical procedures.
MIXED REALITY AND VISUALIZATION OF ORTHOPEDIC SURGICAL ANATOMY
In a review of the literature on the implementation of VR, AR, and MR in orthopedics Verhey et al stated that similar to AR, an MR system produces stereoscopic images formed by combining the real world with three-dimensional (3D) virtual models. It was also stated that in MR systems, virtual objects are not simply projected on real ones, as in AR, but the user can interact with both the real and digital objects. The definition, provided by Verhey et al, is different from that provided by Moro et al, according to which, AR does allow for interaction. Also, Verhey et al argued that both MR systems and AR, produce stereoscopic images. Stereoscopic visualization has been defined as the combined view of two digital images seen separately by each eye, using special devices. In contrast, monoscopic visualization comprises digital objects that can be three-dimensionally rotated but are projected on a two-dimensional screen. According to the aforementioned definitions of stereoscopic visualization and AR, it can be noted that stereopsis is not an essential characteristic of AR, thus there is a disagreement with Verhey et al.
Condino et al described an MR-based orthopedic surgery simulator for which hip arthroplasty was chosen as a benchmark for evaluation. The authors performed quantitative tests to “estimate the accuracy of the system by evaluating the perceived position of AR targets”. According to Condino et al, the results of their study supported the use of MR to develop a simulator for orthopedic surgery. However, as can be concluded by the aforementioned purpose of the study, Condino et al did not distinguish MR from AR.
Gregory et al reported a case of a patient who underwent reverse shoulder arthroplasty performed with the aid of an MR headset. The authors noted that the system enabled accurate visualization of the patient’s anatomy, which was beneficial for the safety of the procedure. A postoperative CT scan confirmed the satisfactory position of the prosthesis, and the patient experienced no peri- or postoperative complications (Table 1). Nevertheless, in the introduction of their article, the authors stated that AR is commonly referred to as MR, thus they did not differentiate the two technologies.
Table 1 Studies of implementation of mixed reality for visualization of orthopedic surgical anatomy, with patient outcomes.
Effects of MR on visualization of orthopedic surgical anatomy
The patient's virtual bone, as well as the other anatomical structures, were accurately superimposed on the patient’s body
The range of motion for the hip joint was within the normal range, while the patient’s recovery was reported to be good and he was discharged without obvious surgical complications
CT: Computed tomography; MR: Mixed reality.
Wu et al reported a case of a patient with traumatic high paraplegia who underwent a complicated cervical spine fracture procedure with the use of MR technology. The authors noted that the MR system enabled the surgeon to clearly visualize the anatomy in the operative field, and that CT with 3D reconstruction could not adequately depict neuronal and vascular components around the fracture. However, in the introduction, the authors defined MR as “the merging of the real world and the virtual world,” and did not explain the difference between MR and AR.
Wei et al evaluated the clinical outcome of MR-assisted percutaneous kyphoplasty to treat an osteoporotic vertebral compression fracture with intravertebral vacuum cleft. It was concluded that percutaneous kyphoplasty assisted by MR provided the surgeon with accurate guidance to the intravertebral vacuum cleft area during the operation. A group of patients who underwent MR-assisted percutaneous kyphoplasty to treat an osteoporotic vertebral compression fracture with intravertebral vacuum cleft was compared with a group who underwent the same procedure with traditional C-arm fluoroscopy instead of MR. Vertebral height improvement, cement diffusion, and pain relief were significantly improved by MR assistance (Table 1). The authors stated that MR is a combination of AR and VR and that it permits accurate combination of virtual objects with the real world, without further explanation.
Wu et al assessed the safety and accuracy of pedicle-screw placement in a 3D printed model of an upper cervical spine fracture under MR-based navigation. The authors noted that MR could effectively help surgeons visualize intraoperative anatomy, especially in complex cases involving the upper cervical spine. The authors highlighted the advantages of MR, which “generates computer graphics onto the holographic display of real scenes”, and cited a study by Volonté et al. However, Volonté et al dealt with AR and not MR technology.
A study by Gu et al included patients who were randomly divided in two groups. The first with MR-based lumbar pedicle-screw placement and the second with traditional screw placement. The implantation accuracy was significantly better in the first group than in the second one. Also, there was significantly less bleeding, shorter operative time, and faster recovery in the first group. One month postoperatively, the pain scores were significantly better in the first than in the second group (Table 1). The authors defined MR as a technology that combines virtual with physical objects, without further clarification.
Lei et al performed a complicated total hip arthroplasty combining 3D printing technology with MR. It was noted that the virtual bone and other anatomical structures were accurately superimposed on the patient’s body. Postoperatively, the range of motion for the hip joint was within the normal range, the patient’s recovery was reported to be good, and he was discharged without obvious surgical complications (Table 1). The authors stated that “the unsatisfied accuracy of registration in MR technology is an urgent problem yet to be resolved” and cited an article by Fida et al. However, Fida et al reviewed the use of AR in open surgery, and both Lei et al and Fida et al used the terms “AR” and “MR” interchangeably.
According to the literature, MR can be a valuable tool in the orthopedic surgeon’s hands for visualization of anatomy. Although the two technologies are distinct, the interchangeable use of the terms “AR” and “MR” in the orthopedic surgery literature does not permit researchers and surgeons to extract safe conclusions about the possible superiority of AR or MR. Because MR has been defined as a technology that provides depth and perspective in the virtual environment, in contrast to AR[2,4], it seems that the two technologies may have different values in perceiving orthopedic surgical anatomy. The literature suggests that the two technologies may have different anatomy teaching potential. Currently, there is a lack of research to permit comparison between AR and MR in terms of their value in orthopedic surgical practice. The possible difference between the value of two technologies needs further investigation, which should proceed with a clear description of the technology under investigation and with differentiation between AR and MR.