Editorial Open Access
Copyright ©The Author(s) 2021. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Orthop. Oct 18, 2021; 12(10): 727-731
Published online Oct 18, 2021. doi: 10.5312/wjo.v12.i10.727
Mixed reality for visualization of orthopedic surgical anatomy
Dimitrios Chytas, Vasileios S Nikolaou
Dimitrios Chytas, Department of Physiotherapy, University of Peloponnese, Sparta 23100, Greece
Vasileios S Nikolaou, 2nd Department of Orthopedics, National and Kapodistrian University of Athens, Athens 15124, Greece
ORCID number: Dimitrios Chytas (0000-0003-2746-7121); Vasileios S Nikolaou (0000-0001-7422-4195).
Author contributions: Chytas D collected the data and wrote the original draft; Nikolaou VS critically reviewed and edited the manuscript.
Conflict-of-interest statement: The authors declare that they have no competing interests
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Vasileios S Nikolaou, MD, MSc, PhD, Associate Professor, 2nd Department of Orthopedics, National and Kapodistrian University of Athens, 21 Dimitriou Ralli Str, Marousi, Athens 15124, Greece. vassilios.nikolaou@gmail.com
Received: March 18, 2021
Peer-review started: March 18, 2021
First decision: June 7, 2021
Revised: June 16, 2021
Accepted: August 30, 2021
Article in press: August 30, 2021
Published online: October 18, 2021

Abstract

In the modern era, preoperative planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing the safety and accuracy of surgical interventions. In the field of orthopedic surgery, the increase in safety and accuracy improves treatment quality and orthopedic patient outcomes. Artificial reality technologies, which include virtual reality (VR), augmented reality (AR), and mixed reality (MR), use digital images obtained from computed tomography or magnetic resonance imaging. VR replaces the user’s physical environment with one that is computer generated. AR and MR have been defined as technologies that permit the fusing of the physical with the virtual environment, enabling the user to interact with both physical and virtual objects. MR has been defined as a technology that, in contrast to AR, enables users to visualize the depth and perspective of the virtual models. We aimed to shed light on the role that MR can play in the visualization of orthopedic surgical anatomy. The literature suggests that MR could be a valuable tool in orthopedic surgeon’s hands for visualization of the anatomy. However, we remark that confusion exists in the literature concerning the characteristics of MR. Thus, a more clear description of MR is needed in orthopedic research, so that the potential of this technology can be more deeply understood.

Key Words: Orthopedic surgery, Mixed reality, Anatomy, Augmented reality, Three-dimensional visualization technologies, Artificial reality technologies

Core Tip: Mixed reality could be a valuable tool in orthopedic surgeon’s hands for visualization of anatomy, but a more clear description of this technology is needed in the orthopedic literature.



INTRODUCTION

In the modern era, surgical planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing safety and accuracy[1]. Among artificial reality technologies, virtual reality (VR) has been defined as a technology that completely replaces the user’s physical environment with one that is computer generated[2]. Augmented reality (AR) enables specific devices, to fuse digital models with physical objects and allow for interaction with both[3]. MR, like AR, permits fusing of physical with virtual environment, but in contrast to AR, enables users to visualize depth and perspective in the virtual models[2,4]. The models are derived from preoperative images, obtained by computed tomography (CT) or magnetic resonance imaging[5]. Of these technologies, VR and AR allow for adequate visualization of orthopedic surgical anatomy, thus facilitating the performance of several types of orthopedic interventions[5]. The technologies provide surgeons with the ability to visualize patient data in real time, improve preoperative planning, and offer accuracy in performance of interventions, thus leading to upgrades of treatment quality and orthopedic patient outcomes[5]. We aimed to shed light on the role that MR can play in the perception of orthopedic surgical anatomy. We consider that, in contrast with VR and AR, the confusion that exists in the literature impedes the understanding of the value of this technology for the visualization of anatomy in orthopedic surgical procedures.

MIXED REALITY AND VISUALIZATION OF ORTHOPEDIC SURGICAL ANATOMY

In a review of the literature on the implementation of VR, AR, and MR in orthopedics Verhey et al[5] stated that similar to AR, an MR system produces stereoscopic images formed by combining the real world with three-dimensional (3D) virtual models[5]. It was also stated that in MR systems, virtual objects are not simply projected on real ones, as in AR, but the user can interact with both the real and digital objects. The definition, provided by Verhey et al[5], is different from that provided by Moro et al[3], according to which, AR does allow for interaction. Also, Verhey et al[5] argued that both MR systems and AR, produce stereoscopic images. Stereoscopic visualization has been defined as the combined view of two digital images seen separately by each eye, using special devices[6]. In contrast, monoscopic visualization comprises digital objects that can be three-dimensionally rotated but are projected on a two-dimensional screen[6]. According to the aforementioned definitions of stereoscopic visualization and AR, it can be noted that stereopsis is not an essential characteristic of AR, thus there is a disagreement with Verhey et al[5].

Condino et al[7] described an MR-based orthopedic surgery simulator for which hip arthroplasty was chosen as a benchmark for evaluation. The authors performed quantitative tests to “estimate the accuracy of the system by evaluating the perceived position of AR targets”. According to Condino et al[7], the results of their study supported the use of MR to develop a simulator for orthopedic surgery. However, as can be concluded by the aforementioned purpose of the study, Condino et al[7] did not distinguish MR from AR.

Gregory et al[8] reported a case of a patient who underwent reverse shoulder arthroplasty performed with the aid of an MR headset. The authors noted that the system enabled accurate visualization of the patient’s anatomy, which was beneficial for the safety of the procedure. A postoperative CT scan confirmed the satisfactory position of the prosthesis, and the patient experienced no peri- or postoperative complications (Table 1). Nevertheless, in the introduction of their article, the authors stated that AR is commonly referred to as MR, thus they did not differentiate the two technologies.

Table 1 Studies of implementation of mixed reality for visualization of orthopedic surgical anatomy, with patient outcomes.
Ref.
Operation
Effects of MR on visualization of orthopedic surgical anatomy
Patient outcomes
Gregory et al[8]Reverse shoulder arthroplastyAccurate visualization of the patient’s anatomyA postoperative CT scan confirmed the adequate position of the prosthesis, while the patient experienced no peri- or postoperative complications
Wei et al[10]Percutaneous kyphoplasty to treat an osteoporotic vertebral compression fracture with intravertebral vacuum cleftThe surgeon could obtain accurate guidance to the intravertebral vacuum cleft area during the operationVertebral height improvement, cement diffusion and pain relief were significantly better in the MR group in comparison with the traditional C-arm fluoroscopy group
Gu et al[13]Lumbar pedicle-screw placementThe implantation accuracy with the use of MR was significantly higher in comparison with traditional screw placementSignificantly less bleeding and operative time, faster recovery, significantly better pain scores at 1 month postoperatively with MR, in comparison with traditional screw placement
Lei et al[14]Total hip arthroplastyThe patient's virtual bone, as well as the other anatomical structures, were accurately superimposed on the patient’s bodyThe range of motion for the hip joint was within the normal range, while the patient’s recovery was reported to be good and he was discharged without obvious surgical complications

Wu et al[9] reported a case of a patient with traumatic high paraplegia who underwent a complicated cervical spine fracture procedure with the use of MR technology. The authors noted that the MR system enabled the surgeon to clearly visualize the anatomy in the operative field, and that CT with 3D reconstruction could not adequately depict neuronal and vascular components around the fracture. However, in the introduction, the authors defined MR as “the merging of the real world and the virtual world,” and did not explain the difference between MR and AR.

Wei et al[10] evaluated the clinical outcome of MR-assisted percutaneous kyphoplasty to treat an osteoporotic vertebral compression fracture with intravertebral vacuum cleft. It was concluded that percutaneous kyphoplasty assisted by MR provided the surgeon with accurate guidance to the intravertebral vacuum cleft area during the operation. A group of patients who underwent MR-assisted percutaneous kyphoplasty to treat an osteoporotic vertebral compression fracture with intravertebral vacuum cleft was compared with a group who underwent the same procedure with traditional C-arm fluoroscopy instead of MR. Vertebral height improvement, cement diffusion, and pain relief were significantly improved by MR assistance (Table 1). The authors stated that MR is a combination of AR and VR and that it permits accurate combination of virtual objects with the real world, without further explanation.

Wu et al[11] assessed the safety and accuracy of pedicle-screw placement in a 3D printed model of an upper cervical spine fracture under MR-based navigation. The authors noted that MR could effectively help surgeons visualize intraoperative anatomy, especially in complex cases involving the upper cervical spine. The authors highlighted the advantages of MR, which “generates computer graphics onto the holographic display of real scenes”, and cited a study by Volonté et al[12]. However, Volonté et al[12] dealt with AR and not MR technology.

A study by Gu et al[13] included patients who were randomly divided in two groups. The first with MR-based lumbar pedicle-screw placement and the second with traditional screw placement. The implantation accuracy was significantly better in the first group than in the second one. Also, there was significantly less bleeding, shorter operative time, and faster recovery in the first group. One month postoperatively, the pain scores were significantly better in the first than in the second group (Table 1). The authors defined MR as a technology that combines virtual with physical objects, without further clarification.

Lei et al[14] performed a complicated total hip arthroplasty combining 3D printing technology with MR. It was noted that the virtual bone and other anatomical structures were accurately superimposed on the patient’s body. Postoperatively, the range of motion for the hip joint was within the normal range, the patient’s recovery was reported to be good, and he was discharged without obvious surgical complications (Table 1). The authors stated that “the unsatisfied accuracy of registration in MR technology is an urgent problem yet to be resolved” and cited an article by Fida et al[15]. However, Fida et al[15] reviewed the use of AR in open surgery, and both Lei et al[14] and Fida et al[15] used the terms “AR” and “MR” interchangeably.

CONCLUSION

According to the literature, MR can be a valuable tool in the orthopedic surgeon’s hands for visualization of anatomy. Although the two technologies are distinct, the interchangeable use of the terms “AR” and “MR” in the orthopedic surgery literature does not permit researchers and surgeons to extract safe conclusions about the possible superiority of AR or MR. Because MR has been defined as a technology that provides depth and perspective in the virtual environment, in contrast to AR[2,4], it seems that the two technologies may have different values in perceiving orthopedic surgical anatomy. The literature suggests that the two technologies may have different anatomy teaching potential[16]. Currently, there is a lack of research to permit comparison between AR and MR in terms of their value in orthopedic surgical practice. The possible difference between the value of two technologies needs further investigation, which should proceed with a clear description of the technology under investigation and with differentiation between AR and MR.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Orthopedics

Country/Territory of origin: Greece

Peer-review report’s scientific quality classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Ghannam WM, Yu JC S-Editor: Wang JL L-Editor: Filipodia P-Editor: Wang LYT

References
1.  Teatini A, Kumar RP, Elle OJ, Wiig O. Mixed reality as a novel tool for diagnostic and surgical navigation in orthopaedics. Int J Comput Assist Radiol Surg. 2021;16:407-414.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
2.  Brigham TJ. Reality Check: Basics of Augmented, Virtual, and Mixed Reality. Med Ref Serv Q. 2017;36:171-178.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 37]  [Cited by in F6Publishing: 18]  [Article Influence: 9.3]  [Reference Citation Analysis (0)]
3.  Moro C, Štromberga Z, Raikos A, Stirling A. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ. 2017;10:549-559.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 211]  [Cited by in F6Publishing: 118]  [Article Influence: 42.2]  [Reference Citation Analysis (0)]
4.  Goo HW, Park SJ, Yoo SJ. Advanced Medical Use of Three-Dimensional Imaging in Congenital Heart Disease: Augmented Reality, Mixed Reality, Virtual Reality, and Three-Dimensional Printing. Korean J Radiol. 2020;21:133-145.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 25]  [Cited by in F6Publishing: 16]  [Article Influence: 12.5]  [Reference Citation Analysis (0)]
5.  Verhey JT, Haglin JM, Verhey EM, Hartigan DE. Virtual, augmented, and mixed reality applications in orthopedic surgery. Int J Med Robot. 2020;16:e2067.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 9]  [Article Influence: 9.0]  [Reference Citation Analysis (0)]
6.  Meyer ER, Cui D. Anatomy Visualizations Using Stereopsis: Assessment and Implication of Stereoscopic Virtual Models in Anatomical Education. Adv Exp Med Biol. 2020;1235:117-130.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 4]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
7.  Condino S, Turini G, Parchi PD, Viglialoro RM, Piolanti N, Gesi M, Ferrari M, Ferrari V. How to Build a Patient-Specific Hybrid Simulator for Orthopaedic Open Surgery: Benefits and Limits of Mixed-Reality Using the Microsoft HoloLens. J Healthc Eng. 2018;2018:5435097.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 39]  [Cited by in F6Publishing: 21]  [Article Influence: 9.8]  [Reference Citation Analysis (0)]
8.  Gregory TM, Gregory J, Sledge J, Allard R, Mir O. Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthop. 2018;89:480-483.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 40]  [Cited by in F6Publishing: 28]  [Article Influence: 10.0]  [Reference Citation Analysis (0)]
9.  Wu X, Liu R, Yu J, Xu S, Yang C, Yang S, Shao Z, Ye Z. Mixed Reality Technology Launches in Orthopedic Surgery for Comprehensive Preoperative Management of Complicated Cervical Fractures. Surg Innov. 2018;25:421-422.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 7]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
10.  Wei P, Yao Q, Xu Y, Zhang H, Gu Y, Wang L. Percutaneous kyphoplasty assisted with/without mixed reality technology in treatment of OVCF with IVC: a prospective study. J Orthop Surg Res. 2019;14:255.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 7]  [Article Influence: 2.3]  [Reference Citation Analysis (0)]
11.  Wu X, Liu R, Xu S, Yang C, Yang S, Shao Z, Li S, Ye Z. Feasibility of mixed reality-based intraoperative three-dimensional image-guided navigation for atlanto-axial pedicle screw placement. Proc Inst Mech Eng H. 2019;233:1310-1317.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 5]  [Article Influence: 2.3]  [Reference Citation Analysis (0)]
12.  Volonté F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P. Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci. 2011;18:506-509.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 91]  [Cited by in F6Publishing: 61]  [Article Influence: 8.3]  [Reference Citation Analysis (0)]
13.  Gu Y, Yao Q, Xu Y, Zhang H, Wei P, Wang L. A Clinical Application Study of Mixed Reality Technology Assisted Lumbar Pedicle Screws Implantation. Med Sci Monit. 2020;26:e924982.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 1]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
14.  Lei PF, Su SL, Kong LY, Wang CG, Zhong D, Hu YH. Mixed Reality Combined with Three-Dimensional Printing Technology in Total Hip Arthroplasty: An Updated Review with a Preliminary Case Presentation. Orthop Surg. 2019;11:914-920.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 3]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
15.  Fida B, Cutolo F, di Franco G, Ferrari M, Ferrari V. Augmented reality in open surgery. Updates Surg. 2018;70:389-400.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 20]  [Article Influence: 7.8]  [Reference Citation Analysis (0)]
16.  Chytas D, Piagkou M, Salmas M, Johnson EO. Mixed and Augmented Reality: Distinct Terms, Different Anatomy Teaching Potential. Anat Sci Educ. 2021;14:519-520.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in F6Publishing: 1]  [Reference Citation Analysis (0)]