Exploring the use of virtual reality in surgical education

Abstract

Virtual reality (VR) technologies have rapidly developed in the past few years. The most common application of the technology, apart from gaming, is for educational purposes. In the field of healthcare, VR technologies have been applied in several areas. Among them is surgical education. With the use of VR, surgical pathways along with the training of surgical skills can be explored safely, in a cost-effective manner. The aim of this mini-review was to explore the use of VR in surgical education and in the 3D reconstruction of internal organs and viable surgical pathways. Finally, based on the outcomes of the included studies, an ecosystem for the implementation of surgical training was proposed.

Key Words: Surgical education, Virtual reality, Abdominal surgery, Simulation

Core Tip: This mini-review aims to explore the use of virtual reality in surgical education and in the 3D reconstruction of internal organs and viable surgical pathways. For this purpose, a non-systematic literature review was conducted and three highly influential scientific papers were selected and discussed. The main topics addressed are the use of technologies in surgical education, the methodologies for the implementation of the training systems, the evaluation approaches and the strengths and limitations of the studies. Finally, the review concluded with a comparative synthesis of the main findings and a discussion on the proposal of a system for implementing these findings on surgical education in the field of organ transplantation.

INTRODUCTION

During the past few years, the use of virtual reality (VR) has increased rapidly in a number of sectors, like education[1,2] transportation[3,4] and healthcare[5]. In the case of education, the main advantage of using VR, is the immersion it provides, by using personalized experiences, promoting engagement, and providing hints that it may enhance learning[6], through the motivation aligned with the active participation of students.

As the immersion of the system increases, the effectiveness of the training module increases [7,8]. Additionally, the level of immersion of VR has been found to be proportional to the number of modalities involved[9].

VR has also found numerous applications in medical education[10-12]. More specifically, in the case of surgical education, the use of VR has been favored, due to many reasons, such as lack of mentors, reduction in training hours and various issues concerning operative procedures[13]. In order to exploit all these advantages, many solutions have been implemented, like the da Vinci Skills Simulator[14] and the LAP Mentor VR laparoscopic surgical simulator[15].

The aim of the present mini-review was to explore the use of VR simulators either alone or in combination with head-mounted displays (HMDs) in surgical education and in the construction of 3D models of internal organs[16]. For this purpose, three highly influential scientific papers were selected and discussed. The main topics addressed were the use of technologies in surgical education, the methodologies for the implementation of the training systems, the evaluation approaches and the strengths and limitations of the studies. Finally, the review concluded with a comparative synthesis of the main findings and a discussion on the proposal of a system for implementing these findings on surgical education in the field of organ transplantation.

METHODOLOGY

In April 2022, we performed a non-systematic literature search on the Google Scholar database using the terms “Virtual Reality”, “surgical education”, “surgery”, “medical education” to identify peer-reviewed articles, written in the English language, published after 2016, that seemingly explored the area of interest. The selected articles adhered to the following inclusion criteria: (1) Implement training in surgical skills with the use of VR technology; (2) Perform skill or full procedure training in abdominal surgeries; and (3) Include participants who were either surgical trainees or experienced surgeons.

All the information of interest was extracted from the selected articles. The information was used for the authors to identify main opportunities and limitations in the use of VR systems in surgical education and finally propose an infrastructure for extended reality (XR) technologies in order to implement a surgical training ecosystem.

TECHNOLOGIES

The devices used for promoting surgical education with the use of VR are mostly expensive[17,18] simulators (LapSim and Lap Mentor), often combined with some additional HMDs[18,19], like HTC Vive 360 or Google VR, to create an immersive and engaging user experience. Most simulation technologies include special controllers (some with haptic feedback) that accurately simulate the use of surgical instruments[17]. The LapSim emulator includes Simball 4D Joystick hardware and the Lap Mentor includes a syringe allowing realistic fluid delivery and BAL performance, while a wide variety of bronchoscopy instruments, such as biopsy forceps, cytology brush, suction and more can also be simulated. Both simulators offer a high-resolution display of the virtual environment (VE). The combination of VR HMDs and the VR simulators promotes immersiveness and enhances the interaction between the participants and the VE (Table 1).

Table 1 Comparison of technologies.

	Beyer-Berjot et al[17], 2016–Lap Mentor VR	Huber et al[18], 2017-LapSim	Sampogna et al[19], 2017-Oculus Rift and Leap motion
Technology used for training	Virtual reality
Equipment used for training	Custom hardware and software-Lap Mentor VR	Custom hardware and software-LapSim	Windows 10-Oculus Quest Rift S
Additional technology used for training	Haptic
Additional equipment used for training	Lap Mentor realistic tactile surgical tools	LapSim realistic tactile surgical tools	Oculus gestures + Leap Motion
Operating system	Lap Mentor software	LapSim software	Windows 10

VR: Virtual reality.

While the aforementioned devices offer a unique interactive experience, their cost can be extremely high. During the past years, there has been a rapid shift in the exploration of low-cost devices, offering the possibility of a larger market to the creators of any application. Such devices are the Oculus Rift, Meta Quest, HTC Vive, Pico[18,19]. The cost of these devices does not exceed $500, making surgical training more accessible to any hospital setting and open to more participants. Sampogna et al[19] used the Oculus Rift device combined with the Leap Motion sensor. The Oculus Rift requires a wired computer connection as well as the installation of the Oculus software on the computer and then through screencast displays the 3D VE on the glasses of the Rift device. The device includes two controllers, but in this study, they used the Leap Motion in order to keep the participants’ hands free. Leap Motion is a motion sensor that recognizes users’ actions and translates them into commands on a VR device or computer.

IMPLEMENTATION METHODS

When implementing surgical training in VR, the simulation can include either some basic tasks that are performed during specific surgeries[17,18], or full surgical procedures[17]. Simulators that specialize in specific surgeries, like the LAP simulator, have already integrated most of the corresponding tasks and require no further configurations in order to be ready for use. Huber et al[18] combined such a VR laparoscopic simulator with a 360^o video depicting an operating room, thus creating a highly immersive scenario, and offering, for the first time, a structured surrounding environment for the simulation to be accumulated in.

All the images and 3D models contained in the aforementioned simulators, are based on magnetic resonance imaging (MRI) and recordings of in vivo procedures. In order to create realistic 3D models of internal organs, a collection of computed tomography scans and MRIs are required. Sampogna et al[19] described in detail the procedure of recreating 3D reconstructions based on medical imaging.

EVALUATION AND OUTCOME MEASURES

When implementing an evaluation of the efficacy of new training methodologies, usually the learning impact of the new method needs to be compared to traditional methods. In the selected studies there was heterogeneity in the outcome measures, which did not follow a common evaluation protocol (Table 2).

Table 2 Beyer-Berjot et al[17], 2016 and Huber et al[18], 2017 outcome measures.

Beyer-Berjot et al[17], 2016–Lap Mentor VR		Huber et al[18], 2017-LapSim
Tasks	Outcome measures	Tasks	Outcome measures
Initial assessment	Time (s)	Peg transfer	Time (s)
Clipping and grasping	No. of movements	Fine dissection	Left time (s)
2-Handed maneuvers	Path length (cm)	Cholecystectomy	Right time (s)
Full laparoscopic sigmoid colectomy			Time (z-score)
Median dissection			Left path length (m)
Lateral dissection			Left angular path (degree)
Anastomosis			Left grasps (n)
Full LSC			Right path length (m)
			Right angular path (degree)
			Right grasps (n)
			Economics (z-score)
			Maximum drops (n)
			Errors (z-score)
			Total (z-score)

VR: Virtual reality; LSC: Large single copy.

There are some common measurements between the study of Beyer-Berjot et al[17] and Huber et al[18] such as the completion time of each task and the number of errors, but other than that, the focus of the evaluation was shifted in opposite directions.

The outcome measures used in the study of Beyer-Berjot et al[17] were: (1) Time taken to complete the task; (2) Time spent per hand; (3) Accuracy of the surgical procedure; (4) Depth of incisions; (5) Number of errors; (6) Number of ripped and burned vessels; and (7) Overall score of the LapSim system based on the calculation of all the components. Questionnaires were also administered, evaluating the degree of interaction, concentration and realism.

In the study of Huber et al[18], different outcome measures were considered, focusing on the degree of interaction of clipping and grasping, 2-handed maneuvers (time, number of movements, and path length) in 4 tasks, medial dissection, lateral dissection, anastomosis and full large single copy. The fidelity and content validity were measured on a Likert scale.

Sampogna et al[19] developed questionnaires to measure simplicity, precision and fidelity, guidance, satisfaction, 3D reconstruction quality, VR immersiveness.

CRITICAL REVISIT

As mentioned before, the main advantage of using VR in surgical education is the immersiveness the technology provides. This advantage was exploited in full when VR was implemented with the use of HMD, as described by Huber et al[18], Sampogna et al[19]. Furthermore, Huber et al[18] introduced noise cancelling headphones for increasing immersion. Haptic feedback is a modality often used in VR environments in order to engage the sense of touch. Beyer-Berjot et al[17] used a simulator that integrated with haptic feedback.

The enrolment of participants of different gaming and surgical skills can prove beneficial when evaluating a VR surgical education application. Huber et al[18] used participants of 3 different laparoscopic experience levels, while about half of them had never played video games or had any exposure to VR. Beyer-Berjot et al[17] implemented a similar design for the selection of the participants, but additionally they recruited a small number of video game players. The fact that the participants of these two studies had varying gaming skills, can offer a more subjective view on the usability and acceptability of the system, while the different surgical levels can assess the effectiveness of the system in terms of education.

LIMITATIONS

Despite the great advantages of using VR technology in surgical education, there are also a couple of limitations that need to be considered. The use of VR simulators implemented without the use of HMDs, as described by Beyer-Berjot et al[17], did not exploit the full potential of the technology, lacking in immersion and users’ engagement. Furthermore, the limited number of participants when performing a feasibility study along with the non-comparison of a new teaching method versus the traditional one[17-19] can lead to barriers in evaluating the impact on learning and skill development. Also, limitations of the use of VR may appear in older adults due to lack of acquittance with the technology. Finally, as Sampogna et al[19] pointed out, if the first operators have rich experience on the skills the new systems aspire to train, the effect of the developed applications on speeding-up the learning curve cannot be evaluated properly. The aforementioned limitations should be considered in terms of the publishing date and in the context of the technological advances of the time. Since then, VR technology has made major progress and the scope of its capabilities has improved vastly.

COMPARATIVE SYNTHESIS

Among the selected studies, two used high-end VR simulation equipment and performed their study with precision sensors[17,18]. In the two studies, during each simulated task, a variety of data were collected, and they were displayed after the completion of the simulation. Some of them were time-on-task, and number of errors as well as some other indicators were designed during the implementation of the systems. The main difference between the simulators used the surgical task they focus on. LipSim can perform fine dissection, peg transfer, and cholecystectomy while LapMentor offers the option of training in sigmoid colectomy. In the study of Sampogna et al[19], MRIs were collected from different patients and then reconstruction of the internal organs was performed. The 3D models were imported in the Unity3D environment, and an application was created for Oculus Rift.

In all selected studies, the participants were either surgeons or surgical residents. Beyer-Berjot et al[17] and Huber et al[18] divided their participants into experimental groups based on the number of operations they had carried out in their careers and on their expertise, while Sampogna et al[19] did not categorize their participants. In all three studies, before the beginning of the studies, participants had the opportunity to perform some warm-up tasks in order to get acquainted with the VR technology. The main aim of this exercise was to minimize the errors caused due to difficulties in operating the simulators and the HDMs.

FUTURE STEPS

In the past few years, expensive devices and applications have been used in the field of surgical education and surgical procedure, which imposes significant limitations to their extensive use. So, it is important to explore the use of less expensive XR technologies (augmented reality, VR, mixed reality). It is also important to explore the difficulties in co-surgery and in team surgery, due to cooperation problems that may arise. There is also a lack of intra-operative applications that focus on surgeon interactions. In addition, although some studies have been conducted on VR applications in the field of transplantation in general, there is a lack of studies on abdominal transplantations. Also, it will be useful to explore XR not only in surgical training but also during the surgical procedure.

Based on the findings of the comparative synthesis of the already existing approaches, we proposed a roadmap and its application could foster the training of surgeries (Figure 1). A 5-layered system could be constructed according to the following paradigm.

Figure 1 Extended reality proposed ecosystem[21-27]. XR: Extended reality; VR: Virtual reality; AR: Augmented reality; MR: Mixed reality.

The first layer includes low-cost devices XR. More specifically, future studies should investigate VR devices such as Meta Quest 2, Pico and AR devices such as NReal Light, Toshiba DynaEdge, which cost no more than $500 each and are affordable for not only surgeons but also mass purchases by hospitals and universities. Also, within the same layer we propose the inclusion of IoT devices such as bands and smartwatches as well as Arduino and Raspberry devices that allow sensorial, real-world, big data acquisition, like speech and motion capture analysis. The second layer focuses on co-designing and co-creating virtual and augmented surgeons’ training, based on participatory activities that will take place among healthcare and technology-oriented professionals[20]. In the third layer, a big data acquisition system is designed during the training activities. Data are gathered from heterogeneous sources such as training metrics, biomarkers, and sensory recordings[21] that could help assess the quality of the surgical procedure. In the fourth layer, biosensors are programmed to collect periodic data from the surgeons, which are uploaded on a cloud-based infrastructure where they are stored in a suitable database for analysis. Additional factors could be studied, such as the noise in the virtual surgery as well as the fatigue of the surgeon during the sessions. The analysis of these data is likely to create new approaches to deal with medical errors in operating rooms. In the fifth layer a platform is constructed that graphically presents the training analytics and the course of the surgeries for each surgeon.

CONCLUSION

VR technologies are becoming more accessible and are a potential cognitive enhancer in the field of surgical education. The findings of this mini-review offer insight into the devices and systems used to train surgeons, as well as to low-cost devices that are rapidly being developed to offer a solution in surgical training. Interestingly, we found a lack of VR training in the field of organ transplantation. In order to tackle this, an ecosystem for promoting learning through XR systems is proposed to be implemented for use in training for transplantation. In order to assess the proposed architecture, a feasibility study along with a cost-effectiveness analysis should be performed. The implementation and evaluation of the system falls outside the scope of this mini-review. Nevertheless, it could prove to be a valuable tool in the field of surgical and more specifically transplantation training, especially if evaluated against a transplantation simulator.

ACKNOWLEDGEMENTS

We thank our colleagues from Laboratory Medical Physics and Digital Innovation lab, Faculty of Health Sciences, School of Medicine, Aristotle University of Thessaloniki who provided insight and expertise that greatly assisted the research.