Augmented reality: The use of the PicoLinker smart glasses improves wire insertion under fluoroscopy

Abstract

AIM

To demonstrate the feasibility of the wearable smart glasses, PicoLinker, in guide wire insertion under fluoroscopic guidance.

METHODS

Under a fluoroscope, a surgeon inserted 3 mm guide wires into plastic femurs from the lateral cortex to the femoral head center while the surgeon did or did not wear PicoLinker, which are wearable smart glasses where the fluoroscopic video was displayed (10 guide wires each).

RESULTS

The tip apex distance, radiation exposure time and total insertion time were significantly shorter while wearing the PicoLinker smart glasses.

CONCLUSION

This study indicated that the PicoLinker smart glasses can improve accuracy, reduce radiation exposure time, and reduce total insertion time. This is due to the fact that the PicoLinker smart glasses enable surgeons to keep their eyes on the operation field.

Key Words: Smart glasses, Imaging, Wearable devices, Fluoroscopy, Guide wire insertion

Core tip: Smart glasses are a kind of wearable device that has a head-mounted monitor enabling augmented reality. The fluoroscopic video was displayed on the head-mounted monitor of the smart glasses, PicoLinker. A surgeon was asked to insert 3 mm guide wires into plastic femoral bones under fluoroscopic control while wearing the PicoLinker smart glasses or by viewing the conventional fluoroscope monitor. Total insertion time, radiation exposure time and tip apex distance were shorter while wearing the PicoLinker smart glasses than while viewing the conventional monitor. Smart glasses are an innovative device that enables surgeons to keep their eyes on the operation field during procedures carried out under fluoroscopic control.

INTRODUCTION

Wearable technology has entered the medical field and will change surgery dramatically. Wearable smart glasses are a kind of computer that displays information on a head-mounted display. For procedures performed under fluoroscopic guidance, the head-mounted display enables surgeons to perform procedures under fluoroscopic control while keeping their eyes on the operative field. Google Glass^® (Google Inc., Mountain View, CA, United States) is the most well-known wearable glasses and has been used for various medical purposes such as medical education[1,2], surgery navigation[3,4] and as vital sign monitors[5]. However, it is not commercially available for general use. PicoLinker (Westunitis Co., Ltd., Osaka, Japan) (Figure 1) is a kind of wearable glasses that can display images or videos that are on existing monitor screens, on the head-mounted monitor. Unlike Google Glass, PicoLinker can capture video from various types of video sources via a transmission box that is connected to PicoLinker and contains various types of video connectors. We assumed that PicoLinker can be used as an alternative screen for fluoroscopic images. This technology is a kind of augmented reality (AR) where virtual images are added on real images. Virtual reality is where the wearer can see completely virtual or synthesized images in a completely shielded eyewear without viewing the real image. Through the AR, the wearer (operator) can obtain additional information (fluoroscopic video) simultaneously with the real image (operation field). Using this device, the operator can glance at the fluoroscope video on the PicoLinker smart glasses while keeping his/her eyes on the operation field. This would improve accuracy, reduce radiation exposure, and reduce the procedure time. The aim of this pilot study was to evaluate the effectiveness of PicoLinker in guide wire insertion into an artificial femoral head under fluoroscopic control.

Figure 1 An operator wearing the PicoLinker (wearable smart glasses).

MATERIALS AND METHODS

Wearable device

The PicoLinker is a kind of wearable smart glasses and contains a prism monitor with resolution of 428 × 240 pixels. PicoLinker can be mounted on any type of normal glasses and is connected to a video box by a wire cable. The video box has six types of video-connectors: HDMI, USB, VGA, video composite, S-video composite, and encrypted and secured micro-SD slot. Various types of monitors such as a fluoroscope monitor can be connected to the video box, and can be viewed by the operator on the prism monitor of PicoLinker. Although most types of video lines can be connected, in the case where there is no available connector nor video signals, a video camera can be placed in front of the existing monitor so that the screen image can be transferred to the prism monitor via the video box. The prism screen is set on normal glasses. In the present study, a fluoroscope monitor was connected to the video box of PicoLinker so that the surgeon could view the fluoroscopic video through PicoLinker.

Evaluation of wire insertion

A surgeon performed guide wire insertion into five plastic femoral bones (Sawbone^®, Sawbones Inc., Malmo, Sweden). The operator was instructed to introduce a Kirschner wire of 3.0 mm in diameter using an electric driver from the lateral cortex of the femur towards the femoral head, parallel to the femoral neck axis on anteroposterior view, under fluoroscopic control with the aim of minimizing the tip apex distance (TAD)[6]. The driver was initially placed beside the operator. The operator was instructed to pick up the driver, pick up the wire with the driver, find the best insertion point, and advance the drill into the femoral head. The drill insertion was performed ten times with PicoLinker (Wearable group) (Figure 2) and another ten times without PicoLinker while viewing the conventional fluoroscope monitor (Conventional group) (Figure 3). The time from picking up the wire to completion of insertion, total radiation time and the TAD on anteroposterior view were measured and evaluated.

Figure 2 The operator inserting a 3. 0 mm Kirschner wire in a plastic femoral bone under a fluoroscope while viewing the fluoroscopic video on PicoLinker.

Figure 3 The operator inserting a 3. 0 mm Kirschner wire in a plastic femoral bone under a fluoroscope without PicoLinker, while viewing a conventional fluoroscope monitor.

Statistical analysis

Results are expressed as the mean. The significance of differences between the groups was evaluated using unpaired t-test. A level of P < 0.05 was considered to be significant. Statistical analyses were performed using Microsoft Excel 2016 (Microsoft Corp., Redmond, WA, United States).

RESULTS

TAD, radiation exposure time and total insertion time were significantly shorter in the Wearable group than in the Conventional group [2.6 mm (mean) vs 4.1 mm, P = 0.02; 11.6 s vs 15.0 s, P = 0.00001; and 14.5 s vs 19.3 s, P < 0.00001, respectively].

DISCUSSION

Smart glasses are an innovative device for medical purposes that can be used in emergency surgery and surgical education including live streaming as well as remote instruction and monitoring. Although many papers on this device have been reported, evidence on the usefulness of the device has seldom been reported. Chimenti et al[7] reported a similar study using Google Glass in which an operator inserted Kirschner wires into cadaveric phalangeal and metacarpal bones. The results showed reductions in radiation time and operation time. Their results were similar to our results, which also showed reductions in radiation time and operation time. In addition, we found that the accuracy of pin insertion was significantly improved using PicoLinker. This is due to the fact that the operator can see both the operation field and fluoroscopic images without moving his/her eyes and head.

Numerous medical procedures are performed under images displayed on monitor screens. In most such procedures, the operator needs to take his/her eyes off from the operation field to see the monitor (Figure 3). In addition to fluoroscopy, vital sign monitor, endoscope, and computer navigation as well as conventional images such as X-ray image, computed tomography and magnetic resonance imaging are images on monitors. Using the PicoLinker allows the operator to keep his/her eyes on the operative field while seeing the images (Figure 2).

Unlike Google Glass, PicoLinker has a video box that can be connected by various types of connectors. Furthermore, even if there is no available connector from a monitor, a video camera can be used to capture the monitor images and the images can be transferred to PicoLinker; consequently, the images on all types of monitors can be transferred to PicoLinker. In addition to connectivity, the simple structure is another advantage of PicoLinker over Google Glass. The PicoLinker contains no more than a prism monitor. It has no CPU, camera nor battery; therefore, it has a light weight and battery life is never a problem. As PicoLinker is connected to a video source via a wire cable, there is no image delay. On the other hand, Google Glass images are transferred via the internet, and therefore a certain degree of latency is inevitable. Commercial availability is another distinct advantage of PicoLinker over Google Glass.

Nearly all reports on the use of smart glasses for medical purposes have involved the use of Google Glass. However, some other devices are available for surgery. We reported another type of smart glasses, InfoLinker (Westunitis Inc., Osaka, Japan), which has both a head-mounted monitor and head-mounted video camera and can connect to the internet for surgical video streaming[8]. This device is suitable for sending images in which the operator can see the same images through a head-mounted video camera. Although Google Glass has been used for this purpose, there are some advantages of the InfoLinker over Google Glass such as internet connection flexibility, battery durability and usability. Both PicoLinker and InfoLinker are currently available in Japan. They will soon become available in other countries.

Limitations of this study were that the trial was done by a single surgeon and the relatively small trial numbers. Nevertheless, the superiority of the use of PicoLinker smart glasses was demonstrated. The results on the usefulness of PicoLinker are encouraging. Further usage is expected.

COMMENTS

Background

There are few reports on head-mounted visualization of video from a fluoroscope monitor. Most operators have to see the fluoroscopic video on a fluoroscope monitor apart from the operation field. This may cause technical difficulties and inconvenience to the operator.

Research frontiers

Smart glasses are an innovative tool for visualization and image recording using a head-mounted monitor and head-mounted camera. Nearly all medical articles on smart glasses have involved the use of Google Glass. Although Google Glass can be utilized in various medical settings, head-mounted visualization of fluoroscopic video has rarely been reported.

Innovations and breakthroughs

The authors have presented a new type of smart glasses named PicoLinker. It is already commercially available in Japan and will soon be available in other countries. PicoLinker has some advantages over Google Glass with regard to direct image transfer via cable connection without image latency, its simple and light body, and unlimited battery life. The introduction of PicoLinker to the medical field will inspire new ideas to improve surgeries.

Peer-review

This is an interesting study testing a devise “coming from the future”.