augmented_reality

Augmented reality

Virtual reality simulators allows the development of novel methods to analyze neurosurgical performance.


Virtual reality (VR) simulators have been proposed as tools to understand, assess, and train neurosurgery residents 1) 2) 3) 4) 5).

An important element of simulator performance is the capacity of simulators to distinguish operator expertise. Most studies on operator performance have utilized “metrics.” 6) 7) 8) 9) 10) 11) 12) 13) 14) 15) 16).


Chan et al., highlights a selection of recent developments in research areas related to virtual reality simulation, including anatomic modeling, computer graphics and visualization, haptics, and physics simulation, and discusses their implication for the simulation of neurosurgery 17).


Medicine and surgery are turning towards simulation to improve on limited patient interaction during residency training. Many simulators today utilize virtual reality with augmented haptic feedback with little to no physical elements.

To optimize the learning exercise, it is essential that both visual and haptic simulators are presented to best present a real-world experience. Many systems attempt to achieve this goal through a total virtual interface.

Bova et al., approach has been to create a mixed-reality system consisting of a physical and a virtual component. A physical model of the head or spine is created with a 3-dimensional printer using deidentified patient data. The model is linked to a virtual radiographic system or an image guidance platform. A variety of surgical challenges can be presented in which the trainee must use the same anatomic and radiographic references required during actual surgical procedures.

Using the aforementioned techniques, they have created a ventriculostomy simulators, percutaneous radiofrequency trigeminal rhizotomy, and spinal instrumentation.

The system has provided the residents an opportunity to understand and appreciate the complex 3-dimensional anatomy of the 3 neurosurgical procedures simulated. The systems have also provided an opportunity to break procedures down into critical segments, allowing the user to concentrate on specific areas of deficiency 18).


Shakur et al., developed a real-time augmented reality simulator for percutaneous trigeminal rhizotomy using the ImmersiveTouch platform. Ninety-two neurosurgery residents tested the simulator at American Association of Neurological Surgeons Top Gun 2014. Postgraduate year (PGY), number of fluoroscopy shots, the distance from the ideal entry point, and the distance from the ideal target were recorded by the system during each simulation session. Final performance score was calculated considering the number of fluoroscopy shots and distances from entry and target points (a lower score is better). The impact of PGY level on residents' performance was analyzed.

Seventy-one residents provided their PGY-level and simulator performance data; 38% were senior residents and 62% were junior residents. The mean distance from the entry point (9.4 mm vs 12.6 mm, P = .01), the distance from the target (12.0 mm vs 15.2 mm, P = .16), and final score (31.1 vs 37.7, P = .02) were lower in senior than in junior residents. The mean number of fluoroscopy shots (9.8 vs 10.0, P = .88) was similar in these 2 groups. Linear regression analysis showed that increasing PGY level is significantly associated with a decreased distance from the ideal entry point (P = .001), a shorter distance from target (P = .05), a better final score (P = .007), but not number of fluoroscopy shots (P = .52).

Because technical performance of percutaneous rhizotomy increases with training, they proposed that the skills in performing the procedure in there virtual reality model would also increase with PGY level, if this simulator models the actual procedure. The results confirm this hypothesis and demonstrate construct validity 19).


Lemole et al., use the ImmersiveTouch (ImmersiveTouch, Inc., Chicago, IL) virtual reality platform, developed at the University of Illinois at Chicago, to simulate the task of ventriculostomy catheter placement as a proof-of-concept. Computed tomographic data are used to create a virtual anatomic volume.

Haptic feedback offers simulated resistance and relaxation with passage of a virtual three-dimensional ventriculostomy catheter through the brain parenchyma into the ventricle. A dynamic three-dimensional graphical interface renders changing visual perspective as the user's head moves. The simulation platform was found to have realistic visual, tactile, and handling characteristics, as assessed by neurosurgical faculty, residents, and medical students.

They developed a realistic, haptics-based virtual reality simulator for neurosurgical education. The first module recreates a critical component of the ventriculostomy placement task. This approach to task simulation can be assembled in a modular manner to reproduce entire neurosurgical procedures 20).

The virtual reality surgical training of thoracic pedicle screw instrumentation effectively improves surgical performance of novice residents compared to those with traditional teaching method, and can help new beginners to master the surgical technique within shortest period of time 21).

Augmented reality technology has been used for intraoperative image guidance through the overlay of virtual images, from preoperative imaging study, onto the real-world surgical field.

The direct projection of a virtual image to the patients head, skull, or brain surface in real-time is an augmented reality system that can be used for Image-Guided Neurosurgery 22).

Information supplied by an image-guidance system can be superimposed on the operating microscope oculars or on a screen, generating augmented reality. Recently, the outline of a patient's head and skull, injected in the oculars of a standard operating microscope, has been used to check the registration accuracy of image guidance.

A commercially available image-guidance system and a standard operating microscope were used. Segmentation of the brain surface and cortical blood vessel relief was performed manually on preoperative computed tomography and magnetic resonance images. The overlay of segmented digital and real operating-microscope images was used to monitor image-guidance accuracy. Adjustment for brain shift was performed by manually matching digital images on real structures.

Experimental manipulation on a phantom proved that the brain surface relief could be used to restore accuracy if the primary registration shifted. Afterward, the technique was used to assist during surgery of 5 consecutive patients with 7 deep-seated brain tumors. The brain surface relief could be successfully used to monitor registration accuracy after craniotomy and during the whole procedure. If a certain degree of brain shift occurred after craniotomy, the accuracy could be restored in all cases, and corticotomies were correctly centered in all cases.

The proposed method was easy to perform and augmented image-guidance accuracy when operating on small deep-seated lesions 23).

Although setups based on augmented reality have been used for various neurosurgical pathologies, very few cases have been reported for the surgery of arteriovenous malformations (AVM).

5 patients underwent AVM resection assisted by augmented reality. Virtual three-dimensional models of patients' heads, skulls, AVM nidi, and feeder and drainage vessels were selectively segmented and injected into the microscope's eyepiece for intraoperative image guidance, and their usefulness was assessed in each case.

Although the setup helped in performing tailored craniotomies, in guiding dissection and in localizing drainage veins, it did not provide the surgeon with useful information concerning feeder arteries, due to the complexity of AVM angioarchitecture.

The difficulty in intraoperatively conveying useful information on feeder vessels may make augmented reality a less engaging tool in this form of surgery, and might explain its underrepresentation in the literature. Integrating an AVM's hemodynamic characteristics into the augmented rendering could make it more suited to AVM surgery 24).

Solves the problem of view switching in traditional image-guided neurosurgery systems by integrating computer-generated objects into the actual scene. However, the state-of-the-art AR solution using head-mounted displays has not been widely accepted in clinical applications because it causes some inconvenience for the surgeon during surgery.

The easy-to-use Tablet-AR system presented in a study is accurate and feasible in clinical applications and has the potential to become a routine device in AR neuronavigation 25).

Augmented reality technology has been used for intraoperative image guidance through the overlay of virtual images, from preoperative imaging studies, onto the real-world surgical field. Although setups based on augmented reality have been used for various neurosurgical pathologies, very few cases have been reported for the surgery of arteriovenous malformations (AVM).

The difficulty in intraoperatively conveying useful information on feeder vessels may make augmented reality a less engaging tool in this form of surgery, and might explain its underrepresentation in the literature. Integrating an AVM's hemodynamic characteristics into the augmented rendering could make it more suited to AVM surgery 26).

Although further studies need to be performed to evaluate whether certain groups of aneurysms are more likely to benefit from it. Further technological development is required to improve its user friendliness 27).

Augmented reality navigation.

Strickland BA, Zada G, Lee DJ. Commentary: Application of Augmented Reality in Percutaneous Procedures-Rhizotomy of the Gasserian Ganglion. Oper Neurosurg (Hagerstown). 2021 Jun 7:opab179. doi: 10.1093/ons/opab179. Epub ahead of print. PMID: 34097742.

Spine Surgery Assisted by Augmented Reality.

Virtual Reality for Chronic Pain Treatment.

Sakamoto Y, Miwa T, Kajita H, Takatsume Y. Practical use of augmented reality for posterior distraction in craniosynostosis. J Plast Reconstr Aesthet Surg. 2022 Aug 27:S1748-6815(22)00515-0. doi: 10.1016/j.bjps.2022.08.072. Epub ahead of print. PMID: 36057505.


1)
Kockro RA, Serra L, Tseng-Tsai Y, Chan C, Yih-Yian S, Gim-Guan C et al (2000) Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery. 46(1):118–137
2)
Bernardo A, Preul MC, Zabramski JM, Spetzler RF (2003) A threedimensional interactive virtual dissection model to simulate transpetrous surgical avenues. Neurosurgery. 52(3):499–505 discussion 504–505
3)
Radetzky A, Rudolph M (2001) Simulating tumour removal in neurosurgery. Int J Med Inform 64(2–3):461–472
4)
Lemole GM Jr, Banerjee PP, Luciano C, Neckrysh S, Charbel FT (2007) Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery. 61(1):142–149
5)
Delorme S, Laroche D, DiRaddo R, Del Maestro RF (2012) NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71(suppl_1):ons32– ons42
6)
6. Choudhury N, Gelinas-Phaneuf N, Delorme S, Del Maestro R (2013) Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 80(5):e9– e19
7)
Gelinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery: integrating theory into practice. Neurosurgery 73(suppl_1):S30–S38
8)
Gelinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V et al (2014) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J Comput Assist Radiol Surg 9(1):1–9
9)
Azarnoush H, Alzhrani G, Winkler-Schwartz A, Alotaibi F, Gelinas-Phaneuf N, Pazos V, Choudhury N, Fares J, DiRaddo R, del Maestro R (2015) Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection. Int J Comput Assist Radiol Surg 10(5):603–618
10)
Cline BC, Badejo AO, Rivest II, Scanlon JR, Taylor WC, Gerling GJ (2008) Human performance metrics for a virtual reality simulator to train chest tube insertion. IEEE SIEDS :168–173
11)
Kazemi H, Rappel JK, Poston T, Hai Lim B, Burdet E, Leong TC (2010) Assessing suturing techniques using a virtual reality surgical simulator. Microsurgery. 30(6):479–486
12)
Trejos AL, Patel RV, Malthaner RA, Schlachta CM (2014) Development of force-based metrics for skills assessment in minimally invasive surgery. Surg Endosc 28(7):2106–2119
13)
Kovac ERA, Azhar A, Quirouet J, Delisle, Anidjar M (2012) Construct validity of the lapSim virtual reality laparoscopic simulator within a urology residency program. CUAJ 6(4):253
14)
Alotaibi FE, Al Zhrani G, Bajunaid K, Winkler-Schwartz A, Azarnoush H et al (2015) Assessing neurosurgical psychomotor performance: role of virtual reality simulators, current and future potential. SOJ Neurol 2(1):1–7
15)
Alotaibi FE, AlZhrani GA, Mullah MA, Sabbagh AJ, Azarnoush H, Winkler-Schwartz A et al (2015) Assessing bimanual performance in brain tumor resection with NeuroTouch, a virtual reality simulator. Oper Neurosurg 11(1):89–98
16)
Alotaibi FE, AlZhrani GA, Sabbagh AJ, Azarnoush H, WinklerSchwartz A, Del Maestro RF (2015) Neurosurgical assessment of metrics including judgment and dexterity using the virtual reality simulator NeuroTouch (NAJD Metrics). Surg Innov 22(6):636–642
17)
Chan S, Conti F, Salisbury K, Blevins NH. Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery. 2013 Jan;72 Suppl 1:154-64. doi: 10.1227/NEU.0b013e3182750d26. PubMed PMID: 23254804.
18)
Bova FJ, Rajon DA, Friedman WA, Murad GJ, Hoh DJ, Jacob RP, Lampotang S, Lizdas DE, Lombard G, Lister JR. Mixed-reality simulation for neurosurgical procedures. Neurosurgery. 2013 Oct;73 Suppl 1:138-45. doi: 10.1227/NEU.0000000000000113. PubMed PMID: 24051877.
19)
Shakur SF, Luciano CJ, Kania P, Roitberg BZ, Banerjee PP, Slavin KV, Sorenson J, Charbel FT, Alaraj A. Usefulness of a Virtual Reality Percutaneous Trigeminal Rhizotomy Simulator in Neurosurgical Training. Neurosurgery. 2015 Sep;11 Suppl 3:420-5; discussion 425. doi: 10.1227/NEU.0000000000000853. PubMed PMID: 26103444.
20)
Lemole GM Jr, Banerjee PP, Luciano C, Neckrysh S, Charbel FT. Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery. 2007 Jul;61(1):142-8; discussion 148-9. Review. PubMed PMID: 17621029.
21)
Hou Y, Lin Y, Shi J, Chen H, Yuan W. Effectiveness of the Thoracic Pedicle Screw Placement Using the Virtual Surgical Training System: A Cadaver Study. Oper Neurosurg (Hagerstown). 2018 Mar 14. doi: 10.1093/ons/opy030. [Epub ahead of print] PubMed PMID: 29554379.
22)
Mahvash M, Besharati Tabrizi L. A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir (Wien). 2013 May;155(5):943-7. doi: 10.1007/s00701-013-1668-2. Epub 2013 Mar 15. PubMed PMID: 23494133.
23)
Kantelhardt SR, Gutenberg A, Neulen A, Keric N, Renovanz M, Giese A. Video-Assisted Navigation for Adjustment of Image-Guidance Accuracy to Slight Brain Shift. Neurosurgery. 2015 Jul 30. [Epub ahead of print] PubMed PMID: 26230043.
24)
Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir (Wien). 2014 Sep;156(9):1769-74. doi: 10.1007/s00701-014-2183-9. Epub 2014 Jul 20. PubMed PMID: 25037466.
25)
Deng W, Li F, Wang M, Song Z. Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg. 2014;92(1):17-24. doi: 10.1159/000354816. Epub 2013 Nov 8. PubMed PMID: 24216673.
26)
Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir (Wien). 2014 Jul 20. [Epub ahead of print] PubMed PMID: 25037466.
27)
Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery. 2014 Jun;10 Suppl 2:252-60; discussion 260-1. doi: 10.1227/NEU.0000000000000328. PubMed PMID: 24594927.
  • augmented_reality.txt
  • Last modified: 2022/09/05 00:20
  • by administrador