Home | About JCVJS | Editorial board | Ahead of print | Current Issue | Archives | Instructions | Subscribe | Advertise | Contact us |   Login 
Journal of Craniovertebral Junction and Spine
Search Articles   
Advanced search   

   Table of Contents  
Year : 2020  |  Volume : 11  |  Issue : 2  |  Page : 81-85  

Augmented reality and artificial intelligence-assisted surgical navigation: Technique and cadaveric feasibility study

1 Department of Research, HoloSurgical Inc., Chicago, IL, USA
2 Department of Orthopaedics, University of Illinois, Chicago, IL, USA

Date of Submission13-Apr-2020
Date of Acceptance02-May-2020
Date of Web Publication5-Jun-2020

Correspondence Address:
Dr. Kris B Siemionow
HoloSurgical Inc., 1415 W 37th Street, Chicago, IL 60609
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/jcvjs.JCVJS_48_20

Rights and Permissions

Purpose: Augmented reality-based image overlay of virtual bony spine anatomy can be projected onto real spinal anatomy using computer tomography-generated DICOM images acquired intraoperatively. The aim of the study was to develop a technique and assess the accuracy and feasibility of lumbar vertebrae pedicle instrumentation using augmented reality-assisted surgical navigation.
Subjects and Methods: An augmented reality and artificial intelligence (ARAI)-assisted surgical navigation system was developed. The system consists of a display system which hovers over the surgical field and projects three-dimensional (3D) medical images corresponding with the patient's anatomy. The system was registered to the cadaveric spine using an optical tracker and arrays with reflective markers. The virtual image overlay from the ARAI system was compared to 3D generated images from intraoperative scans and used to percutaneously navigate a probe to the cortex at the corresponding pedicle starting point. Intraoperative scan was used to confirm the probe position. Virtual probe placement was compared to the actual probe position in the bone to determine the accuracy of the navigation system.
Results: Four cadaveric thoracolumbar spines were used. The navigated probes were correctly placed in all attempted levels (n = 24 levels), defined as Zdichavsky type 1a, Ravi type I, and Gertzbein type 0. The virtual overlay image corresponded to the 3D generated image in all the tested levels.
Conclusions: The ARAI surgical navigation system correctly and accurately identified the starting points at all the attempted levels. The virtual anatomy image overlay precisely corresponded to the actual anatomy in all the tested scenarios. This technology may lead more uniform outcomes between surgeons and decrease minimally invasive spine surgery learning curves.

Keywords: Augmented reality, minimally invasive spine surgery, pedicle screw accuracy, surgical navigation

How to cite this article:
Siemionow KB, Katchko KM, Lewicki P, Luciano CJ. Augmented reality and artificial intelligence-assisted surgical navigation: Technique and cadaveric feasibility study. J Craniovert Jun Spine 2020;11:81-5

How to cite this URL:
Siemionow KB, Katchko KM, Lewicki P, Luciano CJ. Augmented reality and artificial intelligence-assisted surgical navigation: Technique and cadaveric feasibility study. J Craniovert Jun Spine [serial online] 2020 [cited 2022 Aug 18];11:81-5. Available from: https://www.jcvjs.com/text.asp?2020/11/2/81/285936

   Introduction Top

Minimally invasive approaches to spine surgery provide many potential advantages over open procedures. These include decreased paraspinal dissection and other stabilizing tissue injuries, blood loss, and pain.[1] Computer-assisted three-dimensional (3D) navigation was introduced in spine surgery to improve pedicle screw placement accuracy.[2] Spine surgery navigation utilizes preoperative 3D imaging of spinal anatomy obtained by computed tomography (CT), magnetic resonance imaging, or X-rays.[3] There are several limitations which are common to all of the currently available navigation systems. All force surgeons to look away and verify the navigated instrument position on a flat screen monitor located away from the surgical field. This requires surgeons to develop complex and unnatural hand–eye coordination skills. These skills have a steep learning curve and rely on the surgeon's visuospatial skills and ability to translate 2D images into 3D anatomical structures. They also do not afford surgeons a means to visualize internal anatomical structures without soft-tissue exposure. Finally, they are not intuitive or ergonomic for surgeons, which limits their functionality and prolongs surgical time. Augmented reality has the potential to address the limitations of the current surgical navigation systems by improving anatomic visualization, ergonomics, and intraoperative workflow. The complexity of vertebral anatomy leads to a steep learning curve for pedicle screw placement in trainees.[4] An augmented reality navigation system may decrease the learning time required to safely perform minimally invasive surgical (MIS) procedures. The aim of this study was to assess the accuracy and feasibility of lumbar pedicle screw placement using a newly developed augmented reality and artificial intelligence (ARAI) surgical navigation system.

   Subjects and Methods Top

Four intact human cadaver lower torso and spine (T12-pelvis) with no history of the spine surgery were used. The study was conducted in compliance with the ethical guidelines for human cadaver studies.

The cadaver was placed prone on the surgical table. A surgical registration grid (RG) with reflective markers was applied topically to the lumbar area [Figure 1]. A spine array with reflective markers was inserted in the posterior superior iliac spine. Next, a preoperative CT scan was performed with O-arm Surgical Imaging System (Medtronic Inc., Minneapolis, MN, USA). The RG was registered to the spine array using an artificial intelligence sphere identification algorithm, while the reflective spheres were identified and their position was recorded by the V120 infrared optical tracker (NaturalPoint, Inc. Corvallis, OR, USA). The ARAI (Holo Surgical Inc., Chicago, IL, USA) surgical navigation system was brought into the operative field and registered to the spine array.
Figure 1: Registration grid placed over the patient (black) and anatomy array clamped to the spinous process. Both the registration grid and anatomy array were fully visible by the optical tracker

Click here to view

Using the systems' augmented reality 3D display, a metal probe (Medtronic Inc., Minneapolis, MN, USA) was hovered over each lumbar spine vertebrae. The probe's virtual projection, as seen on the 3D display, was used to localize the level and mark the location of the skin incision. After a stab incision was made, the probe was percutaneously advanced through the soft tissues and docked on the pedicle starting point using the ARAI system guidance. The probe was then advanced into the pedicle with a mallet, and its position was secured [Figure 2]. [Figure 3]a demonstrates a surgeon using the system and advancing the navigated probe into the pedicle. Next, an intraoperative CT scan was obtained to assess the adequacy of probe placement [Figure 3]b. A probe was used for analysis instead of pedicle screws in order to facilitate image analysis and decrease the amount of imaging artifacts. Starting point placement adequacy was judged by two orthopedic spine surgeons using intraoperative imaging [Figure 4]. The adequacy of the pedicle probe placement was assessed with the Gertzbein grading system,[5] Ravi score,[6] and Zdichavsky score.[7] In the Gertzbein scale, cortical breaches are described by the extent of extra-cortical screw violation[5] and the relative position of the screw to the pedicle is graded as follows: (i) completely within the pedicle; (ii) pedicle wall breach <2 mm; (iii) pedicle wall breach = 2–4 mm; and (iv) pedicle wall breach >4 mm5.[8] In the Ravi scale, the trajectory was classified as acceptable, straight, lateral, or medial.[6] The Zdichavsky scale classification is depicted in [Figure 5].[7]
Figure 2: (a) Virtual three-dimensional projection of the lumbar spine; virtual Jamshidi needle along with the orthogonal planes demonstrating placement of the navigated needle. Cadaver in prone position, head to the left. (b) Same cadaver specimen in prone position demonstrating the actual Jamshidi needle placement. Head to the left

Click here to view
Figure 3: (a) User placing Jamshidi needle in the lumbar spine pedicle under ARAI system guidance. (b) During the procedure, validation was performed by identifying the relationship of the virtual Jamshidi needle (pictured on computer monitor) to the actual Jamshidi needle as seen on intraoperative scans

Click here to view
Figure 4: (a) Axial image from intraoperative computed tomography scan demonstrating the starting point with Jamshidi needle in place on the L5 pedicle. (b) Sagittal image from intraoperative computed tomography scan demonstrating starting point with Jamshidi needle in place in the L4 pedicle

Click here to view
Figure 5: Zdichavsky scale

Click here to view

In such fashion, the lumbar pedicles L3, L4, and L5 were instrumented with the navigated probe bilaterally. After surgery, the pedicle's morphology was measured on the presurgery CT: sagittal and transverse width and sagittal and transverse angle with the use of Osirix [Figure 6] and [Table 1]. The measurements were performed according to the descriptions presented in the literature.[9],[10],[11] The transverse pedicle angle was measured as an angle between the cord and midvertebral line [Figure 6]b. A total of four board-certified orthopedic surgeons, one per cadaver, took part in the experiment. The time it took each surgeon to cannulate the pedicle was recorded.
Figure 6: (a) Sagittal pedicle angle. (b) Transverse pedicle angle

Click here to view
Table 1: The average measurements of instrumented pedicles

Click here to view

   Results Top

A total of 24 pedicles were instrumented with a navigated probe in four cadavers, six levels per cadaver. All the navigated probes were placed in an excellent position via the percutaneous technique as described above. Excellent position was defined as Zdichavsky type 1a, Ravi type I, and Gertzbein type 0 [Table 2] placement. The highest level of instrumentation was L3 and the lowest was L5. The average time to cannulate a pedicle using the system was 38.2 s (range 29–47 s) per surgeon [Table 3].
Table 2: The accuracy of screw placement

Click here to view
Table 3: Average time to cannulate pedicle per surgeon

Click here to view

   Discussion Top

The current trends toward MIS techniques represent an important shift in the practice of contemporary spinal surgery.[7] Accurate placement of pedicle screws provides optimal mechanical strength while improving patient safety,[7] and decreasing the morbidity of open spine procedures.[1]

The development of augmented reality computer-assisted spine surgery requires that 3D spine images be derived from CT and two-dimensional (2D) X-rays using 3D/2D registration.[12] Proper pedicle screw placement is dependent on the accuracy of this 3D/2D registration. In this study, extrinsic and intrinsic methods for 3D/2D registration were developed. Extrinsic methods rely on the implantation of instruments or markers into the bone.[12],[13],[14]

The markers visualized on both 2D and 3D images and their dimensional position are adjusted as needed.[15] The adjusted markers are then utilized to align the spine. Intrinsic methods utilize anatomical structures to perform 3D/2D registration. This can be anatomic feature based,[12] imaging intensity based[16],[17] or gradient based.[18] Feature-based registrations are concerned with ending the transformation that minimizes the distances between 3D features, extracted from the preinterventional image and the corresponding 2D features.[12] The features utilize geometrical entities such as isolated points, contours, or surfaces.[12] In point-based methodology, the tip of the spinous or transverse processes is identified in the preoperative CT images and localized during the surgery in the physical space of the patient by touching with a probe.[17] Surface-based methodology uses a probe which is moved along the surface of the vertebra. The recorded surface points are matched to that of a vertebral bone surface model extracted from the CT image.[17] Contour-based methods require segmentation of the object surface in both preoperative CT images and intraoperative X-ray images. Registration is in turn performed by minimizing the distance between the 3D surface model and lines presenting the contour of anatomical structures derived from the X-rays.[17],[19] A significant advantage of utilizing an intrinsic versus an extrinsic method is the noninvasive character of the registration. Intrinsic methods do not need the implementation of markers on or into the patient's body. This study demonstrated that our new intrinsic, feature-based 3D/2D registration method for CT and X-rays images was able to accurately identify pedicle screw starting points, with 3D image projection that surgeons could visualize and use to alter pedicle screw trajectory in real time.

Efficient pedicle instrumentation placement shortens surgical time, total anesthesia time, and in percutaneous spine surgery, radiation exposure for both patient and operating room staff.[20] He et al. reported a mean time of intraoperative X-ray imaging for inserting one pedicle screw of 7.99 ± 2.45 min (range 5.25–14 min).[21] In addition, when comparing fluoroscopic X-ray assistance with robot-assisted pedicle screw placement, a decrease in overall screw placement time was found, although there were no significant differences in single screw placement times.[22] The time required to place a single pedicle screw remained between 5 and 6 min using both techniques.[22] In this cadaver study, we demonstrated that the average time it took a board-certified orthopedic spine surgeon to cannulate the pedicle with a navigated pedicle probe was 38.2 s. Although this did not include the time it took the time from cannulation to screw insertion, this technique could significantly shorten the pedicle instrumentation portion of the surgical procedure while drastically decreasing radiation exposure to the operating room staff.

The ARAI surgical navigation system correctly and accurately identified the starting points at all attempted levels. The virtual anatomy image overlay precisely corresponded to the actual anatomy in all the tested scenarios. The virtual anatomy image projected was responsive, and surgeon's perspective was updated in real time. These encouraging preliminary results warrant further investigation, as ARAI surgical navigation systems could provide a powerful tool to increase the speed and accuracy of minimally invasive spine surgeons, leading to improved patient outcomes.

   Conclusion Top

The ARAI surgical navigation system correctly and accurately identified the starting points at all attempted levels. The virtual anatomy image overlay precisely corresponded to the actual anatomy in all the tested scenarios. The virtual anatomy image projected was responsive, and surgeon's perspective was updated in real time. These encouraging preliminary results warrant further investigation, as ARAI surgical navigation systems could provide a powerful tool to increase the speed and accuracy of minimally invasive spine surgeons, leading to improved patient outcomes.

Financial support and sponsorship

This study was financially supported by HoloSurgical Inc.

Conflicts of interest

The following authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Dr. Siemionow – Holosurgical Inc.; Dr. Luciano – Holosurgical Inc., Dr. Lewicki – Holosurgical Inc.

   References Top

Hu W, Tang J, Wu X, Zhang L, Ke B. Minimally invasive versus open transforaminal lumbar fusion: A systematic review of complications. Int Orthop 2016;40:1883-90.  Back to cited text no. 1
Abul-Kasim K, Ohlin A. Patients with adolescent idiopathic scoliosis of Lenke type-1 curve exhibit specific pedicle width pattern. Eur Spine J 2012;21:57-63.  Back to cited text no. 2
Arlien-Søborg P, Kjaer L, Praestholm J. Myelography, CT, and MRI of the spinal canal in patients with myelopathy: A prospective study. Acta Neurol Scand 1993;87:95-102.  Back to cited text no. 3
Gonzalvo A, Fitt G, Liew S, de la Harpe D, Turner P, Ton L, et al. The learning curve of pedicle screw placement: How many screws are enough? Spine (Phila Pa 1976) 2009;34:E761-5.  Back to cited text no. 4
Puvanesarajah V, Liauw JA, Lo SF, Lina IA, Witham TF. Techniques and accuracy of thoracolumbar pedicle screw placement. World J Orthop 2014;5:112-23.  Back to cited text no. 5
Ravi B, Zahrai A, Rampersaud R. Clinical accuracy of computer-assisted two-dimensional fluoroscopy for the percutaneous placement of lumbosacral pedicle screws. Spine (Phila Pa 1976) 2011;36:84-91.  Back to cited text no. 6
Heintel TM, Berglehner A, Meffert R. Accuracy of percutaneous pedicle screws for thoracic and lumbar spine fractures: A prospective trial. Eur Spine J 2013;22:495-502.  Back to cited text no. 7
Barr SJ, Schuette AM, Emans JB. Lumbar pedicle screws versus hooks. Results in double major curves in adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 1997;22:1369-79.  Back to cited text no. 8
Catan H, Buluç L, Anik Y, Ayyildiz E, Sarlak AY. Pedicle morphology of the thoracic spine in preadolescent idiopathic scoliosis: Magnetic resonance supported analysis. Eur Spine J 2007;16:1203-8.  Back to cited text no. 9
Vaccaro AR, Rizzolo SJ, Balderston RA, Allardyce TJ, Garfin SR, Dolinskas C, et al. Placement of pedicle screws in the thoracic spine. Part II: An anatomical and radiographic assessment. J Bone Joint Surg Am 1995;77:1200-6.  Back to cited text no. 10
Zindrick MR, Wiltse LL, Doornik A, Widell EH, Knight GW, Patwardhan AG, et al. Analysis of the morphometric characteristics of the thoracic and lumbar pedicles. Spine (Phila Pa 1976) 1987;12:160-6.  Back to cited text no. 11
Markelj P, Tomaževič D, Likar B, Pernuš F. A review of 3D/2D registration methods for image-guided interventions. Med Image Anal 2012;16:642-61.  Back to cited text no. 12
Gall KP, Thornton AF, Munzenrider J, Rosenthal S. Experience using radiopaque Fiducial points for patient alignment during radiotherapy. Int J Radiat Oncol 1993;27:161.  Back to cited text no. 13
Tang TS, MacIntyre NJ, Gill HS, Fellows RA, Hill NA, Wilson DR, et al. Accurate assessment of patellar tracking using Fiducial and intensity-based fluoroscopic techniques. Med Image Anal 2004;8:343-51.  Back to cited text no. 14
Markelj P, Tomaževič D, Likar B, Pernuš F. Registration of 3D Pre-Interventional to 2D Intra-Interventional Medical Images. Berlin and Heidelberg: Springer, Berlin, Heidelberg; 2009. p. 1924-7.  Back to cited text no. 15
Knaan D, Joskowicz L. Effective intensity-based 2D/3D rigid registration between fluoroscopic X-ray and CT. In: Lecture Notes Computer Science (Including Subseries Lecture Notes in Artificial Intelligence Lecture Notes Bioinformatics). Berlin: Springer; 2003. p. 351-8.  Back to cited text no. 16
Russakoff DB, Rohlfing T, Adler JR Jr., Maurer CR Jr. Intensity-based 2D-3D spine image registration incorporating a single Fiducial marker. Acad Radiol 2005;12:37-50.  Back to cited text no. 17
Livyatan H, Yaniv Z, Joskowicz L. Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT. IEEE Trans Med Imaging 2003;22:1395-406.  Back to cited text no. 18
Feldmar J, Ayache N. Rigid, affine and locally affine registration of free-form surfaces. Int J Comput Vis 1996;18:99-119.  Back to cited text no. 19
Konieczny MR, Krauspe R. Navigation versus fluoroscopy in multilevel mis pedicle screw insertion: Separate analysis of exposure to radiation of the surgeon and of the patients. Clin Spine Surg 2019;32:E258-65.  Back to cited text no. 20
He EX, Cui JH, Yin ZX, Li C, Tang C, He YQ, et al. A minimally invasive posterior lumbar interbody fusion using percutaneous long arm pedicle screw system for degenerative lumbar disease. Int J Clin Exp Med 2014;7:3964-73.  Back to cited text no. 21
Feng S, Tian W, Sun Y, Liu Y, Wei Y. Effect of robot-assisted surgery on lumbar pedicle screw internal fixation in patients with osteoporosis. World Neurosurg 2019;125:e1057-62.  Back to cited text no. 22


  [Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6]

  [Table 1], [Table 2], [Table 3]

This article has been cited by
1 Clinical applications of augmented reality in orthopaedic surgery: a comprehensive narrative review
Johnathan R. Lex, Robert Koucheki, Jay Toor, David J. Backstein
International Orthopaedics. 2022;
[Pubmed] | [DOI]
2 Advancing Surgical Education: The Use of Artificial Intelligence in Surgical Training
David T. Guerrero, Malke Asaad, Aashish Rajesh, Abbas Hassan, Charles E. Butler
The American Surgeon. 2022; : 0003134822
[Pubmed] | [DOI]
3 Machine Learning Applications of Surgical Imaging for the Diagnosis and Treatment of Spine Disorders: Current State of the Art
Paramesh Karandikar, Elie Massaad, Muhamed Hadzipasic, Ali Kiapour, Rushikesh S. Joshi, Ganesh M. Shankar, John H. Shin
Neurosurgery. 2022; Publish Ah
[Pubmed] | [DOI]
4 Feasibility and Accuracy of Thoracolumbar Pedicle Screw Placement Using an Augmented Reality Head Mounted Device
Henrik Frisk, Eliza Lindqvist, Oscar Persson, Juliane Weinzierl, Linda K. Bruetzel, Paulina Cewe, Gustav Burström, Erik Edström, Adrian Elmi-Terander
Sensors. 2022; 22(2): 522
[Pubmed] | [DOI]
5 Autonomous lumbar spine pedicle screw planning using machine learning: A validation study
KrisB Siemionow, CraigW Forsthoefel, MichaelP Foy, Dominik Gawel, ChristianJ Luciano
Journal of Craniovertebral Junction and Spine. 2021; 12(3): 223
[Pubmed] | [DOI]
6 How precise are oral splints for frameless stereotaxy in guided ear, nose, throat, and maxillofacial surgery: a cadaver study
Manfred Nilius, Minou Hélène Nilius
European Radiology Experimental. 2021; 5(1)
[Pubmed] | [DOI]
7 Augmented reality navigation in spine surgery: a systematic review
Gustav Burström, Oscar Persson, Erik Edström, Adrian Elmi-Terander
Acta Neurochirurgica. 2021; 163(3): 843
[Pubmed] | [DOI]
8 The Clinical Application of Augmented Reality in Orthopaedics: Where Do We Stand?
J. Hunter Matthews, John S. Shields
Current Reviews in Musculoskeletal Medicine. 2021; 14(5): 316
[Pubmed] | [DOI]
9 Augmented Reality (AR) in Orthopedics: Current Applications and Future Directions
Andrew A. Furman, Wellington K. Hsu
Current Reviews in Musculoskeletal Medicine. 2021;
[Pubmed] | [DOI]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article
   Subjects and Methods
    Article Figures
    Article Tables

 Article Access Statistics
    PDF Downloaded204    
    Comments [Add]    
    Cited by others 9    

Recommend this journal