The project TeleSTAR uses a full digital surgical microscope and augmented/mixed reality to provide new tools that permit teaching and intraoperative assistance. EIT Health is supported by EIT, a body of the European Union.
Goals and Motivation
TeleSTAR-Telepresence for Surgical Assistance and Training using Augmented Reality, utilizes a fully digital surgical microscope and augmented/mixed reality to provide new tools that permit teaching and intraoperative assistance. The system offers new ways to visualise relevant surgical information, allowing surgeons to improve, and also explain, their decisions. Surgical decisions can be easily visualised during course units, on high-resolution screens or a head-mounted display, in any local or remote location. The system guarantees the exact same surgeon’s field-of-view for all attending healthcare professionals. “Simple” standard procedures, as well as rare and complicated procedures, can be visually annotated and augmented – and supplemented with audio comments – to make every step of the surgery transparent. A bidirectional communication platform allows all attendees to address questions to the surgeon at the right time, which creates a dynamic teaching environment despite local distance. The teaching concept can be scaled to other surgical domains and existing degree programmes.
Current Problem
There is currently a lack of pedagogical and technical education and training measures for surgical interventions to reduce the associated digital illiteracy. The intraoperative surgeon’s view can only be conveyed to third parties (senior/chief physicians, biomedical engineering students) to a limited extent during advanced trainings and surgical workshops. This specifically concerns the surgical field-of-view as well as the lack of image-based annotation and commentary options that are directly linked to the anatomical scene. Within the framework of EIT-Health, two projects are currently being funded that try to address the above-mentioned problems with VR. But, there are currently no new training concepts for the promising approaches based on augmented reality (AR). Furthermore, there is a lack of networked IP-based solutions that are able to transfer intraoperative assistance and teaching systems with additional AR information to remote operating rooms or lecture halls. Especially in Germany, there is currently no program that is awarded with CME.
Solution
TeleSTAR will develop a training concept for healthcare professionals namely senior and chief physicians in ENT surgery using augmented reality (AR) in combination with computer vision algorithms. The healthcare professional networks of TU Delft, Charité – Universitätsmedizin Berlin and ARRI Medical will be brought together. ENT procedures will be performed using the full digital surgical microscope “ARRISCOPE”. ENT chief and senior physicians from the German network will attend the operation in Berlin. The procedures will be transmitted to remote lecture rooms in Delft. The recorded annotated video material will then be made available to all participants for discussion and teaching purposes. The training effect will be monitored with a digital questionnaire. Because of limited me for physicians, we will organize intensive 1-2 days sessions 0. Technical Demo, 1. Description of patient’s history, 2. Preoperative examinations and imaging will be presented to the learners, 3. Participants can suggest therapeutic strategy, which is discussed with the expert surgeon, 4. We will use teledialog tools to ensure interactivity and make live transmissions in 3D, 5. During live transmission, relevant surgical information will be highlighted with AR tools, 6. Finally, a 2nd expert will create an ad hoc questionnaire based on the performed intervention and sent to remote learners. The learning outcome will be monitored based on the correct answers and will be awarded with CME points.
Learning Outcomes
The course concept addresses three pillars for two focus groups (1) healthcare professionals and (2) assistant healthcare workers. The three pillars are as follows.
- Surgical Pillar
Teaching the anatomy of the ear with focus on temporal bone and middle ear - Digital literacy Pillar
Discussing opportunities and pitfalls of advanced imaging systems (3D,AR) - Remote/interactive Teaching Pillar for surgical training/education
Best practice using interactive tools in a remote teaching environment by offering audio, video, touch-based interface to support collaboration between trainer and trainees.
The course will be officially awarded with CME points.
