Medical procedures are an essential part of healthcare delivery, and the acquisition of procedural skills is a critical component of medical education. Unfortunately, procedural skill is not homogeneously distributed among medical providers. Skills may vary within departments or institutions, and across geographic regions, depending on provider’s training and ongoing experience.

We present a mixed reality real-time communication system to increase access to procedural skill training and to improve remote emergency assistance. Our system allows a remote expert to guide a local operator through a medical procedure. RGBD cameras capture a volumetric view of the local scene including the patient, the operator, and the medical equipment. The volumetric capture is augmented onto the remote expert's view to allow the expert to spatially guide the local operator using visual and verbal instructions.  
We evaluated our mixed reality communication system in a study in which experts teach the ultrasound-guided placement of a central venous catheter (CVC) to students in a simulation setting. The study compares state-of-the-art video communication against our system. The results indicate that our system enhances and offers new possibilities for visual communication compared to video teleconference-based training.

Publications:

1. Manuel Rebol, Krzysztof Pietroszek, Claudia Ranniger, Colton Hood, Adam Rutenberg, Neal Sikka, Christian Guetl. Collaborative System Design of Mixed Reality Communication for Medical Training. Proceedings of the 56th Hawaii International Conference on System Sciences, 2023. https://hdl.handle.net/10125/102680
2. Manuel Rebol, Claudia Ranniger, Colton Hood, Erin Horan, Adam Rutenberg, Neal Sikka, Yasser Ajabnoor, Safinaz Alshikah, and Krzysztof Pietroszek. 2022. Mixed Reality Communication System for Procedural Tasks. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces (AVI 2022). Association for Computing Machinery, New York, NY, USA, Article 72, 1–3. https://doi.org/10.1145/3531073.3534497
3. Manuel Rebol, Krzysztof Pietroszek, Claudia Ranniger, Colton Hood, Adam Rutenberg, Neal Sikka, David Li, Christian Gütl, "Mixed Reality Communication for Medical Procedures: Teaching the Placement of a Central Venous Catheter," 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Singapore, Singapore, 2022, pp. 346-354, doi: 10.1109/ISMAR55827.2022.00050.
4. Manuel Rebol, Colton Hood, Claudia Ranniger, Adam Rutenberg, Neal Sikka, Erin Maria Horan, Christian Gütl, Krzysztof Pietroszek. "Remote Assistance with Mixed Reality for Procedural Tasks," 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal, 2021, pp. 653-654, doi: 10.1109/VRW52623.2021.00209.