BodyExplorer: A Next-Generation Simulator for Healthcare Training, Providing Hands-On Learning and Practice via Augmented Reality Visualization



Health & Body


University of Pittsburgh


Researchers: Joseph Samosky, Douglas Nelson, and John O'Donnell

BodyExplorer is a next-generation medical simulator designed to enhance the ability of healthcare trainees to learn anatomy and physiology and practice treating patients though naturalistic interaction with an augmented reality-enhanced, full-body simulated patient. Simulation has been recognized as the most prominent innovation in healthcare education in the past two decades, but current systems require substantial resources, including technicians to run the simulator and instructors to lead scenarios, assess student performance, and provide guided feedback. Learning how to operate current simulators requires advanced training, so students typically cannot use them on their own for self-learning. BodyExplorer was designed to enable 24/7 on-demand training and self-learning for students by providing an intuitive interface, autonomous operation, and automated instruction using a highly sensorized physical body, projected augmented reality (AR), and an integrated virtual instructor. AR enables x-ray vision views inside the body, so trainees can see the internal effects of administering simulated medications or performing procedures, such as inserting a breathing tube. BodyExplorer is designed to expand access to the benefits of simulation-based learning for medical and nursing students, first responders, combat medics, and other healthcare practitioners, enabling them to practice skills and receive quantitative feedback on their performance before treating actual patients.