Augmented reality for remote surgery mentoring

The system, described in journal The Visual Computer, enables medical expertise to be shared using vocal instructions and digital annotations, which appear on a clear tablet between the operating surgeon and the patient. The mentoring surgeon can see the patient via the camera on the tablet, and can draw lines indicating where incisions should be made.

STAR uses algorithms to keep the annotations aligned with the changing images of the surgery, and the transparent tablet screen allows the medic to view their hands, the patient and the overlaid instructions all in the same frame.

“The surgeon sees the operating field, the instruments, and their hands as if the display were not there, yet the operating field is enhanced with the mentor’s graphical annotations,” said Voicu S. Popescu, a Purdue associate professor of computer science.

While telementoring has been around for some time, existing methods require the acting surgeons to alternate their attention between the patient and a screen. STAR removes the need for surgeons to shift their focus away from the operating table. According to the researchers, the technology could be used in a number of scenarios.