Augmented reality warzone system will help inexperienced doctors

When casualties occur far away from hospitals, doctors on the ground often have to deal with situations with which they are not familiar. Currently, a system known as a telestrator is used, where a more experienced mentor in a hospital can draw on to a video screen, with the sketches and notes appearing simultaneously on the screen at the remote location. Many of us are most familiar with telestrators from sports broadcasting, where studio-based pundits draw on their screen to indicate the paths taken by footballers during a replay (or the paths they think they ought to have taken) which appears on our screens.

The problem with telestrators is they require the remote practitioner to split his or her focus between the patient and the screen. Moreover, they failed to show upcoming steps in procedures and give the practitioner an incomplete picture of the ongoing procedure.

The team at Purdue, led by Juan Wachs, propose replacing this technology with an augmented reality system which they presented at a recent military health system research symposium in Florida. The system consists of a transparent headset screen display that allows the doctor in the field to see the patient in front of them, along with real-time on-screen feedback from their mentor, who uses a video monitor to see the augmented reality feed. The system uses computer vision algorithms to track and align the marks and notes made by the mentor with the surgical region in front of the field surgeon.