Cameras and machine learning take pulse of remote patients

Telemedicine could be improved with a method that uses the camera on an electronic device to take pulse and respiration signals from a real-time video of a patient’s face. 

The advance by a University of Washington-led team was presented in December, 2020 at the Neural Information Processing Systems conference. The team is now proposing a better method to measure these physiological signals with a system less likely to be hindered by different cameras, lighting conditions or facial features. The researchers will present these findings April 8 at theACM Conference on Health, Inference, and Learning.

"Machine learning is pretty good at classifying images. If you give it a series of photos of cats and then tell it to find cats in other images, it can do it. But for machine learning to be helpful in remote health sensing, we need a system that can identify the region of interest in a video that holds the strongest source of physiological information - pulse, for example - and then measure that over time," said lead author Xin Liu, a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering.