Tech that reads lips through masks offers boost to hearing-impaired

When integrated with conventional hearing aid technology, the system could help tackle the so-called ‘cocktail party effect’, a common shortcoming of traditional hearing aids.

Hearing aids assist hearing-impaired people by amplifying all ambient sounds around them, but in noisy situations the hearing aids’ broad spectrum of amplification can make it difficult for users to focus on specific sounds, like conversation with a particular person. 

One potential solution to the cocktail party effect is to make ‘smart’ hearing aids, which combine conventional audio amplification with a second device to collect additional data from reading lips.

There has been success in using cameras to aid with lip reading but collecting video footage of people without their consent raises privacy concerns. Cameras are also unable to read lips through masks, a regular challenge for people who wear face coverings for cultural or religious purposes and a broader issue in the COVID-19 era. 

In a paper published in Nature Communications, the Glasgow University led team outline how they set out to harness advanced sensing technology to read lips. Their system is said to preserve privacy by only collecting radio-frequency data, with no accompanying video footage.