Smart textile gloves enable deafblind people to understand live conversations

NTU said the world-first study will eventually allow the spoken word to be translated into a form of braille and communicated through e-textile gloves to the fingers of people who cannot hear or lipread.

Artificial intelligence (AI) will interpret conversations and provide a summary of what’s said via haptic actuators embedded in the fingers of the electronic gloves.

These actuators vibrate in a code to relay words, numbers and grammar to the wearer in a similar way to which raised dots communicate the braille alphabet on a six-figure grid.

The actuators - situated on the tops of the index and ring fingers below the knuckles - can vibrate at different amplitudes, frequencies and for varying lengths of time, allowing for numerous communication possibilities.

These can include the potential to provide directions, communicate phone notifications, fire alarms and doorbells, as well as interpretations of music or visual art.

Trials with ten participants achieved recognition rates of up to 80 per cent by optimising time intervals (125- 2000ms) and pulse widths (10- 60ms). 

In a statement, Dr Hughes-Riley, an associate professor in NTU’s Advanced Textiles Research Group (ATRG), said: “Thousands of people who are born deafblind could, for the first time ever, be able to understand live conversations thanks to the development of this new technology.

“By this we mean conversations in the street in everyday life, and not just those that are online, as the AI when activated will listen to the spoken word on the wearer’s behalf and provide a summary in real time of what’s being said.

“Combining this technology with a pair of wearable gloves allows people to integrate it into their everyday lives, which will bring about enormous benefits for them.

“This could include things that many people take for granted, such as understanding what other people are saying in conversation, hearing announcements at train stations or even enjoying an interpretation of music.”

The study is based on a working prototype for which a conference proceeding was published on 15 March.

Researcher Malindu Ehelagasthenna, who is undertaking a PhD as part of the study, first suggested that textile embedded haptics could be used in this assistive way.

“This technology has enormous potential to enrich the lives of thousands of people around the world who cannot see or hear and who are underrepresented,” he said. “As well as supporting people who are completely deafblind, this technology may have vast benefits for people who are partially sighted and have partial hearing, who may otherwise have difficulties communicating, accessing information and navigating their immediate environment.”