Clever TESSA talks to the deaf

Avatars incorporated into TESSA – Text and Sign Support Assistant – could offer new hope to the one-in-seven people in the UK who suffer from a hearing disability.

TESSA has been designed to translate normal speech into British Sign language and has particular ramifications with the introduction of the Disability Discrimination Act, which makes it mandatory for retailers and service providers to improve the way they communicate with disabled customers.

TESSA has been developed by the Post Office in conjunction with the University of East Anglia and the motion capture company, Televirtual.

Translation from English into BSL can be complicated given that the grammar and syntax of sign language, which can be likened to Chinese, is derived from a combination of the signs and facial expression.

Dr Stephen Cox and Dr Mike Lincoln developed the sophisticated speech recognition system so that it recognises specially chosen phrases spoken by post office counter clerks.

The recogniser can cope with varying amounts of money, days of the week, countries and the system has been designed to work on different accents and within a noisy post office environment.

Recognised phrases are relayed automatically to deaf customers in sign language. Early experiments were conducted at the Post Office branch at UEA and TESSA is now being tested at ten selected sites around the country.

In addition, work is going on to translate the Post Office clerk’s speech to other spoken languages such as French, Spanish, Punjabi, Somali, Welsh and to either display the output to the customer as text or to speak it to them using speech synthesis.

‘The next phase of the project is to move from speech recognition to speech understanding so that the Post Office clerk will not be so constrained in what he or she can say to the system,’ said Dr Stephen Cox. ‘Also, we are investigating the possibility of recognising a limited number of signs made by a deaf customer’

TESSA has already attracted interest from Tesco and a major high street bank and the project was recently awarded a European grant to extend its work.

More on the web at: www.sys.uea.ac.uk/Research/projects/ViSiCAST/NewProjects.html