Well worn words

A wearable captioning system could give people who are deaf or hard of hearing greater access and involvement in their communities.


For people who are deaf or hard of hearing, the voices of actors, teachers, sports announcers and clergy are often silenced.



With captioning typically available only in selected cinemas, a lack of access to information often limits community involvement for people who are deaf or hard of hearing. But a wearable captioning system developed at the Georgia Tech Research Institute (GTRI) and recently licensed by the Georgia Tech Research Corporation’s Office of Technology Licensing to a metro Atlanta company may change that situation for the 28 million Americans (about 10 percent of the population) who are deaf or hard of hearing.



The system, which relies on mobile wireless technology, will allow users to easily receive information that is being presented audibly to the general public in a variety of public venues, such as cinemas, museums, playhouses, schools, government meetings, sports arenas, and places of worship. Because the system can transmit multiple text streams, it may also be used for language translation because its software is said to be internationally compatible.



“The venue must generate the captions for patrons, but our system is an easy way to get captioning to patrons in a way they want – customised for each person,” said GTRI project director Leanne West. “Right now, captioning is typically available only in cinemas – and just a limited number of them – but this device gives us a way to deliver captioning at any venue.”



Peacock Communications, of Marietta, Georgia, plans to sell the captioning system software called COMMplements and consult with venues on hardware and installation issues, said chair and CEO Jack Peacock. He is seeking investor funding to finalise enhancements of the software. Peacock will market the software in the United States initially and then in Canada and Western Europe. The company has not announced a price for the software yet.



In COMMplements, captions are sent by a venue’s transmitter via standard wireless technology – the 802.11b wireless protocol – to a receiver device, such as a personal digital assistant (PDA), which also displays captions, West explained. The 802.11b wireless technology is already installed in many places, such as baseball stadiums, coffee shops and restaurants, and even urban business districts.



The venue operates the transmitter, and the patron can borrow the receiver and display from the venue or bring their own, West said. Wireless-enabled PDAs and laptop computers can serve as receivers and displays, or patrons can use a micro display that plugs into a PDA and attaches to their glasses or is worn on a headband. Although positioned close to the eye, the micro display uses optics that make its screen appear to float several feet away, giving users relaxed viewing of text seemingly overlaid on their visual field.



Captions can be pre-recorded or text can be generated in real time with a shorthand typing method such as CART (Communication Access Real-Time Translation). As voice recognition technology improves, captioning could be derived in this way, West added. Before West and her colleagues began the project, she consulted with the Georgia Council for the Hearing Impaired to gauge the need for and interest in a new type of captioning system.



With their interest confirmed, the project got funding from GTRI and a grant from the WirelessRehabilitationEngineeringResearchCenter (RERC) at Georgia Tech.



With an early prototype in hand, researchers tested the system on two different occasions, including the annual Self-Help for Hard of Hearing People (SHHH) conference. The tests included 63 volunteers who were deaf or hard of hearing and ranged in age from 15 to 75.



In their tests, researchers simulated a cinema setting, playing anywhere from 15 to 90 minutes of video. Participants wore glasses with a micro display attached.



Most volunteers reported that the display “took some getting used to,” but 65 percent of users said it took less than 10 minutes to get the display in comfortable reading position. Also, 65 percent said the text was easy to read.



“We took their comments and included some of the features they wanted to see,” West said. “Some people wanted an option for the font style, but we decided for several reasons to use the Tiresias font developed by the Royal National Institute for the Blind for people with low vision. Now, we get lots of compliments on the font.”



West and her colleagues also created customisable features such as text size and colour, background colour and focus adjustment. Most study participants — 84 percent — said they would not feel self-conscious about wearing the communication assistant in a public venue.



The idea for the captioning system developed about five years ago out of a luncheon discussion session in West’s research division at GTRI. Participants were discussing the litigation – now settled – over the lack of captioning in some cinemas. The group agreed that there must be a better way to provide captioning and to offer it in a variety of venues.



Right now, the leading technology for providing captions in a cinema is called the Rear Window Captioning System, which was developed by the NationalCenter for Accessible Media. The system is available in some cinemas and displays reversed captions on a light-emitting diode (LED) text display mounted in the rear of a theatre. Patrons who are deaf or hard of hearing use transparent acrylic panels mounted in their seat cup holders to reflect the captions so that they appear superimposed on the movie screen.



West said the GTRI system is more accommodating than the Rear Window system. The GTRI system handles multiple text streams, is fully customisable and discreet, and can be used in any public venue willing to provide captioning. Also, it works on a personal electronic device that users can affordably own and easily carry with them. The Rear Window system requires patrons to borrow equipment from the cinema.


GTRI researchers are now enhancing the software with security measures to prevent hacking and are updating it to accommodate a newer, more rugged, and less expensive micro display. They also plan to create a customised version of the captioning system for venues to distribute to patrons who don’t have a PDA.