TESSA

 

TESSA, the TExt and Sign Support Assistant, has been developed to assist people who are deaf or hard of hearing in transactions at a Post Office.

The TESSA system combines speech recognition technology and state of the art virtual human animation to enable Post Office assistants to communicate with deaf customers. The assistant speaks into a microphone which is then recognised by a computer speech recognition system. The speech is converted to British Sign Language and signed by the virtual human for the customer. English text can also be displayed for those who do not use sign language.

TESSA is being developed as a collaboration between The School of Information Systems at the University of East Anglia (UEA), Televirtual , Consignia (the UK Post Office) and the Royal National Institute for Deaf People. UEA carried out system design, speech, linguistic and interface development, while Televirtual developed the Avatar and animation. Consignia and the RNID are involved in assessment of the system.

The movements of the virtual human are ‘copies’ of those of a native sign language user. Software specially developed for the project captures the signer’s hand, mouth and body movements using a variety of electronic sensors. These movements are then stored and used to animate the avatar when required. The Royal National Institute for Deaf People, the UK’s largest charity representing deaf people, are closely involved in the project. Their advice concerning the quality of the avatar and animation as well as information concerning the needs of deaf people have helped to shape the system’s development. During the summer of 2001, the system was on display at the Science Museum in London and was simultaneously on trial at the Post Office there. You can see some pictures from this event.

 

System Overview

The Tessa Virtual Human or 'Avatar'

 

'

Signs Being Motion Captured

 

The project uses commercially available speech-to-text software, which has been optimised to work in a noisy Post Office environment with several different speakers. TESSA enables over 370 of the most frequent Post Office transactions including those involving variable quantities such as money and days of the week to be completed more effectively. The system has been developed around the principle that British Sign Language is simply another language. Thus the technology can also be used to translate into other written languages: French, German, Welsh and Somali are currently implemented. This means the system is particularly useful in Post Offices with large numbers of tourist visitors.

 
While the system is currently targeted at transactions in the UK Post Office, the ease with which new signing lexicons can be produced means that the system may be easily adapted to other areas. A recent development of TESSA has been to move from simple speech recognition to speech "understanding" so that the PO clerk is not so constrained in what he or she can say to the system. Investigations are also being undertaken into the possibility of the system recognising a limited number of signs made by a deaf customer. TESSA is just part of the EU funded ViSiCAST project aimed at benefitting deaf citizens by allowing them access to information and services in sign language. The ViSiCAST partners are also developing the technology to facilitate the use of sign language on television and the Internet.

TESSA has recently won both a gold medal and the overall IT award at the prestigious British Computer Society's Information Technology Awards.

TESSA development is funded by Consignia. Further research into TESSA is also being funded by ViSiCAST , a European Union Information Societies Technology (IST) project.

Further information:

For general enquiries about deafness and hearing loss, contact the RNID Information Line, 0808 808 0123 (voice) or 0808 808 9000 (text).