Read My Lips

(Ivanhoe Newswire) -- A new study by the University of East Anglia (UEA) suggests computers are better at lip-reading than humans.

A research team from the School of Computing Sciences at UEA compared the performance of a machine-based lip-reading system with that of 19 human lip-readers. They found the automated system significantly out-performed the human lip-readers, scoring a recognition rate of 80 per cent, compared with 32 per cent for human viewers.

The study showed that rather than the traditional approach to lip-reading training, in which viewers are taught to spot key lip-shapes from static, often drawn, images, the dynamics and the full appearance of speech gestures are very important. The machines are able to exploit very simplistic features that represent only the shape of the face, but human lip-readers require full view of people speaking.

Using a new video-based training system, viewers with very limited training significantly improved their ability to lip-read monosyllabic words -- a very difficult task. It is hoped this research might lead to novel methods of lip-reading training for the deaf and hard of hearing.

"This pilot study is the first time an automated lip-reading system has been benchmarked against human lip-readers and the results are perhaps surprising," the study's lead author Sarah Hilder was quoted as saying. "With just four hours of training it helped them improve their lip-reading skills markedly. We hope this research will represent a real technological advance for the deaf community."

Agnes Hoctor, campaigns manager at the Royal National Institute for Deaf People (RNID), was quoted as saying, "This research confirms how difficult the vital skill of lip-reading is to learn and why RNID is campaigning for people who are deaf or hard of hearing to have improved access to classes. We would welcome the development of video-based or online training resources to supplement the teaching of lip-reading. Hearing loss affects 55 per cent of people over 60 so, with the ageing population, demand to learn lip-reading is only going to increase."

SOURCE:  Presented at the International Conference on Auditory-Visual Speech Processing (AVSP), September 10 – 13, 2009

If this story or any other Ivanhoe story has impacted your life or prompted you or someone you know to seek or change treatments, please let us know by contacting Melissa Medalie at mmedalie@ivanhoe.com