Joy Weaks
College of Computing
Researchers in the College of Computing
at Georgia Tech have developed technology, in partnership
with the Atlanta
Area School for the Deaf (AASD), to help hearing-impaired
children improve their language development. The CopyCat
project is an instructive computer game that uses a virtual
sign language tutor to augment the classroom experience
for deaf children.
Funded in part by the National
Science Foundation and the U.S. Department of
Education’s National Institute on Disability and Rehabilitation Research,
CopyCat enables these children to interact with and sign to on-screen
characters, while improving their communication skills. Results from fourth and
fifth grade students at AASD already suggest CopyCat is a powerful tool for enhancing
the communication skills of hearing-impaired children.
“By encouraging deaf children to practice signing
in complete thoughts and ideas, this project has proved
to be a great use of computing technology,” says
Thad Starner, an assistant professor in the College of Computing.”
 |
|
Doctoral candidate Seungyon
Lee demonstrates the CopyCat program, a pilot project
developed in coordination with the Atlanta Area
School for the Deaf. |
|
Early
childhood is a critical period for language acquisition, and exposure to
language is key to linguistic development. However,
since 90 percent of
deaf
children are born to hearing parents who do not know or have low levels of
proficiency in sign language, a hearing-impaired child’s only exposure
to sign language is often at school. In addition, American Sign Language (ASL)
is a distinct language
from English, involving a significantly different grammar and lexicon. This
limited exposure slows the pace of their linguistic development.
“The effects of late linguistic development
for hearing-impaired children can lead to a lifetime
of difficulty, hindering their ability to communicate
with people, including their own family members, on a daily basis,” said
Harley Hamilton, educational technology specialist at AASD. “While
using CopyCat, the children display noticeable increases in enthusiasm,
focus and fluent
signing. The computer provides a patient, skilled, communicative partner
for the children anytime they choose, and that level of interaction is
invaluable.”
CopyCat’s interface was designed by doctoral
candidates Seungyon
Lee and Valerie Henderson in collaboration
with Hamilton, while
Starner, postdoctoral
fellow Kwang-Hyun Park and doctoral student Helene Brashear refined the
sign language recognition system.
Using a video camera and wrist-mounted
sensors, the user and the character of the game, Iris
the cat, communicate with ASL. The game is designed
with a limited,
age-appropriate phrase set. For example, the child will sign to Iris, “you
go play balloon.” If the child signs poorly, Iris looks puzzled,
and the child is encouraged to attempt the phrase again. If the child
signs clearly,
Iris frolics and plays with a red balloon.
If the child cannot remember
the correct phrase to direct Iris, she can click on a button bearing
the picture of the object with which she would
like Iris
to play. The system shows a short video with a teacher demonstrating
the correct ASL phrase. The child can then mimic the teacher to communicate
with Iris.
In the current game, there are eight phrases per level, and the child
must
correctly
sign each phrase before moving on to the next level. |
