11/03/08: Sight for the Blind and Speech for the Deaf
11/03/08: Sight for the Blind and Speech for the Deaf
Technology is an amazing thing. This story is about how people in the technology field can make a difference by using their knowledge and creativity to help others. After viewing, don’t forget to take the corresponding Blackboard Assessment for extra credit!
Here is a link to blackboard: https://blackboard.ilstu.edu/webct/entryPageIns.dowebct
Sight for the Blind and Speech for the Deaf
A professor turns cellphones into aids for the disabled
By CATHERINE RAMPELL
Three years ago, in the depths of a Pittsburgh winter, Priya Narasimhan saw a blind man trying to catch a bus.
Stepping in and out of pools of slush, the man called out to passing pedestrians to ask if a vehicle he heard arriving was his ride home. Buses passed by.
“We can do better than that,” Ms. Narasimhan said to herself.
Ms. Narasimhan, an associate professor of electrical and computer engineering at Carnegie Mellon University, soon became the hub for student research projects that develop technologies to assist the disabled by doing such tasks as identifying buses or translating sign language into spoken words. Their creations turn the most ubiquitous device on a college campus — the cellphone — into an independence-enhancing machine.
Some of these endeavors are now being spun off into a small company. Ms. Narasimhan’s and her students’ accomplishments have come after countless hours of work, some for credit but much uncredited, and almost always not financed save for a small grant cadged from the university.
Shortly after the bus incident, Ms. Narasimhan began kicking around ideas about ways to make blind people’s lives easier using technology. Her main priorities were convenience and affordability, so her first inclination was to upgrade something many blind people already use: canes. Perhaps, she thought, she could create a cane that would give audio clues to the surrounding environment.
In the process, she began consulting with Dan Rossi, a systems administrator at Carnegie Mellon who has been blind since childhood. Mr. Rossi has strong views about what kinds of technologies can help blind people. He told Ms. Narasimhan flatly that upgrading the cane, as other inventors have tried to do, was a terrible idea.
“A cane is a cheap tool,” he said. “You know, it’s 20 bucks. You can break them, you can throw them away, you can get them wet, and they don’t have to be recharged. It’s like a pencil. You really don’t want to soup up a pencil.”
Casting canes aside, the budding engineers starting looking at cellphones, which can be bought already outfitted with text-to-speech software and which many disabled people also already use. So far Ms. Narasimhan has advised three student projects that adapt cellphones for use by the blind, and one for use by the deaf.
The first adaptation helps solve the problem faced by the blind man waiting for the bus. Her students’ software program allows users to retrieve scheduled bus routes on their smart phones from the transit system’s Web site. The schedules are then read aloud by the phone.
But buses tend to be off-schedule, so Ms. Narasimhan said she is also lobbying the local transit authority to give her access to buses’ GPS locations. That way a blind person can know for certain if the vehicle he hears approaching is the one he needs to board.
The second project assists blind people in shopping for groceries or other goods by connecting a tiny bar-code reader to a cellphone, which retrieves product names from a free Universal Product Code database that is already available on the Internet. This way, Mr. Rossi said, he doesn’t need a sighted person to help him determine if the cookie box he is holding is oatmeal raisin or chocolate chip.
Ms. Narasimhan is hoping to build a new version of the public UPC database that will include nutritional information, pricing, and other details that a visually impaired shopper might want to know.
Devices already exist that allow people to create custom-made bar-codes, which could be added to the new database so that blind users could label and then identify objects at home or at work.
The last vision-related project Ms. Narasimhan and her students have been working on may receive more attention thanks to a major lawsuit.
In May a U.S. appeals court ruled that the U.S. Treasury must change U.S. paper currency to make bills accessible to the blind. Unlike paper currency from most other countries, U.S. bills of different denominations are the same size and have the same texture. Blind people thus must ask sighted people to identify the bills they are given, and then usually rely on folding or organizing tricks to remember which bills are which.
Ms. Narasimhan’s students have provided an alternative. They have populated a database with images of bills, crisp and crumpled, well lit and shadowed. With special software, a blind person can take a picture of a bill using a cellphone camera. The software will transmit the picture to the database and name the bill based on an image match.
There are already text-reading currency identifiers that can also read words from a variety of other sources. A blind person using these products must zoom in directly on the word “FIVE” or number “5,” though, rather than any other part of the bill. Image matching, with the Carnegie Mellon system, does not have this limitation, though it has the disadvantage of not being able to identify unknown text such as that on menus.
While trying to secure backing for the technology projects for the blind, Ms. Narasimhan has also been advising a nascent project that uses text-to-speech software on cellphones to assist the deaf. This project involves a gesture-recognition glove that can translate hand movements, such as American Sign Language, into spoken words. When a deaf person wearing the glove makes a sign, sensors in the glove translate each hand position into words that are then read aloud by the cellphone’s text-to-speech software. That way, the deaf person can communicate with a hearing person who doesn’t know ASL.
This project is still in the early stages and right now can translate only a few test gestures — a thumbs-up sign triggers the phrase “Go, Pens!,” for example, in honor of the Pittsburgh Penguins.
Despite the financial straits Ms. Narasimhan’s students say they are in, and the fact that they are no longer receiving course credit for this work, they devote many late nights and weekends to the assistive-technology projects.
“I spend a little more time on this stuff than I should be, at least if I want to graduate anytime soon,” said Patrick E. Lanigan, a graduate student who has been working on the technologies for the blind. But, he and his colleagues say, in this kind of work, they are motivated by more than the desire to obtain a degree, and have learned to get a lot of work done even when resources are scarce.
“This has mostly been a soup-kitchen kind of project,” says Ms. Narasimhan.
Section: Information Technology
Volume 54, Issue 42, Page A13
Entry filed under: Uncategorized.