Engineering students, teen create tech for the blind
A 17-year-old Ypsilanti high school student who is visually impaired worked with a software engineering class to develop technology that could make life easier for the blind.
Michigan Engineering News
India West remembers what it was like to see. She was born with full vision, but lost it at about 4 years old when a benign brain tumor tangled with her optic nerve. A 22-hour-surgery saved her life, but took away most of her sight.
“I wasn’t used to walking through the world in the dark,” said West, 17. “It was really scary, not knowing how to rely on my other senses.”
In the years since, some vision has returned. She tests at 20/400 on the doctor’s office eye chart. That’s right at the border of severe and profound visual impairment.
West can see the sky, she says, and make out light and dark. She can identify things around her house. She uses a cane to navigate. But she can’t see faces or facial expressions. She can’t tell colors. She can’t read words on a computer screen.
Much of the world remains closed off to West, and to the visually-impaired community as a whole. It doesn’t have to be that way, say West and her mother, Elena Kaltsas. In an effort to help change the status quo this past semester, West worked with students at the University of Michigan to develop technologies to make life easier for the visually impaired.
Once a week, she came to David Chesney’s Software Engineering class (EECS 481) – a capstone senior-level course. Chesney, an instructor in computer science and engineering, is known around campus for his Gaming for the Greater Good initiative. For nearly a decade, he’s been combining video game development, software engineering, and altruism – teaching his students to use their skills to benefit diverse populations.
At first, students focused solely on video game development, but several years ago, he expanded the scope. Now, each semester they work with groups of children or individuals with disabilities to engineer a variety of assistive technologies tailored to meet their needs. Chesney has involved people with autism, cerebral palsy, Bell’s palsy and now visual impairment. He and his students see those who agree to work them as clients.
The instructor knew West was a good fit for the role when she called him “Dave” the first time they met. She addressed him that way all semester, even while his students called him “Dr. Chesney.”
She owned the room, even if she found it hard to navigate.
“Dave really knows how to make this classroom complicated,” she muttered one afternoon as she moved from project to project.
“How you doing, India,” a student asked her.
“I’m livin’ life!” she answered cheerfully. “ Whatcha workin’ on?”
“We’ve got some cool stuff to show you,” the student replied.
Before he could, West showed off a necklace she made by herself. She offered to make him one, as long as he didn’t care what color it was.
West had come to the initial class with some ideas for the students: A talking oven. A smart grocery cart that would tell you where you are in a store. A way to know where her friends are sitting in a busy lunchroom.
Given the class’s constraints, those weren’t ideas they could bring to life. But the students listened to West’s experiences and came up with their own solutions to some of her day-to-day difficulties.
Noah Duchan and his group made Smart Shelf, a talking piece of furniture that can tell West what’s on it and where, and also describe the location of open areas. To build it, they linked a Raspberry Pi computer with an Amazon’s Echo smart speaker and an Amazon Alexa intelligent personal assistant.
Duchan demonstrated: “Alexa, ask Smart Shelf where the water bottle is.”
Alexa answered: “Your water bottle is on the right side of the top shelf.”
West was awestruck. “Oh my god!” she responded.
She tried it, asking what areas were open to set stuff down on. Alexa listed locations while Duchan looked on, smiling.
“It’s a really cool project,” Chesney said. “The students are taking this interesting off-the-shelf technology and combining it in ways the producers of the technology would never envision. That’s what we try to do here.”
Working with a customer… who says these are the things I like and these are the things I don’t, is absolutely invaluable…
Michigan Engineering Lecturer David Chesney
Many of the teams will be working to give West access to these technologies next semester. Other projects include:
- Steereo, a Google Map-based iOS navigation app that stores favorite routes, plays sound in the direction the user is supposed to travel, and adjusts distances to number of steps. The steps conversion idea came from West, said team member Leda Daehler. “Many visually impaired people count steps to measure distance, so we used the iPhone’s internal pedometer to put all the directions in terms of the user’s personal step size,” Daehler said. “Estimated times are also in terms of the user’s pace.”
Several fun, tablet-based games that use only audio cues. Chesney says theses students filled a void. “There are not many fun games that depend heavily upon only audio.” - A standard for putting RFID tags on objects so that they can announce themselves to a blind person’s smartphone as they move through a building. This project will continue in Winter 2017.
- A series of online coding lessons for people who are visually impaired. West had expressed interest in learning programming. “We looked into whether there were any coding tutorials for the visually impaired and there weren’t,” said student Gabriel Pascualy. “Programming’s a very visual thing and translating it into the context of not being able to see was difficult.”
West could see her future in programming – specifically in software development for the visually impaired. She’s taken a short course and volunteered to test apps for accessibility. Her work with this software engineering class may have bridged the gap in a more indirect way.
“Working with these customers with different abilities gives students a completely different perspective on how to develop software,” Chesney said. “There’s this computer science myth that we build this thing and then we wait in our garage and somebody shows up to buy this thing and lavish money on us. But the reality is working with a customer, someone who looks at the project and says these are the things I like and these are the things I don’t, is absolutely invaluable to a computer scientist, particularly when their usage patterns are completely different than the developers’.
Leda Daehler, who worked on the Steereo team and graduated in December, says she’ll bring this with her when she starts a job next year at Ford, working on autonomous vehicle systems.
“By talking with India and seeing her perspective, I started thinking about the importance of considering users different from yourself during development,” she said. “There has been quite a bit of news about the blind community and self-driving cars. Now I am much more apt to think about small things that can be implemented to make driving accessible for the blind, or other people that have a lot to gain through autonomous vehicles.”
Next year will be West’s last as a high school student. She plans to continue collaborating with Michigan Engineering, working with the students who developed the standard for putting RFID tags on objects. Through the Multidisciplinary Design Program, they’ll turn the Bob and Betty Beyster building into a “smart building,” where objects identify themselves as West moves through.
And in the summer, against the odds, West will graduate high school. Less than a third of visually impaired students ever do. And as a result, nearly two thirds of that population live at or beneath the poverty level.
“This is a reality that does not have to exist,” said Elena Kaltsas, West’s mom. “It’s not because blind students can’t do the work. Rather, it is due to an unfortunate reality in which many public school systems fail to embrace and integrate teaching methods that adapt to diverse and unique needs of the blind community, despite individualized special education mandates.”
To celebrate West’s achievements, the mother and daughter will head to the ocean in May, on West’s 18th birthday.
“We have a trip planned to California and I’m thinking of all the things that I take in visually and I’m sad that she’ll miss that,” Kaltsas said. “But she’s going to experience it in a way that I can’t. I almost think her experience will be richer than my own, as she always tunes into her environment in the most beautiful way, despite her visual loss.
“There’s never a negative word that comes out of her mouth. India’s never angry, she’s never disparaging. She’s never once said, ‘Why me?’ She has only talked about her blindness as if it were a gift. She wants to leverage her “disability” to create change in the world, to challenge people to think differently, to live out of the best part of themselves. I don’t know anyone else in my life who is that way. Usually, it’s the parent teaching the child, but not in this situation.”