Computer Science with Soul

So instead of one market for 70 million, there are 70 million different markets. Everyone has different needs and desires. Because of this, a lot of the companies that make assistive devices tend to be small — and their products very expensive.
david l. jaffe

lecturer at stanford university

On a desk in a lamplit corner of a sixth-grader’s bedroom are two computers – a laptop and a tablet, each tricked out to be a remarkable machine.

They belong to Grace Simon, 13, of Westphalia, Mich. They came to her from a community of students, professors, doctors and high-tech companies that formed through a class at Michigan Engineering. Grace is at the center of its syllabus.

She’s a petite teen with sandy blonde bangs and a contagious smile. She likes math, movies and chapter books. The oldest of Jennifer and Eric Simon’s four children, she enjoys hanging out with her family in her free time. But it’s hard for her to communicate extensively with them – with anyone, actually.

Grace has cerebral palsy. In her case, that means she doesn’t have the muscle control to speak or use sign language. She understands what people say to her. But for the most part, she can only respond “yes,” with a raise of her right fist, or “no,” with a shake of her head. She carries an alphabet board she can use to spell out words by pointing to one letter at a time. But that can be slow and difficult.

For the past two semesters, students in David Chesney’s senior-level software engineering class have been making tools that might broaden Grace’s abilities to connect with the world around her. While they’ve come up with several promising custom technologies, they’re also aiming for something bigger – to help fill a void in the marketplace for affordable assistive technologies.

Low-cost Solutions to High-Cost Problems

Because of who they are and what their motivations are, these students are making low-cost solutions to high-cost problems.
jacqueline kaufman

clinical assistant professor of physical medicine and rehabilitation at the u-m medical school

“There are about 70 million people with disabilities in the U.S.,” says David L. Jaffe, a lecturer at Stanford University who’s been teaching an assistive technology class in mechanical engineering there for eight years.

“That might sound like a big market. But it’s fragmented down to the individual level. So instead of one market for 70 million, there are 70 million different markets. Everyone has different needs and desires. Because of this, a lot of the companies that make assistive devices tend to be small – and their products very expensive.”

With little competition and virtually no economies of scale, prices stay high, sometimes prohibitively so. Pupil-tracking tools, for example, can let users write with their eye movements. They range from about $3,000 to more than $10,000.

“There’s not a lot of motivation to bring the cost down because so few people use it and they can contract with insurance companies,” said Jacqueline Kaufman, a clinical assistant professor of physical medicine and rehabilitation at the U-M Medical School who has advised Chesney’s classes. “Because of who they are and what their motivations are, these students are making low-cost solutions to high-cost problems.”

A mechanism on Grace’s laptop called SET Keyboard can mimic some of these pupil-tracking features with a $150 webcam – Intel’s Creative Interactive Gesture Camera. It recognizes motion in three dimensions as opposed to the typical two. The company donated six to the class.

SET Keyboard puts an oversized “qwerty” display on a computer screen and lets users pick letters by looking at them for about two seconds. In this way, it can be used to compose a text or an email. It auto-completes, or suggests words the user might be working on. And it exports to a text-to-speech clipboard. You’d need to set up another program to turn the sentences into sound. The students say those are easy to find.

The tool impressed an Intel manager who came to the class’s demo day.

“I was excited to see them using the cameras in ways that could help build the foundation for real products,” said Mike Tucker, Michigan public sector program manager at Intel. “Big companies develop things with a set of uses that serves the general purpose market and we don’t always address the areas of people with special needs.”

The cameras will be embedded in next-generation laptops and tablets that currently use Intel technology.

“Then all of a sudden,” Tucker said, “the market for these types of uses expands dramatically.”

An assistive tech incubator

It would be very exciting for her to put her own thoughts on to paper.
jennifer simon

mother of grace simon

The improvised pupil-tracker is still a work in progress for the developers as well as the first customer. Because it’s not easy for Grace to keep her head still, the students also hooked up a joystick. Grace’s mother, Jennifer Simon, is optimistic.

“It would be very exciting,” she said, “for her to put her own thoughts on to paper.”

At the moment, Grace is still getting used to all the tools that one of the students installed in her home in March. It will take time before she can use them proficiently. Though she’s an ace at driving her powered wheelchair today, she’s had ten years to practice.

For the tools themselves, Chesney is setting up a nonprofit incubator this summer to help turn them and other promising projects into viable products. He’ll involve U-M Medical School faculty, assistive technology experts and technology transfer mentors. After students take his class, they can be involved for either independent study credit or, in some cases, a paid internship.

Chesney, a lecturer in computer science and engineering, has had students who take this course develop software for people with special needs for the past three years. At first the assignment was general. The second year, students focused on autism. And the third, they narrowed not just to one condition, but one person – Grace. Concepts with commercial potential have come out of every cohort.

In 2012, there was PATH. Students designed the game to help autistic children improve their focus and gross motor skills. It’s a Kinect game, so players use Microsoft’s body-movement sensor and trace in the air the shape that they see on the screen. A neurosurgeon saw another possibility: Maybe it could work as a therapy aid for people with brachial plexus palsy. The condition is caused by birth damage to the nerves between a baby’s neck and arm, and its main treatment is regular exercise of the affected arm. Informal tests were promising, so in 2013, researchers conducted a clinical trial with patients to see if it’s actually effective. The results haven’t been published yet, but Chesney says they appear positive.

“I expect it to be the first product in the company store,” he said.

Also on the cusp is ASK Interfaces, an operating system for Android mobile devices. Several years ago students designed it as an app to bring email and texting to people with cerebral palsy or other conditions that make it hard to tap small buttons. Fairly quickly, they realized connecting it to email would be tricky because of proprietary issues. So they expanded it to become a new platform that could run behind all programs on an Android device. The system essentially turns the whole screen into a button and cycles through options.

The team has given away thousands of copies of ASK to people around the globe, said Chris McMeeking, a computer science undergrad who’s been on the ASK team from the start. It’s also the enabler of a lot of the apps and games students in Chesney’s class have made since.

McMeeking is excited about the incubator and the support it could give students. He understands how hard it is to move technology from the classroom to the marketplace. He’s been trying to do that with ASK for several years.

“The population of people who need these things doesn’t interest venture capitalists,” he said. “It’s not that they don’t care, per se. But your typical VC is looking for something that would turn a $100,000 investment into a million dollar company. The population is just not large enough to justify that investment.”

But McMeeking, who’s been a teaching assistant in Chesney’s class for more than two years, isn’t giving up. He’s wanted to run an assistive tech company since he was a teenager in Homer, Mich.

His mother cared for a young woman with special needs. She had spent years fine-tuning the speech recognition software and macros on her laptop so she could use it productively.

One day, the laptop crashed.

“She lost it all. And it just killed her will to use the computer again. It was heartbreaking.”

He remembers thinking that it shouldn’t be so hard.

The broader perspective

I’ve seen it time and time again: someone has a great idea and a prototype and they imagine it being on the market in a year’s time. But in reality, what it will take to move things from prototype to product is three to five years and several million dollars.
david l. jaffe

lecturer at stanford university

But putting in place alternatives is challenging too. Jaffe, the Stanford instructor, applauds the effort here at his undergrad alma mater. (He happens to have received his bachelor’s in electrical engineering from Michigan in 1970.) This endeavor, he says, has something important going for it.

The students are using off-the-shelf technologies like the Kinect sensors (donated by Microsoft), Intel cameras and all-purpose computers and mobile devices. Making special needs’ solutions out of mass-market tech is one good way to address demand in this $8 billion market, which IT research firm Gartner described as “underserved” in a 2013 report.

“If you’re producing a piece of assistive tech uses a mass-produced device, it’s not going to be as expensive because the device is already made in millions. That will definitely help make it marketable, as will anything that is adjustable. You can imagine if you have a variety of people, you can set it up differently for each person to optimize,” Jaffe said.

Still, he offers a grounded perspective.

“I’ve seen it time and time again: Someone has a great idea and a prototype and they imagine it being on the market in a year’s time. But in reality, what it will take to move things from prototype to product is three to five years and several million dollars,” Jaffe said. “Not a lot of people can stick with a project that long.”

Chesney hopes he’s opening students’ eyes to why they might want to. And his efforts to “impart souls into computer science people,” as School of Information senior Christopher Liu put it, seem to be sticking. At least they did for Liu.

“This class got me thinking about the morality of coding,” Liu said, “and about who needs our abilities the most. A lot of people can benefit from them, but are we placing the right amount of time and resources in the right places?”