The Michigan Engineer News Center

Gaming for the Greater Good

“If anyone deserves to play a fantasy computer game, it’s this kid.”| Long Read

The inspiration

He chalks it up to his Catholic upbringing. David Chesney can’t help but shoehorn some good into unexpected places. “I’ve always tried to build social context into the courses I teach,” said the University of Michigan computer science and engineering instructor.

So his section of ENGIN 100, a required course for Michigan Engineering freshmen, is “Gaming for the Greater Good.” Through this class and a more advanced, senior-level software engineering version, students have developed games and other apps designed to help people with special needs. So far, they’ve focused on cerebral palsy and autism.

The story of how these courses came to take their current shape begins about seven years ago when Chesney toured a special needs classroom in his school district. There, he saw the technology set-up for a quadriplegic child who had been injured in a car accident as a toddler.

“The device they had set up for him was a camera on top of his PC, and they’d put a metallic dot on his head that would move the cursor,” Chesney said. “That’s the main way he’d interface with his computer.”

Chesney was struck. He remembers thinking, “If anyone deserves to play a fantasy computer game, it’s this kid.” He lifted a Nintendo controller from a table in his office. “This is what today’s gaming interface looks like. I can barely use it.”

At the time of his tour, students in Gaming for the Greater Good were writing a lot of math and physics tutoring programs. He hadn’t yet geared the assignments toward kids with special needs. The tour had touched a nerve, but he wouldn’t figure out how to do anything about it for a few more years.

“Sometimes you have to get poked a couple times before you realize things are changing,” Chesney said.

That second poke came from Joe Kryza, the executive director of infrastructure & system operations at the U-M Health System. After the charity Mott Golf Classic worked with Dell, Cisco and Microsoft to donate laptops for patient rooms in 2002, Kryza’s office got a lot of requests to unlock the machines. Rehabilitation engineers were trying to customize them for patients with severe disabilities. Kryza started looking into customized technologies and he learned that many of them—a system that could help a patient with cerebral palsy to type, for example—were often prohibitively expensive.

“There’s got to be a way to make this cheaper,” Kryza remembers thinking. Eventually he reached out to Computer Science and Engineering for ideas. A meeting was scheduled.

“To be honest, it was a meeting I almost blew off,” Chesney said. “I’m not a big proponent of meetings.”

This one changed the course of his courses.

EnlargeMan sits at computer desk facing screen
IMAGE:  David Chesney at his desk.

A different kind of design classroom

“The end goal for the class is more than a grade. It’s to bring the best projects out into society.”

The results of Software Engineering 481, an upper-level design class, go far beyond grades both for the students and their most promising projects.

“481 is a tech incubator,” said David Chesney, Computer Science and Engineering (CSE) instructor.

In the course, he gives 50 juniors and seniors state-of-the-art tools and sics them on a tough and specialized problem. In winter 2011, the tools were Microsoft Xbox 360s and Kinect sensors. The Kinect replaces a game controller by recognizing players’ body movements. The assignment was to design games that could help kids with autism improve their focus, communication or social skills.

Each of the 50 students submitted a proposal. Chesney, health professionals and the students in the class whittled the ideas down to 10 that teams of students would push forward. Here’s a sampling:

  • DJ Recess used music to help players recognize tone, as autistic people often struggle to understand emotion through a person’s tone of voice.
  • Tickle Monster helped players assign meaning to facial expressions. Players used hand motions to tickle an avatar until the look on its face suggested the player should stop.
  • Jump Rope is a virtual version of the playground pastime that trains players to focus, even in the midst of distractions.
  • Path challenges players to trace a particular shape in the air as a way to improve concentration and gross motor skills.

Roger Archbold, a strategic alliance manager for U.S. education at Microsoft, visited the class several times to witness the progress. The company had donated two Xbox 360s and six Kinect sensors to the effort.

“I was just amazed at how sharp these students were—how far and fast they’d come since our original meeting,” Archbold said. “It was mind-boggling.”

After some game trials and expert consultation, Chesney chose to advance Jump Rope and Path toward commercialization. A group of eight students from 481 and the more basic, freshman-level Gaming for the Greater Good course committed to the projects over the summer. CSE senior Devin Smith was one of the 481 students.

“Getting the chance to work on new technology with a professor who is just as excited as you are to see the possibilities that open up from it is a great experience for everyone involved,” Smith said.

And each semester, the possibilities unfold in real time.

Two years ago, before the class was focused on autism, a student team created a solution that could enable cerebral palsy patients to use a tablet computer as a lower-cost device to translate typed text into audible speech. Individuals with cerebral palsy have trouble pushing small buttons, so they need alternatives to traditional keyboards to input text. The students’ app turned the whole screen into an adaptive technology switch—a button with rotating letters and functions, explained Joe Kryza, executive director of infrastructure and system operations at the U-M Health System.

“This app would drop the cost of this technology by 90 percent. Otherwise, you have to buy a customized high-end PC,” said Kryza, who started the Gaming for the Greater Good cascade several years ago when he approached Chesney’s department with a proposal to work together to develop beneficial technologies for patients.

The students have received inquiries from all over the world about their app. They’re refining it and hope to make it commercially available soon.

“It was a total surprise to us that the app was so good,” Chesney said. “There were wonderful students working on it, but at the end of the semester, it was like, ‘Holy Cow! We’ve pressed the right buttons, and there’s a little bit of magic here.'”

EnlargeStudents confer around a table
IMAGE:  David Chesney's students brainstorm during the Summer Working Group meeting. From right bottom to the left are: Connor Davidson, Patrick Godwin, Jonathan Hyde, Joseph Bickerstaff, Devin Smith, Sharif Moosa and David Chesney.

Beyond the classroom

An autistic child plays virtual jumprope with a Michigan Engineering senior.

An 11-year-old with a weakened arm stretches during a Kinect video game called Path. But she enjoys it.

A young tech-savvy cerebral palsy patient gives a computer science student tips to improve her tablet app.

These are scenes from an important phase in the development of the most promising projects to emerge from computer science and engineering instructor David Chesney’s classes. Teams of students who carried ideas beyond the classroom gathered input from the people they aimed to help.

EnlargeA person points to a projection on the screen
IMAGE:  ENGIN 481 student Devin Smith goes over modifications to the Jump Rope game during the Summer Working Group meeting.

Future possibilities

The students are confident that Path and Jump Rope will soon be in releasable condition for Windows 8. They’re still working out how the games will be available. Getting them on the Xbox 360 platform is a longer-term goal.

Path will be used as a brachial plexus palsy therapy tool by U-M Health System occupational therapists. They are also working to identify a family to take the game home and test it on a long-term basis.

Several dozen games—all that were developed this year and last in both classes—have been loaded on computers at Mott Children’s Hospital for use by therapists and patients if they desire.

Gaming for the Greater Good instructor David Chesney is working with Jackie Kauffman, assistant professor of rehabilitation psychology and neuropsychology, to design experiments to test the efficacy of the games.

Autism will continue to be the focus of Chesney’s freshman- and senior-level classes for the foreseeable future, but the curriculum will change as technologies do. For example, Microsoft’s latest software development kit for Kinect on Windows has a face-tracking feature that can recognize certain movements of the eye brows, lips and nose. Chesney plans to have several student groups explore that feature in Winter 2013.

And students in both classes are carrying forward the social context Chesney exposed them to. Connor Davidson, who took Gaming for the Greater Good as a freshman, figured out that he wants his career to make the world a better place. Davidson, who is now interning with Mott Family Network at the U-M Health System, is considering rehabilitation engineering.

“The class showed me that there’s a lot more I can do with my degree, and with computer programming,” Davidson said. “Now I know I would like to do some social good. That’s kind of a requirement for any job I have.”

Once you get a couple of the right people, then other people are naturally attracted to those efforts and they just started showing up to the party.David Chesney

The Magic of the Common Good

Below is a list of the various entities who came together to make this project possible:

  • David Chesney: Computer science instructor who refocused his classes to make games and other software that could help those with special needs.
  • ENGIN 481: Software Engineering: a senior-level design course with 50 students.
  • ENGIN 100: Gaming for the Greater Good: a section of a required freshman course with 60 students.
  • Joe Kryza: Executive director of infrastructure & system operations at U-M Health System. About a decade ago, Kryza noticed a need for lower-cost technologies for children with special needs. Several years ago, he reached out to Computer Science and Engineering for ideas. He found a collaborator in David Chesney.
  • Jackie Kauffman: U-M assistant professor of rehabilitation psychology and neuropsychology who came to talk to Chesney’s classes about autism and continues to work on moving the project forward.
  • Ned Kirsch: U-M associate professor of physical medicine & rehabilitation at the Medical School who is working with Chesney to find applications for the students’ games.
  • Microsoft: Donated 6 Kinects and 2 Xbox 360s. Learned of the efforts through Joe Kryza.
  • Marc Sirkin: Chief digital marketing officer for Autism Speaks. A former Microsoft employee who heard of the effort through contacts there. Sirkin flew to Ann Arbor to see the class’ final presentations and blogged about it. Sirkin has been a tireless advocate and recently featured the games at NYC Maker Faire.
  • Summer working group: Two students from the freshman class and six from the software engineering class worked over the summer to advance the Path and Jump Rope games toward releasable products.
  • Mott Family Network: A nonprofit volunteer organization that helps patients and families stay connected to life outside the hospital during their hospital stay. Freshman Connor Davidson and Patrick Godwin interned there over the summer and helped to install games from both classes into patient rooms and rehab clinics in the Health System. Joe Kryza is also affiliated with this group.
  • Brachial Plexus Palsy camp at U-M Health System: Invited U-M students to bring the games so that patients could try them out and give feedback.
  • Office of Tech Transfer: Mentor Norm Rapino provided help getting patents and advice on how to patent and commercialize Path and Jump Rope.
  • Eastern Michigan University Autism Collaborative Center: with assistance from Betty Lou Rowe, director of the summer program at the center, U-M students let autistic children play the games.
  • Autistic children: About one in 88 children in the U.S. is autistic, according to the Centers for Disease Control and Prevention. Michigan Engineering students developed video games that could help them with social interaction and concentration amid distractions.
  • Patients with brachial plexus palsy: About one in every 1,500 children in the U.S. has this disorder, which is caused by trauma to the nerves between the neck and an arm during childbirth. Michigan Engineering students developed video games that could potentially improve their therapy.
Explore:Features
Man sits at computer desk facing screen
Students confer around a table
A person points to a projection on the screen
Portrait of Nicole Casal Moore

Contact

Nicole Casal Moore
Media Relations & Research News Director

Michigan Engineering
Communications & Marketing

(734) 647-7087

3214 SI-North

A robotic, prosthetic leg

Space motor helps make robotic prosthetic leg more comfortable and extends battery life

Getting rid of some gears enabled a free-swinging knee, regenerative braking and brought the noise level down from vacuum cleaner to fridge. | Short Read