New sensation: Mind-controlled prosthetic hand goes beyond precise control

Our cover story for the fall 2019 issue invited readers into a hopeful future for the millions of individuals suffering from limb loss. Mind-controlled prostheses offer to give amputees reliable, intuitive control of advanced robotic hands using only thought. The technology has developed in the past few years in some key ways.

At the center of this work is a convergent team of U-M researchers with backgrounds in engineering, medicine, kinesiology and other areas. They work closely with upper-limb amputees and later decided to join the team as research participants to test prosthetic hands and control algorithms repeatedly, with the hopes of one day making the lives of people with similar injuries a little bit easier. 

“Every time I’ve had the opportunity to talk to a potential future user of this technology, whether it’s a person with a spinal cord injury or a person with an amputation, I always learn something that we can take with us into making a better design,” said Cindy Chestek, an associate professor of biomedical engineering. “It’s really important to start talking with future users very early on in the process of designing anything.”

Karen Sussex is one of those future users, and she’s continued to work with the group since our last story. Back then, she would meet with the researchers in a small room off a zigzagging, nondescript medical hallway between hospital buildings and be hooked up to a large computer cart in order to operate one of the prosthetic hands. Now, she meets the team in a large, modern research lab outfitted with motion capture technology in the newly renovated Kinesiology Building (formerly the Natural Sciences Building) and the computer needed to operate the hand now fits in a backpack. Sussex even had the opportunity to get outside, walk across the street to Panera Bread, open the doors with the prosthesis, and grab and carry the bag holding her lunch back to the lab.

One of the primary components of this team’s research is the regenerative peripheral nerve interface, or RPNI, which is a tiny piece of muscle surgically attached to the end of a nerve in the patient’s arm. The nerve grows into this piece of muscle, and soon neural signals meant for the patient’s phantom hand instead cause the RPNI muscle to contract and relax in different ways. This generates easily detectable signals that can be recorded by implanted electrodes and then interpreted and used for controlling the advanced robotic hands. 

One advantage of the team’s RPNI technology was the reliability of the signals they could record. This meant that a user would only need to briefly train the control algorithm on the first day, and could use the hand for months without needing to retrain. This was in contrast with most commercially available myoelectric prosthetic systems that require daily retraining. Now, thanks to Sussex’s persistence with the research, the team has shown that this reliability lasts up to three years. 

Another exciting branch of this work has been finding a way to not just receive signals from the brain via the RPNI, but also to send signals back and provide sensation for the participant, which Sussex has been testing out.

“We can actually modulate the signal properties and move sensation from one side of her hand to the other. So she feels like it’s moving,” said Deanna Gates, an associate professor of kinesiology and biomedical engineering.  “And in another participant, when we stimulated the RPNIs, he was able to improve performance by getting stimulation during movement.”

By being able to feel what that they are touching, participants would be better able to choose how to hold something as well as how much pressure to use. The sensation could also improve proprioception, or the ability to intuit where the hand is in physical space.

While the functionality of the RPNI interface is progressing, the limitations of the robotic hands connected to that interface have been a challenge. One hand the team has used offers a great range of motion, but is large and heavy, which is hard for Sussex to use. The hand she uses most is smaller and lighter, but doesn’t offer the same range of motion. To address this, the team is developing a virtual reality setup that would allow participants to practice with a virtual prosthesis.

Moving forward, the team continues to work on improving their algorithms and reducing the size of the computer needed to operate the hand. Their ideal setup would be a system about the size of a bandage that would go on the arm and send signals wirelessly to the hand. They will also continue studying stimulation to understand how that can help the participant better use their prosthesis. 

---

Listen to our 2020 podcast on this research:

Video transcript

[Background music]
[Robotic hand closing]

Hamilton: It brings you back to a sense of normalcy. It’s like you have a hand again.

Chestek: Advanced prosthetic hands on the market can do lots of things.

Cederna: These are beautiful prosthetics with incredible capabilities, but we’ve had no way to control them.

Chestek: To get that kind of control, you really have to go to the nerves.

Cederna: The problem with most of the technologies we have is that the signals are really tiny. You have tiny little peripheral nerve signals and you have noise in those signals that’s about the same size. So when you try to hear what a peripheral nerve is saying you actually can’t hear it.

We designed a way to connect up with the peripheral nerves with a piece of muscle and then what happens is when a tiny little peripheral nerve signal comes down the nerve, it goes into the muscle and it becomes a huge muscle signal.

Chestek: We’ve now seen to my knowledge the largest voltage recorded from a nerve compared to all previous results. That makes these signals big enough that we can record them and interpret them for controlling a prosthetic hand.

Hamilton: It brought back into my mind the thought of: “well, if I had something like this I could actually be out working without risking hurting myself.”

Sussex: I think it’s a really good step into the future. It’s a good way to move forward for not only me but for other people.

Chestek: You can make a prosthetic hand do a lot of things but that doesn’t mean that the person is intuitively controlling it. So the difference is the person just thinks about moving.

This worked on the very first time we tried it. So now we can access signals associated with individual thumb movement, multi-degree-of-freedom thumb movement, individuated fingers, and this opens up a whole new world for people who are upper-limb prosthesis users.

[Chuckle]
[Background music gets louder]

More Michigan Engineer Look Back stories

Moving out

While flood-resiliency efforts continue, Indonesia plans to abandon Jakarta as its capital.

Focused ambitions

While hunger for an artificial intelligence that can think like a human remains unsated, AI continues to appear in our lives in smaller ways.

Peecycling

Progress continues for upcycling urine as a safe agricultural fertilizer.

Bittersweet business

Askanya Chocolates continues to churn out opportunity in an increasingly troubled Haiti.

Photos by Evan Dougherty