How to make the robot revolution serve the people
In the midst of outfitting a new building, Michigan roboticists shape a world where robots empower, instead of imperil, our humanity.
Michigan Engineering News
Written by Dan Newman
A new robot docent that helps museum visitors experience exhibits is practicing how to navigate among paintings and sculptures at the University of Michigan Museum of Art.
Like works of art, works of engineering carry forward the thinking of their creators. This is especially true for robotics, among engineering fields, as the machines themselves will interact, work with, and respond to humans based on their creator’s thoughts and design.
A lead on the robot docent project, Xi Jessie Yang explains. “The success of an effective autonomous system depends on the approach to human-robot interaction.”
An assistant professor in industrial and operations engineering, Yang learned early on in the robot docent project that the robot must do more than repeat a script or facts. In what is a common process for robotics researchers, Yang has to collaborate and depend on experts from multiple niches, not only museology, but robot hardware, mapping, navigation, and artificial intelligence.
As robotics advances, so have the challenges researchers face, both societal and technical.
“The easy problems have been solved,” said Michael Gonzalez, a Robotics PhD candidate. “Now the interesting problems require knowledge about mechanics, electronics, computing, and human-centric design, in a more interdisciplinary approach than most other fields.”
We are also at a societal juncture, when robotics confronts the status quo in many fields and the public rightfully fears the consequences of a more automated future: loss of jobs, increased inequality, and dehumanized interactions. These worries pervade even as robots remain rudimentary in real life compared to our imaginations. Roombas struggle to vacuum our floors without getting stuck, autonomous cars fail to see and react to their environments, and prosthetics cannot quickly adapt to a wearer’s biomechanics and intentions.
Leave no human behind
A decade ago at the University of Michigan, a team aware of these issues set out to confront them with the goal of creating smart machines that serve society. They began their work recruiting researchers who were separated in labs across campus in order to bridge the traditional engineering disciplines and include medicine, humanities, social sciences, architecture, design, and law. They created the Robotics Institute.
The Robotics Institute, in the College of Engineering, began a graduate program from scratch. It now trains 75 PhDs and over 100 master’s students. Its first class of four PhDs, graduated in 2019, split careers in industry and academia.
Now, the robotics team is throwing open the doors on the most visible and pivotal point of their efforts: a new building that will bring together labs across campus to become a national centerpiece for robotics research, education, and collaboration.
The Ford Robotics Building, a $75 million and 140,000 square-foot complex on North Campus, has the sleek features to enable groundbreaking work: a three-story indoor fly zone for autonomous aerial vehicles, an outdoor testing playground for walking robots, high-bay garage space for self-driving cars, a lab with one of the most scientifically-advanced floors for rehabilitation and mobility robots such as prosthetics and exoskeletons, a makerspace shop to emphasize hardware along with software, and a Mars yard with imitation red soil to test rovers.
While stunning in resources and architecture, the most important aspect of the Robotics Institute might not be replicated so easily elsewhere.
“It’s the people,” Jessy Grizzle, director of robotics and the Elmer G. Gilbert Distinguished University Professor of Engineering and the Jerry W. and Carol L. Levin Professor of Engineering, says to a crop of potential PhD students.
It might sound like a typical talking point of directors, but Grant Gibson, PhD student and recent president of the family-like and vibrant Robotics Graduate Student Council, agrees.
“The culture is what set Michigan apart when I was deciding between schools. Everyone is eager to work together and help each other. Everyone.”
This culture of collaboration physically manifests as shared lab spaces among tangential research groups: students focusing in walking robotics will work next to those in prosthetics, those in robotic arms next to those in robotic sensing, and those in computer vision next to those in autonomous driving. For breaks, everyone can meet at the building’s cafe. To inspire even more roboticists, building tours will be able to peer into these labs through their glass walls.
Designing for us all
For these researchers, the first law of robotics goes far beyond not harming humans. These roboticists build robots for humanity, and work to realize the potential that the field of robotics itself has to enable vast populations, especially those previously underserved.
“I’m from Appalachia, and one of the biggest, most exciting things for me about robotics is that you don’t leave that community behind you,” said Ella Atkins, professor of aerospace engineering.
“There’s a real possibility to include them.”
Atkins will lead research in the three-story fly zone, which sits across the street from M-Air, the 10,000-square-foot outdoor autonomous vehicle testing lab. Matthew Romano, one of Atkins’ PhD students, utilizes the 30 motion capture cameras at M-Air in his work studying how teams of drones can cooperatively transport heavy payloads or assist in tasks, such as roofing.
Such applications require knowledge from those with related experience, creating the opportunity for the public to weigh in and contribute. To better design a roofing quadcopter, Atkins and Romano discussed with a roofer how best an autonomous machine could help their work. While attaching a nail gun to a drone was the team’s first thought in design, they found that a roofer would rather have the shingles delivered while the roofer nails.
“If you’re respecting the truck driver, or roofer in this case, as much as you respect the mathematician, it’s been a good day,” Atkins said.
This culture and it’s intense focus on enabling all humans will be Michigan’s hallmark on the robots, technology, students, and startups that come out of the new building’s doors, garage bays, and aerial netting.
“Just because someone invents a new technology doesn’t mean we have to use it,” said Lionel Robert, professor of information. “For us, we look at how the technology acts and behaves, and how people interact to see if it benefits all of society and shapes our world in a positive direction.”
Robert, a core faculty member of Robotics, studies teamwork and trust with automated systems and tries to answer questions such as, “How can pedestrians trust that an autonomous car will stop at a crosswalk?” or, “How can a construction worker feel safe and comfortable working alongside a robot?”
Robert stresses the importance of not only adopting the right technology, but getting that technology to be inclusive from the start.
“For example, right now, most of the pedestrian models being built for autonomous vehicles assume able-bodied people, but of course that’s not necessarily the case.”
“And once something is put on the road, it becomes part of the infrastructure, and it’s very difficult to go back and change. So we need to get it right, for all people, from the start.”
Pick-and-place collaboration
Robert works with Dawn Tilbury, a professor in mechanical engineering, and Yang, a lead on the museum robot docent project.
Like Robert, Yang focuses on how humans interact with autonomy and robots. However, while Robert is an expert in surveying and asking participants the right questions, Yang is an expert in computational models.
“Ironically, when more autonomy is introduced into a system, the more human factors matter,” Yang said. “This is because humans become increasingly out of the loop, understand less of what’s going on, and what to do when a system fails. As one example in our experiments, when an autonomous vehicle decides to hand over control to the human driver, there is a high likelihood of crashes.”
While Yang provides the expertise in human-robot interaction, she leans on the museum curators to inform the team on how a docent guides a visitor’s thoughts, and on recently hired assistant professor Maani Ghaffari for how the robot will know where it is and where to go among crowds, Claude Monet, and clay sculptures.
Ghaffari, a global scholar who earned a master’s degree in Iran and PhD in Australia, came to Michigan to work with professors Ryan Eustice and Jessy Grizzle on applying math and machine learning to enable robots to plan when they are uncertain of their environment.
In robotics, this research focus is called simultaneous localization and mapping, or SLAM. Ghaffari works with sensors like lidar, the laser-based scanning of environments, and pairs them with algorithms that allow robots to plan the best move forward, no matter if it’s a step up stairs, over debris, or down a ramp.
Ghaffari’s work complements other efforts in Grizzle’s lab, which revolve around Cassie, the bipedal robot that is often the star of University videos. Trying to get a bipedal robot to walk like a human, however, has shown that two-legged walking is a complex behavior difficult to replicate with machines.
But as progress on bipedal robots marches forward, it also translates to other disciplines. In the past decade, large gantries supporting such robots have given way to simple safety mounts, and exclusively in-lab testing with robots walking in fixed circles is replaced with outdoor trials among controlled burns of forests and active construction sites.
The algorithms that control Cassie’s movements can be applied in prosthetics, as well.
“There are not many schools that have a robotics institute, as well as an amazing medical school, and an amazing controls group,” Bobby Gregg, professor of electrical and computer engineering, said. While studying electrical engineering and computer science as an undergrad at Berkeley, Gregg was inspired to pursue robotics after attending a seminar given by Grizzle on his early work on walking and running robots. Gregg joined the faculty in 2019, and now, he adapts Grizzle’s work on bipedal robots and uses it for powered knee and ankle prosthetics that can better mimic a natural walk.
Elliott Rouse, a professor in mechanical engineering, will share a lab with Gregg and the “world’s most scientifically capable floor,” sporting in-ground force plates that measure pressure of foot falls, a two-belt treadmill that can vary speeds between legs, and a Stewart platform that, like an earthquake, can jolt who or whatever is standing on it in any direction to study recovery from unexpected movement.
In this lab, Rouse and his students will continue work on the Open Source Leg, a project that allows researchers to save millions in development while standardizing and focusing efforts on one piece of equipment to create better controls for lower limb prosthetics.
Gregg and Rouse’s prosthetics rely on anticipating where they are in a user’s gait, and what activity, such as running or climbing stairs, is being performed. In the future, such prosthetics might be better controlled with a user’s mind, which is just what Cindy Chestek, professor of biomedicine, is working on.
“No matter where you are in robotics, we’re all dealing with sensor data that we wish was better,” said Chestek. “We’re using the same suites of machine learning algorithms to try to make our devices smarter. And, we’re all dealing with what we have in terms of actuators.”
With a team that includes Alex Vaskov, a robotics PhD student, Chestek surgically amplifies the signal of nerve endings in amputees in order to give them better control of prosthetic hands.
Repeatedly from lab to lab, researchers weave together threads of their expertise, forming the core of robotics at Michigan. With contributions from all fields, the Institute advances solutions for the increasingly complex hurdles in the way of wide robotics adoption.
A building set in concrete
Unlike the newly-laid slabs of stone, the robotics community is malleable and its members easily leave their marks.
“With a new program, we have the opportunity to build robotics into a truly inclusive quantitative engineering discipline before its culture becomes set in stone,” said Chestek, who is a member of the Robotics Institute executive committee.
“I’m like a cycling coach with my graduate students: I’m relentlessly positive and I tell them I’m going to help them do this incredibly difficult thing. With this type of culture, instead of weeding people out, we can recruit from groups that typically haven’t had much of a voice in this really cool discipline.”
Tilbury, the collaborator of Robert and Yang and a founding member of the Robotics Institute, will share a lab that represents the future of manufacturing with Kira Barton, a professor in mechanical engineering. Their space will show how flexible and adaptable future factories will be, and need to be in the face of supply chain risks like pandemics.
Barton, like Chestek, was drawn to the new institute because of the culture, and opportunity to shape it.
“One of the exciting things is that you get to pull from different backgrounds, experiences and perspectives,” Barton said. “It’s this diversity in thinking that actually brings together those creative, innovative ideas. We want to make sure that people feel comfortable to come, even with a non-engineering background and domain.”
And Barton, like other Michigan roboticists, recognizes the power of the developing technologies, and the need to think critically about their effects.
“We push for students to be aware of the impacts that they can have in the world, in their research and beyond.”
The ethos of inclusivity and impact are concentrated in the latest efforts of the institute: developing an undergraduate program. Led by Chad Jenkins, Associate Director and professor in computer science, the program will transform a “Sputnik-era” sequence of math courses into a new combination of linear algebra and calculus. By exposing students to math in combination with software and hardware early on, the team hopes to inspire further robotics study instead of discouraging potential roboticists.
The first pilot course, ROB 101, will be taught Fall 2020.
The students who take that class will learn that current robots, like our stuck Roombas, have yet to meet many of our needs. And when they do, we fear the potential consequences, like negotiating with the Terminator.
Michigan confronts these two looming issues and leads a new future for robotics, keeping humans top of mind. The hopeful result? A future where robotics is a force multiplier for achieving our goals: from enabling people with assistive services and advanced prosthetics, to empowering individuals in their work, to increasing opportunity and equality to those who are disadvantaged.
Robotics will be the greatest technological influence on culture in history. Michigan Robotics is aiming to make it a positive influence.