The Michigan Engineer News Center

How to make the robot revolution serve the people

Story by: Dan Newman

In a new building soon to open, Michigan roboticists will shape a world where robots empower, instead of imperil, our humanity.| Long Read

A new robot docent that helps museum visitors experience exhibits is practicing how to navigate among paintings and sculptures at the University of Michigan Museum of Art.

Like works of art, works of engineering carry forward the thinking of their creators. This is especially true for robotics, among engineering fields, as the machines themselves will interact, work with, and respond to humans based on their creator’s thoughts and design.

Xi Jessie Yang, a lead on the robot docent project, explains: “The success of an effective autonomous system depends on the approach to human-robot interaction.”

EnlargeThe robot docent at UMMA.
IMAGE:  Interacting with the robot docent at the University of Michigan Museum of Art. Photo: UMMA

An assistant professor in industrial and operations engineering, Yang learned early on in the robot docent project that the robot must do more than repeat a script or facts. In what is a common process for robotics researchers, Yang has to collaborate and depend on experts from multiple niches, not only museology, but robot hardware, mapping, navigation, and artificial intelligence.

As robotics advances, so have the challenges researchers face, both societal and technical.

“The easy problems have been solved,” said Michael Gonzalez, a Robotics PhD candidate. “Now the interesting problems require knowledge about mechanics, electronics, computing, and human-centric design, in a more interdisciplinary approach than most other fields.”

EnlargeA look at the underwater robot
IMAGE:  Gideon Billings, Robotics PhD student, and Nick Goumas, research and development engineer, both with the DROP Lab, test an underwater robot on the Great Lakes. Photo: Robert Coelius/Michigan Engineering

We are also at a societal juncture, when robotics confronts the status quo in many fields and the public rightfully fears the consequences of a more automated future: loss of jobs, increased inequality, and dehumanized interactions. These worries pervade even as robots remain rudimentary in real life compared to our imaginations. Roombas struggle to vacuum our floors without getting stuck, autonomous cars fail to see and react to their environments, and prosthetics cannot quickly adapt to a wearer’s biomechanics and intentions.

EnlargeRobotics building
IMAGE:  The Ford Robotics Building captured just after sunrise on July of 2020 on the North Campus of the University of Michigan in Ann Arbor. Photo: Levi Hutmacher/Michigan Engineering

Leave no human behind

A decade ago at the University of Michigan, a team aware of these issues set out to confront them with the goal of creating smart machines that serve society. They began their work recruiting researchers who were separated in labs across campus in order to bridge the traditional engineering disciplines and include medicine, humanities, social sciences, architecture, design, and law. They created the Robotics Institute.

Located in the College of Engineering, the institute began a graduate program from scratch. This fall, it will train 75 PhDs and over 100 master’s students. Its first class of four PhDs, graduated in 2019, split careers in industry and academia.

Now, in their most visible and pivotal effort, the robotics team will throw open the doors to a new building in 2021 that will bring together labs across campus to become a national centerpiece for robotics research, education, and collaboration.

The Ford Robotics Building, a $75 million and 140,000 square-foot complex on North Campus, has the sleek features to enable groundbreaking work: a three-story indoor fly zone for autonomous aerial vehicles, an outdoor testing playground for walking robots, high-bay garage space for self-driving cars, a lab with one of the most scientifically-advanced floors for rehabilitation and mobility robots such as prosthetics and exoskeletons, a makerspace shop to emphasize hardware along with software, and a Mars yard with imitation red soil to test rovers.

EnlargeJesse Grizzle
IMAGE:  Jessy Grizzle, Elmer G Gilbert Distinguished University Professor Jerry W, Carol L Levin Professor of Engineering, and Director of the Robotics Institute, adjusts components of MARLO in the EECS Building on June 4, 2014. Photo: Joseph Xu/Michigan Engineering

While stunning in resources and architecture, the most important aspect of the Robotics Institute might not be replicated so easily elsewhere.

“It’s the people,” Jessy Grizzle, director of robotics and the Elmer G. Gilbert Distinguished University Professor of Engineering and the Jerry W. and Carol L. Levin Professor of Engineering, says to a crop of potential PhD students. 

It might sound like a typical talking point of directors, but Grant Gibson, PhD student and recent president of the family-like and vibrant Robotics Graduate Student Council, agrees.

The culture is what set Michigan apart when I was deciding between schools. Everyone is eager to work together and help each other. Everyone.

This culture of collaboration physically manifests as shared lab spaces among tangential research groups: students focusing in walking robotics will work next to those in prosthetics, those in robotic arms next to those in robotic sensing, and those in computer vision next to those in autonomous driving. For breaks, everyone can meet at the building’s cafe. To inspire even more roboticists, building tours will be able to peer into these labs through their glass walls.

EnlargeElla Atkins with a drone
IMAGE:  Ella Atkins, professor of aerospace engineering, readies to fly a drone at the first flight of M-Air. Photo: Joseph Xu/Michigan Engineering

Designing for us all

For these researchers, the first law of robotics goes far beyond not harming humans. These roboticists build robots for humanity, and work to realize the potential that the field of robotics itself has to enable vast populations, especially those previously underserved.

“I’m from Appalachia, and one of the biggest, most exciting things for me about robotics is that you don’t leave that community behind you,” said Ella Atkins, professor of aerospace engineering. 

“There’s a real possibility to include them.”

Atkins will lead research in the three-story fly zone, which sits across the street from M-Air, the 10,000-square-foot outdoor autonomous vehicle testing lab. Matthew Romano, one of Atkins’ PhD students, utilizes the 30 motion capture cameras at M-Air in his work studying how teams of drones can cooperatively transport heavy payloads or assist in tasks, such as roofing.

Such applications require knowledge from those with related experience, creating the opportunity for the public to weigh in and contribute. To better design a roofing quadcopter, Atkins and Romano discussed with a roofer how best an autonomous machine could help their work. While attaching a nail gun to a drone was the team’s first thought in design, they found that a roofer would rather have the shingles delivered while the roofer nails.

“If you’re respecting the truck driver, or roofer in this case, as much as you respect the mathematician, it’s been a good day,” Atkins said.

This culture and its intense focus on enabling all humans will be Michigan’s hallmark on the robots, technology, students, and startups that come out of the new building’s doors, garage bays, and aerial netting.

“Just because someone invents a new technology doesn’t mean we have to use it,” said Lionel Robert, associate professor of information. “For us, we look at how the technology acts and behaves, and how people interact to see if it benefits all of society and shapes our world in a positive direction.”

Robert, a core faculty member of Robotics, studies teamwork and trust with automated systems and tries to answer questions such as, “How can pedestrians trust that an autonomous car will stop at a crosswalk?” or, “How can a construction worker feel safe and comfortable working alongside a robot?”

Robert stresses the importance of not only adopting the right technology, but getting that technology to be inclusive from the start.

“For example, right now, most of the pedestrian models being built for autonomous vehicles assume able-bodied people, but of course that’s not necessarily the case.”

“And once something is put on the road, it becomes part of the infrastructure, and it’s very difficult to go back and change. So we need to get it right, for all people, from the start.”

IMAGE:  Chad Jenkins, associate director and professor of computer science, shows the 3-D environment that Odd Job and Cookie, two of his research group's robots, perceive. Photo: Joseph Xu/Michigan Engineering

Pick-and-place collaboration

Robert works with Dawn Tilbury, a professor in mechanical engineering, and Yang, a lead on the museum robot docent project.

Like Robert, Yang focuses on how humans interact with autonomy and robots. However, while Robert is an expert in surveying and asking participants the right questions, Yang is an expert in computational models.

“Ironically, when more autonomy is introduced into a system, the more human factors matter,” Yang said. “This is because humans become increasingly out of the loop, understand less of what’s going on, and what to do when a system fails. As one example in our experiments, when an autonomous vehicle decides to hand over control to the human driver, there is a high likelihood of crashes.”

EnlargeProfessors Lionel Robert and X. Jessie Yang stand in front of an autonomous vehicle at Mcity
IMAGE:  Lionel Robert, associate professor of information, and X. Jessie Yang, assistant professor of industrial and operations engineering, stand in front of an autonomous vehicle at Mcity, an autonomous vehicle testing ground. Photo: Jeffrey M. Smith/School of Information.

While Yang provides the expertise in human-robot interaction, she leans on the museum curators to inform the team on how a docent guides a visitor’s thoughts, and on recently hired assistant professor Maani Ghaffari for how the robot will know where it is and where to go among crowds, Claude Monet, and clay sculptures.

Ghaffari, a global scholar who earned a master’s degree in Iran and PhD in Australia, came to Michigan to work with professors Ryan Eustice and Jessy Grizzle on applying math and machine learning to enable robots to plan when they are uncertain of their environment.

In robotics, this research focus is called simultaneous localization and mapping, or SLAM. Ghaffari works with sensors like lidar, the laser-based scanning of environments, and pairs them with algorithms that allow robots to plan the best move forward, no matter if it’s a step up stairs, over debris, or down a ramp.

Ghaffari’s work complements other efforts in Grizzle’s lab, which revolve around Cassie, the bipedal robot that is often the star of University videos. Trying to get a bipedal robot to walk like a human, however, has shown that two-legged walking is a complex behavior difficult to replicate with machines.

But as progress on bipedal robots marches forward, it also translates to other disciplines. In the past decade, large gantries supporting such robots have given way to simple safety mounts. And testing exclusively in labs with robots walking in fixed circles has been replaced with outdoor trials among controlled burns of forests and active construction sites.

The algorithms that control Cassie’s movements can be applied in prosthetics, as well.

“There are not many schools that have a robotics institute, as well as an amazing medical school, and an amazing controls group,” Bobby Gregg, associate professor of electrical and computer engineering, said. While studying electrical engineering and computer science as an undergrad at Berkeley, Gregg was inspired to pursue robotics after attending a seminar given by Grizzle on his early work on walking and running robots. Gregg joined the faculty in 2019, and now, he adapts Grizzle’s work on bipedal robots and uses it for powered knee and ankle prosthetics that can better mimic a natural walk. 

Elliott Rouse, assistant professor of mechanical engineering, will share a lab with Gregg and the “world’s most scientifically capable floor,” sporting in-ground force plates that measure pressure of foot falls, a two-belt treadmill that can vary speeds between legs, and a Stewart platform that, like an earthquake, can jolt who or whatever is standing on it in any direction to study recovery from unexpected movement.

In this lab, Rouse and his students will continue work on the Open Source Leg, a project that allows researchers to save millions in development while standardizing and focusing efforts on one piece of equipment to create better controls for lower limb prosthetics.

Gregg and Rouse’s prosthetics rely on anticipating where they are in a user’s gait, and what activity, such as running or climbing stairs, is being performed. In the future, such prosthetics might be better controlled with a user’s mind, which is just what Cindy Chestek, associate professor of biomedical engineering, is working on.

“No matter where you are in robotics, we’re all dealing with sensor data that we wish was better,” said Chestek. “We’re using the same suites of machine learning algorithms to try to make our devices smarter. And, we’re all dealing with what we have in terms of actuators.”

With a team that includes Alex Vaskov, a robotics PhD student, Chestek surgically amplifies the signal of nerve endings in amputees in order to give them better control of prosthetic hands.

Repeatedly from lab to lab, researchers weave together threads of their expertise, forming the core of robotics at Michigan. With contributions from all fields, the institute advances solutions for the increasingly complex hurdles in the way of wide robotics adoption.

EnlargeKaren Sussex with her prosthetic hand
IMAGE:  Karen Sussex, an upper-limb amputee from Jackson, Mich., operates a Touch Bionics I-LIMB prosthetic hand as Alex Vaskov, robotics Ph.D. candidate, looks on during a testing session at a lab in the University of Michigan Hospital in Ann Arbor. Photo: Robert Coelius/Michigan Engineering

Only the building set in concrete

Unlike the newly-laid slabs of stone, the robotics community is malleable and its members easily leave their marks.

“With a new program, we have the opportunity to build robotics into a truly inclusive quantitative engineering discipline before its culture becomes set in stone,” said Chestek, who is a member of the Robotics Institute executive committee.

“I’m like a cycling coach with my graduate students: I’m relentlessly positive and I tell them I’m going to help them do this incredibly difficult thing. With this type of culture, instead of weeding people out, we can recruit from groups that typically haven’t had much of a voice in this really cool discipline.”

Tilbury, the collaborator of Robert and Yang and a founding member of the Robotics Institute, will share a lab with Kira Barton, an associate professor of mechanical engineering. Their lab represents the future of manufacturing, with a space that will show how flexible and adaptable future factories will be – and need to be – in the face of supply chain risks like pandemics.

Barton, like Chestek, was drawn to the new institute because of the culture, and opportunity to shape it.

“One of the exciting things is that you get to pull from different backgrounds, experiences and perspectives,” Barton said. “It’s this diversity in thinking that actually brings together those creative, innovative ideas. We want to make sure that people feel comfortable to come, even with a non-engineering background and domain.”

IMAGE:  Ding Zhang, ME PhD Student, and Kira Barton, associate professor of mechanical engineering, discuss results from a recent test of a robot that uses cooperative algorithms. Photo: Joseph Xu/Michigan Engineering

And Barton, like other Michigan roboticists, recognizes the power of the developing technologies, and the need to think critically about their effects.

“We push for students to be aware of the impacts that they can have in the world, in their research and beyond.”

The ethos of inclusivity and impact are concentrated in the latest efforts of the institute: developing an undergraduate program. Led by Chad Jenkins, associate director and professor in computer science, the program will transform a “Sputnik-era” sequence of math courses into a new combination of linear algebra and calculus. By exposing students to math in combination with software and hardware early on, the team hopes to inspire further robotics study instead of discouraging potential roboticists. 

The first pilot course, ROB 101, is being taught this semester.

The students who take that class will learn that current robots, like our stuck Roombas, have yet to meet many of our needs. And when they do, we fear the potential consequences, like negotiating with the Terminator.

Michigan confronts these two looming issues and leads a new future for robotics, keeping humans top of mind. The hopeful result? A future where robotics is a force multiplier for achieving our goals: from enabling people with assistive services and advanced prosthetics, to empowering individuals in their work, to increasing opportunity and equality to those who are disadvantaged.

Robotics will be the greatest technological influence on culture in history. Michigan Robotics is aiming to make it a positive influence.

EnlargeStudents of MEZ with their robot
IMAGE:  Zannatun Alim, Tayebah Chowdhury, and Tunzzina Chowdhury, Detroit International Academy Students, rush to finish their team's robot at the annual Michigan Engineering Zone Open House in 2016. Photo: Joseph Xu/Michigan Engineering

This article is part of the Fall 2020 issue of The Michigan Engineer magazine. Visit the issue home page to see other articles.

The robot docent at UMMA.
A look at the underwater robot
Robotics building
Jesse Grizzle
Ella Atkins with a drone
Professors Lionel Robert and X. Jessie Yang stand in front of an autonomous vehicle at Mcity
Karen Sussex with her prosthetic hand
Students of MEZ with their robot
Portrait of Brad Whitehouse


Brad Whitehouse
Editor for Alumni Communications

Michigan Engineering
Communications & Marketing

(734) 647-7089

3214 SI-North

The outside of the Ford Robotics building

U-Michigan, Ford open world-class robotics complex

The facility will accelerate the future of advanced and more equitable robotics and mobility | Medium Read