For years now, Daniel Stone has been peopling many of his construction sites in part with SAMs—one-armed robots that work alongside humans, mortaring and placing brick after brick after brick in the exterior walls of massive commercial buildings. A brick-laying robot might seem like a radical new technology, but Stone knows it’s just one small step toward reinventing construction, one of the most dangerous, least efficient and most polluting human endeavors.

“It’s like we’re in a traffic jam, and we take out five cars and replace them with really fast cars,” said Stone, director of innovation at Southfield Michigan-based Barton Malow. “It’s an improvement in some ways, but you’re still in a traffic jam. What you need to do is break out of the whole mess and find a totally new way.”

A team led by Vineet Kamat, civil and environmental engineering (CEE) professor, and Carol Menassa, CEE associate professor, is working to do just that. They’re backed by the National Science Foundation, which also funded the research that underpins SAM.

The research is driven by long-festering problems in construction like chronic safety issues and a worker shortage driven by the retirement of the Baby Boom generation. The Covid-19 pandemic has only made things worse, upending jobsite dynamics and making it even more difficult for workers to get the job done while staying safe.

Even without the worker shortage, it’s clear that the jobsite is in serious need of an update—a 2020 study by consulting firm McKinsey and Company showed that productivity growth in the industry has grown by less than one percent over the past two decades, far behind the 2.8% productivity growth in the economy as a whole.

Stone and his team at Barton Malow are convinced that fundamental change is what’s needed, and that’s exactly what the U-M research team has in mind. Far from simply farming out human jobs to robots, they envision construction sites where humans and robots work as teams, replicating the master-apprentice model that has served humans for centuries—only with robotic subordinates.

A job where you’re going to blow out your shoulders in a few years lifting masonry blocks, that’s a hard sell.”

Daniel Stone

Such a shift won’t be cheap, but the money to fund it is largely already there—the McKinsey study found that research and development spending among top construction companies jumped 77% between 2013 and 2017.

Stone is ready to help make it happen, and in fact, the move to construction itself was a similarly seismic change for him. He spent his college years on North Campus, graduating from U-M with a degree in classical music performance. He was a professional bassoon player who dabbled in computer coding until a friend talked him into learning about the construction business. He found the combination of construction and technology surprisingly fascinating; after joining Barton Malow 25 years ago, he hasn’t looked back.

“Construction is really broken, and I’m going to work every day to make all of it make more sense, be more interesting, be safer, be faster, use technology better,” he said. “Really, it’s just a very interesting space to be in right now,” 

Stone believes that in addition to offloading some of the work to robot apprentices, a reinvented construction industry will make it easier to attract more human workers. With less heavy lifting and more problem solving, he predicts that the business could attract a more diverse workforce, made up partly of workers who would never have considered construction in the past.

“A job where you’re going to blow out your shoulders in a few years lifting masonry blocks, that’s a hard sell,” Stone said. “We’re trying to attract a wider range of people to the trades, people who are interested in technology. When you take the burden of lifting off people, we can attract new workers and also enable our current workers to extend their careers.”

Synergy, not replacement

More robots begetting more humans might sound counterintuitive, but broader economic data suggests that increased productivity is ultimately good for workers.

“In occupations where productivity gains have been greatest, there’s space to pay workers more,” said Betsey Stevenson, a professor in U-M’s Gerald R Ford School of Public Policy and a former economic advisor in the Obama administration. “Low productivity growth in construction is bad for workers. It’s not great to be a worker in a space where you’re not producing any more than workers of 20 or 30 years ago because that serves as a limit to wage growth.”

But there’s a reason why the construction industry has missed out on technological advances over the years. The workflow tactics that have reinvigorated other industries, like manufacturing, simply don’t translate well to the construction site.

“Construction is much more dynamic and unpredictable than an environment like a factory, so we’re working to redefine the balance between human and robot workers,” Menassa said. “Humans and robots need to coexist, and that’s the premise of what we’re doing right now.”

The team recognized early on that if they were to build a true human-robot apprentice system, they needed much more than a better robot. And that has led to partnerships that reach outside the engineering college. They’re working with School of Information associate professor Lionel Robert to find better ways to bring humans into the loop.

Robert believes that the idea of robots simply taking over human jobs is misguided, and not just because it feeds the narrative that robots are coming for our jobs. Plugging robots into existing jobs, he says, is simply inefficient. It’s a poor solution even in automation-friendly fields and will be even more problematic if it’s applied to construction sites.

“We’ve basically been treating automation as a substitution,” he said. “We take our existing five steps and maybe we have a robot do steps two through four. What we should be saying is, ‘How can a robot help a human do these steps more efficiently?’, or maybe when we have a human-robot team, we can rethink those steps altogether.”

In fact, Robert believes that if we’re to truly maximize what robots can do for us, we would be wise to learn from one of humankind’s longest-serving and most successful collaborators: the dog.

“We work together with dogs to find drugs, chase criminals, help the disabled and so many other things,” he said. “And the key to all of those tasks is for the dog and the human to work as a team. Each has capabilities that the other doesn’t, they’ve both been trained to work together, and the human is in charge. I think we’re going to see a lot of that as a model when it comes to humans and robots working together collaboratively.”

Robert is quick to point out that efficiency is only one piece of the puzzle. If robots are to make construction workplaces safer—and particularly if they’re to help attract a new generation of workers—he says we need to be intentional about making sure that the tools we create help as many humans as possible, and recognize that the responsibility for making that happen rests with everyone involved in the process.

“I worry that people think it’s all done, that the robots are coming and it’s a foregone conclusion, and we can’t shape how robots are going to be used in our society,” he said. “That’s not true at all. We can pick and choose for us as a society, and we need to take an active role in shaping the way this technology is used going forward.”

Designing for human-robot collaboration

Today, those ideas are taking some tentative steps toward being put into practice with a fresh batch of experimental robots. One of them, made by KUKA, is being tested in the lab of the George G. Brown Laboratories. Its orange, suction-tipped arm, with the help of a human coworker located anywhere on Earth, can do construction tasks like placing drywall and ceiling panels.

Menassa, a member of the team that created the robot system, explains that the challenge of creating a robot that works with humans to do construction work is as big as it sounds—but in a completely different way than one might think. The trouble lies not in the mechanics of the arm itself, which is a relatively common off-the-shelf machine, but in building a system that enables real teamwork between human and machine.

The advantages of such a system are many, starting with less need for humans to do strenuous and potentially dangerous heavy lifting. The human installer no longer needs to travel to job sites. They can now be in multiple places at once and are less likely to make a fatigue-related mistake after a long day of heaving drywall panels. And during a pandemic, fewer humans on the job site means fewer opportunities for disease transmission.

Menassa’s team devised a system called a real-time, process-level, immersive virtual-reality twin. At its heart is a virtual-reality copy of both the jobsite and the robot, which the human operator interacts with through an Oculus-style VR headset.

The human sees a video game-like clone of the workspace, informed by the building’s digital 3D CAD model, as well as data from a network of sensors on the jobsite. (The team is currently using Microsoft Kinect cameras that go for around $50.) In addition to representations of what’s on-site (for example a studded-in wall and a stack of drywall sheets), the operator sees a billboard with a set of commands and two robots—one robot is a virtual twin and one represents the actual robot.

The process starts when the human operator, using a joystick-style controller and a pointer, shows the system what needs to be done, for example, picking up a sheet of drywall and aligning it on the studded wall. The human makes planning decisions like the order in which panels are installed and their precise alignment. They can also make adjustments to compensate for built features that don’t quite match the original plans.

We’re working to design a different kind of construction site where robots perform tasks and humans are planners, facilitators and problem solvers.”

Carol Menassa

Based on these instructions, the robot devises the most efficient way to make it happen and develops a sequence of actions called a motion plan. It then demonstrates its plan to the operator on-screen using the virtual twin robot.

At this point, it’s up to the human to determine whether the proposed plan will get the job done. They can either approve the plan as-is, make changes or demand a completely new plan. When the plan is agreed upon, the human tells the robot to make it happen, and then watches in near-real time as the actual robot does the work.

The system is currently undergoing lab testing, and for now, the robot’s work is as slow and cumbersome as it sounds. But Menassa explains that the project’s immediate goal is not to create a work-ready robot, but rather to design a framework of rules and systems under which humans and robots can collaborate in the uniquely challenging constraints of a construction site. Once the ground rules are established, they can be applied to any number of construction tasks, potentially opening the door to versatile robots that can work with humans on a variety of tasks to maximize their cost-effectiveness.

“It’s important to realize that the goal of this project isn’t simply to install drywall faster,” Menassa said. “We’re working to design a different kind of construction site where robots perform tasks and humans are planners, facilitators and problem-solvers. This kind of construction could be much safer, more efficient and cost-effective.”

Another group of U-M researchers is at work in an area of construction that’s in particular need of safety and efficiency improvements—roofing. With workers spending their days perched high off the ground in searing heat, the job is one of the most difficult and dangerous in construction. And Ella Atkins, a U-M professor of aerospace engineering and robotics, is working out the fundamentals of using drones to make it more efficient and safer.

Like Menassa’s robots, Atkins’ roofing drones are no match for humans in terms of speed. When Atkins’ team developed an autonomous drone that could nail down shingles in 2019, human roofers crowed that it was far too slow to replace them. But again, the goal wasn’t to put an army of drones to work nailing shingles. It was to develop a fundamental understanding of how humans and machines might work together in a specific environment.

“For me, the biggest excitement from this is in recognizing that autonomous, useful physical interaction between drones and humans is possible in construction,” Atkins said. “My husband and I re-roofed our house when we were grad students, and honestly, we weren’t much faster than this robot. That evolution from being a complete novice to being successful needs to happen in mechanical systems as well, and a proof of concept like this nailing drone is an important step.”

Will we welcome robot underlings?

Perhaps it’s not surprising that technologists, engineers and information architects are all-in on robot apprentices. But what about the humans who might one day be the masters of those apprentices? The team recognizes that, without buy-in from human workers, their robots—no matter what they call them—are just another piece of equipment.

Here again, Stevenson points out that there are lessons to be learned from history—in this case from the initial encounters between desktop computers and office workers.

“In the early days of desktop computers, companies just plopped computers on people’s desks, but those companies didn’t know how to use computers to boost the productivity of their workforce,” she said. “This became known as the productivity paradox—famously summarized by Robert Solow, who said ‘Computers are everywhere except in the productivity numbers.’”

It’s important, she says, to avoid falling into the same trap with other kinds of automation, and one key step is investment in training and other measures that will not only help workers with nuts-and-bolts issues, but also help them feel good about next-gen job sites.

That might sound a little new-agey for a construction site. But researchers have found that feeling good about your co-workers—human or otherwise—is about more than just happiness. It also has a big effect on safety.

U-M civil and environmental engineering professor SangHyun Lee has spent years researching construction site dynamics. He explains that the safety of workers is keenly affected not just by actual dangers but by the perception of danger. If a worker feels unsafe around a piece of equipment—like a robot teammate—it actually impacts their performance and can increase the likelihood of an accident. That means that building human workers’ confidence in their robotic counterparts is a top priority.

“Most construction injuries are related to unsafe behavior by humans, and stress is a major indicator of potentially unsafe behavior,” Lee said. “If a worker feels stressed and begins to make mistakes or cut corners, that can increase the risk of an accident.”

Lee has spent years working to improve the safety of construction sites. By outfitting construction workers with wearable biosensors that measure brain activity and physiological status, as well as stress responses like increases in heart rate and perspiration, he looks for ways to reduce worker stress and accidents.

On the jobsite, Stone has found that the rudimentary robots they’ve developed so far—the SAM masonry robot and a second robot, called the MULE, that lifts heavy concrete landscaping blocks—have actually reduced worker stress by reducing heavy lifting and enabling human workers to space themselves further apart during the pandemic. And because there’s a serious shortage of human workers, they don’t cost any humans their jobs.

But a robot that places bricks is a far cry from a true robot “apprentice” that would have humans and robots collaborating in a more sophisticated way. And Lee’s research has found that humans are not always as willing to buddy up with robots as the researchers had hoped.

In a recent study, he and his team had construction workers interact with the robots in the lab. Outfitted with a set of the same sensors Lee uses on construction sites, participants interacted with actual robots or immersed themselves in a virtual-reality construction site, where they built a simple masonry wall with the help of a robot.

The virtual setup enabled the team to test multiple scenarios; for example, some environments had human and robot working side-by-side, while others separated them into distinct work areas divided by a safety fence.

Study participants found working with the robot to be off-putting, fearing they might be hit by a robotic arm and struggling to predict the movements of the unfamiliar machines. Ultimately, the researchers found that the safety fence was important not because it protected the workers physically, but because it made them feel safer.

Lee says the study shows the unique complexities of constantly-changing construction sites, where maintaining physical separation and barriers can be much more difficult than in more regimented workplaces like factories. It also underscores how important it is for engineers to look beyond technical solutions and think about the needs of human workers.

“Previous research has focused mostly on the characteristics of the robots themselves—their speed, how they moved, whether they should warn humans when they’re nearby,” Lee said. “But it’s not just about making the robots work better. It’s about making the team work better.”

New opportunities in construction

Still, construction workers have always been a resourceful group. And there’s some evidence that the shifts that need to happen, like new kinds of training, are happening organically within the industry itself.

Stone’s daughter Adrionna, for example, is a 27-year-old technologist who currently works in IT tech support. She grew up around construction sites, where she was generally the only woman, and she sees workers adapting to new technology and taking on jobs far more advanced than those of the past.

Adrionna Stone, in fact, is looking at getting into the business herself. She sees opportunity in the field of 3D modeling, where she would create the digital blueprints that are already common in large construction projects. Those models might one day power robots like the ones Menassa’s team is developing.

“When I go to construction sites, there are a lot of people who are doing very advanced jobs like project management and digital modeling, that they didn’t even know existed before they got into construction,” she said. “I wish more people knew about the opportunities that exist, knew that it’s not just backbreaking labor.”

She believes that, in addition to changing the way work is done, technology may drive a shift in the culture of the construction industry as well, from a business that’s overwhelmingly male and change-averse, to a more diverse field that’s open to new ways of doing things.

“I think people of my generation and younger are just naturally comfortable with technology, and if they see it being used in construction, it gives them something in common with the business, sort of an in,” she said. “I think attracting more kinds of people will make the business more efficient and better at solving problems, and in time, I want to do everything I can to be a part of that change.”