Every sixty seconds in the United States, an excavator inadvertently hits a buried utility. Striking a cable, sewer, water, gas, or fiber optic line can cause major damage as well as cut communities off from critical infrastructure. In addition, human workers often work in close proximity of excavating machines to manually ensure that engineering standards are met during construction. In order to prevent accidental utility damage, ensure the safety of construction workers and improve the excavation productivity, University of Michigan researchers have developed a new way to efficiently and safely operate an excavator using a new computer vision-based guidance system:
“In any excavation site, the operator should be constantly aware of where to dig and where not to dig,” said Vineet R. Kamat, an associate professor of civil and environmental engineering. “Half the problem is that operators need to rely on grade-checkers to manually monitor the evolving trench profile, which is where to dig. And the other half problem is that operators need to know how to avoid hitting existing underground utilities, which is where not to dig.”
This research entitled ”Georeferenced Augmented Reality For Knowledge-Based Excavator Control” was funded by the US National Science Foundation and led by Kamat and postdoctoral researcher Suyang Dong. The latest results of this work are about to be published in the proceedings of the 32nd International Symposium on Automation and Robotics in Construction and Mining.
There are several existing solutions to the problem, including state-of-the-art GPS, angular sensors and laser catchers. “However in order to allow the GPS to function accurately, it needs clear line of sight to the satellites,” said Dong. “Unfortunately GPS accuracy deteriorates in job sites with natural or artificial obstacles like trees and urban canyons, not to mention its cost which is usually too high to be affordable to small excavation contractors.”
The computer vision-based grade control system, SmartDig, works in a completely local context, avoiding the costly and somewhat unreliable nature of GPS systems. “The system consists of several low-cost off-the-shelf cameras that we link together to compose a camera network, and a set of fiducial markers, like Quick-Response (QR) Codes, placed at specific locations on the site as well as on the machine. The cameras can look at the markers and locate the position of the excavator’s articulated arms at any given time,” Kamat explained.
The most significant challenge is to achieve a localization accuracy of one inch with off-the-shelf cameras. The one inch accuracy is required by most trenching and excavation projects.
“People mainly apply QR Codes in e-commerce and entertainment context where localization accuracy is not crucial at all,” Dong said. “However, little work has been done in the engineering domain where the only thing that matters is accuracy. We have been able to achieve the goal by exploring a wide range of vision tracking algorithms, improving camera calibration methods, maintaining camera integrity against changing illumination, as well as testing different combinations of marker settings.”
The system has been tested in a 200ft water main installation project which was part of the extensive renovation of U-M West Quadrangle Residence Hall and the Cambridge House. “We had the operator dig a section of the trench using SmartDig, and then let the grade-checkers confirm the accuracy,” Dong said. ”we were thrilled to be told by the grade-checkers that it was very close. They also provided a lot of valuable feedback in addition to vouching for the system accuracy.”
Based on this feedback, the research team is currently working on improving the system speed, robustness, and working range. In addition, based on an Augmented Reality (AR) platform developed in the laboratory before, they will deliver an AR user interface that will provide the operator with visual guidance on the target profile and location of the utilities in a spatial manner.
SmartDig can not only produce highly accurate projections for excavator operators to follow, it even opens the doors to the possibility of completely autonomous excavation systems. “It is very conceivable that a computer in the excavator cabin can read a digital terrain model, understand where it is located at any given time, where the soil to be dug is located, and use this information in combination with SmartDig technology to fully autonomously conduct the excavation,” Kamat said.
About the Professor
Vineet Kamat is the Associate Professor of Civil and Environmental Engineering and Frank and Brooke Transue CEE Faculty Scholar at the University of Michigan College of Engineering. He is active in research involving Automation and Robotics, and its applications in the construction, operation, and maintenance of civil infrastructure systems. He is also conducting research in Real-Time Visualization and its applications in construction process monitoring and control.