A new software system developed at the University of Michigan uses video game technology to help solve one of the most daunting hurdles facing self-driving cars – the high cost of the laser scanners self-driving cars use to determine their location.
Ryan Wolcott, a doctoral candidate in computer science and engineering, estimates that it could shave thousands of dollars from the cost of self-driving vehicles. The technology enables them to navigate using a single video camera, delivering the same level of accuracy as laser scanners at a fraction of the cost. His paper detailing the system recently was named best student paper at the Conference on Intelligent Robots and Systems in Chicago.
“The laser scanners used by most self-driving cars in development today cost tens of thousands of dollars, and I thought there must be a cheaper sensor that could do the same job,” he said. “Cameras only cost a few dollars each and they’re already in a lot of cars. So they were an obvious choice.”
His system builds on the navigation systems used in other self-driving cars that are currently in development, including Google’s vehicle. They use three-dimensional laser scanning technology to create a real-time map of their environment, then compare that real-time map to a pre-drawn map stored in the system. By making thousands of comparisons per second, they’re able to determine the vehicle’s location within a few centimeters.
Wolcott’s system uses the same approach, with one crucial difference: his software converts the map data into a three-dimensional picture much like a video game. The car’s navigation system can then compare these synthetic pictures with the real-world pictures streaming in from a conventional video camera.
Ryan Eustice, a U-M associate professor of naval architecture and marine engineering who is working with Wolcott on the technology, said one of the key challenges was designing a system that could process a massive amount of video data in real time.
“Visual data takes up much more space than any other kind of data,” he said. “So one of the challenges was to build a system that could do that heavy lifting and still deliver an accurate location in real time.”
To do the job, the team again turned to the world of video games, building a system out of graphics processing technology that’s well known to gamers. The system is inexpensive, yet able to make thousands of complex decisions every second.
“When you’re able to push the processing work to a graphics processing unit, you’re using technology that’s mass produced and widely available,” Eustice said. “It’s very powerful, but it’s also very cost-effective.”
The team has successfully tested the system on the streets of downtown Ann Arbor. While they kept the car under manual control for safety, the navigation system successfully provided accurate location information. Further testing is slated for this year at U-M’s new Mobility Transformation Facility test track, set to open this summer.
The system won’t completely replace laser scanners, at least for now—they’re still needed for other functions like long-range obstacle detection. But the researchers say it’s an important step toward building lower-cost navigation systems. Eventually, their research may also help driverless vehicle technology move past map-based navigation and pave the way to systems that see the road more like humans do.
“Map-based navigation is going to be an important part of the first wave of driverless vehicles, but it does have limitations; you can’t drive anywhere that’s not on the map,” Eustice said. “Putting cameras in cars and exploring what we can do with them is an early step toward cars that have human-level perception.”
The camera-based system still faces many of the same hurdles as laser-based navigation, including how to adapt to varying weather conditions and light levels, as well as unexpected changes in the road. But it’s a valuable new tool in the still-evolving arsenal of technology that’s moving driverless cars toward reality.
The paper is titled “Visual Localization within LIDAR Maps for Automated Urban Driving.” This work was supported by a grant from Ford Motor Company via the Ford-UM Alliance under award N015392; Wolcott was supported by The SMART Scholarship for Service Program by the United States Department of Defense.