Michigan Engineering News
Debevec’s groundbreaking imaging work revolutionized the film and television industry, helping create the special effects seen in Spider-Man 2, Avatar, The Hobbit, Blade Runner 2049, Gravity, The Mandalorian, and many more.
Great Scott! A flying Chevette!
Well, actually, a computer-generated visualization of Paul Debevec’s car, inspired by the opening sequence of Back to the Future, Part 2. Debevec created the model by combining techniques from computer vision and computer graphics while he was an undergrad at the University of Michigan in 1991. The passion project set him on a career path that would one day revolutionize filmmaking and earn him The Charles F. Jenkins Lifetime Achievement Award given by the Emmys.
“This all started with the Chevette Project,” Debevec said, while accepting the award at the 74th Engineering, Science & Technology Emmy Awards on September 28th, 2022. “So, thanks to my mom for giving me a 1980 Chevette. If she’d given me a cooler car, I wouldn’t have had to put so much effort into trying to make it fly using computer graphics.”
Debevec was honored for his pioneering work combining high dynamic range imagery with image-based lighting, which makes it possible to record and reproduce the light of both real and virtual environments to create a seamless transition between the two. His methods are now essential techniques used in computer graphics for VFX and Virtual Production.
What’s more, Debevec pioneered the Light Stage, which uses LED lighting in virtual production and is becoming the mainstay tool for lighting actors on virtual stages. For example, the Disney+ Star Wars show The Mandalorian uses this technology to project digital environments behind the actors on the set, creating a more detailed, realistic environment for the actors and crew to interact with. This involved building a 75’-diameter performance space made of LED walls that projects dynamic digital environments, allowing for pixel-accurate tracking and perspective-correct 3D imagery rendered at high resolution via systems powered by NVIDIA GPUs. This tool is expected to become the norm for all kinds of film and television productions going forward.
Debevec has worked directly on many films, including Spider-Man 2, where his techniques made possible the famous close-up shot of Doctor Octopus (Doc Ock) drowning in the river at the end. He’s also known for Avatar, The Hobbit, Blade Runner 2049, and Free Guy.
But one of his most impactful works was his 1999 short film Fiat Lux. In it, he digitally recreates St. Peter’s Basilica. Falling mirrored globes, as well as tumbling life-sized dominoes, are an abstract representation of the conflict between Galileo and the church. The film showcased new techniques in image-based modeling, rendering, lighting, and high dynamic range rendering, and it led Debevec to his next project – applying the same techniques to accurately recreate human faces.
Debevec created the first Light Stage to study how light reflects off skin, eyeballs, and bone structure at different angles. By incorporating details about the translucency of skin, which makes us look different that plastic mannequins, Debevec paved the way for digital renderings of humans that look almost real.
“All computer graphics challenges are about, how do you render realistic environments? How do you render realistic people and clothing hair and all of this?” Debevec said. “What’s going to happen in the future is that AI will be able to generate animations of people who don’t exist, and it will be so real you won’t be able to tell the difference.”
What’s going to happen in the future is that AI will be able to generate animations of people who don’t exist, and it will be so real you won’t be able to tell the difference.
Paul Debevec
This new frontier of filmmaking will not require living actors to simulate realistic-looking people on screen. It has the potential to revolutionize not only movies, but video games, advertising, and other media as well. But Debevec does not think this will replace movie-making as we know it – or actors for that matter.
“I think back to when movies first came out,” Debevec said. “I wonder what actors thought of this new medium where you can be recorded on film and your performance can be re-lived even after you’re gone. What a mind warp that would have been! But people adapted to it, and then they adapted to talkies.”
For Debevec, the new technology simply opens up more avenues for creative storytelling, and, he hopes, will allow more people to tell more kinds of stories. In addition, it can enrich interactive virtual entertainment, like online games, but there’s still much that can’t be replaced.
“We’ll always have that prerendered, pre-authored entertainment,” Debevec said. “There’s a great deal of artistry required for really interesting plot twists and well-developed characters. These new tools really help democratize the process, and I’m thrilled to be part of that. I hope to honor all of the history and help take it forward in a good way.”
Debevec grew up in Illinois. His father was a professor of nuclear physics at the University of Illinois, which partly inspired Debevec’s interest in academia, and his mother was a social worker. Debevec loved both art and math, and he especially loved movies. As a kid, he was fascinated by a documentary about the making of the special effects seen in the Star Wars films.
“It was adults getting to play with grown up toys – stop motion animation and blue screens and compositing and motion control cameras – and just doing very cool stuff,” Debevec said. “I loved the idea of contributing to a cool storytelling process, like making it look like people are flying through space, because it felt like a way to be connected to a larger audience. The final result is something that you can share with anyone.”
I loved the idea of contributing to a cool storytelling process, like making it look like people are flying through space, because it felt like a way to be connected to a larger audience.
Paul Debevec
When Debevec entered high school, he joined the school yearbook team as the photo editor.
“I developed tons of film in the in the school dark room, which served me very well when I began doing movies,” Debevec said. “Some of the work in high dynamic range imaging that the Emmy is recognizing me for was inspired from my work on photography in those early days.”
But Debevec’s biggest interests always lay in computer programming and mathematics. He spent a lot of time tinkering around with old commodore computers, getting familiar with the rudimentary aspects of computer graphics and design.
“I had no idea that you could actually study computer science,” Debevec said. “It felt so new, like my own little hobby. But when my dad drove me to Michigan for campus visit day, I was looking through the list of majors and saw you could study computer engineering, and I thought, ‘I like computers! I should study that.’”
Debevec chose Michigan for its midwestern charm and excellent academic reputation, but he also found himself enjoying the sport culture (in Debevec’s freshman year, Michigan football won the Big10 championship and defeated USC in the Rose Bowl, and Michigan Basketball won the NCAA National Championship). Above all, he was most excited about the variety of opportunities available to him as a wolverine.
“I am very glad that I went to Michigan,” Debevec said. “Michigan prepared me a lot for embracing the sheer variety of ideas out there, and it’s a place where you can find what you need. If you want to do something that might be a little different, Michigan will make it happen.”
Michigan prepared me a lot for embracing the sheer variety of ideas out there, and it’s a place where you can find what you need.
Paul Debevec
Debevec ended up double majoring in Computer Engineering and Mathematics, which meant he got to embrace the experiences of both the College of Engineering and the College of Literature, Science, and the Arts (LSA). In LSA, he took classes on Shakespeare, Egyptology, and modern European History. These experiences sparked his interest in archaeology and later inspired the subject of his groundbreaking 2004 short film, The Parthenon, where he debuted some of his early techniques using high dynamic range imaging, image-based lighting, and photogrammetry.
As an undergrad living on North Campus, Debevec bonded with his synthesizer-playing roommate and took the graduate-level Computer Vision class taught by Prof. Ramesh Jain, a renowned researcher in Computer Vision, Artificial Intelligence, Multimedia Computing, Experiential computing, and Digital Health, who founded and directed Michigan’s Artificial Intelligence Laboratory in 1987.
“That was the most influential class I took at Michigan,” said Debevec, who also acknowledged Jain during his Emmy-accepted speech. “I got really excited about this idea that you can throw images into a computer and pop out a 3-dimensional model that you can fly around. That’s the course where I had the idea that if you solve for a 3D model of a scene from photos and you re-project that photo back onto the scene, it creates a very photo-real look. It’s like a texture map of the scene.”
Jain wrote Debevec a letter of recommendation for grad school at UC Berkeley, and it was at Berkeley where Debevec’s influence on the film industry became official. His 1997 short film, The Campanile Movie, demonstrated new image-based modeling and rendering techniques, in particular those from the “Facade” photogrammetric modeling system that was the focus of Debevec’s PhD thesis.
The film premiered at the SIGGRAPH 97 Electronic Theater in Los Angeles, where it attracted the attention of visual effects supervisor by the name of John Gaeta. Gaeta used these techniques to create the Oscar-winning, iconic visual effects of the bullet scene in The Matrix.
“That was an incredible moment,” Debevec said. “Here was this little project I did in lab, and it ended up playing a small but significant part of something that was absolutely huge –that reached so much of the world.”
Here was this little project I did in lab, and it ended up playing a small but significant part of something that was absolutely huge –that reached so much of the world.
Paul Debevec
While Debevec was becoming known in the movie industry, he decided to stay rooted in academia.
“I was always looking for ways to stay within academia and publish papers and do research, because that was a really good place,” Debevec said. “We get to decide what to do with a problem, and then Hollywood adopts it, and it gets associated with something much bigger than us. Research is everything.”
Today, Debevec is Adjunct Research Professor at the USC Institute for Creative Technologies, as well as the Director of Research, Creative Algorithms, and Technology at Netflix. He has won two Academy Awards – a Technical Achievement Award and a Scientific and Engineering Award. He is also a governor of visual effects branch of the Academy of Motion Picture Arts and Sciences. The Emmy lifetime achievement award recognizes “a living individual whose ongoing contributions have significantly affected the state of television technology and engineering.”
“I couldn’t possibly be happier about receiving the award,” Debevec said. “It recognizes a lot of things, especially the fundamental work that went into this. Basically, every major visual effects movie for the past 20 years has made use of this tech, so I really appreciate the Emmy’s for digging into that history. I’m looking forward to working with my group at Netflix to innovate even further, though it’s a lifetime achievement award, so I guess it means I could just go to the beach.”
