Turning faces into thermostats

As lockdown requirements ease, COVID-19 is changing the way we use indoor spaces. That presents challenges for those who manage those spaces, from homes to offices and factories. Not least among those challenges is heating and cooling, which is the largest consumer of energy in American homes and commercial buildings. There’s a need for smarter, more flexible climate control that keeps us comfortable without heating and cooling entire empty buildings.

Now, a group of researchers at the University of Michigan has developed a solution that could provide more efficient, more personalized comfort, completely doing away with the wall-mounted thermostats we’re accustomed to. Human Embodied Autonomous Thermostat or “HEAT” is detailed in a paper published in the July 2020 issue of Building and Environment. 

The system pairs thermal cameras with three-dimensional video cameras to measure whether occupants are hot or cold by tracking their facial temperature. It then feeds the temperature data to a predictive model, which compares it with information about occupants’ thermal preferences. Finally, the system determines the temperature that will keep the largest number of occupants comfortable with minimum energy expenditure. The new study shows how the system can effectively and efficiently maintain the comfort of 10 occupants in a lab setting.

“COVID presents a variety of new climate control challenges, as buildings are occupied less consistently and people struggle to stay comfortable while wearing masks and other protective gear. HEAT could provide an unobtrusive way to maximize comfort while using less energy,” said Carol Menassa, who is an associate professor of civil and environmental engineering, the principal investigator of the project and an author on the paper. 

“The key innovation here is that we’re able to measure comfort without requiring users to wear any detection devices and without the need for a separate camera for each occupant.”

HEAT works a bit like today’s internet-enabled learning thermostats. When it’s newly installed, occupants teach the system about their preferences by periodically giving it feedback from their smartphones on a three-point scale: “too hot,” “too cold,” or “comfortable.” After a few days, HEAT learns their preferences and operates independently.

The research team is looking for commercialization partners to help bring the technology to market, and is also working with power company Southern Company Research and Development to begin testing HEAT in its Alabama offices. Test cameras will be mounted on tripods in the corners of offices; Menassa explains that cameras would be placed less obtrusively in a permanent installation. The cameras collect temperature data without identifying individuals, and all footage is deleted immediately after processing, usually within a few seconds. 

Facial temperature is a good predictor of comfort, Menassa said. When we’re too hot, the blood vessels expand to radiate additional heat, raising facial temperature; when we’re too cold, they constrict, cooling the face. While earlier iterations of the system also used body temperature to predict comfort, they required users to wear wristbands that measured body temperature directly, and to provide frequent feedback about their comfort level.

“The cameras we’re using are common and inexpensive, and the model works very well in a residential context,” said Vineet Kamat, a U-M professor of civil and environmental engineering and electrical engineering and computer science and an author on the paper. “Internet-enabled thermostats that detect you and learn from you have sort of built a platform for the next phase, where there’s no visible thermostat at all.”

HEAT’s predictive model was built by U-M industrial operations and engineering associate professor Eunshin Byon, who is also an author on the paper. She believes that tweaks to the model could make the system useful in applications beyond homes and offices—in hospitals, for example, where care providers struggle to stay comfortable under masks and other protective equipment.

“The COVID-19 pandemic requires nurses and other hospital workers to wear a lot of protective gear, and they’ve struggled to stay comfortable in the fast-faced hospital environment,” Byon said.

“The HEAT system could be adapted to help them stay comfortable by adjusting room temperature or even by signaling to them when they need to take a break.” 

In partnership with the U-M school of nursing, Menassa’s research group has already conducted a pilot study that explored how the system can be used to provide personalized thermal comfort for nurses working in healthcare environments such as chemotherapy administration units.

HEAT is available as a licensable technology through the U-M Office of Technology Transfer. 

The paper is titled “HEAT – Human Embodied Autonomous Thermostat.” The U-M Office of Technology Transfer has applied for patent protection for the technology. The research was supported by the National Science Foundation, grant numbers CBET 1349921 and CBET 1804321. The research team has filed patent applications related to the technology. The research team also includes former U-M graduate research assistant Da Li, who is now an assistant professor of civil engineering at Clemson University.