PRISM: A neural network component for better robotic motor control
Learning physical interactions helps robots move more reliably, handle objects and adapt to unfamiliar conditions—without larger networks or extra sensors.
Key takeaways
- A new neural network input module—called PRISM (Polynomial Representations for Interaction-Structured Motor Control)—improves humanoid locomotion and robotic manipulation beyond what larger conventional neural networks can achieve, according to a University of Michigan Engineering study.
- PRISM learns physical interactions from data, enabling adaptive contact behavior without manually designed compliance controllers or force sensors.
- Experiments showed that PRISM outperforms conventional neural networks even when physical conditions change beyond those encountered during training.
A new neural network component, developed at University of Michigan Engineering, challenges a common assumption in robot learning: that the same general-purpose neural networks used for image recognition are equally suited to controlling physical movement.
In image classification, a neural network maps an image to a label but does not change the image. During robotic motor control, actions—like a humanoid robot moving a limb—change the robot’s physical state, which in turn changes what sensors observe next.
Robot movement and goals involve multiplicative relationships among physical variables. For example, the forces acting on one joint depend on the positions and velocities of other joints, while quantities such as mechanical power and contact multiply forces and motion.
Yet conventional neural networks used in robotics, called multilayer perceptrons or MLPs, do not explicitly multiply variables. Instead, they learn these interactions indirectly through simpler operations, consuming network capacity.
The new component—called PRISM (Polynomial Representations for Interaction-Structured Motor Control)—learns multiplicative interactions among sensor observations, past actions and task commands, including joint positions, velocities and desired walking speeds. This eliminates the need to specify physical equations or manually select variables to multiply.
“Many useful clues about physical interaction are already present in a robot’s observations, through combinations of signals such as joint positions and velocities. PRISM gives the controller a compact way to learn and use these combinations,” said Seung Hyun Lee, a doctoral student of computer science and engineering at U-M and lead author of the study.
Because PRISM only replaces the initial processing of robot state and command inputs, it can be incorporated into existing robot learning systems without changing their training objectives, sensing requirements or controllers.
The study, supported by the National Science Foundation, was presented today at the 10th Conference on Robot Learning (CoRL 2026) in Austin, Texas.
Text on screen: Eight panels set up in two rows of four. In each panel, simulated humanoid robots begin walking. The robots in the first three columns step with different gaits like the robots in the last column wear a large, orange backpack. All robots in the top row fall after two seconds, but the robots in the bottom row all remain upright.
Better performance without larger networks
To evaluate PRISM, the researchers tested it across humanoid locomotion and robotic manipulation, using different learning methods and network architectures.
In simulated humanoid walking tests, PRISM enabled the robot to complete 92.5% of 24-second trials without falling, compared to 51% for a conventional MLP. Even with about 43% more total parameters than the original MLP, matching PRISM’s size, the conventional network completed only 52.3% of trials.
Likewise, PRISM improved manipulation task success rates for a robotic arm and a large vision-language-action model, demonstrating that how a network represents physical interactions matters more than simply scaling its size.
Better physical control without extra sensors
Rather than relying on manually designed physical models and control rules, PRISM learns useful physical interactions directly from data. In a simulated recovery trial, it rapidly reduced contact forces after touching an object without force feedback or an explicit compliance controller.
In a simulated cube-stacking experiment where the researchers deliberately induced the robotic arm to drop a cube, PRISM recovered and completed the task in 52.3% of trials, compared to 40.9% for the conventional imitation-learning policy. PRISM also demonstrated practical benefits on physical hardware. Using identical training demonstrations to pick up a roll of tape with an SO-101 robotic arm, PRISM achieved 94% success across 50 attempts, compared to 68% for the conventional approach.
“What surprised me was how such a small change in representation could affect the robot’s response to contact,” said Lee.
The research team probed further to understand what PRISM learned internally. They found that PRISM’s representations made physical quantities related to joint power, slipping and contact forces more predictable than those learned by conventional networks.
Better generalization beyond training conditions
PRISM’s advantages persisted when the physical conditions changed without retraining or modifying the networks.
Across six conditions involving changes in friction, mass and payload, PRISM-controlled humanoid robots successfully completed 92.3% of trials on flat terrain, compared with 78.3% for a parameter-matched standard MLP. On uneven terrain, PRISM completed 87.1% of the trials, compared with 80% for the conventional network.
“Robotic motor control is not an arbitrary mapping from observations to actions. It is governed by physical interactions,” said Stella Yu, a professor of computer science and engineering at U-M and senior author of the study.
“Our results make a strong case for adopting PRISM in place of conventional MLP input layers in robot policy learning. It gives neural networks a structure better suited to controlling movement in the physical world.”
Rather than relying on increasingly large networks or manually engineered physical control mechanisms, PRISM shows how a simple change in representation can help robots learn more capable and adaptable behavior.
This research was supported by the National Science Foundation (2313151), with additional computing resources provided through its NAIRR Pilot (CIS240421 and CIS240431).