The work from Duke University, North Carolina, has been accepted to the IEEE International Conference on Robotics and Automation (ICRA 2025), which will be held May 19-23, 2025, in Atlanta, Georgia.
“WildFusion opens a new chapter in robotic navigation and 3D mapping,” said Boyuan Chen, the Dickinson Family Assistant Professor of Mechanical Engineering and Materials Science, Electrical and Computer Engineering, and Computer Science at Duke University. “It helps robots to operate more confidently in unstructured, unpredictable environments like forests, disaster zones and off-road terrain.”
"Typical robots rely heavily on vision or LiDAR alone, which often falter without clear paths or predictable landmarks," said Yanbaihui Liu, the lead student author and a second-year PhD student in Chen’s lab. “Even advanced 3D mapping methods struggle to reconstruct a continuous map when sensor data is sparse, noisy or incomplete, which is a frequent problem in unstructured outdoor environments. That’s exactly the challenge WildFusion was designed to solve.”
Built on a quadruped robot, WildFusion integrates multiple sensing modalities, including an RGB camera, LiDAR, inertial sensors, plus contact microphones and tactile sensors. The camera and the LiDAR capture the environment’s geometry, colour, distance and other visual details. According to Duke, the unique elements of WildFusion are its use of acoustic vibrations and touch.
As the robot walks, contact microphones record the vibrations generated by each step, capturing subtle differences, such as the crunch of dry leaves versus the soft squish of mud. Meanwhile, the tactile sensors measure how much force is applied to each foot, helping the robot sense stability or slipperiness in real time. These added senses are also complemented by the inertial sensor that collects acceleration data to assess how much the robot is wobbling, pitching or rolling as it traverses uneven ground.
Each type of sensory data is then processed through specialised encoders and fused into a single, rich representation. At the heart of WildFusion is a deep learning model based on the idea of implicit neural representations. Unlike traditional methods that treat the environment as a collection of discrete points, this approach models complex surfaces and features continuously, allowing the robot to make more intuitive decisions about where to step, even when its vision is blocked or ambiguous.
“WildFusion’s multimodal approach lets the robot ‘fill in the blanks’ when sensor data is sparse or noisy, much like what humans do,” said Chen.
WildFusion was tested at the Eno River State Park in North Carolina near Duke’s campus where the robot navigated dense forests, grasslands and gravel paths.
“Watching the robot confidently navigate terrain was incredibly rewarding,” said Liu. “These real-world tests proved WildFusion’s remarkable ability to accurately predict traversability, significantly improving the robot’s decision-making on safe paths through challenging terrain.”
The team now plans to expand the system by incorporating additional sensors, such as thermal or humidity detectors, to further enhance a robot’s ability to understand and adapt to complex environments.
This research was supported by DARPA and the US Army Research Laboratory.
Comment: The UK is closer to deindustrialisation than reindustrialisation
"..have been years in the making" and are embedded in the actors - thus making it difficult for UK industry to move on and develop and apply...