Imagine a robot moving through a collapsed building or a dark cave, where every second determines the success of a rescue operation. Researchers at MIT have developed a new system based on artificial intelligence (AI) that allows robots to create detailed three-dimensional (3D) maps of complex environments within seconds. This system combines modern machine learning with classic computer vision principles and could change how rescue robots operate in dangerous locations such as collapsed mines or disaster areas.
The system processes an unlimited number of images from cameras on the robot, creates accurate 3D reconstructions, and simultaneously estimates the robot's position in real time. Unlike older methods, which fail in cluttered or visually complex environments and require pre-calibrated cameras, this new approach works without special calibration and can quickly handle thousands of images.
Old vs. New Solutions
Robots traditionally use a technique called SLAM (simultaneous localization and mapping) to create a map of their surroundings and determine their position. However, these methods often do not work well in chaotic environments. Machine learning models simplified the process, but they could only process about 60 images at a time, which is insufficient for real-world missions because a robot needs to analyze thousands of images quickly.
MIT researchers, including graduate Dominic Maggio, postdoctoral researcher Hyungtae Lim, and professor of aeronautics and astronautics Luca Carlone, set out to solve this problem. Their approach divides the scene into smaller "submaps," which are created and aligned incrementally. The system then combines these submaps into a single coherent 3D model, allowing the robot to move quickly while maintaining accuracy.
Dominic Maggio said that when he first tried this simple approach, he was surprised that it did not work as well as he had expected. After examining earlier computer vision research, he discovered the reason: machine learning models often introduce subtle deformations into submaps, making it difficult to align them correctly using only rotation and translation.
Combining Geometry and Artificial Intelligence
To solve the problem, Luca Carlone's team adopted techniques from traditional geometry. They developed a mathematical framework that captures and corrects deformations in each submap, enabling the system to align them consistently. Luca Carlone explained that it is necessary to ensure that all submaps are deformed in a consistent way, allowing them to be aligned properly.
Once Dominic Maggio combined the power of machine learning with classical optimization methods, the results were immediate. Luca Carlone noted that once Dominic Maggio had the intuition to bring these two worlds together—learning-based approaches and traditional optimization methods—the implementation was relatively straightforward. This efficient and simple approach has potential for many applications.
The system is based on VGGT-SLAM, which uses the feed-forward VGGT reconstruction model to incrementally build a dense map from uncalibrated monocular cameras. It aligns submaps both locally and globally through loop closure and performs optimization on the SL(4) manifold, a novel approach to the SLAM problem.
Laboratory Tests and Real-World Use
The MIT system proved faster and more accurate than existing mapping techniques. It does not require special camera calibration or additional processing tools. In one demonstration, the researchers recorded a short video of the interior of the MIT Chapel using a mobile phone and reconstructed an accurate 3D model within seconds. The reconstructed scenes had an average error of less than five centimeters.
The team believes this simplicity will help deploy the method in real robots, wearable augmented reality (AR) or virtual reality (VR) systems, and even warehouse automation. Luca Carlone emphasized that knowledge of traditional geometry pays off. If you deeply understand what is happening in the model, you can achieve much better results and make things far more scalable.
The research will be presented at the Conference on Neural Information Processing Systems (NeurIPS) and is available on arXiv under the identifier 2505.12549v2.
Benefits for Rescue Operations
This system could change how rescue robots navigate collapsed mines or disaster sites, where speed and accuracy can mean the difference between life and death. For example, in tests on datasets such as 7-Scenes and TUM RGB-D, VGGT-SLAM achieved an average trajectory error of 0.067 meters on 7-Scenes and 0.053 meters on TUM RGB-D, which is comparable to or better than other methods such as DROID-SLAM or MASt3R-SLAM.
In one example, the system processed a 55-meter route through an office corridor using 22 submaps and created a globally consistent map with loop closure. Additional tests showed that optimization on SL(4) resolves projective ambiguity where Sim(3) fails, improving map quality in general cases.
The team acknowledged limitations such as degeneracy in planar scenes, where homography estimation leads to unstable solutions, as occurred in a floor scene from the TUM dataset. Nevertheless, the system represents the first factor-graph SLAM optimized on the SL(4) manifold, opening new possibilities for dense, accurate mapping in real time.
Sources: news.mit.edu and interestingengineering.com
