Spatial intelligence plays a key role in how people perceive and interact with the world. We use it every day during ordinary activities such as parking a car, catching keys, or navigating through a crowd. Firefighters move through burning buildings thanks to an intuitive understanding of space and stability, while children learn about the world through play even before they begin to speak. This ability is the foundation of our imagination—from building sandcastles to playing Minecraft.
Throughout human history, spatial intelligence has led to major discoveries. Eratosthenes in ancient Greece measured the circumference of the Earth using shadows and the 7-degree angle between Alexandria and Syene. Hargreaves invented the Spinning Jenny, which arranged multiple spindles side by side and increased productivity eightfold. Watson and Crick discovered the structure of DNA by building 3D models from metal plates and wires until the correct arrangement of base pairs became apparent.
Spatial intelligence is the foundation on which our knowledge rests. It helps us understand complex things through visual and physical interactions in ways that words alone cannot.
The Need to Change Current AI Models
Today's large language models (LLMs) are excellent at working with text, code, or images, but they lack a deeper understanding of space. They can generate photorealistic images or short videos, but they fail at estimating distances, orientation, or object rotation. They cannot navigate mazes, predict physics, or maintain consistency in videos for more than a few seconds.
Fei-Fei Li, who created ImageNet —a large database for visual learning—sees this as a problem. ImageNet was one of the three key elements that enabled modern AI, along with neural networks and GPU processors. In her Stanford laboratory, she combines computer vision with robotic learning. Together with Justin Johnson, Christoph Lassner, and Ben Mildenhall, she founded World Labs to overcome these limitations.
According to Li, spatial intelligence is essential for progress in robotics, scientific discovery, and creativity. Without it, AI remains disconnected from reality.
The Path to Spatial Intelligence
For AI to achieve spatial intelligence, it needs world models—a new type of generative model that goes beyond LLMs. These models must have three properties: they must be generative, multimodal, and interactive.
Generative means that they create consistent worlds with geometry, physics, and dynamics, whether real or virtual. Multimodal means that they can process inputs such as images, videos, text, or gestures and generate complete world states. Interactive means that they predict subsequent states based on actions, and potentially also actions directed toward a goal.
World Labs focuses on research: a new universal training objective similar to next-token prediction in LLMs, but more complex because of space. They need vast amounts of internet data, such as images and videos, as well as synthetic data incorporating depth and touch. New architectures, such as RTFM, use spatial images for memory and rapid generation.
The first step is Marble—a model that creates consistent 3D environments for exploration and creation based on multimodal inputs.
Applications in Creativity, Robotics, and Other Fields
Spatial intelligence will open up new possibilities for creativity. Filmmakers and game designers will use Marble to create 3D worlds without expensive software, enabling interactive stories through VR or XR. Architects will visualize buildings, while industrial designers will test objects in space.
In robotics, it will help scale learning through simulations that bridge the gap between the virtual and real worlds. Robots will become companions—helping in laboratories or assisting seniors at home and predicting actions aligned with human goals. It will support a variety of forms, from nanobots to machines for space exploration.
In the long term, it will influence science through experiment simulations, healthcare through molecular modeling or patient monitoring, and education through interactive lessons in which students explore cells or history in 3D.
Fei-Fei Li emphasizes that AI should empower people, not replace them. Inspired by Alan Turing, this progress will bring smarter machines for a better life.
Source: drfeifei.substack.com
