Do you remember the periodic table of elements from school? The table that at first glance looked like a jumble of numbers and abbreviations, but actually concealed the perfect order of the entire material world? Physicists at Emory University have just done something very similar, only this time not for chemistry, but for artificial intelligence. And the result could fundamentally influence how AI systems are designed and trained.
What is multimodal AI, and why do developers struggle with it so much?
Artificial intelligence stopped working with text alone a long time ago. Modern systems combine text, images, audio, and video all at once. They are known as multimodal AI, and they underpin things we use every day, from voice assistants and medical diagnostic tools to autonomous driving systems.
But this is where the problem arises. Developers building such systems repeatedly face one difficult question: which algorithm should they actually choose? There are literally hundreds of different approaches and methods. Each has what is known as a loss function—a mathematical formula that tells the system how wrong it is and how it should improve. Until now, choosing the right one has been more art than science. Lots of trial, lots of error, and lots of wasted computing power.
This is exactly the chaos that Emory University physics professor Ilya Nemenman wanted to end.
Years at the blackboard: How the breakthrough emerged
Nemenman and his team approached the work in a way that is fairly unusual in the machine learning community. Instead of running experiments on powerful servers, they reached for chalk and a blackboard. Literally.
Doctoral student Eslam Abdelaleem and postdoctoral researcher Michael Martini spent months working through equations by hand. "We spent a lot of time in my office writing on the blackboard," Martini recalls. "Sometimes I wrote on paper while Eslam looked over my shoulder."
It was not a quick sprint. It was a years-long search, full of dead ends and returns to first principles. The team worked on the mathematical foundations, consulted with Nemenman, tested ideas on a computer, and then went back to the blackboard whenever a path led nowhere. "There was a lot of trial and error," Martini admits candidly.
What were they looking for? They wanted to determine whether the hundreds of different AI methods might share a simple, common principle. And eventually, they found it.
One principle to rule them all: Compress, but do it intelligently
The breakthrough came when the team identified one fundamental idea connecting seemingly unrelated AI methods. Compress the data enough to retain only what truly predicts the outcome. Nothing more, nothing less.
"We found that many of the most successful AI methods boil down to one simple idea: compress different types of data just enough to preserve the pieces that actually predict what you need," Nemenman explains. "This gives us a kind of periodic table of AI methods. Different methods fall into different cells depending on which information their loss function preserves or discards."
The team named its approach the Variational Multivariate Information Bottleneck Framework. The name is complex, but the principle is elegantly simple: the framework works like a control knob. A developer "turns" it to the desired value, and the system tells them which information to retain for a given task and which to discard.
"Our approach is general and principled," Abdelaleem says. "Our goal is to help people design AI models tailored to the specific problem they want to solve, while also enabling them to understand how and why each part of the model works."
The smartwatch that mistook science for cycling
Science sometimes has an unexpectedly human dimension. Abdelaleem vividly remembers the day the team achieved its breakthrough. After an exhausting but euphoric final push, he headed home from campus. On the way, he checked his Samsung Galaxy smartwatch, which uses AI to track health data, including heart rate.
The watch displayed a message that made him laugh. "My watch said I had been cycling for three hours," Abdelaleem recalls. "That was how it interpreted the level of excitement I was feeling. I thought, 'Wow, this is something! Apparently, science can have that kind of effect.'"
It is a little ironic, isn't it? The system whose shortcomings the team was trying to correct could not distinguish scientific excitement from physical exertion. But that is precisely the point of the entire study: AI needs better tools to truly understand the context of the data it processes.
Less data, less energy, more meaning
The research findings, published in the prestigious journal The Journal of Machine Learning Research, have practical implications that extend beyond academia.
First, the framework allows developers to design new algorithms systematically instead of groping in the dark. They can predict which approaches will work, estimate how much training data will be needed, and identify potential system weaknesses in advance.
Second, and perhaps even more interestingly, the framework can significantly reduce the computational demands of AI. By helping eliminate irrelevant features from data, it reduces the amount of data needed for training. Less data means less computing power. Less computing power means lower energy consumption. And that is good news for anyone watching enormous data centers consume as much electricity as small cities.
"The less data a system needs, the less computing power it requires to operate, and the less harm it does to the environment," Nemenman emphasizes. "This may also open the door to groundbreaking experiments on problems we cannot solve right now because there is not enough data."
Third, the framework brings something that has long been missing from the AI community: clarity. The physicists approached the problem differently from typical machine learning researchers. The latter focus on the accuracy of results without necessarily understanding why the system works. Physicists want to know why. And it was precisely this perspective that led to the breakthrough.
"The machine learning community focuses on achieving system accuracy without necessarily understanding why the system works," Abdelaleem explains. "As physicists, however, we want to understand how and why something works. We therefore focused on finding fundamental, unifying principles that connect different AI methods."
Where AI and biology converge
The team is not stopping there. It intends to continue expanding the framework, and one of its most ambitious areas of interest is biology and cognitive science. Abdelaleem wants to understand how the human brain simultaneously compresses and processes multiple sources of information.
"I want to understand how your brain simultaneously compresses and processes multiple sources of information," he says. "Can we develop a method that allows us to see similarities between a machine learning model and the human brain? That could help us better understand both systems."
It is a bold vision. But when we consider what the team accomplished—years of patient work at the blackboard, hundreds of equations, dead ends, and finally one elegant principle that connected everything—it is hard to doubt that they have what it takes to achieve it.
It took chemists centuries to compile the periodic table of elements. The periodic table for AI was created in just a few years. And perhaps this is the best evidence of how quickly science moves forward when physicists decide to pick up a piece of chalk and start from the beginning.
Source: sciencedaily.com
