Meta AI TRIBE: An AI Model That Decodes the Brain While Watching Videos

Imagine an artificial intelligence that can predict how your brain responds to a movie based solely on video, audio, and text. That is exactly what TRIBE, a model from Meta AI, has achieved, taking first place in the Algonauts 2025 competition. Developed by Meta FAIR's Brain & AI team, this 1-billion-parameter trimodal brain encoder predicts brain responses to video, audio, and text stimuli, significantly outperforming 262 competitors.

What Is TRIBE and How Does It Work?

TRIBE, short for TRImodal Brain Encoder, is the first deep neural network trained to predict whole-brain responses to videos across multiple individuals. According to the paper's abstract by authors including Stéphane d'Ascoli from Meta AI, the model combines pretrained text, audio, and video representations from models such as Llama-3.2-3B, Wav2Vec-Bert-2.0, and VideoJEPA 2 gigantic. It processes these inputs using a transformer that accounts for temporal dynamics and predicts fMRI BOLD signals across 1,000 brain parcels based on the Schaefer atlas.

Results in the Algonauts 2025 Competition

In the Algonauts 2025 competition, TRIBE achieved an average score of 0.2146, outperforming the second-place team's score of 0.2096. Using data from the Courtois NeuroMod dataset, in which four subjects watched the TV series Friends and movies such as The Bourne Supremacy and Hidden Figures, the model predicted responses with a normalized Pearson correlation of 0.54 ± 0.1. It achieved its best results in auditory and language regions, where it approached the noise ceiling. Ablation studies showed that the multimodal approach outperforms unimodal models, particularly in association cortical regions such as the prefrontal and parieto-occipito-temporal cortex.

Why Is the Multimodal Approach Essential?

TRIBE integrates modalities nonlinearly and across multiple subjects, addressing the limitations of previous models. For example, text embeddings are extracted with a context of up to 1,024 words, audio from 60-second segments, and video from 64 frames spanning 4 seconds. Training on 80 hours of fMRI data took 24 hours on a GPU, with an ensemble of 1,000 models used to improve generalization. The model also performed well on out-of-distribution data, such as Pulp Fiction and Charlie Chaplin, achieving scores as high as 0.2604.

Future Prospects and Limitations

TRIBE paves the way for an integrative model of cognition, but it has limitations: it works with coarse brain parcellation, uses only fMRI data, and was trained on just four subjects. The authors emphasize its potential for in silico experiments in neuroscience. The code is available on GitHub.