Meta has released an update that will please anyone working with computer vision. SAM 3.1, the latest version of the Segment Anything model, brings major improvements to video processing without requiring the entire workflow to be rewritten. It is a direct replacement for the previous version, SAM 3.
But what exactly is SAM? The acronym stands for Segment Anything Model, a model capable of finding, outlining, and tracking any object in an image or video. Whether it is a person, an umbrella, or even a platypus.
A new trick called multiplexing
The previous version of the model processed each tracked object separately, one after another. This was slow and unnecessarily computationally demanding.
SAM 3.1 introduces what is known as object multiplexing, a technique that allows the model to process up to 16 objects at once in a single pass. The result? Video throughput on a single H100 GPU has doubled, from 16 to 32 frames per second. All while tracking more objects simultaneously.
This means that real-time tracking in complex footage with many moving elements, which previously required expensive and powerful hardware, can now be handled on significantly more modest machines. That is a major benefit for developers and smaller teams.
What could the earlier SAM 3 model do?
To understand what SAM 3.1 has inherited, it is worth explaining what made SAM 3 stand out from its initial release. SAM 3 is a unified model for detecting, segmenting, and tracking objects in both images and video. It accepts text prompts (such as "striped red umbrella"), example images, and visual cues such as masks or points. This sets it apart significantly from older models that worked only with a predefined list of categories.
The model can segment an enormous number of concepts because Meta built a data engine combining human data annotators and AI models. This produced more than 4 million unique concepts in the training data. Data annotation was approximately 5x faster than purely human work for negative examples and 36% faster for the more difficult ones.
Meta did not start from scratch when building SAM 3. The model is based on several proven components. The text and image encoder comes from Meta Perception Encoder, an open model that Meta shared in April of the previous year. The detector builds on the DETR model, the first system to use transformers for object detection. The memory bank is based on the previous generation, SAM 2.
The result is a system that outperforms both large language models such as Gemini 2.5 Pro and specialized tools such as OWLv2 or GLEE in benchmarks. In testing, users preferred SAM 3 outputs over OWLv2 by a ratio of approximately three to one.
Where is such a model used?
Research results are impressive, but what about practical applications? Meta has not locked the model away in a laboratory. On Facebook Marketplace, SAM 3 powers the "View in Room" feature, which allows people to virtually place a piece of furniture in their space before buying it. It is a kind of "IKEA effect" for online shopping.
In the Edits app, Instagram's video creation tool, effects based on SAM 3 will be added soon. Creators will be able to apply dynamic effects to a specific person or object in a video with a single tap. What once took hours in the editing room can now be done by the model in seconds.
Meta also collaborated with Conservation X Labs and Osa Conservation to create a publicly available dataset for wildlife tracking. The SA-FARI dataset contains more than 10,000 camera-trap videos featuring over 100 animal species, with bounding boxes and segmentation masks for every animal in every frame. This is exactly the kind of application that makes all the work around AI worthwhile.
What can still be improved
It would be unfair to claim that SAM 3.1 can handle everything. The model still struggles to recognize specialized terms requiring expert knowledge, such as "blood platelet" in a medical image. It works extremely well with short, general descriptions, but more complex instructions such as "the second book from the right on the top shelf" are still beyond its capabilities.
Meta openly states that the model can quickly adapt to new concepts when fine-tuned on a small amount of data. The code for fine-tuning the model is included in the public release, allowing the research community and companies to adapt SAM 3.1 to their specific needs. Collaboration with the Roboflow platform will also make it possible to annotate data, fine-tune the model, and deploy it in practice without deep technical expertise.
Both the SAM 3.1 model and checkpoint are freely available on HuggingFace and GitHub, so anyone can explore and try them out.
