Meta has released Muse Spark Open, an open-weight model that accepts text, images, audio and video and produces text responses. It gives developers a common interface for different input types. Meta positions it for coding and long-running agentic tasks in which it is designed to assess results and adjust its approach as it works.
Coding with visual feedback
According to Meta, training included refactoring across multiple files and extended debugging sessions. The model is designed to plan steps, execute them, observe the results and use that feedback to continue through longer tasks. Meta also cites browsing the web with images.
Hugging Face demonstrated how coding and visual input can work together to create 3D assets. It paired a two-bit quantized version running in llama.cpp with the Pi agent, which asks the model to write Three.js code, renders the result in a browser and captures views from several angles. The same model assesses those images, and Pi repeatedly revises the code using its feedback. The authors say the quantized version needs more guidance.
Another Hugging Face demonstration connected the model on Inference Endpoints to OpenEnv and Chromium in a remote sandbox. To choose its next action, the model receives previous actions, the system prompt, the URL and a screenshot. In this demonstration, it has no access to the page’s document structure (DOM) or HTML code.
A full-size Muse model with million-token context
Muse Spark Open follows Muse Glimmer, the small dense model Meta released in August. The new release is the full-size member of the series, using a mixture-of-experts architecture. Both models share a tokenizer and vocabulary; the conversation template builds on Glimmer’s, with additions for the new input types.
The model has approximately 816 billion parameters, with around 42 billion active for each token. Each MoE layer selects eight of its 256 experts. All experts remain in memory, so activating fewer parameters does not remove the memory requirements of the full model.
The inference context window supports up to a million tokens, with the configuration setting an exact maximum of 1,048,576 positions. Text shares that allowance with tokens representing images, audio and video.
Preparing video, audio and tool calls
The Transformers integration includes MuseSparkForConditionalGeneration for producing text from multimodal inputs and MuseSparkProcessor for preparing conversations. Tool calling works with both text-only tasks and tasks involving other input types.
Unlike Muse Glimmer, the processor can handle video with its accompanying audio, thanks to an audio encoder. By default, it samples two frames per second, up to 96 frames. A higher sampling rate preserves more visual detail over time but increases memory requirements and input-processing latency. Audio from video is enabled with load_audio_from_video=True.
Audio passed directly to the processor must be mono at a sample rate of 24 kHz; other sample rates require conversion first. The encoder processes 80-millisecond segments, producing 12.5 audio tokens per second.
Reasoning effort can be set to low, medium, high or xhigh, corresponding to values of 16, 64, 256 and 512. Reasoning cannot be switched off entirely. For simpler tasks, the documentation recommends low to reduce token consumption.
Two checkpoints and a Mac test
Available checkpoints are Muse-Spark-Open-816B-A42B in BF16 and Muse-Spark-Open-816B-A42B-NVFP4. The BF16 version occupies approximately 1.5 TB. The four-bit NVFP4 version targets NVIDIA Blackwell GPUs and, according to Hugging Face, incurs negligible accuracy loss. In the described Transformers configuration, BF16 runs across four nodes with eight H200 GPUs each and TP=32, while NVFP4 runs on one B200 node.
Support is also included in llama.cpp’s prebuilt binaries. In a test on a Mac M3 Ultra with 512 GB of memory, Hugging Face measured 18 tokens per second using an imatrix-calibrated two-bit quantization. The model itself occupied 304 GB in that configuration.
Server choice determines input support and context length
SGLang accepts all four input types, and its NVFP4 reference configuration runs on four GB300 GPUs. By contrast, the vLLM reference integration supports text and images with the NVFP4 checkpoint, requires Blackwell GPUs and uses a 128,000-token serving context in the specified configuration.
Hugging Face offers NVFP4 deployment through Inference Endpoints using a prevalidated SGLang recipe. Eight NVIDIA RTX Pro 6000 98 GB GPUs provide a 131,072-token context; the configuration for 1,048,576 tokens uses eight NVIDIA H200 141 GB GPUs. The OpenAI-compatible interface supports both Chat Completions and Responses. Clients use a Hugging Face token as the API key.
Fine-tuning with LoRA adapters
Hugging Face also demonstrated fine-tuning with TRL SFTTrainer across eight nodes containing eight H100 GPUs each. The recipe trains LoRA adapters in the attention layers while keeping the experts frozen. In the demonstration adapting the conversation template, 500 steps took approximately two hours on 64 H100s, at around 8,000 tokens per second.
