Microsoft Releases VibeVoice-1.5B: A New Open Model for Long Conversations

Whether you are an artificial intelligence researcher or simply a curious technology enthusiast, Microsoft’s VibeVoice-1.5B model is sure to catch your attention. This open text-to-speech model introduces innovations that make it possible to create long, natural conversations with multiple speakers. Let’s take a look at exactly what it can do, how it works, and how it compares with the competition.

What Exactly Can VibeVoice-1.5B Do?

VibeVoice-1.5B is designed to generate expressive, long-form audio with multiple speakers, such as podcasts or dialogues. It can synthesize up to 90 minutes of speech with as many as four distinct speakers in a single output. That is a major leap over conventional models, which are typically limited to one or two speakers. The model uses continuous speech tokenizers (acoustic and semantic) with an ultra-low frequency of 7.5 Hz, ensuring high audio fidelity while efficiently processing long sequences.

At its core is the transformer-based large language model (LLM) Qwen2.5-1.5B, which understands textual context and the flow of dialogue. It also adds a diffusion head to generate audio details. The acoustic tokenizer is based on a σ-VAE variant with an encoder-decoder structure that achieves a 3,200-fold reduction in sampling rate from a 24 kHz input. Each part of the encoder and decoder has approximately 340 million parameters. The semantic tokenizer mirrors the acoustic tokenizer’s architecture but is trained on an automatic speech recognition task.

The model supports expressive elements, including spontaneous singing, and works primarily in English and Chinese. During inference, it requires only about 7 GB of GPU memory, making it accessible even on standard hardware. Training took place in stages: first the tokenizers were trained separately, followed by the entire model with sequence lengths gradually increasing to 65,536 tokens. In total, the model has 2.7 billion parameters and is released under the MIT license for research purposes.

How Does It Work in Practice, and What Are Its Limitations?

In practice, VibeVoice-1.5B excels at creating natural conversations in which speakers take turns. It uses a diffusion process with classifier-free guidance for better quality. The model automatically inserts an audible notice, “This segment was generated by AI,” into every output and adds an invisible watermark to verify its origin, helping to prevent misuse.

But not everything is perfect. The model focuses solely on speech, so it does not generate music, sound effects, or overlapping speech. It fully supports only English and Chinese—other languages may lead to unintelligible/inaccurate results. It is not intended for commercial uses such as voice impersonation without consent or the spread of disinformation. The risks include the potential for deepfakes, so Microsoft recommends responsible use and reporting issues via email at [email protected].

Comparison with the Competition: Where Does VibeVoice Excel?

Compared with competitors such as OpenAI’s Whisper or ElevenLabs, VibeVoice-1.5B holds its own. Whisper is primarily a speech-to-text (ASR) model, not a synthesis model, so it does not compete directly—it excels at transcription in dozens of languages but does not generate audio. VibeVoice, by contrast, leads in output length (up to 90 minutes) and multi-speaker support (up to 4 speakers), neither of which Whisper offers.

Compared with ElevenLabs, a commercial TTS service supporting 29 languages and hundreds of voices, VibeVoice offers openness (it is open-source) and efficiency on standard hardware. ElevenLabs is strong in emotional realism and short-form outputs, but it is limited to shorter durations and is proprietary. VibeVoice adds unique features such as singing and ethical safeguards, including watermarks, which ElevenLabs addresses with its own classifier.

Other models, such as Deepgram or wav2vec 2.0, focus more on ASR than TTS and differ in speed or specialization. VibeVoice-1.5B therefore stands out in open, long-form, multi-speaker synthesis, but it lags behind commercial solutions in language diversity. Overall, it is an excellent research tool that pushes the boundaries of what TTS can do.

If you would like to learn more, take a look at the technical report on arXiv or the microsoft/VibeVoice repository on GitHub. This model shows how artificial intelligence is approaching natural human dialogue—all openly and responsibly. What do you think?