Miso Labs has released MisoTTS, an open‑weights 8‑billion‑parameter text‑to‑speech model that solves two common pain points for developers building voice applications. First, most TTS systems are limited by a fixed token vocabulary, which cannot capture the full richness of human speech such as pitch, rhythm, emphasis, emotion, and accent. Second, many models condition only on text, ignoring the speaker’s tone and producing output that feels flat or falls into the uncanny valley.
MisoTTS tackles these issues with residual vector quantization (RVQ). Instead of predicting a single token, the model outputs a vector of 32 codebook indices drawn from 2048‑way codebooks. This approach expands the addressable vocabulary to roughly 2048³² (about 10¹⁰⁵ distinct sounds) without adding any parameters, letting the model represent a far wider sonic range while keeping its size fixed at 8 B parameters. The architecture splits the work between a 7.7 B‑parameter backbone that processes text and audio context over time, and a 300 M‑parameter decoder that refines the audio codebook indices over depth. By conditioning on both text and prior audio, the model can adapt its output to the interlocutor’s tone, making conversations sound more natural.
Practical benefits for teams include open‑source availability under a modified MIT license, the ability to run the model locally on a CUDA‑capable GPU for data‑privacy‑sensitive use cases, and a claimed latency of around 110 ms—significantly lower than many commercial alternatives. Although the current release supports half‑duplex, single‑turn generation, the core design already provides a scalable foundation for future full‑duplex and turn‑taking extensions.
To get started, developers can load the model from Hugging Face, feed text and optional audio context, and synthesize waveform files with just a few lines of code. This makes MisoTTS a ready‑to‑use solution for expressive, low‑latency voice generation in products ranging from virtual assistants to interactive storytelling platforms.
#AI #TTS #OpenSource #MachineLearning #SpeechSynthesis #RVQ