Zyphra/ZONOS3-Mini-hf

ZONOS3-Mini for 🤗 Transformers

ZONOS3-Mini is Zyphra's compact text-to-speech model supporting 43 languages, with about 100 million active parameters per token (800 million total in the backbone). This repo packages it for plain transformers (remote code, trust_remote_code=True): the language model, the causal multi-resolution iSTFT codec decoder (keep2_plain) and the MECT-B2 speaker encoder in one checkpoint, with the SuperBPE tokenizer and the 27 release voices. It synthesizes 44.1 kHz mono speech on CPU or GPU with nothing but torch and transformers.

Model details, samples and evaluations: the blog post and technical report. The native runtimes (CUDA, MLX, ggml, WebGPU) and the full HTTP API live in the ZONOS3-Mini repository. Try the browser studio, its Lite voice-cloning interface, or Zyphra Cloud.

Languages

The same 43 languages as the main model, with the native runtimes' request codes (English has two locales):

Region	Languages (language code)
Western and Southern Europe	English (en_us, en_gb), French (fr_fr), Spanish (es), German (de), Italian (it), Portuguese (pt_br), Dutch (nl), Greek (el)
Northern Europe	Swedish (sv), Danish (da), Norwegian (no), Finnish (fi)
Central and Eastern Europe	Polish (pl), Czech (cs), Slovak (sk), Hungarian (hu), Romanian (ro), Bulgarian (bg), Russian (ru), Ukrainian (uk)
Middle East and Caucasus	Turkish (tr), Arabic (ar), Hebrew (he), Georgian (ka)
East Asia	Mandarin Chinese (cmn), Japanese (ja), Korean (ko)
Southeast Asia	Vietnamese (vi), Indonesian (id), Malay (ms), Thai (th), Tagalog (tl)
South Asia	Hindi (hi), Bengali (bn), Tamil (ta), Telugu (te), Marathi (mr), Gujarati (gu), Kannada (kn), Malayalam (ml), Punjabi (pa), Odia (or), Urdu (ur)

Text normalization uses NeMo grammars via zyphra_textnorm for English, Mandarin, Japanese, Korean, German, Spanish, French, Italian and Portuguese. Other languages pass through as written, with line breaks collapsed; they do not require the optional normalizer. language selects normalization, not a language token in the prompt.

Usage

pip install "transformers>=4.57" torch soundfile
pip install zyphra-textnorm   # optional, recommended: reads numbers, dates, currencies and units aloud

import soundfile as sf
import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("Zyphra/ZONOS3-Mini-hf", trust_remote_code=True)
model = model.to("cuda" if torch.cuda.is_available() else "cpu")

out = model.generate("Hello! This is ZONOS3-Mini running in transformers.", voice="Savannah", seed=0)
sf.write("hello.wav", out.waveform.numpy(), out.sampling_rate)

AutoModelForTextToWaveform loads the same class. The checkpoint's LM is bfloat16 (the release dtype): transformers 5 loads it as stored, transformers 4.x in float32 unless you pass dtype=torch.bfloat16. The codec decoder and the speaker encoder always stay in float32. dtype=torch.float32 gives the reference numerics on CPU.

Voices

Pick one voice source per call:

Argument	Conditioning
voice="Savannah"	a release voice (below): its reference codes, or its speaker embedding with speaker_mode="embedding"
speaker_audio=wav, speaker_sample_rate=sr	a clip ([samples] or [channels, samples], any rate) through the speaker encoder
speaker_embedding=emb	a 192-d speaker embedding (model.embed_speaker(wav, sr) computes one)
speaker_reference_codes=codes	precomputed codec codes [frames, 10] of a clip (at most 60 s used)

speaker_mode defaults to "reference" for voices and codes and to "embedding" for clips and embeddings. speaker_cfg_scale (default 1.0) above 1 strengthens the voice through classifier-free guidance (a second LM pass per step). Encoding a raw clip into codec codes needs the codec encoder of the native runtimes; here clips clone through the speaker embedding.

import soundfile as sf

wav, sr = sf.read("my_voice.wav", dtype="float32", always_2d=True)
out = model.generate("Cloned from a clip.", speaker_audio=wav.T, speaker_sample_rate=sr, seed=1)

emb = model.embed_speaker(wav.T, sr)  # reuse it across calls
out = model.generate("Same voice, from its embedding.", speaker_embedding=emb, seed=1)

Voice	Language	Style	Gender
Cillian	English	Irish	male
Imogen	English	British	female
Jett	English	energetic	male
Morgana	English	witch	female
Rupert	English	British	male
Sakura	English	anime	female
Savannah	English	American	female
Thaddeus	English	announcer	male
Willow	English	ASMR	female
Wyatt	English	American	male
Gaspard	French	—	male
Liesel	German	—	female
Wolfgang	German	—	male
Ananya	Hindi	—	female
Arjun	Hindi	—	male
Ginevra	Italian	—	female
Lorenzo	Italian	—	male
Minjun	Korean	—	male
Seoyeon	Korean	—	female
Haoran	Mandarin Chinese	energetic	male
Meilin	Mandarin Chinese	energetic	female
Leonor	Portuguese	—	female
Thiago	Portuguese	—	male
Rodrigo	Spanish	—	male
Ximena	Spanish	—	female
Chaitanya	Telugu	—	male
Sravya	Telugu	—	female

Options

Argument	Default	Meaning
language	"en_us"	a request code from Languages; case-insensitive, accepts - for _
text_normalization	True	zyphra_textnorm when installed (otherwise a one-time warning and line breaks collapsed only)
condition_values	unknown	{"loudness": LUFS, "fmax": Hz, "speaking_rate": UTF-8 bytes per second} (or a list of 3)
condition_buckets	unknown	the same as bucket ids (loudness / fmax 0..9, speaking rate 0..29)
temperature, topk, top_p, min_p	0.75, 80, 0.69, 0.0	sampling
repetition_penalty, repetition_window, repetition_codebooks	2.4, 96, 6	repetition penalty
silence_prefill_frames	3	codec silence frames opening the take (0..30)
decode_context, decode_context_frames	False, 30	decode the reference tail as left context (reference mode)
max_new_frames	context left	frame budget (15 frames per second)
seed	random	the take's seed (returned in out.seed)
headroom_db, fade_out_ms	3.0, 0.0	output gain (then a soft limiter at -1 dBFS) and fade-out

generate returns waveform (float32 [samples]), sampling_rate (44100), seed and codes (the decoded codec frames [frames, 10]). model.decode_codes(codes) turns codes into audio; model(input_ids=rows, cache=cache) runs one LM step over prompt rows [rows, 11] for custom decoding loops.

Architecture

Component	ZONOS3-Mini
Backbone	11 layers, dimension 1,024; latent MoE with 64 experts, top-2 routing from layer 1
Sequence mixer	9 Gated DeltaNet layers + 2 global attention layers, Full AttnRes; context 32,768 positions
Text	SuperBPE, 8,192 entries + 4 special tokens
Audio tokens	15 Hz; 10 codebooks of 4,096 entries (1 semantic + 9 acoustic); hop 2,940 samples at 44.1 kHz
Speaker prompt	codec reference prefix (up to 60 s) or a 192-d MECT-B2 embedding
Decoder	bundled keep2_plain causal multi-resolution iSTFT head; the native release also offers full_mr (its default)

Notes

• The backbone configuration, tokenizer and 27 release voices match upstream revision 727302c1. Request languages follow the native language registry.
• The prompt layout, sampler, EOS handling, codec and speaker encoder follow the reference implementation of the ZONOS3-Mini repository. The LM is vectorized (chunked gated-delta rule, grouped experts), so its logits match the reference within float tolerance and long sampled takes may drift from it.
• Not included: speech editing, steering controls and accents, LACI, phoneme input, tag CFG, streaming decode and the codec encoder. The native runtimes serve them.
• The text-to-speech pipeline is not supported: it tokenizes before the model sees the text, which skips the text normalization. Call generate with the raw text.

Citation

@misc{zyphra2026zonos3mini,
  title         = {{ZONOS3-Mini Technical Report}},
  author        = {Gabriel Clark and Sofian Mejjoute and George Close and Beren Millidge},
  year          = {2026},
  eprint        = {2611.16525},
  archivePrefix = {arXiv},
  primaryClass  = {cs.SD},
  url           = {https://arxiv.org/abs/2611.16525},
}

License

The weights are released under the Apache License 2.0. The remote code is ported from the ZONOS3-Mini repository (MIT) and includes adapted code from AntSpeaker (Apache-2.0), Hugging Face Transformers' Qwen3-Next model (Apache-2.0) and torchaudio (BSD-2-Clause); see NOTICE and licenses/.