Heitorkk2/Kitsune-TTS-V1

Ultra-lightweight Text-to-Speech for anime personas and virtual assistants.

Brazilian Portuguese 🇧🇷   ·   ~39M parameters   ·   5 voices

Kitsune-TTS V1

A compact, non-autoregressive speech model built on VITS2-Slim, trained on synthetic data for expressive anime-style voices. Run it locally with PyTorch or use the ONNX export for CPU and web inference.

This repository contains the model weights and matching configuration. The Python API, JavaScript client, training tools and examples live in the GitHub repository.

At a glance

Feature	Details
Architecture	VITS2-Slim with stochastic duration prediction and speaker embeddings
Parameters	Approximately 39M
Language	Brazilian Portuguese (PT-BR)
Audio	22,050 Hz, mono
Voices	5 built-in speakers
Checkpoints	FP32 and FP16
ONNX	FP32 inference graph, no quantization
License	GPL-3.0

Meet the voices

Use the speaker key with the Python API, or the numeric ID with the JavaScript client.

ID	Voice	Speaker key	Character
0	Emilia	emilia	Soft, sweet and gentle
1	Frieren	frieren	Calm, serene and steady
2	Zero Two	zerotwo	Energetic, teasing and playful
3	Violet	violet	Formal, composed and expressive
4	Hiro	hiro	Youthful, calm male voice

Choose your format

File	Use	Approximate size
latest_model_fp16.pth	Smaller PyTorch checkpoint; also used by the fine-tuning notebook	80 MB
latest_model_fp32.pth	Full-precision PyTorch checkpoint and ONNX export source	159 MB
kitsune39M.onnx	Single-file inference graph for ONNX Runtime	121 MB
model_config.json	Architecture, sample rate and speaker map	1 KB

Keep model_config.json beside the checkpoint or ONNX file. Always use weights and configuration from the same model release.

FP16 describes the stored checkpoint precision. The current Python API loads PyTorch weights into an FP32 model by default.

Quick start

Install eSpeak NG on your system for phonemization, then set up the source code:

git clone https://github.com/Heitorkk2/Kitsune-TTS.git
cd Kitsune-TTS
pip install -e ".[torch,onnx]" huggingface_hub scipy

PyTorch

Download the compact checkpoint and configuration:

hf download Heitorkk2/Kitsune-TTS-V1 latest_model_fp16.pth model_config.json --local-dir model

Generate your first WAV:

from scipy.io.wavfile import write
from kitsune.api import KitsuneSynthesizer

synth = KitsuneSynthesizer(
    checkpoint="model/latest_model_fp16.pth",
    device="cpu",
)
print(synth.list_speakers())

audio = synth.synthesize("Olá! Eu sou a Frieren.", speaker="frieren")
write("output.wav", synth.sample_rate, audio)

ONNX

Download the graph and its configuration:

hf download Heitorkk2/Kitsune-TTS-V1 kitsune39M.onnx model_config.json --local-dir model

Use the same API with the ONNX backend:

from kitsune.api import KitsuneSynthesizer

synth = KitsuneSynthesizer(onnx_path="model/kitsune39M.onnx")
audio = synth.synthesize("Uma voz pequena, com muito a dizer.", speaker="emilia")

For browser/Node.js usage and export instructions, see the JavaScript client and ONNX guide.

Add your own voice

Configure your recordings and transcripts in the notebook to create a separate checkpoint for one or more new speakers. The model architecture stays the same; each new speaker adds one embedding row.

Fine-tuning is experimental; defaults may evolve as we test more voices.

Full-generator fine-tuning can change the original voices in the adapted model. Keep this base checkpoint if you want to continue using the original voices.

Usage notes

• V1 supports Brazilian Portuguese. Other languages are not supported by this release.
• Speech quality, pacing and pronunciation can vary with the input text.
• Runtime performance depends on your processor, backend, thread settings and text length.
• Use voice recordings and datasets you have permission to use.

Special thanks

Special thanks to Everteson for helping build the Kitsune-TTS model.

Credits & licensing

The Kitsune model weights, code and synthetic dataset are original work, licensed under GPL-3.0.

• Architecture: daniilrobnikov/vits2, MIT. Used as the architectural starting point via weight transplant from a VCTK-pretrained checkpoint. Text encoder, flow and posterior encoder layers were carried over; duration prediction, speaker embeddings and vocabulary embeddings were re-initialized from scratch to support the new model.
• Phonemization: eSpeak NG (GPL-3.0), through the phonemizer library.
• Initial transplant checkpoint: VCTK Corpus, CC BY 4.0.

Acknowledgments

• VITS2: base architecture inspiration.
• OmniVoice (k2-fsa): zero-shot voice cloning used to bootstrap the training corpus; no real recordings were used for that corpus.
• XTTS (Coqui): additional synthetic dataset generation.
• Kokoro TTS: inspiration for compact speech models.
• Piper TTS: ONNX export and CPU inference reference.