wav2vec2-10L-serbian-v1

A Serbian speech recognition model with 10 encoder layers and 139.1M parameters. It takes mono 16 kHz audio and produces lowercase Serbian Latin text with diacritics. The weights are stored in FP16 (265.4 MiB).

The architecture follows wav2vec 2.0, introduced by Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli at Facebook AI (now Meta).

Inference

Install torch, transformers, soundfile, and scipy. Set audio.wav to the path of your audio file.

from math import gcd

import soundfile as sf
import torch
from scipy.signal import resample_poly
from transformers import AutoModelForCTC, AutoProcessor

model_id = "adlinga/wav2vec2-10L-serbian-v1"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dtype = torch.float16 if device.type == "cuda" else torch.float32

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCTC.from_pretrained(model_id, dtype=dtype).to(device).eval()

audio, sample_rate = sf.read("audio.wav", dtype="float32", always_2d=True)
audio = audio.mean(axis=1)
if sample_rate != 16_000:
    divisor = gcd(sample_rate, 16_000)
    audio = resample_poly(audio, 16_000 // divisor, sample_rate // divisor)

inputs = processor(audio=audio, sampling_rate=16_000, return_tensors="pt").to(device, dtype)
with torch.inference_mode():
    token_ids = model(**inputs).logits.argmax(dim=-1)
print(processor.batch_decode(token_ids.cpu())[0])

Evaluation

Model Params FP16 weights (MiB) Common Voice 27.0 SR WER FLEURS SR WER Median TTFT (ms) Time per 30 s (ms)
wav2vec2-10L-serbian-v1 139.1M 265.4 15.46% 24.02% 5.70 38.9
CLASSLA XLS-R 300M Južne Vesti 315.5M 601.7 24.21% 29.15% 9.43 54.4
Meta OmniASR CTC 300M v2 326.0M 621.8 38.12% 22.76% 15.85 89.5
Whisper Large v3 Turbo 808.9M 1,542.8 36.52% 14.22% 28.96 612.9
Istomin Whisper Small SR 241.7M 461.1 6.54% 1.47% 24.84 969.5
Whisper Large v3 1,543.5M 2,944.0 17.37% 11.82% 49.79 2,134.1
Sagicc Whisper Large v3 SR CMB 1,543.5M 2,944.0 6.35% 7.65% 47.84 2,524.9

WER uses greedy decoding and FP16 weights. References and hypotheses are normalized to Serbian Latin, with written numerals expanded into words by the same rules. The rules use a fixed reading for decimals, dates, and other numeric expressions. Alternative spoken readings can still affect WER. WER runs used batch sizes of 12 for Whisper, 8 for CLASSLA and this model, and 1 for OmniASR. Speed was measured separately with batch size 1 on a warmed-up RTX 5090. TTFT starts at waveform input and ends at the first text token from Whisper or the first non-blank token after a full-clip CTC forward pass. Time per 30 s is 30 times the ratio of total inference time to total audio duration across both test sets. These same test sets were used to select the checkpoint average for wav2vec2-10L-serbian-v1 under the previous normalization.

Downloads last month
55
Safetensors
Model size
0.1B params
Tensor type
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for adlinga/wav2vec2-10L-serbian-v1