Darija-Translator

A Nano-Transformer for Tunisian Darija-to-English translation, built from scratch. Pre-trained on Moroccan Darija data (RTX 3050 Laptop), then fine-tuned on hand-crafted Tunisian pairs (RTX 4070 Desktop). Early baseline, not a production translator.

Model Details

• Architecture: Encoder-Decoder Transformer
• Parameters: ~15.6M
• Tokenizer: SentencePiece BPE (16,000 tokens, Arabizi-aware; markers 3/7/9/5 protected)
• Training data: 35,977 cleaned Moroccan Darija pairs
• Fine-tuning data: 500 hand-crafted Tunisian pairs across 50 categories
• Train/val/test split: stratified 8/1/1 per category
• BLEU score: 3.89 (sacreBLEU, 0-100 scale) — v1 baseline, June 2026, test set never trained on
• Best checkpoint: epoch 7/20 (by validation loss)

Files

• best_model.pt — fine-tuned weights (dict: epoch, avg_loss, model)
• model.py — the DarijaTransformer architecture (needed to load the weights)
• darija_bpe.model — the SentencePiece tokenizer
• config.json — architecture + eval metadata
• inference.py — self-contained greedy decoding

Usage

pip install torch sentencepiece
# put all the files in one folder, then:
python inference.py

Or in code:

from inference import translate
print(translate("9adeh soumha"))

Limitations

500 Tunisian pairs is a small foundation. Simple sentences show recognizable structure; complex cultural expressions are still limited. The dataset is growing through community and crowdsourcing collection, with provenance and quality prioritized over volume; retrains will be documented with updated BLEU scores.

Links

• Dataset: https://huggingface.co/datasets/Dhiadev-tn/tunisian-darija-english
• Code: https://github.com/dhiadev-tn/darija-translator