Instructions to use muscgab/manga-ocr-base-CPT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use muscgab/manga-ocr-base-CPT with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("image-to-text", model="muscgab/manga-ocr-base-CPT")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("muscgab/manga-ocr-base-CPT") model = AutoModelForMultimodalLM.from_pretrained("muscgab/manga-ocr-base-CPT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
manga-ocr-base-CPT (v2)
OCR for Japanese manga text boxes. A drop-in replacement for kha-white/manga-ocr-base: same architecture (ViT-B/16 encoder + 2-layer BERT decoder, 111M parameters), same input (stretched to 224 ร 224), same Hugging Face format. The original manga_ocr package loads it without code changes. v2 is v1 of this repository trained further on about 40 million manga crops.
Versions
| Version | Git tag | Format | Load with |
|---|---|---|---|
| v2 (main) | v2 |
Hugging Face VisionEncoderDecoder | original manga_ocr package, or transformers |
| v1 (2026-10-15) | v1 |
custom format | manga_ocr_cpt |
Usage
pip install manga-ocr
from manga_ocr import MangaOcr
mocr = MangaOcr("muscgab/manga-ocr-base-CPT")
text = mocr("crop.png")
- Input: a cropped text box. There is no text detection.
- Checked with manga-ocr 0.1.14 under transformers 4.38.2 and 5.14.1.
tokenizer.jsonis included because transformers 5.x cannot build the tokenizer fromvocab.txtalone. - Vocabulary: 6,817 characters. Trained on labels of up to 128 characters.
- Same architecture, so speed is the same as manga-ocr-base.
Accuracy
JMangaBench_Read v1.3.3, 7,789 boxes, RIN v1.2 normalisation. v1 and v2: original manga_ocr package 0.1.14, CPU fp32. manga-ocr-base: the predictions published with the benchmark.
| Model | Composite โ | CER % โ | EM % โ | EM printed | EM non-printed | EM new boxes only |
|---|---|---|---|---|---|---|
| manga-ocr-base-CPT v2 | 98.65 | 0.86 | 95.13 | 96.80 | 90.01 | 95.41 |
| manga-ocr-base-CPT v1 | 97.46 | 1.93 | 91.30 | 94.16 | 82.48 | 91.50 |
| manga-ocr-base (original) | 90.27 | 4.81 | 78.52 | 88.34 | 48.33 | 78.48 |
JMangaBench_Hard v1.0, 3,015 boxes (onomatopoeia and handwriting).
| Model | 1โNED โ | EM % โ |
|---|---|---|
| manga-ocr-base-CPT v2 | 95.39 | 87.10 |
| manga-ocr-base-CPT v1 | 89.46 | 73.53 |
| manga-ocr-base (original) | 55.67 | 31.24 |
Read v1.2 (a subset of Read v1.3.3) was used during development to monitor training. The released weights were chosen on Read v1.3.3 (see Training). "New boxes only" is the 4,660 boxes of v1.3.3 that are not in v1.2. Composite = mean of the printed and non-printed 1โNED, as on the Read leaderboard.
Training
- Initialised from v1, which was initialised from kha-white/manga-ocr-base.
- 84.4% of each batch: crops from publicly available manga pages with pseudo-labels. Text boxes were found by a detector and read by five OCR models: manga-ocr-nar-small, manga-ocr-base-CPT-flex, manga-ocr-charspot-small, Hayai Nova and PaddleOCR-VL-For-Manga.
- 38.3M boxes were accepted by vote: 4 or more models agree, or the votes split 3-1-1.
- 2.28M boxes with split votes (3-2, 2-2-1, 2-1-1-1) were trained on all candidate readings with marginal likelihood.
- 15.6% of each batch: 2.49M crops of the same kinds as v1's training data:
- AnimeText-OCR real crops, 695,702;
- synthetic printed text, 995,156;
- synthetic handwriting, 800,000.
- 137,725 steps with global batch 1,024 (about 141M crops seen).
- Hardware and precision: 8 ร Ascend 910B, bf16.
- Optimiser: AdamW. LR was 5e-4 for the decoder and 1e-4 for the encoder, held constant, then decayed to 0 over the last 10,000 steps.
- From step 24,000, the decoder attention logits were capped at 1,000 (QK-clip) for stability. The architecture is unchanged.
- Released weights: the EMA (ฮฒ = 0.9997) of the weights at the last step. It was chosen over the plain last-step weights on Read v1.3.3 (EM 95.13 vs 94.95; JMangaBench_Hard 87.10 vs 87.06).
Limitations
- Japanese only. Input is stretched to a square, so long or horizontal boxes are weaker; manga-ocr-base-CPT-flex handles them better but needs its own package.
- Labels are machine-made, so errors shared by the teacher models can be learned.
- The training pages cover most publicly downloadable manga. Benchmarks built from public manga pages may overlap with training pages. Boxes whose text matched a Read label (8+ characters) were removed, but this filter is not complete.
License
- Weights: CC BY-NC-SA 4.0 (
LICENSE_MODEL); commercial use requires separate permission. AnimeText-OCR is CC BY-NC-SA 4.0. - Initialised from kha-white/manga-ocr-base (Apache License 2.0). See
NOTICE.md.
- Downloads last month
- 160