hdae/karume-umt5-xxl

umT5-XXL text encoder (int8) — Karume

What is this

The encoder of umT5-XXL (google/umt5-xxl) as a standalone text encoder for the WebGPU inference runtime Karume: the encoder converted into Karume's container format (a .krm part sequence whose first part carries the graph and model descriptors) with int8 weights, and the checkpoint's tokenizer converted into one JSON table.

• One graph, text_encoder: the token ids input_ids [1, L] and the relative-position bucket indices relative_position_buckets [L, L] (both int32) in, the encoder's last hidden states [1, L, 4096] in float32 out. L is the number of tokens of the prompt itself (2 to 512, the end token </s> included) — there is no padding and no attention mask.
• One asset, tokenizer (format karume-umt5-tokenizer/1): the tokenizer of google/umt5-xxl as one JSON table — the vocabulary and its scores, the added tokens, the set of whitespace characters and the maximum length of 512 tokens. It carries no prompt cleaning.
• The relative-position bucket indices depend on L only and are computed on the host: entry [i, j] is the bucket of the distance j − i (key position minus query position) under the upstream T5 rule for a bidirectional encoder with 32 buckets and a maximum distance of 128 (relative_attention_num_buckets / relative_attention_max_distance in the upstream config.json). Each direction has 16 buckets (keys after the query use 16–31): distances 0 to 7 get one bucket each, and longer distances fall into 8 logarithmically spaced buckets that start at 8, 12, 16, 23, 32, 46, 64 and 91 (the last one covers every distance from 91 on).
• Exporter used for the conversion: karume/0.14.0. The distribution manifest is karume.json (karume/5).

Base weights and attribution

Converted into the container format — the original checkpoint is not distributed here.

• xxl: the encoder weights (shared.weight and encoder.*) of google/umt5-xxl at commit 66cb9e7e85526fe440a945569e42c72fb6cbc0ad, licensed apache-2.0 (as of retrieval; full text — a verbatim copy is in LICENSE.md). The decoder and lm_head are not included.
• xxl tokenizer: the files tokenizer.json (SHA-256 af904105ce1071b1…) and tokenizer_config.json (SHA-256 616015e2b6c26f9a…) of the same repository and commit, converted into one JSON table.
• Relation to the Wan checkpoints: every tensor is bit-identical to the float32 text_encoder folder of Wan-AI/Wan2.1-T2V-1.3B-Diffusers (commit 0fad780a…), and the bfloat16 text_encoder of Wan-AI/Wan2.2-TI2V-5B-Diffusers (commit b8fff731…) is its round-to-nearest-even rounding (checked on 2026-10-04).
• Changes made here (listed in full in NOTICE.md, per Apache 2.0 §4(b)): only the encoder converted (no decoder, no lm_head); conversion into the Karume container format; the weight matrices of the linear layers and the vocabulary embedding quantized to int8 with one scale per output channel (per row for the embedding), the relative-position bias tables and the RMSNorm weights kept in float32 at the source values; gelu_new replaced by the equivalent GELU(approximate="tanh"); the graph runs on the valid tokens with the bucket indices as an input; the tokenizer converted into one JSON table. No retraining and no fine-tuning.

Models

Model	Pipeline	Quants	Default quant
xxl (default)	umt5-encoder/1	i8	i8

model selects one of these; omitted, it is xxl. quant defaults to that model's own default quant.

Usage

import { Umt5Encoder } from "jsr:@karume/models/umt5";

const encoder = await Umt5Encoder.fromPretrained({
  repo: "hdae/karume-umt5-xxl",
  // Pin a commit for reproducible builds — without it you track `main`, and a future
  // repo update (renamed files, new manifest format) may break your app.
  // Copy the full hash from this repo's "Files and versions" tab:
  // revision: "<full commit sha>",
});
const { inputIds, hiddenStates } = await encoder.encode("a cat"); // hiddenStates: [L, 4096] f32, row-major
await encoder.dispose();

encode tokenizes the text with the bundled tokenizer, builds the relative-position bucket indices, runs the graph and returns the token ids (L of them, ending with </s>) with the encoder's last hidden states, one row of 4,096 floats per token.

Input rules

The text is tokenized as it is — no prompt cleaning and no Unicode normalization — and </s> is appended. The token ids then equal those of AutoTokenizer in transformers 5.14.1 and of the original tokenizer.json of google/umt5-xxl alike. Inputs on which those two disagree, or that the graph cannot take, are rejected with an error instead of being encoded:

• characters outside the vocabulary;
• special tokens written in the text (</s>, <extra_id_0> and the other added tokens);
• ▁ (U+2581) right after a space;
• whitespace other than the plain space U+0020 (tabs, line breaks, no-break spaces, the ideographic space and so on), and a trailing space — leading spaces and runs of spaces between words are accepted;
• more than 512 tokens (the upstream would truncate silently);
• fewer than 2 tokens (an empty text).

Used by

hdae/karume-wan2.2 (Wan2.2 TI2V 5B) references this encoder from its own karume.json at a pinned commit of this repository (the size and the SHA-256 of every part are declared there too), so WanTi2vPipeline.fromPretrained on that repository fetches the text_encoder from here — you do not pass this repository yourself. It does not use this repository's tokenizer: its own tokenizer asset adds Wan's prompt cleaning.

To run Wan2.2 with a compatible umT5-XXL encoder instead of this one (for example a fine-tune converted with the Karume export recipe), pass it through the component swap seat: components: { text_encoder: { source: { repo, revision } } } in the options of WanTi2vPipeline.fromPretrained. Pin the replacement with a commit — a bare repository name follows main, so the encoder would move with every push while the Wan manifest pins this one. The swap needs the GPU text encoder: textEncoder: "precomputed" never loads umT5, so a swap has no effect there. The replacement is admitted only if its graph description has the same SHA-256 as the text_encoder declared by the Wan manifest and its weights are int8 with float32 tables; the tokenizer still comes from the Wan distribution and is not checked against the replacement.

Model: xxl

Quants

Quant	What it is	Download	Weights	Compute
i8 (default)	int8 weights — Linear and vocabulary-embedding weights stored as int8 (one scale per output channel or row) and computed in f32; norms and relative-position tables stay f32.	5.30 GiB	text_encoder = i8	—

If no quant is given, it runs as i8 (this model's recommended default). Per-file size and sha256 live in karume.json — verify against that at the fetch layer. Dtype labels use the runtime's storage dtype vocabulary (f16 / i8 / i4 / i2), not the fp16 spelling common elsewhere in the ecosystem. Weights ship as Karume container files (.krm), split into numbered parts; a part is fetched and verified on its own.

Device limits

• i8: maxBufferSize ≥ 1,050,148,864 bytes / maxStorageBufferBindingSize ≥ 1,050,148,864 bytes.

The largest resident buffer is the int8 vocabulary embedding, held as one buffer, so the device has to grant these limits.