Add verified INT8 ONNX model with KV cache

Browser-ready Transformers.js layout with Python Optimum support, model card, tokenizer compatibility fixes, quantization metadata, and validated cached generation.

README.md

@@ -0,0 +1,115 @@
+---
+library_name: transformers.js
+pipeline_tag: text-generation
+base_model: SupraLabs/Supra-Router-51M
+tags:
+- onnx
+- transformers.js
+- browser
+- web
+- int8
+- quantized
+- llama
+- router
+- text-generation
+---
+
+# Supra-Router-51M ONNX INT8
+
+Browser-ready ONNX conversion of [SupraLabs/Supra-Router-51M](https://huggingface.co/SupraLabs/Supra-Router-51M). The graph is dynamically quantized to signed INT8 and packaged using the Hugging Face ONNX repository layout.
+
+- **Parameters:** 51.8M
+- **ONNX size:** approximately 66 MB
+- **Architecture:** `LlamaForCausalLM`
+- **Execution:** ONNX Runtime Web (WASM) or ONNX Runtime Python
+- **Conversion:** FP32 export, opset 17, per-channel dynamic INT8 weight quantization
+- **KV cache:** included for efficient autoregressive generation
+
+## Transformers.js pipeline
+
+```bash
+npm install @huggingface/transformers
+```
+
+```javascript
+import { pipeline } from "@huggingface/transformers";
+
+const modelId = "Sharjeelbaig/Supra-Router-51M-ONNX";
+const router = await pipeline("text-generation", modelId, {
+  dtype: "int8",
+});
+
+const userPrompt = "Write Python code to find all primes below one million efficiently.";
+const input = `Task: ${userPrompt}\nAnalysis: `;
+
+const output = await router(input, {
+  max_new_tokens: 128,
+  do_sample: false,
+  return_full_text: false,
+});
+
+console.log(output[0].generated_text.trim());
+```
+
+The model returns a pipe-separated routing record:
+
+```text
+Domain: ... | Complexity: 1-5 | Math: True/False | Code: True/False | Route: small model/big model | Justification: ...
+```
+
+## Python with Optimum ONNX Runtime
+
+```bash
+pip install "optimum-onnx[onnxruntime]" transformers
+```
+
+```python
+from transformers import AutoTokenizer, pipeline
+from optimum.onnxruntime import ORTModelForCausalLM
+
+model_id = "Sharjeelbaig/Supra-Router-51M-ONNX"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = ORTModelForCausalLM.from_pretrained(
+    model_id,
+    subfolder="onnx",
+    file_name="model_int8.onnx",
+    use_cache=False,
+)
+
+router = pipeline("text-generation", model=model, tokenizer=tokenizer)
+prompt = "Explain why database deadlocks occur and provide code to prevent them."
+result = router(
+    f"Task: {prompt}\nAnalysis: ",
+    max_new_tokens=128,
+    do_sample=False,
+    return_full_text=False,
+)
+print(result[0]["generated_text"].strip())
+```
+
+## Direct ONNX Runtime loading
+
+The graph accepts:
+
+- `input_ids`: `int64[batch, sequence]`
+- `attention_mask`: `int64[batch, past_sequence + sequence]`
+- `position_ids`: `int64[batch, sequence]`
+- `past_key_values.{0..11}.{key,value}`: cached attention tensors
+
+It returns `logits`: `float32[batch, sequence, 32000]` plus `present.{0..11}.{key,value}` cache tensors. For direct integration, initialize each cache with shape `[batch, 4, 0, 64]`, then pass each returned `present` tensor back as the corresponding `past_key_values` input on the next step. Hugging Face pipelines manage this automatically.
+
+## Validation
+
+- Passed `onnx.checker`.
+- Tested with dynamic sequence lengths.
+- Cached FP32 ONNX maximum absolute logit error versus PyTorch was below `1.4e-4` during export validation.
+- INT8 and FP32 generated identical complete routing strings on representative `small model` and `big model` prompts.
+- The uploaded INT8 interface was run end-to-end through both a Transformers.js text-generation pipeline and a Python Optimum text-generation pipeline.
+
+## Limitations and responsible use
+
+This is a small routing model trained on 992 examples. Treat its route as a heuristic, not a security boundary or sole safety control. Add deterministic policy checks, timeouts, input-length limits, and a conservative fallback in production.
+
+The upstream model card does not declare a license at the time of this conversion. This repository does not add or replace upstream rights. Confirm use and redistribution terms with SupraLabs before commercial deployment or redistribution.
+
+This conversion was produced independently and is not an official SupraLabs release. Refer to the [upstream model card](https://huggingface.co/SupraLabs/Supra-Router-51M) for intended input formatting and training details.

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 512,
+  "initializer_range": 0.02,
+  "intermediate_size": 1408,
+  "max_position_embeddings": 5120,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 12,
+  "num_key_value_heads": 4,
+  "pad_token_id": 1,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "factor": 1.0,
+    "rope_theta": 10000.0,
+    "rope_type": "linear",
+    "type": "linear"
+  },
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.6",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "pad_token_id": 1,
+  "transformers_version": "4.57.6"
+}

onnx/model_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc09518497a8fee5a449207e58aaa5163a8d6e5b409d9ec992480edea6e177ff
+size 69447683

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc09518497a8fee5a449207e58aaa5163a8d6e5b409d9ec992480edea6e177ff
+size 69447683

quantize_config.json

@@ -0,0 +1,9 @@
+{
+  "modes": ["int8", "q8"],
+  "per_channel": true,
+  "reduce_range": false,
+  "block_size": null,
+  "is_symmetric": true,
+  "accuracy_level": null,
+  "quant_type": 1
+}

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "backend": "tokenizers",
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 5120,
+  "pad_token": "<pad>",
+  "tokenizer_class": "LlamaTokenizerFast",
+  "unk_token": "<unk>"
+}