kybrcore/Qwen3.8-27B-QUASAR-NVFP4-NInfer

Qwen3.8-27B QUASAR NVFP4 — NInfer v3 artifact

Single-file NInfer v3 artifact of the QUASAR-QAT W4A4 model, re-containerized for the NInfer v3 runtime with the vision tower, MTP head, DFlash2 drafter and the indexed proposal head assembled in.

Property	Value
File	qwen3_8_27b_quasar.ninfer
Size	19,782,448,132 bytes (~18.42 GiB)
SHA256	d196bf1d8294f1a684e01870967134eab59bec1692c61736f2452d9fa2d63594
Format	NInfer artifact schema v3 (single file, embedded chat template)
Weights	≈18.0 GiB materialized on device (vision + dflash2 + mtp lanes included)

This is a re-containerization and component assembly, not a new training run. The text trunk is imported bit-exactly from the QUASAR-QAT checkpoint (packed NVFP4 words and stored divisors reproduced verbatim); the vocabulary, vision tower, MTP head and DFlash2 companion are quantized/assembled at conversion time from the pinned sources below. See Composition for per-component provenance.

Composition

Component	Encoding	Source
Text projections (496)	NVFP4 W4A4, import_encoded (bit-exact)	QUASAR-QAT checkpoint
GDN a/b control gates	BF16 (decoded from the same checkpoint)	QUASAR-QAT checkpoint
token_embedding / output_head	q8_g32 groupwise, computed locally	Qwen3.8-27B BF16 base
Vision tower (333 tensors)	mixed Q4/Q5/Q6/Q8 by role	Qwen3.8-27B BF16 base
MTP head	Q8 groupwise	Qwen3.8-27B BF16 base
DFlash2 companion	W8G32	incoai/z-lab drafter checkpoint
Proposal head	Q4 groupwise, frequency-ranked	derived from output_head
Chat template	official qwen3_8.jinja (embedded, xhigh default)	Qwen3.8-27B

Quality

Measured on a single RTX 5090 32 GB through the NInfer v3 runtime (unified OpenAI-compatible endpoint). Serving configuration for the runs: NVFP4 KV, DFlash2=7 speculation, vision enabled, native 262K context envelope, serve-side thinking budget 131,072 (effectively uncapped).

Benchmark	Score	Protocol
GPQA-Diamond	89.39%	default profile, rule-judged, seed 42
AIME 2025	100% (30/30)	temp 1.0 / top_p 0.95 / top_k 20, seed 42
AIME 2024	100% (30/30)	same
IFBench	80.33% strict / 83.33% loose	same, 300 prompts
NIAH 64K / 128K	100% / 100% (EN + ZH)	greedy, thinking off, LLM judge
needle_v2 (9K–195K depth)	4/4	internal regression, 243,638-token frozen text

Quality matches the published bf16-teacher numbers of the QUASAR-QAT line (GPQA-D 89.22 ± 2.49) — no measurable quantization cliff from the W4A4 text trunk. DFlash2 draft acceptance and decode, measured with the upstream serving-corpus method (tools/bench/run_serve_corpus.py, stochastic sampling, 75 requests: the three long_decode_aime26_* reasoning fixtures at a 65,536-token budget plus Code/Story/Translation/Structured fixtures at 4,096). Each column shows this artifact / the published official mixed-profile nvfp4 reference:

Fixture / category	Decode tok/s	Acceptance	tok/round
long_decode_aime26_01	388 / 321	64.3% / 67.2%	5.50 / 5.70
long_decode_aime26_15	232 / 183	35.9% / 35.2%	3.51 / 3.47
long_decode_aime26_30	251 / 200	39.1% / 38.6%	3.74 / 3.70
Code	337 / 266	54.5% / 53.9%	4.82 / 4.77
Story	148 / 121	15.9% / 16.8%	2.11 / 2.17
Translation	308 / 256	48.2% / 51.1%	4.37 / 4.58
Structured	450 / 357	77.7% / 78.1%	6.44 / 6.46
Corpus aggregate	242 / 193	37.2% / 37.0%	3.60 / -

Every acceptance difference is within the per-sample standard deviation: the uniform W4A4 trunk carries no measurable draft-acceptance cost relative to the mixed-profile reference. The decode columns were recorded on the current engine revision (upstream 5b4303c0); the published reference tables date from 2026-09-06 on an earlier revision, so that gap reflects engine kernel-dispatch work rather than an artifact difference.

WikiText-2 perplexity (the standard quantization-damage probe; upstream ninfer-perplexity, context 4096 / stride 2048, wikitext-2-raw test): 6.266 for this W4A4 artifact vs 6.100 for the official mixed-profile nvfp4 artifact (+2.7%) - the full price of uniform W4A4, measured. Scoring bandwidth is ~47% higher on this artifact (NVFP4 tensor-core path).

Serving

Requires an NInfer v3 runtime (the upstream Neroued/ninfer engine serves this artifact as-is). 64-bit Linux, CUDA toolkit with sm_120a, one RTX 5090.

Tested engine revision: Neroued/ninfer @ 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5. This artifact was built and validated against that commit; use it, or a revision whose conversion and kernel paths are known-compatible with it. The artifact format is not versioned against later engine releases, so a newer upstream revision is not guaranteed to load or execute this file — pin the engine, not just the artifact.

git clone https://github.com/Neroued/ninfer
cd ninfer && git checkout 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5

# native MTP=3 speculation, native 262K context
ninfer-serve qwen3_8_27b_quasar.ninfer \
  --model-id qwen3.8-27b --vision \
  --spec mtp --draft-tokens 3 --lm-head-draft \
  --max-context 262144 --kv-dtype nvfp4 \
  --host 0.0.0.0 --port 30000

Memory notes (RTX 5090 32 GB, NVFP4 KV, native 262K context, auto sizing), measured with the upstream engine:

Speculation	Weights	KV pool (auto)	Runtime	Free
MTP=3	16.3 GiB	634,624 tokens	13.5 GiB	1.17 GiB
DFlash2=7	18.0 GiB	492,032 tokens	11.9 GiB	2.29 GiB

For contrast, the official mixed-profile nvfp4 artifact auto-sizes to 433,472 (MTP) and 278,464 (DFlash2) tokens on the same engine and card - the uniform W4A4 trunk leaves ~200K more tokens of pool.

Lineage and attribution

Qwen/Qwen3.8-27B                    (BF16 base, Apache-2.0)
  └─ QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4     (W4A4 QAT checkpoint, Apache-2.0)
       ├─ incoai/Qwen3.8-27B-DFlash2         (DFlash2 drafter, Apache-2.0; z-lab mirror)
       └─ Neroued/ninfer @ 5b4303c0          (v3 engine, converter, artifact format)

• DFlash2 is by Inco AI (block-diffusion drafter, ICML 2026); this artifact embeds the drafter weights quantized to W8G32 as the speculative companion.
• The v3 artifact format, converter and all kernel paths are from Neroued/ninfer (upstream), at the revision pinned in Serving. Later upstream revisions have not been validated against this artifact.
• This repository is an independent packaging of these weights for the upstream runtime; it is not an official release by the upstream engine maintainers or the QUASAR-QAT authors.

Provenance notes

The artifact was produced with the upstream NInfer v3 converter (tools/convert) using a QUASAR-QAT packed-weight import recipe, consuming the three pinned source snapshots listed above.

Reproduction: conversion/reproduce.sh in this repository re-downloads the three pinned sources and reruns the conversion with the upstream converter (Neroued/ninfer @ 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5) and the frozen recipe (conversion/qwen3_8_27b_quasar.py). Reruns are byte-identical to the published artifact outside the random metadata header (~first 452 KB); conversion/SOURCES.md lists the source pins, and conversion/reproduce.sh checks the engine commit out before converting.

Caveats

• The quality numbers above were measured on a single RTX 5090 with the NInfer v3 runtime; treat them as reproducibility anchors, not vendor-published figures.
• Sampling follows the model's officially recommended parameters (temperature 1.0 / top_p 0.95 / top_k 20) for the reasoning suites; the NIAH probes run greedy with thinking disabled.
• The W8 DFlash2 companion differs from the NVFP4 companion used in the earlier v2-format release of this model line; expect slightly larger files and slower draft loads, with draft fidelity preserved.