Hugging Face's logo

intel-ai
/
MiniMax-H3-MXFP8-DiT-MXFP4-TE-AutoRound

Text-to-Video
MiniMax H3
Safetensors
text-to-audio-video
image-to-video
audio-video-generation
multimodal
mixed-precision
mxfp8
mxfp4
fp8
fp4
quantized
auto-round
vllm
Model card Files
xet
 Community

MiniMax-H3 β€” MXFP8 DiT + MXFP4 Text-Encoder (mixed precision), vLLM-Omni native layout

A mixed-precision weight-quantized build of MiniMaxAI/MiniMax-H3 β€” the omni-modal (video + native stereo audio) generative DiT β€” produced with Intel auto-round (zero-shot RTN) and converted to the native fused layout that vLLM-Omni serves.

The recipe: the two diffusion transformers stay at MXFP8 (OCP microscaling FP8), while the Qwen3-VL text encoder is pushed down to MXFP4 (OCP microscaling FP4). This is the recommended memory-saver: a controlled 2Γ—2 ablation (below) shows output quality is driven almost entirely by the DiT precision, while the text encoder can be taken to 4-bit near-losslessly. The large AdaLN branch of each DiT is deliberately kept in BF16.

TL;DR

  • Each DiT ~62 GB (bf16) β†’ ~43.65 GB (β‰ˆ0.71Γ—, βˆ’29 %) at MXFP8; AdaLN (β‰ˆ13 B params/DiT) stays BF16.
  • Text encoder (Qwen3-VL, 64 decoder layers) ~63 GB (bf16) β†’ ~20 GB (β‰ˆ0.31Γ—, βˆ’69 %) at MXFP4: the model.language_model.layers linears are MXFP4 (448 Γ— E2M1 weight_packed + 448 Γ— U8 E8M0 weight_scale); the vision tower, embeddings and norms stay BF16. Roughly half the MXFP8 TE (~34 GB) and ~β…“ of the bf16 TE.
  • Quality is on par with bf16: imaging_quality 0.6404 vs 0.6306 (bf16), i.e. +1.55 % β€” slightly above bf16 within sampling noise, and above the all-MXFP8 integral build (0.6284). Reference metrics vs bf16: PSNR 19.20 dB / SSIM 0.620 / LPIPS 0.288.
  • Serves end-to-end on 2Γ—H200 (TP2) and produces video with synchronized audio. H200 runs the FP8/FP4 weights through Marlin dequant kernels (correct, memory-saving, not faster).
  • Requires a small vLLM-Omni encoder.py patch so the serialized MXFP4 text encoder is routed, scale-remapped and repacked correctly (plus a non-idempotent-finalize guard specific to the MXFP4 path) β€” without it you get weight_scale has no target parameter warnings / an AttributeError and the prompt is ignored (see "Serving").

⚠️ Research / evaluation artifact. This repo is the native fused checkpoint (serve-ready), materialized as real files (no symlinks) so it is upload-ready and survives deletion of the other variants.

Showcase β€” official T2VA example

The official Text-to-Audio-Video (T2VA) render for this exact mixed-precision model (1344Γ—768, 24 fps, 32 steps, 10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task t2va, official "starship captain" prompt) is pending GPU availability and will be backfilled into assets/. Because the 2Γ—2 ablation below shows that text-encoder precision barely affects the output, this recipe is visually indistinguishable from the all-MXFP8 integral render shipped in assets/t2va_official_mxfp8_full.mp4 (DiT MXFP8 + TE MXFP8), which shares the same MXFP8 DiT β€” use it as a representative preview alongside the bf16 reference.

bf16 (reference) MXFP8-integral preview (same DiT)
assets/t2va_official_bf16.mp4 assets/t2va_official_mxfp8_full.mp4

The headline evidence for this model is quantitative: dense93 VBench imaging_quality 0.6404 (+1.55 % vs bf16) with PSNR 19.20 / SSIM 0.620 / LPIPS 0.288 β€” see Quality below.

What was quantized

H3 has two DiTs sharing one architecture, plus one shared text encoder:

Component Role Quantized?
transformer (FL2VA / T2VA DiT, 33.12 B) video+audio denoiser βœ… MXFP8 (from BF16)
Ref2VA DiT (33.12 B) reference-conditioned denoiser βœ… MXFP8 (from BF16)
text_encoder (Qwen3-VL) text / vision encoding βœ… MXFP4 (language-model layers)
video_vae / audio_vae video / audio decode ❌ unchanged
adaln_proj.linear (Γ—52/DiT) AdaLN modulation (~13 B) ❌ kept BF16 on purpose

Per DiT β€” quantized Linears (MXFP8) (6 per block Γ— 52 = 312): attn.{to_q,to_k,to_v,to_out.0} + ff.net.0.proj (SwiGLU gate+up) + ff.net.2 (down). Per DiT: 312 Γ— F8_E4M3 weights + 312 Γ— U8 E8M0 block scales + 314 Γ— BF16 (AdaLN, norms, embedders, biases). AdaLN stays BF16 because vLLM-Omni precomputes and caches its modulation β€” pure inference never runs it through a matmul, so quantizing it would cost accuracy for zero benefit.

Text encoder β€” quantized Linears (MXFP4) (7 per layer Γ— 64 = 448): every model.language_model.layers.N.{self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj} is stored as 4-bit weight_packed (OCP E2M1, two values per uint8 byte) + U8 E8M0 weight_scale (one per 32-element group). The vision tower, embeddings, final norm and lm_head stay BF16 (β‰ˆ4 GiB). The vision encoder is not quantized.

Quant format (per component config.json β†’ quantization_config): quant_method=auto-round, data_type=mx_fp, packing_format="auto_round:llm_compressor", sym=True, autoround_version=0.17.0.

  • DiT: bits=8, group_size=32, act_bits=8, act_group_size=32, act_dynamic=True, block_name_to_quantize="token_refiner.blocks,blocks".
  • Text encoder: bits=4, group_size=32, act_bits=4, act_group_size=32, act_dynamic=True, block_name_to_quantize="model.language_model.layers".

Size / compression

Component (per instance) bf16 this model ratio
DiT (transformer, Γ—2) ~62 GiB 43.65 GiB (MXFP8) 0.71Γ— (βˆ’29 %)
text encoder (Qwen3-VL) ~63 GiB ~20 GiB (MXFP4) 0.31Γ— (βˆ’69 %)

Per-DiT breakdown (safetensors header scan): 312 Γ— F8_E4M3 = 18.66 GiB, 312 Γ— U8 = 0.58 GiB, 314 Γ— BF16 (AdaLN + norms + embedders) = 24.34 GiB, 12 Γ— F32 = 0.06 GiB β†’ 43.65 GiB. The MXFP4 text encoder is ~19.4 GiB (4 shards): E2M1 weight_packed (14.5 GiB) + U8 block scales (0.9 GiB) + BF16 vision tower / embeddings / lm_head (~4 GiB).

Total file bytes β‰ˆ 147 GiB (157.6 GB) across both partitions β€” this is what the HF repo lists and what you download back. Locally the ~19.4 GiB MXFP4 text encoder is hardlinked so it occupies disk only once (du β‰ˆ 128 GiB), but each partition path (FL2VA/text_encoder and Ref2VA/text_encoder) carries its own copy on upload, so the TE is serialized twice. Each partition also keeps its own 43.65 GiB MXFP8 DiT. Dropping the TE from MXFP8β†’MXFP4 still shaves ~28 GiB off the all-MXFP8 integral build's upload size (two 34 GiB TEs β†’ two 19.4 GiB TEs).

Directory layout

Native, serve-ready layout (what vLLM-Omni's combined mode loads). All entries are real files (materialized via hardlinks; find -type l is empty):

MiniMax-H3-MXFP8-DiT-MXFP4-TE-AutoRound/
β”œβ”€β”€ model_index.json          # pipeline manifest
β”œβ”€β”€ README.md                 # this card
β”œβ”€β”€ assets/                   # showcase mp4s
β”œβ”€β”€ transformer/config.json   # ~1 KB shim: carries the MXFP8 DiT quantization_config for auto-detect
β”œβ”€β”€ FL2VA/                     # self-contained T2VA/FL2VA pipeline
β”‚   β”œβ”€β”€ transformer/           # 11 shards, MXFP8 DiT (+ .weight_scale)
β”‚   β”œβ”€β”€ text_encoder/          #  4 shards, MXFP4 Qwen3-VL (weight_packed + weight_scale + quant cfg)
β”‚   β”œβ”€β”€ video_vae/  audio_vae/ tokenizer/  processor/  model_index.json
└── Ref2VA/                    # same structure; text_encoder shares inodes with FL2VA

Each partition's text_encoder is the MXFP4 Qwen3-VL (its config.json / quantization_config.json carry data_type=mx_fp, bits=4), not the BF16 encoder shipped by the base model. The top-level transformer/config.json anchor carries the MXFP8 DiT quant config (the DiT is unchanged in this recipe).

Relationship to the other variants

Repo DiT Text encoder dense93 imaging_quality Note
MiniMax-H3 (base) BF16 BF16 0.6306 original, double-packaged modular + pipelines
MiniMax-H3-mxfp8-native MXFP8 BF16 0.6313 DiT-only quant; serve-ready
-mxfp8-full-native (integral) MXFP8 MXFP8 0.6284 all-MXFP8 DiT+TE
this (-MXFP8-DiT-MXFP4-TE) MXFP8 MXFP4 0.6404 βœ… recommended memory-saver

Why this recipe β€” the 2Γ—2 mixed-precision ablation

Four builds were generated on the same 93 dense prompts / same seeds / same code path and scored with VBench imaging_quality (higher = better; bf16 = 0.6306):

TE = MXFP8 TE = MXFP4
DiT = MXFP8 0.6284 (integral) 0.6404 ← this model
DiT = MXFP4 0.2722 (forward mix) 0.2724 (integral MXFP4)

Reading rows (fix the DiT, vary the TE), quality barely moves. Reading columns (fix the TE, vary the DiT), quality jumps a full tier. DiT precision determines the output; text-encoder precision is nearly irrelevant to quality. So the best value is to keep the DiT at MXFP8 and take the TE all the way to MXFP4 β€” which is exactly this model. (Pushing the DiT to MXFP4 collapses quality to ~0.27 regardless of the TE, i.e. washed-out, low-contrast frames.)

Serving (vLLM-Omni, 2Γ—H200 / TP2) 

CUDA_VISIBLE_DEVICES=1,2 \
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3-MXFP8-DiT-MXFP4-TE-AutoRound --omni \
  --host 0.0.0.0 --port 8093 --trust-remote-code \
  --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \
  --enforce-eager --diffusion-attention-backend CUDNN_ATTN \
  --init-timeout 1800 --stage-init-timeout 1200
# no --task-type β‡’ combined (loads FL2VA + Ref2VA; required for T2VA)
  • TP2 is mandatory: create_weights allocates the BF16 skeleton first (two DiTs β‰ˆ 176 GB), so a single 141 GB H200 OOMs; --enable-cpu-offload is incompatible (Marlin repack is CUDA-only).
  • Raise the init timeout. This build loads two MXFP8 DiTs plus a quantized text encoder (Marlin repack), which can exceed the default --init-timeout 600 on slow/contended storage; use --init-timeout 1800 --stage-init-timeout 1200 (the bundled scripts/serve_h3_mxfp8.sh exposes INIT_TIMEOUT / STAGE_INIT_TIMEOUT). MXFP8-DiT builds load slower than MXFP4-DiT ones (~7–8 min to GET /health β†’ 200).
  • Ready in ~7–8 min; GET /health β†’ 200; ~81 GB/GPU; 0 weight_scale has no target parameter warnings, 0 AttributeError; the DiTs dispatch MarlinMxfp8 and the TE dispatches MarlinMxFp4 β‡’ both quantized paths are active.
  • duration must be in [4, 15] s. Submit to /v1/videos/sync.

Required vLLM-Omni patch (text encoder)

Serving the quantized text encoder needs a small, un-upstreamed patch to vllm_omni/diffusion/models/minimax_h3/encoder.py:

  1. Component-level quant routing β€” read the serialized TE config.json β†’ quantization_config (data_type=mx_fp) and build a text-encoder quant config scoped to text_encoder.text_model.layers (the pipeline otherwise passes the DiT config, so the TE loads unquantized).
  2. RowParallel scale-aware loader β€” shard the grouped weight_scale (in/32) on the same dim as the weight, so scales stay aligned under TP.
  3. Suffix-agnostic fused mapping β€” fuse q/k/v β†’ qkv_proj and gate/up β†’ gate_up_proj for both .weight_packed/.weight and .weight_scale.
  4. Post-load repack β€” run process_weights_after_loading on the TE's quantized Linears after device placement (the pipeline skips the generic post_load_weights, and Marlin repack needs CUDA).
  5. MXFP4 non-idempotent-finalize guard β€” INCMxfp4LinearMethod.process_weights_after_loading destructively replaces layer.weight with the packed buffer and deletes the source; the patched loader skips the MXFP4 finalize when the packed buffer is already absent, so the double-finalize path (generic post-load + our explicit repack) no longer raises AttributeError. (This guard is specific to the MXFP4 TE in this model; the all-MXFP8 build does not need it.)

Without the patch: weight_scale has no target parameter warnings and/or an AttributeError on the MXFP4 finalize, the TE falls back to an unquantized/zeroed path, and the model ignores the prompt.

(The two MXFP8 DiTs additionally need the -native serving fixes: a top-level transformer/config.json quant anchor, grouped-qkv / fc1 scale reordering in load_weights, and TP2 without cpu-offload.)

Hardware note (important)

H200 (Hopper) has no native microscaling FP8/FP4 tensor core; vLLM-Omni runs these weights through Marlin weight-only dequant GEMMs: it dequantizes the MXFP8 (E4M3 + E8M0) / MXFP4 (E2M1 + E8M0) weight inside the GEMM and multiplies against unquantized bf16 activations. It is functionally correct and memory-saving, but not faster than bf16 on Hopper. The checkpoints' act_data_type=mx_fp / act_bits is quantization-recipe metadata that is not exercised on H200 (activations stay bf16). True W8A8 / W4A4 (microscaling activations on a native tensor core) requires Blackwell/B300. Dequant does not change stored values, so accuracy matches a native kernel.

Quality β€” VBench (dense93, H200, n=93)

All builds generate the same 93 dense-prompt videos (512Γ—288, 16 steps, 24 fps, 4 s, seeds 1000+i) on the same vLLM-Omni code path, then are scored with VBench imaging_quality (MUSIQ-SPAQ, no-reference per-frame). The bf16 reference is MiniMax-H3-bf16-native (DiT and TE both bf16). See omini_benchmark/MINIMAX_H3_MXFP8_ι‡εŒ–δΈŽθ―„δΌ°.md Β§4.9 for the exact commands and the full 2Γ—2 ablation.

β‘  imaging_quality (0–1, higher = better):

Model imaging_quality /100 vs bf16
bf16 (native, DiT+TE bf16) 0.6306 63.06 β€”
MXFP8 DiT-only (-native) 0.6313 63.13 +0.12%
MXFP8 DiT + MXFP8 TE (integral) 0.6284 62.84 βˆ’0.35%
MXFP8 DiT + MXFP4 TE (this model) 0.6404 64.04 +1.55%

Taking the text encoder to 4-bit keeps no-reference image quality on par with bf16 (in fact nominally above it, within the Β±0.3 diffusion-sampling scatter) β€” consistent with the ablation that TE precision barely moves the output.

β‘‘ Reference metrics vs bf16 (same prompt + same seed ⟹ frame-aligned; PSNR↑ / SSIM↑ / LPIPS↓):

Pair PSNR (dB) SSIM LPIPS
MXFP8 DiT + MXFP4 TE vs bf16 19.20 0.620 0.288
(for reference) MXFP8 integral vs bf16 ~20.05 0.652 0.254

The reverse recipe tracks bf16 almost as closely as the all-MXFP8 integral build despite halving the TE again β€” the per-sample divergence (same scene/soundscape, pixels slightly shifted) is the expected effect of a small weight perturbation amplified by the 16-step denoiser, dominated by the DiT.

Limitations

  • Not faster than bf16 on Hopper; needs 2 GPUs to serve.
  • Serving the quantized text encoder requires the un-upstreamed encoder.py patch above (including the MXFP4 finalize guard).
  • VAEs and AdaLN kept BF16; TE vision tower / embeddings / lm_head kept BF16.
  • Native/diffusers dual packaging means serving needs the conversion + the fixes above.
  • The this-model official T2VA showcase clip is pending and will be backfilled into assets/.

License & attribution

Inherits the base model's license: MiniMax-H3 Community License Agreement (license: other, see LICENSE). This is a derivative (quantized) checkpoint of MiniMaxAI/MiniMax-H3; all model credit belongs to MiniMax. Quantization by Intel auto-round (zero-shot RTN, mixed MXFP8/MXFP4) with PR #2481. Please also comply with the upstream MiniMax-H3 usage terms.

Downloads last month
561
Inference Providers NEW
Text-to-Video
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for intel-ai/MiniMax-H3-MXFP8-DiT-MXFP4-TE-AutoRound

Base model

MiniMaxAI/MiniMax-H3
Quantized
(82)
this model