taobao-content-ai/MiniMax-H3-PTD-RefAsymOPD-LoRAs

MiniMax-H3 PTD + RefAsymOPD Ref2VA Acceleration LoRA

A few-step Ref2VA (reference-image → video + audio) accelerator for MiniMax-H3, combining PTD (Parametric Trajectory Distillation) with RefAsymOPD — Reference-Asymmetric On-Policy Distillation. We release an 8-step and a 6-step variant.

It improves background plausibility, sharpness and realism over plain PTD at the same step count.

At inference each checkpoint is structurally identical to a plain PTD LoRA: one independent low-rank adapter per inference step, applied on top of the per-interval output heads introduced by Parallel Decoding Distillation (PDD). The 8-step variant runs in 8 transformer evaluations, the 6-step in 6, needing no extra input and adding no cost over plain PTD.

This is a sibling of MiniMax-H3-PTD-LoRAs (plain PTD, 8-step and 4-step), not a replacement. Because it is structurally identical, the inference adapter, Diffusers script and ComfyUI nodes are the same code. Pick the 8-step checkpoint for the highest quality, or the 6-step checkpoint for lower latency.

Weights

Name	Base Model	Task	NFE	Description
MiniMax-H3-Ref2VA-PTD-RefAsymOPD-8Step.safetensors	MiniMax-H3	Ref2VA	8	Per-hop PTD LoRA trained with RefAsymOPD, 8 hops (rank=128, network_alpha=128, BF16).
MiniMax-H3-Ref2VA-PTD-RefAsymOPD-6Step.safetensors	MiniMax-H3	Ref2VA	6	Per-hop PTD LoRA trained with RefAsymOPD, 6 hops (rank=128, network_alpha=128, BF16).

The checkpoint carries its configuration in the safetensors __metadata__ (ptd_num_hops, pdd_num_steps=32, pdd_block_size, lora_rank, lora_alpha, lora_targets); apply_ptd_lora reads it and derives num_inference_steps.

Demo (Ref2VA)

Five cases from the training-monitor conditioning set, portrait 768×1344, 124 frames, 24 fps, with audio.

The first column is the MiniMax-H3 base at 40 sequential steps with no LoRA — the quality ceiling the few-step students are chasing, at 5× the compute of the 8-step columns.

Still frames at the native 768×1344 so the detail is visible (mid-point of each clip):

MiniMax-H3 base 40-step (teacher)	PTD + RefAsymOPD 8-step	PTD + RefAsymOPD 6-step
sample-0
sample-1
sample-2
sample-3
sample-4

Full five-way video comparison (base, PDD, PTD, PTD + RefAsymOPD 8-step, PTD + RefAsymOPD 6-step) with audio:

Reference images	MiniMax-H3 base 40-step (teacher)	PDD 8-step	PTD 8-step	PTD + RefAsymOPD 8-step	PTD + RefAsymOPD 6-step
sample-0
sample-1
sample-2
sample-3
sample-4

Everything needed to reproduce a row is in asset/: the reference images in presentation order (sample-N_ref{0..}.jpg, addressed as <Picture 1..> in the prompt), the prompt (sample-N_prompt.txt), and demo_samples.json with the seed, resolution and frame count per sample.

Quick Start

Identical to the plain PTD release — the checkpoint format is the same.

# NFE is read from the checkpoint metadata automatically.
# --case selects one of the five demo rows above (1 = sample-0); demo cases default to this repo.
python predict_ref2v.py --ptd-lora MiniMax-H3-Ref2VA-PTD-RefAsymOPD-8Step.safetensors --case 1
python predict_ref2v.py --ptd-lora MiniMax-H3-Ref2VA-PTD-RefAsymOPD-6Step.safetensors --case 1

import torch
from diffusers import ModularPipeline
from minimax_h3_ptd import apply_ptd_lora

pipeline = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", workflow="ref2va")
pipeline.load_components(dtype=torch.bfloat16, pretrained_model_name_or_path="MiniMaxAI/MiniMax-H3")
# Encode references at the training short edge (768). The pipeline default is 2048, which feeds the
# distilled student ~5x the conditioning tokens it saw in training and degrades output.
pipeline.register_to_config(reference_image_short_edge=768)

nfe = apply_ptd_lora(
    pipeline.transformer_ref,
    "MiniMax-H3-Ref2VA-PTD-RefAsymOPD-8Step.safetensors",  # or -6Step
    pipeline.scheduler.shift,        # video_shift (12.0)
    pipeline.audio_scheduler.shift,  # audio_shift (3.0)
)
result = pipeline(prompt=..., references=[...], num_inference_steps=nfe + 1, ...)

Inference needs diffusers >= 0.40.0 with the MiniMax-H3 ref2va workflow. The base model, text encoder and VAEs are MiniMax's own weights and are downloaded separately — this repository ships only the adapter.

ComfyUI

comfyui_node/ComfyUI-MiniMaxH3-PTD/ holds custom nodes for ComfyUI's native MiniMax-H3; they require ComfyUI-PTD alongside them. This checkpoint loads through them unchanged. See that directory's README for setup and the example workflow.

Reference

1. Lan Feng, et al. "Parametric Trajectory Distillation for Few-Step Video Generation". Preprint (2026). https://alan-lanfeng.github.io/PTD/
2. Neta Shaul, et al. "Parallel Decoding Distillation for Fast Image and Video Generation". arXiv preprint arXiv:2607.26004 (2026).