Motion adapter pilot r16: weights, model card, ComfyUI graph

LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2026 MatlowAI (Dylan Matlow)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

README.md

@@ -0,0 +1,104 @@
+---
+license: mit
+base_model: MiniMaxAI/MiniMax-H3
+tags:
+  - minimax-h3
+  - lora
+  - video
+  - comfyui
+  - motion
+library_name: diffusers
+pipeline_tag: image-to-video
+---
+
+# MiniMax-H3 motion adapter (pilot, r16)
+
+A small LoRA for MiniMax-H3 that makes the de-rope pass in
+[ComfyUI-MAINodes](https://github.com/matlowai/ComfyUI-MAINodes) behave
+better on fast motion. It is a pilot: rank 16, trained for a few hours on
+38 clips we generated ourselves. It works, it is measurable, and it has
+clear rough edges listed below. We are publishing it as an intermediate
+release while we try something considerably more ambitious that may or may
+not work (an all-in-one adapter that internalises the whole two-pass
+pipeline). If that lands it will replace this file; if it does not, this
+one still does a useful job.
+
+## What it does
+
+MAINodes' Motion Lab regenerates the fast span of a clip on a stretched
+clock (the "de-rope") and recovers it back to real time. That pass has a
+known bad habit: on very fast motion the base model over-produces, and the
+result alternates between advance and snap frame to frame. This adapter
+was trained on that exact task (frames held out of the middle of a burst,
+the model asked to fill them back in), and it teaches the base model to
+spend the extra clock on smoothness instead of invention.
+
+Measured on four clips the adapter never saw during training (a fight,
+an anime sword ring, a character swap, a dialogue close-up), against the
+same graph without the adapter and against a same-seed noise floor:
+
+- advance/snap alternation improved on 4 of 4 clips (4.6 to 6.1 times the
+  floor)
+- frame-to-frame over-production dropped from 1.76 to 2.93 down to 1.09
+  to 1.23 on all four
+- blind operator picks, adapter vs no adapter, same seed: 7 for, 0
+  against, 1 no preference
+- it transfers to the Ref2VA checkpoint (same wins on both), so one file
+  serves both fl2va and ref2va graphs
+
+Where it is not great (also measured):
+
+- it costs about 1 dB of anchor fidelity on native keyframes and it
+  over-corrects calm chains (it smooths things that did not need
+  smoothing); keep it off on calm content
+- on the Ref2VA checkpoint it mutes colour. A prismatic creature came out
+  as a plain calico; mean saturation dropped 46 to 38 at denoise 0.40 and
+  54 to 40 at 0.70. If your reference has strong colour, either lower the
+  strength or accept the trade
+- it is anime-heavy in effect: on very fast anime it wins by a mile at
+  every setting; on live-action character work it needs the lower inject
+  values below
+
+## Settings that work
+
+- Strength 1.0. Strength and inject turned out to be the same dial, and
+  1.0 wins on alternation in every paired cell we ran; 0.75 was
+  in-distribution tuning on one fight plate and does not hold elsewhere.
+- Inject is the dial. Starting points: 0.45 for character or dialogue
+  where the base model over-produces hard; 0.30 where identity or props
+  are the deliverable and the base is only mildly inflated; on very fast
+  anime it does not matter (every setting wins).
+- Keep the tail guide on. Turning it off costs identity about 3 times the
+  floor across seeds and checkpoints, adapter or not.
+- Apply it to the de-rope pass only, not to the first (text-to-video)
+  pass.
+
+## Use in ComfyUI
+
+Put the file in `models/loras/minimax_h3/`. Load it with a stock
+`LoraLoaderModelOnly` at strength 1.0 on the model that feeds the de-rope
+pass. A ready graph is in ComfyUI-MAINodes:
+`examples/motion_pipeline_adapter_api.json` (API format, loads through the
+API and through the frontend's API import). The stock
+`LoraLoaderModelOnly` is fine for this file; the key mismatch that bites
+turbo LoRAs does not apply here.
+
+## Training facts
+
+- Base: MiniMax-H3 (fl2va), rank 16, alpha 16, 375 steps, lr 1e-4,
+  adamw8bit, bf16, about 4.6 h on one GPU
+- Data: 38 holdout-infilling items cut from 15 clips we generated with
+  MiniMax-H3 (physics scenes, fights, chains, a dialogue plate); no real
+  footage, no third-party footage
+- Task: hold out the frames inside a motion burst, keep the rest as clean
+  context (kept_label 1.0), predict the held tokens
+- Trainer: our ai-toolkit extension for H3 (span-aware packing verified
+  bit-exact against ComfyUI)
+
+## Files
+
+- `minimax_h3_motion_adapter_pilot_r16.safetensors` (63 MB)
+- `motion_pipeline_adapter_api.json` (the same graph as in the node pack)
+
+Examples, before/after clips and the measurement write-up live on the
+[ComfyUI-MAINodes page](https://matlowai.github.io/ComfyUI-MAINodes/).

minimax_h3_motion_adapter_pilot_r16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b31e67d0399eab21ae45fadddcefe82293b1c6cf677b87f8f19bcc745d02fe4
+size 63103768

motion_pipeline_adapter_api.json

@@ -0,0 +1,543 @@
+{
+  "119": {
+    "class_type": "VAELoader",
+    "inputs": {
+      "vae_name": "minimax_h3/minimax_h3_video_vae_fp16.safetensors"
+    },
+    "_meta": {
+      "title": "VAELoader"
+    }
+  },
+  "120": {
+    "class_type": "VAELoader",
+    "inputs": {
+      "vae_name": "minimax_h3/minimax_h3_audio_vae_fp32.safetensors"
+    },
+    "_meta": {
+      "title": "VAELoader"
+    }
+  },
+  "121": {
+    "class_type": "VAEDecodeAudio",
+    "inputs": {
+      "samples": [
+        "125",
+        0
+      ],
+      "vae": [
+        "120",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "VAEDecodeAudio"
+    }
+  },
+  "122": {
+    "class_type": "VAEDecode",
+    "inputs": {
+      "samples": [
+        "125",
+        0
+      ],
+      "vae": [
+        "119",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "VAEDecode"
+    }
+  },
+  "125": {
+    "class_type": "SamplerCustomAdvanced",
+    "inputs": {
+      "noise": [
+        "129",
+        0
+      ],
+      "guider": [
+        "126",
+        0
+      ],
+      "sampler": [
+        "135",
+        0
+      ],
+      "sigmas": [
+        "124",
+        0
+      ],
+      "latent_image": [
+        "131",
+        1
+      ]
+    },
+    "_meta": {
+      "title": "SamplerCustomAdvanced"
+    }
+  },
+  "126": {
+    "class_type": "BasicGuider",
+    "inputs": {
+      "model": [
+        "127",
+        0
+      ],
+      "conditioning": [
+        "131",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "BasicGuider"
+    }
+  },
+  "129": {
+    "class_type": "RandomNoise",
+    "inputs": {
+      "noise_seed": 20260813
+    },
+    "_meta": {
+      "title": "RandomNoise"
+    }
+  },
+  "128": {
+    "class_type": "CLIPLoader",
+    "inputs": {
+      "clip_name": "minimax_h3/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
+      "type": "minimax",
+      "device": "default"
+    },
+    "_meta": {
+      "title": "CLIPLoader"
+    }
+  },
+  "127": {
+    "class_type": "UNETLoader",
+    "inputs": {
+      "unet_name": "minimax_h3/minimax_h3_fl2va_pruned_int8_convrot.safetensors",
+      "weight_dtype": "default"
+    },
+    "_meta": {
+      "title": "UNETLoader"
+    }
+  },
+  "130": {
+    "class_type": "CreateVideo",
+    "inputs": {
+      "fps": 24,
+      "bit_depth": 8,
+      "images": [
+        "122",
+        0
+      ],
+      "audio": [
+        "121",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "CreateVideo"
+    }
+  },
+  "92": {
+    "class_type": "SaveVideo",
+    "inputs": {
+      "filename_prefix": "video/mainodes_adapter_baseline",
+      "format": "auto",
+      "codec": "auto",
+      "video": [
+        "130",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "SaveVideo"
+    }
+  },
+  "131": {
+    "class_type": "MiniMaxH3ImageToVideo",
+    "inputs": {
+      "prompt": "integrated_multimodal_description: [Shot 1] 2D anime, cel-shaded, cinematic, a medium-wide shot frames a young battle-mage swordswoman alone on a stone platform before a great vermilion torii gate at a cliff edge above a rolling sea of dawn clouds, warm gold light breaking across the platform, small white seabirds far below. She has raven-black hair in two long twin-tails tied with white ribbons that hang to her waist. She wears a sleeveless crimson and white shrine-battle outfit with a bare back and shoulders, a short pleated white skirt with a long open crimson half-skirt trailing behind her knees, white ribbon tassels at both hips, fingerless long black gloves, and lace-topped white thigh-high stockings with red cord ties; she carries a curved crimson-lacquered blade trailing two long white streamers from its hilt. She snaps her free hand up and four translucent spectral swords of pale golden light materialize around her, orbiting her body in a fast level ring at waist height, each trailing thin light. As the ring spins she launches sideways into a one-handed cartwheel directly through the orbit, her legs sweeping overhead in turn, twin-tails, ribbons, half-skirt and streamers whipping through the gaps between the circling spectral blades, and lands back on both feet without breaking the ring's spin. She then drops low and whirls her real blade in a rising diagonal cut; the four spectral swords break from their orbit, streak outward away from her in four directions, and shatter against the stone floor of the platform into brief showers of golden petals of light that fade where they fall. She finishes upright with the blade raised before the torii gate, streamers and twin-tails swinging once and settling. The camera holds a Static Shot at chest height for the whole clip; the platform and gate stay fixed while only the swordswoman, her blade, the spectral swords, her hair, ribbons, skirts and streamers, and the golden petals move. Every frame is crisp and sharp as if shot at a very high shutter speed, her limbs, each ribbon, streamer and skirt pleat held in fine clean detail through every instant of the motion, deep depth of field front to back.\n\noverall_soundscape: A bright ringing shimmer as the four spectral swords materialize and orbit with a low circling hum, the sharp rustle and snap of ribbons, pleats and streamers through the cartwheel, both soles landing on stone with a clean double tap, a hard whoosh with the rising cut, four quick glassy bursts as the spectral blades shatter into petals, and a soft high wind over the sea of clouds with distant seabird cries.\n\nnon_diegetic_music: N/A",
+      "width": [
+        "115",
+        0
+      ],
+      "height": [
+        "115",
+        1
+      ],
+      "length": [
+        "132",
+        1
+      ],
+      "clip": [
+        "128",
+        0
+      ],
+      "vae": [
+        "119",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "MiniMaxH3ImageToVideo"
+    }
+  },
+  "124": {
+    "class_type": "BasicScheduler",
+    "inputs": {
+      "scheduler": "simple",
+      "steps": 25,
+      "denoise": 1,
+      "model": [
+        "127",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "BasicScheduler"
+    }
+  },
+  "133": {
+    "class_type": "PrimitiveFloat",
+    "inputs": {
+      "value": 5.0
+    },
+    "_meta": {
+      "title": "Float (duration)"
+    }
+  },
+  "132": {
+    "class_type": "ComfyMathExpression",
+    "inputs": {
+      "expression": "max(5, round(a * 24)) + (5 - (max(5, round(a * 24)) % 17)) % 17",
+      "values.a": [
+        "133",
+        0
+      ]
+    },
+    "_meta": {
+      "title": "ComfyMathExpression"
+    }
+  },
+  "115": {
+    "class_type": "ResolutionSelector",
+    "inputs": {
+      "aspect_ratio": "1:1 (Square)",
+      "megapixels": 1.0,
+      "multiple": 32
+    },
+    "_meta": {
+      "title": "ResolutionSelector"
+    }
+  },
+  "135": {
+    "class_type": "KSamplerSelect",
+    "inputs": {
+      "sampler_name": "res_multistep"
+    },
+    "_meta": {
+      "title": "KSamplerSelect"
+    }
+  },
+  "200": {
+    "class_type": "H3JerkOracle",
+    "inputs": {
+      "samples": [
+        "125",
+        0
+      ],
+      "length": [
+        "132",
+        1
+      ],
+      "q": 0.75,
+      "d_max": 4,
+      "ramp": true
+    }
+  },
+  "201": {
+    "class_type": "H3TimeSmear",
+    "inputs": {
+      "images": [
+        "122",
+        0
+      ],
+      "dilation": 4,
+      "hold_map": [
+        "200",
+        0
+      ]
+    }
+  },
+  "202": {
+    "class_type": "VAEEncode",
+    "inputs": {
+      "pixels": [
+        "201",
+        0
+      ],
+      "vae": [
+        "119",
+        0
+      ]
+    }
+  },
+  "203": {
+    "class_type": "H3V2VInit",
+    "inputs": {
+      "samples": [
+        "202",
+        0
+      ]
+    }
+  },
+  "204": {
+    "class_type": "H3InjectSchedule",
+    "inputs": {
+      "model": [
+        "600",
+        0
+      ],
+      "scheduler": "simple",
+      "total_steps": 25,
+      "inject": 0.45
+    }
+  },
+  "205": {
+    "class_type": "RandomNoise",
+    "inputs": {
+      "noise_seed": 20260899
+    }
+  },
+  "206": {
+    "class_type": "MiniMaxH3ImageToVideo",
+    "inputs": {
+      "prompt": "integrated_multimodal_description: [Shot 1] 2D anime, cel-shaded, cinematic, a medium-wide shot frames a young battle-mage swordswoman alone on a stone platform before a great vermilion torii gate at a cliff edge above a rolling sea of dawn clouds, warm gold light breaking across the platform, small white seabirds far below. She has raven-black hair in two long twin-tails tied with white ribbons that hang to her waist. She wears a sleeveless crimson and white shrine-battle outfit with a bare back and shoulders, a short pleated white skirt with a long open crimson half-skirt trailing behind her knees, white ribbon tassels at both hips, fingerless long black gloves, and lace-topped white thigh-high stockings with red cord ties; she carries a curved crimson-lacquered blade trailing two long white streamers from its hilt. She snaps her free hand up and four translucent spectral swords of pale golden light materialize around her, orbiting her body in a fast level ring at waist height, each trailing thin light. As the ring spins she launches sideways into a one-handed cartwheel directly through the orbit, her legs sweeping overhead in turn, twin-tails, ribbons, half-skirt and streamers whipping through the gaps between the circling spectral blades, and lands back on both feet without breaking the ring's spin. She then drops low and whirls her real blade in a rising diagonal cut; the four spectral swords break from their orbit, streak outward away from her in four directions, and shatter against the stone floor of the platform into brief showers of golden petals of light that fade where they fall. She finishes upright with the blade raised before the torii gate, streamers and twin-tails swinging once and settling. The camera holds a Static Shot at chest height for the whole clip; the platform and gate stay fixed while only the swordswoman, her blade, the spectral swords, her hair, ribbons, skirts and streamers, and the golden petals move. Every frame is crisp and sharp as if shot at a very high shutter speed, her limbs, each ribbon, streamer and skirt pleat held in fine clean detail through every instant of the motion, deep depth of field front to back.\n\noverall_soundscape: A bright ringing shimmer as the four spectral swords materialize and orbit with a low circling hum, the sharp rustle and snap of ribbons, pleats and streamers through the cartwheel, both soles landing on stone with a clean double tap, a hard whoosh with the rising cut, four quick glassy bursts as the spectral blades shatter into petals, and a soft high wind over the sea of clouds with distant seabird cries.\n\nnon_diegetic_music: N/A",
+      "width": [
+        "115",
+        0
+      ],
+      "height": [
+        "115",
+        1
+      ],
+      "length": [
+        "201",
+        2
+      ],
+      "clip": [
+        "128",
+        0
+      ],
+      "vae": [
+        "119",
+        0
+      ]
+    }
+  },
+  "207": {
+    "class_type": "BasicGuider",
+    "inputs": {
+      "model": [
+        "600",
+        0
+      ],
+      "conditioning": [
+        "206",
+        0
+      ]
+    }
+  },
+  "208": {
+    "class_type": "SamplerCustomAdvanced",
+    "inputs": {
+      "noise": [
+        "205",
+        0
+      ],
+      "guider": [
+        "207",
+        0
+      ],
+      "sampler": [
+        "135",
+        0
+      ],
+      "sigmas": [
+        "204",
+        0
+      ],
+      "latent_image": [
+        "203",
+        0
+      ]
+    }
+  },
+  "209": {
+    "class_type": "VAEDecode",
+    "inputs": {
+      "samples": [
+        "208",
+        0
+      ],
+      "vae": [
+        "119",
+        0
+      ]
+    }
+  },
+  "210": {
+    "class_type": "VAEDecodeAudio",
+    "inputs": {
+      "samples": [
+        "208",
+        0
+      ],
+      "vae": [
+        "120",
+        0
+      ]
+    }
+  },
+  "211": {
+    "class_type": "CreateVideo",
+    "inputs": {
+      "images": [
+        "209",
+        0
+      ],
+      "audio": [
+        "210",
+        0
+      ],
+      "fps": 24,
+      "bit_depth": 8
+    }
+  },
+  "212": {
+    "class_type": "SaveVideo",
+    "inputs": {
+      "video": [
+        "211",
+        0
+      ],
+      "filename_prefix": "video/mainodes_adapter_dilated",
+      "format": "auto",
+      "codec": "auto"
+    }
+  },
+  "213": {
+    "class_type": "H3ExactRecover",
+    "inputs": {
+      "images": [
+        "209",
+        0
+      ],
+      "hold_map": [
+        "201",
+        1
+      ]
+    }
+  },
+  "214": {
+    "class_type": "CreateVideo",
+    "inputs": {
+      "images": [
+        "213",
+        0
+      ],
+      "fps": 24,
+      "bit_depth": 8,
+      "audio": [
+        "219",
+        0
+      ]
+    }
+  },
+  "215": {
+    "class_type": "SaveVideo",
+    "inputs": {
+      "video": [
+        "214",
+        0
+      ],
+      "filename_prefix": "video/mainodes_adapter_recovered",
+      "format": "auto",
+      "codec": "auto"
+    }
+  },
+  "216": {
+    "class_type": "H3JerkHeatmap",
+    "inputs": {
+      "images": [
+        "122",
+        0
+      ],
+      "samples": [
+        "125",
+        0
+      ],
+      "alpha": 0.55,
+      "strip_height": 96
+    }
+  },
+  "217": {
+    "class_type": "CreateVideo",
+    "inputs": {
+      "images": [
+        "216",
+        0
+      ],
+      "fps": 24,
+      "bit_depth": 8
+    }
+  },
+  "218": {
+    "class_type": "SaveVideo",
+    "inputs": {
+      "video": [
+        "217",
+        0
+      ],
+      "filename_prefix": "video/mainodes_adapter_oraclemap",
+      "format": "auto",
+      "codec": "auto"
+    }
+  },
+  "219": {
+    "class_type": "H3AudioRecover",
+    "inputs": {
+      "audio": [
+        "210",
+        0
+      ],
+      "hold_map": [
+        "201",
+        1
+      ],
+      "fps": 24,
+      "reference": [
+        "121",
+        0
+      ],
+      "reference_mix": 0.0
+    }
+  },
+  "600": {
+    "class_type": "LoraLoaderModelOnly",
+    "inputs": {
+      "model": [
+        "127",
+        0
+      ],
+      "lora_name": "minimax_h3/minimax_h3_motion_adapter_pilot_r16.safetensors",
+      "strength_model": 1.0
+    },
+    "_meta": {
+      "title": "Motion adapter (de-rope pass only)"
+    }
+  }
+}