Instructions to use lightx2v/Wan2.1-T2V-14B-StepDistill-CfgDistill-Lightx2v with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use lightx2v/Wan2.1-T2V-14B-StepDistill-CfgDistill-Lightx2v with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("lightx2v/Wan2.1-T2V-14B-StepDistill-CfgDistill-Lightx2v", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
lora key error
With the Wan21_T2V_14B_lightx2v_cfg_step_distill_lora_rank64.safetensors, got following error log
model_type FLOW
lora key not loaded: diffusion_model.blocks.0.diff_m
lora key not loaded: diffusion_model.blocks.1.diff_m
lora key not loaded: diffusion_model.blocks.10.diff_m
lora key not loaded: diffusion_model.blocks.11.diff_m
lora key not loaded: diffusion_model.blocks.12.diff_m
lora key not loaded: diffusion_model.blocks.13.diff_m
lora key not loaded: diffusion_model.blocks.14.diff_m
lora key not loaded: diffusion_model.blocks.15.diff_m
lora key not loaded: diffusion_model.blocks.16.diff_m
lora key not loaded: diffusion_model.blocks.17.diff_m
lora key not loaded: diffusion_model.blocks.18.diff_m
lora key not loaded: diffusion_model.blocks.19.diff_m
lora key not loaded: diffusion_model.blocks.2.diff_m
lora key not loaded: diffusion_model.blocks.20.diff_m
lora key not loaded: diffusion_model.blocks.21.diff_m
lora key not loaded: diffusion_model.blocks.22.diff_m
lora key not loaded: diffusion_model.blocks.23.diff_m
lora key not loaded: diffusion_model.blocks.24.diff_m
lora key not loaded: diffusion_model.blocks.25.diff_m
lora key not loaded: diffusion_model.blocks.26.diff_m
lora key not loaded: diffusion_model.blocks.27.diff_m
lora key not loaded: diffusion_model.blocks.28.diff_m
lora key not loaded: diffusion_model.blocks.29.diff_m
lora key not loaded: diffusion_model.blocks.3.diff_m
lora key not loaded: diffusion_model.blocks.30.diff_m
lora key not loaded: diffusion_model.blocks.31.diff_m
lora key not loaded: diffusion_model.blocks.32.diff_m
lora key not loaded: diffusion_model.blocks.33.diff_m
lora key not loaded: diffusion_model.blocks.34.diff_m
lora key not loaded: diffusion_model.blocks.35.diff_m
lora key not loaded: diffusion_model.blocks.36.diff_m
lora key not loaded: diffusion_model.blocks.37.diff_m
lora key not loaded: diffusion_model.blocks.38.diff_m
lora key not loaded: diffusion_model.blocks.39.diff_m
lora key not loaded: diffusion_model.blocks.4.diff_m
lora key not loaded: diffusion_model.blocks.5.diff_m
lora key not loaded: diffusion_model.blocks.6.diff_m
lora key not loaded: diffusion_model.blocks.7.diff_m
lora key not loaded: diffusion_model.blocks.8.diff_m
lora key not loaded: diffusion_model.blocks.9.diff_m
lora key not loaded: diffusion_model.head.diff_m
same issue here.
The diff_m keys correspond to the modulation keys in the original model, but ComfyUI doesn't know how to deal with them.
Hello, diff_m represents the difference of modulation, and this feature is supported in lightx2v. Since actual testing shows that diff_m has basically no impact on model performance, we have removed it for compatibility with ComfyUI. This bug should no longer occur now.
Cool thanks!
But I still have it in the Terminal:
lora key not loaded: diffusion_model.blocks.39.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.39.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.39.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.39.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.39.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.4.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.4.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.4.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.4.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.4.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.4.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.4.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.5.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.5.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.5.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.5.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.5.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.5.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.5.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.6.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.6.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.6.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.6.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.6.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.6.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.6.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.7.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.7.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.7.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.7.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.7.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.7.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.7.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.8.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.8.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.8.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.8.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.8.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.8.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.8.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.9.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.9.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.9.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.9.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.9.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.9.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.9.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.img_emb.proj.0.diff
lora key not loaded: diffusion_model.img_emb.proj.0.diff_b
lora key not loaded: diffusion_model.img_emb.proj.1.diff_b
lora key not loaded: diffusion_model.img_emb.proj.1.lora_down.weight
lora key not loaded: diffusion_model.img_emb.proj.1.lora_up.weight
lora key not loaded: diffusion_model.img_emb.proj.3.diff_b
lora key not loaded: diffusion_model.img_emb.proj.3.lora_down.weight
lora key not loaded: diffusion_model.img_emb.proj.3.lora_up.weight
lora key not loaded: diffusion_model.img_emb.proj.4.diff
lora key not loaded: diffusion_model.img_emb.proj.4.diff_b
If you're using it with Wan 2.2 models, that's normal. The original issue related to the diff_m keys.