Hugging Face's logo

caiovicentino1
/
Qwopus3.5-9B-v3-HLWQ-Q5

Text Generation
Safetensors
qwen3_5
hlwq
gptq
int4
qwen3.5
vllm
marlin
conversational
Eval Results (legacy)
4-bit precision
Model card Files
xet
 Community
3

Model not loading vLLM 0.19.0

#1
by Arien0 - opened
Discussion
Arien0

Hi! I'm trying to load your model into vLLM 0.19.0 with transformers 4.57.6 but there's a bunch of problems:

a) Quantization method declared in config.json doesn't match polarquant-vllm plugin:
(APIServer pid=6394) Value error, Quantization method specified in the model config (polar) does not match the quantization method specified in the quantization argument (polarengine). [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]

SOLVED modifying config.json to "quant_method": "polarengine",

b) Tokenizer not working:
(APIServer pid=6439) RuntimeError: Failed to load the tokenizer. If the tokenizer is a custom tokenizer not yet available in the HuggingFace transformers library, consider setting trust_remote_code=True in LLM or using the --trust-remote-code flag in the CLI.

SOLVED modifying tokenizer_config.json to "tokenizer_class": "Qwen2TokenizerFast",

c) Wrong parameters in model, or vLLM failing mapping the correct ones:
(EngineCore pid=6553) ERROR 04-06 17:42:39 [core.py:1108] ValueError: There is no module or parameter named 'model' in Qwen3_5ForConditionalGeneration. The available parameters belonging to (Qwen3_5ForConditionalGeneration) are: {'language_model.model.layers.6.input_layernorm.weight', 'language_model.model.layers.16.input_layernorm.weight', 'language_model.model.layers.29.post_attention_layernorm.weight', 'language_model.model.layers.7.self_attn.q_norm.weight', 'language_model.model.layers.17.linear_attn.dt_bias', 'language_model.model.layers.11.self_attn.o_proj.weight', 'language_model.model.layers.1.linear_attn.out_proj.weight', 'language_model.model.layers.7.post_attention_layernorm.weight', 'language_model.model.layers.0.linear_attn.dt_bias', 'language_model.model.layers.19.input_layernorm.weight', 'language_model.model.layers.5.mlp.gate_up_proj.weight', 'language_model.model.layers.28.linear_attn.norm.weight', 'language_model.model.layers.24.linear_attn.A_log', 'language_model.model.layers.30.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.16.linear_attn.dt_bias', 'language_model.model.layers.26.input_layernorm.weight', 'language_model.model.layers.31.mlp.down_proj.weight', 'language_model.model.layers.8.linear_attn.dt_bias', 'language_model.lm_head.weight', 'language_model.model.layers.26.linear_attn.norm.weight', 'language_model.model.layers.12.linear_attn.in_proj_ba.weight', 'language_model.model.layers.25.linear_attn.out_proj.weight', 'language_model.model.layers.4.post_attention_layernorm.weight', 'language_model.model.layers.25.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.1.mlp.gate_up_proj.weight', 'language_model.model.layers.13.input_layernorm.weight', 'language_model.model.layers.4.mlp.down_proj.weight', 'language_model.model.layers.28.mlp.down_proj.weight', 'language_model.model.layers.12.mlp.down_proj.weight', 'language_model.model.layers.12.linear_attn.dt_bias', 'language_model.model.layers.7.mlp.gate_up_proj.weight', 'language_model.model.layers.14.post_attention_layernorm.weight', 'language_model.model.layers.25.linear_attn.A_log', 'language_model.model.layers.2.linear_attn.out_proj.weight', 'language_model.model.layers.15.post_attention_layernorm.weight', 'language_model.model.layers.22.linear_attn.conv1d.weight', 'language_model.model.layers.20.input_layernorm.weight', 'language_model.model.layers.9.mlp.gate_up_proj.weight', 'language_model.model.layers.31.post_attention_layernorm.weight', 'language_model.model.layers.27.post_attention_layernorm.weight', 'language_model.model.layers.28.linear_attn.out_proj.weight', 'language_model.model.layers.1.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.20.post_attention_layernorm.weight', 'language_model.model.layers.22.linear_attn.A_log', 'language_model.model.layers.29.linear_attn.conv1d.weight', 'language_model.model.layers.29.mlp.gate_up_proj.weight', 'language_model.model.layers.6.linear_attn.A_log', 'language_model.model.layers.25.input_layernorm.weight', 'language_model.model.layers.28.linear_attn.A_log', 'language_model.model.layers.18.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.15.self_attn.q_norm.weight', 'language_model.model.layers.2.mlp.down_proj.weight', 'language_model.model.layers.2.linear_attn.A_log', 'language_model.model.layers.10.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.27.self_attn.q_norm.weight', 'language_model.model.layers.2.post_attention_layernorm.weight', 'language_model.model.layers.0.input_layernorm.weight', 'language_model.model.layers.11.self_attn.q_norm.weight', 'language_model.model.layers.12.linear_attn.out_proj.weight', 'language_model.model.layers.8.mlp.gate_up_proj.weight', 'language_model.model.layers.15.mlp.down_proj.weight', 'language_model.model.layers.30.linear_attn.dt_bias', 'language_model.model.layers.22.mlp.down_proj.weight', 'language_model.model.layers.24.linear_attn.in_proj_ba.weight', 'language_model.model.layers.13.linear_attn.A_log', 'language_model.model.layers.24.linear_attn.out_proj.weight', 'language_model.model.layers.9.linear_attn.dt_bias', 'language_model.model.layers.25.linear_attn.in_proj_ba.weight', 'language_model.model.layers.22.linear_attn.dt_bias', 'language_model.model.layers.6.linear_attn.norm.weight', 'language_model.model.layers.0.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.18.linear_attn.out_proj.weight', 'language_model.model.layers.28.linear_attn.dt_bias', 'language_model.model.layers.1.mlp.down_proj.weight', 'language_model.model.layers.15.input_layernorm.weight', 'language_model.model.layers.11.self_attn.k_norm.weight', 'language_model.model.layers.5.linear_attn.norm.weight', 'language_model.model.layers.4.mlp.gate_up_proj.weight', 'language_model.model.layers.17.linear_attn.norm.weight', 'language_model.model.layers.8.input_layernorm.weight', 'language_model.model.layers.16.linear_attn.norm.weight', 'language_model.model.layers.20.linear_attn.conv1d.weight', 'language_model.model.layers.17.linear_attn.in_proj_ba.weight', 'language_model.model.layers.21.post_attention_layernorm.weight', 'language_model.model.layers.25.mlp.down_proj.weight', 'language_model.model.layers.8.linear_attn.A_log', 'language_model.model.layers.30.mlp.down_proj.weight', 'language_model.model.layers.14.linear_attn.norm.weight', 'language_model.model.layers.21.mlp.gate_up_proj.weight', 'language_model.model.layers.22.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.1.linear_attn.in_proj_ba.weight', 'language_model.model.layers.14.mlp.gate_up_proj.weight', 'language_model.model.layers.31.self_attn.k_norm.weight', 'language_model.model.layers.26.linear_attn.A_log', 'language_model.model.layers.5.linear_attn.out_proj.weight', 'language_model.model.layers.17.mlp.gate_up_proj.weight', 'language_model.model.layers.2.linear_attn.norm.weight', 'language_model.model.layers.4.linear_attn.dt_bias', 'language_model.model.layers.14.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.23.input_layernorm.weight', 'language_model.model.layers.24.linear_attn.conv1d.weight', 'language_model.model.layers.12.linear_attn.A_log', 'language_model.model.layers.1.linear_attn.norm.weight', 'language_model.model.layers.20.linear_attn.A_log', 'language_model.model.layers.10.mlp.down_proj.weight', 'language_model.model.layers.5.linear_attn.in_proj_ba.weight', 'language_model.model.layers.2.input_layernorm.weight', 'language_model.model.layers.4.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.14.linear_attn.out_proj.weight', 'language_model.model.layers.29.linear_attn.out_proj.weight', 'language_model.model.layers.9.post_attention_layernorm.weight', 'language_model.model.layers.3.self_attn.qkv_proj.weight', 'language_model.model.layers.17.linear_attn.out_proj.weight', 'language_model.model.layers.24.mlp.down_proj.weight', 'language_model.model.layers.3.post_attention_layernorm.weight', 'language_model.model.layers.5.linear_attn.A_log', 'language_model.model.layers.5.input_layernorm.weight', 'language_model.model.layers.19.mlp.gate_up_proj.weight', 'language_model.model.layers.6.linear_attn.conv1d.weight', 'language_model.model.layers.6.post_attention_layernorm.weight', 'language_model.model.layers.21.linear_attn.A_log', 'language_model.model.layers.18.linear_attn.dt_bias', 'language_model.model.layers.1.linear_attn.dt_bias', 'language_model.model.layers.0.linear_attn.out_proj.weight', 'language_model.model.layers.22.post_attention_layernorm.weight', 'language_model.model.layers.28.post_attention_layernorm.weight', 'language_model.model.layers.19.mlp.down_proj.weight', 'language_model.model.layers.8.post_attention_layernorm.weight', 'language_model.model.layers.25.linear_attn.conv1d.weight', 'language_model.model.layers.31.self_attn.o_proj.weight', 'language_model.model.layers.4.input_layernorm.weight', 'language_model.model.layers.1.linear_attn.conv1d.weight', 'language_model.model.layers.8.linear_attn.in_proj_ba.weight', 'language_model.model.layers.6.linear_attn.in_proj_ba.weight', 'language_model.model.layers.18.post_attention_layernorm.weight', 'language_model.model.layers.0.linear_attn.norm.weight', 'language_model.model.layers.2.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.10.linear_attn.norm.weight', 'language_model.model.layers.18.linear_attn.norm.weight', 'language_model.model.layers.20.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.23.self_attn.k_norm.weight', 'language_model.model.layers.22.linear_attn.in_proj_ba.weight', 'language_model.model.layers.8.linear_attn.out_proj.weight', 'language_model.model.layers.4.linear_attn.norm.weight', 'language_model.model.layers.4.linear_attn.out_proj.weight', 'language_model.model.layers.0.linear_attn.in_proj_ba.weight', 'language_model.model.layers.9.linear_attn.in_proj_ba.weight', 'language_model.model.layers.21.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.3.input_layernorm.weight', 'language_model.model.layers.27.input_layernorm.weight', 'language_model.model.layers.29.linear_attn.A_log', 'language_model.model.layers.29.input_layernorm.weight', 'language_model.model.layers.12.mlp.gate_up_proj.weight', 'language_model.model.layers.12.post_attention_layernorm.weight', 'language_model.model.layers.23.self_attn.qkv_proj.weight', 'language_model.model.layers.26.linear_attn.out_proj.weight', 'language_model.model.layers.13.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.31.mlp.gate_up_proj.weight', 'language_model.model.layers.18.mlp.down_proj.weight', 'language_model.model.layers.4.linear_attn.conv1d.weight', 'language_model.model.layers.17.mlp.down_proj.weight', 'language_model.model.layers.0.linear_attn.conv1d.weight', 'language_model.model.layers.16.mlp.down_proj.weight', 'language_model.model.layers.16.linear_attn.in_proj_ba.weight', 'language_model.model.layers.6.linear_attn.out_proj.weight', 'language_model.model.layers.27.self_attn.k_norm.weight', 'language_model.model.layers.7.mlp.down_proj.weight', 'language_model.model.layers.25.post_attention_layernorm.weight', 'language_model.model.layers.19.self_attn.o_proj.weight', 'language_model.model.layers.9.linear_attn.conv1d.weight', 'language_model.model.layers.27.self_attn.qkv_proj.weight', 'language_model.model.layers.27.self_attn.o_proj.weight', 'language_model.model.layers.26.mlp.down_proj.weight', 'language_model.model.layers.14.input_layernorm.weight', 'language_model.model.layers.29.linear_attn.norm.weight', 'language_model.model.layers.25.mlp.gate_up_proj.weight', 'language_model.model.layers.10.linear_attn.in_proj_ba.weight', 'language_model.model.layers.1.input_layernorm.weight', 'language_model.model.layers.24.mlp.gate_up_proj.weight', 'language_model.model.layers.13.post_attention_layernorm.weight', 'language_model.model.layers.20.mlp.down_proj.weight', 'language_model.model.layers.3.mlp.down_proj.weight', 'language_model.model.layers.16.linear_attn.A_log', 'language_model.model.layers.5.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.27.mlp.gate_up_proj.weight', 'language_model.model.layers.7.self_attn.k_norm.weight', 'language_model.model.layers.28.linear_attn.in_proj_ba.weight', 'language_model.model.layers.16.post_attention_layernorm.weight', 'language_model.model.layers.21.linear_attn.in_proj_ba.weight', 'language_model.model.layers.6.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.24.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.11.post_attention_layernorm.weight', 'language_model.model.layers.21.input_layernorm.weight', 'language_model.model.layers.24.post_attention_layernorm.weight', 'language_model.model.layers.4.linear_attn.in_proj_ba.weight', 'language_model.model.norm.weight', 'language_model.model.layers.10.input_layernorm.weight', 'language_model.model.layers.12.linear_attn.conv1d.weight', 'language_model.model.layers.1.post_attention_layernorm.weight', 'language_model.model.layers.30.linear_attn.conv1d.weight', 'language_model.model.layers.7.input_layernorm.weight', 'language_model.model.layers.13.linear_attn.out_proj.weight', 'language_model.model.layers.26.linear_attn.conv1d.weight', 'language_model.model.layers.30.input_layernorm.weight', 'language_model.model.layers.17.input_layernorm.weight', 'language_model.model.layers.14.linear_attn.A_log', 'language_model.model.layers.13.linear_attn.in_proj_ba.weight', 'language_model.model.layers.10.linear_attn.conv1d.weight', 'language_model.model.layers.14.linear_attn.conv1d.weight', 'language_model.model.layers.3.self_attn.o_proj.weight', 'language_model.model.layers.20.linear_attn.out_proj.weight', 'language_model.model.layers.31.input_layernorm.weight', 'language_model.model.layers.18.linear_attn.in_proj_ba.weight', 'language_model.model.layers.21.linear_attn.out_proj.weight', 'language_model.model.layers.30.post_attention_layernorm.weight', 'language_model.model.layers.16.mlp.gate_up_proj.weight', 'language_model.model.layers.1.linear_attn.A_log', 'language_model.model.layers.2.linear_attn.dt_bias', 'language_model.model.layers.11.mlp.gate_up_proj.weight', 'language_model.model.layers.14.linear_attn.dt_bias', 'language_model.model.layers.21.mlp.down_proj.weight', 'language_model.model.layers.12.input_layernorm.weight', 'language_model.model.layers.3.self_attn.k_norm.weight', 'language_model.model.layers.10.linear_attn.A_log', 'language_model.model.layers.18.linear_attn.A_log', 'language_model.model.layers.10.post_attention_layernorm.weight', 'language_model.model.layers.17.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.30.linear_attn.in_proj_ba.weight', 'language_model.model.layers.8.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.3.self_attn.q_norm.weight', 'language_model.model.layers.16.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.26.linear_attn.dt_bias', 'language_model.model.layers.8.linear_attn.conv1d.weight', 'language_model.model.layers.23.mlp.down_proj.weight', 'language_model.model.layers.7.self_attn.o_proj.weight', 'language_model.model.layers.12.linear_attn.norm.weight', 'language_model.model.layers.19.post_attention_layernorm.weight', 'language_model.model.layers.19.self_attn.k_norm.weight', 'language_model.model.layers.5.post_attention_layernorm.weight', 'language_model.model.layers.0.post_attention_layernorm.weight', 'language_model.model.layers.13.linear_attn.dt_bias', 'language_model.model.layers.31.self_attn.q_norm.weight', 'language_model.model.layers.14.mlp.down_proj.weight', 'language_model.model.layers.21.linear_attn.dt_bias', 'language_model.model.layers.19.self_attn.qkv_proj.weight', 'language_model.model.layers.0.mlp.down_proj.weight', 'language_model.model.layers.28.input_layernorm.weight', 'language_model.model.layers.24.linear_attn.norm.weight', 'language_model.model.layers.8.linear_attn.norm.weight', 'language_model.model.layers.23.self_attn.o_proj.weight', 'language_model.model.layers.29.linear_attn.dt_bias', 'language_model.model.layers.30.mlp.gate_up_proj.weight', 'language_model.model.layers.17.linear_attn.conv1d.weight', 'language_model.model.layers.29.linear_attn.in_proj_ba.weight', 'language_model.model.layers.30.linear_attn.A_log', 'language_model.model.layers.0.linear_attn.A_log', 'language_model.model.layers.6.mlp.down_proj.weight', 'language_model.model.layers.26.mlp.gate_up_proj.weight', 'language_model.model.layers.22.mlp.gate_up_proj.weight', 'language_model.model.layers.26.post_attention_layernorm.weight', 'language_model.model.layers.5.mlp.down_proj.weight', 'language_model.model.layers.26.linear_attn.in_proj_qkvz.weight', 'language_model.model.embed_tokens.weight', 'language_model.model.layers.17.linear_attn.A_log', 'language_model.model.layers.18.linear_attn.conv1d.weight', 'language_model.model.layers.16.linear_attn.out_proj.weight', 'language_model.model.layers.25.linear_attn.dt_bias', 'language_model.model.layers.24.linear_attn.dt_bias', 'language_model.model.layers.15.self_attn.o_proj.weight', 'language_model.model.layers.0.mlp.gate_up_proj.weight', 'language_model.model.layers.22.input_layernorm.weight', 'language_model.model.layers.5.linear_attn.dt_bias', 'language_model.model.layers.25.linear_attn.norm.weight', 'language_model.model.layers.13.mlp.down_proj.weight', 'language_model.model.layers.18.input_layernorm.weight', 'language_model.model.layers.8.mlp.down_proj.weight', 'language_model.model.layers.15.self_attn.k_norm.weight', 'language_model.model.layers.21.linear_attn.norm.weight', 'language_model.model.layers.9.mlp.down_proj.weight', 'language_model.model.layers.11.mlp.down_proj.weight', 'language_model.model.layers.6.linear_attn.dt_bias', 'language_model.model.layers.24.input_layernorm.weight', 'language_model.model.layers.5.linear_attn.conv1d.weight', 'language_model.model.layers.19.self_attn.q_norm.weight', 'language_model.model.layers.30.linear_attn.out_proj.weight', 'language_model.model.layers.10.mlp.gate_up_proj.weight', 'language_model.model.layers.16.linear_attn.conv1d.weight', 'language_model.model.layers.14.linear_attn.in_proj_ba.weight', 'language_model.model.layers.13.linear_attn.conv1d.weight', 'language_model.model.layers.9.input_layernorm.weight', 'language_model.model.layers.15.mlp.gate_up_proj.weight', 'language_model.model.layers.20.mlp.gate_up_proj.weight', 'language_model.model.layers.20.linear_attn.dt_bias', 'language_model.model.layers.15.self_attn.qkv_proj.weight', 'language_model.model.layers.29.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.31.self_attn.qkv_proj.weight', 'language_model.model.layers.12.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.27.mlp.down_proj.weight', 'language_model.model.layers.28.linear_attn.conv1d.weight', 'language_model.model.layers.28.mlp.gate_up_proj.weight', 'language_model.model.layers.4.linear_attn.A_log', 'language_model.model.layers.10.linear_attn.dt_bias', 'language_model.model.layers.21.linear_attn.conv1d.weight', 'language_model.model.layers.9.linear_attn.norm.weight', 'language_model.model.layers.9.linear_attn.out_proj.weight', 'language_model.model.layers.23.post_attention_layernorm.weight', 'language_model.model.layers.2.linear_attn.conv1d.weight', 'language_model.model.layers.9.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.22.linear_attn.norm.weight', 'language_model.model.layers.23.self_attn.q_norm.weight', 'language_model.model.layers.13.linear_attn.norm.weight', 'language_model.model.layers.9.linear_attn.A_log', 'language_model.model.layers.2.linear_attn.in_proj_ba.weight', 'language_model.model.layers.20.linear_attn.in_proj_ba.weight', 'language_model.model.layers.13.mlp.gate_up_proj.weight', 'language_model.model.layers.29.mlp.down_proj.weight', 'language_model.model.layers.3.mlp.gate_up_proj.weight', 'language_model.model.layers.23.mlp.gate_up_proj.weight', 'language_model.model.layers.22.linear_attn.out_proj.weight', 'language_model.model.layers.2.mlp.gate_up_proj.weight', 'language_model.model.layers.18.mlp.gate_up_proj.weight', 'language_model.model.layers.28.linear_attn.in_proj_qkvz.weight', 'language_model.model.layers.11.input_layernorm.weight', 'language_model.model.layers.26.linear_attn.in_proj_ba.weight', 'language_model.model.layers.10.linear_attn.out_proj.weight', 'language_model.model.layers.20.linear_attn.norm.weight', 'language_model.model.layers.17.post_attention_layernorm.weight', 'language_model.model.layers.30.linear_attn.norm.weight', 'language_model.model.layers.6.mlp.gate_up_proj.weight', 'language_model.model.layers.7.self_attn.qkv_proj.weight', 'language_model.model.layers.11.self_attn.qkv_proj.weight'}

UNSOLVED, or at least i'm not able to find a fix for this one. Tried to upgrade to transformers=>5.0 but nothing changed.

Could you take a look and help me solving this?
Thank you!
caiovicentino1
 Owner

Hi @Arien0 , thank you for the detailed report — excellent debugging on (a) and (b)!

Issue (a) — quant_method: Fixed. We've pushed "quant_method": "polarengine" to config.json just now.

Issue (b) — tokenizer: Fixed. We've pushed "tokenizer_class": "Qwen2TokenizerFast" to tokenizer_config.json.

Issue (c) — weight key prefix mismatch: This is a real bug on our side, and we're fixing it right now. Here's what happened:

Qwen3.5 uses Qwen3_5ForConditionalGeneration (multimodal architecture), where the text model lives under a language_model wrapper. The base model (Jackrong/Qwopus3.5-9B-v3) has weight keys like:

model.language_model.layers.0.mlp.gate_proj.weight

But our quantization notebook loaded the model with AutoModelForCausalLM, which resolves to the text-only part and strips the language_model. segment. So our PQ5 codes have keys like:

model.layers.0.mlp.gate_proj.codes

When vLLM creates Qwen3_5ForConditionalGeneration, it expects parameters at language_model.model.layers.X... — hence the error.

Fix in progress: We're re-uploading the safetensors with corrected key names (model.X → model.language_model.X). This should be done within the hour.

In the meantime, if you want to try immediately after the upload, the full set of fixes is:

  1. quant_method: "polarengine" in config.json (already pushed)
  2. tokenizer_class: "Qwen2TokenizerFast" in tokenizer_config.json (already pushed)
  3. Re-download the model to get the renamed safetensors (uploading now)

We're also updating our quantization pipeline to handle multimodal architectures correctly going forward.

Thanks for the feedback — it helps us improve the deployment experience for everyone!
caiovicentino1
 Owner

Update: All three fixes are now live! 🎉

  • ✅ config.json — quant_method: "polarengine"
  • ✅ tokenizer_config.json — tokenizer_class: "Qwen2TokenizerFast"
  • ✅ Both safetensors shards — keys renamed from model.X to model.language_model.X (924 tensors)

You can re-download the model and try again with vLLM. Let me know if it works!
Arien0

Just tried and different errors found:

(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter embed_tokens.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter embed_tokens.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter embed_tokens.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.0.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.1.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.10.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.qkqkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.qkqkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.qkqkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.o_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.o_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.o_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.qkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.qkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.11.self_attn.qkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.12.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.13.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.14.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.qkqkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.qkqkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.qkqkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.o_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.o_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.o_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.qkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.qkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.15.self_attn.qkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.16.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.17.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.18.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.qkqkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.qkqkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.qkqkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.o_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.o_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.o_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.qkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.qkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.19.self_attn.qkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.2.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.qkqkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.qkqkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.qkqkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.o_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.o_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.o_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.qkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.qkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.3.self_attn.qkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.4.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.5.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.6.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.qkqkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.qkqkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.qkqkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.o_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.o_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.o_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.qkv_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.qkv_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.7.self_attn.qkv_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.8.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.in_proj_ba.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.in_proj_ba.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.in_proj_ba.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.in_proj_qkvz.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.in_proj_qkvz.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.in_proj_qkvz.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.out_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.out_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.linear_attn.out_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.down_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.down_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.down_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.gate_gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.gate_gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.gate_gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.gate_up_proj.codes not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.gate_up_proj.ct not found in params_dict, skip loading
(EngineCore pid=12918) WARNING 04-06 20:58:33 [qwen3_5.py:432] Parameter layers.9.mlp.gate_up_proj.norms not found in params_dict, skip loading
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] EngineCore failed to start.
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] Traceback (most recent call last):
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 1082, in run_engine_core
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return func(*args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 848, in init
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] super().init(
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 114, in init
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] self.model_executor = executor_class(vllm_config)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return func(*args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/executor/abstract.py", line 103, in init
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] self._init_executor()
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/executor/uniproc_executor.py", line 52, in _init_executor
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] self.driver_worker.load_model()
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return func(*args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4751, in load_model
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] self.model = model_loader.load_model(
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/polarengine_vllm/init.py", line 82, in _patched_load
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return _original_load(self, *args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return func(*args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 64, in load_model
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] self.load_weights(model, model_config)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return func(*args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 381, in load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] loaded_weights = model.load_weights(self.get_all_weights(model_config, model))
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_5.py", line 705, in load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return loader.load_weights(weights, mapper=self.hf_to_vllm_mapper)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/model_loader/reload/torchao_decorator.py", line 50, in patched_model_load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return original_load_weights(self, weights, *args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 355, in load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] autoloaded_weights = set(self._load_module("", self.module, weights))
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 302, in _load_module
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] yield from self._load_module(
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 275, in _load_module
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] loaded_params = module_load_weights(weights)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_5.py", line 546, in load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return loader.load_weights(weights)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/model_loader/reload/torchao_decorator.py", line 50, in patched_model_load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] return original_load_weights(self, weights, *args, **kwargs)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 355, in load_weights
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] autoloaded_weights = set(self._load_module("", self.module, weights))
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 302, in _load_module
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] yield from self._load_module(
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] File "/opt/llm/vllm/venv/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 339, in _load_module
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] raise ValueError(msg)
(EngineCore pid=12918) ERROR 04-06 20:58:33 [core.py:1108] ValueError: There is no module or parameter named 'lm_head.codes' in Qwen3_5ForCausalLM. The available parameters belonging to lm_head (ParallelLMHead) are: {'lm_head.weight'}

More debuggin fun ahead!!!
caiovicentino1
 Owner

@Arien0 — thanks for continuing to test this. You're hitting the core limitation: vLLM doesn't natively understand PQ5 codes (.codes/.norms/.ct). Our polarengine plugin is supposed to intercept the weight loading and dequant the codes → standard .weight tensors on-the-fly, but it's clearly not working correctly in your setup.

The honest situation:

  • Our PQ5 format stores weights as quantized codes (not standard .weight tensors)
  • This requires our dequant plugin to work — without it, vLLM just sees unknown tensors
  • The plugin integration with vLLM 0.19.0 needs more work

What works TODAY for this model:

Option 1 — HuggingFace transformers (tested, works):

pip install polarquant

import polarengine_vllm  # auto-registers dequant with transformers
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "caiovicentino1/Qwopus3.5-9B-v3-PolarQuant-Q5",
    device_map="auto", trust_remote_code=True
)

Option 2 — Manual dequant → load in vLLM as standard model:

from polarengine_vllm import PolarQuantModel
model = PolarQuantModel.from_pretrained("caiovicentino1/Qwopus3.5-9B-v3-PolarQuant-Q5")
# Save as standard safetensors
model.model.save_pretrained("/path/to/dequanted")
# Then load in vLLM normally (no quantization flag needed)

What we're working on:

  • Fixing the vLLM plugin to properly intercept weight loading in 0.19.0
  • Longer term: native vLLM support via CompressedTensors format

Sorry for the runaround — the config fixes were real issues but the deeper vLLM integration needs more work. The HF transformers path works reliably today.
caiovicentino1
 Owner

@Arien0 — update: the model now loads natively in vLLM! No plugin needed.

We migrated from our custom PQ5 format to CompressedTensors (vLLM's native quantization format). The model now uses Marlin kernel directly.

How to use (one command):

vllm serve caiovicentino1/Qwopus3.5-9B-v3-PolarQuant-Q5 --language-model-only --max-model-len 4096 --trust-remote-code

Results on A100 80GB:

  • 168 tok/s with Marlin kernel
  • Zero plugin, zero pip install, zero config hacks

The --language-model-only flag is needed because Qwen3.5 is a multimodal architecture — this tells vLLM to skip the vision encoder (we only quantized text weights).

Thank you for reporting the original issue — it pushed us to adopt the industry-standard format. The experience is now what it should be: one command, native loading, maximum speed.
Arien0
•
edited 8 days ago

Is this with vLLM 0.19.0 from pip or do I have to install with git from source? Can't wait to get home and try this piece of art!

EDIT: just tested in plain vLLM 0.19.0 and wow! (1xRTX3090=113t/s):

(APIServer pid=3897) INFO 04-07 16:59:15 [loggers.py:259] Engine 000: Avg prompt throughput: 29.3 tokens/s, Avg generation throughput: 34.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 16:59:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.6 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 16:59:35 [loggers.py:259] Engine 000: Avg prompt throughput: 36.9 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 16:59:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 113.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.9%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 16:59:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.1%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:00:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 111.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:00:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 25.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:00:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:07:15 [loggers.py:259] Engine 000: Avg prompt throughput: 542.1 tokens/s, Avg generation throughput: 34.6 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.7%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:07:25 [loggers.py:259] Engine 000: Avg prompt throughput: 409.8 tokens/s, Avg generation throughput: 90.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:07:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 110.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.7%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:07:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 109.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:07:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 109.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.2%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:08:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 64.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:08:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:08:25 [loggers.py:259] Engine 000: Avg prompt throughput: 928.3 tokens/s, Avg generation throughput: 2.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 0.0%

(APIServer pid=3897) INFO 04-07 17:08:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 105.6 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%

(APIServer pid=3897) INFO 04-07 17:08:45 [loggers.py:259] Engine 000: Avg prompt throughput: 649.0 tokens/s, Avg generation throughput: 82.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:08:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 89.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:09:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:09:25 [loggers.py:259] Engine 000: Avg prompt throughput: 1042.6 tokens/s, Avg generation throughput: 59.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.0%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:09:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 107.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.2%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:09:45 [loggers.py:259] Engine 000: Avg prompt throughput: 680.5 tokens/s, Avg generation throughput: 81.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.2%, Prefix cache hit rate: 0.0%
(APIServer pid=3897) INFO 04-07 17:09:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 78.7 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
caiovicentino1
 Owner

@Arien0 — yes, plain pip install vllm (0.19.0) works! No git source needed. The model uses CompressedTensors format which is native in vLLM.

113 tok/s on RTX 3090 — that's great! Marlin kernel doing its job.

Quick reference:

pip install vllm
vllm serve caiovicentino1/Qwopus3.5-9B-v3-PolarQuant-Q5 --language-model-only --enforce-eager

No plugins, no custom code, no pip install polarquant. Just vLLM.

Sign up or log in to comment