Hugging Face's logo

Qwen
/
Qwen3.5-397B-A17B

Image-Text-to-Text
Transformers
Safetensors
qwen3_5_moe
conversational
Eval Results
Model card Files
xet
 Community
74

Add evaluation results for HLE, MMLU-Pro

#9
by SaylorTwift HF Staff - opened
base: refs/heads/main
←
from: refs/pr/9
Discussion Files changed
+55
-0

YAML Metadata Error:Invalid content in Eval Result file .eval_results/hle.yaml

Check out the documentation for more information.

Show details
Task ID "hle" does not match any task in dataset "cais/hle". Available: none

YAML Metadata Error:Invalid content in Eval Result file .eval_results/hle_with_tools.yaml

Check out the documentation for more information.

Show details
Task ID "hle" does not match any task in dataset "cais/hle". Available: none

YAML Metadata Error:Invalid content in Eval Result file .eval_results/mmlu_pro.yaml

Check out the documentation for more information.

Show details
Task ID "mmlu_pro" does not match any task in dataset "TIGER-Lab/MMLU-Pro". Available: none
Files changed (6) hide show
  1. .eval_results/gpqa_diamond.yaml +9 -0
  2. .eval_results/hle.yaml +9 -0
  3. .eval_results/hle_with_tools.yaml +10 -0
  4. .eval_results/mmlu_pro.yaml +9 -0
  5. .eval_results/swe_bench_verified.yaml +9 -0
  6. .eval_results/terminalbench_2.yaml +9 -0
.eval_results/gpqa_diamond.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: Idavidrein/gpqa
3
+ task_id: diamond
4
+ value: 88.4
5
+ date: '2026-02-16'
6
+ source:
7
+ url: https://huggingface.co/Qwen/Qwen3.5-397B-A17B
8
+ name: Model Card
9
+ user: SaylorTwift
.eval_results/hle.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: cais/hle
3
+ task_id: hle
4
+ value: 28.7
5
+ date: '2026-02-16'
6
+ source:
7
+ url: https://huggingface.co/Qwen/Qwen3.5-397B-A17B
8
+ name: Model Card
9
+ user: SaylorTwift
.eval_results/hle_with_tools.yaml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: cais/hle
3
+ task_id: hle
4
+ value: 48.3
5
+ date: '2026-02-16'
6
+ source:
7
+ url: https://huggingface.co/Qwen/Qwen3.5-397B-A17B
8
+ name: Model Card
9
+ user: SaylorTwift
10
+ notes: With tools
.eval_results/mmlu_pro.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: TIGER-Lab/MMLU-Pro
3
+ task_id: mmlu_pro
4
+ value: 87.8
5
+ date: '2026-02-16'
6
+ source:
7
+ url: https://huggingface.co/Qwen/Qwen3.5-397B-A17B
8
+ name: Model Card
9
+ user: SaylorTwift
.eval_results/swe_bench_verified.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: SWE-bench/SWE-bench_Verified
3
+ task_id: swe_bench_%_resolved
4
+ value: 76.4
5
+ date: '2026-02-03'
6
+ source:
7
+ url: https://huggingface.co/Qwen/Qwen3.5-397B-A17B
8
+ name: Model Card
9
+ user: SaylorTwift
.eval_results/terminalbench_2.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: harborframework/terminal-bench-2.0
3
+ task_id: terminalbench_2
4
+ value: 52.5
5
+ date: '2026-02-03'
6
+ source:
7
+ url: https://huggingface.co/Qwen/Qwen3.5-397B-A17B
8
+ name: Model Card
9
+ user: SaylorTwift