Phai-A1-35B-1M

English | ็ฎ€ไฝ“ไธญๆ–‡

Phai-A1-35B-1M is a model built upon Qwen3.6-35B-A3B, extensively post-trained and optimized for Coding & Agent workloads. The entire post-training pipeline, including Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), was conducted on Huawei Ascend NPUs.

The model delivers substantial improvements in code generation, tool calling, long-horizon task planning, and execution across agentic workflows. It also natively supports a context window of up to 1 million (1M) tokens.

Model Highlights

  • Large-Scale Post-Training for Coding & Agent Tasks: Extensive post-training covering core agentic capabilities, including code understanding, generation, debugging, tool calling, multi-step reasoning, and long-horizon planning.
  • Reinforcement Learning with Real Sandbox Environments: The RL stage employs Group Relative Policy Optimization (GRPO) across 4,000 sandbox environments, providing execution-based feedback from code execution, unit testing, and tool interactions. This significantly improves reliability and task success rates in real-world coding and tool-use scenarios.
  • Ultra-Long Context Support: Training data spans context lengths from 16K to 1M tokens. Throughout post-training, YaRN (factor=4) is consistently enabled to extend and stabilize the 1M-token context window, enabling repository-scale code understanding and long-horizon agent interactions.

Training Details

Stage Description
Base Model Qwen3.6-35B-A3B (Mixture-of-Experts)
SFT High-quality Coding & Agent instruction data*, covering multi-turn conversations, tool calling, long-chain reasoning, and related scenarios
RL Primarily based on GRPO, leveraging 4,000 sandbox environments to provide real-world feedback through code execution, unit testing, and tool interactions

* Training data covers sequence lengths ranging from 16K to 1M tokens. YaRN with a scaling factor of 4 is enabled throughout post-training to extend the context window to 1M tokens while maintaining training stability.

Quick Start

from llama_cpp import Llama

model_path = "<your GGUF model path>"

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_gpu_layers=-1,
    verbose=False
)

messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "ๅธฎๆˆ‘ๅฎž็Žฐไธ€ไธชๆ”ฏๆŒๅนถๅ‘้™ๆต็š„ๅผ‚ๆญฅไปปๅŠก้˜Ÿๅˆ—"},
]

output = llm.create_chat_completion(
    messages=messages,
    max_tokens=2048,
    temperature=0.7,
)

print(output["choices"][0]["message"]["content"])

Long-Context Inference Recommendations

  • Built-in YaRN Configuration: The model has undergone additional training with YaRN (factor=4) to adapt to extended context lengths. RoPE scaling is already enabled in config.json by default, supporting context windows of up to 1M tokens without requiring additional configuration.
  • Efficient Long-Context Inference: Ultra-long contexts impose significant memory and KV-cache overhead. We recommend using inference frameworks with long-context support, such as vLLM or SGLang, and enabling optimizations such as prefix caching and chunked prefill when appropriate.
  • Short-Context Compatibility: After additional long-context training, the model largely preserves the inference behavior of the base model on short-context tasks. No special handling is required for standard context lengths.

Evaluation

We evaluate Phai-A1-35B-1M on a comprehensive suite of Coding & Agent benchmarks, including SWE-Bench (Verified / Pro / Multilingual), TAU3-Bench, IFBench, and AA-LCR.

The results demonstrate consistent improvements over the base model of comparable size, with performance approaching or surpassing that of significantly larger models on several benchmarks.

Benchmark results

Note: "โ€”" indicates that benchmark results were not reported for the corresponding model.

Limitations and Usage Guidelines

  • Domain Specialization: The model is primarily optimized for Coding & Agent workloads. Its performance on other general-purpose tasks may differ from that of the base model.
  • Output Reliability: Model-generated outputs may contain errors, inaccuracies, or contextually inappropriate content. Thorough evaluation and human verification are recommended before production deployment.
  • Responsible Use: Users are responsible for complying with applicable laws, regulations, and the licensing terms of the underlying base model.

License

Phai-A1-35B-1M is post-trained from Qwen3.6-35B-A3B. Users must comply with the license terms and conditions of the underlying base model when using or distributing the model weights.

Downloads last month
151
GGUF
Model size
35B params
Architecture
qwen35moe
Hardware compatibility
Log In to add your hardware

3-bit

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for PhaiModelLab/Phai-A1-35B-1M-GGUF

Quantized
(1,014)
this model