Aijuus/sage-a1.5-fp8-df2

Sage-A1.5 · FP8 · DFlash2 · for radiance

InternScience/Sage-A1.5 as 128x128 block FP8 with incoai/Qwen3.6-35B-A3B-DFlash2 merged in place of the model's MTP head, packed into one .rad container for radiance.

Sage's text_config is Qwen3.6-35B-A3B's key for key, so the drafter's geometry matches the trunk it was trained against: six Qwen3-shaped layers at 2048, eight taps (target_layer_ids 1 6 11 16 22 27 32 37), block 8, selector rank 256 / top 16. It proposes seven tokens in one block pass; the MTP head is not carried.

container	sage-a1.5-fp8-df2.rad, 34.45 GiB
plugin	qwen35moe_fp8 (architecture qwen35moe, fp8_e4m3)
weights	21,417: the block-FP8 trunk and vision tower, the drafter's linears at block FP8, its 2-bit draft head
drafter	DFlash2, depth 7, one block pass a step
context	262,144 tokens
licence	Apache-2.0, as the base model

Built from the published FP8 release and the drafter repository with sage-a1.5-fp8-df2.recipe, included here:

rad-convert InternScience/Sage-A1.5-FP8 \
    --draft-model incoai/Qwen3.6-35B-A3B-DFlash2 \
    --recipe sage-a1.5-fp8-df2.recipe \
    -o sage-a1.5-fp8-df2.rad

Serve

hf download Aijuus/sage-a1.5-fp8-df2 sage-a1.5-fp8-df2.rad --local-dir ~/models
docker run -d --init --name sage --network host --group-add video --group-add render \
    --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
    -v ~/models:/models:ro -e ROCR_VISIBLE_DEVICES=0,1 \
    stilldeadcode/radiance:1.9.0 --model /models/sage-a1.5-fp8-df2.rad --host 0.0.0.0 --port 8000 \
    --tp 2 --kv-cache-dtype fp8 --max-model-len 32768 --max-num-seqs 4 \
    --max-num-batched-tokens 2048 --num-speculative-tokens auto \
    --temp 1.0 --top-p 0.95 --top-k 20 --presence-penalty 1.5

That is four 32,768-token sessions with the whole expert plane in VRAM; for one full-context session use --max-num-seqs 1 --max-model-len 262144.

auto takes the depth the drafter states (7). At --tp 1 the experts do not fit and the profile needs --placement expert_tiered --host-pool-mib 8192 --max-num-seqs 1 --max-model-len 131072; sage-a1.5-df2.yaml in the radiance repository carries the tp2 profile. Sampling is the model card's; enable_thinking and verification are per-request chat_template_kwargs.

Measured (2x R9700, warm, 3 runs, median, 256 tokens)

	MTP head	DFlash2
tp2 decode, mean of 8 categories	281.6 tok/s	392.9 tok/s (1.40x)
tp2 decode range	264.6-293.0	329.1-477.7
tp1 decode, mean of 8 categories	168.3 tok/s	231.6 tok/s (1.38x)
tp1 decode range	143.3-188.2	201.8-278.7

Acceptance at depth 7: 49.3% of proposed tokens at tp2 (position 0: 81.4%) and 47.4% at tp1. Prefill is unchanged (11.6k / 11.3k / 11.0k / 10.7k tok/s at 1.6k / 6.5k / 13.1k / 26.1k prompt tokens, tp2). Replies are verified by the trunk, so the drafter changes speed only. Chat, tool calls and image prompts were exercised on both profiles.

BetterBench

A full BetterBench 0.6.0 run (corpus v1.0, 20 passes a category, all three phases, 2x R9700, --max-model-len 65536), with thinking off because the model's template enables it by default and every corpus run would otherwise spend its whole token budget thinking:

single-stream decode, weighted median	365.9 tok/s
TTFT p50 / update p99	31 ms / 14 ms
concurrency 1 / 2 / 4 / 8, aggregate	305 / 434 / 575 / 734 tok/s
prefill, 1.5k / 5.9k / 11.8k / 23.6k / 47k prompt tokens	11.7k / 11.2k / 11.2k / 10.5k / 9.3k tok/s

Per-category decode ranges from 191.5 tok/s (prose) to 529.0 (json). The full report is in benchmarks/betterbench.md and the raw run in benchmarks/betterbench-results.json.