shubhambaid/LFM2.5-VL-3B-MLX-mixed-v6p6b4e4

LFM2.5-VL-3B — MLX mixed-precision (v6/p6/b4/e4)

A component-wise mixed-precision MLX quantization of LiquidAI/LFM2.5-VL-3B: 2.04 GB at 5.19 bits/weight, matching the standard 4-bit MLX quant on all five benchmarks tested while being 14.4% smaller.

As far as I can tell this is the only published LFM2.5-VL quant with the vision tower quantized at all — a survey of 56 quant repos, including LiquidAI's own MLX releases, found none with a vision_tower entry in their quantization config. They all inherit mlx-vlm's default, which skips the vision path entirely.

What the numbers say

model	bits/weight	size	mean	DocVQA	ChartQA	OCRBench	ScreenSpot-v2	RefCOCO
bf16	16.00	6.25 GB	81.5	85.3	86.0	80.3	71.0	85.0
stock MLX 4-bit	6.07	2.39 GB	81.0	85.8	85.7	80.7	68.0	85.0
this model	5.19	2.04 GB	81.3	86.1	86.3	80.3	68.3	85.3
uniform 4-bit	5.00	1.97 GB	79.3	84.7	83.7	78.0	67.3	83.0

n=300 per benchmark, 512-token vision budget, seed 0. 95% CIs are ±4.0 to ±5.3 points.

This is an equal-accuracy claim, not a better-accuracy claim. Every difference against the stock 4-bit quant is under 0.7 points against combined CIs of ~5.5 — well inside noise. The claim is same quality, less memory, and the intervals are wide enough that "same" is a bounded statement rather than a tight one.

Reproduce: lfm2vl-lab · raw results: lfm2vl-lab-results

The recipe

component	bits	params	note
vision tower (SigLIP2 NaFlex)	6	412M	67.2% coverage — see below
multimodal projector	6	13.6M
LFM2.5 backbone	4	2.43B	carries essentially all the memory
embeddings (tied)	4	262M

Important: vision coverage is 67.2%, not 100%

MLX silently skips any module whose input dimension isn't divisible by the group size. LFM2.5-VL's SigLIP2 shape-optimized tower has intermediate_size = 4304 (= 16 × 269, and 269 is prime), so all 27 vision_tower.encoder.layers.*.mlp.fc2 modules are unquantizable at group size 32, 64 and 128 alike.

So "vision at 6 bits" here means 67.2% of vision parameters at 6 bits and 32.8% still at bf16 — 133.9M params. This isn't a defect in this model; it's a property of the architecture that any quantization of this tower inherits. It is stated explicitly because a coverage number that isn't 100% should never be implied to be.

Two modules are deliberately left at full precision

vision_tower.embeddings.position_embedding and .patch_embedding are excluded unconditionally. Both have their .weight read as a raw tensor rather than invoked through __call__:

• position_embedding — reshaped directly, so a quantized (packed uint32) weight yields the wrong dimension and generation dies with a broadcast error.
• patch_embedding — its .weight.dtype is used to cast pixel values. Quantized, that dtype is uint32, so every normalized pixel truncates to 0 or 1. This one raises nothing. The model loads, reports 100% coverage, and answers from its text prior with the image destroyed.

Usage

pip install mlx-vlm
python -m mlx_vlm generate \
  --model shubhambaid/LFM2.5-VL-3B-MLX-mixed-v6p6b4e4 \
  --image your_image.png \
  --prompt "Read all the text in this image." \
  --max-tokens 128

Set the vision token budget deliberately

The shipped default is max_image_tokens=256. On this model that is not a good default for screens or documents, and the effect dwarfs anything quantization does:

max_image_tokens	ScreenSpot-v2	DocVQA	RefCOCO	s/sample
128	57.0	55.4	83.0	1.37
256 (default)	53.0	80.3	89.0	1.78
512	79.0	86.6	86.0	2.67
1024	70.0	81.3	89.0	4.42

n=100. It is non-monotonic — 1024 is worse than 512 on two of three tasks — and the optimum is task-dependent: 512 for screens and documents, 256 for natural images. "Use the maximum" is wrong.

from mlx_vlm import load
model, processor = load("shubhambaid/LFM2.5-VL-3B-MLX-mixed-v6p6b4e4")
processor.image_processor.max_image_tokens = 512
processor.image_processor.max_num_patches  = 2048   # must scale with the budget

max_num_patches sizes the positional-embedding buffer. Raising the token budget without raising it raises a broadcast error rather than degrading gracefully.

The defaults in this repo are unchanged from the base model — deviating silently from what Liquid shipped would be worse than documenting it.

Grounding output format is prompt-dependent

Asked for a bounding box four different ways, the base model returns four different things — and three of the four return no coordinates at all:

prompt	result
"Where is the element to X? Return its bounding box."	parses (~1% failure)
"Click on: X"	prose — 100% parse failure
"Point to the element to X."	prose — 100% parse failure
Locate "X".	prose — 100% parse failure

When it does emit coordinates the format varies between JSON bbox_2d, 0–1 floats, and 0–1000 integers. The model card documents 0–1000 as the convention. Use the "Return its bounding box" phrasing and parse defensively.

What was measured, and what wasn't

Five benchmarks at n=300 with a 512-token budget: DocVQA (ANLS), ChartQA (relaxed accuracy), OCRBench (substring match), ScreenSpot-v2 (point-in-box), RefCOCO (Prec@1 at IoU 0.5). Not measured: multilingual performance, tool calling, multi-image reasoning, long-context behaviour, or anything on the base model's 16 non-English languages.

The bf16 baseline reproduces the published card within 5 points on four of five benchmarks. ScreenSpot-v2 lands at 71.0 vs a published 80.7; a per-segment breakdown localizes the gap to desktop/web icons (macOS 0.524, Windows 0.667 vs iOS 0.872), and Liquid publishes neither their token budget nor their prompt, so the absolute figure isn't reproducible from the card alone. Comparisons here are all against the same pinned bf16 baseline, so they remain attributable.

License

LFM Open License v1.0, inherited from the base model. Apache-2.0-derived; commercial rights terminate above $10M annual revenue.

Modifications from the base model: weights quantized to mixed precision per the recipe above. No fine-tuning, no architecture changes, no config changes. All attribution notices retained.