Wonderfull model
#1
by clayboby - opened
Good Quality
Much Better than LilaRest/gemma-4-31B-it-NVFP4-turbo and cyankiwi/gemma-4-31B-it-AWQ-4bit
Any optimize skill？
PS: I patched vllm and run it in single 5090 32G with 200K context and 1 image input
Performance wise I am just trying to quantize what makes sense and calibration wise the calibration dataset is a mix of coding and tool call corpus as those are the my top usages. Recipe and dataset composition are in the model card so it should be easy to evaluate and replicate.
Really good. I test GSM8k-Platinum 、MMLU-CoT、IFEval and HumanEval and my own test, almost no quanlity loss compare to goole. Best model for agent like openclaw or hermes