Instructions to use kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4") model = AutoModelForMultimodalLM.from_pretrained("kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4
- SGLang
How to use kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 with Docker Model Runner:
docker model run hf.co/kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4
- Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 — Blackwell Native NVFP4 (
20.7 GiB/ Single 24 GB GPU Ready)- 1. What Kataguru Changed (
254 + 2Sovereign Language MoE Architecture) - 2. Native Sceptic Multi-Mode, Multimodal & Agentic Capabilities (6 Engines in 1 Model)
- 3. Verified Benchmark & Live Audit Summary
- 4. Honest Limitations & Hardware Sizing
- Laburpena, Industria-Programazioa eta Sceptic Gaitasun Natiboak (Euskaraz)
- 1. What Kataguru Changed (
- Suomenkielinen Yhteenveto / Finnish Summary
Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 — Blackwell Native NVFP4 (20.7 GiB / Single 24 GB GPU Ready)
🏢 Official Release by Datasein Oy (Finland) Developed and published by Datasein Oy (
@kataguru, Chief Architect: Jari Katajisto, Finland).
Kataguru-Sceptic-35B-A3B-Basque-v3.2 is a sovereign Basque (Euskara) Industrial Coder (PLC/CNC: 96.67 %), Systems Software Engineer & 6-in-1 Multi-Mode / Multimodal Mixture-of-Experts (254 + 2 = 256 experts per layer) flagship model built on the Kataguru Sceptic 35B-A3B v3.1 foundation (15-Benchmark Mean: 70.26 %, GPQA Diamond: 70.20 %, LiveCodeBench v6: 39.43 %, BFCL V4 AST: 83.67 %, Industrial PLC/CNC: 96.67 %).
⚙️ Not Just a Language Model — A Full-Stack Industrial & Systems Coder in Basque (
96.67 %PLC/CNC) Because254frozen Sceptic core experts (#0..#253) co-activate in everyTop-8token step alongside the2native Basque experts (#254, #255), Kataguru-Sceptic-35B-A3B-Basque-v3.2 is a production-grade Industrial Automation & Systems Software Coder that writes, audits, and debugs PLC/CNC and systems code directly in Basque (Euskara), Finnish, or English.
🛠️ Specialized Industrial Coding & Engineering Skills (#0..#253 + #EU)
- IEC 61131-3 PLC Programming (
96.67 %Verified Benchmark):- Beckhoff TwinCAT 3 Structured Text (
ST): Deterministic state machines (CASE), motion control (MC_MoveAbsolute,MC_Power,MC_GearIn), EtherCAT distributed clocks, and ADS diagnostics. - Siemens TIA Portal (
SCL/ S7-1500): Function Blocks (FB/FC), instance Data Blocks (DB), PID closed-loop control (PID_Compact), and PROFINET IO diagnostics.
- Beckhoff TwinCAT 3 Structured Text (
- CNC G-Code, Post-Processors & Coordinate Kinematics:
- Fanuc / Siemens Sinumerik 840D / Heidenhain: Multi-axis lathe & milling G-code (
G41/G42tool radius compensation,G54–G59work offsets, canned cyclesG71/G76/G81–G89), spindle/feedrate load verification, and collision-free toolpath auditing.
- Fanuc / Siemens Sinumerik 840D / Heidenhain: Multi-axis lathe & milling G-code (
- Industrial Protocols, Fieldbuses & SCADA/MES Diagnostics:
- OPC UA (
IEC 62541), Modbus TCP/RTU, EtherCAT, PROFINET, CANopen, and SQL Server / WCF production telemetry troubleshooting.
- OPC UA (
- Functional Safety & Machinery Standards:
- ISO 13849-1 (
PLa–PLe), IEC 62061 / IEC 61508 (SIL 1–SIL 3), Safe Torque Off (STO), two-channel emergency-stop interlocks, and watchdogs.
- ISO 13849-1 (
- Systems, Embedded & Algorithmic Software Engineering (
LiveCodeBench v6: 39.43 %,Easy: 90.70 %):- Modern
C++20,Rust,C(embedded/RTOS),Python 3.12,C#/.NET,SQL, andBash/Linuxkernel/network hardening, concurrency, and memory-safe systems architecture.
- Modern
- Deterministic
<tool_call>AST Agent & Schematic Vision (BFCL V4: 83.67 %):- Reads P&ID diagrams, electrical wiring schematics, HMI error screens, and engineering plots via its
27-layer ViT Vision Encoder, and executes multi-step<tool_call>AST tool chains (0syntax errors).
- Reads P&ID diagrams, electrical wiring schematics, HMI error screens, and engineering plots via its
Unlike single-purpose language fine-tunes, Kataguru-Sceptic-35B-A3B-Basque-v3.2 preserves 100 % of Kataguru Sceptic's native multi-mode operating system, 27-layer multimodal vision encoder, <tool_call> AST agentic engine, and graduate STEM/industrial coding core inside a single model — switching roles dynamically per request with zero server restarts:
- Interactive Capability & Code Examples:
kataguru/Kataguru-Sceptic-35B-Usage-Examples|kataguru/Kataguru-Sovereign-AI-Guide
Official Basque (v3.2) Release Suite
- Full BF16 Master Weights (
~67.5 GB):kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2 - Blackwell Native NVFP4 (
~20.9 GiB, Single 24 GB GPU Ready):kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4 - Universal vLLM W4A16-AWQ (
~19.5 GiB, Single 24 GB GPU Ready):kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-W4A16-AWQ - GGUF APEX-Trio (
APEX-Small,APEX-Medium,APEX-Good+mmproj— 16 GB GPU Ready):kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-GGUF
1. What Kataguru Changed (254 + 2 Sovereign Language MoE Architecture)
| Field | Description / Value |
|---|---|
| Base model | kataguru/Kataguru-Sceptic-35B-A3B-v3.1 (254 frozen core experts #0..#253, 3B active parameters per token) |
Kataguru v3.2 architecture |
254 + 2 = 256 Experts per MoE layer (40 layers):• Experts #0..#253 (100 % Frozen Core): Preserves the full scientific reasoning (GPQA Diamond 70.20 %), competitive coding (LiveCodeBench v6 39.43 %), agentic tool calling (BFCL V4 83.67 %), and industrial PLC/CNC capabilities (96.67 %) with zero catastrophic forgetting.• Expert #254 (Basque Grammar, Morphology & Technical Specialist): Ergative-absolutive case alignment, polypersonal auxiliary verb agreement (Nor-Nori-Nork), and agglutinative postpositional suffixes, plus industrial engineering, PLC/CNC, clinical medicine, and statutory/patent terminology.• Expert #255 (Basque Literature, Nonfiction & Bidirectional Translator): Full-page literary prose, modern nonfiction, natural dialogue, and faithful EN <-> EU translation (/translate / /itzuli). |
| Language Bank Training Set | 900 human-verified golden samples (100 % Pass on Mechanical & Quality Gate) + contrastive router calibration (--moe-mode new_experts_only --recalibrate-router, r=32, alpha=64) |
| Dedicated Chat Template & Calibration | Native Basque Multi-Mode chat_template.jinja (/eu, /translate, /itzuli, /erraza, [INSPECT], [CURATE], [SURGEON], [DECISION]) + Basque quantization calibration (calib_nvfp4_awq_256_eu.jsonl & imatrix_calibration_eu.txt) |
| Format & Footprint | Blackwell NVFP4 (compressed-tensors, ~20.7 GiB on disk, ~19.4 GiB in vLLM --language-model-only — fits on a single 24 GB GPU!) |
| What was NOT changed | All 254 core reasoning/STEM/coding experts (#0..#253), 27-layer multimodal vision encoder (333 tensors), Layer 40 MTP speculative head (785 tensors), 262,144 context window |
2. Native Sceptic Multi-Mode, Multimodal & Agentic Capabilities (6 Engines in 1 Model)
Every Kataguru-Sceptic-35B-A3B-Basque-v3.2 checkpoint includes the complete Kataguru Sovereign Multi-Mode Master Architecture out of the box:
| Built-In Mode / Engine | Trigger Tag or Template Arg | Native Capability & Production Role |
|---|---|---|
| 1. Forensic Quality Inspector | [INSPECT] or sceptic_mode="inspect" |
Audits code, math derivations, and SFT datasets; emits deterministic verdict-first JSON ({"verdict": "ACCEPT|REJECT|REVIEW", "fatal_error": false, "learning_value": 9.2, "score": 9.2, ...}) with <100 ms early-exit rejection. |
| 2. First-Principles Data Curator & Surgeon | [CURATE] / [SURGEON] |
Synthesizes high-learning-value (>= 8.5) training samples from raw technical documents ([CURATE]) or self-heals buggy/truncated code and flawed proofs ([SURGEON]). |
| 3. 20 ms System 1 Reflex Judge | [DECISION] or sceptic_mode="decision" |
Closes <think> automatically and scores candidates via single-token (max_tokens=1, logprobs=True) 0–9 / A/B logprob distributions (>1M samples/day throughput). |
| 4. Full-Page Bidirectional Translator | /translate or /itzuli |
Zero-chatter, paragraph-preserving literary, legal, medical, and engineering translation (EN <-> EU & FI <-> EU), keeping LaTeX formulas and code blocks intact (0.00 % refusal). |
5. Native Basque & Plain-Language Control |
/eu, /en, /fi, /erraza |
Locks response language across all modes (including code comments and [INSPECT] rationales), or switches to accessible plain Basque (/erraza). |
6. Multimodal Vision, <tool_call> Agent & PLC/CNC |
Default Chat + images / tools=[...] |
27-Layer ViT Vision (schematics, charts, UI screenshots, OCR, video), BFCL V4 83.67 % <tool_call> AST, Systems SWE (C, C++20, Rust, Python), and Industrial Automation (96.67 %) (Beckhoff TwinCAT 3 ST, Siemens TIA Portal SCL, CNC G-code, OPC UA). |
Empirical Cognitive Reasoning Modes (chat_template.jinja)
Control reasoning structure and latency per prompt via inline tags (automatically stripped before rendering) or chat_template_kwargs={"reasoning_effort": "..."}:
{REASON:ihyper}(Instant MECE):+8.69 pphigher accuracy and-31 %faster (0<think>overhead; structured MECE synthesis directly in the answer).{REASON:hyper}/{REASON:logic}: Deliberate MECE decomposition or formal first-principles axiom/unit verification inside<think>...</think>.{REASON:007}(Fault-Tolerant Agentic Loop): Deterministic state-machine execution for<tool_call>AST tool use and PLC/CNC control logic.{REASON:spoon}/{REASON:einstein}/{REASON:help}/{REASON:off}: Deep premise validation, multi-angle scientific synthesis, interactive mode guide, or instant zero-think reply.
3. Verified Benchmark & Live Audit Summary
| Benchmark / Capability Pillar | Score | Notes |
|---|---|---|
| 15-Benchmark Core Foundation Mean | 70.26 % | Inherited 100 % intact via frozen Experts #0..#253 |
| GPQA Diamond (PhD-level STEM) | 70.20 % (139 / 198) |
Graduate physics, chemistry & molecular biology |
| Industrial PLC / CNC / TwinCAT / SCL | 96.67 % | IEC 61131-3 Structured Text, Siemens SCL & G-code verification |
| BFCL V4 Single-Turn AST (Tool Calling) | 83.67 % (Non-Live: 88.44 %) |
Native <tool_call> AST & irrelevance detection (80.26 %) |
LiveCodeBench v6 (Pass@1) |
39.43 % (Easy: 90.70 %) |
Competitive algorithmic programming |
Live Basque 10/10 Audit (v3.2) |
10 / 10 PASS (100 %) |
HiTZ EvalEU Suite (EusProficiency, EusExams, EusReading, EusTrivia) and Full-Page Literary & Technical Translation (EN/ES <-> EU) |
4. Honest Limitations & Hardware Sizing
- High-Temperature Creative Fiction: For strict morphological and proofreading accuracy in Basque, use
temperature=0.2–0.3withpresence_penalty=0.3. - VRAM Footprint:
W4A16-AWQ(~19.5 GiBdisk /~18.0 GiBVRAM) &NVFP4(~20.9 GiBdisk /~19.4 GiBVRAM): Run on a single24 GBGPU (RTX 3090 / 4090 / 5090 / L4) in vLLM (--language-model-only) or2x 16–32 GBGPUs with full multimodal vision!GGUF APEX-Small / Medium / Good(~11.9–15.4 GiB): Run on a single16 GBGPU or Apple Silicon Mac (llama.cpp, LM Studio, Ollama).- Full
BF16(~67.5 GB): Requires~72 GBVRAM (2x 48–80 GB).
Laburpena, Industria-Programazioa eta Sceptic Gaitasun Natiboak (Euskaraz)
Kataguru-Sceptic-35B-A3B-Basque-v3.2 ez da hizkuntza-eredu arrunt bat soilik, 256 adituko MoE arkitektura bakarrean integratutako Industria-Programatzailea (PLC/CNC: 96,67 %) eta 6-batean sistema multifuntzional subiranotasunezkoa baizik (254 Sceptic oinarrizko aditu izoztu #0..#253 + 2 euskara batuko aditu espezializatu #254 eta #255):
- Industria- eta Sistema-Programatzaile Osoa:
- PLC eta industria-automatizazioa (
IEC 61131-3: 96,67 %): Beckhoff TwinCAT 3 Structured Text (ST), Siemens TIA Portal (SCL/ S7-1500), egoera-makinak, higidura-kontrola, PID kontrolagailuak eta EtherCAT/PROFINET diagnostikoa. - CNC G-kodea eta mekanizazioa: Fanuc / Sinumerik 840D / Heidenhain (
G41/G42erradio-konpentsazioa,G54–G59). - Industria-protokoloak eta segurtasun funtzionala: OPC UA (
IEC 62541), Modbus TCP/RTU, ISO 13849-1 (PLe), IEC 62061 (SIL 3/STO). - Sistema- eta algoritmo-programazioa (
LiveCodeBench v6: 39,43 %,Easy: 90,70 %):C++20,Rust,C,Python 3.12,C#/.NET,SQLetaLinux/Bash.
- PLC eta industria-automatizazioa (
- Euskara Batua eta orrialde osoko itzulpena (
/eu,/translate,/itzuli,/erraza): Ergativitatea etaNor-Nori-Norkaditz-komunztadura zehatza, terminologia teknikoa eta bi noranzkoko itzulpena. - Sceptic-en 6 lan-modu natiboak:
[INSPECT],[CURATE]&[SURGEON],[DECISION](<20 ms),27geruzako ViT ikusmen multimodala,<tool_call>agentea (BFCL V4: 83,67 %) eta{REASON:ihyper}/{REASON:007}.
Suomenkielinen Yhteenveto / Finnish Summary
Kataguru-Sceptic-35B-A3B-Basque-v3.2 on Kataguru Sceptic 35B-A3B (v3.1, 15-benchmark: 70,26 %) -pohjalle rakennettu suvereeni 254 + 2 = 256 ekspertin Teollisuuskoodari (PLC/CNC: 96,67 %) ja 6-in-1-monitoimimalli (Basque / Euskara):
- Täyden tehon Teollisuuskoodari & Järjestelmäohjelmoija (Ei pelkkä kielimalli):
- PLC- ja automaatio-ohjelmointi (
96,67 %): Kirjoittaa, tarkastaa ja korjaa Beckhoff TwinCAT 3 Structured Text (ST)- ja Siemens TIA Portal (SCL/ S7-1500) -ohjelmia (tilakoneet, liikeohjaus, PID-säätö, EtherCAT/PROFINET-diagnostiikka). - CNC G-koodi ja koneistus: Fanuc / Sinumerik 840D / Heidenhain -työstöradat (
G41/G42-teräsädekompensaatio,G54–G59, kiinteät työstökierrot ja törmäystarkastus). - Teollisuusväylät ja koneturvallisuus: OPC UA (
IEC 62541), Modbus TCP/RTU, ISO 13849-1 (PLe) ja IEC 62061 (SIL 3/STO). - Järjestelmä- ja algoritmikoodaus (
LiveCodeBench v6: 39,43 %,Easy: 90,70 %): TuotantotasonC++20,Rust,C,Python 3.12,C#/.NET,SQLjaLinux/Bashsuoraan kohdekielellä (Euskara), suomeksi tai englanniksi.
- PLC- ja automaatio-ohjelmointi (
- Natiivi Sceptic-monitoimiarkkitehtuuri yhdessä mallissa: Sisältää jäädytetyn
254ekspertin tieteellisen ja teknisen ytimen (GPQA Diamond 70,20 %,BFCL V4 <tool_call> 83,67 %),27-kerroksisen multimodaalisen vision (mmprojmukana kytkentäkaavioiden ja HMI-ruutujen lukuun), laaduntarkastajan ([INSPECT]), aineistosyntetisoijan ja koodikirurgin ([CURATE]/[SURGEON]),<20 msSysteemi 1 -pikatuomarin ([DECISION]) sekä empiiriset{REASON:ihyper}/{REASON:007}-päättelytilat. - Kaksi dedikoitua kielieksperttiä (
#254ja#255,900näytettä):- Ekspertti
#254: Kohdekielen (Euskara) taivutusoppi, morfologia sekä tekniikan, automaation, lääketieteen ja lakikielen erikoissanasto. - Ekspertti
#255: Kohdekielen kaunokirjallisuus, tietokirjallisuus ja koko sivun kaksisuuntaiset käännökset (/translate//itzuli).
- Ekspertti
- 24 GB vLLM & 16 GB GGUF -valmis: Täydet
BF16-painot, yhdelle24 GBnäytönohjaimelle mahtuvatNVFP4(~20,9 GiB) jaW4A16-AWQ(~19,5 GiB) sekä kielikohtaisellaimatrix-korpuksella kalibroidutGGUF APEX-kvantit (APEX-Small,APEX-Medium,APEX-Good+mmproj-f16). - Julkaisija (
Datasein Oy, Suomi): Kehittäjä Datasein Oy (@kataguru, pääarkkitehti Jari Katajisto).
- Downloads last month
- 120
Model tree for kataguru/Kataguru-Sceptic-35B-A3B-Basque-v3.2-NVFP4
Base model
Qwen/Qwen3.5-35B-A3B-Base