Image-Text-to-Text
GGUF
English
Portuguese
ollama
llama.cpp
local-ai
on-device
low-vram
small-model
tiny-llm
vision
multimodal
thinking
reasoning
tool-calling
code
coding-assistant
typescript
javascript
qwen
qwen3.5
iq3_xxs
imatrix
conversational
Eval Results (legacy)
Instructions to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS # Run inference directly in the terminal: llama cli -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS # Run inference directly in the terminal: llama cli -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS # Run inference directly in the terminal: ./build/bin/llama cli -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Use Docker
docker model run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
- Llama new
- LM Studio
- Jan
- vLLM
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
- Ollama
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with Ollama:
ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
- Unsloth Desktop
- Pi
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with Docker Model Runner:
docker model run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
- Lemonade
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Run and chat with the model
lemonade run user.Sagui-4B-Vision-Thinking-Code-GGUF-IQ3_XXS
List all available models
lemonade list
- Hermes Agent
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 10.9 kB
-
https://huggingface.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/resolve/main/README.md
- Command line
-
hf download hf://sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/README.md
-
curl -L -o README.md https://huggingface.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/resolve/main/README.md
10.9 kB
| license: apache-2.0 | |
| base_model: Qwen/Qwen3.5-4B | |
| base_model_relation: quantized | |
| library_name: gguf | |
| pipeline_tag: image-text-to-text | |
| language: | |
| - en | |
| - pt | |
| thumbnail: https://huggingface.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/resolve/main/assets/banner.png | |
| tags: | |
| - gguf | |
| - ollama | |
| - llama.cpp | |
| - local-ai | |
| - on-device | |
| - low-vram | |
| - small-model | |
| - tiny-llm | |
| - vision | |
| - multimodal | |
| - thinking | |
| - reasoning | |
| - tool-calling | |
| - code | |
| - coding-assistant | |
| - typescript | |
| - javascript | |
| - qwen | |
| - qwen3.5 | |
| - iq3_xxs | |
| - imatrix | |
| - conversational | |
| model-index: | |
| - name: Sagui-4B | |
| results: | |
| - task: | |
| type: text-generation | |
| name: TypeScript code generation | |
| dataset: | |
| type: nuprl/MultiPL-E | |
| name: HumanEval-TS (MultiPL-E) | |
| config: humaneval-ts | |
| split: test | |
| metrics: | |
| - type: pass@1 | |
| name: pass@1 | |
| value: 44.0 | |
| - task: | |
| type: text-generation | |
| name: TypeScript code generation | |
| dataset: | |
| type: nuprl/MultiPL-E | |
| name: MBPP-TS (MultiPL-E) | |
| config: mbpp-ts | |
| split: test | |
| metrics: | |
| - type: pass@1 | |
| name: pass@1 | |
| value: 54.6 | |
| # Sagui-4B: tiny 2.4 GB local AI that thinks, sees and codes (GGUF) | |
|  | |
| **Sagui-4B is a tiny open-weights model for [Ollama](https://ollama.com) and [llama.cpp](https://github.com/ggml-org/llama.cpp).** It reasons step by step, reads images and screenshots, writes TypeScript, and calls tools, all offline. It runs on a 4 GB GPU from 2014. | |
| > **Size:** 2.4 GB in total: a 1.71 GB model plus a 0.67 GB vision projector. Without images, you only need the 1.71 GB file. | |
| ```bash | |
| ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS | |
| ``` | |
| <p align="center"><a href="https://www.paypal.com/donate/?hosted_button_id=JA6B7RKJURXNW" title="Donate to Bugio and Sagui via PayPal"><img src="https://huggingface.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/resolve/main/assets/donate.png" alt="Support Bugio and Sagui: donate via PayPal" width="520"></a></p> | |
| If Sagui saves you a subscription or a GPU upgrade, you can [support the project](#support-the-project): [☕ PayPal](https://www.paypal.com/donate/?hosted_button_id=JA6B7RKJURXNW) | |
| ## Why Sagui-4B | |
| **40% smaller than the Q4_K_M it comes from, for 5 to 9 points less on TypeScript benchmarks.** Thinking, vision and tool calling all stay. | |
| | | Sagui-4B | Qwen3.5-4B Q4_K_M (source model) | | |
| |---|---|---| | |
| | Total with vision | **2.38 GB** | 3.51 GB | | |
| | Without vision | **1.71 GB** | 2.83 GB | | |
| | Thinking, vision, tool calling | ✅ | ✅ | | |
| | HumanEval-TS pass@1 | **44.0%** | 52.8% | | |
| | MBPP-TS pass@1 | **54.6%** | 59.7% | | |
| | Speed on a GTX 970 (llama.cpp, 4 GB) | **21.2 tok/s** | 20.5 tok/s | | |
| | Languages | English, Portuguese and other Latin scripts | Many, including non-Latin scripts | | |
| A sagui is a marmoset, one of the smallest monkeys in the world, native to Brazil. Tiny, sharp-eyed and very quick. | |
| ## See it working | |
| These are unedited outputs from this exact GGUF. Prompts and raw responses are in [`examples/transcripts.json`](examples/transcripts.json), and `scripts/run_demos.py` reproduces them. Sampling isn't seeded, so your wording will vary. | |
| ### Screenshot → bug fix | |
|  | |
| > **Prompt:** This is a screenshot from my editor. Explain the error in one sentence, then give the corrected line 4. | |
| > The error occurs because the `reduce` method's initial value of `"0"` is a string, which cannot be added to a number; it should be replaced with the numeric literal `0`. | |
| `return items.reduce((sum, i) => sum + i.price * i.qty, 0);` | |
| *63 tokens, 26.8 s on a GTX 970.* | |
| ### UI screenshot → working React component | |
| | Input screenshot | Sagui's code, rendered | | |
| |---|---| | |
| |  |  | | |
| *Thinking on, 2332 tokens, 174.2 s on a GTX 970. The code is unedited and the page has no CSS reset. It missed the field placeholders. Without thinking, the same prompt produced correct text and fields but ignored the inline-style request.* | |
| ### Thinking mode | |
| > **Prompt:** A train leaves at 14:35 and arrives at 17:10. How long is the trip in minutes? | |
| ```text | |
| To find the duration of the trip in minutes, we can calculate the difference between the arrival time and the departure time. | |
| **Step 1: Calculate the time difference** | |
| * **Departure:** 14:35 (2:35 PM) | |
| * **Arrival:** 17:10 (5:10 PM) | |
| First, look at the hours: $17 - 14 = 3$ hours. However, since we are subtracting minutes ($10 - 35$), we cannot simply subtract directly because 10 is less than 35. We need to borrow an hour from the arrival time. | |
| * Borrow 1 hour (60 minutes) from the hours: | |
| $17 \text{ hours } 10 \text{ minutes} = 16 \text{ hours } 70 \text{ minutes}$ | |
| Now, subtract the departure time ($14 \text{ hours } 35 \text{ minutes}$) from this adjusted time: | |
| * **Hours:** $16 - 14 = 2$ hours | |
| * **Minutes:** $70 - 35 = 35$ minutes | |
| So, the trip duration is **2 hours and 35 minutes**. | |
| **Step 2: Convert to minutes** | |
| * 2 hours = $2 \times 60 = 120$ minutes. | |
| * Add the remaining minutes: $120 + 35 = 155$. | |
| The trip is **155** minutes long. | |
| ``` | |
| *3542 tokens, 254.5 s on a GTX 970, most of them reasoning. Thinking checks its work at length, so turn it off when you want speed.* | |
| ## Run it | |
| ### Ollama, one command | |
| ```bash | |
| ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS | |
| ``` | |
| ### Ollama, from the files | |
| ```bash | |
| hf download sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF --local-dir sagui | |
| cd sagui | |
| ollama create sagui-4b -f Modelfile | |
| ollama run sagui-4b "What is in this image? ./photo.png" | |
| ``` | |
| Control thinking per chat with `--think` or `--think=false`, or in the API: | |
| ```bash | |
| curl http://localhost:11434/api/chat -d '{ | |
| "model": "sagui-4b", | |
| "think": false, | |
| "messages": [{"role": "user", "content": "Write a debounce function in TypeScript."}] | |
| }' | |
| ``` | |
| ### llama.cpp | |
| ```bash | |
| llama-server -m Sagui-4B-IQ3_XXS.gguf --mmproj mmproj-Sagui-4B-F16.gguf -ngl 99 -c 16384 | |
| ``` | |
| ### Tip for 4 GB GPUs | |
| Ollama can split the model between CPU and GPU to be safe. On our GTX 970 that gave 15.0 tok/s. Forcing every layer onto the GPU gave 22.7 tok/s: | |
| ``` | |
| PARAMETER num_gpu 99 | |
| ``` | |
| Add that line to the `Modelfile` and run `ollama create` again. If you run out of memory, remove it or lower `num_ctx`. | |
| ## Files | |
| | File | Size | Needed for | | |
| |---|---|---| | |
| | `Sagui-4B-IQ3_XXS.gguf` | 1.71 GB | Everything | | |
| | `mmproj-Sagui-4B-F16.gguf` | 0.67 GB | Images only | | |
| | `Modelfile`, `template`, `params` | small | Ollama setup | | |
| ## Benchmarks | |
| TypeScript code generation on [MultiPL-E](https://huggingface.co/datasets/nuprl/MultiPL-E) with greedy decoding. A solution passes only if it type-checks with `tsc` and passes its tests in Node. | |
| | Benchmark | Sagui-4B IQ3_XXS | Qwen3.5-4B Q4_K_M | | |
| |---|---|---| | |
| | HumanEval-TS | 70/159 (**44.0%**) | 84/159 (52.8%) | | |
| | MBPP-TS | 213/390 (**54.6%**) | 233/390 (59.7%) | | |
| These measure raw completion without thinking mode. Completions, failures and the script are in [`benchmarks/`](benchmarks/). | |
| Measured on an Intel i5-10400, 16 GB RAM and an NVIDIA GTX 970 (4 GB). With Ollama's default split, about 3.4 GB of VRAM was in use with the desktop running. CPU only: 8.1 tok/s. | |
| ## Limitations | |
| - **Latin scripts only.** Removing other alphabets is part of how the model got this small. Asked to translate into Japanese and Russian, it answered in romanized text and got the Japanese wrong. Use the original Qwen3.5-4B for Chinese, Japanese, Korean, Russian, Arabic and similar languages. | |
| - **Heavy quantization.** IQ3_XXS trades accuracy for size, and it was calibrated on TypeScript. Expect weaker results outside code than from larger builds. | |
| - **Thinking is slow on small hardware.** A simple question can take thousands of tokens and several minutes. | |
| - **It makes mistakes.** About half of the benchmark solutions fail. In our `groupBy` demo, the code ran correctly but failed `tsc --strict` with TS2322. Review generated code before running it, and don't use the model for medical, legal or safety-critical decisions. Vision was tested on screenshots and simple images, not on photos at scale. | |
| ## FAQ | |
| **How small is a local model with vision and thinking?** Sagui-4B does vision, reasoning and tool calling in 2.4 GB. Without images, it needs 1.71 GB. | |
| **Can it run on 4 GB of VRAM?** Yes. Every result here comes from a 4 GB GTX 970. | |
| **Does it run without a GPU?** Yes, on CPU, at 8.1 tok/s on an i5-10400. | |
| **Is it good for coding offline?** It's useful for small TypeScript tasks and for explaining errors from screenshots. Larger models are more accurate, as the benchmarks above show. | |
| **Was it retrained?** No. It has the same weights as Qwen3.5-4B, with a trimmed vocabulary and heavier quantization. See the [recipe](recipe/README.md). | |
| **Can I use it commercially?** Yes, under Apache 2.0. Keep the `LICENSE` and `NOTICE` files. | |
| ## Em português | |
| **Sagui-4B é uma IA local de 2,4 GB que pensa, enxerga e programa.** São 1,71 GB de modelo mais 0,67 GB do projetor de visão. Sem imagens, basta o arquivo de 1,71 GB. Roda offline com Ollama ou llama.cpp, inclusive em GPU de 4 GB, e responde em português e inglês. Foi calibrado para TypeScript. | |
| ```bash | |
| ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS | |
| ``` | |
| <p align="center"><a href="https://www.paypal.com/donate/?hosted_button_id=JA6B7RKJURXNW" title="Donate to Bugio and Sagui via PayPal"><img src="https://huggingface.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/resolve/main/assets/donate.png" alt="Support Bugio and Sagui: donate via PayPal" width="520"></a></p> | |
| Gostou? [Apoie o projeto](#support-the-project) e compartilhe. | |
| ## Support the project | |
| Sagui-4B is free and stays free. Donations pay for the compute and the time behind the next tiny models. | |
| <p align="center"><a href="https://www.paypal.com/donate/?hosted_button_id=JA6B7RKJURXNW" title="Donate to Bugio and Sagui via PayPal"><img src="https://huggingface.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF/resolve/main/assets/donate.png" alt="Support Bugio and Sagui: donate via PayPal" width="520"></a></p> | |
| - **[PayPal](https://www.paypal.com/donate/?hosted_button_id=JA6B7RKJURXNW)** | |
| No money to spare? A ❤️ on this page and a post about what you built help just as much. | |
| ## Credits and license | |
| Sagui-4B is a modified version of [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) by the Qwen team, converted from the GGUF by [Unsloth](https://huggingface.co/unsloth) and quantized with [llama.cpp](https://github.com/ggml-org/llama.cpp). It is licensed under [Apache 2.0](LICENSE), and the changes are listed in [`NOTICE`](NOTICE). This is an independent community project, not affiliated with or endorsed by the Qwen team or Alibaba Cloud. | |