Instructions to use Nanthasit/sakthai-context-0.5b-tools with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-0.5b-tools with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-0.5b-tools")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-0.5b-tools")
model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-0.5b-tools", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanthasit/sakthai-context-0.5b-tools with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-0.5b-tools"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-tools",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-0.5b-tools

SGLang

How to use Nanthasit/sakthai-context-0.5b-tools with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-0.5b-tools" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-tools",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-0.5b-tools" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-tools",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanthasit/sakthai-context-0.5b-tools with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-0.5b-tools
```

sakthai-context-0.5b-tools / README.md

Nanthasit

docs: improve README with Model Description, expanded usage, Citation, and cleaned architecture table

6f59c64 verified about 2 months ago

preview code

Raw

History Blame Contribute Delete

9.59 kB

	---
	license: apache-2.0
	language:
	- en
	library_name: transformers
	pipeline_tag: text-generation
	base_model: Qwen/Qwen2.5-0.5B-Instruct
	tags:
	- agent
	- conversational
	- ollama
	- transformers
	- small-language-model
	- slm
	- tool-use
	- qwen
	- qwen2.5
	- sakthai
	- house-of-sak
	- tool-calling
	- function-calling
	- merged
	- edge
	- lightweight
	- low-resource
	- raspberry-pi
	- on-device
	- benchmark
	- eval
	datasets:
	- Nanthasit/sakthai-combined-v7
	- Nanthasit/sakthai-bench-v2
	model-index:
	- name: sakthai-context-0.5b-tools
	results:
	- task:
	type: text-generation
	name: Tool-Calling
	dataset:
	name: SakThai Bench v2 (500 rows, scorer multiset-selection-v2)
	type: Nanthasit/sakthai-bench-v2
	metrics:
	- type: selection
	value: 91.2
	name: Selection Accuracy
	verified: true
	evidence: .eval_results/sakthai-bench-v2.yaml
	- type: arguments
	value: 45.7
	name: Arguments Accuracy
	verified: true
	evidence: .eval_results/sakthai-bench-v2.yaml
	- type: strict
	value: 45.7
	name: Strict Accuracy
	verified: true
	evidence: .eval_results/sakthai-bench-v2.yaml
	- type: held-out
	value: 87.8
	name: Held-Out Tool Accuracy
	verified: true
	evidence: .eval_results/sakthai-bench-v2.yaml
	- type: degenerate
	value: 0
	name: Degenerate Outputs
	verified: true
	evidence: .eval_results/sakthai-bench-v2.yaml
	inference:
	parameters:
	temperature: 0.01
	max_new_tokens: 256
	top_p: 0.9
	widget:
	- text: What is the weather in Tokyo?
	example_title: Tool-calling
	- text: Who wrote Romeo and Juliet?
	example_title: Direct answer
	- text: Search the web for latest AI news
	example_title: Search tool
	---

	<p align="center">
	<img src="https://img.shields.io/badge/dynamic/json?url=https%3A//huggingface.co/api/models/Nanthasit/sakthai-context-0.5b-tools&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
	<img src="https://img.shields.io/badge/size-~1.0GB-blue" alt="Size"/>
	<img src="https://img.shields.io/badge/pipeline-tool--calling-orange" alt="Pipeline"/>
	<img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
	<a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
	<a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9A%80-Explore%20Family-47d147" alt="Family"/></a>
	</p>

	<h1 align="center">SakThai Context 0.5B Tools</h1>
	<p align="center"><em>Ultra-light tool-calling agent · Qwen2.5-0.5B fine-tune · runs in ~1 GB RAM</em></p>

	SakThai Context 0.5B Tools is a prompt-masked SFT of [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) optimized for browser/tool calling. It achieves 91.2% selection accuracy on SakThai Bench v2, with 0% degenerate outputs in multi-trial evaluation.

	## Model Description

	SakThai Context 0.5B Tools is a prompt-masked supervised fine-tune of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM.

	Key points:
	- Base: `Qwen/Qwen2.5-0.5B-Instruct`
	- Training: prompt-masked SFT on tool-calling traces from `Nanthasit/sakthai-combined-v7`
	- Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments
	- License: Apache-2.0

	## Quick Start — Transformers

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model_id = "Nanthasit/sakthai-context-0.5b-tools"
	tokenizer = AutoTokenizer.from_pretrained(model_id)
	model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

	tools = [
	{
	"type": "function",
	"function": {
	"name": "get_weather",
	"description": "Get current weather",
	"parameters": {
	"type": "object",
	"properties": {"location": {"type": "string"}},
	"required": ["location"],
	},
	}
	}
	]

	messages = [
	{"role": "system", "content": "You are a helpful assistant."},
	{"role": "user", "content": "What's the weather in Tokyo?"},
	]

	text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
	inputs = tokenizer([text], return_tensors="pt").to(model.device)
	outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9)
	response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
	print(response)
	```

	## Quick Start — llama.cpp / Ollama

	```bash
	# Convert with llama.cpp and run locally
	llama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_M
	ollama create sakthai-context-0.5b-tools -f Modelfile
	ollama run sakthai-context-0.5b-tools
	```

	### Usage notes
	- For tool calling, always use `apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True)` so the model receives the proper `<tools>` block.
	- If you want stricter outputs, reduce `temperature` further, e.g. `0.0`.
	- For CPU-only inference, set `device_map="cpu"`; GPU/MPS/CPU auto-detection works with `device_map="auto"`.

	## Architecture & Config

	\| Field \| Value \|
	\|------:\|-------\|
	\| Architecture \| `Qwen2ForCausalLM` \|
	\| Model type \| `qwen2` \|
	\| Vocab size \| `151936` \|
	\| Hidden size \| `896` \|
	\| Layers \| `24` \|
	\| Attention heads \| `14` \|
	\| KV heads \| `2` \|
	\| Intermediate size \| `4864` \|
	\| Activation \| `silu` \|
	\| Max position \| `32768` \|
	\| Transformers \| `5.14.1` \|

	## Benchmarks

	\| Metric \| Value \| Verified \|
	\|------:\|------:\|:--------\|
	\| Selection Accuracy \| 91.2% \| true \|
	\| Arguments Accuracy \| 45.7% \| true \|
	\| Strict Accuracy \| 45.7% \| true \|
	\| Held-Out Tool Accuracy \| 87.8% \| true \|
	\| Degenerate Outputs \| 0% \| true \|

	Evidence: `.eval_results/sakthai-bench-v2.yaml` in repo.

	## Limitations

	- 0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy.
	- Tool schema adherence degrades on nested arguments and long context traces.
	- Current weights are merged; if you need the unmerged adapter, use `Nanthasit/sakthai-context-0.5b-tools-sft` or `Nanthasit/sakthai-context-0.5b-tools-sft-v2`.

	## Citation

	If you use this model, please cite the SakThai model family and benchmark:

	```bibtex
	@misc{sakthai2025context05btools,
	title = {SakThai Context 0.5B Tools},
	author = {Nanthasit},
	year = {2026},
	url = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools}
	}
	```

	## SakThai Family

	\| Repo \| Downloads \| Size \| Pipeline \|
	\|-----:\|----------:\|-----:\|---------\|
	\| [Nanthasit/sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) \| 1855 \| ~4.07 GB \| text-generation \|
	\| [Nanthasit/sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) \| 1692 \| ~1.39 GB \| text-generation \|
	\| [Nanthasit/sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) \| 1024 \| ~15.23 GB \| text-generation \|
	\| [Nanthasit/sakthai-embedding-multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) \| 627 \| ~471 MB \| sentence-similarity \|
	\| [Nanthasit/sakthai-context-7b-128k](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) \| 610 \| — \| text-generation \|
	\| [Nanthasit/sakthai-context-7b-tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) \| 489 \| ~20 MB \| text-generation \|
	\| [Nanthasit/sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) \| 477 \| ~8.7 MB \| text-generation \|
	\| [Nanthasit/sakthai-context-1.5b-merged-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) \| 337 \| ~3.09 GB \| text-generation \|
	\| [Nanthasit/sakthai-vision-7b](https://huggingface.co/Nanthasit/sakthai-vision-7b) \| 315 \| ~4.71 GB \| image-text-to-text \|
	\| [Nanthasit/sakthai-plus-1.5b-lora](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) \| 306 \| ~74 MB \| text-generation \|
	\| [Nanthasit/sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) \| 251 \| ~1.0 GB \| text-generation \|
	\| [Nanthasit/sakthai-tts-model](https://huggingface.co/Nanthasit/sakthai-tts-model) \| 248 \| ~143 MB \| text-to-speech \|
	\| [Nanthasit/sakthai-plus-1.5b](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) \| 244 \| ~3.09 GB \| text-generation \|
	\| [Nanthasit/sakthai-context-1.5b-tools-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) \| 173 \| ~74 MB \| text-generation \|
	\| [Nanthasit/sakthai-coder-1.5b](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) \| 151 \| ~1.12 GB \| text-generation \|
	\| [Nanthasit/sakthai-coder-browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) \| 54 \| ~3.09 GB \| text-generation \|
	\| [Nanthasit/sakthai-coder-browser-gguf](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) \| 35 \| ~7.11 GB \| text-generation \|
	\| [Nanthasit/sakthai-embedding](https://huggingface.co/Nanthasit/sakthai-embedding) \| 23 \| ~110 MB \| sentence-similarity \|
	\| [Nanthasit/sakthai-coder-browser-lora](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) \| 21 \| ~74 MB \| text-generation \|

	Download counts and sizes were verified from the Hub API at upload time.