How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf maylore-ai/Temper-2-Nodel-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf maylore-ai/Temper-2-Nodel-GGUF:BF16
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf maylore-ai/Temper-2-Nodel-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf maylore-ai/Temper-2-Nodel-GGUF:BF16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf maylore-ai/Temper-2-Nodel-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama cli -hf maylore-ai/Temper-2-Nodel-GGUF:BF16
Use Docker
docker model run hf.co/maylore-ai/Temper-2-Nodel-GGUF:BF16
Quick Links

Temper-2-Nodel-GGUF

GGUF version of Temper-2-Nodel, a 4B model for Rust code. Results and details are on the main model page.

Website: maylore.com/temper-2-nodel

File Quantization Size
Temper-2-Nodel-Q8_0.gguf Q8_0 4.48 GB
Temper-2-Nodel-BF16.gguf BF16 8.42 GB

The recommended temperature is 0.2. The Modelfile and params in this repository set it for Ollama along with top_p 0.95, the settings of the published results.

llama.cpp

llama-cli -hf maylore-ai/Temper-2-Nodel-GGUF:Q8_0 --temp 0.2 --top-p 0.95

LM Studio

LM Studio does not take the temperature from the GGUF file and starts every model at 0.8. Set it to 0.2 in the model settings.

Ollama

ollama run hf.co/maylore-ai/Temper-2-Nodel-GGUF:Q8_0

Or locally with the Modelfile from this repository.

ollama create temper-nodel -f Modelfile
ollama run temper-nodel

License

Apache 2.0, same as Qwen3.5-4B. Derivative works, including fine-tunes, merges and quantizations, must keep the attributions in NOTICE (Apache 2.0, section 4(d)).

Downloads last month
266
GGUF
Model size
4B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for maylore-ai/Temper-2-Nodel-GGUF

Finetuned
Qwen/Qwen3.5-4B
Quantized
(2)
this model