Instructions to use tensorblock/SmolLM2-360M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tensorblock/SmolLM2-360M-GGUF with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("tensorblock/SmolLM2-360M-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tensorblock/SmolLM2-360M-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/SmolLM2-360M-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/SmolLM2-360M-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/SmolLM2-360M-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/SmolLM2-360M-GGUF:Q2_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf tensorblock/SmolLM2-360M-GGUF:Q2_K # Run inference directly in the terminal: ./llama-cli -hf tensorblock/SmolLM2-360M-GGUF:Q2_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf tensorblock/SmolLM2-360M-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf tensorblock/SmolLM2-360M-GGUF:Q2_K
Use Docker
docker model run hf.co/tensorblock/SmolLM2-360M-GGUF:Q2_K
- LM Studio
- Jan
- Ollama
How to use tensorblock/SmolLM2-360M-GGUF with Ollama:
ollama run hf.co/tensorblock/SmolLM2-360M-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use tensorblock/SmolLM2-360M-GGUF with Docker Model Runner:
docker model run hf.co/tensorblock/SmolLM2-360M-GGUF:Q2_K
- Lemonade
How to use tensorblock/SmolLM2-360M-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tensorblock/SmolLM2-360M-GGUF:Q2_K
Run and chat with the model
lemonade run user.SmolLM2-360M-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
Upload folder using huggingface_hub
Browse files- README.md +5 -8
- SmolLM2-360M-Q2_K.gguf +2 -2
- SmolLM2-360M-Q3_K_L.gguf +2 -2
- SmolLM2-360M-Q3_K_M.gguf +2 -2
- SmolLM2-360M-Q3_K_S.gguf +2 -2
- SmolLM2-360M-Q4_0.gguf +2 -2
- SmolLM2-360M-Q4_K_M.gguf +2 -2
- SmolLM2-360M-Q4_K_S.gguf +2 -2
- SmolLM2-360M-Q5_0.gguf +2 -2
- SmolLM2-360M-Q5_K_M.gguf +2 -2
- SmolLM2-360M-Q5_K_S.gguf +2 -2
- SmolLM2-360M-Q6_K.gguf +2 -2
- SmolLM2-360M-Q8_0.gguf +2 -2
README.md
CHANGED
|
@@ -1,15 +1,12 @@
|
|
| 1 |
---
|
| 2 |
-
base_model: unsloth/SmolLM2-360M
|
| 3 |
-
language:
|
| 4 |
-
- en
|
| 5 |
library_name: transformers
|
| 6 |
license: apache-2.0
|
|
|
|
|
|
|
| 7 |
tags:
|
| 8 |
-
- llama
|
| 9 |
-
- unsloth
|
| 10 |
-
- transformers
|
| 11 |
- TensorBlock
|
| 12 |
- GGUF
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
<div style="width: auto; margin-left: auto; margin-right: auto">
|
|
@@ -23,9 +20,9 @@ tags:
|
|
| 23 |
</div>
|
| 24 |
</div>
|
| 25 |
|
| 26 |
-
##
|
| 27 |
|
| 28 |
-
This repo contains GGUF format model files for [
|
| 29 |
|
| 30 |
The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
|
| 31 |
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
library_name: transformers
|
| 3 |
license: apache-2.0
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
tags:
|
|
|
|
|
|
|
|
|
|
| 7 |
- TensorBlock
|
| 8 |
- GGUF
|
| 9 |
+
base_model: HuggingFaceTB/SmolLM2-360M
|
| 10 |
---
|
| 11 |
|
| 12 |
<div style="width: auto; margin-left: auto; margin-right: auto">
|
|
|
|
| 20 |
</div>
|
| 21 |
</div>
|
| 22 |
|
| 23 |
+
## HuggingFaceTB/SmolLM2-360M - GGUF
|
| 24 |
|
| 25 |
+
This repo contains GGUF format model files for [HuggingFaceTB/SmolLM2-360M](https://huggingface.co/HuggingFaceTB/SmolLM2-360M).
|
| 26 |
|
| 27 |
The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
|
| 28 |
|
SmolLM2-360M-Q2_K.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:14921eebdd2adfdef2a5b73b3c45236e2e76be539728b925854372f189775f35
|
| 3 |
+
size 218673248
|
SmolLM2-360M-Q3_K_L.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:393b8f1b46c517cf86e19f1214e72aaff3c06e3febb5e9c70f838b2a7d808a32
|
| 3 |
+
size 246321248
|
SmolLM2-360M-Q3_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e0d91546eed8f6d7ee95d5e51457c9ec95a2c3c03a61fcc70d04ba5540d5ea26
|
| 3 |
+
size 234686048
|
SmolLM2-360M-Q3_K_S.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95f68f307a83b2ea71b357d310a1b2fdaecf4dd4a037e71510caf3c50b0d9150
|
| 3 |
+
size 218673248
|
SmolLM2-360M-Q4_0.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ae2e3d5220f08369729b1f0d590076d9d044aef2da30100ec20558b0043f9225
|
| 3 |
+
size 229118048
|
SmolLM2-360M-Q4_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2e8594e539a8e78d7fadb34074ccbc8ea7939d57c70c57925765db41b865e1b9
|
| 3 |
+
size 270590048
|
SmolLM2-360M-Q4_K_S.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7eba4ecbc4e09eaab8418bdb64628da22c9dcf263eea806035f7a3d3501fa37c
|
| 3 |
+
size 259914848
|
SmolLM2-360M-Q5_0.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1cb485565884c8a9c80d888dbc5f6c0dea4f2c3dcf4bb4a4a25bd4f9fd04183e
|
| 3 |
+
size 268439648
|
SmolLM2-360M-Q5_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4302e73437a93dbe803e92ad9101b71061e46004a15c0435322fcecbe11882fc
|
| 3 |
+
size 289943648
|
SmolLM2-360M-Q5_K_S.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:51441f14b05743b3c071abf7c1b95ae9fbc5c4432b07b5b6af52b49d388fd3d9
|
| 3 |
+
size 283185248
|
SmolLM2-360M-Q6_K.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e78ed6ba1d3f9818ba70d421de6f3197366acc04c2d0bdebaa306d86cc953b0d
|
| 3 |
+
size 367358048
|
SmolLM2-360M-Q8_0.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:847fd6d45eaa185623524ccdbf75acdcdfabb29e63ee1005073c95360ba12aca
|
| 3 |
+
size 386404448
|