Instructions to use tensorblock/OLMo-7B-hf-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tensorblock/OLMo-7B-hf-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/OLMo-7B-hf-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/OLMo-7B-hf-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/OLMo-7B-hf-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/OLMo-7B-hf-GGUF:Q2_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf tensorblock/OLMo-7B-hf-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama cli -hf tensorblock/OLMo-7B-hf-GGUF:Q2_K
Use Docker
docker model run hf.co/tensorblock/OLMo-7B-hf-GGUF:Q2_K
- Llama new
- LM Studio
- Jan
- Ollama
How to use tensorblock/OLMo-7B-hf-GGUF with Ollama:
ollama run hf.co/tensorblock/OLMo-7B-hf-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use tensorblock/OLMo-7B-hf-GGUF with Docker Model Runner:
docker model run hf.co/tensorblock/OLMo-7B-hf-GGUF:Q2_K
- Lemonade
How to use tensorblock/OLMo-7B-hf-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tensorblock/OLMo-7B-hf-GGUF:Q2_K
Run and chat with the model
lemonade run user.OLMo-7B-hf-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
Remove .gguf files (keep Q2_K.gguf)
Browse files- OLMo-7B-hf-Q3_K_L.gguf +0 -3
- OLMo-7B-hf-Q3_K_M.gguf +0 -3
- OLMo-7B-hf-Q3_K_S.gguf +0 -3
- OLMo-7B-hf-Q4_0.gguf +0 -3
- OLMo-7B-hf-Q4_K_M.gguf +0 -3
- OLMo-7B-hf-Q4_K_S.gguf +0 -3
- OLMo-7B-hf-Q5_0.gguf +0 -3
- OLMo-7B-hf-Q5_K_M.gguf +0 -3
- OLMo-7B-hf-Q5_K_S.gguf +0 -3
- OLMo-7B-hf-Q6_K.gguf +0 -3
- OLMo-7B-hf-Q8_0.gguf +0 -3
OLMo-7B-hf-Q3_K_L.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:52775e46097789987444de7175e5b927735952fdb5bfe3f63a1e3751817825f8
|
| 3 |
-
size 3690804320
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q3_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:db4fb74607eea755b540337819f87b65fa6cf1b96495c094b83580b69804a51e
|
| 3 |
-
size 3391698016
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q3_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:07329e152d4146abdd8b993dde4453e122688d89be04c836d941b39472e195b5
|
| 3 |
-
size 3041997920
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q4_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:fb344be53db4a85f0561d020f705b7ef86ccc2bb6c2ed31c45497b59262d0379
|
| 3 |
-
size 3929457760
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q4_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:a584d4f6bfbab9f433af5ca17676c119d50a0a38e1ce5c27c8d01ecc4cd39487
|
| 3 |
-
size 4184654944
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q4_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:6b8d640041aa5a2cf145da3709f547dfafac2df0a03ebff802998b90d385a309
|
| 3 |
-
size 3960390752
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q5_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:813c23c26113b5ed0137a46d41c73dc601c547c7a31ad8b4ff67d9f357fa0212
|
| 3 |
-
size 4764714080
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q5_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:cbbe4a35184c46f1029eb13fc4d7d0de33d7daa09fb084f0697118b49322aa46
|
| 3 |
-
size 4896179296
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q5_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:20c2ad4ad0e64ff2d38cab444e1417d4e9842896dce2dc4e1c3c1a7ab85424fd
|
| 3 |
-
size 4764714080
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q6_K.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:f827f9d8d4b905b9caf7a9208f0fa026530fb6f86e42ffa1c4a5ca7777fd1dfe
|
| 3 |
-
size 5652173920
|
|
|
|
|
|
|
|
|
|
|
|
OLMo-7B-hf-Q8_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:0425f49eb3fb6120f2ed3cd4f98b3ee067f58c05d64f82711c32d3c1fbb849b6
|
| 3 |
-
size 7320384608
|
|
|
|
|
|
|
|
|
|
|
|