Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

Misc

Inference Endpoints

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

155

Base only

Active filters: q2_k

afrideva/dolphin-2_6-phi-2_oasst2_chatML_V2-GGUF

Text Generation • 3B • Updated Jan 19, 2024 • 199 • 5

afrideva/TinyLlama-haiku-dpo-v.0.1-GGUF

Text Generation • 1B • Updated Jan 19, 2024 • 77

afrideva/tinyllama-python-GGUF

Text Generation • 1B • Updated Jan 20, 2024 • 119

afrideva/phi-2-psy-GGUF

Text Generation • 3B • Updated Jan 20, 2024 • 87 • 5

afrideva/beecoder-220M-python-GGUF

Text Generation • 0.2B • Updated Jan 20, 2024 • 254

afrideva/smol_llama-220M-openhermes-GGUF

Text Generation • 0.2B • Updated Jan 20, 2024 • 73

afrideva/TinyLlama-3T-1.1bee-GGUF

Text Generation • 1B • Updated Jan 20, 2024 • 234 • 1

afrideva/smol_llama-220M-bees-internal-GGUF

Text Generation • 0.2B • Updated Jan 20, 2024 • 44

afrideva/smol_llama-220M-open_instruct-GGUF

Text Generation • 0.2B • Updated Jan 20, 2024 • 73

afrideva/zephyr-220m-sft-full-GGUF

Text Generation • 0.2B • Updated Jan 20, 2024 • 43

afrideva/zephyr-220m-dpo-full-GGUF

Text Generation • 0.2B • Updated Jan 20, 2024 • 61

afrideva/stablelm-2-1_6b-GGUF

Text Generation • 2B • Updated Jan 22, 2024 • 228 • 3

afrideva/stablelm-2-zephyr-1_6b-GGUF

Text Generation • 2B • Updated Jan 22, 2024 • 75 • 3

afrideva/LocutusqueXFelladrin-TinyMistral248M-Instruct_oasst2_chatML_V1_DPO_V3-GGUF

Text Generation • 0.2B • Updated Jan 27, 2024 • 41

afrideva/tau-0.5B-GGUF

Text Generation • 0.6B • Updated Mar 25, 2024 • 68

afrideva/tau-0.5B-instruct-GGUF

Text Generation • 0.6B • Updated Mar 25, 2024 • 50

afrideva/NeuralReyna-Mini-1.8B-v0.3-GGUF

Text Generation • 2B • Updated Mar 25, 2024 • 131 • 1

Edmon02/deepseek-coder-7b-instruct-v1.5-GGUF

Text Generation • 7B • Updated 4 days ago • 1.16k

afrideva/gemma-1.1-2b-it_oasst_format_chatML_unsloth_V1_function_calling_V2-GGUF

Text Generation • 3B • Updated Apr 17, 2024 • 119 • 5

afrideva/Bioinspired-Phi-3-mini-4k-GGUF

Text Generation • 4B • Updated May 3, 2024 • 103

afrideva/Phi-3-mini-128k-instruct_function-GGUF

Text Generation • 4B • Updated May 8, 2024 • 132 • 1

afrideva/gemma-1.1-2b-it_orpo_traslate_en_es_V1-GGUF

Text Generation • 3B • Updated May 10, 2024 • 69

tifin-india/sarvam-m-24b-q2-k-gguf

Text Generation • 24B • Updated May 24, 2025 • 18

roylin1003/royal-zhTW-ID-q2_k.gguf

Text Generation • 8B • Updated Jun 3, 2025 • 7

QuantLLM/Llama-3.2-3B-2bit-mlx

Text Generation • 3B • Updated Dec 20, 2025 • 10

QuantLLM/Llama-3.2-3B-2bit-gguf

3B • Updated Dec 20, 2025 • 22

dahus/gemma-4-e2b-it-Q2_K-GGUF

Text Generation • 5B • Updated Apr 16 • 283 • 1

QuantLLM/Qwen3-0.6B-2bit-gguf

0.6B • Updated Apr 25 • 15

bleysg/DeepSeek-V4-Flash-IQ2XXS-Q2K-FP8-120GB-target

Text Generation • 84B • Updated 28 days ago • 806 • 3

opensota/deepseek-v4-gguf

Text Generation • 284B • Updated 27 days ago • 904 • 2