Duplicate from AlexAtomic/qwen36-27b-GGUF

.gitattributes

@@ -0,0 +1,49 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+benchmark.png filter=lfs diff=lfs merge=lfs -text
+hero.png filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+qwen36-27b-UD-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,138 @@
+---
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE
+thumbnail: https://huggingface.co/AlexAtomic/qwen36-27b-GGUF/resolve/main/hero.png
+base_model:
+- Qwen/Qwen3.6-27B
+base_model_relation: quantized
+quantized_by: AlexAtomic
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+- atomic-chat
+- qwen
+- qwen3
+- gguf
+- imatrix
+- quantized
+- llama.cpp
+---
+
+<center>
+
+<div style="display:flex; justify-content:center; align-items:center; gap:10px; flex-wrap:wrap;">
+<a href="https://atomic.chat"><img src="https://huggingface.co/AlexAtomic/qwen36-27b-GGUF/resolve/main/pill_atomic_v3.png" alt="Atomic Chat" width="186"></a>
+<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AlexAtomic/qwen36-27b-GGUF/resolve/main/pill_discord_v3.png" alt="Join Discord" width="184"></a>
+<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AlexAtomic/qwen36-27b-GGUF/resolve/main/pill_github_v3.png" alt="GitHub" width="141"></a>
+</div>
+
+<br/>
+
+<img src="https://huggingface.co/AlexAtomic/qwen36-27b-GGUF/resolve/main/hero.png" alt="Qwen3.6 27B" style="width:420px; max-width:100%; height:auto; margin-bottom:0.6em;"/>
+
+<div style="display:flex; justify-content:center; gap:0.5em;">
+<a href="https://huggingface.co/Qwen/Qwen3.6-27B"><strong>Base model: Qwen/Qwen3.6-27B</strong></a>
+</div>
+</center>
+
+**Qwen3.6 27B**, self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Qwen's original weights with a per-tensor importance matrix. Runs fully offline.
+
+## Highlights
+
+- **First open-weight Qwen3.6 variant**, following the Qwen3.5 series, with a focus on stability and real-world utility.
+- **Agentic coding** that handles frontend workflows and repository-level reasoning with greater fluency and precision.
+- **Thinking Preservation**, a new option to retain reasoning context from historical messages to streamline iterative development.
+- **Base model is multimodal** (vision encoder); these GGUF quants cover the text path.
+- **262,144-token native context**, extensible up to ~1,010,000 tokens.
+- **Full quant ladder** with an importance matrix on every quant over [`calibration_datav3`](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
+
+> [!NOTE]
+> These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.
+
+> [!IMPORTANT]
+> Always pass `--jinja` so the **Qwen3.6 27B chat template** is applied. Without it the model can emit malformed turns.
+
+## Model Overview
+
+| Property | Value |
+|---|---|
+| Base model | `Qwen/Qwen3.6-27B` |
+| Total parameters | 27B |
+| Layers | 64 |
+| Context length | 262,144 native, extensible up to ~1,010,000 |
+| Architecture | Causal LM with vision encoder (Gated DeltaNet + Gated Attention) |
+| This repo | GGUF quants (imatrix), text path |
+
+<img src="https://huggingface.co/AlexAtomic/qwen36-27b-GGUF/resolve/main/benchmark.png" alt="Qwen3.6 27B benchmark scores" style="width:100%; max-width:900px;"/>
+
+Scores are Qwen's published results for the base `Qwen/Qwen3.6-27B`. Quantization preserves the large majority of this; `Q4_K_M` and up sit within a point or two of full precision.
+
+
+## Choosing a quant
+
+| Quant | Size | Notes |
+|---|---|---|
+| `Q2_K` | 10.7 GB | Smallest. Minimal RAM, clear quality drop. |
+| `IQ3_M` | 12.6 GB | Beats Q3 at similar size thanks to imatrix. Best low-RAM pick. |
+| `Q3_K_M` | 13.3 GB | Low quality but usable. |
+| `Q3_K_L` | 14.3 GB | A step above Q3_K_M. |
+| `IQ4_XS` | 15.1 GB | Excellent quality for size. Recommended low-bit. |
+| `Q4_K_S` | 15.6 GB | Compact Q4, fast. |
+| **`Q4_K_M`** | 16.5 GB | **Recommended default. Best balance of size, speed and quality.** |
+| **`UD-Q4_K_XL`** | 17.5 GB | **Dynamic. Embeddings and output kept at Q8_0 for higher quality at a Q4 footprint.** |
+| `Q5_K_S` | 18.7 GB | Higher quality. |
+| `Q5_K_M` | 19.2 GB | Higher quality, low loss. |
+| `Q6_K` | 22.1 GB | Near lossless. |
+| `Q8_0` | 28.6 GB | Effectively lossless, reference quality. |
+
+> [!TIP]
+> Pick the largest file that fits your (V)RAM with room for context. `Q4_K_M` or `UD-Q4_K_XL` is the sweet spot for most setups; `Q6_K` or `Q8_0` for maximum fidelity.
+
+## Get started
+
+Run Qwen3.6 27B locally with:
+
+- **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AlexAtomic/qwen36-27b-GGUF`, pick a quant, hit **Use this model**.
+- **llama.cpp:** `llama-server -hf AlexAtomic/qwen36-27b-GGUF:Q4_K_M --jinja -c 8192`
+- **Ollama:** `ollama run hf.co/AlexAtomic/qwen36-27b-GGUF:Q4_K_M`
+- **LM Studio / Jan:** search the repo id, download any quant.
+
+## Best practices
+
+| Parameter | Value |
+|---|---|
+| temperature | 0.7 |
+| top_p | 0.8 |
+| top_k | 20 |
+| min_p | 0.0 |
+| presence_penalty | 1.5 |
+| repetition_penalty | 1.0 |
+
+Qwen's recommended Instruct (non-thinking) settings. Thinking mode for general tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0.
+
+## Run in llama.cpp
+
+```bash
+git clone https://github.com/ggerganov/llama.cpp
+cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
+cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server
+```
+
+```bash
+./llama.cpp/build/bin/llama-server \
+    -hf AlexAtomic/qwen36-27b-GGUF:UD-Q4_K_XL \
+    --jinja -ngl 99 -c 8192 -fa on
+```
+
+## How these were made
+
+1. Download `Qwen/Qwen3.6-27B` (original weights).
+2. Convert to f16 GGUF with [llama.cpp](https://github.com/ggerganov/llama.cpp).
+3. Build an importance matrix over `calibration_datav3` (100 chunks).
+4. Quantize the full ladder with `--imatrix`.
+5. `UD-Q4_K_XL` additionally pins the token-embedding and output tensors to `Q8_0`.
+
+## License
+
+Released by Qwen under the Apache 2.0 license. Quantized by Atomic Chat.
+

qwen36-27b-IQ3_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b6625343bf101af295fb3aed242452737920e19c4b3ee63d1a77916ea964b00
+size 12580873728

qwen36-27b-IQ4_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e74efeac360ccb8f3b9d623456e2e42e78da79080fd02c677141153550932df
+size 15082505728

qwen36-27b-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcb73624e8bbd8ff6c3d5a8318a16a82d8edba1a46880b4d17b59177ca5a1250
+size 10711664128

qwen36-27b-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9cc538d68590cea4d60137fc12c2d51c26f52a82b7701ce6064330499f0417fb
+size 14344775168

qwen36-27b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb125723eaf01749c7a46d758ae9f567c631825dbb1ad963e28c29d8a50c497c
+size 13301442048

qwen36-27b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0863123e9f79c1f43bb6d4f2d7bc64541a4e92f7242c3ddae649731c03cdd09
+size 16547399168

qwen36-27b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e8040b0f14cbe14a2a531ac520230f7bc9ff3a3547a786d078719efc6256ac9
+size 15586313728

qwen36-27b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbda77b6ca15963976b87b0ac8bdbb967b1b9ab7109f05a7148d4adcf278e432
+size 19231098368

qwen36-27b-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a3ed4bd488c0164c2587499a221b64f17ffba234605a3ca006a196582a02e15
+size 18679612928

qwen36-27b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cec0773975e65be7d7d3d79a3d09131f2f55a948ebb87e152ede1144e61dda03
+size 22082528768

qwen36-27b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1519d51a23a5fe06096d80fb87bb0826c19513f9c0bd142fdd586bae00f1a577
+size 28595762688

qwen36-27b-UD-Q4_K_XL.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f38d7563abb6a9dd80ae2227db831f57d641d8eae575d970f9c085df84bf800
+size 17491015168