Maxilicious20/Velqora-160M-Instruct-GGUF

Velqora-160M-Instruct — GGUF

This repository is intended for GGUF exports of Velqora-160M-Instruct, a 158,622,720-parameter English-first decoder-only language model trained from scratch on a single NVIDIA RTX 4060 8 GB GPU.

Original custom PyTorch model: Maxilicious20/Velqora-160M-Instruct.

Compatibility warning: GGUF is a file format, not a guarantee that the model can be loaded by llama.cpp, LM Studio, Ollama or other GGUF runtimes. Velqora has custom PyTorch tensor names, custom tokenizer serialization and custom chat formatting. A correct converter and validated inference implementation may be required. Compatibility is not claimed or verified by this model card.

Base architecture

Property	Value
Original model	Velqora-160M-Instruct
Original parameter count	158,622,720
Core architecture	Llama-style decoder-only Transformer with GQA, SwiGLU, RoPE and RMSNorm
Attention	16 query heads / 4 KV heads
Layers / width	12 layers / 1,024 hidden size
Maximum trained context	2,048 tokens
Tokenizer	Original 32K byte-level BPE, not a standard Meta Llama tokenizer
Training	Same 3.215 billion processed tokens as the unquantized original
Intended format	GGUF, after successful conversion and validation

Files and quantization variants

Upload only GGUF binaries that were actually generated from the completed Velqora-160M-Instruct export and validated. The following are suggested naming patterns, not claims that these files currently exist:

Example filename	Representation	Status
Velqora-160M-Instruct-F16.gguf	16-bit floating point	Upload only if converted and validated
Velqora-160M-Instruct-Q8_0.gguf	8-bit quantization	Upload only if converted and validated
Velqora-160M-Instruct-Q4_K_M.gguf	4-bit mixed K quantization	Upload only if converted and validated

No file sizes, speedups or quality-retention claims are made here, since actual GGUF artifacts and measurements were not supplied for verification. Once the files are uploaded, replace this section with the exact filenames, sizes, conversion tool version, runtime and test results.

Important conversion requirements

The original repository provides model.pt, config.json, tokenizer.json and custom velqora/model.py and velqora/tokenization.py sources.

A GGUF conversion must preserve:

1. The exact trained tensors and the weight tying relationship.
2. The model dimensions, grouped-query attention layout, RoPE convention and context length.
3. The original trained token IDs, vocabulary, byte-level BPE behavior and special tokens.
4. The assistant/user role serialization and assistant stop token.
5. Numerical agreement with the original PyTorch model on a small token-level or logits comparison, within quantization tolerance.

Do not simply rename model.pt to .gguf. Do not assume convert_hf_to_gguf.py works on Velqora unchanged: that converter normally recognizes supported Hugging Face Transformers model classes and their tensor layouts. A custom adaptation or converter may be necessary.

Inference — only after runtime validation

Once a specific GGUF binary has been confirmed to load and generate correctly in the intended runtime, a local llama.cpp invocation would look like this (replace the filename):

llama-cli -m ./ACTUAL_VALIDATED_FILE.gguf -p "Hello!" -n 128

This is an illustrative command, not evidence of working support. A raw -p prompt may also differ from Velqora's expected role template. If the runtime supports its custom chat template, verify it carefully before using conversation mode.

Original chat prompt shape:

<|im_start|>user
Hello!<|im_end|><|im_start|>assistant

Assistant completions stop at <|eot_id|> in the original PyTorch implementation. The trained maximum context window is 2,048, not 4K/8K/16K.

Evaluation and quality

The only available quantitative results are from the unquantized PyTorch model, evaluated with a local, non-standard zero-shot protocol. They must not be represented as scores for any GGUF variant until quantized inference is independently tested.

For reference, the original PyTorch evaluation recorded ARC-Easy 46.55%, ARC-Challenge 22.44%, HellaSwag 28.10% (9,609 of 10,042 tasks), BoolQ 58.26%, MMLU 23.13% (13,957 of 14,042 tasks), and VelqoraBench-400 0.00% with a strict exact-output scorer. These values are not official leaderboard-comparable results, and HellaSwag/MMLU are incomplete. See the original model card for methodology and limitations.

If quantized variants are published, record per-file validation and benchmark results rather than assuming identical accuracy to the original.

Intended use and limitations

Research, educational experiments and local inference after validation. This small experimental model may hallucinate, miscalculate, misinterpret prompts, repeat phrases and fail to follow instructions. It is not suitable for high-stakes or autonomous decision-making. Quantization can introduce additional errors. Input/output safety was not comprehensively established.

License

No model-weights license has been confirmed for this release. The license field is intentionally omitted. The publisher must choose an accurate license for the original weights and verify that the intended GGUF distribution follows it; the GGUF repository should not silently claim a more permissive license than the parent model.