TinyTalk release

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,27 @@
+MIT License
+
+Copyright (c) 2026 REZOR
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+This license covers the source code in this repository. The generated model
+artifacts (model_data.cpp, tok_data.cpp, embed/*.bin) additionally derive
+from third-party works — see NOTICE.md for their licenses and attributions.

NOTICE.md

@@ -0,0 +1,54 @@
+# Third-party components and attributions
+
+The source code in this repository is MIT-licensed (see LICENSE). The
+embedded model and tokenizer artifacts derive from the following works:
+
+## Base model — TinyStories-Instruct-3M
+
+- Ronen Eldan & Yuanzhi Li, *TinyStories: How Small Can Language Models Be
+  and Still Speak Coherent English?* (arXiv:2305.07759).
+- Weights: https://huggingface.co/roneneldan/TinyStories-Instruct-3M —
+  published without an explicit license tag. The companion TinyStories
+  dataset is CDLA-Sharing-1.0, which places no restrictions on results
+  (e.g. trained models). If the licensing of the base weights matters for
+  your use case, contact the model author.
+
+## Fine-tuning data
+
+- **SODA** (https://huggingface.co/datasets/allenai/soda), CC BY 4.0.
+  Kim et al., *SODA: Million-scale Dialogue Distillation with Social
+  Commonsense Contextualization* (arXiv:2212.10465). The chat fine-tune
+  embedded in `model_data.cpp` was trained on a filtered subset.
+- **TinyStoriesInstruct**
+  (https://huggingface.co/datasets/roneneldan/TinyStoriesInstruct),
+  CDLA-Sharing-1.0 (per the TinyStories dataset family).
+
+## Tokenizer
+
+- GPT-2 byte-level BPE vocabulary and merges (`vocab.json` / `merges.txt`),
+  from OpenAI's GPT-2 release (https://github.com/openai/gpt-2), MIT
+  (Modified MIT License, Copyright (c) 2019 OpenAI). Embedded here in
+  pruned, re-encoded form inside `tok_data.cpp`.
+
+## Alternative model (not embedded by default)
+
+- **Maykeye/TinyLLama-v0** (https://huggingface.co/Maykeye/TinyLLama-v0),
+  Apache-2.0. Supported by the engine via
+  `tools/convert_tinyllama_v0.py`.
+
+## Vendored keyboard driver (`main/keyboard/`)
+
+- Ported from **M5Cardputer** v1.1.1
+  (https://github.com/m5stack/M5Cardputer), MIT,
+  Copyright (c) 2025 M5Stack Technology CO LTD. Arduino GPIO/interrupt
+  calls were replaced with ESP-IDF `driver/gpio` equivalents.
+- Includes M5Stack's adaptation of the **Adafruit TCA8418** keypad driver
+  (https://github.com/adafruit/Adafruit_TCA8418), BSD, Copyright (c)
+  Limor Fried (Adafruit Industries).
+
+## Acknowledgements
+
+- The inference engine follows the structure of Andrej Karpathy's
+  **llama2.c** (https://github.com/karpathy/llama2.c), MIT. The code here
+  is an independent implementation extended with Q4_0 quantization, a
+  GPT-Neo forward path, int8 KV cache, and a flash-walking tokenizer.

README.md

@@ -0,0 +1,122 @@
+---
+license: cc-by-4.0
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+base_model: roneneldan/TinyStories-Instruct-3M
+datasets:
+- allenai/soda
+- roneneldan/TinyStoriesInstruct
+tags:
+- gpt-neo
+- tinystories
+- tiny
+- embedded
+- esp32
+- microcontroller
+- chat
+- small-talk
+---
+
+# TinyTalk — human-like small talk on a microcontroller
+
+**TinyTalk** is a ~8.3M-parameter (≈1.6M non-embedding) **GPT-Neo** chatbot built
+to do one thing: hold short, friendly, human-sounding small-talk conversations
+on **low-end hardware that can't run a normal LLM** — think an **ESP32-S3**.
+
+It is the model embedded in the [Cardputer AI](https://github.com/rezor/cardputer_ai)
+firmware, where it runs **fully offline on the device** in ~2 MB of flash after
+Q4_0 quantization. This repository hosts the full-precision PyTorch /
+`safetensors` weights so the model can be used, fine-tuned, or re-quantized on
+its own.
+
+## What it's for
+
+Most chat models assume a datacenter GPU. TinyTalk asks the opposite question:
+*how small can a model be and still feel like talking to someone?* It trades
+away knowledge, reasoning, and long context to fit on a microcontroller, keeping
+only the ability to make warm, coherent small talk:
+
+```
+User: hey, how are you?
+Bot: I am good! I played outside today. It was so much fun!
+User: nice! what did you play?
+Bot: I played with my ball. Do you want to play too?
+```
+
+Good fits: an offline conversational toy or companion on an ESP32 / handheld;
+a teaching example of an end-to-end on-device LLM; a tiny base to fine-tune for
+embedded chat. **Not** a fit: anything needing facts, instructions, reasoning,
+or safety guarantees.
+
+## What it is, technically
+
+- **Architecture:** GPT-Neo (`GPTNeoForCausalLM`) — 8 layers, hidden size 128,
+  16 heads, alternating global/local attention (window 256), learned position
+  embeddings, tied input/output embeddings, GPT-2 byte-level BPE tokenizer
+  (vocab 50257).
+- **Base:** [`roneneldan/TinyStories-Instruct-3M`](https://huggingface.co/roneneldan/TinyStories-Instruct-3M).
+- **Fine-tune:** ~70K filtered, simple-English dialogues from
+  [`allenai/SODA`](https://huggingface.co/datasets/allenai/soda), reformatted as
+  `User:`/`Bot:` turns, mixed with a slice of
+  [`TinyStoriesInstruct`](https://huggingface.co/datasets/roneneldan/TinyStoriesInstruct).
+  Loss is masked to the bot replies / story bodies, so the model never trains on
+  producing the user's turns.
+
+## Prompt format
+
+Trained on this exact format, with `<|endoftext|>` (token 50256) between
+exchanges:
+
+```
+User: <message>
+Bot: <reply><|endoftext|>
+User: <message>
+Bot:
+```
+
+Feed `User: <message>\nBot: ` and generate until `<|endoftext|>`.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("TheREZOR/TinyTalk")
+model = AutoModelForCausalLM.from_pretrained("TheREZOR/TinyTalk")
+
+prompt = "User: hi, what's your name?\nBot:"
+ids = tok(prompt, return_tensors="pt").input_ids
+out = model.generate(
+    ids, max_new_tokens=40, do_sample=True, temperature=0.7, top_k=40,
+    eos_token_id=tok.eos_token_id,
+)
+print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
+```
+
+## Honest limitations
+
+- **Kindergarten English only.** Short, simple sentences.
+- **No world knowledge.** Factual questions get friendly confabulation.
+- **Short memory.** Trained/served with a tiny context (~80 tokens on device,
+  256 max). Not instruction-following, not safe for any production use.
+- A toy/educational model — interesting because it fits on a microcontroller,
+  not because it is good.
+
+## License & attribution
+
+Released under **CC BY 4.0**, the binding term inherited from the SODA training
+data. You must retain the following attributions:
+
+- **Base model:** TinyStories-Instruct-3M — Ronen Eldan & Yuanzhi Li,
+  *TinyStories: How Small Can Language Models Be and Still Speak Coherent
+  English?* (arXiv:2305.07759). Published without an explicit license tag; the
+  TinyStories dataset family is CDLA-Sharing-1.0, which places no restriction on
+  trained models.
+- **Fine-tune data:** SODA (CC BY 4.0) — Kim et al., *SODA: Million-scale
+  Dialogue Distillation with Social Commonsense Contextualization*
+  (arXiv:2212.10465); and TinyStoriesInstruct (CDLA-Sharing-1.0).
+- **Tokenizer:** GPT-2 byte-level BPE — OpenAI GPT-2 (MIT).
+
+See `NOTICE.md` for the full provenance.

config.json

@@ -0,0 +1,52 @@
+{
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPTNeoForCausalLM"
+  ],
+  "attention_dropout": 0,
+  "attention_layers": [
+    "global",
+    "local",
+    "global",
+    "local",
+    "global",
+    "local",
+    "global",
+    "local"
+  ],
+  "attention_types": [
+    [
+      [
+        "global",
+        "local"
+      ],
+      4
+    ]
+  ],
+  "bos_token_id": 50256,
+  "classifier_dropout": 0.1,
+  "dtype": "float32",
+  "embed_dropout": 0,
+  "eos_token_id": 50256,
+  "gradient_checkpointing": false,
+  "hidden_size": 128,
+  "initializer_range": 0.02,
+  "intermediate_size": null,
+  "layer_norm_epsilon": 1e-05,
+  "max_position_embeddings": 2048,
+  "model_type": "gpt_neo",
+  "num_heads": 16,
+  "num_layers": 8,
+  "pad_token_id": null,
+  "resid_dropout": 0,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "use_cache": true,
+  "vocab_size": 50257,
+  "window_size": 256
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "5.11.0",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7510ceac1d4f2a7d00b6f2da92cdce38706faa60a96c60884819d00ef58af77
+size 33124616

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 2048,
+  "pad_token": null,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}