Update README.md

README.md

@@ -1,3 +1,60 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+tags:
+- text-generation
+- instruction-following
+- smollm
+- distillation
+- smoltalk
+language:
+- en
+pipeline_tag: text-generation
+---
+
+# Distil-SmolLM2-135M
+
+**Distil-SmolLM2-135M** is a distilled version of [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct), trained on a filtered subset of the [Smoltalk dataset](https://huggingface.co/datasets/HuggingFaceTB/smoltalk). This release aims to provide a more capable and performant ultra-small 135M generative large language model for small tasks on-edge or at-scale.
+
+## Model Details
+
+*   **Student Model:** [SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) (this model)
+*   **Teacher Model:** [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)
+*   **Parameters:** ~135 Million
+*   **Language:** English
+*   **Description:** This model was created by distilling the knowledge from `SmolLM2-1.7B-Instruct` into `SmolLM2-135M-Instruct`. The distillation process utilized the Smoltalk dataset, with specific exclusions.
+
+## Intended Uses & Limitations
+
+**Intended Uses:**
+This model is intended for research, experimentation, and general use in instruction-following and chat applications where a smaller model footprint is desired. It can be used for:
+*   Answering questions based on provided context.
+*   Classifying text.
+*   Simple conversational tasks.
+*   More complex tasks upon further fine-tuning.
+
+**Limitations:**
+*   **Reduced Capacity:** Being a smaller model (135M parameters), its performance will generally be significantly lower than its larger teacher model (1.7B parameters) and other state-of-the-art large language models, especially on complex reasoning or knowledge-intensive tasks.
+*   **Hallucinations:** Like all LLMs, this model can generate incorrect or nonsensical information (hallucinate).
+*   **Bias:** The model may reflect biases present in the training data.
+*   **Safety:** The model has not undergone extensive safety fine-tuning or alignment beyond the original instruction tuning of the teacher. It may generate harmful, unethical, or offensive content. Use with caution and appropriate safeguards.
+*   **Dataset Exclusions:** The model was not trained on the `apigen-80k` or `longalign` sources from Smoltalk, which might affect its performance on tasks related to function calling or very long context alignment.
+
+## How to Get Started
+
+You can use this model with the `transformers` library for text generation tasks.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "OxxoCodes/distil-SmolLM2-135M-Instruct"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+# Make sure to use a prompt format the model was trained on
+# This example uses a generic instruction format.
+# Refer to SmolLM2-1.7B-Instruct or SmolLM2-135M-Instruct for specific prompt templates if applicable.
+prompt = f"<|im_start|>system\nYou are a helpful AI assistant.<|im_end|>\n<|im_start|>user\nWhat is the world's largest sea mammal?\n<|im_end|>\n<|im_start|>assistant\n"
+inputs = tokenizer(prompt, return_tensors="pt")
+
+outputs = model.generate(**inputs, max_new_tokens=50, num_return_sequences=1, temperature=0.3, top_p=0.95)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))