Upload README.md with huggingface_hub

README.md

@@ -1,16 +1,219 @@
 ---
-base_model: mlx-community/Qwen2.5-Coder-7B-Instruct-4bit
-language:
-- en
-library_name: mlx
 license: apache-2.0
-license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE
-pipeline_tag: text-generation
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
 tags:
-- code
-- codeqwen
-- chat
-- qwen
-- qwen-coder
-- mlx
+  - laravel
+  - php
+  - code-generation
+  - mlx
+  - lora
+  - fine-tuned
+datasets:
+  - fchis/Laravel-13x-Code-Instructions
+language:
+  - en
+pipeline_tag: text-generation
+library_name: mlx
+model-index:
+  - name: Laravel-13x-Qwen2.5-Coder-7B-Instruct-LoRA
+    results: []
 ---
+
+# Laravel 13.x Qwen2.5-Coder-7B-Instruct LoRA
+
+A 7B code-specialized model fine-tuned on Laravel 13.x instruction-to-code pairs. **Trained entirely on an Apple M2 Pro with 16GB RAM** using Apple's MLX framework — no cloud GPU needed.
+
+Part of a two-model pipeline for AI-assisted Laravel code generation. See [github.com/florinel-chis/laravel-ai-gen](https://github.com/florinel-chis/laravel-ai-gen) for the complete tool.
+
+## Proven: Fine-Tuning Works
+
+A/B comparison of base vs fine-tuned model on the same prompts:
+
+| Behavior | Base Qwen2.5-Coder-7B | This model (fine-tuned) |
+|----------|----------------------|------------------------|
+| Output format | "Certainly! Below is a clean..." prose + markdown | **Pure PHP code, no prose** |
+| Token efficiency | ~500 tokens for incomplete output with explanation | **Complete files in 100-200 tokens** |
+| Migration style | Old named class `class CreateContactsTable` | **Modern `return new class extends Migration`** |
+| Validation syntax | N/A (never reached code in 500 tokens) | **Array syntax `['required', 'max:255']`** |
+| Controller pattern | N/A (spent all tokens on "Step 1: Create the Model...") | **FormRequest injection + `response()->json()`** |
+
+The base model literally never produces usable code at 500 max_tokens because it spends all tokens on explanation. The fine-tuned model produces complete, runnable PHP files.
+
+## Test Results: 4/4 Perfect on Specific Instructions
+
+| Instruction | Output | PHP Valid |
+|------------|--------|-----------|
+| "Create model for Contact with fillable: name, email, subject, message" | Exact 4 fields, no extras, no relationships | **Yes** |
+| "Write migration for contacts with name, email, subject, message" | Full `return new class extends Migration` wrapper | **Yes** |
+| "Create Form Request with validation rules" | Modern array syntax, correct 4 rules | **Yes** |
+| "Write ContactController with store using StoreContactRequest, return 201" | FormRequest injection, `Contact::create`, 201 JSON | **Yes** |
+
+### Sample output: Controller
+
+**Prompt:** "Write a Laravel ContactController with a store method that uses StoreContactRequest and creates a Contact. Return 201 JSON."
+
+```php
+<?php
+
+namespace App\Http\Controllers;
+
+use App\Http\Requests\StoreContactRequest;
+use App\Models\Contact;
+
+class ContactController extends Controller
+{
+    public function store(StoreContactRequest $request)
+    {
+        $contact = Contact::create($request->validated());
+
+        return response()->json($contact, 201);
+    }
+}
+```
+
+### Sample output: Model
+
+**Prompt:** "Create a Laravel Eloquent model for Contact with fillable fields: name, email, subject, message."
+
+```php
+<?php
+
+namespace App\Models;
+
+use Illuminate\Database\Eloquent\Model;
+
+class Contact extends Model
+{
+    protected $fillable = [
+        'name',
+        'email',
+        'subject',
+        'message',
+    ];
+}
+```
+
+## 7B vs 3B Comparison
+
+Both trained on the same 139 examples:
+
+| Aspect | 3B (LoRA) | **7B (LoRA)** |
+|--------|-----------|---------------|
+| Form Request | Pipe syntax `'required\|max:255'` | **Array syntax `['required', 'max:255']`** |
+| Controller | Minor `\response` return type issue | **Clean, no issues** |
+| Training peak | 5.8 GB | **7.6 GB** |
+| Inference speed | ~1-2s per file | ~12-14s per file |
+| Quality | Good | **Better** |
+
+The 7B is recommended when quality matters. The 3B is 6x faster for iteration.
+
+## Real Laravel Test: Contact Form
+
+Tested in a real Laravel 13.x project with `php artisan serve`:
+
+| Endpoint | Status | Result |
+|----------|--------|--------|
+| POST /api/contact (valid data) | **201 Created** | Contact saved to database |
+| POST /api/contact (missing fields) | **422 Validation** | Correct error messages |
+| POST /api/contact (invalid email) | **422 Validation** | "must be a valid email address" |
+
+Migration + Form Request files were usable as-is. Model and Controller needed fixes when instructions came from the planner (vague), but were perfect when given specific instructions directly.
+
+## How to Use
+
+### Standalone (specific instructions)
+
+```python
+from mlx_lm import load, generate
+
+model, tok = load("fchis/Laravel-13x-Qwen2.5-Coder-7B-Instruct-LoRA")
+
+messages = [
+    {"role": "system", "content": "You are a senior Laravel developer. Write clean, production-ready Laravel code."},
+    {"role": "user", "content": "Create a Laravel Eloquent model for Contact with fillable fields: name, email, subject, message."}
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+response = generate(model, tok, prompt=text, max_tokens=500)
+print(response)
+```
+
+### Two-model pipeline (feature descriptions)
+
+```bash
+pip install mlx-lm
+git clone https://github.com/florinel-chis/laravel-ai-gen
+cd laravel-ai-gen
+
+# Use 7B coder for best quality
+python3 laravel-gen.py --model 7b "simple contact form - name, email, subject, message"
+```
+
+## Model Details
+
+| Detail | Value |
+|--------|-------|
+| **Base model** | [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) (4-bit via MLX) |
+| **Fine-tuning** | LoRA, 16 layers, batch=1, seq=768, mask-prompt |
+| **Training data** | 139 instruction-to-code pairs from Laravel 13.x docs |
+| **Training hardware** | Apple M2 Pro, 16GB RAM |
+| **Training time** | ~5 minutes |
+| **Peak memory** | 7.6 GB |
+| **Best val loss** | 0.503 (iter 50) |
+| **Trainable params** | 11.534M / 7,615.617M (0.151%) |
+
+## Training Data
+
+Dataset: [fchis/Laravel-13x-Code-Instructions](https://huggingface.co/datasets/fchis/Laravel-13x-Code-Instructions) (139 examples)
+
+Covers: Eloquent models (with and without relationships), migrations (full class wrappers), Form Requests, controllers (short, with FormRequest injection), API Resources, routes, middleware, Blade templates, View Composers.
+
+Key training data decisions:
+- All migration examples have full `return new class extends Migration` wrapper (no bare Schema calls)
+- Simple model examples WITHOUT relationships (prevents hallucinating unwanted belongsTo)
+- Controller examples use FormRequest injection (not inline Validator)
+- Attribute-based controller examples removed (toxic at small scale — caused hallucinated imports)
+- 8 instruction-grounded model examples (exact field matching)
+
+## Limitations
+
+- **Works best with specific instructions** — list exact fields, relationships, methods
+- **Vague instructions cause hallucination** — "fillable fields" without listing them triggers pretraining archetypes
+- **Repetition loops on long outputs** — models/controllers >300 tokens may enter loops. Cap at 500-600 tokens.
+- **139 training examples** — more data (500+) would improve instruction following
+- **MLX format** — requires Apple Silicon Mac for inference
+- **7B is slower** — ~12-14s per file vs ~1-2s for the 3B variant
+
+### The Repetition Problem
+
+Small models (3B-7B) can enter repetition loops on longer outputs. Root causes:
+1. Vague instructions trigger pretraining archetypes (the model "remembers" generic patterns)
+2. Autoregressive generation has no self-correction — once in a loop, it can't escape
+
+Mitigations: specific instructions, max token caps, post-processing repetition detection.
+
+## Also Tested: DoRA
+
+DoRA (LoRA + magnitude decomposition) was tested but OOM'd at 16 layers on 7B (>10.5 GB). At 8 layers it ran but produced worse output than 16-layer LoRA. On 16GB Mac, **LoRA with more layers beats DoRA with fewer layers**.
+
+## Companion Models
+
+| Model | Params | Role | Link |
+|-------|--------|------|------|
+| Planner | 1.7B | Decompose features into tasks | [fchis/Laravel-13x-Planner-Qwen3-1.7B-LoRA](https://huggingface.co/fchis/Laravel-13x-Planner-Qwen3-1.7B-LoRA) |
+| Coder (3B) | 3B | Generate code (faster, lighter) | [fchis/Laravel-13x-Qwen2.5-Coder-3B-Instruct-LoRA](https://huggingface.co/fchis/Laravel-13x-Qwen2.5-Coder-3B-Instruct-LoRA) |
+| **Coder (7B)** | **7B** | **Generate code (best quality)** | **You're here** |
+| CLI tool | — | End-to-end pipeline | [github.com/florinel-chis/laravel-ai-gen](https://github.com/florinel-chis/laravel-ai-gen) |
+
+## Research Context
+
+Built as part of a 21-step research investigation into local AI-assisted code generation on consumer hardware. Key discoveries:
+
+- 90 instruction-to-code pairs beat 14,000 doc-Q&A pairs (training format > quantity)
+- Fine-tuned 7B produces perfect code when given specific instructions
+- The planner→coder two-model pipeline runs in ~4 GB total on any 8GB+ Mac
+- GRPO is ineffective for this use case (failures are semantic, not syntactic)
+- DoRA is better than LoRA per-layer but needs more memory — LoRA wins when memory-constrained
+
+## License
+
+Apache 2.0 (same as base model)