fuhaddesmond commited on
Commit
72aa359
·
verified ·
1 Parent(s): 2560c03

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +98 -5
README.md CHANGED
@@ -1,21 +1,114 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
3
  ---
4
- This is BLIP3o-NEXT-GRPO-TexT checkpoint trained on the BLIP3o-NEXT-SFT.
5
 
 
6
 
 
7
 
8
- ### Download
 
 
 
9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  ```
 
 
 
 
11
  from huggingface_hub import snapshot_download
12
  snapshot_download(
13
- repo_id="BLIP3o/BLIP3o-NEXT-GRPO-TexT-3B",
14
  repo_type="model"
15
  )
16
  ```
17
 
18
- Clone the repo (if you haven’t already) and install the environment:
 
 
 
 
 
 
 
 
 
 
 
19
 
 
 
 
 
 
 
 
20
  ```
21
- git clone https://github.com/JiuhaiChen/BLIP3o.git
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: text-to-image
4
+ tags:
5
+ - text-to-image
6
+ - image-generation
7
+ - blip3o
8
+ - sana
9
+ - grpo
10
+ - custom-handler
11
  ---
 
12
 
13
+ # 🌌 Illuma - Truly Open Source Image Generation
14
 
15
+ Illuma is an image generation model cloned from **Salesforce/BLIP3o-NEXT-GRPO-TexT-3B** — the first truly open-source image generation model with:
16
 
17
+ - ✅ **Training code** released (Apache 2.0)
18
+ - ✅ **Datasets** released (BLIP3o-Pretrain + BLIP3o-60K)
19
+ - ✅ **No usage restrictions** (Apache 2.0 license)
20
+ - ✅ **Can be renamed, rebranded, and refined**
21
 
22
+ ## Architecture
23
+
24
+ **AR (3B Qwen2.5-VL) + SANA 1.5 Diffusion Decoder**
25
+
26
+ Illuma uses a two-stage generation process:
27
+ 1. **Autoregressive model** generates visual tokens from text prompt
28
+ 2. **SANA 1.5 diffusion decoder** converts visual tokens to a high-quality image
29
+
30
+ The GRPO (Group Relative Policy Optimization) RL training improves text rendering in generated images (GenEval 0.73 → 0.90).
31
+
32
+ ## 🚀 Deploy on Hugging Face Inference Endpoints
33
+
34
+ This model includes a **custom handler** (`handler.py`) for deployment on HF Inference Endpoints:
35
+
36
+ 1. Go to [Inference Endpoints](https://ui.endpoints.huggingface.co/)
37
+ 2. Click **"+ New endpoint"**
38
+ 3. Select **`fuhaddesmond/illuma`** as the model repository
39
+ 4. Select **AWS** → **NVIDIA T4** ($0.50/hr) or **NVIDIA A10G** ($1.00/hr)
40
+ 5. Set **Task** to **Custom**
41
+ 6. Click **Create Endpoint**
42
+ 7. Once deployed, call the API:
43
+
44
+ ```python
45
+ import requests
46
+
47
+ API_URL = "https://YOUR_ENDPOINT_ID.aws.endpoints.huggingface.cloud"
48
+ headers = {"Authorization": "Bearer hf_YOUR_TOKEN"}
49
+
50
+ payload = {
51
+ "inputs": "A neon sign that says 'ILLUMA' glowing in purple against a dark wall",
52
+ "parameters": {
53
+ "seq_len": 729,
54
+ "top_p": 0.95,
55
+ "top_k": 1200
56
+ }
57
+ }
58
+
59
+ response = requests.post(API_URL, headers=headers, json=payload)
60
+ import base64
61
+ from PIL import Image
62
+ from io import BytesIO
63
+
64
+ image_data = base64.b64decode(response.json()["image"])
65
+ image = Image.open(BytesIO(image_data))
66
+ image.save("illuma_output.png")
67
+ ```
68
+
69
+ ## 🔧 Local Inference
70
+
71
+ ```bash
72
+ # Clone BLIP3o repo (BLIP3o-NEXT branch)
73
+ git clone --branch BLIP3o-NEXT --single-branch https://github.com/JiuhaiChen/BLIP3o.git
74
+ cd BLIP3o
75
+ pip install -e .
76
+
77
+ # Download model
78
+ python -c "from huggingface_hub import snapshot_download; print(snapshot_download(repo_id='fuhaddesmond/illuma', repo_type='model'))"
79
+
80
+ # Run inference
81
+ python inference.py /path/to/downloaded/model
82
  ```
83
+
84
+ ## Download
85
+
86
+ ```python
87
  from huggingface_hub import snapshot_download
88
  snapshot_download(
89
+ repo_id="fuhaddesmond/illuma",
90
  repo_type="model"
91
  )
92
  ```
93
 
94
+ ## Model Details
95
+
96
+ | Detail | Value |
97
+ |--------|-------|
98
+ | **Base Model** | BLIP3o-NEXT-GRPO-TexT-3B |
99
+ | **Parameters** | ~4B (3B AR + diffusion decoder) |
100
+ | **Architecture** | Qwen2.5-VL + SANA 1.5 |
101
+ | **License** | Apache 2.0 |
102
+ | **GRPO Training** | GenEval 0.73 → 0.90 |
103
+ | **Specialty** | Text rendering in images |
104
+
105
+ ## Citation
106
 
107
+ ```bibtex
108
+ @article{chen2025blip3,
109
+ title={BLIP3-o: A Family of Fully Open Unified Multimodal Models},
110
+ author={Chen, Jiuhai and others},
111
+ journal={arXiv preprint arXiv:2505.09568},
112
+ year={2025}
113
+ }
114
  ```