Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+architecture.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -130,7 +130,7 @@ doc = converter.convert(source=source).document
 
 print(doc.export_to_markdown())
 
-````
+```
 </details>
 
 
@@ -139,7 +139,7 @@ Alternatively, you can use bare **transformers**, **vllm**, **onnx** or **mlx-vl
 <details>
 <summary>📄 Single page image inference using plain 🤗 tranformers 🤖</summary>
 
-
+```python
 # Run batch inference
 start_time = time.time()
 outputs = llm.generate(batched_inputs, sampling_params=sampling_params)
@@ -196,7 +196,11 @@ The evaluation can be performed using the [docling-eval](https://github.com/docl
     </tr>
     <tr>
       <td><b>granite-docling-258m</b></td>
-      <td><b>0.27</b></td><td><b>0.86</b></td><td><b>0.92</b></td><td><b>0.88</b></td>
+      <td>0.27</td><td>0.86</td><td>0.92</td><td>0.88</td>
+    </tr>
+    <tr>
+      <td><b>granite-docling-2stage_258m</b></td>
+      <td><b>0.31</b></td><td><b>0.90</b></td><td><b>0.93</b></td><td><b>0.92</b></td>
     </tr>
   </tbody>
 </table>
@@ -222,98 +226,18 @@ The evaluation can be performed using the [docling-eval](https://github.com/docl
     </tr>
     <tr>
       <td><b>granite-docling-258m</b></td>
-      <td><b>0.45</b></td><td><b>0.84</b></td><td><b>0.91</b></td>
-      <td><b>0.83</b></td><td><b>0.65</b></td><td><b>0.72</b></td>
+      <td>0.45</td><td>0.84</td><td>0.91</td>
+      <td><b>0.83</b></td><td>0.65</td><td>0.72</td>
     </tr>
-  </tbody>
-  <thead>
-    <tr><th colspan="7"><b>Code Recognition</b></th></tr>
     <tr>
-      <th></th>
-      <th>Edit-distance ↓</th>
-      <th>F1 ↑</th>
-      <th>Precision ↑</th>
-      <th>Recall ↑</th>
-      <th>BLEU ↑</th>
-      <th>Meteor ↑</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td><b>smoldocling-256m-preview</b></td>
-      <td>0.114</td><td>0.915</td><td>0.94</td><td>0.909</td><td>0.875</td><td>0.889</td>
-    </tr>
-    <tr>
-      <td><b>granite-docling-258m</b></td>
-      <td><b>0.013</b></td><td><b>0.988</b></td><td><b>0.99</b></td><td><b>0.988</b></td>
-      <td><b>0.983</b></td><td><b>0.986</b></td>
-    </tr>
-  </tbody>
-  <thead>
-    <tr><th colspan="7"><b>Equation Recognition</b></th></tr>
-    <tr>
-      <th></th>
-      <th>Edit-distance ↓</th>
-      <th>F1 ↑</th>
-      <th>Precision ↑</th>
-      <th>Recall ↑</th>
-      <th>BLEU ↑</th>
-      <th>Meteor ↑</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td><b>smoldocling-256m-preview</b></td>
-      <td>0.119</td><td>0.947</td><td>0.959</td><td>0.941</td><td>0.824</td><td>0.878</td>
-    </tr>
-    <tr>
-      <td><b>granite-docling-258m</b></td>
-      <td><b>0.073</b></td><td><b>0.968</b></td><td><b>0.968</b></td><td><b>0.969</b></td>
-      <td><b>0.893</b></td><td><b>0.927</b></td>
-    </tr>
-  </tbody>
-</table>
-<table>
-  <thead>
-    <tr><th colspan="3"><b>Table Recognition (FinTabNet 150dpi)</b></th></tr>
-    <tr>
-      <th></th>
-      <th>TEDS (structure) ↑</th>
-      <th>TEDS (w/content) ↑</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td><b>smoldocling-256m-preview</b></td>
-      <td>0.82</td><td>0.76</td>
-    </tr>
-    <tr>
-      <td><b>granite-docling-258m</b></td>
-      <td><b>0.97</b></td><td><b>0.96</b></td>
-    </tr>
-  </tbody>
-</table>
-<table>
-  <thead>
-    <tr><th colspan="3"><b>Other Benchmarks</b></th></tr>
-    <tr>
-      <th></th>
-      <th>MMStar ↑</th>
-      <th>OCRBench ↑</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td><b>smoldocling-256m-preview</b></td>
-      <td>0.17</td><td>338</td>
-    </tr>
-    <tr>
-      <td><b>granite-docling-258m</b></td>
-      <td><b>0.30</b></td><td><b>500</b></td>
+      <td><b>granite-docling-2stage_258m</b></td>
+      <td><b>0.27</b></td><td><b>0.85</b></td><td><b>0.92</b></td>
+      <td><b>0.83</b></td><td><b>0.70</b></td><td><b>0.79</b></td>
     </tr>
   </tbody>
 </table>
 
+
 💻 Local inference on Apple Silicon with MLX: [see here](https://huggingface.co/ibm-granite/granite-docling-258M-mlx)
 
 ## Supported Instructions
@@ -370,6 +294,8 @@ The evaluation can be performed using the [docling-eval](https://github.com/docl
 
 # Model Architecture:
 
+<img src="https://huggingface.co/docling-project/granite-docling-2stage-258m/resolve/main/architecture.png" alt="2 stage architecutre" style="width: 500px; height: auto; margin-right: 20px;">
+
 The architecture of granite-docling-258m consists of the following components:
 
 (1) RT-DETRS object detector: [docling-layout-heron](https://huggingface.co/docling-project/docling-layout-heron).
@@ -444,4 +370,3 @@ Its training, which includes both human-annotated and synthetic data informed by
 
    llm = LLM(model=MODEL_PATH, revision="untied", limit_mm_per_prompt={"image": 1}, dtype="float32")
    ```
-````