Update README.md
Browse files
README.md
CHANGED
|
@@ -40,8 +40,10 @@ The transformer runs on the Torq NPU in bf16; the token embeddings run on the ho
|
|
| 40 |
|
| 41 |
| File | Size | Role |
|
| 42 |
|---|--:|---|
|
| 43 |
-
| `model.vmfb` | 461 MB | **monolithic** build — decoder + LM head in one graph (logits output) |
|
| 44 |
-
| `body.vmfb` | 327 MB | **split** build — decoder body only, emits hidden states (pairs with `lm_head.vmfb`) |
|
|
|
|
|
|
|
| 45 |
| `lm_head.vmfb` | 134 MB | **split** build — standalone LM head (hidden → 65 536 logits) |
|
| 46 |
| `token_embeddings.npy` | 134 MB | CPU embedding lookup table (bf16) |
|
| 47 |
| `config.json` | — | model configuration |
|
|
@@ -78,15 +80,21 @@ The `onnx/` export is provided for reference / portability to other runtimes.
|
|
| 78 |
| Platform | Model / Stage | Environment | NPU Clock | Inference Time | Infer / s |
|
| 79 |
| --------- | --------- | --------- | --------- | --------- | --------- |
|
| 80 |
| SL2619 2GB | LFM2.5-230M | Torq v2.0.0 | 1 GHz | TBD | 7.1 |
|
|
|
|
| 81 |
|
| 82 |
|
| 83 |
## Deployment
|
| 84 |
|
| 85 |
The models have been tested with the following environment.
|
| 86 |
|
| 87 |
-
- Torq Compiler:
|
| 88 |
-
-
|
| 89 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
### Usage Tutorials / Example Apps
|
| 92 |
|
|
|
|
| 40 |
|
| 41 |
| File | Size | Role |
|
| 42 |
|---|--:|---|
|
| 43 |
+
| `model.vmfb` (legacy) | 461 MB | **monolithic** build — decoder + LM head in one graph (logits output) |
|
| 44 |
+
| `body.vmfb` (legacy) | 327 MB | **split** build — decoder body only, emits hidden states (pairs with `lm_head.vmfb`) |
|
| 45 |
+
| `transformer.vmfb` | 326 MB | **split** build — decoder body only, emits hidden states (pairs with `lm_head.vmfb`) |
|
| 46 |
+
| `transformer_prefill.vmfb` | 326 MB | **split** build — `transformer.vmfb` with a batch size of 64, only used for batched prefill |
|
| 47 |
| `lm_head.vmfb` | 134 MB | **split** build — standalone LM head (hidden → 65 536 logits) |
|
| 48 |
| `token_embeddings.npy` | 134 MB | CPU embedding lookup table (bf16) |
|
| 49 |
| `config.json` | — | model configuration |
|
|
|
|
| 80 |
| Platform | Model / Stage | Environment | NPU Clock | Inference Time | Infer / s |
|
| 81 |
| --------- | --------- | --------- | --------- | --------- | --------- |
|
| 82 |
| SL2619 2GB | LFM2.5-230M | Torq v2.0.0 | 1 GHz | TBD | 7.1 |
|
| 83 |
+
| SL2619 2GB | LFM2.5-230M | Torq v2.2.0 | 1 GHz | TBD | 7.5 |
|
| 84 |
|
| 85 |
|
| 86 |
## Deployment
|
| 87 |
|
| 88 |
The models have been tested with the following environment.
|
| 89 |
|
| 90 |
+
- Torq Compiler:
|
| 91 |
+
- v2.0.0
|
| 92 |
+
- v2.2.0
|
| 93 |
+
- v2.2.1
|
| 94 |
+
- Torq Runtime:
|
| 95 |
+
- *v2.0.0* included in Astra SDK release *scarthgap_6.12_v2.4.0*
|
| 96 |
+
- *v2.2.0* in Astra SDK release *scarthgap_6.12_v2.4.0*
|
| 97 |
+
- *v2.2.1* in Astra SDK release *scarthgap_6.12_v2.4.0*
|
| 98 |
|
| 99 |
### Usage Tutorials / Example Apps
|
| 100 |
|