Upload bench_content.py
Browse files- bench_content.py +39 -0
bench_content.py
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
TAB_BENCH = """## π Benchmark Results
|
| 2 |
+
|
| 3 |
+
### ποΈ Oxford Spires β Large-Scale Trajectory Estimation (Sparse: 320 frames)
|
| 4 |
+
|
| 5 |
+
| Method | Type | AUC@15 β | AUC@30 β | ATE β | FPS |
|
| 6 |
+
|---|---|---|---|---|---|
|
| 7 |
+
| VGGT | Offline | 23.84 | 35.09 | 24.78 | β |
|
| 8 |
+
| DA3 | Offline | 49.84 | 56.68 | 12.87 | β |
|
| 9 |
+
| Pi3 | Offline | 38.64 | 48.65 | 14.03 | β |
|
| 10 |
+
| VIPE | Optim | 45.35 | 51.88 | 10.52 | β |
|
| 11 |
+
| DroidSLAM | Optim | 8.58 | 21.41 | 21.84 | β |
|
| 12 |
+
| CUT3R | Online | 5.98 | 14.95 | 18.16 | 29.2 |
|
| 13 |
+
| TTT3R | Online | 13.92 | 25.90 | 19.35 | 29.0 |
|
| 14 |
+
| Wint3R | Online | 11.61 | 23.42 | 21.10 | 3.9 |
|
| 15 |
+
| **LingBot-Map** | **Online** | **61.64** | **75.16** | **6.42** | **20.3** |
|
| 16 |
+
|
| 17 |
+
> π LingBot-Map beats ALL offline, optimization, and online methods despite being streaming!
|
| 18 |
+
|
| 19 |
+
### Dense Setting (3,840 frames) β Drift Resistance
|
| 20 |
+
|
| 21 |
+
| Method | ATE sparse β | ATE dense β | ΞATE |
|
| 22 |
+
|---|---|---|---|
|
| 23 |
+
| CUT3R | 18.16 | 32.47 | +14.31 |
|
| 24 |
+
| Wint3R | 21.10 | 32.90 | +11.80 |
|
| 25 |
+
| **LingBot-Map** | **6.42** | **7.11** | **+0.69** |
|
| 26 |
+
|
| 27 |
+
> LingBot-Map's ATE barely increases over 12Γ longer sequences!
|
| 28 |
+
|
| 29 |
+
### 3D Reconstruction (F1 Score)
|
| 30 |
+
|
| 31 |
+
| Method | ETH3D | 7-Scenes | NRGBD |
|
| 32 |
+
|---|---|---|---|
|
| 33 |
+
| Wint3R | 77.28 | 78.81 | 56.96 |
|
| 34 |
+
| TTT3R | 68.48 | 77.25 | 53.55 |
|
| 35 |
+
| Stream3R | 72.87 | 78.79 | 54.07 |
|
| 36 |
+
| **LingBot-Map** | **98.98** | **80.39** | **64.26** |
|
| 37 |
+
|
| 38 |
+
> ETH3D F1 of **98.98** β nearly perfect reconstruction!
|
| 39 |
+
"""
|