amaye15/toto-gguf

Toto-2.0 — GGUF

GGUF conversions of Datadog's Toto-2.0 time-series foundation model, all 5 published sizes × 3 dtypes (F32/F16/Q8_0) — 15 files total. Converted and run with zsfm, a Rust workspace that ports zero-shot forecasting and tabular foundation models to GGUF + candle, verified bit-exact against the original PyTorch checkpoint. No PyTorch, no Python runtime required to run inference.

Size	F32	F16	Q8_0
4M	toto-4m-f32.gguf	toto-4m-f16.gguf	toto-4m-q8.gguf
22M	toto-22m-f32.gguf	toto-22m-f16.gguf	toto-22m-q8.gguf
313M	toto-313m-f32.gguf	toto-313m-f16.gguf	toto-313m-q8.gguf
1B	toto-1b-f32.gguf	toto-1b-f16.gguf	toto-1b-q8.gguf
2.5B	toto-2.5b-f32.gguf	toto-2.5b-f16.gguf	toto-2.5b-q8.gguf

F16 is the recommended default (~2× smaller than F32, ~0.07–0.18 MAE difference on the 313M+ sizes). Q8_0 trades more accuracy for the smallest files — useful when memory, not disk, is the constraint.

There's no config.json in this repo — GGUF embeds its own architecture metadata, but the Python bindings below still need Toto's config.json (patch size, quantile levels) from Datadog's original repo for the matching size.

Context must be at least 32 timesteps (Toto's patch_size) — a shorter context fails with context too short. The examples below use a 32-value context.

Use it

Python (pip install zsfm)

pip install zsfm huggingface_hub

import zsfm
from huggingface_hub import hf_hub_download

# the GGUF weights come from this repo; config.json (patch size, quantile
# levels, etc.) comes from Datadog's original repo for the same size
gguf_path   = hf_hub_download("amaye15/toto-gguf", "toto-2.5b-f16.gguf")
config_path = hf_hub_download("Datadog/Toto-2.0-2.5B", "config.json")

model = zsfm.TotoModel(gguf_path, config_path)

# Toto's patch_size is 32 — context must be at least 32 timesteps long
context = [0.85, 0.93, 1.01, 1.09, 1.17, 1.25, 1.33, 1.06, 1.14, 1.22, 1.3, 1.38,
           1.46, 1.54, 1.27, 1.35, 1.43, 1.51, 1.59, 1.67, 1.75, 1.48, 1.56, 1.64,
           1.72, 1.8, 1.88, 1.96, 1.69, 1.77, 1.85, 1.93]

point = model.forecast(context, horizon=64)
# -> List[float], the median (q0.5) forecast

# batch: one point-forecast list per series (each series independently needs >=32 timesteps)
points = model.forecast_batch([context, context], horizon=64)

# full 9-quantile distribution (added in zsfm 0.2.7)
quantile_matrix = model.forecast_quantiles(context, horizon=64)  # 9 rows, one per level below
levels = model.quantiles()  # [0.1, 0.2, ..., 0.9] — quantile_matrix[i] is the level[i] forecast

forecast()/forecast_batch() return the median point forecast only (a plain list of floats); forecast_quantiles() returns the full q0.10–q0.90 matrix, matching the CLI's JSON output below.

Rust / CLI (cargo install zsfm)

cargo install zsfm --locked

# downloads the original Datadog weights and converts to GGUF locally
# (produces the same bytes as toto-2.5b-f16.gguf in this repo):
zsfm toto convert --dtype f16
echo '{"context": [0.85,0.93,1.01,1.09,1.17,1.25,1.33,1.06,1.14,1.22,1.3,1.38,1.46,1.54,1.27,1.35,1.43,1.51,1.59,1.67,1.75,1.48,1.56,1.64,1.72,1.8,1.88,1.96,1.69,1.77,1.85,1.93], "horizon": 64}' \
  | zsfm toto infer --gguf gguf/toto-2.5b-f16.gguf

zsfm toto convert -m Datadog/Toto-2.0-<size> --dtype <f32|f16|q8> -o gguf/toto-<size>-<dtype>.gguf reproduces any of the 15 files in this repo directly from the original weights — -m needs the full HuggingFace repo id (Datadog/Toto-2.0-4m, -22m, -313m, -1B, or -2.5B; a bare size like -m 4m will fail with a 401). Always pass -o explicitly: it defaults to gguf/toto-2.5b-f16.gguf regardless of which -m/--dtype you chose, so without it every run overwrites the same file under the wrong name. Example for the 4M Q8_0 file:

zsfm toto convert -m Datadog/Toto-2.0-4m --dtype q8 -o gguf/toto-4m-q8.gguf

To skip conversion and run a file already published here, download it directly and point --config at the matching size's original config.json:

huggingface-cli download amaye15/toto-gguf toto-2.5b-f16.gguf --local-dir .
huggingface-cli download Datadog/Toto-2.0-2.5B config.json --local-dir .
echo '{"context": [0.85,0.93,1.01,1.09,1.17,1.25,1.33,1.06,1.14,1.22,1.3,1.38,1.46,1.54,1.27,1.35,1.43,1.51,1.59,1.67,1.75,1.48,1.56,1.64,1.72,1.8,1.88,1.96,1.69,1.77,1.85,1.93], "horizon": 64}' \
  | zsfm toto infer --gguf toto-2.5b-f16.gguf --config config.json

See the usage guide for the full request/response format (batch and multivariate input, quantile output, --context-length/--f64 flags).

Source, the other 10 time-series forecasters + 5 tabular models, and full docs: amaye15/zsfm-rs.

Response format

{
  "id": "forecast-000001932b7a1234",
  "object": "forecast",
  "created": 1736290000,
  "model": "toto",
  "choices": [{
    "index": 0,
    "forecast": {
      "point": [2.1, 2.3, 2.5],
      "quantiles": {
        "0.10": [1.8, 2.0, 2.2],
        "0.50": [2.1, 2.3, 2.5],
        "0.90": [2.4, 2.6, 2.8]
      }
    },
    "finish_reason": "stop"
  }],
  "usage": {"context_length": 32, "forecast_length": 64}
}

point is the median (q0.5); all 9 quantile levels (q0.10–q0.90) are included. Pass a batch of series ("context": [[...], [...]]) for one choice per series, or a 3D array ([batch][variate][time]) for Toto's native multivariate mode, which returns a variates array per choice instead of a flat point/quantiles.

Architecture

Toto-2.0 is an encoder-only time-series transformer:

• Input: multivariate time series, instance-normalized, patched (patch_size=32), projected to d_model.
• Encoder: multi-layer causal transformer with xPos RoPE (standard RoPE plus per-dimension exponential decay, for long-range stability).
• Output: hidden states decoded through a residual block to 9 quantile forecasts (q0.1–q0.9) per variate.
• All 5 sizes (4M–2.5B) share this architecture, differing in width/depth only.

License

Conversion code: MIT (amaye15/zsfm-rs). Weights: Apache-2.0, per Datadog's original release — unrestricted, including commercial use.