Tabby-Prompt

A prompt module for the frozen Tabby-Pretrain time-series backbone. 729,286 trainable parameters — 0.50 % of the 145.8 M backbone, which is never updated.

Which repository do you want? Three are published:

Repository	What it is
Tabby	the full post-trained model, backbone and prompt merged — load it and forecast
Tabby-Pretrain	the frozen backbone alone, for zero-shot use or as a base to adapt
Tabby-Prompt (this one)	the prompt module alone, ~3 MB, for people who already hold the backbone

If you just want to run Tabby, use Tabby. This repository is for reproducing the adaptation or swapping prompts against a backbone you already have.

The prompt is prepended to the patch-token sequence and is built as

P(x) = P_shared + sigma(g) * P_adaptive(x)

P_shared is a set of 160 learned vectors. P_adaptive(x) is generated from summary statistics of the input window through a rank-4 map, so the prompt depends on the series rather than being one global prefix. The context window is split into 16 adaptive segments and a cross-attention refiner lets each prompt slot read them, which is what makes the prompt sensitive to where in the history the information sits. g is a learned scalar gate, initialised at sigma(+5) ≈ 0.993.

Results

GIFT-Eval, 97 configs, and the TIME benchmark, 98 out-of-pool tasks. All values are Seasonal-Naive normalised geometric means at context_length=8096.

	GIFT-Eval MASE	GIFT-Eval CRPS	TIME MASE	TIME CRPS
Backbone, zero-shot	0.7192	0.4922	0.6915	0.5715
Tabby-Prompt (this checkpoint, seed 4)	0.6966 (−3.14 %)	0.4806 (−2.36 %)	0.6770 (−2.10 %)	0.5644 (−1.24 %)

Two further seeds trained with the identical recipe reached GIFT-Eval 0.6983 / 0.4827 (seed 42) and 0.7018 / 0.4826 (seed 8).

Run-to-run variation is real. With identical settings and different seeds the aggregate MASE spans roughly 0.698–0.710, so a single number should not be read as precise. This checkpoint is the seed-4 run. Report a seed mean with its spread rather than this row alone.

Files

File	What it is
tabby_prompt_plen160_seed4.pt	the prompt module, 3.4 MB, 25 tensors
config.json	a readable summary of the settings embedded in the .pt

The .pt carries everything needed to rebuild the prompt — length, gate init, segment settings, context length — under its config key, so you do not re-specify the architecture. Only --context_length is passed on the command line, because it defines the evaluation protocol rather than the model.

Checkpoint provenance: epoch=19, val_loss=1.975004, seed=4. The config.pretrain_ckpt field has been rewritten from its original cluster path to the public backbone reference.

Usage

A prompt is tied to the backbone it was trained on. This one requires Tabby-Pretrain (pretraining step 165000); an earlier or later backbone will not reproduce the numbers and will not raise an error.

python evaluate.py \
    --mode prompt --ckpt tabby_prompt_plen160_seed4.pt \
    --pretrain_ckpt <path to Tabby-Pretrain snapshot> --pretrain_code <path to the code> \
    --gift_eval_repo <gift-eval checkout> --dataset_properties dataset_properties.json \
    --context_length 8096 --batch_size 256 \
    --datasets all --out_csv tabby_seed4

Expected startup, which confirms the checkpoint was understood:

Backbone loaded: step=-1
Loading prompt checkpoint: .../tabby_prompt_plen160_seed4.pt
Loaded: epoch=19, val_loss=1.97...

and at the end:

[SN-agg] geo-mean MASE=0.6966 CRPS=0.4806 over 97 configs (patchtstfm_prompted, ctx 8096)

If the number does not match, check in this order: --context_length is 8096; the backbone is the 175000-step one; the [SN-agg] line covers 97 configs and not fewer. Those three account for essentially every mismatch. Differences of ±0.001 between machines come from non-deterministic kernels.

License

CC BY-NC 4.0 — non-commercial use only.