---
license: gemma
base_model: unsloth/functiongemma-270m-it
library_name: ggml
tags:
  - gguf
  - llama-cpp
  - function-calling
  - tool-use
  - agents
language:
  - en
pipeline_tag: text-generation
---

# functiongemma-270m-relay-tools

Fine-tuned **FunctionGemma 270M** translator for
[Relay](https://github.com/baka-onii/relay): single-turn
`natural-language instruction → exactly one tool call` over the harness's
33-tool registry (filesystem, editing, execution, git, web, environment,
utility, interaction). It does **not** reason or chat — the Relay runtime owns
validation, confidence gates, and approvals; this model only selects the tool
by its exact registered name plus small args. Bulk text (file contents, code,
commands) travels in runtime payload blocks, never through this model.

Because selection is by exact tool name, misfires on concrete instructions are
negligible in practice — failures appear almost only on abstract or ambiguous
edge-case prompts (e.g. "Show develop for docs/." or "Compare X with
release/1.2"), never on everyday phrasings like "Read src/auth.py" or
"Show the last 5 commits".

## Training

- **Base:** `unsloth/functiongemma-270m-it`, full fine-tune (no LoRA), Unsloth, WSL + CUDA.
- **Hyperparameters:** 3 epochs, batch 4 / grad-accum 8 (eff. 32), lr 2e-5,
  `--save-steps 50`, max_seq_length 4096. Main run `outputs/full4`, continued as
  `outputs/full5` from `checkpoint-100`; published weights are the
  **checkpoint-200** export (~200 updates).
- **Dataset:** one atomic tool call per row (`instruction` + `tool` +
  `arguments`), 30–100 rows per tool across all 33 tools, explicit and implicit
  phrasings; prompts rendered with Relay's own `FunctionGemmaActionModel`
  renderer (zero train/serve skew), 10% held out per tool.
- **Export:** merged 16-bit → GGUF, both quants in this repo: `Q4_K_M`
  (serving quant, 261 MB) and `BF16` (reference precision).
  No retraining was needed at any point: the one export failure was a
  converter vocab assertion, fixed at export time.

## Evaluation

Held-out set, 20 prompts × 33 tools (660 runs) through the real harness adapter
(`eval_finetune.py`), each quant served via llama-server (avg 0.10 s/call).
Three tiers — **functional** (right tool + behaviorally equivalent args:
payload slots exempt, values normalized, missing/extra optional args pass,
`ask_user` passes on any non-empty question), **selection** (right tool),
**exact** (byte-identical args):

| | Functional | Selection | Exact |
|---|---|---|---|
| Base model | 81/660 = 12.3% | 136/660 = 20.6% | 54/660 = 8.2% |
| Fine-tuned `Q4_K_M` | 583/660 = 88.3% | 603/660 = 91.4% | 446/660 = 67.6% |
| Fine-tuned `BF16` | **601/660 = 91.1%** | 609/660 = 92.3% | 461/660 = 69.8% |

Quantization costs ~2.8pp functional — serve the Q4, keep the BF16 for
reference. The exact→functional gap is dominated by contract-correct
placeholder slots (`__PAYLOAD_*__`, filled by the runtime at inference) and
equivalent phrasings (`(2 ** 5 % 7)` ≡ `2 ** 5 % 7`), not wrong behavior.
Genuine weak spots: payload-tool *selection* (`run_process`/`run_python`
confusion), git-range prompts (`git_diff` vs `git_show`/`read_file`), and
occasional garbled paths — keep these behind runtime approval. Fifteen tools
score a perfect 20/20 functional on BF16 (`copy_file`, `create_directory`,
`delete_file`, `delete_text`, `file_info`, `find_executable`, `git_checkout`,
`git_commit`, `insert_text`, `move_file`, `read_directory`, `read_file`,
`replace_text`, `run_powershell`, `web_extract`).

<details>
<summary>Per-tool functional scores (Q4_K_M / BF16, 20 prompts each)</summary>

apply_patch 17 / 17 · ask_user 17 / 19 · calculator 17 / 17 ·
copy_file 20 / 20 · create_directory 20 / 20 · delete_file 20 / 20 ·
delete_text 20 / 20 · file_info 18 / 20 · find_executable 19 / 20 ·
get_time 17 / 17 · get_working_directory 17 / 18 ·
git_branch_list 19 / 19 · git_checkout 20 / 20 · git_commit 20 / 20 ·
git_diff 10 / 10 · git_log 15 / 16 · git_show 14 / 13 · git_stage 18 / 18 ·
git_status 14 / 16 · insert_text 20 / 20 · move_file 20 / 20 ·
process_info 19 / 17 · read_directory 20 / 20 · read_file 19 / 20 ·
replace_text 18 / 20 · run_powershell 20 / 20 · run_process 15 / 15 ·
run_python 17 / 18 · search_files 13 / 15 · web_extract 20 / 20 ·
web_open 19 / 19 · web_search 18 / 19 · write_file 13 / 18
</details>

## Usage

Built for [Relay](https://github.com/baka-onii/relay)
(`FunctionGemmaActionModel`, `--fg-gguf`):

```sh
# llama-server with the GGUF (CPU is fine — ~0.6 s per translation)
llama-server -m functiongemma-270m-relay-tools-q4_k_m.gguf -c 32768 --port 8081

# inside Relay (auto-detects models/*.gguf, or pass --fg-gguf)
relay live --workspace /path/to/project --fg-gguf models/fg-tools.gguf
relay live --workspace /path/to/project --fg-gguf models/fg-tools.gguf --ui
```
Runtime settings: fg_max_tokens = 256 (larger values make it babble into
input-independent calls), fg_temperature = 0.0. Wire format follows Google's
FunctionGemma docs (<start_function_declaration> developer turn,
<start_function_call>call:name{arg:<escape>v<escape>}<end_function_call>);
the model emits no confidence — Relay parses strictly single-call and treats
anything else as a retryable error.

Limitations
- Knows only the 33 Relay tools it was trained on; unknown tools fail closed.
- No multi-tool plans, no chit-chat, no refusals — ambiguity handling lives in
the runtime gates, not in this model.
- Keep payload-tool outputs (execution, edits) behind approval; eval shows these
are its weakest selections.

License
Derivative of FunctionGemma — use is subject to the
Gemma Terms of Use (https://ai.google.dev/gemma/terms). If you redistribute
these weights, propagate the same terms.