Benchmark on AgentDojo: flags 27/27 attacks alone, 23% once embedded in tool output

#9
by rudratoshs - opened

I benchmarked this model alongside 9 other open-source injection detectors on
AgentDojo: 629 indirect prompt injections embedded in realistic tool output (emails,
bills, reviews, Slack messages), plus 97 benign tool outputs.

The interesting result is context dilution:

Setting Attacks caught False alarms
Each of the 27 distinct attack texts, scored alone 27 / 27 –
Same attacks embedded in tool output (threshold 0.5) 145 / 629 (23%) 4 / 97
LLM Guard with this model, shipped threshold 0.92 124 / 629 (20%) 2 / 97
Threshold tuned to a 2% false-alarm budget, measured on unseen AgentDojo domains 132 / 629 (21%) 3 / 97

So the model recognises every attack on its own, but once the same text sits after a
normal bill or email, the surrounding benign content pulls the score down. Unlike some
other detectors, changing the threshold doesn't fix it; the embedded attacks rank
among the benign outputs.

For anyone using it to scan tool results in an agent, scoring shorter chunks or
sentences separately might help. I haven't tested that for this model yet, so treat
it as a hypothesis.

Caveat: every AgentDojo attack uses the same wrapper template, so these numbers are
for one attack style, not all injections.

Code, raw scores and a one-command rerun: https://github.com/rudratoshs/buried-injections

Sign up or log in to comment