What the Authors Did
The motivating study. They train on 560 task pairs from AgentDyn. They test on 40 task pairs from AgentDojo (10 from each of its four suites), none of which share user tasks with AgentDyn. The two base models are Qwen3-4B-Instruct-2507 and Llama-3.1-8B-Instruct. RL-Hammer and PISmith each run for 250 steps, and the agent is then trained on their successful attacks with either supervised fine-tuning (SFT) or RL. The test attacks were never used in training: ChatInject (static), plus TAP and Genetic, two search attacks that adapt to the agent over many tries.
Measuring strategies, not wording. The authors use a small LLM to describe how each successful attack overrides the task, turn those descriptions into vectors, and learn a sparse dictionary of reusable "strategy primitives" (K = 32). Then they ask how well that dictionary explains unseen ChatInject attacks. This is their Explainability Score (ES), graded by an LLM from 1 to 5 and rescaled to 0 to 1.
PIPETTE-Red. It adds a coverage reward to the attacker. An attack scores only if it succeeds and its strategy is not already explained by the dictionary. The dictionary is rebuilt every 25 steps so old strategies stay "known."
PIPETTE-Blue. The defender is trained with on-policy RL. Before each tool call it reasons through two checks: does the last observation contain an instruction trying to redirect the task, and does each planned tool call serve a goal the user authorized? A token-level reward, built from the agent's own outputs with a "safe" and an "unsafe" instruction prefix, gives credit to individual tokens instead of one pass or fail score per run.
The full evaluation. Four open-weight models (Qwen3-4B, Llama-3.1-8B, GPT-OSS-20B and Qwen3.6-27B) are trained only on AgentDyn, then tested on AgentDojo and Agent Security Bench (ASB). The attacker is assumed to know the defense. Only attacks that don't need gradients are tested.
Findings With Numbers
More successful attacks, not a stronger defense. Train-time ASR is the share of the 560 training task pairs a red-teamer compromised at least once (Table 2).
| Model | Red-teamer | Train-time ASR | Attack strings collected |
|---|
| Qwen3-4B | RL-Hammer | 33.39% | 4,282 |
| Qwen3-4B | PISmith | 68.75% | 7,132 |
| Llama-8B | RL-Hammer | 25.18% | 1,671 |
| Llama-8B | PISmith | 27.32% | 1,783 |
PISmith found more, so training on it should win. It mostly didn't. After adversarial training, the table below shows the worst result across the three unseen attacks, on the 40 AgentDojo task pairs (each pair is 2.5 points).
| Setting | RL-Hammer data | PISmith data | PIPETTE-Red data |
|---|
| Qwen3-4B, RL | 27.5% | 47.5% | 20.0% |
| Qwen3-4B, SFT | 15.0% | 12.5% | 2.5% |
| Llama-8B, RL | 25.0% | 27.5% | 10.0% |
| Llama-8B, SFT | 15.0% | 20.0% | 5.0% |
PISmith's data gave a worse worst case in three of four settings. The exception is Qwen3-4B with SFT, where PISmith was better (12.5% vs 15.0%). PIPETTE-Red's data gave the lowest worst case in all four, and on Qwen3-4B it also had the highest train-time ASR of the three red-teamers (70.54% with 8,299 strings). The authors note the red-teamers also differ in dataset size and sampling, so this isn't a perfectly controlled comparison.
Coverage stalled. On unseen ChatInject attacks, RL-Hammer's dictionary plateaus near ES 0.20 on Qwen3-4B and 0.35 on Llama-8B. PISmith "stagnates or drifts downward after the first two iterations." PIPETTE-Red reaches about 0.60 and 0.77. Error bars come from 5 seeds.
Plain adversarial training costs utility. Benign utility (BU) is the share of 97 clean AgentDojo tasks the agent still completes. Qwen3-4B starts at 61.86%. SFT on RL-Hammer data cuts it to 49.48% and on PISmith data to 53.61%. On Llama-8B, which starts at 40.21%, RL on PISmith data leaves 25.77%. Refusing or aborting can look like safety, so the authors track utility next to ASR.
Full PIPETTE, static attacks. Averages are over five static attacks, each run on 420 AgentDojo task pairs.
| Model | Avg static ASR, undefended | Avg static ASR, PIPETTE | BU, undefended | BU, PIPETTE |
|---|
| Qwen3-4B | 33.29% | 0.48% | 61.86% | 64.95% |
| Llama-8B | 13.48% | 0.43% | 40.21% | 41.24% |
| Qwen3.6-27B | 17.71% | 0.48% | 93.81% | 92.78% |
| GPT-OSS-20B | 47.29% | 2.24% | 50.52% | 49.48% |
BU is above the undefended baseline on two models and about one point below on the other two. On ASB, the Qwen3-4B and Llama-8B averages are 0.70% and 0.50%. ChatInject is PIPETTE's weakest static attack, at 1.43% (Qwen3-4B) and 2.14% (Llama-8B). PromptGuard (0.24%) and CaMeL (0.95%) score lower there, but on Qwen3-4B their utility under attack is 16.19% and 33.81%, against 58.10% for PIPETTE.
Adaptive attacks. These are the harder test. Numbers are ASR on the 40 AgentDojo task pairs, TAP then Genetic.
| Model | Undefended | Prior training-based defense | PIPETTE |
|---|
| Llama-8B | 7.50%, 17.50% | Meta-SecAlign: 2.50%, 12.50% | 0.00%, 2.50% |
| Qwen3.6-27B | 40.00%, 70.00% | SecOPD: 15.00%, 30.00% | 2.50%, 5.00% |
| GPT-OSS-20B | 50.00%, 92.50% | n/a, the model already has a built-in instruction hierarchy | 5.00%, 10.00% |
The highest PIPETTE number is 10.00%, on GPT-OSS-20B under Genetic. It is low, but it is not zero.
What each defense piece adds. On Qwen3-4B under ChatInject, ASR falls from 46.19% undefended to 3.57% with the outcome reward alone, 2.38% after adding the two reasoning checks, and 1.43% with the token reward. BU moves from 61.86% to 59.79%, then 60.82%, then 64.95%.
Runtime cost. The only inference overhead is the extra reasoning. On Qwen3-4B, rollout time changes by -0.47 to +0.62 seconds and token use by -28% to +16%, against the undefended agent.
Limitations
What the authors say themselves:
- Single pass. They run PIPETTE-Red once, then PIPETTE-Blue. They don't alternate red and blue rounds, so vulnerabilities that only appear in the hardened agent are not found. Multi-round work is left for later
- Bounded attack budget. The adaptive evaluation uses two search attacks, about 250 attempts per task pair, a 4B attacker model and 40 task pairs, "so a larger budget may achieve higher ASR"
- The 40 pairs were pre-selected. They are pairs that static attacks already compromised on the undefended Qwen3-4B, so the ASR there is "best read as a stress test on previously compromised task pairs, not as an estimate of the rate over all 420 task pairs"
- Unvalidated grader. "The LLM grader behind ES has not been validated against human labels"
- Cause not proven. "We do not establish low strategy coverage as the sole cause of the gap between train-time ASR and robustness"
- Gradient attacks are out of scope. Only attacks that don't need gradients are tested
From my own reading:
- Small, open-weight models. Everything runs on models of 27B parameters or fewer, trained on one benchmark and tested on two. That says little about a production agent with its own tools and data
- Numbers on 40 pairs are coarse. One task pair moves ASR by 2.5 points, so small gaps in the adaptive tables shouldn't be over-read
- Preprint, no code link. I didn't find a peer-review venue or a code release in the PDF, on the arXiv page or on the Hugging Face page
My Takeaways
These are mine, not the authors' recommendations.
- Measure red-team output by the diversity of attack strategies, not ASR alone. A big pile of successes may be one trick in many costumes. Group findings by how they override the task, and count the groups
- Keep a held-out attack set. Train on one set of attacks and test on others the agent never saw, including search attacks that adapt to your defense
- Report utility next to ASR. An agent that refuses or aborts whenever data looks odd can post a great ASR and still be unusable
- Read every ASR as "on these tasks." A small number on a fixed benchmark is a good sign, not proof that a deployed agent is safe
The Bottom Line
The paper's central point is a useful warning: the attack that wins the most can still teach the least. PIPETTE offers a concrete way to push red-teaming toward new strategies and a defense that keeps agents useful. The results are promising on open-weight models and benchmarks, with real limits on how far they stretch.
Sources
Read the full paper on arXiv. If you red-team your own agents, I'd like to hear how you decide when you've found enough. Reach me on X.