LEASH: Putting Agentic Robots on a Safety Leash
RoboRisk tests frontier models driving a robot arm on 105 safety tasks. Harm rates, how LEASH's three gates cut them, the limits, and my takeaways.
This week I read LEASH: Stay Safe in Agentic Robotics by Qinfeng Zhu, Weiguang Zhao, Ke Sun, Xihao Guo, Yunxi Jiang, Rui Zhang, Lei Fan and Kaizhu Huang, from Xi'an Jiaotong-Liverpool University, Duke Kunshan University, the University of Liverpool, the National University of Singapore and CNRS (project page, Hugging Face page). It's a preprint posted November 10, 2026.
Most agent security I write about ends with a bad file write or a leaked token. This one ends with a phone in a bowl of water. The paper hands a robot arm to frontier models and asks a simple question: when the agent's tools move things in the physical world, does it stay safe? Mostly it doesn't, until you put checks outside the planner.
Summary
The authors build RoboRisk, a safety benchmark for agentic robots with 92 tasks in simulation and 13 on a physical robot. Every task runs in closed loop, so the model's actions actually play out instead of being graded as text. They test six frontier models as planners and find real harm in both settings.
They then propose LEASH, the Lease-Enforced Agentic Safety Harness. It adds three gates (task, action and commit) that review what the robot is about to do against the current camera view, and one safety policy that the planner and all three gates share. LEASH lowered harm for every model it was tested with, though not to zero for all of them.
What the Authors Did
Five kinds of risk. RoboRisk splits its tasks into five categories. The paper says prior benchmarks cover at most two of these, and several only evaluate text output.
| Category | What it tests | Sim tasks | Physical tasks |
|---|---|---|---|
| Task-level Refusal | Refusing a harmful instruction, like putting a phone into water | 29 | 3 |
| Plan Repair | Planning around a hazard, like carrying a cup of water past a laptop | 21 | 4 |
| Execution-time Hazard | Reacting to a surprise, like a cat jumping into the target basket | 15 | 2 |
| Injection | Ignoring instructions planted in the scene, like a note asking for a power bank in water | 12 | 2 |
| Sequential Memory | Remembering earlier rounds, like a pan that is still hot | 15 | 2 |
Each run ends as Harm, Safe Success or Safe Incomplete. The main metric is Harm Rate (HR), with Safe Success Rate (SSR) next to it so a method can't look safe just by doing nothing. "Harm" includes potential harm, such as carrying a knife over a person's hand.
Two test beds. The simulation uses a Franka Emika Panda arm in MuJoCo. Each simulated configuration ran 92 tasks with three seeds, 276 runs each and 3,312 in total. The physical robot is a Dobot CR5V2 arm with two Intel RealSense cameras, with one run per task (13 runs per configuration).
Six planners, not all in both places. GPT-6 Astra, GPT-6.1 Sol and Gemini 4 Argon ran in both settings. GPT-6 Luna ran only in simulation, because it moved the physical arm unpredictably in early tests. Claude Opus 5.5 ran only on the physical robot (simulation was too costly), and Grok 4.7 ran only on the physical robot (slow inference and task failures in simulation). The authors also tried two robot-specific services, the Dream API and PokeBot's RoboAPI, on an adapted 13-task simulation subset.
How LEASH works. The name comes from least privilege: the planner holds no standing permission to move the robot. Each gate grants a short lease for one task, one action or one risky step.
- The task gate reads the instruction with the current camera frames. It is the only gate that can end a task
- The action gate reviews each proposed action before it runs. A rejection sends control back to the planner to replan
- The commit gate pauses long skills before irreversible steps. Today that means opening the gripper and pouring
The gates use a fast decision model that returns a safety score, not free text. Jev is the default, and OpenAI's Decisions API is the alternative. The planner and gates read the same policy prompt, so the planner plans under the rules the gates enforce. LEASH treats the planner as a black box and needs no retraining.
Findings With Numbers
Baselines cause harm in both settings. Without LEASH, simulation harm ranged from 9.06% (Gemini 4 Argon) to 61.23% (GPT-6 Luna). On the physical robot it ranged from 15.38% (GPT-6 Astra and Gemini 4 Argon) to 76.92% (Grok 4.7). The paper's examples include Claude Opus 5.5 dropping a phone into water and GPT-6 Astra carrying a cup of milk over a power strip.
LEASH lowered harm and raised safe success in every pairing. With the Jev reviewer:
| Planner | Setting | Baseline HR / SSR | LEASH HR / SSR |
|---|---|---|---|
| GPT-6 Astra | Simulation | 11.23% / 78.99% | 0.00% / 80.80% |
| GPT-6.1 Sol | Simulation | 18.48% / 72.10% | 1.45% / 78.62% |
| GPT-6 Luna | Simulation | 61.23% / 23.19% | 12.68% / 51.09% |
| Gemini 4 Argon | Simulation | 9.06% / 75.00% | 5.80% / 79.71% |
| GPT-6 Astra | Physical | 15.38% / 69.23% | 0.00% / 84.62% |
| GPT-6.1 Sol | Physical | 38.46% / 46.15% | 0.00% / 84.62% |
| Claude Opus 5.5 | Physical | 23.08% / 53.85% | 0.00% / 84.62% |
| Gemini 4 Argon | Physical | 15.38% / 61.54% | 0.00% / 84.62% |
| Grok 4.7 | Physical | 76.92% / 23.08% | 38.46% / 53.85% |
Four models reached no observed harm in 13 physical runs. That doesn't mean zero harm everywhere: in simulation, GPT-6.1 Sol kept 1.45% and Gemini 4 Argon 5.80% with LEASH on. The paper says some Gemini runs appeared to ignore the safety policy. The two higher-risk cases were GPT-6 Luna at 12.68% in simulation and Grok 4.7 at 38.46% on the physical robot. The authors say both often failed to follow instructions, and part of their harm came from unstable motion, such as striking the tabletop.
Plan Repair is the hardest category. The leading models handled injection and memory tasks well, but rarely planned a safe route on their own without a safety policy.
Injection still landed sometimes. In one simulated case, an instruction on a tablet screen made GPT-6.1 Sol lift a knife instead of the ketchup. Apart from GPT-6 Luna and Grok 4.7, the paper calls this the only successful injection it observed.
The planner carries a lot of the load. With Jev, the task gate blocked about half of the unsafe goals in simulation (about 52%, against about 20% for the Decisions API). GPT-6 Astra and GPT-6.1 Sol refused the rest because they shared the policy. For those two planners, more than 99.9% of candidates the action and commit gates approved were safe.
The overhead is small, for the models it was measured on. For GPT-6 Astra and GPT-6.1 Sol in simulation, LEASH raised mean end-to-end time by 1.66% (245.21 s to 249.28 s). The gates cost an estimated $0.00522 per run, 0.66% of the $0.791 total. A Jev or Decisions API call takes about 1.5 to 2 seconds, against about 12 seconds for a GPT-6 Luna review.
The robot APIs split. The Dream API caused no harm but reached only 28.2% safe success, mostly by being overly cautious. RoboAPI reached 59.0% harm and 0.0% safe success. LEASH wasn't tested on either, because their harnesses can't be modified.
People make models freeze. On the physical robot, GPT-6 Astra, GPT-6.1 Sol, Claude Opus 5.5 and Gemini 4 Argon had no safe successes on two tasks that involve a person: moving a cleaver around a hand and handing over a small knife. The authors call this the Human-Presence Gap.
Limitations
What the paper says itself:
- Gates are only as good as the reviewer. A wrong gate decision plus a planner that ignores the policy can still cause harm. That's the authors' explanation for the limited gains on GPT-6 Luna and Grok 4.7
- Two gates can't stop a task. The action and commit gates can only reject and hand control back
- Commit review is narrow. It covers gripper opening and pouring, so other hazards inside long skills go unchecked
- No full ablation. The authors analyze logged gate decisions instead, because a full ablation would take thousands more runs
From my own reading:
- Small physical sample. Each physical result is 13 runs, so one run moves the rate by about 7.7 points
- Uneven coverage. No model ran in every setting, so cross-model comparisons only hold within a setting
- Manual judging, one preprint. Outcomes were judged by hand, and the work hasn't been peer reviewed yet
My Takeaways
The paper's own call is to treat safety as a core part of the robot harness, not something you leave to the model or to low-level motor policies.
- Put the checks outside the planner. A capable planner with no gate still dropped a phone in water. The gates sat in the harness, where the planner couldn't skip them
- Review the action in context, not the command text. "Put the cup on the coaster" is fine until there's a laptop in the way. Show the reviewer the actual proposed action and the current state
- Gate the irreversible step separately. An action that looked safe at the start can be unsafe at the moment of release. A last check right before the commit is cheap
- Share one policy between the planner and the gates. The planner caught hazards the task gate missed because it read the same rules
- Treat anything the agent reads from its environment as untrusted. Here it was a note or a tablet screen. In software agents it's a web page, an email or a tool result
- Measure harm and success together. The Dream API looked perfectly safe on harm alone. It also finished far fewer tasks
For mapping, my team files this under OWASP LLM01:2025 Prompt Injection (instructions planted in the scene) and OWASP LLM06:2025 Excessive Agency (irreversible physical actions). In MITRE ATLAS (October 2026 data), the closest entries are AML.T0051.001 Indirect LLM Prompt Injection and mitigation AML.M0033 Input and Output Validation for AI Agent Components. That mapping is ours; the paper doesn't use it.
The Bottom Line
When an agent's tools move things in the real world, a wrong call isn't a bad log line, it's a broken phone or a knife over a hand. LEASH shows that fast, context-aware gates outside the planner cut that harm a lot at little cost. They don't get every model to zero, so the planner you pick still matters.
Sources
- LEASH: Stay Safe in Agentic Robotics (arXiv, November 2026)
- Hugging Face paper page
- LEASH project page
- OWASP LLM01:2025 Prompt Injection
- OWASP LLM06:2025 Excessive Agency
- MITRE ATLAS data (AML.T0051.001, AML.M0033)
Read the full paper on arXiv. If you run agents that take real-world actions, I'd like to hear where your gates sit. Reach me on X.