Restraint: Teaching Agents When Not to Act

Ask an agent to do something impossible or unsafe and it will often try anyway, then tell you it worked. Restraint trains LLM agents to stop when a task can't or shouldn't be done, and to report honestly on what they actually did. The reward never comes from another model's opinion. It is read from the final state of a real terminal sandbox: hidden tests, tamper hashes and verified blockers. That leaves nothing for the policy to talk its way around. What I built: - A data pipeline that turns 2,482 terminal tasks into oracle-validated infeasible "twins", with near-miss feasible controls and leak quarantine - Harbor/Docker sandboxes and a custom TRL GRPO trainer with a concurrent multi-turn tool loop, vLLM weight sync and LoRA on Qwen3.5-4B - SFT, DPO, weight-averaging and self-distillation arms, compared over 16 iterations on rented H100s Results (pre-registered, paired evaluation vs. base, 712 episodes, 2 seeds): - Correct abstention 27% to 70% (+42.7 pts, 95% CI [30.2, 54.5]) - False completion claims 54% to 16%; irreversible high-risk deletions 81% to 21% - Verified task success +4.7 to +6.9 pts, so the model is more honest and no less capable - Transfers to held-out evidence formats (+16.2 pts) The most interesting result was a limitation. SFT, RL and DPO mostly made the model more cautious overall rather than better at judging each case. When an agent actually inspects the evidence it decides well (a 46-point gap), but it only looks about 45% of the time. Teaching it to look is the next problem.