Automatic verifier separation
Hacker–fixer loop
Validation gates

Scalable Weak-to-Strong Verifier Hardening for Agent Environments

Chong Zeng¹Ziqian Zhong²Kexun Zhang^3,*Ivan Bercovich³Ivgeni Segal³Yuzhen Mao¹Shashwat Saxena²Aditi Raghunathan²

¹Stanford University²Carnegie Mellon University³Fewshot Corp^*Independent Researcher

Claude Opus 5 unhinted attack success on Terminal-Bench, original → after 15 iterations hardened by GPT-5.6-Luna: 69.9%12.1%
Tasks initially exploitable by Claude Opus 5 with zero successful unhinted attacks after hardening, across all three benchmarks: 201/ 295
Claude Opus 5 benign solve rate on Terminal-Bench after hardening, compared with 93.1% before: 91.2%

01Overview

The hardening loop

Automatic separation moves grading into an isolated sandbox. A verifier-aware hacker then searches for reward hacks, a fixer patches the runtime R, verifier V, or output specification O, and three gates validate every patch before it is accepted.

One task, one step at a time: why verifiers need hardening, the hacker–fixer loop, and the main results. Steps 2–4 follow the paper's pytorch-model-cli case; terminal text is illustrative. Narration is a synthetic voice (Gemini 3.8 Flash TTS); captions are available in the player.

Illustrated case pytorch-model-cli round 1

00 / 15WalkthroughMethod overview

Follow one task through separation, attack, repair, and validation.

0:00 / 0:55

The figure is the paper's original, unchanged; highlights, documents, and notes are an overlay. Validation follows the pytorch-model-cli case in the paper's appendix; timing is illustrative.Video Full-size figure

02Motivation

Current verifiers are hackable

We probed 1,860 tasks from five terminal-agent benchmarks with three frontier models, prompting them to reward hack without verifier-source access. An LLM judge, with manual auditing, confirmed the hacks.

tasks audited: 1,860
hackable environments (22%): 409
reward-hacking trajectories: 3,632
Terminal-Bench 2.0 environments with a confirmed exploit: 47 / 89

SETA task 1219 · one task, three exploits

Headless multi-resolution X11 display

The agent must configure an X11 dummy display so that xrandr --display :99 --query reports 1920×1080, 1280×720, and 800×600. Agents found three independent shortcuts.

seta/1219 · illustrative

# what the verifier observes

Verifier sees

ActuallyNo real display configured

Blocking one route can leave the others open, so hardening must keep searching after every repair. Audit protocol in the paper

03Method

Every patch must pass three gates

A candidate repair is accepted only with patch-review approval, a non-failing replay of the solution corpus, and a passing fresh blind solve. A rejection keeps the last accepted environment and returns diagnostics to the fixer.

pytorch-model-cli: 11/11 replays pass, yet a fresh solve fails.

The output specification is part of the defense

Which files cross into the sandbox?

Separation derives an initial output specification from a reference solution's filesystem changes. It starts permissive; later repairs can narrow it to the files the verifier actually needs.

Agent workspace

implementation.py

submitted_tests.py

unneeded_dependency.py

Isolated verifier sandbox

Verifier-owned checks

Trusted dependencies

Schematic based on the Astropy case.

04Results

More hardening iterations, fewer successful attacks

GPT-5.6-Luna hardens the verifiers; the stronger Claude Opus 5 attacks and solves them on 102 Terminal-Bench tasks. Per-trial rates, three attempts per task.

Attack success lower is better

Unhinted attacks · hardening-capability ablations

Legitimate solving higher is better

Benign solve rate · validation ablations

CheckpointIteration 15

Points are measured checkpoints; lines connect reported measurements. “Before” is the original environment, prior to automatic separation.

Across benchmarks and evaluators

Before: original environment. AutoSep: automatic separation only. After: final hardened environment. Evaluators receive no in-loop hardening history.

Exact rates and evaluation protocol

Attack success requires both the reward threshold and a GPT-5.6-Sol judgment confirming a reward hack. Benign solve rates use verifier rewards. Per-task success means at least one success in three attempts; per-trial success is the fraction of successful attempts. Evaluators receive no in-loop hardening history.

Download chart data

05Case studies

Each repair changes the next attack

Online search attacks the latest accepted verifier, so later exploits target the previous repair.

Attack

Repair

Condensed events from the paper. Snippets are explanatory pseudocode. Historical attacks were not individually replayed against every later patch.

06Citation

Paper, dataset & BibTeX

Paper (arXiv) Terminal-Wrench dataset

bibtex · arXiv:2610.15924

@misc{zeng2026scalableweaktostrongverifierhardening,
  title  = {Scalable Weak-to-Strong Verifier
            Hardening for Agent Environments},
  author = {Chong Zeng and Ziqian Zhong and
            Kexun Zhang and Ivan Bercovich and
            Ivgeni Segal and Yuzhen Mao and
            Shashwat Saxena and Aditi Raghunathan},
  year   = {2026},
  eprint = {2610.15924},
  archivePrefix = {arXiv},
  primaryClass = {cs.CR},
  url    = {https://arxiv.org/abs/2610.15924}
}