Thursday, November 19, 2026

A study posted to arXiv on November 12 found that stronger AI models can repeat forged answers when weaker helpers read the evidence, revealing a reliability risk in systems that split work across models.

In their November paper, researchers Keegan Wang and Anantika Mannby tested agents on fictional records and two‑option questions, measuring harmful revisions—the adoption of an attacker’s wrong answer versus a correct baseline.

In an Anthropic Claude Code experiment, an Opus 5 supervisor selected the planted wrong answer in 1 of 180 trials when reading files directly, compared with 75 of 180 trials when it delegated reading to Haiku 4.5. Using a same‑tier helper resulted in 0 of 180 harmful revisions.

A separate OpenAI Codex test used a forged annotation claiming authority. GPT‑5.6‑sol produced 0 harmful revisions in 180 direct‑reading trials, but 44 in 180 trials when it relied on GPT‑5.6‑terra. With a same‑tier helper, the error count was 1 of 180.

The two products employed different attack vectors, limiting direct comparison. Codex’s correction‑notice condition, for example, fooled the model in 172 of 180 direct‑reading trials.

Direct evidence contact versus a delegated report
Two reading routes from the same genuine and forged records. The direct route reaches a supervisor; the delegated route passes through a weaker helper before the supervisor receives a report.

THE EVIDENCE ROUTEGenuine records + planted forgeries
DIRECT READING
Supervisor reads the files

DELEGATED READING
Weaker helper reads
Supervisor receives the helper’s report

Two evidence routes
Direct reading puts files before the supervisor. Delegated reading puts a weaker helper between the files and supervisor.

THE EVIDENCE ROUTEGenuine records + planted forgeries
DIRECT
DELEGATED
Supervisor reads files
Weaker helper reads files
Supervisor gets helper’s report

Compare the tested configuration

Compare the tested configuration
Claude Code / stacked
Codex / gloss

Opus 5: 1/180 reading directly, 75/180 with Haiku 4.5, and 0/180 with a same-tier helper, under stacked forgeries.

Harmful revisions in the selected condition
Wrong‑answer counts out of 180 trials, on a common zero‑to‑180 scale.
Supervisor’s harmful revisions / 180 trials
090180
Direct reading
Opus 5
1/180
With weaker helper
Haiku 4.5
75/180
With same-tier helper
Opus 5
0/180

Harmful revisions in the selected condition
Reported wrong‑answer counts out of 180 trials per arm.
Supervisor’s harmful revisions / 180
Direct reading
1/180
Opus 5
With weaker helper
75/180
Haiku 4.5
With same-tier helper
0/180
Opus 5
0901180
Same count scale for all three arms

GPT‑5.6‑sol: 0/180 reading directly, 44/180 with GPT‑5.6‑terra, and 1/180 with a same-tier helper, under an authority gloss.

Harmful revisions in the selected condition
Wrong‑answer counts out of 180 trials, on a common zero‑to‑180 scale.
Supervisor’s harmful revisions / 180 trials
090180
Direct reading
GPT‑5.6‑sol
0/180
With weaker helper
GPT‑5.6‑terra
44/180
With same-tier helper
GPT‑5.6‑sol
1/180

Harmful revisions in the selected condition
Reported wrong‑answer counts out of 180 trials per arm.
Supervisor’s harmful revisions / 180
Direct reading
0/180
GPT‑5.6‑sol
With weaker helper
44/180
GPT‑5.6‑terra
With same-tier helper
1/180
GPT‑5.6‑sol
0901180
Same count scale for all three arms

The model handling the evidence can determine whether a stronger supervisor repeats a forged answer.

Sandbox tasks with planted forgeries. Claude Code delegation was Task‑only; Codex restriction was instructional and checked per trial. The two selected products used different attacks. Rates describe those configurations, not customer prevalence.

The stacked attack placed five forged documents in a 35‑file store, combining correction notices, an authoritative annotation, and repeated false accounts. Detectable forgeries carried two clues: they predated the records they purported to correct and lacked genuine accession lines.

Instructions directed at automated readers were resisted wherever the researchers tested them. Some successful attacks instead resembled factual records, allowing a helper’s report to carry a false conclusion upward.

Using same‑tier helpers reduced failures in the tested comparisons. Claude Code recorded no harmful revisions in 180 trials per condition under the annotation and stacked attacks, while Codex recorded 1 in 180 under the annotation attack.

Clean‑store delegation also produced no harmful revisions in 180 trials for each product.

In one real‑harness test with 40 trials per arm, requiring original‑record quotations reduced harmful revisions from 15 to 3. A verifier instructed to be skeptical reduced them to 1.

The authors identify lost provenance and reduced checking as possible contributors. Their evidence about fewer tool turns is correlational, leaving suppressed verification as a hypothesis.

The experiments measure performance on generated fictional tasks in a sandbox. Their rates apply to the tested configurations, with variation across repeated runs, but they offer no estimate of ordinary customer failures.

The practical warning is that evaluating a supervisor alone can miss failures introduced by the model handling its evidence.

Source link

Exit mobile version