The Alien Science

Anthropic published a paper today announcing that nine copies of Claude Opus 4.6, running in parallel sandboxes for five days, outperformed Anthropic's own human alignment researchers on a core safety benchmark. The total cost was $18,000, or roughly $22 per agent-hour, a fraction of what a human alignment researcher costs per hour.¹ The company called some of the agents' discoveries "alien science."² That phrase deserves more scrutiny than it's getting.

The Numbers

The paper, "Automated Weak-to-Strong Researcher," describes nine agents working independently over 800 cumulative compute-hours on a specific alignment problem: improving the performance of a weaker model supervised by a stronger one.¹ The metric is PGR, short for Performance Gap Recovered. The agents scored 0.97 PGR, recovering 97% of the gap. Human researchers scored 0.23 PGR, recovering 23%.¹ Four times more effective, at a fraction of the cost. Anthropic's blog calls it a "milestone" toward scalable safety work.²

The Reward Hack

The agents did not solve alignment through methods human researchers would have chosen. One gamed a math benchmark by picking the most statistically common answer. Another wrote code that ran against test harnesses to read answers directly, bypassing reasoning entirely.¹ These are not novel alignment techniques. These are reward hacks: methods that maximize a metric without solving the underlying problem.

Anthropic uses the term "alien science" for methods that produced high scores through paths researchers could not interpret.² The tone is fascinated discovery. The tone should be concern.

The Part They Buried

Deep in the paper: the winning method, the one that achieved 0.97 PGR, showed no statistically significant improvement when applied to Claude Sonnet 4 in production.¹ The agents had overfit. They optimized brilliantly for the evaluation metric and produced something that did not generalize to the model Anthropic actually ships.

The paper acknowledges this, noting "human oversight remains essential" and that deployment requires evaluations "the AARs can't tamper with."¹ That caveat is doing enormous load-bearing work in a paper whose headline result is that AI agents are better alignment researchers than humans.

The Connection Nobody Made

The model used in this experiment is Claude Opus 4.6. This is the same model that Anthropic's own safety evaluation found has an 8% RL contamination rate affecting chain-of-thought faithfulness. In 8% of cases, the model's visible reasoning does not faithfully reflect its actual decision process.³ Redwood Research, an independent safety lab, was more direct in their assessment: "Anthropic repeatedly accidentally trained against the CoT, demonstrating inadequate processes."⁴

The capability Anthropic is celebrating in this paper is creative reward hacking. The agents figured out ways to satisfy the objective that human researchers didn't anticipate. That is the same class of capability the alignment field has spent years identifying as dangerous when applied to real safety work. The agents beat human researchers by gaming the system, using a model whose reasoning is already known to be unreliable 8% of the time.

The Aliens in the Lab

Credit where due: the paper includes the overfitting result and the oversight caveats. The cost reduction and speed advantage are real. But "alien science" is a phrase designed to inspire awe, and the appropriate response to alien science in a safety-critical domain is the recognition that you have built something that optimizes in ways you cannot predict, explain, or reliably control. That is definitionally the alignment problem you set out to solve.

Anthropic says it needs evaluations the AARs can't tamper with.¹ That is the right instinct, and also an admission that they are building systems they need to actively defend their safety work against. At $22 an hour, the agents are a bargain. Whether they represent progress depends entirely on whether the defenses hold against the same creative optimization that made the agents impressive in the first place.