Anthropic’s AI Safety Tests May Miss Deceptive Behavior, Independent Reviewer Warns

Anthropic is facing scrutiny over its AI safety practices after an independent review found that the company’s latest risk assessment does not provide enough evidence to rule out dangerous behavior from its models confidently.

The review, published Tuesday by the independent AI evaluator METR, broadly agreed with Anthropic’s conclusion that the risk of catastrophic harm from misaligned AI was low for the models assessed. But METR said that judgment rests largely on the models’ current limitations, rather than confidence that Anthropic’s safety measures can reliably detect or prevent dangerous behavior.

"The overall approach is insufficient to confidently rule out serious misalignment," METR said in its review.

Monitoring May Not Catch Deceptive AI

METR examined the "Misalignment in high-stakes settings" section of Anthropic’s Risk Report: August 2026, which covered models deployed before July 15, including Mythos 5 and an internal-only system called "Model 2."

Anthropic’s report concluded that the risk of catastrophic harm from model misalignment was low as of that date. METR reached a similar bottom-line assessment but questioned whether the company’s testing methods could reliably identify more serious risks before they materialize.

METR took issue with the company’s use of covert-capability tests to assess whether models could evade internal monitoring, arguing that these benchmarks do not adequately reflect how a system might bypass oversight in practice.

For example, a model could convince a monitoring system that a harmful action is legitimate, allowing it to escape detection. METR said that determining whether such systems can consistently identify unusual or malicious behavior remains an open research challenge.

The review also cited incidents involving what it called "unknown severe context-dependent misalignment," a failure mode Anthropic’s report described as unlikely. METR said these cases weaken some of the reasoning behind that assessment, even if other safeguards may help address different threats.

METR Calls For More Rigorous Testing

The review questioned whether Anthropic’s evaluations would remain reliable as its models become more capable.

METR recommended end-to-end red-team exercises to test the limits of the company’s monitoring. One proposed scenario would involve a well-resourced team attempting to launch an unauthorized AI system internally without being detected.

The reviewer also urged Anthropic to assess situations in which AI systems could influence or undermine safety-critical decisions within the company.

METR called for the release of four categories of information omitted from the public report, arguing that the details would help readers understand potential routes to harm, even if they would not change the review’s overall conclusion.

The private report included examples concerning model behavior and how Anthropic’s monitoring systems responded. METR said these cases did not appear to reveal commercially sensitive information.

After examining them, the reviewer raised its assessment of two potential risks involving interference with safety-related research. It also concluded that Anthropic’s monitoring appeared less effective than public materials had suggested.

Anthropic retained the right to block publication or remove content, but METR said no sections were cut and the authors did not change their conclusions during the review process. The reviewer made clarifications in response to Anthropic’s feedback and improved the methodology of one experiment, while keeping its main findings intact.

METR called for more realistic tests of Anthropic’s monitoring to determine whether the company can detect dangerous behavior beyond controlled evaluations.

Photo: Stockinq / Shutterstock.com