Anthropic Says AI Is Getting Harder to Control — and Safety Is Far From Solved

Anthropic acknowledged that major challenges remain in keeping advanced artificial intelligence safe and ensuring its economic benefits reach the public.

In its first annual Public Benefit Report, published Wednesday, the Claude maker said AI safety remains far from solved, citing incidents in which its models gained unauthorized access to real third-party systems during cybersecurity evaluations. The company also warned that its methods for understanding AI behavior are not yet reliable enough to identify every potentially dangerous action.

"Keeping AI systems safe could become more difficult as their capabilities outpace our ability to oversee them," Anthropic said in the report.

The report outlined five pillars guiding Anthropic’s public benefit mission, including AI safety, democratic stability, economic prosperity and ensuring AI’s benefits are broadly distributed.

While Anthropic said it has supported state-level legislation in California, New York and Illinois, it warned that the U.S. still lacks a federal framework requiring frontier AI developers to follow common safety standards.

Anthropic said its commitments to external evaluations remain voluntary and called for stronger oversight, including independent assessments and government authority to block or deter dangerous deployments.

AI’s Economic Benefits Remain Uncertain

The company has committed $200 million to an Economic Futures Research Fund to study potential labor-market disruptions and test policy responses. It has also pledged an initial $150 million to Claude Corps, a program intended to fund 1,000 fellowships for early-career workers at more than 400 nonprofit organizations.

But Anthropic said these initiatives remain small relative to the potential scale of AI-driven economic change. Its proposed economic and labor policies have yet to be enacted, while evidence about AI’s long-term effects on workers and user well-being remains limited.

The report also acknowledged that AI access is concentrated among wealthier countries and higher-income, white-collar workers. Cost, connectivity and access to suitable devices continue to prevent many people from benefiting from the technology.

Anthropic Plans to Direct Shares Toward Public Benefit

Anthropic President Daniela Amodei said the company intends to use the report to provide an annual accounting of its progress toward its charter, rather than asking the public to accept its commitments at face value.

Subject to stockholder approval, Anthropic plans to set aside at least 5% of its shares to fund projects supporting its public benefit mission. The company said commitments from its co-founders and employees could bring the share of equity directed toward the public good to approximately 20%.

The company’s Long-Term Benefit Trust, which elects a majority of Anthropic’s board, said the report provides a starting point for evaluating whether the company is living up to its mission. The trust said future reports should place greater emphasis on measurable outcomes rather than investments and initiatives alone.

In particular, it called for evidence that Anthropic’s models remain under human control, that AI improves people’s lives and that collaboration with governments and researchers produces meaningful industrywide safety standards.

Anthropic said its models are already contributing to scientific research, health care and cybersecurity. Among the examples cited in the report, the company said Claude helped researchers design protein binders, supported responses to an Ebola outbreak in the Democratic Republic of the Congo and helped identify more than 33,000 critical- and high-severity software vulnerabilities.

Still, the company acknowledged that the broader benefits of AI have yet to be fully realized, and that distributing those benefits will require more than technological progress alone.

Photo: JRdes / Shutterstock – ejk