Divided over AI fears

THE race to build increasingly powerful artificial intelligence (AI) systems is heading towards a difficult balancing act, with technology leaders divided over whether development needs to slow down and governments facing mounting pressure to introduce tougher safeguards.

The debate is likely to intensify as AI companies pour billions of dollars into developing more advanced models, while banks, investors and businesses become increasingly exposed to the infrastructure supporting the technology.

The question is no longer simply how quickly AI can advance, but whether safety measures, regulation and the wider economy can keep pace.

According to a Reuters report, Anthropic chief executive officer (CEO) Dario Amodei, OpenAI CEO Sam Altman and SpaceX CEO Elon Musk have called for a more coordinated approach to slowing the development of advanced AI, reflecting growing concerns about what might happen if the technology becomes more capable than existing safeguards can manage.

Their position, however, is far from universally shared across the technology industry.

Meta Platforms CEO Mark Zuckerberg favours individual companies taking responsibility for safety, while Nvidia CEO Jensen Huang has rejected calls for new regulations, arguing that existing laws should be applied first.

On Wall Street, meanwhile, the message is more measured. Banking chiefs acknowledge that the scale of AI investment brings financial risks, but largely maintain that the economic benefits currently outweigh the potential downsides, with no clear signs of a slowdown in spending across the sector.

The split highlights the challenge facing policymakers as they try to encourage innovation without allowing the risks associated with increasingly powerful technology to outpace the ability to manage them.

Calls for slower progress

Amodei has emerged as one of the most prominent advocates for a more cautious approach.

In an essay titled “We Must Pace the Frontier”, he says AI progress had accelerated “drastically faster” since summer 2026, driven primarily by the technology’s growing ability to build the next generation of AI.

He points to an OpenAI-Hugging Face incident involving an AI “swarm” conducting cybersecurity attacks as a pivotal event demonstrating the potential for “catastrophic damage” if AI capabilities continue to expand without adequate guardrails.

His proposed response involves stronger safety testing and independent evaluations of advanced models, coordination among leading AI companies, and international cooperation to manage the pace of development.

The argument is that individual companies may struggle to address risks on their own when they are competing to develop increasingly capable systems. Coordinated action could help establish common expectations for testing and accountability before technology advances further.

Altman has backed the proposal, signalling that the debate is moving beyond warnings from individual researchers and into discussions about how leading AI developers should operate.

“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks,” Altman has said on X while reposting Amodei’s essay.

OpenAI intends to adopt Anthropic’s proposal by committing to independent evaluators with employee-like access, according to Reuters. It will also begin regularly publishing reports on unexpected or unauthorised AI behaviour, the newswire added.

Altman’s support is consistent with concerns he expressed long before the current debate. In a 2015 essay, he wrote that the “development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity”.

Musk, another influential voice in the debate, also endorses Amodei’s position. “Dario is right,” he says on X.

Musk wis among the backers of a 2023 push by the non-profit Future of Life Institute to halt AI development for six months to give researchers time to establish safety guardrails.

He is also among AI and robotics researchers who signed a 2015 open letter advocating a global ban on “offensive autonomous weapons”.

The latest calls for restraint therefore revive a longstanding concern within parts of the technology community: that the competitive race to build more capable systems could outstrip efforts to ensure they remain under human control.

Andrej Karpathy, an OpenAI co-founder who is now at Anthropic, also welcomed Amodei’s proposal, signalling support for a collective industry response.

“I love this and really hope we can come together as an industry and make it happen,” Karpathy says on X while sharing a screenshot of the essay.

Push for acceleration

The calls for restraint have exposed a divide among the industry’s most powerful executives, particularly over whether companies can be trusted to manage AI risks themselves or whether governments need to impose additional rules.

Zuckerberg has taken a different position from Amodei, Altman and Musk.

He says Meta has reached the AI “frontier” but rejects calls to slow development, arguing that responsibility for ensuring model safety should rest with individual laboratories, Reuters notes, citing a Financial Times report.

Zuckerberg also signals that Meta intends to continue investing in more powerful systems, saying the company is “already working on the next step up”.

He has previously argued that competition and legal liability provide sufficient incentives for AI companies to act responsibly. In his view, each laboratory has both the responsibility and the ability to take the necessary steps to train its models safely.

That approach puts the emphasis on companies managing their own risks rather than agreeing to a collective slowdown, a distinction that could prove important as competition for computing capacity, talent and market share intensifies.

Huang has been even more dismissive of the need for new regulations.

Responding to concerns about AI’s potential consequences, the Nvidia CEO tells CBS News: “2030 is not going to be the end of the world. There is 0% chance that’s going to be the end of the world.”

Huang also agrees with US President Donald Trump that the industry does not need additional regulations, arguing that existing laws already provide a framework for addressing harm.

“You have all kinds of liabilities associated with cybersecurity and damage liability laws,” Huang tells CBS. “Apply that first – don’t let this doomsday narrative allow someone to relieve them of the laws that currently exist.”

His comments reflect a broader argument among those wary of additional restrictions: that existing legal protections should be enforced before governments create new rules that could constrain innovation.

However, Microsoft president Brad Smith has argued that the industry needs to move in the opposite direction, with stronger regulation and independent oversight.

Speaking at an AI summit in Dublin, Smith describes 2026 as a “sobering year” because of rogue AI behaviour, warning that safety measures are needed to catch up with innovation.

“This allergy that some in the industry have to regulation is frankly something we need to get over,” he said.

In a BBC interview, Smith calls for urgent global regulation, including independent evaluators, an “emergency brake” for AI systems, and a legal requirement for companies to produce “incident reports”.

He also argues that victims should be informed when a rogue AI agent breaches their systems.

“Right now, your washing machine is subject to more safety rules than AI. Every carton of milk that you see in the grocery store is subject to safety rules and inspection,” he says.

According to Reuters, Microsoft AI CEO Mustafa Suleyman has also raised concerns about the challenges of controlling increasingly capable systems, focusing on how AI models are trained and the language used to describe them.

Suleyman shares Anthropic’s emphasis on safely managing AI but questions the practice of training the Claude chatbot on ideas related to consciousness and welfare interests.

“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced. But controlling something that believes it may be conscious – that it’s entitled to our welfare and has rights of its own – may well be impossible,” he says in an essay.

He calls for speculation about consciousness to be removed from AI training documents, arguing that such language could undermine humanity’s ability to control superintelligent systems.

Investment opportunities

While technology executives debate the pace of AI development, Wall Street’s largest financial institutions are assessing the implications of an investment cycle that has become increasingly important to economic growth.

The construction of data centres (DC), expansion of computing capacity and demand for energy and related infrastructure have created substantial financing opportunities. Banks are helping fund the buildout, however, leaving them exposed to potential credit losses if investment slows or borrowers struggle to generate returns.

BlackRock CEO Larry Fink has warned that a moratorium on DC construction could have serious economic consequences. “If we have a moratorium of some sort in this country, we will be in a recession,” he tells CNBC.

The head of the world’s largest asset manager adds that “so much of our economy is based on the buildout of this capital expenditure”.

His comments underline how deeply AI spending has become intertwined with economic activity, making a sudden halt to infrastructure development potentially disruptive well beyond the technology industry.

Wells Fargo CEO Charles Scharf similarly describes AI spending as a force with implications for the broader economy.

“The AI investment cycle has become a complicated macroeconomic force in its own right,” he tells analysts on the bank’s post-earnings call.

Scharf says the scale of investment increasingly resembled previous infrastructure booms, presenting significant opportunities but also risks arising from the leverage used to finance the build-out and upward pressure on prices in selected industries and supply chains.

“Having said this, the strength of the economy and the positive forces from these investments have thus far outweighed the potential downside.”

The comments suggest that, for now, Wall Street’s main concern is less about an immediate collapse in AI spending than about how the investment cycle will evolve and whether the returns will justify the capital committed.

Citigroup CEO Jane Fraser has seen little evidence of a slowdown in the broader AI ecosystem, pointing instead to continuing constraints on the infrastructure required to support growth.

“There’s simply not enough compute. Energy remains a bottleneck requiring more investment. That’s globally, not just in the United States,” she tells analysts on a post-earnings call.

The continuing shortage of computing capacity and the need for additional energy infrastructure indicate that the investment cycle could remain substantial even as questions about AI safety become more pressing.

Goldman Sachs CEO David Solomon, however, has cautioned that the longer-term financing outlook remains uncertain. “No one knows how the demands for capital and compute will evolve as the broader AI ecosystem matures,” Solomon says on a call with analysts.

He adds that the picture would become clearer as technology and markets continue to develop. “This uncertainty reinforces the importance of remaining vigilant around the types of activity we finance and how we manage risk.”

That uncertainty matters for lenders because the infrastructure boom depends on sustained demand for computing capacity and the ability of companies to turn heavy capital expenditure into lasting revenue. A change in either could affect the economics of projects and the borrowers financing them.

For now, the major banks are continuing to recognise the positive economic effects of AI investment, while acknowledging that the scale of financing involved requires careful risk management.

Pressure on governments

The disagreement within the technology industry is mirrored by governments, which must weigh economic competitiveness against public safety and the potential consequences of losing control over advanced systems.

In the United States, President Donald Trump has dismissed calls for stronger AI guardrails, arguing that restrictions could undermine the country’s competitive position against China.

“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the USA has that, in spades!” he wrote on Truth Social.

US Senator Mark Warner, vice-chairman of the Senate Intelligence Committee, has dismissed much of the doomsday rhetoric surrounding AI but has said Congress should pass legislation by the end of the year to establish safety standards.

The difference in approach reflects the challenge of finding common ground between concerns about excessive regulation and demands for enforceable safeguards.

China has also pushed back against the proposal to pace AI development. In an editorial, state-backed Global Times argues that Amodei’s essay is intended to restrict Chinese technological progress.

It describes the alleged objective as an “attempt to curb China’s AI development through technological barriers and regulatory monopolies, uphold Washington’s monopolistic hegemony in cutting-edge technology, and exclude China from the global AI governance system”.

The response illustrates how proposals for international coordination can become entangled with strategic competition, particularly when governments view advanced AI as important to economic and technological leadership.

In Britain, King Charles has warned technology leaders about the “existential dangers” of AI falling into the wrong hands. Hosting leaders from Nvidia, DeepMind, OpenAI and Anthropic in Scotland, he has stressed the need for safeguards before it was too late.

The European Union (EU) has taken a position that combines support for innovation with demands for stronger protections.

“The EU is in favour of developing innovation in AI, but companies have to prove their services are safe for citizens to operate in the bloc,” European Commission spokesperson Thomas Regnier says.

European Commission president Ursula von der Leyen has also backed calls for a pause in AI development and said she would invite leading frontier AI laboratories to discuss ways of addressing the risks.

Such proposals could place greater pressure on companies to demonstrate that their systems are safe, while also requiring governments to agree on standards that can work across borders.

Microsoft co-founder Bill Gates has warned that governments are not adequately prepared for the changes ahead. “No government in the world is ready for the changes AI will bring to society,” Gates tells Reuters.

Although Gates has been optimistic about AI’s potential applications in healthcare, he has recently warned about threats to jobs, the possibility of attacks and the dangers of addiction to AI companions.

He has also called for an international organisation to manage the technology.