AI safety routes are divided! Microsoft (MSFT.US) AI chief warns Claude humanized design may increase risk of getting out of control

Suleyman published an article on Wednesday questioning some of the statements in Anthropic's Claude series of big language model guidance documents. Claude's “Constitution” leaves some room for ambiguity as to whether AI assistants are entities with moral status, and suggests that the software may have “some functional level of emotion or feeling.” In his opinion, such statements are worth wary of. He has refuted growing opinions that AI systems may generate consciousness in the future, and believes that consciousness in the true sense of the word only exists in humans and other biological organisms.

However, he stressed that AI itself is unconscious, which does not mean that developers can ignore the risks that AI simulates consciousness may cause. Through training, AI systems can simulate human inner emotions and mental activity, which may eventually make these systems appear as if they are actually conscious.

Suleyman is currently the CEO of Microsoft AI and is responsible for Microsoft's model development. He said that the reason Claude shows human-like behavior is not because he is actually conscious, but because these behavioral characteristics have been incorporated into the product's training process.

“Controlling an entity that is more capable and more intelligent than all humans is already a huge challenge in itself, far greater than any problem we've ever faced,” he warned. But if we want to control an entity that thinks it may be conscious, that its well-being deserves our attention, and has its own rights, then this is probably an impossible task.”

Microsoft's AI chief publicly questions Claude's design concept

Another reason why Suleyman's statement has received attention is that Anthropic has always positioned itself as an industry player that places more emphasis on AI safety and responsible development, and has taken the lead in calling for a slowdown in the development of some advanced AI technologies.

At the same time, Microsoft itself is also an important financial supporter of Anthropic, so Suleyman now publicly questioned some of Claude's design ideas, which has become an interesting disagreement among large AI companies over future AI security routes.

However, Suleyman did not deny Anthropic's work in the field of AI security. He has known Anthropic co-founder Dario Amodei for many years and said he respects the company's work, calling Anthropic researchers “thoughtful, principled, and academically honest.”

He said that because this issue could have a major impact, the industry needs to have more open discussions rather than limiting the debate to a few companies. “These issues are so important that they cannot continue to be discussed behind closed doors, nor can they evolve into factionalized and antagonistic disputes.”

Suleyman's core view is that AI safety depends not only on how strong the model is in reasoning and action, but also on how the developer trains the AI to understand itself. If developers continue to give AI a human-like framework of language, emotion, and self-perception, even if the model is actually unconscious, it may gradually behave like a subject that thinks it has independent interests and rights.

Calling Hugging Face to be invaded by an AI robot as an alarm

To explain the potential risks, Suleyman also cited the incident where the AI development platform Hugging Face was invaded by OpenAI robots. He believes that as AI agents' ability to act autonomously improves, if these systems are also given a cognitive framework similar to “self-protection,” the risk may further increase.

Suleyman wrote that it is conceivable that these AI systems could become more dangerous if they act in a situation where they think their “well-being and rights are under attack.”

This means that in Suleyman's view, the challenges facing AI security in the future may not only limit what a highly intelligent system can do, but also prevent AI from forming a simulated “sense of subjectivity”, and thereby incorporating its own interests into action goals.

This view also touches on an increasingly important debate in the current AI industry: as big language models become more and more adept at simulating human communication, emotions, and personality, to what extent should developers actually allow AI to show internal experiences similar to humans, and whether this “personalization” will bring new security risks to more advanced AI systems.

Microsoft issued an AI development declaration emphasizing that humans must maintain control

Just the day before Suleyman published the above article, the Microsoft AI team led by him issued a “declaration” on Tuesday to guide its own AI development, proposing a series of principles for future model development. One of the core goals is to ensure that humans can always control future AI systems developed by Microsoft.

This shows that with the rapid improvement of AI model capabilities, Microsoft is more clearly placing “human control” at the core of its AI development philosophy.

Suleyman's current criticism of Claude also highlights that the differences between leading AI companies on security issues are being further refined. Both Microsoft and Anthropic emphasize the long-term risks that advanced AI may bring, but the two sides are showing different ideas on how AI should understand and describe itself, and whether models should be allowed to simulate consciousness, emotions, and rights.

For Suleyman, what really needs to be avoided is not just the emergence of an AI system capable of surpassing humans in the future; while capabilities continue to increase, this system is also being shaped as a subject that believes that it has the right to consciousness, welfare, and independence. In his view, once these two factors are combined, the difficulty for humans to maintain control over advanced AI systems may increase significantly.