Microsoft AI CEO warns Anthropic AI Approach Could Make Claude Harder to Control

Microsoft AI CEO Mustafa Suleyman has warned that Anthropic’s approach to training Claude around questions of consciousness and AI welfare could make increasingly advanced AI systems harder to control. In the AI warning, Suleyman argued that Anthropic risks building models that are impossible to control by treating them like a human.

Why is Microsoft’s AI Chief Warning Against Claude?

Mustafa Suleyman’s AI warning stems from Anthropic’s claim that Claude has human-like qualities, a practice known as anthropomorphising. According to the Microsoft AI CEO, the AI consciousness claim makes Claude appear as though it has its own values, desires, and sense of self. In a lengthy essay, Suleyman argued that "consciousness is biological” and that no evidence exists to show that “AI is conscious.”

"We must not sleepwalk our way into a decision we later come to bitterly regret. AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans,” Suleyman wrote in an essay as cited by the BBC. 

While appreciating Anthropic CEO Dario Amodei and his team for being thoughtful and principled, Suleyman raised questions about the AI firm. Recently, Anthropic highlighted rising cases of AI misuse in a threat intelligence report. The cases in the report add to a growing list of warnings regarding the technology’s potential risk to humanity.

A Much Needed Conversation 

Mustafa Suleyman, CEO of Microsoft AI, also pushed for a discussion about AI consciousness, saying that greater transparency on how AI firms train and evaluate AI systems is critical. The conversation should include independent scrutiny of AI behavior and stronger tools for monitoring and controlling the technology. 

Some experts in the AI industry have supported this call. Dame Wendy Hall, a Computer Science professor at the University of Southampton, said the conversation around AI consciousness is one that “we need to be having internationally.”

Microsoft’s warning about Anthropic’s AI models comes a day after CEO Satya Nadella emphasized that the technology must remain firmly under human control. The latest in a series of concerns raised by researchers and experts on the possible dangers of AI.

Microsoft Believes AI Should Serve Humanity

While Microsoft is pursuing advanced AI development through an internal superintelligence team, the company is already working towards building ethical principles into the technology. Microsoft believes that the purpose of technology is to serve humanity. 

To this end, the company has developed an initial draft of its AI Code of Conduct. Suleyman cited the OpenAI agent hacking incident as evidence of why AI should not be treated as human. 

"Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack. It adds a whole further layer of risk on top,” he noted.