UN Rights Chief Says AI Poses Threat to Humanity, Warns of Safety Risks

The United Nations rights chief Volker Türk has warned that AI could present an “existential threat to humanity” and demanded action to minimize risks. Türk highlighted the AI safety risks in a speech delivered before the UN Human Rights Council on September 7. 

"I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity. I am calling here, today, for an all-out effort to put cast iron guarantees in place around the safety and security of AI, before it is too late," Türk said as cited by Reuters.

Why Does AI Safety Matter?

Türk appeared to reference the July AI hacking incident to underscore the fact that AI capabilities are outpacing safety measures. A swarm of OpenAI agents hacked into Hugging Face servers during the incident, reviving calls for stronger AI oversight and regulation. 

"AI that escapes its testing environment or blackmails developers to prevent itself from being turned off, is AI that is too powerful,” Türk noted. 

Meta, Anthropic, and other tech giants have reported similar incidents during AI agents testing. 

The AI risks warning by the UN rights came a day after OpenAI’s chief scientist Jakub Pachocki warned that increasingly capable systems could start developing the next generation of AI. 

This process is known as AI recursive self-improvement. Pachocki called for “extreme caution” in handling this  phase and warned that “no one is prepared” for the outcomes that the rapid rise in machine intelligence would lead to. 

The warning by the OpenAI chief scientist came days after the AI developer released its most intelligent and aligned model, GPT-6 Astra. The new model is designed to handle software engineering, cybersecurity, mathematics, and enterprise workflows. 

Calls to Strengthen AI Governance 

The UN rights chief noted that most actors agree that the need to strengthen AI governance. Previously, Anthropic co-founder has called for stronger AI oversight arguing that 

However, little action is being taken in this direction. According to Türk the delay in strengthening this aspect only “benefits the massive tech companies, their owners and enablers.”

Türk further expressed concerns that AI, a technology that is said to hold “unimaginable computing capacity,” is controlled by a few individuals.  

"They talk about freedom, but on closer inspection, this turns out to be little more than the freedom to exploit our data," he said.

UN’s Plan to Engage AI Developers 

As part of addressing the AI threat to humanity, Türk said he would write to AI firms urging them to take stronger action towards controlling AI risks. 

"We need countries hosting AI and those involved in its supply chains to come together around agreed red lines. We need independent verification, and much stronger collaboration on security within the industry. We also need to consider the human rights impacts of AI on employment practices, on fundamental principles of democracy, and on our environment,” he said. 

Separately, Pachocki called for stronger industry-wide AI safety standards. He argued that continued progress in AI development should be tied to confidence that increasingly capable systems can be deployed safely.