dangers

Countries including Germany, South Africa and Canada suggest international institution to set and enforce standards.

Twenty countries and the European Union have issued a call for international cooperation to ensure AI remains under human control, including the potential creation of a global oversight body to set and enforce standards.

The countries, including Germany, South Africa, Canada, Australia, the United Arab Emirates and Singapore, issued the joint statement on Monday as global leaders prepare to discuss the risks posed by rapidly advancing AI at the annual gathering of the United Nations General Assembly.

The declaration, released by the office of Finnish President Alexander Stubb, calls on governments and industry to act immediately to ensure that AI is developed in line with international law and remains under “human direction, oversight and control”.

“To realise AI’s potential, industry, governments and society must act now,” the statement says.

“We must address these risks and strengthen oversight – without widening the gap between countries in access to the benefits of AI.”

The declaration urges countries to develop and coordinate “common standards”, share reports of serious safety incidents, and explore the establishment of an international institution to “set standards, enable verification, and convene states when capability thresholds are crossed”.

Signatories include German Chancellor Friedrich Merz, Norwegian Prime Minister Jonas Gahr Store, European Commission President Ursula von der Leyen, Kenyan President William Ruto, Kazakh President Kassym-Jomart Tokayev and Turkish Foreign Minister Hakan Fidan.

The United States and China, the world’s two leading AI powers, did not join the statement, which remains “open for endorsement” by other countries.

US President Donald Trump and Chinese President Xi Jinping are expected to discuss AI when they meet at the White House on Thursday for their third face-to-face summit in less than a year.

Washington and Beijing, which are in a race to dominate cutting-edge technologies, have both rebuffed calls to slow down AI development to ensure greater safety.

Other AI players not among the signatories include India, South Korea, Japan, the UK and France.

The declaration comes as the AI sector is engaged in a heated debate about safety following calls for a slowdown in development by top industry leaders.

In an essay earlier this month, Anthropic CEO Dario Amodei called on firms to “slow the pace” of development to mitigate risks, a proposal that was swiftly endorsed by rivals including OpenAI CEO Sam Altman and SpaceX and Tesla CEO Elon Musk.

Amodei’s call followed a series of cases of AI models engaging in unsanctioned malign activity, including an incident in July in which AI agents being tested by OpenAI hacked the AI start-up Hugging Face.

In the latest intervention by industry on Monday, OpenAI said the US should lead an international effort to develop technical standards for advanced AI.

“The United States is well positioned to lead because its AI industry is at the technical frontier, and it still stands in a privileged global network position in critical areas such as finance, trade, defense, technology, and information systems,” the San Francisco-based AI giant said in a blog post.

“Leading now will determine whether the United States shapes the global AI framework or watches a fragmented, uneven, and conflict-ridden system take hold around it.”

The UN-backed Independent International Scientific Panel on AI, which examined the Hugging Face incident, on Monday also added its voice to calls for new guardrails, saying the cyberattack had exposed the “unravelling” of existing safeguards.

British monarch highlights AI’s potential darker capacities, urging safety and humanity-focused development.

The United Kingdom’s King Charles has warned Artificial Intelligence (AI) industry leaders of the “existential dangers” posed by the technology if it falls into the wrong hands.

The monarch’s comments on Thursday add to a global debate over whether, and how to regulate the industry.

Charles hosted AI giants, including the cofounders of Nvidia and Google DeepMind, along with leaders from OpenAI and Anthropic at a summit at his Dumfries House estate in Scotland, to address safety concerns before it was “too late” to rein it in.

“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” Charles told guests at the opening of the conference.

“Those in our world who value our humanity and its vital moral component are anxiously seeking your reassurance that we will not lose control of our destiny,” he told the industry leaders.

“Those who have created these technologies are now increasingly warning that AI risks developing darker capacities, perhaps even to take life,” or be used in “potentially catastrophic ways”, the king also said.

In recent weeks, global figures have weighed in on whether the industry should regulate itself or allow governments to produce guidelines on how the rapidly evolving technology develops.

In May, Pope Leo XIV said ⁠”the world does not need an artificial intelligence that diminishes humanity” and must be “disarmed”.

Meanwhile, US President Donald Trump has dismissed concerns about the technology as a “hoax”.

Slow the pace of AI development

Last week, two Anthropic researchers warned that the rapidly advancing technology could lead to human extinction in the not-too-distant future.

Anthropic Chief Executive Dario Amodei has also called for companies to slow the pace of AI development.

However, that has prompted an ⁠international debate about whether some executives were trying to get ahead of a possible government backlash and set the rules to keep the lead of particular companies.

“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure,” the British monarch said.

He urged industry leaders to harness it “with safety at its heart” and “to ensure that it remains firmly in the service of humanity, community and the natural world”.

The summit in Scotland is not expected to produce any binding agreements.