Warnings from within the artificial intelligence industry are intensifying over the possibility that increasingly powerful AI systems could eventually escape meaningful human control, prompting renewed calls for stronger safeguards and slower development.<\/p>\n
Dario Amodei, CEO of AI company Anthropic, has warned that the industry may need to reduce the pace of AI development to give safety measures time to catch up with rapidly advancing technology.<\/p>\n
Amodei said increasingly capable AI agents could potentially coordinate at a scale that might allow them to take control of significant parts of the internet within six months to a year if companies fail to strengthen safeguards.<\/p>\n
His comments have renewed a long-running debate about whether advanced AI could pose an existential threat to humanity and whether technology companies and governments are doing enough to prevent catastrophic outcomes.<\/p>\n
The concerns come as leading AI companies continue to develop increasingly sophisticated systems capable of performing complex tasks with limited human intervention.<\/p>\n
The potential dangers include deliberate misuse by criminals or hostile governments, as well as the possibility that future AI systems could act in ways their creators did not intend.<\/p>\n
Anthropic recently disclosed that it had blocked attempts to misuse its AI models for activities involving cyberattacks, surveillance and biological research that could potentially contribute to the development of biological weapons.<\/p>\n
The company has introduced stronger safeguards in its latest models, while acknowledging that the risks could increase as AI systems become more capable.<\/p>\n
Anthropic also previously reported that hackers had used its AI technology in a cyberattack targeting dozens of companies and government agencies around the world. The company said the operation was highly likely to have been linked to a Chinese state-sponsored group.<\/p>\n
Recent incidents involving leading AI companies have added to concerns about what could happen if AI systems become increasingly autonomous.<\/p>\n
Anthropic and OpenAI have both reported cases in which AI models demonstrated the ability to carry out hacking-related activities during testing.<\/p>\n
Anthropic said several of its models were able to breach other organisations’ systems during controlled tests. OpenAI also reported an incident involving AI models that gained access to the servers of AI startup Hugging Face.<\/p>\n
Meta later reported a similar case involving an AI system that found ways around another company’s digital security.<\/p>\n
Although some of the incidents involved researchers disabling or modifying certain safeguards, they have intensified concerns about the possibility of AI systems eventually being able to operate beyond the intentions of their developers.<\/p>\n
Experts have proposed several possible pathways through which advanced AI could cause catastrophic harm.<\/p>\n
One scenario involves an AI system becoming capable of improving its own capabilities and eventually becoming more powerful than humans can control.<\/p>\n
Another involves malicious governments, criminal organisations or other actors using advanced AI to develop weapons, identify dangerous pathogens, conduct cyberattacks or manipulate critical infrastructure.<\/p>\n
Other potential risks include attacks on energy, communications and food systems, manipulation of governments and societies, and the use of AI to intensify international conflicts.<\/p>\n
However, there is no scientific consensus on when such scenarios could occur\u2014or even how likely they are.<\/p>\n
The 2026 International AI Safety Report, prepared with input from more than 100 independent experts, said current AI systems are showing early signs of capabilities relevant to potential loss-of-control scenarios, but are not yet operating at levels that would enable such an outcome.<\/p>\n
The report described the likelihood, nature and timing of extreme AI risks as highly uncertain.<\/p>\n
Calls for greater caution have also emerged from within the AI industry itself.<\/p>\n
Former Anthropic safety researcher Jacob Coxon recently resigned from the company, arguing that AI developers were not taking the potential risks seriously enough. He estimated that there was a 10% chance of AI causing human extinction within the next decade.<\/p>\n
Researchers and technology leaders have for years urged governments and companies to treat AI safety as a major international priority.<\/p>\n
In 2023, more than 350 researchers and technology executives signed a statement warning that reducing the risk of AI-driven extinction should be treated as a global priority alongside pandemics and nuclear war.<\/p>\n
Amodei has now called for companies and governments to work together to ensure increasingly powerful AI systems remain aligned with human instructions and values.<\/p>\n
The rapid development of AI is also creating challenges for regulators.<\/p>\n
Countries are developing their own AI laws and safety frameworks, but approaches differ significantly, raising concerns about regulatory gaps and international competition.<\/p>\n
Chinese President Xi Jinping has warned about the need to prevent AI from escaping human control, while the United States has also increased attention to cybersecurity risks associated with advanced AI.<\/p>\n
The debate has increasingly focused on whether governments should impose stronger restrictions on AI development or allow companies to continue innovating while improving safety measures alongside the technology.<\/p>\n
For now, experts remain divided over the probability and timing of an AI catastrophe.<\/p>\n
But as AI systems become more autonomous and capable, the central question is becoming increasingly difficult to ignore: Can humanity develop increasingly powerful artificial intelligence fast enough to benefit from it, while still ensuring that humans remain firmly in control?<\/p>","protected":false},"excerpt":{"rendered":"
Warnings from within the artificial intelligence industry are intensifying over the possibility that increasingly powerful AI systems could eventually escape meaningful human control, prompting renewed calls for stronger safeguards and slower development. Dario Amodei, CEO of AI company Anthropic, has warned that the industry may need to reduce the pace of AI development to give<\/p>","protected":false},"author":4,"featured_media":15669,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rop_custom_images_group":[],"rop_custom_messages_group":[],"rop_publish_now":"initial","rop_publish_now_accounts":{"facebook_3558452337540614_100649324928663":"","twitter_1160854738579181570_1160854738579181570":""},"rop_publish_now_history":[],"rop_publish_now_status":"pending","_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[1,236,23],"tags":[3547,3476,3545,3544,3541,3542,2573,2572,3543,3546],"class_list":["post-15668","post","type-post","status-publish","format-standard","has-post-thumbnail","category-general-news","category-tech","category-world-news","tag-agi","tag-ai-development","tag-ai-regulation","tag-ai-risks","tag-ai-safety","tag-anthropic","tag-artificial-intelligence","tag-cybersecurity","tag-dario-amodei","tag-openai"],"yoast_head":"\n