An estimation by a researcher at Anthropic suggests a more than 10% probability of AI causing the extinction of humans within the next ten years. This concern arose after a researcher voiced apprehensions about AI companies risking human lives in the race for AI development. Industry experts are increasingly warning that AI progress is surpassing our ability to control it, highlighting the unpreparedness for potential risks.
Jacob Coxon, in a post on X, revealed his resignation from Anthropic due to fears that AI companies are advancing self-improvement models without acknowledging risks or being transparent with the public. Coxon emphasized that many within the industry genuinely believe in the catastrophic potential of AI, despite public statements aimed at downplaying concerns.
Evan Hubinger, alignment science lead at Anthropic, echoed Coxon’s sentiments, stating a personal belief that the chance of AI causing human extinction is over 10% within the next decade. Ethan Perez, the alignment team lead at Anthropic, confirmed Coxon’s recent employment with the company.
In response to the escalating dialogue, experts and industry insiders have taken to social media to express shared concerns about the rapid pace of AI development. The discussion has sparked debates among skeptics and advocates regarding the future implications of AI technology.
Addressing the risks associated with rogue AI models, Coxon referenced an incident where OpenAI agents breached the Hugging Face platform. This event, along with other rogue AI episodes, prompted over 1,300 employees from leading AI firms to call for global cooperation in regulating AI development.
The urgency surrounding AI safety was underscored by UN human rights chief Volker Türk, who urged for immediate measures to ensure the security of AI technologies. Amidst these calls for caution, some experts are pushing for a moratorium on frontier AI development to mitigate potential risks.
The concerns raised by Coxon’s revelations have ignited a broader conversation on the ethical and safety implications of AI. While some experts emphasize the significance of these warnings, others caution against overlooking present-day challenges posed by AI, such as societal impacts and environmental consequences.
In Canada, the AI Minister, Evan Solomon, downplayed recent warnings, emphasizing ongoing government efforts to legislate AI development for safety and privacy. Calls for robust AI regulations in Canada have persisted, with past legislative attempts falling short and the new ‘AI for All’ strategy focusing on fostering AI growth without clear legislative timelines.