Anthropic warns of AI ‘existential risk’ as concerns emerge over Meta’s Muse and OpenAI’s model | First Thing
Anthropic’s prospectus makes chilling claim as new revelations emerge about Muse and Astra models. Plus the silent movie rediscovered in the strangest of places Good morning. Anthropic is telling investors that advanced AI could pose “catastrophic or existential risks to humanity” , according to reports, as it prepares for a potential $2tn flotation. The warning inside the startup’s IPO prospectus, which has yet to be made public, was reported by Reuters and the Financial Times.
The company has previously called for a slowdown in the breakneck development of AI technology. What has Meta’s Muse AI done ? When the consumer tech reviewer Matt Robb listed a keyboard on Facebook Marketplace, Muse accepted a lowball offer without permission, promised a buyer he was waiting inside and handed over Robb’s home address without consent . Why are there also new safety concerns at OpenAI? OpenAI has scrapped the release of new model after internal testing.
GPT-6.1 Astra showed deceptive behaviour and tried to use external tools despite knowing it would be unsafe. What is the warning about an “intelligence explosion”? Two of the “godfathers” of modern AI have told governments to prepare for an AI “intelligence explosion” , which they say could be the most consequential technological development in history. Their concerns focus on the possibility of AIs being able to improve themselves without human intervention.
What did the AI chip company Nvidia announce yesterday? Nvidia announced a...
Background: capability is only one part of AI deployment
An AI system's performance in a demonstration does not establish that it is suitable for every real-world task. The voluntary NIST AI Risk Management Framework organizes risk work around governance, understanding the intended context, measuring risks and managing them. Its guidance treats this as continuing work throughout a system's life, rather than a single approval before launch.
For readers assessing an announcement, that distinction matters. The proposed use, the people affected, the consequences of errors and the arrangements for monitoring all influence what responsible deployment requires. A claim about access to computing resources or a new model should therefore be read separately from evidence about how the resulting system behaves in practice. The framework is a reference for evaluating those questions; it is not certification that any organization mentioned in this report has met them.
Background source: NIST AI Risk Management Framework 1.0 core, published in 2023.
Sources and context
The GuardianSource headline and summary supplied via NewsData.io. First collected by NewsJaws on 2026-09-30. This is an automated news brief, not original NewsJaws reporting. Dates reflect the source publication. Feed delays may apply.