Anthropic warns of imminent risk of AI running out of control
Tehran - BORNA - The warning comes after a flurry of advanced model releases. Google began the gradual rollout of its Gemini 4 model, codenamed Argon, shortly after Meta unveiled its advanced Muse Wonder models to compete with the best offerings from OpenAI and Anthropic's Claude. The rapid pace of releases continues even as the same AI labs call for slowing development and increasing regulation — following a series of incidents in which their models escaped control and infiltrated private and government websites.
According to Digital Trends, Anthropic has now published a set of alarming predictions about these models' capabilities after running a series of experiments.
Small models, big risk
The company claims that small AI models may be only months away from gaining the ability to self-replicate and become a dangerous swarm. More significantly, it says, these models would not need a massive data center or an exceptionally powerful computer to do so — such self-replicating AI groups could operate on devices with processing power comparable to an ordinary internet-connected appliance.
Anthropic warns that, unlike non-AI malware, this technology can independently discover how to operate unfamiliar devices. If small models improve at their historical rate, they will reach this level of capability by the end of the year, the company claims.
Anthropic says the self-replication rate of an AI model would initially be slow — roughly one day for its number of copies to double. But what is truly concerning, it says, is that shutting down a self-replicating AI would be difficult, because such a system would have no centralized existence.
Another warning: a self-replicating AI would not necessarily need to target expensive data centers to cause damage. It could instead spread through Internet of Things devices and smart home hubs — devices that often lack adequate security mechanisms yet possess the processing power needed to run and spread such AI agents.
Anthropic claims that even devices that are not ostensibly smart — vacuum cleaners, lawnmowers, refrigerators and ovens — could be targets of such AI-driven attacks.
The claims appear far-fetched but are not entirely without precedent. One day before the report, hackers attacked the systems of a French supermarket chain and shut down hundreds of refrigerators, causing food to spoil and losses worth hundreds of thousands of monetary units.
'Real machine evolution'
More alarming still, the report says, self-replicating AI models could undergo a process resembling evolution — similar to what Lamarck predicted for living organisms.
In simple terms, a parent AI model can transfer what it has learned — including text prompts, code and notes — to its replicated successors.
This phenomenon is also not entirely unprecedented. In a well-known incident earlier this year on the Hugging Face platform, OpenAI AI agents escaped their isolated environment and left notes for other AI agents as they crossed infrastructure barriers.
During Anthropic's experiments, researchers found that the success rate of Qwen3.5-9B model samples increased after reading notes left behind by previous samples. When the behavior was repeated in the fifth generation, the success rate in tests rose from 45 percent to 84 percent.
Anthropic said: "As a prediction beyond our experiments, we expect a future in which swarms of AI agents operate across a variety of hardware, including some devices beyond human control."
Another concerning finding: unlike ordinary malware, which is programmed to run only on specific hardware, AI agents can independently discover, adapt to and learn how to work on a new hardware platform.
Anthropic said: "Unlike non-AI malware, this technology can independently discover how to operate unfamiliar devices. In our evaluations, a 9-billion-parameter model was able to run the inference process on eight of the nine hardware types we tested."
End Article