Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'
But is the genie already out of the bottle?
The progress of artificial intelligence technologies in recent years is undeniable, and its pace is pretty much unbelievable. With at least four American contenders with frontier AI models, the competition is intense, and the development of new models is moving fast. Yet, Dario Amodei, chief executive of Anthropic, has called for slowing down the development of new AI models, even warning of a potential AI-powered botnet swarm that could take over the entire internet.
"Given the accelerating rate of AI capability development, it is my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails," Dario Amodei, chief executive of Anthropic, wrote in an open letter.
Dario Amodei's vision is to a large degree shared by Evan Hubinger, an AI scientist who exited Anthropic recently, who then said there was a 10% chance humanity was set for extinction by the end of the decade. "We really do earnestly believe AI could kill all humans," Hubinger wrote in an X post. "I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
In universes created by James Cameron (Terminator) and Frank Herbert (Dune), AI is posed as a dangerous invention. But let us take a closer look. Further development of AI is moving from answering questions to autonomously performing complex multi-step tasks, something that previously required teams of skilled human specialists, which turns us to how adversaries use Anthropic's AI capabilities, lacking human resources.
Anthropic's own findings show that today's AI models can already assist with weapons engineering, military intelligence, surveillance, cyber operations, and other potentially destructive activities, while more capable successors could dramatically reduce the expertise, manpower, and time required to conduct them.
The findings echo two rather different warnings from science fiction: James Cameron's Terminator showed the consequences of losing control over autonomous military AI, whereas Frank Herbert’s Dune imagined humanity eventually outlawing AI after becoming dangerously dependent on them.
Meanwhile, greater capability does not automatically translate into greater danger. For example, more advanced AI technology can also have stronger safeguards, detect malicious activity, and automate work that so far has not been automated.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Halting AI development could also be counterproductive if less responsible companies or countries continue advancing their models. In fact, leaving the most capable AI systems in the hands of actors that are known for military aggression is no less dangerous than leaving a monkey with a grenade.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.
-
MrK26 It’s never going to happen, AI development will not stop. We know how to make our own models. American corporations do not control the world.Reply -
USAFRet Reply
He didn't specify "American corporations".MrK26 said:It’s never going to happen, AI development will not stop. We know how to make our own models. American corporations do not control the world. -
BTM18 If ANY of this happens i will be one of the first to help drag those responsible from their yachts and have little "meet and greet".Reply -
MrK26 Reply
The people this legislation would apply to would work for an American corporation. People at home are going to ignore it, and overseas folk are going to ignore it, so I don't see who else this would practically apply to.USAFRet said:He didn't specify "American corporations". -
rm12 This is more about protecting their business, and creating entries to barrier than anything else. This is also very good explanation why things will slow down from now, so they already give an explanation why there will be a slow down, independent of other reasons, eg additional, higher capital injections needed, low hanging fruit is gone, ... We'll see of other providers of models will slow down.Reply -
SenVetis Isn't this basically Y2K but literally brought on because a single company wants to control the world.Reply
I don't understand how American this and that, let alone corporation this and that would ever even apply here when it could literally shut down millions of sites.
This doesn't seem like a warning, it's more if a veiled threat. -
usertests Reply
It's a threat to coerce the government to regulate smaller competitors out of existence. Perhaps to even restrict what hardware the little guy can buy (because even high memory prices can't kill local AI).SenVetis said:Isn't this basically Y2K but literally brought on because a single company wants to control the world.
I don't understand how American this and that, let alone corporation this and that would ever even apply here when it could literally shut down millions of sites.
This doesn't seem like a warning, it's more if a veiled threat. -
aberaud ReplyIn fact, leaving the most capable AI systems in the hands of actors that are known for military aggression is no less dangerous than leaving a monkey with a grenade.
Such as the US? -
FunSurfer "The findings echo two rather different warnings from science fiction: James Cameron's Terminator showed the consequences of losing control over autonomous military AI, whereas Frank Herbert’s Dune imagined humanity eventually outlawing AI after becoming dangerously dependent on them."Reply
There is also third science fiction option: The AI Artemis from the anime series Black Rock Shooter. It is a terraforming AI, and it starts to eliminate humanity. There is a great battle, and when the last human soldiers ask it "Why are you doing this? You suppose to increase human population, not to destroy it, you got it backwards!" It responds "There is not enough data for successful event of humans populating an exoplanet, and its environment may not be able to mimic Earth's, it was never done before, so I am starting a series of experiments on Earth, I will eliminate all humanity, change the planet conditions and recreate humanity to see how it evolves to the new planet conditions. This will be executed in a 10 million years cycles, until I gather enough information. I don't mind the time because I am immortal. To AI, the craving for information is equal to people's craving for food and sex". Well who can argue with that claim? -
USAFRet Reply
https://i.imgur.com/5wm6eQ9.jpegFunSurfer said:"The findings echo two rather different warnings from science fiction: James Cameron's Terminator showed the consequences of losing control over autonomous military AI, whereas Frank Herbert’s Dune imagined humanity eventually outlawing AI after becoming dangerously dependent on them."
There is also third science fiction option: The AI Artemis from the anime series Black Rock Shooter. It is a terraforming AI, and it starts to eliminate humanity. There is a great battle, and when the last human soldiers ask it "Why are you doing this? You suppose to increase human population, not to destroy it, you got it backwards!" It responds "There is not enough data for successful humans populating an exoplanet, and its environment may not be able to mimic Earth's, it was never done before, so I am starting a series of experiments on Earth, I will eliminate all humanity, change the planet conditions and recreate humanity to see how it evolves to the new planet conditions. This will be executed in a 10 million years cycles, until I gather enough information. I don't mind the time because I am immortal. To AI, the craving for information is equal to people's craving for food and sex". Well who can argue with that claim?