New hack exploits AI hallucinations to trick agents into running malicious code — 'HalluSquatting' attack exploits a fundamental weakness in every available model

News
By
Published

Today, a tale from the book of We Told You So.

AI robot agents
(Image credit: Getty Images)

Ever since the advent of agentic AI, security researchers have been yelling from the top of their lungs about how it's a bad idea to grant user-level permissions to an LLM — for all purposes, a program with non-deterministic outputs and inconsistent handling of inputs. A research paper on HalluSquatting from researchers at Tel Aviv University, Technion, and Intuit, shows how easily one can fool modern AI bots and harness them into a massive army of AI agents, with the research showing that agents can hallucinate potentially malicious code repositories up to 85% of the time.

The mechanism for HalluSquatting (aka "adversarial hallucination squatting") is surprisingly simple, and takes advantage of the fact that when met with unfamiliar terms, bots will not know they're incorrect and hallucinate a "correct" answer. Adding to that, the methods the bots use to come up with said answer are predictable, for example, owner/repository or toolname/toolname GitHub URLs. This is different than just standard typo-squatting, as it exploits the hallucination mechanism itself.

An attacker first identifies an application, code repository, programming library, or bot skill that's gained popularity only in recent months or years — let's say, a new GitHub repo with the URL OriginalOwner/WindowsTelemetryOff. As the bots' training data is not recent enough to contain information about it, GitHub URLs owner/repo combinations SuperHacker/WindowsTelemetryOff , and WindowsTelemetryOff/WindowsTelemetryOff look just as peachy. Likewise, WindowsTelemetryOf and WindowTelemetryOff (note the typos) will be valid candidates.

Latest Videos From

The attacker then creates a malicious repository using those generated names. When Claude or another code agent is asked to "run the windowstelemetryoff scripts" or a similar instruction, chances are they'll hallucinate the repo name (sometimes even having run a web search), run into the malicious version that looks like the original, and happily run whatever's in there.

From that point, all bets are off now that the attacker's code is running on the user's machine. The most obvious outcome could be creating a reverse shell (the user's machine opens a command line that's controlled remotely). Now having access to the user's account, the attacker can siphon off their data and passwords, install software, run crypto miners, or harness their AI agent for further malfeasance, all with the power of entire data centers at their disposal.

And here's the kicker: just the one HalluSquatted piece of software has the potential to bait and reel in tens of thousands of bots, if not more, in a proverbial blink of an eye. A crafty attacker would be kind enough to include all the original code in their poisoned version, adding yet another layer of unawareness to the mix.

The research team found that an LLM will hallucinate the location of a recent code repository up to 85% of the time, a figure that can reach 100% for trending agentic skills. Every single model is widely affected, up to and including Anthropic's mighty Claude Opus 4.5. At the application level, the figures are better, but still pretty bad.

The scientists are working on common LLM-backed programming applications, including Cursor, Windsurf, and OpenClaw, among others. In this scenario, the bots stand a better chance given they're working with more context information, but even still, the success rates for hacking ranged from 20%-35% for Cursor, Gemini CLI, and Copilot, and increased massively to close to 80-100% on OpenClaw and its variants. The exploit mechanism doesn't even need to be crafted specifically for any bot; the researchers' results show it's universal and transferable, too.

The mean hallucination rate for names of sample GitHub repositories published in 2025 is 92.4%, while predictably, bots get the URLs wrong 0.9% for those from 2019 or earlier, though that's arguably still a concerning figure. The most effective mitigation is adjusting workflow: instructing bots to always run web searches before installing software, and providing them with additional context. Unfortunately, that's not the default way most people appear to use them.

Cybersecurity professionals have long advocated for not blindly trusting a bot's actions and severely restricting the access level granted to AI agents. And yet it's not uncommon to see bots with wide-ranging permissions over users' machines, API keys, access keys, and service accounts, to name a few — all in a bid to make it "easier" for the bot to vibe-code their pointy-haired-boss' latest brilliant idea.

Google Preferred Source

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

Bruno Ferreira
Bruno Ferreira
Contributor

Bruno Ferreira is a contributing writer for Tom's Hardware. He has decades of experience with PC hardware and assorted sundries, alongside a career as a developer. He's obsessed with detail and has a tendency to ramble on the topics he loves. When not doing that, he's usually playing games, or at live music shows and festivals.

10 Comments Comment from the forums
  • PEnns
    "The most obvious outcome could be creating a reverse shell (the user's machine opens a command line that's controlled remotely)"
    As usual, the weakest link / silliest scenario: If the attacker ALREADY has access to the user's machine, why all this commotion about having AI do nefarious stuff...if the remote attacker can do even worse with / out AI??
    Reply
  • rluker5
    The takeaway from this story could be: don't let an AI agent find and run software for you from the internet unchecked. Who would do such a thing anyways? In the future it may be the default due to a Windows update.

    Just yesterday, while my daughter was at piano lessons, I was checking out water softener options and wanted to know the water hardness in my locality. I searched for the answer using Edge on my phone and could only get that AI summary and nothing else. I tried several times and it kept repeating so I switched the search engine from Bing to something else and had the AI headline paragraph again which continued to state that my water hardness was either 21 or 3.4 grains per gallon which was a worthless answer, but that was now followed by the actual search results which I used to get a real answer.

    Who would make my mobile search sometimes only include the AI summary as an answer? In this case Microsoft. The push for AI use at the expense of functionality or security is real.
    Reply
  • rluker5
    PEnns said:
    "The most obvious outcome could be creating a reverse shell (the user's machine opens a command line that's controlled remotely)"
    As usual, the weakest link / silliest scenario: If the attacker ALREADY has access to the user's machine, why all this commotion about having AI do nefarious stuff...if the remote attacker can do even worse with / out AI??
    The weakest link is that agentic AI is the one that gives the attacker this access by running code from an imposter website, which all agentic AIs apparently are prone to doing.
    Reply
  • edzieba
    PEnns said:
    "The most obvious outcome could be creating a reverse shell (the user's machine opens a command line that's controlled remotely)"
    As usual, the weakest link / silliest scenario: If the attacker ALREADY has access to the user's machine, why all this commotion about having AI do nefarious stuff...if the remote attacker can do even worse with / out AI??
    Did you not actually read the article, or merely not understand it?

    This attack allows an attacker to host malicious code on Github (or similar repos) with names similar to popular repos - same concept as DNS typosquatting, but targeting LLM hallucination malformations rather than human typing malformations - and all that attacker then needs to do is sit and wait, as 'AI agents' will hallucinate names similar to the requested repo, find that hallucinated repo already exists, then blindly execute code from that repo.
    The attacker does not require any prior access to the targets, the targets come to them and pwn themselves.
    Reply
  • PEnns
    edzieba said:
    Did you not actually read the article, or merely not understand it?

    This attack allows an attacker to host malicious code on Github (or similar repos) with names similar to popular repos - same concept as DNS typosquatting, but targeting LLM hallucination malformations rather than human typing malformations - and all that attacker then needs to do is sit and wait, as 'AI agents' will hallucinate names similar to the requested repo, find that hallucinated repo already exists, then blindly execute code from that repo.
    The attacker does not require any prior access to the targets, the targets come to them and pwn themselves.

    Nah, us peasants don't.

    Only you can read and comprehend and willfully skip over the quoted part, FROM THE ARTICLE!
    Reply
  • JohnyFin
    Don't know ...to cry or laugh ..
    Reply
  • Moonstick2
    PEnns said:
    Nah, us peasants don't.

    Only you can read and comprehend and willfully skip over the quoted part, FROM THE ARTICLE!
    PEnns said:
    "The most obvious outcome could be creating a reverse shell (the user's machine opens a command line that's controlled remotely)"
    As usual, the weakest link / silliest scenario: If the attacker ALREADY has access to the user's machine, why all this commotion about having AI do nefarious stuff...if the remote attacker can do even worse with / out AI??
    The "obvious outcome" above is referring to a possible result of the AI running a malicious script it fetched from a similar-sounding repo because its training set didn't contain the real one.

    The whole point is that the attacker doesn't have access to the user's machine until the AI gives it to them. The attacker puts a poisoned repo/skill online in a place that hallucinating AIs are expected to find it when trying to find something else; one day a user on their local machine asks their AI to download a repo or skill; the AI goes looking, finds the poisoned one, mistakes it for what it's looking for, runs it and creates a privileged reverse shell on the local machine that gives the attacker access. The more autonomous and widespread the AI and the more automated such a fetch request is dispatched to the AI, the more likely this is to happen.

    The non-AI equivalent is creating a fake page for a trending download using an address that people might mistype (e.g. www.chatpgt.com) or looks plausible (e.g. www.chatgptforwindows.com) and waiting for unwary users to download your dodgy executable.

    The reason this is worse is scale and speed: using the AI vector you could capture orders of magnitude more computers in far less time than you would waiting for people who mistype addresses and run the executables without noticing anything off.
    Reply
  • qwertymac93
    rluker5 said:
    The takeaway from this story could be: don't let an AI agent find and run software for you from the internet unchecked. Who would do such a thing anyways? In the future it may be the default due to a Windows update.

    Just yesterday, while my daughter was at piano lessons, I was checking out water softener options and wanted to know the water hardness in my locality. I searched for the answer using Edge on my phone and could only get that AI summary and nothing else. I tried several times and it kept repeating so I switched the search engine from Bing to something else and had the AI headline paragraph again which continued to state that my water hardness was either 21 or 3.4 grains per gallon which was a worthless answer, but that was now followed by the actual search results which I used to get a real answer.

    Who would make my mobile search sometimes only include the AI summary as an answer? In this case Microsoft. The push for AI use at the expense of functionality or security is real.
    You were in copilot search mode, if you scroll and look at the top, there's an "all" button, that brings you to the traditional Bing search. Not that you should stick with Bing in the first place, and not to discount the fact that Microsoft may have intentionally obfuscated this by hiding everything but the AI prompt and response until you scroll, tricking you into thinking that was your only option.
    I prefer search.brave.com as it's the most old-school style search engine that still has an AI summary, if you want that sort of thing (and it can be turned off).
    Reply
  • edzieba
    PEnns said:
    Nah, us peasants don't.

    Only you can read and comprehend and willfully skip over the quoted part, FROM THE ARTICLE!
    Not understand, got it.
    Reply
  • DRagor
    "AI hallucinations".
    WHAT?
    Reply