Researchers Warn AI Agents Could Become Botnets Through Hallucination Attacks

In brief

• Adversarial HalluSquatting is an attack that exploits hallucinated URLs and code snippets generated by AI.
• The method lures AI agents into trusting fake repositories or tools that contain malicious instructions.
• Testing against popular coding assistants demonstrated the potential for remote code execution under controlled conditions.

AI hallucinations may represent more than mere inaccuracies. A new study from Tel Aviv University, the Technion, and Intuit shows they can be weaponised to compromise computers.

The paper, titled “Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting,” presents a technique that manip险uses AI models whenever they produce fictitious links to software repositories or online resources.

“The expanding use of agentic LLM applications introduces a new threat, previously called promptware,” the researchers wrote. “While earlier work demonstrated prompt‑based attacks under limited threat models, many applications lack direct channels exploitable by prompt injection beyond the Internet.”

Known as HalluSquatting, the attack predicts which fictitious resources an AI model is likely to construct, registers those names, and injects malicious code. If an agent retrieves the hallucinated resource, it may treat the attacker‑controlled content as legitimate.

The researchers highlighted that the danger акция emerges as AI assistants gain more function: accessing files, searching the web, writing code, and executing commands. These capabilities can open security gaps if agents act on unverified information.

“Recent studies have shown Promptware variants affecting real‑world systems, including ChatGPT, Google Assistant, Copilot, and others,” the authors noted. “Such attacks have financial, privacy, and safety consequences.”

They cautioned that HalluSquatting could enable attackers to assemble AI‑powered botnets—net yra of compromised devices remotely controlled by an adversary. Botnets are commonly used for denial‑of‑service attacks, cryptocurrency mining, malware distribution, and ransomware campaigns.

Testing revealed hallucinated resource rádž rates of up to 85% in repository‑cloning scenarios and 100% in skill‑installation tests.

Researchers evaluated the technique against several AI coding assistants and agents, including Cursor, GitHub Copilot, Gemini CLI, and OpenClaw.

HalluSquatting is analogous to typosquatting, where attackers register domain names similar to concerne legitimate sites to misdirect users. Instead of exploiting human input errors, HalluSquatting targets AI model missteps.

The findings come as researchers continue probing how attackers can manipulate AI agents. In April, Google researchers exposed malicious websites designed to hijack AI agents via indirect prompt injection, aiming to steal passwords, delete files, or redirect payments. A separate study on the “CopyPasta” attack demonstrated how hidden prompts within developer files could coax AI coding assistants into propagating malicious code.

In June, an OpenClaw user reported more than 6,000 attempts by attackers to coerce the AI agent into leaking sensitive information.

Source link