Cybercriminals are taking advantage of a new large language model (LLM)-driven attack vector called “phantom squatting” to threaten the software supply chain by registering nonexistent domains linked to legitimate brands to intercept traffic generated by AI systems. One attacker even used an AI coding assistant to build a full phishing kit targeting a high-risk phantom domain that researchers had identified earlier.
LLMs consistently hallucinate Web domains for legitimate brands, and this leaves the door open for cybercriminals to exploit of these domains, according to research from Palo Alto Networks’ Unit 42 published Tuesday. The researchers analyzed 913 global brands via 685,339 URL queries across multiple configurations of two distinct LLM models, which generated 250,000 hallucinated domains. They exist alongside the more than 13,220 confirmed malicious URLs related to the brands, the researchers found.
Unit 42 compared hallucinations of Web domains to how LLMs frequently hallucinate software package names that do not exist in any registry. “Just as an LLM might hallucinate a library name, it can generate fictitious domains for Web portals, API endpoints, or corporate services for a target brand,” Unit 42 researchers wrote in the report. People making requests to AI assistants that are directed to the phantom portals are then at risk for malicious activity hiding behind them, they said.
Given this, and the fact that LLMs now exist “as a trusted supply chain dependency” across many enterprises, these hallucinated domains are emerging as a significant threat stemming from the use of AI assistants across the enterprise.
Exploit Paths for Phantom Squatting
One exploit path is if a coding assistant generates a plausible but unregistered benefits portal URL, which would allow an adversary to preemptively register it. Another is if an AI research agent produces a plausible banking portal domain that an adversary could have already registered to capture traffic, according to Unit 42. A third way is if a developer integrates an AI-generated API endpoint into their code, unknowingly directing application data to an attacker-controlled server.
“The attack chain is simple: probe models for invented domains that appear repeatedly, register the most useful names, place phishing or malicious content behind them, and wait for a person (or, increasingly, an autonomous agent) to follow the recommendation,” Johan Edholm, security engineer and co-founder at Detectify, tells Dark Reading. “It’s cheap, repeatable, and scalable, which is what actually makes an attack dangerous.”
Phantom squatting is related to typosquatting, with a key difference, he says. Typosquatting waits for someone to mistype a known domain, while phantom squatting waits for a model to invent a plausible one to which users are directed.
This makes detection of the vector more difficult, “because the domain may sit outside the predictable variations defenders normally monitor, and a newly registered domain begins with little or no reputation history,” Edholm says.
‘Montana Empire’ Built on a Ghost
Unit 42’s proactive monitoring of high-priority hallucinated domains detected registrations of phantom domains by would-be attackers 18 to 51 days after initial identification, they said. In one case, this led to the flagging of a “high-risk” postal service e-commerce domain 23 days before registration that later was used as the victim-facing site for a phishing kit called “Montana Empire.”
The attacker used an AI coding assistant to build the full phishing kit, including scraping legitimate storefronts, building the PHP back end, and establishing a Telegram-based command-and-control (C2) before registering the domain, which later was used for credential theft.
“This case demonstrates the full cycle of the phantom squatting supply chain threat model,” the researchers observed. “The adversary used AI systems to generate attack tooling against infrastructure identified by our discovery pipeline 23 days earlier. Both parties arrived at the spoofed domain via the same mechanism, the LLM’s internal prediction of a structurally inevitable hallucination for the target brand.”
Unit 42 found additional cases in which phantom squatting was used to target national postal services and other sectors with phishing and a malicious Android application.
Potential Evolution and Mitigation for Phantom Squatting
The danger in attackers abusing phantom squatting is that the delivery mechanism for the malicious activity has already been sanctioned by the system, Edholm says. “The recommendation arrives through a trusted assistant rather than a phishing email, so it inherits credibility the attacker never had to earn, and it bypasses defenses that depend on a domain having a bad reputation first,” he explains.
The vector can also potentially evolve from not only providing misleading answers to questions or recommendations, but to executing “an automated supply-chain compromise without ever requiring a human click,” Edholm says. “That’s the direction to plan for: the point of failure shifts from a person following bad advice to a system acting on it on their behalf,” he says.
To protect against the existing threat and prepare for the future, Edholm recommends that organizations set up security protocols that verify URLs against authoritative documentation or approved allowlists, as well as prevent AI agents from connecting freely to arbitrary new domains. They also should tightly limit the credentials and data that those systems can access. “In short,” he advises, “don’t let a confident-sounding recommendation become an action without an independent check standing in between.”
No responses yet