Artificial intelligence (AI) is enabling threat actors to send unholy volumes of fraudulent emails, personalized to a degree that mass emailers of old couldn’t have touched.
Last month, Microsoft researchers tracked a phishing campaign in which an unattributed threat actor sent out more than 1 million emails in just three days. Despite the volume of content they had to juggle, the threat actor managed to specifically target relevant accounts payable departments and lightly personalize each message with the real names of targeted employees’ executive leadership, most likely through serious assistance from AI. A few other bells and whistles, too, made the emails much more convincing than your average Automated Clearing House (ACH) fraud scam.
Personalized Mass Phishing
The basic premise of the emails was that victims’ companies owed just shy of $50,000 to the enterprise cloud services company ServiceNow for an annual subscription. Attached invoices were detailed — with credible line items and dollar amounts that didn’t add up to round numbers — and featured visual elements and branding true to the impersonated company.
In a clever little twist, the attacker wrapped each phishing email and invoice in a forged email “thread.” They created a brief back-and-forth conversation, made to seem like it occurred before the victim received the email, between an executive at a victim’s company and the president of ServiceNow. The executive asked their colleague at ServiceNow to forward the invoice directly to the victim in their finance department.
A sample of a spoofed message from an executive team member in a massive ACH scam campaign. SOURCE: Microsoft
To give extra credence to these exchanges, the attackers identified CEOs, CFOs, and presidents at victim organizations, and plugged them into the phishing emails’ signatures. It would have been an easy task to throw at a chatbot.
“Gathering information about a victim organization can now be a matter of minutes,” says Merium Khalid, director of AI and automation for the Office of the CTO at Barracuda Networks. “AI can rapidly process publicly available information on the Internet about an organization — such as company websites, executive information, employee roles, press releases, and other public sources — and use that context to help construct more convincing impersonation emails.”
She adds that attackers can build multiple AI-driven processes for different parts of a campaign, including information gathering, generating personalized content, and creating templates that can be used for a variety of targeted organizations. Indeed, Microsoft found a few telltale signs that the threat actor behind this mass email campaign used AI to assist in personalizing its email template to specific victims.
The end result was relatively convincing mass phishing, beyond what would have been possible even a few years ago. Phishers used to be able to send generic bulk emails, or personalized emails for select targets.
Here, the attacker generated more than 1 million, personalized to each recipient, in a roughly 48-hour period, from Aug. 3 to Aug. 5. The emails were blasted to organizations across industries — IT, consumer goods, and real estate companies being the most common targets — 87.7% of which resided in the US.
AI’s Greatest Threat (for Now)
Despite this summer’s head-spinning, futuristic headlines — of rogue waves of AI agents attacking innocent companies with wanton abandon — experts emphasize that AI’s greatest proven threat so far is its ability to enhance old-fashioned kinds of cyberattacks.
“New AI-native threats like adversarial agents and prompt injection will create new risks, but phishing, impersonation, and fraud already have proven paths to success,” Joshua Bartolomie, vice president and global head of threat intelligence at Doppel, tells Dark Reading. “AI makes those attacks faster, cheaper, more personalized, and easier to scale. For example, near-simultaneous impersonation campaigns spanning multiple surfaces have grown roughly sevenfold in two years. The bigger near-term risk is not necessarily a new attack, but the industrialization of effective ones.”
Since AI is enhancing existing kinds of attacks more than creating new ones, long-established best practices for cyber defense still apply. In its blog post, Microsoft’s recommendations for companies — properly configuring email authentication and spoof protection, implementing email security filters and extended detection and response (XDR) — included little about those cutting-edge, AI-powered cyber defenses common in marketing campaigns today.
Khalid recommends that organizations balance investing in straightforward cybersecurity hygiene and frontier tooling. “This campaign is a good example,” she says, since standard email filters can pick out malicious artifacts in incoming messages, but AI-powered email protection can analyze those signals at machine speed.
“AI isn’t a replacement for fundamental security practices,” she says. “Ultimately, the best defense is layered: good cyber hygiene and processes, educated employees, and AI-powered security technologies capable of detecting and responding at the same speed attackers are increasingly operating.”
Comments are closed