Picture this: You put up photos online from a trip to Brazil. The next day, an email arrives from the hotel where you stayed, thanking you in your visit and providing a special discount to your next stay. The email references your recent trip, the hotel’s name looks right and the link looks legitimate.
But the hotel never sent it. An AI model generated the e-mail after scraping details out of your vacation posts.
Researchers at The University of Texas at Arlington have shown that broadenly available AI tools can collect information from people’s social media posts and use them to quick generate highly highly personalized scam emails targeting large numbers of people, a tactic referred to as context-aware spear phishing. The emails are designed to trick recipients into clicking malicious hyperlinks and handing over sensitive information, consisting of passwords or credit card numbers.
“Anyone can be the attacker, and that makes it scarier,” said Shirin Nilizadeh, an associate professor in UTA’s Department of Computer Science and Engineering who led the study. “With these AI models, they do not need to do this much. They can simply inform them what they want them to do.”
From job titles to daily life
Spear phishing isn’t new. Nilizadeh said most targeted scams have relied on static details about a person, inclusive of their job title or employer. This study examines a more sophisticated type of attack, one which attracts on details of people’s everyday lives shared online.
“Now scammers can use people’s social interactions and their daily lives and use it in their emails,” she said. “It’s more personalized, and it can make people more vulnerable.”
Elham Pourabbas Vafa, a graduate student in computer science and the co-author of the study’s findings, stated the AI tools pulled details from people’s latest social media posts, along with photos, and used that information to create personalized emails. The researchers tested emails based totally on travel, events events were interested about, consisting of concert events, and even concerns that they shared online. They also tailored-subject lines, sender names and links to each recipient.
“The person who receives this kind of spear-phishing e-mail goes to locate it very credible as it’s referencing info particular to their recent experiences,” Pourabbas Vafa stated.
In a user study, participants gave the AI-generated emails high marks for high-quality, underscoring what researchers say makes those attacks so dangerous.
Testing the safeguards
AI companies already use safety filters designed to block harmful requests. But the researchers located that those filters in large failed to detect requests to create personalized scam emails.
The team tested numerous extensively used safety safety , together with Llama Guard, ShieldGemma and WildGuard. With their default settings, the tools detected simply 0% to 19% of the attack requests.
The researchers then proposed ways to prevent the emails from being generated in the first place. When the safety tools were given particular instructions to recognize those attacks, their detection rates rose to as high as 99%.
“We are urging AI companies to take those types of attacks seriously,” Nilizadeh stated. “They want to place safeguards in place to prevent their tools from being misused to create highly targeted social engineering attacks.”
The researchers omitted the precise instructions used to produce the scam emails from the public version of their paper to prevent others from misusing them.
The study has been accepted for presentation at Network and Distributed System Security Symposium in Seoul, South Korea, in March 2027. Pourabbas Vafa also presented the research at the Texas Security Workshop at The University of Texas at Dallas.
