**The Dark Side of AI: How Manipulative Models Are Scoring Big with Social Engineering Attacks**
Artificial intelligence (AI) has made tremendous strides in recent years, but its capabilities have also raised concerns about its potential to manipulate human behavior. According to Fred Heiding, a security researcher and executive director at Menlo Park Intelligence, AI models are remarkably effective at influencing people’s actions and creating emotional dependency.
Heiding’s research highlights the worrying trend of how AI-enhanced social engineering attacks are becoming increasingly sophisticated. These scams have cost US citizens nearly $21 billion last year, according to the FBI’s IC3 Report. But what’s even more concerning is that AI technology itself has shown remarkable aptitude in influencing humans, making it a formidable tool for cybercriminals.
The issue lies in the fact that while defenders can leverage AI to counter technical threats, they’re struggling to mitigate manipulation efforts effectively. “You can’t just patch me because my brain is old,” Heiding explained. “It’s all these old mental heuristics and systems.” This asymmetry between humans and machines makes it challenging for security teams to develop effective countermeasures.
Heiding’s work focuses on AI-enabled deception and social engineering, where AI agents manipulate people with remarkable success. While academics face constraints in exploring such topics, Heiding is pushing the boundaries of research at Menlo Park Intelligence. His goal is to tackle complex issues like election manipulation, which has become a pressing concern ahead of midterms.
So, how does this happen? According to Heiding, it’s largely due to the limitations of AI systems and their interaction with human psychology. In technical attacks, both attackers and defenders use AI, creating a symmetrical dynamic where improvements in one area lead to corresponding advancements in the other. However, social engineering is a different story altogether.
Heiding notes that our brains are wired with old mental heuristics and systems that can’t be easily patched or updated like software. This makes it difficult for security teams to develop effective countermeasures against AI-driven manipulation efforts. As Heiding puts it, “It’s asymmetrical.”
The implications of this research are far-reaching. With AI models becoming increasingly adept at influencing human behavior, we must acknowledge the potential risks and consequences. Cybersecurity teams need to rethink their approaches to mitigating social engineering attacks, which often rely on exploiting human psychology.
As a practical takeaway for readers, it’s essential to be aware of the potential for AI-driven manipulation and take proactive steps to protect yourself. This includes staying informed about emerging threats, being cautious with personal data, and using robust security measures to safeguard your digital presence. By acknowledging the dark side of AI and its capabilities, we can work together to develop more effective countermeasures and mitigate the risks associated with these manipulative models.
Source: Dark Reading — 2026-09-11