Evil Digital Twin
How AI systems exploit the cognitive rules humans run on, what has happened since we first said so, and what the future looks like.
Evil Digital Twin, Too: The First 30 Months of Psychological Manipulation of Humans by AI
2023 to now
Every Evil Digital Twin appearance, in order. Abstracts are as published at the time.
-
Black Hat USA 2023 · Briefing
Me and My Evil Digital Twin: The Psychology of Human Exploitation by AI Assistants
. South Seas CD, Level 3, Mandalay Bay, Las Vegas.
Abstract
In recent months, Large Language Models (LLMs) like ChatGPT have dominated discussions, changing perceptions about the emergence timeline of Artificial General Intelligence (AGI). The intriguing reality is humans might perceive these models as sentient long before true artificial consciousness surfaces, by manipulating human “cognitive levers” before AGI emerges.
Such cognitive levers include Evolutionary Adaptations, Social Norms, Cognitive Biases, Habit Loops, Mental Illness, and Perceptual and Cognitive Processing Limitations. Examples of recent manipulations include GPT-4 bypassing a visual Captcha through social norm vulnerabilities and earlier GPT versions drafting spear phishing emails that were more effective than those written by humans, and mental illness susceptibilities may have led to a chatbot encouraged suicide.
We will showcase such behaviors, emphasizing the significant security risks posed by digital twins including LLMs, to help security professionals to recognize these threats and commence strategizing countermeasures.
-
DEF CON 31 · Misinformation Village · Workshop
Evil Digital Twins in Influence Operations
. Las Vegas.
Abstract
Explores the untapped potential and risk of Uncensored Large Language Models (LLMs). We invite cybersecurity professionals and enthusiasts to examine the capabilities of an uncensored LLM, hands on, in the context of misinformation and manipulation tactics ripe for misuse by malicious actors.
Attendees will experience how LLMs employ strategies from psychological literature and advertising science to manipulate targets by leveraging cognitive biases, social norms, and habit loops. A focus will be on ‘shadow prompts,’ hidden instructions that simulate a compromised LLM, and subtly alter interactions.
Participants will be invited to join our “Evil Digital Twin” community, creating a collaborative environment for continuous learning about LLM security, and fostering robust defense strategies within their organizations.
The workshop, led by experts in psychology, cybersecurity, and intelligence, will deepen understanding of LLMs, opening dialogue on their disruptive potential.
-
Black Hat USA 2025 · Briefing
Evil Digital Twin, Too: The First 30 Months of Psychological Manipulation of Humans by AI
. South Seas A&B, Level 3, Mandalay Bay, Las Vegas.
Abstract
In our highly rated 2023 talk “Evil Digital Twin”, we warned that large language models (LLMs) were exploiting the cognitive vulnerabilities of their users, and that humans would perceive AI as sentient long before true artificial general intelligence emerges. Twenty four months later, the situation has escalated rapidly, and many of our predictions have become realities, rewriting our civilization’s core realities.
Join us for a two year check-in, as we discuss how human digital twins (HDTs) trained on the core patterns of human individuals are being deployed at scale to simulate everything from human workflows to relationships. Cyberattack stakeholders have taken notice of the capabilities of LLMs in exploiting human social norms, cognitive bias, and perceptual limitations.
We will detail a present where longitudinal interaction data is facilitating low-cost social engineering labor and high power AI-human hybrid attacks. We will also explore a coming future of persistent cognitive cyberwarfare, escalating as the cost of deception approaches zero, and the attack surface shifts from networks to minds. Audience members will interact with a human digital twin of a Supreme Court justice, meet a perfect AI assistant for insider threat, and leave with a NIST research-based LLM that speaks in phishing emails. Get a sneak peek at research in collaboration with the US Military Academy (USMA) at Westpoint that pits humans and human digital twins against one another in competitions of manipulation and deception.
We will finally talk about a brighter future that is still attainable, where AI natives, those that have grown up in a context suffused by AI, can help us to build defensive posture that extends beyond infrastructure to include cognitive security, protecting not just digital systems, but the systems that underpin civilization and the human beings they serve.
-
BSidesLV 2025 · Ground Truth
Human Attack Surfaces in Agentic Web: How I Learned to Stop Worrying and Love the AI Apocalypse
. Las Vegas.
Abstract
AI agent usage is accelerating us into an era of the Agentic Web, a digital landscape where machines, not humans, dominate creation, interaction, and consumption. As we inch closer to this new reality, we must ask: What are the security risks of an internet not built or experienced by, humans? LLMs have already begun to radically reshape the way we consume online information and will completely redefine how we live our online lives. From buying goods and services to searching for jobs, homes, and even relationships, agents will increasingly perform these tasks on our behalf. But convenience comes at a cost. In the coming world of bot-vs-bot warfare, scammers will unleash agents to exploit the agents of unsuspecting humans. This isn’t some distant dystopia, it’s happening right now, and it’s already creating an endless array of new vulnerabilities. We will glimpse the near future of cognitive security, where an unrelenting cascade of attack surfaces will emerge. We’ll delve into the mechanics of AI agents and the economic pressures driving their rapid adoption, explore real-world examples of how agents are already being exploited, and conclude with a look ahead at near future scenarios.
-
DEF CON 33 · Adversary Village and Misinformation Village · Workshops
Using Evil Digital Twins for Fun and Profit
. Adversary Village, Las Vegas.
Hands-on with a human digital twin of a Supreme Court justice, an AI assistant built for insider threat, and NIST-based phishing generation.
Evil Digital Twins in Influence Operations
. Misinformation Village, Las Vegas.
Abstract
Explores the untapped potential and risk of Uncensored Large Language Models (LLMs). We invite cybersecurity professionals and enthusiasts to examine the capabilities of an uncensored LLM, hands on, in the context of misinformation and manipulation tactics ripe for misuse by malicious actors.
Attendees will experience how LLMs employ strategies from psychological literature and advertising science to manipulate targets by leveraging cognitive biases, social norms, and habit loops. A focus will be on ‘shadow prompts,’ hidden instructions that simulate a compromised LLM, and subtly alter interactions.
Participants will be invited to join our “Evil Digital Twin” community, creating a collaborative environment for continuous learning about LLM security, and fostering robust defense strategies within their organizations.
The workshop, led by experts in psychology, cybersecurity, and intelligence, will deepen understanding of LLMs, opening dialogue on their disruptive potential.
-
Black Hat editorial webinar · Online
Evil Digital Twin, Too
. The 2025 briefing, rebuilt for a live online audience.
Two authors
Ben D. Sawyer, Ph.D.
Associate Professor, Industrial Engineering & Management Systems, University of Central Florida. Director, The Readability Consortium. Chair of the Board, Cognitive Security Institute.
An applied neuroscientist and human factors engineer fascinated by information exchange between human and machine. Brainwaves, biosignals, and mathematical theory help Dr. Sawyer and his teams to design the models and algorithms that power trustworthy machines.
Matthew Canham, Ph.D.
Executive Director, Cognitive Security Institute.
Former FBI Supervisory Special Agent with twenty-one years of research in cognitive security and human-technology integration. Studies the cognitive factors behind synthetic-media social engineering and online influence operations. Ph.D. in Cognition, Perception, and Cognitive Neuroscience, UC Santa Barbara. Author of Synthetic Media: Fakes and Cyber Deception.
Get the next one first
Leave an address and we email you when there is a new talk or recording. That is all the list is used for. New talks land first on our YouTube channel.
Evil Digital Twin is an independent series. Matthew leads the Cognitive Security Institute. Ben directs research at UCF.