• Last time: AI and wellbeing (attention, mental health, environment)
• Algorithms optimise engagement, not wellbeing. Filter bubbles look overstated; anxiety, sleep, and body-image effects don’t
• AI mental-health tools: short-term promise, thin long-term evidence. Augmentation, not replacement
• Training is a one-off cost; inference at scale is what keeps growing
• Common thread: incentive structures beat intentions
• Today: what happens when AI is used to deceive?

Part 1: Misinformation and AI

• What is misinformation?
• AI and synthetic media
• Why it matters now

Part 2: Deepfakes

• Technical foundations
• Real-world examples

Part 3: Why we fall for it

• Evolutionary wiring and fake news
• System 1 vs System 2
• Confirmation bias and motivated reasoning
• Bandwagon effect and the “in the know” feeling

Part 4: Responses

• Technical detection
• Platform policies
• Legal frameworks
• Media literacy

Source: System Card: Claude Mythos Preview

Source: X.com

The trichotomy (Wardle & Derakhshan, 2017, Council of Europe):

A related but broader concept: synthetic media (content created or manipulated by AI)

Why the distinction matters:

• Intent shapes the appropriate response
• Different actors, different motivations
• Legal treatment and solutions vary by category

The old problem:

• Misinformation is ancient
• Technology changes scale and speed

Before generative AI:

• Fake images required Photoshop skills
• Fake videos needed studios, fake audio needed voice actors
• Production was expensive and slow
• Detection was often possible

After generative AI:

• Anyone can create convincing fakes
• Production cost approaches zero
• Quality improves rapidly
• Detection becomes harder as generators improve

The scale problem:

• Thousands of fake articles, generated in minutes
• Targeted content for specific audiences
• Human moderators can’t keep up (Lazer et al., 2018)

How platforms amplify misinformation:

• Engagement = revenue
• Outrage drives engagement
• False information is often more engaging
• Algorithms optimise for clicks, not truth

The correction problem:

• Lies spread faster than corrections
• False stories reach millions
• Corrections reach thousands
• Even debunked claims persist in memory

Vosoughi et al. (2018):

• Analysed 126,000 stories on Twitter
• False stories 6x more likely to be retweeted
• Reached more people, spread faster

Political deepfakes:

Financial fraud:

• CEO voice clone authorised £200K transfer
• Estimated $25 million scam in Hong Kong (2024)

Non-consensual intimate imagery:

• 96% of deepfakes online (Sensity/Deeptrace, 2019)
• Women overwhelmingly targeted: a tool for harassment and extortion
• 2024 update from Home Security Heroes put the share at 98%
• Legal response is lagging

• Deepfakes are one tool among many
• “Cheap fakes” often just as effective (Paris & Donovan, 2019 coined the term)
• Context matters more than technical quality
• Social vulnerability enables technical attacks

The liar’s dividend:

• When anything could be fake…
• Real evidence can be dismissed as fake
• “That video of me is a deepfake!”
• Truth becomes contested terrain

RAND’s threat assessment:

• Near-term: Targeted harassment, fraud
• Medium-term: Election manipulation
• Long-term: Erosion of shared reality

Chesney & Citron (2019):

• Frame deepfakes as an epistemic threat: a danger to public knowledge itself, separate from privacy or fraud harms
• News loses the ability to verify its own sources
• Politicians get plausible deniability for everything caught on camera
• People retreat further into the views they already hold

What the empirical work has found since:

• Vaccari & Chadwick (2020): UK survey (N=2,005) showed people are more uncertain after seeing deepfakes than directly deceived, and this lowers trust in news overall
• Schiff, Schiff & Bueno (2024): five experiments, 15,000 Americans. False “deepfake!” claims help politicians dodge scandals, but mostly for text-based reporting, not video (so far)

The evolutionary angle:

• Our brains evolved for small groups and immediate threats
• Novelty once meant survival-relevant information
• We still react to new, surprising claims the same way
• False news is almost by definition more novel than truth

Why outrage spreads:

• Threat detection is fast and automatic
• Emotionally charged content bypasses careful evaluation
• Fear and disgust trigger sharing behaviour
• We evolved to warn the group, not fact-check first

Kahneman’s (2011) dual-process theory:

Social media is a System 1 environment:

• Infinite scroll rewards quick reactions
• You see a video, react, share, move on
• System 2 never kicks in unless something jolts you
• Deepfakes are designed to feel real at System 1 speed

The problem:

• Catching a fake requires System 2
• But strong emotion (outrage, fear) suppresses System 2
• Pennycook & Rand (2019) tested this directly: people who score higher on cognitive reflection tests are better at spotting fake news, regardless of party

What it is:

• We seek evidence that confirms what we already believe
• We dismiss evidence that contradicts it
• This isn’t laziness; it’s how brains manage information overload
• Nickerson (1998) called it the most pervasive bias in human reasoning

How it works with deepfakes:

• Who said it? If it’s someone we trust, we’re more likely to believe it, even if it’s fake (the opposite is also true)
• Taber & Lodge (2006): people argue back when corrected, sometimes hardening their position
• Corrections still feel like an attack on identity, even when they aren’t

What Asch (1956) showed in the 1950s:

• People will deny what their own eyes tell them if enough others disagree
• About 75% of participants conformed at least once on an obvious task
• Bond & Smith (1996) replicated Asch across 17 countries and confirmed the effect, though weaker in individualistic cultures

On social media, numbers are social proof:

• A video with 2 million views “must be real”
• We outsource our judgment to the crowd

How this helps deepfakes spread:

• Early shares create a snowball
• Each share adds perceived legitimacy
• By the time fact-checkers respond, millions have seen it
• Taking it down can actually increase belief (“they’re hiding something”)

Sharing as social currency:

• Breaking news makes you feel like an insider
• “Did you see this?!” = status in your group
• Being first to share gets the attention

Why this matters:

• Creates a race to share before verifying
• People share content they haven’t fully read (Gabielkov et al., 2016: 59% of links shared on Twitter were never clicked)

Combined with the other biases:

• Confirmation bias picks what we share (things we agree with)
• The bandwagon effect picks when (once it’s already popular)
• The “in the know” feeling picks how fast (immediately, no checking)

Detection approaches:

Why it’s hard:

• Detectors can be fooled
• Adversarial training improves generators
• Real-time detection is difficult
• Scale: Millions of images, few moderators
• Groh et al. (2022) tested ordinary people against a state-of-the-art detector on 5,000 deepfakes: humans and the model made different mistakes, and combining them beat either alone

A new idea:

• Instead of trying to catch fakes after the fact, verify the real thing at the source.
• The C2PA standard (Adobe, Microsoft, BBC, the New York Times and others) embeds a cryptographic signature when a camera takes a picture, then logs every edit
• Viewers can check the chain back to source, and unsigned content becomes the suspicious one

Why it isn’t a fix on its own:

• Needs near-universal adoption to be useful at all
• Legacy content has no provenance to attach
• Signatures can be stripped, screenshots wash them out
• Tracks creation, but doesn’t prevent it
• Real privacy concerns about tying every image to a device and identity

What platforms actually do:

• YouTube requires creators to label realistic AI-generated content
• Meta tags AI-generated images and video using C2PA
• TikTok requires disclosure
• X has rolled back most of its 2020 synthetic-media rules

What happens when platforms remove content:

• Moves to less moderated platforms
• Removal can increase distrust
• Bak-Coleman et al. (2021) argue platforms function as a planetary-scale behavioural system and need the same crisis discipline as climate science
• Piecewise content rules cannot fix system-level effects

Emerging legislation:

• US: Texas SB 751 and similar state laws target election deepfakes
• EU: AI Act Article 50 requires deepfake disclosure
• UK: Online Safety Act plus a 2025 amendment criminalising the creation of non-consensual sexual deepfakes
• China: Deep Synthesis Provisions (2023) require disclosure of AI-generated content

Challenges:

• Jurisdictional: creators can be anywhere
• Enforcement: finding perpetrators is hard
• Definition: what counts as a “deepfake”?
• Speech: satire, art, legitimate uses

Misinformation and AI

• Misinformation is old; AI changes scale
• Attention economy amplifies false content
• Platform incentives work against truth

Deepfakes

• Synthetic media quality improving rapidly
• Political manipulation, NCII, fraud
• The “liar’s dividend” undermines all evidence

Why we fall for it

• Evolutionary wiring makes novelty irresistible
• System 1 scrolling, System 2 sleeping
• Confirmation bias is the vulnerability no patch can fix
• Bandwagon and “in the know” feeling accelerate sharing

Responses

• Detection: necessary, insufficient
• Provenance: promising, years away
• Legal: slow, jurisdictional
• Literacy: helps, doesn’t scale
• No single fix works; layering defences is the best we have