• We covered AI regulation around the world
• EU AI Act: World’s first comprehensive AI law with risk-based approach (banned, high-risk, limited, minimal risk tiers)
• US approach: Sectoral regulation with state-level initiatives
• China: State control over content with innovation goals; requires algorithm registration
• No global consensus yet, but convergence on key principles is likely
• Today: How does AI intersect with privacy?

Part 1: Privacy in the AI age

• Why AI makes privacy harder
• Data collection at unprecedented scale
• Inference and prediction of sensitive attributes

Part 2: Legal frameworks

• GDPR and data protection principles
• Rights of individuals
• US privacy landscape

Part 3: Technical approaches

• Differential privacy
• Federated learning
• Privacy-preserving machine learning (synthetic data)

Part 4: Challenges and tensions

• Case study: Clearview AI
• Discussion: where’s your line on privacy?
• What can you do?

AI makes privacy harder in ways that older laws did not anticipate:

Scale of collection:

• Traditional surveillance was limited by staffing; AI-enabled collection runs around the clock at almost no cost
• Every interaction becomes a data point, creating “data exhaust” (data generated as a byproduct of normal use) from ordinary activities

Inference capabilities:

• AI can predict sensitive information from seemingly innocuous data, e.g. shopping patterns reveal health conditions, typing speed reveals mood, social networks reveal politics
• Things you never disclosed can be inferred

Persistence:

• Data doesn’t decay; models trained today affect you forever and decisions follow you across contexts
• “Right to be forgotten” is technically hard

Research has shown AI can predict:

Famous example: Target and pregnancy

• Target’s marketing algorithm identified a pregnant teenager
• Sent baby product coupons to her home
• Her mother complained to Target, saying her daughter was “only in high school”
• Days later, the father called Target, apologising and confirming the pregnancy
• The algorithm knew before her family did (NYT)

LLMs have a training data problem:

• Trained on internet-scale data
• That data includes personal information
• Models can memorise and regurgitate training data
• Your name, address, phone number might be in there!

Demonstrated attacks:

• Researchers extracted verbatim training data from GPT-2 (Carlini et al., 2021)
• Including names, phone numbers, email addresses
• Extraction attacks continue to improve

The consent problem:

• Most people don’t know what’s in training sets
• “Publicly available” ≠ “consented to AI training”

You have the right to:

Article 22: automated decisions

• You have the right not to be subject to purely automated decisions
• There must be human involvement in consequential decisions
• Exceptions exist for contracts and explicit consent

Purpose limitation vs model training:

• You gave data for one purpose
• Can it train a model for another?
• AI companies claim legitimate interest

Data minimisation vs big data:

• AI works better with more data
• GDPR says collect only what’s necessary
• What’s “necessary” for a foundation model? Everything?

Right to explanation vs black boxes:

• If an AI makes a decision about you, you can ask why
• But many AI systems can’t explain themselves

Right to erasure vs model training:

• Can you demand your data be removed from a trained model?
• “Unlearning” is technically very difficult

No comprehensive federal privacy law, as we saw in previous lectures

Instead, a patchwork of sector-specific rules:

State laws filling the gap:

• California (CCPA/CPRA): Most comprehensive, GDPR-like rights
• Virginia, Colorado, Connecticut: Similar frameworks
• Illinois BIPA: Biometric data, with private right of action
• Many more states following, as we saw before

Major fines (selected):

Patterns:

• Big tech companies are primary targets
• Data transfers, consent, children are focus areas
• Fines are getting larger
• Enforcement varies by country (Ireland is lax)

Differential privacy is a mathematical framework for privacy-preserving data analysis (Dwork & Roth, 2014).

• Add carefully calibrated random noise to query results before releasing them
• For example: “How many people in this dataset have diabetes?” The true answer is 137, but the system returns 137 ± some noise (say, 134 or 141)
• Individual records stay hidden, but aggregate patterns remain visible

Formal guarantee:

• The output is nearly the same whether or not any single individual is in the dataset
• Privacy loss is quantified by epsilon (ε): small ε = strong privacy but noisier results; large ε = weaker privacy but more accurate results
• This is a provable, mathematical guarantee, not just a promise

What if you could train a model on data that looks real but isn’t? That is the idea behind synthetic data (Jordon et al., 2022).

How it works:

• Train a generative model on real data
• Model learns statistical patterns and correlations
• Generate new, fake data points that look real
• Train your AI on synthetic data instead

Privacy benefits and limitations:

• No real personal data in training pipeline
• Individuals can’t be identified from synthetic records
• Can share data freely for research and collaboration
• However, synthetic data may not capture rare cases well
• Risk of “overfitting” to original data (memorisation)

What they can do:

• Allow useful research on sensitive datasets (medical, financial) while protecting privacy
• Give engineers concrete tools, not just good intentions
• Provide mathematical guarantees (differential privacy) or architectural separation (federated learning)

What they cannot do:

• Stop companies from collecting data in the first place
• Fix the power imbalance between users and platforms
• Prevent governments from demanding access to data
• Make users understand how their data is used

However…

The best privacy protection is not collecting data at all. But AI companies have the opposite incentive: more data = better models = more profit.

Technical fixes work within the system. They don’t change the system itself.

• In January 2020, the New York Times revealed that Clearview AI had secretly scraped over 30 billion photos from Facebook, Instagram, LinkedIn, and other platforms
• Built a facial recognition database without anyone’s consent!
• Sold access to over 2,400 US law enforcement agencies, plus corporations and wealthy individuals
• US police used the database nearly a million times
• People never knew their faces were in the database

The legal fallout:

• American Civil Liberties Union (ACLU) sued under Illinois BIPA (2020). Settlement (2022): Clearview permanently banned from selling its database to private companies nationwide
• France (CNIL): €20M fine for illegal data collection
• Italy: €20M fine
• Netherlands: €30.5M fine (2024)
• UK, Australia, Canada: ordered to delete data

Individual actions:

• Review privacy settings and use privacy-focused tools (Signal, DuckDuckGo, Firefox)
• Limit location sharing and use different email addresses for different services
• Exercise your GDPR/CCPA rights (access, deletion requests) and be skeptical of “personalisation”

Limitations of individual action:

• Opting out often means losing service. Privacy is collective
• Your data can be inferred from others’ data, and power imbalance is structural

Collective action:

• Support privacy legislation and demand transparency from companies
• Support organisations fighting for privacy (EFF, EPIC, noyb)
• Choose services that respect privacy and vote for privacy-protecting candidates

AI and privacy

• AI enables collection at unprecedented scale
• Inference makes “non-sensitive” data sensitive
• Models can memorise and leak personal information

Legal frameworks

• GDPR: Comprehensive rights-based approach
• US: Patchwork of sectoral laws
• Tension between AI development and data protection

Technical approaches

• Differential privacy: Add noise, preserve patterns
• Federated learning: Train without centralising
• Synthetic data: Train on artificial data, protect originals
• All have trade-offs; none is a silver bullet

Tensions

• Same data enables benefits and surveillance
• Trade-off between utility and privacy exists but is overstated

Key insights

• Collection is the core problem
• Privacy is collective, not just individual
• Technical fixes don’t address power imbalances
• Rules and oversight determine whether data helps or harms