• We discussed the foundation of AI systems: (good) data!
• Garbage in, garbage out!
• Different AI/ML tasks require different types of labels (classification, regression, etc.)
• Selection bias is dangerous and hard to fix after the fact
• Labelling is harder than it looks, and clear guidelines and multiple annotators improve quality
• Inter-annotator agreement (Cohen’s Kappa) measures labelling reliability
• Today: How do machines actually learn from data? 🤔

• The three paradigms of machine learning
• Supervised learning: Learning from labelled examples
  • Linear regression, decision trees, neural networks
• The bias-variance trade-off
• Unsupervised learning: Finding hidden structure
  • Clustering and dimensionality reduction
• Reinforcement learning: Learning from interaction
  • Agents, rewards, and policies

• Supervised learning: The model learns from labelled examples
• Training data: Input-output pairs \((x_i, y_i)\)
• Goal: Learn a function \(f\) such that \(f(x) \approx y\)
• Like learning with a teacher who provides correct answers
• The “supervision” comes from known labels
• Most common paradigm in real-world applications
• Examples:
  • Email → Spam/Not spam
  • Image → Cat/Dog/Bird
  • Patient data → Disease risk

Classification 🏷️

• Predict a discrete category
• Binary: Yes/No, Spam/Ham
• Multi-class: Cat/Dog/Bird/Fish
• Output: Class label (or probabilities)
• Examples:
  • Fraud detection
  • Disease diagnosis
  • Sentiment analysis

Regression 📈

• Predict a continuous value
• Output: A number on a continuous scale
• Examples:
  • House price prediction
  • Stock price forecasting
  • Temperature prediction
  • Age estimation from photo

• Linear regression: Predict y as a weighted sum of features

\[\hat{y} = w_0 + w_1 x_1 + w_2 x_2 + \ldots + w_n x_n\]

• Learn weights \(w\) that minimise prediction error
• Logistic regression: Classification via sigmoid function

\[P(y=1|x) = \frac{1}{1 + e^{-(w_0 + w_1 x_1 + \ldots)}}\]

• Simple, interpretable, fast to train
• Works well when relationships are approximately linear
• Often a strong baseline before trying complex models

• Decision trees: Learn a hierarchy of yes/no questions
• Each node splits data based on a feature
• Leaves contain predictions
• Easy to interpret: “If income > £50k AND age > 30, then approve loan”
• Can capture non-linear relationships
• Prone to overfitting (memorising training data)
• Solution: Random forests, which combine many trees
  • Each tree sees different data/features
  • Average their predictions for robustness

• Neural networks: Layers of interconnected nodes
• Each layer transforms the input
• Can learn arbitrarily complex functions
• Deep networks (many layers) = deep learning
• Require more data than simpler models
• Less interpretable (“black box”)
• State-of-the-art for:
  • Image classification (CNNs)
  • Speech recognition
  • Natural language processing
• Remember from Lecture 02: backpropagation enables training!

• Every model makes errors! Two sources:
• Bias: Error from oversimplified assumptions
  • Underfitting: Model too simple
  • Misses real patterns in the data
• Variance: Error from sensitivity to training data
  • Overfitting: Model too complex
  • Memorises noise, fails on new data

\[\text{Total Error} = \text{Bias}^2 + \text{Variance} + \text{Noise}\]

• Trade-off: Reducing one often increases the other
• Goal: Find the sweet spot where total error is minimised

• Problem: Single train/test split can be misleading
  • Why? It doesn’t account for enough random variation
• Cross-validation: Multiple train/test splits
• K-fold CV: Split data into K parts
  • Train on K-1 folds, test on 1 fold
  • Repeat K times, rotate the test fold
  • Average the results
• Benefits:
  • Every data point is tested once
  • More reliable performance estimate
  • Helps detect overfitting
• Common choice: K = 5 or K = 10

• Unsupervised learning: Learn from data without labels
• No “correct answers” provided
• Goal: Discover hidden patterns or structure
• Like learning without a teacher, explore and discover
• Why use it?
  • Labels are expensive (experts needed)
  • Sometimes labels don’t exist (“How many customer types?”)
  • Exploration: Understand data before modelling
  • Pre-training: Learn representations, then fine-tune
• Some questions to ask before using these models:
  • Are there natural groups in the data?
  • Can we represent data more compactly?

• Curse of dimensionality: High-dimensional data is hard to work with
  • Distances become meaningless
  • Need exponentially more data
  • Visualisation impossible
• Dimensionality reduction: Find lower-dimensional representation
• Preserve important structure, discard noise
• Two main approaches:
  • Linear: PCA (Principal Component Analysis)
  • Non-linear: t-SNE, UMAP
• Applications: Visualisation, preprocessing, compression
• Super cool example (we will see it in lecture 06!): https://projector.tensorflow.org/

Linear: PCA (Principal Component Analysis)

• Find directions of maximum variance
• Keep top K components to reduce dimensions
• Fast, well-understood, preserves global structure
• Limitation: Only captures linear relationships

Non-linear: t-SNE and UMAP

• Preserve local structure: Nearby points stay nearby
• Excellent for visualisation
• Reveal clusters that PCA misses
• Caution: Cluster sizes and distances can be misleading

• Reinforcement learning (RL): Learn by trial and error
• An agent interacts with an environment
• Takes actions, receives rewards (or punishments)
• Goal: Learn a policy that maximises cumulative reward
• No labelled examples, machines learn from experience
• Like training a dog (or kids!): Reward good behaviour!
• Different from supervised: No “correct” action given
• Different from unsupervised: There IS a goal (maximise reward)

• Exploration: Try new actions to discover better strategies
• Exploitation: Use known good actions to maximise reward
• The dilemma:
  • Too much exploration: Waste time on bad actions
  • Too much exploitation: Miss better strategies
• Example: Restaurant choice
  • Exploit: Go to your favourite restaurant
  • Explore: Try a new restaurant (might be better!)
• RL algorithms must balance both
• Common approach: ε-greedy (explore with probability ε, also called multi-armed bandit)

• When it comes to LLMs, you can make it do (almost) anything with the right prompt
• This is more art than science (as most things in AI are), but it can be fun!
• Please note that I’m not encouraging you to jailbreak anything, as you may get permanently banned from using LLMs or worse
• One of the best people in this field is/was Pliny the Liberator, who jailbroke pretty much all LLMs he could get his hands on 😂
• If you’re interested in jailbreaking, check out this repository: https://github.com/elder-plinius/L1B3RT4S
• Here’s Claude Sonnet teaching how to produce drugs: https://x.com/elder_plinius/status/1972885831955484830

• Alignment problem: Ensuring models do what we mean, not just what we say
• Specification gaming: Models finding perverse ways to maximise rewards (e.g., reward hacking)
• Scalable oversight: How to supervise models that are more capable than their human evaluators
• Red teaming: Proactively finding vulnerabilities and jailbreaks
• Safety is not an afterthought: It must be built into the learning process from the start