Topic 14: Classification

class: center, middle, inverse, title-slide

.title[
# Topic 14: Classification
]
.subtitle[
## Part 2: Examples
]
.author[
### Nick Hagerty <br> ECNS 460/560 Fall 2023 <br> Montana State University
]

---

# Table of contents

1. [Setup: Credit card default data](#setup)

1. [Logistic regression and KNN](#simple)

1. [Cross-validation](#cv)

1. [Decision trees](#trees)

1. [Teach your laptop how to read](#read)

---
layout: true
# Setup: Credit card default data

---
name: setup
class: inverse, middle

---

**1.** Load some libraries:

```r
library(pacman)
p_load(tidyverse, skimr, janitor, tidymodels, magrittr, tune,
       glmnet, readxl, kknn, rpart.plot)
```

And this data on credit card defaults:

```r
default = 
  # skip the first row to load the correct variable names
  read_excel("data/default of credit card clients.xls", skip=1) %>%
  # clean variable names with janitor::clean_names()
  clean_names()
```

---

.scroll-output-full[

```r
skim(default)
```

<table style='width: auto;'
      class='table table-condensed'>
<caption>Data summary</caption>
<tbody>
  <tr>
   <td style="text-align:left;"> Name </td>
   <td style="text-align:left;"> default </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Number of rows </td>
   <td style="text-align:left;"> 30000 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Number of columns </td>
   <td style="text-align:left;"> 25 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> _______________________ </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Column type frequency: </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> numeric </td>
   <td style="text-align:left;"> 25 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ________________________ </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Group variables </td>
   <td style="text-align:left;"> None </td>
  </tr>
</tbody>
</table>

**Variable type: numeric**

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> skim_variable </th>
   <th style="text-align:right;"> n_missing </th>
   <th style="text-align:right;"> complete_rate </th>
   <th style="text-align:right;"> mean </th>
   <th style="text-align:right;"> sd </th>
   <th style="text-align:right;"> p0 </th>
   <th style="text-align:right;"> p25 </th>
   <th style="text-align:right;"> p50 </th>
   <th style="text-align:right;"> p75 </th>
   <th style="text-align:right;"> p100 </th>
   <th style="text-align:left;"> hist </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> id </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 15000.50 </td>
   <td style="text-align:right;"> 8660.40 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 7500.75 </td>
   <td style="text-align:right;"> 15000.5 </td>
   <td style="text-align:right;"> 22500.25 </td>
   <td style="text-align:right;"> 30000 </td>
   <td style="text-align:left;"> ▇▇▇▇▇ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> limit_bal </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 167484.32 </td>
   <td style="text-align:right;"> 129747.66 </td>
   <td style="text-align:right;"> 10000 </td>
   <td style="text-align:right;"> 50000.00 </td>
   <td style="text-align:right;"> 140000.0 </td>
   <td style="text-align:right;"> 240000.00 </td>
   <td style="text-align:right;"> 1000000 </td>
   <td style="text-align:left;"> ▇▃▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> sex </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 1.60 </td>
   <td style="text-align:right;"> 0.49 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 1.00 </td>
   <td style="text-align:right;"> 2.0 </td>
   <td style="text-align:right;"> 2.00 </td>
   <td style="text-align:right;"> 2 </td>
   <td style="text-align:left;"> ▅▁▁▁▇ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> education </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 1.85 </td>
   <td style="text-align:right;"> 0.79 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1.00 </td>
   <td style="text-align:right;"> 2.0 </td>
   <td style="text-align:right;"> 2.00 </td>
   <td style="text-align:right;"> 6 </td>
   <td style="text-align:left;"> ▆▇▃▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> marriage </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 1.55 </td>
   <td style="text-align:right;"> 0.52 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1.00 </td>
   <td style="text-align:right;"> 2.0 </td>
   <td style="text-align:right;"> 2.00 </td>
   <td style="text-align:right;"> 3 </td>
   <td style="text-align:left;"> ▁▇▁▇▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> age </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 35.49 </td>
   <td style="text-align:right;"> 9.22 </td>
   <td style="text-align:right;"> 21 </td>
   <td style="text-align:right;"> 28.00 </td>
   <td style="text-align:right;"> 34.0 </td>
   <td style="text-align:right;"> 41.00 </td>
   <td style="text-align:right;"> 79 </td>
   <td style="text-align:left;"> ▇▇▂▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_0 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> -0.02 </td>
   <td style="text-align:right;"> 1.12 </td>
   <td style="text-align:right;"> -2 </td>
   <td style="text-align:right;"> -1.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> ▇▂▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_2 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> -0.13 </td>
   <td style="text-align:right;"> 1.20 </td>
   <td style="text-align:right;"> -2 </td>
   <td style="text-align:right;"> -1.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_3 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> -0.17 </td>
   <td style="text-align:right;"> 1.20 </td>
   <td style="text-align:right;"> -2 </td>
   <td style="text-align:right;"> -1.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_4 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> -0.22 </td>
   <td style="text-align:right;"> 1.17 </td>
   <td style="text-align:right;"> -2 </td>
   <td style="text-align:right;"> -1.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_5 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> -0.27 </td>
   <td style="text-align:right;"> 1.13 </td>
   <td style="text-align:right;"> -2 </td>
   <td style="text-align:right;"> -1.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_6 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> -0.29 </td>
   <td style="text-align:right;"> 1.15 </td>
   <td style="text-align:right;"> -2 </td>
   <td style="text-align:right;"> -1.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_amt1 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 51223.33 </td>
   <td style="text-align:right;"> 73635.86 </td>
   <td style="text-align:right;"> -165580 </td>
   <td style="text-align:right;"> 3558.75 </td>
   <td style="text-align:right;"> 22381.5 </td>
   <td style="text-align:right;"> 67091.00 </td>
   <td style="text-align:right;"> 964511 </td>
   <td style="text-align:left;"> ▇▃▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_amt2 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 49179.08 </td>
   <td style="text-align:right;"> 71173.77 </td>
   <td style="text-align:right;"> -69777 </td>
   <td style="text-align:right;"> 2984.75 </td>
   <td style="text-align:right;"> 21200.0 </td>
   <td style="text-align:right;"> 64006.25 </td>
   <td style="text-align:right;"> 983931 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_amt3 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 47013.15 </td>
   <td style="text-align:right;"> 69349.39 </td>
   <td style="text-align:right;"> -157264 </td>
   <td style="text-align:right;"> 2666.25 </td>
   <td style="text-align:right;"> 20088.5 </td>
   <td style="text-align:right;"> 60164.75 </td>
   <td style="text-align:right;"> 1664089 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_amt4 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 43262.95 </td>
   <td style="text-align:right;"> 64332.86 </td>
   <td style="text-align:right;"> -170000 </td>
   <td style="text-align:right;"> 2326.75 </td>
   <td style="text-align:right;"> 19052.0 </td>
   <td style="text-align:right;"> 54506.00 </td>
   <td style="text-align:right;"> 891586 </td>
   <td style="text-align:left;"> ▇▃▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_amt5 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 40311.40 </td>
   <td style="text-align:right;"> 60797.16 </td>
   <td style="text-align:right;"> -81334 </td>
   <td style="text-align:right;"> 1763.00 </td>
   <td style="text-align:right;"> 18104.5 </td>
   <td style="text-align:right;"> 50190.50 </td>
   <td style="text-align:right;"> 927171 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_amt6 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 38871.76 </td>
   <td style="text-align:right;"> 59554.11 </td>
   <td style="text-align:right;"> -339603 </td>
   <td style="text-align:right;"> 1256.00 </td>
   <td style="text-align:right;"> 17071.0 </td>
   <td style="text-align:right;"> 49198.25 </td>
   <td style="text-align:right;"> 961664 </td>
   <td style="text-align:left;"> ▁▇▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_amt1 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 5663.58 </td>
   <td style="text-align:right;"> 16563.28 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1000.00 </td>
   <td style="text-align:right;"> 2100.0 </td>
   <td style="text-align:right;"> 5006.00 </td>
   <td style="text-align:right;"> 873552 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_amt2 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 5921.16 </td>
   <td style="text-align:right;"> 23040.87 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 833.00 </td>
   <td style="text-align:right;"> 2009.0 </td>
   <td style="text-align:right;"> 5000.00 </td>
   <td style="text-align:right;"> 1684259 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_amt3 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 5225.68 </td>
   <td style="text-align:right;"> 17606.96 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 390.00 </td>
   <td style="text-align:right;"> 1800.0 </td>
   <td style="text-align:right;"> 4505.00 </td>
   <td style="text-align:right;"> 896040 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_amt4 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 4826.08 </td>
   <td style="text-align:right;"> 15666.16 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 296.00 </td>
   <td style="text-align:right;"> 1500.0 </td>
   <td style="text-align:right;"> 4013.25 </td>
   <td style="text-align:right;"> 621000 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_amt5 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 4799.39 </td>
   <td style="text-align:right;"> 15278.31 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 252.50 </td>
   <td style="text-align:right;"> 1500.0 </td>
   <td style="text-align:right;"> 4031.50 </td>
   <td style="text-align:right;"> 426529 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> pay_amt6 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 5215.50 </td>
   <td style="text-align:right;"> 17777.47 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 117.75 </td>
   <td style="text-align:right;"> 1500.0 </td>
   <td style="text-align:right;"> 4000.00 </td>
   <td style="text-align:right;"> 528666 </td>
   <td style="text-align:left;"> ▇▁▁▁▁ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> default_payment_next_month </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 0.22 </td>
   <td style="text-align:right;"> 0.42 </td>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.0 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:left;"> ▇▁▁▁▂ </td>
  </tr>
</tbody>
</table>
]

---

**2.** Convert categorical variables to factors (because `tidymodels` isn't so great for this step)

.scroll-output-75[

```r
default2 = default %>%
  mutate(across(sex:marriage | pay_0:pay_6 | default_payment_next_month,
                as_factor))
skim(default2)
```

<table style='width: auto;'
      class='table table-condensed'>
<caption>Data summary</caption>
<tbody>
  <tr>
   <td style="text-align:left;"> Name </td>
   <td style="text-align:left;"> default2 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Number of rows </td>
   <td style="text-align:left;"> 30000 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Number of columns </td>
   <td style="text-align:left;"> 25 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> _______________________ </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Column type frequency: </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> factor </td>
   <td style="text-align:left;"> 10 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> numeric </td>
   <td style="text-align:left;"> 15 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ________________________ </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Group variables </td>
   <td style="text-align:left;"> None </td>
  </tr>
</tbody>
</table>

**Variable type: factor**

**Variable type: numeric**

---

**3.** Split the sample into training (80%) and test (20%).

```r
set.seed(101)
default_split = default2 %>% initial_split(prop = 0.8)
default_train = default_split %>% training()
default_test  = default_split %>% testing()
```

**4.** Define a recipe. We'll be predicting whether each customer defaults on their credit card payment in the following month.

```r
default_recipe = default_train %>% 
  recipe(default_payment_next_month ~ .) %>%
  update_role(id, new_role = "ID") %>%
  step_dummy(all_nominal_predictors()) %>%
  step_normalize(all_numeric_predictors()) %>%
  step_nzv(all_predictors())
default_clean = default_recipe %>% prep() %>% juice()
```

---
layout: false
name: simple
class: inverse, middle
# Logistic regression and KNN

---
layout: true
# Logistic regression example

---

Fit a simple logistic regression:

```r
# Define a simple logistic model
model_logistic = logistic_reg() %>%
  set_engine("glm")

# Define the workflow
workflow_logistic = workflow() %>%
  add_recipe(default_recipe) %>%
  add_model(model_logistic)

# Fit the model in the training set, make predictions in the test set
predict_logistic = workflow_logistic %>%
  fit(data = default_train) %>%
  augment(new_data = default_test)
```

---

View the results:

```r
head(predict_logistic %>% select(default_payment_next_month | starts_with(".")), 10)
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-9">
<col><col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">default_payment_next_month</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.pred_class</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.pred_0</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.pred_1</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.779</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.221</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.889</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.111</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.77 </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.23 </td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.884</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.116</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">1</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">1</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.225</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.775</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.805</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.195</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">1</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.664</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.336</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.897</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.103</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.861</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.139</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">1</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">1</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.409</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.591</td></tr>
</table>

---

Calculate accuracy and the confusion matrix:

```r
accuracy(predict_logistic, 
         truth = default_payment_next_month, 
         estimate = .pred_class)
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-10">
<col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.metric</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimator</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimate</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">binary</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.815</td></tr>
</table>

```r
conf_mat(predict_logistic, 
         truth = default_payment_next_month, 
         estimate = .pred_class)
```

```
#>           Truth
#> Prediction    0    1
#>          0 4457  893
#>          1  215  435
```

---
layout: false
# KNN example

Fit a k-nearest neighbors model instead (choosing `$k=10$`):

```r
# Define a simple KNN model
model_knn = nearest_neighbor(neighbors = 10, mode = "classification") %>%
  set_engine("kknn", scale = TRUE)

# Define the workflow
workflow_knn = workflow() %>%
  add_recipe(default_recipe) %>%
  add_model(model_knn)

# Fit the model in the training set, make predictions in the test set
predict_knn = workflow_knn %>%
  fit(data = default_train) %>%
  augment(new_data = default_test)

# Calculate accuracy
accuracy(predict_knn, truth = default_payment_next_month, estimate = .pred_class)
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-11">
<col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.metric</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimator</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimate</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">binary</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.786</td></tr>
</table>

---
# Tuning K in KNN

We can optimize *k*-nearest neighbors by tuning the value of `$k=$` `neighbors` through cross-validation.

Note you can use KNN for regression too - just switch to `mode = "regression"`.

---
layout: false
name: cv
class: inverse, middle
# Cross-validation

---
layout: true
# Logistic regression + shrinkage

---

We can add regularization to a logistic regression in almost exactly the same way as a linear regression.

```r
# Create a 5-fold cross-validation split
default_cv = default_train %>% vfold_cv(v = 5)

# Define a logistic model with ridge regularization
model_logridge = logistic_reg(penalty = tune(), mixture = 0) %>%
  set_engine("glmnet")

# Define the workflow
workflow_logridge = workflow() %>%
  add_recipe(default_recipe) %>%
  add_model(model_logridge)

# Define a sequence of lambdas to try
lambdas = 10 ^ seq(from = 4, to = -2, length = 50)
```

---

The difference is that we have to specify a metric appropriate to classification (i.e., `accuracy` instead of `rmse`).
- You could also specify `sensitivity`, `specificity`, or `precision`.

```r
# Cross-validate
cv_logridge = workflow_logridge %>% tune_grid(
    default_cv,
    grid = data.frame(penalty = lambdas),
    metrics = metric_set(accuracy)
  )

# Show the best model
cv_logridge %>% show_best(n=3)
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-13">
<col><col><col><col><col><col><col><tr>
<th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">penalty</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.metric</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimator</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">mean</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">n</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">std_err</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.config</th></tr>
<tr>
<td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.01  </td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">binary</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.814</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">5</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.00379</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">Preprocessor1_Model01</td></tr>
<tr>
<td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.0133</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">binary</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.814</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">5</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.00379</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">Preprocessor1_Model02</td></tr>
<tr>
<td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.0176</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">binary</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.813</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">5</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.00367</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">Preprocessor1_Model03</td></tr>
</table>

---

.small[

```r
# Fit the best model in the training set, make predictions in the test set
predict_logridge = workflow_logridge %>%
  finalize_workflow(select_best(cv_logridge, 'accuracy')) %>%
  last_fit(default_split) %>%
  collect_predictions()

# Calculate accuracy and the confusion matrix
accuracy(predict_logridge, 
         truth = default_payment_next_month, estimate = .pred_class)
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-14">
<col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.metric</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimator</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimate</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">binary</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.814</td></tr>
</table>

```r
conf_mat(predict_logridge, 
         truth = default_payment_next_month, estimate = .pred_class)
```

```
#>           Truth
#> Prediction    0    1
#>          0 4467  910
#>          1  205  418
```
]

---
layout: true
# Decision trees

---
name: trees
class: inverse, middle

---

Fit a decision tree (give it the raw data, not the normalized or dummy variables):

```r
# Define a decision tree model
model_tree = decision_tree(mode = "classification", cost_complexity = 0.003) %>%
  set_engine("rpart")

# Define the workflow & fit the model
fit_tree = workflow() %>%
  add_formula(default_payment_next_month ~ .) %>%
  add_model(model_tree) %>%
  fit(data = default_train)
```

---

.small[
Output:

```
#> ══ Workflow [trained] ══════════════════════════════════════════════════════════
#> Preprocessor: Formula
#> Model: decision_tree()
#> 
#> ── Preprocessor ────────────────────────────────────────────────────────────────
#> default_payment_next_month ~ .
#> 
#> ── Model ───────────────────────────────────────────────────────────────────────
#> n= 24000 
#> 
#> node), split, n, loss, yval, (yprob)
#>       * denotes terminal node
#> 
#>  1) root 24000 5308 0 (0.7788333 0.2211667)  
#>    2) pay_0=-2,-1,0,1 21472 3549 0 (0.8347150 0.1652850)  
#>      4) pay_2=-2,-1,0,1,8 19713 2821 0 (0.8568965 0.1431035) *
#>      5) pay_2=2,3,4,5,6,7 1759  728 0 (0.5861285 0.4138715)  
#>       10) pay_6=-2,-1,0,4 1253  459 0 (0.6336792 0.3663208) *
#>       11) pay_6=2,3,5,8 506  237 1 (0.4683794 0.5316206) *
#>    3) pay_0=2,3,4,5,6,7,8 2528  769 1 (0.3041930 0.6958070) *
```
]

---

Visualize the tree:

```r
rpart.plot(extract_fit_engine(fit_tree))
```

---
layout: false
# Tuning (and pruning) the tree

Decision trees have several possible tuning parameters:
- `cost_complexity`
- `tree_depth`
- `min_n`

You can read more about them in the R documentation, [ISLR](https://web.stanford.edu/~hastie/ISLR2/ISLRv2_website.pdf) chapter 8, or by Googling (try [here](https://www.tidymodels.org/start/tuning/), [here](https://emilhvitfeldt.github.io/ISLR-tidymodels-labs/tree-based-methods.html), or [here](https://juliasilge.com/blog/wind-turbine/)).

---
layout: true
# Teach your laptop how to read

---
name: read
class: inverse, middle

This section is adapted from [*Introduction to Data Science*](http://rafalab.dfci.harvard.edu/dsbook/machine-learning-in-practice.html) by Rafael A. Irizarry, used under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0).

---

.small[
Believe it or not, we've already gotten far enough with machine learning that you can now teach your computer how to read.
]

.small[
The **MNIST database** is a set of images of handwritten numerical digits. They look like this:

- The images are scanned, centered, and represented in `$28 \times 28=784$` pixels.
- Each pixel's value is a grayscale intensity between 0 (white) and 255 (black).
- Each image has also been read by a human and identified as a digit (0-9).
]

---

**1. Load the data** and take a subset (for quicker computation):

```r
p_load(dslabs)
mnist = read_mnist()
set.seed(1990)
index = sample(nrow(mnist$train$images), 10000)
x = mnist$train$images[index,]
y = factor(mnist$train$labels[index])
numbers = bind_cols("y"=y, as_tibble(x))
```

---

**2. Split the sample** and take a look at the data:

```r
numbers_split = numbers %>% initial_split(prop = 0.8)
numbers_train = numbers_split %>% training()
numbers_test  = numbers_split %>% testing()
head(numbers_test, 8)
```

---

**3. Set up a recipe** & remove near-zero-variance predictors:

```r
numbers_recipe = numbers_train %>%
  recipe(y ~ .) %>%
  step_nzv(all_predictors())
numbers_clean = numbers_recipe %>% prep() %>% juice()
head(numbers_clean, 8)
```

---

**4. Fit a KNN model,** k=10 (this will take a minute or so):

```r
# Define the model
numbers_model = nearest_neighbor(neighbors = 10, mode = "classification") %>%
  set_engine("kknn", scale = TRUE)

# Fit the model
numbers_workflow = workflow() %>%
  add_model(numbers_model) %>%
  add_recipe(numbers_recipe) %>%
  fit(numbers_train)

# Predict in the test set
test_predictions = numbers_workflow %>% 
  last_fit(numbers_split) %>% 
  collect_predictions()
```

---

**Take a look at the results!**

```r
head(test_predictions)
```
<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-6a49cf08077e0108e9d0" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-6a49cf08077e0108e9d0">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86"],[0,0.06,0,0.763,0,0,0,0,0.94,1,0.409,0.928,0,0,0,0,0.971,0.862,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0.029,0,0.08,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.855,0.016,0,0,0,0,0.171,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.766,0,0,0,0,0,0,0,0,0.484,0,0,1,0,0,1,0,0,0,0,0.894,0,0,0,0,0,0,0,0.971,0.127,0.005,0.405,0.005,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,1,0,0,0.595,0.894,0,0,0,0,0,0.406,0],[1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0.4,0,0,0,0,0,0,0,0,0.966,0,0,0,1,0,1,0,0,0,0,0,0.065,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.08,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.86,0,0,0,0,0,0.072,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0.005,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,1,0,1,1,0,0,1,0,0,0.971,0,0,0.765,0,0,0.955,0,0,0,0,0.472,1,0,0,1],[0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0.702,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.995,0,0,0.865,0,0,0,0,0,0,0,0,0,0,0,0.86,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.252,0,0,0,0,0,0,0,0],[0,0.94,0,0,0,0.14,0,0,0.06,0,0.591,0,0,0,0,0,0,0.138,0.298,0,0,0,0,0,0,0,0,0,1,0,0,0.043,0,1,0,0,0,0,0,0,0.135,0,0,0,0,0,0.914,0,0,0,0,0,0,0,0,0,0,0.186,0,0,0,0,0,0,0,0,0,0,0,0.212,0.029,0,0,0.235,0,0.145,0.029,0,0,0,0,0.357,0,0,0.358,0],["6","9","1","4","2","7","3","1","4","4","9","4","6","8","5","6","4","4","8","2","1","6","7","5","3","1","5","4","9","5","1","2","0","9","5","2","3","8","0","0","8","4","5","3","6","5","9","1","6","0","6","0","8","3","0","7","0","0","1","3","7","7","0","7","7","5","5","7","1","3","7","3","0","7","5","4","7","5","5","1","1","7","7","0","5","7"],["6","9","1","4","2","7","3","1","4","4","9","4","6","8","5","6","4","4","8","2","1","6","7","5","3","1","3","4","9","5","1","2","0","9","5","6","3","8","0","0","8","4","5","3","6","5","9","1","6","0","6","0","8","3","0","7","0","9","1","3","7","7","0","7","7","5","5","7","1","3","7","3","0","7","5","4","7","8","5","1","1","7","7","0","5","7"]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>.pred_4<\/th>\n      <th>.pred_5<\/th>\n      <th>.pred_6<\/th>\n      <th>.pred_7<\/th>\n      <th>.pred_8<\/th>\n      <th>.pred_9<\/th>\n      <th>.pred_class<\/th>\n      <th>y<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"t","columnDefs":[{"className":"dt-right","targets":[1,2,3,4,5,6]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script>

---

**Confusion matrix:**

```r
conf_mat(test_predictions, truth=y, estimate=.pred_class)
```

```
#>           Truth
#> Prediction   0   1   2   3   4   5   6   7   8   9
#>          0 200   0   0   0   0   0   0   0   0   1
#>          1   0 230   2   1   2   1   0   0   6   0
#>          2   1   0 179   1   0   0   1   0   3   2
#>          3   0   1   0 207   0   3   0   0   5   5
#>          4   0   1   0   0 185   1   0   0   0   0
#>          5   1   0   0   2   0 168   1   1   4   0
#>          6   1   0   0   1   2   1 221   0   1   0
#>          7   0   0   2   1   2   2   0 200   0   2
#>          8   0   0   0   1   0   2   0   0 142   0
#>          9   0   0   0   2  11   2   0   3   2 184
```

---

**Overall accuracy:**

```r
accuracy(test_predictions, truth=y, estimate=.pred_class)
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-26">
<col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.metric</th><th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimator</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">.estimate</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">accuracy</td><td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">multiclass</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.958</td></tr>
</table>

---

Let's calculate accuracy metrics by digit...

```r
pred_numbers = test_predictions %>%
  dplyr::select(y_hat = .pred_class, y) %>%
  mutate(correct = 1*(y==y_hat))
sens = pred_numbers %>%
  group_by(y) %>%
  summarize(sensitivity = mean(correct))
prec = pred_numbers %>%
  group_by(y_hat) %>%
  summarize(precision = mean(correct))
by_class = bind_cols(sens, prec) %>% dplyr::select(-y_hat)
```

---

**Sensitivity and precision by digit:**

```r
by_class
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto;  " id="tab:unnamed-chunk-28">
<col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">y</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">sensitivity</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: bold;">precision</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.985</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.995</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">1</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.991</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.95 </td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">2</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.978</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.957</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">3</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.958</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.937</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">4</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.916</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.989</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">5</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.933</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.949</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">6</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.991</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.974</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">7</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.98 </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.957</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">8</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.871</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt;    padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.979</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0.4pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">9</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.948</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0.4pt 0pt;    padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.902</td></tr>
</table>

---

Our error rate was **4.2%.**

* [Others](https://en.wikipedia.org/wiki/MNIST_database) have gotten as low as **0.52%** with KNN.

* And as low as **0.17%** with convolutional neural networks.

</br>

Now you basically know how to make a self-driving car!